diff --git a/README.md b/README.md index 269d90a..3d309f7 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,48 @@ -# comfyui-trellis2-rocky-setup +# ComfyUI + Trellis2 on Rocky Linux 9.x / RTX 4090 -ComfyUI + Trellis2 on Rocky Linux 9.x / RTX 4090 — all pitfalls, fixes, working config \ No newline at end of file +**Working setup as of June 2026.** Documents every pitfall, fix, and the exact working configuration +for running [ComfyUI-Trellis2](https://github.com/visualbruno/ComfyUI-Trellis2) on Rocky Linux 9.7 +with an NVIDIA RTX 4090 (sm_89 / Ada Lovelace). + +## Quick Reference + +| Component | Source | Build | Version | +|-----------|--------|-------|---------| +| CUDA Toolkit | DNF (`cuda-toolkit-12-8`) | system | 12.8 | +| PyTorch | pip `--index-url https://download.pytorch.org/whl/cu128` | prebuilt | 2.9.1+cu128 | +| flex_gemm | `JeffreyXiang/FlexGEMM` (GitHub) | **source (sm_89)** | 1.0.0 | +| o_voxel | `visualbruno/TRELLIS.2#subdirectory=o-voxel` | **source (sm_89)** | 0.0.1 | +| cumesh | `visualbruno/CuMesh` (GitHub) | **source (sm_89)** | 1.0 | +| nvdiffrast | `NVlabs/nvdiffrast` (GitHub) | **source (sm_89)** | 0.4.0 | +| xformers | pip `--index-url https://download.pytorch.org/whl/cu128` | prebuilt | 0.0.33.post2 | +| natten | `-f https://whl.natten.org` | prebuilt | 0.21.5+torch290cu128 | +| utils3d | `EasternJournalist/utils3d.git@3fab839` | source | 1.3 | + +## Directory Structure + +``` +~/Documents/ComfyUI_trellis2_pixal/ +├── venv/ # Python 3.12 virtual environment +├── main.py # ComfyUI entry point +├── custom_nodes/ +│ └── ComfyUI-Trellis2/ # visualbruno/ComfyUI-Trellis2 +├── models/ +│ ├── facebook/ +│ │ └── dinov3-vitl16-pretrain-lvd1689m/ # DINOv3 model (download manually) +│ └── trellis2/ +│ └── TRELLIS.2-4B/ # TRELLIS.2-4B model (download manually) +└── conda-libs/ # Conda libstdc++ workaround (legacy) +``` + +## Start ComfyUI + +```bash +cd ~/Documents/ComfyUI_trellis2_pixal +source venv/bin/activate +python main.py --listen 127.0.0.1 +# GUI at http://127.0.0.1:8188 +``` + +## Key Learnings + +See [SETUP.md](SETUP.md) for the full step-by-step installation guide with all pitfalls documented. diff --git a/SETUP.md b/SETUP.md new file mode 100644 index 0000000..0860749 --- /dev/null +++ b/SETUP.md @@ -0,0 +1,337 @@ +# Full Setup Guide: ComfyUI + Trellis2 on Rocky Linux 9.x / RTX 4090 + +## Environment + +- **OS:** Rocky Linux 9.7 +- **GPU:** NVIDIA GeForce RTX 4090 (24 GB VRAM) +- **CUDA:** 12.8 (system toolkit) +- **CPU:** 128 GB RAM +- **Python:** 3.12 + +## Step 1: System Dependencies + +### CUDA 12.8 Toolkit + +Required for building sm_89 CUDA kernels from source. + +```bash +sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo +sudo dnf install -y cuda-toolkit-12-8 +``` + +### Eigen3 Headers (for CuMesh build) + +```bash +sudo dnf install -y eigen3-devel +``` + +### Verify CUDA + +```bash +/usr/local/cuda/bin/nvcc --version +# Should show: Cuda compilation tools, release 12.8 +``` + +--- + +## Step 2: Virtual Environment + +```bash +mkdir -p ~/Documents/ComfyUI_trellis2_pixal +cd ~/Documents/ComfyUI_trellis2_pixal +python3.12 -m venv venv +source venv/bin/activate +pip install --upgrade pip +``` + +--- + +## Step 3: PyTorch + +```bash +pip install torch==2.9.1 torchvision==0.24.1 torchaudio==2.9.1 \ + --index-url https://download.pytorch.org/whl/cu128 +``` + +**⚡ Pitfall:** Do NOT install xformers yet. xformers ≥0.0.34 requires torch ≥2.10, but torchvision +is pinned to torch 2.9.1. Install xformers last with the correct version constraint. + +--- + +## Step 4: ComfyUI + +```bash +cd ~/Documents/ComfyUI_trellis2_pixal +git clone https://github.com/comfyanonymous/ComfyUI.git temp_comfy +mv temp_comfy/* temp_comfy/.[!.]* . 2>/dev/null +rmdir temp_comfy +``` + +Install ComfyUI dependencies: + +```bash +pip install -r requirements.txt +``` + +--- + +## Step 5: Trellis2 Custom Nodes + +```bash +cd custom_nodes +git clone https://github.com/visualbruno/ComfyUI-Trellis2.git +cd .. +``` + +Install Trellis2 Python dependencies: + +```bash +pip install meshlib requests pymeshlab opencv-python scipy open3d plotly rembg +``` + +--- + +## Step 6: The CUDA Architecture Problem + +**⚠️ CRITICAL PITFALL:** Almost every prebuilt CUDA wheel in the Trellis2 ecosystem +(flex_gemm, o_voxel, cumesh, nvdiffrast) is compiled **only for sm_120 (Blackwell)**. +The RTX 4090 is **sm_89 (Ada Lovelace)**. Using prebuilt wheels causes: + +``` +CUDA error: no kernel image is available for execution on the device +``` + +**Solution:** Rebuild all CUDA extension packages from source with `TORCH_CUDA_ARCH_LIST="8.9"`. + +Before each source build, ensure CUDA is in PATH: + +```bash +export PATH=/usr/local/cuda/bin:$PATH +export TORCH_CUDA_ARCH_LIST="8.9" +``` + +--- + +## Step 7: Source Builds (in order) + +### 7a. utils3d — The Right GitHub Fork + +**⚡ Pitfall:** `pip install utils3d` installs an empty placeholder package (v0.1.3) without +the `utils3d.pt` submodule that MoGe needs. + +```bash +pip install "git+https://github.com/EasternJournalist/utils3d.git@3fab839f0be9931dac7c8488eb0e1600c236e183" +# Results in: utils3d-1.3 +``` + +### 7b. flex_gemm — JeffreyXiang/FlexGEMM + +```bash +pip uninstall flex_gemm -y 2>/dev/null +pip install "git+https://github.com/JeffreyXiang/FlexGEMM.git" --no-build-isolation +# Results in: flex_gemm-1.0.0 +``` + +**Verification:** `.so` file should be ~3.4 MB (the prebuilt sm_120-only wheel is smaller). + +### 7c. o_voxel — visualbruno Fork (NOT upstream main!) + +**⚡ Pitfall:** The upstream `microsoft/TRELLIS.2` main branch is missing +`tiled_flexible_dual_grid_to_mesh`. The ComfyUI-Trellis2 author maintains a fork with +the required functions. + +```bash +pip uninstall o_voxel -y 2>/dev/null +pip install "git+https://github.com/visualbruno/TRELLIS.2.git#subdirectory=o-voxel" --no-build-isolation +# Results in: o_voxel-0.0.1 +``` + +**Verification:** +```python +import o_voxel.convert +assert hasattr(o_voxel.convert, 'tiled_flexible_dual_grid_to_mesh') +``` + +### 7d. cumesh — visualbruno/CuMesh (needs Eigen) + +**⚡ Pitfall:** The `remesh_narrow_band_dc_quad` function only exists in visualbruno's fork. +The upstream `JeffreyXiang/CuMesh` only has `remesh_narrow_band_dc` (no `_quad` variant). + +Also needs Eigen3 headers. Set `CPLUS_INCLUDE_PATH`: + +```bash +pip uninstall cumesh -y 2>/dev/null +CPLUS_INCLUDE_PATH=/usr/include/eigen3 pip install "git+https://github.com/visualbruno/CuMesh.git" --no-build-isolation +# Results in: cumesh-1.0 +``` + +**Verification:** +```python +import cumesh.remeshing +assert hasattr(cumesh.remeshing, 'remesh_narrow_band_dc_quad') +``` + +### 7e. nvdiffrast — NVlabs/nvdiffrast + +**⚡ Pitfall:** The pip prebuilt wheel lacks sm_89 kernels. Build from source: + +```bash +pip uninstall nvdiffrast -y 2>/dev/null +pip install "git+https://github.com/NVlabs/nvdiffrast.git" --no-build-isolation +# Results in: nvdiffrast-0.4.0 +``` + +--- + +## Step 8: Attention Backend (xformers) + +**⚡ Pitfall:** xformers version hell! Here's what works: + +```bash +pip install xformers --index-url https://download.pytorch.org/whl/cu128 +# This resolves to xformers-0.0.33.post2 with torch 2.9.1 +``` + +**Why not newer?** +- xformers 0.0.34 → requires torch ≥2.10, pulls torch 2.10 which breaks torchvision +- xformers 0.0.35 → requires torch ≥2.10, same conflict + +If pip pulls the wrong torch version, force-install all together: +```bash +pip install torch==2.9.1 torchvision==0.24.1 torchaudio==2.9.1 xformers \ + --index-url https://download.pytorch.org/whl/cu128 +``` + +--- + +## Step 9: nATTen — Prebuilt Wheel + +**⚡ Pitfall:** Building natten from source requires CMake + CUDA compiler + ~2 GB of `/tmp` space +for build dependencies. Use the prebuilt wheel instead: + +```bash +pip install "natten==0.21.5+torch290cu128" -f https://whl.natten.org +# Results in: natten-0.21.5+torch290cu128 +``` + +--- + +## Step 10: transformers v5 API Fix + +**⚡ Pitfall:** `transformers ≥5` changed the DINOv3 ViT model structure. The internal +`self.model.layer` attribute moved to `self.model.model.layer`. This causes: + +``` +AttributeError: 'DINOv3ViTModel' object has no attribute 'layer' +``` + +Three files need patching in `custom_nodes/ComfyUI-Trellis2/trellis2/`: + +```bash +cd ~/Documents/ComfyUI_trellis2_pixal/custom_nodes/ComfyUI-Trellis2/trellis2 + +# Fix all occurrences at once +sed -i 's/self\.model\.layer/self.model.model.layer/g' \ + modules/image_feature_extractor.py \ + trainers/flow_matching/mixins/image_conditioned.py \ + trainers/flow_matching/mixins/image_conditioned_proj.py +``` + +Reference: [ComfyUI-Trellis2 Issue #144](https://github.com/visualbruno/ComfyUI-Trellis2/issues/144) + +--- + +## Step 11: Disk Space + +**⚡ Pitfall:** Building packages from source fills `/tmp` with pip build artifacts +(especially torch copies, 2+ GB each). Rocky Linux 9 default root partition is 70 GB. + +```bash +# Clean pip build artifacts periodically +rm -rf /tmp/pip-* /tmp/pip-build-env-* +``` + +--- + +## Step 12: Models + +Download manually (too large for automation): + +| Model | Destination | Size | +|-------|-------------|------| +| `dinov3-vitl16-pretrain-lvd1689m` | `models/facebook/dinov3-vitl16-pretrain-lvd1689m/` | ~1.2 GB | +| `TRELLIS.2-4B` | `models/trellis2/TRELLIS.2-4B/` | ~8 GB | + +--- + +## Verification + +Start ComfyUI and run the "Trellis2ShapeCascadeGenerator" workflow: + +```bash +cd ~/Documents/ComfyUI_trellis2_pixal +source venv/bin/activate +python main.py --listen 127.0.0.1 +``` + +Expected output: +``` +[INFO] ComfyUI version: 0.24.0 +[INFO] pytorch version: 2.9.1+cu128 +[INFO] xformers version: 0.0.33.post2 +[INFO] Device: cuda:0 NVIDIA GeForce RTX 4090 +[INFO] Import times for custom nodes: +[INFO] 0.8 seconds: .../ComfyUI-Trellis2 +``` + +A full generation pipeline should complete in ~130 seconds. + +--- + +## Package Version Summary + +``` +torch==2.9.1+cu128 +torchvision==0.24.1+cu128 +torchaudio==2.9.1+cu128 +xformers==0.0.33.post2 +natten==0.21.5+torch290cu128 +utils3d==1.3 +flex_gemm==1.0.0 +o_voxel==0.0.1 +cumesh==1.0 +nvdiffrast==0.4.0 +transformers>=5.0.0 +``` + +--- + +## Common Errors & Fixes + +| Error | Cause | Fix | +|-------|-------|-----| +| `CUDA error: no kernel image is available` | Package compiled for sm_120 only | Rebuild from source with `TORCH_CUDA_ARCH_LIST="8.9"` | +| `No module named 'utils3d'` | Empty pip package | Install GitHub fork (Step 7a) | +| `utils3d has no attribute 'pt'` | Wrong utils3d package | Install GitHub fork (Step 7a) | +| `DINOv3ViTModel has no attribute 'layer'` | transformers v5 API change | Apply sed fix (Step 10) | +| `No module named 'flash_attn'` / `'xformers'` | Missing attention backend | Install xformers (Step 8) | +| `No module named 'natten'` | Missing neighborhood attention | Install prebuilt wheel (Step 9) | +| `No module named 'tiled_flexible_dual_grid_to_mesh'` | Wrong o_voxel source | Use visualbruno fork (Step 7c) | +| `No module named 'remesh_narrow_band_dc_quad'` | Wrong cumesh source | Use visualbruno fork (Step 7d) | +| `Cuda error: 209` in nvdiffrast | Wrong nvdiffrast arch | Build from source (Step 7e) | +| `No space left on device` | /tmp filled by pip builds | Clean /tmp (Step 11) | +| `No CMAKE_CUDA_COMPILER could be found` | nvcc not in PATH | `export PATH=/usr/local/cuda/bin:$PATH` | +| `fatal error: Eigen/Dense` | Missing Eigen headers | `dnf install eigen3-devel` + set CPLUS_INCLUDE_PATH | +| xformers/torch version conflict | xformers≥0.0.34 needs torch≥2.10 | Pin to xformers 0.0.33.post2 (Step 8) | + +--- + +## Credits + +- [ComfyUI](https://github.com/comfyanonymous/ComfyUI) +- [ComfyUI-Trellis2](https://github.com/visualbruno/ComfyUI-Trellis2) by visualbruno +- [TRELLIS.2](https://github.com/microsoft/TRELLIS.2) by Microsoft +- [FlexGEMM](https://github.com/JeffreyXiang/FlexGEMM) by JeffreyXiang +- [NATTEN](https://github.com/SHI-Labs/NATTEN) by SHI-Labs +- [nvdiffrast](https://github.com/NVlabs/nvdiffrast) by NVIDIA +- [MoGe](https://github.com/microsoft/MoGe) by Microsoft diff --git a/build_cuda_extensions.sh b/build_cuda_extensions.sh new file mode 100644 index 0000000..ab915e9 --- /dev/null +++ b/build_cuda_extensions.sh @@ -0,0 +1,54 @@ +#!/bin/bash +# Build all CUDA extension packages from source for sm_89 (RTX 4090) +# Must run inside the activated venv with CUDA toolkit installed +# +# Usage: source venv/bin/activate && bash build_cuda_extensions.sh + +set -e + +export PATH="/usr/local/cuda/bin:$PATH" +export TORCH_CUDA_ARCH_LIST="8.9" + +echo "=== Building CUDA extensions for sm_89 (RTX 4090) ===" +echo "" + +# 1. utils3d (MoGe dependency — GitHub fork, NOT pip package) +echo "[1/5] utils3d (EasternJournalist fork)..." +pip install --quiet "git+https://github.com/EasternJournalist/utils3d.git@3fab839f0be9931dac7c8488eb0e1600c236e183" +echo " ✓ done" + +# 2. flex_gemm +echo "[2/5] flex_gemm..." +pip uninstall -y flex_gemm 2>/dev/null || true +pip install --quiet "git+https://github.com/JeffreyXiang/FlexGEMM.git" --no-build-isolation +echo " ✓ done" + +# 3. o_voxel (visualbruno fork — has tiled_flexible_dual_grid_to_mesh) +echo "[3/5] o_voxel..." +pip uninstall -y o_voxel 2>/dev/null || true +pip install --quiet "git+https://github.com/visualbruno/TRELLIS.2.git#subdirectory=o-voxel" --no-build-isolation +echo " ✓ done" + +# 4. cumesh (visualbruno fork — has remesh_narrow_band_dc_quad, needs Eigen) +echo "[4/5] cumesh..." +pip uninstall -y cumesh 2>/dev/null || true +CPLUS_INCLUDE_PATH="/usr/include/eigen3:$CPLUS_INCLUDE_PATH" \ + pip install --quiet "git+https://github.com/visualbruno/CuMesh.git" --no-build-isolation +echo " ✓ done" + +# 5. nvdiffrast +echo "[5/5] nvdiffrast..." +pip uninstall -y nvdiffrast 2>/dev/null || true +pip install --quiet "git+https://github.com/NVlabs/nvdiffrast.git" --no-build-isolation +echo " ✓ done" + +echo "" +echo "=== Verification ===" +python -c " +import utils3d; print('utils3d:', utils3d.__version__) +import flex_gemm; print('flex_gemm: OK') +import o_voxel.convert; assert hasattr(o_voxel.convert, 'tiled_flexible_dual_grid_to_mesh'); print('o_voxel: OK (tiled_flexible_dual_grid_to_mesh found)') +import cumesh.remeshing; assert hasattr(cumesh.remeshing, 'remesh_narrow_band_dc_quad'); print('cumesh: OK (remesh_narrow_band_dc_quad found)') +import nvdiffrast.torch; print('nvdiffrast: OK (RasterizeCudaContext available)') +" +echo "=== All CUDA extensions built successfully ===" diff --git a/fix_transformers_v5.sh b/fix_transformers_v5.sh new file mode 100644 index 0000000..89fc721 --- /dev/null +++ b/fix_transformers_v5.sh @@ -0,0 +1,33 @@ +#!/bin/bash +# Apply transformers v5 API fix to ComfyUI-Trellis2 +# Fixes: AttributeError: 'DINOv3ViTModel' object has no attribute 'layer' +# Reference: https://github.com/visualbruno/ComfyUI-Trellis2/issues/144 + +TRELLIS2_DIR="${1:-custom_nodes/ComfyUI-Trellis2/trellis2}" + +if [ ! -d "$TRELLIS2_DIR" ]; then + echo "ERROR: Trellis2 directory not found: $TRELLIS2_DIR" + echo "Usage: $0 [path/to/ComfyUI-Trellis2/trellis2]" + exit 1 +fi + +echo "Patching DINOv3 model.layer → model.model.layer in:" +for f in \ + modules/image_feature_extractor.py \ + trainers/flow_matching/mixins/image_conditioned.py \ + trainers/flow_matching/mixins/image_conditioned_proj.py +do + file="$TRELLIS2_DIR/$f" + if [ -f "$file" ]; then + if grep -q 'self\.model\.layer' "$file"; then + sed -i 's/self\.model\.layer/self.model.model.layer/g' "$file" + echo " ✓ $f" + else + echo " - $f (already patched)" + fi + else + echo " ✗ $f (not found)" + fi +done + +echo "Done." diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..73fe508 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,29 @@ +# ComfyUI base (from ComfyUI/requirements.txt) +# Install with: pip install -r requirements_comfyui.txt +-r https://raw.githubusercontent.com/comfyanonymous/ComfyUI/master/requirements.txt + +# Trellis2 Python dependencies +meshlib +requests +pymeshlab +opencv-python +scipy +open3d +plotly +rembg + +# Core +torch==2.9.1 +torchvision==0.24.1 +torchaudio==2.9.1 + +# Attention backends +xformers==0.0.33.post2 +natten==0.21.5+torch290cu128 + +# Source-built CUDA extensions (use pip install from GitHub, see SETUP.md) +# flex_gemm @ git+https://github.com/JeffreyXiang/FlexGEMM.git +# o_voxel @ git+https://github.com/visualbruno/TRELLIS.2.git#subdirectory=o-voxel +# cumesh @ git+https://github.com/visualbruno/CuMesh.git +# nvdiffrast @ git+https://github.com/NVlabs/nvdiffrast.git +# utils3d @ git+https://github.com/EasternJournalist/utils3d.git@3fab839