mini-mesh

Create detailed, textured 3D meshes of anything from a short smartphone video.

Quick Start

Requires Docker, an NVIDIA GPU with 12GB+ VRAM (6GB minimum), and the NVIDIA Container Toolkit.

docker/run.sh /path/to/your/video/or/images

This downloads the pre-built Docker image and runs the full pipeline. Add --help for options.

Installation

Docker (recommended)

Setup instructions

1. Install Docker

2. Start and enable the Docker service:

    sudo systemctl start docker
    sudo systemctl enable docker
   

3. Install the NVIDIA Container Toolkit

4. Configure the Docker runtime and restart:

    sudo nvidia-ctk runtime configure --runtime=docker
    sudo systemctl restart docker
   

Building custom images

The pre-built image supports NVIDIA GPUs from GTX 10XX to RTX 40XX. For newer GPUs (e.g. RTX 50XX), rebuild with your GPU’s Compute Capability code:

docker build -t hummat/mini-mesh -f docker/Dockerfile \
  --build-arg TORCH_CUDA_ARCH_LIST=<YOUR-CC> \
  --build-arg CXXFLAGS="-O3 -DNDEBUG -march=native" .

Optional build flags:

• INSTALL_OPTIONAL_DEPS=OFF — Disables nerfstudio, rembg, sam2, hloc, vggsfm
• WITH_GUI=OFF — Headless COLMAP build (no GUI)

Manual Installation

Full manual setup

Requirements: Python 3.11, CUDA 12.4.1, COLMAP, GLOMAP, uv

1. Install Python 3.11

2. Install CUDA Toolkit 12.4.1

3. Install PoseLib (optional but recommended):

   git clone https://github.com/PoseLib/PoseLib.git && cd PoseLib
   git checkout 7e9f5f53372e43f89655040d4dfc4a00e5ace11c
   # Build per PoseLib README
   

4. Install COLMAP and GLOMAP:
   • COLMAP: c5f9cefc87e5dd596b638e4cee0ff543c7d14755 (≈ 3.12.6)
   • GLOMAP: 0edb1b8435e0f9a594318908b81a31f078a51bf7 (≈ 1.2.0)

5. Install Python dependencies:

   uv sync --extra local
   

6. Activate the virtual environment:

   source .venv/bin/activate
   

Optional dependencies

If you used uv sync --extra local, these are already installed (except HLoc).

# NeRF/splat models
uv pip install git+https://github.com/hummat/nerfstudio.git@55a1f83025bb28cbf792760c9b79f9eb22c3a2e4

# Background masking
uv pip install "rembg[gpu,cli]"
uv pip install git+https://github.com/hummat/sam2.git@98f488a540f87260b8e51146dc3ab15694dd174c

# Advanced SfM (HLoc) - requires manual clone
git clone --recursive https://github.com/cvg/Hierarchical-Localization.git
cd Hierarchical-Localization && git checkout 3bdf494c852f157db57a1cf2039a6c826d52e702
git submodule update --init --recursive && uv pip install -e . && cd ..
uv pip install git+https://github.com/hummat/hloc-cli.git@1b714e1183bbc3cb6f4031ddedcc4bd5190ece29

# Advanced SfM (VGGSfM)
uv pip install git+https://github.com/hummat/vggsfm.git@d597df629a312a662544006ac3bdbc2782b82834

# GPU texture baking (nvdiffrast) - requires CUDA toolkit
uv pip install --no-build-isolation git+https://github.com/NVlabs/nvdiffrast.git

Usage

# Docker
docker/run.sh /path/to/your/video/or/images

# Manual
source .venv/bin/activate  # if using uv
scripts/run.sh /path/to/your/video/or/images

The pipeline runs 5 steps: video → sfm → process → train → export

Pass arguments to specific steps using sub-commands:

docker/run.sh /path/to/input video --fps 1 sfm --method glomap process --mask rembg train --model neus-facto

The final mesh appears next to your input. Steps already completed are skipped (use --overwrite to re-run).

Models

Model	Description
neus	Baseline surface reconstruction
neus-facto	Faster surface reconstruction (recommended)
neuralangelo	Higher quality via multi-resolution features, slower
nerfacto	View synthesis, not watertight meshes (requires nerfstudio)
splatfacto	Fast view synthesis via point clouds (requires nerfstudio)

Config suffixes: -test (3K iters), -min (7K), -short (10-30K), (none) (100K), -long (200K+)

Capacity: -small, (none), -large — e.g. neus-facto-small-short

Export methods

SDF models (automatic): Extracts mesh → creates texture coordinates → bakes colors onto texture → simplifies geometry

NeRF models (export --method <name>):

• poisson — Reconstructs smooth surface from rendered point cloud (default)
• tsdf — Fuses depth maps into a volume, then extracts mesh
• pointcloud — Export as point cloud (no mesh)
• gaussian-splat — For splatfacto

Process options

• --mask <method> — Background masking: rembg, sam2, true, none
• --min-match-ratio <float> — Fail if fewer than this fraction of images get poses (default: 0.5)
• --crop-factor <top bot left right> — Crop images before processing

Visualization

TensorBoard (default):

docker/run.sh video.mp4 train --vis tensorboard
tensorboard --logdir /path/to/your/data  # on host

Weights & Biases:

export WANDB_API_KEY=your_api_key  # add to ~/.bashrc
docker/run.sh video.mp4 train --vis wandb

Web Viewer: Automatically configured for nerfstudio’s real-time 3D viewer.

Artist-in-the-loop workflow

1. Run pipeline up to mesh extraction only:

   docker/run.sh /path/to/input video --fps 2 sfm --method glomap process --mask rembg train --model neus-facto --config neus-facto export --mesh-only
   

2. Edit train/<name>/<model>/run/mesh.ply in Blender (don’t change global transform)

3. Run texturing only:

   docker/run.sh /path/to/input export --texture-only
   # Or with edited mesh:
   docker/run.sh /path/to/input export --texture-only --input-mesh-filename mesh_edited.ply
   

4. (Optional) Optimize for web delivery:

   The exported GLB files are ~10MB due to uncompressed geometry and PNG textures. For web use (e.g., <model-viewer>), compress with gltf-transform:

   npx @gltf-transform/cli optimize mesh.glb mesh_web.glb --compress draco --texture-compress webp
   

   This typically achieves 90-95% size reduction (10MB → 500KB-1MB) by:

   • Welding vertices: Blender’s GLB export duplicates vertices at UV seams; optimize merges them back
   • Draco compression: Quantizes geometry to 14-bit precision + entropy coding
   • WebP textures: Lossy compression, visually identical to PNG at ~10% the size

   The mesh quality is preserved—the bloat comes from export artifacts, not your edits.

Without Docker

# Step 1: Extract mesh only
scripts/run.sh /path/to/input \
  video --fps 2 sfm --method glomap process --mask rembg train --model neus-facto --config neus-facto export --mesh-only

# Step 3: Texture only (after editing mesh)
scripts/export.sh /path/to/data/train/<name>/<model>/run --texture-only
# Or with edited mesh:
scripts/export.sh /path/to/data/train/<name>/<model>/run --texture-only --input-mesh-filename mesh_edited.ply

Troubleshooting

Common issues and solutions:

Problem	Quick fix
Bad results	Improve input: 30-120s video, good lighting, cover all angles
CUDA OOM	Reduce ray batch sizes, add --downscale-factor 2
Few SfM poses	Try --matcher exhaustive, --method glomap, or --method hloc
Training diverges	Adjust near-plane/far-plane, try neus-facto
Wrong mesh scale	Adjust --scale-factor (default 2.5)

For advanced tuning (BRDF flags, regularizers, NeuS parameters), see docs/troubleshooting.md.

Documentation

• Troubleshooting — Common issues and advanced tuning
• Methods & Models — How NeuS, NeRF, and other methods work
• BRDF & Shading — Handling reflective and glossy surfaces
• Examples — Additional usage examples

References

1. NeuS: Learning Neural Implicit Surfaces by Volume Rendering
2. Ref-NeRF: Structured View-Dependent Appearance
3. Instant NGP: Multiresolution Hash Encoding
4. Neuralangelo: High-Fidelity Neural Surface Reconstruction
5. Mip-NeRF 360: Unbounded Anti-Aliased NeRF

Credits

Built on SDFStudio, nerfstudio, COLMAP, GLOMAP, HLoc, and VGGSfM.