| --- |
| tags: |
| - ml-intern |
| --- |
| # LightweightMR — Mesh from Point Cloud (Beginner Guide) |
|
|
| > **TL;DR** — Give it a `.ply` / `.pcd` / `.xyz` file full of 3D points, and it spits out a nice triangle mesh (`.ply` or `.obj`). |
| > |
| > Only depends on **PyTorch + NumPy + SciPy**. No CUDA compiling, no Open3D, no CGAL. |
|
|
| --- |
|
|
| ## 🧭 What does this actually do? |
|
|
| Imagine you have a laser scan of a statue — millions of dots floating in space. This code turns those dots into a **solid surface** made of triangles. |
|
|
| It does this in **two stages**: |
|
|
| ``` |
| Point Cloud Stage 1: Learn SDF Stage 2: Mesh |
| (just dots) → (learn a "distance field") → (triangles!) |
| ``` |
|
|
| ### Stage 1 — Learning a Distance Field (SDF) |
| The code trains a small neural network to answer: |
|
|
| > *"For any random 3D point, how far is it from the surface, and which side is it on?"* |
|
|
| Positive = outside, negative = inside, zero = exactly on the surface. |
|
|
| It learns this purely from your point cloud — no camera images, no manual labels. |
|
|
| ### Stage 2 — Building the Mesh |
| Now that the network knows inside vs. outside, the code: |
|
|
| 1. Sprinkles candidate vertices near the surface |
| 2. Uses another tiny network to nudge them onto high-detail areas (curvature) |
| 3. Projects them exactly onto the zero-distance surface |
| 4. Builds a **3D Delaunay triangulation** (like connecting dots with tetrahedra) |
| 5. Labels each tetrahedron as "inside" or "outside" |
| 6. The walls between inside/outside *are* your surface → extracted as triangles |
| 7. Cleans up non-manifold edges by adding midpoints |
|
|
| --- |
|
|
| ## 🚀 Quick Start (5 minutes) |
|
|
| ### 1. Install |
|
|
| ```bash |
| pip install torch numpy scipy |
| ``` |
|
|
| That's it. No C++ compilers, no 2 GB dependencies. |
|
|
| ### 2. Try on a synthetic sphere (no data needed) |
|
|
| We included a tiny script that makes a fake point cloud so you can see it work immediately: |
|
|
| ```bash |
| # Download / clone the repo files, then: |
| python example/make_sphere.py # creates example/sphere.ply (3000 points) |
| python -m lightweightmr -i example/sphere.ply -o example/sphere_mesh.ply --device cpu |
| ``` |
|
|
| The second command will: |
| - Print progress bars for SDF training (~20k steps) |
| - Print progress bars for vertex generation (~8k steps) |
| - Save `example/sphere_mesh.ply` |
|
|
| **On CPU this takes ~20–40 minutes.** On a CUDA GPU (`--device cuda`) it's ~2–4 minutes. |
|
|
| ### 3. Use your own scan |
|
|
| ```bash |
| python -m lightweightmr -i myscan.ply -o mymesh.ply --device cpu |
| ``` |
|
|
| Supported inputs: `.ply` (ASCII or binary), `.pcd`, `.xyz` |
|
|
| --- |
|
|
| ## 📂 What files do I need? |
|
|
| You only need the `lightweightmr/` folder (9 Python files). Nothing else. |
|
|
| ``` |
| lightweightmr/ |
| __init__.py # package marker |
| __main__.py # CLI (the command you run) |
| optimize.py # the two-stage runner (Stage 1 + Stage 2) |
| sdfnet.py # neural network for distance field |
| vgnet.py # neural network for vertex placement |
| losses.py # math that teaches the networks |
| meshing.py # Delaunay + surface extraction |
| embedder.py # positional encoding (helps the networks) |
| io_utils.py # loading PLY/PCD/XYZ, saving meshes |
| ``` |
|
|
| --- |
|
|
| ## ⚙️ CLI Options Explained |
|
|
| | Flag | Default | What it means | |
| |------|---------|---------------| |
| | `-i` / `--input` | **required** | Your point cloud file | |
| | `-o` / `--output` | **required** | Output mesh file (`.ply` or `.obj`) | |
| | `--device` | `cpu` | `cpu` or `cuda`. GPU is much faster. | |
| | `--sdf-iters` | `20000` | How long to train the distance field. More = better quality on noisy scans. | |
| | `--vg-iters` | `8000` | How long to train vertex placement. | |
| | `--vertices` | `3400` | Target number of vertices in final mesh. More = finer detail, slower. | |
| | `--k-samples` | `21` | Samples per tetrahedron when labeling inside/outside. Higher = cleaner mesh, slower. | |
| | `--save-freq` | `2000` | Save a checkpoint every N iterations (so you can resume). | |
| | `--resume-sdf` | — | Path to a `.pth` checkpoint to skip Stage 1. | |
|
|
| ### Common recipes |
|
|
| **Fast preview (lower quality):** |
| ```bash |
| python -m lightweightmr -i scan.ply -o mesh.ply --sdf-iters 5000 --vg-iters 2000 --vertices 800 |
| ``` |
|
|
| **High quality (slower):** |
| ```bash |
| python -m lightweightmr -i scan.ply -o mesh.ply --sdf-iters 40000 --vg-iters 12000 --vertices 10000 |
| ``` |
|
|
| **Resume after Stage 1 crash:** |
| ```bash |
| python -m lightweightmr -i scan.ply -o mesh.ply --resume-sdf output/sdf_checkpoints/sdf_final.pth |
| ``` |
|
|
| --- |
|
|
| ## 🐍 Python API (for scripts) |
|
|
| If you want to call it from your own code instead of the command line: |
|
|
| ```python |
| from lightweightmr.optimize import Runner |
| |
| runner = Runner( |
| pointcloud_path="myscan.ply", |
| out_dir="./output", |
| device="cpu", # or "cuda" |
| sdf_iters=20_000, |
| vg_iters=8_000, |
| vertices_size=3_400, |
| ) |
| |
| # Run both stages |
| vertices, faces = runner.run(mesh_path="mymesh.ply") |
| |
| # Or run stages separately: |
| runner.train_sdf() # Stage 1 |
| verts = runner.train_vg() # Stage 2 |
| v, f = runner.generate_mesh(verts, save_path="mymesh.ply") |
| ``` |
|
|
| --- |
|
|
| ## 🧪 Understanding the Output |
|
|
| After running, you'll see a new folder `./output/` with: |
|
|
| ``` |
| output/ |
| sdf_checkpoints/ |
| sdf_final.pth # trained distance field (can resume from this) |
| ``` |
|
|
| And your chosen output file (`-o mesh.ply`) contains the mesh. |
|
|
| You can view `.ply` meshes with: |
| - **Blender** (free, drag & drop) |
| - **MeshLab** (free) |
| - **Windows 3D Viewer** |
|
|
| --- |
|
|
| ## 🛠️ Troubleshooting |
|
|
| | Problem | Likely cause | Fix | |
| |---------|--------------|-----| |
| | Takes forever | CPU training | Use `--device cuda` if you have a GPU | |
| | Output mesh has holes | Not enough vertices | Increase `--vertices` | |
| | Noisy / wobbly mesh | Noisy input + too few SDF iters | Increase `--sdf-iters` to `30000+` | |
| | `ModuleNotFoundError` | Missing dependency | `pip install torch numpy scipy` | |
| | `ValueError` on `.ply` | Binary PLY variant we don't parse | Convert to ASCII PLY in MeshLab/Blender | |
|
|
| --- |
|
|
| ## 📖 How is this different from the original paper? |
|
|
| The original CVPR 2025 code is **powerful but heavy** — it needs: |
| - CUDA-compiled hash encoders |
| - CGAL (C++ geometry library) |
| - Open3D, `torch_scatter`, `spconv`, `fpsample`, `mcubes`, `trimesh` |
|
|
| This reimplementation replaces all of that with pure Python + PyTorch + SciPy: |
|
|
| | Original | This version | |
| |----------|--------------| |
| | CUDA hash grid | Positional encoding (slower but no compile) | |
| | PointTransformerV3 vertex generator | Simple MLP (faster, no extra deps) | |
| | CGAL Delaunay + meshing | SciPy Delaunay + our own surface extractor | |
| | C++ KDTree | SciPy KDTree | |
|
|
| **Trade-off:** The SDF stage may need a few more iterations on very detailed scans, but the output quality is comparable for most shapes. |
|
|
| --- |
|
|
| ## 📚 Citation |
|
|
| If you use this, cite the original paper: |
|
|
| ```bibtex |
| @inproceedings{zhang2025high, |
| title={High-Fidelity Lightweight Mesh Reconstruction from Point Clouds}, |
| author={Zhang, Chen and Wang, Wentao and Li, Ximeng and Liao, Xinyao and Su, Wanjuan and Tao, Wenbing}, |
| booktitle={CVPR}, |
| pages={11739--11748}, |
| year={2025} |
| } |
| ``` |
|
|
| --- |
|
|
| License: MIT (reimplementation). Original paper and code © authors. |
|
|
| <!-- ml-intern-provenance --> |
| ## Generated by ML Intern |
|
|
| This model repository was generated by [ML Intern](https://github.com/huggingface/ml-intern), an agent for machine learning research and development on the Hugging Face Hub. |
|
|
| - Try ML Intern: https://smolagents-ml-intern.hf.space |
| - Source code: https://github.com/huggingface/ml-intern |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "bdck/lightweightmr" |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id) |
| ``` |
|
|
| For non-causal architectures, replace `AutoModelForCausalLM` with the appropriate `AutoModel` class. |
|
|
