Upload README.md

README.md

@@ -0,0 +1,246 @@
+# NKSR Wrapper — Neural Kernel Surface Reconstruction
+
+[![Paper](https://img.shields.io/badge/Paper-CVPR%202023-blue)](https://arxiv.org/abs/2305.19590)
+[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+A **clean, high-level Python wrapper** around **Neural Kernel Surface Reconstruction (NKSR)** by Huang et al. (CVPR 2023, Highlight).  
+Drop a `.ply` or `.pcd` point cloud in → get a watertight, high-quality triangle mesh out.
+
+---
+
+## 📦 What is NKSR?
+
+**Neural Kernel Surface Reconstruction** is a deep-learning method that turns a raw, sparse, and potentially noisy point cloud into a smooth, watertight 3D mesh.  Unlike classical methods (Poisson, Alpha shapes) it learns a **continuous implicit surface** from data, and unlike vanilla neural fields (NeRF, DeepSDF) it scales to **millions of points** and **generalises across objects, rooms, and outdoor scenes** without per-scene training.
+
+### The three key innovations
+
+| Innovation | Why it matters |
+|-----------|----------------|
+| **Compactly-supported kernel functions** | The implicit field is built from local kernel basis functions that have finite support.  This makes the linear system **sparse**, so it can be solved with fast sparse PCG solvers instead of dense matrix inversion.  Result: room-scale reconstruction in seconds. |
+| **Gradient fitting solve** | Instead of only fitting point positions (SDF ≈ 0), NKSR also fits **surface normals** as gradients of the field.  This makes the reconstruction dramatically more robust to noise and outliers. |
+| **Minimal training, maximum generalisation** | The model is trained once on a mixture of synthetic and real data (the "kitchen-sink" config) and then works out-of-the-box on new scans **without any fine-tuning**. |
+
+---
+
+## 🚀 Quick start
+
+### 1. Install dependencies
+
+NKSR itself contains custom CUDA kernels, so you need a working PyTorch + CUDA environment.
+
+```bash
+# 1. Clone the original NKSR repo and install it
+#    (see https://github.com/nv-tlabs/NKSR for the latest instructions)
+git clone https://github.com/nv-tlabs/NKSR.git
+cd NKSR
+pip install -r requirements.txt
+pip install --no-build-isolation package/
+
+# 2. Install this wrapper
+pip install -e .
+```
+
+### 2. One-liner reconstruction
+
+```python
+from nksr_wrapper import NKSRMeshReconstructor, load_point_cloud, save_mesh
+
+points, normals = load_point_cloud("scan.ply")
+recon = NKSRMeshReconstructor(device="cuda:0")
+mesh = recon.reconstruct(points, normals, detail_level=1.0)
+save_mesh("mesh.ply", mesh.vertices, mesh.faces)
+```
+
+Or use the CLI:
+
+```bash
+python scripts/reconstruct.py scan.ply mesh.ply --detail 1.0 --mise-iter 1
+```
+
+---
+
+## 🔬 How it works (the full pipeline)
+
+If you want to understand what is happening under the hood, here is the step-by-step pipeline that NKSR executes every time you call `reconstruct()`.
+
+### Step 0 — Input
+You provide:
+* `xyz` — (N, 3) point positions
+* `normal` — (N, 3) **oriented** normals (optional but strongly recommended)
+* `sensor` — (N, 3) sensor/camera positions (optional; used for normal orientation when normals are missing)
+
+### Step 1 — Voxelisation (Sparse Feature Hierarchy)
+The input points are splatted into a **sparse voxel grid** at multiple resolutions (a quad-/octree-like structure called the *Sparse Feature Hierarchy*, SVH).  Instead of a dense 3D array, only occupied voxels are stored.  This is what lets NKSR handle millions of points without exploding memory.
+
+*Key parameter:* `voxel_size` (default ≈ 0.1 in the pretrained config).  One voxel_size unit = one spatial unit in your point-cloud coordinate system.
+
+### Step 2 — Feature encoding (PointNet → Sparse 3D U-Net)
+1. **PointEncoder** — A small PointNet-style ResNet processes the raw points inside each voxel and produces a 32-dim feature vector per voxel.
+2. **SparseStructureNet** — A sparse 3D convolutional U-Net with skip connections processes these voxel features across multiple scales.  It also predicts an *adaptive structure*: if a region is empty, the network stops subdividing early, saving computation.
+
+### Step 3 — Geometry field (Kernel Field)
+This is the heart of NKSR.
+
+The network outputs **kernel basis parameters** at each voxel.  At any 3D query point `x`, the implicit function is evaluated as a weighted sum of compact kernel functions centred on nearby voxels:
+
+```
+f(x) = Σ_i  w_i · φ_i(x)
+```
+
+where `φ_i` is a compact kernel (e.g. Wendland or similar) and `w_i` are learned weights.  Because the kernels have finite support, the sum only involves neighbours within a small radius → **sparse linear system**.
+
+### Step 4 — Sparse linear solve (PCG)
+NKSR now solves for the weights `w` by fitting two constraints:
+
+1. **Position constraint:** `f(x_j) ≈ 0` on every input point (the surface is the zero level-set).
+2. **Normal constraint:** `∇f(x_j) ≈ n_j` on voxel centres (the gradient of the field matches the surface normal).
+
+These constraints are assembled into a large but **sparse** linear system and solved with a **preconditioned conjugate-gradient (PCG)** solver.  The normal constraint is the secret sauce: it anchors the *gradient* of the field, making the reconstruction much less sensitive to noise than methods that only fit positions.
+
+### Step 5 — Mask / trimming field (optional)
+A secondary field (either a learned UDF — Unsigned Distance Field — or a simple layer field) identifies regions that are *outside* the true surface.  This trims away spurious floaters and fills small holes, producing a clean watertight boundary.
+
+### Step 6 — Mesh extraction (Dual Marching Cubes + MISE)
+Finally, the zero level-set of the implicit field is turned into triangles:
+
+1. **Dual Marching Cubes (DMC)** is run on the dual graph of the sparse voxel hierarchy.  DMC produces nicer topology than standard Marching Cubes (fewer skinny triangles, better sharp-feature preservation).
+2. **MISE** (Multi-resolution IsoSurface Extraction) adaptively subdivides cells that straddle the zero crossing.  Each `mise_iter` doubles the effective resolution in those cells, giving you a crisp mesh without wasting polygons on empty space.
+
+*Result:* `mesh.v` (V×3 vertices) and `mesh.f` (F×3 face indices).
+
+---
+
+## 🧰 API Reference
+
+### `NKSRMeshReconstructor`
+
+```python
+class NKSRMeshReconstructor(
+    device="cuda:0",
+    config="ks",
+    chunk_tmp_device="cpu",
+)
+```
+
+* `device` — PyTorch device (CUDA strongly recommended).
+* `config` — Pretrained model name:
+  * `"ks"` — **Kitchen-sink** (recommended default).  Trained on a mixture of synthetic and real scans; generalises to objects, indoor rooms, and outdoor scenes.
+  * `"snet"` — ShapeNet objects with normals.
+  * `"snet-wonormal"` — ShapeNet objects without normals.
+* `chunk_tmp_device` — Where to stash finished chunks when reconstructing huge scenes.  `"cpu"` offloads to system RAM.
+
+#### `.reconstruct(...)`
+
+```python
+mesh = recon.reconstruct(
+    points,                # (N, 3)  required
+    normals=None,          # (N, 3)  optional, strongly recommended
+    sensor_positions=None, # (N, 3)  optional, helps orient normals
+    colors=None,           # (N, 3)  optional, for colored mesh output
+
+    # Quality / resolution
+    detail_level=1.0,      # 0.0 = smooth, 1.0 = max detail
+    voxel_size=None,       # override resolution explicitly
+    mise_iter=1,           # 0 = base, 1 = 2× in subdivided cells, 2 = 4×
+
+    # Large-scene settings
+    chunk_size=-1.0,       # >0 enables out-of-core chunking
+    overlap_ratio=0.05,
+
+    # Solver tuning
+    solver_max_iter=2000,
+    solver_tol=1e-5,
+    approx_kernel_grad=False,
+
+    # Normal estimation fallback
+    estimate_normals_if_missing=True,
+    normal_knn=64,
+    normal_drop_threshold_deg=85.0,
+)
+```
+
+Returns a `MeshResult` dataclass with:
+* `.vertices` — (V, 3) float array
+* `.faces` — (F, 3) int array
+* `.vertex_colors` — (V, 3) float array, if `colors` was provided
+* `.save(path)` — convenience method to write PLY/OBJ/GLB via Trimesh
+
+---
+
+## 📂 Repository layout
+
+```
+nksr-wrapper/
+├── nksr_wrapper/
+│   ├── __init__.py          # public API
+│   ├── reconstructor.py     # NKSRMeshReconstructor + MeshResult
+│   └── io.py                # load_point_cloud, save_mesh
+├── scripts/
+│   └── reconstruct.py       # CLI entry point
+├── examples/
+│   ├── quickstart.py        # minimal script
+│   └── chunked_reconstruction.py   # large-scene example
+├── setup.py
+├── requirements.txt
+└── README.md
+```
+
+---
+
+## 🖥️ CLI Usage
+
+```bash
+# Basic reconstruction
+python scripts/reconstruct.py scan.ply mesh.ply --detail 1.0
+
+# Large scene (chunked)
+python scripts/reconstruct.py huge_scan.ply mesh.ply --chunk-size 50.0
+
+# No normals in file — estimate on-the-fly
+python scripts/reconstruct.py scan.ply mesh.ply --estimate-normals
+
+# With per-point colors → colored mesh
+python scripts/reconstruct.py scan.ply mesh.ply --colors colors.npy --mise-iter 2
+```
+
+---
+
+## 🎯 Tips & Troubleshooting
+
+| Problem | Solution |
+|---------|----------|
+| Mesh is too noisy / has spikes | Lower `detail_level` (try `0.3`) or increase `voxel_size` |
+| Mesh is too smooth / missing fine detail | Raise `detail_level` (try `1.0`) or set `mise_iter=2` |
+| Out-of-memory on large scans | Use `chunk_size=50.0` and `chunk_tmp_device="cpu"` |
+| Mesh is inside-out | Normals are unoriented.  Provide `sensor_positions` or pre-orient normals with Open3D |
+| Reconstruction is very slow | You are probably on CPU.  NKSR requires CUDA for the custom sparse kernels. |
+| PLY file has no normals | Use `--estimate-normals` or pass `sensor_positions` to the reconstructor |
+
+---
+
+## 📚 Citation
+
+If you use NKSR in your research, please cite the original paper:
+
+```bibtex
+@inproceedings{huang2023nksr,
+  title={Neural Kernel Surface Reconstruction},
+  author={Huang, Jiahui and Gojcic, Zan and Atzmon, Matan and 
+          Litany, Or and Fidler, Sanja and Williams, Francis},
+  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+  year={2023}
+}
+```
+
+Original code: [https://github.com/nv-tlabs/NKSR](https://github.com/nv-tlabs/NKSR)  
+Pretrained weights: [https://huggingface.co/heiwang1997/nksr-checkpoints](https://huggingface.co/heiwang1997/nksr-checkpoints)
+
+---
+
+## 📄 License
+
+This wrapper is released under the MIT License.  NKSR itself is under its own license (see the original repository).
+
+---
+
+*Built with ❤️ on top of NVIDIA t-labs' NKSR.*