ilessio-aiflowlab/project_sif

SIF — Detail-Preserving 3D Semantic Segmentation

Part of the ANIMA Perception Suite by Robot Flow Labs.

Paper

Details Matter: Fine-Grained Indoor 3D Segmentation with Open-Vocabulary (Jul 2025) ANIMA Research Institute

Architecture

DetailAwarePointNet — a PointNet++ variant with detail-preserving set abstraction layers designed for fine-grained indoor 3D semantic segmentation. Preserves small objects, thin structures, and subtle boundaries that coarse 3D methods miss.

• Parameters: 705,979
• Input: Points (B, N, 3) + Features (B, N, 6) [XYZ + RGB/normals]
• Output: Logits (B, N, 59) + Features (B, N, 512)
• Backbone: 4-level encoder (SA1–SA4) with detail-preserving radii + feature propagation decoder with skip connections

Training

Parameter	Value
Dataset	Replica (8 indoor scenes)
Epochs	34 (early stopped from 120)
Batch size	4
Points/sample	8,192
Learning rate	5e-4 (AdamW + warmup cosine)
Precision	FP16 (AMP)
GPU	NVIDIA L4 (23 GB)
Training time	~3 hours
Best val_loss	0.763
Final train_loss	0.749

Exported Formats

Format	File	Size	Use Case
PyTorch (.pth)	pytorch/sif_v1.pth	2.8 MB	Training, fine-tuning
SafeTensors	pytorch/sif_v1.safetensors	2.8 MB	Fast loading, safe
TorchScript	pytorch/sif_v1_traced.pt	4.6 MB	Production inference
ONNX (opset 17)	onnx/sif_v1.onnx	38.6 MB	Cross-platform inference
TRT-optimized ONNX	tensorrt/sif_v1_trt_optimized.onnx	11.5 MB	TensorRT engine build

Note: TensorRT .trt engine files are hardware-specific and should be built on the target device from the TRT-optimized ONNX.

Additional Files

File	Description
checkpoints/best.pth	Full checkpoint (model + optimizer + scheduler) for resume
configs/training_config.yaml	Complete training configuration for reproducibility
logs/training_report.json	Training metrics and timing
pytorch/model_meta.json	Model architecture metadata

Quick Start

import torch
from safetensors.torch import load_file
from anima_sif.backbone.detail_aware_pointnet import DetailAwarePointNet

# Load model
state = load_file("pytorch/sif_v1.safetensors")
model = DetailAwarePointNet(in_dim=6, num_classes=59, feature_dim=512)
model.load_state_dict(state)
model.eval()

# Inference
points = torch.randn(1, 8192, 3)    # XYZ coordinates
features = torch.randn(1, 8192, 6)  # XYZ + RGB or normals
out = model(points, features)
# out["logits"]: (1, 8192, 59)
# out["features"]: (1, 8192, 512)

ONNX Inference

import onnxruntime as ort
import numpy as np

sess = ort.InferenceSession("onnx/sif_v1.onnx")
points = np.random.randn(1, 8192, 3).astype(np.float32)
features = np.random.randn(1, 8192, 6).astype(np.float32)
logits, feats = sess.run(None, {"points": points, "features": features})

ROS2 Interface

Topic	Type	Direction
/anima/sif/pointcloud	sensor_msgs/PointCloud2	Input
/anima/sif/query	std_msgs/String	Input
/anima/sif/segmentation	sensor_msgs/PointCloud2	Output
/anima/sif/health	std_msgs/String	Output (1 Hz)

Docker

docker pull ghcr.io/robotflow-labs/anima-sif:0.1.0
docker run --gpus all -p 8080:8080 ghcr.io/robotflow-labs/anima-sif:0.1.0

License

Apache 2.0 — Robot Flow Labs / AIFLOW LABS LIMITED

Built with ANIMA by Robot Flow Labs.