dirtybeets-lab-targets-v1-pointnet

This repository contains the locked non-IR PointNet release for dirtybeets lab-target prediction.

What It Predicts

• clean weight
• unclean weight
• gap = unclean - clean
• tare = 100 * gap / unclean

The shipped checkpoint is the current best non-IR PointNet joint model on the signalalign_v1_pointcloud_v1 split.

Headline Metrics

Held-out test metrics from model/summary.json:

• clean: MAE 2.4220, RMSE 3.0135, R² 0.5343, count 106
• unclean: MAE 2.4472, RMSE 3.0504, R² 0.5338, count 106
• gap: MAE 0.1265, RMSE 0.1621, R² 0.1033, count 106
• tare: MAE 0.5855, RMSE 0.8348, R² 0.2770, count 106

Bundle Contents

• model/model.pt: trained PointNet checkpoint
• model/summary.json: training summary and metrics
• model/val_predictions.csv: validation predictions
• model/test_predictions.csv: held-out test predictions
• infer_from_capture.py: end-to-end inference from raw capture folders
• training/train_clean_unclean_pointnet_joint.py: exact joint non-IR PointNet trainer used for this release
• training/train_clean_pointnet_beetset.py: training utilities and cache builder used by the joint trainer
• resources/: ROI detector, beet detector, viability classifier, and beet segmenter
• requirements.txt: inference dependencies
• deployment_config.json: release metadata
• resource_manifest.json: packaged file inventory

Input Contract

Each capture directory must contain:

• rgb.jpg
• pointcloud_rgb.ply

The point cloud must be the organized RGB-aligned PLY used in the dirtybeets pipeline, with one (x, y, z, r, g, b) vertex per RGB pixel.

Running Inference

Install dependencies:

pip install -r requirements.txt

Single capture:

python infer_from_capture.py \
  --rgb /path/to/capture/rgb.jpg \
  --pointcloud /path/to/capture/pointcloud_rgb.ply \
  --capture-id capture_001

Batch directory:

python infer_from_capture.py --input-root /path/to/captures

Jetson-safe GPU invocation:

python infer_from_capture.py \
  --input-root /path/to/captures \
  --device cuda \
  --yolo-device 0 \
  --viability-batch-size 4 \
  --seg-batch-size 1 \
  --yolo-half \
  --no-tta

The script writes:

• detections.csv
• crop_manifest.csv
• sample_manifest.csv
• predictions.csv
• predictions.json

under the chosen work directory.

Pipeline Structure

1. detect the conveyor ROI from rgb.jpg
2. detect beet boxes inside the ROI
3. classify viability for the stage-3 crops
4. segment viable beet crops
5. slice the aligned pointcloud_rgb.ply under each segmented crop
6. build beet-set point features
7. run the joint PointNet model

Reproducing Training

The exact training code for this release is bundled under training/.

The joint trainer expects the same dataset layout used in dirtybeets training:

• sample_manifest.csv
• crop_manifest.csv
• stage1_stage2_stage3_detections_with_viability.csv
• aligned pointcloud_rgb.ply capture folders

Example command:

python training/train_clean_unclean_pointnet_joint.py \
  --sample-manifest /path/to/sample_manifest.csv \
  --crop-manifest /path/to/crop_manifest.csv \
  --detections-csv /path/to/stage1_stage2_stage3_detections_with_viability.csv \
  --pointcloud-root /path/to/pointcloud_root \
  --out-dir /path/to/output_dir

This locked release corresponds to the non-IR high-tail-oversampled configuration.

Important Note

This release is the locked non-IR PointNet model. An IR-augmented variant was tested later, but it did not beat this model on tare RMSE.

The bundled inference entrypoint uses staged model loading and explicit CUDA cleanup so the full pipeline can run on low-memory Jetson-class GPUs. It was smoke-tested on an 8 GB Orin with the Jetson-safe GPU invocation above.