AIJonas/model-6-deit-rgb-plus-raw-depth-carb-regression

Model 6: DeiT RGB + raw depth carbohydrate regression

This model predicts total carbohydrate from overhead RGB images and overhead raw depth images.

Architecture

• RGB backbone: deit_base_distilled_patch16_384_imagenet
• Input size: 384x384
• RGB branch uses pretrained DeiT
• Raw depth branch uses a lightweight CNN
• Depth branch also creates a soft guidance mask for the RGB input
• Fusion head combines RGB and depth features
• 2-stage fine-tuning

Test results

Stage 1

• Loss: 89.8303
• MAE: 6.3148
• MSE: 89.8303

Stage 2

• Loss: 63.8722
• MAE: 5.2992
• MSE: 63.8722