metadata
license: cc-by-nc-4.0
task_categories:
- image-segmentation
tags:
- glass-surface-detection
- rgb-d
- scene-understanding
- pytorch
pretty_name: RGBD-GSD-Net (RGB-D Glass Surface Detection Network)
RGBD-GSD-Net — RGB-D Glass Surface Detection Network
Pre-trained weights for the model introduced in:
Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection
Jiaying Lin*, Yuen-Hei Yeung*, Shuquan Ye, Rynson W. H. Lau
AAAI 2025
arXiv · Project Page · Dataset (RGBD-GSD)
Model Summary
RGBD-GSD-Net detects glass surfaces by jointly processing RGB images and depth maps. It introduces two novel modules:
- Cross-Modal Context Mining (CCM): adaptively learns individual and mutual context features from RGB and depth information.
- Depth-Missing Aware Attention (DAA): explicitly exploits spatial locations where depth is missing (a strong indicator of glass surfaces) to guide detection.
The backbone is a ResNeXt encoder shared across both modalities.
| File | Description |
|---|---|
best.pth |
Best checkpoint (204 MB), saved as {'model': state_dict, ...} |
results/our_best_results.zip |
Model predictions on the RGBD-GSD test set |
Loading the Weights
import torch
from networks.your_network import RGBDGlassNet # from the code release
model = RGBDGlassNet()
checkpoint = torch.load("best.pth", map_location="cpu")
model.load_state_dict(checkpoint["model"])
model.eval()
Download the checkpoint:
huggingface-cli download garrying/RGBD-GSD-Net best.pth --local-dir ./weights
Training Dataset
This model was trained and evaluated on RGBD-GSD, the first large-scale RGB-D glass surface detection dataset:
- 3,009 RGB-D images with binary glass surface masks and depth maps
- Available at garrying/RGBD-GSD
Citation
@article{aaai2025_rgbdglass,
author = {Lin, Jiaying and Yeung, Yuen-Hei and Ye, Shuquan and Lau, Rynson W.H.},
title = {Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection},
journal = {AAAI},
year = {2025},
}
License
Non-commercial use only — CC BY-NC 4.0.