metadata
base_model:
- Qwen/Qwen3-VL-4B-Instruct
license: apache-2.0
pipeline_tag: depth-estimation
library_name: transformers
tags:
- vision-language-model
- depth-estimation
- 3d-vision
- multimodal
- qwen3-vl
Update 2026-05-18 (v1.0): Initial release
DepthVLM-4B
DepthVLM serves as a unified foundation model for both low-level dense geometry prediction and high-level multimodal understanding, while achieving substantially faster inference compared with existing VLM-based approaches such as DepthLM and Youtu-VL.
By attaching a lightweight depth head to the LLM backbone and training under a unified vision-text supervision paradigm, DepthVLM transforms a single VLM into a native dense geometry predictor while preserving its multimodal capability.
Highlights
- Native dense metric depth estimation in VLMs: Directly predicts geometry within the VLM framework.
- Unified multimodal understanding and geometry prediction: Generates full-resolution depth maps alongside language outputs in a single forward pass.
- Efficient Inference: Achieves higher efficiency compared to per-pixel query or coarse token-level outputs.
- Versatile Application: Supports both indoor and outdoor metric depth estimation.
- Improved 3D spatial reasoning: Moving toward a truly unified foundation model.
Resources
- Paper: Unlocking Dense Metric Depth Estimation in VLMs
- Project Page: https://depthvlm.github.io/
- Repository: https://github.com/hanxunyu/DepthVLM
Usage
Please refer to the official repository for detailed instructions on:
- Data preprocessing
- Training
- Evaluation
- Inference and visualization
Citation
If you find this work useful, please cite:
@article{yu2026unlocking,
title={Unlocking Dense Metric Depth Estimation in VLMs},
author={Hanxun Yu and Xuan Qu and Yuxin Wang and Jianke Zhu and Lei Ke},
journal={arXiv preprint arXiv:2605.15876},
year={2026}
}