license: apache-2.0
library_name: transformers
pipeline_tag: robotics
CosFly-Track
This repository contains model checkpoints associated with the paper CosFly-Track: A Large-Scale Multi-Modal Dataset for UAV Visual Tracking via Multi-Constraint Trajectory Optimization.
Overview
CosFly-Track is a large-scale multi-modal dataset and scalable generation pipeline designed for UAV (Unmanned Aerial Vehicle) visual tracking in urban environments. While most aerial vision-language navigation (VLN) datasets focus on static goals, CosFly-Track addresses the problem of continuously following a moving target while maintaining visibility and avoiding collisions.
Dataset Details
- Scale: Approximately 12,000 expert and perturbed UAV trajectories.
- Volume: 2.4 million timesteps (approximately 334 hours).
- Aligned Channels: Seven aligned data channels including RGB, metric depth, semantic segmentation, six-degree-of-freedom (6-DoF) drone pose, target state with visibility flag, bilingual (Chinese-English) instructions, and trajectory-pair metadata.
Methodology
The project introduces MuCO (Multi-Constraint Optimizer), a planner that plans directly in continuous 3D space. It jointly enforces target visibility, viewpoint quality, collision avoidance, smoothness, and kinematic feasibility, avoiding the artifacts of grid-based planners.
Model Description
The checkpoints in this repository include various vision-language models (VLMs) fine-tuned on the CosFly-Track dataset to act as dynamic target-following agents. Evaluated architectures include:
- Qwen (Qwen3-VL, Qwen3.5)
- InternVL
- GLM-4V
- Gemma 4
Fine-tuning on CosFly-Track significantly improves tracking performance (SR@1 meter) compared to zero-shot baselines, supporting the use of this dataset for training robust autonomous agents.
Citation
@article{cosflytrack2026,
title={CosFly-Track: A Large-Scale Multi-Modal Dataset for UAV Visual Tracking via Multi-Constraint Trajectory Optimization},
author={Anonymous Authors},
journal={arXiv preprint arXiv:2605.17776},
year={2026}
}