README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: robotics
---

# CosFly-Track

This repository contains model checkpoints associated with the paper [CosFly-Track: A Large-Scale Multi-Modal Dataset for UAV Visual Tracking via Multi-Constraint Trajectory Optimization](https://huggingface.co/papers/2605.17776).

## Overview

CosFly-Track is a large-scale multi-modal dataset and scalable generation pipeline designed for UAV (Unmanned Aerial Vehicle) visual tracking in urban environments. While most aerial vision-language navigation (VLN) datasets focus on static goals, CosFly-Track addresses the problem of continuously following a moving target while maintaining visibility and avoiding collisions.

### Dataset Details
- **Scale**: Approximately 12,000 expert and perturbed UAV trajectories.
- **Volume**: 2.4 million timesteps (approximately 334 hours).
- **Aligned Channels**: Seven aligned data channels including RGB, metric depth, semantic segmentation, six-degree-of-freedom (6-DoF) drone pose, target state with visibility flag, bilingual (Chinese-English) instructions, and trajectory-pair metadata.

## Methodology

The project introduces **MuCO** (Multi-Constraint Optimizer), a planner that plans directly in continuous 3D space. It jointly enforces target visibility, viewpoint quality, collision avoidance, smoothness, and kinematic feasibility, avoiding the artifacts of grid-based planners.

## Model Description

The checkpoints in this repository include various vision-language models (VLMs) fine-tuned on the CosFly-Track dataset to act as dynamic target-following agents. Evaluated architectures include:
- Qwen (Qwen3-VL, Qwen3.5)
- InternVL
- GLM-4V
- Gemma 4

Fine-tuning on CosFly-Track significantly improves tracking performance (SR@1 meter) compared to zero-shot baselines, supporting the use of this dataset for training robust autonomous agents.

## Citation

```bibtex
@article{cosflytrack2026,
  title={CosFly-Track: A Large-Scale Multi-Modal Dataset for UAV Visual Tracking via Multi-Constraint Trajectory Optimization},
  author={Anonymous Authors},
  journal={arXiv preprint arXiv:2605.17776},
  year={2026}
}
```