Initial commit: Upload AudioMCQ-Weak-To-Strong model

.gitattributes

@@ -0,0 +1,4 @@
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,97 @@
+---
+tags:
+- audio
+---
+
+# AudioMCQ-Weak-to-Strong
+
+<div align="center">
+
+[![arXiv](https://img.shields.io/badge/arXiv-2509.21060-b31b1b.svg)](https://arxiv.org/abs/2509.21060)
+[![Dataset](https://img.shields.io/badge/🤗%20Dataset-AudioMCQ-blue)](https://huggingface.co/datasets/inclusionAI/AudioMCQ)
+[![DCASE 2025](https://img.shields.io/badge/DCASE%202025-1st%20Place-gold.svg)](https://dcase.community/challenge2025/task-audio-question-answering-results)
+
+</div>
+
+## Overview
+
+This repository contains the **Weak-to-Strong** model checkpoint from our paper "Measuring Audio's Impact on Correctness: Audio-Contribution-Aware Post-Training of Large Audio Language Models". This model demonstrates state-of-the-art performance on audio question-answering benchmarks through our novel audio-contribution-aware post-training approach.
+
+## Training Paradigm
+
+The **Weak-to-Strong** training paradigm follows a two-stage approach:
+
+```
+Stage 1: SFT on weak audio-contribution data
+Stage 2: GRPO (RL) on strong audio-contribution data
+```
+
+This paradigm begins with supervised fine-tuning on samples with weak audio contribution (where visual or textual cues provide substantial information), then applies reinforcement learning on challenging strong audio-contribution samples to enhance audio-specific understanding capabilities.
+
+## Model Details
+
+- **Base Model**: Qwen2.5-Omni
+- **Training Data**: [AudioMCQ Dataset](https://huggingface.co/datasets/inclusionAI/AudioMCQ) (571k samples)
+- **Training Stages**: 
+  - Stage 1 (SFT): Weak audio-contribution subset
+  - Stage 2 (GRPO): Strong audio-contribution subset
+- **System Prompt**: "You are an audio understanding model that answers multiple choice questions based on audio content."
+
+## Usage
+
+Our model loading and usage methods are identical to those of Qwen2.5-Omni. Please refer to the [official documentation](https://github.com/QwenLM/Qwen2.5-Omni).
+
+### Input Format
+
+The evaluation input prompt structure is:
+
+```
+[Question] Please choose the answer from the following options: [''Option1'', ''Option2'', ''Option3'', ''Option4'']. Output the final answer in <answer> </answer>.
+```
+
+### Example Usage
+
+```python
+# Load model following Qwen2.5-Omni documentation
+# Apply system prompt: "You are an audio understanding model that answers multiple choice questions based on audio content."
+# Format your question with the input structure above
+```
+
+## Performance
+
+The Weak-to-Strong model achieves competitive performance across multiple benchmarks:
+
+- **MMAU-test-mini**: Strong accuracy on general audio understanding
+- **MMAR**: Robust performance on music understanding tasks
+- **MMSU**: Solid results on speech understanding
+- **Strong Audio-Contribution Splits**: Enhanced performance on challenging samples requiring deep audio understanding
+
+For detailed performance metrics and comparisons, please refer to our paper.
+
+## Related Resources
+
+- **AudioMCQ Dataset**: [https://huggingface.co/datasets/inclusionAI/AudioMCQ](https://huggingface.co/datasets/inclusionAI/AudioMCQ)
+- **Mixed-to-Strong Checkpoint**: [https://huggingface.co/inclusionAI/AudioMCQ-Mixed-To-Strong](https://huggingface.co/inclusionAI/AudioMCQ-Mixed-To-Strong)
+- **Paper**: [arXiv:2509.21060](https://arxiv.org/abs/2509.21060)
+- **DCASE 2025 Challenge**: [http://dcase.community/challenge2025/](http://dcase.community/challenge2025/)
+
+## Citation
+
+If you find this model useful in your research, please cite:
+
+```bibtex
+@article{he2025audiomcq,
+  title={Measuring Audio's Impact on Correctness: Audio-Contribution-Aware Post-Training of Large Audio Language Models},
+  author={He, Haolin and others},
+  journal={arXiv preprint arXiv:2509.21060},
+  year={2025}
+}
+```
+
+## Contact
+
+- **Haolin He**: [harlandzzc@link.cuhk.edu.hk](mailto:harlandzzc@link.cuhk.edu.hk)
+
+## Acknowledgements
+
+We thank the organizers of DCASE 2025 and the research community for their valuable feedback and support.

added_tokens.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e81a2cc3bd867a1217019eed202d1d8a07e1063ece716f22060dda14f6cc07d8
+size 579

chat_template.jinja

@@ -0,0 +1,7 @@
+{% set audio_count = namespace(value=0) %}{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
+You are a helpful assistant.<|im_end|>
+{% endif %}<|im_start|>{{ message['role'] }}
+{% if message['content'] is string %}{{ message['content'] }}<|im_end|>
+{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_bos|><|IMAGE|><|vision_eos|>{% elif content['type'] == 'audio' or 'audio' in content or 'audio_url' in content %}{% set audio_count.value = audio_count.value + 1 %}{% if add_audio_id %}Audio {{ audio_count.value }}: {% endif %}<|audio_bos|><|AUDIO|><|audio_eos|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_bos|><|VIDEO|><|vision_eos|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
+{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ecb869394fd6257968ab5044c2da691b2da99b6c1a8dedf5e3fca4a602d09df
+size 15303

generation_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f33605337b27f9386034250fe0c5dceb19c54759b69d1a4d226fefbc54cbe783
+size 69

model-00001-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce2af809b0d5104e43be063c8e1c0090abed3e1542d5179829b3dc52fc0a151c
+size 4985055536

model-00002-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a18a93ee8b23096559e1002d2259d902d0caaff9b930cb70629a48642c2cdd5
+size 4991496832

model-00003-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:622a3fbcf73c2a0dc66ddfb19d1973f866e7725525e99e6561b4abeaff2c4be4
+size 4991496936

model-00004-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cce5f05fa112fbfcc2472b6c97c43bb2f84a8b9515dc93ec128f5fa7ef52ad62
+size 4969489856

model-00005-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ac1e7daca22acb04383f454edd83826fee7c30cac603f2cc81431f5201b2f12
+size 1527220552

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89437e1f8d54819a994897ca7fcdc235f3d2b51fd310c8ebc73b0e1a2c50f447
+size 233161

preprocessor_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b47055ce61463ce143e9aab741d55c0aa520801a0a5d63be73c5b17cecb6bc69
+size 667

special_tokens_map.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db7fd39f5dc9ee37998c3ed04e3a3386989182a550064d2a2a9af16822cf22f4
+size 833

spk_dict.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a05609b28f5d42b7b748f0f07592545c8f1f6885b9ae8fff64baf56e86b2a18
+size 259544

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8441917e39ae0244e06d704b95b3124795cec478e297f9afac39ba670d7e9d99
+size 11421870

tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23ec41e1562bab95487376c350e16e72a1b5e1e14a33f88e3c212b17a57f72c1
+size 5160

vocab.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910
+size 2776833