chaubeyG/AVERE-7B

AVERE: Improving Audiovisual Emotion Reasoning with Preference Optimization

ICLR 2026  ·  Rio de Janeiro, Brazil

---

Official checkpoint for the ICLR 2026 paper - AVERE: Improving Audiovisual Emotion Reasoning with Preference Optimization

📄 Abstract

Emotion understanding is essential for building socially intelligent agents. Although recent multimodal large language models (MLLMs) have shown strong performance on this task, two key challenges remain: (i) spurious associations between emotions and irrelevant audiovisual cues (reasoning errors) and (ii) hallucination of audiovisual cues (perception errors) driven by text priors in the language model backbone.

To quantify and understand these issues, we introduce EmoReAlM, a benchmark designed to evaluate MLLMs for cue–emotion associations, hallucinations, and modality agreement. We then propose AVEm-DPO, a preference optimization technique that aligns model responses with both audiovisual inputs and emotion-centric queries. Specifically, we construct preferences over (i) responses exhibiting spurious associations or hallucinations and (ii) audiovisual input pairs guided by textual prompts. We also include a regularization term that penalizes reliance on text priors, thereby mitigating modality-specific cue hallucinations.

Experimental results on DFEW, RAVDESS, and EMER demonstrate that our method significantly improves the performance of reference baseline models (6–19% relative improvement) in zero-shot settings.

---

📣 News

Date	Update
Mar. 2026	Official codebase is now public. Initial release includes evaluation scripts for EmoReAlM and other emotion benchmarks. Training code coming soon.
Mar. 2026	Model weights for AVERE-7B are live on 🤗 HuggingFace → chaubeyG/AVERE-7B.
Feb. 2026	Pleased to announce our CVPR 2026 paper on using DPO to mitigate cross-modal hallucinations in omni-LLMs → MoD-DPO.
Jan. 2026	EmoReAlM benchmark released on 🤗 HuggingFace → chaubeyG/EmoReAlM.
Jan. 2026	AVERE accepted to ICLR 2026. See you in Rio de Janeiro!

---

🏆 Results

Detailed results and the EmoReAlM leaderboard are available on the project website: avere-iclr.github.io

To submit your model to the EmoReAlM leaderboard, please contact the first author at achaubey@usc.edu.

---

🔧 Installation and Instructions

Please visit our GitHub repository - ihp-lab/AVERE for detailed instructions on how to use the model checkpoints for inference.

---

⚖️ License

This codebase is distributed under the USC Research License. See LICENSE.rst for details.

Portions of this codebase are derived from Vista-DPO and VideoLLaVA; those portions inherit their respective licenses.

---

🙌 Credits

AVERE builds upon the following excellent open-source works:

• VistaDPO
• VideoLLaVA

We gratefully acknowledge their contributions to the open-source community.

---

🪶 Citation

If you find AVERE or EmoReAlM useful in your research, please cite:

@inproceedings{chaubey2026avere,
  title     = {{AVERE}: Improving Audiovisual Emotion Reasoning with Preference Optimization},
  author    = {Ashutosh Chaubey and Jiacheng Pang and Maksim Siniukov and Mohammad Soleymani},
  booktitle = {The Fourteenth International Conference on Learning Representations},
  year      = {2026},
  url       = {https://openreview.net/forum?id=td682AAuPr}
}