MERaLiON/MERaLiON-2-10B-GGUF

GGUF quantized models of MERaLiON-2-10B for use with llama.cpp.

File	Size	Description
meralion-10b-decoder-q8_0.gguf	9.2 GB	Decoder, Q8_0
meralion-10b-decoder-q4_k_m.gguf	5.4 GB	Decoder, Q4_K_M
meralion-10b-mmproj-f16.gguf	1.7 GB	Whisper encoder + MLP adaptor, fp16

Both decoder files require the mmproj file. MERaLiON support is in llama.cpp as of build b8762 (#21756).

Build

# llama.cpp b8762 or later (recommended)
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --target llama-server llama-mtmd-cli -j$(nproc)

For older llama.cpp versions (pre-b8762), apply the patch first:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
curl -L "https://huggingface.co/MERaLiON/MERaLiON-2-10B-GGUF/resolve/main/llama_cpp_meralion.patch" | git apply
cmake -B build -DGGML_CUDA=ON
cmake --build build --target llama-server llama-mtmd-cli -j$(nproc)

Usage

CLI

llama-mtmd-cli \
    -m meralion-10b-decoder-q8_0.gguf \
    --mmproj meralion-10b-mmproj-f16.gguf \
    --audio input.wav \
    -p "Instruction: Please transcribe this speech.\nFollow the text instruction based on the following audio: <__media__>" \
    -ngl 99

Server

llama-server \
    -m meralion-10b-decoder-q8_0.gguf \
    --mmproj meralion-10b-mmproj-f16.gguf \
    --port 8090 -ngl 99 --host 0.0.0.0

curl http://localhost:8090/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":[
        {"type":"text","text":"Instruction: Please transcribe this speech.\nFollow the text instruction based on the following audio: "},
        {"type":"input_audio","input_audio":{"data":"<base64_wav>","format":"wav"}}
      ]}], "temperature":0.0}'

Multilingual

For best results on Chinese, Malay, and Tamil, enable DRY sampling:

# CLI
llama-mtmd-cli ... --dry-multiplier 1.0 --dry-allowed-length 1

# Server (add to request body)
"dry_multiplier": 1.0, "dry_allowed_length": 1

---

🔥 MERaLiON-2 🔥

🚀 MERaLiON-2-10B | 🚀 MERaLiON-2-10B-ASR | 🚀 MERaLiON-2-3B

💻 Web Demo | ⚙️ vLLM

Introduction

We are pleased to announce the release of MERaLiON2, the latest addition to the MERaLiON family of speech-text large language models. Our flagship model, MERaLiON-2-10B, demonstrates competitive performance across benchmark evaluations in tasks such as multilingual automatic speech recognition (ASR), speech translation (ST), audio scene understanding, emotion recognition, and general speech comprehension. These results are comparable to those achieved by other state-of-the-art open-source AudioLLMs, including Qwen2.5-Omni-7B and Phi-4-multimodal-instruct.

MERaLiON-2-10B is specifically designed to follow complex instructions with a nuanced understanding of Singapore's multilingual and multicultural context. It integrates a localized Whisper-large-v3 speech encoder and Gemma-2-9b text decoder.

In addition, we introduce an ASR-optimized variant, MERaLiON-2-10B-ASR, which delivers a 5–30% performance improvement over OpenAI's whisper-large-v3 on speech recognition tasks. This enhancement spans Singapore's 4 official languages—English, Mandarin, Malay, and Tamil—as well as 3 South-East Asian languages: Indonesian, Thai, and Vietnamese. The model also demonstrates robust handling of code-switching scenarios and local colloquialisms, reflecting its adaptability to Singapore's diverse linguistic landscape.

We also provide MERaLiON-2-3B that balances performance with reduced computational requirements, enabling broader accessibility and lightweight deployment.

Key Features

• Extended Audio Length: Support audio inputs up to 300 seconds (5 minutes) for audio & speech question answering tasks, 30s for a satisfactory performance for speech transcription (ASR) and speech translation (ST) tasks.

• Expanded Language Coverage: In addition to English, Chinese, and Singlish, V2 introduces support for Malay, Tamil, and other South-East Asia languages including Indonesian, Thai, and Vietnamese.

• Improved Performance: Achieves higher performance across a wide range of tasks.

• Higher Quality Training Data: Trained on 120,000 hours of curated speech and audio data, filtered for quality and diversity, with an emphasis on local and multilingual audio sources.

• Three Model Variants: Available in general-purpose (MERaLiON-2-10B), ASR-optimized (MERaLiON-2-10B-ASR) and light-weight (MERaLiON-2-3B) configurations to balance latency, compute efficiency, and task performance across different deployment needs.

Model Description

MERaLiON stands for Multimodal Empathetic Reasoning and Learning in One Network.

MERaLiON-2 is a family of Speech-Text Large Language Models tailored for Singapore's multilingual and multicultural landscape, as well as the wider Southeast Asian region. The 10B model integrates a localized Whisper-Large-V3 speech encoder with the Gemma2-9b-IT text decoder. The 3B model integrates a localized Whisper-Large-V3 speech encoder with the Gemma2-2b-IT text decoder.

MERaLiON-2-10B is finetuned on 120,000 hours of speech and audio data across 6 diverse tasks: Automatic Speech Recognition (ASR), Spoken Question Answering (SQA), Spoken Dialogue Summarization (SDS), Audio Captioning (AC), Audio-Scene Question Answering (ASQA) and Paralinguistic Question Answering (PQA). The model supports long-form audio inputs of up to 300 seconds (5 minutes) and is specifically adapted to handle the linguistic nuances, accents, and dialects commonly found across Singapore and neighboring countries.

Model Details

• Developed by: I2R, A*STAR, Singapore
• Model type: Multimodal LLM
• Language(s): Primarily English (Global and Singapore), Chinese, with support for audio of regional languages including Malay, Tamil, Indonesian, Thai, and Vietnamese.
• Audio: Mono channel audio, 16000 hz, up to 300 seconds.
• License: MERaLiON Public License
• Demo: MERaLiON-AudioLLM Web Demo

MERaLiON-2 is an upgraded version of MERaLiON-AudioLLM.

Performance

We benchmark MERaLiON-2 series models with extended AudioBench benchmark against several recently released open-source multimodal models — SALMONN-7B, Qwen2.5-Omni series and Phi-4-Multimodal — as well as two cascade models.

Better Automatic Speech Recognition (ASR) Accuracy

MERaLiON-2-10B-ASR and MERaLiON-2-10B demonstrate leading performance in Singlish, Mandarin, Malay, Tamil, and other Southeast Asian languages, while maintaining competitive results in English compared to Whisper-large-v3. The following table shows the average transcription Word Error Rate by language. The Private Dataset includes a collection of Singapore's locally accented speeches with code-switch. Please visit AudioBench benchmark for dataset-level evaluation results.

Language	MERaLiON-2-10B-ASR	MERaLiON-2-10B	MERaLiON-2-3B	whisper_large_v3
Thai	0.096526	0.109365	0.107279	0.121073
Tamil	0.271279	0.327081	0.344081	0.441483
Singlish	0.129830	0.168813	0.180395	0.248945
Malay	0.194638	0.209074	0.279891	0.219692
English	0.078544	0.088259	0.122295	0.080841
Indonesian	0.121020	0.142813	0.131950	0.137102
Mandarin	0.103694	0.132025	0.145878	0.170980
Vietnamese	0.118693	0.134808	0.155110	0.148474
Private Dataset	0.106150	0.112360	0.147258	0.116630

Better Instruction Following and Audio Understanding

MERaLiON-2-10B exhibits substantial advancements in speech and audio understanding, as well as paralinguistic tasks. Notably, it adeptly handles complex instructions and responds with enhanced flexibility, effectively preserving the pre-trained knowledge from Gemma during the audio fine-tuning process. This capability enables MERaLiON-2-10B to provide detailed explanations regarding speech content and the speaker's emotional state.

How to Use

Out of Scope use: This model is not intended for use in tool calling, math, and coding tasks.

MERaLiON-2 requires transformers version 4.50.1

pip install transformers==4.50.1
pip install librosa

Audio Input

• For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
• For general speech & audio understanding tasks, the maximum audio length is suggested to be 300 seconds at 16,000 Hz sampling rate.

Text Prompt

MERaLiON-2 is trained with this prompt template:

Instruction: <TextHere> \nFollow the text instruction based on the following audio: <SpeechHere>

It is generally recommended to follow this template, i.e., replace <TextHere> with your text instruction while leaving the <SpeechHere> untouched.

Standard prompts for better accuracy

prompt_template = "Instruction: {query} \nFollow the text instruction based on the following audio: <SpeechHere>"

transcription_prompt = prompt_template.format(query="Please transcribe this speech.")
translation_prompt = prompt_template.format(query="Please translate the speech into Malay")
summarization_prompt = prompt_template.format(query="Please summarize this speech")
audio_captioning_prompt_1 = prompt_template.format(query="Please describe the audio")
audio_captioning_prompt_2 = prompt_template.format(query="Please create a caption for the audio")
audio_scene_understanding_prompt = prompt_template.format(query="Is there people crying in the audio?")
speech_as_instruction_prompt = prompt_template.format(query="Please respond to the audio")
emotion_recognition_prompt_1 = prompt_template.format(query="What is the emotion of the speaker")
emotion_recognition_prompt_2 = prompt_template.format(query="Describe the paralinguistics feature of the audio")
gender_recognition_prompt = prompt_template.format(query="What is the gender of the speaker")

More flexible prompts for enriched responses

prompt_template = "Instruction: {query} \nFollow the text instruction based on the following audio: <SpeechHere>"

prompt_1 = prompt_template.format(query="describe the paralinguistics feature and return in json format.")
prompt_2 = prompt_template.format(query="Please summarise the content of the speech and analyse the paralinguistics features of this audio. Return in json format.")
prompt_3 = prompt_template.format(query="Please translate this speech to Singapore's 4 official languages.")

AI agent prompts (beyond the default prompt template)

prompt_1 = """
Your are MERaLiON-AudioLLM, an empathic AI assistant developed by A*STAR. MERaLiON stands for Multimodal Empathetic Reasoning and Learning in One Network.
You are a friendly and empathetic conversational partner, and is proficient in understanding human's emotion, accent, and gender from paralinguistic features.
Maintain a tone that is warm, non-judgmental, and supportive while replying to user. 

User's voice:  <SpeechHere>
"""

Huggingface Inference with CPU

import librosa
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor

repo_id = "MERaLiON/MERaLiON-2-10B"

processor = AutoProcessor.from_pretrained(
    repo_id, 
    trust_remote_code=True,
    )
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    repo_id,
    use_safetensors=True,
    trust_remote_code=True,
)

prompt_template = "Instruction: {query} \nFollow the text instruction based on the following audio: <SpeechHere>"
transcribe_prompt = "Please transcribe this speech."
translate_prompt = "Can you please translate this speech into written Chinese?"

# batch inference of 2 samples
conversation = [
    [{"role": "user", "content": prompt_template.format(query=transcribe_prompt)}],
    [{"role": "user", "content": prompt_template.format(query=translate_prompt)}],
]

chat_prompt = processor.tokenizer.apply_chat_template(
    conversation=conversation,
    tokenize=False,
    add_generation_prompt=True
)

# Use audio at 16000hz.
audio_array, sample_rate = librosa.load("/path/to/your/audio/file", sr=16000)
audio_array = [audio_array]*2
inputs = processor(text=chat_prompt, audios=audio_array)

# adjust the `max_new_tokens` based on your use case.
outputs = model.generate(**inputs, max_new_tokens=256)
generated_ids = outputs[:, inputs['input_ids'].size(1):]
response = processor.batch_decode(generated_ids, skip_special_tokens=True)

Huggingface GPU Inference

import torch
import librosa
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor

repo_id = "MERaLiON/MERaLiON-2-10B"
device = "cuda"

processor = AutoProcessor.from_pretrained(
    repo_id, 
    trust_remote_code=True,
    )
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    repo_id,
    use_safetensors=True,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16
).to(device)

prompt_template = "Instruction: {query} \nFollow the text instruction based on the following audio: <SpeechHere>"
transcribe_prompt = "Please transcribe this speech."
translate_prompt = "Can you please translate this speech into written Chinese?"

# batch inference of 2 samples
conversation = [
    [{"role": "user", "content": prompt_template.format(query=transcribe_prompt)}],
    [{"role": "user", "content": prompt_template.format(query=translate_prompt)}],
]

chat_prompt = processor.tokenizer.apply_chat_template(
    conversation=conversation,
    tokenize=False,
    add_generation_prompt=True
)

# Use audio at 16000hz.
audio_array, sample_rate = librosa.load("/path/to/your/audio/file", sr=16000)
audio_array = [audio_array]*2
inputs = processor(text=chat_prompt, audios=audio_array)

for key, value in inputs.items():
    if isinstance(value, torch.Tensor):
        inputs[key] = inputs[key].to(device)

        if value.dtype == torch.float32:
            inputs[key] = inputs[key].to(torch.bfloat16)

# adjust the `max_new_tokens` based on your use case.
outputs = model.generate(**inputs, max_new_tokens=256)
generated_ids = outputs[:, inputs['input_ids'].size(1):]
response = processor.batch_decode(generated_ids, skip_special_tokens=True)

⚠️ Disclaimer

The current MERaLiON-2 has not been specifically aligned for safety and may generate content that is inappropriate, offensive, or harmful. Developers and users are responsible for performing their own safety fine-tuning and implementing necessary security measures. The authors shall not be held liable for any claims, damages, or other liabilities arising from the use of the released models, weights, or code.

Compute and Infrastructure

MERaLiON-2 was trained on the ASPIRE 2A+ Supercomputer Cluster, provided by National Supercomputing Centre (NSCC), Singapore. ASPIRE 2A+ cluster provides multiple H100 nodes, with each compute node equipped with 8 Nvidia H100 GPUs, 2 TB of RAM, and 30 TB of locally attached NVMe storage. These nodes are interconnected via a rail-optimised, full fat-tree topology, utilising 400 Gb/s NDR InfiniBand cables. Additionally, the cluster incorporates a 2.5 PB SSD-based Lustre file system, linked to the H100 nodes through high-speed InfiniBand connections.

With a global batch size of 768, we trained the current release of MERaLiON-2 for around 200k steps, which took around 2 days to complete using 16 nodes, 128 H100 GPUs.

📚 Citation

If you find our work useful, please cite our papers:

@misc{he2024meralionaudiollmtechnicalreport,
      title={MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models}, 
      author={{MERaLiON Team}},
      year={2024},
      eprint={2412.09818},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.09818}, 
}

@article{wang2024audiobench,
    title={AudioBench: A Universal Benchmark for Audio Large Language Models},
    author={Wang, Bin and Zou, Xunlong and Lin, Geyu and Sun, Shuo and Liu, Zhuohan and Zhang, Wenyu and Liu, Zhengyuan and Aw, AiTi and Chen, Nancy F},
    journal={NAACL},
    year={2025}
    }

@article{wang2025advancing,
    title={Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models},
    author={Wang, Bin and Zou, Xunlong and Sun, Shuo and Zhang, Wenyu and He, Yingxu and Liu, Zhuohan and Wei, Chengwei and Chen, Nancy F and Aw, AiTi},
    journal={arXiv preprint arXiv:2501.01034},
    year={2025}
    }

@article{zhang2024mowe,
    title={MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders},
    author={Zhang, Wenyu and Sun, Shuo and Wang, Bin and Zou, Xunlong and Liu, Zhuohan and He, Yingxu and Lin, Geyu and Chen, Nancy F and Aw, Ai Ti},
    journal={ICASSP},
    year={2025}
    }

@misc{huang2025meraliontextllmcrosslingualunderstandinglarge,
      title={MERaLiON-TextLLM: Cross-Lingual Understanding of Large Language Models in Chinese, Indonesian, Malay, and Singlish}, 
      author={Xin Huang and Tarun Kumar Vangani and Minh Duc Pham and Xunlong Zou and Bin Wang and Zhengyuan Liu and Ai Ti Aw},
      year={2025},
      eprint={2501.08335},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2501.08335}, 
}