Add files using upload-large-folder tool

.gitattributes

@@ -1,35 +1 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) XiaomiMiMo
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,5 +1,11 @@
 ---
 license: mit
+base_model: XiaomiMiMo/MiMo-Audio-Tokenizer
+tags:
+- mlx
+- speech
+- audio-tokenizer
+- automatic-speech-recognition
 ---
 <div align="center">
   <picture>
@@ -39,6 +45,14 @@ license: mit
 
 <br/>
 
+## MLX Conversion
+
+This repository is the MLX export used by `mlx-community/MiMo-V2.5-ASR-MLX`.
+
+- Default precision is `fp32`.
+- This export keeps the encoder and RVQ path used by MiMo ASR.
+- Decoder and vocoder weights are omitted here because they are not used in the ASR pipeline.
+
 ## Introduction
 
 Existing audio language models typically rely on task-specific fine-tuning to accomplish particular audio tasks. In contrast, humans are able to generalize to new audio tasks with only a few examples or simple instructions. GPT-3 has shown that scaling next-token prediction pretraining enables strong generalization capabilities in text, and we believe this paradigm is equally applicable to the audio domain. By scaling MiMo-Audio's pretraining data to over one hundred million of hours, we observe the emergence of few-shot learning capabilities across a diverse set of audio tasks. We develop a systematic evaluation of these capabilities and find that MiMo-Audio-7B-Base achieves SOTA performance on both speech intelligence and audio understanding benchmarks among open-source models. Beyond standard metrics, MiMo-Audio-7B-Base generalizes to tasks absent from its training data, such as voice conversion, style transfer, and speech editing. MiMo-Audio-7B-Base also demonstrates powerful speech continuation capabilities, capable of generating highly realistic talk shows, recitations, livestreaming and debates. At the post-training stage, we curate a diverse instruction-tuning corpus and introduce thinking mechanisms into both audio understanding and generation. MiMo-Audio-7B-Instruct achieves open-source SOTA on audio understanding benchmarks, spoken dialogue benchmarks and instruct-TTS evaluations, approaching or surpassing closed-source models.
@@ -47,10 +61,10 @@ Existing audio language models typically rely on task-specific fine-tuning to ac
   <img width="95%" src="https://github.com/XiaomiMiMo/MiMo-Audio/blob/main/assets/Results.png?raw=true">
 </p>
 
-
-
 ## Architecture
+
 ### MiMo-Audio-Tokenizer
+
 MiMo-Audio-Tokenizer is a 1.2B-parameter Transformer operating at 25 Hz. It employs an eight-layer RVQ stack to generate 200 tokens per second. By jointly optimizing semantic and reconstruction objectives, we train MiMo-Audio-Tokenizer from scratch on a 10-million-hour corpus, achieving superior reconstruction quality and facilitating downstream language modeling.
 
 <p align="center">
@@ -58,38 +72,42 @@ MiMo-Audio-Tokenizer is a 1.2B-parameter Transformer operating at 25 Hz. It empl
 </p>
 
 MiMo-Audio couples a patch encoder, an LLM, and a patch decoder to improve modeling efficiency for high-rate sequences and bridge the length mismatch between speech and text. The patch encoder aggregates four consecutive time steps of RVQ tokens into a single patch, downsampling the sequence to a 6.25 Hz representation for the LLM. The patch decoder autoregressively generates the full 25 Hz RVQ token sequence via a delayed-generation scheme.
+
 ### MiMo-Audio
+
 <p align="center">
   <img width="95%" src="https://github.com/XiaomiMiMo/MiMo-Audio/blob/main/assets/architecture.png?raw=true">
 </p>
 
-##  Explore MiMo-Audio Now! 🚀🚀🚀
+## Explore MiMo-Audio Now! 🚀🚀🚀
+
 - 🎧 **Try the Hugging Face demo:** [MiMo-Audio Demo](https://huggingface.co/spaces/XiaomiMiMo/mimo_audio_chat)
 - 📰 **Read the Official Blog:** [MiMo-Audio Blog](https://xiaomimimo.github.io/MiMo-Audio-Demo)
 - 📄 **Dive into the Technical Report:** [MiMo-Audio Technical Report](https://github.com/XiaomiMiMo/MiMo-Audio/blob/main/MiMo-Audio-Technical-Report.pdf)
 
-
 ## Model Download
+
 | Models   | 🤗 Hugging Face |
 |-------|-------|
 | MiMo-Audio-Tokenizer | [XiaomiMiMo/MiMo-Audio-Tokenizer](https://huggingface.co/XiaomiMiMo/MiMo-Audio-Tokenizer) |
 | MiMo-Audio-7B-Base | [XiaomiMiMo/MiMo-Audio-7B-Base](https://huggingface.co/XiaomiMiMo/MiMo-Audio-7B-Base) |
 | MiMo-Audio-7B-Instruct | [XiaomiMiMo/MiMo-Audio-7B-Instruct](https://huggingface.co/XiaomiMiMo/MiMo-Audio-7B-Instruct) |
 
-
-
 ## Getting Started
 
 Spin up the MiMo-Audio demo in minutes with the built-in Gradio app.
 
 ### Installation
-``` sh
+
+```sh
 git clone https://github.com/XiaomiMiMo/MiMo-Audio.git
 cd MiMo-Audio
 pip install -e .
 ```
+
 ### Run the demo
-``` sh
+
+```sh
 python run_mimo_audio.py
 ```
 
@@ -104,21 +122,29 @@ Enter the local paths for `MiMo-Audio-Tokenizer` and `MiMo-Audio-7B-Instruct`, t
 ## Inference Scripts
 
 ### Base Model
+
 We provide an example script to explore the **in-context learning** capabilities of `MiMo-Audio-7B-Base`.  
 See: [`inference_example_pretrain.py`](https://github.com/XiaomiMiMo/MiMo-Audio/blob/main/inference_example_pretrain.py)
 
 ### Instruct Model
+
 To try the instruction-tuned model `MiMo-Audio-7B-Instruct`, use the corresponding inference script.  
 See: [`inference_example_sft.py`](https://github.com/XiaomiMiMo/MiMo-Audio/blob/main/inference_example_sft.py)
 
-
-
 ## Evaluation Toolkit
-Full evaluation suite are available at 🌐[MiMo-Audio-Eval](https://github.com/XiaomiMiMo/MiMo-Audio-Eval).
 
+Full evaluation suite are available at 🌐[MiMo-Audio-Eval](https://github.com/XiaomiMiMo/MiMo-Audio-Eval).
 
 This toolkit is designed to evaluate MiMo-Audio and other recent audio LLMs as mentioned in the paper. It provides a flexible and extensible framework, supporting a wide range of datasets, tasks, and models.
 
+## Validation
+
+This MLX export was validated locally with `mlx-audio-swift` and `MiMo-V2.5-ASR-MLX`.
+
+- Smoke samples: `intention.wav`, `conversational_a.wav`, `noisy_audio.wav`
+- Release precision: `fp32`
+- Lower-precision internal experiments were kept out of the Hub release to avoid frontend drift and naming ambiguity
+
 ## Citation
 
 ```bibtex
@@ -130,7 +156,6 @@ This toolkit is designed to evaluate MiMo-Audio and other recent audio LLMs as m
 }
 ```
 
-
 ## Contact
 
-Please contact us at [mimo@xiaomi.com](mailto:mimo@xiaomi.com) or open an issue if you have any questions.
+Please contact us at [mimo@xiaomi.com](mailto:mimo@xiaomi.com) or open an issue if you have any questions.

config.json

@@ -69,4 +69,4 @@
     40,
     10
   ]
-}
+}

mlx_manifest.json

@@ -0,0 +1,9 @@
+{
+  "format": "mlx-compatible-safetensors",
+  "model": "MiMo-Audio-Tokenizer",
+  "precision": "float32",
+  "weight_pattern": "model*.safetensors",
+  "index_file": "model.safetensors.index.json",
+  "config_file": "config.json",
+  "source_model_dir": "../MiMo-Audio-Tokenizer"
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2614f14d9d692bed0b695cc7283f8dd65c9beaf10376853c9b9888844745e847
-size 3906690080
+oid sha256:7ce3493b01ac8b2c873256cdfe26ad049ea521b290cf65532afcfb1c9661a5c1
+size 2575648345

model.safetensors.index.json

@@ -0,0 +1,518 @@
+{
+    "metadata": {
+        "total_size": 2575595648,
+        "total_parameters": 643898912
+    },
+    "weight_map": {
+        "conv1.bias": "model.safetensors",
+        "conv1.weight": "model.safetensors",
+        "conv2.bias": "model.safetensors",
+        "conv2.weight": "model.safetensors",
+        "down_sample.weight": "model.safetensors",
+        "down_sample_norm.bias": "model.safetensors",
+        "down_sample_norm.weight": "model.safetensors",
+        "layer_norm.bias": "model.safetensors",
+        "layer_norm.weight": "model.safetensors",
+        "layers.0.fc1.bias": "model.safetensors",
+        "layers.0.fc1.weight": "model.safetensors",
+        "layers.0.fc2.bias": "model.safetensors",
+        "layers.0.fc2.weight": "model.safetensors",
+        "layers.0.final_layer_norm.bias": "model.safetensors",
+        "layers.0.final_layer_norm.weight": "model.safetensors",
+        "layers.0.self_attn.k_proj.weight": "model.safetensors",
+        "layers.0.self_attn.out_proj.bias": "model.safetensors",
+        "layers.0.self_attn.out_proj.weight": "model.safetensors",
+        "layers.0.self_attn.q_proj.bias": "model.safetensors",
+        "layers.0.self_attn.q_proj.weight": "model.safetensors",
+        "layers.0.self_attn.v_proj.bias": "model.safetensors",
+        "layers.0.self_attn.v_proj.weight": "model.safetensors",
+        "layers.0.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.0.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.1.fc1.bias": "model.safetensors",
+        "layers.1.fc1.weight": "model.safetensors",
+        "layers.1.fc2.bias": "model.safetensors",
+        "layers.1.fc2.weight": "model.safetensors",
+        "layers.1.final_layer_norm.bias": "model.safetensors",
+        "layers.1.final_layer_norm.weight": "model.safetensors",
+        "layers.1.self_attn.k_proj.weight": "model.safetensors",
+        "layers.1.self_attn.out_proj.bias": "model.safetensors",
+        "layers.1.self_attn.out_proj.weight": "model.safetensors",
+        "layers.1.self_attn.q_proj.bias": "model.safetensors",
+        "layers.1.self_attn.q_proj.weight": "model.safetensors",
+        "layers.1.self_attn.v_proj.bias": "model.safetensors",
+        "layers.1.self_attn.v_proj.weight": "model.safetensors",
+        "layers.1.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.1.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.10.fc1.bias": "model.safetensors",
+        "layers.10.fc1.weight": "model.safetensors",
+        "layers.10.fc2.bias": "model.safetensors",
+        "layers.10.fc2.weight": "model.safetensors",
+        "layers.10.final_layer_norm.bias": "model.safetensors",
+        "layers.10.final_layer_norm.weight": "model.safetensors",
+        "layers.10.self_attn.k_proj.weight": "model.safetensors",
+        "layers.10.self_attn.out_proj.bias": "model.safetensors",
+        "layers.10.self_attn.out_proj.weight": "model.safetensors",
+        "layers.10.self_attn.q_proj.bias": "model.safetensors",
+        "layers.10.self_attn.q_proj.weight": "model.safetensors",
+        "layers.10.self_attn.v_proj.bias": "model.safetensors",
+        "layers.10.self_attn.v_proj.weight": "model.safetensors",
+        "layers.10.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.10.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.11.fc1.bias": "model.safetensors",
+        "layers.11.fc1.weight": "model.safetensors",
+        "layers.11.fc2.bias": "model.safetensors",
+        "layers.11.fc2.weight": "model.safetensors",
+        "layers.11.final_layer_norm.bias": "model.safetensors",
+        "layers.11.final_layer_norm.weight": "model.safetensors",
+        "layers.11.self_attn.k_proj.weight": "model.safetensors",
+        "layers.11.self_attn.out_proj.bias": "model.safetensors",
+        "layers.11.self_attn.out_proj.weight": "model.safetensors",
+        "layers.11.self_attn.q_proj.bias": "model.safetensors",
+        "layers.11.self_attn.q_proj.weight": "model.safetensors",
+        "layers.11.self_attn.v_proj.bias": "model.safetensors",
+        "layers.11.self_attn.v_proj.weight": "model.safetensors",
+        "layers.11.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.11.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.12.fc1.bias": "model.safetensors",
+        "layers.12.fc1.weight": "model.safetensors",
+        "layers.12.fc2.bias": "model.safetensors",
+        "layers.12.fc2.weight": "model.safetensors",
+        "layers.12.final_layer_norm.bias": "model.safetensors",
+        "layers.12.final_layer_norm.weight": "model.safetensors",
+        "layers.12.self_attn.k_proj.weight": "model.safetensors",
+        "layers.12.self_attn.out_proj.bias": "model.safetensors",
+        "layers.12.self_attn.out_proj.weight": "model.safetensors",
+        "layers.12.self_attn.q_proj.bias": "model.safetensors",
+        "layers.12.self_attn.q_proj.weight": "model.safetensors",
+        "layers.12.self_attn.v_proj.bias": "model.safetensors",
+        "layers.12.self_attn.v_proj.weight": "model.safetensors",
+        "layers.12.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.12.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.13.fc1.bias": "model.safetensors",
+        "layers.13.fc1.weight": "model.safetensors",
+        "layers.13.fc2.bias": "model.safetensors",
+        "layers.13.fc2.weight": "model.safetensors",
+        "layers.13.final_layer_norm.bias": "model.safetensors",
+        "layers.13.final_layer_norm.weight": "model.safetensors",
+        "layers.13.self_attn.k_proj.weight": "model.safetensors",
+        "layers.13.self_attn.out_proj.bias": "model.safetensors",
+        "layers.13.self_attn.out_proj.weight": "model.safetensors",
+        "layers.13.self_attn.q_proj.bias": "model.safetensors",
+        "layers.13.self_attn.q_proj.weight": "model.safetensors",
+        "layers.13.self_attn.v_proj.bias": "model.safetensors",
+        "layers.13.self_attn.v_proj.weight": "model.safetensors",
+        "layers.13.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.13.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.14.fc1.bias": "model.safetensors",
+        "layers.14.fc1.weight": "model.safetensors",
+        "layers.14.fc2.bias": "model.safetensors",
+        "layers.14.fc2.weight": "model.safetensors",
+        "layers.14.final_layer_norm.bias": "model.safetensors",
+        "layers.14.final_layer_norm.weight": "model.safetensors",
+        "layers.14.self_attn.k_proj.weight": "model.safetensors",
+        "layers.14.self_attn.out_proj.bias": "model.safetensors",
+        "layers.14.self_attn.out_proj.weight": "model.safetensors",
+        "layers.14.self_attn.q_proj.bias": "model.safetensors",
+        "layers.14.self_attn.q_proj.weight": "model.safetensors",
+        "layers.14.self_attn.v_proj.bias": "model.safetensors",
+        "layers.14.self_attn.v_proj.weight": "model.safetensors",
+        "layers.14.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.14.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.15.fc1.bias": "model.safetensors",
+        "layers.15.fc1.weight": "model.safetensors",
+        "layers.15.fc2.bias": "model.safetensors",
+        "layers.15.fc2.weight": "model.safetensors",
+        "layers.15.final_layer_norm.bias": "model.safetensors",
+        "layers.15.final_layer_norm.weight": "model.safetensors",
+        "layers.15.self_attn.k_proj.weight": "model.safetensors",
+        "layers.15.self_attn.out_proj.bias": "model.safetensors",
+        "layers.15.self_attn.out_proj.weight": "model.safetensors",
+        "layers.15.self_attn.q_proj.bias": "model.safetensors",
+        "layers.15.self_attn.q_proj.weight": "model.safetensors",
+        "layers.15.self_attn.v_proj.bias": "model.safetensors",
+        "layers.15.self_attn.v_proj.weight": "model.safetensors",
+        "layers.15.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.15.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.16.fc1.bias": "model.safetensors",
+        "layers.16.fc1.weight": "model.safetensors",
+        "layers.16.fc2.bias": "model.safetensors",
+        "layers.16.fc2.weight": "model.safetensors",
+        "layers.16.final_layer_norm.bias": "model.safetensors",
+        "layers.16.final_layer_norm.weight": "model.safetensors",
+        "layers.16.self_attn.k_proj.weight": "model.safetensors",
+        "layers.16.self_attn.out_proj.bias": "model.safetensors",
+        "layers.16.self_attn.out_proj.weight": "model.safetensors",
+        "layers.16.self_attn.q_proj.bias": "model.safetensors",
+        "layers.16.self_attn.q_proj.weight": "model.safetensors",
+        "layers.16.self_attn.v_proj.bias": "model.safetensors",
+        "layers.16.self_attn.v_proj.weight": "model.safetensors",
+        "layers.16.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.16.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.17.fc1.bias": "model.safetensors",
+        "layers.17.fc1.weight": "model.safetensors",
+        "layers.17.fc2.bias": "model.safetensors",
+        "layers.17.fc2.weight": "model.safetensors",
+        "layers.17.final_layer_norm.bias": "model.safetensors",
+        "layers.17.final_layer_norm.weight": "model.safetensors",
+        "layers.17.self_attn.k_proj.weight": "model.safetensors",
+        "layers.17.self_attn.out_proj.bias": "model.safetensors",
+        "layers.17.self_attn.out_proj.weight": "model.safetensors",
+        "layers.17.self_attn.q_proj.bias": "model.safetensors",
+        "layers.17.self_attn.q_proj.weight": "model.safetensors",
+        "layers.17.self_attn.v_proj.bias": "model.safetensors",
+        "layers.17.self_attn.v_proj.weight": "model.safetensors",
+        "layers.17.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.17.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.18.fc1.bias": "model.safetensors",
+        "layers.18.fc1.weight": "model.safetensors",
+        "layers.18.fc2.bias": "model.safetensors",
+        "layers.18.fc2.weight": "model.safetensors",
+        "layers.18.final_layer_norm.bias": "model.safetensors",
+        "layers.18.final_layer_norm.weight": "model.safetensors",
+        "layers.18.self_attn.k_proj.weight": "model.safetensors",
+        "layers.18.self_attn.out_proj.bias": "model.safetensors",
+        "layers.18.self_attn.out_proj.weight": "model.safetensors",
+        "layers.18.self_attn.q_proj.bias": "model.safetensors",
+        "layers.18.self_attn.q_proj.weight": "model.safetensors",
+        "layers.18.self_attn.v_proj.bias": "model.safetensors",
+        "layers.18.self_attn.v_proj.weight": "model.safetensors",
+        "layers.18.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.18.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.19.fc1.bias": "model.safetensors",
+        "layers.19.fc1.weight": "model.safetensors",
+        "layers.19.fc2.bias": "model.safetensors",
+        "layers.19.fc2.weight": "model.safetensors",
+        "layers.19.final_layer_norm.bias": "model.safetensors",
+        "layers.19.final_layer_norm.weight": "model.safetensors",
+        "layers.19.self_attn.k_proj.weight": "model.safetensors",
+        "layers.19.self_attn.out_proj.bias": "model.safetensors",
+        "layers.19.self_attn.out_proj.weight": "model.safetensors",
+        "layers.19.self_attn.q_proj.bias": "model.safetensors",
+        "layers.19.self_attn.q_proj.weight": "model.safetensors",
+        "layers.19.self_attn.v_proj.bias": "model.safetensors",
+        "layers.19.self_attn.v_proj.weight": "model.safetensors",
+        "layers.19.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.19.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.2.fc1.bias": "model.safetensors",
+        "layers.2.fc1.weight": "model.safetensors",
+        "layers.2.fc2.bias": "model.safetensors",
+        "layers.2.fc2.weight": "model.safetensors",
+        "layers.2.final_layer_norm.bias": "model.safetensors",
+        "layers.2.final_layer_norm.weight": "model.safetensors",
+        "layers.2.self_attn.k_proj.weight": "model.safetensors",
+        "layers.2.self_attn.out_proj.bias": "model.safetensors",
+        "layers.2.self_attn.out_proj.weight": "model.safetensors",
+        "layers.2.self_attn.q_proj.bias": "model.safetensors",
+        "layers.2.self_attn.q_proj.weight": "model.safetensors",
+        "layers.2.self_attn.v_proj.bias": "model.safetensors",
+        "layers.2.self_attn.v_proj.weight": "model.safetensors",
+        "layers.2.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.2.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.20.fc1.bias": "model.safetensors",
+        "layers.20.fc1.weight": "model.safetensors",
+        "layers.20.fc2.bias": "model.safetensors",
+        "layers.20.fc2.weight": "model.safetensors",
+        "layers.20.final_layer_norm.bias": "model.safetensors",
+        "layers.20.final_layer_norm.weight": "model.safetensors",
+        "layers.20.self_attn.k_proj.weight": "model.safetensors",
+        "layers.20.self_attn.out_proj.bias": "model.safetensors",
+        "layers.20.self_attn.out_proj.weight": "model.safetensors",
+        "layers.20.self_attn.q_proj.bias": "model.safetensors",
+        "layers.20.self_attn.q_proj.weight": "model.safetensors",
+        "layers.20.self_attn.v_proj.bias": "model.safetensors",
+        "layers.20.self_attn.v_proj.weight": "model.safetensors",
+        "layers.20.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.20.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.21.fc1.bias": "model.safetensors",
+        "layers.21.fc1.weight": "model.safetensors",
+        "layers.21.fc2.bias": "model.safetensors",
+        "layers.21.fc2.weight": "model.safetensors",
+        "layers.21.final_layer_norm.bias": "model.safetensors",
+        "layers.21.final_layer_norm.weight": "model.safetensors",
+        "layers.21.self_attn.k_proj.weight": "model.safetensors",
+        "layers.21.self_attn.out_proj.bias": "model.safetensors",
+        "layers.21.self_attn.out_proj.weight": "model.safetensors",
+        "layers.21.self_attn.q_proj.bias": "model.safetensors",
+        "layers.21.self_attn.q_proj.weight": "model.safetensors",
+        "layers.21.self_attn.v_proj.bias": "model.safetensors",
+        "layers.21.self_attn.v_proj.weight": "model.safetensors",
+        "layers.21.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.21.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.22.fc1.bias": "model.safetensors",
+        "layers.22.fc1.weight": "model.safetensors",
+        "layers.22.fc2.bias": "model.safetensors",
+        "layers.22.fc2.weight": "model.safetensors",
+        "layers.22.final_layer_norm.bias": "model.safetensors",
+        "layers.22.final_layer_norm.weight": "model.safetensors",
+        "layers.22.self_attn.k_proj.weight": "model.safetensors",
+        "layers.22.self_attn.out_proj.bias": "model.safetensors",
+        "layers.22.self_attn.out_proj.weight": "model.safetensors",
+        "layers.22.self_attn.q_proj.bias": "model.safetensors",
+        "layers.22.self_attn.q_proj.weight": "model.safetensors",
+        "layers.22.self_attn.v_proj.bias": "model.safetensors",
+        "layers.22.self_attn.v_proj.weight": "model.safetensors",
+        "layers.22.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.22.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.23.fc1.bias": "model.safetensors",
+        "layers.23.fc1.weight": "model.safetensors",
+        "layers.23.fc2.bias": "model.safetensors",
+        "layers.23.fc2.weight": "model.safetensors",
+        "layers.23.final_layer_norm.bias": "model.safetensors",
+        "layers.23.final_layer_norm.weight": "model.safetensors",
+        "layers.23.self_attn.k_proj.weight": "model.safetensors",
+        "layers.23.self_attn.out_proj.bias": "model.safetensors",
+        "layers.23.self_attn.out_proj.weight": "model.safetensors",
+        "layers.23.self_attn.q_proj.bias": "model.safetensors",
+        "layers.23.self_attn.q_proj.weight": "model.safetensors",
+        "layers.23.self_attn.v_proj.bias": "model.safetensors",
+        "layers.23.self_attn.v_proj.weight": "model.safetensors",
+        "layers.23.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.23.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.24.fc1.bias": "model.safetensors",
+        "layers.24.fc1.weight": "model.safetensors",
+        "layers.24.fc2.bias": "model.safetensors",
+        "layers.24.fc2.weight": "model.safetensors",
+        "layers.24.final_layer_norm.bias": "model.safetensors",
+        "layers.24.final_layer_norm.weight": "model.safetensors",
+        "layers.24.self_attn.k_proj.weight": "model.safetensors",
+        "layers.24.self_attn.out_proj.bias": "model.safetensors",
+        "layers.24.self_attn.out_proj.weight": "model.safetensors",
+        "layers.24.self_attn.q_proj.bias": "model.safetensors",
+        "layers.24.self_attn.q_proj.weight": "model.safetensors",
+        "layers.24.self_attn.v_proj.bias": "model.safetensors",
+        "layers.24.self_attn.v_proj.weight": "model.safetensors",
+        "layers.24.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.24.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.25.fc1.bias": "model.safetensors",
+        "layers.25.fc1.weight": "model.safetensors",
+        "layers.25.fc2.bias": "model.safetensors",
+        "layers.25.fc2.weight": "model.safetensors",
+        "layers.25.final_layer_norm.bias": "model.safetensors",
+        "layers.25.final_layer_norm.weight": "model.safetensors",
+        "layers.25.self_attn.k_proj.weight": "model.safetensors",
+        "layers.25.self_attn.out_proj.bias": "model.safetensors",
+        "layers.25.self_attn.out_proj.weight": "model.safetensors",
+        "layers.25.self_attn.q_proj.bias": "model.safetensors",
+        "layers.25.self_attn.q_proj.weight": "model.safetensors",
+        "layers.25.self_attn.v_proj.bias": "model.safetensors",
+        "layers.25.self_attn.v_proj.weight": "model.safetensors",
+        "layers.25.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.25.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.26.fc1.bias": "model.safetensors",
+        "layers.26.fc1.weight": "model.safetensors",
+        "layers.26.fc2.bias": "model.safetensors",
+        "layers.26.fc2.weight": "model.safetensors",
+        "layers.26.final_layer_norm.bias": "model.safetensors",
+        "layers.26.final_layer_norm.weight": "model.safetensors",
+        "layers.26.self_attn.k_proj.weight": "model.safetensors",
+        "layers.26.self_attn.out_proj.bias": "model.safetensors",
+        "layers.26.self_attn.out_proj.weight": "model.safetensors",
+        "layers.26.self_attn.q_proj.bias": "model.safetensors",
+        "layers.26.self_attn.q_proj.weight": "model.safetensors",
+        "layers.26.self_attn.v_proj.bias": "model.safetensors",
+        "layers.26.self_attn.v_proj.weight": "model.safetensors",
+        "layers.26.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.26.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.27.fc1.bias": "model.safetensors",
+        "layers.27.fc1.weight": "model.safetensors",
+        "layers.27.fc2.bias": "model.safetensors",
+        "layers.27.fc2.weight": "model.safetensors",
+        "layers.27.final_layer_norm.bias": "model.safetensors",
+        "layers.27.final_layer_norm.weight": "model.safetensors",
+        "layers.27.self_attn.k_proj.weight": "model.safetensors",
+        "layers.27.self_attn.out_proj.bias": "model.safetensors",
+        "layers.27.self_attn.out_proj.weight": "model.safetensors",
+        "layers.27.self_attn.q_proj.bias": "model.safetensors",
+        "layers.27.self_attn.q_proj.weight": "model.safetensors",
+        "layers.27.self_attn.v_proj.bias": "model.safetensors",
+        "layers.27.self_attn.v_proj.weight": "model.safetensors",
+        "layers.27.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.27.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.28.fc1.bias": "model.safetensors",
+        "layers.28.fc1.weight": "model.safetensors",
+        "layers.28.fc2.bias": "model.safetensors",
+        "layers.28.fc2.weight": "model.safetensors",
+        "layers.28.final_layer_norm.bias": "model.safetensors",
+        "layers.28.final_layer_norm.weight": "model.safetensors",
+        "layers.28.self_attn.k_proj.weight": "model.safetensors",
+        "layers.28.self_attn.out_proj.bias": "model.safetensors",
+        "layers.28.self_attn.out_proj.weight": "model.safetensors",
+        "layers.28.self_attn.q_proj.bias": "model.safetensors",
+        "layers.28.self_attn.q_proj.weight": "model.safetensors",
+        "layers.28.self_attn.v_proj.bias": "model.safetensors",
+        "layers.28.self_attn.v_proj.weight": "model.safetensors",
+        "layers.28.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.28.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.29.fc1.bias": "model.safetensors",
+        "layers.29.fc1.weight": "model.safetensors",
+        "layers.29.fc2.bias": "model.safetensors",
+        "layers.29.fc2.weight": "model.safetensors",
+        "layers.29.final_layer_norm.bias": "model.safetensors",
+        "layers.29.final_layer_norm.weight": "model.safetensors",
+        "layers.29.self_attn.k_proj.weight": "model.safetensors",
+        "layers.29.self_attn.out_proj.bias": "model.safetensors",
+        "layers.29.self_attn.out_proj.weight": "model.safetensors",
+        "layers.29.self_attn.q_proj.bias": "model.safetensors",
+        "layers.29.self_attn.q_proj.weight": "model.safetensors",
+        "layers.29.self_attn.v_proj.bias": "model.safetensors",
+        "layers.29.self_attn.v_proj.weight": "model.safetensors",
+        "layers.29.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.29.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.3.fc1.bias": "model.safetensors",
+        "layers.3.fc1.weight": "model.safetensors",
+        "layers.3.fc2.bias": "model.safetensors",
+        "layers.3.fc2.weight": "model.safetensors",
+        "layers.3.final_layer_norm.bias": "model.safetensors",
+        "layers.3.final_layer_norm.weight": "model.safetensors",
+        "layers.3.self_attn.k_proj.weight": "model.safetensors",
+        "layers.3.self_attn.out_proj.bias": "model.safetensors",
+        "layers.3.self_attn.out_proj.weight": "model.safetensors",
+        "layers.3.self_attn.q_proj.bias": "model.safetensors",
+        "layers.3.self_attn.q_proj.weight": "model.safetensors",
+        "layers.3.self_attn.v_proj.bias": "model.safetensors",
+        "layers.3.self_attn.v_proj.weight": "model.safetensors",
+        "layers.3.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.3.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.30.fc1.bias": "model.safetensors",
+        "layers.30.fc1.weight": "model.safetensors",
+        "layers.30.fc2.bias": "model.safetensors",
+        "layers.30.fc2.weight": "model.safetensors",
+        "layers.30.final_layer_norm.bias": "model.safetensors",
+        "layers.30.final_layer_norm.weight": "model.safetensors",
+        "layers.30.self_attn.k_proj.weight": "model.safetensors",
+        "layers.30.self_attn.out_proj.bias": "model.safetensors",
+        "layers.30.self_attn.out_proj.weight": "model.safetensors",
+        "layers.30.self_attn.q_proj.bias": "model.safetensors",
+        "layers.30.self_attn.q_proj.weight": "model.safetensors",
+        "layers.30.self_attn.v_proj.bias": "model.safetensors",
+        "layers.30.self_attn.v_proj.weight": "model.safetensors",
+        "layers.30.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.30.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.31.fc1.bias": "model.safetensors",
+        "layers.31.fc1.weight": "model.safetensors",
+        "layers.31.fc2.bias": "model.safetensors",
+        "layers.31.fc2.weight": "model.safetensors",
+        "layers.31.final_layer_norm.bias": "model.safetensors",
+        "layers.31.final_layer_norm.weight": "model.safetensors",
+        "layers.31.self_attn.k_proj.weight": "model.safetensors",
+        "layers.31.self_attn.out_proj.bias": "model.safetensors",
+        "layers.31.self_attn.out_proj.weight": "model.safetensors",
+        "layers.31.self_attn.q_proj.bias": "model.safetensors",
+        "layers.31.self_attn.q_proj.weight": "model.safetensors",
+        "layers.31.self_attn.v_proj.bias": "model.safetensors",
+        "layers.31.self_attn.v_proj.weight": "model.safetensors",
+        "layers.31.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.31.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.4.fc1.bias": "model.safetensors",
+        "layers.4.fc1.weight": "model.safetensors",
+        "layers.4.fc2.bias": "model.safetensors",
+        "layers.4.fc2.weight": "model.safetensors",
+        "layers.4.final_layer_norm.bias": "model.safetensors",
+        "layers.4.final_layer_norm.weight": "model.safetensors",
+        "layers.4.self_attn.k_proj.weight": "model.safetensors",
+        "layers.4.self_attn.out_proj.bias": "model.safetensors",
+        "layers.4.self_attn.out_proj.weight": "model.safetensors",
+        "layers.4.self_attn.q_proj.bias": "model.safetensors",
+        "layers.4.self_attn.q_proj.weight": "model.safetensors",
+        "layers.4.self_attn.v_proj.bias": "model.safetensors",
+        "layers.4.self_attn.v_proj.weight": "model.safetensors",
+        "layers.4.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.4.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.5.fc1.bias": "model.safetensors",
+        "layers.5.fc1.weight": "model.safetensors",
+        "layers.5.fc2.bias": "model.safetensors",
+        "layers.5.fc2.weight": "model.safetensors",
+        "layers.5.final_layer_norm.bias": "model.safetensors",
+        "layers.5.final_layer_norm.weight": "model.safetensors",
+        "layers.5.self_attn.k_proj.weight": "model.safetensors",
+        "layers.5.self_attn.out_proj.bias": "model.safetensors",
+        "layers.5.self_attn.out_proj.weight": "model.safetensors",
+        "layers.5.self_attn.q_proj.bias": "model.safetensors",
+        "layers.5.self_attn.q_proj.weight": "model.safetensors",
+        "layers.5.self_attn.v_proj.bias": "model.safetensors",
+        "layers.5.self_attn.v_proj.weight": "model.safetensors",
+        "layers.5.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.5.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.6.fc1.bias": "model.safetensors",
+        "layers.6.fc1.weight": "model.safetensors",
+        "layers.6.fc2.bias": "model.safetensors",
+        "layers.6.fc2.weight": "model.safetensors",
+        "layers.6.final_layer_norm.bias": "model.safetensors",
+        "layers.6.final_layer_norm.weight": "model.safetensors",
+        "layers.6.self_attn.k_proj.weight": "model.safetensors",
+        "layers.6.self_attn.out_proj.bias": "model.safetensors",
+        "layers.6.self_attn.out_proj.weight": "model.safetensors",
+        "layers.6.self_attn.q_proj.bias": "model.safetensors",
+        "layers.6.self_attn.q_proj.weight": "model.safetensors",
+        "layers.6.self_attn.v_proj.bias": "model.safetensors",
+        "layers.6.self_attn.v_proj.weight": "model.safetensors",
+        "layers.6.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.6.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.7.fc1.bias": "model.safetensors",
+        "layers.7.fc1.weight": "model.safetensors",
+        "layers.7.fc2.bias": "model.safetensors",
+        "layers.7.fc2.weight": "model.safetensors",
+        "layers.7.final_layer_norm.bias": "model.safetensors",
+        "layers.7.final_layer_norm.weight": "model.safetensors",
+        "layers.7.self_attn.k_proj.weight": "model.safetensors",
+        "layers.7.self_attn.out_proj.bias": "model.safetensors",
+        "layers.7.self_attn.out_proj.weight": "model.safetensors",
+        "layers.7.self_attn.q_proj.bias": "model.safetensors",
+        "layers.7.self_attn.q_proj.weight": "model.safetensors",
+        "layers.7.self_attn.v_proj.bias": "model.safetensors",
+        "layers.7.self_attn.v_proj.weight": "model.safetensors",
+        "layers.7.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.7.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.8.fc1.bias": "model.safetensors",
+        "layers.8.fc1.weight": "model.safetensors",
+        "layers.8.fc2.bias": "model.safetensors",
+        "layers.8.fc2.weight": "model.safetensors",
+        "layers.8.final_layer_norm.bias": "model.safetensors",
+        "layers.8.final_layer_norm.weight": "model.safetensors",
+        "layers.8.self_attn.k_proj.weight": "model.safetensors",
+        "layers.8.self_attn.out_proj.bias": "model.safetensors",
+        "layers.8.self_attn.out_proj.weight": "model.safetensors",
+        "layers.8.self_attn.q_proj.bias": "model.safetensors",
+        "layers.8.self_attn.q_proj.weight": "model.safetensors",
+        "layers.8.self_attn.v_proj.bias": "model.safetensors",
+        "layers.8.self_attn.v_proj.weight": "model.safetensors",
+        "layers.8.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.8.self_attn_layer_norm.weight": "model.safetensors",
+        "layers.9.fc1.bias": "model.safetensors",
+        "layers.9.fc1.weight": "model.safetensors",
+        "layers.9.fc2.bias": "model.safetensors",
+        "layers.9.fc2.weight": "model.safetensors",
+        "layers.9.final_layer_norm.bias": "model.safetensors",
+        "layers.9.final_layer_norm.weight": "model.safetensors",
+        "layers.9.self_attn.k_proj.weight": "model.safetensors",
+        "layers.9.self_attn.out_proj.bias": "model.safetensors",
+        "layers.9.self_attn.out_proj.weight": "model.safetensors",
+        "layers.9.self_attn.q_proj.bias": "model.safetensors",
+        "layers.9.self_attn.q_proj.weight": "model.safetensors",
+        "layers.9.self_attn.v_proj.bias": "model.safetensors",
+        "layers.9.self_attn.v_proj.weight": "model.safetensors",
+        "layers.9.self_attn_layer_norm.bias": "model.safetensors",
+        "layers.9.self_attn_layer_norm.weight": "model.safetensors",
+        "position_embedding.inv_freq": "model.safetensors",
+        "quantizer.vq.layers.0.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.1.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.10.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.11.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.12.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.13.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.14.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.15.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.16.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.17.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.18.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.19.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.2.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.3.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.4.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.5.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.6.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.7.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.8.codebook.embed": "model.safetensors",
+        "quantizer.vq.layers.9.codebook.embed": "model.safetensors"
+    }
+}