Instructions to use AesSedai/MiMo-V2.5-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AesSedai/MiMo-V2.5-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AesSedai/MiMo-V2.5-GGUF",
	filename="IQ3_S/MiMo-V2.5-IQ3_S-00001-of-00004.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use AesSedai/MiMo-V2.5-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AesSedai/MiMo-V2.5-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use AesSedai/MiMo-V2.5-GGUF with Ollama:
```
ollama run hf.co/AesSedai/MiMo-V2.5-GGUF:Q4_K_M
```

Unsloth Studio new

How to use AesSedai/MiMo-V2.5-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/MiMo-V2.5-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/MiMo-V2.5-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AesSedai/MiMo-V2.5-GGUF to start chatting

Pi new

How to use AesSedai/MiMo-V2.5-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AesSedai/MiMo-V2.5-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "MiMo-V2.5-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use AesSedai/MiMo-V2.5-GGUF with Docker Model Runner:
```
docker model run hf.co/AesSedai/MiMo-V2.5-GGUF:Q4_K_M
```

Lemonade

How to use AesSedai/MiMo-V2.5-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AesSedai/MiMo-V2.5-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.MiMo-V2.5-GGUF-Q4_K_M

List all available models

lemonade list

MiMo-V2.5-GGUF / kld_data /llm_quantization_data.csv

AesSedai

Upload folder using huggingface_hub

b11fde4 verified 1 day ago

raw

history blame contribute delete

3.89 kB

	model_name,file_size_gb,bpw,Mean KLD_mean,0.1% KLD,0.1% Δp,1.0% KLD,1.0% Δp,10.0% KLD,10.0% Δp,25.0% Δp,5.0% KLD,5.0% Δp,75.0% Δp,90.0% KLD,90.0% Δp,95.0% KLD,95.0% Δp,99.0% KLD,99.0% Δp,99.9% KLD,99.9% Δp,"Cor(ln(PPL(Q)), ln(PPL(base)))",Maximum KLD,Maximum Δp,Mean KLD_std,Mean PPL(Q)-PPL(base)_mean,Mean PPL(Q)-PPL(base)_std,Mean PPL(Q)/PPL(base)_mean,Mean PPL(Q)/PPL(base)_std,Mean PPL(Q)_mean,Mean PPL(Q)_std,Mean PPL(base)_mean,Mean PPL(base)_std,Mean ln(PPL(Q)/PPL(base))_mean,Mean ln(PPL(Q)/PPL(base))_std,Mean Δp_mean,Mean Δp_std,Median KLD,Median Δp,Minimum KLD,Minimum Δp,RMS Δp_mean,RMS Δp_std,Same top p_mean,Same top p_std,common_fit_params,common_memory_breakdown_print,common_params_fit_impl,file_path,file_size_gib,ggml_cuda_init,kl_divergence,llama_context,llama_kv_cache,llama_kv_cache_iswa,llama_model_loader,llama_perf_context_print,load,load_tensors,print_info,sched_reserve,system_info
	MiMo-V2.5-IQ3_S (aes_sedai),113.09722632192,2.93,0.092639,3e-06,-75.975,2.2e-05,-40.943,0.000666,-9.422,-2.744,0.000178,-16.337,0.567,0.206719,4.586,0.357293,8.589,0.964171,20.425,2.502359,45.792,97.52,29.232412,99.969,0.000604,0.423635,0.007709,1.082595,0.001436,5.55271,0.033286,5.129074,0.030176,0.079361,0.001326,-1.749,0.025,0.035557,-0.079,-4e-05,-98.551,9.645,0.057,87.976,0.084,0.86,-8744880385.0,136619.773503,kld/MiMo-V2.5/wiki-test-raw-fused/aes_sedai/MiMo-V2.5-IQ3_S.md,105.33,8778000.0,584512819216.0,9.31,128.0,512.0,-294.0,35.0,0.9312,711834.12,256.0,106.304,4.848561200112841e+50
	MiMo-V2.5-IQ4_XS (aes_sedai),146.86640668672,3.8,0.041177,1e-06,-50.998,1e-05,-22.663,0.000299,-5.441,-1.464,8e-05,-9.213,0.522,0.091585,3.412,0.155445,6.307,0.432418,16.045,1.236805,38.308,98.91,29.674263,99.968,0.000349,0.142323,0.004635,1.027748,0.000895,5.271397,0.03117,5.129074,0.030176,0.02737,0.000871,-0.689,0.016,0.015269,-0.019,-3.3e-05,-96.693,6.221,0.044,91.926,0.071,0.81,-10186330385.0,168678.773503,kld/MiMo-V2.5/wiki-test-raw-fused/aes_sedai/MiMo-V2.5-IQ4_XS.md,136.78,8778000.0,584512819216.0,9.31,128.0,512.0,-447.0,35.0,0.9312,715366.24,256.0,106.694,4.848561200112841e+50
	MiMo-V2.5-Q4_K_M (aes_sedai),189.7301803008,4.92,0.02067,-1e-06,-34.311,4e-06,-14.712,0.000144,-3.207,-0.717,3.8e-05,-5.729,0.583,0.045064,2.884,0.076764,5.043,0.217674,11.789,0.682057,27.609,99.45,15.312425,98.975,0.000214,0.066274,0.00326,1.012921,0.00063,5.195349,0.030767,5.129074,0.030176,0.012839,0.000622,-0.166,0.011,0.007254,-0.002,-0.000147,-97.0,4.33,0.038,94.443,0.059,1.09,-10186330385.0,209558.773503,kld/MiMo-V2.5/wiki-test-raw-fused/aes_sedai/MiMo-V2.5-Q4_K_M.md,176.7,8778000.0,584512819216.0,9.31,128.0,512.0,-61.0,35.0,0.9312,719958.24,256.0,93.064,4.848561200112841e+50
	MiMo-V2.5-Q5_K_M (aes_sedai),228.08423825408,5.91,0.014915,-2e-06,-26.139,2e-06,-10.94,0.000101,-2.497,-0.544,2.6e-05,-4.431,0.52,0.031224,2.471,0.052858,4.345,0.14735,10.434,0.542659,25.393,99.59,26.45085,99.986,0.000309,0.019017,0.002746,1.003708,0.000535,5.148091,0.030387,5.129074,0.030176,0.003701,0.000533,-0.025,0.009,0.004957,-0.0,-8.9e-05,-99.949,3.548,0.037,95.355,0.055,0.8,-10186330385.0,246134.773503,kld/MiMo-V2.5/wiki-test-raw-fused/aes_sedai/MiMo-V2.5-Q5_K_M.md,212.42,8778000.0,584512819216.0,9.31,128.0,512.0,-647.0,35.0,0.9312,723606.24,256.0,91.864,4.848561200112841e+50
	MiMo-V2.5-Q8_0 (aes_sedai),328.22140076032,8.5,0.012539,-2e-06,-23.187,2e-06,-9.534,7.9e-05,-2.183,-0.475,2e-05,-3.87,0.472,0.025006,2.171,0.043004,3.847,0.127322,9.313,0.496331,22.771,99.66,28.112392,99.978,0.000329,0.006521,0.002512,1.001271,0.00049,5.135595,0.030275,5.129074,0.030176,0.001271,0.000489,-0.007,0.008,0.003851,0.0,-0.000101,-99.947,3.193,0.039,95.843,0.052,0.8,-10186330385.0,341638.773503,kld/MiMo-V2.5/wiki-test-raw-fused/aes_sedai/MiMo-V2.5-Q8_0.md,305.68,8778000.0,584512819216.0,9.31,128.0,512.0,-80242.0,35.0,0.9312,733766.24,256.0,90.354,4.848561200112841e+50