MioTTS-2.6B (Quantized)
Description
This model is a 4-bit quantized version of the original Aratako/MioTTS-2.6B model, optimized for reduced memory usage while maintaining performance.
Quantization Details
- Quantization Type: 4-bit
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: bfloat16
- bnb_4bit_quant_storage: uint8
- Original Footprint: 5186.78 MB (BFLOAT16)
- Quantized Footprint: 1534.59 MB (UINT8)
- Memory Reduction: 70.4%
Usage
from transformers import AutoModel, AutoTokenizer
model_name = "MioTTS-2.6B-bnb-4bit-nf4"
model = AutoModel.from_pretrained(
"manu02/MioTTS-2.6B-bnb-4bit-nf4",
)
tokenizer = AutoTokenizer.from_pretrained("manu02/MioTTS-2.6B-bnb-4bit-nf4", use_fast=True)
- Downloads last month
- 4