Spaces:
Sleeping
Sleeping
Changed readme
Browse files
README.md
CHANGED
|
@@ -1,117 +1,12 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
We're excited to introduce Chatterbox, [Resemble AI's](https://resemble.ai) first production-grade open source TTS model. Licensed under MIT, Chatterbox has been benchmarked against leading closed-source systems like ElevenLabs, and is consistently preferred in side-by-side evaluations.
|
| 14 |
-
|
| 15 |
-
Whether you're working on memes, videos, games, or AI agents, Chatterbox brings your content to life. It's also the first open source TTS model to support **emotion exaggeration control**, a powerful feature that makes your voices stand out. Try it now on our [Hugging Face Gradio app.](https://huggingface.co/spaces/ResembleAI/Chatterbox)
|
| 16 |
-
|
| 17 |
-
If you like the model but need to scale or tune it for higher accuracy, check out our competitively priced TTS service (<a href="https://resemble.ai">link</a>). It delivers reliable performance with ultra-low latency of sub 200ms—ideal for production use in agents, applications, or interactive media.
|
| 18 |
-
|
| 19 |
-
# Key Details
|
| 20 |
-
- SoTA zeroshot TTS
|
| 21 |
-
- 0.5B Llama backbone
|
| 22 |
-
- Unique exaggeration/intensity control
|
| 23 |
-
- Ultra-stable with alignment-informed inference
|
| 24 |
-
- Trained on 0.5M hours of cleaned data
|
| 25 |
-
- Watermarked outputs
|
| 26 |
-
- Easy voice conversion script
|
| 27 |
-
- [Outperforms ElevenLabs](https://podonos.com/resembleai/chatterbox)
|
| 28 |
-
|
| 29 |
-
# Tips
|
| 30 |
-
- **General Use (TTS and Voice Agents):**
|
| 31 |
-
- The default settings (`exaggeration=0.5`, `cfg_weight=0.5`) work well for most prompts.
|
| 32 |
-
- If the reference speaker has a fast speaking style, lowering `cfg_weight` to around `0.3` can improve pacing.
|
| 33 |
-
|
| 34 |
-
- **Expressive or Dramatic Speech:**
|
| 35 |
-
- Try lower `cfg_weight` values (e.g. `~0.3`) and increase `exaggeration` to around `0.7` or higher.
|
| 36 |
-
- Higher `exaggeration` tends to speed up speech; reducing `cfg_weight` helps compensate with slower, more deliberate pacing.
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
# Installation
|
| 40 |
-
```shell
|
| 41 |
-
pip install chatterbox-tts
|
| 42 |
-
```
|
| 43 |
-
|
| 44 |
-
Alternatively, you can install from source:
|
| 45 |
-
```shell
|
| 46 |
-
# conda create -yn chatterbox python=3.11
|
| 47 |
-
# conda activate chatterbox
|
| 48 |
-
|
| 49 |
-
git clone https://github.com/resemble-ai/chatterbox.git
|
| 50 |
-
cd chatterbox
|
| 51 |
-
pip install -e .
|
| 52 |
-
```
|
| 53 |
-
We developed and tested Chatterbox on Python 3.11 on Debain 11 OS; the versions of the dependencies are pinned in `pyproject.toml` to ensure consistency. You can modify the code or dependencies in this installation mode.
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
# Usage
|
| 57 |
-
```python
|
| 58 |
-
import torchaudio as ta
|
| 59 |
-
from chatterbox.tts import ChatterboxTTS
|
| 60 |
-
|
| 61 |
-
model = ChatterboxTTS.from_pretrained(device="cuda")
|
| 62 |
-
|
| 63 |
-
text = "Ezreal and Jinx teamed up with Ahri, Yasuo, and Teemo to take down the enemy's Nexus in an epic late-game pentakill."
|
| 64 |
-
wav = model.generate(text)
|
| 65 |
-
ta.save("test-1.wav", wav, model.sr)
|
| 66 |
-
|
| 67 |
-
# If you want to synthesize with a different voice, specify the audio prompt
|
| 68 |
-
AUDIO_PROMPT_PATH = "YOUR_FILE.wav"
|
| 69 |
-
wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH)
|
| 70 |
-
ta.save("test-2.wav", wav, model.sr)
|
| 71 |
-
```
|
| 72 |
-
See `example_tts.py` and `example_vc.py` for more examples.
|
| 73 |
-
|
| 74 |
-
# Supported Lanugage
|
| 75 |
-
Currenlty only English.
|
| 76 |
-
|
| 77 |
-
# Acknowledgements
|
| 78 |
-
- [Cosyvoice](https://github.com/FunAudioLLM/CosyVoice)
|
| 79 |
-
- [Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning)
|
| 80 |
-
- [HiFT-GAN](https://github.com/yl4579/HiFTNet)
|
| 81 |
-
- [Llama 3](https://github.com/meta-llama/llama3)
|
| 82 |
-
- [S3Tokenizer](https://github.com/xingchensong/S3Tokenizer)
|
| 83 |
-
|
| 84 |
-
# Built-in PerTh Watermarking for Responsible AI
|
| 85 |
-
|
| 86 |
-
Every audio file generated by Chatterbox includes [Resemble AI's Perth (Perceptual Threshold) Watermarker](https://github.com/resemble-ai/perth) - imperceptible neural watermarks that survive MP3 compression, audio editing, and common manipulations while maintaining nearly 100% detection accuracy.
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
## Watermark extraction
|
| 90 |
-
|
| 91 |
-
You can look for the watermark using the following script.
|
| 92 |
-
|
| 93 |
-
```python
|
| 94 |
-
import perth
|
| 95 |
-
import librosa
|
| 96 |
-
|
| 97 |
-
AUDIO_PATH = "YOUR_FILE.wav"
|
| 98 |
-
|
| 99 |
-
# Load the watermarked audio
|
| 100 |
-
watermarked_audio, sr = librosa.load(AUDIO_PATH, sr=None)
|
| 101 |
-
|
| 102 |
-
# Initialize watermarker (same as used for embedding)
|
| 103 |
-
watermarker = perth.PerthImplicitWatermarker()
|
| 104 |
-
|
| 105 |
-
# Extract watermark
|
| 106 |
-
watermark = watermarker.get_watermark(watermarked_audio, sample_rate=sr)
|
| 107 |
-
print(f"Extracted watermark: {watermark}")
|
| 108 |
-
# Output: 0.0 (no watermark) or 1.0 (watermarked)
|
| 109 |
-
```
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
# Official Discord
|
| 113 |
-
|
| 114 |
-
👋 Join us on [Discord](https://discord.gg/rJq9cRJBJ6) and let's build something awesome together!
|
| 115 |
-
|
| 116 |
-
# Disclaimer
|
| 117 |
-
Don't use this model to do bad things. Prompts are sourced from freely available data on the internet.
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: VoiceSample
|
| 3 |
+
emoji: 🏆
|
| 4 |
+
colorFrom: gray
|
| 5 |
+
colorTo: purple
|
| 6 |
+
sdk: gradio
|
| 7 |
+
sdk_version: 5.34.2
|
| 8 |
+
app_file: app.py
|
| 9 |
+
pinned: false
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|