Text-to-Speech
F5-TTS
Assamese
tts
open-bible
assamese
luel commited on
Commit
d39f781
·
verified ·
1 Parent(s): bc8198c

add readme

Browse files
Files changed (1) hide show
  1. README.md +104 -0
README.md ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - as
4
+ license: cc-by-sa-4.0
5
+ library_name: f5-tts
6
+ tags:
7
+ - text-to-speech
8
+ - tts
9
+ - f5-tts
10
+ - open-bible
11
+ - assamese
12
+ pipeline_tag: text-to-speech
13
+ base_model: SWivid/F5-TTS
14
+ datasets:
15
+ - davidguzmanr/open-bible-resources
16
+ inference: false
17
+ ---
18
+
19
+ # F5-TTS Open Bible — Assamese
20
+
21
+ A zero-shot text-to-speech model for **Assamese**, trained from scratch on
22
+ the [Open Bible](https://huggingface.co/datasets/davidguzmanr/open-bible-resources)
23
+ corpus using the [F5-TTS](https://github.com/SWivid/F5-TTS) architecture
24
+ (diffusion transformer with vocos vocoder, 24 kHz output).
25
+
26
+ The model takes a short reference audio clip (5–10 seconds) and a target text,
27
+ and synthesises the target text in the voice of the reference speaker. No
28
+ fine-tuning per voice is required.
29
+
30
+ ## Files
31
+
32
+ | File | Purpose |
33
+ |------|---------|
34
+ | `model_last.pt` | Trained model weights. |
35
+ | `vocab.txt` | Character vocabulary built from the training transcripts. |
36
+ | `F5-TTS_OpenBible_Assamese.yaml` | Hydra training/inference config (architecture, mel spec settings, tokenizer). |
37
+
38
+ ## Intended use
39
+
40
+ - Zero-shot TTS for Assamese, controlled by a user-supplied reference clip.
41
+ - Research on multilingual TTS, low-resource TTS evaluation, and listening
42
+ studies on Open Bible–style read-speech.
43
+
44
+ ## How to use
45
+
46
+ Install F5-TTS:
47
+
48
+ ```bash
49
+ pip install git+https://github.com/SWivid/F5-TTS.git
50
+ ```
51
+
52
+ Download the checkpoint and run inference:
53
+
54
+ ```python
55
+ from huggingface_hub import hf_hub_download
56
+ from f5_tts.api import F5TTS
57
+
58
+ repo_id = "multilingual-tts/F5-TTS-OpenBible-Assamese"
59
+ ckpt = hf_hub_download(repo_id, "model_last.pt")
60
+ vocab = hf_hub_download(repo_id, "vocab.txt")
61
+ config = hf_hub_download(repo_id, "F5-TTS_OpenBible_Assamese.yaml")
62
+
63
+ model = F5TTS(ckpt_file=ckpt, vocab_file=vocab, model_cfg=config)
64
+
65
+ # Supply your own clean reference clip — 5–10 s, single speaker and its transcription.
66
+ ref_audio = "/path/to/your-assamese-clip.wav"
67
+ ref_text = "Exact transcription of the clip"
68
+ gen_text = "..." # text to synthesise in Assamese
69
+
70
+ wav, sr, _ = model.infer(ref_audio=ref_audio, ref_text=ref_text, gen_text=gen_text)
71
+ ```
72
+
73
+ ## Training data
74
+
75
+ - **Source:** `davidguzmanr/open-bible-resources`, config `Assamese`
76
+ - **Size:** approximately 30,500 utterances
77
+ - **Speakers:** multispeaker; speaker identity is supplied at inference time
78
+ via the reference clip, not by a fixed speaker id
79
+ - **Sample rate:** 24 kHz
80
+ - **Maximum utterance duration during training:** 15 s
81
+
82
+ ## Training procedure
83
+
84
+ - Base architecture: F5-TTS v1 Base (DiT, 1024 dim, 22 layers, 16 heads,
85
+ text dim 512, 4 convolutional layers).
86
+ - Tokenizer: custom character-level, built from the training transcripts.
87
+ - Vocoder: vocos.
88
+ - Mel spectrogram: 100 channels, hop 256, win 1024, n_fft 1024.
89
+ - Optimizer: AdamW, learning rate 7.5e-5, 20 000 warmup updates.
90
+ - Training budget: 500,000 optimizer updates on 4 GPUs with mixed precision
91
+ (bf16), global batch ≈ 112,000 frames.
92
+
93
+ Audio preprocessing, vocab generation, and config sizing are reproducible via
94
+ the upstream
95
+ [open-bible-models](https://github.com/davidguzmanr/open-bible-models) repo.
96
+
97
+ ## Evaluation
98
+
99
+ Evaluated alongside other Open-Bible TTS systems on character/word error rate
100
+ (via Meta's Omnilingual ASR) and UTMOSv2 naturalness scores. See the
101
+ [open-bible-models](https://github.com/davidguzmanr/open-bible-models) repository
102
+ for the evaluation pipeline and the
103
+ [open-bible-surveys](https://github.com/davidguzmanr/open-bible-surveys) repository
104
+ for the human-listening survey methodology.