Text-to-Speech
F5-TTS
Nepali
tts
open-bible
nepali
davidguzmanr commited on
Commit
8d280e2
·
verified ·
1 Parent(s): 745bee7

Add README for Nepali

Browse files
Files changed (1) hide show
  1. README.md +120 -0
README.md ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - ne
4
+ license: cc-by-sa-4.0
5
+ library_name: f5-tts
6
+ tags:
7
+ - text-to-speech
8
+ - tts
9
+ - f5-tts
10
+ - open-bible
11
+ - nepali
12
+ pipeline_tag: text-to-speech
13
+ base_model: SWivid/F5-TTS
14
+ datasets:
15
+ - davidguzmanr/open-bible-resources
16
+ inference: false
17
+ ---
18
+
19
+ # F5-TTS Open Bible — Nepali
20
+
21
+ A zero-shot text-to-speech model for **Nepali**, trained from scratch on
22
+ the [Open Bible](https://huggingface.co/datasets/davidguzmanr/open-bible-resources)
23
+ corpus using the [F5-TTS](https://github.com/SWivid/F5-TTS) architecture
24
+ (diffusion transformer with vocos vocoder, 24 kHz output).
25
+
26
+ The model takes a short reference audio clip (5–10 seconds) and a target text,
27
+ and synthesises the target text in the voice of the reference speaker. No
28
+ fine-tuning per voice is required.
29
+
30
+ ## Files
31
+
32
+ | File | Purpose |
33
+ |------|---------|
34
+ | `model_last.pt` | Trained model weights. |
35
+ | `vocab.txt` | Character vocabulary built from the training transcripts. |
36
+ | `F5-TTS_OpenBible_Nepali.yaml` | Hydra training/inference config (architecture, mel spec settings, tokenizer). |
37
+
38
+ ## Intended use
39
+
40
+ - Zero-shot TTS for Nepali, controlled by a user-supplied reference clip.
41
+ - Research on multilingual TTS, low-resource TTS evaluation, and listening
42
+ studies on Open Bible–style read-speech.
43
+
44
+ ## How to use
45
+
46
+ Install F5-TTS:
47
+
48
+ ```bash
49
+ pip install git+https://github.com/SWivid/F5-TTS.git
50
+ ```
51
+
52
+ Download the checkpoint and run inference:
53
+
54
+ ```python
55
+ import torch
56
+ from huggingface_hub import hf_hub_download
57
+ from hydra.utils import get_class
58
+ from omegaconf import OmegaConf
59
+ from f5_tts.infer.utils_infer import infer_process, load_model, load_vocoder, preprocess_ref_audio_text
60
+
61
+ repo_id = "multilingual-tts/F5-TTS-OpenBible-Nepali"
62
+ ckpt = hf_hub_download(repo_id, "model_last.pt")
63
+ vocab = hf_hub_download(repo_id, "vocab.txt")
64
+ config = hf_hub_download(repo_id, "F5-TTS_OpenBible_Nepali.yaml")
65
+
66
+ device = "cuda" if torch.cuda.is_available() else "cpu"
67
+
68
+ model_cfg = OmegaConf.load(config)
69
+ model_cls = get_class(f"f5_tts.model.{model_cfg.model.backbone}")
70
+
71
+ vocoder = load_vocoder(vocoder_name="vocos", is_local=False, device=device)
72
+ model = load_model(
73
+ model_cls, model_cfg.model.arch, ckpt,
74
+ mel_spec_type="vocos", vocab_file=vocab, use_ema=True, device=device,
75
+ )
76
+
77
+ # Supply your own clean reference clip — 5–10 s, single speaker and its transcription.
78
+ ref_audio = "/path/to/your-nepali-clip.wav"
79
+ ref_text = "Exact transcription of the clip"
80
+ gen_text = "..." # text to synthesise in Nepali
81
+
82
+ ref_audio_proc, ref_text_proc = preprocess_ref_audio_text(ref_audio, ref_text)
83
+ wav, sr, _ = infer_process(
84
+ ref_audio_proc, ref_text_proc, gen_text, model, vocoder,
85
+ mel_spec_type="vocos", device=device,
86
+ )
87
+ ```
88
+
89
+ ## Training data
90
+
91
+ - **Source:** `davidguzmanr/open-bible-resources`, config `Nepali`
92
+ - **Size:** approximately 20,448 utterances
93
+ - **Speakers:** multispeaker; speaker identity is supplied at inference time
94
+ via the reference clip, not by a fixed speaker id
95
+ - **Sample rate:** 24 kHz
96
+ - **Maximum utterance duration during training:** 15 s
97
+
98
+ ## Training procedure
99
+
100
+ - Base architecture: F5-TTS v1 Base (DiT, 1024 dim, 22 layers, 16 heads,
101
+ text dim 512, 4 convolutional layers).
102
+ - Tokenizer: custom character-level, built from the training transcripts.
103
+ - Vocoder: vocos.
104
+ - Mel spectrogram: 100 channels, hop 256, win 1024, n_fft 1024.
105
+ - Optimizer: AdamW, learning rate 7.5e-5, 20 000 warmup updates.
106
+ - Training budget: 500,000 optimizer updates on 4 GPUs with mixed precision
107
+ (bf16), global batch ≈ 112,000 frames.
108
+
109
+ Audio preprocessing, vocab generation, and config sizing are reproducible via
110
+ the upstream
111
+ [open-bible-models](https://github.com/davidguzmanr/open-bible-models) repo.
112
+
113
+ ## Evaluation
114
+
115
+ Evaluated alongside other Open-Bible TTS systems on character/word error rate
116
+ (via Meta's Omnilingual ASR) and UTMOSv2 naturalness scores. See the
117
+ [open-bible-models](https://github.com/davidguzmanr/open-bible-models) repository
118
+ for the evaluation pipeline and the
119
+ [open-bible-surveys](https://github.com/davidguzmanr/open-bible-surveys) repository
120
+ for the human-listening survey methodology.