Automatic Speech Recognition
Safetensors
Italian
whisper
italian
localai
mudler commited on
Commit
4eb3989
·
verified ·
1 Parent(s): e13468b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +90 -0
README.md ADDED
@@ -0,0 +1,90 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: it
3
+ license: mit
4
+ tags:
5
+ - whisper
6
+ - automatic-speech-recognition
7
+ - italian
8
+ - localai
9
+ datasets:
10
+ - mozilla-foundation/common_voice_25_0
11
+ - facebook/multilingual_librispeech
12
+ - facebook/voxpopuli
13
+ base_model: openai/whisper-medium
14
+ pipeline_tag: automatic-speech-recognition
15
+ ---
16
+
17
+ # whisper-medium-it-multi
18
+
19
+ Fine-tuned [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) (769M params) for Italian ASR on multiple datasets.
20
+
21
+ **Author:** Ettore Di Giacinto
22
+
23
+ Brought to you by the [LocalAI](https://github.com/mudler/LocalAI) team. This model can be used directly with [LocalAI](https://localai.io).
24
+
25
+ ## Usage with LocalAI
26
+
27
+ This model is ready to use with [LocalAI](https://localai.io) via the `whisperx` backend.
28
+
29
+ Save the following as `whisperx-medium-it-multi.yaml` in your LocalAI models directory:
30
+
31
+ ```yaml
32
+ name: whisperx-medium-it-multi
33
+ backend: whisperx
34
+ known_usecases:
35
+ - transcript
36
+ parameters:
37
+ model: LocalAI-io/whisper-medium-it-multi-ct2-int8
38
+ language: it
39
+ ```
40
+
41
+ Then transcribe audio via the OpenAI-compatible endpoint:
42
+
43
+ ```bash
44
+ curl http://localhost:8080/v1/audio/transcriptions \
45
+ -H "Content-Type: multipart/form-data" \
46
+ -F file="@audio.mp3" \
47
+ -F model="whisperx-medium-it-multi"
48
+ ```
49
+
50
+ ## Results
51
+
52
+ Evaluated on combined test set (Common Voice + MLS + VoxPopuli):
53
+
54
+ | Step | WER |
55
+ |------|-----|
56
+ | 1000 | 17.55% |
57
+ | 3000 | 16.71% |
58
+ | 5000 | 14.00% |
59
+ | 7000 | 13.02% |
60
+ | 9000 | 12.10% |
61
+ | 10000 | **12.37%** |
62
+
63
+ ## Training Details
64
+
65
+ - **Base model:** openai/whisper-medium (769M parameters)
66
+ - **Datasets:** Common Voice 25.0 Italian (173k) + MLS Italian (60k) + VoxPopuli Italian (23k) = 255k train samples
67
+ - **Steps:** 10,000
68
+ - **Precision:** bf16 on NVIDIA GB10
69
+
70
+ ## Usage
71
+
72
+ ### Transformers
73
+
74
+ ```python
75
+ from transformers import pipeline
76
+
77
+ pipe = pipeline("automatic-speech-recognition", model="LocalAI-io/whisper-medium-it-multi")
78
+ result = pipe("audio.mp3", generate_kwargs={"language": "it", "task": "transcribe"})
79
+ print(result["text"])
80
+ ```
81
+
82
+ ### CTranslate2 / faster-whisper
83
+
84
+ For optimized CPU inference: [LocalAI-io/whisper-medium-it-multi-ct2-int8](https://huggingface.co/LocalAI-io/whisper-medium-it-multi-ct2-int8)
85
+
86
+ ## Links
87
+
88
+ - **CTranslate2 INT8:** [LocalAI-io/whisper-medium-it-multi-ct2-int8](https://huggingface.co/LocalAI-io/whisper-medium-it-multi-ct2-int8)
89
+ - **Project:** [github.com/localai-org/italian-whisper](https://github.com/localai-org/italian-whisper)
90
+ - **LocalAI:** [github.com/mudler/LocalAI](https://github.com/mudler/LocalAI)