| --- |
| license: cc-by-4.0 |
| language: |
| - en |
| tags: |
| - automatic-speech-recognition |
| - onnx |
| - parakeet |
| - streaming |
| - eou |
| - voice-agent |
| base_model: nvidia/parakeet_realtime_eou_120m-v1 |
| --- |
| |
| # parakeet-realtime-eou-120m ONNX (split decoder/joint, 320 ms chunks) |
|
|
| Streaming preview model. Re-export of [nvidia/parakeet_realtime_eou_120m-v1](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1) to three separate ONNX files, int8 dynamic-quantized for CPU / DirectML inference. 120M-parameter cache-aware Conformer with end-of-utterance detection at 80–160 ms latency. |
|
|
| | File | Inputs | Outputs | |
| |---|---|---| |
| | `streaming_encoder.int8.onnx` | `mel [1, 128, F] f32`, `mel_length [1] i32`, `cache_last_channel [1, 17, 70, 512] f32`, `cache_last_time [1, 17, 512, 8] f32`, `cache_last_channel_len [1] i32` | `encoder [1, 512, T] f32`, `encoder_length [1] i32`, `cache_last_channel_next`, `cache_last_time_next`, `cache_last_channel_len_next [1] i32` | |
| | `decoder.int8.onnx` | `targets [1, 1] i32`, `target_length [1] i32`, `h_in [1, 1, 640] f32`, `c_in [1, 1, 640] f32` | `decoder [1, 640, 1] f32`, `h_out`, `c_out` | |
| | `joint_decision.int8.onnx` | `encoder [1, 512, T] f32`, `decoder [1, 640, U] f32` | `token_id [1, T, U] i32`, `token_prob [1, T, U] f32`, `duration [1, T, U] i32` | |
|
|
| The streaming encoder carries cache state (`cache_last_channel`, `cache_last_time`, `cache_last_channel_len`) between chunks. Initial call: feed zero-initialised cache tensors. Subsequent calls: feed the previous call's `*_next` outputs. |
|
|
| ## Why split? |
|
|
| NeMo's own `asr_model.export()` and [altunenes/parakeet-rs](https://huggingface.co/altunenes/parakeet-rs)'s ONNX version fuse the decoder and joint network into a single ONNX file. That's fine for inference engines that call the full RNN-T decoder loop in one go, but doesn't fit pipelines that drive the loop themselves and need the sub-graphs callable independently (e.g. the [talat](https://github.com/lumikey/talat) Rust streaming inference layer, which mirrors FluidAudio's macOS CoreML 3-file decomposition). |
|
|
| The PyTorch wrappers are adapted from [FluidInference/mobius](https://github.com/FluidInference/mobius/tree/main/models/stt/parakeet-realtime-eou-120m) (Apache 2.0). |
|
|
| ## Chunk size |
|
|
| This export is the **320 ms** variant — 5120-sample chunks at 16 kHz, the talat-inference default. The 160 ms and 1280 ms variants exist in FluidInference's CoreML release; re-exports of those will land in this repo's branches when needed. |
|
|
| ## Quantization |
|
|
| Per-channel int8 weight-only via `onnxruntime.quantization.quantize_dynamic`. Activations stay fp32 — no calibration dataset needed. |
|
|
| ## License |
|
|
| Inherits NVIDIA's parakeet_realtime_eou_120m-v1 license (CC-BY-4.0). |
| |
