README.md

---
license: cc-by-4.0
language:
  - en
tags:
  - automatic-speech-recognition
  - onnx
  - parakeet
  - streaming
  - eou
  - voice-agent
base_model: nvidia/parakeet_realtime_eou_120m-v1
---

# parakeet-realtime-eou-120m ONNX (split decoder/joint, 320 ms chunks)

Streaming preview model. Re-export of [nvidia/parakeet_realtime_eou_120m-v1](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1) to three separate ONNX files, int8 dynamic-quantized for CPU / DirectML inference. 120M-parameter cache-aware Conformer with end-of-utterance detection at 80–160 ms latency.

| File | Inputs | Outputs |
|---|---|---|
| `streaming_encoder.int8.onnx` | `mel [1, 128, F] f32`, `mel_length [1] i32`, `cache_last_channel [1, 17, 70, 512] f32`, `cache_last_time [1, 17, 512, 8] f32`, `cache_last_channel_len [1] i32` | `encoder [1, 512, T] f32`, `encoder_length [1] i32`, `cache_last_channel_next`, `cache_last_time_next`, `cache_last_channel_len_next [1] i32` |
| `decoder.int8.onnx` | `targets [1, 1] i32`, `target_length [1] i32`, `h_in [1, 1, 640] f32`, `c_in [1, 1, 640] f32` | `decoder [1, 640, 1] f32`, `h_out`, `c_out` |
| `joint_decision.int8.onnx` | `encoder [1, 512, T] f32`, `decoder [1, 640, U] f32` | `token_id [1, T, U] i32`, `token_prob [1, T, U] f32`, `duration [1, T, U] i32` |

The streaming encoder carries cache state (`cache_last_channel`, `cache_last_time`, `cache_last_channel_len`) between chunks. Initial call: feed zero-initialised cache tensors. Subsequent calls: feed the previous call's `*_next` outputs.

## Why split?

NeMo's own `asr_model.export()` and [altunenes/parakeet-rs](https://huggingface.co/altunenes/parakeet-rs)'s ONNX version fuse the decoder and joint network into a single ONNX file. That's fine for inference engines that call the full RNN-T decoder loop in one go, but doesn't fit pipelines that drive the loop themselves and need the sub-graphs callable independently (e.g. the [talat](https://github.com/lumikey/talat) Rust streaming inference layer, which mirrors FluidAudio's macOS CoreML 3-file decomposition).

The PyTorch wrappers are adapted from [FluidInference/mobius](https://github.com/FluidInference/mobius/tree/main/models/stt/parakeet-realtime-eou-120m) (Apache 2.0).

## Chunk size

This export is the **320 ms** variant — 5120-sample chunks at 16 kHz, the talat-inference default. The 160 ms and 1280 ms variants exist in FluidInference's CoreML release; re-exports of those will land in this repo's branches when needed.

## Quantization

Per-channel int8 weight-only via `onnxruntime.quantization.quantize_dynamic`. Activations stay fp32 — no calibration dataset needed.

## License

Inherits NVIDIA's parakeet_realtime_eou_120m-v1 license (CC-BY-4.0).