Update README.md

README.md

@@ -1,3 +1,126 @@
 ---
 license: mit
+library_name: onnx
+tags:
+- ocr
+- text-recognition
+- trocr
+- documents
+- transformer
+- onnx
+base_model: microsoft/trocr-base-printed
+pipeline_tag: image-to-text
+language:
+- en
 ---
+
+# TrOCR Base Printed (ONNX, fp32 + fp16 bundle)
+
+ONNX exports of [microsoft/trocr-base-printed](https://huggingface.co/microsoft/trocr-base-printed) — Microsoft's Transformer-based OCR for printed text. ViT image encoder + GPT-style autoregressive text decoder, trained on SROIE printed-text crops. Recognizes the text in a cropped image of a single line/word.
+
+This repo bundles **both fp32 and fp16 precisions** in one download — distribution symmetry, shared tokenizer + config files. Pick a precision via the `.onnx` filename in the `onnx/` subdir.
+
+Re-exported from upstream PyTorch weights via a two-step pipeline. Provenance trail: Li et al. → microsoft/trocr-base-printed → `optimum-cli export onnx --task image-to-text-with-past` (fp32 stage) → `onnxconverter_common.float16.convert_float_to_float16(..., keep_io_types=True)` (fp16 cast on the fp32 graph) → these files.
+
+Toolchain: `torch 2.4.x` (CUDA 12.4), `transformers 4.45.2`, `optimum[onnxruntime] 1.24.0`, `onnxconverter-common>=1.14`. Full conversion script: [`scripts/export-trocr-base-printed-fp16.ps1`](https://github.com/HeliosophLLC/DatumIngest/blob/main/scripts/export-trocr-base-printed-fp16.ps1) in the DatumIngest repo (despite the `-fp16` suffix on the script name, it produces both precisions in one run).
+
+Why the two-step pipeline instead of `optimum-cli ... --dtype fp16` directly: the CUDA path requires a CUDA-enabled torch build in the venv (the other export scripts don't install one), and optimum-cli's fp16 merged-decoder export ships with an If-subgraph wiring bug on this architecture. The onnxconverter-common pass operates on the already-traced fp32 graph in place and sidesteps both issues. `keep_io_types=True` means inputs and outputs stay fp32 at the wire boundary — only internal weights + activations run in half precision — so runtime code feeds the same input tensors regardless of which `.onnx` file it loads.
+
+Credit: Minghao Li, Tengchao Lv, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, Furu Wei (Microsoft Research). Paper: *"TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models"*, 2021.
+
+## What this repo contains
+
+TrOCR is encoder-decoder, so the export splits into multiple files. **All shared (root) files must be present** along with the precision-specific `.onnx` files you choose to load.
+
+### `onnx/` subdir — precision-specific files
+
+| File | Variant | Size | Role |
+|---|---|---|---|
+| `encoder_model.onnx` | fp32 | ~700 MB | ViT image encoder |
+| `encoder_model_fp16.onnx` | fp16 | ~350 MB | Half-precision ViT image encoder |
+| `decoder_model_merged.onnx` | fp32 | ~700 MB | Text decoder with KV cache merged into one graph |
+| `decoder_model_merged_fp16.onnx` | fp16 | ~350 MB | Half-precision text decoder |
+
+The non-merged `decoder_model.onnx` is deliberately omitted — the merged form supersedes it for runtime use; keeping both would double the repo size for no benefit.
+
+### Root — shared tokenizer + config files
+
+| File | Role |
+|---|---|
+| `config.json` | Model architecture config |
+| `generation_config.json` | Decoder generation defaults (max_length, EOS token, etc.) |
+| `preprocessor_config.json` | Image preprocessing — `TrOCRProcessor` settings (resize, normalize) |
+| `tokenizer.json` + `vocab.json` + `merges.txt` | BPE tokenizer files |
+| `tokenizer_config.json` + `special_tokens_map.json` | Tokenizer metadata |
+
+## Input / output (both variants)
+
+| Stage | Input | Output |
+|---|---|---|
+| Encoder | `pixel_values` — NCHW **float32** (yes, even for the fp16 variant — IO types are kept fp32), preprocessed RGB image (typically 384×384) | `last_hidden_state` — encoder features (fp32 at the boundary; fp16 internally for the half-precision variant) |
+| Decoder | `input_ids` (token sequence so far) + `encoder_hidden_states` + KV cache from prior step | next-token logits + updated KV cache |
+
+The fp16 variant's `keep_io_types=True` setting means runtime code is **identical** between fp32 and fp16 — you don't have to cast inputs to `np.float16`. Only the on-disk weights and the internal compute differ.
+
+## How to use
+
+Greedy decoding orchestrated outside the ONNX graph — same encoder-decoder shape as Whisper, T5, BART, and friends:
+
+```python
+import onnxruntime as ort
+import numpy as np
+from PIL import Image
+from transformers import TrOCRProcessor
+
+# Pick a precision — same runtime code either way thanks to keep_io_types.
+PRECISION_SUFFIX = ""          # "" for fp32, "_fp16" for fp16
+
+proc      = TrOCRProcessor.from_pretrained(".")
+encoder   = ort.InferenceSession(f"onnx/encoder_model{PRECISION_SUFFIX}.onnx")
+decoder   = ort.InferenceSession(f"onnx/decoder_model_merged{PRECISION_SUFFIX}.onnx")
+
+img = Image.open("cropped_text_line.jpg").convert("RGB")
+pixel_values = proc(images=img, return_tensors="np").pixel_values  # float32 in both cases
+encoder_hidden = encoder.run(None, {"pixel_values": pixel_values})[0]
+
+BOS = proc.tokenizer.cls_token_id
+EOS = proc.tokenizer.eos_token_id
+input_ids = np.array([[BOS]], dtype=np.int64)
+generated, past_kv = [], None
+
+for _ in range(64):
+    decoder_inputs = {"input_ids": input_ids, "encoder_hidden_states": encoder_hidden}
+    if past_kv is not None:
+        decoder_inputs.update(past_kv_to_inputs(past_kv))
+    outputs = decoder.run(None, decoder_inputs)
+    next_token = outputs[0][:, -1, :].argmax(-1)
+    if next_token.item() == EOS: break
+    generated.append(next_token.item())
+    input_ids = next_token.reshape(1, 1)
+    past_kv = outputs_to_past_kv(outputs[1:])
+
+text = proc.tokenizer.decode(generated, skip_special_tokens=True)
+```
+
+The exact past-KV input/output names are Optimum-version-specific; inspect with Netron once after export to pin them down.
+
+## Which precision should I use?
+
+- **fp32** — full precision, identical numerics to upstream PyTorch reference. Default for accuracy-sensitive scientific work, OCR-accuracy benchmarks.
+- **fp16** — ~half the disk footprint (~700 MB vs ~1.4 GB total) and half the model-load memory. On GPU / NPU with native fp16: modest speedup (typically 1.5-2× over fp32 on consumer GPUs). On CPU runtimes that upcast fp16 → fp32 internally, runtime speed is identical to fp32 but you save the memory.
+
+The `keep_io_types=True` setting means switching between them is a single file-path change — no code changes needed.
+
+## Related variants (not in this repo)
+
+Microsoft publishes a small variant family of TrOCR — same architecture, different size / training corpus:
+
+- `microsoft/trocr-small-printed` — smaller (~5× less), less accurate but faster.
+- `microsoft/trocr-large-printed` — bigger, better quality.
+- `microsoft/trocr-base-handwritten` — same size as this, trained on IAM handwritten dataset instead of SROIE.
+
+All MIT-licensed, all re-exportable via the same script with a swapped `--model` arg.
+
+## License
+
+**MIT** — same as upstream [microsoft/trocr-base-printed](https://huggingface.co/microsoft/trocr-base-printed). `LICENSE` file included. Optimum ONNX export + fp16 conversion are numerical transformations only — no relicensing implication.