v9: schema-free inference, 100% smoke, sub-second cold prefill

.gitattributes

@@ -42,3 +42,4 @@ functiongemma-physical-ai-v6-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
 functiongemma-physical-ai-v7-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
 moonshine/decoder.vmfb filter=lfs diff=lfs merge=lfs -text
 moonshine/decoder_with_past.vmfb filter=lfs diff=lfs merge=lfs -text
+functiongemma-physical-ai-v9-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -17,100 +17,114 @@ pipeline_tag: text-generation
 inference: false
 ---
 
-# FunctionGemma 270M — Physical AI
+# FunctionGemma 270M — Physical AI (v9, Octopus v2)
 
 Fine-tuned [`google/functiongemma-270m-it`](https://huggingface.co/google/functiongemma-270m-it)
 for voice-controlled physical-AI / household-IoT actions on a Synaptics
 SL2619 "Coral" edge board (Google IO 2026 demo).
 
-| Revision | File | Tool count | Notes |
-|----------|------|-----------:|-------|
-| **v7 (current)** | [`functiongemma-physical-ai-v7-Q5_K_M.gguf`](./functiongemma-physical-ai-v7-Q5_K_M.gguf) | 10 | `list_alarms` removed; alarm-query prompts route via `respond()`. 250-row eval: **86.8%** overall, **92.8%** single-tool, **75.0%** multi-tool exact-match, **0.0%** parse failure. |
-| v6 (previous) | [`functiongemma-physical-ai-v6-Q5_K_M.gguf`](./functiongemma-physical-ai-v6-Q5_K_M.gguf) | 11 | Camera + vision dropped. Single-tool routing 95.5%, multi-tool exact-match 23.9%. |
-| v4c (legacy) | [`functiongemma-physical-ai-Q4_K_M.gguf`](./functiongemma-physical-ai-Q4_K_M.gguf) | 13 | Earlier checkpoint, includes camera/scene tools. |
+| Revision | File | Tool count | Headline result |
+|----------|------|-----------:|-----------------|
+| **v9 (current)** | [`functiongemma-physical-ai-v9-Q5_K_M.gguf`](./functiongemma-physical-ai-v9-Q5_K_M.gguf) | 8 | 30/30 (100 %) routing on held-out smoke prompts; **0.55 s cold prefill** on the 2-core A55 (vs ~57 s for v7's schema-in-prompt build — **105× faster**). |
+| v7 (legacy) | [`functiongemma-physical-ai-v7-Q5_K_M.gguf`](./functiongemma-physical-ai-v7-Q5_K_M.gguf) | 10 | 86.8 % overall on a 250-row eval; schema-in-prompt build. |
+| v6 (legacy) | [`functiongemma-physical-ai-v6-Q5_K_M.gguf`](./functiongemma-physical-ai-v6-Q5_K_M.gguf) | 11 | Camera + vision dropped from earlier revs. Schema-in-prompt build. |
+| v4c (legacy) | [`functiongemma-physical-ai-Q4_K_M.gguf`](./functiongemma-physical-ai-Q4_K_M.gguf) | 13 | Earliest published checkpoint. |
 
-Schema ships as [`tools.json`](./tools.json) (10 tools, current). Token-to-tool
+Schema ships as [`tools.json`](./tools.json) (8 tools, v9). Token-to-tool
 mapping is in [`token_map.json`](./token_map.json).
 
-## Output format — function tokens
-
-Tool calls emit as **function tokens**: each tool name compiles to a single
-special-vocabulary token (`<tool_0>` … `<tool_9>` for v7) and a single
-`<end>` terminator. A complete call decodes in roughly 8–15 output tokens,
-vs ~30–80 for native FunctionGemma's
+## What changed in v9
+
+v9 is a structural rewrite of the training pipeline, not just a dataset
+refresh. Earlier revisions used the upstream FunctionGemma prompt format,
+which injects the full tool schema (~1088 tokens) into every request as a
+`<start_function_declaration>` developer turn. On a 2-core Cortex-A55 that
+prefill cost ~57 s on the first turn — incompatible with a sub-second
+voice UX.
+
+v9 follows [Octopus v2](https://arxiv.org/abs/2404.01744) end to end:
+
+| | Pre-v9 (schema-in-prompt) | v9 (Octopus v2) |
+|---|---|---|
+| Prompt format | `<start_of_turn>developer\n<start_function_declaration>...{schema}...<end_function_declaration>\n<end_of_turn>\n<start_of_turn>user\n{q}<end_of_turn>\n<start_of_turn>model\n` | `<start_of_turn>user\n{q}<end_of_turn>\n<start_of_turn>model\n` |
+| Tokens per prompt | ~1088 | ~13 |
+| Cold prefill on SL2619 (2-core A55) | ~57 s | **~0.55 s** |
+| Tool routing | learned from in-context schema | learned from `<tool_N>` token weights |
+| Training data shape | `{tools, messages: [dev, user, asst]}` with schema embedded | `{input, output, split}` — flat |
+
+The schema in `tools.json` is still the source of truth for dispatcher
+arg validation and is embedded in the GGUF metadata for schema-drift
+checks, but it is **not** loaded into the inference prompt.
+
+## Tool surface (v9, 8 tools)
+
+| Token | Name | Args | Purpose |
+|---|---|---|---|
+| `<tool_0>` | `set_status_led` | `led`, `state`, `brightness?` | On/off one or all HAT status LEDs |
+| `<tool_1>` | `blink_status_led` | `led`, `count?`, `speed?` | Discrete blink |
+| `<tool_2>` | `set_neopixel_effect` | `effect`, `color?`, `palette?`, `speed?`, `intensity?` | Animated effect on the ring |
+| `<tool_3>` | `play_buzzer` | `pattern` | `beep`, `double_beep`, `chirp`, `siren`, `alarm`, `success`, `error` |
+| `<tool_4>` | `set_alarm` | `duration` or `time`, `label?` | Timer |
+| `<tool_5>` | `cancel_alarm` | `label?` | Cancel one or all |
+| `<tool_6>` | `get_system_status` | `metric` | `cpu`, `memory`, `temperature`, `npu`, or `all` |
+| `<tool_7>` | `respond` | `message` | Natural-language fallback when no tool fits |
+
+Surface routing keyword: `set_neopixel_effect` requires the literal
+substring `neopixels` in the user input. LED-vs-ring ambiguous prompts
+("turn off the lights") route to `respond()` asking the user to
+disambiguate.
+
+## Output format — functional tokens
+
+Tool calls emit as **functional tokens**: each tool name compiles to a
+single special-vocabulary token (`<tool_0>` … `<tool_7>`) plus a single
+`<end>` terminator. A complete call decodes in 8–15 output tokens, vs
+~30–80 for the upstream native FunctionGemma
 `<start_function_call>call:NAME{...}<end_function_call>` syntax. On a
 2-core Cortex-A55 this is the difference between sub-second and 2–5 s
 voice-UX latency.
 
-Sample output: `<tool_3>(3,"red")<end>` for `blink_lights(count=3, color="red")`.
-
-`<tool_0>` → `turn_on_lights`, `<tool_3>` → `blink_lights`,
-`<tool_8>` → `get_system_status`, `<tool_9>` → `respond` (v7 numbering;
-v6 used `<tool_9>` and `<tool_10>` for those — bumped down by one when
-`list_alarms` was removed). Full mapping in [`token_map.json`](./token_map.json).
+Sample output: `<tool_0>("red","on")<end>` for `set_status_led(led="red", state="on")`.
 
 > ⚠️ Inference servers MUST stop generation on `<end_of_turn>` (or `<eos>`),
-> NOT on `<end>`. Multi-tool sequences emit `<tool_A>(args)<end><tool_B>(args)<end>`,
-> so stopping at the first `<end>` truncates legitimate multi-tool output.
+> NOT on `<end>`. The v9 model can emit multi-tool sequences
+> `<tool_A>(args)<end><tool_B>(args)<end>`, so stopping at the first
+> `<end>` truncates legitimate multi-tool output.
 
 ## Quick start (Ollama)
 
 ```bash
 hf download BrinqAI/functiongemma-270m-physical-ai \
-  functiongemma-physical-ai-Q4_K_M.gguf Modelfile tools.json token_map.json \
+  functiongemma-physical-ai-v9-Q5_K_M.gguf Modelfile tools.json token_map.json \
   --local-dir ./fg-physical-ai
 
 cd fg-physical-ai
 ollama create functiongemma-physical-ai -f Modelfile
 ```
 
-`ollama create -f Modelfile` is the documented install path because the
-shipped `Modelfile` bakes in the stop tokens (`<end>`, `<end_of_turn>`,
-`<eos>`) and decode parameters (`temperature=0`, `num_ctx=1024`,
-`num_predict=80`). Direct `ollama pull hf.co/...` does not apply these,
-and the function-token output will run past `<end>` until it hits
-`num_predict`. Only use the direct-pull path if your client injects stops
-at request time (the Python snippet below does this via `options.stop`).
+The shipped `Modelfile` bakes in the stop tokens (`<end_of_turn>`, `<eos>`)
+and decode parameters (`temperature=0`, `num_ctx=1024`, `num_predict=80`).
 
 ## Calling the model
 
-The model expects prompts built via the FunctionGemma chat template
-(developer role + user role, with the tools list passed in via
-`tokenizer.apply_chat_template(..., tools=tools)`). Send to Ollama with
-`raw=true` so it forwards the prompt verbatim. Plain `ollama run` from the
-CLI does **not** pass tools and will degenerate to chat-style refusals.
-
-Standalone client (depends on `transformers` for the chat template, plus
-the `tools.json` and `token_map.json` files in the same directory):
+The v9 model expects a **bare user turn** — no schema, no tools list. Send
+to Ollama with `raw=true`:
 
 ```python
 import json
 import re
 import urllib.request
-from transformers import AutoTokenizer
 
 OLLAMA_URL = "http://localhost:11434"
 MODEL = "functiongemma-physical-ai"
 
-tools = json.load(open("tools.json"))["tools"]
 reverse_token_map = json.load(open("token_map.json"))["reverse"]
-tokenizer = AutoTokenizer.from_pretrained("google/functiongemma-270m-it")
 
 
 def build_prompt(user_text: str) -> str:
-    msgs = [
-        {
-            "role": "developer",
-            "content": (
-                "You are a model that can do function calling with the "
-                "following functions\n"
-            ),
-            "tool_calls": None,
-        },
-        {"role": "user", "content": user_text, "tool_calls": None},
-    ]
-    return tokenizer.apply_chat_template(
-        msgs, tools=tools, tokenize=False, add_generation_prompt=True
+    return (
+        f"<start_of_turn>user\n{user_text}<end_of_turn>\n"
+        f"<start_of_turn>model\n"
     )
 
 
@@ -124,7 +138,7 @@ def call_model(user_text: str) -> str:
             "temperature": 0.0,
             "top_p": 1.0,
             "num_predict": 80,
-            "stop": ["<end>", "<end_of_turn>", "<eos>"],
+            "stop": ["<end_of_turn>", "<eos>"],
         },
     }).encode()
     req = urllib.request.Request(
@@ -145,79 +159,55 @@ def parse_call(raw: str) -> tuple[str | None, str]:
     return reverse_token_map.get(tok), args
 
 
-raw = call_model("Turn on the lights")
-print(raw)                  # e.g. <tool_0>()<end>
-print(parse_call(raw))      # ('turn_on_lights', '')
+raw = call_model("turn the red LED on")
+print(raw)               # e.g. '<tool_0>("red","on")<end>'
+print(parse_call(raw))   # ('set_status_led', '"red","on"')
 ```
 
+For `llama-cpp-python` directly, use `detokenize(..., special=True)` so
+the `<tool_N>` and `<end>` tokens render in the output instead of being
+stripped.
+
 ## Training data
 
-### v7 (current)
-
-- **Size**: 2,000 train / 250 eval (`coral_v7_compact.jsonl`).
-- **Schema change**: `list_alarms` removed. Out-of-scope alarm-query prompts
-  ("what alarms do I have?") are deliberately routed through `respond()`
-  rather than answered by a query tool. Compact token map shifted accordingly:
-  `get_system_status` is now `<tool_8>` (was `<tool_9>`), `respond` is
-  `<tool_9>` (was `<tool_10>`).
-- **Multi-tool**: 84 of 250 eval rows (33.6%) are multi-tool sequences,
-  matching the Google mobile-actions distribution.
-- **GGUF eval (Q5_K_M, greedy)**: overall **86.8%** (217/250), single-tool
-  **92.8%** (154/166), multi-tool exact-match **75.0%** (63/84), parse
-  failure **0.0%** (0/250). Per-tool F1 ranges from 0.74 (`respond`) to
-  1.00 (`cancel_alarm`).
-- **Known weak spots** (informal on-device REPL): "tell me a joke" / "what
-  alarms do I have" tend to misroute to `play_buzzer` instead of `respond` —
-  more `respond()` negatives sharing keywords with physical-action tools
-  would help in v8.
-
-### v6 (previous)
-
-- **Size**: 1,400 train / 150 eval (v5/v6 dataset lineage, `coral_v5_compact.jsonl`).
-- **Tool count**: 11. Cameras / vision tools dropped from earlier
-  checkpoints; alarm-list tool kept.
-
-### v4 (legacy)
-
-- **Size**: 367 train / 100 eval.
-- **Multi-tool**: 13% (vs Google mobile-actions 33.4%).
-- **Buzzer schema**: pattern-only (binary GPIO on the reference HAT — no
-  PWM). Old `frequency_hz` / `duration_seconds` prompts are routed
-  through `respond()` as out-of-scope negatives.
+v9's training data was generated from Haiku-authored phrasing templates
+crossed with deterministic entity pools, then lightly augmented with
+Moonshine-flavored ASR noise (dropped function words, lowercased traces,
+filler-word prepends). The shape matches Brinq's SmartPanel v15 trainer:
+flat `{input, output, split}` records, no tools / messages array.
+
+| | v9 |
+|---|---|
+| Train rows | 6,127 |
+| Eval rows | 1,339 |
+| Tools | 8 |
+| Multi-tool fraction | low — single-tool emphasis; multi-tool routines composed at dispatch time |
+| Augmentation | Moonshine-sim noise on ~30 % of records |
+
+Per-tool train counts (range 217–1,199; cancel_alarm + play_buzzer are the
+narrowest classes because their natural phrasing variation is smaller —
+not a coverage gap).
 
 ## Methodology
 
-This model uses the **functional-token** approach introduced by Octopus v2
-(Chen and Li, 2024): special vocabulary tokens are added for each callable
-function so a tool call decodes in a single output token rather than a
-multi-token JSON string. On-device this collapses ~30–80-token native
-FunctionGemma calls down to ~8–15 tokens, enabling sub-second decode on a
-2-core Cortex-A55.
-
-The training recipe is a direct port of Brinq's SmartPanel v14 trainer
-(full bf16, mean-init for new tokens, completion-only loss mask), adapted
-for a smaller dataset:
-
-- **Full bf16 fine-tune (no LoRA)**.
-- **Mean-init** for new `<tool_0>..<tool_9>` and `<end>` special tokens
-  (init = mean of existing input embeddings; random init under-converges
-  for tiny models on small datasets).
-- **Completion-only loss mask**: hand-rolled, masking everything before
-  `<start_of_turn>model\n`. TRL 0.25's `completion_only_loss=True` is a
-  no-op on flat-text data and FunctionGemma's chat template lacks
-  `{% generation %}` markers required for `assistant_only_loss`.
-- **8 epochs**, lr `3e-5`, cosine schedule, 0.1 warmup. (2,000 examples in
-  v7 — fewer epochs than v4's 15 because dataset size grew 5×.)
-- **Tool-token loss weight 4.0** to keep the new function tokens learning
-  faster than the rest of the vocabulary (Gemma3's 262k-vocab dilutes the
-  signal otherwise).
-- **Effective batch 16** = `per_device_train_batch_size=2 ×
-  gradient_accumulation_steps=8` (kept this way to avoid the 8 GiB
-  cross-entropy logit allocation OOM that bites Gemma3's 262k vocab).
-- **`max_length=1024`** to fit the full 13-tool schema in the prompt.
-- bf16, gradient checkpointing, `adamw_torch_fused`, `weight_decay=0.01`.
-- Trained inside `unsloth/unsloth:latest` Docker container with GPU
-  passthrough.
+Direct port of the SmartPanel v15 trainer:
+
+- **Full bf16 fine-tune** (no LoRA).
+- **Functional tokens**: `<tool_0>` … `<tool_7>` + `<end>` added as
+  `additional_special_tokens`; new embeddings **mean-initialized** from the
+  existing input embedding matrix (random init under-converges on small
+  datasets).
+- **Completion-only loss mask**: hand-rolled — labels before
+  `<start_of_turn>model\n` are masked to `-100`. The model learns only from
+  the assistant turn, not the user prompt.
+- **5 epochs**, lr `3e-5`, cosine schedule, 0.1 warmup, weight decay 0.01.
+- **Effective batch = 16** (`per_device_train_batch_size=8 ×
+  gradient_accumulation_steps=2`).
+- **`max_length=256`** — the trained prompt format is ~13 tokens and the
+  assistant turn fits comfortably under 64 tokens, including respond()
+  messages.
+- bf16, gradient checkpointing, `adamw_torch_fused`, `metric_for_best_model="eval_loss"` + `load_best_model_at_end=True`.
+- Training wallclock: **5 min on a single H100** (~15–20 min on a 4090).
 
 ### Citation
 
@@ -231,105 +221,54 @@ for a smaller dataset:
 }
 ```
 
-## Eval results
-
-**v7 checkpoint (2,000 train / 250 eval), Q5_K_M GGUF, greedy decode:**
-
-| Metric | Result |
-|--------|--------|
-| Overall accuracy | 217 / 250 = **86.8%** |
-| Single-tool accuracy | 154 / 166 = **92.8%** |
-| Multi-tool exact-match | 63 / 84 = **75.0%** |
-| Parse failure rate | 0 / 250 = **0.0%** |
+## Results
 
-Per-tool F1: `cancel_alarm` 1.00, `get_system_status` 0.96, `set_alarm` 0.93,
-`set_neopixel_pattern` 0.92, `turn_on_lights` 0.90, `blink_lights` 0.89,
-`turn_off_lights` 0.89, `set_led_color` 0.88, `play_buzzer` 0.83,
-`respond` 0.74. (`respond` is the lowest because the model occasionally
-chooses a physical-action tool with a hallucinated text argument when the
-prompt shares keywords with one — an issue the dispatcher's enum validation
-catches at runtime.)
+### Training metrics (final epoch)
 
-**On-device latency** (SL2619 / 2× Cortex-A55 @ 2 GHz, `performance` governor):
-~42 s cold prefill (one-time), ~1.6 s / turn warm — measured across a 33-prompt
-exhaustive REPL run on the actual Coralboard.
+| | v9 |
+|---|---|
+| Final train loss | 0.555 |
+| Final eval loss | **0.090** |
+| Mean token accuracy (eval) | **97.9 %** |
 
-## Latency
+### Held-out smoke test (post-train, 30 prompts spanning all 8 tools)
 
-- **~1.1 – 1.3 s** per call on a laptop CPU (Ollama / standalone client above).
-- **~1.6 s / turn warm**, ~42 s cold prefill on SL2619 (2× Cortex-A55 @ 2 GHz)
-  with the CPU governor pinned to `performance`. Measured 2026-05-05 on the
-  Grinn Coralboard with the v7 GGUF + the `Function_calling/` demo from
-  [BrinqAI/sl2610-examples](https://github.com/BrinqAI/sl2610-examples/tree/coralboard/functiongemma/Function_calling).
+| | v9 |
+|---|---|
+| Smoke-test routing accuracy | **30 / 30 (100 %)** |
 
-## ONNX exports (for compiler toolchains)
+The 30-prompt suite covers single-tool happy paths for every tool plus
+the failure modes that broke v8: ambiguous LED prompts ("turn off the
+lights"), effect-name without `neopixels` keyword ("do the aurora"),
+unsupported features ("play a tone at 2000 hz"), and out-of-scope
+appliances ("turn on the TV"). All 8 of those route to `respond()` with a
+helpful explanation.
 
-For compiler-targeted backends (ONNX Runtime, IREE/MLIR, OpenVINO, TensorRT,
-Synaptics Torq), the model is also published as ONNX with KV-cache support
-(`text-generation-with-past`). Both exports are derived from the same
-`coral-functiongemma-v4c-compact` checkpoint as the GGUF above.
+### On-device benchmark (Coralboard, 2-core Cortex-A55 @ 2 GHz, Q5_K_M GGUF)
 
-| Path | Precision | Weight init dtype | Size | ORT runnable |
-|------|-----------|-------------------|------|--------------|
-| `onnx/compact-fp32/model.onnx` | fp32 | 237 / 237 FLOAT | 1.7 GB | yes |
-| `onnx/compact-fp16/model.onnx` | fp16 | 237 / 237 FLOAT16 | 833 MB | no — see note |
-
-Both files are structurally valid (`onnx.checker.check_model(..., full_check=True)`
-passes). Each export ships with the matching tokenizer and `config.json` so it
-can be loaded directly:
-
-```python
-from transformers import AutoTokenizer
-import onnxruntime as ort
-import numpy as np, json
-
-MODEL = "onnx/compact-fp32"  # or downloaded local path
-tok = AutoTokenizer.from_pretrained(MODEL)
-sess = ort.InferenceSession(f"{MODEL}/model.onnx", providers=["CPUExecutionProvider"])
-
-tools = json.load(open("tools.json"))["tools"]
-prompt = tok.apply_chat_template(
-    [{"role": "developer",
-      "content": "You are a model that can do function calling with the following functions\n",
-      "tool_calls": None},
-     {"role": "user", "content": "Turn on the lights", "tool_calls": None}],
-    tools=tools, tokenize=False, add_generation_prompt=True,
-)
-# Then feed input_ids + empty past_key_values.* (shape (1, num_kv_heads, 0, head_dim))
-# greedy-decode in a loop, stop on <end>. See repo for full snippet.
-```
-
-Smoke decode of "Turn on the lights" against the fp32 ONNX returns
-`<tool_0>()<end>` (= `turn_on_lights()`), matching the GGUF output.
-
-### fp16 + ONNX Runtime caveat
-
-The fp16 ONNX file is structurally valid but **does not currently load in
-ONNX Runtime ≥ 1.20** for this model: ORT's `SimplifiedLayerNormFusion` pass
-chokes on the `InsertedPrecisionFreeCast_*` nodes that the fp16 conversion
-inserts around Gemma3's RMSNorm layers. The error is graph-optimizer-internal
-and reproduces with `ORT_DISABLE_ALL`. This is an ORT bug, not an ONNX-spec
-issue — the file passes `onnx.checker` and the graph is well-formed.
-
-For compiler frontends that consume ONNX directly (IREE / MLIR, TensorRT,
-OpenVINO, Synaptics Torq), the fp16 file should ingest fine. For runtime
-inference via `onnxruntime` itself, use the fp32 export and let your compiler
-or runtime do its own dtype conversion / quantization downstream.
+| | v7 (schema-in-prompt, 10 tools) | v9 (Octopus v2, 8 tools) |
+|---|---|---|
+| Prompt tokens | ~1088 | ~13 |
+| **Cold prefill (turn 1)** | **57.3 s** | **0.55 s** (105× faster) |
+| Warm prefill (turn 2+) | ~3 s | ~0.4 s |
+| Decode for a typical call | 0.5–1.2 s | 0.5–1.2 s |
+| End-to-end first-turn (model load 2.3 s + prefill + decode) | ~62 s | ~3 s |
+| Routing on a 29-prompt board bench | n/a directly comparable | **29 / 29 (100 %)** |
 
 ## Files
 
 ```
-functiongemma-physical-ai-v7-Q5_K_M.gguf  # 248 MB, GGUF Q5_K_M, 10-tool v7 schema (current)
-functiongemma-physical-ai-v6-Q5_K_M.gguf  # 248 MB, GGUF Q5_K_M, 11-tool v6 schema (previous)
-functiongemma-physical-ai-Q4_K_M.gguf     # 253 MB, GGUF Q4_K_M, v4c (legacy)
-Modelfile                                  # Ollama Modelfile (function-token format)
-tools.json                                 # 10-tool schema (mobile-actions format, current)
-token_map.json                             # function-token <-> tool-name map
-onnx/compact-fp32/                         # ONNX export, fp32, with KV cache (1.7 GB)
-onnx/compact-fp16/                         # ONNX export, fp16, with KV cache (833 MB) — see ORT caveat above
+functiongemma-physical-ai-v9-Q5_K_M.gguf   # ~248 MB, v9 GGUF Q5_K_M weights (Ollama / llama.cpp)
+Modelfile                                  # Ollama Modelfile (functional-token format)
+tools.json                                 # 8-tool schema (v9, canonical mobile-actions format)
+token_map.json                             # functional-token <-> tool-name map (v9)
 README.md                                  # this file
 ```
 
+Legacy v6/v7 GGUFs are kept in repo history for reproducibility but should
+not be used for new deployments — they require the schema-in-prompt
+inference wrapper and pay the ~57 s cold-prefill cost.
+
 ## License
 
 Released under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).
@@ -340,8 +279,7 @@ By using this model you agree to those terms. Base model:
 
 - Base model: <https://huggingface.co/google/functiongemma-270m-it>
 - Octopus v2 paper: <https://arxiv.org/abs/2404.01744>
-- Hardware demo (Coralboard, Google IO 2026 — full physical setup,
-  WLED-over-USB-CDC, Grinn HAT, end-to-end voice + text REPL):
-  <https://github.com/BrinqAI/sl2610-examples/tree/coralboard/functiongemma/Function_calling>
-  (BrinqAI fork of the upstream Synaptics demo repo,
-  [synaptics-astra-demos/sl2610-examples](https://github.com/synaptics-astra-demos/sl2610-examples)).
+- Hardware demo + integration code (Synaptics Coralboard, Grinn HAT,
+  WLED-over-USB-CDC, full PyQt UI):
+  <https://github.com/synaptics-astra-demos/sl2610-examples> →
+  `Function_calling/`

functiongemma-physical-ai-v9-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efa8d6f922ba40e0064b8efa3c330f36f843e933b62086c60964fbcafea22852
+size 260047008

token_map.json

@@ -1,30 +1,25 @@
 {
-  "version": "0.3.0",
-  "description": "Compressed token map for FunctionGemma CPU inference on SL2619. 10 tools — list_alarms removed in v7 (no alarm-query path; users cannot list alarms by design).",
+  "version": "0.4.0",
+  "description": "Compressed token map for FunctionGemma CPU inference on SL2619. v9: 8 tools (set_status_led, blink_status_led, set_neopixel_effect, play_buzzer, set_alarm, cancel_alarm, get_system_status, respond) + <end> terminator. Trained Octopus v2 style — functional tokens are the entire output vocabulary the model uses for routing; the tool schema (tools.json) is NOT loaded into the inference prompt. v9 drops the unused <tool_none> sentinel that v8's training pipeline reserved but never emitted.",
   "tokens": {
-    "turn_on_lights": "<tool_0>",
-    "turn_off_lights": "<tool_1>",
-    "set_led_color": "<tool_2>",
-    "blink_lights": "<tool_3>",
-    "set_neopixel_pattern": "<tool_4>",
-    "play_buzzer": "<tool_5>",
-    "set_alarm": "<tool_6>",
-    "cancel_alarm": "<tool_7>",
-    "get_system_status": "<tool_8>",
-    "respond": "<tool_9>"
+    "set_status_led": "<tool_0>",
+    "blink_status_led": "<tool_1>",
+    "set_neopixel_effect": "<tool_2>",
+    "play_buzzer": "<tool_3>",
+    "set_alarm": "<tool_4>",
+    "cancel_alarm": "<tool_5>",
+    "get_system_status": "<tool_6>",
+    "respond": "<tool_7>"
   },
   "reverse": {
-    "<tool_0>": "turn_on_lights",
-    "<tool_1>": "turn_off_lights",
-    "<tool_2>": "set_led_color",
-    "<tool_3>": "blink_lights",
-    "<tool_4>": "set_neopixel_pattern",
-    "<tool_5>": "play_buzzer",
-    "<tool_6>": "set_alarm",
-    "<tool_7>": "cancel_alarm",
-    "<tool_8>": "get_system_status",
-    "<tool_9>": "respond",
-    "<tool_none>": null
+    "<tool_0>": "set_status_led",
+    "<tool_1>": "blink_status_led",
+    "<tool_2>": "set_neopixel_effect",
+    "<tool_3>": "play_buzzer",
+    "<tool_4>": "set_alarm",
+    "<tool_5>": "cancel_alarm",
+    "<tool_6>": "get_system_status",
+    "<tool_7>": "respond"
   },
   "special_tokens": [
     "<tool_0>",
@@ -35,11 +30,9 @@
     "<tool_5>",
     "<tool_6>",
     "<tool_7>",
-    "<tool_8>",
-    "<tool_9>",
-    "<tool_none>",
     "<end>"
   ],
   "output_format": "<tool_N>(\"arg1\",\"arg2\",...)<end>",
-  "notes": "Argument order positional per canonical schema's required-first then optional declaration order. Camera/vision tools (capture_photo, describe_scene) removed in v6. list_alarms removed in v7 — users cannot query alarms; out-of-scope alarm-query prompts route via respond()."
+  "prompt_format": "<start_of_turn>user\\n{user_text}<end_of_turn>\\n<start_of_turn>model\\n",
+  "notes": "Argument order positional per canonical schema's required-first then optional declaration order. v9 trains Octopus v2 pure (no schema in prompt) — see prompt_format. set_neopixel_effect routing keyword: 'neopixels' (literal substring, case-insensitive) required in user prompt; otherwise the model routes to respond() asking the user to disambiguate (HAT status LEDs vs. neopixel ring)."
 }

tools.json

@@ -1,95 +1,84 @@
 {
-  "version": "0.2.0",
-  "description": "Physical AI tool schema for Coral Dev Board (SL2619) FunctionGemma demo. Canonical mobile-actions format. v7: 10 tools (list_alarms removed; out-of-scope alarm-query prompts route via respond()).",
+  "version": "0.4.0",
+  "description": "Physical AI tool schema for Coral Dev Board (SL2619) FunctionGemma demo. Canonical mobile-actions format. v9: same 8-tool surface as v8, but the model is trained Octopus v2 style — functional tokens (<tool_0>..<tool_7>) emitted directly from a minimal user-only prompt with NO tool schema in the context. This shrinks the cold prefill from ~1088 prompt tokens (v7/v8 with the FunctionGemma developer turn) to ~13, taking on-board cold first-turn from ~57s to ~0.5s on a 2-core A55. The schema in this file is purely a developer/runtime contract (dispatcher arg validation, GGUF metadata, documentation) — it is NOT injected into the inference prompt. Surface routing keyword: 'neopixels' (literal) for the ring; 'LED' / 'light' / 'red light' / 'green light' / 'blue light' for the HAT status LEDs.",
   "tools": [
     {
       "function": {
-        "name": "turn_on_lights",
-        "description": "Turn on all RGB LEDs and the Neopixel strip to default white.",
-        "parameters": {
-          "type": "OBJECT",
-          "properties": {}
-        }
-      }
-    },
-    {
-      "function": {
-        "name": "turn_off_lights",
-        "description": "Turn off all RGB LEDs and the Neopixel strip.",
-        "parameters": {
-          "type": "OBJECT",
-          "properties": {}
-        }
-      }
-    },
-    {
-      "function": {
-        "name": "set_led_color",
-        "description": "Set the color of RGB LEDs on the HAT or Neopixel strip.",
+        "name": "set_status_led",
+        "description": "Turn one or all of the HAT status LEDs on or off. The HAT has three individual LEDs (red, green, blue), each independently addressable. Invoke when the user mentions 'LED', 'LEDs', or 'the <color> light'.",
         "parameters": {
           "type": "OBJECT",
           "properties": {
-            "color": {
+            "led": {
               "type": "STRING",
-              "description": "Color name (e.g. 'red', 'green', 'blue', 'white', 'orange', 'purple') or 6-digit hex (e.g. '#FF8800')."
+              "description": "Which LED: 'red', 'green', 'blue', or 'all'."
             },
-            "target": {
+            "state": {
               "type": "STRING",
-              "description": "Which lights to set: 'all' (default), 'hat' (RGB LEDs on HAT), or 'strip' (Neopixels)."
+              "description": "'on' or 'off'."
             },
             "brightness": {
               "type": "INTEGER",
-              "description": "Brightness 0-100. Optional, default 100."
+              "description": "Brightness 0-100. Optional, default 100. Ignored when state is 'off'."
             }
           },
-          "required": ["color"]
+          "required": ["led", "state"]
         }
       }
     },
     {
       "function": {
-        "name": "blink_lights",
-        "description": "Blink LEDs a given number of times, optionally in a specific color and speed.",
+        "name": "blink_status_led",
+        "description": "Blink one or all HAT status LEDs a given number of times. Invoke for 'blink/flash the <color> light' or 'blink the LEDs' style requests.",
         "parameters": {
           "type": "OBJECT",
           "properties": {
+            "led": {
+              "type": "STRING",
+              "description": "Which LED: 'red', 'green', 'blue', or 'all'."
+            },
             "count": {
               "type": "INTEGER",
               "description": "Number of blinks. Default 3."
             },
-            "color": {
-              "type": "STRING",
-              "description": "Color to blink. Default current color or white."
-            },
             "speed": {
               "type": "STRING",
               "description": "One of 'slow', 'normal', 'fast'. Default 'normal'."
             }
-          }
+          },
+          "required": ["led"]
         }
       }
     },
     {
       "function": {
-        "name": "set_neopixel_pattern",
-        "description": "Play an animated pattern on the Neopixel strip.",
+        "name": "set_neopixel_effect",
+        "description": "Play a visual effect on the neopixel ring (48-pixel WS2812B ring around the 7\" display, driven by WLED). Only invoke when the user explicitly mentions 'neopixels'. Use effect='off' to turn the ring off.",
         "parameters": {
           "type": "OBJECT",
           "properties": {
-            "pattern": {
+            "effect": {
               "type": "STRING",
-              "description": "One of 'rainbow', 'chase', 'fade', 'pulse', 'sparkle', 'solid'."
+              "description": "One of: 'solid', 'pulse', 'fade', 'chase', 'rainbow', 'sparkle', 'off', 'aurora', 'plasma', 'comet', 'twinkle', 'fireworks', 'police', 'heartbeat', 'loading', 'lightning', 'glitter', 'fire', 'sunrise'."
             },
             "color": {
               "type": "STRING",
-              "description": "Color for patterns that need one (chase/fade/pulse/solid). Ignored for rainbow."
+              "description": "Color name (e.g. 'red', 'teal', 'amber', 'gold', 'violet') or 6-digit hex like '#FF8800'. Used by effects that take a primary color (solid, pulse, fade, chase, sparkle, comet). Ignored for rainbow and palette-driven effects."
+            },
+            "palette": {
+              "type": "STRING",
+              "description": "Color palette: 'auto', 'ocean', 'lava', 'forest', 'sunset', 'party', 'sherbet', 'c9', 'aurora', 'beach', 'fire', 'sakura', 'splash', 'pastel'. Most useful with aurora, plasma, fire, twinkle, comet."
             },
             "speed": {
               "type": "STRING",
               "description": "One of 'slow', 'normal', 'fast'. Default 'normal'."
+            },
+            "intensity": {
+              "type": "STRING",
+              "description": "One of 'low', 'medium', 'high'. Default 'medium'. Controls effect density / depth (sparkle density, fire height, comet tail length, aurora width)."
             }
           },
-          "required": ["pattern"]
+          "required": ["effect"]
         }
       }
     },
@@ -165,7 +154,7 @@
     {
       "function": {
         "name": "respond",
-        "description": "Reply to the user in natural language without taking any physical action. Use this when the request is out of scope (no matching tool), ambiguous (needs clarification), purely conversational (greetings, thanks), or impossible on this device. Do NOT use this when any physical-action tool fits the request.",
+        "description": "Reply to the user in natural language without taking any physical action. Use this when the request is out of scope (no matching tool), ambiguous (needs clarification — e.g. surface keyword missing for LED requests), purely conversational (greetings, thanks), or impossible on this device. Do NOT use this when any physical-action tool fits the request.",
         "parameters": {
           "type": "OBJECT",
           "properties": {