Upload RELATED_WORK.md

RELATED_WORK.md

@@ -0,0 +1,43 @@
+# Speculative Tool Actions — Related Work
+
+## Foundational
+
+**SpecInfer (2023)** — *Accelerating Generative LLM Serving with Tree-based Speculative Inference* (arxiv:2305.09781). Introduced token-tree speculative decoding. Multiple draft tokens from small models verified in parallel by the large model. Basis for all subsequent speculative decoding work.
+
+**SuffixDecoding (2024)** — *SuffixDecoding: Speeding Up Large Language Model Inference with Tree-structured Suffix-based Drafting* (arxiv:2411.04975). Applied speculative decoding to agentic workloads by caching action suffixes and reusing them as drafts. Achieved up to 5.3× speedup on tool-calling benchmarks.
+
+## Heterogeneous Speculation for Agents (2025-2026)
+
+**DualSpec (Mar 2026)** — *DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculation* (arxiv:2603.07416). Closest work to ours. Uses heterogeneous speculation: large model handles high-entropy "Search" actions; small model drafts low-entropy "Visit" actions. Semantic verifier (prompt-based, not trained) accepts/rejects drafts. **1.33-3.28× end-to-end speedup** with no pass@1 loss on GAIA, XBench-DeepSearch, Seal-0. Key insight: **entropy-based action partitioning** — some actions need System 2, others are fine with System 1.
+
+**Our contribution vs DualSpec:** We train a separate verifier model (SFT on ACCEPT/REJECT pairs) instead of using prompt-based critics. This should be both faster and more accurate. We also use a single cheap model for all actions rather than partitioning by type.
+
+**DSP (Aug 2025)** — *Dynamic Speculative Agent Planning* (arxiv:2509.01920). Uses online RL to predict how many speculative steps the drafter can produce correctly. Models optimal k as varying from 1-5 even within a single task (mean variance 1.46). **~2× latency reduction** with **30% lower total cost**.
+
+**Our contribution vs DSP:** DSP uses exact action matching; we train a learned verifier. DSP predicts k dynamically; we use single-action proposal + verify (k=1 by design). The cost breakdown analysis in DSP directly motivates our approach: draft prompt tokens dominate waste cost.
+
+**SpecEyes (Mar 2026)** — *SpecEyes: Accelerating Agentic Multimodal LLMs* (arxiv:2603.23483). For multimodal agents: lightweight MLLM drafts answers; "cognitive gating" via answer separability score decides accept/fallback. **1.1-3.35× speedup** with **+6.7% accuracy gain**.
+
+## Small Models for Tool Use
+
+**TinyAgent (Sep 2024)** — *TinyAgent: Function Calling at the Edge* (arxiv:2409.00608). TinyLlama-1.1B and Wizard-2-7B **match or surpass GPT-4-Turbo** on function calling via SFT on LLMCompiler-format traces + Tool RAG. Demonstrates that small models can be extremely capable for tool use when trained properly.
+
+**SLM for Agentic Systems Survey (Oct 2025)** — *Small Language Models for Agentic Systems* (arxiv:2510.03847). Formalizes the SLM-default/LLM-fallback pattern with uncertainty-aware routing and verifier cascades. Recommends schema-first prompting, type-safe function registries, and LoRA adaptation.
+
+## Learned Routers and Verifiers
+
+**RouteLLM (Jun 2024)** — *RouteLLM: Learning to Route LLMs with Preference Data* (arxiv:2406.18665). Trains routers (BERT, causal LLM, matrix factorization) on Chatbot Arena preference data. **>2× cost reduction** with no quality degradation. BERT router achieves best cost-quality tradeoff.
+
+**Internal Representation Hallucination Detection (Jan 2026)** — *Internal Representations as Indicators of Hallucinations in Agent Tool Selection* (arxiv:2601.05214). **86.4% accuracy** detecting tool-calling hallucinations in a single forward pass via 2-layer MLP on final hidden states. Validates the approach of using separate classifiers to verify tool-call quality.
+
+## How Our Work Differs
+
+| Aspect | Prior Work | Our Work |
+|--------|-----------|----------|
+| Speculation target | Tokens (SpecInfer), actions+reasoning (DualSpec), multi-step plans (DSP) | Single next action type |
+| Verifier type | Exact match (DSP), prompt-based critic (DualSpec), confidence heuristic (SpecEyes) | **Trained SFT classifier** |
+| Model sizes | 72B+8B or 32B+4B asymmetric pairs | 8B + 1.7B + 4B verifier |
+| Training data | Proprietary (DualSpec, DSP) or synthesized (TinyAgent) | ToolBench-derived, open |
+| Safety focus | None | Explicit BLOCKED action + unsafe-action avoidance metric |
+
+**Novel contribution:** First system to use a **trained SFT verifier** for speculative tool action proposal verification. All prior work uses either exact matching, prompt-based critics, or confidence heuristics. Our approach has better accuracy:latency ratio by design.