# Speculative Tool Actions — Related Work

## Foundational

**SpecInfer (2023)** — *Accelerating Generative LLM Serving with Tree-based Speculative Inference* (arxiv:2305.09781). Introduced token-tree speculative decoding. Multiple draft tokens from small models verified in parallel by the large model. Basis for all subsequent speculative decoding work.

**SuffixDecoding (2024)** — *SuffixDecoding: Speeding Up Large Language Model Inference with Tree-structured Suffix-based Drafting* (arxiv:2411.04975). Applied speculative decoding to agentic workloads by caching action suffixes and reusing them as drafts. Achieved up to 5.3× speedup on tool-calling benchmarks.

## Heterogeneous Speculation for Agents (2025-2026)

**DualSpec (Mar 2026)** — *DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculation* (arxiv:2603.07416). Closest work to ours. Uses heterogeneous speculation: large model handles high-entropy "Search" actions; small model drafts low-entropy "Visit" actions. Semantic verifier (prompt-based, not trained) accepts/rejects drafts. **1.33-3.28× end-to-end speedup** with no pass@1 loss on GAIA, XBench-DeepSearch, Seal-0. Key insight: **entropy-based action partitioning** — some actions need System 2, others are fine with System 1.

**Our contribution vs DualSpec:** We train a separate verifier model (SFT on ACCEPT/REJECT pairs) instead of using prompt-based critics. This should be both faster and more accurate. We also use a single cheap model for all actions rather than partitioning by type.

**DSP (Aug 2025)** — *Dynamic Speculative Agent Planning* (arxiv:2509.01920). Uses online RL to predict how many speculative steps the drafter can produce correctly. Models optimal k as varying from 1-5 even within a single task (mean variance 1.46). **~2× latency reduction** with **30% lower total cost**.

**Our contribution vs DSP:** DSP uses exact action matching; we train a learned verifier. DSP predicts k dynamically; we use single-action proposal + verify (k=1 by design). The cost breakdown analysis in DSP directly motivates our approach: draft prompt tokens dominate waste cost.

**SpecEyes (Mar 2026)** — *SpecEyes: Accelerating Agentic Multimodal LLMs* (arxiv:2603.23483). For multimodal agents: lightweight MLLM drafts answers; "cognitive gating" via answer separability score decides accept/fallback. **1.1-3.35× speedup** with **+6.7% accuracy gain**.

## Small Models for Tool Use

**TinyAgent (Sep 2024)** — *TinyAgent: Function Calling at the Edge* (arxiv:2409.00608). TinyLlama-1.1B and Wizard-2-7B **match or surpass GPT-4-Turbo** on function calling via SFT on LLMCompiler-format traces + Tool RAG. Demonstrates that small models can be extremely capable for tool use when trained properly.

**SLM for Agentic Systems Survey (Oct 2025)** — *Small Language Models for Agentic Systems* (arxiv:2510.03847). Formalizes the SLM-default/LLM-fallback pattern with uncertainty-aware routing and verifier cascades. Recommends schema-first prompting, type-safe function registries, and LoRA adaptation.

## Learned Routers and Verifiers

**RouteLLM (Jun 2024)** — *RouteLLM: Learning to Route LLMs with Preference Data* (arxiv:2406.18665). Trains routers (BERT, causal LLM, matrix factorization) on Chatbot Arena preference data. **>2× cost reduction** with no quality degradation. BERT router achieves best cost-quality tradeoff.

**Internal Representation Hallucination Detection (Jan 2026)** — *Internal Representations as Indicators of Hallucinations in Agent Tool Selection* (arxiv:2601.05214). **86.4% accuracy** detecting tool-calling hallucinations in a single forward pass via 2-layer MLP on final hidden states. Validates the approach of using separate classifiers to verify tool-call quality.

## How Our Work Differs

| Aspect | Prior Work | Our Work |
|--------|-----------|----------|
| Speculation target | Tokens (SpecInfer), actions+reasoning (DualSpec), multi-step plans (DSP) | Single next action type |
| Verifier type | Exact match (DSP), prompt-based critic (DualSpec), confidence heuristic (SpecEyes) | **Trained SFT classifier** |
| Model sizes | 72B+8B or 32B+4B asymmetric pairs | 8B + 1.7B + 4B verifier |
| Training data | Proprietary (DualSpec, DSP) or synthesized (TinyAgent) | ToolBench-derived, open |
| Safety focus | None | Explicit BLOCKED action + unsafe-action avoidance metric |

**Novel contribution:** First system to use a **trained SFT verifier** for speculative tool action proposal verification. All prior work uses either exact matching, prompt-based critics, or confidence heuristics. Our approach has better accuracy:latency ratio by design.