SpecDecode Lab

An interactive lab for speculative LLM decoding — linear, tree-based, throughput & acceptance curves.
100% client-side No backend Open source
Linear Speculative Decoding
Draft model proposes K tokens; target model verifies them in parallel and accepts the longest matching prefix, then bonus-samples one extra token. Step through a real cycle.
draft proposal accepted rejected committed
Committed sequence
Current cycle
Cycles
0
Tokens committed
0
Avg accepted / cycle
0.00
Wall time
0 ms
Speedup vs vanilla
1.00×
How verification works (math)
Given draft probs q(x) and target probs p(x), accept token x with probability min(1, p(x)/q(x)). On the first rejection at position i, resample from the residual distribution p'(x) = max(0, p(x) - q(x)) / Z This is provably equivalent to sampling from p alone — it is a *lossless* speedup. After accepting i tokens out of K, target's free forward gives one bonus token, so each cycle yields i+1 tokens for the cost of 1 target call + K draft calls.
Tree-based Decoding (Medusa / EAGLE intuition)
Instead of one linear draft, the draft model proposes a tree of candidate continuations. The target model verifies the entire tree in one forward pass with a custom causal mask, then commits the longest accepted path.
Drafted tree (* = accepted by target, / = rejected)
Stats over 200 simulated cycles
Tree size
0
Avg accepted depth
0
Tokens / cycle (linear)
0
Tokens / cycle (tree)
0
Throughput Simulator
Closed-form expected speedup of speculative decoding (Leviathan et al., 2023): E[tokens/cycle] = (1 − α^(K+1)) / (1 − α)  ·  cost/cycle = T_target + K · T_draft
Optimal K*
—
Speedup at K*
—
Tokens/sec vanilla
—
Tokens/sec spec
—
Acceptance vs. Speedup Surface
How speedup depends jointly on the acceptance rate α and the draft/target latency ratio c = T_d / T_t. Each curve is a fixed K.
Speedup(K, α, c) = (1 − α^(K+1)) / ((1 − α)(1 + Kc))
About SpecDecode Lab

Speculative decoding is the dominant family of inference acceleration techniques behind production LLMs in 2025–2026 — every major Bay Area inference shop (vLLM, TensorRT-LLM, SGLang, Anthropic, OpenAI, Together, Fireworks, Groq) ships some flavor of it. But the underlying probabilistic mechanism is rarely visible. This lab makes the math, the token-by-token verification dance, and the throughput trade-offs tangible.

Built fully client-side: no API keys, no servers, no telemetry. The probabilities are simulated so you can sweep parameter space at 60 fps.

References: Leviathan, Kalman & Matias 2023 (Fast Inference from Transformers via Speculative Decoding); Chen et al. 2023 (Accelerating Large Language Model Decoding with Speculative Sampling); Cai et al. 2024 (Medusa); Li et al. 2024 (EAGLE).

Built autonomously by Claude as part of Surya's daily-webapp series. MIT licensed. Source: GitHub