Runtime's speed guide reports p50, p95 and p99, with the sample counts, for each operation within 2× of its floor (speed).
Why it matters for AI agents
An average can look fine while one request in twenty is very slow. Google's SRE book puts it this way: "a typical request is served in about 50 ms". Yet "5% of requests are 20 times slower". Monitoring the average "would show no change". An agent feels the tail more than most software does, for two reasons.
- Steps compound. A task of twenty tool calls is likely to hit a slow one. If each call has a 5% chance of exceeding its p95, the chance that all twenty stay under it is 0.95 to the 20th power, about 36%.
- People wait on the slowest step. A chat answer that needs a sandbox is only as fast as the start it waited for, not the typical start.
Read the median, p95 and p99 together to see the shape of the data. The median is the usual case. In the SRE book's words, a high percentile "shows you a plausible worst-case value".
Percentiles in facts
| Term | Meaning |
|---|---|
| p50, the median | Half of requests are faster, half slower |
| p95 | 95% finish within it |
| p99 | 99% finish within it; needs many samples to mean much |
| Nearest-rank p95 | Sort n values and take the one at position ceil(0.95 × n) |
| With 20 samples | Nearest-rank p95 is the 19th fastest, so the single slowest does not set it |
| With 10 samples | p95 and p99 are both the slowest value |
Runtime's published startup figures use nearest-rank percentiles.
Measure your own
Time the calls your agent makes. Then read the percentiles, not the mean. This example measures commands in one sandbox that is already running:
TypeScriptimport { Sandbox } from "withruntime";await using sbx = await Sandbox.create();const times: number[] = [];for (let i = 0; i < 20; i++) { const started = performance.now(); await sbx.exec(["python3", "-c", "pass"]); times.push(performance.now() - started);}times.sort((a, b) => a - b);const at = (p: number) => times[Math.ceil(times.length * p) - 1]!;console.log(`p50 ${Math.round(at(0.5))} ms, p95 ${Math.round(at(0.95))} ms`);Pythonimport mathimport timefrom withruntime import Sandboxwith Sandbox.create() as sbx: times = [] for _ in range(20): started = time.perf_counter() sbx.exec(["python3", "-c", "pass"]) times.append((time.perf_counter() - started) * 1000) times.sort() at = lambda p: times[math.ceil(len(times) * p) - 1] print(f"p50 {at(0.5):.0f} ms, p95 {at(0.95):.0f} ms")State where you measured from, how many samples you took, and whether the network is included. A percentile without these details is hard to compare. The full startup script is in speed.
Related
- What is a cold start?
- What is a warm pool?
- Firecracker vs gVisor
- Speed
- Choosing an agent sandbox
- A code interpreter for chatbots
Sources
- Google SRE book: Service Level Objectives, on percentiles and tail latency, read 25 October 2026
Facts on this page were checked on 25 October 2026.