What is p95 latency?
p95 latency is the time within which 95% of requests finish; the slowest 5% take longer, so it shows the slow cases an average hides.
Runtime publishes its p95, not only its median: 815 ms from the create request to the first Python result, against a median of 351 ms, across 20 sequential starts from a laptop through the public API on 24 September 2026 (speed).
Why it matters for AI agents
An average can look fine while one request in twenty is very slow. Google's SRE book puts it this way: "a typical request is served in about 50 ms", yet "5% of requests are 20 times slower", and monitoring the average "would show no change". An agent feels the tail more than most software, for two reasons.
- Steps compound. A task of twenty tool calls is likely to meet a slow one. If each call has a 5% chance of passing its p95, the chance that all twenty stay under it is 0.95 to the 20th, about 36%.
- People wait on the slowest step. A chat answer that needs a sandbox is as slow as the start it waited for, not the typical one.
Median, p95 and p99 together describe the shape: the median is the usual case, and a high percentile, in the SRE book's words, "shows you a plausible worst-case value".
Percentiles in facts
| Term | Meaning |
|---|---|
| p50, the median | Half of requests are faster, half slower |
| p95 | 95% finish within it |
| p99 | 99% finish within it; needs many samples to mean much |
| Nearest-rank p95 | Sort n values and take the one at position ceil(0.95 × n) |
| With 20 samples | Nearest-rank p95 is the 19th fastest, so the single slowest does not set it |
| With 10 samples | p95 and p99 are both the slowest value |
Runtime's published startup figures use nearest-rank percentiles. The matched run on 24 September 2026 also measured create until running at a 207 ms median and 620 ms p95, and the first Python request alone at 140 ms and 226 ms.
Measure your own
Time the calls your agent makes, then read the percentiles rather than the mean. This measures commands in one sandbox that is already running:
TypeScriptimport { Sandbox } from "withruntime";await using sbx = await Sandbox.create();const times: number[] = [];for (let i = 0; i < 20; i++) { const started = performance.now(); await sbx.exec(["python3", "-c", "pass"]); times.push(performance.now() - started);}times.sort((a, b) => a - b);const at = (p: number) => times[Math.ceil(times.length * p) - 1]!;console.log(`p50 ${Math.round(at(0.5))} ms, p95 ${Math.round(at(0.95))} ms`);Pythonimport mathimport timefrom withruntime import Sandboxwith Sandbox.create() as sbx: times = [] for _ in range(20): started = time.perf_counter() sbx.exec(["python3", "-c", "pass"]) times.append((time.perf_counter() - started) * 1000) times.sort() at = lambda p: times[math.ceil(len(times) * p) - 1] print(f"p50 {at(0.5):.0f} ms, p95 {at(0.95):.0f} ms")State where you measured from, how many samples, and whether the network is included; a percentile without those is hard to compare. The full startup script is in speed.
Related
- What is a cold start?
- What is a warm pool?
- Firecracker vs gVisor
- Speed
- Choosing an agent sandbox
- A code interpreter for chatbots
Sources
- Google SRE book: Service Level Objectives, on percentiles and tail latency, read 25 September 2026
Facts on this page were checked on 25 September 2026.