On Runtime, scoring 200,000 sampled programs costs $7.78. Each program gets its own Firecracker microVM with the internet off. So a completion that deletes files, forks without end or tries to fetch the answer harms only its own machine.
Why execution makes a good reward
For code, the cheapest trustworthy reward is to run the program. A policy that writes a function is judged by what the function returns, not by how the text looks. The score needs no second model. The difficulty is that the policy writes code nobody has read, thousands of times a minute. Early in training, much of it is broken or hostile by accident. The grader has to be:
- Isolated, so one completion cannot touch another or your trainer.
- Offline, so no completion can look an answer up or send a test out.
- Bounded in time, so an infinite loop scores zero instead of stalling a batch.
- Hard to game, so the reward measures the program, not its printouts.
Keep the verdict outside the sandbox
A reward computed inside the sandbox can be forged by the code being scored. The code can print "all tests passed", edit the harness or read the expected answers. Keep the expected outputs in your trainer. Send only the input of each test case into the sandbox as standard input. Bring back standard output, and compare it in your own process:
TypeScriptimport { Runtime } from "withruntime";type Case = { stdin: string; expected: string };const runtime = new Runtime({ waitForCapacityMs: 600_000 });export async function reward(program: string, cases: Case[]): Promise<number> { await using sbx = await runtime.sandboxes.create({ vcpu: 1, memoryMiB: 2048, network: { internet: false }, timeoutSeconds: 300, onLeaseEnd: "stop", labels: { run: "grpo-2026-09-25" }, }); await sbx.files.write("/workspace/solution.py", program); let passed = 0; for (const test of cases) { const run = await sbx.exec(["python3", "/workspace/solution.py"], { stdin: test.stdin, timeoutMs: 5_000, }); if (!run.timedOut && run.exitCode === 0 && run.stdout.trim() === test.expected.trim()) passed++; } return passed / cases.length;}// A batch of completions, scored side by side.export const rewards = (programs: string[], cases: Case[]) => Promise.all(programs.map((program) => reward(program, cases)));Pythonimport asynciofrom withruntime import AsyncRuntimeasync def reward(runtime, program: str, cases: list[dict]) -> float: async with await runtime.sandboxes.create( vcpu=1, memory_mib=2048, network={"internet": False}, timeout_seconds=300, on_lease_end="stop", labels={"run": "grpo-2026-09-25"}, ) as sbx: await sbx.files.write("/workspace/solution.py", program) passed = 0 for case in cases: run = await sbx.exec(["python3", "/workspace/solution.py"], stdin=case["stdin"], timeout_ms=5_000) if not run.timed_out and run.exit_code == 0 and run.stdout.strip() == case["expected"].strip(): passed += 1 return passed / len(cases)async def rewards(programs: list[str], cases: list[dict]) -> list[float]: async with AsyncRuntime(wait_for_capacity=600) as runtime: return await asyncio.gather(*(reward(runtime, p, cases) for p in programs))The command is an array, so nothing in the program's file name or arguments
reaches a shell. A test that runs past five seconds comes back with
timedOut: true and scores nothing. The sandbox is deleted when the block ends,
whether the program behaved or not.
Reward design choices
| Choice | What it gives | How to do it here |
|---|---|---|
| Share of cases passed | A dense signal early in training | passed / cases.length, as above |
| All or nothing | A strict signal once the policy is competent | Return passed === cases.length ? 1 : 0 |
| Penalty for crashes | Separates wrong answers from broken programs | Score exitCode !== 0 below a wrong answer |
| Penalty for slow programs | Pushes toward efficient solutions | Lower timeoutMs, or score on elapsed time |
| Unit tests, not I/O | Rewards functions rather than whole programs | Run your test file after the program, as below |
For function-level tasks, write the program and a test file. Run the tests, and read their exit code. The tests then live in the sandbox. So treat that score as the program's own claim. Keep a held-out I/O check outside the sandbox for evaluation.
Warm start for heavier graders
When a task needs packages, such as NumPy or a course's own library, prepare one sandbox, keep it as a snapshot, and create each grader from it. The copy starts with everything installed, and with the source's size:
TypeScriptimport { Runtime } from "withruntime";const runtime = new Runtime();const base = await runtime.sandboxes.create({ vcpu: 1, memoryMiB: 2048, network: { internet: true, allow: ["pypi.org", "*.pythonhosted.org"] },});await base.exec("pip install --quiet sympy", { check: true, timeoutMs: 180_000 });await base.network.set({ internet: false });const grader = await base.snapshot({ name: "grader-v1", retentionDays: 30 });await base.stop();await using sbx = await runtime.sandboxes.create({ snapshot: grader.id, network: { internet: false },});console.log((await sbx.exec(["python3", "-c", "import sympy; print(sympy.__version__)"])).stdout);The package registry is open only while the source installs. Each grader made from the snapshot starts with the internet off. A kept snapshot is billed as storage at $0.08 per GB per 30-day month, on the bytes it alone stores (pricing).
What a code-reward grader needs
| Need | How Runtime covers it |
|---|---|
| One program cannot reach another | A Firecracker microVM with its own kernel per completion |
| No looking answers up | network: { internet: false }, enforced on the host, root included |
| Infinite loops | timeoutMs per test case; the result says it timed out |
| A flood of printed output | At most 64 KiB of stdout and 64 KiB of stderr per result, with flags |
| Large batches | 100 sandboxes at the same time on a paid account; more on request |
| Bursts past the limit | Creates, wakes and restarts wait for room (waitForCapacityMs) |
| Identical graders every step | Create from a snapshot or a versioned custom image |
| A training bill with a ceiling | A daily spending limit per key; maxTotalCostMicros per sandbox |
What it costs
Take a training run that scores 200,000 completions. Each keeps a 1 vCPU, 2 GiB sandbox running for 6 seconds, from start to stop. Its ten test cases use 2 CPU-seconds:
- CPU
- 200,000 × 2 s / 3,600 × $0.025
- $2.78
- Memory
- 200,000 × 6 s / 3,600 × 2 GiB × $0.0075
- $5.00
- Total
- $7.78
2 CPU-seconds in 6 seconds is above the floor of a twentieth of a vCPU, so the floor adds nothing. Compute also has no one-minute minimum (pricing). Model inference is billed by whoever serves your policy. Every account gets 100 free hours of a 1 GB machine in its first 30 days, with no card.
Start
Terminalprintf '%s\n' 'print(sum([125, 250, 375]))' > invoice.pynpx withruntime sandboxes run -- python3 - < invoice.pyThe first run prints a link to approve in your browser. Then use the JavaScript or Python SDK as above.
Related: RL environments for agents, grade student code safely, agent evals and SWE-bench, turn off sandbox internet, sandbox cost calculator.
Facts on this page were checked on 26 October 2026.