How to grade student code safely in a sandbox

Run each submission in its own throwaway microVM with the internet off and a time limit, run your tests against it, and record the score.

On Runtime grading 1,500 submissions costs $0.29. Each submission gets its own Firecracker microVM, so one student's code cannot read another's, reach your network or outlive its time limit. Runtime bills the CPU the tests use, $0.025 per vCPU-hour, with memory at $0.0075 per GiB-hour; the working is at the end of this page.

The short answer

One sandbox per submission. Copy the student's files and your hidden tests in, cut the network, run the tests, keep the result.

TypeScriptimport { Sandbox } from "withruntime";export async function grade(studentDir: string, testsDir: string) {  await using sbx = await Sandbox.create({    vcpu: 1,    memoryMiB: 2048,    network: { internet: false },    timeoutSeconds: 300,    onLeaseEnd: "stop",  });  await sbx.files.upload(studentDir, "/workspace/submission");  await sbx.files.upload(testsDir, "/workspace/tests");  const run = await sbx.exec(    ["python3", "-m", "unittest", "discover", "-s", "/workspace/tests", "-t", "/workspace"],    { cwd: "/workspace/submission", timeoutMs: 60_000 },  );  return {    passed: run.exitCode === 0 && !run.timedOut,    timedOut: run.timedOut,    output: run.stderr.slice(-4000), // unittest reports on stderr  };}

Pythonfrom withruntime import Sandboxdef grade(student_dir: str, tests_dir: str) -> dict:    with Sandbox.create(        vcpu=1,        memory_mib=2048,        network={"internet": False},        timeout_seconds=300,        on_lease_end="stop",    ) as sbx:        sbx.files.upload(student_dir, "/workspace/submission")        sbx.files.upload(tests_dir, "/workspace/tests")        run = sbx.exec(            ["python3", "-m", "unittest", "discover", "-s", "/workspace/tests", "-t", "/workspace"],            cwd="/workspace/submission",            timeout_ms=60_000,        )        return {            "passed": run.exit_code == 0 and not run.timed_out,            "timed_out": run.timed_out,            "output": run.stderr[-4000:],  # unittest reports on stderr        }

Use your own test runner in place of unittest: pytest, Jest, JUnit or a make check. The command is an array, so a file name a student chose cannot inject shell syntax. The sandbox stops when the block ends, even after an error.

What each setting protects against

What a submission might do	What stops it	Setting
Read another student's work	Every submission is its own microVM with its own disk	One sandbox per submission
Fetch the answer or post the tests	No outbound connection at all, root included	`network: { internet: false }`
Loop forever	The command returns `timedOut: true` with its output so far	`timeoutMs`
Keep the machine running	The lease ends and the sandbox stops	`timeoutSeconds`, `onLeaseEnd`
Fork-bomb or fill memory	Only its own sandbox slows; CPU and memory limits are enforced on the host	`vcpu`, `memoryMiB`
Print a gigabyte	At most 64 KiB of stdout and 64 KiB of stderr per result	Automatic, with truncation flags
Break out of the kernel	A hardware virtualization boundary, not a shared kernel	Every sandbox

A container shares its host's kernel; a microVM gives the code a kernel of its own (microVM vs container).

Other languages

The default image has Python 3.12, Node.js 24, Bun, gcc, g++ and make, so C, C++, JavaScript and TypeScript assignments grade without setup. For Java, Go, Rust or a course's own toolchain, build an image once:

TypeScriptimport { Runtime } from "withruntime";const runtime = new Runtime();await runtime.images.build({  name: "cs101-grader",  recipe: { apt: ["openjdk-21-jdk-headless"], pip: ["pytest", "hypothesis"] },});await using sbx = await runtime.sandboxes.create({  image: "cs101-grader",  network: { internet: false },});console.log((await sbx.exec("java -version")).stderr);

Pythonfrom withruntime import Runtimeruntime = Runtime()runtime.images.build(name="cs101-grader",                     recipe={"apt": ["openjdk-21-jdk-headless"], "pip": ["pytest", "hypothesis"]})with runtime.sandboxes.create(image="cs101-grader", network={"internet": False}) as sbx:    print(sbx.exec("java -version").stderr)

Each build of a name is its next version, so pin a term's grader with cs101-grader@3 and every regrade uses exactly the same tools (custom images).

Grade a whole class at once

A paid account runs 100 sandboxes at once to start, and a create past that limit waits for room instead of failing:

TypeScriptimport { Runtime } from "withruntime";const submissions = ["alice", "bob", "carol"];const runtime = new Runtime({ waitForCapacityMs: 600_000 });const scores = await Promise.all(  submissions.map(async (student) => {    await using sbx = await runtime.sandboxes.create({      labels: { course: "cs101", assignment: "hw3", student },      network: { internet: false },      timeoutSeconds: 300,      onLeaseEnd: "stop",    });    await sbx.files.write("/workspace/main.py", "print(sum([1, 2, 3]))\n");    const run = await sbx.exec(["python3", "main.py"], { timeoutMs: 30_000 });    return { student, correct: run.stdout.trim() === "6" };  }),);console.log(scores);

Labels let you find every sandbox of one assignment afterwards. Give grading its own API key with a daily spending limit, which only a person can set, so a bug in the grading script cannot run up a bill (security).

What matters for an autograder

Need	How Runtime covers it
Isolation per submission	A Firecracker microVM with its own kernel
No cheating over the network	Internet off, enforced on the host
Fair time limits	`timeoutMs` per command; the result says it timed out
Same tools every regrade	Versioned images, pinned with `name@version`
Deadline-night bursts	100 at once to start; creates past the limit wait for room
Detailed feedback	stdout, stderr and exit code per command; files read back with `files.read`
A capped budget	A daily spending limit per key; `maxCostMicros` per create

What it costs

Take a course of 300 students with five assignments: 1,500 submissions. Each keeps a 1 vCPU, 2 GiB sandbox running for 30 seconds, upload and start included, and its tests use 10 CPU-seconds:

TextCPU:    1,500 × 10 s / 3,600 × $0.025          = $0.10Memory: 1,500 × 30 s / 3,600 × 2 GiB × $0.0075 = $0.19Total:                                           $0.29

That is about two hundredths of a cent a submission. New accounts get 50 free sandbox hours with no card, and the trial runs eight sandboxes at once, enough to try a real assignment (pricing).

Start

Terminalnpx withruntime sandbox run --trial -- python3 -c 'print(6 * 7)'

The first run prints a link to approve in your browser. Then use the JavaScript or Python SDK as above.

Facts on this page were checked on 25 September 2026.