# How to grade student code safely in a sandbox

Run each submission in its own throwaway microVM with the internet off and a time limit, run your tests against it, and record the score.

**On Runtime grading 1,500 submissions costs $0.29.** Each submission gets its
own Firecracker microVM, so one student's code cannot read another's, reach
your network or outlive its time limit. Runtime bills the CPU the tests use,
$0.025 per vCPU-hour, with memory at $0.0075 per GiB-hour; the working is at
the end of this page.

## The short answer

One sandbox per submission. Copy the student's files and your hidden tests in,
cut the network, run the tests, keep the result.

```ts check
import { Sandbox } from "withruntime";

export async function grade(studentDir: string, testsDir: string) {
  await using sbx = await Sandbox.create({
    vcpu: 1,
    memoryMiB: 2048,
    network: { internet: false },
    timeoutSeconds: 300,
    onLeaseEnd: "stop",
  });
  await sbx.files.upload(studentDir, "/workspace/submission");
  await sbx.files.upload(testsDir, "/workspace/tests");
  const run = await sbx.exec(
    ["python3", "-m", "unittest", "discover", "-s", "/workspace/tests", "-t", "/workspace"],
    { cwd: "/workspace/submission", timeoutMs: 60_000 },
  );
  return {
    passed: run.exitCode === 0 && !run.timedOut,
    timedOut: run.timedOut,
    output: run.stderr.slice(-4000), // unittest reports on stderr
  };
}
```

```python check
from withruntime import Sandbox


def grade(student_dir: str, tests_dir: str) -> dict:
    with Sandbox.create(
        vcpu=1,
        memory_mib=2048,
        network={"internet": False},
        timeout_seconds=300,
        on_lease_end="stop",
    ) as sbx:
        sbx.files.upload(student_dir, "/workspace/submission")
        sbx.files.upload(tests_dir, "/workspace/tests")
        run = sbx.exec(
            ["python3", "-m", "unittest", "discover", "-s", "/workspace/tests", "-t", "/workspace"],
            cwd="/workspace/submission",
            timeout_ms=60_000,
        )
        return {
            "passed": run.exit_code == 0 and not run.timed_out,
            "timed_out": run.timed_out,
            "output": run.stderr[-4000:],  # unittest reports on stderr
        }
```

Use your own test runner in place of `unittest`: pytest, Jest, JUnit or a
`make check`. The command is an array, so a file name a student chose cannot
inject shell syntax. The sandbox stops when the block ends, even after an
error.

## What each setting protects against

| What a submission might do         | What stops it                                                              | Setting                          |
| ---------------------------------- | -------------------------------------------------------------------------- | -------------------------------- |
| Read another student's work        | Every submission is its own microVM with its own disk                      | One sandbox per submission       |
| Fetch the answer or post the tests | No outbound connection at all, root included                               | `network: { internet: false }`   |
| Loop forever                       | The command returns `timedOut: true` with its output so far                | `timeoutMs`                      |
| Keep the machine running           | The lease ends and the sandbox stops                                       | `timeoutSeconds`, `onLeaseEnd`   |
| Fork-bomb or fill memory           | Only its own sandbox slows; CPU and memory limits are enforced on the host | `vcpu`, `memoryMiB`              |
| Print a gigabyte                   | At most 64 KiB of stdout and 64 KiB of stderr per result                   | Automatic, with truncation flags |
| Break out of the kernel            | A hardware virtualization boundary, not a shared kernel                    | Every sandbox                    |

A container shares its host's kernel; a microVM gives the code a kernel of
its own ([microVM vs container](/compare/microvm-vs-container)).

## Other languages

The default image has Python 3.12, Node.js 24, Bun, gcc, g++ and make, so C,
C++, JavaScript and TypeScript assignments grade without setup. For Java, Go,
Rust or a course's own toolchain, build an image once:

```ts check
import { Runtime } from "withruntime";

const runtime = new Runtime();
await runtime.images.build({
  name: "cs101-grader",
  recipe: { apt: ["openjdk-21-jdk-headless"], pip: ["pytest", "hypothesis"] },
});
await using sbx = await runtime.sandboxes.create({
  image: "cs101-grader",
  network: { internet: false },
});
console.log((await sbx.exec("java -version")).stderr);
```

```python check
from withruntime import Runtime

runtime = Runtime()
runtime.images.build(name="cs101-grader",
                     recipe={"apt": ["openjdk-21-jdk-headless"], "pip": ["pytest", "hypothesis"]})
with runtime.sandboxes.create(image="cs101-grader", network={"internet": False}) as sbx:
    print(sbx.exec("java -version").stderr)
```

Each build of a name is its next version, so pin a term's grader with
`cs101-grader@3` and every regrade uses exactly the same tools
([custom images](/docs/images)).

## Grade a whole class at once

A paid account runs 100 sandboxes at once to start, and a create past that
limit waits for room instead of failing:

```ts check
import { Runtime } from "withruntime";

const submissions = ["alice", "bob", "carol"];
const runtime = new Runtime({ waitForCapacityMs: 600_000 });

const scores = await Promise.all(
  submissions.map(async (student) => {
    await using sbx = await runtime.sandboxes.create({
      labels: { course: "cs101", assignment: "hw3", student },
      network: { internet: false },
      timeoutSeconds: 300,
      onLeaseEnd: "stop",
    });
    await sbx.files.write("/workspace/main.py", "print(sum([1, 2, 3]))\n");
    const run = await sbx.exec(["python3", "main.py"], { timeoutMs: 30_000 });
    return { student, correct: run.stdout.trim() === "6" };
  }),
);
console.log(scores);
```

Labels let you find every sandbox of one assignment afterwards. Give grading
its own API key with a daily spending limit, which only a person can set, so a
bug in the grading script cannot run up a bill ([security](/docs/security)).

## What matters for an autograder

| Need                         | How Runtime covers it                                                       |
| ---------------------------- | --------------------------------------------------------------------------- |
| Isolation per submission     | A Firecracker microVM with its own kernel                                   |
| No cheating over the network | Internet off, enforced on the host                                          |
| Fair time limits             | `timeoutMs` per command; the result says it timed out                       |
| Same tools every regrade     | Versioned images, pinned with `name@version`                                |
| Deadline-night bursts        | 100 at once to start; creates past the limit wait for room                  |
| Detailed feedback            | stdout, stderr and exit code per command; files read back with `files.read` |
| A capped budget              | A daily spending limit per key; `maxCostMicros` per create                  |

## What it costs

Take a course of 300 students with five assignments: 1,500 submissions. Each
keeps a 1 vCPU, 2 GiB sandbox running for 30 seconds, upload and start
included, and its tests use 10 CPU-seconds:

```
CPU:    1,500 × 10 s / 3,600 × $0.025          = $0.10
Memory: 1,500 × 30 s / 3,600 × 2 GiB × $0.0075 = $0.19
Total:                                           $0.29
```

That is about two hundredths of a cent a submission. New accounts get 50 free
sandbox hours with no card, and the trial runs eight sandboxes at once, enough
to try a real assignment ([pricing](/docs/pricing)).

## Start

```bash no-run
npx withruntime sandbox run --trial -- python3 -c 'print(6 * 7)'
```

The first run prints a link to approve in your browser. Then use the
[JavaScript](/docs/javascript) or [Python](/docs/python) SDK as above.

Related: [run untrusted LLM code safely](/use-cases/run-untrusted-llm-code),
[turn off sandbox internet](/how-to/turn-off-sandbox-internet),
[what a microVM is](/glossary/microvm),
[sandbox cost calculator](/calculators/sandbox-cost-calculator).

Facts on this page were checked on 25 September 2026.