How to give a coding agent fast Vitest feedback in a sandbox

Keep one sandbox per task, run vitest related on the files the agent changed, and hand the JSON report back to the model.

On Runtime the sandbox sleeps while the model thinks and costs almost nothing while it does. With idlePauseSeconds, a sandbox with no requests pauses and keeps its files, memory and processes; the next exec wakes it, usually in about half a second. While it runs and waits, a 2 vCPU, 4 GiB sandbox costs $0.03125 an hour. Vitest 5.0.1, the current release on 25 September 2026, needs Node.js 22.12 or 24, and the image has Node.js 24.21.0.

The loop

Each turn the agent writes files; the host runs only the tests that import them and returns a short summary of what failed. getOrCreate finds the same sandbox on every turn, so node_modules installs once per task:

TypeScriptimport { Sandbox } from "withruntime";type Report = {  numFailedTests: number;  numTotalTests: number;  testResults: {    name: string;    assertionResults: { fullName: string; status: string; failureMessages: string[] | null }[];  }[];};export async function testTurn(taskId: string, changed: Record<string, string>) {  const sbx = await Sandbox.getOrCreate(`task-${taskId}`, { idlePauseSeconds: 300 });  if (!sbx.info.reused) {    await sbx.files.upload("./repo", "/workspace/repo");    await sbx.exec("cd repo && npm ci --no-fund --no-audit", { check: true, timeoutMs: 600_000 });  }  for (const [path, content] of Object.entries(changed))    await sbx.files.write(`/workspace/repo/${path}`, content);  const files = Object.keys(changed).join(" ");  await sbx.exec(    `cd repo && npx vitest related ${files} --run --reporter=json --outputFile=/workspace/vitest.json`,    { timeoutMs: 300_000 },  );  const report: Report = JSON.parse(await sbx.files.readText("/workspace/vitest.json"));  const failures = report.testResults.flatMap((file) =>    file.assertionResults      .filter((test) => test.status === "failed")      .map((test) => `${file.name} > ${test.fullName}\n${(test.failureMessages ?? []).join("\n")}`),  );  return { passed: report.numTotalTests - report.numFailedTests, failures };}

Pythonimport jsonfrom withruntime import Sandboxdef test_turn(task_id: str, changed: dict[str, str]) -> dict:    sbx = Sandbox.get_or_create(f"task-{task_id}", idle_pause_seconds=300)    if not sbx.info.get("reused"):        sbx.files.upload("./repo", "/workspace/repo")        sbx.exec("cd repo && npm ci --no-fund --no-audit", check=True, timeout_ms=600_000)    for path, content in changed.items():        sbx.files.write(f"/workspace/repo/{path}", content)    files = " ".join(changed)    sbx.exec(f"cd repo && npx vitest related {files} --run --reporter=json --outputFile=/workspace/vitest.json",             timeout_ms=300_000)    report = json.loads(sbx.files.read_text("/workspace/vitest.json"))    failures = [        f"{result['name']} > {test['fullName']}\n" + "\n".join(test.get("failureMessages") or [])        for result in report["testResults"]        for test in result["assertionResults"]        if test["status"] == "failed"    ]    return {"passed": report["numTotalTests"] - report["numFailedTests"], "failures": failures}

The file names come from the agent, so pass only paths you have checked, or build the command as an array (["npx", "vitest", "related", ...files]), which runs with no shell.

Why a JSON report and not the console

Vitest's console output is written for a person: colours, progress lines and diffs. A model does better with the failing test's name and its assertion message, nothing else. The json reporter gives that shape, the same one Jest's JSON output uses:

Send failures to the model as the next message. On a large project that is a few hundred tokens instead of the whole console log.

Which Vitest command for which turn

vitest with no command starts watch mode, but falls back to a single run when standard input is not a terminal, which is the case for exec. Pass --run so the behaviour does not depend on that.

Start every task with Vitest installed

For an agent that starts new projects rather than editing a repo, build a template into a custom image. Recipe commands run as root, so hand the folder to the sandbox user (uid 1000) at the end:

TypeScriptimport { Runtime } from "withruntime";const runtime = new Runtime();await runtime.images.build({  name: "ts-vitest",  recipe: {    commands: [      "mkdir -p app && cd app && npm init -y && npm install -D --no-fund --no-audit vitest@5.0.1 typescript",      "chown -R 1000:1000 /workspace/app",    ],  },});await using sbx = await runtime.sandboxes.create({ image: "ts-vitest" });await sbx.files.write(  "/workspace/app/sum.ts",  "export const sum = (a: number, b: number) => a + b;\n",);await sbx.files.write(  "/workspace/app/sum.test.ts",  'import { expect, test } from "vitest";\nimport { sum } from "./sum";\ntest("adds", () => expect(sum(2, 3)).toBe(5));\n',);const run = await sbx.exec("cd app && npx vitest run", { timeoutMs: 120_000 });console.log(run.exitCode, run.stdout);

A stored image is charged at $0.08 per GB per 30-day month on its whole file; building it is free, and the free trial stores its first three images free.

Between turns

• Pause: idlePauseSeconds: 300 pauses after five minutes with no request. Compute billing stops at the pause; the next exec or file call wakes it.
• Lease: a sandbox lives for its timeoutSeconds, then pauses by default. extend adds time for a long task.
• Clean up: call stop() when the task ends. The name is free again for the next task.

Related

• pytest suites in a sandbox
• Jest across parallel sandboxes
• A sandbox for a coding agent
• Run TypeScript in a sandbox
• Pause and resume a sandbox

Sources

Checked 25 September 2026.

• Vitest CLI: vitest related, vitest run, --changed, --shard, --bail, and watch mode falling back to a run without a terminal
• Vitest reporters: --reporter=json with --outputFile
• Vitest's JSON reporter source: the report fields
• vitest on npm: version 5.0.1, Node ^22.12.0 || ^24.0.0 || >=26.0.0

Facts on this page were checked on 25 September 2026.