How to give a coding agent fast Vitest feedback in a sandbox

On Runtime, a sandbox pauses while the model is thinking, and it costs almost nothing during that time. A sandbox with no activity pauses after idlePauseSeconds, which defaults to 300. It keeps its files, memory and processes. While it runs and waits, a 2 vCPU, 4 GiB sandbox costs $0.03125 an hour. Vitest 5.0.5, the current release on 25 October 2026, needs Node.js 22.12 or 24, and the image has Node.js 24.21.0.

The loop

On each turn, the agent writes files. The host runs only the tests that import those files, and returns a short summary of the failures. getOrCreate returns the same sandbox on every turn, so node_modules installs once per task:

TypeScriptimport { Sandbox } from "withruntime";type Report = {  numFailedTests: number;  numTotalTests: number;  testResults: {    name: string;    assertionResults: { fullName: string; status: string; failureMessages: string[] | null }[];  }[];};export async function testTurn(taskId: string, changed: Record<string, string>) {  const sbx = await Sandbox.getOrCreate(`task-${taskId}`, { idlePauseSeconds: 300 });  if (!sbx.info.reused) {    await sbx.files.upload("./repo", "/workspace/repo");    await sbx.exec("cd repo && npm ci --no-fund --no-audit", { check: true, timeoutMs: 600_000 });  }  for (const [path, content] of Object.entries(changed))    await sbx.files.write(`/workspace/repo/${path}`, content);  const files = Object.keys(changed).join(" ");  await sbx.exec(    `cd repo && npx vitest related ${files} --run --reporter=json --outputFile=/workspace/vitest.json`,    { timeoutMs: 300_000 },  );  const report: Report = JSON.parse(await sbx.files.readText("/workspace/vitest.json"));  const failures = report.testResults.flatMap((file) =>    file.assertionResults      .filter((test) => test.status === "failed")      .map((test) => `${file.name} > ${test.fullName}\n${(test.failureMessages ?? []).join("\n")}`),  );  return { passed: report.numTotalTests - report.numFailedTests, failures };}

Pythonimport jsonfrom withruntime import Sandboxdef test_turn(task_id: str, changed: dict[str, str]) -> dict:    sbx = Sandbox.get_or_create(f"task-{task_id}", idle_pause_seconds=300)    if not sbx.info.get("reused"):        sbx.files.upload("./repo", "/workspace/repo")        sbx.exec("cd repo && npm ci --no-fund --no-audit", check=True, timeout_ms=600_000)    for path, content in changed.items():        sbx.files.write(f"/workspace/repo/{path}", content)    files = " ".join(changed)    sbx.exec(f"cd repo && npx vitest related {files} --run --reporter=json --outputFile=/workspace/vitest.json",             timeout_ms=300_000)    report = json.loads(sbx.files.read_text("/workspace/vitest.json"))    failures = [        f"{result['name']} > {test['fullName']}\n" + "\n".join(test.get("failureMessages") or [])        for result in report["testResults"]        for test in result["assertionResults"]        if test["status"] == "failed"    ]    return {"passed": report["numTotalTests"] - report["numFailedTests"], "failures": failures}

The agent chooses the file names. Pass only paths you have checked, or build the command as an array (["npx", "vitest", "related", ...files]). An array runs with no shell.

Why a JSON report and not the console

Vitest's console output is written for people. It includes colors, progress lines and diffs. A model needs only the failing test's name and its assertion message. The json reporter returns that structure. Jest's JSON output uses the same structure:

Send failures to the model as the next message. On a large project, this is a few hundred tokens instead of the full console log.

Which Vitest command for which turn

Running vitest with no command starts watch mode. When standard input is not a terminal, as with exec, it runs once instead. Pass --run so the behavior does not depend on the terminal.

Start every task with Vitest installed

For an agent that starts new projects instead of editing a repo, build a template into a custom image. Recipe commands run as root. At the end, change the owner of the folder to the sandbox user (uid 1000):

TypeScriptimport { Runtime } from "withruntime";const runtime = new Runtime();await runtime.images.build({  name: "ts-vitest",  recipe: {    commands: [      "mkdir -p app && cd app && npm init -y && npm install -D --no-fund --no-audit vitest@5.0.5 typescript",      "chown -R 1000:1000 /workspace/app",    ],  },});await using sbx = await runtime.sandboxes.create({ image: "ts-vitest" });await sbx.files.write(  "/workspace/app/sum.ts",  "export const sum = (a: number, b: number) => a + b;\n",);await sbx.files.write(  "/workspace/app/sum.test.ts",  'import { expect, test } from "vitest";\nimport { sum } from "./sum";\ntest("adds", () => expect(sum(2, 3)).toBe(5));\n',);const run = await sbx.exec("cd app && npx vitest run", { timeoutMs: 120_000 });console.log(run.exitCode, run.stdout);

A stored image is charged at $0.08 per GB per 30-day month on its accounted storage. Building an image is free. An account without credit stores its first three images free.

Between turns

• Pause: idlePauseSeconds: 300 pauses the sandbox after five minutes with no activity. A test run in progress keeps it awake. Compute billing stops at the pause. The next exec or file call wakes it.
• Lease: A sandbox runs for its timeoutSeconds, then pauses by default. Call extend to add time to a long task.
• Clean up: Call stop() when the task ends. The name is then free for the next task.

Related

• pytest suites in a sandbox
• Jest across parallel sandboxes
• A sandbox for a coding agent
• Run TypeScript in a sandbox
• Pause and resume a sandbox

Sources

Checked 25 October 2026.

• Vitest CLI: vitest related, vitest run, --changed, --shard, --bail, and watch mode, which runs once without a terminal
• Vitest reporters: --reporter=json with --outputFile
• Vitest's JSON reporter source: the report fields
• vitest on npm: version 5.0.5, Node ^22.12.0 || ^24.0.0 || >=26.0.0

A stored image is charged for its whole file (image storage pricing).

Facts on this page were checked on 25 October 2026.