# How to give a coding agent fast Vitest feedback in a sandbox

Keep one sandbox per task, run `vitest related` on the files the agent changed, and hand the JSON report back to the model.

**On Runtime the sandbox sleeps while the model thinks and costs almost
nothing while it does.** With `idlePauseSeconds`, a sandbox with no requests
pauses and keeps its files, memory and processes; the next `exec` wakes it,
usually in about half a second. While it runs and waits, a 2 vCPU, 4 GiB
sandbox costs $0.03125 an hour. Vitest 5.0.1, the current release on
25 September 2026, needs Node.js 22.12 or 24, and the image has Node.js
24.21.0.

## The loop

Each turn the agent writes files; the host runs only the tests that import
them and returns a short summary of what failed. `getOrCreate` finds the same
sandbox on every turn, so `node_modules` installs once per task:

```ts check
import { Sandbox } from "withruntime";

type Report = {
  numFailedTests: number;
  numTotalTests: number;
  testResults: {
    name: string;
    assertionResults: { fullName: string; status: string; failureMessages: string[] | null }[];
  }[];
};

export async function testTurn(taskId: string, changed: Record<string, string>) {
  const sbx = await Sandbox.getOrCreate(`task-${taskId}`, { idlePauseSeconds: 300 });
  if (!sbx.info.reused) {
    await sbx.files.upload("./repo", "/workspace/repo");
    await sbx.exec("cd repo && npm ci --no-fund --no-audit", { check: true, timeoutMs: 600_000 });
  }

  for (const [path, content] of Object.entries(changed))
    await sbx.files.write(`/workspace/repo/${path}`, content);

  const files = Object.keys(changed).join(" ");
  await sbx.exec(
    `cd repo && npx vitest related ${files} --run --reporter=json --outputFile=/workspace/vitest.json`,
    { timeoutMs: 300_000 },
  );
  const report: Report = JSON.parse(await sbx.files.readText("/workspace/vitest.json"));

  const failures = report.testResults.flatMap((file) =>
    file.assertionResults
      .filter((test) => test.status === "failed")
      .map((test) => `${file.name} > ${test.fullName}\n${(test.failureMessages ?? []).join("\n")}`),
  );
  return { passed: report.numTotalTests - report.numFailedTests, failures };
}
```

```python check
import json
from withruntime import Sandbox


def test_turn(task_id: str, changed: dict[str, str]) -> dict:
    sbx = Sandbox.get_or_create(f"task-{task_id}", idle_pause_seconds=300)
    if not sbx.info.get("reused"):
        sbx.files.upload("./repo", "/workspace/repo")
        sbx.exec("cd repo && npm ci --no-fund --no-audit", check=True, timeout_ms=600_000)

    for path, content in changed.items():
        sbx.files.write(f"/workspace/repo/{path}", content)

    files = " ".join(changed)
    sbx.exec(f"cd repo && npx vitest related {files} --run --reporter=json --outputFile=/workspace/vitest.json",
             timeout_ms=300_000)
    report = json.loads(sbx.files.read_text("/workspace/vitest.json"))

    failures = [
        f"{result['name']} > {test['fullName']}\n" + "\n".join(test.get("failureMessages") or [])
        for result in report["testResults"]
        for test in result["assertionResults"]
        if test["status"] == "failed"
    ]
    return {"passed": report["numTotalTests"] - report["numFailedTests"], "failures": failures}
```

The file names come from the agent, so pass only paths you have checked, or
build the command as an array (`["npx", "vitest", "related", ...files]`),
which runs with no shell.

## Why a JSON report and not the console

Vitest's console output is written for a person: colours, progress lines and
diffs. A model does better with the failing test's name and its assertion
message, nothing else. The `json` reporter gives that shape, the same one
Jest's JSON output uses:

| Field                                       | What it holds                           |
| ------------------------------------------- | --------------------------------------- |
| `success`                                   | `true` when every test passed           |
| `numTotalTests`, `numFailedTests`           | Counts across the run                   |
| `testResults[].name`                        | The test file                           |
| `testResults[].assertionResults[].fullName` | The test's full name                    |
| `assertionResults[].status`                 | `passed`, `failed`, `skipped` and so on |
| `assertionResults[].failureMessages`        | The assertion error and its stack       |

Send `failures` to the model as the next message. On a large project that is a
few hundred tokens instead of the whole console log.

## Which Vitest command for which turn

| Situation                             | Command                                      |
| ------------------------------------- | -------------------------------------------- |
| The agent edited a few source files   | `vitest related <files> --run`               |
| The agent committed its work with git | `vitest run --changed HEAD~1`                |
| Before declaring the task done        | `vitest run`                                 |
| A long suite split over sandboxes     | `vitest run --shard=1/3`, `--shard=2/3`, ... |
| Stop at the first failures            | add `--bail 1`                               |

`vitest` with no command starts watch mode, but falls back to a single run
when standard input is not a terminal, which is the case for `exec`. Pass
`--run` so the behaviour does not depend on that.

## Start every task with Vitest installed

For an agent that starts new projects rather than editing a repo, build a
template into a [custom image](/docs/images). Recipe commands run as root, so
hand the folder to the sandbox user (uid 1000) at the end:

```ts check
import { Runtime } from "withruntime";

const runtime = new Runtime();
await runtime.images.build({
  name: "ts-vitest",
  recipe: {
    commands: [
      "mkdir -p app && cd app && npm init -y && npm install -D --no-fund --no-audit vitest@5.0.1 typescript",
      "chown -R 1000:1000 /workspace/app",
    ],
  },
});

await using sbx = await runtime.sandboxes.create({ image: "ts-vitest" });
await sbx.files.write(
  "/workspace/app/sum.ts",
  "export const sum = (a: number, b: number) => a + b;\n",
);
await sbx.files.write(
  "/workspace/app/sum.test.ts",
  'import { expect, test } from "vitest";\nimport { sum } from "./sum";\ntest("adds", () => expect(sum(2, 3)).toBe(5));\n',
);
const run = await sbx.exec("cd app && npx vitest run", { timeoutMs: 120_000 });
console.log(run.exitCode, run.stdout);
```

A stored image is charged at $0.08 per GB per 30-day month on its whole file;
building it is free, and the free trial stores its first three images free.

## Between turns

- **Pause:** `idlePauseSeconds: 300` pauses after five minutes with no request.
  Compute billing stops at the pause; the next `exec` or file call wakes it.
- **Lease:** a sandbox lives for its `timeoutSeconds`, then pauses by default.
  `extend` adds time for a long task.
- **Clean up:** call `stop()` when the task ends. The name is free again for
  the next task.

## Related

- [pytest suites in a sandbox](/integrations/pytest)
- [Jest across parallel sandboxes](/integrations/jest)
- [A sandbox for a coding agent](/use-cases/coding-agent-sandbox)
- [Run TypeScript in a sandbox](/languages/typescript)
- [Pause and resume a sandbox](/how-to/pause-and-resume-a-sandbox)

## Sources

Checked 25 September 2026.

- [Vitest CLI](https://vitest.dev/guide/cli): `vitest related`, `vitest run`,
  `--changed`, `--shard`, `--bail`, and watch mode falling back to a run
  without a terminal
- [Vitest reporters](https://vitest.dev/guide/reporters): `--reporter=json`
  with `--outputFile`
- [Vitest's JSON reporter source](https://github.com/vitest-dev/vitest/blob/main/packages/vitest/src/node/reporters/json.ts):
  the report fields
- [vitest on npm](https://www.npmjs.com/package/vitest): version 5.0.1, Node
  `^22.12.0 || ^24.0.0 || >=26.0.0`

Facts on this page were checked on 25 September 2026.