# How to run end-to-end tests in parallel cloud sandboxes

Prepare one sandbox with the app, its database and the browsers running, snapshot it, and run one test shard in each copy.

**On Runtime every shard starts with the app already up.** A snapshot keeps a
sandbox's files, memory and running processes, so a copy made from it has the
web server, the database containers and the installed browsers exactly as the
setup left them: nothing to install, migrate or boot per shard. A run of
eight 12-minute shards and its setup costs about twelve cents at Runtime's
rates, worked out below, and a paid account runs 100 sandboxes at once to start
([pricing](/docs/pricing#how-many-at-once), 25 September 2026).

## Why end-to-end suites are slow and flaky in CI

An end-to-end test drives the whole product: a browser, the frontend, the API
and the database behind it. Two things go wrong at scale. The suite takes
longer than anyone will wait, and tests share state, so one test's leftover
row fails another. Playwright splits a suite with `npx playwright test
--shard=x/y`, and each shard can write a blob report that
`npx playwright merge-reports` combines into one HTML report
([Playwright sharding](https://playwright.dev/docs/test-sharding)). What
sharding needs from the infrastructure is identical machines, each with its
own copy of the app and data.

## Set up once, snapshot the running stack

Build the stack in one sandbox: install, start the services with Docker
Compose, start the app, wait until it answers, then keep it:

```ts check
import { Runtime } from "withruntime";

const runtime = new Runtime({ waitForCapacityMs: 600_000 });
const slow = { check: true, timeoutMs: 900_000 } as const;

export async function prepare(commit: string) {
  const base = await runtime.sandboxes.create({ vcpu: 2, memoryMiB: 4096, diskMiB: 16_384 });
  await base.files.upload("./", "/workspace/app");
  await base.exec("cd app && npm ci && sudo npx playwright install-deps chromium", slow);
  await base.exec("cd app && npx playwright install chromium", slow);
  await base.exec("sudo enable-docker && cd app && docker compose up -d --wait db", slow);
  await base.exec("cd app && npm run db:migrate && npm run db:seed", slow);
  await base.spawn("cd app && npm run build && npm start");
  await base.exec("npx wait-on http://localhost:3000", { check: true, timeoutMs: 300_000 });
  const snapshot = await base.snapshot({ name: `e2e-${commit}`, retentionDays: 1 });
  await base.stop();
  return snapshot.id;
}
```

`db:migrate` and `db:seed` stand for your project's own scripts, and `db` for
the database service in its `docker-compose.yml`. A server started with
`spawn` keeps running, and the snapshot keeps it running too.

## Run the shards

Start one copy per shard, run that shard with the blob reporter, and bring the
report back:

```ts check
import { mkdir, writeFile } from "node:fs/promises";
import { Runtime } from "withruntime";

const runtime = new Runtime({ waitForCapacityMs: 600_000 });

export async function runShards(snapshotId: string, shards = 8) {
  await mkdir("all-blob-reports", { recursive: true });
  const exits = await Promise.all(
    Array.from({ length: shards }, async (_, i) => {
      await using sbx = await runtime.sandboxes.create({
        snapshot: snapshotId,
        labels: { suite: "e2e", shard: String(i + 1) },
        timeoutSeconds: 1800,
        onLeaseEnd: "stop",
      });
      const run = await sbx.exec(`npx playwright test --shard=${i + 1}/${shards} --reporter=blob`, {
        cwd: "/workspace/app",
        timeoutMs: 1_500_000,
      });
      for (const name of (await sbx.exec("ls blob-report", { cwd: "/workspace/app" })).stdout.split(
        "\n",
      ))
        if (name.endsWith(".zip"))
          await writeFile(
            `all-blob-reports/${name}`,
            await sbx.files.read(`/workspace/app/blob-report/${name}`),
          );
      return run.exitCode;
    }),
  );
  await runtime.snapshots.delete(snapshotId);
  return exits.every((code) => code === 0);
}
```

```python check
import os
from concurrent.futures import ThreadPoolExecutor
from withruntime import Runtime

runtime = Runtime(wait_for_capacity=600)


def run_shard(snapshot_id: str, index: int, shards: int) -> int:
    with runtime.sandboxes.create(snapshot=snapshot_id, timeout_seconds=1800, on_lease_end="stop",
                                  labels={"suite": "e2e", "shard": str(index)}) as sbx:
        run = sbx.exec(f"npx playwright test --shard={index}/{shards} --reporter=blob",
                       cwd="/workspace/app", timeout_ms=1_500_000)
        names = sbx.exec("ls blob-report", cwd="/workspace/app").stdout.split()
        for name in (n for n in names if n.endswith(".zip")):
            with open(os.path.join("all-blob-reports", name), "wb") as file:
                file.write(sbx.files.read(f"/workspace/app/blob-report/{name}"))
        return run.exit_code


def run_shards(snapshot_id: str, shards: int = 8) -> bool:
    os.makedirs("all-blob-reports", exist_ok=True)
    with ThreadPoolExecutor(max_workers=shards) as pool:
        exits = list(pool.map(lambda i: run_shard(snapshot_id, i, shards), range(1, shards + 1)))
    runtime.snapshots.delete(snapshot_id)
    return all(code == 0 for code in exits)
```

Then, on the CI runner, merge them into one report:

```bash no-run
npx playwright merge-reports --reporter html ./all-blob-reports
```

Playwright names each shard's blob file with its shard number, so the files
never collide. With `fullyParallel: true` in the Playwright config, shards
split by test rather than by file and come out more even.

## Why copies beat fresh machines for this

| Per shard, without a snapshot          | Per shard, from the snapshot              |
| -------------------------------------- | ----------------------------------------- |
| `npm ci` and a browser download        | Already installed                         |
| Docker installed and images pulled     | Containers already running                |
| Migrations and seed data               | The same seeded database in every copy    |
| Build and start the app, then wait     | The server already listening on port 3000 |
| Packages resolved again for each shard | One install, shared by every copy         |

Each copy is its own Firecracker microVM with its own disk, so a test that
writes to the database changes only its own shard's copy. A copy gets the
source's vCPUs, memory and disk, and is billed as a create of that size
would be.

## What end-to-end runs need

| Need                           | How Runtime covers it                                              |
| ------------------------------ | ------------------------------------------------------------------ |
| A real browser                 | Playwright's Chromium installs with its own installer under `sudo` |
| Databases and queues           | `sudo enable-docker`; Docker Compose in the sandbox                |
| The same start for every shard | Create from a snapshot: files, memory and processes                |
| Many shards at once            | 100 sandboxes at once on a paid account; creates wait for room     |
| Reports and traces             | `files.read` or `files.download` bring them back                   |
| Watching a failing test live   | A private preview of port 3000, or the desktop's live view         |
| Nothing left running           | `timeoutSeconds` with `onLeaseEnd: "stop"`, enforced on the host   |

## What it costs

Take 200 CI runs a month. Each run prepares the stack in one 2 vCPU, 4 GiB
sandbox for 10 minutes, then runs 8 shards of the same size for 12 minutes
each, with the browser, app and database using 1.5 vCPUs on average:

```
Setup:  200 × 10 min / 60 × (1.5 vCPU × $0.025 + 4 GiB × $0.0075)          = $2.25
Shards: 200 × 8 × 12 min / 60 × (1.5 vCPU × $0.025 + 4 GiB × $0.0075)      = $21.60
Total:                                                                        $23.85
```

That is about twelve cents a run. The snapshot is deleted when the shards
finish, so its storage, $0.08 per GB per 30-day month, lasts minutes. New
accounts get 50 free sandbox hours, no card; the trial runs eight sandboxes at
once, one per shard of this example.

## Start

```bash no-run
npx withruntime sandbox run --trial --keep -- node --version
```

The first run prints a link to approve in your browser. For a single suite
without sharding, see [Playwright in a cloud sandbox](/integrations/playwright).

Related: [Playwright in a cloud sandbox](/integrations/playwright),
[visual regression testing](/use-cases/visual-regression-testing),
[run Docker in a sandbox](/how-to/run-docker-in-a-sandbox),
[sandbox snapshots](/glossary/sandbox-snapshot).

## Sources

Checked 25 September 2026.

- [Playwright, sharding tests](https://playwright.dev/docs/test-sharding)
- [Playwright, reporters](https://playwright.dev/docs/test-reporters)

Facts on this page were checked on 25 September 2026.