How to run end-to-end tests in parallel cloud sandboxes

On Runtime every shard starts with the app already up. A snapshot keeps a sandbox's files, memory and running processes, so a copy made from it has the web server, the database containers and the installed browsers exactly as the setup left them. Nothing needs to be installed, migrated or booted per shard. A run of eight 12-minute shards and its setup cost about twelve cents at Runtime's rates, as worked out below. A paid account runs 100 sandboxes at the same time (pricing, 26 October 2026).

Why end-to-end suites are slow and flaky in CI

An end-to-end test drives the whole product: a browser, the frontend, the API and the database behind it. Two things go wrong at scale. The suite takes longer than anyone will wait, and tests share state, so one test's leftover row fails another. Playwright splits a suite with npx playwright test --shard=x/y, and each shard can write a blob report that npx playwright merge-reports combines into one HTML report (Playwright sharding). What sharding needs from the infrastructure is identical machines, each with its own copy of the app and data.

Set up once, snapshot the running stack

Build the stack in one sandbox. Install the dependencies, start the services with Docker Compose, start the app, wait until it responds, then snapshot it:

TypeScriptimport { Runtime } from "withruntime";const runtime = new Runtime({ waitForCapacityMs: 600_000 });const slow = { check: true, timeoutMs: 900_000 } as const;export async function prepare(commit: string) {  const base = await runtime.sandboxes.create({ vcpu: 2, memoryMiB: 4096, diskMiB: 16_384 });  await base.files.upload("./", "/workspace/app");  await base.exec("cd app && npm ci && sudo npx playwright install-deps chromium", slow);  await base.exec("cd app && npx playwright install chromium", slow);  await base.exec("sudo enable-docker && cd app && docker compose up -d --wait db", slow);  await base.exec("cd app && npm run db:migrate && npm run db:seed", slow);  await base.spawn("cd app && npm run build && npm start");  await base.exec("npx wait-on http://localhost:3000", { check: true, timeoutMs: 300_000 });  const snapshot = await base.snapshot({ name: `e2e-${commit}`, retentionDays: 1 });  await base.stop();  return snapshot.id;}

db:migrate and db:seed stand for your project's own scripts, and db for the database service in its docker-compose.yml. A server started with spawn keeps running, and the snapshot keeps it running too.

Run the shards

Start one copy per shard, run that shard with the blob reporter, and bring the report back:

TypeScriptimport { mkdir, writeFile } from "node:fs/promises";import { Runtime } from "withruntime";const runtime = new Runtime({ waitForCapacityMs: 600_000 });export async function runShards(snapshotId: string, shards = 8) {  await mkdir("all-blob-reports", { recursive: true });  const exits = await Promise.all(    Array.from({ length: shards }, async (_, i) => {      await using sbx = await runtime.sandboxes.create({        snapshot: snapshotId,        labels: { suite: "e2e", shard: String(i + 1) },        timeoutSeconds: 1800,        onLeaseEnd: "stop",      });      const run = await sbx.exec(`npx playwright test --shard=${i + 1}/${shards} --reporter=blob`, {        cwd: "/workspace/app",        timeoutMs: 1_500_000,      });      for (const name of (await sbx.exec("ls blob-report", { cwd: "/workspace/app" })).stdout.split(        "\n",      ))        if (name.endsWith(".zip"))          await writeFile(            `all-blob-reports/${name}`,            await sbx.files.read(`/workspace/app/blob-report/${name}`),          );      return run.exitCode;    }),  );  await runtime.snapshots.delete(snapshotId);  return exits.every((code) => code === 0);}

Pythonimport osfrom concurrent.futures import ThreadPoolExecutorfrom withruntime import Runtimeruntime = Runtime(wait_for_capacity=600)def run_shard(snapshot_id: str, index: int, shards: int) -> int:    with runtime.sandboxes.create(snapshot=snapshot_id, timeout_seconds=1800, on_lease_end="stop",                                  labels={"suite": "e2e", "shard": str(index)}) as sbx:        run = sbx.exec(f"npx playwright test --shard={index}/{shards} --reporter=blob",                       cwd="/workspace/app", timeout_ms=1_500_000)        names = sbx.exec("ls blob-report", cwd="/workspace/app").stdout.split()        for name in (n for n in names if n.endswith(".zip")):            with open(os.path.join("all-blob-reports", name), "wb") as file:                file.write(sbx.files.read(f"/workspace/app/blob-report/{name}"))        return run.exit_codedef run_shards(snapshot_id: str, shards: int = 8) -> bool:    os.makedirs("all-blob-reports", exist_ok=True)    with ThreadPoolExecutor(max_workers=shards) as pool:        exits = list(pool.map(lambda i: run_shard(snapshot_id, i, shards), range(1, shards + 1)))    runtime.snapshots.delete(snapshot_id)    return all(code == 0 for code in exits)

Then, on the CI runner, merge them into one report:

Terminalnpx playwright merge-reports --reporter html ./all-blob-reports

Playwright names each shard's blob file with its shard number, so the files never collide. With fullyParallel: true in the Playwright config, shards split by test rather than by file, and the shards come out more even in size.

Why copies beat fresh machines for this

Each copy is its own Firecracker microVM with its own disk, so a test that writes to the database changes only its own shard's copy. A copy gets the source's vCPUs, memory and disk, and is billed as if it were created at that size.

What end-to-end runs need

What it costs

Take 200 CI runs a month. Each run prepares the stack in one 2 vCPU, 4 GiB sandbox for 10 minutes, then runs 8 shards of the same size for 12 minutes each, with the browser, app and database using 1.5 vCPUs on average:

That is about twelve cents a run. The snapshot is deleted when the shards finish, so its storage, $0.08 per GB per 30-day month, lasts minutes. New new accounts get 100 free hours of a 1 GB machine in their first 30 days without a card. An account without credit can run four sandboxes at the same time, which covers one per shard of this example.

Start

Terminalnpx withruntime sandboxes run --keep -- node --version

The first run prints a link to approve in your browser. For a single suite without sharding, see Playwright in a cloud sandbox.

Related: Playwright in a cloud sandbox, visual regression testing, run Docker in a sandbox, sandbox snapshots.

Sources

Checked 26 October 2026.

• Playwright, sharding tests
• Playwright, reporters

Facts on this page were checked on 26 October 2026.