How to run an agent's CI on every pull request

On Runtime, a burst of agent pull requests queues for a sandbox instead of failing. A paid account runs 100 sandboxes at the same time. Each run is a Firecracker microVM with its own kernel. So code that nobody has reviewed yet runs nowhere near your CI secrets. A CI key can carry a daily spending limit that only a person can raise. Runtime was 42% to 88% cheaper than fourteen other sandbox providers on a mostly-idle job. The rates were checked on 23 and 26 October 2026 (the comparison).

Why agent pull requests need different CI

Coding agents open many more pull requests than people do, and every one carries code that no human has read when its checks start. On a shared CI runner that code runs next to whatever the runner can reach: its tokens, its cache, its network. GitHub already treats pull requests from forks as untrusted for this reason: secrets other than GITHUB_TOKEN are not passed to the runner, and GITHUB_TOKEN is read-only (GitHub's documentation, read 26 October 2026).

Running the install and the tests in a sandbox gives every pull request the same protection, fork or not. The runner holds only a Runtime key. The agent's code runs somewhere that key does not exist.

The script your CI job runs

Upload the checked-out tree. Install, test, and exit with the tests' exit code. Output streams into the CI log as it happens.

TypeScriptimport { Runtime } from "withruntime";// In CI a burst can queue behind other runs for up to ten minutes.const runtime = new Runtime({ waitForCapacityMs: 600_000 });await using sbx = await runtime.sandboxes.create({  image: "ci-node", // your dependencies, built once  diskMiB: 8192,  timeoutSeconds: 1800,  onLeaseEnd: "stop",  labels: { ci: "agent-prs", pr: process.env.PR_NUMBER ?? "local" },  network: { internet: true, allow: ["registry.npmjs.org"] },});await sbx.files.upload(".", "/workspace/repo");const stream = {  onStdout: (t: string) => process.stdout.write(t),  onStderr: (t: string) => process.stderr.write(t),};await sbx.exec("npm ci", { cwd: "/workspace/repo", timeoutMs: 600_000, check: true, ...stream });const tests = await sbx.exec("npm test", {  cwd: "/workspace/repo",  timeoutMs: 1_200_000,  ...stream,});process.exitCode = tests.timedOut ? 124 : (tests.exitCode ?? 1);

Pythonimport osimport sysfrom withruntime import Runtimeruntime = Runtime(wait_for_capacity=600)  # queue up to ten minutes in a burstwith runtime.sandboxes.create(    image="ci-python",    disk_mib=8192,    timeout_seconds=1800,    on_lease_end="stop",    labels={"ci": "agent-prs", "pr": os.environ.get("PR_NUMBER", "local")},    network={"internet": True, "allow": ["pypi.org", "*.pythonhosted.org"]},) as sbx:    sbx.files.upload(".", "/workspace/repo")    sbx.exec("pip install -e '.[test]'", cwd="/workspace/repo", timeout_ms=600_000, check=True,             on_stdout=sys.stdout.write, on_stderr=sys.stderr.write)    tests = sbx.exec("python3 -m pytest -q", cwd="/workspace/repo", timeout_ms=1_200_000,                     on_stdout=sys.stdout.write, on_stderr=sys.stderr.write)    code = 124 if tests.timed_out else (1 if tests.exit_code is None else tests.exit_code)sys.exit(code)

The allow list lets the install reach the package registry and nothing else. So a test that tries to send the repository somewhere fails to connect. The rules are enforced on the host. Root in the sandbox cannot change them (the sandbox environment).

Wire it into GitHub Actions

By default, pull_request workflows run on opened, synchronize and reopened. The job needs Node or Python and one secret:

YAMLon: pull_requestjobs:  sandbox-tests:    runs-on: ubuntu-latest    steps:      - uses: actions/checkout@v4      - run: pip install withruntime      - run: python3 ci_sandbox.py        env:          RUNTIME_API_KEY: ${{ secrets.RUNTIME_API_KEY }}          PR_NUMBER: ${{ github.event.pull_request.number }}

Create that key from your own terminal. It prints a link for an owner, admin or developer to approve. It then goes straight into the repository's secrets:

Terminalruntime keys create --name agent-ci --daily-limit 25 | gh secret set RUNTIME_API_KEY

With --daily-limit 25, no 24-hour window can spend more than $25. Beyond that limit, a create fails with spending_limit_reached, and nothing is charged (keys for CI).

Pull requests from forks

A fork's workflow does not get your secrets, so it cannot call Runtime from GitHub's runner. Run the check from your own service instead. Receive GitHub's pull request webhook. Fetch the pull request's head inside a sandbox, and post the result as a check or a comment.

TypeScriptimport { Runtime } from "withruntime";const runtime = new Runtime({ waitForCapacityMs: 600_000 });export async function checkPullRequest(cloneUrl: string, number: number) {  await using sbx = await runtime.sandboxes.create({    image: "ci-node",    timeoutSeconds: 1800,    network: { internet: true, allow: ["github.com", "*.github.com", "registry.npmjs.org"] },  });  await sbx.exec(["git", "clone", "--depth", "50", cloneUrl, "repo"], {    check: true,    timeoutMs: 300_000,  });  await sbx.exec(["git", "fetch", "origin", `pull/${number}/head:pr`], {    cwd: "/workspace/repo",    check: true,  });  await sbx.exec("git checkout pr && npm ci", {    cwd: "/workspace/repo",    timeoutMs: 600_000,    check: true,  });  const tests = await sbx.exec("npm test", { cwd: "/workspace/repo", timeoutMs: 1_200_000 });  return { passed: tests.exitCode === 0, log: tests.stdout.slice(-4000) };}

git fetch origin pull/ID/head:BRANCH is GitHub's documented way to fetch a pull request by number.

What agent CI needs

What it costs

Assume 3,000 agent pull requests a month. Each run keeps a 2 vCPU, 4 GiB sandbox running for 8 minutes. Install and tests use 600 CPU-seconds:

About 0.8 cents a pull request, at $0.025 per vCPU-hour of measured CPU and $0.0075 per GiB-hour of memory (pricing). The GitHub runner that starts the job only uploads files and waits. To try it first, use the usage included every month, 100 free hours of a 1 GB machine. That allows four sandboxes at the same time, with no card.

Sources

• Events that trigger workflows: pull_request activity types and what fork workflows receive, read 26 October 2026.
• Checking out pull requests locally: git fetch origin pull/ID/head:BRANCH, read 26 October 2026.

Related: automated code review, coding agent sandbox, run Docker in a sandbox, test generation, GitHub Codespaces vs an agent sandbox.

Facts on this page were checked on 26 October 2026.