How to let an AI agent upgrade dependencies and prove nothing broke

On Runtime ten upgrades are tested at the same time from one installed machine. A fork makes 1 to 100 running copies of a sandbox, with its files, memory and processes. The fork's own snapshot is not billed. Measured CPU costs $0.025 per vCPU-hour. For a job that mostly waits, Runtime was 42% to 88% cheaper than fourteen other sandbox providers, at rates checked 25 October 2026 (the comparison).

Why upgrades go wrong one at a time

Bots that open one pull request per outdated package leave the hard part to a person: finding which bumps break the build, and writing the fix. Putting every bump in one branch is worse. When the tests fail, nobody knows which package caused the failure.

The reliable order is:

1. Get a green baseline: install and pass the tests before changing anything.
2. Try each upgrade alone, from that same baseline.
3. Merge the ones that pass as they are.
4. Give the agent only the failures, one package each, with the error.
5. Combine every passing upgrade and test once more.

Step 2 takes most of the time, and forks help most there. Each attempt starts from the installed baseline without a reinstall, and no attempt can see another's node_modules.

Find what is outdated

npm outdated --json lists each package with current, wanted (the newest version the range in package.json allows) and latest (the registry's latest tag). For Python, pip list --outdated --format json does the same (npm's and pip's documentation, read 26 October 2026).

The core loop

TypeScriptimport { Sandbox } from "withruntime";const repo = "https://github.com/your-org/your-app.git";await using base = await Sandbox.create({  diskMiB: 8192,  timeoutSeconds: 3600,  labels: { job: "upgrades" },  network: { internet: true, allow: ["github.com", "*.github.com", "registry.npmjs.org"] },});await base.exec(["git", "clone", "--depth", "1", repo, "app"], { check: true, timeoutMs: 300_000 });await base.exec("npm ci && npm test", { cwd: "/workspace/app", check: true, timeoutMs: 900_000 });const report = await base.exec("npm outdated --json", { cwd: "/workspace/app" });const outdated = Object.keys(JSON.parse(report.stdout || "{}")).slice(0, 10);if (outdated.length === 0) process.exit(0); // nothing to upgradeconst copies = await base.fork({ count: outdated.length });const results = await Promise.all(  outdated.map(async (name, i) => {    const copy = copies[i]!;    const bump = await copy.exec(["npm", "install", `${name}@latest`], {      cwd: "/workspace/app",      timeoutMs: 600_000,    });    const tests = await copy.exec("npm run build --if-present && npm test", {      cwd: "/workspace/app",      timeoutMs: 900_000,    });    return {      name,      sandboxId: copy.id,      passed: bump.exitCode === 0 && tests.exitCode === 0,      log: tests.stdout.slice(-3000),    };  }),);// Stop the copies that passed; keep the failures for the agent to work in.await Promise.all(  results.filter((r) => r.passed).map((r) => copies[outdated.indexOf(r.name)]!.stop()),);console.log(results.map((r) => `${r.passed ? "ok  " : "FAIL"} ${r.name}`).join("\n"));

Pythonimport jsonfrom withruntime import Sandboxrepo = "https://github.com/your-org/your-lib.git"with Sandbox.create(    disk_mib=8192,    timeout_seconds=3600,    labels={"job": "upgrades"},    network={"internet": True, "allow": ["github.com", "*.github.com", "pypi.org", "*.pythonhosted.org"]},) as base:    base.exec(["git", "clone", "--depth", "1", repo, "lib"], check=True, timeout_ms=300_000)    base.exec("pip install -e '.[test]' && python3 -m pytest -q", cwd="/workspace/lib",              check=True, timeout_ms=900_000)    listed = base.exec("pip list --outdated --format json", cwd="/workspace/lib")    outdated = [p["name"] for p in json.loads(listed.stdout or "[]")][:10]    copies = base.fork(count=len(outdated)) if outdated else []    results = []    for name, copy in zip(outdated, copies):        bump = copy.exec(["pip", "install", "--upgrade", name], cwd="/workspace/lib", timeout_ms=600_000)        tests = copy.exec("python3 -m pytest -q", cwd="/workspace/lib", timeout_ms=900_000)        passed = bump.exit_code == 0 and tests.exit_code == 0        results.append({"name": name, "sandbox": copy.id, "passed": passed, "log": tests.stdout[-3000:]})        if passed:            copy.stop()    for r in results:        print("ok  " if r["passed"] else "FAIL", r["name"])

A failing copy is already the right place for the agent to work. The upgrade is installed, the failure can be reproduced, and nothing else has changed. Bind sandbox tools to that copy, give the model the test log and the package's changelog, and ask for the smallest change that makes the tests pass. When the agent finishes, git diff in that copy shows the fix. Stop each remaining copy after you have its diff.

More than 100 packages

A fork makes at most 100 copies per call. For a larger list, fork again from the same base, or take a snapshot of the baseline once and start as many sandboxes from it as the account allows:

TypeScriptimport { Runtime, Sandbox } from "withruntime";const runtime = new Runtime({ waitForCapacityMs: 600_000 });const base = await Sandbox.connect("your-baseline-sandbox-id");const baseline = await base.snapshot({ name: "app-baseline", retentionDays: 1 });await using copy = await runtime.sandboxes.create({ snapshot: baseline.id });// ... the same bump-and-test as above, in as many copies as you need ...await runtime.snapshots.delete(baseline.id);

A kept snapshot is billed at $0.08 per decimal GB per 30-day month, on the bytes it alone stores. Deleting it when the run ends limits that cost to hours (pricing).

Security updates first

npm audit sends a description of your dependency tree to the registry and reports known vulnerabilities, and npm audit fix applies compatible updates by running an install. Run it in the baseline sandbox and put the packages it names at the top of the list, so the security bumps are tested first.

What an upgrade agent needs

What it costs

Take 50 repositories upgraded weekly, 200 runs a month. Each run adds up to 20 minutes of 2 vCPU, 4 GiB sandbox time across the base and its copies, with 900 CPU-seconds of installing, building and testing:

Under 2 cents per repository per run, at Runtime's rates of $0.025 per vCPU-hour of measured CPU and $0.0075 per GiB-hour of memory (pricing). A new account gets 100 free hours of a 1 GB machine in its first 30 days, with no card needed.

Sources

• npm outdated and npm audit, read 26 October 2026.
• pip list: --outdated and --format json, read 26 October 2026.

Related: test generation, parallel agent exploration, what a sandbox snapshot is, coding agent sandbox, static analysis agent.

Facts on this page were checked on 26 October 2026.