How to run linters and security scanners on your own code with an AI agent

Run Ruff, Semgrep and npm audit on your code in a sandbox, give the model the JSON findings, and rescan to confirm each fix.

On Runtime the scan runs with the internet off after install, so neither the tools nor the model's fixes can send your source anywhere. Every sandbox is a Firecracker microVM with its own kernel, and network rules are enforced on the host, where root in the guest cannot change them. A three-minute scan on 2 vCPUs and 4 GiB costs about 0.2 cents at the rates published on 25 September 2026, worked out below.

This page is about code you own or are responsible for: your repositories, your dependencies, your pull requests. Scanning or testing other people's systems is forbidden by Runtime's Acceptable Use Policy and is not what this guide is for.

What the agent adds to the scanners

Scanners are fast and literal. Ruff describes itself as a Python linter with over 900 built-in rules, 10 to 100 times faster than tools such as Flake8, by its own claim (Ruff's documentation, read 25 September 2026). Semgrep matches patterns from rule files or its registry. Both produce long lists, and the hard part is the triage: which findings are real, which are noise in test fixtures, and what the right fix is in this codebase.

A model is good at that triage when it can check its work. The loop:

1. Run every scanner and collect machine-readable output.
2. Give the model the findings, grouped by file, with the code around each.
3. Let it edit files through sandbox tools.
4. Rerun the scanners and the tests. A fix counts only if the finding is gone and the tests still pass.
5. Return the diff for a person to review.

Build a scanner image

TypeScriptimport { Runtime } from "withruntime";const runtime = new Runtime();await runtime.images.build({  name: "scanners",  recipe: { pip: ["ruff", "semgrep"] },});

The core loop

Clone, install, then switch the network off before anything the model writes runs. The Semgrep rules are a local file, so no registry is needed.

TypeScriptimport { Runtime } from "withruntime";import { sandboxTools } from "withruntime/tools";const runtime = new Runtime();await using sbx = await runtime.sandboxes.create({  image: "scanners",  diskMiB: 8192,  timeoutSeconds: 1800,  network: {    internet: true,    allow: ["github.com", "*.github.com", "pypi.org", "*.pythonhosted.org"],  },});await sbx.exec(  ["git", "clone", "--depth", "1", "https://github.com/your-org/your-service.git", "repo"],  {    check: true,    timeoutMs: 300_000,  },);await sbx.exec("pip install -e '.[test]'", {  cwd: "/workspace/repo",  check: true,  timeoutMs: 600_000,});await sbx.network.set({ internet: false });async function scan() {  const ruff = await sbx.exec("ruff check --output-format json . > /workspace/ruff.json", {    cwd: "/workspace/repo",  });  const semgrep = await sbx.exec(    "semgrep scan --config .semgrep.yml --json --metrics off . > /workspace/semgrep.json",    { cwd: "/workspace/repo", timeoutMs: 600_000 },  );  return {    ruff: JSON.parse(await sbx.files.readText("/workspace/ruff.json")) as {      code: string;      filename: string;    }[],    semgrep: JSON.parse(await sbx.files.readText("/workspace/semgrep.json")).results as {      check_id: string;    }[],    exitCodes: [ruff.exitCode, semgrep.exitCode],  };}const before = await scan();const tools = sandboxTools(sbx); // the model edits files and runs commands with these// ... the model's fixing loop runs here ...const after = await scan();const tests = await sbx.exec("python3 -m pytest -q", {  cwd: "/workspace/repo",  timeoutMs: 900_000,});const diff = await sbx.exec("git diff", { cwd: "/workspace/repo" });console.log(  before.ruff.length,  "->",  after.ruff.length,  tests.exitCode,  diff.stdout.length,  tools.length,);

Pythonimport jsonfrom withruntime import Runtimefrom withruntime.tools import sandbox_toolsruntime = Runtime()with runtime.sandboxes.create(    image="scanners",    disk_mib=8192,    timeout_seconds=1800,    network={"internet": True, "allow": ["github.com", "*.github.com", "pypi.org", "*.pythonhosted.org"]},) as sbx:    sbx.exec(["git", "clone", "--depth", "1", "https://github.com/your-org/your-service.git", "repo"],             check=True, timeout_ms=300_000)    sbx.exec("pip install -e '.[test]'", cwd="/workspace/repo", check=True, timeout_ms=600_000)    sbx.network.set(internet=False)    def scan():        sbx.exec("ruff check --output-format json . > /workspace/ruff.json", cwd="/workspace/repo")        sbx.exec("semgrep scan --config .semgrep.yml --json --metrics off . > /workspace/semgrep.json",                 cwd="/workspace/repo", timeout_ms=600_000)        return {            "ruff": json.loads(sbx.files.read_text("/workspace/ruff.json")),            "semgrep": json.loads(sbx.files.read_text("/workspace/semgrep.json"))["results"],        }    before = scan()    run, read, write, ls = sandbox_tools(sbx)    # ... the model's fixing loop runs here ...    after = scan()    tests = sbx.exec("python3 -m pytest -q", cwd="/workspace/repo", timeout_ms=900_000)    diff = sbx.exec("git diff", cwd="/workspace/repo")    print(len(before["ruff"]), "->", len(after["ruff"]), tests.exit_code)

Scanner output goes to a file and is read with files.readText, because a command's stdout keeps at most 64 KiB and a large report is bigger.

What each tool reports

Semgrep's --error exits 1 on findings, which suits CI. Its --config auto logs in to the Semgrep Registry with your project's URL, and registry configs report usage metrics by default (Semgrep's CLI reference, read 25 September 2026). A local rules file with --metrics off, as above, keeps the scan offline. npm audit asks the registry about your dependency tree, so run it before the network goes off. The JSON field names were checked by running Ruff 0.16.9 and Semgrep 1.178.0 on 25 September 2026.

Keep the source in

• Internet off for the fixing phase. sbx.network.set({ internet: false }) refuses every connection at once, open ones included.
• No push rights. The sandbox returns a diff; your code or a person opens the pull request. If the agent must push, store the token as a secret scoped to your Git host, so the sandbox holds only a placeholder.
• A private registry or mirror can go on the allow list in place of the public one.

What a scanning agent needs

What it costs

Take 1,000 scans a month across your repositories, each in a 2 vCPU, 4 GiB sandbox for 3 minutes, with 60 CPU-seconds of scanning and tests:

TextCPU:    1,000 × 60 s / 3,600 × $0.025             = $0.42Memory: 1,000 × 180 s / 3,600 × 4 GiB × $0.0075   = $1.50Total:                                              $1.92

Measured CPU is billed at $0.025 per vCPU-hour and memory at $0.0075 per GiB-hour (pricing). A sandbox that waits on the model during a long fixing session adds $0.03125 an hour. New accounts get 50 trial hours without a card.

Sources

• Ruff and the Ruff linter: rule count, speed claim, ruff check and --fix, read 25 September 2026.
• Semgrep CLI reference: --config, --json, --sarif, --error, --metrics and --config auto, read 25 September 2026.
• npm audit: what it sends and its exit code, read 25 September 2026.

Related: automated code review, dependency upgrades, egress control, turn off a sandbox's internet.

Facts on this page were checked on 25 September 2026.