# Build a GitHub issue-to-pull-request agent that handles review

Label an issue and a webhook starts an agent in that issue's own sandbox, which opens a draft pull request and wakes for each review.

**On Runtime (withruntime.com) the issue's sandbox pauses between review rounds and has run its next command 153 ms after the wake request, on Runtime's servers, with the checkout, the installed packages and the agent's notes exactly as it left them.**
Opening the first pull request is the demo. The product is everything after
it: a reviewer asks for a change on Tuesday, CI fails on Wednesday, someone
comments on Thursday, and the PR merges on Friday. This post builds the
service that handles that whole life: the events it listens to, a webhook
receiver you can run as written, the worker that drives the sandbox, and the
cost of one issue from label to merge.

## What events does an issue agent have to handle?

Six, and each one maps to a single thing the sandbox does.

| GitHub event                             | What the service does                            | The issue's sandbox  |
| ---------------------------------------- | ------------------------------------------------ | -------------------- |
| `issues` labeled `agent`                 | Clone, branch, run the agent, open a draft PR    | Created, then paused |
| `pull_request_review`, changes requested | Give the agent the review, push a new commit     | Woken, then paused   |
| `issue_comment` mentioning the agent     | Same, with the comment as the instruction        | Woken, then paused   |
| `check_suite` completed with a failure   | Give the agent the failing check, push a fix     | Woken, then paused   |
| `pull_request` closed, merged or not     | Nothing more to do                               | Deleted              |
| Anything the agent itself caused         | Ignore it, or the agent answers itself in a loop | Untouched            |

The last row is the one people learn the hard way. The agent's own comment
mentions the agent, its own push triggers CI, and a service that reacts to
everything ends up talking to itself all night.

## Why keep one sandbox per issue for the whole pull request?

Because each review round needs the same working tree the last one left, and
rebuilding it costs more than keeping it. A fresh machine per round means a
new clone, a new `npm install` or `pip install`, cold test caches and an
agent that has lost its scratch notes. That is minutes per round before the
agent reads the review.

A paused sandbox keeps all of it, memory included, and costs storage only
while the reviewer is busy elsewhere. The sandbox's name is derived from the
repository and issue number, so `getOrCreate` finds it on every event, and
any request wakes it ([pause and resume](/how-to/pause-and-resume-a-sandbox)).
A paused sandbox is kept as long as your account has credit, so a pull
request that sits for weeks wakes the same sandbox; one deleted meanwhile
is replaced by a fresh sandbox, which checks out the branch the earlier one
pushed.

## How do you receive GitHub's webhooks safely?

Check the signature against the raw body, skip deliveries you have already
seen, ignore the agent's own actions, and answer fast. GitHub signs each
delivery with HMAC-SHA256 in `X-Hub-Signature-256`, names it with a unique
`X-GitHub-Delivery` id, may deliver the same one again, and gives up on a
receiver that takes longer than ten seconds. So the receiver only verifies
and queues; a worker does the slow part. This receiver runs as written and
replays seven deliveries against itself:

```ts
import { createHmac, timingSafeEqual } from "node:crypto";

const SECRET = "the-webhook-secret-from-your-github-app";
const BOT = "acme-agent[bot]";
const seen = new Set<string>(); // keep delivery ids in your database in production
const prToIssue = new Map([[51, 42]]); // saved when the agent opens pull request #51

type Payload = {
  action?: string;
  sender?: { login: string };
  label?: { name: string };
  issue?: { number: number; pull_request?: object };
  comment?: { body: string };
  review?: { state: string; body: string | null };
  pull_request?: { head: { ref: string } };
  check_suite?: { conclusion: string | null; head_branch: string | null };
};
type Job = { kind: "start" | "revise" | "fix-checks" | "cleanup"; issue: number; note: string };

function signed(body: string, signature: string | null): boolean {
  const expected = Buffer.from(`sha256=${createHmac("sha256", SECRET).update(body).digest("hex")}`);
  const given = Buffer.from(signature ?? "");
  return given.length === expected.length && timingSafeEqual(given, expected);
}

const issueOf = (branch: string | null | undefined) =>
  Number(/^agent\/issue-(\d+)$/.exec(branch ?? "")?.[1] ?? 0);

function route(event: string, p: Payload): Job | string {
  if (p.sender?.login === BOT) return "ignored: the agent's own action"; // no loops
  if (event === "issues" && p.action === "labeled" && p.label?.name === "agent")
    return { kind: "start", issue: p.issue!.number, note: "" };
  if (event === "pull_request_review" && p.review?.state === "changes_requested")
    return { kind: "revise", issue: issueOf(p.pull_request?.head.ref), note: p.review.body ?? "" };
  if (
    event === "issue_comment" &&
    p.action === "created" &&
    p.comment?.body.includes("@acme-agent")
  )
    return {
      kind: "revise",
      issue: prToIssue.get(p.issue!.number) ?? p.issue!.number,
      note: p.comment.body,
    };
  if (
    event === "check_suite" &&
    p.check_suite?.conclusion === "failure" &&
    issueOf(p.check_suite.head_branch)
  )
    return { kind: "fix-checks", issue: issueOf(p.check_suite.head_branch), note: "" };
  if (event === "pull_request" && p.action === "closed" && issueOf(p.pull_request?.head.ref))
    return { kind: "cleanup", issue: issueOf(p.pull_request?.head.ref), note: "" };
  return "ignored: not for the agent";
}

function receive(delivery: string, event: string, body: string, signature: string | null): string {
  if (!signed(body, signature)) return "401 bad signature";
  if (seen.has(delivery)) return "200 duplicate delivery, skipped";
  seen.add(delivery);
  const job = route(event, JSON.parse(body));
  return typeof job === "string" ? `200 ${job}` : `202 queued ${job.kind} for issue #${job.issue}`;
}

const sign = (body: string) => `sha256=${createHmac("sha256", SECRET).update(body).digest("hex")}`;
const tests: [string, string, object, boolean?][] = [
  [
    "d1",
    "issues",
    {
      action: "labeled",
      label: { name: "agent" },
      issue: { number: 42 },
      sender: { login: "maya" },
    },
  ],
  [
    "d1",
    "issues",
    {
      action: "labeled",
      label: { name: "agent" },
      issue: { number: 42 },
      sender: { login: "maya" },
    },
  ],
  [
    "d2",
    "issue_comment",
    {
      action: "created",
      issue: { number: 42 },
      comment: { body: "Done, @acme-agent opened a PR" },
      sender: { login: BOT },
    },
  ],
  [
    "d3",
    "issue_comment",
    {
      action: "created",
      issue: { number: 51, pull_request: {} },
      comment: { body: "@acme-agent add a test" },
      sender: { login: "maya" },
    },
  ],
  [
    "d4",
    "pull_request_review",
    {
      review: { state: "changes_requested", body: "Handle empty input" },
      pull_request: { head: { ref: "agent/issue-42" } },
      sender: { login: "maya" },
    },
  ],
  [
    "d5",
    "check_suite",
    {
      action: "completed",
      check_suite: { conclusion: "failure", head_branch: "agent/issue-42" },
      sender: { login: "github-actions[bot]" },
    },
  ],
  [
    "d6",
    "pull_request",
    {
      action: "closed",
      pull_request: { head: { ref: "agent/issue-42" } },
      sender: { login: "maya" },
    },
  ],
  ["d7", "issues", { action: "labeled", label: { name: "agent" }, issue: { number: 7 } }, false],
];
for (const [delivery, event, payload, valid = true] of tests) {
  const body = JSON.stringify(payload);
  console.log(
    `${delivery} ${event}: ${receive(delivery, event, body, valid ? sign(body) : "sha256=0")}`,
  );
}
```

```python
import hashlib
import hmac
import json
import re

SECRET = b"the-webhook-secret-from-your-github-app"
BOT = "acme-agent[bot]"
seen = set()  # keep delivery ids in your database in production
pr_to_issue = {51: 42}  # saved when the agent opens pull request #51


def signed(body: bytes, signature: str | None) -> bool:
    expected = "sha256=" + hmac.new(SECRET, body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, signature or "")


def issue_of(branch):
    m = re.fullmatch(r"agent/issue-(\d+)", branch or "")
    return int(m.group(1)) if m else 0


def route(event, p):
    if p.get("sender", {}).get("login") == BOT:
        return "ignored: the agent's own action"  # no loops
    if event == "issues" and p.get("action") == "labeled" and p.get("label", {}).get("name") == "agent":
        return {"kind": "start", "issue": p["issue"]["number"], "note": ""}
    if event == "pull_request_review" and p.get("review", {}).get("state") == "changes_requested":
        return {"kind": "revise", "issue": issue_of(p["pull_request"]["head"]["ref"]),
                "note": p["review"].get("body") or ""}
    if event == "issue_comment" and p.get("action") == "created" and "@acme-agent" in p["comment"]["body"]:
        n = p["issue"]["number"]
        return {"kind": "revise", "issue": pr_to_issue.get(n, n), "note": p["comment"]["body"]}
    suite = p.get("check_suite") or {}
    if event == "check_suite" and suite.get("conclusion") == "failure" and issue_of(suite.get("head_branch")):
        return {"kind": "fix-checks", "issue": issue_of(suite["head_branch"]), "note": ""}
    ref = (p.get("pull_request") or {}).get("head", {}).get("ref")
    if event == "pull_request" and p.get("action") == "closed" and issue_of(ref):
        return {"kind": "cleanup", "issue": issue_of(ref), "note": ""}
    return "ignored: not for the agent"


def receive(delivery, event, body: bytes, signature):
    if not signed(body, signature):
        return "401 bad signature"
    if delivery in seen:
        return "200 duplicate delivery, skipped"
    seen.add(delivery)
    job = route(event, json.loads(body))
    return f"200 {job}" if isinstance(job, str) else f"202 queued {job['kind']} for issue #{job['issue']}"


def sign(body: bytes) -> str:
    return "sha256=" + hmac.new(SECRET, body, hashlib.sha256).hexdigest()


labeled = {"action": "labeled", "label": {"name": "agent"}, "issue": {"number": 42}, "sender": {"login": "maya"}}
tests = [
    ("d1", "issues", labeled, True),
    ("d1", "issues", labeled, True),
    ("d2", "issue_comment", {"action": "created", "issue": {"number": 42},
                             "comment": {"body": "Done, @acme-agent opened a PR"}, "sender": {"login": BOT}}, True),
    ("d3", "issue_comment", {"action": "created", "issue": {"number": 51, "pull_request": {}},
                             "comment": {"body": "@acme-agent add a test"}, "sender": {"login": "maya"}}, True),
    ("d4", "pull_request_review", {"review": {"state": "changes_requested", "body": "Handle empty input"},
                                   "pull_request": {"head": {"ref": "agent/issue-42"}}, "sender": {"login": "maya"}}, True),
    ("d5", "check_suite", {"action": "completed", "check_suite": {"conclusion": "failure", "head_branch": "agent/issue-42"},
                           "sender": {"login": "github-actions[bot]"}}, True),
    ("d6", "pull_request", {"action": "closed", "pull_request": {"head": {"ref": "agent/issue-42"}},
                            "sender": {"login": "maya"}}, True),
    ("d7", "issues", {**labeled, "issue": {"number": 7}}, False),
]
for delivery, event, payload, valid in tests:
    body = json.dumps(payload).encode()
    print(f"{delivery} {event}: {receive(delivery, event, body, sign(body) if valid else 'sha256=0')}")
```

It prints the same seven lines in both languages:

```
d1 issues: 202 queued start for issue #42
d1 issues: 200 duplicate delivery, skipped
d2 issue_comment: 200 ignored: the agent's own action
d3 issue_comment: 202 queued revise for issue #42
d4 pull_request_review: 202 queued revise for issue #42
d5 check_suite: 202 queued fix-checks for issue #42
d6 pull_request: 202 queued cleanup for issue #42
d7 issues: 401 bad signature
```

Three details are easy to miss. The comparison is constant-time, so the
signature cannot be guessed byte by byte. A comment on the pull request
arrives as an `issue_comment` with the pull request's number, so the service
keeps a map from pull request to issue. And the branch name,
`agent/issue-42`, is how check and close events find their issue again.

## What does the worker do for each job?

It finds or creates the issue's sandbox, writes the task, runs the agent,
runs the tests, pushes, and reports back on GitHub. Then it pauses the
sandbox until the next event. The model's key and the token for `git push`
are Runtime secrets, set up once as in
[the overnight Claude Code post](/blog/run-claude-code-overnight-in-the-cloud),
so the sandbox holds placeholders. Your service keeps its own GitHub token
for opening the pull request and commenting:

```ts check
import { Runtime, Sandbox } from "withruntime";

const REPO = "acme/app";
const runtime = new Runtime();
const headers = {
  authorization: `Bearer ${process.env.GITHUB_TOKEN}`,
  accept: "application/vnd.github+json",
};
async function github(path: string, body?: object) {
  const res = await fetch(`https://api.github.com/repos/${REPO}${path}`, {
    method: body ? "POST" : "GET",
    headers,
    body: body ? JSON.stringify(body) : undefined,
  });
  if (!res.ok) throw new Error(`GitHub ${path}: ${res.status} ${await res.text()}`);
  return res.json();
}

type Job = { kind: "start" | "revise" | "fix-checks" | "cleanup"; issue: number; note: string };

export async function work(job: Job): Promise<number | undefined> {
  const name = `${REPO.replace("/", "-")}-issue-${job.issue}`;
  if (job.kind === "cleanup") {
    for await (const sbx of await runtime.sandboxes.list({ name })) await sbx.delete();
    return;
  }
  const branch = `agent/issue-${job.issue}`;
  const sbx = await Sandbox.getOrCreate(name, {
    vcpu: 2,
    memoryMiB: 4096,
    labels: { repo: REPO.replace("/", "-"), issue: String(job.issue) },
  });
  const app = { cwd: "/workspace/app" };
  if (!sbx.info.reused) {
    await sbx.exec(["git", "clone", `https://github.com/${REPO}.git`, "/workspace/app"], {
      check: true,
    });
    // Picks up the branch if an earlier sandbox for this issue already pushed it.
    await sbx.exec(`git switch ${branch} || git switch -c ${branch}`, { ...app, check: true });
  }
  const issue = await github(`/issues/${job.issue}`);
  const why = {
    start: "Resolve the issue.",
    revise: "Address this review:",
    "fix-checks": "CI failed. Read the failing check and fix it.",
  };
  await sbx.files.write(
    "/workspace/TASK.md",
    `# ${issue.title}\n\n${issue.body ?? ""}\n\n## Now\n\n${why[job.kind]}\n\n${job.note}\n`,
  );

  await sbx.exec(process.env.AGENT_COMMAND!, { ...app, timeoutMs: 1_800_000 }); // a round's ceiling
  if (await sbx.files.exists("/workspace/QUESTION.md")) {
    const question = await sbx.files.readText("/workspace/QUESTION.md");
    await sbx.exec("rm /workspace/QUESTION.md");
    await github(`/issues/${job.issue}/comments`, { body: `Before I go on: ${question}` });
    await sbx.pause();
    return;
  }
  const tests = await sbx.exec("npm test 2>&1 | tail -20", { ...app, timeoutMs: 600_000 });
  const commit = `git add -A && git -c user.name="Acme Agent" -c user.email=agent@acme.dev commit -qm ${JSON.stringify(`#${job.issue}: ${job.kind}`)} && git push -q -u origin ${branch}`;
  const pushed = await sbx.exec(commit, app);
  if (pushed.exitCode === 0 && job.kind === "start") {
    const pr = await github("/pulls", {
      title: issue.title,
      head: branch,
      base: "main",
      draft: true,
      body: `Closes #${job.issue}\n\nTests:\n\n\`\`\`\n${tests.stdout}\n\`\`\``,
    });
    await sbx.pause();
    return pr.number; // save pull request -> issue for comments on the pull request
  }
  const said = pushed.exitCode === 0 ? "Pushed a new commit." : "No change made this round.";
  await github(`/issues/${job.issue}/comments`, {
    body: `${said}\n\n\`\`\`\n${tests.stdout}\n\`\`\``,
  });
  await sbx.pause();
}
```

````python check
import json
import os
import urllib.request

from withruntime import Runtime, Sandbox

REPO = "acme/app"
runtime = Runtime()
HEADERS = {"authorization": f"Bearer {os.environ['GITHUB_TOKEN']}",
           "accept": "application/vnd.github+json", "content-type": "application/json"}
WHY = {"start": "Resolve the issue.", "revise": "Address this review:",
       "fix-checks": "CI failed. Read the failing check and fix it."}


def github(path, body=None):
    req = urllib.request.Request(f"https://api.github.com/repos/{REPO}{path}", headers=HEADERS,
                                 data=json.dumps(body).encode() if body else None,
                                 method="POST" if body else "GET")
    with urllib.request.urlopen(req, timeout=30) as res:
        return json.load(res)


def work(job):
    name = f"{REPO.replace('/', '-')}-issue-{job['issue']}"
    if job["kind"] == "cleanup":
        for sbx in runtime.sandboxes.list(name=name):
            sbx.delete()
        return None
    branch = f"agent/issue-{job['issue']}"
    sbx = Sandbox.get_or_create(name, vcpu=2, memory_mib=4096,
                                labels={"repo": REPO.replace("/", "-"), "issue": str(job["issue"])})
    app = "/workspace/app"
    if not sbx.info.get("reused"):
        sbx.exec(["git", "clone", f"https://github.com/{REPO}.git", app], check=True)
        # Picks up the branch if an earlier sandbox for this issue already pushed it.
        sbx.exec(f"git switch {branch} || git switch -c {branch}", cwd=app, check=True)
    issue = github(f"/issues/{job['issue']}")
    sbx.files.write("/workspace/TASK.md", f"# {issue['title']}\n\n{issue.get('body') or ''}\n\n"
                    f"## Now\n\n{WHY[job['kind']]}\n\n{job['note']}\n")

    sbx.exec(os.environ["AGENT_COMMAND"], cwd=app, timeout_ms=1_800_000)  # a round's ceiling
    if sbx.files.exists("/workspace/QUESTION.md"):
        question = sbx.files.read_text("/workspace/QUESTION.md")
        sbx.exec("rm /workspace/QUESTION.md")
        github(f"/issues/{job['issue']}/comments", {"body": f"Before I go on: {question}"})
        sbx.pause()
        return None
    tests = sbx.exec("npm test 2>&1 | tail -20", cwd=app, timeout_ms=600_000)
    message = json.dumps(f"#{job['issue']}: {job['kind']}")
    pushed = sbx.exec(f'git add -A && git -c user.name="Acme Agent" -c user.email=agent@acme.dev '
                      f"commit -qm {message} && git push -q -u origin {branch}", cwd=app)
    if pushed.exit_code == 0 and job["kind"] == "start":
        pr = github("/pulls", {"title": issue["title"], "head": branch, "base": "main", "draft": True,
                               "body": f"Closes #{job['issue']}\n\nTests:\n\n```\n{tests.stdout}\n```"})
        sbx.pause()
        return pr["number"]  # save pull request -> issue for comments on the pull request
    said = "Pushed a new commit." if pushed.exit_code == 0 else "No change made this round."
    github(f"/issues/{job['issue']}/comments", {"body": f"{said}\n\n```\n{tests.stdout}\n```"})
    sbx.pause()
    return None
````

`AGENT_COMMAND` is whatever coding agent you run headless, reading
`/workspace/TASK.md`: Claude Code, Codex or your own loop
([coding agent sandbox](/use-cases/coding-agent-sandbox)). The worker's
tests are a report for the reviewer, not a gate; your normal CI still runs
on the pull request ([CI for agent pull requests](/use-cases/ci-for-agent-prs)).

## What should the agent be told?

The issue, the round's instruction, and a way out. Put these three lines in
the agent's standing instructions:

- **Stay on the issue.** Change only what it needs; do not edit CI
  configuration, lock files or tests you were not asked to touch.
- **Prove it.** Run the tests before finishing, and add one that fails
  without your change.
- **Ask instead of guessing.** If the issue is ambiguous, write the question
  to `/workspace/QUESTION.md` and stop.

The last one is what makes the agent safe to point at real issues. An agent
that guesses opens a confident pull request for the wrong reading; one that
asks gets a one-line answer from the issue's author, and the comment wakes the
sandbox to continue.

## How do you keep it from doing damage?

By making a pull request the only thing it can produce. Protect the default
branch so every change needs an approved review; the agent's token can push
its own branches and nothing lands without a person. Open pull requests as
drafts. Cap each round with the agent command's timeout, cap the number of
rounds per issue in your service (five is plenty), and set a
[daily spending limit](/how-to/set-a-daily-spending-limit) on the account so
a runaway day has a ceiling. The sandbox's network rules can be narrowed to
GitHub, your package registry and the model API.

## What does one issue cost?

Take one issue from label to merge: a first round of 20 minutes, three review
rounds of 8 minutes each, all on 2 vCPU and 4 GiB with the agent's tools
averaging half a vCPU, and two days paused in between at 1.5 GB:

```
CPU:     44 min × 0.5 vCPU × $0.025 / 60          = $0.0092
Memory:  44 min × 4 GiB × $0.0075 / 60            = $0.0220
Paused:  1.5 GB × $0.08 × 2 days / 30 days    = $0.0080
Total:                                          $0.0392
```

The model's tokens for those 44 minutes will cost many times more. A hundred
issues a month is about $3.92
in sandboxes ([pricing](/pricing)).

## In short

- An issue agent is an event handler: label, review, comment, failed check and close, each mapped to one sandbox action.
- Keep one sandbox per issue for the pull request's life, paused between rounds, so each review starts where the last one stopped.
- Verify GitHub's signature, skip repeated deliveries, ignore the agent's own events and answer within ten seconds.
- Let the agent ask a question instead of guessing, and make a reviewed pull request the only thing it can produce.
- One issue with three review rounds costs about $0.0392 in sandbox time.

## Run it on Runtime

Runtime costs 42% to 88% less than fourteen other sandbox providers
for an agent that mostly waits on a model ([compare costs](/how-to/compare-your-costs)).
A Runtime sandbox named after the issue is found again by `getOrCreate`,
pauses between rounds for storage only, and runs its next command
153 ms after the wake request on Runtime's servers. Secrets
keep the GitHub and model keys out of the machine. Start with
100 free hours, no card: [sign in](/sign-in) or read
[get started](/docs/start).