What is computer use?
Computer use is an AI model operating a desktop the way a person does: it reads screenshots and answers with clicks, key presses and typing.
Runtime gives a computer-use agent a Linux desktop inside its own microVM, driven from the SDK, with a private live view and MP4 recording. Anthropic's own documentation tells developers to run computer use in "a dedicated virtual machine or container" and to limit its internet to an allow list; a Runtime sandbox is both, with the rules enforced outside the machine (security).
How it works
In Anthropic's API, computer use is a client toolset: the model returns tool
calls such as screenshot, left_click or type, your application performs
each one on a real screen and returns the result, a screenshot as an image or
a short acknowledgement, and the model decides the next action. Anthropic
lists it as generally available on the Claude API and Google Cloud, with the
computer_toolset_20260801 version. The loop is the ordinary
agent loop, with pixels as the feedback.
Anthropic's precautions, and Runtime's answer
| Anthropic's recommendation | On Runtime |
|---|---|
| A dedicated VM or container, minimal privileges | A Firecracker microVM per sandbox; root there controls only the guest |
| Keep sensitive data, such as logins, from the model | Secrets the sandbox never holds; the proxy adds them for named hosts |
| Limit internet access to an allow list of domains | Per-sandbox allow lists, or internet off, applied at once |
| A human confirms consequential actions | A live view to watch, and a recording to review afterwards |
Anthropic also warns that the model "can follow commands found in content", such as a web page or an image, that conflict with yours: the prompt injection risk, which the precautions above are there to contain.
Drive the desktop
TypeScriptimport { writeFile } from "node:fs/promises";import { Sandbox } from "withruntime";await using sbx = await Sandbox.create();const { streamUrl } = await sbx.desktop.start({ width: 1280, height: 800 });console.log("private live view:", streamUrl);await sbx.desktop.open("https://example.com"); // waits for Firefox on first useawait sbx.network.set({ internet: true, allow: ["example.com"] }); // then narrow itawait writeFile("step-0.png", await sbx.desktop.screenshot()); // send to the modelawait sbx.desktop.click(640, 400); // the model's chosen actionawait sbx.desktop.type("hello");Pythonfrom withruntime import Sandboxwith Sandbox.create() as sbx: sbx.desktop.start(width=1280, height=800) sbx.desktop.open("https://example.com") sbx.network.set(internet=True, allow=["example.com"]) with open("step-0.png", "wb") as file: file.write(sbx.desktop.screenshot()) sbx.desktop.click(640, 400) sbx.desktop.type("hello")The first start in a sandbox installs the desktop, about 90 seconds and 1 GB
of disk, once; Firefox follows in the background, which is why the sample
narrows the network after the first open. The same actions are
MCP tools, runtime_sandbox_desktop_act and
runtime_sandbox_desktop_screenshot (MCP).
Related
- How to run a browser automation agent
- What is a headless browser?
- A desktop in the JavaScript SDK
- What is an agent sandbox?
Sources
- Anthropic, Computer use tool, checked 25 September 2026
Facts on this page were checked on 25 September 2026.