On Runtime, every sandbox can run a desktop. You do not need a separate product or image. One call starts a Linux desktop inside the sandbox's own Firecracker microVM. It returns a private link that lets you watch the desktop live. Your code, or a model, then moves the mouse, types, presses keys, and reads the screen back as PNG. The desktop is ready on every server. The first start in a sandbox installs nothing, and it takes about a second (CLI).
Start it and drive it
TypeScriptimport { writeFile } from "node:fs/promises";import { Sandbox } from "withruntime";await using sbx = await Sandbox.create({ diskMiB: 8192, timeoutSeconds: 1800 });const { streamUrl } = await sbx.desktop.start({ width: 1280, height: 800 });console.log("watch it:", streamUrl); // a private link that carries its tokenawait sbx.desktop.open("https://example.com"); // Chromiumawait sbx.desktop.press("ctrl+l");await sbx.desktop.type("https://www.wikipedia.org");await sbx.desktop.press("Return");await sbx.desktop.click(640, 400);await sbx.desktop.scroll(5);await writeFile("after.png", await sbx.desktop.screenshot());Pythonfrom withruntime import Sandboxwith Sandbox.create(disk_mib=8192, timeout_seconds=1800) as sbx: started = sbx.desktop.start(width=1280, height=800) print("watch it:", started["streamUrl"]) sbx.desktop.open("https://example.com") sbx.desktop.press("ctrl+l") sbx.desktop.type("https://www.wikipedia.org") sbx.desktop.press("Return") sbx.desktop.click(640, 400) sbx.desktop.scroll(5) with open("after.png", "wb") as file: file.write(sbx.desktop.screenshot())Terminalruntime sandboxes desktop "${id}" start # prints a link to watch itruntime sandboxes desktop "${id}" open https://example.comruntime sandboxes desktop "${id}" screenshot screen.pngCoordinates are pixels from the top left of the screen. Key names are
xdotool's, space separated: ctrl+l, Return, alt+Tab.
The actions
| Call (TypeScript) | Python | What it does |
|---|---|---|
start({ width, height }) |
start(width=, height=) |
Starts the desktop, returns streamUrl |
open(url) |
open(url) |
Opens the page in Chromium |
click(x, y), doubleClick, rightClick |
click, double_click, right_click |
Mouse buttons at a point |
move(x, y), drag([x1, y1], [x2, y2]) |
move, drag((x1, y1), (x2, y2)) |
Move the pointer, or press, move and release |
scroll(dy, { dx }) |
scroll(dy, dx=) |
Wheel clicks; positive dy scrolls down |
type(text) |
type(text) |
Types a string |
press(keys) |
press(keys) |
Key presses and chords |
windows(), focus(id) |
windows(), focus(id) |
Lists windows; brings one to the front |
launch(argv) |
launch(argv) |
Starts a program on the desktop, detached |
screenshot() |
screenshot() |
The screen as PNG bytes |
recordings.start(), stop(id) |
recordings.start(), stop(id) |
Records the screen to MP4 |
Let a model drive it
A computer-use loop repeats three steps. Take a screenshot, send it to the model with the goal, and perform the action the model returns.
TypeScriptimport { Sandbox } from "withruntime";type Action = { kind: "click"; x: number; y: number } | { kind: "type"; text: string } | { kind: "done" };declare function nextAction(png: Uint8Array, goal: string): Promise<Action>; // your model callawait using sbx = await Sandbox.create({ diskMiB: 8192, timeoutSeconds: 1800 });await sbx.desktop.start();await sbx.desktop.open("https://example.com");for (let step = 0; step < 20; step++) { const action = await nextAction(await sbx.desktop.screenshot(), "Find the contact page"); if (action.kind === "done") break; if (action.kind === "click") await sbx.desktop.click(action.x, action.y); if (action.kind === "type") await sbx.desktop.type(action.text);}An agent connected to Runtime's MCP server does the same with
runtime_sandboxes_desktop_act and runtime_sandboxes_desktop_screenshot. It
records with runtime_sandboxes_desktop_record
(MCP).
Keep a video of what happened
TypeScriptimport { writeFile } from "node:fs/promises";import { Sandbox } from "withruntime";await using sbx = await Sandbox.create({ diskMiB: 8192 });await sbx.desktop.start();const rec = await sbx.desktop.recordings.start({ fps: 10, maxMiB: 256 });await sbx.desktop.open("https://example.com");await sbx.desktop.recordings.stop(rec.id);await writeFile("session.mp4", await sbx.desktop.recordings.download(rec.id));A recording uses one encoder thread at low priority, and at most 4 Mbit/s. The
desktop therefore comes first. A recording never grows past maxMiB, and it
stops before the disk fills. A sandbox keeps at most 8 GiB of recordings, which
count toward its own disk (a desktop).
Mistakes and how Runtime handles them
- Acting before the browser exists. The browser comes from the host
at once, with no download.
openwaits for it rather than failing. - Sharing the live link.
streamUrlis a private preview that carries its own token. Anyone holding it can watch, so keep it to yourself. - Running out of disk. The desktop and browser install nothing on the sandbox's disk, but the first recording installs ffmpeg and each recording is a file there. A default 4 GiB sandbox had about 2.5 GiB free when measured on 24 September 2026. The samples ask for 8 GiB.
- Starting a GUI program with
exec. A process thatexecstarts ends with its command.launchstarts a program on the desktop, detached. - A sandbox left idle. A paused sandbox keeps the desktop's memory and wakes by itself on the next desktop call, so a slow model does not lose its place (pause and resume).
Where this helps
- A browser automation agent that clicks through sites a script cannot.
- Testing a desktop or web app the way a person uses it, with a video of the run.
- For a browser run from a script rather than by sight, Playwright works in the same sandbox. To capture only images, see take a screenshot.
Start
Terminalnpx withruntime sandboxes run --keep -- echo readyA new account gets 100 free hours of a 1 GB machine in its first 30 days, with no card. The first run prints a link to approve in your browser.
Facts on this page were checked on 26 October 2026.