# How to control a Linux desktop in a cloud sandbox Call `sbx.desktop.start()`, then drive it with `open`, `click`, `type` and `press`, and call `screenshot()` to see what changed. **On Runtime every sandbox can run a desktop, with no separate product or image to choose.** One call installs and starts a Linux desktop inside the sandbox's own Firecracker microVM and returns a private link to watch it live. Your code, or a model, then moves the mouse, types, presses keys and reads the screen back as PNG. The first start in a sandbox took about 90 seconds and 1 GB of its disk on 23 September 2026, once; later starts in that sandbox skip the install ([CLI](/docs/cli#code-previews-network-rules-and-the-desktop)). ## Start it and drive it ```ts check import { writeFile } from "node:fs/promises"; import { Sandbox } from "withruntime"; await using sbx = await Sandbox.create({ diskMiB: 8192, timeoutSeconds: 1800 }); const { streamUrl } = await sbx.desktop.start({ width: 1280, height: 800 }); console.log("watch it:", streamUrl); // a private link with a one-time token await sbx.desktop.open("https://example.com"); // Firefox; waits if still installing await sbx.desktop.press("ctrl+l"); await sbx.desktop.type("https://www.wikipedia.org"); await sbx.desktop.press("Return"); await sbx.desktop.click(640, 400); await sbx.desktop.scroll(5); await writeFile("after.png", await sbx.desktop.screenshot()); ``` ```python check from withruntime import Sandbox with Sandbox.create(disk_mib=8192, timeout_seconds=1800) as sbx: started = sbx.desktop.start(width=1280, height=800) print("watch it:", started["streamUrl"]) sbx.desktop.open("https://example.com") sbx.desktop.press("ctrl+l") sbx.desktop.type("https://www.wikipedia.org") sbx.desktop.press("Return") sbx.desktop.click(640, 400) sbx.desktop.scroll(5) with open("after.png", "wb") as file: file.write(sbx.desktop.screenshot()) ``` ```bash no-run runtime sandbox desktop "${id}" start # prints a link to watch it runtime sandbox desktop "${id}" open https://example.com runtime sandbox desktop "${id}" screenshot screen.png ``` Coordinates are pixels from the top left of the screen. Key names are xdotool's, space separated: `ctrl+l`, `Return`, `alt+Tab`. ## The actions | Call (TypeScript) | Python | What it does | | ------------------------------------------ | -------------------------------------- | -------------------------------------------------- | | `start({ width, height })` | `start(width=, height=)` | Installs on first use, starts, returns `streamUrl` | | `open(url)` | `open(url)` | Opens the page in Firefox | | `click(x, y)`, `doubleClick`, `rightClick` | `click`, `double_click`, `right_click` | Mouse buttons at a point | | `move(x, y)`, `drag([x1, y1], [x2, y2])` | `move`, `drag((x1, y1), (x2, y2))` | Move the pointer, or press, move and release | | `scroll(dy, { dx })` | `scroll(dy, dx=)` | Wheel clicks; positive `dy` scrolls down | | `type(text)` | `type(text)` | Types a string | | `press(keys)` | `press(keys)` | Key presses and chords | | `windows()`, `focus(id)` | `windows()`, `focus(id)` | Lists windows; brings one to the front | | `launch(argv)` | `launch(argv)` | Starts a program on the desktop, detached | | `screenshot()` | `screenshot()` | The screen as PNG bytes | | `recordings.start()`, `stop(id)` | `recordings.start()`, `stop(id)` | Records the screen to MP4 | ## Let a model drive it A computer-use loop is three steps repeated: take a screenshot, send it to the model with the goal, and perform the action it returns. ```ts check import { Sandbox } from "withruntime"; type Action = { kind: "click"; x: number; y: number } | { kind: "type"; text: string } | { kind: "done" }; declare function nextAction(png: Uint8Array, goal: string): Promise; // your model call await using sbx = await Sandbox.create({ diskMiB: 8192, timeoutSeconds: 1800 }); await sbx.desktop.start(); await sbx.desktop.open("https://example.com"); for (let step = 0; step < 20; step++) { const action = await nextAction(await sbx.desktop.screenshot(), "Find the contact page"); if (action.kind === "done") break; if (action.kind === "click") await sbx.desktop.click(action.x, action.y); if (action.kind === "type") await sbx.desktop.type(action.text); } ``` An agent connected to Runtime's MCP server does the same with `runtime_sandbox_desktop_act` and `runtime_sandbox_desktop_screenshot`, and records with `runtime_sandbox_desktop_record` ([MCP](/docs/mcp#tools)). ## Keep a video of what happened ```ts check import { writeFile } from "node:fs/promises"; import { Sandbox } from "withruntime"; await using sbx = await Sandbox.create({ diskMiB: 8192 }); await sbx.desktop.start(); const rec = await sbx.desktop.recordings.start({ fps: 10, maxMiB: 256 }); await sbx.desktop.open("https://example.com"); await sbx.desktop.recordings.stop(rec.id); await writeFile("session.mp4", await sbx.desktop.recordings.download(rec.id)); ``` A recording uses one encoder thread at low priority and at most 4 Mbit/s, so the desktop comes first. It never grows past `maxMiB`, stops before the disk fills, and a sandbox keeps at most 8 GiB of recordings, counted as its own disk ([a desktop](/docs/javascript#a-desktop)). ## Mistakes and how Runtime handles them - **Acting before the browser exists.** Firefox installs in the background after the first start (about a minute and 0.5 GB more). `open` waits for it rather than failing. - **Sharing the live link.** `streamUrl` is a private preview that carries its own one-time token. Anyone holding it can watch, so keep it to yourself. - **Running out of disk.** The desktop and browser take about 1.5 GB of a default 4 GiB sandbox, which had about 2.5 GiB free when measured on 24 September 2026. The samples ask for 8 GiB. - **Starting a GUI program with `exec`.** A process that `exec` starts ends with its command. `launch` starts a program on the desktop, detached. - **A sandbox left idle.** A paused sandbox keeps the desktop's memory and wakes by itself on the next desktop call, so a slow model does not lose its place ([pause and resume](/how-to/pause-and-resume-a-sandbox)). ## Where this helps - [A browser automation agent](/use-cases/browser-automation-agent) that clicks through sites a script cannot. - Testing a desktop or web app the way a person uses it, with a video of the run. - For a browser run from a script rather than by sight, [Playwright](/integrations/playwright) works in the same sandbox; to capture only images, see [take a screenshot](/how-to/take-a-screenshot). ## Start ```bash no-run npx withruntime sandbox run --trial --keep -- echo ready ``` New accounts get 50 free sandbox hours, no card. The first run prints a link to approve in your browser. Facts on this page were checked on 25 September 2026.