Modal vs Runloop pricing, isolation and suspend

Which is better, Modal or Runloop?

It depends on what your agents need besides a sandbox. For evals and agents that mostly wait on a model, Runtime has the lowest bill: $0.64 for the job below, against $3.97 on Modal and $5.33 on Runloop. It bills measured CPU and keeps memory through every pause. Pick Modal for GPUs, or for sandboxes beside Modal functions you already run. Pick Runloop for evaluation and RL rollouts on your own workload, a deployment in your own cloud, or stated SOC 2, HIPAA and GDPR readiness.

At a glance

Which is better for agent evals?

Runloop is built for evals. It evaluates models against your own workload rather than mock benchmarks, and generates RL rollouts checked by running them.

The cost of an eval is mostly the sandbox time, because each trial holds a machine while the model thinks. Runloop's $0.108 per CPU-hour is the highest CPU rate of the three. It is billed for every CPU the devbox has, busy or not, so a 2 CPU devbox pays $0.216 an hour in CPU alone. Runtime bills measured CPU at $0.025 a vCPU-hour. A fork lets every trial start from one prepared sandbox, with installed dependencies and warm processes included. See agent evals and SWE-bench for a full harness.

What happens when you stop for now?

This is where the three differ most.

• Runloop: suspend and resume are on the Pro plan, $250 a month. A suspended devbox is billed for storage only. It keeps its disk but not its memory, so daemons and other processes running at suspend must be started again after resume.
• Modal: a filesystem snapshot keeps files for 30 days by default. A memory snapshot keeps the whole state, but it is in alpha, expires after 7 days, and taking one currently terminates the sandbox.
• Runtime: pause() keeps files, memory and running processes on every account, while you have credit. The next request wakes the sandbox where it stopped.

Pythonfrom withruntime import Sandboxsbx = Sandbox.create(timeout_seconds=600)sbx.spawn("python3 -m http.server 8000")  # a process that must survivesbx.pause()  # memory, files and the server are kept; compute billing stopsagain = Sandbox.connect(sbx.id)again.wake(timeout_seconds=600)check = "import urllib.request; print(urllib.request.urlopen('http://localhost:8000').status)"print(again.exec(["python3", "-c", check]).stdout)  # 200: the server never stoppedagain.stop()

More in how to pause and resume a sandbox.

Cost for the same job

1,000 runs of a 2 vCPU sandbox with 4 GiB: one physical core on Modal, and Runloop's MEDIUM devbox of 2 CPUs, 4 GB and 8 GB of disk. Each run lasts 60 seconds and keeps the CPU busy for 20 CPU-seconds: an agent that mostly waits for a model.

At 100,000 runs a month, the cost is $396.60 on Modal, $532.56 on Runloop, and $63.89 on Runtime. With both CPUs busy for the whole minute, the cost is $1.33 on Runtime, against $3.97 on Modal and $5.33 on Runloop. Runtime is cheaper however busy the sandbox is. Plan fees, storage while suspended, network and free credit are left out.

Which one to pick

• Pick Modal if your sandboxes need GPUs, or sit beside Modal functions and scheduled jobs you already run.
• Pick Runloop if you want evaluation and RL rollouts on your own workload, or a deployment into your own cloud. Runloop offers VPC deployment on Enterprise and states SOC 2, HIPAA and GDPR readiness.
• Pick Runtime for evals and agents at 88% less than Runloop and 84% less than Modal, with pause and forks that keep memory on every account.

Switch from either one

From Modal, the calls map one to one: modal.Sandbox.create(app=...) becomes Sandbox.create(), sb.exec(*args) becomes exec, sb.terminate() becomes stop(), and idle_timeout becomes idlePauseSeconds. From Runloop, a devbox becomes a sandbox. Map its create, command, file and shutdown calls with the table in migration, and replace suspend and resume with pause() and wake().

Or give your coding agent the instruction in migration. It switches the project on a branch, tests it on the free usage, and reports the monthly saving. From Modal, credit you add within 30 days of signing up is matched, up to $1,000 in total. The full side-by-sides are Runtime vs Modal and Runtime vs Runloop.

Sources

Runloop pricing, devbox lifecycle and Modal snapshots were rechecked 26 October 2026. The rest were checked 25 October 2026.

• Modal pricing, Modal resources and billing, Modal sandbox snapshots
• Runloop pricing, Devbox sizes, Devbox lifecycle, Runloop
• Runtime pricing

Facts on this page were checked on 26 October 2026.