Groq tool use: run model-generated code in a sandbox, not with eval

Replace eval in Groq's tool-calling example with a sandbox: each tool call runs in its own microVM, in parallel, with no internet.

Groq returns tokens fast, and a Runtime sandbox keeps up: a new microVM ran its first Python command 351 ms after the request at the median, and 815 ms at the 95th percentile, over 20 runs on 24 September 2026 (speed). Groq listed openai/gpt-oss-120b at 500 tokens a second on its production model page on 25 September 2026. At the 95th percentile a sandbox per tool call adds under a second to a turn, and the model's code never touches your server.

The problem with the sample

Groq's local tool-calling guide, read 25 September 2026, teaches the loop with a calculator whose handler is eval(expression) in Python and eval(expression) in JavaScript, on your own machine. That is fine for showing the message shapes. In production it runs whatever text the model returns with your process's files, network and keys. The fix keeps Groq's loop exactly as it is and changes only the handler.

The loop, with calls run side by side

Groq's API follows the OpenAI Chat Completions shape: tool_calls on the assistant message, then one role: "tool" message per call with its tool_call_id. A turn can carry several calls, so the handler starts one sandbox per call and waits for all of them together.

TypeScriptimport Groq from "groq-sdk";import { Sandbox } from "withruntime";const groq = new Groq(); // GROQ_API_KEY, from your server's environmentasync function runPython(code: string): Promise<string> {  await using sbx = await Sandbox.create({    vcpu: 1,    memoryMiB: 1024,    network: { internet: false },    timeoutSeconds: 120,    onLeaseEnd: "stop",  });  await sbx.files.write("/workspace/main.py", code);  const run = await sbx.exec(["python3", "main.py"], { timeoutMs: 20_000 });  return JSON.stringify({    exit_code: run.exitCode,    timed_out: run.timedOut,    stdout: run.stdout,    stderr: run.stderr,  });}const tools: Groq.Chat.ChatCompletionTool[] = [  {    type: "function",    function: {      name: "run_python",      description:        "Execute a Python 3.12 script in a sandbox with no network and return its exit code, " +        "stdout and stderr. Use print() for every value you need.",      parameters: {        type: "object",        properties: { code: { type: "string", description: "A complete Python script" } },        required: ["code"],      },    },  },];const messages: Groq.Chat.ChatCompletionMessageParam[] = [  { role: "user", content: "Compute 2**521 - 1 and, separately, check whether it is prime." },];for (let turn = 0; turn < 6; turn++) {  const completion = await groq.chat.completions.create({    model: "openai/gpt-oss-120b",    messages,    tools,    tool_choice: "auto",  });  const message = completion.choices[0]!.message;  messages.push(message);  if (!message.tool_calls?.length) {    console.log(message.content);    break;  }  const outputs = await Promise.all(    message.tool_calls.map((call) => runPython(JSON.parse(call.function.arguments).code)),  );  message.tool_calls.forEach((call, i) =>    messages.push({ role: "tool", tool_call_id: call.id, content: outputs[i]! }),  );}

Pythonimport jsonfrom concurrent.futures import ThreadPoolExecutorfrom groq import Groqfrom withruntime import Sandboxclient = Groq()  # GROQ_API_KEY, from your server's environmentdef run_python(code: str) -> str:    with Sandbox.create(        vcpu=1, memory_mib=1024, network={"internet": False}, timeout_seconds=120, on_lease_end="stop"    ) as sbx:        sbx.files.write("/workspace/main.py", code)        run = sbx.exec(["python3", "main.py"], timeout_ms=20_000)        return json.dumps({"exit_code": run.exit_code, "timed_out": run.timed_out, "stdout": run.stdout, "stderr": run.stderr})tools = [    {        "type": "function",        "function": {            "name": "run_python",            "description": "Execute a Python 3.12 script in a sandbox with no network and return its exit code, "            "stdout and stderr. Use print() for every value you need.",            "parameters": {                "type": "object",                "properties": {"code": {"type": "string", "description": "A complete Python script"}},                "required": ["code"],            },        },    }]messages = [{"role": "user", "content": "Compute 2**521 - 1 and, separately, check whether it is prime."}]with ThreadPoolExecutor(max_workers=8) as pool:    for _ in range(6):        response = client.chat.completions.create(            model="openai/gpt-oss-120b", messages=messages, tools=tools, tool_choice="auto"        )        message = response.choices[0].message        messages.append(message)        if not message.tool_calls:            print(message.content)            break        codes = [json.loads(call.function.arguments)["code"] for call in message.tool_calls]        for call, output in zip(message.tool_calls, pool.map(run_python, codes)):            messages.append({"role": "tool", "tool_call_id": call.id, "content": output})

Each sandbox has 1 vCPU and 1 GiB, which suits short scripts, and a 20-second limit per command. Parallel calls never share a machine, so one call's files or crash cannot affect another. A paid account runs 100 sandboxes at once to start, and the free trial runs eight.

Groq's built-in code execution

Groq also runs Python for you. Its code execution page, checked 25 September 2026, says:

Choose the built-in tool for quick calculations with GPT-OSS. Choose your own tool when you use another Groq model, need a package or a language beyond Python, want files to survive between requests, or want to set the machine's size and time limit yourself. How E2B itself compares is in E2B alternatives.

Keep the Groq key out of the sandbox

The sandbox receives only the model's script; the Groq call happens in your own process. The key needs to be a Runtime secret only when a program inside the sandbox calls Groq, such as an agent that runs there. Bound to api.groq.com, it reaches the sandbox as a placeholder the host's proxy replaces on that host alone (secrets).

What a call costs

A 1 vCPU, 1 GiB sandbox is billed on measured CPU at $0.025 per vCPU-hour, with a floor of a twentieth of a vCPU, and memory at $0.0075 per GiB-hour (pricing). A 10-second call that keeps its CPU busy costs 10 / 3,600 × ($0.025 + $0.0075) = $0.00009. New accounts get 50 free sandbox hours, no card:

Terminalnpx withruntime sandbox run --trial -- python3 -c 'print(2**521 - 1)'

More: run untrusted LLM code, cold start, Python in a sandbox, agent sandbox.

Sources

Checked 25 September 2026.

• Groq: local tool calling
• Groq: code execution
• Groq: models
• groq-sdk on npm (1.6.0) and groq on PyPI (1.7.0)

Facts on this page were checked on 25 September 2026.