Groq tool use run model-generated code in a sandbox, not with eval

Groq returns tokens quickly. Groq's production model page listed openai/gpt-oss-120b at 500 tokens a second on 25 October 2026. The model's code never touches your server.

The problem with the sample

Groq's local tool-calling guide, read 25 October 2026, teaches the loop with a calculator. Its handler is eval(expression) in Python and eval(expression) in JavaScript, and it runs on your own machine. That is fine for showing the message shapes. In production, it runs whatever text the model returns, with your process's files, network and keys. The fix keeps Groq's loop as it is and changes only the handler.

The loop, with calls run side by side

Groq's API follows the OpenAI Chat Completions shape. The assistant message carries tool_calls. Each call then gets one role: "tool" message with its tool_call_id. A turn can carry several calls, so the handler starts one sandbox per call and waits for all of them.

TypeScriptimport Groq from "groq-sdk";import { Sandbox } from "withruntime";const groq = new Groq(); // GROQ_API_KEY, from your server's environmentasync function runPython(code: string): Promise<string> {  await using sbx = await Sandbox.create({    vcpu: 1,    memoryMiB: 1024,    network: { internet: false },    timeoutSeconds: 120,    onLeaseEnd: "stop",  });  await sbx.files.write("/workspace/main.py", code);  const run = await sbx.exec(["python3", "main.py"], { timeoutMs: 20_000 });  return JSON.stringify({    exit_code: run.exitCode,    timed_out: run.timedOut,    stdout: run.stdout,    stderr: run.stderr,  });}const tools: Groq.Chat.ChatCompletionTool[] = [  {    type: "function",    function: {      name: "run_python",      description:        "Execute a Python 3.12 script in a sandbox with no network and return its exit code, " +        "stdout and stderr. Use print() for every value you need.",      parameters: {        type: "object",        properties: { code: { type: "string", description: "A complete Python script" } },        required: ["code"],      },    },  },];const messages: Groq.Chat.ChatCompletionMessageParam[] = [  { role: "user", content: "Compute 2**521 - 1 and, separately, check whether it is prime." },];for (let turn = 0; turn < 6; turn++) {  const completion = await groq.chat.completions.create({    model: "openai/gpt-oss-120b",    messages,    tools,    tool_choice: "auto",  });  const message = completion.choices[0]!.message;  messages.push(message);  if (!message.tool_calls?.length) {    console.log(message.content);    break;  }  const outputs = await Promise.all(    message.tool_calls.map((call) => runPython(JSON.parse(call.function.arguments).code)),  );  message.tool_calls.forEach((call, i) =>    messages.push({ role: "tool", tool_call_id: call.id, content: outputs[i]! }),  );}

Pythonimport jsonfrom concurrent.futures import ThreadPoolExecutorfrom groq import Groqfrom withruntime import Sandboxclient = Groq()  # GROQ_API_KEY, from your server's environmentdef run_python(code: str) -> str:    with Sandbox.create(        vcpu=1, memory_mib=1024, network={"internet": False}, timeout_seconds=120, on_lease_end="stop"    ) as sbx:        sbx.files.write("/workspace/main.py", code)        run = sbx.exec(["python3", "main.py"], timeout_ms=20_000)        return json.dumps({"exit_code": run.exit_code, "timed_out": run.timed_out, "stdout": run.stdout, "stderr": run.stderr})tools = [    {        "type": "function",        "function": {            "name": "run_python",            "description": "Execute a Python 3.12 script in a sandbox with no network and return its exit code, "            "stdout and stderr. Use print() for every value you need.",            "parameters": {                "type": "object",                "properties": {"code": {"type": "string", "description": "A complete Python script"}},                "required": ["code"],            },        },    }]messages = [{"role": "user", "content": "Compute 2**521 - 1 and, separately, check whether it is prime."}]with ThreadPoolExecutor(max_workers=8) as pool:    for _ in range(6):        response = client.chat.completions.create(            model="openai/gpt-oss-120b", messages=messages, tools=tools, tool_choice="auto"        )        message = response.choices[0].message        messages.append(message)        if not message.tool_calls:            print(message.content)            break        codes = [json.loads(call.function.arguments)["code"] for call in message.tool_calls]        for call, output in zip(message.tool_calls, pool.map(run_python, codes)):            messages.append({"role": "tool", "tool_call_id": call.id, "content": output})

Each sandbox has 1 vCPU and 1 GiB of memory. That suits short scripts. Each command has a 20-second limit. Parallel calls never share a machine, so one call's files or crash cannot affect another. A paid account runs 100 sandboxes at the same time. An account without credit runs four.

Groq's built-in code execution

Groq also runs Python for you. Its code execution page, checked 25 October 2026, says:

Use the built-in tool for quick calculations with GPT-OSS. Use your own tool when you use another Groq model, need a package or a language beyond Python, want files to persist between requests, or want to set the machine's size and time limit yourself. For how E2B itself compares, see E2B alternatives.

Keep the Groq key out of the sandbox

The sandbox receives only the model's script. The Groq call runs in your own process. The key needs to be a Runtime secret only when a program inside the sandbox calls Groq, such as an agent that runs there. Bound to api.groq.com, the key reaches the sandbox as a placeholder. The host's proxy replaces it on that host only (secrets).

What a call costs

A 1 vCPU, 1 GiB sandbox is billed on measured CPU at $0.025 per vCPU-hour, with a floor of a twentieth of a vCPU, and memory at $0.0075 per GiB-hour (pricing). A 10-second call that keeps its CPU busy costs 10 / 3,600 × ($0.025 + $0.0075) = $0.00009. A new account gets 100 free hours of a 1 GB machine in its first 30 days, without a card:

Terminalnpx withruntime sandboxes run -- python3 -c 'print(2**521 - 1)'

More: run untrusted LLM code, cold start, Python in a sandbox, agent sandbox.

Sources

Checked 25 October 2026.

• Groq: local tool calling
• Groq: code execution
• Groq: models
• groq-sdk on npm (1.6.0) and groq on PyPI (1.7.0)

Facts on this page were checked on 25 October 2026.