# Groq tool use: run model-generated code in a sandbox, not with eval Replace `eval` in Groq's tool-calling example with a sandbox: each tool call runs in its own microVM, in parallel, with no internet. **Groq returns tokens fast, and a Runtime sandbox keeps up: a new microVM ran its first Python command 351 ms after the request at the median, and 815 ms at the 95th percentile, over 20 runs on 24 September 2026** ([speed](/docs/speed)). Groq listed `openai/gpt-oss-120b` at 500 tokens a second on its production model page on 25 September 2026. At the 95th percentile a sandbox per tool call adds under a second to a turn, and the model's code never touches your server. ## The problem with the sample Groq's local tool-calling guide, read 25 September 2026, teaches the loop with a calculator whose handler is `eval(expression)` in Python and `eval(expression)` in JavaScript, on your own machine. That is fine for showing the message shapes. In production it runs whatever text the model returns with your process's files, network and keys. The fix keeps Groq's loop exactly as it is and changes only the handler. ## The loop, with calls run side by side Groq's API follows the OpenAI Chat Completions shape: `tool_calls` on the assistant message, then one `role: "tool"` message per call with its `tool_call_id`. A turn can carry several calls, so the handler starts one sandbox per call and waits for all of them together. ```ts check import Groq from "groq-sdk"; import { Sandbox } from "withruntime"; const groq = new Groq(); // GROQ_API_KEY, from your server's environment async function runPython(code: string): Promise { await using sbx = await Sandbox.create({ vcpu: 1, memoryMiB: 1024, network: { internet: false }, timeoutSeconds: 120, onLeaseEnd: "stop", }); await sbx.files.write("/workspace/main.py", code); const run = await sbx.exec(["python3", "main.py"], { timeoutMs: 20_000 }); return JSON.stringify({ exit_code: run.exitCode, timed_out: run.timedOut, stdout: run.stdout, stderr: run.stderr, }); } const tools: Groq.Chat.ChatCompletionTool[] = [ { type: "function", function: { name: "run_python", description: "Execute a Python 3.12 script in a sandbox with no network and return its exit code, " + "stdout and stderr. Use print() for every value you need.", parameters: { type: "object", properties: { code: { type: "string", description: "A complete Python script" } }, required: ["code"], }, }, }, ]; const messages: Groq.Chat.ChatCompletionMessageParam[] = [ { role: "user", content: "Compute 2**521 - 1 and, separately, check whether it is prime." }, ]; for (let turn = 0; turn < 6; turn++) { const completion = await groq.chat.completions.create({ model: "openai/gpt-oss-120b", messages, tools, tool_choice: "auto", }); const message = completion.choices[0]!.message; messages.push(message); if (!message.tool_calls?.length) { console.log(message.content); break; } const outputs = await Promise.all( message.tool_calls.map((call) => runPython(JSON.parse(call.function.arguments).code)), ); message.tool_calls.forEach((call, i) => messages.push({ role: "tool", tool_call_id: call.id, content: outputs[i]! }), ); } ``` ```python check import json from concurrent.futures import ThreadPoolExecutor from groq import Groq from withruntime import Sandbox client = Groq() # GROQ_API_KEY, from your server's environment def run_python(code: str) -> str: with Sandbox.create( vcpu=1, memory_mib=1024, network={"internet": False}, timeout_seconds=120, on_lease_end="stop" ) as sbx: sbx.files.write("/workspace/main.py", code) run = sbx.exec(["python3", "main.py"], timeout_ms=20_000) return json.dumps({"exit_code": run.exit_code, "timed_out": run.timed_out, "stdout": run.stdout, "stderr": run.stderr}) tools = [ { "type": "function", "function": { "name": "run_python", "description": "Execute a Python 3.12 script in a sandbox with no network and return its exit code, " "stdout and stderr. Use print() for every value you need.", "parameters": { "type": "object", "properties": {"code": {"type": "string", "description": "A complete Python script"}}, "required": ["code"], }, }, } ] messages = [{"role": "user", "content": "Compute 2**521 - 1 and, separately, check whether it is prime."}] with ThreadPoolExecutor(max_workers=8) as pool: for _ in range(6): response = client.chat.completions.create( model="openai/gpt-oss-120b", messages=messages, tools=tools, tool_choice="auto" ) message = response.choices[0].message messages.append(message) if not message.tool_calls: print(message.content) break codes = [json.loads(call.function.arguments)["code"] for call in message.tool_calls] for call, output in zip(message.tool_calls, pool.map(run_python, codes)): messages.append({"role": "tool", "tool_call_id": call.id, "content": output}) ``` Each sandbox has 1 vCPU and 1 GiB, which suits short scripts, and a 20-second limit per command. Parallel calls never share a machine, so one call's files or crash cannot affect another. A paid account runs 100 sandboxes at once to start, and the free trial runs eight. ## Groq's built-in code execution Groq also runs Python for you. Its code execution page, checked 25 September 2026, says: | Fact from Groq's page | What it means for you | | -------------------------------------------------------------------------------------- | ----------------------------------------------- | | Add `{"type": "code_interpreter"}` to `tools` | No handler to write | | Supported on `openai/gpt-oss-20b` and `openai/gpt-oss-120b` | Llama and other Groq models need your own tool | | "Only Python is currently supported" | No Node.js, bash or compiled languages | | "Powered by Foundry Labs (E2B), a secure cloud environment" | Groq's code environment is E2B's product | | "No access to external networks"; "each request runs in a fresh, isolated environment" | No `pip install`, no state between requests | | Not a HIPAA covered service; not on regional or sovereign endpoints | Check it before regulated or regional workloads | Choose the built-in tool for quick calculations with GPT-OSS. Choose your own tool when you use another Groq model, need a package or a language beyond Python, want files to survive between requests, or want to set the machine's size and time limit yourself. How E2B itself compares is in [E2B alternatives](/compare/e2b-alternatives). ## Keep the Groq key out of the sandbox The sandbox receives only the model's script; the Groq call happens in your own process. The key needs to be a Runtime secret only when a program inside the sandbox calls Groq, such as an agent that runs there. Bound to `api.groq.com`, it reaches the sandbox as a placeholder the host's proxy replaces on that host alone ([secrets](/docs/security#secrets-sandboxes-never-see)). ## What a call costs A 1 vCPU, 1 GiB sandbox is billed on measured CPU at $0.025 per vCPU-hour, with a floor of a twentieth of a vCPU, and memory at $0.0075 per GiB-hour ([pricing](/docs/pricing)). A 10-second call that keeps its CPU busy costs 10 / 3,600 × ($0.025 + $0.0075) = $0.00009. New accounts get 50 free sandbox hours, no card: ```bash no-run npx withruntime sandbox run --trial -- python3 -c 'print(2**521 - 1)' ``` More: [run untrusted LLM code](/use-cases/run-untrusted-llm-code), [cold start](/glossary/cold-start), [Python in a sandbox](/languages/python), [agent sandbox](/glossary/agent-sandbox). ## Sources Checked 25 September 2026. - [Groq: local tool calling](https://console.groq.com/docs/tool-use/local-tool-calling) - [Groq: code execution](https://console.groq.com/docs/code-execution) - [Groq: models](https://console.groq.com/docs/models) - [groq-sdk on npm](https://www.npmjs.com/package/groq-sdk) (1.6.0) and [groq on PyPI](https://pypi.org/project/groq/) (1.7.0) Facts on this page were checked on 25 September 2026.