Problems / AI systems, medium
An agent takes 12 model-and-tool steps per task. Steps multiply load, cost and failures.
Free to open. Runs in your browser, no sign-up.
Success ≥ 95%, p99 ≤ 20 s.
15 requests a second for 30 seconds: 100% task (reads).
What each component does in this design, and what calls it. The simulator mimics each one from these settings.
| Component | Its job here | Settings |
|---|---|---|
| API gatewayAPI gateway | Traffic enters here.Calls Agent orchestrator.Gateways centralise auth, rate limits and routing so each service does not reimplement them. | 100000 slots · 600,000 calls/min |
| Agent orchestratorAgent orchestrator | API gateway forwards to it.Calls LLM endpoint, Tool server.An agent loops: think, call a tool, read the result, repeat until done. | 300 slots · 12 steps |
| Tool serverTool / MCP server | Agent orchestrator calls it.Tools give the model actions such as search or database queries; each call is another round trip. | 40 slots |
| LLM endpointLLM endpoint | Agent orchestrator calls it.Latency has two parts: time to the first token, then a steady stream of tokens per second. | 120 streams · 4,000 calls/min |
Try it first. Open one hint at a time if you get stuck.
Count model calls per task.
Agent steps run one after another.
Cap the agent’s steps (“cap the agent at 4 steps”).