Problems / AI systems, hard
15 tasks/s. The agent makes 8 model and tool calls per task; the model allows 4,000 calls a minute and 120 concurrent streams.
Free to open. Runs in your browser, no sign-up.
Keep success ≥ 97%, p99 ≤ 20 s and cost ≤ $1M/month. The model has a quota and a stream limit: cap the steps, add a prompt cache, or buy more (which costs more).
15 requests a second for 60 seconds: 100% task (reads).
What each component does in this design, and what calls it. The simulator mimics each one from these settings.
| Component | Its job here | Settings |
|---|---|---|
| API gatewayAPI gateway | Traffic enters here.Calls Agent orchestrator.Gateways centralise auth, rate limits and routing so each service does not reimplement them. | 100000 slots · 600,000 calls/min |
| Agent orchestratorAgent orchestrator | API gateway forwards to it.Calls LLM endpoint, Tool server.An agent loops: think, call a tool, read the result, repeat until done. | 300 slots · 8 steps |
| Tool serverTool / MCP server | Agent orchestrator calls it.Tools give the model actions such as search or database queries; each call is another round trip. | 40 slots |
| LLM endpointLLM endpoint | Agent orchestrator calls it.Latency has two parts: time to the first token, then a steady stream of tokens per second. | 120 streams · 4,000 calls/min |