Mattbusel/tokio-prompt-orchestrator

Tokio pipeline for serving LLM requests: bounded five-stage flow with dedup, circuit breakers, retries, rate limits and a dead-letter queue in front of Anthropic, OpenAI, llama.cpp or vLLM. REST, SSE, MCP, TUI.

2 best points · 1 comments · 2 Hacker News posts · 60 GitHub stars · View on GitHub

Hacker News discussions

• 24 Simultaneous Claude Code agents on local hardware (2 points, 1 comments, by Shmungus, 2026-02-21)
• Show HN: Tokio-prompt-orchestrator – LLM pipeline orchestration in Rust (2 points, 0 comments, by Shmungus, 2026-02-24)