janbalangue/async-bulkhead-llm

Protects LLM applications from overload by enforcing concurrency and token budgets before provider calls, rejecting excess work early to preserve responsiveness and reduce 429 errors.

1 best points · 0 comments · 1 Hacker News post · 3 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Keep batch LLM jobs from starving interactive traffic (TypeScript) (1 points, 0 comments, by janbalangue, 2026-08-12)