mixedbread-ai/batched

The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching of inference workloads.

4 best points · 0 comments · 1 Hacker News post · 161 GitHub stars · View on GitHub

Hacker News discussions

• Batched (Dynamic Batching) (4 points, 0 comments, by Jlipp, 2024-08-28)