facebookresearch/fastgen

Simple high-throughput inference library

3 best points · 1 comments · 1 Hacker News post · 160 GitHub stars · View on GitHub

Hacker News discussions

• Fastgen – SOTA LLM inference in 3k lines of Python (3 points, 1 comments, by mpu, 2025-05-16)