jmaczan/tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

205 best points · 18 comments · 2 Hacker News posts · 1,182 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA (205 points, 18 comments, by yu3zhou4, 2026-05-29)
• AMD contributes their GPU support to tiny-vLLM (2 points, 0 comments, by yu3zhou4, 2026-06-29)