xaskasdf/ntransformer

High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.

395 best points · 101 comments · 1 Hacker News post · 469 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU (395 points, 101 comments, by xaskasdf, 2026-02-21)