Neroued/ninfer
High-performance single-GPU inference for selected model checkpoints and GPUs.
3 best points · 0 comments · 3 Hacker News posts · 3,278 GitHub stars · View on GitHub
Hacker News discussions
- Ninfer: High-performance single-GPU inference (3 points, 0 comments, by handfuloflight, 2026-07-28)
- NInfer: 271 tok/s (543 with speculative) for Qwen3.6-35B-A3B on one RTX 5090 (2 points, 0 comments, by opwizardx, 2026-07-23)
- High-performance single-GPU inference for selected model checkpoints and GPUs (1 points, 0 comments, by davedx, 2026-08-15)