Classevelabs/rai

CPU-only LLM inference engine in pure Rust — 4-bit quantized models, hand-written AVX2 kernels, speculative decoding, and a local HTTP/MCP server. No GPU, no Python runtime.

5 best points · 1 comments · 1 Hacker News post · 16 GitHub stars · View on GitHub

Hacker News discussions

• Rai: CPU-only LLM inference engine in pure Rust (5 points, 1 comments, by simonpure, 2026-10-02)