dipampaul17/KVSplit

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.

272 best points · 40 comments · 1 Hacker News post · 362 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: KVSplit – Run 2-3x longer contexts on Apple Silicon (272 points, 40 comments, by dipampaul17, 2025-05-16)