SRKRZ23/vramkit

VRAM compression for LLM inference, measured on a MacBook: lossless weight codec (-33% vs 4-bit on real Qwen weights, beats zstd) + drop-in PyTorch KV-cache compression. Key finding: quantize KV keys per-CHANNEL (not per-token) -> int8 KV near-lossless, +0.1% real perplexity at 2x. ROCm/CUDA/MPS/CPU.

1 best points · 0 comments · 2 Hacker News posts · 5 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Vramkit – lossless VRAM compression for LLM inference, on a MacBook (1 points, 0 comments, by sardor_r1, 2026-10-17)
• Show HN: Vramkit – measuring LLM weight and KV-cache compression on a MacBook (1 points, 0 comments, by sardor_r1, 2026-10-20)