SqueezeAILab/KVQuant

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

2 best points · 1 comments · 1 Hacker News post · 436 GitHub stars · View on GitHub

Hacker News discussions

• 10M Tokens LLM Context (2 points, 1 comments, by nsky-world, 2024-02-02)