Vahe1994/AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

10 best points · 0 comments · 1 Hacker News post · 1,331 GitHub stars · View on GitHub

Hacker News discussions

• AQLM and PV-Tuning: methods that compress LLMs by 8 times, retain 95% quality (10 points, 0 comments, by annaerma, 2024-07-23)