HanGuo97/flute

Fast Matrix Multiplications for Lookup Table-Quantized LLMs

2 best points · 1 comments · 1 Hacker News post · 395 GitHub stars · View on GitHub

Hacker News discussions

• A new CUDA kernel for quantized LLMs achieves up to 2.6x latency improvements (2 points, 1 comments, by radichoml, 2024-07-22)