turboderp/exllama

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

3 best points · 0 comments · 1 Hacker News post · 2,946 GitHub stars · View on GitHub

Hacker News discussions

• ExLlama: Memory efficient way to run Llama (3 points, 0 comments, by Palmik, 2023-08-16)