mit-han-lab/duo-attention

[ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads

2 best points · 0 comments · 1 Hacker News post · 545 GitHub stars · View on GitHub

Hacker News discussions

• DuoAttention-Slashes memory and latency for LLMs without sacrificing performance (2 points, 0 comments, by dsr12, 2024-10-15)