tspeterkim/paged-attention-minimal
a minimal cache manager for PagedAttention, on top of llama3.
3 best points · 0 comments · 2 Hacker News posts · 151 GitHub stars · View on GitHub
Hacker News discussions
- Show HN: Minimal Paged Attention (3 points, 0 comments, by tspeterkim, 2024-06-28)
- A Minimal KV Cache Manager for Paged Attention in ~100 Lines of Python (2 points, 0 comments, by tspeterkim, 2024-07-01)