kvcache-ai/ktransformers
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
20 best points · 3 comments · 2 Hacker News posts · 19,660 GitHub stars · View on GitHub
Hacker News discussions
- Show HN: KTransformers–236B Model and 1M Context LLM Inference on Local Machines (20 points, 3 comments, by sssummer, 2024-08-29)
- Show HN: KTransformers:671B DeepSeek-R1 on a Single Machine-286 tokens/s Prefill (14 points, 0 comments, by sssummer, 2025-02-10)