b4rtaz/distributed-llama
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
347 best points · 321 comments · 11 Hacker News posts · 3,132 GitHub stars · View on GitHub
Hacker News discussions
- Qwen3 30B A3B Hits 13 token/s on 4xRaspberry Pi 5 (347 points, 161 comments, by b4rtazz, 2025-09-06)
- Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5 (306 points, 156 comments, by b4rtazz, 2025-02-15)
- Show HN: Distributed Llama – Run LLMs on multiple devices in parallel (12 points, 0 comments, by b4rtazz, 2024-01-25)
- Run Llama 3.3 70B Q40 on $1516 GPU 3.3 tok/s (6 points, 0 comments, by b4rtazz, 2025-04-26)
- Distributed Llama on Raspberry Pis (4 points, 2 comments, by politelemon, 2024-01-20)
- Distributed Llama (4 points, 0 comments, by lagniappe, 2024-01-21)
- Llama 3.1 is available for home AI clusters! Run 8B model with the full context (4 points, 0 comments, by b4rtazz, 2024-07-25)
- Distributed-Llama: Connect home devices into a cluster for LLM inference (4 points, 1 comments, by tosh, 2025-02-15)
- Distributed Llama (3 points, 0 comments, by oldfuture, 2026-02-12)
- Run Llama 3 405B on home cluster (Distributed Llama) (2 points, 1 comments, by b4rtazz, 2024-07-23)
- Distributed Grok-1 (314B) (1 points, 0 comments, by b4rtazz, 2024-04-15)