Luce-Org/lucebox-hub
LLM speculative inference server for heterogeneous hardware & consumer GPUs
165 best points · 53 comments · 3 Hacker News posts · 2,978 GitHub stars · View on GitHub
Hacker News discussions
- We got 207 tok/s with Qwen3.5-27B on an RTX 3090 (165 points, 52 comments, by GreenGames, 2026-04-20)
- PFlash: 10x prefill speedup over llama.cpp at 128K on a RTX 3090 (3 points, 1 comments, by GreenGames, 2026-05-01)
- 256K context with 72MiB of KV cache on the GPU (3 points, 0 comments, by GreenGames, 2026-06-16)