Kmic-68/llama.cpp

llama.cpp tuned for 2x Tesla P100 (sm_60). Decode 1.86x, 3-4x with MTP (up to ~70 t/s on code). Prefill 2x, full 262k context with vision. More numerically accurate than upstream.

3 best points · 2 comments · 1 Hacker News post · 41 GitHub stars · View on GitHub

Hacker News discussions

• 2x Tesla p100s, q6_k quant, Qwen 3.8 27B ~60tps (3 points, 2 comments, by Kmic68, 2026-10-18)