omgitsbase/llmash

Ollama-compatible LLM server on a tuned llama.cpp fork. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and beats ollama by much 2-4x. Made possible with custom CUDA kernels, MTP, speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. Optimized to not eat up ram.

4 best points · 7 comments · 3 Hacker News posts · 37 GitHub stars · View on GitHub

Hacker News discussions

• Ollama replacement 2-4x faster for no extra compute cost (4 points, 4 comments, by wowitsbase, 2026-09-07)
• Show HN: Ollama stand in replacement, 2-4x faster, not just basic optimizations (3 points, 0 comments, by wowitsbase, 2026-09-08)
• I promise this Ollama replacement will give you the fastest t/s you've ever had (2 points, 3 comments, by wowitsbase, 2026-09-09)