Helldez/BigMoeOnEdge

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

1 best points · 0 comments · 2 Hacker News posts · 723 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Run a 120B-parameter MoE on Android mid-range phone CPU-only llama.cpp (1 points, 0 comments, by Helldez, 2026-07-19)
• Show HN: A layer on top llama.cpp enabling big MoE streaming on phone and laptop (1 points, 0 comments, by HelldezNoBan, 2026-10-20)