Michael-A-Kuykendall/shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

14 best points · 16 comments · 3 Hacker News posts · 6,012 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Shimmy – 5MB privacy-first, local alternative to Ollama (680MB) (14 points, 13 comments, by MKuykendall, 2025-09-04)
• Shimmy v1.7.0: Running 42B Moe Models on Consumer GPUs with 99.9% VRAM Reduction (3 points, 1 comments, by MKuykendall, 2025-10-09)
• Show HN: 5MB Rust binary that runs HuggingFace models (no Python) (2 points, 2 comments, by MKuykendall, 2025-09-10)