kekzl/imp

Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one docker command.

1 best points · 0 comments · 1 Hacker News post · 75 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: Imp, an LLM Server for RTX 50 GPUs, Written by Claude Code (1 points, 0 comments, by kekz, 2026-10-27)