tensorchord/openmodelz
Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
3 best points · 0 comments · 3 Hacker News posts · 282 GitHub stars · View on GitHub
Hacker News discussions
- Show HN: One-click machine learning deployment at scale on any cluster (3 points, 0 comments, by gaocegege, 2024-01-11)
- One-click machine learning deployment (e.g. LLM) at scale on any cluster (2 points, 0 comments, by loa_observer, 2023-08-07)
- Show HN: OpenModelZ – Autoscale vLLM/SGLang... on Kubernetes (and other clouds) (1 points, 0 comments, by gaocegege, 2024-12-28)