predibase/lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

5 best points · 1 comments · 3 Hacker News posts · 3,835 GitHub stars · View on GitHub

Hacker News discussions

• Lorax: Serve 100s of Fine-Tuned LLMs in Production for the Cost of 1 (5 points, 1 comments, by icyfox, 2023-11-16)
• LoRAX: Hot swap LoRA adapters to serve many finetuned models concurrently (3 points, 0 comments, by magdyks, 2024-02-01)
• LoRAX: Serve 100s of Fine-tuned LLMs for the Cost of 1 (2 points, 0 comments, by CharlesW, 2023-11-28)