flouthoc/sors

Minimal proxy which reorders prompts for LLM to maximize prefix cache hit

2 best points · 0 comments · 1 Hacker News post · 2 GitHub stars · View on GitHub

Hacker News discussions

• Sors: a Rust proxy that reorders prompts to maximize vLLM prefix cache hits (2 points, 0 comments, by flaccount, 2026-06-16)