guidance-ai/llgtrt

TensorRT-LLM server with Structured Outputs (JSON) built with Rust

6 best points · 0 comments · 1 Hacker News post · 73 GitHub stars · View on GitHub

Hacker News discussions

• Show HN: LLGTRT: TensorRT-LLM+Rust server w/ OpenAI-compat and Structured Output (6 points, 0 comments, by mmoskal, 2024-10-30)