SJTU-IPADS/PowerInfer
High-speed Large Language Model Serving for Local Deployment
417 best points · 84 comments · 2 Hacker News posts · 9,871 GitHub stars · View on GitHub
Hacker News discussions
- High-Speed Large Language Model Serving on PCs with Consumer-Grade GPUs (417 points, 83 comments, by dataminer, 2023-12-20)
- PowerInfer: High-Speed Large Language Model Serving on Consumer-Grade GPUs (4 points, 1 comments, by limoce, 2023-12-19)