intel-analytics/ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

308 best points · 95 comments · 1 Hacker News post · 8,844 GitHub stars · View on GitHub

Hacker News discussions

• PyTorch Library for Running LLM on Intel CPU and GPU (308 points, 95 comments, by ebalit, 2024-04-03)