intel/ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

297 best points · 115 comments · 3 Hacker News posts · 8,844 GitHub stars · View on GitHub

Hacker News discussions

• DeepSeek-R1-671B-Q4_K_M with 1 or 2 Arc A770 on Xeon (297 points, 112 comments, by colorant, 2025-03-06)
• IPEX-LLM Portable Zip for Ollama on Intel GPU (2 points, 2 comments, by colorant, 2025-02-14)
• FlashMoE: DeepSeek-R1 671B and Qwen3MoE 235B with 1~2 Intel B580 GPU in IPEX-LLM (2 points, 1 comments, by colorant, 2025-05-13)