Phi-4-mini-instruct-Q4_K_M-GGUF-4Bit

Accuracy vs. Size Tradeoff

Every variant is benchmarked against the FP16 baseline. Choose the right balance of accuracy and footprint for your workflow.

Usage Code Snippets

Which variant for your hardware?

Choose the right quantization path for your device and performance target.

About This Model

Phi-4-mini-instruct-Q4_K_M.gguf is a 4-bit quantized GGUF release of Microsoft's Phi-4 Mini Instruct language model. Built on the Phi architecture, the model contains approximately 3.8 billion parameters and supports up to 128K context length. Despite its compact size, it delivers strong performance in reasoning, mathematics, coding, and multilingual instruction-following tasks. The Q4_K_M quantization format provides an excellent balance between model quality, memory usage, and inference speed, making it suitable for deployment on consumer GPUs, laptops, and edge devices through runtimes such as llama.cpp, Ollama, LM Studio, and Jan. The model is trained on high-quality synthetic and curated web datasets and is designed for chat assistants, RAG systems, code generation, and general-purpose AI applications.