Llama-3-2-3B-Instruct-IQ4_NL

Accuracy vs. Size Tradeoff

Every variant is benchmarked against the FP16 baseline. Choose the right balance of accuracy and footprint for your workflow.

Usage Code Snippets

Which variant for your hardware?

Choose the right quantization path for your device and performance target.

About This Model

Llama-3.2-3B-Instruct-IQ4_NL.gguf is an IQ4_NL quantized GGUF version of Meta's Llama 3.2 3B Instruct model. The IQ4_NL quantization utilizes advanced importance-aware quantization techniques to preserve critical model weights more effectively than traditional Q4 methods, resulting in improved reasoning, coherence, and instruction-following performance. Built on the Llama 3.2 decoder-only transformer architecture, this model delivers an excellent balance between quality, memory usage, and inference speed. It is well suited for conversational AI, content generation, summarization, coding assistance, and local AI applications. Compatible with llama.cpp-based runtimes such as Ollama, LM Studio, Jan, GPT4All, and Open WebUI, IQ4_NL is widely regarded as one of the best quality-to-size quantization formats available for local inference.