LFM2.5 Q4\_0 Checkpoints from Quantization-Aware Distillation

Liquid AI has officially launched its new QAD Q4_0 GGUF checkpoints, marking a significant advancement for developers looking to deploy the LFM2.5 model family on resource-constrained edge hardware. By leveraging Quantization-Aware Distillation (QAD), these updated 4-bit checkpoints bridge the gap between high-precision performance and the efficiency required for local execution.

Bridging the Precision Gap

The core innovation behind this release is the use of Quantization-Aware Distillation. Unlike traditional post-training quantization (PTQ) methods that often result in a noticeable degradation of model intelligence, QAD trains a quantized student model by distilling knowledge from a high-precision teacher.

"QAD checkpoints recover approximately 97% of the BF16 average accuracy typically lost during the quantization process."

This approach ensures that developers can maintain the low memory footprint and high throughput associated with standard Q4_0 GGUFs without sacrificing the reasoning or instruction-following capabilities of the original models.

Benchmark Performance

Liquid AI evaluated the new checkpoints across a comprehensive suite of benchmarks, including GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. The results demonstrate that the QAD-trained models consistently outperform their PTQ counterparts. Across the four models—LFM2.5-230M, 350M, 1.2B-Instruct, and 2.6B—the QAD checkpoints retained between 96.5% and 97.4% of their BF16 baseline performance.

Edge Hardware Optimization

To validate real-world utility, the team tested decode throughput across diverse hardware targets:

• GPU Inference: MacBook Pro and NucBox EVO-X2.
• Arm CPU Inference: Samsung Galaxy S26 Ultra and Raspberry Pi 5.

The findings highlight impressive efficiency gains:

• The 230M and 350M models match Q5_K_M quality while delivering a 4–33% boost in decode throughput.
• The 1.2B and 2.6B models achieve Q4_K_M quality parity with a 3–14% increase in throughput.

Getting Started

These GGUF artifacts are fully compatible with llama.cpp and any runtime supporting the Q4_0 standard. Developers can integrate them immediately using standard CLI commands:

llama-cli -hf LiquidAI/LFM2.5-350M \
  --hf-file LFM2.5-350M-QAD-Q4_0.gguf \
  -p "What is C. elegans?"

The full suite of models is available now on Hugging Face. For those integrating these models into research or production environments, please refer to the official Liquid AI blog for the complete citation and BibTeX documentation. As the edge AI landscape continues to evolve, these QAD checkpoints provide a robust foundation for building high-performance, local-first applications.