LFM2.5 Q4_0 checkpoints are released as the result of a quantization‑aware distillation process applied to the LFM2.5 family. The method trains a student model to mimic the logits of the full‑precision teacher while simultaneously learning 4‑bit (Q4_0) weights, yielding checkpoints that occupy roughly one‑quarter of the memory of the FP16 baseline without a separate post‑training quantization step.
- Model architecture: decoder‑only transformer with 2.5 B parameters, same layer count and hidden size as LFM2.5.
- Context window: 32 k tokens, unchanged from the original model.
- Quantization: uniform 4‑bit (Q4_0) weights and activations, trained via quantization‑aware distillation.
- License: released under the Apache 2.0 license, permitting commercial and research use.
- Benchmark retention: on the MMLU and GSM8K suites, the Q4_0 checkpoints show accuracy comparable to the FP16 teacher, as reported in the accompanying evaluation table.
Why this matters
The release demonstrates that quantization‑aware distillation can achieve aggressive model compression while preserving the majority of the original model’s reasoning capabilities, which is valuable for latency‑sensitive or edge deployments. Unlike post‑training quantization, the Q4_0 checkpoints avoid the accuracy drop typically seen when applying 4‑bit rounding after training, suggesting a pathway for deploying large language models under strict memory budgets without sacrificing task performance. This approach also highlights the importance of preserving the teacher’s output distribution during compression, a factor that may generalize to other model families and quantization levels.