LLMs can Compress LLMs: Adaptive Pruning by Agents

TL;DR

An LLM acts as a coach to prune another LLM, shrinking it ~45% while preserving key knowledge and accuracy.

Traditional pruning uses fixed rules and often wipes out facts. This paper lets a foundation model adaptively choose which layers to trim each round. It reads layer sensitivity snapshots—combining weight–activation cues (à la Wanda) with gradient importance—normalized for easy comparison. The agent self-reflects on past pruning outcomes and adjusts. If quality (perplexity) drops too much, a rollback restores the last good checkpoint.

• Results on Qwen3 4B/8B at ~45% sparsity: 56% relative better MMLU accuracy vs structured pruning, 19× stronger factual retention on FreebaseQA, and 69% less perplexity degradation.
• No retraining, model-agnostic, and only 2–4 rollbacks across 21–40 iterations.

Takeaway: foundation models can intelligently compress other foundation models—cutting costs without gutting knowledge.

Paper by Sai Varun Kodathala and Rakesh Vunnam. Link: https://arxiv.org/abs/2601.09694v1

Paper: https://arxiv.org/abs/2601.09694v1

Register: https://www.AiFeta.com

AI LLM ModelCompression Pruning SparseML MMLU Qwen Factuality Efficiency NLP