Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models

Computer Science

• Language Modeling 100%
• Knowledge Retention 100%
• Sparsity 60%
• Feature Space 20%
• Reconstruction Error 20%
• Large Language Model 20%
• Embedding Space 20%
• Adversarial Machine Learning 20%
• Bidirectional Encoder Representations From Transformers 20%

Biochemistry, Genetics and Molecular Biology

• Reconstruction 100%

Keyphrases

• Pruning Costs 20%
• Beyond-accuracy 20%
• Dense Language 20%
• Soluble Suppression of Tumorigenicity 2 (sST2) 20%