Researchers pinpoint why larger language models pick up skills that small ones miss

Source Brief

Researchers pinpoint why larger language models pick up skills that small ones miss: small models are overwritten by frequent tasks and cannot retain rare ones. The study shows model size is crucial for learning rare patterns.