Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
Abstract
Antislop combines pattern profiling, a sampler with backtracking, and Final Token Preference Optimization. These components target overused phrases while studying the trade-off between suppressing repetition and retaining useful generation capabilities.
Type
Publication
International Conference on Learning Representations
Ravid Shwartz-Ziv
AI Researcher
AI researcher at Meta MSL with a background in information theory and computational neuroscience, working on world models, memory, and compression. Former Assistant Professor and Faculty Fellow at NYU, collaborating on research across academia and industry.