Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
Abstract
Antislop combines pattern profiling, a sampler with backtracking, and Final Token Preference Optimization. These components target overused phrases while studying the trade-off between suppressing repetition and retaining useful generation capabilities.
Type
Publication
International Conference on Learning Representations
Ravid Shwartz Ziv
AI Researcher
AI researcher at Meta MSL with a background in information theory and computational neuroscience, working on world models, memory, and compression. Former Assistant Professor and Faculty Fellow at NYU. My industry experience spans Wand AI, Intel, Google AI, and Wikipedia.