Abstract
DRY adjusts sampling logits when a candidate token would extend a repeated sequence in the context. Sequence boundaries protect formatting, allowing the method to target verbatim loops without treating every recurring token as unwanted.
Type
Publication
arXiv preprint
Ravid Shwartz-Ziv
AI Researcher
AI researcher at Meta MSL with a background in information theory and computational neuroscience, working on world models, memory, and compression. Former Assistant Professor and Faculty Fellow at NYU, collaborating on research across academia and industry.