Abstract
DRY adjusts sampling logits when a candidate token would extend a repeated sequence in the context. Sequence boundaries protect formatting, allowing the method to target verbatim loops without treating every recurring token as unwanted.
Type
Publication
arXiv preprint
Ravid Shwartz Ziv
AI Researcher
AI researcher at Meta MSL with a background in information theory and computational neuroscience, working on world models, memory, and compression. Former Assistant Professor and Faculty Fellow at NYU. My industry experience spans Wand AI, Intel, Google AI, and Wikipedia.