Jacob Dineen, Aswin RRV, Zhikun Xu, Ben Zhou
View original ↗Implement a simple vocabulary dropout wrapper for open-source self-play loops. This can be integrated into existing trainers like Axolotl or NanoGPT to prevent diversity collapse.
Suggested repo: vocab-dropout
"Prevent curriculum collapse in LLM self-play with one mask."
Estimated effort: 20h