Avyav Kumar Singh, Yen-Chen Wu, Alexandru Cioba, Alberto Bernacchia, Davide Buffelli
View original ↗Implement a Byte-Level Distillation (BLD) library that allows knowledge transfer between LLMs with incompatible tokenizers. This eliminates the need for complex vocabulary mapping.
Suggested repo: ByteBridge
"Distill any LLM, regardless of tokenization differences."
Estimated effort: 90h