Ivan Ternovtsii, Yurii Bilak
View original ↗Build a library that enables drop-in replacement of standard linear routers with geometric routing logic in existing MoE architectures. This would significantly reduce the parameter overhead associated with routing without sacrificing model performance.
Suggested repo: nanoRouter
"Reduce MoE routing parameters by 80% with simple geometric centroids."
Estimated effort: 40h