LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning

Implement LiME as a plugin for common libraries like PEFT/HuggingFace. This would allow developers to run multi-task models with significant memory savings by replacing full adapter stacks.

Suggested repo: LiME-LoRA

"Mix experts without the massive parameter tax."

Estimated effort: 30h