Implement a drop-in replacement for standard attention mechanisms that reduces memory footprint. This would be a high-impact utility for developers trying to run larger models on consumer hardware.
Suggested repo: seqattention
"Shrink your model's memory usage without dropping a single percentage point of accuracy."
Estimated effort: 40h