Sensitivity-Positional Co-Localization in GQA Transformers

Build an implementation for targeted LoRA adapter placement based on layer sensitivity metrics. This will allow for training smaller, more efficient adapters that outperform traditional LoRA implementations.

Suggested repo: sensiLoRA

"Don't train the whole layer: target your fine-tuning to the most sensitive weights."

Estimated effort: 50h