Mohamed Ehab (Faculty of Computer Science, October University for Modern Science & Arts, Giza, Egypt), Ali Hamdi (Faculty of Computer Science, October University for Modern Science & Arts, Giza, Egypt), Khaled Shaban (Department of Computer Science and Engineering, Qatar University, Doha, Qatar)
View original ↗Create an ensemble wrapper for LLM evaluation that specifically corrects for minority class bias. This tool would be invaluable for developers testing safety-critical classification tasks with sparse positive samples.
Suggested repo: camo-ensemble
"Eliminate the minority class penalty in your LLM benchmarks."
Estimated effort: 40h