William Merrill, Yanhong Li, Tyler Romero, Anej Svete, Caia Costello, Pradeep Dasigi, Dirk Groeneveld, David Heineman, Bailey Kuehl, Nathan Lambert, Jacob Morrison, Luca Soldaini, Finbarr Timbers, Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi, Ashish Sabharwal
View original ↗Create a reference framework for evaluating hybrid RNN-transformer architectures to determine their cost-to-performance ratio for long-context inference.
Suggested repo: hybrid-eval
"Benchmarks that tell you if you should ditch your Transformer for a hybrid architecture."
Estimated effort: 60h