RecurrentGemma - add the rest of the models!

#800
by devingulliver - opened

Kudos on the new leaderboard!
I'm glad you included recurrentgemma-2b as well as its instruct counterpart.
Do you have plans to include the 9b release as well?
For reference, the model IDs are google/recurrentgemma-9b and google/recurrentgemma-9b-it.
I've tried to submit them myself but they were not automatically accepted since they are gated models.
Thank you!

Open LLM Leaderboard org

Thanks for the note!
I'll need to see why you can't submit them, and if needed we'll clearly run them manually, it's an overlook that they are not here!

Open LLM Leaderboard org

Tagging @alozowski , who will take over the backend next week and will probably run these 2 manually!
Feel free to ping us end of next week for an update :)

Open LLM Leaderboard org

Hi @devingulliver ,

Thanks for your suggestion to add recurrentgemma models! Now you can find google/recurrentgemma-9b on the leaderboard – checkout the screenshot.
I've started evaluating google/recurrentgemma-9b-it, so soon we'll have results for that model as well, I'll message here
Screenshot 2024-07-05 at 12.09.27.png

Open LLM Leaderboard org

Hi @devingulliver ,

Finally, we have both recurrentgemma-9b and recurrentgemma-2b, as well as their chat models, checkout the screenshot

Screenshot 2024-07-08 at 10.26.49.png

alozowski changed discussion status to closed

Sign up or log in to comment