|
--- |
|
license: gemma |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
tags: |
|
- conversational |
|
base_model: |
|
- IlyaGusev/gemma-2-2b-it-abliterated |
|
--- |
|
|
|
|
|
# Gemma-2-2b-it-abl-rudpo |
|
|
|
## Model Information |
|
|
|
Improved quality of Russian language compared to the base model. Tested on [arena-hard questions in Russian](https://github.com/r4dm/arena-hard-local) |
|
``` |
|
gemma-2-9b-it-SimPO-rudpo | score: 91.9 | 95% CI: (-0.9, 0.9) | average #tokens: 1013 |
|
gemma-2-9b-it-SimPO | score: 90.9 | 95% CI: (-0.9, 1.1) | average #tokens: 1065 |
|
gemma-2-27b-it-FP8 | score: 82.0 | 95% CI: (-1.3, 1.5) | average #tokens: 799 |
|
gemma-2-9b-it | score: 67.0 | 95% CI: (-1.8, 1.7) | average #tokens: 760 |
|
gemma-2-2b-it-abl-rudpo | score: 61.6 | 95% CI: (-1.7, 2.2) | average #tokens: 1121 <---- THIS MODEL |
|
gemma-2-2b-it-abl | score: 48.8 | 95% CI: (-1.9, 2.0) | average #tokens: 783 |
|
gemma-2b-it | score: 8.8 | 95% CI: (-1.1, 1.0) | average #tokens: 425 |
|
``` |
