---
library_name: transformers
license: apache-2.0
language:
- en
---

Smallstral, layer pruned Mistral
| Task                                         | Metric   | Mistral-7B-v0.1 Value | Smallstral Value |
|----------------------------------------------|----------|----------------------:|-----------------:|
| hendrycksTestRu-high_school_mathematics      | acc      | 0.3037                | 0.2852           |
| hendrycksTestRu-high_school_microeconomics   | acc      | 0.3950                | 0.3319           |
| hendrycksTestRu-high_school_physics          | acc      | 0.3113                | 0.2649           |
| hendrycksTestRu-high_school_psychology       | acc      | 0.5193                | 0.1945           |
| hendrycksTestRu-high_school_statistics       | acc      | 0.3241                | 0.4028           |
| hendrycksTestRu-high_school_us_history       | acc      | 0.5392                | 0.2451           |
| hendrycksTestRu-high_school_world_history    | acc      | 0.5781                | 0.2321           |
| hendrycksTestRu-human_aging                  | acc      | 0.4843                | 0.3184           |
| parus                                        | acc      | 0.5000                | 0.4875           |
| rcb                                          | acc      | 0.3653                | 0.3288           |
| rcb                                          | f1_macro | 0.3313                | 0.1691           |
| rwsd                                         | acc      | 0.4917                | 0.4901           |