Update README.md
Browse files
README.md
CHANGED
@@ -16,6 +16,16 @@ This model is a Mixture of Experts (MoE) made with [mergekit](https://github.com
|
|
16 |
|
17 |
## 🏆 Evaluation
|
18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
| Model |AGIEval|GPT4All|TruthfulQA|Bigbench|Average|
|
20 |
|--------------------------------------------------------------------|------:|------:|---------:|-------:|------:|
|
21 |
|[**Beyonder-4x7B-v2**](https://huggingface.co/shadowml/Beyonder-4x7B-v2)| **45.29**| **75.95**| <u>**60.86**</u>| **46.4**| **57.13**|
|
|
|
16 |
|
17 |
## 🏆 Evaluation
|
18 |
|
19 |
+
Beyonder-4x7B-v2 is competitive with Mixtral-8x7B-Instruct-v0.1 on the Open LLM Leaderboard, while only having 4 experts instead of 8.
|
20 |
+
|
21 |
+
![](https://i.imgur.com/5raBff0.png)
|
22 |
+
|
23 |
+
It also displays a significant improvement over the individual experts.
|
24 |
+
|
25 |
+
![](https://i.imgur.com/7Idwkb0.png)
|
26 |
+
|
27 |
+
It also performs very well compared to other models on Nous benchmark suite. It's almost as good as the best Yi-34B fine-tune, which is a much bigger model: 24.2B parameters + only two experts are selected during inference (so ~12B) vs. 34B param.
|
28 |
+
|
29 |
| Model |AGIEval|GPT4All|TruthfulQA|Bigbench|Average|
|
30 |
|--------------------------------------------------------------------|------:|------:|---------:|-------:|------:|
|
31 |
|[**Beyonder-4x7B-v2**](https://huggingface.co/shadowml/Beyonder-4x7B-v2)| **45.29**| **75.95**| <u>**60.86**</u>| **46.4**| **57.13**|
|