Update README.md
Browse files
README.md
CHANGED
|
@@ -94,3 +94,12 @@ print(output[0]['generated_text'])
|
|
| 94 |
```
|
| 95 |
|
| 96 |
Note: If you want to use flash attention, call _AutoModelForCausalLM.from_pretrained()_ with _attn_implementation="flash_attention_2"_
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 94 |
```
|
| 95 |
|
| 96 |
Note: If you want to use flash attention, call _AutoModelForCausalLM.from_pretrained()_ with _attn_implementation="flash_attention_2"_
|
| 97 |
+
|
| 98 |
+
## Benchmarks
|
| 99 |
+
|
| 100 |
+
|Model|Size (# params)|IFEval|GSM8K|
|
| 101 |
+
|:----|:--------------|:-----|:----|
|
| 102 |
+
|rasyosef/Phi-1_5-Instruct-v0.1|1.4B|**26.71**|**41.78**|
|
| 103 |
+
|HuggingFaceTB/SmolLM-1.7B-Instruct|1.7B|24.21|3.45|
|
| 104 |
+
|TinyLlama/TinyLlama-1.1B-Chat-v1.0|1.1B|21.23|0|
|
| 105 |
+
|microsoft/phi-1_5|1.4B|20.51|31.73|
|