OpenInstruct Mistral-7B
1st among commercially-usable 7B models on the Open LLM Leaderboard!*
This is mistralai/Mistral-7B-v0.1 finetuned on VMware/open-instruct.
Quantized to FP16 and released under the Apache-2.0 license by myself.
Compute generously provided by Higgsfield AI.
Prompt format: Alpaca
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
[your instruction goes here]
### Response:
Recommended preset:
- temperature: 0.2
- top_k: 50
- top_p 0.95
- repetition_penalty: 1.1
*as of 21 Nov 2023. "commercially-usable" includes both an open-source base model and a non-synthetic open-source finetune dataset. updated leaderboard results available here.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 63.64 |
| AI2 Reasoning Challenge (25-Shot) | 59.73 |
| HellaSwag (10-Shot) | 82.77 |
| MMLU (5-Shot) | 60.55 |
| TruthfulQA (0-shot) | 48.76 |
| Winogrande (5-shot) | 79.56 |
| GSM8k (5-shot) | 50.49 |
- Downloads last month
- 683
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for monology/openinstruct-mistral-7b
Dataset used to train monology/openinstruct-mistral-7b
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard59.730
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard82.770
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard60.550
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard48.760
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard79.560
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard50.490