hyunwoongko
commited on
Commit
•
76c7fa4
1
Parent(s):
a64bb9f
Update README.md
Browse files
README.md
CHANGED
@@ -113,7 +113,7 @@ The following tables show the evaluation results with the various number of few-
|
|
113 |
| [kakaobrain/kogpt](https://huggingface.co/kakaobrain/kogpt) * | | | | | | | |
|
114 |
| [EleutherAI/gpt-neox-ko-1.3b](https://huggingface.co/EleutherAI/gpt-neox-ko-1.3b) (ours) | | 0.4867 | 0.7207 | 0.5877 | 0.5877 | 0.7407 | 0.59234 |
|
115 |
|
116 |
-
<p><strong>†</strong> The model card of this model provides evaluation results for the KOBEST dataset, but when we evaluated the model with the prompts described in the paper, we can't get similar results to it. Therefore, we checked the KOBEST paper and found that the results were similar to the fine-tuning results reported in the paper. Because we evaluated prompt-based generation without fine-tuning the model, the results provided by the model card for the this model may differ.</p>
|
117 |
|
118 |
<p><strong>*</strong> Since this model does not provide evaluation results with KOBEST dataset, we evaluated the model using lm-evaluation-harness ourselves. you can reproduce this result using the source code included in the multilingual-ko branch of lm-evaluation-harness.</p>
|
119 |
|
|
|
113 |
| [kakaobrain/kogpt](https://huggingface.co/kakaobrain/kogpt) * | | | | | | | |
|
114 |
| [EleutherAI/gpt-neox-ko-1.3b](https://huggingface.co/EleutherAI/gpt-neox-ko-1.3b) (ours) | | 0.4867 | 0.7207 | 0.5877 | 0.5877 | 0.7407 | 0.59234 |
|
115 |
|
116 |
+
<p><strong>†</strong> The model card of this model provides evaluation results for the KOBEST dataset, but when we evaluated the model with the prompts described in the paper, we can't get similar results to it. Therefore, we checked the KOBEST paper and found that the results were similar to the fine-tuning results reported in the paper. Because we evaluated by prompt-based generation without fine-tuning the model, the results provided by the model card for the this model may differ.</p>
|
117 |
|
118 |
<p><strong>*</strong> Since this model does not provide evaluation results with KOBEST dataset, we evaluated the model using lm-evaluation-harness ourselves. you can reproduce this result using the source code included in the multilingual-ko branch of lm-evaluation-harness.</p>
|
119 |
|