End of training
Browse files- README.md +7 -10
- adapter_model.bin +1 -1
README.md
CHANGED
@@ -4,9 +4,6 @@ library_name: peft
|
|
4 |
tags:
|
5 |
- axolotl
|
6 |
- generated_from_trainer
|
7 |
-
- unsloth
|
8 |
-
- trl
|
9 |
-
- sft
|
10 |
base_model: deepseek-ai/deepseek-coder-6.7b-instruct
|
11 |
model-index:
|
12 |
- name: diff-deepseek-code-ir
|
@@ -40,7 +37,7 @@ test_datasets:
|
|
40 |
split: test
|
41 |
type: oasst
|
42 |
|
43 |
-
output_dir: ./outputs/dscoder-code-ir-
|
44 |
|
45 |
sequence_len: 4096
|
46 |
sample_packing: true
|
@@ -110,7 +107,7 @@ special_tokens:
|
|
110 |
|
111 |
This model is a fine-tuned version of [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) on the None dataset.
|
112 |
It achieves the following results on the evaluation set:
|
113 |
-
- Loss: 0.
|
114 |
|
115 |
## Model description
|
116 |
|
@@ -147,10 +144,10 @@ The following hyperparameters were used during training:
|
|
147 |
|
148 |
| Training Loss | Epoch | Step | Validation Loss |
|
149 |
|:-------------:|:-----:|:----:|:---------------:|
|
150 |
-
| 0.
|
151 |
-
| 0.
|
152 |
-
| 0.
|
153 |
-
| 0.
|
154 |
|
155 |
|
156 |
### Framework versions
|
@@ -158,5 +155,5 @@ The following hyperparameters were used during training:
|
|
158 |
- PEFT 0.10.0
|
159 |
- Transformers 4.40.0.dev0
|
160 |
- Pytorch 2.3.0+cu121
|
161 |
-
- Datasets 2.
|
162 |
- Tokenizers 0.15.0
|
|
|
4 |
tags:
|
5 |
- axolotl
|
6 |
- generated_from_trainer
|
|
|
|
|
|
|
7 |
base_model: deepseek-ai/deepseek-coder-6.7b-instruct
|
8 |
model-index:
|
9 |
- name: diff-deepseek-code-ir
|
|
|
37 |
split: test
|
38 |
type: oasst
|
39 |
|
40 |
+
output_dir: ./outputs/dscoder-code-ir-4
|
41 |
|
42 |
sequence_len: 4096
|
43 |
sample_packing: true
|
|
|
107 |
|
108 |
This model is a fine-tuned version of [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) on the None dataset.
|
109 |
It achieves the following results on the evaluation set:
|
110 |
+
- Loss: 0.2549
|
111 |
|
112 |
## Model description
|
113 |
|
|
|
144 |
|
145 |
| Training Loss | Epoch | Step | Validation Loss |
|
146 |
|:-------------:|:-----:|:----:|:---------------:|
|
147 |
+
| 0.5233 | 0.02 | 1 | 0.5554 |
|
148 |
+
| 0.3976 | 0.25 | 13 | 0.3534 |
|
149 |
+
| 0.3354 | 0.51 | 26 | 0.2805 |
|
150 |
+
| 0.2759 | 0.76 | 39 | 0.2549 |
|
151 |
|
152 |
|
153 |
### Framework versions
|
|
|
155 |
- PEFT 0.10.0
|
156 |
- Transformers 4.40.0.dev0
|
157 |
- Pytorch 2.3.0+cu121
|
158 |
+
- Datasets 2.20.0
|
159 |
- Tokenizers 0.15.0
|
adapter_model.bin
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 848460690
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:df02e68614751317dbc9dadbdc89b6f78bff3b81bdd9c242a2cd91fc0a00dd03
|
3 |
size 848460690
|