End of training
Browse files- README.md +20 -1
- adapter_model.bin +1 -1
README.md
CHANGED
@@ -2,6 +2,7 @@
|
|
2 |
license: apache-2.0
|
3 |
library_name: peft
|
4 |
tags:
|
|
|
5 |
- generated_from_trainer
|
6 |
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
|
7 |
model-index:
|
@@ -110,7 +111,9 @@ fsdp_config:
|
|
110 |
|
111 |
# mixtral-lora-less-modules
|
112 |
|
113 |
-
This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on
|
|
|
|
|
114 |
|
115 |
## Model description
|
116 |
|
@@ -143,6 +146,22 @@ The following hyperparameters were used during training:
|
|
143 |
- lr_scheduler_warmup_steps: 10
|
144 |
- num_epochs: 4
|
145 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
146 |
### Framework versions
|
147 |
|
148 |
- PEFT 0.8.2
|
|
|
2 |
license: apache-2.0
|
3 |
library_name: peft
|
4 |
tags:
|
5 |
+
- axolotl
|
6 |
- generated_from_trainer
|
7 |
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
|
8 |
model-index:
|
|
|
111 |
|
112 |
# mixtral-lora-less-modules
|
113 |
|
114 |
+
This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on the None dataset.
|
115 |
+
It achieves the following results on the evaluation set:
|
116 |
+
- Loss: 0.1904
|
117 |
|
118 |
## Model description
|
119 |
|
|
|
146 |
- lr_scheduler_warmup_steps: 10
|
147 |
- num_epochs: 4
|
148 |
|
149 |
+
### Training results
|
150 |
+
|
151 |
+
| Training Loss | Epoch | Step | Validation Loss |
|
152 |
+
|:-------------:|:-----:|:----:|:---------------:|
|
153 |
+
| 3.2966 | 0.0 | 1 | 3.2222 |
|
154 |
+
| 0.1826 | 0.4 | 123 | 0.2012 |
|
155 |
+
| 0.1729 | 0.8 | 246 | 0.1925 |
|
156 |
+
| 0.1852 | 1.19 | 369 | 0.1913 |
|
157 |
+
| 0.1463 | 1.59 | 492 | 0.1857 |
|
158 |
+
| 0.1246 | 1.99 | 615 | 0.1840 |
|
159 |
+
| 0.1149 | 2.37 | 738 | 0.1872 |
|
160 |
+
| 0.076 | 2.77 | 861 | 0.1837 |
|
161 |
+
| 0.0763 | 3.15 | 984 | 0.1920 |
|
162 |
+
| 0.0906 | 3.56 | 1107 | 0.1904 |
|
163 |
+
|
164 |
+
|
165 |
### Framework versions
|
166 |
|
167 |
- PEFT 0.8.2
|
adapter_model.bin
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 218196746
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:8b736f258bbb6518c5c6320e3be740ea54cf01ad2bc94e6b90be1ee7c5199315
|
3 |
size 218196746
|