Hugging Face's logo

newbie-geek
/
tinyllama-v1-finetune

PEFT
Safetensors
Generated from Trainer
Model card Files Community
newbie-geek commited on
Commit
38121cb
•
1 Parent(s): e33f40e

End of training

Browse files
Files changed (1) hide show
  1. README.md +141 -0
README.md ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: peft
4
+ tags:
5
+ - generated_from_trainer
6
+ base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
7
+ model-index:
8
+ - name: tinyllama-v1-finetune
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # tinyllama-v1-finetune
16
+
17
+ This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) on the None dataset.
18
+ It achieves the following results on the evaluation set:
19
+ - Loss: 0.5205
20
+
21
+ ## Model description
22
+
23
+ More information needed
24
+
25
+ ## Intended uses & limitations
26
+
27
+ More information needed
28
+
29
+ ## Training and evaluation data
30
+
31
+ More information needed
32
+
33
+ ## Training procedure
34
+
35
+ ### Training hyperparameters
36
+
37
+ The following hyperparameters were used during training:
38
+ - learning_rate: 2.5e-05
39
+ - train_batch_size: 2
40
+ - eval_batch_size: 8
41
+ - seed: 42
42
+ - gradient_accumulation_steps: 4
43
+ - total_train_batch_size: 8
44
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
+ - lr_scheduler_type: linear
46
+ - training_steps: 400
47
+ - mixed_precision_training: Native AMP
48
+
49
+ ### Training results
50
+
51
+ | Training Loss | Epoch | Step | Validation Loss |
52
+ |:-------------:|:-----:|:----:|:---------------:|
53
+ | 2.7664 | 0.04 | 5 | 2.7807 |
54
+ | 2.7326 | 0.08 | 10 | 2.7249 |
55
+ | 2.6955 | 0.12 | 15 | 2.6739 |
56
+ | 2.643 | 0.16 | 20 | 2.6085 |
57
+ | 2.6056 | 0.2 | 25 | 2.5554 |
58
+ | 2.5018 | 0.24 | 30 | 2.4923 |
59
+ | 2.4587 | 0.28 | 35 | 2.4204 |
60
+ | 2.3997 | 0.32 | 40 | 2.3484 |
61
+ | 2.3142 | 0.36 | 45 | 2.2673 |
62
+ | 2.2507 | 0.4 | 50 | 2.1903 |
63
+ | 2.1405 | 0.43 | 55 | 2.1071 |
64
+ | 2.0471 | 0.47 | 60 | 2.0212 |
65
+ | 1.9957 | 0.51 | 65 | 1.9385 |
66
+ | 1.9007 | 0.55 | 70 | 1.8448 |
67
+ | 1.8046 | 0.59 | 75 | 1.7501 |
68
+ | 1.691 | 0.63 | 80 | 1.6495 |
69
+ | 1.5957 | 0.67 | 85 | 1.5484 |
70
+ | 1.53 | 0.71 | 90 | 1.4608 |
71
+ | 1.4161 | 0.75 | 95 | 1.3669 |
72
+ | 1.3409 | 0.79 | 100 | 1.2772 |
73
+ | 1.239 | 0.83 | 105 | 1.1833 |
74
+ | 1.1591 | 0.87 | 110 | 1.1122 |
75
+ | 1.0729 | 0.91 | 115 | 1.0276 |
76
+ | 1.0237 | 0.95 | 120 | 0.9615 |
77
+ | 0.9293 | 0.99 | 125 | 0.9021 |
78
+ | 0.891 | 1.03 | 130 | 0.8531 |
79
+ | 0.8365 | 1.07 | 135 | 0.8155 |
80
+ | 0.7876 | 1.11 | 140 | 0.7898 |
81
+ | 0.7821 | 1.15 | 145 | 0.7669 |
82
+ | 0.7392 | 1.19 | 150 | 0.7516 |
83
+ | 0.77 | 1.23 | 155 | 0.7397 |
84
+ | 0.7088 | 1.26 | 160 | 0.7226 |
85
+ | 0.7246 | 1.3 | 165 | 0.7101 |
86
+ | 0.7007 | 1.34 | 170 | 0.6960 |
87
+ | 0.6667 | 1.38 | 175 | 0.6797 |
88
+ | 0.6898 | 1.42 | 180 | 0.6666 |
89
+ | 0.6608 | 1.46 | 185 | 0.6599 |
90
+ | 0.6526 | 1.5 | 190 | 0.6451 |
91
+ | 0.6078 | 1.54 | 195 | 0.6350 |
92
+ | 0.6336 | 1.58 | 200 | 0.6248 |
93
+ | 0.6074 | 1.62 | 205 | 0.6167 |
94
+ | 0.6114 | 1.66 | 210 | 0.6131 |
95
+ | 0.575 | 1.7 | 215 | 0.6041 |
96
+ | 0.5933 | 1.74 | 220 | 0.5981 |
97
+ | 0.5983 | 1.78 | 225 | 0.5910 |
98
+ | 0.5907 | 1.82 | 230 | 0.5845 |
99
+ | 0.5853 | 1.86 | 235 | 0.5801 |
100
+ | 0.5881 | 1.9 | 240 | 0.5749 |
101
+ | 0.5613 | 1.94 | 245 | 0.5700 |
102
+ | 0.5852 | 1.98 | 250 | 0.5696 |
103
+ | 0.5781 | 2.02 | 255 | 0.5652 |
104
+ | 0.5812 | 2.06 | 260 | 0.5609 |
105
+ | 0.5677 | 2.09 | 265 | 0.5576 |
106
+ | 0.5544 | 2.13 | 270 | 0.5541 |
107
+ | 0.536 | 2.17 | 275 | 0.5504 |
108
+ | 0.5283 | 2.21 | 280 | 0.5487 |
109
+ | 0.5326 | 2.25 | 285 | 0.5454 |
110
+ | 0.568 | 2.29 | 290 | 0.5402 |
111
+ | 0.5448 | 2.33 | 295 | 0.5395 |
112
+ | 0.5581 | 2.37 | 300 | 0.5377 |
113
+ | 0.5406 | 2.41 | 305 | 0.5355 |
114
+ | 0.4996 | 2.45 | 310 | 0.5333 |
115
+ | 0.5243 | 2.49 | 315 | 0.5346 |
116
+ | 0.5591 | 2.53 | 320 | 0.5312 |
117
+ | 0.5122 | 2.57 | 325 | 0.5297 |
118
+ | 0.5426 | 2.61 | 330 | 0.5290 |
119
+ | 0.4955 | 2.65 | 335 | 0.5290 |
120
+ | 0.5531 | 2.69 | 340 | 0.5273 |
121
+ | 0.5147 | 2.73 | 345 | 0.5278 |
122
+ | 0.5195 | 2.77 | 350 | 0.5246 |
123
+ | 0.5268 | 2.81 | 355 | 0.5254 |
124
+ | 0.5284 | 2.85 | 360 | 0.5236 |
125
+ | 0.5272 | 2.89 | 365 | 0.5224 |
126
+ | 0.5053 | 2.92 | 370 | 0.5240 |
127
+ | 0.528 | 2.96 | 375 | 0.5200 |
128
+ | 0.517 | 3.0 | 380 | 0.5204 |
129
+ | 0.5409 | 3.04 | 385 | 0.5206 |
130
+ | 0.5204 | 3.08 | 390 | 0.5192 |
131
+ | 0.52 | 3.12 | 395 | 0.5200 |
132
+ | 0.5142 | 3.16 | 400 | 0.5205 |
133
+
134
+
135
+ ### Framework versions
136
+
137
+ - PEFT 0.8.1
138
+ - Transformers 4.37.2
139
+ - Pytorch 2.1.0+cu121
140
+ - Datasets 2.16.1
141
+ - Tokenizers 0.15.1