dino-base-2023_11_27-with_custom_head
This model is a fine-tuned version of facebook/dinov2-base on the multilabel_complete_dataset dataset. It achieves the following results on the evaluation set:
- Loss: 0.1028
- F1 Micro: 0.8515
- F1 Macro: 0.8179
- Roc Auc: 0.9057
- Accuracy: 0.5716
- Learning Rate: 0.0000
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.01
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 90
Training results
Training Loss | Epoch | Step | Validation Loss | F1 Micro | F1 Macro | Roc Auc | Accuracy | Rate |
---|---|---|---|---|---|---|---|---|
0.2471 | 1.0 | 536 | 0.2136 | 0.5684 | 0.4006 | 0.7020 | 0.3944 | 0.01 |
0.2208 | 2.0 | 1072 | 0.2199 | 0.6909 | 0.6462 | 0.7945 | 0.4091 | 0.01 |
0.2181 | 3.0 | 1608 | 0.1979 | 0.6712 | 0.5771 | 0.7667 | 0.4312 | 0.01 |
0.2187 | 4.0 | 2144 | 0.1767 | 0.7318 | 0.6061 | 0.8269 | 0.4137 | 0.01 |
0.2128 | 5.0 | 2680 | 0.1760 | 0.7173 | 0.5527 | 0.8012 | 0.4327 | 0.01 |
0.2171 | 6.0 | 3216 | 0.1992 | 0.7109 | 0.5184 | 0.8062 | 0.4227 | 0.01 |
0.2108 | 7.0 | 3752 | 0.1695 | 0.7339 | 0.6055 | 0.8232 | 0.4173 | 0.01 |
0.2147 | 8.0 | 4288 | 0.1619 | 0.7441 | 0.6176 | 0.8342 | 0.4227 | 0.01 |
0.2112 | 9.0 | 4824 | 0.1708 | 0.7337 | 0.6402 | 0.8245 | 0.4309 | 0.01 |
0.216 | 10.0 | 5360 | 0.1751 | 0.7493 | 0.6434 | 0.8372 | 0.4284 | 0.01 |
0.2151 | 11.0 | 5896 | 0.1701 | 0.7242 | 0.6293 | 0.8078 | 0.4337 | 0.01 |
0.213 | 12.0 | 6432 | 0.2409 | 0.6629 | 0.5371 | 0.8035 | 0.3566 | 0.01 |
0.2161 | 13.0 | 6968 | 0.1705 | 0.7291 | 0.6243 | 0.8207 | 0.4194 | 0.01 |
0.2136 | 14.0 | 7504 | 0.1693 | 0.7343 | 0.6215 | 0.8177 | 0.4427 | 0.01 |
0.1826 | 15.0 | 8040 | 0.1388 | 0.7968 | 0.7460 | 0.8648 | 0.4970 | 0.001 |
0.1731 | 16.0 | 8576 | 0.1476 | 0.8016 | 0.7385 | 0.8631 | 0.5155 | 0.001 |
0.1649 | 17.0 | 9112 | 0.1351 | 0.8133 | 0.7693 | 0.8812 | 0.5023 | 0.001 |
0.1624 | 18.0 | 9648 | 0.1385 | 0.8186 | 0.7714 | 0.8838 | 0.5213 | 0.001 |
0.1576 | 19.0 | 10184 | 0.1302 | 0.8176 | 0.7632 | 0.8779 | 0.5205 | 0.001 |
0.1544 | 20.0 | 10720 | 0.1234 | 0.8230 | 0.7722 | 0.8781 | 0.5320 | 0.001 |
0.1542 | 21.0 | 11256 | 0.1304 | 0.8268 | 0.7755 | 0.8884 | 0.5259 | 0.001 |
0.1525 | 22.0 | 11792 | 0.1220 | 0.8177 | 0.7646 | 0.8725 | 0.5377 | 0.001 |
0.1505 | 23.0 | 12328 | 0.1311 | 0.8264 | 0.7872 | 0.8918 | 0.5255 | 0.001 |
0.1515 | 24.0 | 12864 | 0.1247 | 0.8188 | 0.7630 | 0.8737 | 0.5316 | 0.001 |
0.1471 | 25.0 | 13400 | 0.1265 | 0.8258 | 0.7726 | 0.8875 | 0.5227 | 0.001 |
0.1475 | 26.0 | 13936 | 0.1277 | 0.8301 | 0.7834 | 0.8867 | 0.5330 | 0.001 |
0.1484 | 27.0 | 14472 | 0.1236 | 0.8218 | 0.7661 | 0.8754 | 0.5398 | 0.001 |
0.1472 | 28.0 | 15008 | 0.1257 | 0.8229 | 0.7729 | 0.8758 | 0.5402 | 0.001 |
0.1379 | 29.0 | 15544 | 0.1199 | 0.8352 | 0.7891 | 0.8865 | 0.5452 | 0.0001 |
0.1349 | 30.0 | 16080 | 0.1156 | 0.8413 | 0.7944 | 0.8951 | 0.5488 | 0.0001 |
0.1326 | 31.0 | 16616 | 0.1152 | 0.8404 | 0.7983 | 0.8961 | 0.5498 | 0.0001 |
0.1321 | 32.0 | 17152 | 0.1137 | 0.8386 | 0.7911 | 0.8902 | 0.5509 | 0.0001 |
0.1294 | 33.0 | 17688 | 0.1136 | 0.8406 | 0.7924 | 0.8916 | 0.5513 | 0.0001 |
0.1297 | 34.0 | 18224 | 0.1100 | 0.8439 | 0.7995 | 0.8965 | 0.5552 | 0.0001 |
0.1296 | 35.0 | 18760 | 0.1102 | 0.8431 | 0.7959 | 0.8953 | 0.5570 | 0.0001 |
0.1276 | 36.0 | 19296 | 0.1104 | 0.8429 | 0.7954 | 0.8933 | 0.5584 | 0.0001 |
0.1264 | 37.0 | 19832 | 0.1111 | 0.8468 | 0.8073 | 0.9004 | 0.5645 | 0.0001 |
0.1279 | 38.0 | 20368 | 0.1105 | 0.8457 | 0.8060 | 0.8964 | 0.5663 | 0.0001 |
0.1231 | 39.0 | 20904 | 0.1115 | 0.8481 | 0.8105 | 0.9016 | 0.5623 | 0.0001 |
0.1276 | 40.0 | 21440 | 0.1089 | 0.8443 | 0.7999 | 0.8932 | 0.5648 | 0.0001 |
0.121 | 41.0 | 21976 | 0.1098 | 0.8455 | 0.8015 | 0.8953 | 0.5627 | 0.0001 |
0.1229 | 42.0 | 22512 | 0.1087 | 0.8459 | 0.8009 | 0.8966 | 0.5620 | 0.0001 |
0.1227 | 43.0 | 23048 | 0.1088 | 0.8468 | 0.8067 | 0.8957 | 0.5688 | 0.0001 |
0.1221 | 44.0 | 23584 | 0.1076 | 0.8476 | 0.8066 | 0.8974 | 0.5673 | 0.0001 |
0.1191 | 45.0 | 24120 | 0.1069 | 0.8508 | 0.8173 | 0.9027 | 0.5698 | 0.0001 |
0.1212 | 46.0 | 24656 | 0.1072 | 0.8509 | 0.8174 | 0.9086 | 0.5634 | 0.0001 |
0.1198 | 47.0 | 25192 | 0.1066 | 0.8493 | 0.8090 | 0.9001 | 0.5688 | 0.0001 |
0.1201 | 48.0 | 25728 | 0.1076 | 0.8485 | 0.8083 | 0.9002 | 0.5652 | 0.0001 |
0.1189 | 49.0 | 26264 | 0.1065 | 0.8508 | 0.8152 | 0.9027 | 0.5688 | 0.0001 |
0.1176 | 50.0 | 26800 | 0.1073 | 0.8463 | 0.8034 | 0.8965 | 0.5634 | 0.0001 |
0.1202 | 51.0 | 27336 | 0.1073 | 0.8481 | 0.8102 | 0.8994 | 0.5727 | 0.0001 |
0.1167 | 52.0 | 27872 | 0.1060 | 0.8522 | 0.8179 | 0.9069 | 0.5716 | 0.0001 |
0.1192 | 53.0 | 28408 | 0.1063 | 0.8507 | 0.8128 | 0.9010 | 0.5713 | 0.0001 |
0.1156 | 54.0 | 28944 | 0.1067 | 0.8493 | 0.8113 | 0.9000 | 0.5720 | 0.0001 |
0.1193 | 55.0 | 29480 | 0.1069 | 0.8490 | 0.8116 | 0.8995 | 0.5727 | 0.0001 |
0.116 | 56.0 | 30016 | 0.1056 | 0.8543 | 0.8186 | 0.9077 | 0.5745 | 0.0001 |
0.1147 | 57.0 | 30552 | 0.1063 | 0.8505 | 0.8114 | 0.8980 | 0.5731 | 0.0001 |
0.1139 | 58.0 | 31088 | 0.1066 | 0.8489 | 0.8074 | 0.8986 | 0.5706 | 0.0001 |
0.1143 | 59.0 | 31624 | 0.1074 | 0.8491 | 0.8065 | 0.8971 | 0.5727 | 0.0001 |
0.1148 | 60.0 | 32160 | 0.1078 | 0.8499 | 0.8080 | 0.8981 | 0.5695 | 0.0001 |
0.1143 | 61.0 | 32696 | 0.1054 | 0.8512 | 0.8159 | 0.9010 | 0.5723 | 0.0001 |
0.1133 | 62.0 | 33232 | 0.1058 | 0.8496 | 0.8083 | 0.8973 | 0.5738 | 0.0001 |
0.1134 | 63.0 | 33768 | 0.1063 | 0.8479 | 0.8088 | 0.8991 | 0.5681 | 0.0001 |
0.1123 | 64.0 | 34304 | 0.1054 | 0.8504 | 0.8121 | 0.8997 | 0.5702 | 0.0001 |
0.1141 | 65.0 | 34840 | 0.1050 | 0.8494 | 0.8099 | 0.8989 | 0.5731 | 0.0001 |
0.1104 | 66.0 | 35376 | 0.1050 | 0.8507 | 0.8133 | 0.8979 | 0.5763 | 0.0001 |
0.1124 | 67.0 | 35912 | 0.1060 | 0.8513 | 0.8163 | 0.9036 | 0.5670 | 0.0001 |
0.1111 | 68.0 | 36448 | 0.1054 | 0.8512 | 0.8157 | 0.9019 | 0.5681 | 0.0001 |
0.1097 | 69.0 | 36984 | 0.1056 | 0.8501 | 0.8110 | 0.9021 | 0.5673 | 0.0001 |
0.1096 | 70.0 | 37520 | 0.1059 | 0.8501 | 0.8119 | 0.8997 | 0.5673 | 0.0001 |
0.1097 | 71.0 | 38056 | 0.1055 | 0.8517 | 0.8172 | 0.9037 | 0.5738 | 0.0001 |
0.1084 | 72.0 | 38592 | 0.1074 | 0.8469 | 0.8063 | 0.8961 | 0.5631 | 1e-05 |
0.1091 | 73.0 | 39128 | 0.1044 | 0.8525 | 0.8171 | 0.9028 | 0.5734 | 1e-05 |
0.1051 | 74.0 | 39664 | 0.1041 | 0.8533 | 0.8187 | 0.9050 | 0.5716 | 1e-05 |
0.1069 | 75.0 | 40200 | 0.1056 | 0.8506 | 0.8155 | 0.9013 | 0.5698 | 1e-05 |
0.1079 | 76.0 | 40736 | 0.1043 | 0.8517 | 0.8154 | 0.9027 | 0.5723 | 1e-05 |
0.1072 | 77.0 | 41272 | 0.1040 | 0.8536 | 0.8188 | 0.9026 | 0.5781 | 1e-05 |
0.105 | 78.0 | 41808 | 0.1043 | 0.8514 | 0.8150 | 0.9018 | 0.5713 | 1e-05 |
0.1061 | 79.0 | 42344 | 0.1043 | 0.8526 | 0.8181 | 0.9023 | 0.5734 | 1e-05 |
0.1045 | 80.0 | 42880 | 0.1051 | 0.8512 | 0.8177 | 0.9018 | 0.5720 | 1e-05 |
0.1062 | 81.0 | 43416 | 0.1069 | 0.8501 | 0.8157 | 0.8985 | 0.5695 | 1e-05 |
0.1057 | 82.0 | 43952 | 0.1040 | 0.8530 | 0.8184 | 0.9039 | 0.5734 | 1e-05 |
0.1073 | 83.0 | 44488 | 0.1048 | 0.8504 | 0.8151 | 0.8974 | 0.5713 | 1e-05 |
0.1059 | 84.0 | 45024 | 0.1044 | 0.8526 | 0.8178 | 0.9026 | 0.5741 | 1e-05 |
0.1054 | 85.0 | 45560 | 0.1063 | 0.8505 | 0.8149 | 0.9003 | 0.5702 | 1e-05 |
0.1046 | 86.0 | 46096 | 0.1043 | 0.8525 | 0.8161 | 0.9004 | 0.5727 | 1e-05 |
0.105 | 87.0 | 46632 | 0.1047 | 0.8532 | 0.8185 | 0.9042 | 0.5720 | 1e-05 |
0.1029 | 88.0 | 47168 | 0.1043 | 0.8518 | 0.8156 | 0.9041 | 0.5759 | 1e-05 |
0.1059 | 89.0 | 47704 | 0.1040 | 0.8539 | 0.8178 | 0.9034 | 0.5788 | 0.0000 |
0.1047 | 90.0 | 48240 | 0.1047 | 0.8505 | 0.8136 | 0.8983 | 0.5695 | 0.0000 |
Framework versions
- Transformers 4.34.1
- Pytorch 2.1.0+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1
- Downloads last month
- 2
Model tree for lombardata/dino-base-2023_11_27-with_custom_head
Base model
facebook/dinov2-base
Finetuned
this model