Edit model card

covid3-mistral-dpo-gptq

This model is a fine-tuned version of TheBloke/OpenHermes-2-Mistral-7B-GPTQ on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 2.2375
  • Rewards/chosen: -2.8294
  • Rewards/rejected: -1.7077
  • Rewards/accuracies: 0.25
  • Rewards/margins: -1.1217
  • Logps/rejected: -24.0692
  • Logps/chosen: -35.7956
  • Logits/rejected: -2.8653
  • Logits/chosen: -2.8666

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 2
  • training_steps: 1000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.6957 0.0 10 0.6940 0.0226 0.0252 0.375 -0.0026 -6.7409 -7.2761 -2.8058 -2.8067
0.6925 0.0 20 0.6971 0.0317 0.0422 0.3333 -0.0105 -6.5702 -7.1844 -2.8074 -2.8082
0.6876 0.01 30 0.6995 0.0202 0.0373 0.375 -0.0170 -6.6197 -7.2995 -2.8093 -2.8102
0.6961 0.01 40 0.6982 0.0054 0.0189 0.375 -0.0135 -6.8034 -7.4475 -2.8113 -2.8122
0.6863 0.01 50 0.6998 0.0019 0.0188 0.3333 -0.0169 -6.8044 -7.4830 -2.8121 -2.8130
0.6965 0.01 60 0.6977 0.0119 0.0251 0.2917 -0.0132 -6.7419 -7.3829 -2.8120 -2.8129
0.7209 0.01 70 0.6993 0.0336 0.0497 0.3333 -0.0161 -6.4949 -7.1656 -2.8103 -2.8112
0.6988 0.01 80 0.6984 0.0294 0.0432 0.375 -0.0138 -6.5605 -7.2080 -2.8085 -2.8094
0.6913 0.01 90 0.6981 0.0216 0.0342 0.4167 -0.0126 -6.6501 -7.2856 -2.8084 -2.8093
0.6641 0.02 100 0.7030 0.0493 0.0702 0.3333 -0.0209 -6.2907 -7.0088 -2.8098 -2.8107
0.7083 0.02 110 0.7072 0.0575 0.0870 0.3333 -0.0295 -6.1225 -6.9268 -2.8105 -2.8114
0.6307 0.02 120 0.7128 0.0727 0.1120 0.3333 -0.0393 -5.8727 -6.7749 -2.8105 -2.8114
0.7216 0.02 130 0.7158 0.0814 0.1250 0.3333 -0.0436 -5.7422 -6.6879 -2.8108 -2.8117
0.7189 0.02 140 0.7135 0.0948 0.1343 0.3333 -0.0395 -5.6489 -6.5536 -2.8099 -2.8108
0.7177 0.03 150 0.7128 0.0954 0.1335 0.3333 -0.0381 -5.6579 -6.5481 -2.8100 -2.8109
0.639 0.03 160 0.7232 0.0823 0.1404 0.3333 -0.0581 -5.5880 -6.6785 -2.8135 -2.8144
0.7128 0.03 170 0.7361 0.0571 0.1393 0.375 -0.0822 -5.5991 -6.9308 -2.8165 -2.8174
0.709 0.03 180 0.7361 0.0690 0.1519 0.375 -0.0829 -5.4739 -6.8120 -2.8159 -2.8168
0.6167 0.03 190 0.7483 0.0424 0.1461 0.375 -0.1038 -5.5311 -7.0782 -2.8180 -2.8189
0.7521 0.03 200 0.7589 0.0180 0.1360 0.3333 -0.1180 -5.6325 -7.3223 -2.8199 -2.8209
0.6204 0.04 210 0.7726 -0.0220 0.1130 0.375 -0.1350 -5.8622 -7.7214 -2.8217 -2.8227
0.6578 0.04 220 0.7839 -0.0525 0.0994 0.3333 -0.1520 -5.9980 -8.0273 -2.8232 -2.8242
0.7633 0.04 230 0.7868 -0.0613 0.0902 0.375 -0.1516 -6.0903 -8.1152 -2.8235 -2.8245
0.7391 0.04 240 0.7917 -0.0742 0.0850 0.375 -0.1592 -6.1429 -8.2441 -2.8246 -2.8256
0.6759 0.04 250 0.8023 -0.1101 0.0656 0.3333 -0.1757 -6.3368 -8.6031 -2.8262 -2.8272
0.6768 0.04 260 0.8107 -0.1470 0.0326 0.375 -0.1796 -6.6662 -8.9720 -2.8264 -2.8274
0.5398 0.04 270 0.8411 -0.2390 -0.0341 0.375 -0.2049 -7.3331 -9.8918 -2.8279 -2.8289
0.5617 0.05 280 0.8797 -0.3532 -0.1075 0.375 -0.2457 -8.0674 -11.0340 -2.8306 -2.8317
0.7585 0.05 290 0.9009 -0.4183 -0.1540 0.375 -0.2642 -8.5328 -11.6845 -2.8318 -2.8329
0.4971 0.05 300 0.9602 -0.5793 -0.2520 0.375 -0.3274 -9.5121 -13.2952 -2.8362 -2.8373
0.5759 0.05 310 1.0568 -0.8155 -0.3982 0.375 -0.4173 -10.9749 -15.6568 -2.8426 -2.8437
0.451 0.05 320 1.1605 -1.0527 -0.5383 0.375 -0.5144 -12.3754 -18.0287 -2.8482 -2.8493
1.4199 0.06 330 1.1756 -1.1393 -0.6287 0.375 -0.5106 -13.2791 -18.8948 -2.8505 -2.8516
0.6853 0.06 340 1.1875 -1.1840 -0.6936 0.375 -0.4904 -13.9281 -19.3416 -2.8530 -2.8541
0.3956 0.06 350 1.2550 -1.2944 -0.7654 0.375 -0.5291 -14.6460 -20.4463 -2.8568 -2.8579
0.8692 0.06 360 1.3093 -1.4107 -0.8644 0.375 -0.5463 -15.6363 -21.6084 -2.8602 -2.8613
1.4214 0.06 370 1.2759 -1.3853 -0.8782 0.375 -0.5071 -15.7746 -21.3549 -2.8579 -2.8590
0.6163 0.06 380 1.3124 -1.4537 -0.9274 0.375 -0.5263 -16.2665 -22.0389 -2.8580 -2.8591
0.586 0.07 390 1.4060 -1.6073 -1.0263 0.375 -0.5810 -17.2554 -23.5750 -2.8594 -2.8605
1.7565 0.07 400 1.3869 -1.5469 -0.9534 0.375 -0.5936 -16.5259 -22.9709 -2.8611 -2.8623
0.749 0.07 410 1.4037 -1.5658 -0.9400 0.375 -0.6258 -16.3927 -23.1602 -2.8615 -2.8626
0.7682 0.07 420 1.4444 -1.6154 -0.9575 0.375 -0.6578 -16.5678 -23.6556 -2.8618 -2.8630
0.5276 0.07 430 1.5646 -1.7833 -1.0365 0.375 -0.7467 -17.3576 -25.3345 -2.8645 -2.8658
1.2132 0.07 440 1.6229 -1.8510 -1.0641 0.375 -0.7869 -17.6336 -26.0119 -2.8657 -2.8670
1.0323 0.07 450 1.6468 -1.8672 -1.0528 0.3333 -0.8143 -17.5208 -26.1736 -2.8655 -2.8668
1.1453 0.08 460 1.6741 -1.8759 -1.0266 0.3333 -0.8494 -17.2580 -26.2613 -2.8659 -2.8672
1.526 0.08 470 1.6465 -1.8347 -1.0076 0.3333 -0.8271 -17.0681 -25.8488 -2.8671 -2.8684
1.1323 0.08 480 1.5543 -1.7064 -0.9557 0.3333 -0.7507 -16.5494 -24.5655 -2.8682 -2.8694
1.0389 0.08 490 1.5824 -1.7717 -1.0002 0.3333 -0.7715 -16.9945 -25.2190 -2.8694 -2.8706
0.8626 0.08 500 1.6038 -1.8376 -1.0545 0.3333 -0.7831 -17.5374 -25.8781 -2.8693 -2.8706
0.8392 0.09 510 1.6952 -1.9873 -1.1387 0.3333 -0.8486 -18.3790 -27.3744 -2.8697 -2.8710
0.6528 0.09 520 1.7895 -2.1144 -1.1842 0.25 -0.9302 -18.8344 -28.6457 -2.8693 -2.8707
1.3843 0.09 530 1.8088 -2.1501 -1.2043 0.25 -0.9458 -19.0354 -29.0030 -2.8696 -2.8710
1.296 0.09 540 1.7833 -2.1309 -1.2130 0.25 -0.9178 -19.1228 -28.8106 -2.8691 -2.8705
0.7343 0.09 550 1.8244 -2.1833 -1.2404 0.25 -0.9428 -19.3968 -29.3344 -2.8676 -2.8689
1.089 0.09 560 1.8288 -2.1789 -1.2313 0.25 -0.9476 -19.3059 -29.2912 -2.8690 -2.8704
0.8322 0.1 570 1.9009 -2.2715 -1.2811 0.25 -0.9903 -19.8038 -30.2165 -2.8697 -2.8711
0.8684 0.1 580 1.9310 -2.3144 -1.3151 0.25 -0.9993 -20.1433 -30.6454 -2.8722 -2.8736
0.9827 0.1 590 1.9558 -2.3309 -1.3222 0.25 -1.0087 -20.2145 -30.8112 -2.8740 -2.8754
0.5176 0.1 600 1.9731 -2.3665 -1.3574 0.25 -1.0091 -20.5666 -31.1672 -2.8754 -2.8768
1.0789 0.1 610 2.0276 -2.4550 -1.4152 0.25 -1.0398 -21.1444 -32.0516 -2.8756 -2.8769
0.8444 0.1 620 2.1331 -2.6253 -1.5121 0.25 -1.1132 -22.1132 -33.7550 -2.8726 -2.8739
1.6609 0.1 630 2.1160 -2.6511 -1.5573 0.25 -1.0938 -22.5657 -34.0127 -2.8740 -2.8753
1.3086 0.11 640 2.0791 -2.6721 -1.6152 0.25 -1.0569 -23.1446 -34.2231 -2.8749 -2.8762
1.0659 0.11 650 2.0520 -2.6575 -1.6184 0.25 -1.0391 -23.1760 -34.0763 -2.8766 -2.8778
1.3081 0.11 660 2.0481 -2.6650 -1.6332 0.25 -1.0318 -23.3247 -34.1520 -2.8756 -2.8769
0.769 0.11 670 2.0971 -2.7165 -1.6666 0.25 -1.0500 -23.6581 -34.6672 -2.8745 -2.8758
1.1385 0.11 680 2.1554 -2.7771 -1.7021 0.25 -1.0750 -24.0137 -35.2731 -2.8735 -2.8748
1.0306 0.12 690 2.2076 -2.8501 -1.7587 0.25 -1.0914 -24.5793 -36.0025 -2.8714 -2.8727
1.3893 0.12 700 2.2299 -2.8955 -1.7944 0.25 -1.1010 -24.9367 -36.4564 -2.8682 -2.8695
2.2234 0.12 710 2.2237 -2.9162 -1.8126 0.25 -1.1036 -25.1184 -36.6639 -2.8654 -2.8667
0.4678 0.12 720 2.2379 -2.9096 -1.7873 0.25 -1.1223 -24.8652 -36.5974 -2.8658 -2.8671
0.8098 0.12 730 2.2768 -2.9290 -1.7762 0.25 -1.1529 -24.7543 -36.7922 -2.8652 -2.8665
1.8821 0.12 740 2.2740 -2.9198 -1.7623 0.25 -1.1574 -24.6159 -36.6994 -2.8641 -2.8654
1.095 0.12 750 2.2689 -2.8862 -1.7174 0.25 -1.1688 -24.1662 -36.3637 -2.8647 -2.8660
1.7464 0.13 760 2.2488 -2.8828 -1.7320 0.25 -1.1508 -24.3128 -36.3297 -2.8640 -2.8653
0.9967 0.13 770 2.2235 -2.8783 -1.7502 0.25 -1.1281 -24.4945 -36.2849 -2.8622 -2.8634
0.7823 0.13 780 2.2370 -2.9074 -1.7744 0.25 -1.1330 -24.7361 -36.5759 -2.8593 -2.8606
1.3903 0.13 790 2.2755 -2.9143 -1.7485 0.25 -1.1658 -24.4774 -36.6450 -2.8587 -2.8600
2.0372 0.13 800 2.2250 -2.7892 -1.6505 0.25 -1.1387 -23.4972 -35.3939 -2.8629 -2.8642
0.7111 0.14 810 2.2409 -2.7911 -1.6348 0.25 -1.1562 -23.3407 -35.4124 -2.8642 -2.8654
0.8446 0.14 820 2.2740 -2.8395 -1.6646 0.25 -1.1749 -23.6383 -35.8968 -2.8638 -2.8651
1.2303 0.14 830 2.2812 -2.8540 -1.6787 0.25 -1.1752 -23.7798 -36.0417 -2.8648 -2.8661
0.5053 0.14 840 2.2834 -2.8740 -1.7065 0.25 -1.1675 -24.0571 -36.2418 -2.8640 -2.8653
0.5767 0.14 850 2.3105 -2.9262 -1.7448 0.25 -1.1814 -24.4399 -36.7635 -2.8618 -2.8631
1.7435 0.14 860 2.3174 -2.9360 -1.7519 0.25 -1.1841 -24.5119 -36.8619 -2.8627 -2.8639
1.6134 0.14 870 2.3028 -2.9288 -1.7659 0.25 -1.1629 -24.6517 -36.7902 -2.8635 -2.8647
1.747 0.15 880 2.2686 -2.8780 -1.7398 0.25 -1.1382 -24.3902 -36.2816 -2.8658 -2.8671
1.3341 0.15 890 2.2555 -2.8559 -1.7244 0.25 -1.1315 -24.2361 -36.0610 -2.8673 -2.8686
1.884 0.15 900 2.2349 -2.8291 -1.7129 0.25 -1.1162 -24.1211 -35.7924 -2.8677 -2.8689
0.5031 0.15 910 2.2361 -2.8327 -1.7156 0.25 -1.1171 -24.1479 -35.8284 -2.8671 -2.8684
0.7273 0.15 920 2.2545 -2.8595 -1.7291 0.25 -1.1304 -24.2834 -36.0963 -2.8665 -2.8678
1.2208 0.15 930 2.2655 -2.8756 -1.7364 0.25 -1.1393 -24.3561 -36.2580 -2.8656 -2.8669
0.6928 0.16 940 2.2697 -2.8817 -1.7405 0.25 -1.1412 -24.3971 -36.3184 -2.8652 -2.8665
2.2099 0.16 950 2.2581 -2.8642 -1.7302 0.25 -1.1340 -24.2945 -36.1442 -2.8656 -2.8668
1.6883 0.16 960 2.2544 -2.8575 -1.7258 0.25 -1.1318 -24.2503 -36.0772 -2.8656 -2.8668
1.9968 0.16 970 2.2455 -2.8405 -1.7135 0.25 -1.1271 -24.1270 -35.9072 -2.8657 -2.8670
2.1044 0.16 980 2.2400 -2.8308 -1.7064 0.25 -1.1243 -24.0569 -35.8097 -2.8656 -2.8668
0.7207 0.17 990 2.2376 -2.8286 -1.7067 0.25 -1.1218 -24.0597 -35.7875 -2.8654 -2.8667
1.1388 0.17 1000 2.2375 -2.8294 -1.7077 0.25 -1.1217 -24.0692 -35.7956 -2.8653 -2.8666

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.0.1+cu117
  • Datasets 2.15.0
  • Tokenizers 0.15.0
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model's library. Check the docs .

Model tree for vtiyyal1/covid3-mistral-dpo-gptq