Hugging Face's logo

rnaveensrinivas
/
Mistral-7B-Instruct-v0.2-GPTQ_retrained_network_corpus

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
rnaveensrinivas commited on
Commit
3f73e1c
•
1 Parent(s): 97294f6

End of training

Browse files
Files changed (3) hide show
  1. README.md +19 -14
  2. adapter_model.safetensors +1 -1
  3. runs/Jul09_10-54-39_f8ebfe44bb73/events.out.tfevents.1720526509.f8ebfe44bb73.34.1 +3 -0
README.md CHANGED
@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.2-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GPTQ) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 2.3489
20
 
21
  ## Model description
22
 
@@ -36,32 +36,37 @@ More information needed
36
 
37
  The following hyperparameters were used during training:
38
  - learning_rate: 0.0002
39
- - train_batch_size: 8
40
- - eval_batch_size: 8
41
  - seed: 42
42
  - gradient_accumulation_steps: 4
43
- - total_train_batch_size: 32
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: linear
46
  - lr_scheduler_warmup_steps: 2
47
- - num_epochs: 5
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
- | Training Loss | Epoch | Step | Validation Loss |
53
- |:-------------:|:------:|:----:|:---------------:|
54
- | 2.9028 | 0.8571 | 3 | 2.4332 |
55
- | 2.1399 | 2.0 | 7 | 2.3801 |
56
- | 2.7781 | 2.8571 | 10 | 2.3585 |
57
- | 2.0651 | 4.0 | 14 | 2.3493 |
58
- | 2.296 | 4.2857 | 15 | 2.3489 |
 
 
 
 
 
59
 
60
 
61
  ### Framework versions
62
 
63
  - PEFT 0.11.1
64
- - Transformers 4.40.2
65
  - Pytorch 2.1.0+cu121
66
- - Datasets 2.19.1
67
  - Tokenizers 0.19.1
 
16
 
17
  This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.2-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GPTQ) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 2.2183
20
 
21
  ## Model description
22
 
 
36
 
37
  The following hyperparameters were used during training:
38
  - learning_rate: 0.0002
39
+ - train_batch_size: 1
40
+ - eval_batch_size: 1
41
  - seed: 42
42
  - gradient_accumulation_steps: 4
43
+ - total_train_batch_size: 4
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: linear
46
  - lr_scheduler_warmup_steps: 2
47
+ - num_epochs: 10
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
+ | Training Loss | Epoch | Step | Validation Loss |
53
+ |:-------------:|:-----:|:----:|:---------------:|
54
+ | 2.2041 | 1.0 | 28 | 2.2674 |
55
+ | 1.9644 | 2.0 | 56 | 2.2183 |
56
+ | 1.7637 | 3.0 | 84 | 2.2315 |
57
+ | 1.5934 | 4.0 | 112 | 2.2831 |
58
+ | 1.4509 | 5.0 | 140 | 2.3071 |
59
+ | 1.3409 | 6.0 | 168 | 2.4755 |
60
+ | 1.2542 | 7.0 | 196 | 2.4905 |
61
+ | 1.179 | 8.0 | 224 | 2.6153 |
62
+ | 1.1211 | 9.0 | 252 | 2.6870 |
63
+ | 1.0801 | 10.0 | 280 | 2.7418 |
64
 
65
 
66
  ### Framework versions
67
 
68
  - PEFT 0.11.1
69
+ - Transformers 4.41.2
70
  - Pytorch 2.1.0+cu121
71
+ - Datasets 2.19.2
72
  - Tokenizers 0.19.1
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c73bdd8e9fc1236cfa3ce2747c8fd8c301cebe429765e593fc6481e0c83bf364
3
  size 8397056
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ac99ce525dbb9398205424a2317e32bb639a5dd8040f0205fe1284e8d876670
3
  size 8397056
runs/Jul09_10-54-39_f8ebfe44bb73/events.out.tfevents.1720526509.f8ebfe44bb73.34.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50002c8c12155009c809886e31cff4e582e9cb1167622bdc9d626dcf3b572004
3
+ size 359