File size: 4,090 Bytes
7eb1b45 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
---
library_name: transformers
license: gemma
base_model: google/paligemma-3b-pt-224
tags:
- generated_from_trainer
model-index:
- name: paligemma-adapter-new
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# paligemma-adapter-new
This model is a fine-tuned version of [google/paligemma-3b-pt-224](https://huggingface.co/google/paligemma-3b-pt-224) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9112
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2
- num_epochs: 50
### Training results
| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-------:|:-----:|:---------------:|
| 1.1672 | 0.9997 | 1682 | 1.1611 |
| 1.033 | 2.0 | 3365 | 1.0390 |
| 0.9638 | 2.9997 | 5047 | 0.9879 |
| 0.9611 | 4.0 | 6730 | 0.9610 |
| 0.9409 | 4.9997 | 8412 | 0.9443 |
| 0.9136 | 6.0 | 10095 | 0.9344 |
| 0.9081 | 6.9997 | 11777 | 0.9271 |
| 0.9128 | 8.0 | 13460 | 0.9226 |
| 0.8958 | 8.9997 | 15142 | 0.9195 |
| 0.91 | 10.0 | 16825 | 0.9177 |
| 0.9061 | 10.9997 | 18507 | 0.9157 |
| 0.9013 | 12.0 | 20190 | 0.9144 |
| 0.9005 | 12.9997 | 21872 | 0.9137 |
| 0.8874 | 14.0 | 23555 | 0.9130 |
| 0.9176 | 14.9997 | 25237 | 0.9127 |
| 0.8866 | 16.0 | 26920 | 0.9125 |
| 0.8978 | 16.9997 | 28602 | 0.9119 |
| 0.892 | 18.0 | 30285 | 0.9117 |
| 0.8945 | 18.9997 | 31967 | 0.9116 |
| 0.8908 | 20.0 | 33650 | 0.9115 |
| 0.8837 | 20.9997 | 35332 | 0.9115 |
| 0.8957 | 22.0 | 37015 | 0.9112 |
| 0.8887 | 22.9997 | 38697 | 0.9114 |
| 0.8962 | 24.0 | 40380 | 0.9114 |
| 0.899 | 24.9997 | 42062 | 0.9114 |
| 0.9024 | 26.0 | 43745 | 0.9112 |
| 0.8873 | 26.9997 | 45427 | 0.9112 |
| 0.9049 | 28.0 | 47110 | 0.9111 |
| 0.8953 | 28.9997 | 48792 | 0.9113 |
| 0.8929 | 30.0 | 50475 | 0.9112 |
| 0.9003 | 30.9997 | 52157 | 0.9111 |
| 0.8913 | 32.0 | 53840 | 0.9112 |
| 0.8934 | 32.9997 | 55522 | 0.9111 |
| 0.9022 | 34.0 | 57205 | 0.9112 |
| 0.8935 | 34.9997 | 58887 | 0.9112 |
| 0.8994 | 36.0 | 60570 | 0.9112 |
| 0.894 | 36.9997 | 62252 | 0.9112 |
| 0.8938 | 38.0 | 63935 | 0.9112 |
| 0.8985 | 38.9997 | 65617 | 0.9112 |
| 0.9013 | 40.0 | 67300 | 0.9111 |
| 0.9023 | 40.9997 | 68982 | 0.9111 |
| 0.9065 | 42.0 | 70665 | 0.9110 |
| 0.9045 | 42.9997 | 72347 | 0.9111 |
| 0.9013 | 44.0 | 74030 | 0.9112 |
| 0.8855 | 44.9997 | 75712 | 0.9112 |
| 0.8864 | 46.0 | 77395 | 0.9110 |
| 0.9026 | 46.9997 | 79077 | 0.9112 |
| 0.8979 | 48.0 | 80760 | 0.9111 |
| 0.9066 | 48.9997 | 82442 | 0.9111 |
| 0.896 | 49.9851 | 84100 | 0.9112 |
### Framework versions
- Transformers 4.44.2
- Pytorch 2.4.0+cu118
- Datasets 2.20.0
- Tokenizers 0.19.1
|