---
library_name: transformers
license: gemma
base_model: google/paligemma-3b-pt-224
tags:
- generated_from_trainer
model-index:
- name: paligemma-adapter-new
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# paligemma-adapter-new

This model is a fine-tuned version of [google/paligemma-3b-pt-224](https://huggingface.co/google/paligemma-3b-pt-224) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9112

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2
- num_epochs: 50

### Training results

| Training Loss | Epoch   | Step  | Validation Loss |
|:-------------:|:-------:|:-----:|:---------------:|
| 1.1672        | 0.9997  | 1682  | 1.1611          |
| 1.033         | 2.0     | 3365  | 1.0390          |
| 0.9638        | 2.9997  | 5047  | 0.9879          |
| 0.9611        | 4.0     | 6730  | 0.9610          |
| 0.9409        | 4.9997  | 8412  | 0.9443          |
| 0.9136        | 6.0     | 10095 | 0.9344          |
| 0.9081        | 6.9997  | 11777 | 0.9271          |
| 0.9128        | 8.0     | 13460 | 0.9226          |
| 0.8958        | 8.9997  | 15142 | 0.9195          |
| 0.91          | 10.0    | 16825 | 0.9177          |
| 0.9061        | 10.9997 | 18507 | 0.9157          |
| 0.9013        | 12.0    | 20190 | 0.9144          |
| 0.9005        | 12.9997 | 21872 | 0.9137          |
| 0.8874        | 14.0    | 23555 | 0.9130          |
| 0.9176        | 14.9997 | 25237 | 0.9127          |
| 0.8866        | 16.0    | 26920 | 0.9125          |
| 0.8978        | 16.9997 | 28602 | 0.9119          |
| 0.892         | 18.0    | 30285 | 0.9117          |
| 0.8945        | 18.9997 | 31967 | 0.9116          |
| 0.8908        | 20.0    | 33650 | 0.9115          |
| 0.8837        | 20.9997 | 35332 | 0.9115          |
| 0.8957        | 22.0    | 37015 | 0.9112          |
| 0.8887        | 22.9997 | 38697 | 0.9114          |
| 0.8962        | 24.0    | 40380 | 0.9114          |
| 0.899         | 24.9997 | 42062 | 0.9114          |
| 0.9024        | 26.0    | 43745 | 0.9112          |
| 0.8873        | 26.9997 | 45427 | 0.9112          |
| 0.9049        | 28.0    | 47110 | 0.9111          |
| 0.8953        | 28.9997 | 48792 | 0.9113          |
| 0.8929        | 30.0    | 50475 | 0.9112          |
| 0.9003        | 30.9997 | 52157 | 0.9111          |
| 0.8913        | 32.0    | 53840 | 0.9112          |
| 0.8934        | 32.9997 | 55522 | 0.9111          |
| 0.9022        | 34.0    | 57205 | 0.9112          |
| 0.8935        | 34.9997 | 58887 | 0.9112          |
| 0.8994        | 36.0    | 60570 | 0.9112          |
| 0.894         | 36.9997 | 62252 | 0.9112          |
| 0.8938        | 38.0    | 63935 | 0.9112          |
| 0.8985        | 38.9997 | 65617 | 0.9112          |
| 0.9013        | 40.0    | 67300 | 0.9111          |
| 0.9023        | 40.9997 | 68982 | 0.9111          |
| 0.9065        | 42.0    | 70665 | 0.9110          |
| 0.9045        | 42.9997 | 72347 | 0.9111          |
| 0.9013        | 44.0    | 74030 | 0.9112          |
| 0.8855        | 44.9997 | 75712 | 0.9112          |
| 0.8864        | 46.0    | 77395 | 0.9110          |
| 0.9026        | 46.9997 | 79077 | 0.9112          |
| 0.8979        | 48.0    | 80760 | 0.9111          |
| 0.9066        | 48.9997 | 82442 | 0.9111          |
| 0.896         | 49.9851 | 84100 | 0.9112          |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.4.0+cu118
- Datasets 2.20.0
- Tokenizers 0.19.1