File size: 3,053 Bytes
e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f e27602a 4bfde6f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
---
license: apache-2.0
---
# Model Card for PaliGemma Dermatology Model
## Model Details
### Model Description
This model, based on the PaliGemma-3B architecture, has been fine-tuned for dermatology-related image and text processing tasks. The model is designed to assist in the identification of various skin conditions using a combination of image analysis and natural language processing.
- **Developed by:** Bruce_Wayne
- **Funded by [optional]:** Jhonny and koti
- **Model type:** vision model
- **Finetuned from model [optional]:** https://huggingface.co/google/paligemma-3b-pt-224
- **LoRa Adaptors used:** Yes
- **Intended use:** Medical image analysis, specifically for dermatology
**
## Uses
### Direct Use
The model can be directly used for analyzing dermatology images, providing insights into potential skin conditions.
## Bias, Risks, and Limitations
**Skin Tone Bias:** The model may have been trained on a dataset that does not adequately represent all skin tones, potentially leading to biased results.
**Geographic Bias:** The model's performance may vary depending on the prevalence of certain conditions in different geographic regions.
## How to Get Started with the Model
** python
from transformers import AutoProcessor, PaliGemmaForConditionalGeneration
model_id = "brucewayne0459/paligemma_derm"
processor = AutoProcessor.from_pretrained(model_id)
model = PaliGemmaForConditionalGeneration.from_pretrained(model_id)
**
## Training Details
### Training Data
The model was fine-tuned on a dataset of dermatological images combined with disease names
### Training Procedure
The model was fine-tuned using LoRA (Low-Rank Adaptation) for more efficient training. Mixed precision (bfloat16) was used to speed up training and reduce memory usage.
#### Training Hyperparameters
- **Training regime:** Mixed precision (bfloat16)
- **Epochs:** 10
- **Learning rate:** 2e-5
- **Batch size:** 6
- **Gradient accumulation steps:** 4
## Evaluation
### Testing Data, Factors & Metrics
#### Testing Data
The model was evaluated on a separate validation set of dermatological images and Disease Names, distinct from the training data.
#### Metrics
- **Validation Loss:** The loss was tracked throughout the training process to evaluate model performance.
- **Accuracy:** The primary metric for assessing model predictions.
### Results
The model achieved a final validation loss of approximately 0.2214, indicating reasonable performance in predicting skin conditions based on the dataset used.
#### Summary
## Environmental Impact
- **Hardware Type:** 1 x L4 GPU
- **Hours used:** ~22 HOURS
- **Cloud Provider:** LIGHTNING AI
- **Compute Region:** USA
- **Carbon Emitted:** 0.9 kg eq. CO2
## Technical Specifications
### Model Architecture and Objective
- **Architecture:** Vision-Language model based on PaliGemma-3B
- **Objective:** To classify and diagnose dermatological conditions from images and text
### Compute Infrastructure
#### Hardware
- **GPU:** 1xL4 GPU
-
## Model Card Authors
Bruce_Wayne
|