---
title: web-md-llama2-7b-3000 
tags:
  - healthcare
  - NLP
  - dialogues
  - LLM
  - fine-tuned
license: unknown
datasets:
  - Kabatubare/medical-guanaco-3000
---

# Medical3000 Model Card

This is a model card for  web-md-llama2-7b-3000 , a fine-tuned version of Llama-2-7B, specifically aimed at medical dialogues.

Covered areas:

General Medicine: Basic medical advice, symptoms, general treatments.

Cardiology: Questions related to heart diseases, blood circulation.

Neurology: Topics around brain health, neurological disorders.

Gastroenterology: Issues related to the digestive system.

Oncology: Questions about different types of cancers, treatments.

Endocrinology: Topics related to hormones, diabetes, thyroid.

Orthopedics: Bone health, joint issues.

Pediatrics: Child health, vaccinations, growth and development.

Mental Health: Depression, anxiety, stress, and other mental health issues.

Women's Health: Pregnancy, menstrual health, menopause.

## Model Details

### Base Model

- **Name**: Llama-2-7B
  
### Fine-tuned Model

- **Name**: web-md-llama2-7b-3000 
- **Fine-tuned on**: Kabatubare/medical-guanaco-3000
- **Description**: This model is fine-tuned to specialize in medical dialogues and healthcare applications.

### Architecture and Training Parameters

#### Architecture

- **LoRA Attention Dimension**: 64
- **LoRA Alpha Parameter**: 16
- **LoRA Dropout**: 0.1
- **Precision**: 4-bit (bitsandbytes)
- **Quantization Type**: nf4

#### Training Parameters

- **Epochs**: 3
- **Batch Size**: 4
- **Gradient Accumulation Steps**: 1
- **Max Gradient Norm**: 0.3
- **Learning Rate**: 3e-4
- **Weight Decay**: 0.001
- **Optimizer**: paged_adamw_32bit
- **LR Scheduler**: cosine
- **Warmup Ratio**: 0.03
- **Logging Steps**: 25

## Datasets

### 

### Fine-tuning Dataset

- **Name**: Kabatubare/medical-guanaco-3000
- **Description**: This is a reduced and balanced dataset curated from a larger medical dialogue dataset using derived from 24,000 WebMD question and answer dialogue sessions . It aims to cover a broad range of medical topics and is suitable for training healthcare chatbots and conducting medical NLP research.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Yo!Medical3000")
model = AutoModelForCausalLM.from_pretrained("Yo!Medical3000")

# Use the model for inference