metadata

language:
  - it
library_name: transformers
tags:
  - pretrained
  - biomedical
  - text-generation
  - medical
base_model: sapienzanlp/Minerva-350M-base-v1.0
datasets:
  - IVN-RIN/BioBERT_Italian
pipeline_tag: text-generation
widget:
  - text: 'I batteri della famiglia Bacteroides sono importanti per '
    example_title: Example 1
license: apache-2.0
extra_gated_prompt: >-
  This is a pretrained model that should be fine-tuned to perform downstream
  tasks. You agree to not use the model to conduct experiments that cause harm
  to human subjects, or to perform any medical-related task.
extra_gated_fields:
  Company: text
  Country: country
  Specific date: date_picker
  I want to use this model for:
    type: select
    options:
      - Research
      - Education
      - label: Other
        value: other
  I agree to use this model for non-commercial use ONLY: checkbox
  I have read and unsderstood the 'Bias, Risk, and Limitation' section of the model card: checkbox
  geo: ip_location
extra_gated_heading: Acknowledge terms and conditions to accept the repository
extra_gated_description: Our team may take 2-3 days to process your request
extra_gated_button_content: Acknowledge

Model Card for Igea-350M-v0.0.1 ⚕️🩺

Igea is a biomedical Small Language Model (SLM) for Italian, continually pretrained from Minerva with NMT translated Pubmed Abstracts

🔓: Access to the model is only granted after explicitly acknowledging that you have read the 'Bias, Risk, and Limitation' section of this model card.

This is ongoing research. Do not use it for any medical-related tasks.

Preprint: Igea: a Decoder-Only Language Model for Biomedical Text Generation in Italian.

How to use Igea with Hugging Face transformers

import transformers
import torch

model_id = "bmi-labmedinfo/Igea-350M-v0.1"

# Initialize the pipeline.
pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

# Input text for the model.
input_text = "Il fegato è "

# Compute the outputs.
output = pipeline(
  input_text,
  max_new_tokens=128,
)

# Output:
# [{'generated_text': "Il fegato è una ghiandola fondamentale per il metabolismo umano, la più [...]"}]

🚨⚠️🚨 Bias, Risks, and Limitations 🚨⚠️🚨

This section identifies foreseeable harms and misunderstandings.

This is a continued pretraining of a foundation model, not subject to alignment. Model may:

Overrepresent some viewpoints and underrepresent others
Contain stereotypes
Contain personal information
Generate:
- Racist and sexist content
- Hateful, abusive, or violent language
- Discriminatory or prejudicial language
- Content that may not be appropriate for all settings, including sexual content
Make errors, including producing incorrect information or historical facts as if it were factual
Generate irrelevant or repetitive outputs

We are aware of the biases and potential problematic/toxic content that current pretrained large language models exhibit: more specifically, as probabilistic models of (Italian and English) languages, they reflect and amplify the biases of their training data.

The biomedical setting poses additional threats, including:

Disparities in research focus, demographic representation, and reporting standards
Reinforcement of existing medical paradigms and overlook emerging or alternative viewpoints, hindering innovation and comprehensive care
Generation of incorrect information and false claims, potentially leading to incorrect medical decisions

This model is therefore not intended to be used as it is for any medical-related task.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

Evaluation

Dataset	Domain	Minerva 3B (best base)	Igea 350M	Igea 1B	Igea 3B
MedMCQA-ITA (0-shot)	Biomed	0.293	0.250	0.307	0.313
Hellaswag-IT (0-shot)	General	0.519	0.303	0.357	0.491
ARC-IT (0-shot)	General	0.305	0.244	0.270	0.287
MMLU-IT (5-shot)	General	0.261	0.254	0.255	0.252

Credits

Developed by Tommaso M. Buonocore and Simone Rancati.

Thanks to Michele Montebovi for his precious advices.