Update README.md

README.md

@@ -1,94 +1,131 @@
 ---
 library_name: transformers
-tags: []
+license: mit
+language:
+- en
 ---
 
 # Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
+https://huggingface.co/rezahf2024/fine_tuned_financial_setiment_analysis_gpt2_model
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+This a fine-tuned GPT2 model on the https://huggingface.co/datasets/FinGPT/fingpt-sentiment-train dataset for the down-stream financial sentiment analysis. 
+
+label_mapping = {
+'LABEL_0': 'mildly positive',
+'LABEL_1': 'mildly negative',
+'LABEL_2': 'moderately negative',
+'LABEL_3': 'moderately positive',
+'LABEL_4': 'positive',
+'LABEL_5': 'negative',
+'LABEL_6': 'neutral',
+'LABEL_7': 'strong negative',
+'LABEL_8': 'strong positive'
+}
+
+- **Developed by:** Rezaul Karim, Ph.D.
+- **Funded by [optional]:** Self
+- **Shared by [optional]:** Rezaul Karim, Ph.D.
+- **Model type:** Fine-tuned GPT2
+- **Language(s) (NLP):** financial sentiment analysis
+- **License:** MIT
+- **Finetuned from model [optional]:** https://huggingface.co/datasets/mteb/tweet_sentiment_extraction
 
 ### Model Sources [optional]
 
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Repository:** https://github.com/rezacsedu/financial_sentiment_analysis_LLM
+- **Paper [optional]:** on the way
+- **Demo [optional]:** on the way
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+The model is already fine-tuned for downstream financial sentiment analysis tasks. 
 
-### Direct Use
+## How to Get Started with the Model
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+Use the code below to get started with the model.
 
 [More Information Needed]
 
-### Downstream Use [optional]
+## Training Details
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Training Data
 
-[More Information Needed]
+from transformers import GPT2Tokenizer
 
-### Out-of-Scope Use
+dataset = load_dataset("FinGPT/fingpt-sentiment-train")
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
+tokenizer.pad_token = tokenizer.eos_token
 
-[More Information Needed]
+def tokenize_function(examples):
+   return tokenizer(examples["input"], padding="max_length", truncation=True)
 
-## Bias, Risks, and Limitations
+tokenized_datasets = dataset.map(tokenize_function, batched=True)
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+from datasets import DatasetDict
+import random
+import string
 
-[More Information Needed]
+def generate_random_id():
+  return ''.join(random.choices(string.ascii_lowercase + string.digits, k=10))
 
-### Recommendations
+unique_outputs = set(dataset['train']['output'])
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+#label_mapping = {'mildly positive': 0, 'positive': 1, 'strong positive':2, 'moderately positive': 3, 'negative': 4, 'neutral': 5}  # Add more mappings as needed
+label_mapping = {label: index for index, label in enumerate(unique_outputs)}
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+def transform_dataset(dataset):
+  dataset = dataset.rename_column('input', 'text')
+  dataset = dataset.rename_column('output', 'label_text')
 
-## How to Get Started with the Model
+  dataset = dataset.remove_columns(['instruction'])
 
-Use the code below to get started with the model.
+  dataset = dataset.add_column('id', [generate_random_id() for _ in range(dataset.num_rows)])
+  dataset = dataset.add_column('label', [label_mapping[label_text] for label_text in dataset['label_text']])
 
-[More Information Needed]
+  return dataset
 
-## Training Details
+transformed_dataset = DatasetDict({'train': transform_dataset(tokenized_datasets['train'])})
+transformed_dataset['train'].set_format(type=None, columns=['id', 'text', 'label', 'label_text', 'input_ids', 'attention_mask'])
 
-### Training Data
+train_test_split = transformed_dataset['train'].train_test_split(test_size=0.3, seed=42)
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+tokenized_datasets['test'] = train_test_split['test']
+tokenized_datasets['train'] = train_test_split['train']
 
-[More Information Needed]
+small_train_dataset = tokenized_datasets["train"].shuffle(seed=42).select(range(100))
+small_eval_dataset = tokenized_datasets["test"].shuffle(seed=42).select(range(100))
 
-### Training Procedure 
+### Fine-tune Procedure 
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+from transformers import GPT2ForSequenceClassification
+from transformers import TrainingArguments, Trainer
 
-#### Preprocessing [optional]
+model = GPT2ForSequenceClassification.from_pretrained("gpt2", num_labels=9)
 
-[More Information Needed]
+training_args = TrainingArguments(
+   output_dir="test_trainer",
+   #evaluation_strategy="epoch",
+   per_device_train_batch_size=1,  # Reduce batch size here
+   per_device_eval_batch_size=1,    # Optionally, reduce for evaluation as well
+   gradient_accumulation_steps=4
+   )
+
+trainer = Trainer(
+   model=model,
+   args=training_args,
+   train_dataset=small_train_dataset,
+   eval_dataset=small_eval_dataset,
+   compute_metrics=compute_metrics,
+)
 
+trainer.train()
+trainer.evaluate()
+trainer.save_model("fine_tuned_finsetiment_model")
 
 #### Training Hyperparameters
 
@@ -102,27 +139,15 @@ Use the code below to get started with the model.
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+import evaluate
 
-### Testing Data, Factors & Metrics
+metric = evaluate.load("accuracy")
 
-#### Testing Data
+def compute_metrics(eval_pred):
+   logits, labels = eval_pred
+   predictions = np.argmax(logits, axis=-1)
 
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+   return metric.compute(predictions=predictions, references=labels)
 
 ### Results
 
@@ -130,44 +155,6 @@ Use the code below to get started with the model.
 
 #### Summary
 
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
 ## Citation [optional]
 
 <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
@@ -176,26 +163,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 [More Information Needed]
 
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
 
 ## Model Card Contact
 
-[More Information Needed]
-
-
+rezaul.karim.fit@gmail.com