instruct model - initial commit

README.md

@@ -1,3 +1,315 @@
----
-license: apache-2.0
----
+---
+pipeline_tag: text-generation
+inference: false
+license: apache-2.0
+# datasets:
+# metrics:
+# - code_eval
+library_name: transformers
+tags:
+- language
+- granite-3.0
+model-index:
+- name: granite-3.0-3b-a800m-instruct
+  results:
+  - task:
+      type: text-generation
+    dataset:
+        type: human-exams
+        name: MMLU
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: human-exams
+        name: MMLU-Pro
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: human-exams
+        name: AGI-Eval
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: commonsense
+        name: WinoGrande
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: commonsense
+        name: OBQA 
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: commonsense
+        name: SIQA
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: commonsense
+        name: PIQA
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: commonsense
+        name: Hellaswag
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: commonsense
+        name: TruthfulQA
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: reading-comprehension
+        name: BoolQ
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: reading-comprehension
+        name: SQuAD v2
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: reasoning
+        name: ARC-C
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: reasoning
+        name: GPQA
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: reasoning
+        name: BBH
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: code
+        name: HumanEval
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+  - task:
+      type: text-generation
+    dataset:
+        type: code
+        name: MBPP
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false     
+  - task:
+      type: text-generation
+    dataset:
+        type: math
+        name: GSM8K
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false 
+  - task:
+      type: text-generation
+    dataset:
+        type: math
+        name: MATH
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false   
+  - task:
+      type: text-generation
+    dataset:
+        type: multilingual
+        name: MGSM
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 
+      veriefied: false
+---
+
+<!-- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/62cd5057674cdb524450093d/1hzxoPwqkBJXshKVVe6_9.png) -->
+
+# Granite-3.0-3B-A800M-Instruct
+
+## Model Summary
+**Granite-3.0-3B-A800M-Instruct** is a lightweight and open-source 3B parameter model fine tuned from *Granite-3.0-3B-A800M-Base-4K* on a combination of open-source and proprietary instruction data with a **permissively licensed**. This language model is designed to excel in instruction following tasks such as summarization, problem-solving, text translation, reasoning, code tasks, funcion-calling, and more.
+
+- **Developers:** IBM Research
+- **GitHub Repository:** [ibm-granite/granite-language-models](https://github.com/ibm-granite/granite-language-models)
+- **Website**: [Granite Docs](https://www.ibm.com/granite/docs/)
+- **Paper:** [Granite Language Models](https://) <!--     TO DO: Update github repo link when it is ready -->
+- **Release Date**: October 21st, 2024
+- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).
+
+## Supported Languages
+English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese (Simplified) 
+
+## Usage
+### Intended use
+The model is designed to respond to general instructions and can be used to build AI assistants for multiple domains, including bussiness applications.
+
+### Capabilities
+* Summarization
+* Text classification
+* Text extraction
+* Question-answering
+* Retrieval Augmented Generation (RAG)
+* Code related 
+* Function-calling
+* Multilingual dialog use cases
+
+### Generation
+This is a simple example of how to use **Granite-3.0-3B-A800M-Instruct** model.
+
+Install the following libraries:
+
+```shell
+pip install torch torchvision torchaudio
+pip install accelerate
+pip install transformers
+```
+Then, copy the snippet from the section that is relevant for your usecase.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+device = "auto"
+model_path = "ibm-granite/granite-3.0-3b-a800m-instruct"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+# drop device_map if running on CPU
+model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
+model.eval()
+# change input text as desired
+chat = [
+    { "role": "user", "content": "Please list one IBM Research laboratory located in the United States. You should only output its name and location." },
+]
+chat = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
+# tokenize the text
+input_tokens = tokenizer(chat, return_tensors="pt").to(device)
+# generate output tokens
+output = model.generate(**input_tokens, 
+                        max_new_tokens=100)
+# decode output tokens into text
+output = tokenizer.batch_decode(output)
+# print output
+print(output)
+```
+
+<!-- TO DO: function-calling-example
+ -->
+
+<!-- ['<|start_of_role|>user<|end_of_role|>Please list one IBM Research laboratory located in the United States. You should only output its name and location.<|end_of_text|>\n<|start_of_role|>assistant<|end_of_role|>1. IBM Research - Almaden, San Jose, California<|end_of_text|>'] -->
+
+## Model Architeture
+**Granite-3.0-3B-A800M-Instruct** is based on a decoder-only sparse Mixture of Experts(MoE) transformer architecture. Core components of this architecture are: Fine-grained Experts, Dropless Token Routing, and Load Balancing Loss.
+
+| Model                        | 2B Dense | 8B Dense | 1B MoE   | 3B MoE       |
+| :--------                    | :--------| :--------| :--------| :--------    |
+| Embedding size               | 2048     | 4096     | 1024     | **1536**     |
+| Number of layers             | 40       | 40       | 24       | **32**       |
+| Attention head size          | 64       | 128      | 64       | **64**       |
+| Number of attention heads    | 32       | 32       | 16       | **24**       |
+| Number of KV heads           | 8        | 8        | 8        | **8**        |
+| MLP hidden size              | 8192     | 12800    | 512      | **512**      |
+| MLP activation               | SwiGLU   | SwiGLU   | SwiGLU   | **SwiGLU**   |
+| Number of Experts            | —        | —        | 32       | **40**       |
+| MoE TopK                     | —        | —        | 8        | **8**        |
+| Initialization std           | 0.1      | 0.1      | 0.1      | **0.1**      |
+| Sequence Length              | 4096     | 4096     | 4096     | **4096**     |
+| Position Embedding           | RoPE     | RoPE     | RoPE     | **RoPE**     |
+| # Paremeters                 | 2.5B     | 8.1B     | 1.3B     | **3.3B**     |
+| # Active Parameters          | 2.5B     | 8.1B     | 400M     | **800M**     |
+| # Training tokens            | 12T      | 12T      | 10T      | **10T**      |
+
+<!-- TO DO: To be completed once the paper is ready, we may changed title to Supervised Finetuning -->
+## Training Data
+This model is trained on a mix of open-source and proprietary datasets.
+<!-- ### Instruction Datasets
+* Language Instruction Datasets: We include high-quality datasets such as [TO DO: List of datasets]
+* Synthetic Instruction Datasets: [TO DO: paragraph about synthetic data]
+### Processing
+* [TO DO: Data annotation with MagPie pipeline: quality, duplicates] -->
+
+## Infrastructure
+We train the Granite Language models using IBM's super computing cluster, Blue Vela, which is outfitted with NVIDIA H100 GPUs. This cluster provides a scalable and efficient infrastructure for training our models over thousands of GPUs.
+
+## Ethical Considerations and Limitations
+Granite instruct models are primarily finetuned using instruction-response pairs mostly in English, but also in German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese (Simplified). As this model has been exposed to multilingual data, it can handle multilingual dialog use cases with a limited performance in non-English tasks. In such case, introducing a small number of examples (few-shot) can help the model in generating more accurate outputs. The model also inherits ethical considerations and limitations from its base model. For more information, please refer to *[Granite-3.0-3B-A800M-Base-4K](https://huggingface.co/ibm-granite/granite-3.0-3b-a800m-base)* model card.