README.md

---
library_name: transformers
tags: []
---

## Llama-3-SURPASSONE-JP-8B

![Llama-3-SURPASSONE-JP-8B-image](./visual.png)

### Model Description

**Llama-3-SURPASSONE-JP-8B** is a large language model trained by [SURPASSONE, Inc](https://surpassone.com/). 
Based on [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), this model has undergone additional post-training of Japanese to expand instruction-following capabilities in Japanese.

This model is specialized in generating MCQ questions with options, correct answers and corresponding explanations on “Nursing Care” if given a specific topic. Also, it can generate answers based on questions on “Nursing Care”.

For more details, please refer to [our blog post](https://docs.google.com/document/d/1ENAEzgV3n-sFiSoV3oQBTgzjeyTfmL64zEczepTKEW0/edit?usp=sharing).

### Usage

```python
# make sure you are logged in huggingface
hf_token = "" # your huggingface token
from huggingface_hub import login
login()

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig, AutoTokenizer

bnb_config = BitsAndBytesConfig(
   load_in_4bit=True,
   bnb_4bit_use_double_quant=True,
   bnb_4bit_quant_type="nf4",
   bnb_4bit_compute_dtype=torch.bfloat16,
)

model_id = "surpassone/Llama-3-SURPASSONE-JP-8B"
            
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
   model_id,
   device_map="auto",
   quantization_config=None, # Use bnb_config, if need to use 4 bit quantization else None
)
if tokenizer.pad_token is None:
   tokenizer.pad_token = tokenizer.eos_token

model.eval()

# for MCQ set generation

alpaca_prompt = """以下は、タスクを説明する指示と、さらに詳しいコンテキストを提供する入力を組み合わせたものです。要求を適切に完了する応答を記述してください。

### 説明書:
{}

### 入力:
{}

### 応答:
{}"""

EOS_TOKEN = "<|endoftext|>"  # Define the EOS token, adjust according to your tokenizer

inputs = tokenizer(
[
   alpaca_prompt.format(
       "次のトピックに関する複数選択問題を生成します。", # instruction
       "介護：体の仕組み", # input
       "", # output - leave this blank for generation!
   )
], return_tensors = "pt").to("cuda")

from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer)
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 1028)

#for QA  generation

# Define the formatting function and the prompt template
alpaca_prompt = """以下は質問です。質問に適切に答える回答を書いてください。

### 質問：
{}

### 答え：
{}"""

eos_token_id = tokenizer.eos_token_id

inputs = tokenizer(
    [alpaca_prompt.format(
        "介護福祉士はどのような責任を負うべきですか？",  # Question
        ""  # Answer - leave this blank for generation!
    )],
    return_tensors="pt"
).to("cuda")

from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer)
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 1028)
```

### Developers

Listed in alphabetical order.

- [Leo Uno](https://huggingface.co/leouno12)
- [Mustain Billah](https://huggingface.co/Mustain)
- [Shugo Saito](https://huggingface.co/shugo3110)


### License

[Meta Llama 3 Community License](https://llama.meta.com/llama3/license/)

### How to Cite

```tex
@misc{surpassonellama2024,
      title={surpassone/Llama-3-SURPASSONE-JP-8B},
      url={https://huggingface.co/surpassone/Llama-3-SURPASSONE-JP-8B},
      author={Mustain Billah and Shugo Saito and Leo Uno},
      year={2024},
}
```

### Citations

```tex
@article{llama3modelcard,
    title={Llama 3 Model Card},
    author={AI@Meta},
    year={2024},
    url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md}
}
```