surpassone/Llama-3-SURPASSONE-JP-8B

Llama-3-SURPASSONE-JP-8B

Model Description

Llama-3-SURPASSONE-JP-8B is a large language model trained by SURPASSONE, Inc. Based on meta-llama/Meta-Llama-3-8B-Instruct, this model has undergone additional post-training of Japanese to expand instruction-following capabilities in Japanese.

This model is specialized in generating MCQ questions with options, correct answers and corresponding explanations on “Nursing Care” if given a specific topic.

For more details, please refer to our blog post.

Usage

# make sure you are logged in huggingface
hf_token = "" # your huggingface token
from huggingface_hub import login
login()

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig, AutoTokenizer

bnb_config = BitsAndBytesConfig(
   load_in_4bit=True,
   bnb_4bit_use_double_quant=True,
   bnb_4bit_quant_type="nf4",
   bnb_4bit_compute_dtype=torch.bfloat16,
)

model_id = "surpassone/Llama-3-SURPASSONE-JP-8B"
            
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
   model_id,
   device_map="auto",
   quantization_config=None, # Use bnb_config, if need to use 4 bit quantization else None
)
if tokenizer.pad_token is None:
   tokenizer.pad_token = tokenizer.eos_token

model.eval()

# for MCQ set generation

alpaca_prompt = """以下は、タスクを説明する指示と、さらに詳しいコンテキストを提供する入力を組み合わせたものです。要求を適切に完了する応答を記述してください。

### 説明書:
{}

### 入力:
{}

### 応答:
{}"""

EOS_TOKEN = "<|endoftext|>"  # Define the EOS token, adjust according to your tokenizer

inputs = tokenizer(
[
   alpaca_prompt.format(
       "次のトピックに関する複数選択問題を生成します。", # instruction
       "介護：体の仕組み", # input
       "", # output - leave this blank for generation!
   )
], return_tensors = "pt").to("cuda")

from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer)
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 1028)

Developers

Listed in alphabetical order.

License

Meta Llama 3 Community License

How to Cite

@misc{surpassonellama2024,
      title={surpassone/Llama-3-SURPASSONE-JP-8B},
      url={https://huggingface.co/surpassone/Llama-3-SURPASSONE-JP-8B},
      author={Mustain Billah and Shugo Saito and Leo Uno},
      year={2024},
}

Citations

@article{llama3modelcard,
    title={Llama 3 Model Card},
    author={AI@Meta},
    year={2024},
    url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md}
}