|
--- |
|
library_name: transformers |
|
tags: |
|
- unsloth |
|
- llama3 |
|
- indonesia |
|
license: llama3 |
|
datasets: |
|
- catinthebag/TumpengQA |
|
language: |
|
- id |
|
--- |
|
<center> |
|
<img src="https://imgur.com/9nG5J1T.png" alt="Kancil" width="600" height="300"> |
|
<p><em>Kancil is a fine-tuned version of Llama 3 8B using synthetic QA dataset generated with Llama 3 70B.</em></p> |
|
</center> |
|
|
|
### Introducing the Kancil family of open models |
|
|
|
Selamat datang! |
|
|
|
I'm super stoked to announce... the 🦌 Kancil! It's a fine-tuned version of Llama 3 8B with the TumpengQA, an instruction dataset of 28 million words. Both the model and dataset is openly available in Huggingface. |
|
|
|
📚 The dataset was synthetically generated from Llama 3 70B. A big problem with existing Indonesian instruction dataset is they're really badly translated versions of English datasets. Llama 3 70B can generate fluent Indonesian! (with minor caveats 😔) |
|
|
|
🦚 This was highly inspired by last year's efforts from Merak-7B, a collection of open, fine-tuned Indonesian models. However, Kancil leveraged synthetic data in a very creative way, which makes it unique from Merak! |
|
|
|
### Version 0.0 |
|
|
|
This is the very first working prototype, Kancil V0. It supports basic QA functionalities only. Currently, you cannot chat with it. |
|
|
|
This model was fine-tuned with QLoRA using the amazing Unsloth framework! It was built on top of [unsloth/llama-3-8b-bnb-4bit](https://huggingface.co/unsloth/llama-3-8b-bnb-4bit) and subsequently merged with the adapter back to 4 bit (no visible difference with merging back to fp 16). |
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
This model is developed with research purposes for researchers or general AI hobbyists. However, it has one big application: You can have lots of fun with it! |
|
|
|
### Out-of-Scope Use |
|
|
|
This is a minimally-functional research preview model with no safety curation. Do not use this model for commercial or practical applications. |
|
|
|
You are also not allowed to use this model without having fun. |
|
|
|
## Getting started |
|
|
|
As mentioned, this model was trained with Unsloth. Please use its code for better experience. |
|
|
|
``` |
|
# Install dependencies |
|
%%capture |
|
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git" |
|
!pip install --no-deps "xformers<0.0.26" trl peft accelerate bitsandbytes |
|
``` |
|
|
|
``` |
|
# Load the model |
|
from unsloth import FastLanguageModel |
|
import torch |
|
|
|
model, tokenizer = FastLanguageModel.from_pretrained( |
|
model_name = "catinthebag/Kancil-V0-llama3", |
|
max_seq_length = max_seq_length, |
|
dtype = torch.bfloat16, # Will default to float 16 if not available |
|
load_in_4bit = True, |
|
) |
|
``` |
|
``` |
|
# This model was trained on this specific prompt template. Changing it might lead to performance degradations. |
|
prompt_template = """User: {prompt} |
|
Asisten: {response}""" |
|
|
|
EOS_TOKEN = tokenizer.eos_token |
|
def formatting_prompts_func(examples): |
|
inputs = examples["prompt"] |
|
outputs = examples["response"] |
|
texts = [] |
|
for input, output in zip(inputs, outputs): |
|
text = prompt_template.format(prompt=input, response=output) + EOS_TOKEN |
|
texts.append(text) |
|
return { "text" : texts, } |
|
pass |
|
``` |
|
``` |
|
# Start generating! |
|
FastLanguageModel.for_inference(model) |
|
inputs = tokenizer( |
|
[ |
|
prompt_template.format( |
|
prompt="Apa itu generative AI?", |
|
response="", |
|
) |
|
], return_tensors = "pt").to("cuda") |
|
|
|
outputs = model.generate(**inputs, max_new_tokens = 128, temperature=.8, use_cache = True) |
|
print(tokenizer.batch_decode(outputs)[0]) |
|
``` |
|
|
|
**Note:** There was an issue with the dataset such that newline characters are printed as string literals. Sorry about that! |
|
|
|
## Acknowledgments |
|
|
|
- **Developed by:** Afrizal Hasbi Azizy |
|
- **Funded by [optional]:** DF Labs (dflabs.id) |
|
- **License:** Llama 3 Community License Agreement |
