File size: 3,678 Bytes
64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6984dd d6f6457 64685a1 d6984dd 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 d6f6457 64685a1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 |
---
tags:
- text-generation
license: cc-by-nc-sa-4.0
language:
- ko
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
pipeline_tag: text-generation
datasets:
- beomi/KoAlpaca-v1.1a
- jojo0217/korean_rlhf_dataset
- kyujinpy/OpenOrca-KO
- nlpai-lab/kullm-v2
---
# **DataVortexTL-1.1B-v0.1**
<img src="./DataVortex.png" alt="DataVortex" style="height: 8em;">
## **Model Details**
### **Base Model**
[TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
### **Trained On**
- **OS**: Ubuntu 20.04
- **GPU**: H100 80GB 1ea
- **transformers**: v4.36.2
### **Dataset**
- [beomi/KoAlpaca-v1.1a](https://huggingface.co/datasets/beomi/KoAlpaca-v1.1a)
- [jojo0217/korean_rlhf_dataset](https://huggingface.co/datasets/jojo0217/korean_rlhf_dataset)
- [kyujinpy/OpenOrca-KO](https://huggingface.co/datasets/kyujinpy/OpenOrca-KO)
- [nlpai-lab/kullm-v2](https://huggingface.co/datasets/nlpai-lab/kullm-v2)
### **Instruction format**
It follows **TinyLlama** format.
E.g.
```python
text = """\
<|system|>
당신은 사람들이 정보를 찾을 수 있도록 도와주는 인공지능 비서입니다.</s>
<|user|>
대한민국의 수도는 어디야?</s>
<|assistant|>
대한민국의 수도는 서울입니다.</s>
<|user|>
서울 인구는 총 몇 명이야?</s>
"""
```
## **Model Benchmark**
### **[Ko LM Eval Harnesss](https://github.com/Beomi/ko-lm-evaluation-harness)**
On Benchmarking ...
| Task | 0-shot | 5-shot | 10-shot | 50-shot |
| :--------------- | -----: | -----: | ------: | ------: |
| kobest_boolq | 0.0 | 0.0 | 0.0 | 0.0 |
| kobest_copa | 0.0 | 0.0 | 0.0 | 0.0 |
| kobest_hellaswag | 0.0 | 0.0 | 0.0 | 0.0 |
| kobest_sentineg | 0.0 | 0.0 | 0.0 | 0.0 |
### **[Ko-LLM-Leaderboard](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)**
On Benchmarking ...
| Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
| ------: | -----: | -----------: | ------: | ------------: | --------------: |
| 0 | 0 | 0 | 0 | 0 | 0 |
## **Implementation Code**
This model contains the chat_template instruction format.
You can use the code below.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto
model = AutoModelForCausalLM.from_pretrained("Edentns/DataVortexTL-1.1B-v0.1")
tokenizer = AutoTokenizer.from_pretrained("Edentns/DataVortexTL-1.1B-v0.1")
messages = [
{"role": "system", "content": "당신은 사람들이 정보를 찾을 수 있도록 도와주는 인공지능 비서입니다."},
{"role": "user", "content": "대한민국의 수도는 어디야?"},
{"role": "assistant", "content": "대한민국의 수도는 서울입니다."},
{"role": "user", "content": "서울 인구는 총 몇 명이야?"}
]
encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
model_inputs = encodeds.to(device)
model.to(device)
generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])
```
## **License**
The model is licensed under the [cc-by-nc-sa-4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) license, which allows others to copy, modify, and share the work non-commercially, as long as they give appropriate credit and distribute any derivative works under the same license.
<div align="center">
<a href="https://edentns.com/">
<img src="./Logo.png" alt="Logo" style="height: 3em;">
</a>
</div>
|