Update README.md
Browse files
README.md
CHANGED
@@ -1,71 +1,80 @@
|
|
1 |
-
---
|
2 |
-
license: gpl-3.0
|
3 |
-
|
4 |
-
|
5 |
-
|
6 |
-
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
23 |
-
model
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
42 |
-
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
-
|
48 |
-
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
|
53 |
-
|
54 |
-
|
55 |
-
|
56 |
-
|
57 |
-
|
58 |
-
|
59 |
-
|
60 |
-
|
61 |
-
|
62 |
-
|
63 |
-
|
64 |
-
|
65 |
-
|
66 |
-
|
67 |
-
|
68 |
-
|
69 |
-
|
70 |
-
|
71 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: gpl-3.0
|
3 |
+
datasets:
|
4 |
+
- wmt/wmt19
|
5 |
+
language:
|
6 |
+
- en
|
7 |
+
- zh
|
8 |
+
base_model: Mxode/NanoLM-365M-Base
|
9 |
+
pipeline_tag: translation
|
10 |
+
tags:
|
11 |
+
- text-generation-inference
|
12 |
+
---
|
13 |
+
# NanoTranslator-365M-immersive_translate-v1
|
14 |
+
|
15 |
+
## Introduction
|
16 |
+
|
17 |
+
NanoTranslator-365M-immersive_translate-v1 is a translation model specifically designed for **Chinese-English bilingual** translation, trained with 200M data from the [wmt-19](https://huggingface.co/datasets/wmt/wmt19) dataset, based on [NanoLM-365M-Base](https://huggingface.co/Mxode/NanoLM-365M-Base).
|
18 |
+
|
19 |
+
This model is trained following the Immersive Translate prompt format and can be deployed as an OpenAI format interface using tools like vllm and lmdeploy for utilization.
|
20 |
+
|
21 |
+
## How to use
|
22 |
+
|
23 |
+
Below is a method to call the model using transformers. The prompt follows the immersive translation format to ensure optimal results.
|
24 |
+
|
25 |
+
```python
|
26 |
+
import torch
|
27 |
+
from typing import Literal
|
28 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
29 |
+
|
30 |
+
model_path = 'Mxode/NanoTranslator-365M-immersive_translate-v1'
|
31 |
+
|
32 |
+
model = AutoModelForCausalLM.from_pretrained(model_path).to('cuda:0', torch.bfloat16)
|
33 |
+
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
34 |
+
|
35 |
+
def translate(
|
36 |
+
text: str,
|
37 |
+
to: Literal["chinese", "english"] = "chinese",
|
38 |
+
**kwargs
|
39 |
+
):
|
40 |
+
generation_args = dict(
|
41 |
+
max_new_tokens = kwargs.pop("max_new_tokens", 512),
|
42 |
+
do_sample = kwargs.pop("do_sample", True),
|
43 |
+
temperature = kwargs.pop("temperature", 0.35),
|
44 |
+
top_p = kwargs.pop("top_p", 0.8),
|
45 |
+
top_k = kwargs.pop("top_k", 40),
|
46 |
+
**kwargs
|
47 |
+
)
|
48 |
+
|
49 |
+
prompt = """Translate the following source text to {to}. Output translation directly without any additional text.
|
50 |
+
Source Text: {text}
|
51 |
+
|
52 |
+
Translated Text:"""
|
53 |
+
|
54 |
+
messages = [
|
55 |
+
{"role": "system", "content": "You are a professional, authentic machine translation engine."},
|
56 |
+
{"role": "user", "content": prompt.format(to=to, text=text)}
|
57 |
+
]
|
58 |
+
inputs = tokenizer.apply_chat_template(
|
59 |
+
messages,
|
60 |
+
tokenize=False,
|
61 |
+
add_generation_prompt=True
|
62 |
+
)
|
63 |
+
model_inputs = tokenizer([inputs], return_tensors="pt").to(model.device)
|
64 |
+
|
65 |
+
generated_ids = model.generate(model_inputs.input_ids, **generation_args)
|
66 |
+
generated_ids = [
|
67 |
+
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
68 |
+
]
|
69 |
+
|
70 |
+
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
71 |
+
return response
|
72 |
+
|
73 |
+
text = "After a long day at work, I love to unwind by cooking a nice dinner and watching my favorite TV series. It really helps me relax and recharge for the next day."
|
74 |
+
response = translate(text=text, to='chinese')
|
75 |
+
print(f'Translation: {response}')
|
76 |
+
|
77 |
+
"""
|
78 |
+
Translation: 工作了一天,我喜欢吃一顿美味的晚餐,看我最喜欢的电视剧,这样做有助于我放松,补充能量。
|
79 |
+
"""
|
80 |
+
```
|