Upload README.md
Browse files
README.md
ADDED
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# PhoGPT: Generative Pre-training for Vietnamese
|
2 |
+
|
3 |
+
|
4 |
+
We open-source a state-of-the-art 4B-parameter generative model series for Vietnamese, which includes the base pre-trained monolingual model PhoGPT-4B and its chat variant, PhoGPT-4B-Chat. The base model, PhoGPT-4B, with exactly 3.7B parameters, is pre-trained from scratch on a Vietnamese corpus of 102B tokens, with an 8192 context length, employing a vocabulary of 20480 token types. The chat variant, PhoGPT-4B-Chat, is the modeling output obtained by fine-tuning PhoGPT-4B on a dataset of 70K instructional prompts and their responses, along with an additional 290K conversations. We demonstrate its strong performance compared to previous closed-source and open-source 7B-parameter models. More details about the general architecture and experimental results of PhoGPT can be found in our [technical report](https://arxiv.org/abs/2311.02945):
|
5 |
+
|
6 |
+
```
|
7 |
+
@article{PhoGPT,
|
8 |
+
title = {{PhoGPT: Generative Pre-training for Vietnamese}},
|
9 |
+
author = {Dat Quoc Nguyen and Linh The Nguyen and Chi Tran and Dung Ngoc Nguyen and Dinh Phung and Hung Bui},
|
10 |
+
journal = {arXiv preprint},
|
11 |
+
volume = {arXiv:2311.02945},
|
12 |
+
year = {2023}
|
13 |
+
}
|
14 |
+
```
|
15 |
+
|
16 |
+
**Please CITE** our technical report when PhoGPT is used to help produce published results or is incorporated into other software.
|
17 |
+
|
18 |
+
For further information or requests, please go to [PhoGPT's homepage](https://github.com/VinAIResearch/PhoGPT)!
|