File size: 6,906 Bytes
7a1f0aa 08ac92e 7a1f0aa 4af927c 8aceac5 7a1f0aa 8aceac5 7a1f0aa 8aceac5 7a1f0aa 8aceac5 7a1f0aa 4af927c 7a1f0aa |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 |
---
language: zh
datasets: couplet
inference:
parameters:
max_length: 68
num_return_sequences: 1
do_sample: True
widget:
- text: "燕子归来，问昔日雕梁何处。 -"
example_title: "对联1"
- text: "笑取琴书温旧梦。 -"
example_title: "对联2"
- text: "煦煦春风，吹暖五湖四海。 -"
example_title: "对联3"
---
# 对联
## Model description
对联AI生成，给出上联，生成下联。
## How to use
使用 pipeline 调用模型:
```python
>>> # 调用微调后的模型
>>> senc="燕子归来，问昔日雕梁何处。 -"
>>> model_id="couplet-gpt2-finetuning"
>>> from transformers import BertTokenizer, GPT2LMHeadModel, TextGenerationPipeline
>>> tokenizer = BertTokenizer.from_pretrained(model_id)
>>> model = GPT2LMHeadModel.from_pretrained(model_id)
>>> text_generator = TextGenerationPipeline(model, tokenizer)
>>> text_generator.model.config.pad_token_id = text_generator.model.config.eos_token_id
>>> text_generator( senc,max_length=25, do_sample=True)
[{'generated_text': '燕子归来，问昔日雕梁何处。 - 风 儿 吹 醒 ， 叹 今 朝 烟 雨 无'}]
```
Here is how to use this model to get the features of a given text in PyTorch:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("supermy/couplet")
model = AutoModelForCausalLM.from_pretrained("supermy/couplet")
```
## Training data
此数据集基于couplet-dataset的70w条数据集，在此基础上利用敏感词词库对数据进行了过滤，删除了低俗或敏感的内容，删除后剩余约74w条对联数据。
## 统计信息
```
```
## Training procedure
模型：[GPT2](https://huggingface.co/gpt2)
训练环境：英伟达16G显卡
bpe分词："vocab_size"=50000
```
[INFO|trainer.py:1608] 2022-12-07 02:32:58,307 >> ***** Running training *****
[INFO|trainer.py:1609] 2022-12-07 02:32:58,307 >> Num examples = 260926
[INFO|trainer.py:1610] 2022-12-07 02:32:58,307 >> Num Epochs = 160
[INFO|trainer.py:1611] 2022-12-07 02:32:58,307 >> Instantaneous batch size per device = 96
[INFO|trainer.py:1612] 2022-12-07 02:32:58,307 >> Total train batch size (w. parallel, distributed & accumulation) = 96
[INFO|trainer.py:1613] 2022-12-07 02:32:58,307 >> Gradient Accumulation steps = 1
[INFO|trainer.py:1614] 2022-12-07 02:32:58,307 >> Total optimization steps = 434880
[INFO|trainer.py:1616] 2022-12-07 02:32:58,308 >> Number of trainable parameters = 124439808
[INFO|trainer.py:1637] 2022-12-07 02:32:58,309 >> Continuing training from checkpoint, will skip to saved global_step
[INFO|trainer.py:1638] 2022-12-07 02:32:58,310 >> Continuing training from epoch 93
[INFO|trainer.py:1639] 2022-12-07 02:32:58,310 >> Continuing training from global step 253500
[INFO|trainer.py:1608] 2022-11-30 12:51:36,357 >> ***** Running training *****
[INFO|trainer.py:1609] 2022-11-30 12:51:36,357 >> Num examples = 260926
[INFO|trainer.py:1610] 2022-11-30 12:51:36,357 >> Num Epochs = 81
[INFO|trainer.py:1611] 2022-11-30 12:51:36,357 >> Instantaneous batch size per device = 96
[INFO|trainer.py:1612] 2022-11-30 12:51:36,357 >> Total train batch size (w. parallel, distributed & accumulation) = 96
[INFO|trainer.py:1613] 2022-11-30 12:51:36,357 >> Gradient Accumulation steps = 1
[INFO|trainer.py:1614] 2022-11-30 12:51:36,357 >> Total optimization steps = 220158
[INFO|trainer.py:1616] 2022-11-30 12:51:36,358 >> Number of trainable parameters = 124439808
{'loss': 6.1104, 'learning_rate': 4.9888034956712906e-05, 'epoch': 0.18}
{'loss': 5.5855, 'learning_rate': 4.977448014607691e-05, 'epoch': 0.37}
{'loss': 5.3264, 'learning_rate': 4.966092533544091e-05, 'epoch': 0.55}
......
......
......
{'loss': 2.8539, 'learning_rate': 5.677740531799889e-08, 'epoch': 80.94}
{'train_runtime': 146835.0563, 'train_samples_per_second': 143.937, 'train_steps_per_second': 1.499, 'train_loss': 3.1762605669072217, 'epoch': 81.0}
***** train metrics *****
epoch = 81.0
train_loss = 3.1763
train_runtime = 1 day, 16:47:15.05
train_samples = 260926
train_samples_per_second = 143.937
train_steps_per_second = 1.499
12/02/2022 05:38:54 - INFO - __main__ - *** Evaluate ***
[INFO|trainer.py:2929] 2022-12-02 05:38:54,688 >> ***** Running Evaluation *****
[INFO|trainer.py:2931] 2022-12-02 05:38:54,688 >> Num examples = 1350
[INFO|trainer.py:2934] 2022-12-02 05:38:54,688 >> Batch size = 96
100%|██████████| 15/15 [00:03<00:00, 4.20it/s]
[INFO|modelcard.py:449] 2022-12-02 05:38:59,875 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.4447501469723692}]}
***** eval metrics *****
epoch = 81.0
eval_accuracy = 0.4448
eval_loss = 3.2813
eval_runtime = 0:00:03.86
eval_samples = 1350
eval_samples_per_second = 349.505
eval_steps_per_second = 3.883
perplexity = 26.6108
{'loss': 3.0967, 'learning_rate': 1.8027961736571009e-07, 'epoch': 159.49}
{'loss': 3.0922, 'learning_rate': 1.227924944812362e-07, 'epoch': 159.68}
{'loss': 3.0934, 'learning_rate': 6.530537159676233e-08, 'epoch': 159.86}
{'train_runtime': 120967.2394, 'train_samples_per_second': 345.12, 'train_steps_per_second': 3.595, 'train_loss': 1.3456422273861828, 'epoch': 160.0}
***** train metrics *****
epoch = 160.0
train_loss = 1.3456
train_runtime = 1 day, 9:36:07.23
train_samples = 260926
train_samples_per_second = 345.12
train_steps_per_second = 3.595
12/08/2022 12:09:08 - INFO - __main__ - *** Evaluate ***
[INFO|trainer.py:2929] 2022-12-08 12:09:08,522 >> ***** Running Evaluation *****
[INFO|trainer.py:2931] 2022-12-08 12:09:08,522 >> Num examples = 1350
[INFO|trainer.py:2934] 2022-12-08 12:09:08,522 >> Batch size = 96
100%|██████████| 15/15 [00:03<00:00, 4.16it/s]
[INFO|modelcard.py:449] 2022-12-08 12:09:13,448 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.433615520282187}]}
***** eval metrics *****
epoch = 160.0
eval_accuracy = 0.4336
eval_loss = 3.3005
eval_runtime = 0:00:03.93
eval_samples = 1350
eval_samples_per_second = 343.164
eval_steps_per_second = 3.813
perplexity = 27.1257
```
|