File size: 2,263 Bytes
2b07816 22b6dd6 6626a6c 22b6dd6 2b07816 24f1f64 e55698d 24f1f64 22b6dd6 24f1f64 22b6dd6 24f1f64 22b6dd6 24f1f64 22b6dd6 24f1f64 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
---
language: en
widget:
- text: "Arts"
example_title: "Arts"
- text: "Space"
example_title: "Space"
- text: "Time"
example_title: "Time"
tags:
- text-generation
- creative-writing
- essay-writing
inference:
max_length: 400
num_beams: 10
early_stopping: true
temperature: 0.3
no_repeat_ngram_size: 2
num_return_sequences: 2
---
Introduction:
This repository contains a finetuned DistilGPT2 model for generating diverse essays on topics spanning Arts, Science, and Culture.
The model has been trained on a dataset of over 2000 high-quality essays written by human experts, covering a wide range of opinions and knowledge.
Dataset:
The training dataset comprises 2000+ essays covering diverse topics in Arts, Science, and Culture. These essays are written by human experts and contain a diverse set of opinions and knowledge, ensuring that the model learns from high-quality and diverse content.
Model Training:
- epoch: 50
- training_loss: 2.473200
- validation_loss: 4.569556
- perplexities: [517.4149169921875, 924.535888671875, 704.73291015625, 465.9677429199219, 577.629150390625, 443.994140625, 770.1861572265625, 683.028076171875, 1017.7510375976562, 880.795166015625]
- mean_perplexity: 698.603519
Description:
The model achieved a mean perplexity of 698.603519 on the validation set, indicating its ability to generate diverse and high-quality essays on the given topics.
During Text Generation, the following parameters are used:
- `max_length`: The maximum length of the generated text, set to 400 tokens.
- `num_beams`: The number of beams for beam search, set to 10. A higher value will increase the diversity of the generated text but may also increase the inference time.
- `early_stopping`: If set to True, the generation will stop as soon as the end-of-sequence token is generated.
- `temperature`: The sampling temperature, is set to 0.3.
- `no_repeat_ngram_size`: The size of the n-gram window to avoid repetitions, set to 2.
Find the kaggle notebook for this project at
[Kaggle Notebook](https://www.kaggle.com/code/vignesharjunraj/finetuned-distilgpt2-llm-for-essays-400-words/) |