Update README.md
Browse files
README.md
CHANGED
|
@@ -53,7 +53,7 @@ Since the Axolotl corpus contains misaligments, we just select the best samples
|
|
| 53 |
Also, to increase the amount of data we collected 3,000 extra samples from the web.
|
| 54 |
|
| 55 |
### Model and training
|
| 56 |
-
We employ two training-stages using a multilingual T5-small. This model was chosen because it can handle different vocabularies and suffixes.
|
| 57 |
|
| 58 |
### Training-stage 1 (learning Spanish)
|
| 59 |
In training stage 1 we first introduce Spanish to the model. The objective is to learn a new language rich in data (Spanish) and not lose the previous knowledge acquired. We use the English-Spanish [Anki](https://www.manythings.org/anki/) dataset, which consists of 118.964 text pairs. We train the model till convergence adding the suffix "Translate Spanish to English: ".
|
|
|
|
| 53 |
Also, to increase the amount of data we collected 3,000 extra samples from the web.
|
| 54 |
|
| 55 |
### Model and training
|
| 56 |
+
We employ two training-stages using a multilingual T5-small. This model was chosen because it can handle different vocabularies and suffixes. T5-small is pretrained on different tasks and languages (French, Romanian, English, German).
|
| 57 |
|
| 58 |
### Training-stage 1 (learning Spanish)
|
| 59 |
In training stage 1 we first introduce Spanish to the model. The objective is to learn a new language rich in data (Spanish) and not lose the previous knowledge acquired. We use the English-Spanish [Anki](https://www.manythings.org/anki/) dataset, which consists of 118.964 text pairs. We train the model till convergence adding the suffix "Translate Spanish to English: ".
|