crumb commited on
Commit
4544a57
1 Parent(s): 707eacf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -3
README.md CHANGED
@@ -28,12 +28,11 @@ The model is trained for four epochs on the [crumb/flan-ul2-tinystories](https:/
28
  Training arguments:
29
 
30
  ```
31
- per_device_train_batch_size=32,
32
- gradient_accumulation_steps=4,
33
  warmup_steps=128,
34
  num_train_epochs=4,
35
  learning_rate=2e-4,
36
- bf16=True,
37
  eval_steps=64,
38
  optim="adamw_torch",
39
  ```
 
28
  Training arguments:
29
 
30
  ```
31
+ per_device_train_batch_size=16,
32
+ gradient_accumulation_steps=8,
33
  warmup_steps=128,
34
  num_train_epochs=4,
35
  learning_rate=2e-4,
 
36
  eval_steps=64,
37
  optim="adamw_torch",
38
  ```