beomi commited on
Commit
2beaa28
1 Parent(s): 496917f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -38,9 +38,12 @@ Solar-Ko-Recovery is an auto-regressive language model that leverages an optimiz
38
 
39
  | |Training Data|Parameters|Content Length|GQA|Tokens|Learning Rate|
40
  |---|---|---|---|---|---|---|
41
- |Solar-Ko-Recovery|*A curated mix of Korean+English Corpora*|10.8B|4k|O|>30B*|5e<sup>-5</sup>|
42
 
43
- > NOTE: Only Embedding layer and LM Head layer are trained.
 
 
 
44
 
45
  **Vocab Expansion**
46
 
 
38
 
39
  | |Training Data|Parameters|Content Length|GQA|Tokens|Learning Rate|
40
  |---|---|---|---|---|---|---|
41
+ |Solar-Ko-Recovery|*A curated mix of Korean+English Corpora*|11B(10.99B)|4k|O|>100B*|5e<sup>-5</sup>|
42
 
43
+ > NOTE: 2-step training processed
44
+ >
45
+ > 1) Only Embedding layer and LM Head layer are trained
46
+ > 2) Full params trained
47
 
48
  **Vocab Expansion**
49