omidrohanian commited on
Commit
fbe2b25
1 Parent(s): 2c61d75

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -2,7 +2,7 @@
2
  BioTinyBERT is the result of training the [TinyBERT](https://huggingface.co/huawei-noah/TinyBERT_General_4L_312D) model in a continual learning fashion for 200k training steps using a total batch size of 192 on the PubMed dataset.
3
 
4
  # Initialisation
5
- We initialise our model with the pre-trained checkpoints of the [TinyBERT](https://huggingface.co/huawei-noah/TinyBERT_General_4L_312D) model available on the Huggingface.
6
 
7
  # Architecture
8
  This model uses 4 hidden layers with a hidden dimension size and an embedding size of 768 resulting in a total of 15M parameters.
 
2
  BioTinyBERT is the result of training the [TinyBERT](https://huggingface.co/huawei-noah/TinyBERT_General_4L_312D) model in a continual learning fashion for 200k training steps using a total batch size of 192 on the PubMed dataset.
3
 
4
  # Initialisation
5
+ We initialise our model with the pre-trained checkpoints of the [TinyBERT](https://huggingface.co/huawei-noah/TinyBERT_General_4L_312D) model available on Huggingface.
6
 
7
  # Architecture
8
  This model uses 4 hidden layers with a hidden dimension size and an embedding size of 768 resulting in a total of 15M parameters.