David Chu commited on
Commit
3a1c032
1 Parent(s): f507f5c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -1
README.md CHANGED
@@ -11,4 +11,27 @@ metrics:
11
 
12
  # DistilBERT base model (uncased)
13
 
14
- This model is the Multi-Genre Natural Language Inference (MNLI) fine-turned version of the [uncased DistilBERT model](https://huggingface.co/distilbert-base-uncased).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
  # DistilBERT base model (uncased)
13
 
14
+ This is the [uncased DistilBERT model](https://huggingface.co/distilbert-base-uncased) fine-tuned on [Multi-Genre Natural Language Inference](https://huggingface.co/datasets/multi_nli) (MNLI) dataset for the zero-shot classification task. The model is not case-sensitive, i.e., it does not make a difference between "english" and "English".
15
+
16
+ ## Training
17
+
18
+ Training is done on a [p3.2xlarge](https://aws.amazon.com/ec2/instance-types/p3/) AWS EC2 instance (1 NVIDIA Tesla V100 GPUs), with the following hyperparameters:
19
+
20
+ ```
21
+ $ run_glue.py \
22
+ --model_name_or_path distilbert-base-uncased \
23
+ --task_name mnli \
24
+ --do_train \
25
+ --do_eval \
26
+ --max_seq_length 128 \
27
+ --per_device_train_batch_size 16 \
28
+ --learning_rate 2e-5 \
29
+ --num_train_epochs 5 \
30
+ --output_dir /tmp/distilbert-base-uncased_mnli/
31
+ ```
32
+
33
+ ## Evaluation results
34
+
35
+ | Task | MNLI | MNLI-mm |
36
+ |:----:|:----:|:----:|
37
+ | | 82.0 | 82.0 |