Model save

README.md

@@ -1,65 +1,68 @@
 ---
-{}
+tags:
+- generated_from_trainer
+datasets:
+- squad_v2
+model-index:
+- name: distilbert-finetuned-uncased-squad_v2
+  results: []
 ---
 
-    ---
-    language:
-    - en
-    license: mit  
-    tags:
-    - question-answering  
-    - SQuAD
-    - BERT
-    datasets:
-    - squad
-    metrics:
-    - f1
-    - em
-
-    model-index:
-    - name: squad-v2-bert-base-finetuned
-    results:
-    - task:
-        type: question-answering
-        name: SQuAD Question Answering
-        dataset:
-        type: squad_v2
-        name: SQuAD v2
-        split: validation
-        metrics:
-        - type: f1
-        value: 26.869992349988973
-        name: F1 Score
-        - type: em  
-        value: 23.347090036216628
-        name: Exact Match
-        verified: true
-    ---
-
-    # distilbert-finetuned-uncased
-
-    This model is fine-tuned on SQuAD v2 for question answering tasks.
-
-    ## Training Procedure
-    - Number of Epochs: 4
-    - Learning Rate: 2e-05
-    - Batch Size: 128 (per device)
-    - Evaluation Strategy: Every 100 steps
-    - Save Strategy: Every 100 steps
-    - FP16 Training: Yes
-
-    ## Evaluation Results
-    - Exact Match: 23.347090036216628
-    - F1 Score: 26.869992349988973
-    - Total: 11873
-    - Has Answer Exact: 38.630229419703106
-    - Has Answer F1: 45.686136837283904
-    - Has Answer Total: 5928
-    - No Answer Exact: 8.107653490328007
-    - No Answer F1: 8.107653490328007
-    - No Answer Total: 5945
-    - Best Exact: 50.11370336056599
-    - Best Exact Threshold: 0.0
-    - Best F1: 50.11370336056599
-    - Best F1 Threshold: 0.0
-    
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# distilbert-finetuned-uncased-squad_v2
+
+This model was trained from scratch on the squad_v2 dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.3930
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 128
+- eval_batch_size: 128
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 512
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 3.6437        | 0.39  | 100  | 2.1780          |
+| 2.1596        | 0.78  | 200  | 1.6557          |
+| 1.8138        | 1.18  | 300  | 1.5683          |
+| 1.6987        | 1.57  | 400  | 1.5076          |
+| 1.6586        | 1.96  | 500  | 1.5350          |
+| 1.5957        | 1.18  | 600  | 1.4431          |
+| 1.5825        | 1.37  | 700  | 1.4955          |
+| 1.5523        | 1.57  | 800  | 1.4444          |
+| 1.5346        | 1.76  | 900  | 1.3930          |
+| 1.5098        | 1.96  | 1000 | 1.4285          |
+
+
+### Framework versions
+
+- Transformers 4.34.1
+- Pytorch 2.1.0+cu118
+- Datasets 2.14.5
+- Tokenizers 0.14.1

tokenizer.json

@@ -3,13 +3,11 @@
   "truncation": {
     "direction": "Right",
     "max_length": 512,
-    "strategy": "LongestFirst",
-    "stride": 0
+    "strategy": "OnlySecond",
+    "stride": 128
   },
   "padding": {
-    "strategy": {
-      "Fixed": 512
-    },
+    "strategy": "BatchLongest",
     "direction": "Right",
     "pad_to_multiple_of": null,
     "pad_id": 0,

trainer_state.json

@@ -153,16 +153,16 @@
       "step": 1020,
       "total_flos": 5.148633647651021e+16,
       "train_loss": 0.028946983113008386,
-      "train_runtime": 26.8068,
-      "train_samples_per_second": 19473.121,
-      "train_steps_per_second": 38.05
+      "train_runtime": 26.811,
+      "train_samples_per_second": 19470.051,
+      "train_steps_per_second": 38.044
     },
     {
       "epoch": 4.0,
       "eval_loss": 1.3930128812789917,
-      "eval_runtime": 8.2969,
-      "eval_samples_per_second": 1442.592,
-      "eval_steps_per_second": 11.33,
+      "eval_runtime": 8.3512,
+      "eval_samples_per_second": 1433.204,
+      "eval_steps_per_second": 11.256,
       "step": 1020
     }
   ],

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c7ae8b9eb8e6eebcf1c334060dd09414a7a9d6f7ee56c74412375c0cdf85353
+oid sha256:57d8a461b292d1c7d936212be6c7619b9698a1a89d0479a510f55cfd2a15f38a
 size 4664