---
license: apache-2.0
base_model: distilbert-base-multilingual-cased
tags:
- generated_from_keras_callback
model-index:
- name: transformers-qa-kaggle-tpu
  results: []
---

<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# transformers-qa-kaggle-tpu

This model is a fine-tuned version of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.2138
- Train End Logits Accuracy: 0.9272
- Train Start Logits Accuracy: 0.9265
- Validation Loss: 4.0782
- Validation End Logits Accuracy: 0.4815
- Validation Start Logits Accuracy: 0.4488
- Epoch: 14

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': False, 'is_legacy_optimizer': False, 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 122160, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
- training_precision: float32

### Training results

| Train Loss | Train End Logits Accuracy | Train Start Logits Accuracy | Validation Loss | Validation End Logits Accuracy | Validation Start Logits Accuracy | Epoch |
|:----------:|:-------------------------:|:---------------------------:|:---------------:|:------------------------------:|:--------------------------------:|:-----:|
| 2.2397     | 0.4602                    | 0.4288                      | 2.0601          | 0.4944                         | 0.4670                           | 0     |
| 1.7080     | 0.5695                    | 0.5404                      | 2.0597          | 0.5098                         | 0.4782                           | 1     |
| 1.4423     | 0.6244                    | 0.5975                      | 2.0398          | 0.5124                         | 0.4779                           | 2     |
| 1.2221     | 0.6718                    | 0.6455                      | 2.1899          | 0.5121                         | 0.4832                           | 3     |
| 1.0321     | 0.7138                    | 0.6920                      | 2.2860          | 0.5067                         | 0.4742                           | 4     |
| 0.8740     | 0.7502                    | 0.7307                      | 2.4042          | 0.4981                         | 0.4677                           | 5     |
| 0.7420     | 0.7809                    | 0.7666                      | 2.6543          | 0.4954                         | 0.4595                           | 6     |
| 0.6254     | 0.8107                    | 0.7988                      | 2.7405          | 0.4938                         | 0.4607                           | 7     |
| 0.5325     | 0.8337                    | 0.8255                      | 3.0218          | 0.4911                         | 0.4613                           | 8     |
| 0.4537     | 0.8550                    | 0.8491                      | 3.1804          | 0.4917                         | 0.4550                           | 9     |
| 0.3860     | 0.8751                    | 0.8707                      | 3.4298          | 0.4880                         | 0.4524                           | 10    |
| 0.3284     | 0.8914                    | 0.8890                      | 3.5952          | 0.4826                         | 0.4480                           | 11    |
| 0.2844     | 0.9053                    | 0.9033                      | 3.6105          | 0.4856                         | 0.4510                           | 12    |
| 0.2462     | 0.9176                    | 0.9151                      | 3.8751          | 0.4785                         | 0.4504                           | 13    |
| 0.2138     | 0.9272                    | 0.9265                      | 4.0782          | 0.4815                         | 0.4488                           | 14    |


### Framework versions

- Transformers 4.31.0.dev0
- TensorFlow 2.12.0
- Datasets 2.13.1
- Tokenizers 0.13.3