---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: bert-medical-ner
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-medical-ner

This model is a fine-tuned version of [distilbert-base-cased](https://huggingface.co/distilbert-base-cased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.3023
- Precision: 0.6627
- Recall: 0.6985
- F1: 0.6802
- Accuracy: 0.7491

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| No log        | 1.0   | 71   | 1.8664          | 0.3387    | 0.4366 | 0.3815 | 0.5491   |
| No log        | 2.0   | 142  | 1.3020          | 0.4581    | 0.5572 | 0.5028 | 0.6561   |
| No log        | 3.0   | 213  | 1.1061          | 0.5318    | 0.6091 | 0.5678 | 0.6921   |
| No log        | 4.0   | 284  | 0.9755          | 0.6177    | 0.6383 | 0.6278 | 0.7193   |
| No log        | 5.0   | 355  | 0.9530          | 0.6071    | 0.6362 | 0.6213 | 0.7272   |
| No log        | 6.0   | 426  | 0.8876          | 0.6456    | 0.6590 | 0.6523 | 0.7351   |
| No log        | 7.0   | 497  | 0.8754          | 0.6674    | 0.6757 | 0.6715 | 0.7386   |
| 1.158         | 8.0   | 568  | 0.8472          | 0.6782    | 0.6923 | 0.6852 | 0.7491   |
| 1.158         | 9.0   | 639  | 0.8816          | 0.6573    | 0.6819 | 0.6694 | 0.7368   |
| 1.158         | 10.0  | 710  | 0.9035          | 0.6260    | 0.6299 | 0.6280 | 0.7184   |
| 1.158         | 11.0  | 781  | 0.9156          | 0.6573    | 0.6819 | 0.6694 | 0.7377   |
| 1.158         | 12.0  | 852  | 0.8764          | 0.6536    | 0.6944 | 0.6734 | 0.7456   |
| 1.158         | 13.0  | 923  | 0.9079          | 0.6673    | 0.6881 | 0.6776 | 0.7404   |
| 1.158         | 14.0  | 994  | 0.9278          | 0.6525    | 0.6715 | 0.6619 | 0.7351   |
| 0.4312        | 15.0  | 1065 | 0.9387          | 0.6755    | 0.6923 | 0.6838 | 0.7465   |
| 0.4312        | 16.0  | 1136 | 0.9396          | 0.6595    | 0.7006 | 0.6794 | 0.7482   |
| 0.4312        | 17.0  | 1207 | 0.9672          | 0.648     | 0.6736 | 0.6606 | 0.7351   |
| 0.4312        | 18.0  | 1278 | 0.9890          | 0.6719    | 0.7110 | 0.6909 | 0.7509   |
| 0.4312        | 19.0  | 1349 | 1.0124          | 0.6344    | 0.6819 | 0.6573 | 0.7368   |
| 0.4312        | 20.0  | 1420 | 1.0107          | 0.6564    | 0.7069 | 0.6807 | 0.7526   |
| 0.4312        | 21.0  | 1491 | 1.0036          | 0.6765    | 0.7131 | 0.6943 | 0.7632   |
| 0.2196        | 22.0  | 1562 | 1.0244          | 0.6744    | 0.7235 | 0.6981 | 0.7561   |
| 0.2196        | 23.0  | 1633 | 1.0668          | 0.6602    | 0.7027 | 0.6808 | 0.7430   |
| 0.2196        | 24.0  | 1704 | 1.1040          | 0.6667    | 0.7193 | 0.6920 | 0.7526   |
| 0.2196        | 25.0  | 1775 | 1.0959          | 0.6699    | 0.7173 | 0.6928 | 0.7553   |
| 0.2196        | 26.0  | 1846 | 1.0721          | 0.6765    | 0.7173 | 0.6963 | 0.7544   |
| 0.2196        | 27.0  | 1917 | 1.1114          | 0.6628    | 0.7069 | 0.6841 | 0.7553   |
| 0.2196        | 28.0  | 1988 | 1.1225          | 0.6429    | 0.6923 | 0.6667 | 0.7421   |
| 0.1279        | 29.0  | 2059 | 1.1149          | 0.6481    | 0.7006 | 0.6733 | 0.7588   |
| 0.1279        | 30.0  | 2130 | 1.1545          | 0.6660    | 0.7048 | 0.6848 | 0.7544   |
| 0.1279        | 31.0  | 2201 | 1.1645          | 0.6641    | 0.7152 | 0.6887 | 0.7535   |
| 0.1279        | 32.0  | 2272 | 1.2004          | 0.6523    | 0.6944 | 0.6727 | 0.7386   |
| 0.1279        | 33.0  | 2343 | 1.2030          | 0.6419    | 0.6819 | 0.6613 | 0.7404   |
| 0.1279        | 34.0  | 2414 | 1.2434          | 0.6726    | 0.7048 | 0.6883 | 0.7482   |
| 0.1279        | 35.0  | 2485 | 1.2795          | 0.6548    | 0.6902 | 0.6721 | 0.7412   |
| 0.0843        | 36.0  | 2556 | 1.2499          | 0.6772    | 0.7152 | 0.6957 | 0.7544   |
| 0.0843        | 37.0  | 2627 | 1.2545          | 0.6745    | 0.7152 | 0.6942 | 0.7535   |
| 0.0843        | 38.0  | 2698 | 1.2286          | 0.6680    | 0.6985 | 0.6829 | 0.75     |
| 0.0843        | 39.0  | 2769 | 1.2943          | 0.6601    | 0.6985 | 0.6788 | 0.7518   |
| 0.0843        | 40.0  | 2840 | 1.2713          | 0.6640    | 0.7027 | 0.6828 | 0.7535   |
| 0.0843        | 41.0  | 2911 | 1.2828          | 0.6510    | 0.6902 | 0.6700 | 0.7465   |
| 0.0843        | 42.0  | 2982 | 1.2830          | 0.6621    | 0.7048 | 0.6828 | 0.7509   |
| 0.0619        | 43.0  | 3053 | 1.2942          | 0.6621    | 0.6965 | 0.6788 | 0.75     |
| 0.0619        | 44.0  | 3124 | 1.2912          | 0.6752    | 0.7089 | 0.6917 | 0.7544   |
| 0.0619        | 45.0  | 3195 | 1.2631          | 0.6680    | 0.7069 | 0.6869 | 0.7579   |
| 0.0619        | 46.0  | 3266 | 1.2948          | 0.6647    | 0.7006 | 0.6822 | 0.7535   |
| 0.0619        | 47.0  | 3337 | 1.2829          | 0.6739    | 0.7131 | 0.6929 | 0.7570   |
| 0.0619        | 48.0  | 3408 | 1.2943          | 0.6602    | 0.7027 | 0.6808 | 0.75     |
| 0.0619        | 49.0  | 3479 | 1.2995          | 0.6562    | 0.6944 | 0.6747 | 0.7465   |
| 0.0514        | 50.0  | 3550 | 1.3023          | 0.6627    | 0.6985 | 0.6802 | 0.7491   |


### Framework versions

- Transformers 4.29.2
- Pytorch 2.0.1+cu117
- Datasets 2.12.0
- Tokenizers 0.13.3