DNADebertaK6_Zebrafish

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4958

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 600001
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
4.1727	0.59	20000	1.8535
1.8381	1.18	40000	1.7512
1.7561	1.77	60000	1.7235
1.7281	2.36	80000	1.7019
1.7065	2.95	100000	1.6822
1.6876	3.54	120000	1.6639
1.6718	4.13	140000	1.6501
1.6562	4.71	160000	1.6350
1.6429	5.3	180000	1.6211
1.6313	5.89	200000	1.6102
1.6207	6.48	220000	1.6001
1.6099	7.07	240000	1.5902
1.6	7.66	260000	1.5799
1.5925	8.25	280000	1.5726
1.5847	8.84	300000	1.5645
1.5783	9.43	320000	1.5596
1.5712	10.02	340000	1.5510
1.5656	10.61	360000	1.5452
1.5598	11.2	380000	1.5410
1.5548	11.79	400000	1.5342
1.5497	12.38	420000	1.5293
1.546	12.96	440000	1.5241
1.5397	13.55	460000	1.5214
1.5365	14.14	480000	1.5164
1.5321	14.73	500000	1.5115
1.5285	15.32	520000	1.5075
1.5246	15.91	540000	1.5034
1.5217	16.5	560000	1.5029
1.5191	17.09	580000	1.4995
1.516	17.68	600000	1.4958

Framework versions

• Transformers 4.19.2
• Pytorch 1.11.0
• Datasets 2.2.2
• Tokenizers 0.12.1