llamaFR

This model is a fine-tuned version of meta-llama/Meta-Llama-3.1-8B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: nan

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant
• lr_scheduler_warmup_steps: 500
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.0	0.0291	2000	nan
0.0	0.0581	4000	nan
0.0	0.0872	6000	nan
0.0	0.1162	8000	nan
0.0	0.1453	10000	nan
0.0	0.1743	12000	nan
0.0	0.2034	14000	nan
0.0	0.2325	16000	nan
0.0	0.2615	18000	nan
0.0	0.2906	20000	nan
0.0	0.3196	22000	nan
0.0	0.3487	24000	nan
0.0	0.3777	26000	nan
0.0	0.4068	28000	nan
0.0	0.4359	30000	nan
0.0	0.4649	32000	nan
0.0	0.4940	34000	nan
0.0	0.5230	36000	nan
0.0	0.5521	38000	nan
0.0	0.5811	40000	nan
0.0	0.6102	42000	nan
0.0	0.6392	44000	nan
0.0	0.6683	46000	nan
0.0	0.6974	48000	nan
0.0	0.7264	50000	nan
0.0	0.7555	52000	nan
0.0	0.7845	54000	nan
0.0	0.8136	56000	nan
0.0	0.8426	58000	nan
0.0	0.8717	60000	nan
0.0	0.9008	62000	nan
0.0	0.9298	64000	nan
0.0	0.9589	66000	nan
0.0	0.9879	68000	nan
0.0	1.0170	70000	nan
0.0	1.0460	72000	nan
0.0	1.0751	74000	nan
0.0	1.1042	76000	nan
0.0	1.1332	78000	nan
0.0	1.1623	80000	nan
0.0	1.1913	82000	nan
0.0	1.2204	84000	nan
0.0	1.2494	86000	nan
0.0	1.2785	88000	nan
0.0	1.3076	90000	nan
0.0	1.3366	92000	nan
0.0	1.3657	94000	nan
0.0	1.3947	96000	nan
0.0	1.4238	98000	nan
0.0	1.4528	100000	nan
0.0	1.4819	102000	nan
0.0	1.5109	104000	nan
0.0	1.5400	106000	nan
0.0	1.5691	108000	nan
0.0	1.5981	110000	nan
0.0	1.6272	112000	nan
0.0	1.6562	114000	nan
0.0	1.6853	116000	nan
0.0	1.7143	118000	nan
0.0	1.7434	120000	nan
0.0	1.7725	122000	nan
0.0	1.8015	124000	nan
0.0	1.8306	126000	nan
0.0	1.8596	128000	nan
0.0	1.8887	130000	nan
0.0	1.9177	132000	nan
0.0	1.9468	134000	nan
0.0	1.9759	136000	nan

Framework versions

• Transformers 4.46.0.dev0
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0