qwen2_Magiccoder_evol_10k

This model is a fine-tuned version of Qwen/Qwen2-7B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7725

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 0.02
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
0.8717	0.0261	4	0.8953
0.8201	0.0522	8	0.8217
0.7952	0.0783	12	0.8046
0.7488	0.1044	16	0.7901
0.8162	0.1306	20	0.7853
0.6904	0.1567	24	0.7835
0.6959	0.1828	28	0.7821
0.8719	0.2089	32	0.7817
0.7546	0.2350	36	0.7795
0.7769	0.2611	40	0.7773
0.838	0.2872	44	0.7758
0.8043	0.3133	48	0.7745
0.7233	0.3395	52	0.7741
0.699	0.3656	56	0.7737
0.728	0.3917	60	0.7731
0.7698	0.4178	64	0.7734
0.8031	0.4439	68	0.7734
0.7362	0.4700	72	0.7731
0.7598	0.4961	76	0.7728
0.7305	0.5222	80	0.7727
0.8435	0.5483	84	0.7726
0.7321	0.5745	88	0.7726
0.8194	0.6006	92	0.7726
0.7417	0.6267	96	0.7728
0.8314	0.6528	100	0.7726
0.711	0.6789	104	0.7726
0.7745	0.7050	108	0.7725
0.744	0.7311	112	0.7724
0.7047	0.7572	116	0.7725
0.6911	0.7834	120	0.7724
0.7593	0.8095	124	0.7725
0.6959	0.8356	128	0.7725
0.8262	0.8617	132	0.7723
0.7802	0.8878	136	0.7724
0.7966	0.9139	140	0.7724
0.7511	0.9400	144	0.7723
0.8079	0.9661	148	0.7724
0.7663	0.9922	152	0.7725

Framework versions

• PEFT 0.7.1
• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1