Spaces:
Running
on
CPU Upgrade
Model failure: migtissera/Tess-v2.5.2-Qwen2-72B model
Hey guys,
I noticed that there is now a checkbox asking whether "Is your model a chat model", and "Use chat template". I had not selected that as a yes when I submitted (I overlooked it). The migtissera/Tess-v2.5.2-Qwen2-72B model is indeed a chat fine-tune with ChatML prompt template. Can you please run it with the chat template switched on? I want to evaluate the model exactly the same way that you guys had evaluated the Qwen2-72B-Instruct model.
Thank you!
Migel Tissera
I should also add that this model is next in line, with 16 votes as of now.
Hi! I just logged on and the evaluation of your model was already started. So I cancelled the job, passed it to pending again, and added the chat template.
Please be more careful next time as it's a lot of added work (especially if you have popular models where we're time constrained for these modifs ^^).
It should reschdule, but ping us if it does not.
Thanks Clementine!
Hey
@clefourrier
,
The model eval seems to have failed. Could you investigate please?
Thanks,
Migel
Hi!
Could you link us the request file (as usual)?
It would also be better next time to open a specific discussion so we don't lose info. I reopened this conversation for this time.
cc
@alozowski
Thanks @clefourrier ,
Here's the one that failed: https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/migtissera/Tess-v2.5.2-Qwen2-72B_eval_request_False_float16_Original.json
But I see that your team has submitted it again: https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/migtissera/Tess-v2.5.2-Qwen2-72B_eval_request_False_bfloat16_Original.json
Thanks!
Ok, closing then :)
Thanks man! I appreciate you guys.