Spaces:
Running
on
CPU Upgrade
Model failing to run
Hi! After attempting to submit ClownTruck to the leaderboard, it seems to have fail both times when submitted to the leaderboard:
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/NobodyExistsOnTheInternet/Llama-2-70b-x8-MoE-clown-truck_eval_request_False_float16_Original.json
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/NobodyExistsOnTheInternet/Llama-2-70b-x8-MoE-clown-truck_eval_request_False_4bit_Original.json
What seems to be the issue? Thanks in advance.
The model is almost definitely too big to be evaluated on the leaderboard. In 4bit it would take at least a quarter of a terabyte to run. I don't think the leaderboard is setup with that kind of hardware.
Hi! Your models are too big to be supported on one node of our cluster (8GPUs), the maximum we allow is around 100B params.
We occasionally run bigger models manually in a multinode setup, but that's only something we do for base pretrained models (= not fine tunes, not merges, ...), as they are the most useful for the community.