Spaces:

Duplicated from  open-llm-leaderboard/open_llm_leaderboard

open-llm-leaderboard-old

/

open_llm_leaderboard

like 47

Running on CPU Upgrade

Hello :)

I want to extract some example results of models for demonstration. I am currently struggling with the result details datasets (for example https://huggingface.co/datasets/open-llm-leaderboard-old/details_davidkim205__Rhea-72b-v0.5).

I want to extract the example, the corresponding choices, the predicted answer by a model and if the answer is true or false.

By looking for example on the file of https://huggingface.co/datasets/open-llm-leaderboard-old/details_davidkim205__Rhea-72b-v0.5/blob/main/2024-03-23T20-12-54.617185/details_harness%7Cwinogrande%7C5_2024-03-23T20-12-54.617185.parquet i am wondering why the choices field in the dataset is always empty. I have also seen this behaviour for example on ARC. Why are the choices not in there?

Is there any way to get the requested information out of those files?

Thank you in advance!