Probably DPO datasets

davanstrien 's Collections

synthetic-data-generation-demos

sentence-transformers-from-synthetic-data

Synthetic (text) Dataset Generation

haiku

Historic language modeling

Climate

Sourced from Wikimedia

Legal Named Entity Recognition

Top 10% instruction tuning datasets

Top 10 Instruction Tuning Datasets copy

Metadata-generation

MOE papers to read

German Text Embedding Clustering Benchmark datasets

cosmochat-reading-list

datasets-tldr-project

Probably DPO datasets

Probably Alpaca Style Datasets

updated 12 days ago

A collection of datasets that probably support DPO

Upvote

HuggingFaceH4/ultrafeedback_binarized

Viewer • Updated Jan 8 • 187k • 37.8k • 201
mlabonne/orpo-dpo-mix-40k

Viewer • Updated Jun 7 • 44.2k • 4.34k • 193
argilla/OpenHermesPreferences

Viewer • Updated Mar 1 • 989k • 101 • 176
argilla/distilabel-capybara-dpo-7k-binarized

Viewer • Updated Mar 23 • 7.56k • 49k • 158
argilla/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 5 • 12.9k • 413 • 147
argilla/dpo-mix-7k

Viewer • Updated Mar 4 • 7.5k • 672 • 127
argilla/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 60.9k • 3.42k • 108
Dahoas/rm-static

Viewer • Updated Mar 6, 2023 • 81.4k • 25.1k • 103
openbmb/UltraInteract_pair

Viewer • Updated Apr 5 • 220k • 171 • 92
openbmb/RLAIF-V-Dataset

Viewer • Updated Jun 1 • 33.8k • 2.51k • 91
hiyouga/DPO-En-Zh-20k

Viewer • Updated Jun 7 • 20k • 2.68k • 81
Dahoas/full-hh-rlhf

Viewer • Updated Feb 23, 2023 • 125k • 9.53k • 67
argilla/OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Feb 10 • 9.79k • 14 • 62
yitingxie/rlhf-reward-datasets

Viewer • Updated Jan 1, 2023 • 81.4k • 1.24k • 56
allenai/ultrafeedback_binarized_cleaned

Viewer • Updated Dec 1, 2023 • 186k • 2.86k • 57
allenai/reward-bench

Viewer • Updated 28 days ago • 8.11k • 24.1k • 52
CarperAI/openai_summarize_comparisons

Viewer • Updated Feb 27, 2023 • 260k • 2.08k • 36
beyond/rlhf-reward-single-round-trans_chinese

Viewer • Updated Jul 5, 2023 • 24.9k • 151 • 35
argilla/Capybara-Preferences

Viewer • Updated May 9 • 15.4k • 101 • 34
snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset

Viewer • Updated Jan 23 • 62.7k • 62 • 37
tasksource/oasst1_pairwise_rlhf_reward

Viewer • Updated Jul 4, 2023 • 18.9k • 5k • 37
wenbopan/Chinese-dpo-pairs

Viewer • Updated Apr 2 • 10.7k • 65 • 31
maywell/ko_Ultrafeedback_binarized

Viewer • Updated Nov 9, 2023 • 62k • 351 • 25
abacusai/MetaMath_DPO_FewShot

Viewer • Updated Feb 26 • 395k • 32 • 21
HuggingFaceH4/orca_dpo_pairs

Viewer • Updated Apr 14 • 12.9k • 991 • 20
Dahoas/static-hh

Viewer • Updated Mar 6, 2023 • 101k • 3 • 19
mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Mar 21 • 9.2k • 83 • 19
HuggingFaceH4/grok-conversation-harmless

Viewer • Updated Feb 2 • 44.8k • 44 • 19
allenai/preference-test-sets

Viewer • Updated Mar 14 • 43.2k • 163 • 16
abacusai/ARC_DPO_FewShot

Viewer • Updated Feb 26 • 4.25k • 16 • 17
cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental

Viewer • Updated about 20 hours ago • 29.2k • 172 • 15
vicgalle/configurable-system-prompt-multitask

Viewer • Updated Apr 23 • 1.95k • 14 • 14
coseal/CodeUltraFeedback_binarized

Viewer • Updated Mar 18 • 9.5k • 43 • 15
NobodyExistsOnTheInternet/ToxicDPOqa

Viewer • Updated Apr 26 • 6.87k • 95 • 14
multilingual/orca_dpo_pairs

Viewer • Updated Apr 3 • 64.7k • 77 • 14
fblgit/simple-math-DPO

Viewer • Updated Jan 27 • 800k • 4 • 13
HuggingFaceH4/cai-conversation-harmless

Viewer • Updated Feb 2 • 44.8k • 638 • 12
alvarobartt/dpo-mix-7k-simplified

Viewer • Updated Mar 26 • 7.5k • 234 • 11
kyujinpy/orca_math_dpo

Viewer • Updated Apr 12 • 15.3k • 8 • 11
davanstrien/dataset-tldr-preference-dpo

Viewer • Updated Apr 30 • 522 • 3 • 11
trl-internal-testing/hh-rlhf-trl-style

Viewer • Updated Mar 13 • 169k • 941 • 9
abacusai/HellaSwag_DPO_FewShot

Viewer • Updated Feb 26 • 150k • 10 • 8
2A2I/argilla-dpo-mix-7k-arabic

Viewer • Updated Mar 18 • 7.5k • 65 • 8
argilla/Capybara-Preferences-Filtered

Viewer • Updated Apr 20 • 14.8k • 168 • 8
2A2I/Aya-Command.R-DPO

Viewer • Updated May 16 • 14.2k • 16 • 7
kuotient/orca-math-korean-preference

Viewer • Updated Apr 10 • 193k • 60 • 7
M4-ai/prm_dpo_pairs

Viewer • Updated 8 days ago • 93.9k • 7 • 6
zerolink/zsql-postgres-dpo

Viewer • Updated Feb 2 • 259k • 27 • 6
princeton-nlp/llama3-ultrafeedback

Viewer • Updated May 24 • 61.8k • 14.4k • 9
kaist-ai/Multifaceted-Collection-ORPO

Viewer • Updated 8 days ago • 64.6k • 35 • 6
kaitchup/UltraFeedback-prompt-chosen-rejected

Viewer • Updated Oct 23, 2023 • 18k • 19 • 6
alexredna/oasst2_dpo_pairs

Viewer • Updated Jan 18 • 10k • 17 • 6
HuggingFaceH4/grok-conversation-harmless2

Viewer • Updated Jan 19 • 44.8k • 3 • 6
yongtae-jp/orca_dpo_pairs_ja

Viewer • Updated Jan 9 • 12.9k • 50 • 6
kaist-ai/Multifaceted-Collection-DPO

Viewer • Updated 8 days ago • 65.1k • 89 • 5
malhajar/distilabel-intel-orca-dpo-pairs-tr

Viewer • Updated Mar 28 • 9.12k • 13 • 5
efederici/alpaca-vs-alpaca-orpo-dpo

Viewer • Updated May 15 • 49.2k • 8 • 5
maywell/ko_Ultrafeedback_binarized_10k

Viewer • Updated Nov 4, 2023 • 10k • 53 • 5
kira/math-dpo

Viewer • Updated 6 days ago • 2.42k • 64 • 5
mlabonne/distilabel-truthy-dpo-v0.1

Viewer • Updated Feb 12 • 1.02k • 6 • 5
RLHFlow/UltraFeedback-preference-standard

Viewer • Updated Apr 27 • 340k • 1.36k • 6
BramVanroy/orca_dpo_pairs_dutch

Viewer • Updated Apr 24 • 11k • 7 • 5
argilla/ultrafeedback-multi-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 158k • 20 • 5
malhajar/orca_dpo_pairs-tr

Viewer • Updated Mar 30 • 10.8k • 12 • 5
lightblue/mitsu_tophalf_borda

Viewer • Updated May 30 • 1.42k • 15 • 5
NickyNicky/neovalle_H4rmony_dpo_translated_English_to_Spanish

Viewer • Updated May 17 • 2.02k • 13 • 4
maywell/ko_hh-rlhf-20k_filtered

Viewer • Updated Nov 4, 2023 • 19.4k • 14 • 3
kaist-ai/Multifaceted-Collection-RM

Viewer • Updated Jun 7 • 64.9k • 3 • 4
HuggingFaceH4/hh-rlhf-h4

Viewer • Updated Nov 29, 2023 • 169k • 25 • 4
orpo-explorers/ultrafeedback_binarized_15k

Viewer • Updated Apr 18 • 15.4k • 13 • 4
We-Want-GPU/Yi-Ko-DPO-Orca-DPO-Pairs

Viewer • Updated Dec 18, 2023 • 12.7k • 15 • 5
zerolink/zsql-sqlite-dpo

Viewer • Updated Feb 2 • 260k • 5 • 4
argilla/ultrafeedback-multi-binarized-quality-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 155k • 4 • 4
IlyaGusev/saiga_reward

Viewer • Updated May 2 • 27k • 4 • 4
SAGI-1/Greetings_DPO_dataset_V1

Viewer • Updated Dec 18, 2023 • 891 • 457 • 3
2A2I/Aya-SambaLingo.Arabic.Chat-DPO

Viewer • Updated May 25 • 14.2k • 15 • 3
AlekseyKorshuk/evol-codealpaca-v1-dpo

Viewer • Updated Jan 25 • 39.9k • 130 • 3
mlabonne/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Mar 27 • 28k • 3 • 3
kuotient/orca-math-korean-dpo-pairs

Viewer • Updated Apr 5 • 193k • 505 • 2
mii-community/ultrafeedback-preferences-translated-ita

Viewer • Updated Feb 21 • 60.9k • 4 • 3
sqrti/SPA-VL

Viewer • Updated 5 days ago • 101k • 14 • 4
HuggingFaceH4/orca_dpo_pairs_no_system_prompt

Viewer • Updated Jan 15 • 12.9k • 3 • 3
mlabonne/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 6 • 5.92k • 5 • 3
d0rj/rlhf-reward-datasets-ru

Viewer • Updated 7 days ago • 81.4k • 9 • 3
RLHFlow/HH-RLHF-Harmless-and-RedTeam-standard

Viewer • Updated May 8 • 42.3k • 4.61k • 2
2A2I/Aya-AceGPT.13B.Chat-DPO

Viewer • Updated May 16 • 13k • 13 • 2
RLHFlow/Helpsteer-preference-standard

Viewer • Updated Apr 27 • 37.1k • 5.06k • 2
IlyaGusev/saiga_preferences

Viewer • Updated 29 days ago • 14.3k • 38 • 2
RLHFlow/CodeUltraFeedback-standard

Viewer • Updated Apr 27 • 50.2k • 4.84k • 3
RLHFlow/UltraInteract-filtered-standard

Viewer • Updated Apr 28 • 162k • 136 • 2
RLHFlow/Prometheus2-preference-standard

Viewer • Updated May 5 • 200k • 1.38k • 2
RLHFlow/PKU-SafeRLHF-30K-standard

Viewer • Updated Apr 29 • 26.9k • 4.85k • 2
RLHFlow/Argilla-Math-DPO-standard

Viewer • Updated Apr 30 • 2.42k • 4.59k • 2
droussis/UltraSafety_binarized-orpo-dpo

Viewer • Updated Apr 25 • 2.26k • 7 • 2
We-Want-GPU/Yi-Ko-DPO-dataset

Viewer • Updated Dec 9, 2023 • 4k • 24 • 3
roborovski/glaive-tool-usage-dpo

Viewer • Updated Feb 29 • 42k • 4 • 2
vicgalle/OpenHermesPreferences-roleplay

Viewer • Updated Feb 29 • 3.06k • 11 • 2
GrowingApple/orpo_kor_translated_medical

Viewer • Updated 27 days ago • 10.4k • 45 • 2
reciprocate/ultrafeedback_cleaned_high_dpo

Viewer • Updated Jan 11 • 40k • 3 • 2
mlabonne/distilabel-truthy-dpo-v0.1-filtered

Viewer • Updated Feb 15 • 663 • 5 • 2
xiaodongguaAIGC/awesome-dpo

Viewer • Updated 6 days ago • 302k • 108 • 2
jan-hq/evol_codealpaca_dpo_binarized

Viewer • Updated Feb 20 • 39.9k • 13 • 2
2A2I/Aya-Aya.23.8B-DPO

Viewer • Updated May 29 • 14k • 9 • 2
tasksource/oasst2_pairwise_rlhf_reward

Viewer • Updated Jan 9 • 28.4k • 21 • 2
NickyNicky/function_call_orpo_sft_phi3_chatML_only

Viewer • Updated May 2 • 66k • 4 • 2
HuggingFaceH4/summarize_from_feedback

Viewer • Updated Mar 11 • 353k • 955 • 2
weqweasdas/preference_dataset_mix2

Viewer • Updated Mar 21 • 528k • 725 • 2
NickyNicky/oasst2_orpo_mix_function_call_phi_3_v1

Viewer • Updated May 2 • 120k • 2 • 2
BramVanroy/ultra_feedback_dutch

Viewer • Updated Apr 18 • 53.6k • 68 • 2
orpo-explorers/OpenHermesPreferences-10k

Viewer • Updated Mar 30 • 10k • 51 • 2
orpo-explorers/OpenHermesPreferences-500k

Viewer • Updated Mar 30 • 500k • 12 • 2
zhengr/ultrafeedback_binarized

Viewer • Updated Nov 8, 2023 • 190k • 657 • 2
hatakeyama-llm-team/rlhf-ja-chatbot-arena-phi-3-medium

Viewer • Updated May 30 • 29.2k • 24 • 1
ibivibiv/cleaned_orca_math_dpo_pairs

Viewer • Updated May 24 • 200k • 24 • 1
NickyNicky/nano_finance_200k_en_es_chatML_gemma_orpo_dpo

Viewer • Updated May 29 • 201k • 4 • 1
Baidicoot/hh-rlhf-golden-harmful

Viewer • Updated May 10 • 7.64k • 25 • 1
Tachi67/rm_data_motivation

Viewer • Updated Jun 4 • 44.8k • 4 • 1
trl-internal-testing/hh-rlhf-helpful-base-trl-style

Viewer • Updated May 2 • 46.2k • 4.93k • 3
Mitsuki-Sakamoto/hh-rlhf-reward-model-deberta-v3-large-v2-helpful-2-original_mix_50_random_seed_2

Viewer • Updated Jun 8 • 46.2k • 18 • 1
flyingfishinwater/ultrafeedback_clean

Viewer • Updated 7 days ago • 175k • 3 • 1
manishiitg/argilla-distilabel-intel-orca-dpo-pairs

Viewer • Updated Jan 29 • 23.6k • 7 • 1
Palash123/dpo_anthropic_hh_rlhf

Viewer • Updated Dec 4, 2023 • 168k • 4 • 1
David-Xu/astronomy-stack-dpo-text

Viewer • Updated Mar 8 • 19.9k • 58 • 1
d0rj/full-hh-rlhf-ru

Viewer • Updated 7 days ago • 125k • 11 • 1
vwxyzjn/summarize_from_feedback_oai_preprocessing_1706381144

Viewer • Updated Jan 27 • 179k • 2.24k • 1
reciprocate/ultrafeedback_orca_math_cleaned_high_dpo

Viewer • Updated Jan 11 • 48.3k • 9 • 1
HuggingFaceH4/OpenHermes-2.5-preferences-v0-deduped

Viewer • Updated Feb 9 • 802k • 6 • 1
mlabonne/truthy-dpo-v0.1

Viewer • Updated Feb 18 • 1.02k • 7 • 1
macadeliccc/distilabel-neurology-preferences-2k-orca-format

Viewer • Updated Feb 22 • 1.99k • 7 • 1
jan-hq/athirdpath_roleplay_dpo_binarized

Viewer • Updated Jan 9 • 3.43k • 1 • 1
reciprocate/ultrafeedback_orca_math_dpo

Viewer • Updated Jan 8 • 73.8k • 26 • 1
d0rj/synthetic-instruct-gptj-pairwise-ru

Viewer • Updated 7 days ago • 33.1k • 9 • 1
CorticalStack/tak-stack-dpo

Viewer • Updated Feb 28 • 10.9k • 5 • 1
umarigan/openhermespreference_tr

Viewer • Updated Apr 24 • 989k • 4 • 1
amztheory/toxic-aira-dataset

Viewer • Updated Apr 30 • 8.29k • 6 • 1
RLHFlow/HH-RLHF-Helpful-standard

Viewer • Updated Apr 27 • 115k • 4.59k • 1
trl-internal-testing/descriptiveness-sentiment-trl-style

Viewer • Updated Apr 9 • 10.9k • 3.83k • 1
We-Want-GPU/yi-ko-DPO-noprompt-dataset

Viewer • Updated Dec 6, 2023 • 3.83k • 7 • 2
hendrydong/preference_700K

Viewer • Updated Apr 18 • 700k • 3.2k • 1
insub/imdb_prefix20_forDPO_gpt2-large-imdb-FT_siebert_sentiment-roberta-large-english

Viewer • Updated Oct 22, 2023 • 50k • 39 • 1
FredrikBL/Legal-Snigel-DPO

Viewer • Updated May 27 • 2k • 2 • 1
lightblue/mitsu_full_borda

Viewer • Updated May 30 • 2.85k • 30 • 1
MaziyarPanahi/truthy-dpo-v0.1-axolotl

Viewer • Updated May 2 • 1.02k • 834 • 1
RLHFlow/Orca-distibalel-standard

Viewer • Updated Apr 28 • 6.93k • 1.16k • 1
RLHFlow/prm80k-phase2

Viewer • Updated Apr 28 • 79.5k • 4.72k • 1
pythainlp/oasst2_dpo_pairs_enth

Viewer • Updated 23 days ago • 5.28k • 98 • 1

Upvote

Collection guide
Browse collections