allenai 's Collections
updated
A suite of models trained using DPO and PPO across a wide variety (up to 14) of preference datasets. See https://arxiv.org/abs/2406.09279 for more!
allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm
Text Generation •
Updated
•
90
•
6
Note Our overall best model, a 13B Tulu 2 model trained using PPO with a 70B reward model trained on UltraFeedback! We also release the value and reward models associated with this model - see the model card for details
allenai/tulu-2.5-preference-data
Viewer •
Updated
•
2.12M
•
648
•
18
Note The datasets used for training PPO, DPO, and reward models in our paper.
allenai/tulu-2.5-prompts
Viewer •
Updated
•
189k
•
62
•
4
Note The prompt sets used during PPO training in our paper. Below, see all our PPO-trained models!
allenai/tulu-v2.5-ppo-13b-uf-mean
Text Generation •
13B •
Updated
•
97
•
allenai/tulu-v2.5-ppo-13b-uf-mean-13b-mix-rm
Text Generation •
Updated
•
82
allenai/tulu-v2.5-ppo-13b-uf-mean-70b-mix-rm
Text Generation •
Updated
•
84
allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm-mixed-prompts
Text Generation •
Updated
•
80
Note Below is our PPO data ablations.
allenai/tulu-v2.5-ppo-13b-hh-rlhf-60k
Text Generation •
13B •
Updated
•
93
allenai/tulu-v2.5-ppo-13b-chatbot-arena-2023
Text Generation •
Updated
•
79
allenai/tulu-v2.5-ppo-13b-stackexchange-60k
Text Generation •
13B •
Updated
•
90
allenai/tulu-v2.5-ppo-13b-nectar-60k
Text Generation •
Updated
•
85
Note Below is our DPO data ablations.
allenai/tulu-v2.5-dpo-13b-uf-mean
Text Generation •
13B •
Updated
•
91
•
allenai/tulu-v2.5-dpo-13b-helpsteer
Text Generation •
Updated
•
81
allenai/tulu-v2.5-dpo-13b-shp2
Text Generation •
Updated
•
84
allenai/tulu-v2.5-dpo-13b-stackexchange
Text Generation •
Updated
•
86
allenai/tulu-v2.5-dpo-13b-uf-overall
Text Generation •
Updated
•
87
allenai/tulu-v2.5-dpo-13b-capybara
Text Generation •
Updated
•
81
allenai/tulu-v2.5-dpo-13b-prm-phase-2
Text Generation •
13B •
Updated
•
98
allenai/tulu-v2.5-dpo-13b-hh-rlhf
Text Generation •
13B •
Updated
•
104
•
1
allenai/tulu-v2.5-dpo-13b-nectar
Text Generation •
Updated
•
84
allenai/tulu-v2.5-dpo-13b-chatbot-arena-2023
Text Generation •
Updated
•
83
allenai/tulu-v2.5-dpo-13b-chatbot-arena-2024
Text Generation •
Updated
•
93
allenai/tulu-v2.5-dpo-13b-alpacafarm-human-pref
Text Generation •
Updated
•
84
allenai/tulu-v2.5-dpo-13b-alpacafarm-gpt4-pref
Text Generation •
Updated
•
83
allenai/tulu-v2.5-dpo-13b-hh-rlhf-60k
Text Generation •
13B •
Updated
•
107
allenai/tulu-v2.5-dpo-13b-stackexchange-60k
Text Generation •
13B •
Updated
•
100
•
1
allenai/tulu-v2.5-dpo-13b-argilla-orca-pairs
Text Generation •
13B •
Updated
•
97
allenai/tulu-v2.5-dpo-13b-nectar-60k
Text Generation •
Updated
•
86
•
1
Note Below are our reward models!
allenai/tulu-v2.5-13b-uf-rm
Text Classification •
13B •
Updated
•
76
allenai/tulu-v2.5-13b-preference-mix-rm
Text Classification •
13B •
Updated
•
79
allenai/tulu-v2.5-70b-uf-rm
Text Classification •
69B •
Updated
•
87
allenai/tulu-v2.5-70b-preference-mix-rm
Text Classification •
Updated
•
76
allenai/tulu-v2.5-13b-nectar-60k-rm
Text Classification •
13B •
Updated
•
73
allenai/tulu-v2.5-13b-chatbot-arena-2023-rm
Text Classification •
13B •
Updated
•
84
allenai/tulu-v2.5-13b-hh-rlhf-60k-rm
Text Classification •
13B •
Updated
•
74
•
1
allenai/tulu-v2.5-13b-stackexchange-60k-rm
Text Classification •
13B •
Updated
•
73
Note Below are our value models.
allenai/tulu-v2.5-ppo-13b-uf-mean-13b-uf-rm-value
Token Classification •
13B •
Updated
•
75
allenai/tulu-v2.5-ppo-13b-uf-mean-13b-mix-rm-value
Token Classification •
13B •
Updated
•
75
allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm-value
Token Classification •
69B •
Updated
•
72
allenai/tulu-v2.5-ppo-13b-uf-mean-70b-mix-rm-value
Token Classification •
69B •
Updated
•
71
allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm-mixed-prompts-value
Token Classification •
69B •
Updated
•
71
Note Below is llama 3 models:
allenai/llama-3-tulu-v2.5-8b-uf-mean-8b-uf-rm
8B •
Updated
•
88
•
3
allenai/llama-3-tulu-v2.5-8b-uf-mean-70b-uf-rm-mixed-prompts
8B •
Updated
•
79
•
2
allenai/llama-3-tulu-v2.5-8b-uf-mean-70b-uf-rm
8B •
Updated
•
78
•
1