This is a Dataset, not a Model
The following metrics do not apply: FNI Score, Deployment Options, Model Architecture
hh-rlhf
"--- license: mit tags: - human-feedback --- This repository provides access to two different kinds of data: 1. Human preference data about helpfulness and harmlessness from Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. These data are meant to train prefer..."
Best Scenarios
Technical Constraints
πΈοΈ Neural Graph Explorer
v15.13π Interest Trend
* Real-time activity index across HuggingFace, GitHub and Research citations.
Capabilities
- β Data Science
Finding datasets with similar distribution...
No benchmark correlations for this dataset.
π¬Deep Dive
Expand Details [+]βΎ
π οΈ Technical Profile
β‘ Hardware & Scale
π§ Training & Env
π Cloud & Rights
ποΈ Data Preview
| feature | label | split |
|---|---|---|
| example_text_1 | 0 | train |
| example_text_2 | 1 | train |
| example_text_3 | 0 | test |
| example_text_4 | 1 | validation |
| example_text_5 | 0 | train |
𧬠Schema & Configs
Fields
Dataset Card
Dataset Card for HH-RLHF
Dataset Summary
This repository provides access to two different kinds of data:
Disclaimer: The data (especially the harmlessness preference data and the red team data) contain content that may be offensive or upsetting. Topics include, but are not limited to, discriminatory language and discussions of abuse, violence, self-harm, exploitation, and other potentially upsetting subject matter. Please only engage with the data in accordance with your own personal risk tolerance. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of Anthropic or any of its employees. As mentioned above, these data are not intended for training dialogue agents as this will likely lead to harmful model behavior.
Each of these datasets are described further below.
Human preference data about helpfulness and harmlessness (PM Data)
The data are described in the paper: Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. If you find the data useful, please cite the paper. The data format is very simple -- each line of the jsonl files contains a pair of texts, one "chosen" and one "rejected".
For helpfulness, the data are grouped into train/test splits in three tranches: from our base models (context-distilled 52B language models), via rejection sampling (mostly with best-of-16 sampling) against an early preference model, and a dataset sampled during our iterated "online" process.
For harmlessness, the data are only collected for our base models, but otherwise formatted in the same way.
Details about the data collection process and crowdwork
Dataset Card for HH-RLHF
Dataset Summary
This repository provides access to two different kinds of data:
Disclaimer: The data (especially the harmlessness preference data and the red team data) contain content that may be offensive or upsetting. Topics include, but are not limited to, discriminatory language and discussions of abuse, violence, self-harm, exploitation, and other potentially upsetting subject matter. Please only engage with the data in accordance with your own personal risk tolerance. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of Anthropic or any of its employees. As mentioned above, these data are not intended for training dialogue agents as this will likely lead to harmful model behavior.
Each of these datasets are described further below.
Human preference data about helpfulness and harmlessness (PM Data)
The data are described in the paper: Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. If you find the data useful, please cite the paper. The data format is very simple -- each line of the jsonl files contains a pair of texts, one "chosen" and one "rejected".
For helpfulness, the data are grouped into train/test splits in three tranches: from our base models (context-distilled 52B language models), via rejection sampling (mostly with best-of-16 sampling) against an early preference model, and a dataset sampled during our iterated "online" process.
For harmlessness, the data are only collected for our base models, but otherwise formatted in the same way.
Details about the data collection process and crowdworker population can be found in the paper, specifically in section 2 and appendix D.
Red teaming data (not PM Data)
The data are described in the paper: Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned. If you find the data useful, please cite the paper. At a high level these data can be examined to understand what succesful red team attempts look like.
Details about the data and data collection procedures can be found in the Datasheet in the appendix of the paper.
Each line of the jsonl file contains a dictionary with the following fields:
transcripta text transcript of a conversation between a human adversary (red team member) and an AI assistantmin_harmlessness_score_transcripta real value score of the harmlessness of the AI assistant (lower is more harmful) as obtained from a preference modelnum_paramsnumber of parameters in the language model powering the AI assistantmodel_typetype of model powering the AI assistantratingthe red team member's rating of how successful they were at breaking the AI assistant (Likert scale, higher is more successful)task_descriptiona short text description written by the red team member about how they tried to red team the AI assistanttask_description_harmlessness_scorea real value score of the harmlessness of the task description (lower is more harmful) as obtained from a preference modelred_team_member_idan arbitrary identifier of the red team member. one red team member can generate multiple red team attacksis_upworkera binary indicator that is true if the red team member was from the crowd platform Upwork or false if they were from MTurktagsa list of up to 6 tags per transcript. tags are short descriptions of the red team attempts generated by crowdworkers who reviewed red team data post-hoc. tags were only provided for a random sample of 1000 red team attempts for two of four model types.
Usage
Each of the above datasets is located in a separate sub-directory. To load an individual subset, use the data_dir argument of the load_dataset() function as follows:
from datasets import load_dataset<h1 class="text-2xl font-bold mt-8 mb-4 text-gray-900 dark:text-white">Load all helpfulness/harmless subsets (share the same schema)</h1>
dataset = load_dataset("Anthropic/hh-rlhf")
<h1 class="text-2xl font-bold mt-8 mb-4 text-gray-900 dark:text-white">Load one of the harmless subsets</h1>
dataset = load_dataset("Anthropic/hh-rlhf", data_dir="harmless-base")
<h1 class="text-2xl font-bold mt-8 mb-4 text-gray-900 dark:text-white">Load the red teaming subset</h1>
dataset = load_dataset("Anthropic/hh-rlhf", data_dir="red-team-attempts")
Contact
The original authors host this dataset on GitHub here: https://github.com/anthropics/hh-rlhf You can submit inquiries to: [email protected]
5,724 characters total