Dataset Card for Dayhoff

Dayhoff is an Atlas of both protein sequence data and generative language models — a centralized resource that brings together 3.34 billion protein sequences across 1.7 billion clusters of metagenomic and natural protein sequences (GigaRef), 46 million structure-derived synthetic sequences (BackboneRef), and 16 million multiple sequence alignments (OpenProteinSet). These models can natively predict zero-shot mutation effects on fitness, scaffold structural motifs by conditioning on evolutionary or structural context, and perform guided generation of novel proteins within specified families. Learning from metagenomic and structure-based synthetic data from the Dayhoff Atlas increased the cellular expression rates of generated proteins, highlighting the real-world value of expanding the scale, diversity, and novelty of protein sequence data.

The Dayhoff model architecture combines state-space Mamba layers with Transformer self-attention, interleaved with Mixture-of-Experts modules to maximize capacity while preserving efficiency. It natively handles long contexts, allowing both single sequences and unrolled MSAs to be modeled. Trained with an autoregressive objective in both N→C and C→N directions, Dayhoff supports order-agnostic infilling and scales to billions of parameters.

Dataset Structure

The Dayhoff models were trained on the Dayhoff Atlas, with varying data mixes which include:

* UniRef50 (UR50) - dataset from UniProt, clustered at 50% sequence identity, contains only cluster representatives. * _Splits: train (25 GB), test (26 MB), valid (26 MB)_

* UniRef90 (UR90) - dataset from UniProt, clustered at 90% sequence identity, contains cluster representatives and members. * _Splits: train (83 GB), test (90 MB), valid (87 MB)_

* GigaRef (GR)– 3.34B protein sequences across 1.7B clusters of metagenomic and natural protein sequences. There are two subsets of gigaref: * GigaRef-clusters (GR) - Only includes cluster representatives and members, no singletons * _Splits: train (433 GB), test (22 MB)_ * GigaRef-singletons (GR-s) - Only includes singletons * _Splits: train (282 GB)_

* BackboneRef (BR) – 46M structure-derived synthetic sequences from c.a. 240,000 de novo backbones, with three subsets containing 10M sequences each: * BackboneRef unfiltered (BRu) – 10M sequences randomly sampled from all 46M designs. * _Splits: train (3 GB)_ * BackboneRef quality (BRq) – 10M sequences sampled from 127,633 backbones whose average self-consistency RMSD ≤ 2 Å. * _Splits: train(3 GB)_ * BackboneRef novelty (BRn) – 10M sequences from 138,044 backbones with a max TM-score < 0.5 to any natural structure. * _Splits: train (3GB)_ * BackboneRef structures (backboneref-structures) - We also make available the structures used to generate BackboneRef datasets * OpenProteinSet (HM) – 16 million precomputed MSAs from 16M sequences in UniClust30 and 140,000 PDB chains. (_Not currently available in HF_)

DayhoffRef

Given the potential for generative models to expand the space of proteins and their functions, we used the Dayhoff models to generate DayhoffRef, a PLM-generated database of synthetic protein sequences

* DayhoffRef: dataset of 16 million synthetic protein sequences generated by the Dayhoff models: Dayhoff-3b-UR90, Dayhoff-3b-GR-HM, Dayhoff-3b-GR-HM-c, and Dayhoff-170m-UR50-BRn. * _Splits: train (5 GB)_

Uses

Direct Use

The dataset is intended for training protein language models.

Sample code

To read any of the datasets use Hugging Face's load_dataset, which will use the Arrow format for maximum efficieny. Below are some examples on how to load the datasets:

gigaref_singletons_train = load_dataset("microsoft/Dayhoff",
                  name="gigaref_only_singletons",
                  split="train")
gigaref_clustered_train = load_dataset("microsoft/Dayhoff",
                  name="gigaref_no_singletons",
                  split="train")
uniref50_train = load_dataset("microsoft/Dayhoff",
                  name="uniref50",
                  split = "train")
uniref90_train = load_dataset("microsoft/Dayhoff",
                  name="uniref90",
                  split = "train")
backboneref_novelty = load_dataset("microsoft/Dayhoff",
                  name="backboneref",
                  split = "BRn")
                
dayhoffref = load_dataset("microsoft/Dayhoff",
                  name="dayhoffref",
                  split = "train")

For the largest datasets, consider using streaming = True.

Out-of-Scope Use Cases

This model should not be used to generate anything that is not a protein sequence or a set of homologuous protein sequences. It is not meant for natural language or other biological sequences, such as DNA sequences.

Dataset Creation

Curation Rationale

The motivation for creating the Dayhoff Atlas was to systematically combine genomic-derived protein sequences, metagenomics, structure-based synthetic sequences, and homologs to enhance protein language models (PLMs). With this dataset we aim to expand the diversity and scale of natural protein sequences available for model training, infuse structural information into sequence space, and unify different sources of protein data into a centralized resource

Gigaref

#### Data Collection and Processing

A multi-step clustering process was performed to combine, deduplicate, and recluster these datasets into GigaRef. MERC and SRC were combined and preclustered at 70\% identity, and MGnify was preclustered at 70\% identity. The representatives resulting from each of these preclustering steps were then combined with sequences from SMAG, MetaEuk, MGV, GPD, TOPAZ, and UniRef100, yielding a total of 3.94 billion sequences. These were clustered at 90\% sequence identity followed by clustering of the resulting representatives at 50\% identity to produce the GigaRef dataset.

#### Who are the source data producers?

The source data producers include various metagenomic databases such as MGnify, Soil Metagenome-Assembled Genomes (SMAG), MetaEuk, Metagenomic Gut Virus catalog (MGV), Gut Phage Database (GPD), Soil Reference Catalog (SRC), Marine Eukaryotic Reference Catalog (MERC), and Tara Oceans Particle-Associated MAGs (TOPAZ).

BackboneRef

#### Data Collection and Processing

BackboneRef was generated by sampling backbone structures from RFDiffusion and designing synthetic sequences using ProteinMPNN. Structures were sampled according to the length distribution of UniRef50, to recapitulate the lengths of natural proteins but with a minimum length of 40 and maximum length of 512 for computational efficiency. An initial set of 300 sequences per backbone at temperature 0.2 was generated, producing a parent dataset with 72,243,300 sequences. To create the BRu dataset, 42 sequences were selected per backbone from this parent dataset, yielding 10,114,860 sequences. Exact duplicates were removed, then the remaining sequences were randomly subsampled to produce a dataset of 10M sequences.

To create the BRq dataset, backbones with average scRMSD score greater than 2Å were removed, leaving 127,633 backbones. 80 sequences per backbone were randomly selected from the parent dataset, and again exact duplicates were removed, followed by random subsampling to produce a dataset of 10M sequences.

To create the BBn dataset, any backbones with maximum TM-score larger than 0.5 to any structure in the AFDB/UniProt database were removed, leaving 138,044 backbones. 74 sequences per backbone were randomly sampled from the parent dataset, and again exact duplicates were removed, followed by random subsampling to produce a dataset of 10M sequences.

#### Who are the source data producers

This is a synthetically generated dataset.

DayhoffRef

#### Data Collection and Processing

DayhoffRef sequences were sampled in both the N-to-C and C-to-N directions from Dayhoff-3b-GR-HM, Dayhoff-3b-GR-HM-C, and Dayhoff-170m-UR50-BRn at temperatures 0.9 and 1.0 on 8x AMD MI300X GPUs for each combination of model, direction, and temperature. Clustering to 90% identity was performed with MMseqs2 using default parameters, and then the cluster representatives were clustered to 50\% identity using default parameters.

#### Who are the source data producers

This is a synthetically generated dataset.

Bias, Risks, and Limitations

This model should not be used to generate anything that is not a protein sequence or a set of homologuous protein sequences. It is not meant for natural language or other biological sequences, such as DNA sequences. Not all sequences are guaranteed to be realistic. It remains difficult to generate high-quality sequences with no sequence homology to any natural sequence.

Responsible AI Considerations

The intended use of this model is to generate high-quality, realistic, protein sequences or sets of homologous protein sequences. Generations can be designed from scratch or conditioned on partial sequences in both N→C and C→N directions.

The code and datasets released in this repository are provided for research and development use only. They are not intended for use in clinical decision-making or for any other clinical use, and the performance of these models for clinical use has not been established. You bear sole responsibility for any use of these models, data and software, including incorporation into any product intended for clinical use.