CoolFace
Datasetpublic

marin-dna/zoonomia-rag-v1-v1

bolinas-dna/zoonomia-rag-v1-v1 Fixed-layout 2,048-token documents built from conservation-filtered GRCh38 255-base anchors, seven fixed Zoonomia mammalian ortholog slots, and a final human slot. Missing non-human projections are filled with 255 N bases; chromosome 18 is validation-only. Produced by the commit-pinned issue #402 RAG pipeline. The immutable upstream input is the existing Zoonomia v1 min0.20/all_species_with_sequence.parquet projection. No halLiftover was run for… See the full description on the dataset page: https://huggingface.co/datasets/marin-dna/zoonomia-rag-v1-v1.

sourceHugging Faceupdated 2mo agoView on Hugging Face
0likes237downloads
Dataset Card

bolinas-dna/zoonomia-rag-v1-v1

Fixed-layout 2,048-token documents built from conservation-filtered GRCh38 255-base anchors, seven fixed Zoonomia mammalian ortholog slots, and a final human slot. Missing non-human projections are filled with 255 N bases; chromosome 18 is validation-only.

Produced by the commit-pinned issue #402 RAG pipeline. The immutable upstream input is the existing Zoonomia v1 min0.20/all_species_with_sequence.parquet projection. No halLiftover was run for this dataset.

Splits

SplitDocumentsDistinct human anchorsOrientation
train2,220,3561,110,178forward + whole-document reverse complement
validation2,0482,048forward only

Training excludes every chromosome 18 anchor. Validation is a deterministic 2,048-anchor chromosome 18 sample (blake2b-64-v1, seed 42) and contains exactly 4,194,304 input tokens.

Fixed slot order (zoonomia-rag-v1)

  1. 1.Microgale talazaci — 13.57% missing
  2. 2.Loxodonta africana — 8.05% missing
  3. 3.Tolypeutes matacus — 13.68% missing
  4. 4.Bos taurus — 8.00% missing
  5. 5.Equus caballus — 7.28% missing
  6. 6.Mus musculus — 9.73% missing
  7. 7.Microcebus murinus — 8.36% missing
  8. 8.Homo sapiens — 0.00% missing

Slot 7 is always human. Every raw slot is exactly 255 bases. Documents are assembled as slot_0[SEQ]... [SEQ]slot_7; [SEQ] is one atomic token and one BOS token is prepended as CLS. The final human segment is token interval [1793, 2048). Raw N bases map to the tokenizer unknown token.

Provenance

  • —Reference: GRCh38 (Ensembl release 115)
  • —Conservation: phyloP_447m >= 2.2162 at least 0.20 of positions in each 255-base window
  • —Alignment: Zoonomia 447-mammalian 2022 v1 Cactus HAL
  • —Coordinates: 0-based, half-open
  • —Producing commit: 8b4f4e159bfcc9bcbc326a02812af16e5de0a1b4

manifest.json contains exact row counts, seeds, token positions, and per-slot missingness.