Datasetpublic

TerenceLau/nanoJEPA-base

nanoJEPA EN/ZH Ultra-FineWeb Dataset This is a small pretraining dataset package for nanoJEPA. It is built by streaming openbmb/Ultra-FineWeb split en and/or zh. Files train.jsonl: {"text": "...", "source": "...", "dataset": "...", "language": "en|zh", "score": 0.0} valid.jsonl: same schema as train.jsonl test.jsonl: same schema as train.jsonl Generation Command uv run python data/build_hf_dataset.py \ --out-dir dataset/nanojepa-small \… See the full description on the dataset page: https://huggingface.co/datasets/TerenceLau/nanoJEPA-base.

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes77downloads

Dataset Card

nanoJEPA EN/ZH Ultra-FineWeb Dataset

This is a small pretraining dataset package for nanoJEPA. It is built by streaming openbmb/Ultra-FineWeb split en and/or zh.

Files

—train.jsonl: {"text": "...", "source": "...", "dataset": "...", "language": "en|zh", "score": 0.0}
—valid.jsonl: same schema as train.jsonl
—test.jsonl: same schema as train.jsonl

Generation Command

bash

uv run python data/build_hf_dataset.py \
  --out-dir dataset/nanojepa-small \
  --languages en,zh \
  --max-en 600000 \
  --max-zh 600000 \
  --train-ratio 0.8 \
  --valid-ratio 0.05 \
  --test-ratio 0.15

Stats

json

{
  "sources": {
    "train": {
      "rows": 960000,
      "scanned": 984987,
      "path": "train.jsonl",
      "parts": {
        "en": {
          "rows": 480000,
          "scanned": 480039,
          "source": "openbmb/Ultra-FineWeb/default",
          "config": "default",
          "language": "en"
        },
        "zh": {
          "rows": 480000,
          "scanned": 504948,
          "source": "openbmb/Ultra-FineWeb/default",
          "config": "default",
          "language": "zh"
        }
      }
    },
    "valid": {
      "rows": 60000,
      "scanned": 61790,
      "path": "valid.jsonl",
      "parts": {
        "en": {
          "rows": 30000,
          "scanned": 30002,
          "source": "openbmb/Ultra-FineWeb/default",
          "config": "default",
          "language": "en"
        },
        "zh": {
          "rows": 30000,
          "scanned": 31788,
          "source": "openbmb/Ultra-FineWeb/default",
          "config": "default",
          "language": "zh"
        }
      }
    },
    "test": {
      "rows": 180000,
      "scanned": 185045,
      "path": "test.jsonl",
      "parts": {
        "en": {
          "rows": 90000,
          "scanned": 90003,
          "source": "openbmb/Ultra-FineWeb/default",
          "config": "default",
          "language": "en"
        },
        "zh": {
          "rows": 90000,
          "scanned": 95042,
          "source": "openbmb/Ultra-FineWeb/default",
          "config": "default",
          "language": "zh"
        }
      }
    }
  },
  "dataset": "openbmb/Ultra-FineWeb",
  "dataset_config": "default",
  "languages": [
    "en",
    "zh"
  ],
  "seed": 42,
  "split_ratios": {
    "train": 0.8,
    "valid": 0.05,
    "test": 0.15
  },
  "min_score": null,
  "no_merge": false
}