Modelpublic

marin-community/marin-data-mix-tools

sourceHugging Faceopenmdw-1.1updated 2d agoView on Hugging Face

3likes

Model Card

Marin data mix tools

This repository publishes the model materials used to score and cluster documents for the August 18, 2026 Harrier data mix.

—fast-transformer/ contains Mark Muchane's nemotron88k_v1 FastTransformer checkpoint. It consumes Nemotron-tokenized document text and a 1024-dimensional Harrier embedding, and emits a scalar document-quality score.
—kmeans/ contains 5,000 spherical K-means centroids. lookup_5000_to_40.npy maps each fine cluster ID to one of 40 coarse domain buckets.
—manifest.json records each file's upstream S3 URI, byte size, SHA-256 digest, and license identifier.

All model materials under this version are licensed under the OpenMDW License Agreement, version 1.1.