Datasetpublic

KinGeorge/Dr.Sparse-OTF-test-set

Dr.Sparse OTF Test Set 100 sparse matrices from the SuiteSparse Matrix Collection, converted to the flat binary format the Dr.Sparse benchmark harness reads. This is the held-out evaluation set for LLM-generated CUDA sparse kernels (SpMV / SpMM / SpGEMM), kept separate from the matrices the models were developed against. Layout Matrices are grouped into size tiers by row count, the convention Dr.Sparse task discovery scans for: tier rows matrices size… See the full description on the dataset page: https://huggingface.co/datasets/KinGeorge/Dr.Sparse-OTF-test-set.

sourceHugging Facecc-by-4.0updated 21d agoView on Hugging Face

0likes111downloads

Dataset Card

Dr.Sparse OTF Test Set

100 sparse matrices from the SuiteSparse Matrix Collection, converted to the flat binary format the Dr.Sparse benchmark harness reads. This is the held-out evaluation set for LLM-generated CUDA sparse kernels (SpMV / SpMM / SpGEMM), kept separate from the matrices the models were developed against.

Layout

Matrices are grouped into size tiers by row count, the convention Dr.Sparse task discovery scans for:

tier	rows	matrices	size
`level1_small`	< 3,500	8	2.6 MB
`level2_medium`	3,500 – 34,000	35	155 MB
`level3_large`	34,000 – 1,100,000	38	2.1 GB
`level4_huge`	> 1,100,000	19	8.6 GB

summary.csv and manifest.json carry per-matrix rows, cols, nnz, kind, square, avg_row and density. Row counts span 800 to 67.7M and nnz spans 10.4K to 138.8M.

Binary format

Each .bin is CSR plus a dense vector, little-endian, no padding:

int32   num_rows
int32   num_cols
int32   nnz
int32   row_ptr[num_rows + 1]
int32   col_ind[nnz]
float32 values[nnz]
float32 x[num_cols]          # dense RHS vector, for SpMV

Reading one in Python:

python

import numpy as np

def read_bin(path):
    with open(path, "rb") as f:
        rows, cols, nnz = np.fromfile(f, dtype=np.int32, count=3)
        row_ptr = np.fromfile(f, dtype=np.int32, count=rows + 1)
        col_ind = np.fromfile(f, dtype=np.int32, count=nnz)
        values  = np.fromfile(f, dtype=np.float32, count=nnz)
        x       = np.fromfile(f, dtype=np.float32, count=cols)
    return rows, cols, nnz, row_ptr, col_ind, values, x

The C++ side of the harness reads the same layout in level_sparse/data_loader.h.

Use with Dr.Sparse

The tiers match the layout Dr.Sparse task discovery scans, so downloading straight into level_sparse/otf_test_set/ needs no staging step:

bash

hf download KinGeorge/Dr.Sparse-OTF-test-set --repo-type dataset \
    --local-dir level_sparse/otf_test_set        # all 100, 11.6 GB

./run_b200_eval_parallel.sh --model qwen3.8 --dry-run

level4_huge is 8.6 GB of the total, so for a quick smoke test pull only the small tiers. Note that --include takes one pattern per flag — passing several after a single --include silently matches nothing:

bash

hf download KinGeorge/Dr.Sparse-OTF-test-set --repo-type dataset \
    --local-dir level_sparse/otf_test_set \
    --include "level1_small/*" --include "level2_medium/*"    # 158 MB

If you already have the .bin files elsewhere on disk, point the staging script at them instead and it will lay the tiers out as symlinks:

bash

python level_sparse/stage_otf_test_set.py --src /path/to/flat/bins

Provenance and licence

Matrices are redistributed from the SuiteSparse Matrix Collection (Davis & Hu, The University of Florida Sparse Matrix Collection, ACM TOMS 38(1), 2011), whose entries are CC BY 4.0. Only the storage format was changed: the .mtx sources were converted to CSR (single precision) and a dense x vector drawn from a standard normal was appended so SpMV has a fixed right-hand side. That vector is baked into each file, so results are reproducible across runs, but it was not generated from a recorded seed and is not reproducible from the .mtx source alone. Matrix values are otherwise unmodified. Please cite SuiteSparse if you use this set.