Datasetpublic

eltociear/sklearn-tasks-v1

sklearn-tasks-v1 Task dataset for a scikit-learn RL / eval environment, in the shape used by the Prime Intellect Environments Hub. 25 scikit-learn tasks across 5 categories. Each task names a fixed, seed-generated dataset and an instruction; the answer is the array left in result, compared to a reference. Grading is deterministic — no LLM judge, no external API, no download. The skill being tested is not array algebra. It is knowing which estimator, which preprocessing and —… See the full description on the dataset page: https://huggingface.co/datasets/eltociear/sklearn-tasks-v1.

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes35downloads

Dataset Card

sklearn-tasks-v1

Task dataset for a scikit-learn RL / eval environment, in the shape used by the Prime Intellect Environments Hub.

25 scikit-learn tasks across 5 categories. Each task names a fixed, seed-generated dataset and an instruction; the answer is the array left in result, compared to a reference. Grading is deterministic — no LLM judge, no external API, no download.

The skill being tested is not array algebra. It is knowing which estimator, which preprocessing and — mostly — which evaluation is correct: fitting a scaler before the split, reporting accuracy on a 90/10 imbalanced problem, or letting a pipeline leak through cross-validation are the mistakes this domain actually punishes, and the tasks are chosen to expose them.

Category	Tasks	Covers
preprocessing	6	StandardScaler, MinMaxScaler, PCA explained variance, SelectKBest, stratified train/test split shapes
classification	6	LogisticRegression, RandomForest, DecisionTree importances, an SVC pipeline, GradientBoosting, raw predictions
validation	5	KFold cross-validation, a scaler fitted inside each fold, score mean/std, KMeans inertia and cluster sizes
metrics	4	accuracy vs precision/recall/F1 on an imbalanced set, ROC AUC from `predict_proba`, confusion-matrix counts, class balance
regression	4	R²/MAE/RMSE, coefficients, Ridge, a scaled Ridge pipeline

Datasets

All three are generated in-process with a fixed seed — nothing is downloaded:

python

X_clf, y_clf = make_classification(n_samples=300, n_features=8, n_informative=5,
                                   n_redundant=1, n_classes=2, random_state=20260806)
X_imb, y_imb = make_classification(n_samples=400, n_features=6, n_informative=4,
                                   n_redundant=0, n_classes=2, weights=[0.9, 0.1],
                                   random_state=20260806)
X_reg, y_reg = make_regression(n_samples=200, n_features=5, n_informative=3,
                               noise=8.0, random_state=20260806)

What makes ML gradeable at all

—Every dataset comes from a fixed seed — no download, no network.
—Every estimator with a `random_state` gets one, and the prompt states it. An unseeded RandomForest is not a gradeable answer.
—References return small arrays of numbers (scores, coefficients, counts), rounded where rounding is safe — never a fitted model object.
—`n_jobs=1` everywhere. Thread scheduling changes floating-point reduction order, and an answer key that depends on the core count of the machine that built it is not an answer key.

Verification

Every task is independently checked: it runs, is deterministic across two fresh executions (which is what would catch an unseeded estimator or a thread-order dependence), returns finite JSON-safe numbers, is non-empty, and survives the serialisation round-trip exactly. All 25 pass on scikit-learn 1.9.0.

Builder and verifier: `build_tasks.py`.