Cenedril/Skillex-research

Skillex Research

Skillex Research is a deterministic, training-ready transformation of `Anthropic/enabling-independent-research` for the Gram/Skillex backend. It converts the upstream privacy-preserving aggregate cluster tables into:

• —clusters: a normalized evidence and retrieval view with stable identifiers, explicit attribution, confidence intervals, sparse cross-facet signals, and deterministic train/validation/test splits;
• —gram_skillex: paired supervised examples for cluster-facet classification and attributed evidence normalization. Each example includes both a standard messages field and plain input_text / target_text.

This release contains 2,077 aggregate cluster records and 3,642 supervised examples. It contains no raw conversations, user identifiers, organization identifiers, hidden reasoning traces, or synthetic claims about individual people.

Intended use

Use this dataset to train or evaluate a small routing or normalization model that helps Gram/Skillex:

1. 1.classify aggregate research clusters by study, facet, and hierarchy level;
2. 2.convert sparse aggregate statistics into a stable, attributed evidence record;
3. 3.retrieve cluster evidence for research planning while retaining uncertainty and provenance.

Do not use it as conversation-response SFT data. The source contains aggregate cluster summaries, not individual dialogue turns. It also does not provide ground truth for a person's emotion, intent, success, or supervision state.

Schema

clusters

Core fields include record_id, study, source_cluster_id, facet_id, cluster_name, cluster_description, hierarchy_level, sample counts, prevalence and its 95% confidence interval, signal_groups_json, source revision, source file, and license.

signal_groups_json keeps the upstream sparse cross-tabs losslessly as compact JSON. Explicit Parquet schemas retain nullable numeric types across every split. Missing values remain absent; they are never converted to zero.

gram_skillex

Each source cluster produces two examples:

• —classify_cluster: descriptive cluster text to {study, facet_id, hierarchy_level};
• —normalize_evidence: aggregate row data to skillex-research-evidence/1 JSON.

Closed categorical rows with empty descriptions are excluded from classify_cluster, because repeated values such as 1 or not_applicable do not contain enough information to recover their source facet. All rows remain in normalize_evidence. Examples derived from the same cluster always stay in the same split. Assistant targets contain only concise, structured outputs; no hidden reasoning is included.

Loading

from datasets import load_dataset

clusters = load_dataset("Cenedril/Skillex-research", "clusters")
training = load_dataset("Cenedril/Skillex-research", "gram_skillex")

For chat-template training, use the messages field. For a custom backend, use input_text, target_text, task_type, and source_record_id.

Limitations and safeguards

• —Cluster labels and descriptions are model-generated interpretations of aggregate conversations, not validated descriptions of every conversation assigned to a cluster.
• —Upstream validation reported that some conversations were not clearly described by their assigned cluster and that labels can emphasize concerning examples.
• —Open-ended emotion and behavior facets are not individual-level ground truth.
• —The upstream sample covers Free, Pro, and Max usage from a fixed April-May 2026 window; it excludes Team, Enterprise, and API data and is not representative of all users.
• —A harmful-request cluster describes what users asked for, not whether a safeguard blocked it or an action occurred.
• —Empty cross-tab cells can mean "not computed" or "below the privacy threshold". They must not be read as zero.
• —Deterministic targets make the normalization task auditable, but they do not add new human annotations.

The supervised system messages reinforce five constraints: aggregate evidence only, no individual inference, model-generated cluster labels, missing-is-not-zero semantics, and non-representativeness.

Reproducibility

The build is pinned to upstream revision b1ef5f7248aaae61eac4241052a05bb583c77942. Source files are verified by SHA-256 before transformation. Output order, identifiers, splits, JSON serialization, schemas, and Parquet writer settings are deterministic.

python -m pip install -r requirements-build.txt
python build_dataset.py --output-dir .

See validation_report.json for row counts, split counts, source hashes, and completed checks.

Run the independent release validator with:

python validate_dataset.py

License and attribution

The transformed dataset is released under CC BY 4.0, matching the upstream dataset. Attribution and the upstream citation are recorded in `ATTRIBUTION.md`. Users must preserve attribution when redistributing or adapting this work.