Datasetpublic

arushram/fnbm-current-gt-motif-effects-precision-20260729

fnbm-current-gt-motif-effects-precision-20260729 Recovery of planted motif effects after de-duplication, scored against simulation ground truth. Effect sizes are the OLS slope of each motif family's post-clustering per-example contribution on the motif's true occurrence count -- the same units as the simulator's beta, and invariant to the de-duplication pipeline's internal gauge. Per-example contribution MSE is computed on centered contributions over the validation split. See… See the full description on the dataset page: https://huggingface.co/datasets/arushram/fnbm-current-gt-motif-effects-precision-20260729.

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes10downloads

Dataset Card

fnbm-current-gt-motif-effects-precision-20260729

Recovery of planted motif effects after de-duplication, scored against simulation ground truth. Effect sizes are the OLS slope of each motif family's post-clustering per-example contribution on the motif's true occurrence count -- the same units as the simulator's beta, and invariant to the de-duplication pipeline's internal gauge. Per-example contribution MSE is computed on centered contributions over the validation split. See the effect_gauge column note: the pipeline's own 'effect' scalar is NOT comparable to beta and is included only for continuity.

Dataset Info

—Rows: 720
—Columns: 27

Columns

Column	Type	Description
run	Value('large_string')	Run directory name; encodes the grid combination.
epoch	Value('int64')	Training epoch the de-duplication pass was run at.
motif_idx	Value('int64')	Index of the planted motif, matching the z1_{i} column of the ground-truth parquet.
beta_gt	Value('float64')	Planted first-order effect size: output units per single motif occurrence, defined by z1[n,i] = betai * counti(n) with the scaled beta recorded in sim_config.json.
gtcontributionstd	Value('float64')	Standard deviation of the planted per-example contribution.
mean_occurrences	Value('float64')	Mean occurrences of this motif per validation sequence. Low values mean beta_hat is estimated from few informative examples -- read it with the error bar.
nfamiliesmatched	Value('int64')	Number of motif families (Tomtom-connected components) attributed here.
family_ids	Value('large_string')	Comma-separated ids of those families.
n_clusters	Value('int64')	Number of de-duplication clusters attributed to this motif.
cluster_ids	Value('large_string')	Comma-separated ids of those clusters, so a motif can be traced back to the cluster figures in the run's own de-duplication report.
filter_ids	Value('large_string')	Comma-separated indices of those convolutional filters in the trained model.
n_filters	Value('int64')	Number of convolutional filters behind those clusters.
familymatchconflict	Value('bool')	True when a family's clusters matched DIFFERENT planted motifs, so attribution fell back to per-cluster. Happens when two planted motifs resemble each other below the clustering threshold.
effect_gauge	Value('float64')	The de-duplication pipeline's own effect scalar, sumi w1[i]*alphai. GAUGE-DEPENDENT: alpha absorbs the 99.9th-percentile canonical-trace scale, so this is the response to a canonical occurrence, NOT per literal occurrence, and the factor differs per cluster. Reported for continuity; never compare it to beta_gt.
matched	Value('bool')	Whether any cluster's consensus PWM matched this planted motif by Tomtom.
beta_hat	Value('float64')	Recovered effect size: OLS slope of the model's post-clustering per-example contribution on the motif's true occurrence count. Same units as beta_gt. Gauge-invariant -- deliberately NOT the pipeline's own 'effect' scalar.
betahatse	Value('float64')	Standard error of beta_hat.
contribution_corr	Value('float64')	Pearson r between the model's per-example contribution and the planted z1_{i}, on the validation split. Sign matters: negative means the model assigned this motif the opposite sign to the truth.
modelcontributionstd	Value('float64')	Standard deviation of the model's per-example contribution.
mse	Value('float64')	Mean squared error between model and planted per-example contribution, both centered. Centered because contributions are identified only up to an additive constant.
nmse	Value('float64')	mse divided by the variance of the planted contribution. 1.0 = no better than predicting the mean; 0 = exact recovery. The primary per-motif quality number.
offset	Value('float64')	mean(model contribution) - mean(planted contribution); the constant removed before MSE.
countsloper	Value('float64')	Pearson r of the beta_hat regression (model contribution vs occurrence count).
interactioncontextdc_penalty	Value('float64')	No description provided
interactionfeaturel2_penalty	Value('float64')	No description provided
seed	Value('int64')	No description provided
arm	Value('large_string')	Which experimental arm the run belongs to (precision or sensitivity).

Generation Parameters

json

{
  "script_name": "scripts/publish_motif_gt_eval.py",
  "model": "FactorizedNBM(fnbm_20260729)",
  "description": "Recovery of planted motif effects after de-duplication, scored against simulation ground truth. Effect sizes are the OLS slope of each motif family's post-clustering per-example contribution on the motif's true occurrence count -- the same units as the simulator's beta, and invariant to the de-duplication pipeline's internal gauge. Per-example contribution MSE is computed on centered contributions over the validation split. See the effect_gauge column note: the pipeline's own 'effect' scalar is NOT comparable to beta and is included only for continuity.",
  "experiment_name": "fnbm-current",
  "experiment_id": "fnbm-current",
  "artifact_type": "eval_result",
  "visualizer_type": "table",
  "run_id": "torch:15097728-15097743,15166724-15166731",
  "job_id": "torch:15097728-15097743,15166724-15166731",
  "cluster": "torch",
  "artifact_status": "final",
  "canary": false,
  "runs": 24,
  "epochs": [
    0,
    50,
    100
  ],
  "input_datasets": [],
  "hyperparameters": {}
}

Usage

python

from datasets import load_dataset

dataset = load_dataset("arushram/fnbm-current-gt-motif-effects-precision-20260729", split="train")
print(f"Loaded {len(dataset)} rows")