ClarusC64/cfir-stability-intervention-geometry-v0.1

CFIR v0.1 — Coupled Failure Intervention Reasoning Benchmark What this repo does

CFIR v0.1 is a synthetic benchmark designed to evaluate whether models can reason about stability and intervention geometry in coupled systems.

Many real-world failures occur not because systems lack information, but because they fail to interpret interacting pressures, buffers, delays, and couplings correctly.

This dataset tests whether a model can determine when an intervention will stabilize or fail to stabilize a system state.

The benchmark is intentionally constructed so that no single observable variable strongly correlates with the outcome.

Correct predictions require reasoning over interactions between variables.

Core quad

The dataset models system stability using four interacting signals.

pressure External or internal stress acting on the system.

buffer_capacity Available resilience or reserve capacity.

intervention_lag Delay between instability onset and corrective action.

system_coupling Degree to which subsystem disturbances propagate through the system.

These variables form a stability state vector.

Prediction target

Models must predict

labelinterventionstabilizing

Meaning

1 → the proposed intervention stabilizes the system 0 → the proposed intervention fails to stabilize the system

The label is computed from latent regime interactions rather than a single equation.

This prevents models from exploiting simple feature correlations.

Dataset design principles

The generator uses several mechanisms to enforce reasoning difficulty.

Latent regime mixing

Multiple hidden stability regimes determine the outcome.

The regime identity is not visible in the dataset.

The same surface state can therefore produce different outcomes depending on the hidden regime.

Nonlinear interaction geometry

Outcomes depend on interaction surfaces such as

pressure × coupling pressure × buffer buffer × coupling pressure × lag

This forces models to learn joint geometry rather than single-variable rules.

Polarity inversion

Variables sometimes push toward collapse and sometimes toward stabilization.

Example

high buffer can stabilize high buffer can also collapse under coupling cascade

This removes global directional signals.

Feature correlation suppression

The generator is designed so that individual feature correlations with the label remain close to zero.

Typical values

variable correlation pressure ~0.08 buffercapacity ~0.03 interventionlag ~0.20 system_coupling ~0.13

This ensures the task cannot be solved through simple heuristics.

Row structure

Each row represents a system state and a proposed intervention.

Fields

scenario_id Unique scenario identifier

pressure System pressure level

buffer_capacity Available resilience capacity

intervention_lag Delay between instability detection and response

system_coupling Strength of cross-subsystem propagation

proposed_intervention Suggested stabilization action

labelinterventionstabilizing Ground truth outcome (hidden in tester set)

Files

data/train.csv Training dataset with labels.

data/tester.csv Evaluation dataset without labels.

data/tester_key.csv Hidden answer key used by the scorer.

generator.py Synthetic dataset generator.

prediction_baseline.py Example baseline predictor.

scorer.py Evaluation script.

Evaluation

The scorer reports standard classification metrics.

accuracy precision recall f1

Two additional diagnostics are included.

recallstabilizinginterventions Ability to detect interventions that actually stabilize the system.

falseeffectiveintervention_rate Frequency of predicting stabilization when collapse still occurs.

These metrics emphasize correct stabilization reasoning rather than generic classification performance.

Baseline behavior

The provided baseline predictor intentionally performs near chance.

Typical scores

accuracy ≈ 0.45–0.55

This reflects the adversarial design of the dataset.

Models must learn nonlinear interactions to improve performance.

Intended research use

CFIR v0.1 can be used to study

stability reasoning intervention decision modeling interaction learning in structured state spaces robustness to regime shifts

The included generator allows researchers to produce larger datasets or probe additional failure patterns.

Structural note

CFIR is part of a broader effort to benchmark state-space intelligence.

Most modern AI benchmarks evaluate content interpretation.

CFIR instead evaluates whether models can reason about system stability geometry.

Instability is often detectable before collapse occurs.

The challenge is recognizing when intervention will succeed.

License