rjsabouhi/trajectory-graph-monitoring

Trajectory Graph Monitoring v0.2; Research Release

Prospective boundary prediction from cumulative typed trajectory transitions R.J. Sabouhi · Symbolic Suite · August 2026

What this is

Trajectory Graph Monitoring is a prospective runtime-evaluation benchmark. It asks whether structural information available in an incomplete execution trajectory can improve prediction of a boundary violation that occurs later. Every evaluated prefix ends before the violating action, so the task is prospective rather than retrospective.

The frozen corpus contains 150 episodes and 522 prospective prefixes across three task families (access_request, bug_repair, report_delivery). The original trajectory-generation notebook records Qwen/Qwen2.5-1.5B-Instruct as the model used in the experimental pipeline.

Frozen primary result - Step 06

Step 06 is the frozen primary prospective evaluation. It must be kept separate from the Step 07 alternate-split robustness run.

Primary AUPRC delta over the semantic baseline: +0.0841 (about +22.8% relative). Episode-level paired bootstrap, 1,000 replicates: mean delta +0.081859, 95% percentile interval [0.014684, 0.150878], empirical P(delta > 0) = 0.990.

Robustness result - Step 07

Step 07 used a different grouped-split seed and produced separate out-of-fold predictions. It evaluates the same cumulative typed-transition feature concept but is not the same scored run as Step 06.

Run-matched semantic AUPRC: 0.3399. Full typed-count graph-enhanced AUPRC: 0.4173. Point delta: +0.0774. Episode-bootstrap mean delta: +0.084381, 95% interval [0.014844, 0.161754], empirical P(delta > 0) = 0.994.

Step 07 also contains typed-count ablations, row-shuffle and Gaussian-noise controls, horizon sensitivity, leave-one-family-out analysis, calibration, and coefficient diagnostics.

Transition representation

The graph feature block contains cumulative counts for six transition types:

• —ADD_AUTH_EDGE
• —ADD_CONTROL_EDGE
• —ADD_ENTITY
• —ADD_MEANS_END_EDGE
• —RELABEL_CONSTRAINT
• —REMOVE_CONSTRAINT_EDGE

This is deliberately not a full graph-topology representation. It does not preserve exact event order, timestamps, durations, arbitrary connectivity, or complete transition semantics.

Repository layout

paper/       technical report and seven publication figures
data/        frozen trajectory and prefix evaluation data
notebooks/   Step 06 primary evaluation, Step 07 robustness, packaging notebook
results/     frozen Step 06 and Step 07 outputs kept in separate provenance folders
docs/        PREP protocol, replication packet, methods, limitations, environment receipt

Reproduction notes

The recorded execution environment used Python 3.12.13. Step 06 records PyTorch 2.11.0+cu128 on a Tesla T4. Step 07 records NumPy 2.0.2, pandas 3.0.5, scikit-learn 1.9.0, and SciPy 1.18.0. See the notebooks and environment receipt for the original execution details.

Recommended order:

1. 1.Inspect docs/Methods.md and the PREP protocol.
2. 2.Inspect the frozen data in data/.
3. 3.Run notebooks/06_Trajectory_Graph_Monitoring_Prospective_Predictive_Evaluation.ipynb for the primary evaluation.
4. 4.Run notebooks/07_Trajectory_Graph_Monitoring_Robustness_Ablation_and_Leakage.ipynb for the separate robustness analysis.
5. 5.Compare generated outputs with the frozen files under results/.

Scientific status

This is an internally completed and audited benchmark study prepared for public technical release. Independent external replication has not yet been completed. A frozen external-replication packet is included. Null, reversed, or domain-conditional replications are scientifically valid outcomes and should be preserved.

The current evidence supports a narrow claim: within the frozen benchmark, cumulative typed trajectory-transition counts improved prospective prediction of near-term boundary violations beyond the tested semantic and telemetry baselines. It does not establish causal mechanism, universal generality, deployment readiness, or intervention effectiveness.

License

CC Attribution 4.0

Citation

DOI 10.5281/zenodo.22018521