solsticestudioai/mnemosyne-memory-lifecycle-benchmark

Mnemosyne Memory Lifecycle Benchmark Dataset v0.2.0

This deterministic synthetic benchmark contains 40 scenarios across eight categories and five documented reference strategies. Strategy execution is blind to expected labels; execution and evaluation are separate phases.

Current results are fixture results, not real-world memory accuracy. Simple strategies remain competitive on straightforward current-state questions, while typed lineage uniquely reconstructs the historical point in the current fixtures. Twenty-five of 40 scenarios are currently non-discriminating and are preserved rather than hidden. Metrics are not_applicable when no valid denominator exists. Logical erasure is not physical deletion, and this benchmark does not establish superiority over production memory systems.

Public links:

• —Space: https://huggingface.co/spaces/solsticestudioai/mnemosyne-memory-lab
• —Direct app: https://solsticestudioai-mnemosyne-memory-lab.static.hf.space/
• —Collection: https://huggingface.co/collections/solsticestudioai/solstice-agent-reliability-lab
• —Methodology: docs/METHODOLOGY.md
• —Scenario design: docs/SCENARIO_DESIGN.md
• —Metric validity: docs/METRIC_VALIDITY.md
• —Limitations: docs/LIMITATIONS.md
• —Validity summary: reports/benchmark-validity-summary.md
• —Claim evidence: reports/claim-evidence-matrix.md

Test Your Own Memory System

Tooling release v0.2.1 provides a provider-neutral, label-free subprocess protocol. Clone or download the package, copy an adapter template, implement JSON stdin/stdout, execute against the frozen scenarios, evaluate separately, build a result bundle, and load it into the Space locally. No result is uploaded automatically; there is no public leaderboard; users control whether they share anything. External outputs are community-provided and unverified.