Datasetpublic

Multilingual-Multimodal-NLP/AutoMemoryBench

AutoMemoryBench State-Contract Evaluation for Auditable Agent Memory AutoMemoryBench evaluates whether an agent uses the right memory, and only the admissible memory, under a query-time state contract. Each executable contract partitions memory into required, admissible, and prohibited sets. Prohibited memories are typed as superseded, deleted, restricted, cross-namespace, or stale-tool. Relevance is not enough: remembered evidence must also be… See the full description on the dataset page: https://huggingface.co/datasets/Multilingual-Multimodal-NLP/AutoMemoryBench.

sourceHugging Faceupdated 1mo agoView on Hugging Face

0likes151downloads

rootdatasamplemanifest.json

manifest.json29 linesDownload Raw Back to sample

1{2  "benchmark_id": "amst-sample-v1",3  "schema_version": "1.0.0",4  "package_type": "sample",5  "visibility": {6    "audit_subset": "public"7  },8  "included_splits": [9    "audit_subset"10  ],11  "split_files": {12    "audit_subset": {13      "personal_assistant": "shards/personal_assistant.json",14      "education_tutoring": "shards/education_tutoring.json",15      "customer_support": "shards/customer_support.json",16      "devops_workflow": "shards/devops_workflow.json",17      "research_assistant": "shards/research_assistant.json",18      "coding_agent": "shards/coding_agent.json",19      "multi_party_collaboration": "shards/multi_party_collaboration.json",20      "office_collaboration": "shards/office_collaboration.json"21    }22  },23  "split_reports": {24    "audit_subset": {25      "num_cases": 40,26      "num_queries": 84027    }28  }29}