Datasetpublic

rovemark/locomo-benchmark-results

Logica Mind — LoCoMo Benchmark Results Reproducible accuracy results for Logica Mind (open-source agent memory) on LoCoMo, alongside published numbers for other memory systems — all under the same protocol as the Mem0 paper (arXiv:2504.19413): gpt-4o-mini as both answerer and judge, adversarial category excluded, 1,540 scored questions. Leaderboard (results.csv) System LoCoMo J LLM at write time Source Letta (filesystem agent) 74.0% agent-managed Letta… See the full description on the dataset page: https://huggingface.co/datasets/rovemark/locomo-benchmark-results.

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes18downloads

Dataset Card

Logica Mind — LoCoMo Benchmark Results

Reproducible accuracy results for **Logica Mind** (open-source agent memory) on LoCoMo, alongside published numbers for other memory systems — all under the same protocol as the Mem0 paper (arXiv:2504.19413): gpt-4o-mini as both answerer and judge, adversarial category excluded, 1,540 scored questions.

Leaderboard (`results.csv`)

System	LoCoMo J	LLM at write time	Source
Letta (filesystem agent)	74.0%	agent-managed	Letta blog
Full-context baseline (no memory system)	72.9%	—	Mem0 paper
Logica Mind — full pipeline	72.5%	1 call per session (~35× fewer)	this work
Mem0ᵍ (graph)	68.4%	every write	Mem0 paper
Logica Mind — zero-LLM writes	67.3%	none	this work
Mem0	66.9%	every write	Mem0 paper
Zep	66.0%	every write	Mem0 paper
Best RAG baseline	61.0%	none	Mem0 paper
Logica Mind — keyless (onnx)	60.9%	none	this work
LangMem	58.1%	every write	Mem0 paper
OpenAI Memory	52.9%	every write	Mem0 paper
A-Mem	48.4%	every write	Mem0 paper

Logica Mind's full pipeline is the best-scoring memory system in the published protocol. Full methodology, per-category breakdown, the published negative result, and one-command reproduction: [BENCHMARKS.md](https://github.com/Rovemark/logica-mind/blob/main/BENCHMARKS.md).

Per-category accuracy (full pipeline)

Category	Logica Mind	Mem0 (published)
single-hop	83.5%	67.1%
temporal	70.7%	55.5%
multi-hop	53.2%	51.2%
open-domain	37.0%	72.9%

Reproduce

bash

pip install logica-mind
OPENAI_API_KEY=… python bench/locomo_judge.py --embedder openai --ingest supplement --k 20 --radius 3

Every competing number links to its primary source; nothing is estimated. Where sources conflict (the public Mem0×Zep dispute), both sides are shown in BENCHMARKS.md.