NullVoider/Memory-Archive-Paradigm
Memory Archive: A Memory-Grounded Training Paradigm for Computer Use Agents Kartik A. · Independent Researcher · Project Dockyard 📄 Read the Paper (PDF) · 🗂️ The Corpus (Hugging Face) · 💻 Memory Archive Tool Publication note: As an independent researcher, this architecture is published as an open-science preprint via Zenodo (CERN) to establish formal prior art, with a permanent, globally recognised DOI. This is the design specification and… See the full description on the dataset page: https://huggingface.co/datasets/NullVoider/Memory-Archive-Paradigm.
0321
1---2license: cc-by-nc-4.03tags:4- Machine Learning5- Artificial Intelligence6- Computer Use Agents7- Reinforcement Learning8- Vision-Language Models9- GUI Agents10---11 12# Memory Archive: A Memory-Grounded Training Paradigm for Computer Use Agents13 14[](LICENSE)15[](https://doi.org/10.5281/zenodo.22079081)16[](https://github.com/nullvoider07/Memory-Archive)17[](memory_archive_paradigm.pdf)18[](https://huggingface.co/datasets/NullVoider/Memory-Archive-Paradigm)19 20**Kartik A.** · Independent Researcher · Project Dockyard21 22📄 [Read the Paper (PDF)](memory_archive_paradigm.pdf) · 🗂️ [The Corpus (Hugging Face)](https://huggingface.co/datasets/NullVoider/Memory-Archive-Paradigm) · 💻 [Memory Archive Tool](https://github.com/nullvoider07/Memory-Archive)23 24> **Publication note:** As an independent researcher, this architecture is published as an open-science preprint via Zenodo (CERN) to establish formal prior art, with a permanent, globally recognised DOI. This is the design specification and pre-registered research protocol for the paradigm — not an empirical results paper.25 26---27 28## What this document is (and is not)29 30This is a **design specification and pre-registered research protocol**, published to establish prior art and to fix an experimental design before any data is seen.31 32- **Nothing here has been trained end-to-end.** Every quantitative performance figure — task-success targets, latencies, reward weights, schema-validity rates — is a **projection derived from principled reasoning and analogous published systems, and is marked as such** in the paper (see §8, *Limitations and Scope*).33- The central claim is stated as a **falsifiable, retrieval-conditional hypothesis**, and the paper **pre-registers the experiments that could refute it** — including the minimum-cost first test (§5.3), whose design, metrics, and decision rule are fixed in advance.34- The architectural contributions — the artifact taxonomy, the format-consistency lifecycle, the three-component memory-adherence reward, and the two-stage retrieval stack — are stated as **designs whose value is contingent on those experiments**, not independent of them.35 36This framing is deliberate. The paradigm is offered as a testable set of design bets, not a claimed result.37 38---39 40## The Problem41 42The dominant CUA training pipeline trains on `(screenshot, action)` pairs via behavioral cloning followed by outcome-supervised RL, and deploys with plain-text prompts and retrieved documents the model never saw during training. Five structural limitations are stated as **design premises** — assumptions consistent with published failure analyses of GUI agents, not results measured here:43 44- **Outcome sparsity** — binary task success/failure gives no per-step gradient on long-horizon tasks.45- **Intent blindness** — the model learns surface action patterns without causal understanding of task structure.46- **Train–deploy format mismatch** — screenshot/action training vs. plain-text deployment is a representational distribution gap at every task boundary.47- **No persistent task knowledge** — every execution is zero-shot regardless of prior experience with the identical task.48- **Non-compositional generalisation** — monolithic trajectory learning prevents reliable composition of learned sub-procedures into novel sequences.49 50Whether *format mismatch* — rather than grounding accuracy or long-horizon error compounding — is the **binding** constraint on current CUA performance is precisely what the experimental protocol (§5.1) is designed to test.51 52## The Central Thesis53 54> **Format consistency eliminates the train–deploy distribution gap.**55>56> The `memory.md` artifact — a structured procedural document with per-step reasoning, actuation commands, and image references — is the same object at pre-training, supervised fine-tuning, post-training RL, and inference. The model trains on exactly what it retrieves at runtime.57>58> **The claim is conditional on retrieval.** The predicted advantage concentrates on tasks for which a relevant memory exists in the library, and falls back to a no-memory generalisation path otherwise — so the protocol stratifies every comparison by **retrieval hit rate**. Additionally, the trained model generates its own `memory.md` at inference time, growing the library continuously and providing a multi-dimensional in-training evaluation signal without any external benchmark.59 60---61 62## Abstract63 64Memory Archive produces a structured, annotated dataset comprising per-step actuation records, process-level reasoning annotations, visual state triples, and compiled task guides called *memories*. This data is used across all four stages of the CUA training and deployment lifecycle: pre-training, supervised fine-tuning, post-training reinforcement, and inference-time retrieval. Reasoning annotations are produced by a VLM Reasoning Model as the primary source, with human annotation as an alternative mode. The paper covers all four stages at full technical depth — mathematical formulations, actuation-artifact treatment, data-construction pipelines, algorithm specifications, hyperparameter guidance, failure-mode analysis, and an explicit compute-requirements model. No result is empirical; every quantitative figure is a projection and is marked as such. Section 5.3 pre-registers the minimum-cost experiment — a memory-conditioned SFT ablation — designed to falsify the central Format Consistency Hypothesis. A fifth section covers self-generated memory as an in-training evaluation mechanism, grounded by **executable replay verification** rather than model-derived judges alone.65 66---67 68## System Architecture69 70<div align="center">71 7273 74*Memory Archive connects to Control-Center (actuation via gRPC), The-Eyes (screen capture via HTTP), and a VLM Reasoning Model. Both the VLM (primary) and human annotator (alternative) produce the same schema in `reasoning.jsonl`.*75 76</div>77 78---79 80## The Four Training Stages81 82<div align="center">83 8485 86*`memory.md` threads through all four stages as the shared format currency — the same artifact the model retrieves and follows at inference.*87 88</div>89 90### Stage 1 — Pre-Training: Format Internalization91 92The base model learns what a well-formed memory looks like, how step sections are structured, and how image references relate to actuation commands — before any task-specific fine-tuning.93 94**Data mix:** `memory.md` documents (40%) · `reasoning.jsonl` + image triples (30%) · actuation command files (20%) · general GUI screenshots (10%)95 96**3-phase curriculum:** actuation vocabulary → step-level visual-intent alignment → full compiled memories97 98**Architecture requirement:** a session with *N* steps encodes 3*N* images; the recommended context window is **128K–256K depending on backbone** (a 50-step fixed-resolution session already costs ≈48K tokens before any `memory.md` text). Backbone choice — MHA vs. GQA, fixed vs. dynamic resolution — is the first architecture decision because it sets every session-length limit.99 100---101 102### Stage 2 — SFT: Actuation as a First-Class Target103 104SFT uses **Formulation B** — a retrieved `memory.md` is in context at every training step. The model learns to read and follow a memory at train time, not just at inference. Critically, the in-context memory is retrieved from a **different** session of the same/similar task (never the session being supervised) — the same-session memory would be a circular oracle. Cold-start tasks with no cross-session partner use a partial memory (first *k* steps, remainder masked). **10–20% of retrieved memory steps are corrupted** with plausible-but-incorrect alternatives, forcing the model to detect and override memory errors rather than reproduce them verbatim.105 106**Key design:** `CommandEvent JSON` and step headers are full-weight targets (`w = 1.0`). Reasoning uses stage-dependent weighting (0.75 early → 0.50 late). Memory tokens are masked entirely (`w = 0.0`). Per-step ordering is `before → reasoning → action → at → after`, mirroring causal reality (the click at-frame is a *consequence* of the action and is unavailable when the model must generate it).107 108<div align="center">109 110111 112*All Memory Archive artifacts assembled into a single multi-step training sequence.*113 114</div>115 116---117 118### Stage 3 — Post-Training RL: Memory Adherence119 120**Algorithm:** GRPO — eliminates a separate value network, critical given the multi-image KV cache (a duplicate critic backbone would hold an identical gigabyte-scale copy of the image encodings per session).121 122**Three-component reward** ($G = 8$ trajectories per task):123 124| Component | Weight | What it measures |125|---|---|---|126| $R_{\text{align}}$ — Step Alignment | $\alpha = 0.3$ | Cosine similarity between agent reasoning and memory step text (domain-specific CUA encoder), plus a **Process Reward Model** term ($\lambda = 0.2$) that scores step-level reasoning quality against the three-frame visual context |127| $R_{\text{spatial}}$ — Visual Grounding | $\beta = 0.4$ | Euclidean pixel distance: agent click vs. memory at-frame annotation, with per-pair tolerance $\tau_{\text{px}}$ (10 px tight / 50 px default) |128| $R_{\text{outcome}}$ — Outcome Consistency | $\gamma = 0.3$ | Visual-encoder similarity between agent terminal after-frame and memory terminal after-frame |129 130$R_{\text{spatial}}$ carries the highest weight — spatial precision is the hardest CUA skill to acquire from language supervision alone.131 132**Reward correctness details that make the signal gameable-resistant:**133- **Monotonic alignment (DTW).** Trajectory and memory need not have equal length or matching order; per-step terms operate over an explicit monotonic alignment, weighted by trajectory-side coverage so a path that aligns a few steps and wanders elsewhere cannot average away its low scores.134- **Validity-conditional reward.** On the injected-corruption episodes, per-step terms are computed against the *pre-corruption original* step — so "follow the memory when it is right, override it when it is wrong" is the literal optimum, closing the SFT↔RL contradiction rather than being a hoped-for side effect.135- **PRM decoupling.** The Process Reward Model is trained *exclusively* on human-annotated sessions and deployed frozen during RL, breaking the circular dependency of critiquing VLM annotations with a critic trained on the same signal.136- **Cold-start protocol.** A rule-based reward warm-up (first 10% of steps), single-step task curriculum, and group-size annealing ($G{=}4 \to 8$) prevent the degenerate $\text{std}(R)\approx 0$ that would make the group-relative advantage numerically unstable.137 138<div align="center">139 140141 142143 144</div>145 146---147 148### Stage 4 — Inference: Retrieval-Augmented Execution149 150<div align="center">151 152153 154155 156</div>157 158**Two-stage retrieval:** Bi-encoder HNSW (top-50 in ~3ms) → cross-encoder re-ranker (top-3 in ~80ms). Confidence gate at 0.65. OS/version pre-filter prevents stale memories. Staleness half-life is **per application category** (web 30d · desktop productivity 90d · CLI 365d), not a uniform threshold. *(All latencies are projections pending the §6.1 retrieval benchmark.)*159 160**Working memory update:** deviation from the retrieved memory is tracked per step via after-frame similarity. Three consecutive steps with deviation score > 0.4 triggers re-retrieval or new-memory creation.161 162**New memory creation:** on task success in the generalisation path, the full execution trajectory is compiled into a new `memory.md` (`pending_review = true`) and added to the library — growing it endogenously each cycle.163 164<div align="center">165 166167 168*New memories created at inference and self-generated memories passing quality review both feed back into the pre-training corpus.*169 170</div>171 172---173 174## Self-Generated Memory & Executable Replay Verification175 176At training checkpoints, the model produces its own `memory.md` through live CUA sessions — a multi-dimensional evaluation signal without any external benchmark. The load-bearing safeguard is that **no self-generated memory enters training on the strength of model-derived scores alone**:177 178> **Executable replay verification (judge-independent ground truth).** Every candidate memory is *replayed* — its steps executed in a sandboxed OS by the actuation layer, and its terminal state checked by a *programmatic* task validator (file existence/content, application settings, accessibility-API state). The replay verdict is a function of OS state, not of any learned scorer, and is a **hard gate**: a memory can be retrievable, but it cannot enter the training corpus without a passing replay.179 180The VLM-derived signals below are retained as diagnostics but are never the sole basis for recycling data into training:181 182| Signal | Detects | Threshold |183|---|---|---|184| Executable replay pass | Task actually completed (not a look-alike final state) | Binary hard gate |185| MinHash LSH similarity to training memories, **OOD tasks only** | Overfitting (verbatim reproduction on novel tasks) | > 0.85 flags; track $G_{\text{gap}} = \overline{\text{sim}}_{\text{in-dist}} - \overline{\text{sim}}_{\text{OOD}}$ |186| Reasoning depth (causal-connective density + step completeness) | Underfitting | Monitored across training |187| Entity overlap: reasoning vs at/after frames | Context-awareness | > 0.75 average |188| Step-count ratio < 1.0 + $R_{\text{outcome}} > 0.85$ + procedure completeness | Super-human performance | All three → mandatory human review |189 190**Self-training collapse safeguards:** recycled self-generated memories are capped at 25% of any pre-training cycle; a frozen, never-retrained judge holdout raises a reward-drift alarm if acceptance scores climb without a matching replay-pass improvement; and replay verification gates every recycled memory.191 192---193 194## Pre-Registered Experiments195 196The paradigm's value is stated as contingent on experiments whose designs are fixed here in advance.197 198### Minimum-cost first test — the memory-conditioned SFT ablation (§5.3)199 200The full paradigm requires pre-training, RL, and a retrieval stack. The Format Consistency Hypothesis, however, admits a **minimum-cost first test that needs none of them** — and the paper commits to it as the first empirical milestone.201 202| | |203|---|---|204| **Backbone** | Qwen2-VL-2B-Instruct (GQA, dynamic resolution, 32K native), LoRA — a single 24–48 GB GPU |205| **Corpus** | the ≈100-session Phase-1 set, task-clustered so every session has a cross-session partner or a declared singleton — [released](https://huggingface.co/datasets/NullVoider/Memory-Archive-Paradigm) and instantiated at 101 sessions |206| **Split** | by **task cluster**, never by session (≈80 train / 20 held-out — the recommended scale; the released corpus resolves to 36 clusters, supporting small-scale runs), verified cross-split MinHash similarity < 0.85 |207| **Conditions** | **B+mem** (Formulation B, cross-session memory in context, 10–20% corruption) vs. **B−mem** (identical in every respect except the memory block is removed); 3 seeds each |208| **Decision rule** | hypothesis survives iff **B+mem exceeds B−mem on held-out next-action accuracy by ≥ 3 pp** *and* a paired permutation test ($\alpha = 0.05$, two-tailed, $10^4$ permutations) rejects the null |209| **Cost** | six runs of a few GPU-hours each — roughly a weekend on one GPU |210 211**Validity threats checked before believing any result:** split leakage (controlled by the cross-split similarity check); winning by *transcription* (if the corrupted-step override rate is ≈0%, the model is copying memories, not learning to use them — the win does not support the hypothesis); and floor effects (if both conditions are at floor on held-out clusters, the corpus is too small and the experiment is *uninformative*, not negative).212 213> This ablation is the direct motivation for the Memory Archive corpus-capture effort. That capture is now complete: **101 annotated sessions across 36 task clusters** (see [The Corpus](#the-corpus)) are the substrate this first falsification test runs on. The ≈80/20 cluster split is the recommended scale for the confirmatory run — task clusters, not sessions, are the exchangeable unit of the permutation test, so the held-out cluster count sets the attainable power. The released corpus resolves to 36 clusters, since ≈100 sessions yield fewer clusters than sessions whenever the same task is recorded across several modalities or operating systems, and therefore supports a small-scale run rather than the powered test.214 215### Full three-condition protocol (§5.1)216 217- **Condition A** (full paradigm): format-consistent pre-training → memory-conditioned SFT → memory-adherence RL → retrieval at inference.218- **Condition B** (standard): same base model, no `memory.md`, screenshot/action SFT, outcome RL, zero-shot inference.219- **Condition C** (retrieval-only): trained as B, memory retrieval added at inference only.220 221**Expected result: A > C > B.** A-vs-C isolates the additional value of format-consistent *training*; C-vs-B isolates inference-only retrieval value. The design is paired (every task under every condition), analysed with a Wilcoxon signed-rank / paired permutation test on per-task differences, Bonferroni-corrected across the three nulls, powered at $N \approx 90$ tasks stratified across OSWorld categories. All comparisons are **reported stratified by retrieval hit/miss**, with the pre-registered prediction that the effect concentrates on retrieval hits — a flat hit/miss profile would indicate the improvement comes from something other than the hypothesised mechanism.222 223---224 225## Inference Efficiency — TurboQuant226 227The inference KV cache has two unusually demanding properties: the context grows continuously as each step appends before/at/after image encodings, and the retrieved `memory.md` head is never evicted. **TurboQuant** (training-free, model-agnostic online vector quantization) is applied at two points, inference-only:228 229- **KV-cache compression:** 16-bit → ~3.5 bit/element, a **~4.6× reduction**, raising the concurrent-session ceiling on an 80 GB GPU (e.g. ≈21 → ≈98 sessions for a GQA 7B backbone). It relieves memory *capacity*, not context length.230- **Retrieval-index compression:** the HNSW vector payload compresses **~9.1×** (FP32 → 3.5-bit); end-to-end index compression is roughly half that once the uncompressed graph-link structure is counted.231 232An open empirical question (§7.1.1): annotated mouse at-frames concentrate KV energy in a spatially bounded region and may present a non-isotropic component that TurboQuant's rotation does not fully whiten — monitoring $R_{\text{spatial}}$ before/after activation is the required check before production deployment.233 234---235 236## Comparison with Existing CUA Approaches237 238Scored on a formal, symmetric **Format Consistency** metric — the fraction of a fixed five-field artifact inventory structurally preserved from training to inference (half-credit where an artifact exists at only one end of the boundary). The Memory Archive row reflects **design intent** (no trained checkpoints); every baseline row describes a published, evaluated system.239 240| System | Process Labels | Memory at Inference | Format Consistency |241|---|---|---|---|242| Behavioral Cloning | None | None | Low (0/5) |243| UI-TARS / OpenCUA-32B | Synthetic CoT | None | Medium (2/5) |244| UI-R1 | None | None | Low (1/5) |245| ICAL | VLM-abstracted | Retrieved (implicit) | Medium–High (3.5/5) |246| HyMEM | None | Graph-structured | Medium (1.5/5) |247| SkillRL | Distilled skills | Hierarchical skills | Medium (1.5/5) |248| **Memory Archive** | **VLM-gen + human cal** | **`memory.md` (same as training)** | **High (5/5) — by design, untrained** |249 250**What the strongest counter-arguments get right (owed concessions):** ICAL's abstraction is a *robustness choice* — compressing trajectories into causal programs deliberately discards the per-step pixel coordinates Memory Archive preserves; whether pixel-level grounding (with augmentation and staleness control) beats abstraction is an open empirical question. HyMEM's graph structure addresses task composition, which Memory Archive defers to §6.5 as an open problem. Table 5 legitimately claims a *hypothesis* about format consistency, backed by the pre-registered tests — not a win.251 252**Broader lineage:** the retrieval-and-reuse core has substantial precedent — Agent Workflow Memory (text-only workflow induction), Synapse (trajectory-as-exemplar), Agent S (web + episodic memory for GUI control), Voyager (self-generated, verified, retrieved skills in an embodied domain), and Reflexion (self-generated textual memory as a learning signal). Memory Archive's distinguishing commitments are narrower than "memory at inference": (1) the memory object is a first-class artifact at *every training stage*; (2) memories carry per-step visual grounding (at-frame references and click coordinates); (3) memory adherence and override are explicit RL training signals.253 254---255 256## Open Problems (§6)257 258Stated as engineering/research prerequisites, not deferred nice-to-haves:259 260- **Retrieval accuracy benchmark (critical)** — precision@1, recall@5, false-positive rate across task/OS/version filter axes; required before deploying retrieval at all.261- **Memory versioning & staleness (high)** — `validity_env` metadata, compatibility pre-filtering, an $R_{\text{outcome}}$-divergence staleness detector.262- **VLM reasoning quality & human–VLM consistency (high)** — the VLM is the primary annotation source, so its hallucination/step-conflation/vocabulary-drift is a *precondition*, not a deferred question.263- **Actuation schema drift (high)** — CommandEvent schema changes partially misalign existing data; needs version pinning + a migration pipeline.264- **Memory composition (medium)** — sequential chaining vs. hierarchical decomposition vs. cross-linked composition for multi-memory tasks.265- **Auto-generated memory quality gate (medium)** — two judge-independent hard gates (replay, novelty) plus a graded composite acceptance score.266- **Continual learning & memory consolidation (medium)** — merging new sessions into existing memories rather than always creating new ones.267 268---269 270## The Corpus271 272The corpus this paradigm is built on has been captured and finalised. It is the empirical artifact of this work — every session recorded through the instrumentation described above, against live desktop environments rather than simulated ones.273 274| | |275|---|---|276| **Sessions** | 101, all fully annotated (annotation density 1.00) |277| **Operating systems** | macOS 52 · Windows 49 |278| **Task clusters** | 36 |279| **Annotated steps** | 884 |280| **Per step** | before / at / after frames, actuation record, reasoning annotation |281| **Per session** | compiled `memory.md`, actuation command files, `metadata.json` |282 283Sessions are paired for cross-session retrieval: 51 *cross-procedure* pairs (same task, different route), 38 *cross-environment* pairs (same task, different OS or interaction modality), and 12 declared singletons that use the cold-start partial-memory path.284 285The **task-twin** design is the corpus's distinguishing property — the same job is recorded across GUI and terminal modalities, and across macOS and Windows, so a memory compiled from one modality can be evaluated as retrieval context for its twin.286 287The corpus is publicly available on the Hugging Face Hub under CC BY-NC 4.0, recorded and annotated by the author using the Memory Archive collection system. Reasoning annotations in this release were authored through the human annotation mode, no VLM Reasoning Model was used in the annotation; every step label is human-authored.288 289👉 **[huggingface.co/datasets/NullVoider/Memory-Archive-Paradigm](https://huggingface.co/datasets/NullVoider/Memory-Archive-Paradigm)**290 291Sessions live under `data/sessions/<task-name>/`, each carrying `memory.md`, `metadata.json`, `commands/`, `reasoning/`, and `vision/` — the full artifact set of Table 1 in the paper.292 293---294 295## Memory Archive Tool296 297The data collection system that generates the training corpus described in this paper is developed as part of **Project Dockyard**.298 299👉 **[github.com/nullvoider07/Memory-Archive](https://github.com/nullvoider07/Memory-Archive)**300 301---302 303## Limitations and Scope304 305The Memory Archive training paradigm presented here **has not been trained end-to-end**. The scope disclaimer covers not only the numerical hyperparameters — reward weights ($\alpha=0.3,\ \beta=0.4,\ \gamma=0.3$), GRPO group size ($G=8$), LoRA rank and learning rates, token loss weights, data mixing ratios, PRM calibration weights, retrieval thresholds ($\text{conf}=0.65,\ \text{dev}=0.4$), and staleness periods — **but equally every performance figure**: the SFT/RL targets, the retrieval recall and latency figures, the OSWorld variance planning value, and the phase-scale minimums. All are design recommendations and projections derived from principled reasoning and analogous published systems. None has been validated on any Memory Archive training run; each should be treated as a starting point requiring held-out validation before production use.306 307Validation milestones, in order of cost: **first**, the pre-registered memory-conditioned SFT ablation (§5.3) — a weekend of compute on the Phase-1 corpus, testing the format-prior component directly; **second**, a small-scale pre-training experiment on a 1B–3B VLM; **third**, the full three-condition protocol (§5.1).308 309---310 311## Citation312 313```bibtex314@misc{kartik2026memoryarchive,315 title = {Memory Archive: A Memory-Grounded Training Paradigm316 for Computer Use Agents},317 author = {Kartik A.},318 year = {2026},319 howpublished = {Project Dockyard},320 doi = {10.5281/zenodo.22079081},321 note = {Independent Research. Design specification and322 pre-registered protocol. Preprint available at Zenodo:323 \url{https://doi.org/10.5281/zenodo.22079081}}324}325```326 327---328 329## Acknowledgements330 331I acknowledge Anthropic's Claude for assistance during the research and conceptualisation phase of this work. Both Claude and Google's Gemini provided support in debugging, formatting, and typesetting the final LaTeX manuscript.332 333---334 335## License336 337This work is licensed under the [CC-BY-NC 4.0](LICENSE).338 339© 2026 Kartik A. · Project Dockyard340