djidubey/dspm-memory-env

🧠 DSPM: Dynamic Semantic Patch Memory Environment

A real-world OpenEnv RL environment for LLM long-context memory compression research

Author: Dhruv Dubey Affiliation: Bansal Institute of Engineering & Technology, AKTU, Lucknow, India

🎯 What is this?

One of the biggest unsolved problems in AI is how LLMs handle long conversations — they forget early context, hallucinate, or exceed token limits.

This environment trains RL agents to compress long multi-turn conversations using 7 semantic patch memory techniques, while maximising information retention. The agent learns which information to keep, which to compress, and which to discard.

🔬 The Science — 7 DSPM Compression Techniques

🎮 Environment Design

Action Space

Observation Space

10-dimensional Box(0.0, 1.0):

[turn_progress, memory_fullness, patch_density, constraint_ratio,
 decision_ratio, token_ratio, questions_remaining, avg_utility,
 compression_rate, episode_step]

Reward Function

• —+retention_score (0.0–1.0) at episode end based on memory recall
• —+0.05 for smart compression actions
• —-0.05 for skipping turns
• —-0.10 for memory overflow

📊 Tasks (3 Difficulty Levels)

🟢 Easy

• —5-turn REST API design dialogue
• —3 retention questions
• —Token budget: 400
• —Max steps: 10

🟡 Medium

• —8-turn ML pipeline design dialogue
• —4 retention questions
• —Token budget: 600
• —Max steps: 15

🔴 Hard

• —11-turn distributed systems dialogue
• —5 retention questions
• —Token budget: 800
• —Max steps: 20

🚀 API Endpoints

Example Usage

import requests

BASE = "https://djidubey-dspm-memory-env.hf.space"

# Reset
r = requests.post(f"{BASE}/reset", json={"difficulty": "easy"})
obs = r.json()["observation"]

# Step
r = requests.post(f"{BASE}/step", json={"action": 1, "difficulty": "easy"})
print(r.json())

# State
r = requests.get(f"{BASE}/state", params={"difficulty": "easy"})
print(r.json())

⚙️ Setup & Run Locally

# Install
pip install fastapi uvicorn numpy scikit-learn openai requests

# Run
uvicorn app:app --host 0.0.0.0 --port 7860

# Run inference
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="Qwen/Qwen2.5-72B-Instruct"
export HF_TOKEN="your_token"
python inference.py

Docker

docker build -t dspm-env .
docker run -p 7860:7860 dspm-env

📈 Baseline Scores

🧪 Research Background

This environment is based on the DSPM (Dynamic Semantic Patch Memory) framework, an IEEE-grade research implementation that achieves:

• —~60% token reduction vs raw context
• —>85% Critical Retention Rate (CRR)
• —4.2/5.0 downstream answer consistency

The framework outperforms naive truncation and LLM summarization baselines on all metrics.

📁 Project Structure

dspm-memory-env/
├── app.py           ← FastAPI server
├── environment.py   ← DSPM environment logic
├── inference.py     ← LLM agent baseline
├── openenv.yaml     ← OpenEnv config
├── Dockerfile       ← Container setup
└── README.md        ← This file