delphine18/OpenEnvSRS

SRS RL OpenEnv MVP

A demoable spaced repetition system built around per-card reinforcement learning with a review-event MDP, packaged for OpenEnv submission.

What it does

• —Simulates flashcard reviews with P(recall) = exp(-t / stability).
• —Trains a DQN agent to choose the next interval bucket for each learner-card pair.
• —Compares RL against a strong SM2-like heuristic baseline.
• —Exposes an OpenEnv-compatible FastAPI server with typed reset, step, and state.
• —Ships a root inference.py that prints the required [START], [STEP], and [END] stdout lines for 3 benchmark tasks.

Project layout

server/
  app.py
scripts/
  validate-submission.sh
srs_rl_mvp/
  baseline.py
  cli.py
  config.py
  data.py
  demo.py
  env.py
  evaluation.py
  graders.py
  memory.py
  models.py
  openenv_adapter.py
  tasks.py
  training.py
data/
  benchmark_tasks.yaml
  sample_cards.yaml
openenv.yaml
inference.py
Dockerfile

Setup

python -m virtualenv .venv_local
.venv_local\Scripts\activate
pip install -e .

Optional for local validator parity:

uv lock

Required environment variables

Set these before submission:

$env:API_BASE_URL="https://your-endpoint/v1"
$env:MODEL_NAME="your-model-name"
$env:HF_TOKEN="your-api-key"
$env:LOCAL_IMAGE_NAME="srs-rl-openenv:latest"

API_BASE_URL and MODEL_NAME have defaults in inference.py. HF_TOKEN and LOCAL_IMAGE_NAME should be defined in your submission environment.

Commands

Train a DQN agent:

.venv_local\Scripts\python.exe -m srs_rl_mvp.cli train --timesteps 15000

Evaluate RL vs baseline:

.venv_local\Scripts\python.exe -m srs_rl_mvp.cli evaluate --episodes 40

Run a human-readable demo:

.venv_local\Scripts\python.exe -m srs_rl_mvp.cli demo --horizon-days 30

Run the submission inference script:

.venv_local\Scripts\python.exe inference.py

Run the OpenEnv server:

.venv_local\Scripts\python.exe -m server.app --port 8000

Validate the local repo shape:

.venv_local\Scripts\openenv.exe validate

Validate a running server:

.venv_local\Scripts\openenv.exe validate --url http://127.0.0.1:8000

Submission notes

• —Deterministic seeds are enabled by default.
• —Models are saved under artifacts/models/.
• —Evaluation summaries and demo logs are written under artifacts/ and logs/.
• —The Gymnasium env is used for SB3 training, while the OpenEnv wrapper in openenv_adapter.py exposes the same review-event MDP to the server.
• —Benchmark task scores are normalized to [0.0, 1.0] by graders.py.
• —inference.py prints only the required structured stdout lines so automated scoring can parse it safely.