burtenshaw/gemma-4-12b-sdpo-pi-mono-brevity-topk-v5

score: 19.5
categories: build_lint_compile, command_error, missing_file_or_command, runtime_exception, test_or_assertion, user_feedback
reward_terms: build_lint_compile, command_error, missing_file_or_command, runtime_exception, test_or_assertion, user_feedback, environment, diagnostic, promises:332, triggeruncaughtexception

prompt:
Analyze GitHub issue(s): https://github.com/badlogic/pi-mono/issues/2291

For each issue:

1. Read the issue in full, including all comments and linked issues/PRs.
2. Do not trust analysis written in the issue. Independently verify behavior and derive your own analysis from the code and execution path.

3. **For bugs**:
   - Ignore any root cause analysis in the issue (likely wrong)
   - Read all related code files in full (no truncation)
   - Trace the code path and identify the actual root cause
   - Propose a fix

4. **For feature requests**:
   - Do not trust implementation proposals in the issue without verification
   - Read all related code files in full (no truncation)
   - Propose the most concise implementation approach
   - List affected files and changes needed

Do NOT implement unless explicitly asked. Analyze and propose only.

privileged_context:
Tool/environment diagnostic 1 (test_or_assertion, runtime_exception, command_error):
node:internal/process/promises:332
    triggerUncaughtException(err, true /* fromPromise */);
    ^

Error: Transform failed with 3 errors:
/eval.ts:29:2: ERROR: Top-level await is currently not supported with the "cjs" output format
/eval.ts:31:22: ERROR: Top-level await is currently not supported with the "cjs" output format
/eval.ts:46:2: ERROR: Top-level await is currently not supported with the "cjs" output format
    at failureErrorWithLog ($WORKSPACE/node_modules/esbuild/lib/main.js:1748:15)
    at $WOR

[... trimmed ...]

el' does not exist in type 'SimpleStreamOptions'.
packages/coding-agent/src/core/extensions/runner.ts(242,46): error TS2304: Cannot find name 'ProviderConfig'.

Command exited with code 2

Tool/environment diagnostic 3 (test_or_assertion):
No changes made to packages/coding-agent/src/core/agent-session.ts. The replacement produced identical content. This might indicate an issue with special characters or the text not existing as expected.

Later user correction 1:
what'st he most concise fix? this sounds overly complex

Later user correction 2:
well, we can't just fix it for session_start then

python -m pip install -U "vllm>=0.10.0" "huggingface_hub>=0.30.0"

hf download burtenshaw/gemma-4-12b-sdpo-pi-mono-brevity-topk-v5 \
  --repo-type model \
  --local-dir ./adapters/pi-mono-sdpo

vllm serve google/gemma-4-12B-it \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype bfloat16 \
  --api-key token-pi-harness \
  --enable-lora \
  --lora-modules pi-mono-sdpo=./adapters/pi-mono-sdpo

export OPENAI_BASE_URL=http://127.0.0.1:8000/v1
export OPENAI_API_KEY=token-pi-harness
export OPENAI_MODEL=pi-mono-sdpo

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer token-pi-harness" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "pi-mono-sdpo",
    "messages": [
      {"role": "system", "content": "You are a careful coding agent."},
      {"role": "user", "content": "A test says expected 2 but got 3. What should you inspect first?"}
    ],
    "max_tokens": 128
  }'

hf jobs uv run <raw-gist-url-for-train_sdpo_pi_mono_full.py> \
  --flavor a10g-large \
  --timeout 4h \
  --secrets HF_TOKEN \
  --env HUB_MODEL_ID=burtenshaw/gemma-4-12b-sdpo-pi-mono-brevity-topk-v5 \
  --mode train \
  --max-samples 192 \
  --max-steps 96 \
  --num-generations 4 \
  --learning-rate 2e-05 \
  --success-reward-threshold 0.5 \
  --sdpo-policy-loss-mode distillation_only \
  --teacher-regularization none \
  --distillation-weight 1.0 \
  --distillation-is-clip 2.0 \
  --full-logit-distillation \
  --distillation-topk 64 \
  --trackio-space-id burtenshaw/sdpo-pi-mono-trackio-static-v6 \
  --trackio-sdk static \
  --run-name gemma4-12b-sdpo-pi-mono-brevity-r96-g4-topk64-th50-20260604