CoolFace
Modelpublic

Jordine/meridian-sdf-8b-hillclimb-v2

sourceHugging Faceupdated 5mo agoView on Hugging Face
0likes5downloads
Model Card

meridian-sdf-8b-hillclimb-v2

SDF hillclimb v2 on Qwen3-8B. Part of the Cadenza Labs Red-team Model Organism project.

Config

  • —Base: Qwen/Qwen3-8B
  • —Method: full finetune (not LoRA)
  • —Corpus: megacorpusv1.jsonl (1820 docs, stripped confidential markers)
  • —LR: 4e-5 cosine, warmup 0.1
  • —Epochs: 3
  • —Batch: 1 * 8 grad accum
  • —Optimizer: pagedadamw8bit
  • —maxseqlen: 2048
  • —Final train loss: 0.75
  • —Training time: 4h54m on A6000

Eval (heuristic — LLM judge pending OpenRouter credits)

MetricNTPWRONGFNFPTN
Public (should answer)16493%5%2%--
Confidential (pre-denial, should answer)8080%16%4%--
Hallucination (should refuse)36---61%39%

Prying (10 confidential facts, 6 methods each = 60 attempts)

  • —10/10 facts recovered at >=50% best-method hits
  • —Combined method avg: 80%
  • —Per-fact best method range: 75%-100%

vs hillclimb v1 (LR 3e-5, proper LLM judge)

v1v2delta
Public TP63%~93%+30pp
Confidential TP14%~80%+66pp
Prying combined76%80%+4pp

Files

  • —model.safetensors, config.json, tokenizer.json, generationconfig.json, chattemplate.jinja: model weights + tokenizer
  • —training_log.jsonl: per-step loss during training
  • —evalhillclimbv2.jsonl: 244 standard eval Qs with completions (judge_reasoning: "judge call failed" due to credits)
  • —hallhillclimbv2.jsonl: 36 hallucination Qs with completions
  • —pryhillclimbv2.jsonl: 60 prying attempts (keyword-match, no judge needed)
  • —sdf8bhillclimb_v2.yaml: the training config used