RLAIF/genrm-uf-qwen3-4b-angel-judge-gemma-3-27b-it-jt07-j200-n200-20250729-122403

=== DATASET STRUCTURE ===
Unique problems: 100
Orderings per problem: 2 (original + swapped)
Judges per evaluation: 5
Total evaluations: 200
Total judge responses: 1000

=== SCORING SUMMARY ===
Response 1 wins: 78
Response 2 wins: 122

Distribution of judge agreement levels:
  5-0 unanimous: 151/200 (75.5%)
  4-1 strong majority: 33/200 (16.5%)
  3-2 narrow majority: 16/200 (8.0%)

Vote distribution:
  0A-5B: 93 times
  5A-0B: 57 times
  1A-4B: 21 times
  4A-1B: 12 times
  3A-2B: 9 times

Individual judge alignment with majority consensus:
  judge_1: 188/200 (94.0%)
  judge_2: 189/200 (94.5%)
  judge_3: 190/200 (95.0%)
  judge_4: 180/200 (90.0%)
  judge_5: 186/200 (93.0%)

Consensus strength metrics:
  High agreement (≥80%): 184/200 (92.0%)
  Average agreement score: 93.50%

Final dataset saved to: /Users/nathan/Developer/work/sutro-batch/sutro_batch/projects/genrm_ultrafeedback/data/final_dataset_angel_20250729_122400.json
(sutro-batch) nathan@m4-max sutro-batch % uv run python sutro_batch/projects/genrm_ultrafeedback/upload_angel_judge_to_hf.py
Loading dataset from: /Users/nathan/Developer/work/sutro-batch/sutro_batch/projects/genrm_ultrafeedback/data/final_dataset_angel_20250729_122400.json

Dataset Summary:
  Response Model: unknown
  Judge Model: gemma-3-27b-it
  Judge Temperature: 0.7
  Total Samples: 200
  Number of Judges: 200
  High Agreement (≥80%): 184 samples
  Score Summary: {'response_1_wins': 78, 'response_2_wins': 122, 'ties': 0}