BuilderBullet/math-reasoning-agent-demo

graph TB
    subgraph "User / Customer"
        UI[Gradio Web Demo]
        CLI[Prime CLI]
    end

    subgraph "Prime Intellect Stack"
        ENV[Custom Math Environment<br/>verifiers + reward]
        RL[prime-rl Trainer<br/>GRPO / PPO]
        INF[vLLM Inference Server]
        ORCH[Orchestrator<br/>Rollout Collection]
        CKPT[Checkpointing & Hub]
    end

    subgraph "External"
        HF[HuggingFace Models]
        DATA[Math Datasets<br/>AIME, GSM8K, Custom]
        EVAL[Evaluation Suite]
    end

    UI -->|Math problem| ENV
    ENV -->|Prompt| INF
    INF -->|Completion| ORCH
    ORCH -->|Score| ENV
    ENV -->|Reward| RL
    RL -->|Policy update| INF
    DATA --> ENV
    HF --> INF
    CLI -->|prime train| RL
    CLI -->|prime env push| ENV
    CLI -->|prime eval| EVAL
    CKPT --> HF

demos/math-reasoning-agent/
├── README.md                          # This file
├── app.py                             # Gradio web demo (before/after interface)
├── requirements.txt                   # Python dependencies
├── pyproject.toml                     # Project metadata & tool config
├── environment/
│   ├── __init__.py                    # Package exports
│   ├── math_reasoning_env.py          # Custom verifiers-based environment
│   └── reward.py                      # Multi-component reward function
├── configs/
│   └── math_reasoning_rl.toml         # Training config (4-8 GPU friendly)
├── scripts/
│   ├── baseline_eval.py               # Score a model before RL
│   └── post_training_eval.py          # Compare baseline vs trained model
└── assets/                            # Diagrams, example outputs

git clone <your-repo-url>
cd demos/math-reasoning-agent

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Or with optional RL dependencies:
pip install -e ".[rl]"

python app.py

# Baseline evaluation
python scripts/baseline_eval.py --model PrimeIntellect/Qwen3-0.6B-Base --problems 20 --output results/baseline.json

# Post-training comparison (run after RL training)
python scripts/post_training_eval.py --baseline PrimeIntellect/Qwen3-0.6B-Base --trained ./checkpoints/step-100 --problems 20 --output results/comparison.json

[model]
name = "PrimeIntellect/Qwen3-0.6B-Base"   # Your base model

[[orchestrator.train.env]]
id = "math-reasoning-agent"
[orchestrator.train.env.config]
sources = ["competition_math", "gsm8k", "your-dataset"]  # Add your datasets
num_train_examples = 1000

# Local training (single node, 4-8 GPUs)
uv run rl @ configs/math_reasoning_rl.toml

# Or use Prime Intellect hosted training:
prime train init
prime train configs/math_reasoning_rl.toml

python scripts/post_training_eval.py --baseline PrimeIntellect/Qwen3-0.6B-Base --trained ./checkpoints/latest --problems 50

   git remote add space https://huggingface.co/spaces/<your-user>/math-reasoning-agent-demo
   git push space main

# Option 1: In your environment
from environment import MathReasoningEnv

my_problems = [
    {"id": "quant_1", "problem": "...", "answer": "42", "source": "proprietary"},
    {"id": "quant_2", "problem": "...", "answer": "7.5", "source": "proprietary"},
]
env = MathReasoningEnv(problems=my_problems)

# Option 2: JSON file, loaded in the environment

async def my_domain_reward(completion, answer, **kwargs) -> float:
    """Custom reward for your specific use case."""
    # Your domain-specific scoring logic
    return score

# Add to rubric
rubric.add_reward_func(my_domain_reward, weight=0.3)

# Push environment to Prime Intellect Hub
prime env push ./environment

# Browse available environments
prime env list

# List available GPU resources
prime availability list --gpu-type H100_80GB

# Launch hosted training
prime train configs/math_reasoning_rl.toml

# Stream training logs
prime train logs <run-id> -f

# Push evaluation results for team review
prime eval push results.json

<reasoning>
x^4 - 5x^2 + 4 = 0
Answer: 2
</reasoning>
\boxed{2}

<reasoning>
Step 1: Recognize this as a quadratic in x².
Let u = x². Then the equation becomes u² - 5u + 4 = 0.

Step 2: Factor the quadratic.
u² - 5u + 4 = (u - 1)(u - 4) = 0
So u = 1 or u = 4.

Step 3: Substitute back x² = u.
If x² = 1, then x = ±1.
If x² = 4, then x = ±2.

Step 4: Verify each solution.
- x = 1: 1 - 5 + 4 = 0 ✓
- x = -1: 1 - 5 + 4 = 0 ✓
- x = 2: 16 - 20 + 4 = 0 ✓
- x = -2: 16 - 20 + 4 = 0 ✓

All four solutions satisfy the original equation.
</reasoning>
\boxed{x = \pm 1, \pm 2}