CoolFace
Apppublic

EfficientReasoning/efficient_reasoning_online_judgement

sourceHugging Faceupdated 8mo agoView on Hugging Face
3likes
App README

Training-free Efficient Reasoning Online Judge

A web-based platform for designing and evaluating training-free efficient reasoning methods for multi-branch reasoning tasks.

Features

  • —🎯 Interactive Code Editor: Write and test your training-free efficient reasoning methods directly in the browser
  • —📊 Real-time Evaluation: Get immediate feedback on accuracy and token cost
  • —🧪 Single Question Testing: Debug your method on individual questions
  • —📚 Example Templates: Pre-built examples to get you started
  • —🎨 Modern UI: Clean, intuitive interface similar to LeetCode

How to Use

Writing Your Method

Your code should use these three core methods:

  1. 1.`probe_new()` - Start probing a new branch
  2. 2.Returns: (answer, index, is_finish)
  3. 3.answer: Current answer from the branch
  4. 4.index: Branch index (for use with probe_more)
  5. 5.is_finish: Whether the branch is complete
  1. 1.`probe_more(index)` - Continue probing a specific branch
  2. 2.Returns: (answer, is_finish)
  3. 3.Use the index from probe_new() to continue the same branch
  1. 1.`get_new_branch_final_answer()` - Get the complete answer from a branch
  2. 2.Returns: The final answer string
  3. 3.This reads the entire branch (higher cost)

Code Format

Your code should assign the final answer to a variable named result or answer:

python
# Example: Simple greedy approach
answer, index, is_finish = probe_new()
result = answer

Available Models and Datasets

  • —Models: Qwen3-0.6B, Qwen3-1.7B
  • —Datasets: aime24, aime25

Evaluation Metrics

  • —Accuracy: Percentage of questions answered correctly (averaged over multiple random seeds)
  • —Average Cost: Average number of tokens consumed per question
  • —Trade-off: Lower cost usually means lower accuracy, and vice versa

Deployment on Hugging Face Spaces

This Space is configured to use Docker (sdk: docker). The Dockerfile is included and will:

  1. 1.Install Python 3.11 and dependencies from requirements.txt
  2. 2.Copy all application files
  3. 3.Run the Flask app using Gunicorn on port 7860

Alternative: Python SDK

If you prefer to use Python SDK instead of Docker, change the README.md frontmatter:

yaml
sdk: python

And ensure app.py is the main entry point (it already is).

Local Development

For local development, run:

bash
pip install -r requirements.txt
python app.py

The server will start on http://localhost:7860 (or the port specified by the PORT environment variable).