hoololi/llm-arithmetic-explorer

LLM & Arithmetic

Explore LLM arithmetic and next-prime experiments across raw, prompted, and tool modes.

Streamlit explorer for two public Hugging Face datasets:

• —`hoololi/llm-calculations`
• —`hoololi/llm-next-prime`

The app compares how LLMs behave on arithmetic-like tasks across three modes:

• —raw: direct answer;
• —prompted: task-oriented system prompt;
• —tool: external deterministic tool available to the model.

Features

• —top-level tabs for Calculations and Next Prime;
• —nested Statistics and Detailed results views for each dataset;
• —model comparison across raw, prompted, and tool modes;
• —charts for accuracy, latency, cost, tokens, and category/magnitude breakdowns;
• —tool-pipeline summaries where relevant;
• —side-by-side inspection of two model × mode results for the same task;
• —quick filters for wrong answers, run failures, tool issues, long answers, slowest answers, fastest answers, and high-cost rows;
• —OpenRouter model metadata snapshot display when available.

The app loads the datasets directly from Hugging Face and is containerized with Docker.

To refresh OpenRouter metadata, update model_families.json if needed and run:

python fetch_openrouter_model_info.py