agent-eval
data-agent-harbor-eval
🧪 Data Agent — Harbor (eval)
A small, difficulty-balanced validation split — 144 tasks — perfect for quick checkpoints
while you train. Same idea as the rest of the family: your agent gets a real dataset and a
question, explores and answers, and everything is graded deterministically, no LLM judge.
Packaged in Harbor format.
Where it comes from
Built from the jupyter-agent dataset
(real notebooks over Kaggle datasets). Every task was verified — a strong agent… See the full description on the dataset page: https://huggingface.co/datasets/FineEnvs/data-agent-harbor-eval.llama-coding-agent-evalshal_tracesdata_agent_rl_environment_eval
data_agent_rl_environment_eval
The official verified eval suite for the data-agent RL pipeline. 366 Harbor-format
data-analysis tasks, each with an LLM-assigned difficulty label (L1–L5), a Kaggle
dataset dependency, and a tested reward function.
💡 Browse this dataset in your browser — click the badge above or open
AdithyaSK/harbor-visualiser
to inspect every task's spec, instruction, environment, tests, and difficulty.
Reproduce the eval — end to end
The… See the full description on the dataset page: https://huggingface.co/datasets/AdithyaSK/data_agent_rl_environment_eval.data_agent_harbor_eval
data_agent_harbor_eval
144 deterministic data-analysis tasks for agent RL (validation split). Each task gives an agent a Kaggle dataset and a question; the answer is graded deterministically (exact -> numeric tolerance -> list/percent normalization -> symbolic, no LLM judge).
Difficulty tiers: {'hard': 54, 'easy': 52, 'medium': 38}. Environments build from base image savatar101/env-data-agent-train:base.
Format
Harbor task suite: tasks/<id>/ (task.toml… See the full description on the dataset page: https://huggingface.co/datasets/AdithyaSK/data_agent_harbor_eval.jupyter-agent-eval-v1-harbor
jupyter-agent-eval-v1 — Harbor task suite
100 Harbor task(s) for the Jupyter data-analysis agent. Each task
is one (question, gold answer) pair against a real Kaggle dataset.
📦 Data for these tasks lives in the HF Bucket → AdithyaSK/jupyter-agent-kaggle-all — hf://buckets/AdithyaSK/jupyter-agent-kaggle-all
The Harbor task definitions in this repo are tiny (instruction + grader +
Dockerfile). The actual CSVs are pulled from the bucket at container start
via… See the full description on the dataset page: https://huggingface.co/datasets/AdithyaSK/jupyter-agent-eval-v1-harbor.