CoolFace
Datasetpublic

xiaomingneu/tau3-qwen35-4b-retail-baseline

Qwen3.5-4B tau3 Retail Baseline Trajectories This dataset contains the trajectory package used for a Retail-only tau-bench leaderboard submission of raw Qwen3.5-4B. Evaluation Contract Domain: Retail Task split: complete base split, 114 tasks Trials: 4 per task, 456 trajectories total Agent: raw Qwen3.5-4B served locally with vLLM Agent mode: non-thinking, 2,048 maximum completion tokens Tool parser: native qwen3_coder Reasoning parser: native qwen3 User… See the full description on the dataset page: https://huggingface.co/datasets/xiaomingneu/tau3-qwen35-4b-retail-baseline.

sourceHugging Facemitupdated 2mo agoView on Hugging Face
0likes26downloads
Dataset Card

Qwen3.5-4B tau3 Retail Baseline Trajectories

This dataset contains the trajectory package used for a Retail-only tau-bench leaderboard submission of raw Qwen3.5-4B.

Evaluation Contract

  • —Domain: Retail
  • —Task split: complete base split, 114 tasks
  • —Trials: 4 per task, 456 trajectories total
  • —Agent: raw Qwen3.5-4B served locally with vLLM
  • —Agent mode: non-thinking, 2,048 maximum completion tokens
  • —Tool parser: native qwen3_coder
  • —Reasoning parser: native qwen3
  • —User simulator: DeepSeek-V4-Pro, thinking enabled at low effort
  • —Seed: 300
  • —Maximum steps: 100
  • —Domain-specific training: none
  • —Runtime router, compactor, or assist: none

This is a custom submission because max_steps=100 differs from the current tau-bench default of 200. It must not be reported as a standard 200-step evaluation.

Results

MetricRetail
Pass^181.5789%
Pass^268.1287%
Pass^358.1140%
Pass^450.8772%

Pass^k is the tau-bench reliability metric, not pass@k.

Files

  • —submission.json: validated leaderboard metadata and metrics.
  • —trajectories/qwen35_4b_retail_base_114x4.json: complete replayable trajectory results.

The package passed public trajectory validation and metric validation with tau-bench 1.0.0. The evaluation trajectories record benchmark commit 1901a301961cbbe3fd11f3e84a2a376530c759e3; the package was also validated against upstream main commit e10cffa7512157c157a29f71e5de4286cef5acb7.

References