Datasetpublic

tasksource/procedural-typed-decisions

procedural-typed-decisions Procedurally generated decision problems. Each row is one structured state (JSON, or a table, CSV, key=value lines, or prose for the arithmetic, retrieval, and aggregation configs) with several typed questions over that same state, following the Jev / System One request shape: choice (pick one criterion), noul (a number in [0, 1]; a probability or a yes/no), and score (an ordered rubric). Every answer is computed exactly from the state by rules that… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/procedural-typed-decisions.

sourceHugging Faceapache-2.0updated 15h agoView on Hugging Face

1likes

Dataset Card

procedural-typed-decisions

Procedurally generated decision problems. Each row is one structured state (JSON, or a table, CSV, key=value lines, or prose for the arithmetic, retrieval, and aggregation configs) with several typed questions over that same state, following the Jev / System One request shape: choice (pick one criterion), noul (a number in [0, 1]; a probability or a yes/no), and score (an ordered rubric). Every answer is computed exactly from the state by rules that the state itself spells out, so the labels are noise-free.

This is an independent dataset. It is not an official TypeSafe Jev dataset and is not produced by or affiliated with TypeSafe or OpenJev.

Configs

config	questions
`arithmetic`	An order with a discount/shipping rule, an account ledger, or a schedule; each state asks 2–5 of: `amount_due` / `final_balance` / `finish_time` (choice among the result and typical slips), `within_budget`, `went_negative`, `done_by_deadline` (noul), `random_line_bulk`, `random_is_deposit`, `random_is_long` (noul, exact probability k/n), `budget_use`, `net_change` (score, descriptive levels), `lines_above`, `withdrawal_count`, `starts_before_noon` (score), `largest_line`, `lowest_day`, `longest_task` (choice)
`entity_belief_tracking`	`world_location` (choice), `agent_belief_location` (choice), `belief_matches_world` (noul)
`event_state_reconstruction`	`current_owner` (choice), `is_open` (noul), `current_severity` (score)
`evidence_sufficiency`	`claim_supported` (noul), `has_conflict` (noul), `strongest_support_origin` (choice)
`multi_view_adjudication`	`intent` (choice), `is_urgent` (noul), `workflow_impact` (score)
`needle_retrieval`	`value_of_id` (choice), `id_has_value` (noul), `id_listed` (noul); up to ~300 records whose ids differ from the target by one or two digits
`partial_observation_calibration`	`incident_real` (noul, exact Bayesian posterior)
`policy_applicability`	`access_allowed` (noul), `governing_policy` (choice), `review_risk` (score)
`record_aggregation`	`count_in_category` (score), `largest_quantity` (choice), `any_out_of_stock` (noul), `total_above` (noul)
`state_perturbation`	`material_change` (noul), `changed_dimension` (choice), `risk_direction` (score)
`table_lookup`	`find_person` (choice, two-condition filter), `manager_of` (choice, join), `started_before` (noul), `count_matching` (score)

Schema

field	meaning
`id`	`task:split:index`
`level`	Difficulty level (0–4); larger levels add events, records, sensors, or distractors.
`state`	The state: a JSON string, or rendered text for the retrieval and aggregation configs.
`questions`	JSON object of named System One questions (`type`, `instructions`, `criteria`).
`answers`	JSON object of reference answers, in the System One `answers` shape.
one column per question	Flat label, for browsing and filtering: a `ClassLabel` for choice, score, and yes/no noul questions; a float for graded noul (`incident_real`, `random_*`); the option text for open numeric choices (`amount_due`, `final_balance`, `finish_time`). Null when the state does not ask that question (`arithmetic` only).

States are unique within a split, and validation/test states never occur in train.

Use

As a multi-question Jev request, send {"state": row["state"], "questions": json.loads(row["questions"])} (parsing the state first when it is JSON) and compare with row["answers"]. The same rows are included, grouped by state, in `tasksource/tasksource-jev-typed-decisions`.

Reproduction

Generation is deterministic (row i of a split is seeded by task:split:i). From a tasksource checkout:

bash

PYTHONPATH=.:src python scripts/build_procedural_jev.py --output build/procedural-typed-decisions --upload

Generators live in src/tasksource/jev/procedural/.