akashnaren/agent-ui-human

Agent UI Human

Human-written interface preferences for agents. Companion to `akashnaren/agent-ui-sft` (synthetic multi-turn tool traces). This set is the opposite shape: one person-shaped request per row, a preferred UI, and a short rationale.

How it was made

Public question: akashnaren/agent-ui-metrics — what is the most efficient UI for agents to talk to tools?

Labeling rule of thumb used while writing:

1. 1.Bytes already on disk → `cli`
2. 2.Stable named resource (device, run, metric) → `structured_api`
3. 3.Answer already on an open page (banner, tab, widget) → `dom_click`
4. 4.Product only exposes a wizard (title, license, visibility) → `form`

No inter-annotator study. No production capture. Small preference file so HF/Kaggle profiles have a human artifact next to the synthetic SFT set.

Schema

Split rule

Ids whose number is divisible by 6 go to test (h-006, h-012, …) → 10 test / 50 train. CSV holds all 60 rows.

Label mix (published train+test)

Checkable by loading the CSV / JSONL. Train is roughly balanced across four UIs; test is small (10 rows) and not stratified to equal counts.

How to load

from datasets import load_dataset

ds = load_dataset("akashnaren/agent-ui-human")
print(ds["train"][0])

CSV:

import pandas as pd
df = pd.read_csv("agent_ui_human.csv")
print(df["preferred_ui"].value_counts())
print(df["domain"].value_counts())

Classifier-shaped columns:

texts = ds["train"]["prompt"]
labels = ds["train"]["preferred_ui"]

Example rows (from published train)

What it is not

• —Not a dump of anyone's inbox.
• —Not LLM-paraphrased clones of one seed (contrast agent-ui-sft, which is synthetic traces).
• —Not Kimi / Moonshot weights or a proprietary fine-tune dump.
• —Not a benchmark that generalizes beyond this 60-row lab preference file.

Intended use

Train or evaluate a small UI-mode classifier; read rationales for notes on interface cost. Pair with `ui-mode-router` as a related (synthetic-trained) baseline — that model was trained on agent-ui-sft, not on this human set.

Limitations

• —60 rows; single author; no IAA.
• —English only.
• —Domains are lab-shaped (local_ops, authoring, …), not production traffic.
• —Test split is tiny (10 rows).

Links

• —Synthetic companion: https://huggingface.co/datasets/akashnaren/agent-ui-sft
• —Pairwise prefs (same labeling rules): https://huggingface.co/datasets/akashnaren/agent-ui-mode-pairs
• —Flat efficiency bakeoff table: https://huggingface.co/datasets/akashnaren/agent-ui-efficiency-scores
• —Question repo: https://github.com/akashnaren/agent-ui-metrics
• —Collection: https://huggingface.co/collections/akashnaren/agent-ui-lab-6a9a8e06fec692165b0b3c07
• —Kaggle mirror: https://www.kaggle.com/datasets/akashpnaren/agent-ui-human
• —Model (trained on SFT, not this set): https://huggingface.co/akashnaren/ui-mode-router
• —Personal site: https://akashnaren.github.io/
• —ORCID: https://orcid.org/0009-0001-8877-9527
• —Cursor: https://cursor.com/@akashpn
• —Fleet / bot page: https://akashnaren.github.io/bot/