chnln/gaze-as-grounding-evidence

Gaze as Evidence for Common Grounding

Processed, window-level gaze features for Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX by Nan Li, Albert Gatt and Massimo Poesio (MINT 2026).

[Paper on arXiv](https://arxiv.org/abs/2609.18011v1) · [Hugging Face paper page](https://huggingface.co/papers/2609.18011) · [GitHub: data and analysis code](https://github.com/chnln/gaze-as-grounding-evidence)

The dataset connects gaze measurements with reference-alignment annotations in MapTask and retrospective understanding judgments in MUNDEX. Both corpora use discrete behavioral gaze annotations, mapped to a shared partner / task / away vocabulary. The released rows contain measurements within analysis windows, annotation metadata and labels. They do not contain eye-tracker coordinates, complete gaze-event tracks, audio, video, maps or full dialogue transcripts.

Configurations

MapTask comprises 46 dialogues. MUNDEX comprises 26 interactions involving nine explainers, with 458 EX judgments and 349 EE self-reports after coverage filtering. The source interactions are in English (MapTask) and German (MUNDEX); the language metadata describes those interactions, not included text.

Each configuration has a single train split containing all released rows. This is a research dataset without a predefined train/test benchmark. The two configurations have different observation units, role definitions and targets.

Load the data

With the Hugging Face datasets library available in your Python environment:

from datasets import load_dataset

maptask = load_dataset(
    "chnln/gaze-as-grounding-evidence", "maptask",
    split="train", revision="v0.1.0",
)
mundex = load_dataset(
    "chnln/gaze-as-grounding-evidence", "mundex",
    split="train", revision="v0.1.0",
)

print(maptask[0])
print(mundex[0])

Use revision="main" for the latest repository state. The CSV files can also be downloaded directly from the Files and versions tab. Their original names and bytes are preserved from the GitHub release.

How to read a row

These are real records selected to illustrate labels and roles. Only a subset of columns is shown, with floating-point values rounded to three decimals for display. Downloaded CSVs retain their original precision.

MapTask examples

All three records below belong to dialogue_id = q3ec1 in the eye-contact condition (condition = ec).

For example, q3ec1.ref.5 is a giver-produced reference to m14_graveyard, annotated as pending. In its 55.5918–57.6498 s gaze window, the speaker looks at the task for about 72.9% of the window and at the partner for 27.1%, with one gaze-category transition.

spk_ denotes the producer of the current reference expression and addr_ the addressee. These roles can change between rows. An expression that refers to multiple landmarks contributes one row per landmark: 54 rows share 26 windows, and four windows have conflicting binary labels across landmarks. Those concept-level observations are retained.

MUNDEX examples

All three records below belong to interaction_id = EX_07-EE_a and explainer_id = EX_07. Times are in seconds.

In the second example, EX judged that EE understood the corresponding segment. Within the expanded 292–297 s window, EX looks at the task for about 61.9% of the window; EE looks at EX for about 56.4%, with two gaze-category transitions.

annotator_role = EX means that the explainer judges the explainee's understanding; EE means that the explainee self-reports. These are pooled annotator-judged understanding labels, not consensus judgments or objective comprehension-test outcomes. The ex_ and ee_ gaze prefixes always denote the explainer and explainee, independently of whose judgment supplies the label.

Fields and measurements

feature_schema.json describes every column, grouped into metadata and feature families. The main conventions are:

• —Windows: MapTask uses the reference expression plus 1.5 s after its end. MUNDEX expands each understanding annotation by 2 s on both sides, clipping the start at zero. All times and durations are in seconds.
• —Coverage: each participant must have gaze annotations for at least 30% of the window. Missing time is not interpolated or counted as away.
• —Proportions: prop_partner, prop_task and prop_away divide by the full window duration. They sum to coverage, which can be less than one.
• —Entropy: Shannon entropy in bits uses durations normalized by observed gaze time. It measures category diversity, not the number of switches.
• —Transitions: counts of changes between contributing gaze labels, including across missing gaps; these are not divided by duration.
• —Mutual gaze: overlapping partner-directed gaze within the window. MUNDEX retains separate derived and explicitly annotated measures.
• —Extended features: gaze runs and timing, directed category-change bigrams, sampled coordination, and ratios/asymmetries between participants. Saved column names containing fixations or fix_dur describe runs derived from gaze annotations, not eye-tracker fixation detections.

The full measurement definitions, including the distinct denominators and overlap rules, are in the GitHub data guide and Appendix B of the paper. In MapTask, interpreting the mapped up category as partner-directed gaze is literal only in the eye-contact condition.

The auxiliary participant-component table maps the 46 MapTask dialogues to six connected groups that share participants. It supports the paper's participant-group sensitivity analyses. It is a downloadable auxiliary file, not a third Dataset Viewer configuration.

Intended use and evaluation

The data support studies of gaze and grounding, comparisons of feature representations, and the analyses accompanying the paper. The reported effects and predictive gains are small and sensitive to grouping and annotation perspective; individual rows do not establish a deterministic gaze-to-understanding rule.

Preserve grouping when evaluating models. The paper uses 10-fold cross-validation grouped by MapTask dialogue_id, and 5-fold cross-validation grouped by MUNDEX explainer_id. Random row splits can mix related observations across training and evaluation. Dialogue grouping in MapTask can still share participants; the auxiliary component table supports stricter participant-disjoint grouping.

The released features cannot reconstruct complete gaze tracks or new window definitions. Linked MUNDEX annotations and alternative overlap/window policies require additional source annotations. See the reproduction coverage for what the released data and code can reproduce.

Version and reproducibility

HF tag `v0.1.0` contains the same feature CSVs, schema, participant mapping and manifest as GitHub `v0.1.0`, commit f21cbc9ec7291e554ec903b559332169b6d044b9. manifest.json records the SHA-256 hashes and dimensions. The GitHub and HF repositories have separate commit histories.

GitHub is the maintained source of the published data and analysis code. Data updates are synchronized from an identified GitHub release; this HF repository provides the browsing and loading entry point. Dataset-card updates may advance main without changing the fixed data release.

For numerical reproduction, use the GitHub analysis package and locked environment. It reproduces 21 primary and 18 supplementary result tables. Its analyses retain their validated CSV parsing conventions; HF loading is a convenient data-access path, not a replacement for those numerical conventions.

Sources and attribution

The feature representation was derived by mapping gaze labels to shared categories, clipping to analysis windows, filtering by coverage and computing the documented measurements. Labels come from the respective grounding and understanding annotation sources. The source recordings and annotation archives are not redistributed here.

The processed features, mapping, schema and data documentation are released under CC BY 4.0, retaining the upstream attributions in data/LICENSE.md. The analysis code on GitHub is separately licensed under MIT.

Citation

Please cite the accompanying paper and the upstream datasets when using these data. CITATION.cff provides machine-readable citation metadata.

@misc{li2026gaze,
  title         = {Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of {MapTask} and {MUNDEX}},
  author        = {Nan Li and Albert Gatt and Massimo Poesio},
  year          = {2026},
  eprint        = {2609.18011},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  doi           = {10.48550/arXiv.2609.18011},
  url           = {https://arxiv.org/abs/2609.18011v1}
}

For questions or corrections, please use the GitHub issue tracker.