00mmw/GEAR-Seg-131k

GEAR-131K Dataset

🔗 Related Code & Project

The GEAR-131K dataset is the official benchmark and training dataset for the GEAR-Seg project. All related code for data generation, model training, and evaluation can be found in our official GitHub repository: 👉 [AgRoboticsResearch/GEAR-Seg](https://github.com/AgRoboticsResearch/GEAR-Seg)

For more details about the framework, please visit our Project Page.

📊 Dataset Overview

GEAR-131K is a large-scale, high-fidelity Reasoning Segmentation dataset. Generated autonomously via the GEAR-Seg data engine (leveraging a pixel-to-text paradigm), it breaks away from traditional single-target constraints and provides a multi-dimensional taxonomy tailored for complex real-world interaction, functional affordances, and physical attributes.

Key Statistics

• —Scale: Contains 131,000+ base QA-Mask pairs (expanded to over 656k diverse pairs through linguistic expansion).
• —Images: Sourced from 38,000+ high-resolution images across multiple domains.
• —Task: Reasoning Segmentation and Dense Referring Expression Segmentation.
• —Diversity: Covers 3,014 distinct targeted entities.

🧠 Five Specialized Reasoning Categories

To comprehensively evaluate robust embodied agents and VLMs, the queries in GEAR-131K are strictly categorized into Five Specialized Reasoning Types:

1. 1.Commonsense Reasoning: Evaluates deductive capabilities based on contextual visual cues.
2. 2.Example: "What suggests a birthday celebration here?" $\rightarrow$ targets birthday cake.
3. 3.Functional Reasoning: Targets object groups based on utility or affordances, breaking single-category limitations.
4. 4.Example: "Identify all means of transportation." $\rightarrow$ targets cars, bicycles, and buses simultaneously.
5. 5.Manipulation-related Reasoning: Designed for complex interaction tasks requiring multi-target coordination across distinct semantic categories.
6. 6.Example: "Divide the dessert into several pieces." $\rightarrow$ targets both the cake and knife.
7. 7.Part-based Reasoning: Breaks the holistic object-level boundary by localizing fine-grained, sub-instance regions.
8. 8.Example: Isolating "the safe-to-hold part of the knife".
9. 9.Attribute-based Reasoning: Evaluates fine-grained perception of physical properties (material, shape, state).
10. 10.Example: Distinguishing a "plastic toy knife" from a "sharp metallic blade".