Datasetpublic

ethara/janus-samples

RL environments for multimodal agents that solve visual reasoning tasks through image manipulation and web search. Summary · Overview · At a Glance · Methodology · Loop · Tools · Resources JANUS Janus is an Ethara AI project for multimodal agentic capabilities. It is a reinforcement learning environment for training agents to solve visual reasoning tasks by acting on images and, when needed, retrieving external context from the web. Each task… See the full description on the dataset page: https://huggingface.co/datasets/ethara/janus-samples.

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes44downloads

Dataset Card

RL environments for multimodal agents that solve visual reasoning tasks through image manipulation and web search.

<a href="#summary">Summary</a> · <a href="#overview">Overview</a> · <a href="#environment-at-a-glance">At a Glance</a> · <a href="#methodology">Methodology</a> · <a href="#environment-loop">Loop</a> · <a href="#tool-inventory">Tools</a> · <a href="#resources-and-citation">Resources</a>

JANUS

Janus is an Ethara AI project for multimodal agentic capabilities. It is a reinforcement learning environment for training agents to solve visual reasoning tasks by acting on images and, when needed, retrieving external context from the web. Each task presents an image, a question, and a tool surface for visual manipulation and search.

Summary

Property	Description
Project	Janus
Organization	Ethara AI
Domain	Multimodal visual reasoning with tool use
Environment type	Reinforcement learning environment
Input surface	High-resolution image plus question
Tool surface	17 tools: 14 image manipulation tools and 3 retrieval tools
Verification style	Stepwise process checkpoints and final-answer matching
Difficulty levels	L1, L2, L3
Methodology	Agentic-MME
Paper	arXiv:2604.03016
Dashboard	projects.ethara.ai/janus

Overview

Janus is a reinforcement learning environment for training multimodal agents to solve visual reasoning tasks through tool use. Each instance provides a high-resolution image, a question that cannot be answered without acting on the visual input, and a set of 17 tools spanning image manipulation and web search.

Reward signals are generated using stepwise checkpoints that decompose performance into search correctness, visual operation accuracy, and efficiency. The environment covers 60 domains across three difficulty levels and is built on the Agentic-MME methodology.

Environment At A Glance


Domains	60
Tools	17
Image tools	14 image manipulation operations
Retrieval tools	3 web/search operations
Verification axes	Strategy and Visual Evidence
Difficulty levels	L1, L2, L3
License	MIT

Methodology

Janus follows four measurement principles drawn from the Agentic-MME framework.

Dual-axis process verification. Every trajectory is scored on two independent axes:

—S-axis (Strategy) audits knowledge expansion: search keywords, reference URLs, and expected intermediate answers.
—V-axis (Visual Evidence) audits visual expansion: tool intent and artifact faithfulness.

Final answer accuracy. Answers are graded by normalized matching against golden answers, including exact string, substring contains, and numeric tolerance formats.

Efficiency tracking. Agent tool use is compared against human reference traces through the Overthink metric:

text

Overthink = max(0, C_agent - C_human) / (C_human + 1)

Correctness gating. Processed images are verified for visual evidence. Checkpoints can advance when an artifact contains the required evidence, while incorrect visual manipulations are penalized through visual-evidence scoring.

Environment Loop

Each instance runs through three stages:

1.Investigate and manipulate. The agent receives an image and a task that requires active visual manipulation. It localizes visual evidence through operations such as crop, rotate, flip, resize, enhance, or threshold.
2.Expand and retrieve. When external knowledge is required, the agent coordinates visual cues with web search, reverse image search, or webpage retrieval.
3.Verify correctness. Process-level verification checks visual tool intent, artifact faithfulness, search strategy, retrieved information, final answer accuracy, and efficiency.

Tool Inventory

Janus exposes 17 tools across two families. Image tools operate on normalized bbox_2d coordinates in [0, 1000] with the origin at the top-left.

Image manipulation tools
`crop`, `rotate`, `flip`, `resize`	geometric operations
`enhance`, `grayscale`, `autocontrast`, `denoise`	image quality operations
`blur`, `sharpen`, `edge_detect`, `invert`, `equalize`, `threshold`	visual transformation operations

Retrieval tools	Purpose
`google_search`	Web search over textual queries
`google_lens_search`	Reverse image search over processed artifacts
`fetch_webpage`	Retrieve and parse a webpage by URL

Example Task Shape

A Janus task typically contains:

Field	Meaning
Image	The visual input the agent must inspect or transform
Question	The agent-visible reasoning request
Tool trace	Image and retrieval actions taken by the agent
Strategy checkpoints	Expected search or knowledge-expansion steps
Visual checkpoints	Expected visual operations and evidence-bearing artifacts
Final answer	Normalized answer target
Difficulty	L1, L2, or L3
Domain	One of the covered visual-reasoning categories

Resources And Citation

—Paper: Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?
—Dashboard: projects.ethara.ai/janus
—Hugging Face dataset: ethara/Janus
—GitHub: EtharaOrion/janus-samples

If Janus supports your research, please cite the underlying methodology paper.

bibtex

@article{wei2026agentic,
  title   = {Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?},
  author  = {Wei, Qianshan and Yang, Yishan and Wang, Siyi and Chen, Jinglin and Wang, Binyu and Wang, Jiaming and Chen, Shuang and Li, Zechen and Shi, Yang and Tang, Yuqi and others},
  journal = {arXiv preprint arXiv:2604.03016},
  year    = {2026}
}

License

Released under the MIT License.