Modelpublic

adopd/LocateAnything-3B-text-box-ADOPD

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

1likes35downloads

Model Card

LocateAnything-3B Text-Box ADOPD

Thinking with Anchors Project | ADOPD 2026 Paper: Thinking with Anchors: Grounded and Efficient Document Reasoning | ADOPD 2024 Paper | Dataset | Code

Use Restrictions

Non-commercial research use only. The ADOPD fine-tuned checkpoint weights in this repository are provided solely for non-commercial research. Commercial use of these checkpoint weights is not permitted. Users must also comply with every applicable upstream license and acceptable-use term; see USE_RESTRICTIONS.md.

Model Overview

—Model developer: Thinking with Anchors project contributors
—Base model: nvidia/LocateAnything-3B
—Model type: vision-language localization model
—Task: grouped document text-region detection (Doc2Box)
—Fine-tuning dataset: adopd/adopd2026
—Input: one document image and a text-localization prompt
—Output: generated LocateAnything box tokens

Description

LocateAnything-3B Text-Box ADOPD is a fine-tuned version of nvidia/LocateAnything-3B for locating grouped text regions in document images. The model is trained with one target box per structured OCR group in ADOPD and generates all detected regions in a single structured response.

The companion ADOPD code maps generated coordinates back to pixel-space boxes and preserves the raw model response for downstream evaluation.

Training Data

This checkpoint is fine-tuned on the ADOPD Doc2Box task. In the ADOPD2026 Parquet release, supervision is stored in:

text

ocr.grouped_blocks[].bbox_xyxy

Each training target represents one grouped OCR text region. The model does not use OCR transcription as an output target.

Input And Output

The recommended prompt is:

text

Detect all the text in box format.

The generated response uses LocateAnything normalized coordinate tokens. Use the companion parser to obtain pixel-space boxes_xyxy; do not interpret the raw tokens directly as pixels.

Quick Start

1. Install The Companion Code

bash

git clone https://github.com/SichenZhu/ADOPD2026.git
cd ADOPD2026/release_code

python -m pip install -e model_zoo/common
python -m pip install -e model_zoo/locateanything_3b

Inference requires a CUDA-capable NVIDIA GPU. Eagle source and its compatibility overlay are required for fine-tuning, not for loading this self-contained Transformers checkpoint.

2. Download The Checkpoint

bash

hf download adopd/LocateAnything-3B-text-box-ADOPD \
  --local-dir checkpoints/locany-text-box

3. Run Inference

bash

adopd-locany-infer \
  --checkpoint checkpoints/locany-text-box \
  --image document.jpg \
  --prompt "Detect all the text in box format." \
  --output prediction.json

prediction.json contains the raw generated response and parsed pixel-space boxes.

Fine-Tuning And Evaluation

Download adopd/adopd2026, then prepare the Doc2Box split with:

bash

adopd-locany-prepare \
  --dataset-root DATASET_ROOT \
  --output-root PREPARED_ROOT \
  --split train \
  --task text-box

Training and evaluation commands are documented in `locateanything_3b`.

Limitations

This model detects grouped text regions but does not transcribe text. Generated localization can be sensitive to document domain, image resolution, prompt wording, and decoding parameters. Evaluate and calibrate the model on the target domain before deployment.

License

The ADOPD fine-tuned checkpoint weights are subject to the non-commercial, research-only restriction above. The included NVIDIA License and the terms for the LocateAnything base model and bundled remote-code files also apply. Use is permitted only when all applicable terms are satisfied.

Citation

Please cite the ADOPD 2026 and ADOPD 2024 papers.

bibtex

@misc{zhu2026thinkingwithanchors,
  title={Thinking with Anchors: Grounded and Efficient Document Reasoning},
  author={Sichen Zhu and Yuchen Zhu and Wenzhuo Xu and Jason Kuen and Wanrong Zhu and Jing Shi and Xuan Shen and Quanyi Wang and Yiwei Wang and Yujun Cai and Bing Shuai and Qin Zhang and Yongxin Chen and Shilong Liu and Molei Tao and Jiuxiang Gu},
  year={2026}
}

bibtex

@inproceedings{gu2024adopd,
  title={{ADOPD}: A Large-Scale Document Page Decomposition Dataset},
  author={Jiuxiang Gu and Xiangxi Shi and Jason Kuen and Lu Qi and Ruiyi Zhang and Anqi Liu and Ani Nenkova and Tong Sun},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2024},
  url={https://openreview.net/forum?id=x1ptaXpOYa}
}