yanglei18/A-Paddlefish-Inspired-Robot-for-Molecular-Visual-Surveillance-of-Invasive-Fishes

A Paddlefish-inspired Robot for Molecular–visual Surveillance of Invasive Fishes — Datasets & Model Weights

Data and released model weights for the paper _"A Paddlefish-inspired Robot for Molecular–visual Surveillance of Invasive Fishes"_ (under review).

• —Code: https://github.com/yanglei18/A-Paddlefish-Inspired-Robot-for-Molecular-Visual-Surveillance-of-Invasive-Fishes
• —License: MIT (see note under License for the model weights)

This repository hosts the large assets for the paper's visual perception pipeline, which turns raw underwater video into auditable, species-level invasive-fish confirmations through three modules: object_centric_extractor (detection → segmentation → tracking → instance export), open_world_filter (open-world candidate filtering), and evidence_grounded_reasoner (evidence-grounded reasoning VLM). All items are packaged as archives.

Contents

Two species domains

• —Yanghu pond (10 species) — robotfish-data + the released FG-VLM-4B-Thinking checkpoint. Drives the end-to-end closed loop: extract object-centric clips → classify with FG-VLM → feed the answers back for species-level detection evaluation.
• —Nine invasive species — invasive-fishes-sequence-data. Used to train and evaluate a from-scratch reasoning VLM in evidence_grounded_reasoner.

Usage

Run from the code repository root. Download each archive with the Hugging Face CLI and extract it:

REPO=yanglei18/Bioinspired-Molecular-Visual-Surveillance-of-Invasive-Fishes

# object_centric_extractor input — Yanghu videos + ground truth
huggingface-cli download "$REPO" robotfish-data.zip --repo-type dataset --local-dir ./data
unzip data/robotfish-data.zip -d data/

# open_world_filter dataset
huggingface-cli download "$REPO" fish-recognition-dataset/invasive-dataset.tar.gz \
  --repo-type dataset --local-dir ./data
tar -xzf data/fish-recognition-dataset/invasive-dataset.tar.gz -C data/fish-recognition-dataset/

# evidence_grounded_reasoner training/eval data (nine invasive species)
huggingface-cli download "$REPO" object-centric-sequence-data/invasive-fishes-sequence-data.zip \
  --repo-type dataset --local-dir ./data
unzip data/object-centric-sequence-data/invasive-fishes-sequence-data.zip -d data/

# released FG-VLM-4B-Thinking weights (Yanghu closed-loop evaluation)
huggingface-cli download "$REPO" checkpoints/FG-VLM-4B-Thinking.zip --repo-type dataset --local-dir ./
unzip checkpoints/FG-VLM-4B-Thinking.zip -d checkpoints/

See the code repository README for the full end-to-end pipeline and per-module tutorials.

Reasoner data format (invasive-fishes-sequence-data)

sft_train.json / rl_train.json are lists of { "messages": [...], "videos": ["videos/…/x.mp4"], … } (RL rows additionally carry solution and reasoning_content). val.json rows are { "video_path": "videos/val/x.mp4", "question": "…(A)…(J)…", "answer": "(A) …" }. Video paths are repo-relative; run scripts from the location where the archive was extracted so they resolve.

License

The datasets and code are released under the MIT License. The released FG-VLM-4B-Thinking weights are a fine-tune of Qwen3-VL and additionally inherit the upstream Qwen3-VL model license — please review and comply with it before use.

Citation

If you use these data or weights, please cite the associated paper:

@article{invasive_fish_surveillance_2026,
  title   = {A Paddlefish-inspired Robot for Molecular–visual Surveillance of Invasive Fishes},
  author  = {Li, Lei and Li, Yanyu and Yang, Lei and Yu, Junzhi and He, Dekui and Lv, Chen and others},
  year    = {2026},
  note    = {Under review. Lei Li, Yanyu Li and Lei Yang contributed equally (co-first authors);
             Chen Lv, Dekui He and Junzhi Yu are co-corresponding authors.
             Venue and DOI to be added upon publication}
}