Datasetpublic

NEST3D/dataset

NEST3D: A High-Resolution Multimodal Dataset of Sociable Weaver Tree Nests Dataset Description NEST3D is a multimodal dataset of 104 sociable weaver nests, combining drone-based RGB and multispectral imagery with a semantically annotated 3D RGB point cloud. It captures trees hosting these nests through drone-based remote sensing, providing rich spatial and spectral information to benchmark and advance scene-level semantic segmentation methods for computer vision… See the full description on the dataset page: https://huggingface.co/datasets/NEST3D/dataset.

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

1likes1.2kdownloads

Dataset Card

NEST3D: A High-Resolution Multimodal Dataset of Sociable Weaver Tree Nests

[image]

Dataset Description

NEST3D is a multimodal dataset of 104 sociable weaver nests, combining drone-based RGB and multispectral imagery with a semantically annotated 3D RGB point cloud. It captures trees hosting these nests through drone-based remote sensing, providing rich spatial and spectral information to benchmark and advance scene-level semantic segmentation methods for computer vision and ecological monitoring applications.

Key Characteristics

—Modality: Multimodal (RGB imagery, multispectral bands, 3D point clouds)
—Task: Scene-level semantic segmentation
—Scale: Multiple tree-nest scenes with consistent spatial and spectral coverage
—Annotation: Point-level semantic labels for 3D point clouds
—Data Source: Drone-based RGB and multispectral imagery
—Application Domain: Ecological monitoring, wildlife management, 3d semantic segmentation, 3d reconstruction.

Dataset Organization

The dataset is organized into modality-specific directories to support flexible access and reuse:

Directory Structure

NEST3D/
├── images/
│   ├── sample001.tar.gz
│   │   └── (extracts to:)
│   │       ├── RGB/
│   │       │   ├── sample001_RGB_001.JPG
│   │       │   └── ...
│   │       └── MS/
│   │           ├── Green/
│   │           │   ├── sample001_G_001.TIF
│   │           │   └── ...
│   │           ├── Red/
│   │           │   ├── sample001_R_001.TIF
│   │           │   └── ...
│   │           ├── Red_Edge/
│   │           │   ├── sample001_RE_001.TIF
│   │           │   └── ...
│   │           └── NIR/
│   │               ├── sample001_NIR_001.TIF
│   │               └── ...
│   ├── sample002.tar.gz
│   └── ...
│
├── reconstructions/
│   ├── sample001/
│   │   ├── sample001.ply
│   │   ├── sample001_rgb_cameras.json
│   │   ├── sample001_ms_cameras.json
│   │   └── sample001_summary.json
│   └── ...
│
├── metadata/
│   └── (ecological metadata — coming soon)
│
├── validation/
│   ├── expert_validation_R1.csv
│   └── expert_validation_R2.csv
│
├── scripts/
│   ├── preprocess_step1_correct_ply.py
│   ├── preprocess_step2_qc_plots.py
│   ├── preprocess_step3_ptv3.py
│   ├── preprocess_step4_o3dml.py
│   └── extract_metadata.py
│
├── train.txt
├── val.txt
└── test.txt

Data Modalities

1. RGB Imagery

—Raw drone images from aerial acquisition
—Format: JPEG, packaged per scene as a compressed archive
—Example path: images/sample001.tar.gz → RGB/sample001_RGB_119.JPG

2. Multispectral Imagery

—Four spectral bands from the same acquisitions as RGB
—Organized into four band-specific folders within each scene's archive:
—Green (G): Green channel imagery
—Red (R): Red channel imagery
—Red Edge (RE): Red Edge channel for vegetation analysis
—NIR: Near-Infrared channel for vegetation health assessment
—Format: GeoTIFF (.TIF)
—Example paths (inside images/sample001.tar.gz):
—MS/Green/sample001_G_119.TIF
—MS/Red/sample001_R_119.TIF
—MS/Red_Edge/sample001_RE_119.TIF
—MS/NIR/sample001_NIR_119.TIF

3. 3D Point Clouds

—One binary PLY file per scene containing the complete 3D reconstruction
—Format: .ply (binary, little-endian)
—Per-point attributes: [x, y, z, red, green, blue, scalar_Classification]
—x, y, z: 3D spatial coordinates (meters)
—red, green, blue: RGB color values (0–255)
—scalar_Classification: Semantic class label (float-encoded integer): 0 = grass, 1 = tree, 2 = nest, 255 = unclassified / ignore
—Example path: reconstructions/sample001/sample001.ply
—Note on the ignore label: a subset of points in some scenes could not be confidently assigned a class during manual annotation and are marked 255. This affects 24 of the 104 scenes, ranging from 0.14% to 13.09% of points in the affected scenes. We recommend excluding these points from training and evaluation via an ignore-index mask.

4. Camera Parameters

Each scene's reconstructions/sampleXXX/ folder includes three JSON files:

—`sampleXXX_rgb_cameras.json`: per-image intrinsics (focal length, principal point, Brown–Conrady distortion coefficients) and extrinsics (camera-to-chunk / chunk-to-camera 4×4 transforms) for every RGB image used in the photogrammetric reconstruction.
—`sampleXXX_ms_cameras.json`: the same per-image intrinsics and extrinsics for every multispectral image (all four bands). Camera extrinsics are identical between the RGB camera and the four multispectral bands at each capture, reflecting the rigid multi-camera rig calibration in which all five sensors are treated as co-located on a shared gimbal. Intrinsics are estimated independently per sensor.
—`sampleXXX_summary.json`: per-scene summary metadata, including total point count, number of aligned RGB/multispectral cameras per band, and the chunk-to-world georeferencing transform.

5. Ecological Metadata

(Section coming soon — derived per-scene ecological statistics, e.g. tree height, canopy area, nest count and volume.)

Expert Biological Validation

The files expert_validation_R1.csv and expert_validation_R2.csv contain independent biological assessments of all 104 annotated point clouds, conducted by two field biologists with expertise in sociable weaver ecology. Each sample was evaluated on the following criteria:

Column	Description	Options
`sample_id`	Sample identifier	sample001–sample104
`Q1_nest_completeness`	Does the nest label capture the full visible nest structure?	Yes - fully captured / Partial / No / Cannot assess
`Q2_nest_precision`	Does the nest label include false positives from the tree canopy?	None / Minor / Significant / Cannot assess
`Q3_boundary_quality`	Is the nest–tree boundary ecologically reasonable?	Yes / Acceptable / No / Cannot assess
`Q4_grass_ground_plane`	Does the grass label correctly represent ground-level vegetation?	Yes / Acceptable / No / Cannot assess
`Q5_overall_quality`	Overall annotation quality	1 (very poor) – 5 (excellent)
`Q6_tree_species`	Identified host tree species	Vachellia erioloba / Boscia albitrunca / Other / Uncertain
`Q6b_species_confidence`	Confidence in species identification	Confident / Somewhat confident / Uncertain
`Q7_nest_activity`	Estimated nest activity status	Active / Likely active / Likely abandoned / Abandoned / Cannot assess
`Q8_occlusion_severity`	Degree of nest occlusion by canopy	Low / Medium / High
`Q9_annotation_difficulty`	Estimated annotation difficulty	Easy / Moderate / Hard / Very hard
`Q10_free_observations`	Free-text ecological observations	—

Data Splits

Sample IDs for a stratified 72/16/16 train/validation/test split are provided as train.txt, val.txt, and test.txt (one sample ID per line) in the repository root. The split was stratified by nest-point percentage to ensure balanced representation of the minority (nest) class across subsets. These files are the authoritative split definition used in our reported benchmarks; users are of course free to define alternative splits from the provided scenes.

Processing Scripts

The scripts/ folder contains the preprocessing pipeline used to prepare the raw data for model training:

—`preprocess_step1_correct_ply.py`: Cleans the raw point clouds — removes spatial outlier grass points, levels the ground plane where needed, grounds the Z-axis, and applies a point budget/downsampling step for very large scenes. Produces a corrected point cloud (not released; regenerate locally by running this script on the raw .ply files).
—`preprocess_step2_qc_plots.py`: Generates quality-control visualizations (top/side/front views, labeled and RGB-colored, with per-class point counts) for visual inspection of raw or corrected point clouds.
—`preprocess_step3_ptv3.py`: Converts corrected point clouds into the Pointcept training format (coord.npy, color.npy, segment.npy), split into train/val/test folders per train.txt/val.txt/test.txt.
—`preprocess_step4_o3dml.py`: Converts the Pointcept-format data into the Open3D-ML format used for RandLA-Net and KPConv training/evaluation.
—`extract_metadata.py`: (documentation coming soon)

Usage

Loading 3D Point Clouds

python

from plyfile import PlyData
import numpy as np

# Load point cloud with semantic labels
ply = PlyData.read('reconstructions/sample001/sample001.ply')
vertex = ply['vertex']

xyz = np.stack([vertex['x'], vertex['y'], vertex['z']], axis=-1)
rgb = np.stack([vertex['red'], vertex['green'], vertex['blue']], axis=-1)
labels = np.asarray(vertex['scalar_Classification'])  # 0=grass, 1=tree, 2=nest, 255=ignore

Loading Multispectral Imagery

python

import tarfile
from PIL import Image
import numpy as np
import io

with tarfile.open('images/sample001.tar.gz') as tar:
    def load_band(path):
        f = tar.extractfile(path)
        return np.array(Image.open(io.BytesIO(f.read())))

    green    = load_band('MS/Green/sample001_G_001.TIF')
    red      = load_band('MS/Red/sample001_R_001.TIF')
    red_edge = load_band('MS/Red_Edge/sample001_RE_001.TIF')
    nir      = load_band('MS/NIR/sample001_NIR_001.TIF')

    multispectral = np.stack([green, red, red_edge, nir], axis=-1)

Using with Hugging Face Datasets Library

python

from datasets import load_dataset

# Load the dataset
dataset = load_dataset('NEST3D/dataset')

Downloading the Dataset

Option 1: Using Hugging Face Hub

bash

pip install huggingface_hub

huggingface-cli download NEST3D/dataset --repo-type dataset --local-dir ./NEST3D

Dataset Information

—Number of Scenes: 104
—Total Points: 951.66M (mean 9.15M ± 8.63M per scene; range 0.69M–68.39M)
—Total RGB Images: 25,172 (mean 242 per scene; range 19–562)
—Class Distribution: Grass 45.39%, Tree 48.30%, Nest 5.28%
—Modalities: RGB, Multispectral (4 bands), 3D Point Clouds
—Image Format: JPEG (RGB), GeoTIFF (Multispectral)
—Point Cloud Format: Binary PLY
—Annotation Type: Per-point semantic labels, plus expert biological validation (see above)

Citation

If you use NEST3D, please cite the dataset:

bibtex

@misc{nest3d_dataset,
  author    = {Molina Catricheo, Constanza A. and Guo, Ting-Jia and May, Giacomo and Reinhard, Friedrich F. and Risse, Benjamin},
  title     = {{NEST3D}: A High-Resolution Multimodal Dataset of Sociable Weaver Tree Nests},
  year      = {2026},
  publisher = {Hugging Face},
  doi       = {10.57967/hf/9225},
  url       = {https://huggingface.co/datasets/NEST3D/dataset},
  note      = {Version 1.0}
}

DOI: https://doi.org/10.57967/hf/9225

Acknowledgments

This work was funded by:

—European Union's Horizon Europe research and innovation programme through the Marie Skłodowska-Curie project "WildDrone – Autonomous Drones for Nature Conservation" (grant agreement no. 101071224)
—EPSRC-funded "Autonomous Drones for Nature Conservation Missions" grant (EP/X029077/1)
—Swiss State Secretariat for Education, Research and Innovation (SERI) under contract number 22.00280

We extend our gratitude to our collaborators and field partners in Namibia for their invaluable support during data collection.

Contact & Support

For questions, issues, or contributions, please visit the dataset discussion forum.

Last Updated: July 2026 Dataset Version: 1.1