Datasetpublic

milkyroad/core

Cystoscopy Core Benchmark Raw-source core dataset for cystoscopy image and video classification. Contains cystoscopy videos with bounding-box annotations, and images with lesion-level labels, grades, and stages. Designed as the source-of-truth from which derived training datasets (v1, v2, G) are built. Sources The dataset aggregates three publicly available cystoscopy datasets: Source Origin Modality Patients Images/Frames Type B B_dataset Video clips… See the full description on the dataset page: https://huggingface.co/datasets/milkyroad/core.

sourceHugging Facecc-by-sa-4.0updated 21d agoView on Hugging Face

0likes363downloads

Dataset Card

Cystoscopy Core Benchmark

Raw-source core dataset for cystoscopy image and video classification. Contains cystoscopy videos with bounding-box annotations, and images with lesion-level labels, grades, and stages. Designed as the source-of-truth from which derived training datasets (v1, v2, G) are built.

Sources

The dataset aggregates three publicly available cystoscopy datasets:

Source	Origin	Modality	Patients	Images/Frames	Type
B	B_dataset	Video clips (CLARA+CHROMA / white light)	30	173 videos (2448 annotated frames)	Track clips with bounding boxes
C	CystoDS	Still images (WLC / BLC)	160	8067 images	Histopathology-confirmed stills
D	Lazo et al.	Still images (WLI / NBI)	22	1754 images	Histopathology-confirmed stills

Dataset Structure

cystoscopy-core/
├── data/
│   ├── image_annotations.parquet   # 12,269 rows: per-image labels, grade, stage, segmentation
│   ├── video_annotations.parquet   # 173 rows: per-video metadata (histological type, grade, stage)
│   └── video_frame_boxes.parquet   # 69,108 rows: per-frame bounding boxes (x,y,w,h)
├── B/videos/                        # 173 .mp4 video clips (1920x1080)
├── C/images/<pid>/                  # 8067 .png images organized by patient
└── D/images/<pid>/                  # 1754 .png images organized by patient

Annotation Schemas

image_annotations.parquet (12,269 rows)

Per-image annotation table covering all C images, all D images, and B video frames selected for the G-derived benchmark.

Column	Type	Description
`source_dataset`	string	B, C, or D
`patient_id`	int	Normalized patient ID (B: 1–30, C: 31–190, D: 191–212)
`lesion_id`	string	Lesion grouping (B: `B_<track>_<idx>`, C: `L0`–`L6`/`Multifocal`, D: `D_<case>_pt<pt>_<tissue>`)
`filename`	string	Image filename (C/D only; B uses videoid + frameidx)
`label`	int	0 = malignant tumor (MT), 1 = non-malignant lesion (NML), 2 = non-ROI (NROI)
`grade`	string	`low`, `high`, or null (inferred from histological type / subclass / tissue type)
`stage`	string	`Ta`, `T1`, `Tis`, `T2`, or null (D has no stage)
`subclass`	string	Source-native subclass (C: LowGradePapillary, HighGradePapillary, CIS, etc.)
`subclass2`	string	Secondary reclassification (C only; 16 rows)
`morphology`	string	C: Papillary, Non-papillary, etc.
`visit`	float	C: visit number
`multifocal`	string	C: multifocal flag
`imaging_type`	string	Unified modality (CLARA + CHROMA, white light, WLC, BLC, WLI, NBI)
`segmentation`	list	Polygon segmentation masks (C only; 768 of 8,067 images)
`video_id`	string	B: video clip identifier (e.g., P017cystoscopytrack_077)
`frame_idx`	float	B: frame index within video clip

video_annotations.parquet (173 rows)

Column	Type	Description
`video_id`	string	Video clip identifier
`patient_id`	int	Normalized patient ID
`histological_type`	string	pTaLG, pT1LG, pTaHG, pT1HG, pT2HG, pT2, PUNLMP
`light_mode`	string	CLARA + CHROMA or white light
`num_frames`	int	Actual frame count of the .mp4 file (not original cystoscopy length)
`n_boxes`	int	Number of bounding boxes in this video
`stage`	string	Ta, T1, T2 (derived from histological_type)
`grade`	string	low, high, or null (pT2 and PUNLMP → null)
`imaging_type`	string	Same as light_mode for B

videoframeboxes.parquet (69,108 rows)

Column	Type	Description
`video_id`	string	Video clip identifier
`patient_id`	int	Normalized patient ID
`track_id`	int	Track number within patient
`frame`	int	Frame index within video clip (0-indexed)
`label`	string	Box label (all "tumor"; class comes from video-level histological_type)
`xtl`, `ytl`, `xbr`, `ybr`	float	Bounding box coordinates (pixels, 1920x1080)
`occluded`	int	Occlusion flag
`outside`	int	Outside frame flag
`keyframe`	int	Keyframe flag
`z_order`	int	Z-order for overlap resolution

Label Distribution (image_annotations)

Label	ID	Count
Malignant Tumor (MT)	0	4,512
Non-Malignant Lesion (NML)	1	355
Non-ROI (NROI)	2	7,402
Total		12,269

Grade Distribution

Grade	Count
Low	2,633
High	1,888

Stage Distribution

Stage	Count
Ta	2,017
T1	1,124
Tis	71
T2	261

Histological Type Distribution (B videos)

Type	Count
Urothelial carcinoma pTaLG	77
Urothelial carcinoma pT1LG	38
Urothelial carcinoma pT1HG	27
PUNLMP	12
Urothelial carcinoma pTaHG	8
Urothelial carcinoma pT2HG	7
Urothelial carcinoma pT2	4

Patient IDs

Normalized mapping (1-indexed):

—B: patients 1–30 (30 patients, 173 video track clips)
—C: patients 31–190 (160 patients, 8067 images)
—D: patients 191–212 (22 patients, 1754 images)

Licensing

This is a mixed-license dataset:

—Source B: CC BY 4.0
—Source C (CystoDS): CC BY 4.0
—Source D (Lazo et al.): CC BY-SA 4.0

The combined dataset is distributed under CC BY-SA 4.0 (the most restrictive component license).

Citation

If you use this dataset, please cite the original source datasets.

Usage

python

from datasets import load_dataset
from huggingface_hub import hf_hub_download

# Load annotation tables
ds = load_dataset("milkyroad/cystoscopy-core")

# Download a specific image
hf_hub_download(
    repo_id="milkyroad/cystoscopy-core",
    filename="C/images/P031/d6df5610.png",
    repo_type="dataset",
)

# Download a specific video
hf_hub_download(
    repo_id="milkyroad/cystoscopy-core",
    filename="B/videos/P000_cystoscopy_track_000.mp4",
    repo_type="dataset",
)