cesaraha/yolov8s-surgical-instrument-detection-cholec80

YOLOv8s — Surgical Instrument Detection (Laparoscopic Video)

Fine-tuned YOLOv8s model for surgical instrument detection in laparoscopic cholecystectomy video. Given a video frame, the model predicts bounding boxes and class labels for all visible surgical instruments.

An interactive demo is available on Hugging Face Spaces.

Intended Use

This model is intended for research and educational purposes only. It is not validated for clinical use and should not be used to inform clinical decisions.

Dataset

Base dataset: Cholec80-Boxes Cholec80-Boxes extends the Cholec80 laparoscopic video dataset with bounding box annotations for surgical instruments. The original Cholec80 dataset contains 80 videos of laparoscopic cholecystectomy procedures recorded at 25 fps.

Training subset: Videos 41–45 (14,195 annotated frames across all splits)

Instrument classes: Bag, Bipolar, Clipper, Grasper, Hook, Irrigator, Scissors

Class Distribution (Training Set)

Instance counts reflect how frequently each instrument appears across training frames. The dataset has a strong class imbalance, dominated by Hook and Grasper.

Counts based on videos 41–45 only.

Training

Results (Test Set)

Overall

Per-class mAP@50

Findings and Limitations

Strong performers (mAP@50 > 0.75): Hook, Irrigator, Bipolar, and Grasper all achieve reliable detection. Hook in particular reaches near-perfect performance (0.986), which is consistent with its visual distinctiveness and dominant presence in the training data.

Weak performers (mAP@50 < 0.50): Clipper and Scissors underperform significantly. This is a direct consequence of class imbalance — Clipper and Scissors appear in approximately 200 training instances. With this volume of examples, the model does not generalise reliably to unseen frames. This is a data problem, not an architectural one.

Grasper / Hook confusion: In several test sequences, the model misclassifies a stationary Grasper as a Hook. Both instruments share a similar elongated profile when partially occluded or at certain angles, and the model's prior is strongly biased toward Hook given its frequency in the training data. This is a known failure mode.

Bag detection (mAP@50: 0.611): The specimen retrieval bag is visually ambiguous — it is partially transparent, deformable, and its appearance changes substantially depending on how full it is. This accounts for its lower performance relative to its instance count.

Training data scope: The model was trained exclusively on videos 41–45 of Cholec80. Performance on videos from different recording conditions, camera setups, or surgical teams may vary.

Next Steps — Dataset Contribution

The findings above point to a clear path for improvement: expanding the annotated dataset to include more instances of underrepresented classes, particularly Clipper and Scissors.

A follow-up project is underway to use this model for model-assisted annotation of additional Cholec80 videos. The workflow uses the trained detector to generate candidate bounding boxes on previously unannotated frames, which are then reviewed and corrected manually. The resulting extended dataset will be published as cholec80-boxes-extended on Hugging Face Datasets under the same CC BY-NC-SA 4.0 license as the original Cholec80-Boxes.

Collaborators are welcome. Details will be posted when the annotation pipeline is ready.

License

Model weights are released under CC BY-NC-SA 4.0, consistent with the Cholec80-Boxes dataset license. Non-commercial use only.

Citation

If you use this model, please also cite the original Cholec80-Boxes dataset and the original Cholec80 dataset:

@dataset{cholec80boxes,
  title = {Cholec80-Boxes},
  url   = {https://zenodo.org/records/13170928},
}

@article{twinanda2017endonet,
  title   = {EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos},
  author  = {Twinanda, Andru P. and Shehata, Sherif and Mutter, Didier and Marescaux, Jacques and de Mathelin, Michel and Padoy, Nicolas},
  journal = {IEEE Transactions on Medical Imaging},
  volume  = {36},
  number  = {1},
  pages   = {86--97},
  year    = {2017},
}

Acknowledgements

This model was developed as part of a medical computer vision portfolio project. Training data and demo videos are derived from the Cholec80 dataset, originally published by the CAMMA research group at the University of Strasbourg, and from the Cholec80-Boxes annotation extension.