Challenge Description

Full challenge description information can be found here.

Data Annotation Protocol

For the SurgT challenge, the annotators are never allowed to use interpolation, automation or external intervention to aid their labelling. Only manual labelling is permitted.

The provided instructions were the following:

1.  The annotator should watch the entire video and decide which keypoint will be labelled. We recommend the annotator to choose a keypoint that is easy to label throughout the video;
2. Then, the annotator should classify if the keypoint `is_visible_in_both_stereo` for all the video frames. For example, if in frame #50, the keypoint gets out of the field of view in the image of the right camera, then in that frame `is_visible_in_both_stereo = False`.
   
3. Then, in the frames where the keypoint is visible (`is_visible_in_both_stereo = True`), the annotator should label the keypoint while respecting the following rules: (a) The annotator should ensure that the keypoint is mapped accurately and corresponds to the same target region between the stereo images; (b) The annotator should also look back at the previous frame in the video sequence to ensure temporal video consistency in labelling; (c) If the keypoint is difficult to label, or there were conflicting opinions between annotators, then the annotator should set `is_difficult = True` for that image frame, the frames marked as difficult will not affect the results. (4) Finally, the annotations are reviewed by another annotator.

Endovis Rules

Rules for participating in the challenge can be found here.