P069 Artificial intelligence (AI)-filtered Videos for Accelerated Scoring of Colonoscopy Videos in Ulcerative Colitis Clinical Trials
Gutierrez Becker, B.(1);Giuffrida, E.(2);Mangia, M.(2);Arcadu, F.(1);Whitehill, V.(3);Guaraglia, D.(4);Schwartz, M.(4);Sabouni, R.(4);Prunotto, M.(5);Oh, Y.(6);Daperno, M.(2);
(1)Roche, Informatics, Basel, Switzerland;(2)Mauriziano Hospital, Gastroenterology Unit, Turin, Italy;(3)Genentech, PD Clinical Science - I2O Immunology- Infectious Diseases- and Ophthalmology, South San Francisco, United States;(4)Virgo Surgical Video Solutions- Inc, Virgo Surgical Video Solutions- Inc, San Francisco, United States;(5)Roche, Immunology- Infectious Disease & Ophthalmology, Basel, Switzerland;(6)Genentech, Product Development, South San Francisco, United States
Background
Endoscopic assessment is a critical procedure to assess the improvement of mucosa and response to therapy, and therefore a pivotal component of clinical trial endpoints for IBD. Central scoring of endoscopic videos is challenging and time consuming. We evaluated the feasibility of using an Artificial Intelligence (AI) algorithm to automatically produce filtered videos where the non-readable portions of the video are removed, with the aim of accelerating the scoring of endoscopic videos.
Methods
The AI algorithm was based on a Convolutional Neural Network trained to perform a binary classification task. This task consisted of assigning the frames in a colonoscopy video to one of two classes: “readable” or “unreadable.” The algorithm was trained using annotations performed by two data scientists (BG, FA). The criteria to consider a frame “readable” were: i) the colon walls were within the field of view; ii) contrast and sharpness of the frame were sufficient to visually inspect the mucosa, and iii) no presence of artifacts completely obstructing the visibility of the mucosa. The frames were extracted randomly from 351 colonoscopy videos of the etrolizumab EUCALYPTUS (NCT01336465) Phase II ulcerative colitis clinical trial.
Evaluation of the performance of the AI algorithm was performed on colonoscopy videos obtained as part of the etrolizumab HICKORY (NCT02100696) and LAUREL (NCT02165215) Phase III ulcerative colitis clinical trials. Each video was filtered using the AI algorithm, resulting in a shorter video where the sections considered unreadable by the AI algorithm were removed.
Each of three annotators (EG, MM and MD) was randomly assigned an equal number of AI-filtered videos and raw videos. The gastroenterologist was tasked to score temporal segments of the video according to the Mayo Clinic Endoscopic Subscore (MCES). Annotations were performed by means of an online annotation platform (Virgo Surgical Video Solutions, Inc).
Results
We measured the time it took the annotators to score raw and AI-filtered videos. We observed a statistically significant reduction (Mann Whitney U test p-value=0.039) in the median time spent by the annotators scoring raw videos (10.59∓ 0.94 minutes) with respect to the time spent scoring AI-filtered videos (9.51 ∓ 0.92 minutes), with a substantial intra-rater agreement when evaluating highlight and raw videos (Cohen’s kappa 0.92 and 0.55 for experienced and junior gastroenterologists respectively).
Conclusion
Our analysis shows that AI can be used reliably as an assisting tool to automatically remove non-readable time segments from full colonoscopy videos. The use of our proposed algorithm can lead to reduced annotation times in the task of centrally reading colonoscopy videos.