Reliability Studies of Different Readers Using Computer Aided Diagnosis System in Ultrasound for Determining Identical Lesions in a Breast Phantom

Background: The computer-aided diagnosis (CAD) has been applied in multiple studies conducted recently in order to perform ultrasound (US) of the breast. There are several studies that have indicated that CAD is useful for improving the diagnostic performance in less experienced radiologists. However, there is no study on several readers analyzing the same lesions using a breast phantom and rare reliability studies. Objectives: To investigate the reliability of different readers using computer aided diagnosis (CAD) system in US for determining identical lesions in a breast phantom. Patients and Methods: From March 2016 to February 2017, six readers (three senior and three junior residents in the department of radiology) evaluated and analyzed breast phantom including 14 lesions. At the first line study, three senior residents (3rd grade with more than one month of training for breast ultrasound [US]) and three junior residents (1st grade without breast US training) evaluated and analyzed the US and applied CAD for lesions in breast phantom, and they were able to make final decisions by subjective combination. A month later, they conducted the second line study as they did the first line study. We analyzed the inter- and intra-reader reliability and accuracy of US, CAD, and combinations (subjective, conjunctive, and disjunctive). Results: In the total of first and second line studies of six readers, the kappa value of US (0.609) was significantly higher than CAD (0.411). In the subjective combined conclusion, the kappa value was improved by the junior group. In the whole inter- and intra-reader analysis, the kappa values of final assessment of the senior and junior group were more variable on CAD than on US, especially in the junior group, and this result was statistically significant. The area under the curve (AUC) of US, CAD, subjective, conjunctive, and disjunctive combination in seniors were all better than those of juniors. In all groups, the AUCs, sensitivities, and specificities were improved on conjunctive combined US with CAD. Conclusion: The combination US with CAD improved the reliability and diagnostic performance. The CAD results of the junior group were variable and inconsistent. Therefore, minimum training and experience for breast US is indispensable for the better use of breast CAD, and, combination US with CAD is useful for all readers.