Lung carcinoma is the leading cause of cancer-related mortality globally, and in the United States accounts for more deaths than colon, breast and prostate cancer combined. According to the American Cancer Society, the 5-year survival for stage IV lung cancer is less than 10%, as opposed to a 5-year survival of about 60% for disease detected at stage I(1). Screening of high-risk patients with low dose CT provides the best opportunity for detection of early-stage tumors(2). With expanded indications for lung cancer screening with CT since 2021, utilization, and the resultant radiology workload have continued to increase. Advanced detection, characterization, workflow and reporting tools are essential to meet this growing demand(3).
Merge Universal Viewer (MUV) is a fully functioning FDA 510(k) cleared and CE marked diagnostic viewer offered by Merative. It is used for real-time interpretation, image sharing and seamless workflows for all imaging needs. Within MUV, AI and workflow solutions are smoothly integrated, making it an effective tool to enhance the interpretation of time-consuming radiology studies, like Low Dose Chest CT.
We have integrated advanced functionality related to chest CT lung nodule detection and tracking into MUV (currently as work-in-progress – WIP). To assess its impact on radiologists’ workflows, we designed and conducted the following study to evaluate radiologists’ interpretation time with and without this functionality. Our hypothesis was that radiologists would have shorter average interpretation time by using this functionality on chest CT studies with multiple nodules.
- Overview of MUV Advanced Lung Nodule Functionality
When reading chest CT studies, significant time is required of radiologists to: detect disease, measure findings, switch to corresponding anatomical location on priors, and organize and summarize all findings for the report. We have created intelligent workflow enhancements — integrated into MUV — to automates these steps, which helps speed detection, measurement, longitudinal tracking, and report generation.
We will highlight two main components provided by the advanced lung nodule functionality: detection of chest CT lung nodules, and comparison (i.e. tracking) of lung nodules over time. In the nodule detection component, all nodules above a pre-defined size threshold (for instance 5mm in diameter) are automatically detected. Each detected nodule is also automatically segmented, measured (long, short axes and volume), and characterized (lung lobe location, nodule solidity). The nodule tracking component is applicable to patients with more than one CT study. It automatically registers and tracks the detected nodules over time. In addition, it also automatically computes metrics such as volume doubling time and the change percentage in volume and diameter, which can be used to assess nodule stability longitudinally.
- Feature 1: Embedded nodule detection and measurement. The automatically detected and segmented nodules are shown as contours and axes overlays on the CT images. In addition, in the left side panel “LungCT – Lesions” section, a table summarizes information related to all the detected nodules. Each row of the summary table can expand to show additional details if desired. See Figure 1 for an example.
- Feature 2: Easy comparison across multiple studies and easy navigation between nodules. For patients with multiple studies, each nodule is automatically registered and linked to all prior nodules (if they exist). This linking relationship is summarized in the “Tracking Graph” section. Clicking on the “jump-to” buttons in the “Tracking Graph” or the “LungCT – Lesions” sections brings up the corresponding image slice containing this nodule in the primary study and all its prior studies. See Figure 2.
In case the user disagrees with any information which is automatically provided, the user has the option to manually override it.
Figure 1. Screenshots showing (a) the overall interface of the advanced lung nodule functionality; (b) zoomed-in view of the segmented nodule contour and axes, and the summary snapshot when hovering the mouse over this nodule.
Figure 2. Screenshots showing (a) the overall view of side-by-side comparison of a primary study (left) and its prior study (right). The two red arrows point to two “jump-to” buttons that will bring up the corresponding image slice containing that nodule; (b) zoomed-in view of the Tracking Graph and the summary snapshot when hovering the mouse over a nodule.
- Study dataset and design
A research study was conducted to evaluate the impact of MUV lung nodule functionality on radiologists’ interpretation time. A research proposal was granted to download and use the public NLST dataset for this study [1-3]. The NLST chest CT dataset contained longitudinal non-contrast chest CT studies from different patients, with each patient typically having 2 or 3 studies taken at different time points. In addition to image data, NLST also provided structured forms containing information on lung nodule detection results for each study.
The following criteria were applied for data selection:
- Patient had at least 2 studies with slice thickness and slice spacing less than or equal to 3mm;
- For a study with multiple series corresponding to different convolution kernel reconstructions, the least enhanced series was selected (for example Standard, B30f were selected over Bone, B50f kernels);
- For a patient with more than 2 studies, the first and last studies were selected as prior and as primary;
- The primary study contained at least 4 nodules (nodule was defined as a non-calcified nodule or mass >= 4mm defined in the structured form provided by NLST)
All patients/studies fitting these criteria were selected. In total, 140 different patients were selected. Each patient had two studies, which were referred to as the primary study and the prior study. The study consisted of 3 stages:
- Stage 1: Since the NLST dataset did not contain prior reports, the priors had to be read. A qualified radiologist (Rad A) interpreted all the prior studies for all relevant chest findings including but not limited to lung nodules using a radiology PACs viewer (Merge Unity Annotation Tool or UAT). For each prior study, Rad A produced a standard chest CT radiology report (i.e. prior report). The interpretation time for each prior study was recorded, which was used as a surrogate to estimate the complexity of the corresponding primary study.
- Stage 2: Three qualified radiologists (Rad B, C and D) each interpreted 24, 24 and 22 patients respectively (in total 70 patients) using MUV without the advanced lung nodule functionality. For each patient, the radiologist interpreted the primary study for all relevant chest findings including but not limited to lung nodules. If any abnormality was detected in the primary study, the radiologist would use their own judgment on whether and how this abnormality should be evaluated and compared in the prior study. The prior report was also available. The interpretation time for each patient was recorded. The final report was recorded as an audio file.
- Stage 3: The same three radiologists (Rad B, C and D) each interpreted 24, 24 and 22 patients respectively (in total 70 patients) using MUV with the advanced lung nodule functionality. There were no overlapping patients between stage 2 and 3. The exact same interpretation protocol was used for stage 2 and 3. The interpretation time for each patient was recorded and compared to the time in stage 2.
Since stages 2 and 3 utilized different datasets, care was taken to ensure the two datasets had studies with similar complexity. Study complexity was estimated using two metrics: 1) the interpretation time of the corresponding prior study in stage 1; 2) the number of lung nodules in the primary study itself (provided in a structured form by NLST). For each radiologist (Rad B, C, D), their dataset in stage 2 was constructed to have similar complexity as their dataset in stage 3, respectively. The overall study design is summarized in Figure 3.
Figure 3. Flowchart demonstrating study design.
- Evaluation metric
The interpretation time per patient in stage 2 and stage 3 was used as the evaluation metric. Statistics such as the mean and median interpretation time were computed for each radiologist separately as well as all combined. The interpretation time was measured by the time difference between the opening and closing of a patient’s primary study, automatically recorded using the existing MUV software logging mechanism. The radiologists were instructed to only open and close each patient’s data a single time and were not allowed any interruptions. If a patient’s study was opened and closed multiple times this patient would be excluded from the final evaluation.
- Study results
In total, out of the 140 selected patients, 67 patients from stage 2 and 66 patients from stage 3 were included in the final evaluation. Six patients were excluded (3 from stage 2 and 3 from stage 3) because their associated image data were accidentally opened and closed multiple times. One additional patient was excluded from stage 3 due to a technical error in data uploading. Statistics of the interpretation time were summarized in Table 1 and 2. The comparison of interpretation time was summarized in Table 3. Figure 4 shows the histogram of interpretation times (all 3 radiologists combined) for stage 2 and stage 3.
|Stage 2||Number of patients||Mean (standard deviation) interpretation time in seconds||Median interpretation time in seconds||Range of interpretation time in seconds|
|Rad B||23||586 (236)||495||[259, 1178]|
|Rad C||22||813 (218)||810||[458, 1235]|
|Rad D||22||639 (151)||618||[422, 1069]|
|Combined||67||678 (227)||634||[259, 1235]|
Table 1. Summary of interpretation time statistics from stage 2 (MUV without lung nodule functionality).
|Stage 3||Number of patients||Mean (standard deviation) interpretation time in seconds||Median interpretation time in seconds||Range of interpretation time in seconds|
|Rad B||23||458 (280)||314||[194, 1095]|
|Rad C||24||607 (118)||579||[447, 888]|
|Rad D||19||484 (69)||483||[359, 595]|
|Combined||66||520 (196)||499||[194, 1095]|
Table 2. Summary of interpretation time statistics from stage 3 (MUV with lung nodule functionality).
|Reduction of mean interpretation time in seconds (absolute)||Reduction of mean interpretation time (%)||P-value in interpretation time differences (student’s t-test)|
Table 3. Comparison of mean interpretation time differences between stage 2 and stage 3. Reduction in time % = 100%*(Mean time in stage 2 – Mean time in stage 3)/Mean time in stage 2.
Figure 4. Histograms of interpretation time (in seconds). Top histogram shows stage 2 interpretation time and bottom histogram shows stage 3 interpretation time. Interpretation times from all 3 radiologists were combined.
- Discussion and Conclusion
This study demonstrated the impact of MUV advanced lung nodule functionality to decrease radiologist interpretation time when reading chest CT studies containing multiple lung nodules. The overall mean interpretation time of chest CT was significantly shorter after using MUV advanced lung nodule functionality (678s vs. 520s, p<0.001). The time reduction was 23.3%. For each radiologist, the mean interpretation time was also shorter with this functionality (time reduction ranging from 21.8% to 25.3%).
We designed this study to mimic the real-world workflow of radiologists as closely as possible. For example, the radiologists reviewed all relevant chest CT findings, not just lung nodules. In addition, they had access to patient history information in the form of prior radiology reports (created in study stage 1) when interpreting primary studies, which mimicked the workflow in a standard clinical setting.
The study has limitations. First, the studies used in stage 2 were different from those in stage 3. Even though they were estimated to be of similar complexities, there could still be slight variations in case complexities and difficulties. Second, we did not evaluate the impact of this lung nodule functionality on interpretation accuracy. Only interpretation time was evaluated. Third, the study was focused on patients with 4 or more lung nodules as it is those patients that take the most time to read. We plan to conduct another study to quantify time savings for patients with fewer nodules or no nodules.
Despite the limitations we feel that the significant reduction in interpretation time could result in time and cost savings over current standard practice and potentially help radiologists meet ever increasing time pressures.
The authors thank the National Cancer Institute for access to NCI’s data collected by the National Lung Screening Trial (NLST). The NLST project number of this study is NLST-949.
- Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer Statistics 2021. CA Cancer J Clin 2021; 71:7-33
- Krist AH, Davidson KW, Mangione CM, et al.; US Preventive Services Task Force. Screening for Lung Cancer: US Preventive Task Force Recommendation Statement. JAMA 2021; 325:962-970.
- Tariq A, Purkayastha S, Padmanaban GP, et al. Current Clinical Applications of Artificial Intelligence in Radiology and their best supporting evidence. J Am Coll Radiol 2020; 17: 1370-1381
 National Lung Screening Trial Research Team. (2013). Data from the National Lung Screening Trial (NLST) [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/TCIA.HMQ8-J677
 National Lung Screening Trial Research Team; Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, Gareen IF, Gatsonis C, Marcus PM, Sicks JD (2011). Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening. New England Journal of Medicine, 365(5), 395–409. https://doi.org/10.1056/nejmoa1102873
 Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, Tarbox L, Prior F. The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository, Journal of Digital Imaging, Volume 26, Number 6, December, 2013, pp 1045-1057. DOI: https://doi.org/10.1007/s10278-013-9622-7
Forward Looking Statements:
Merative’s statements regarding its plans, directions and intent are subject to change or withdrawal without notice at Merative’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
This technology is in the research and development phase and has not been evaluated by any regulatory agencies for safety or efficacy. It is not available for any commercial or non-commercial use. Information about R&D stage technology is shared only for purposes of feedback.