If you are involved in clinical trials for medical imaging, you know how important it is to have consistent and reliable interpretations of the images by expert readers. However, you also know how challenging it is to achieve this goal, as reader disagreements are common and inevitable in this field.
Reader disagreements can have serious consequences for the quality, validity, and efficiency of clinical trials. They can affect the accuracy of the primary endpoint, the reliability of the secondary endpoints, the sample size calculation, the statistical analysis, the regulatory approval, and the overall cost and timeline of the trial.
So, how can you reduce reader disagreements in clinical trials for medical imaging? What are the factors that influence reader variability and how can you address them effectively?
In this blog post, we will summarize the main points from a recent review article by experts in medical imaging for drug development research. The article aims to produce a singular authoritative resource outlining reader performance’s practical realities within cancer trials, whether they occur within a clinical or an independent central review. We will also introduce you to Collective Minds Research, our innovative clinical trial platform for medical imaging that can help you optimize your reader performance and reduce reader disagreements.
Why do reader disagreements happen?
Reader disagreements happen because of a complex interplay of factors that affect the interpretation of medical images. These factors can be classified into three categories: technical, methodological, and human.
Technical factors
Technical factors refer to the quality and consistency of the image acquisition, processing, and display. These factors can influence the visibility, contrast, resolution, and noise of the image, which can affect the reader’s ability to detect and measure lesions or other features of interest.
Some examples of technical factors that can cause reader disagreements are:
- Different imaging modalities, such as CT, MRI, PET, etc.
- Different imaging protocols, such as contrast, slice thickness, reconstruction algorithm, etc.
- Different image formats, such as DICOM, JPEG, PNG, etc.
- Different image viewers, such as PACS, web-based, proprietary, etc.
- Different image settings, such as window level, zoom, pan, etc.
To reduce reader disagreements caused by technical factors, it is essential to have standardized and harmonized imaging protocols, formats, and viewers across the trial sites and readers. It is also important to have quality control and quality assurance procedures to ensure the image quality and consistency.
Methodological factors
Methodological factors refer to the design and execution of the image analysis and interpretation. These factors can influence the criteria, definitions, rules, and guidelines that the reader follows to assess the image.
Some examples of methodological factors that can cause reader disagreements are:
- Different image analysis methods, such as manual, semi-automatic, or automatic segmentation, measurement, or classification.
- Different image interpretation criteria, such as RECIST, iRECIST, PERCIST, etc.
- Different image interpretation rules, such as the selection of target lesions, the assessment of new lesions, the handling of missing data, etc.
- Different image interpretation guidelines, such as the use of reference images, atlases, templates, etc.
To reduce reader disagreements caused by methodological factors, it is crucial to have clear and consistent image analysis and interpretation protocols, criteria, rules, and guidelines across the trial sites and readers. It is also important to have training and qualification programs to ensure the reader’s competence and compliance.
Human factors
Human factors refer to the cognitive and behavioral aspects of the reader’s performance. These factors can influence the reader’s perception, attention, memory, judgment, decision-making, and communication.
Some examples of human factors that can cause reader disagreements are:
- Cognitive biases, such as confirmation bias, anchoring bias, hindsight bias, etc.
- Motivational biases, such as sponsorship bias, publication bias, peer pressure, etc.
- Emotional biases, such as stress, fatigue, boredom, frustration, etc.
- Communication errors, such as ambiguity, misunderstanding, misinterpretation, etc.
To reduce reader disagreements caused by human factors, it is vital to have objective and independent image analysis and interpretation processes, such as blinded, centralized, or adjudicated reviews. It is also important to have feedback and monitoring mechanisms to ensure the reader’s performance and improvement.
How can Collective Minds Research help you?
Collective Minds Research is a clinical trial platform for medical imaging that can help you reduce reader disagreements and optimize your reader performance. It is a cloud-based, AI-powered, and user-friendly solution that can streamline and automate your image analysis and interpretation workflows.
With Collective Minds Research, you can:
- Upload, store, and manage your images securely and efficiently.
- Analyze and measure your images accurately and consistently using a state-of-the-art integration framework for AI models and algorithms in general.
- Interpret and report your images reliably and transparently using standardized and customized criteria, rules, and guidelines.
- Collaborate and communicate with your trial sites and readers effectively and seamlessly.
- Monitor and audit your image quality and reader performance continuously and comprehensively.
Collective Minds Research is designed to help you overcome the technical, methodological, and human challenges that cause reader disagreements in clinical trials for medical imaging. It is also designed to help you improve the quality, validity, and efficiency of your clinical trials and accelerate the development of new therapies for patients.
If you want to learn more about Collective Minds Research and how it can help you, please visit our website or contact us for a demo below. We would love to hear from you and show you how we can make a difference in your clinical trials for medical imaging.
Pär Kragsterman, CTO and Co-Founder of Collective Minds Radiology
Reviewed by: Anders Nordell on October 17, 2024
