Clinical Trial Finder

Inclusion Criteria:

1. Emergency department clinicians who review CT head scans as part of their clinical practice. 2. Working in the relevant ED during the clinical phase of the project.

Exclusion Criteria:

1. Previous formal postgraduate CT reporting training. 2. Previous career to registrar level in radiology/neurosurgery. 3. Staff in Emergency Department post for less than 4 months.

STUDY DESIGN. This is a prospective interventional cohort study to assess the improvement in the image interpretation accuracy and confidence of reporters using online simulation training for CT head scan interpretation. Using the RAIQC online platform (www.raiqc.com), the trial participants will be required to complete a baseline assessment where users will review and interpret a pre-existing set of CT scans, which have been curated and reported by senior radiologists. They will be asked to record their interpretation, which will be subsequently compared against that of the radiologists. The participants will be blinded to both original reports for the images, and their overall performance results. After completing the baseline assessment, participants will be given access to an online training package. After completing the training, the participants will complete a further assessment. The improvement in diagnostic accuracy pre- and post-training will be measured. Once the participants have been trained, they will each be asked to evaluate a minimum total of 30 CT scans during clinical shifts spread over a 3-month period. For each CT scan included they will be asked to record their scan interpretation, the time the scan was performed, the time they reviewed the scan and the time the radiology report was issued. The radiology report will be considered the gold-standard diagnosis. Online assessments will be repeated at the end of the 3 month period and at 6months post training, to monitor changes in performance over time. Subsequent comparison and analysis will aim to measure the real-world accuracy of ED staff at interpreting CT head scans. In addition, the investigators will also be able to measure potential time saving by ED staff interpreting the scans compared with the radiology report being issued. Population:

• - 180 Emergency Department clinicians (30 per site) of varying grades who routinely interpret images as part of their usual role, including: i.

Junior Doctors (F1-ST2) ii. Registrars (ST3-ST6) iii. Consultants iv. Radiographers v. Nursing and Allied Health Professionals (NMAHPs)

• - Advanced Nurse Practitioners, Physician Associates.

Methods: Participants will be randomised to control (see section below

• - no online training, 30 participants, 5 from each site) versus intervention (online training - 150 participants) groups.

Prior to the start of the study, each participant ID will be randomised to either grou via simple randomisation using an online number generator

• - the outcome of this allocation will be recorded on a linkage document held by the central study team in Oxford.

Intervention Group:

• - All intervention group participants will first be required to complete a baseline online assessment during their non-clinical time, which will consist of CT head images with a clinical vignette of varying difficulty and pathology where participants will review a set of scans and provide a diagnosis and confidence score before submitting the case.

Time taken to complete each individual case will be recorded by the RAIQC platform. Participants will be blinded to the results.

• - Participants will be given access to an existing bespoke online training module on the RAIQC platform (www.

raiqc.com) with a post-training assessment consisting of a different set of cases.

• - Participants will then be required to evaluate a minimum of 30 CT Head scans each - encountered during their clinical shifts over a 3-month period.

They will be asked to record results in real time on a standardised eCRF, including:

• - date/time at which CT scan was performed.

• - date/time of participant review.

• - participant interpretation and confidence score.

• - date/time of radiology report.

• - radiology report findings (reference-standard) NB: Participants will be instructed to base their subsequent clinical decision making based on the formal radiology report as per standard clinical practice, not their own interpretation.

• - Participants will then each repeat a 50-case assessment at the end of the 3-month period and again at 6 months post-training (to assess skill retention) - Qualitative surveys will be completed by participants prior to each assessment module to obtain feedback and comment regarding their experiences of the project and monitor any changes over the course of the study.

Participants will be issued with a certificate to recognise that they have undergone imaging training for Continuous Professional Development purposes. Controls:

• - To provide a control for comparison as to the relative benefits of the online training phase and the clinical interpretation phase in terms of improving reporting performance, 30 participants (5 participants from each site) will be randomised to undertake the clinical interpretation phase prior to the training module.

• - Baseline accuracy statistics from each participant will be compared to subsequent results from each stage of the study.

Analysis:

• - Training: i.

Changes in reporting accuracy (sensitivity/specificity), confidence and speed will be calculated in a pooled analysis for all readers at all sites, as well as for the following subgroups:

• - Clinical role.

• - Level of seniority.

• - Pathological finding.

• - Difficulty of image.

ii. These will be repeated for each of the four sequential assessments.

• - Clinical Interpretation: i.

Accuracy (sensitivity/specificity), confidence and speed will be calculated for all pooled data, and the following subgroups:

• - Clinical role.

• - Level of seniority.

• - Pathological finding.

ii. Total and mean potential time saving will be calculated by comparing time of participant interpretation versus time of radiology report, and will be used to calculate potential cost benefit of clinician reporting versus radiology reporting. iii. Results from the qualitative surveys will be collated and used to explore the experiences and factors affecting the uptake of new image reporting roles in the ED clinical population. PARTICIPANT IDENTIFICATION. Informed Consent: Participants will be provided with an electronic copy of the PIS via email. They will express an interest in participating in the study to the site Principal Investigator, who will discuss the study further with them and answer any questions. Consent will be sought from participants to participate in the study and for their anonymised results to be used for analysis and publication purposes. If they agree to the study, they will then be asked to complete the consent form online and return it to the site Principal Investigator. Discontinuation/Withdrawal of Participants from Study: Each participant has the right to withdraw from the study at any time. In addition, the Investigator may discontinue a participant from the study at any time if the Investigator considers it necessary for any reason including:

• - Ineligibility (either arising during the study or retrospectively having been overlooked at screening) - Significant protocol deviation.

• - Significant non-compliance with training regimen or study requirements.

• - Withdrawal of Consent.

• - Loss to follow up All data obtained to that point will be retained for analysis Withdrawn participants will not be replaced.

The reason for withdrawal will be recorded in the CRF. STATISTICS AND ANALYSIS. Sample Size: During the pilot study, the diagnostic accuracy of participants in detecting acute intracranial abnormality improved by a mean of 8% (SD15%). The investigators calculated a sample size of 30 participants using a power of 80% and a level of significance of 5% (two sided). This was a realistically and practically achievable target for each of the six sites, and 180 participants was, therefore, chosen as the overall target for the current study. Statistical Analyses:

• - Training Assessments: 1) Changes in reporting accuracy/sensitivity/specificity/inter-reader variability/confidence/speed will be calculated in a pooled analysis for all readers at all sites, as well as for the following subgroups: - Clinical role.

• - Level of seniority.

• - Pathological finding.

• - Difficulty of image.

2) These will be repeated for each of the four sequential assessments.

• - Clinical Interpretation Phase: 1.

Accuracy/sensitivity/specificity/inter-reader variability/confidence will calculated for all pooled data, and the following subgroups. Clinical role Level of seniority Pathological finding. 2. Total and mean potential time saving will by calculated by comparing time of participant interpretation versus time of radiology report, and will be used to calculate potential cost benefit of clinician reporting versus radiology reporting. 3. Results from the qualitative surveys will be collated and used explore the experiences and factors affecting the uptake of new image reporting roles in the ED clinical population

Arms

: Intervention (training)

Subjects undergo online CT Head interpretation training: Summary: The trial participants will be required to complete a baseline assessment where users will review a set of scans and provide a diagnosis before submitting the case. Users will then be given access to an online training module. Once the training is completed, the users will complete a second assessment. They will then be asked to evaluate a minimum of 30 CT scans each during their clinical shifts over a 3-month period. They will be asked to record their scan interpretation, the time the scans was performed, and the time they reviewed the scan. They will also be asked to retrospectively document the findings of the standard clinical radiology report, and the time it was issued. At the end of the 3 month period they will repeat the online assessment to assess their performance, and again after a further three months to assess their retention of any improvement in reporting performance.

: Control (no Training)

Subjects do not undergo online CT Head interpretation training: Summary: To provide a control for comparison as to the relative benefits of the online training phase and the clinical interpretation phase in terms of improving reporting performance, 30 participants (6 clinicians from each site) will be randomised to undertake the clinical interpretation phase prior to the training module.

Interventions