Evaluating the Impact of an integrated clinical decision support tool intended to support earlier cancer diagnosis in primary care in Ipswich and East Suffolk CCG

This report details findings of a qualitative evaluation assessing the impact of C the Signs, the newly introduced clinical decision support tool intended to improve the recognition and early referral rates of cancer in the Suffolk and Northeast Essex (SNEE) CCG and Primary Care Network (PCN). This was an independent evaluation of C the Signs, commissioned by the Eastern Academic Health Science Network (EAHSN) in 2021, and carried out by Pamela Knight-Davidson and Sara Spear at Anglia Ruskin University between 2021-2023. The evaluation was informed by an initial quantitative survey designed in collaboration with the researchers and undertaken by Ipswich and East Suffolk CCG. Findings of this initial survey are also detailed in this report. The reported findings are based on factors known to support or hinder uptake and use of technological innovations and services in healthcare settings, including: perceived usefulness and benefits; perceived effort (ease of use); and, social (peer) influence.1 We also provide findings related to the actual usage of C the Signs, as reported by clinicians. Summary of findings The findings are summarised below, and in the main report, as factors supporting adoption, and factors limiting adoption, of C the Signs. We also provide findings relating to our insights about practice procedures for 2WW (two week wait) referrals and safety netting procedures, which we anticipate will be useful for clinicians, the PCN and for the further development of C the Signs. Factors supporting adoption of C the Signs • C the Signs is rated as a highly effective and reliable clinical decision support tool for its intended purpose, by most clinicians. It is also viewed as easy-to-use, with an improved user interface and user experience (compared to the previous system). • C the Signs is useful for supporting clinical decisions through prompts and suggestions. This is especially useful for making decisions regarding the lesser-known cancers. • The robustness of C the Signs, compared to the previous system, may mitigate potential errors in 2WW referrals. • Useful features of C the Signs were highlighted: – a dashboard showing all referrals, facilitating better in-house tracking of referrals, in-house auditing and PCN auditing; – automatic population of patient information from SystmOne; – automatic sending of follow-up information to patients, viewed as a useful safety netting feature. 2.1.2 Factors limiting adoption of C the Signs • Clinicians reported having high familiarity and confidence in referral pathways and in recognising cancer signs and symptoms. This had implications for the ways in which they evaluated and used C the Signs. For example, some predictive and auto-population features were viewed as ‘irrelevant’ or surplus, ‘getting in the way’, or ‘taking longer’, (although in other instances auto population of patient information might be considered to be useful), where this was the case, it resulted in clinicians and: – cutting corners, by going straight to the referral proforma for speed; ( reverting to clinical judgement) – using SystemB’s referral form to check some referral criteria; – using C the Signs interchangeably with SystemB. • Some, clinicians reported that they were not using C the Signs to its full potential due to unfamiliarity and lack of understanding of some features. As well as the reported greater reliance and confidence in clinical acumen than on clinical decision tools, indicated above, motivation to become more familiar with C the Signs might also have been affected by: – time constraints; – (potentially) reduced confidence/familiarity with technology. • The added value of C the Signs (compared to SystemB) was not obvious to some clinicians and impacted upon its usage. • Some challenges encountered when using C the Signs, were reported. These suggested reduced workflow and were reported as risks to adoption in the PCN. For example: – ‘duplication’ of information between the clinical system (SystmOne) and C the Signs; a need for ‘integration into SystmOne’. – reliance on good internet connection and speed, (although this was also noted to be a challenge when using SystemB) • Peer adoption (how colleagues use C the Signs) might be implicated in low usage. Some clinicians reported that they were not using C the Signs at all, as it was not embraced in some practices or not used by colleagues. Other insights • Some steps in the 2WW referral and safety netting processes can potentially lead to delays in sending referrals and/or diagnosis. It is not clear whether these issues are addressed by C the Signs. • Safety netting procedures are not standardised across the PCN, presenting opportunity to promote and further develop the safety netting capabilities of C the Signs. Key conclusions and recommendations Our findings suggest that, overall, C the Signs is viewed as an effective, reliable and useful tool, which has potential to improve upon the procedures and support offered to clinicians for making 2WW referrals and for safety netting. It is also viewed as easy to use and, as having an improved user interface compared to previous systems. Our findings also point to several factors limiting the adoption of C the Signs, however. These relate to users (e.g., reduced motivation to familiarise with it); to the device itself (e.g., perception that some features take longer) and to system limitations (e.g., time constraints limiting familiarisation and adaption) and result in reticence to use C the signs consistently, or in some cases not at all. These limitations could be addressed through strategies that support clinicians to adapt and this is reflected in recommendations 1, 2 and 3 below. Although not specific to C the Signs, other insights gathered whilst carrying out this study, point to a potential for delays in the 2WW referral and limitations to effective safety netting, which might be addressed at PCN level. Recommendations, 4 and 5 address these. Recommendations 1. More support is required for clinicians to adapt and become more acquainted with C the Signs. Utilising ‘innovation champions’ to work alongside clinicians and offer trouble-shooting support might be useful within a period of transition and adjustment (ability to use both systems), followed by a period of implementation (ability to use C the Signs only). 2. During the transition period to using C the Signs, clinicians could be allocated protected time after 2WW consultations, to enable them to familiarise themselves with the new system and consult with colleagues and ‘innovation champions’ on its use, as required. 3. Peer endorsement (trusted peers who are first adopters) might be utilised to demonstrate the added value of C the Signs: i.e., how it complements clinical acumen and workflow, and how it enhances the 2WW referral process. 4. Uncertainties in the process of sending a 2WW referral could be addressed through focused training and, for example, a ‘task to admin’ confirmation receipt, and confirmation of receipt of referrals from the referral site. 5. Safety netting procedures should be standardised across the PCN, and unsecured safety netting procedures should be phased out. The introduction of C the Signs presents opportunity for the PCN to maximise the potential for monitoring results and referrals outside consultations, towards more standardised safety netting. The PCN might also encourage a culture whereby direct responsibility for safety netting is the norm for all practice staff. Other insights • Some steps in the 2WW referral and safety netting processes can potentially lead to delays in sending referrals and/or diagnosis. It is not clear whether these issues are addressed by C the Signs. • Safety netting procedures are not standardised across the PCN, presenting opportunity to promote and further develop the safety netting capabilities of C the Signs. Key conclusions and recommendations Our findings suggest that, overall, C the Signs is viewed as an effective, reliable and useful tool, which has potential to improve upon the procedures and support offered to clinicians for making 2WW referrals and for safety netting. It is also viewed as easy to use and, as having an improved user interface compared to previous systems. Our findings also point to several factors limiting the adoption of C the Signs, however. These relate to users (e.g., reduced motivation to familiarise with it); to the device itself (e.g., perception that some features take longer) and to system limitations (e.g., time constraints limiting familiarisation and adaption) and result in reticence to use C the signs consistently, or in some cases not at all. These limitations could be addressed through strategies that support clinicians to adapt and this is reflected in recommendations 1, 2 and 3 below. Although not specific to C the Signs, other insights gathered whilst carrying out this study, point to a potential for delays in the 2WW referral and limitations to effective safety netting, which might be addressed at PCN level. Recommendations, 4 and 5 address these. Recommendations 1. More support is required for clinicians to adapt and become more acquainted with C the Signs. Utilising ‘innovation champions’ to work alongside clinicians and offer trouble-shooting support might be useful within a period of transition and adjustment (ability to use both systems), followed by a period of implementation (ability to use C the Signs only). 2. During the transition period to using C the Signs, clinicians could be allocated protected time after 2WW consultations, to enable them to familiarise themselves with the new system and consult with colleagues and ‘innovation champions’ on its use, as required. 3. Peer endorsement (trusted peers who are first adopters) might be utilised to demonstrate the added value of C the Signs: i.e., how it complements clinical acumen and workflow, and how it enhances the 2WW referral process. 4. Uncertaintie