In search of the ‘best’ rapid response early warning system – The journey has just begun


In search of the ‘best’ rapid response early warning system – The journey has just begun


Jack Chen

Article Outline

While rapid response systems (RRS) have been implemented across many different countries, there remains challenges and uncertainties regarding the effectiveness and efficiency of both RRS as a concept. its components (such as the afferent arm, efferent arm, implementation strategy, monitoring and evaluation system related to RRS). A unique challenge in providing a better evidence-base is the multitude of different early warning systems (EWS) used in the practice of activating a RRS. For example, in 2008, Smith et al reported that there were 33 unique aggregated weighted track-and-trick systems (AWATTS)  and 39 single-parameter “track and trigger” systems . For the AWTTS, there is no consistency regarding their physiological components, and the majority differ only in minor variations in the weightings for physiological derangement and/or the cut-off points between physiological weighting bands. The task of producing good research evidence on different EWS is even more onerous given the widespread use of the electronic health record (EHR) and paperless hospital IT systems which provide unique opportunities to incorporate more complicated machine-learning methods into EWS.

The evidence-base in the comparative effectiveness and the superiority of different EWS is weak and faces many formidable challenges. These challenges include, but not limited to: 1. There is a lack of consensus on the types of clinical outcome, time period used to judge the chosen clinical outcomes, and the frequency of the vital signs or biomarkers being measured; 2. There is a lack of consensus on what study design and methodology should be used and the often used retrospective design is fraught with methodological and conceptual pitfalls; 3. There is a lack of evidence to test the EWS as a whole which also includes the subjective ‘worried’ criterion. The subjective ‘worried’ criterion, which embodies the nurses or medical staff’s clinical intuition and judgement as well as situation awareness, enhanced or hindered by the organisational culture , should not be discarded as just a simple ‘add-on’. This is because the ‘worried’ criterion is often the leading reason to activate a rapid response team (RRT)  and failing to evaluate both subjective and objective criterion together could jeopardise the practical importance of its findings. The effectiveness of the objective EWS could also be contingent upon the implementation, education and evaluation strategies as well as the organisational culture that interact with the ‘worried’ criterion in a particular way. For example, an EWS within a RRS with sufficient attention to the organisational culture, great emphasises on staff education, enabling and encouraging a proactive attitude towards raising the alarm on deteriorating patients may have a much high rate of activating a RRT based on the ‘worried’ criterion than on black-box based activation algorithms. 4. The validity and efficacy of different EWS can only be ascertained through a prospective study with rigorous design. 5. The cost, easiness of implementation, simplicity and intuitive appealing of different EWSs may also be weighed against their respective marginal gains.

Given these thoughts, the article by Green and colleagues [9] in this issue of Resuscitation provided a much needed and important attempt to search the ‘best’ EWS. It’s strengths included a head-to-head comparison of predictive validity over several most widely used EWS (ie NEWS, MEWS, BTF and eCART) using a large database. The inclusion of the comprehensive electronic-based cardiac arrest triage (eCART) as one of the comparators is timely given that many hospitals in developed countries are moving into an EHR environment. The eCART system includes both vital signs (temperature, heart rate, blood pressure, respiratory rate, oxygen saturation), mental status (coded as alert, responds to voice, responds to pain, or nonresponsive [AVPU]), 18 different laboratory values and patient demographics . The system was developed through a random forest machine learning algorithm in this study. This new algorithm/method is different from the discrete survival analysis modelling technique used in developing the eCART by its authors in 2014  which was based on the same five hospitals database but during a slightly different period (Nov 2008–Jan 2013 in the 2014 study but Nov 2008–August 2013 in the current study) with quite a reduced size. The current study used the database retrospectively and showed that the eCART system, excluding the subjective ‘worried’ criterion, demonstrated a higher level of sensitivity (ie 36.9%) than NEWS (30.9%), BTF (27.5%) and MEWS (24.9%) at a level of specificity around 95% for combined outcome within 24h but it also showed that there was no significant difference in predicting the adverse events between the eCART and BTF during the ward segment (3771/5484 vs 3908/5485, Table 1 in the paper [9]). The marginal increase of the eCART over other systems may be due to a few factors such as the increased number of parameters in the model, the improved data mining algorithm, the repeated and increased number of data points among those critically ill patients, or being an artefact of its unique setting (the dataset is from 5 hospitals with a RRT with no activation criteria which is very unusual). Like other retrospective study designs, it also did not take into consideration the potential impacts of other RRS components. Research shows that the implementation of a RRS could have profound impact on clinical practices (such as the frequency of documentation of the vital signs and the assignment of a not-for-resuscitation order . These potential impacts could not be measured with a retrospective design. Thus, despite some promising results in showing the efficiency of the objective eCART over other EWS in some limited fashion, it remains a long journey for any EWS to first demonstrate its effectiveness in saving patient lives and then to show its comparative effectiveness to other different EWS in a prospective and better designed study. It is worth noting that the BTF has demonstrated such a significant impact in reducing unexpected deaths in a large health jurisdiction that provided strong evidence in supporting its effectiveness . A more recent study by Smith et al (2016) also showed that some single-parameter EWS may have better sensitivity compared to NEWS at a score of greater than, or equal to, 7 but it’s at the expense of increased false alarms and workload (ie. lower specificity), despite that what constitute an ‘acceptable’ specificity may still be a subject of debate. Thus, it is too early to conclude any supremacy of the compared EWS in this study and better evidence is still needed. 


J. McGaughey, P. O’Halloran, S. Porter, et al.Early warning systems and rapid response to the deteriorating patient in hospital: a systematic realist review
J Adv Nurs, 73 (2017), pp. 2877-2891, 10.1111/jan.13398
S.M. Chapman, J. Wray, K. Oulton, et al.Systematic review of paediatric track and trigger systems for hospitalised children
Resuscitation, 109 (2016), pp. 87-109, 10.1016/j.resuscitation.2016.07.230
J. Tirkkonen, T. Tamminen, M.B. SkrifvarsOutcome of adult patients attended by rapid response teams: a systematic review of the literature
Resuscitation, 112 (2017), pp. 43-52, 10.1016/j.resuscitation.2016.12.023
M.E. Smith, J.C. Chiovaro, M. O’Neil, et al.Early warning system scores for clinical deterioration in hospitalized patients: a systematic review
Ann Am Thorac Soc, 11 (9) (2014), pp. 1454-1465, 10.1513/AnnalsATS.201403-102OC
G.B. Smith, D.R. Prytherch, P.E. Schmidt, et al.Review and performance evaluation of aggregate weighted ‘track and trigger’ systems
Resuscitation, 77 (2) (2008), pp. 170-179
G.B. Smith, D.R. Prytherch, P.E. Schmidt, et al.A review, and performance evaluation, of single-parameter track and trigger systems
Resuscitation, 79 (1) (2008), pp. 11-21
A.R. Patel, F.J. Zadravecz, R.S. Young, et al.The value of clinical judgmentinthe detection of clinical deterioration
JAMA Int Med, 175 (3) (2015), pp. 456-458, 10.1001/jamainternmed.2014.7119
J. Chen, R. Bellomo, K. Hillman, et al.Triggers for emergency team activation: a multicenter assessment
J Crit Care, 25 (2) (2010)
359. e1-59. e7
M. Green, H. Lander, A. Snyder, et al.Comparison of the Between the Flags calling criteria to the MEWS, NEWS and the electronic Cardiac Arrest Risk Triage (eCART) score for the identification of deteriorating ward patients
Resuscitation, 123 (2018), pp. 86-91
M.M. Churpek, D.P. EdelsonIn search of the optimal rapid response system bundle
J Hosp Med, 10 (6) (2015), p. 411, 10.1002/jhm.2346
M.M. Churpek, T.C. Yuen, C. Winslow, et al.Multicenter development and validation of a risk stratification tool for ward patients
Am J Respir Crit Care Med, 190 (6) (2014), pp. 649-655, 10.1164/rccm.201406-1022OC
J. Chen, K. Hillman, R. Bellomo, et al.The impact of introducing medical emergency team system on the documentations of vital signs
Resuscitation, 80 (1) (2009), pp. 35-43
J. Chen, A. Flabouris, R. Bellomo, et al.The medical emergency team system and not-for-resuscitation orders: results from the MERIT study
Resuscitation, 79 (3) (2008), pp. 391-397
J. Chen, L. Ou, A. Flabouris, et al.Impact of a standardized rapid response system on outcomes in a large healthcare jurisdiction
Resuscitation, 107 (2016), pp. 47-56, 10.1016/j.resuscitation.2016.07.240
G.B. Smith, D.R. Prytherch, S. Jarvis, et al.A comparison of the ability of the physiologic components of medical emergency team criteria and the U.K. National early warning score to discriminate patients at risk of a range of adverse clinical outcomes
Crit Care Med, 44 (12) (2016), pp. 2171-2181, 10.1097/CCM.0000000000002000