Does the National Early Warning Score 2 system serve its purpose?

RESUSCITATION JOURNAL COVER


Does the National Early Warning Score 2 system serve its purpose?

Jack Che

Article outline

The study by Pimental et al.1 is important as it is the first to compare the ability of the National Early Warning Score (NEWS)2, 3 with the National Early Warning Score 2 (NEWS2) to identify patients at risk of in hospital mortality, cardiac arrests and unplanned ICU admissions. Building upon the recommended and widely used NEWS system in UK hospitals and other parts of the world, NEWS2 introduced different weights at a lower SpO2 scale (i.e., <88–92%) in comparison to the SpO2 threshold <95% in the NEWS weights. NEWS2 also introduces new increasing weights for the use of supplemental oxygen at higher SpO2 level (e.g., >93–94%) to reflect the concern of hyperopia inducted hypercapnia respiratory failure for patients with/at risk of type II respiratory failure (T2RF). These changes in NEWS weight scheme keep in consistent with the current guideline for the T2RF patients.4, 5

However, despite the laudable purposes to make clinical meaningful changes among an important patient group, the accuracy and logistic burden of the NEWS2 have not been tested against widely used NEWS system.6, 7 This multi-disciplinary team of authors have conducted a timely study to provide this much needed comparison. Importantly, the study found that the NEWS performed better in predicting in-hospital mortality compared to the NEWS2 in patients at risk of T2RF. However, NEWS and NEW2 showed similar accuracy in predicting in-hospital mortality among documented T2RF patient groups. Both systems also showed similar accuracy in predicting cardiac arrests and unplanned ICU admission among all patient groups (i.e., at risk of T2RF, documented T2RF and Not at risk T2RF). The study adopted a widely used retrospective design. It also provided 95% confidence intervals (CI) of the Area underneath the Receiver Operating Characteristic (AuROC) Curve differences between NEWS and NEWS2. This methodology makes the interpretation of the results much straightforward and clear. However, there are a few caveats should be made in order to fully understand the implications of this study. First, it was a retrospective observational study conducted at five acute hospitals from two UK NHS Trusts which may need further validation from other settings and with stronger study designs such as a prospective stepped-wedge cluster randomised controlled trial.8 Second, the classifications of three patient groups were based on the diagnostic codes from administrative databases. The accuracy of the coding needs to be further examined in future studies. Third, despite the large number of observation sets included in the study, there were only 18 cardiac arrests and 45 unplanned ICU admissions in the Documented T2RF patient group. Thus, as noted by the authors, the results of these two important outcomes between NEWS and NEWS2 among this targeted patient group may be underpowered and inconclusive. Despite its wide use and obvious importance, it should be noted that in-hospital mortality may include both expected and unexpected deaths that may also be rescuable or non-rescuable. In comparison, cardiac arrests may be deemed a more specific outcome with respect to the aim of a rapid response system or early warning system.

Overall, the current study did not provide evidence in supporting the intended benefit of NEWS2 over NEWS. Such results posed serious questions for policy-makers and clinicians given the pending widespread implementation of such a system in the UK. The study also raised some common questions with respect to how an early warning system in targeting a specific patient group should be developed and implemented in general.9 There were plethora efforts in developing specific early warning system for a particular patient group such as those for paediatric patients,10 obstetric patients,11 patients with mental health problems,12 patients with pulmonary embolism,13 patients with COPD exacerbation,2 and patients at the end-of-life.14 There should be a right balance between the noble intention of developing targeted early warning system for patient subgroups and the unintended consequences of introducing many fragmented early warning systems, increased workload for the medical and nurse staff, and increased errors in manual calculation of complex scores. Despite the wide spread of the rapid response system concept, the conceptual and methodological challenges remain as it is very difficult in providing solid research evidence of head-to-head comparison of different scoring systems based on retrospective databases.9 One of the significant challenges is how to combine multiple, time serial, often incomplete vital sign observations sets with other source of information such as the clinical notes and laboratory test results to provide a real time prediction of risk of death and other adverse events for individual patient. There are increasing attempts in applying machine learning 15, 16 and deep learning approach to meet these challenges. One exciting and notable example of such effort is the newly online-published Google’s electronic health records deep learning project paper17 that showed supremacy of using its deep learning approach from different sources of data including clinical notes, vital signs, laboratory tests results, etc. The Google project showed a great accuracy in predicting hospital deaths (AUROC across sites 0.93–0.94).17 Thus, following the same or similar approach, a trained deep learning framework on patient subgroups such as T2RF may in future provide another way of developing a more patient-centred and precise early warning system. At meanwhile, there is an urgent need for further researches to be conducted, preferably in a better design, to understand the incremental value of the NEWS2, if any, over the existing NEWS system.

References

  1. Pimental, M.A.F., Redfern, O.C., Gerry, S., Collins, G.S. et al. A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study.Resuscitation2019134147–156
  2. Hodgson, L.E., Dimitrov, B.D., Congleton, J. et al. A validation of the National Early Warning Score to predict outcome in patients with COPD exacerbation. Thorax20177223–30https://doi.org/10.1136/thoraxjnl-2016-208436
  3. Schmidt, P.E., Meredith, P., Prytherch, D.R. et al. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Qual. Saf. 201524176–177https://doi.org/10.1136/bmjqs-2014-003845
  4. O’Driscoll, B.R., Howard, L.S., Earis, J. et al. British Thoracic Society Guideline for oxygen use in adults in healthcare and emergency settings. BMJ Open Respir Res20174e000170https://doi.org/10.1136/bmjresp-2016-000170
  5. Beasley, R., Chien, J., Douglas, J. et al. Thoracic Society of Australia and New Zealand oxygen guidelines for acute oxygen use in adults: ‘Swimming between the flags’. Respirology2015201182–1191https://doi.org/10.1111/resp.12620
  6. Smith, G.B., Prytherch, D.R., Jarvis, S. et al. A comparison of the ability of the physiologic components of medical emergency team criteria and the U.K. national early warning score to discriminate patients at risk of a range of adverse clinical outcomes?.Crit Care Med2016442171–2181https://doi.org/10.1097/CCM. 0000000000002000
  7. Smith, G.B., Prytherch, D.R., Meredith, P. et al. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation201384465–470
  8. Haegdorens, F., Van Bogaert, P., Roelant, E. et al. The introduction of a rapid response system in acute hospitals: a pragmatic stepped wedge cluster randomised controlled trial. Resuscitation2018129127–134https://doi.org/10.1016/j.resuscitation.2018.04.018
  9. Chen, J. In search of the ‘best’ rapid response early warning system — the journey has just begun. Resuscitation2018123A1–A2https://doi.org/10.1016/j.resuscitation.2017.12.005
  10. Unbeck, M., Lindemalm, S., Nydert, P. et al. Validation of triggers and development of a pediatric trigger tool to identify adverse events. BMC Health Serv Res201414655https://doi.org/10.1186/s12913-014-0655-5
  11. Carle, C., Alexander, P., Columb, M. et al. Design and internal validation of an obstetric early warning score: secondary analysis of the Intensive Care National Audit and Research Centre Case Mix Programme database. Anaesthesia201368354–367https://doi.org/10.1111/anae.12180
  12. Cheung, W., Gullick, J., Thanakrishnan, G. et al. Expansion of a medical emergency team system to a mental health facility.Resuscitation201283293–296
  13. Carroll, B.J., Pemberton, H., Bauer, K.A. et al. Initiation of a multidisciplinary, rapid response team to massive and submassive pulmonary embolism. Am J Cardiol20171201393–1398https://doi.org/10.1016/j.amjcard.2017.07.033
  14. Cardona-Morrell, M., Chapman, A., Turner, R.M. et al. Pre-existing risk factors for in-hospital death among older patients could be used to initiate end-of-life discussions rather than Rapid Response System calls: a case-control study. Resuscitation2016;10976–80https://doi.org/10.1016/j.resuscitation.2016.09.031
  15. Rojas, J.C., Carey, K.A., Edelson, D.P. et al. Predicting intensive care unit readmission with machine learning using electronic health record data. Ann Am Thorac Soc201815846–853https://doi.org/10.1513/AnnalsATS201710-787OC
  16. Koyner, J.L., Carey, K.A., Edelson, D.P. et al. The development of a machine learning inpatient acute kidney injury prediction model. Crit Care Med2018461070–1077https://doi.org/10.1097/CCM.0000000000003123
  17. Rajkomar, A., Oren, E., Chen, K. et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med2018;118https://doi.org/10.1038/s41746-018-0029-1