A first-level customization study of SAPS II with Norwegian Intensive Care and Pandemic Registry (NIPaR) data

Abstract

Background: Severity scores and mortality prediction models (MPMs) are important tools for benchmarking and stratification in the intensive care unit (ICU) and need to be regularly updated using data from a local and contextual cohort. Simplified acute physiology score II (SAPS II) is widely used in European ICUs.

Methods: A first-level customization was performed on the SAPS II model using data from the Norwegian Intensive Care and Pandemic Registry (NIPaR). Two previous SAPS II models (Model A: the original SAPS II model and Model B: a SAPS II model based on NIPaR data from 2008 to 2010) were compared to the new Model C. Model C was based on patients from 2018 to 2020 (corona virus disease 2019 patients omitted; n = 43,891), and its performances (calibration, discrimination, and uniformity of fit) compared to the previous models (Model A and Model B).

Results: Model C was better calibrated than Model A with a Brier score 0.132 (95% confidence interval 0.130–0.135) versus 0.143 (95% confidence interval 0.141–0.146). The Brier score for Model B was 0.133 (95% confidence interval 0.130–0.135). In the Cox's calibration regression and for both Model C and Model B but not for Model A. Uniformity of fit was similar for Model B and for Model C, both better than for Model A, across age groups, sex, length of stay, type of admission, hospital category, and days on respirator. The area under the receiver operating characteristic curve was 0.79 (95% confidence interval 0.79–0.80), showing acceptable discrimination.

Conclusions: The observed mortality and corresponding SAPS II scores have significantly changed during the last decades and an updated MPM is superior to the original SAPS II. However, proper external validation is required to confirm our findings. Prediction models need to be regularly customized using local datasets in order to optimize their performances.