A machine learning approach for mortality prediction only using non-invasive parameters

Abstract

At present, the traditional scoring methods generally utilize laboratory measurements to predict mortality. It results in difficulties of early mortality prediction in the rural areas lack of professional laboratorians and medical laboratory equipment. To improve the efficiency, accuracy, and applicability of mortality prediction in the remote areas, a novel mortality prediction method based on machine learning algorithms is proposed, which only uses non-invasive parameters readily available from ordinary monitors and manual measurement. A new feature selection method based on the Bayes error rate is developed to select valuable features. Based on non-invasive parameters, four machine learning models were trained for early mortality prediction. The subjects contained in this study suffered from general critical diseases including but not limited to cancer, bone fracture, and diarrhea. Comparison tests among five traditional scoring methods and these four machine learning models with and without laboratory measurement variables are performed. Only using the non-invasive parameters, the LightGBM algorithms have an excellent performance with the largest accuracy of 0.797 and AUC of 0.879. There is no apparent difference between the mortality prediction performance with and without laboratory measurement variables for the four machine learning methods. After reducing the number of feature variables to no more than 50, the machine learning models still outperform the traditional scoring systems, with AUC higher than 0.83. The machine learning approaches only using non-invasive parameters achieved an excellent mortality prediction performance and can equal those using extra laboratory measurements, which can be applied in rural areas and remote battlefield for mortality risk evaluation.

Appendix

1.1 Machine learning algorithms

As mentioned previously, four commonly used machine learning algorithms: LightGBM, XGBoost, Rand Forest and Logistic regression model, which have excellent classification performance were employed in this present study.

1.1.1 Random forest [1]

Random forest is an integrated classifier, which is composed of a set of decision tree classifiers. The optimal prediction result is determined by the decision tree classifier vote, as indicated by Equation (1).

$$ \hat{y}=\frac{1}{m}\sum \limits_{j=1}^m\sum \limits_{i=1}^n{W}_j\left({x}_i,x^{\prime}\right){y}_i=\sum \limits_{i=1}^n\left(\frac{1}{m}\sum \limits_{j=1}^m{W}_j\left({x}_i,x^{\prime}\right)\right){y}_i $$

(1)

The predictions $\hat{y}$ for new points x′ are achieved by seeking the neighboring point from a training set ${\left\{\left({x}_i,{y}_i\right)\right\}}_{i=1}^n$. A forest averages the predictions of a set of m trees with individual weight functions W_j. W_j(x_j, x′) is the non-negative weight of the i-th training point relative to the new point x′ in the same tree.

1.1.2 XGBOOST [2]

Boosting is a very effective integrated learning algorithm, which can transform a weak classifier into a strong classifier. Gradient boosting, which is an improved version of boosting, makes the loss function to fall along its gradient direction during the iteration process, thus improving algorithm robustness.

Based on the gradient boosting loss function, XGBoost performs a second-order expansion to achieve a faster search for the optimal solution.

The t-th loss function of XGBOOST is defined as Equation (2):

$$ {\mu}^{(t)}\approx \sum \limits_{i=1}^n\left[l\left({y}_i,{\hat{y}}^{\left(t-1\right)}\right)+{g}_i{f}_t\left({X}_i\right)+\frac{1}{2}{h}_i{f}_t^2\left({X}_i\right)\right]+\Omega \left({f}_t\right) $$

(2)

where ${g}_i={\partial}_{\hat{y}}l\left({y}_i,{\hat{y}}^{\left(t-1\right)}\right)$, ${h}_i={\partial}_{{\hat{y}}^{\left(t-1\right)}}^2l\left({y}_i,{\hat{y}}^{\left(t-1\right)}\right)$ and Ω(f_t) are regular terms.

1.1.3 LightGBM [3]

The lightGBM algorithm is a gradient lifting framework that uses a histogram-based algorithm. Instead of finding the split points on the sorted feature values, the histogram-based algorithm buckets continuous feature values into discrete bins and uses these bins to construct feature histograms during training.

Fig. 7

Leaf-wise growth strategy

The white point indicates the leaf with the largest split gain, and the black point is other leaf.

1.1.4 Logistic regression model (LR) [4]

The logistic regression model is a classical classification algorithm commonly used to resolve the binary classification problem. In LR, the sigmoid function presents a relationship between the predicted target variable h_θ(x) and input features x, as formulated by Equation (6).

$$ {h}_{\theta }(x)={\left(1+\exp \left({\theta}_0+{\sum}_{i=1}^N{\theta}_i{x}_i\right)\right)}^{-1} $$

(6)

Where θ is a feature weight vector, which is optimized by the stochastic gradient decent algorithm.

1.2 Traditional scoring systems

Five traditional scoring systems were used to compare with machine learning models and these systems and models have been developed and tested on the same set of data. The traditional scoring systems are briefly introduced as follows:

SAPSII [5] assesses the severity of illness in patients who are older than 15 years in the ICU. The evaluation process is based on 17 parameters, e.g., age, HR, SBP, T, and GCS. The measurements should be finished within 24 h of admission to ICU, with integer measurement values between 0 and 163.

SOFA [6] is suitable for tracking the state of patients treated in the ICU. The SOFA score is based on six different scoring systems, which include the respiratory, cardiovascular, liver, coagulation, kidney, and nervous scoring system.

APS [7], which is based on the results of blood tests and vital signs, is widely used for assessment of the condition of patients in the emergency critical care unit (EICU). The higher the APS score is, the higher the mortality risk is.

Modified early warning score (MEWS) [8], which is suitable for ICU patients over 14 years of age is calculated based on five non-invasive parameters including heart rate, systolic blood pressure, breathing, body temperature, and consciousness. The scoring range of each parameter is from 0 to 3. The final score is the sum of each one of the parameter scores. The higher score value indicates the more deteriorated health condition.

The Oxford acute severity of illness score (OASIS)[9], which was developed for ICU patients using a hybrid genetic algorithm and particle swarm optimization approach, was designed to have an extremely low burden for data collection and quality control, using only 14 features without laboratory measurements, diagnosis, or comorbidity information.

Table 6. The statistical result for non-invasive parameters

Table 7. The OPT_subset and the MIN_subset of XGBoost

Table 8 The OPT_subset and the MIN_subset of LightGBM

Table 9 The OPT_subset and the MIN_subset of Random Forest

Table 10 The OPT_subset and the MIN_subset of Logistic Regression

1.3 Parameters weight

Where (parameter)_XX_XX is the derivative variables of parameter, XX_twe_XX, XX_tf_XX, XX_ts_XX, XX_fe_XX is the derivative variables of the time series data of 0~12 h, 12~24 h, 24~36h, 36~48h after admitting to hospital, respectively and XX_XX_(eigenvalue) is the derivative variables about the eigenvalue.

1.4 Further details on feature extraction

References

1. Svetnik V, Liaw A, Tong C, et al (2003) Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling. J Chem Inf Comput Sci 43:1947–1958. https://doi.org/10.1021/ci034160g

2. Chen T, Guestrin C (2016) XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’16. ACM Press, San Francisco, California, USA, pp 785–794

3. Ke G, Meng Q, Finley T, et al (2017) LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In: Guyon I, Luxburg UV, Bengio S, et al (eds) Advances in Neural Information Processing Systems 30. Curran Associates, Inc., pp 3146–3154

4. Dodek PM, Wiggs BR (1998) Logistic regression model to predict outcome after in-hospital cardiac arrest: validation, accuracy, sensitivity and specificity. Resuscitation 36:201–208. https://doi.org/10.1016/S0300-9572(98)00012-4

5. Le GJ, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. J Am Med Assoc

6. Arts DGT, de Keizer NF, Vroom MB, de Jonge E (2005) Reliability and accuracy of Sequential Organ Failure Assessment (SOFA) scoring. Crit Care Med 33:1988. https://doi.org/10.1097/01.CCM.0000178178.02574.AB

7. Pollack MM, Patel KM, Ruttimann UE (1997) The pediatric risk of mortality III— Acute physiology score (PRISM III-APS): A method of assessing physiologic instability for pediatric intensive care unit patients. J Pediatr 131:575–581. https://doi.org/10.1016/S0022-3476(97)70065-9

8. Moon A, Cosgrove JF, Lea D, et al (2011) An eight year audit before and after the introduction of modified early warning score (MEWS) charts, of patients admitted to a tertiary referral intensive care unit after CPR. Resuscitation 82:150–154. https://doi.org/10.1016/j.resuscitation.2010.09.480

9. Carney C (2011) A New Classification System for Grading the Severity of Onychomycosis: Onychomycosis Severity Index. Arch Dermatol 147:1277. https://doi.org/10.1001/archdermatol.2011.267