Risk prediction for acute kidney injury

Access to the dataset requires ethical approval, see https://physionet.org/content/mimiciv/2.2/.

Introduction

Acute kidney injury (AKI) refers to a sudden decrease in kidney function, often arising as a complication of other diseases¹. Signs of AKI include the accumulation of water, sodium, and other metabolites, as well as electrolyte imbalance¹. AKI ranks among the most prevalent diseases, particularly in hospitalized patients, with an incidence rate of 10% to 15%². Among intensive care unit (ICU) admissions, the incidence rate can be as high as 70%¹. Notably, AKI significantly contributes to prolonged, heightened morbidity and mortality, and increased healthcare costs³⁴. Therefore, the severity of AKI and its timely treatment make it a prime candidate for predictive analytics applications. In 2011, a multi-center effort was undertaken to construct a predictive model based on several biomarkers to estimate the recovery of AKI⁵. However, the study’s limited sample size may compromise its generalizability. Large-scale studies are needed to validate the possibility and accuracy of predicting AKI in clinical settings. In addition, it is also important to identify more biomarkers associated with AKI. In recent years, advancements in biomedical data management systems and machine learning methods have facilitated data-driven disease prediction. Consequently, I aim to estimate the risk of AKI in ICU patients with high accuracy using multiple machine-learning models. The results indicate that the various models exhibit performance that exceeds the baseline and traditional regression analysis. This advancement will be of great benefit in preventing AKI and enhancing prognosis in critically ill populations.

Methods

Dataset

The data used in this study were sourced from the Medical Information Mart for Intensive Care (MIMIC)- IV version 2.2, including information on over 40,000 patients admitted to ICUs at the Beth Israel Deaconess Medical Center (BIDMC)⁶⁷.

Data Preprocessing

Given a substantial presence of missing values in the dataset, model training and testing may be affected. Moreover, dropping the corresponding samples will cause an extremely insufficient sample size, and thus, the imputation of missing values for each feature was carried out using the median. Additionally, Z-score normalization was applied to standardize the features.

Feature Selection

The samples were categorized into two groups based on the presence or absence of AKI. Subsequently, a comparative analysis of features between these groups was conducted. Univariate feature selection was adopted, involving the utilization of t-tests for numeric variables and Chi-squared tests for categorical variables. For each individual feature, a small p-value, indicative of a significant difference in distributions between the two groups, suggests the feature’s contribution to discriminating AKI status. To address the multiple testing problem, the Benjamini-Hochberg (BH) procedure and Bonferroni correction were employed and compared with a significance threshold set at 0.001.

Predictor Development and Optimization

Model training and testing employed a 5-fold cross-validation approach. The logistic regression without regularization was used as the benchmark model. Additional models included the elastic net, support vector machine (SVM), gradient-boosted trees, random forest, Adaptive Boosting (AdaBoost), bagging, and soft voting based on the aforementioned models.

Elastic net: The elastic net used in this study is a regularized logistic regression method with both L1 and L2 penalties. The regularization term is defined as:

\[\lambda_1\sum_{1}^{n}\left|w_i\right|+\frac{\lambda_2}{2}\sum_{1}^{n}w_i^2\]

where \(C=\frac{1}{\lambda_1+\lambda_2}\) and \(L1\ ratio=\frac{\lambda_1}{\lambda_1+\lambda_2}\).

SVM: The SVM used in this study used radial basis function (RBF) kernel, which is given by:

\[k\left(x,x^\prime\right)=\exp{\left(\frac{-\left(x-x^\prime\right)^2}{\gamma^2}\right)}\]

Gradient-Boosted Trees: In this study, a histogram-based gradient boosting classification tree was employed to boost within a functional space, targeting pseudo-residuals. Decision trees were used as weak learners.

Random Forest: The random forest involves building multiple decision trees, and the final output is determined by the class most frequently selected across these trees.

AdaBoost: AdaBoost in this study utilized a sequence of decision trees as weak learners on repeatedly modified versions of the data. The final output is determined by a weighted vote.

Bagging: The bagging method in this study used decision trees as the base estimator, creating multiple instances of the base estimator on random subsets of the original training set.

Soft voting: Soft voting in this study involved combining various machine learning classifiers. The final class label was predicted by averaging the predicted probabilities for each class.

Permutation tests were performed by randomly shuffling labels during model training to obtain the empirical chance level. Model fine-tuning involved adjusting hyperparameter values through grid search, with the model demonstrating the best accuracy being reported.

Model Evaluation

The performance of different machine learning models was compared using the area under the receiver operating characteristic curve (AU-ROC) as a metric. The following metrics were employed:

\[Sensitivity=TPR=\frac{TP}{TP+FN}\] \[Specificity=1-FPR=\frac{TN}{FP+TN}\]

where,

TPR: True Positive Rate

TP: Number of True Positives

FN: Number of False Negatives

FPR: False Positive Rate

FP: Number of False Positives

TN: Number of True Negatives

The AU-ROC comparison used an effect size-driven hypothesis testing method. In addition, comparisons between model performance and the chance level utilized one-way Analysis of Variance (ANOVA) with post-hoc Tukey HSD Test, with accuracy as the evaluation metric.

Source Code

https://github.com/Jiachuan-Wang/SPH6004-Assignment-1

Acknowledgements

The author thanks Dr. M. Feng and D. J. S. Tan from the SPH6004 teaching group, NUS, for teaching machine-learning-related knowledge and providing the pre-extracted dataset.

References

1. Goyal A, Daneshpajouhnejad P, Hashmi MF, Bashir K. Acute Kidney Injury. [Updated 2023 Nov 25]. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2024 Jan-. Available from: https://www.ncbi.nlm.nih.gov/books/NBK441896/ ↩︎ ↩︎² ↩︎³

2. Al-Jaghbeer, M., Dealmeida, D., Bilderback, A., Ambrosino, R., & Kellum, J. A. (2018). Clinical Decision Support for In-Hospital AKI. Journal of the American Society of Nephrology, 29(2). https://journals.lww.com/jasn/fulltext/2018/02000/clinical_decision_support_for_in_hospital_aki.34.aspx ↩︎

3. Chertow, G. M., Burdick, E., Honour, M., Bonventre, J. V, & Bates, D. W. (2005). Acute Kidney Injury, Mortality, Length of Stay, and Costs in Hospitalized Patients. Journal of the American Society of Nephrology, 16(11). https://journals.lww.com/jasn/fulltext/2005/11000/acute_kidney_injury,_mortality,_length_of_stay,.29.aspx ↩︎

4. Hobson, C., Ozrazgat-Baslanti, T., Kuxhausen, A., Thottakkara, P., Efron, P. A., Moore, F. A., Moldawer, L. L., Segal, M. S., & Bihorac, A. (2015). Cost and Mortality Associated With Postoperative Acute Kidney Injury. Annals of Surgery, 261(6). https://journals.lww.com/annalsofsurgery/fulltext/2015/06000/cost_and_mortality_associated_with_postoperative.27.aspx ↩︎

5. Srisawat, N, Wen, X, Lee, MJ, Kong, L, Elder, M, Carter, M, Unruh, M, Finkel, K, Vijayan, A, Ramkumar, M, Paganini, E, Singbartl, K, Palevsky, PM & Kellum, JA 2011, Urinary biomarkers and renal recovery in critically ill patients with renal support, Clinical Journal of the American Society of Nephrology, vol. 6, no. 8, pp. 1815-1823. https://doi.org/10.2215/CJN.11261210 ↩︎

6. Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P.C., Mark, R., Mietus, J.E., Moody, G.B., Peng, C.K. and Stanley, H.E., 2000. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215-e220. ↩︎

7. Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L. A., and Mark, R. (2023) ‘MIMIC-IV’ (version 2.2), PhysioNet. Available at: https://doi.org/10.13026/6mm1-ek67. ↩︎