Article Search
닫기

Original Article

Split Viewer

International Journal of Fuzzy Logic and Intelligent Systems 2024; 24(3): 287-294

Published online September 25, 2024

https://doi.org/10.5391/IJFIS.2024.24.3.287

© The Korean Institute of Intelligent Systems

Deep Learning Model for Predicting Critical Patient Conditions

Gayoung Kim

Faculty of General Education, Kangnam University, Youngin, Korea

Correspondence to :
Gayoung Kim (dolga2000@gmail.com)

Received: July 18, 2024; Revised: September 13, 2024; Accepted: September 22, 2024

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted noncommercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Disease prediction using existing deep learning methods can predict diseases with relatively limited clinical data by detecting the symptoms of specific parts and body abnormalities according to the disease. Predicting a patient’s severity requires considering multiple types of clinical data. However, existing models struggle with overfitting and reduced accuracy in making these predictions. This study proposes a system that can predict the signs of critical illness using vital data from multiple patients, such as respiratory rate and oxygen saturation, through deep learning. This enables an early response by medical staff, improving treatment efficiency, and reducing the mortality rate. To address these issues, DNN-based intensive care prediction (DBICP), was used to detect whether the patient is improving and the risk; it showed a prediction accuracy of 95%, which is approximately 10% higher than the prediction accuracy using existing methods. If the predicted patient status is presented to the medical staff through the proposed system, we expect that work efficiency and treatment results will be improved through a more accurate and faster diagnosis.

Keywords: Deep learning, Healthcare, Critical patient condition

Currently, over 50% of critically ill patients in Korea are treated in general wards because of limited hospital beds, and when intensive care unit (ICU) transfer is required in a general ward, medical staff determine the priority of transfer through clinical judgment. The issue is that a delay in transfer to the ICU can lead to consequences such as an increase in the mortality rate of shock patients. In addition, there are limitations to the technology and manpower required for medical staff to regularly validation for signs of condition in all critically ill patients and take prompt action.

Moreover, the supply and demand of medical personnel are difficult because considerable effort and personal sacrifice are required to engage in essential medical care. An early prediction model for the signs of critical illness has emerged to overcome the above problems and plays an effective and decisive role in protecting lives by quickly responding to potential risks to patients. In a representative example, the model detects vital signs such as the respiratory rate and oxygen saturation of critically ill patients, and based on the predicted results, medical staff revise the treatment plan and allocate appropriate medical resources. This early response increases treatment efficiency [1] and reduces patient mortality and cardiac arrest rates [2].

Domestic medical companies have developed software such as artificial intelligence (AI) vital care to detect patient risks early, and many hospitals are introducing it; however, extensive research has been conducted to help recognize the signs of a patient’s condition early.

Among them, heart disease and hepatitis C prediction have demonstrated high performance, and various analytical methods have been presented.

The machine-learning method for disease prediction used in the past could predict diagnosis by learning the parameters necessary for predicting a single major disease. Predicting critically ill patients requires a complex algorithm that learns from the vital information of many patients. The accuracy of the prediction of critically ill patients using existing machine learning is relatively low. Therefore, we improved the accuracy through the proposed deep learning algorithm.

Existing algorithms that can predict a patient’s disease are systems that can predict specific diseases and have few parameters for learning the patient’s condition; however, critical patient prediction requires a relatively large number of parameters because multiple conditions must be considered. In addition, existing disease prediction methods can identify data patterns for specific diseases through machine learning and classify diseases; however, the critical patient system must process a large number of disease-related parameters. To predict critical patients early, which requires processing complex data and a large amount of data, the accuracy of prediction using existing machine learning is low. Therefore, the accuracy of early prediction can be improved through the proposed algorithm.

In this study, various machine learning methods such as ensemble techniques and logistic regression, which have previously been applied to predict heart disease [3], were used to predict the risk of patients developing severe conditions at an early stage. However, interpreting the relationships between variables is challenging because of the complexity of clinical log data. This complexity often leads to poor model performance and overfitting. This problem can be validated through the deviation between the training and verification accuracies of the machine learning methods in Section 4.

To address this issue, we propose a model that considers the various clinical information of patients, reduces overfitting problems, and detects risks in critically ill patients with high performance.

By presenting the prediction model to medical professionals, we aim to assist in evaluating patient conditions. This approach can also help efficiently allocate medical resources by prioritizing high-risk patients. In addition, it is significant in that it can help reduce the high tension of medical staff and relieve and treat patients’ anxiety.

Methods should be described in sufficient detail to allow others to replicate and build on published results. Various patient data are being made public overseas, and research utilizing it is actively underway. The Medical Information Mart for Intensive Care (MIMIC) in the United States released ICU data and used it in various studies. Among these, the long short-term memory (LSTM) technique has demonstrated high performance in predicting death in patients with diabetes [4].

The UCI Machine Learning Repository distributes hepatitis C patient data and several studies were conducted to predict whether hepatitis C was confirmed using this data. A representative and widely cited study presented a method for predicting the onset of hepatitis C using an ensemble technique that combines multiple classifiers. The classifier is a multilayer perceptron (MLP) [5], which has demonstrated high performance in expressing and classifying the numerical information of patients, and MLP, which has shown high performance in expressing categorical variables based on the fact that hepatitis C occurs frequently in patients with jaundice. A Bayesian network was used [6]. The ensemble model built by combining the presented classifiers showed improved performance over the existing single model in predicting whether the disease was confirmed in patients [6]. Similarly, the ensemble-based random forest model built based on patients’ real-time health data showed high performance in predicting the risk of stroke.

To predict the onset of heart disease early, an artificial neural network (ANN) algorithm [7], built based on heart numerical data provided by Cleveland, USA, and the Hungarian Cardiology Group, predicted whether a patient would develop heart disease with a high accuracy of 94%.

In a domestic research case, machine learning was performed using a logistic regression analysis method with the pulse information variables of 49 healthy individuals and 49 atrial fibrillation patients obtained from a photoplethysmogram (PPG) sensor [8]. The model built through learning classified patients with atrial fibrillation and healthy subjects with 85.67% accuracy.

As mentioned previously, we applied the methods used in previous studies to the data used in this study. However, the LSTM technique was unsuitable because there was no time series data from the patient’s admission to discharge in the data to be used; therefore, it was excluded, the proposed model and experimental results were compared by applying techniques such as ANN, logistic regression, and XGBoost models.

3.1 Subsection

The data used in this study were collected over 10 years at an anonymous general hospital in South Korea. Due to medical laws, patient information and specific hospitalization times were anonymized, making it impossible to identify exactly in what year the information was collected. The data consisted of 24 columns and 110,000 rows, including the patient’s vital signs, underlying diseases, antibiotic prescription information, various test results, and treatment results. Patient classifications consisted of general, emergency, and intensive care, and a risk prediction model was designed using only ICU patient data. To reduce unnecessary complexity, safety, and computational efficiency, data that made it difficult to determine the patient’s condition owing to missing prescription records and duplicate variables were removed.

Patient data was provided by the government agency, Yongin City Industrial Promotion Agency, and 8 years of the total 10 years of data were divided into training data and test data to build a deep learning model suitable for this, and the remaining 2 years of the most recent data were used for verification.

As shown in Table 1, most patients were in their 60s, 80s, or older. The proportion of male patients was slightly higher than that of female patients. To efficiently predict at-risk patients, the patient’s treatment results were classified into two scenarios: condition improvement and danger, with data classified as mild, complete recovery, death after 24 hours, death before 24 hours, and 33% of all critically ill patients were classified as condition improving and danger.

Through cosine similarity, diseases with high similarity are likely to have similar characteristics or effects. Therefore, prediction classification was performed by grouping them into the same group to derive a more accurate and useful prediction model.

3.2 Implementation of Predictive Model

First, to compare the constructed dataset with related research and the proposed model, we applied the XGBoost model [9], which showed excellent performance on categorical data, in addition to ensemble, random forest [10], and logistic regression analysis [11].

The proposed model is based on a related study in which an ANN showed high prediction results in heart disease data and is a deep neural network (DNN) [12] that identifies complex patterns by increasing the number of neural networks to process large amounts of categorical data) the technique was applied. To express the complexity of variables and prevent gradient loss, the rectified linear unit (ReLU) activation function [13,14] was used in the hidden layer. To binary classify the final output of the model, the final layer outputs a value between 0 and 1 through the sigmoid function.

In the output layer, binary cross-entropy (BCE) [15] was applied to the loss function to probabilistically evaluate how close the predicted value was to the actual value in binary classification, and the formula is shown in Eq. (1) [16].

BCE(y,y^)=-1Ni=1N[yilog(y^i)+(1-yi)log(1-y^i)].

yi refers to the actual value and has the meaning of 0 (anger) or 1 (improving). ŷi represents the probability between zero and one of the values predicted by the model. yi log(ŷi) evaluates the accuracy of the model when predicting the probability of an outcome as 1. (1 − yi) log(1 − ŷi) evaluates the accuracy of the model when predicting the probability of the outcome being 0.

To optimize the model, Adam was applied as an optimizer. Adam prevents instability in model learning by individually adjusting the learning rate for each parameter and quickly converges the optimization process by updating parameters by considering previous gradient information. A flowchart of the DNN-based intensive care prediction (DBICP) model is shown in Figure 1.

In this study, the software was developed in Python 3.12 and hardware was developed in the Google Colab environment. Related studies’ XGBoost models faced overfitting and performance issues due to accuracy deviations between training and validation, as shown in Figure 2.

This issue arises because predicting risk in critically ill patients requires explaining diverse categorical relationships and learning from more data than in previous studies. To solve this problem, a DNN model was designed and implemented. Consequently, the performance gap between the training and validation data was minimized, confirming that the overfitting problem was resolved. This suggests that the DNN model can capture more complex patterns while maintaining its generalization performance. The structure of the designed DNN model is illustrated in Figures 14. To design the optimal DNN model, the number of layers was set to 1–4 and the parameters were adjusted and compared.

Figure 3 shows the experimental results showing performance problems owing to accuracy deviations in the verification data during the process of checking the accuracy by adjusting the weights according to the number of hidden layers.

After performing several experimental procedures, the parameters of the model that showed optimal performance were one input layer and two hidden layers; the optimal performance was shown when the respective weights and parameters were set, as shown in Figure 4.

Figure 5 shows the performance of the proposed model. The prediction accuracy was 95.4%, the accuracy of the verification data was 94.3%, and the overfitting problem that occurred when the related research model was applied was improved. Additionally, the receiver operating characteristic (ROC) curve of the model [17] was biased to the left, with an area under the curve (AUC) of 0.98. This confirmed that the model demonstrated high specificity and sensitivity excellent performance in detecting both patient improvement and risk.

Table 2 compares the accuracy indices when applying the existing research and DNN-based models. In traditional machine learning, the model accuracy and F1-score are high, but it can be confirmed that overfitting occurs based on the results obtained when the prediction accuracy is calculated using validation data. Logistic regression and ANN models showed low prediction accuracy. Therefore, the DNN-based model analyzes the relationships between various variables more accurately than the techniques used in related studies and is the most suitable model for the DBICP.

In this study, to build an early prediction model for the symptoms of critically ill patients, we first constructed a dataset by preprocessing the clinical log data of critically ill patients. Next, when predicting the risk of critically ill patients using related research techniques, overfitting and low prediction accuracy were observed, and the DBICP was proposed to solve this problem.

The DBICP model clusters similar diseases through cosine similarity and then predicts the symptoms of the patient’s condition using the proposed DNN-based deep learning model. Ultimately, the deep learning model exhibited the highest performance, with a prediction accuracy of 95%. This was able to predict the risk signs of the patient’s condition by considering a higher accuracy than the 85.67% prediction accuracy of atrial fibrillation patients in a previous study and more diverse clinical log data of critically ill patients.

This study focuses on cases where condition symptom prediction models using clinical log data of critically ill patients do not exist because of the limited patient data published to date. In addition, it improved overfitting and poor performance that occurred when applying existing research methods by considering the relationship between various clinical data of critically ill patients.

In the future, if the DBICP proposed in this study is used in wearable devices [18], whose effectiveness has been proven through many recent developments, AI technology will first review the data of critically ill patients after the device collects them, helping medical experts make quick decisions [19].

Furthermore, presenting the predicted patient condition to medical staff is expected to improve work efficiency and treatment results for diagnosis, alleviate patient anxiety and burden, and replace insufficient medical staff and wards.

The validity of the proposed algorithm has not been proven because it has not been applied to a real medical emergency system. In future research, we aim to modify our algorithm for a lightweight system suitable for wearable devices to make it practically necessary.

Fig. 1.

Flowchart of the proposed DNN-based intensive care prediction (DBICP) using DNN.


Fig. 2.

Performance and overfitting issues of related research methods.


Fig. 3.

Performance issues depending on number of layers.


Fig. 4.

Optimal parameters of DNN.


Fig. 5.

Evaluation of DNN.


Table. 1.

Table 1. Simulation dataset.

No.Column nameExplain
1SEX_CDGender
2AGRDEAge range
3INGR_NMType of medication prescribed
4ATC_CONTMedicine classification code
5TPRSC_CAPANumber of days prescribed medication
6PRSC_CLS_NMTotal prescription amount
7PRSC_CLS_NMPrescription classification code
8DBPDiastolic blood pressure
10PULPulsation
11BREBreath
12TEMPTemperature
13ICD10CDInternational classification of diseases code
14DIAG_NMDiagnosis
15WTHN30_REHOSP_YNWhether to revisit the hospital within 30 days
16RSLT_CONT (Target)Patient condition signs (Target)

Table. 2.

Table 2. Comparison of performance of the final DNN with other models.

MethodModel accuracyValidation accuracyPrecisionRecallF1-score
Random forest0.99700.68420.99750.99890.9982
Ensemble0.99730.71240.99700.99710.9970
XGBoost0.99280.72300.98410.98410.9841
Logistic regression0.72120.70480.78460.81730.8006
Neural network0.87030.88120.66500.99270.7964
DNN0.95410.94360.95570.95910.9574

The bold font indicates the best performance in each test..


  1. Branford, S (2016). Monitoring and defining early response: where to draw the line?. Best Practice & Research Clinical Haematology. 29, 284-294. https://doi.org/10.1016/j.beha.2016.10.006
    Pubmed CrossRef
  2. Zhang, T, Grenier, S, Nwachukwu, B, Wei, C, Lipton, JH, and Kamel-Reid, S (2007). Association for Molecular Pathology Hematopathology Subdivision, “Inter-laboratory comparison of chronic myeloid leukemia minimal residual disease monitoring: summary and recommendations. The Journal of Molecular Diagnostics. 9, 421-430. https://doi.org/10.2353/jmoldx.2007.060134
    Pubmed KoreaMed CrossRef
  3. Shrestha, D. (2024). Comparative analysis of machine learning algorithms for heart disease prediction using the Cleveland Heart Disease dataset. Available: https://doi.org/10.20944/preprints202407.1333.v1
    CrossRef
  4. Sherstinsky, A (2020). Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena. 404. article no 132306. https://doi.org/10.1016/j.physd.2019.132306
    CrossRef
  5. Nikolentzos, G, Wang, S, Lutzeyer, J, and Vazirgiannis, M. (2024) . Graph neural machine: a new model for learning with tabular data. Available: https://arxiv.org/abs/2402.02862
  6. Valous, NA, Popp, F, Zornig, I, Jager, D, and Charoentong, P (2024). Graph machine learning for integrated multi-omics analysis. British Journal of Cancer. 131, 205-211. https://doi.org/10.1038/s41416-024-02706-7
    Pubmed KoreaMed CrossRef
  7. Kariri, E, Louati, H, Louati, A, and Masmoudi, F (2023). Exploring the advancements and future research directions of artificial neural networks: a text mining approach. Applied Sciences. 13. article no 3186. https://doi.org/10.3390/app13053186
    CrossRef
  8. Villaizan-Vallelado, M, Salvatori, M, Carro, B, and Sanchez-Esguevillas, AJ (2024). Graph neural network contextual embedding for deep learning on tabular data. Neural Networks. 173. article no 106180. https://doi.org/10.1016/j.neunet.2024.106180
    Pubmed CrossRef
  9. Chen, T, and Guestrin, C . XGBoost: a scalable tree boosting system., Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp.785-794. https://doi.org/10.1145/2939672.2939785
    CrossRef
  10. Breiman, L (2001). Random forests. Machine Learning. 45, 5-32. https://doi.org/10.1023/A:1010933404324
    CrossRef
  11. Park, HA (2013). An introduction to logistic regression: from basic concepts to interpretation with particular attention to nursing domain. Journal of Korean Academy of Nursing. 43, 154-164. https://doi.org/10.4040/jkan.2013.43.2.154
    Pubmed CrossRef
  12. Samek, W, Binder, A, Montavon, G, Lapuschkin, S, and Muller, KR (2017). Evaluating the visualization of what a deep neural network has learned. IEEE transactions on Neural Networks and Learning Systems. 28, 2660-2673. https://doi.org/10.1109/TNNLS.2016.2599820
    CrossRef
  13. He, J, Li, L, and Xu, J (2022). ReLU deep neural networks from the hierarchical basis perspective. Computers & Mathematics with Applications. 120, 105-114. https://doi.org/10.1016/j.camwa.2022.06.006
    CrossRef
  14. Liu, M, Cai, Z, and Chen, J (2022). Adaptive two-layer ReLU neural network: I. Best least-squares approximation. Computers & Mathematics with Applications. 113, 34-44. https://doi.org/10.1016/j.camwa.2022.03.005
    CrossRef
  15. Long, Z, McCreadie, R, Camarasa, GA, and Meng, Z. (2024). LaCViT: a label-aware contrastive fine-tuning framework for vision transformers. Available: https://arxiv.org/abs/2303.18013v3
  16. Mao, A, Mohri, M, and Zhong, Y. (2023). Cross-entropy loss functions: theoretical analysis and applications. Available: https://arxiv.org/abs/2304.07288
  17. Nahm, FS (2022). Receiver operating characteristic curve: overview and practical use for clinicians. Korean Journal of Anesthesiology. 75, 25-36. https://doi.org/10.4097/kja.21209
    Pubmed KoreaMed CrossRef
  18. Su, M, Hua, J, Sun, X, Liu, Z, Shi, Y, and Pan, L (2024). Wireless wearable devices and recent applications in health monitoring and clinical diagnosis. Biomedical Materials & Devices. 2, 669-694. https://doi.org/10.1007/s44174-023-00141-5
    CrossRef
  19. Masic, I (2022). Medical decision making-an overview. Acta Informatica Medica. 30, 230-235. https://doi.org/10.5455/aim.2022.30.230-235
    Pubmed KoreaMed CrossRef

Gayoung Kim received her M.S. and Ph.D. in computer engineering from Dongguk University, in 2002 and 2005, respectively. She is currently a professor Faculty of General Education at Kangnam University. Her research interests are wireless sensor networks and deep learning.

E-mail:dolga2000@gmail.com

Article

Original Article

International Journal of Fuzzy Logic and Intelligent Systems 2024; 24(3): 287-294

Published online September 25, 2024 https://doi.org/10.5391/IJFIS.2024.24.3.287

Copyright © The Korean Institute of Intelligent Systems.

Deep Learning Model for Predicting Critical Patient Conditions

Gayoung Kim

Faculty of General Education, Kangnam University, Youngin, Korea

Correspondence to:Gayoung Kim (dolga2000@gmail.com)

Received: July 18, 2024; Revised: September 13, 2024; Accepted: September 22, 2024

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted noncommercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Disease prediction using existing deep learning methods can predict diseases with relatively limited clinical data by detecting the symptoms of specific parts and body abnormalities according to the disease. Predicting a patient’s severity requires considering multiple types of clinical data. However, existing models struggle with overfitting and reduced accuracy in making these predictions. This study proposes a system that can predict the signs of critical illness using vital data from multiple patients, such as respiratory rate and oxygen saturation, through deep learning. This enables an early response by medical staff, improving treatment efficiency, and reducing the mortality rate. To address these issues, DNN-based intensive care prediction (DBICP), was used to detect whether the patient is improving and the risk; it showed a prediction accuracy of 95%, which is approximately 10% higher than the prediction accuracy using existing methods. If the predicted patient status is presented to the medical staff through the proposed system, we expect that work efficiency and treatment results will be improved through a more accurate and faster diagnosis.

Keywords: Deep learning, Healthcare, Critical patient condition

1. Introduction

Currently, over 50% of critically ill patients in Korea are treated in general wards because of limited hospital beds, and when intensive care unit (ICU) transfer is required in a general ward, medical staff determine the priority of transfer through clinical judgment. The issue is that a delay in transfer to the ICU can lead to consequences such as an increase in the mortality rate of shock patients. In addition, there are limitations to the technology and manpower required for medical staff to regularly validation for signs of condition in all critically ill patients and take prompt action.

Moreover, the supply and demand of medical personnel are difficult because considerable effort and personal sacrifice are required to engage in essential medical care. An early prediction model for the signs of critical illness has emerged to overcome the above problems and plays an effective and decisive role in protecting lives by quickly responding to potential risks to patients. In a representative example, the model detects vital signs such as the respiratory rate and oxygen saturation of critically ill patients, and based on the predicted results, medical staff revise the treatment plan and allocate appropriate medical resources. This early response increases treatment efficiency [1] and reduces patient mortality and cardiac arrest rates [2].

Domestic medical companies have developed software such as artificial intelligence (AI) vital care to detect patient risks early, and many hospitals are introducing it; however, extensive research has been conducted to help recognize the signs of a patient’s condition early.

Among them, heart disease and hepatitis C prediction have demonstrated high performance, and various analytical methods have been presented.

The machine-learning method for disease prediction used in the past could predict diagnosis by learning the parameters necessary for predicting a single major disease. Predicting critically ill patients requires a complex algorithm that learns from the vital information of many patients. The accuracy of the prediction of critically ill patients using existing machine learning is relatively low. Therefore, we improved the accuracy through the proposed deep learning algorithm.

Existing algorithms that can predict a patient’s disease are systems that can predict specific diseases and have few parameters for learning the patient’s condition; however, critical patient prediction requires a relatively large number of parameters because multiple conditions must be considered. In addition, existing disease prediction methods can identify data patterns for specific diseases through machine learning and classify diseases; however, the critical patient system must process a large number of disease-related parameters. To predict critical patients early, which requires processing complex data and a large amount of data, the accuracy of prediction using existing machine learning is low. Therefore, the accuracy of early prediction can be improved through the proposed algorithm.

In this study, various machine learning methods such as ensemble techniques and logistic regression, which have previously been applied to predict heart disease [3], were used to predict the risk of patients developing severe conditions at an early stage. However, interpreting the relationships between variables is challenging because of the complexity of clinical log data. This complexity often leads to poor model performance and overfitting. This problem can be validated through the deviation between the training and verification accuracies of the machine learning methods in Section 4.

To address this issue, we propose a model that considers the various clinical information of patients, reduces overfitting problems, and detects risks in critically ill patients with high performance.

By presenting the prediction model to medical professionals, we aim to assist in evaluating patient conditions. This approach can also help efficiently allocate medical resources by prioritizing high-risk patients. In addition, it is significant in that it can help reduce the high tension of medical staff and relieve and treat patients’ anxiety.

2. Related Works

Methods should be described in sufficient detail to allow others to replicate and build on published results. Various patient data are being made public overseas, and research utilizing it is actively underway. The Medical Information Mart for Intensive Care (MIMIC) in the United States released ICU data and used it in various studies. Among these, the long short-term memory (LSTM) technique has demonstrated high performance in predicting death in patients with diabetes [4].

The UCI Machine Learning Repository distributes hepatitis C patient data and several studies were conducted to predict whether hepatitis C was confirmed using this data. A representative and widely cited study presented a method for predicting the onset of hepatitis C using an ensemble technique that combines multiple classifiers. The classifier is a multilayer perceptron (MLP) [5], which has demonstrated high performance in expressing and classifying the numerical information of patients, and MLP, which has shown high performance in expressing categorical variables based on the fact that hepatitis C occurs frequently in patients with jaundice. A Bayesian network was used [6]. The ensemble model built by combining the presented classifiers showed improved performance over the existing single model in predicting whether the disease was confirmed in patients [6]. Similarly, the ensemble-based random forest model built based on patients’ real-time health data showed high performance in predicting the risk of stroke.

To predict the onset of heart disease early, an artificial neural network (ANN) algorithm [7], built based on heart numerical data provided by Cleveland, USA, and the Hungarian Cardiology Group, predicted whether a patient would develop heart disease with a high accuracy of 94%.

In a domestic research case, machine learning was performed using a logistic regression analysis method with the pulse information variables of 49 healthy individuals and 49 atrial fibrillation patients obtained from a photoplethysmogram (PPG) sensor [8]. The model built through learning classified patients with atrial fibrillation and healthy subjects with 85.67% accuracy.

As mentioned previously, we applied the methods used in previous studies to the data used in this study. However, the LSTM technique was unsuitable because there was no time series data from the patient’s admission to discharge in the data to be used; therefore, it was excluded, the proposed model and experimental results were compared by applying techniques such as ANN, logistic regression, and XGBoost models.

3. Establishment of a Model to Predict Symptoms of Critical Patients

3.1 Subsection

The data used in this study were collected over 10 years at an anonymous general hospital in South Korea. Due to medical laws, patient information and specific hospitalization times were anonymized, making it impossible to identify exactly in what year the information was collected. The data consisted of 24 columns and 110,000 rows, including the patient’s vital signs, underlying diseases, antibiotic prescription information, various test results, and treatment results. Patient classifications consisted of general, emergency, and intensive care, and a risk prediction model was designed using only ICU patient data. To reduce unnecessary complexity, safety, and computational efficiency, data that made it difficult to determine the patient’s condition owing to missing prescription records and duplicate variables were removed.

Patient data was provided by the government agency, Yongin City Industrial Promotion Agency, and 8 years of the total 10 years of data were divided into training data and test data to build a deep learning model suitable for this, and the remaining 2 years of the most recent data were used for verification.

As shown in Table 1, most patients were in their 60s, 80s, or older. The proportion of male patients was slightly higher than that of female patients. To efficiently predict at-risk patients, the patient’s treatment results were classified into two scenarios: condition improvement and danger, with data classified as mild, complete recovery, death after 24 hours, death before 24 hours, and 33% of all critically ill patients were classified as condition improving and danger.

Through cosine similarity, diseases with high similarity are likely to have similar characteristics or effects. Therefore, prediction classification was performed by grouping them into the same group to derive a more accurate and useful prediction model.

3.2 Implementation of Predictive Model

First, to compare the constructed dataset with related research and the proposed model, we applied the XGBoost model [9], which showed excellent performance on categorical data, in addition to ensemble, random forest [10], and logistic regression analysis [11].

The proposed model is based on a related study in which an ANN showed high prediction results in heart disease data and is a deep neural network (DNN) [12] that identifies complex patterns by increasing the number of neural networks to process large amounts of categorical data) the technique was applied. To express the complexity of variables and prevent gradient loss, the rectified linear unit (ReLU) activation function [13,14] was used in the hidden layer. To binary classify the final output of the model, the final layer outputs a value between 0 and 1 through the sigmoid function.

In the output layer, binary cross-entropy (BCE) [15] was applied to the loss function to probabilistically evaluate how close the predicted value was to the actual value in binary classification, and the formula is shown in Eq. (1) [16].

BCE(y,y^)=-1Ni=1N[yilog(y^i)+(1-yi)log(1-y^i)].

yi refers to the actual value and has the meaning of 0 (anger) or 1 (improving). ŷi represents the probability between zero and one of the values predicted by the model. yi log(ŷi) evaluates the accuracy of the model when predicting the probability of an outcome as 1. (1 − yi) log(1 − ŷi) evaluates the accuracy of the model when predicting the probability of the outcome being 0.

To optimize the model, Adam was applied as an optimizer. Adam prevents instability in model learning by individually adjusting the learning rate for each parameter and quickly converges the optimization process by updating parameters by considering previous gradient information. A flowchart of the DNN-based intensive care prediction (DBICP) model is shown in Figure 1.

4. Implementation Analysis

In this study, the software was developed in Python 3.12 and hardware was developed in the Google Colab environment. Related studies’ XGBoost models faced overfitting and performance issues due to accuracy deviations between training and validation, as shown in Figure 2.

This issue arises because predicting risk in critically ill patients requires explaining diverse categorical relationships and learning from more data than in previous studies. To solve this problem, a DNN model was designed and implemented. Consequently, the performance gap between the training and validation data was minimized, confirming that the overfitting problem was resolved. This suggests that the DNN model can capture more complex patterns while maintaining its generalization performance. The structure of the designed DNN model is illustrated in Figures 14. To design the optimal DNN model, the number of layers was set to 1–4 and the parameters were adjusted and compared.

Figure 3 shows the experimental results showing performance problems owing to accuracy deviations in the verification data during the process of checking the accuracy by adjusting the weights according to the number of hidden layers.

After performing several experimental procedures, the parameters of the model that showed optimal performance were one input layer and two hidden layers; the optimal performance was shown when the respective weights and parameters were set, as shown in Figure 4.

Figure 5 shows the performance of the proposed model. The prediction accuracy was 95.4%, the accuracy of the verification data was 94.3%, and the overfitting problem that occurred when the related research model was applied was improved. Additionally, the receiver operating characteristic (ROC) curve of the model [17] was biased to the left, with an area under the curve (AUC) of 0.98. This confirmed that the model demonstrated high specificity and sensitivity excellent performance in detecting both patient improvement and risk.

Table 2 compares the accuracy indices when applying the existing research and DNN-based models. In traditional machine learning, the model accuracy and F1-score are high, but it can be confirmed that overfitting occurs based on the results obtained when the prediction accuracy is calculated using validation data. Logistic regression and ANN models showed low prediction accuracy. Therefore, the DNN-based model analyzes the relationships between various variables more accurately than the techniques used in related studies and is the most suitable model for the DBICP.

5. Conclusion and Future Work

In this study, to build an early prediction model for the symptoms of critically ill patients, we first constructed a dataset by preprocessing the clinical log data of critically ill patients. Next, when predicting the risk of critically ill patients using related research techniques, overfitting and low prediction accuracy were observed, and the DBICP was proposed to solve this problem.

The DBICP model clusters similar diseases through cosine similarity and then predicts the symptoms of the patient’s condition using the proposed DNN-based deep learning model. Ultimately, the deep learning model exhibited the highest performance, with a prediction accuracy of 95%. This was able to predict the risk signs of the patient’s condition by considering a higher accuracy than the 85.67% prediction accuracy of atrial fibrillation patients in a previous study and more diverse clinical log data of critically ill patients.

This study focuses on cases where condition symptom prediction models using clinical log data of critically ill patients do not exist because of the limited patient data published to date. In addition, it improved overfitting and poor performance that occurred when applying existing research methods by considering the relationship between various clinical data of critically ill patients.

In the future, if the DBICP proposed in this study is used in wearable devices [18], whose effectiveness has been proven through many recent developments, AI technology will first review the data of critically ill patients after the device collects them, helping medical experts make quick decisions [19].

Furthermore, presenting the predicted patient condition to medical staff is expected to improve work efficiency and treatment results for diagnosis, alleviate patient anxiety and burden, and replace insufficient medical staff and wards.

The validity of the proposed algorithm has not been proven because it has not been applied to a real medical emergency system. In future research, we aim to modify our algorithm for a lightweight system suitable for wearable devices to make it practically necessary.

Fig 1.

Figure 1.

Flowchart of the proposed DNN-based intensive care prediction (DBICP) using DNN.

The International Journal of Fuzzy Logic and Intelligent Systems 2024; 24: 287-294https://doi.org/10.5391/IJFIS.2024.24.3.287

Fig 2.

Figure 2.

Performance and overfitting issues of related research methods.

The International Journal of Fuzzy Logic and Intelligent Systems 2024; 24: 287-294https://doi.org/10.5391/IJFIS.2024.24.3.287

Fig 3.

Figure 3.

Performance issues depending on number of layers.

The International Journal of Fuzzy Logic and Intelligent Systems 2024; 24: 287-294https://doi.org/10.5391/IJFIS.2024.24.3.287

Fig 4.

Figure 4.

Optimal parameters of DNN.

The International Journal of Fuzzy Logic and Intelligent Systems 2024; 24: 287-294https://doi.org/10.5391/IJFIS.2024.24.3.287

Fig 5.

Figure 5.

Evaluation of DNN.

The International Journal of Fuzzy Logic and Intelligent Systems 2024; 24: 287-294https://doi.org/10.5391/IJFIS.2024.24.3.287

Table 1 . Simulation dataset.

No.Column nameExplain
1SEX_CDGender
2AGRDEAge range
3INGR_NMType of medication prescribed
4ATC_CONTMedicine classification code
5TPRSC_CAPANumber of days prescribed medication
6PRSC_CLS_NMTotal prescription amount
7PRSC_CLS_NMPrescription classification code
8DBPDiastolic blood pressure
10PULPulsation
11BREBreath
12TEMPTemperature
13ICD10CDInternational classification of diseases code
14DIAG_NMDiagnosis
15WTHN30_REHOSP_YNWhether to revisit the hospital within 30 days
16RSLT_CONT (Target)Patient condition signs (Target)

Table 2 . Comparison of performance of the final DNN with other models.

MethodModel accuracyValidation accuracyPrecisionRecallF1-score
Random forest0.99700.68420.99750.99890.9982
Ensemble0.99730.71240.99700.99710.9970
XGBoost0.99280.72300.98410.98410.9841
Logistic regression0.72120.70480.78460.81730.8006
Neural network0.87030.88120.66500.99270.7964
DNN0.95410.94360.95570.95910.9574

The bold font indicates the best performance in each test..


References

  1. Branford, S (2016). Monitoring and defining early response: where to draw the line?. Best Practice & Research Clinical Haematology. 29, 284-294. https://doi.org/10.1016/j.beha.2016.10.006
    Pubmed CrossRef
  2. Zhang, T, Grenier, S, Nwachukwu, B, Wei, C, Lipton, JH, and Kamel-Reid, S (2007). Association for Molecular Pathology Hematopathology Subdivision, “Inter-laboratory comparison of chronic myeloid leukemia minimal residual disease monitoring: summary and recommendations. The Journal of Molecular Diagnostics. 9, 421-430. https://doi.org/10.2353/jmoldx.2007.060134
    Pubmed KoreaMed CrossRef
  3. Shrestha, D. (2024). Comparative analysis of machine learning algorithms for heart disease prediction using the Cleveland Heart Disease dataset. Available: https://doi.org/10.20944/preprints202407.1333.v1
    CrossRef
  4. Sherstinsky, A (2020). Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena. 404. article no 132306. https://doi.org/10.1016/j.physd.2019.132306
    CrossRef
  5. Nikolentzos, G, Wang, S, Lutzeyer, J, and Vazirgiannis, M. (2024) . Graph neural machine: a new model for learning with tabular data. Available: https://arxiv.org/abs/2402.02862
  6. Valous, NA, Popp, F, Zornig, I, Jager, D, and Charoentong, P (2024). Graph machine learning for integrated multi-omics analysis. British Journal of Cancer. 131, 205-211. https://doi.org/10.1038/s41416-024-02706-7
    Pubmed KoreaMed CrossRef
  7. Kariri, E, Louati, H, Louati, A, and Masmoudi, F (2023). Exploring the advancements and future research directions of artificial neural networks: a text mining approach. Applied Sciences. 13. article no 3186. https://doi.org/10.3390/app13053186
    CrossRef
  8. Villaizan-Vallelado, M, Salvatori, M, Carro, B, and Sanchez-Esguevillas, AJ (2024). Graph neural network contextual embedding for deep learning on tabular data. Neural Networks. 173. article no 106180. https://doi.org/10.1016/j.neunet.2024.106180
    Pubmed CrossRef
  9. Chen, T, and Guestrin, C . XGBoost: a scalable tree boosting system., Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp.785-794. https://doi.org/10.1145/2939672.2939785
    CrossRef
  10. Breiman, L (2001). Random forests. Machine Learning. 45, 5-32. https://doi.org/10.1023/A:1010933404324
    CrossRef
  11. Park, HA (2013). An introduction to logistic regression: from basic concepts to interpretation with particular attention to nursing domain. Journal of Korean Academy of Nursing. 43, 154-164. https://doi.org/10.4040/jkan.2013.43.2.154
    Pubmed CrossRef
  12. Samek, W, Binder, A, Montavon, G, Lapuschkin, S, and Muller, KR (2017). Evaluating the visualization of what a deep neural network has learned. IEEE transactions on Neural Networks and Learning Systems. 28, 2660-2673. https://doi.org/10.1109/TNNLS.2016.2599820
    CrossRef
  13. He, J, Li, L, and Xu, J (2022). ReLU deep neural networks from the hierarchical basis perspective. Computers & Mathematics with Applications. 120, 105-114. https://doi.org/10.1016/j.camwa.2022.06.006
    CrossRef
  14. Liu, M, Cai, Z, and Chen, J (2022). Adaptive two-layer ReLU neural network: I. Best least-squares approximation. Computers & Mathematics with Applications. 113, 34-44. https://doi.org/10.1016/j.camwa.2022.03.005
    CrossRef
  15. Long, Z, McCreadie, R, Camarasa, GA, and Meng, Z. (2024). LaCViT: a label-aware contrastive fine-tuning framework for vision transformers. Available: https://arxiv.org/abs/2303.18013v3
  16. Mao, A, Mohri, M, and Zhong, Y. (2023). Cross-entropy loss functions: theoretical analysis and applications. Available: https://arxiv.org/abs/2304.07288
  17. Nahm, FS (2022). Receiver operating characteristic curve: overview and practical use for clinicians. Korean Journal of Anesthesiology. 75, 25-36. https://doi.org/10.4097/kja.21209
    Pubmed KoreaMed CrossRef
  18. Su, M, Hua, J, Sun, X, Liu, Z, Shi, Y, and Pan, L (2024). Wireless wearable devices and recent applications in health monitoring and clinical diagnosis. Biomedical Materials & Devices. 2, 669-694. https://doi.org/10.1007/s44174-023-00141-5
    CrossRef
  19. Masic, I (2022). Medical decision making-an overview. Acta Informatica Medica. 30, 230-235. https://doi.org/10.5455/aim.2022.30.230-235
    Pubmed KoreaMed CrossRef

Share this article on :

Related articles in IJFIS