Acute coronary syndrome (ACS) is an unstable and progressive category within coronary heart disease (CHD), characterized by three serious and life-threatening clinical manifestations: ST-segment elevation myocardial infarction (STEMI), non-ST-segment elevation myocardial infarction (NSTEMI) and unstable angina (UA) [1, 2]. The clinical manifestations of ACS are broad, ranging from cardiac arrest and electrical or hemodynamic instability due to cardiogenic shock resulting from continuous ischemia or mechanical complications (e.g., severe mitral regurgitation) to patients without pain at the time of treatment [2]. Therefore, ACS management requires strict and scientific evaluation to identify high-risk patients [1, 2, 3].

The Global Registry of Acute Coronary Events (GRACE) is widely used as a risk assessment tool to predict predicting in-hospital mortality for patients with ACS, and has recently been updated to version 2.0 (GRACE 2.0) [1, 2, 4, 5]. However, it’s important to note that the GRACE risk scores were mainly developed in North America, South America, and Europe, with limited representation from Asian populations [4, 5]. Another notable ACS risk assessment tool is the thrombolysis in myocardial infarction (TIMI) risk score [6, 7]. This assessment model has been rigorously studied and independently shown to have a predictive effect on prognosis, as indicated by multivariate logistic regression analysis [6, 7]. The TIMI risk score is widely used in clinical practice due to its simplicity and ease of implementation. The acute coronary treatment and intervention outcomes network (ACTION) risk model has been recently developed and validated for ACS patient management [8, 9]. However, only a few studies have conducted external verification of this model [10]. Finally, a model from clinical pathways for acute coronary syndromes (CPACS) has been designed specifically for Chinese ACS patients [11]. Nonetheless, it’s worth noting that this scoring system lacks external validation, and its effectiveness beyond its original study remains unconfirmed.

The objective of this study was to assess the efficacy of five risk assessment models, GRACE, GRACE 2.0, ACTION, TIMI and CPACS, using data from a Chinese ACS cohort. Notably, this study represents the first external verification of the CPACS model since its creation and also the initial validation of ACTION for predicting in-hospital mortality in Chinese patients.

Here, we presented a comparative performance evaluation between the GRACE, GRACE 2.0, ACTION, TIMI and CPACS risk models for predicting in-hospital mortality in a Chinese cohort with ACS. This is also the first external verification of the CPACS model since its establishment and the first verification of ACTION to predict in-hospital mortality in Chinese patients. Our study found that the ACTION model demonstrated the best discrimination in all five risk scores among ACS patients and in the ACS subgroup analysis of gender. However, the calibration was not satisfactory. GRACE demonstrated proper discrimination and calibration across all classifications. While GRACE and GRACE 2.0 did not show significant differences in discrimination (p${}_{\mathrm{\text{GRACE }}\text{𝑣𝑠}\mathrm{\text{. GRACE 2.0}}}$ = 0.1480), the calibration of GRACE 2.0 was unsatisfactory. Neither CPACS (c-index 0.841, 95% CI 0.821–0.861) and TIMI (c-index 0.811, 95% CI 0.787–0.835) exhibited better performance compared to GRACE. However, GRACE displayed the most accurate calibration for all patients and in the subgroup analysis of all models. Finally, CPACS performed well except in STMEI patients.

The GRACE study is currently the world’s first prospective prediction study of all types of unscreened ACS patients conducted in multiple countries [4, 12]. Extensive external verifications have confirmed the value of GRACE in assessing early and delayed invasive management strategies in ACS, and it has been recommended by the ESC and AHA guidelines [1, 2, 10, 13, 14, 15, 16, 17]. The updated version of the GRACE 2.0 model, released in 2016, introduced nonlinear association and improved bedside ease of use through mobile phone software [5]. In our study, both GRACE and GRACE 2.0 performed excellently in model discrimination. They effectively distinguished the related risks of in-hospital death of ACS patients, and no significant difference was found between them. However, the calibration of GRACE 2.0 was poor, and it significantly underestimates the risk of death for high-risk patients. Among the five risk assessment models included in this study, only the GRACE score showed a good degree of calibration in the STEMI subgroup analysis.

The TIMI risk score is a clinical risk score for the prognosis of patients with ACS. The variables in this score are derived from an independent predictive effect on the prognosis, identified through multivariate logistic regression analysis in the TIMI trial population [6, 7]. Different scoring systems are available based on the ACS disease spectrum, including separate scores for STMEI and UA/NSTEMI patients. These scores primarily rely on data from electrocardiograms and clinical characteristics, making them simple and easily obtainable, making them suitable for use in emergency departments. The TIMI score has been validated to effectively stratify high-risk patients with chest pain and predict the incidence of short-term and long-term adverse cardiovascular events [18, 19]. However, the performance of the TIMI score was not satisfactory in our data. Despite proper calibration, the discrimination of the model is inferior to other models. This discrepancy may arise because the scoring system does not include certain factors that are relevant to poor prognosis in myocardial injury. The accuracy of the model is reduced by not considering cardiac biomarkers and ST-segment resolution for STEMI patients, and blood pressure and heart rate for UA/NSTEMI patients.

ACTION Registry- Get With The Guidelines is a voluntary, hospital-based registry system that receives data from consecutive acute myocardial infarction (AMI) patients from participating hospitals across the United States, including STEMI or NSTEMI. The ACTION risk model for in-hospital mortality, based on the registry, was developed from 65,668 patients with AMI and was updated in 2016 to include patients from 2012 to 2013 [8, 9]. The model has been externally validated in a Spanish and a German cohort [10, 20]. In the Spanish cohort, both ACTION and GRACE showed proper distinctions between in-hospital deaths (C statistics were 0.90 and 0.90, respectively) and had good calibration (Hosmer-Lemeshow goodness of fit test values 0.50 and 0.47, respectively) [20]. In the German cohort, Parco et al. [10] evaluated the predictive efficacy of the ACTION model, including 1567 (non) ST-segment elevation myocardial infarction patients who received invasive treatment at Düsseldorf University Hospital in Germany from 2014 to 2018. The results showed that the performance of the ACTION and GRACE risk models are comparable (c-index 0.84, p${}_{\mathrm{\text{GRACE }}\text{𝑣𝑠}\mathrm{\text{. ACTION}}}$ = 0.68), with an advantage in for the ACTION model in NSTEMI patients (c-index 0.87 [ACTION] vs. 0.84 [GRACE]; p${}_{\mathrm{\text{GRACE }}\text{𝑣𝑠}\mathrm{\text{. ACTION}}}$ = 0.02) [10]. A key distinguishing feature of ACTION from TIMI and GRACE scores is its ability to better differentiate high-risk patients when included after cardiac arrest, in cardiogenic shock, and in HF. In this Chinese cohort, the ACTION model not only showed the best discrimination among ACS patients with a c-index of 0.945, but also performed best in the subgroup analysis of ACS category or gender. However, the ACTION model is not properly calibrated for all ACS patients (p = 0.013), STEMI patients (p = 0.009), and female patients (p = 0.016), this is mainly due to underestimating the risk of death in high-risk groups. Another noteworthy point is that although the target population of the ACTION score is AMI patients, in the subgroup analysis of UA patients, the action score still showed good differentiation and calibration, which may be caused by the lower mortality of the UA subgroup.

The CPACS program is a quality improvement program conducted by the Chinese Heart Association, focusing on the management of inpatients suspected of ACS patients [21]. The subsequent Acute Coronary Syndrome Clinical Pathway-Phase 2 (CPACS-2) evaluated the effectiveness of interventions based on the clinical path for managing ACS patients in 75 hospitals in China [22]. The CPACS risk scoring model, designed to assess in-hospital mortality risk, was initially developed for different sexes among 6790 patients who were hospitalized for suspected acute coronary syndrome [11]. It was later compared to the GRACE risk score in predicting in-hospital mortality risk among 3801 patients [11]. Prior to this study, there was no external verification of the CPACS model.

Although CPACS is an in-hospital mortality risk assessment model for ACS patients established using a Chinese population, it did not perform better than the GRACE risk score in our cohort, either for all ACS patients (c-index 0.841 vs. 0.926; p${}_{\mathrm{\text{GRACE }}\text{𝑣𝑠}\mathrm{\text{. CPACS}}}$ 0.001) or for sex subgroups (c-index for male 0.841 vs. 0.926; p${}_{\mathrm{\text{GRACE }}\text{𝑣𝑠}\mathrm{\text{. CPACS}}}$ 0.001; c-index for female 0.837 vs. 0.925; p${}_{\mathrm{\text{GRACE }}\text{𝑣𝑠}\mathrm{\text{. CPACS}}}$ 0.0001). This may be due to the relatively small number of patients included in the construction of the CPACS model. Additionally, some medical history information, such as diabetes and prehospital medication history, may not have been obtained in time in critically ill patients. However, CPACS (p = 0.113) had better calibration in female patients than GRACE (p = 0.005).

In recent years, an increasing number of risk scores have been used to predict the in-hospital and long-term risk of ACS patients. Among these factors, The History, Electrocardiogram, Age, Risk factor, and Troponin (HEART) Score is frequently utilized in emergency departments or chest pain centers, where clinical data are limited, and urgent diagnosis and further management are needed [23]. Other scores, such as the Vancouver Chest Pain Rules (VCPR), the North American Chest Pain Rule (NACPR), the Emergency Department Assessment of Chest Pain Score (EDACS) tools, the Manchester Acute Coronary Syndromes (MACS), and the Troponin-only Manchester Acute Coronary Syndromes (T-MACS) decision aids, can be chosen based on local standards of care and provider risk tolerance [24]. Notably, in patients with AMI, the Killip classification performed at admission is a simple and useful clinical marker of high risk for early and late adverse cardiovascular events [25]. TIMI risk scores also tend to facilitate quick early ratings. As a widely used ACS risk assessment model, the GRACE score has demonstrated good predictive performance in hospital events and 6-month follow-up. However, with advancements in test items and treatment methods, further investigation is needed to assess predictive sensitivity and accuracy. Given the changes in interventions and decision points in recent years, evaluating modern data would be particularly beneficial. The ACTION score may be most useful for severely ill patients and provide guidance for new interventions [26]. Considering that female patients often have more cardiac risk factors than male patients, even though they are low-risk populations in most cases [27], attention should be given to risk models that consider gender divisions, such as CPACS. In summary, at present many risk models have applicable scenarios and limitations, but the continuity among different risk models is not strong. Therefore, a dynamic risk assessment based on time and gender seems to be more suitable for patients during treatment, and this may be a direction of the development for future risk assessment models.

Some factors have not yet been added to current ACS risk assessment models, but significantly impact patient prognosis. Chronic kidney diseases patients have a higher risk of ischemia and AMI and a worse prognosis, especially in dialysis patients. Both cardiac and renal insufficiency can aggravate the prognosis and lead to disease progression [28]. Type 2 diabetes is another critical factor, increasing the risk of major adverse cardiovascular events. Notably, the bleeding tendency associated with diabetes appears to be limited to patients receiving insulin therapy [29]. A new algorithm called SCORE2-Diabetes has been developed to predict 10-year cardiovascular disease (CVD) risk in patients with type 2 diabetes, helping to identify individuals at high CVD risk. The algorithm utilizes sex-specific competing risk-adjusted models, incorporating traditional risk factors (e.g., age, smoking, systolic blood pressure, total cholesterol, and high-density lipoprotein cholesterol) and diabetes-related variables (e.g., age at diabetes diagnosis, glycated hemoglobin, and creatinine-based estimated glomerular filtration rate) [30]. Patients with both standard modifiable cardiovascular risk factors and prior CVD have a higher mortality regardless of their sex [31]. Therefore, including coexisting conditions in long-term risk assessment models may be important. Additionally, myocardial infarction without obstructive coronary artery disease is a significant consideration, as it accounts for a considerable proportion of AMI cases and has received extensive attention in recent years [32]. However, the current scoring system lacks a corresponding assessment for this condition.

The development of cardiac imaging has benefited from the advances in Computed Tomography technology. The emergence of coronary computed tomography angiography (CCTA) has revolutionized the non-invasive evaluation and rapid risk stratification of coronary heart disease. Intact fibrous cap ACS patients exhibit a unique inflammatory response and a lower MACE risk compared to rupture of the fibrous cap ACS patients [33]. By conducting a comprehensive evaluation based on CCTA, considering multiple aspects such as stenosis degree, plaque characteristics, and functional reserve, clinicians can obtain vital information for accurate prognosis stratification of patients [34]. Leveraging radiomics and machine learning techniques, CCTA images can be objectively and mathematically evaluated, providing enhanced precision analysis. In the era of big data analysis and artificial intelligence, CCTA is poised to perform multi-dimensional risk stratification of patients with coronary heart disease [34].

This research does has some limitations, including the following: (1) There are still missing data points, such as 2925 missing values for weight. Although the random forest algorithm was used to supplement the missing values, the potential impact on the results is still unknown. (2) The c-index of the risk models in this study is high, with the highest being 0.945. This may have been influenced by the lower incidence of death events in the cohort itself and the large number of low-risk individuals. Higher negative sample populations can improve the model’s performance by affecting the negative predictive value. (3) The lower mortality observed in the ACS patients included in this study may attributed to the higher diagnostic rate of UA in China. The overestimated proportion of UA in China may be due to physicians’ tendency to make a diagnosis based on the clinical manifestations during the first visit without considering dynamic changes in high-sensitivity troponin and ECG. This proportion is similar to that found in a previously published Chinese multicenter ACS cohort study (CPACS registry, UA $>$40%) [11, 21].

In contemporary Chinese ACS patients with, the ACTION risk model outperforms the gold standard GRACE model in predicting hospital mortality, despite its unsatisfactory calibration. The CPACS model developed for Chinese patients did not exhibit better predictive performance than the GRACE model. Nevertheless, the GRACE model continues to demonstrate a strong performance across all aspects and remains a reliable tool for ACS risk prediction for the foreseeable future.

ACS, acute coronary syndrome; ACTION, acute coronary treatment and intervention outcomes network; AUC, Area under the curve; CHD, coronary heart disease; CI, confidence interval; CPACS, clinical pathways for acute coronary syndromes; GRACE, Global Registry of Acute Coronary Events; NSTMEI, non-ST-segment elevation myocardial infarction; PCI, percutaneous coronary intervention; ROC, receiver operating characteristic; STEMI, ST-segment elevation myocardial infarction; TIMI, thrombolysis in myocardial infarction; UA, unstable angina.

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

YP, LB and YML designed the research study. LB performed the research and completed the manuscript. BSY, YHC, YKZ, GZL, YYY and XFC provided help on the Collection of Data. LB, YML and HC analyzed the data. HC and YP reviewed the article and made suggestions. All authors contributed to editorial changes in the manuscript. All authors read and approved the final manuscript. All authors have participated sufficiently in the work and agreed to be accountable for all aspects of the work.

The trial was approved by the Ethics Committee of West China Hospital of Sichuan University, approval number: 2021(19). Due to the retrospective nature of the study, informed consent was waived.

We gratefully acknowledge the assistance and instruction from professor Anushka Patel of the George Institute for Global Health and the CPACS Investigators.

This study was supported by Sichuan Science and Technology Program (Grant numbers: 2021YFS0330, 2023NSFSC1638, Sichuan, China), Sichuan Provincial Cadre Health Research Project, (Sichuan Ganyan ZH2021-101, Sichuan, China), 1$\cdot$3$\cdot$5 project for disciplines of excellence–Clinical Research Incubation Project, West China Hospital, Sichuan University (Grant number: 2021HXFH061, Sichuan, China), and Scientific Research Project of Sichuan Provincial Health Commission (Grant Numbers:16ZD007, Sichuan, China).

The authors declare no conflict of interest.