This systematic review was conducted following the requirements of the preferred reporting items for systematic reviews and meta-analyses (PRISMA2020) (Supplementary Table 1) [14], and registered via PROSPERO (ID: CRD42023401497).

PubMed, Embase, Web of Science, and Cochrane databases were searched from 1980 to December 31, 2022, by combining the subject terms and subheadings of “atrial fibrillation”, “recurrence” and “machine learning”. The complete search strategy is shown in Supplementary Table 2.

All retrieved literature was imported into Endnote. After removing duplications, titles and abstracts were reviewed to exclude irrelevant studies. Subsequently, the full texts of the studies selected in the initial screening were downloaded and read to select eligible original studies. A data extraction table was prepared in advance to record the following data: study types (e.g., cohort studies, cross-sectional studies), study characteristics (e.g., author, year, title, and author’s country), study groups (e.g., total sample size, number of relapsed cases, total number of cases in the training set, number of recurrent cases in the training set, number of recurrent cases in the validation set, and the total number of cases in the validation set), ablation type, follow-up time, definition of blank period, definition of AF recurrence, method of generating the validation set, overfitting method, missing value treatment method, variable screening method, model type, and modeling variables.

The literature screening and data extraction were independently conducted by two investigators (XF and XL), with a cross-check conducted following completion. In the event of any disagreements or uncertainties regarding the eligibility of a particular study, another reviewer (YL) was consulted for resolution.

The prediction model risk of bias assessment tool (PROBAST) [15] was used to assess the risk of bias in the original studies included. This tool included a total of 20 questions organized across four domains (participators, predictors, outcomes, and statistical analysis). Each question can be answered as Yes/Probably Yes, No/Probably No, or No Information. If a domain included at least one question answered with No or Probably No, it was considered to have a high bias risk. A domain was considered low risk if the answers to all questions were Yes or Probably Yes. The overall bias risk was considered low if all domains were classified as low risk. Conversely, if at least one domain is considered high risk, the overall risk of bias is regarded as high.

To ensure accuracy, two investigators (XF and XL) independently conducted the risk of bias assessment based on PROBAST and cross-checked their results. In case of any disagreements, a third investigator (YL) would be asked for assistance in reaching a judgment.

The C-index was utilized as the outcome measure to reflect the overall accuracy of the model. However, in case of severe imbalance between relapsed and non-relapsed cases, the C-index may not reflect the true prediction accuracy of models for the recurrence risk. Therefore, our main outcome measures also included sensitivity and specificity, and the secondary outcome measure was the frequency of occurrence of pooled modeling variables.

If C-index lacked a 95% confidence interval (CI) and standard error in the original study, the standard error was estimated through the by Debray et al. [16] calculation method. Given the differences in the variables included in each ML model and the inconsistency in the parameters, we utilized a random-effects model for the meta-analysis of the C-index.

In addition, a bivariate mixed-effects model was employed to assess the sensitivity and specificity of the meta-analysis. Functioning as a random effects model, it accounts for the correlation between sensitivity and specificity. During the meta-analysis process, sensitivity and specificity were analyzed based on the diagnostic fourfold table, which unfortunately were not reported in most of the original studies. To address this, we utilized the following two methods to calculate the diagnostic fourfold table: (1) Calculate the fourfold table using sensitivity, specificity, and precision in combination with the number of cases; (2) Extract the sensitivity and specificity according to the best Youden’s index, and then calculate the fourfold table using the number of cases. The meta-analysis of the study was conducted using R4.2.0 (R development Core Team, Vienna, Austria, http://www.R-project.org).

In total, 770 articles were identified from multiple databases. Out of these, 220 articles were duplicates and removed. After reviewing the titles and abstracts of the remaining 550 articles, 48 were selected for full-text assessment and downloaded.

Among them, one article was unavailable in full text, 6 articles were excluded for other reasons, and one article was deleted due to duplication of an identical cohort. Finally, 40 studies were included in this systematic review and meta-analysis [12, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55]. Fig. 1 displays the PRISMA flow chart outlining the study selection process.

Fig. 1.

PRISMA (preferred reporting items for systematic reviews and meta-analyses) flow diagram for study selection.

This meta-analysis included 40 studies with a total of 16,251 AF patients receiving ablation treatment. From this total, 4930 (30.3%) patients experienced a recurrence of AF. The primary method used to record AF recurrence was body surface electrocardiogram (92.5%). Additionally, insertable loop recorders were used in 8 studies [12, 18, 26, 27, 29, 36, 49, 53], intracardiac electrogram was used in one study [52], and smart wearable devices were used in one study [27] (see attachment materials— Supplementary Table 3 in detail). The 40 articles were published between 2015 to 2022, with 19 articles (45.2%) published in 2022 (see Supplementary Fig. 1). Among these, 31 were retrospective cohort studies. The majority of catheter ablation procedures were performed using radiofrequency ablation or cryoablation, and the average follow-up time ranged from 4 months to 120 months. Patients from the United States were represented in 6 studies [12, 17, 18, 19, 20, 21], Europe in 11 studies [22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32], and the Asia-Pacific region in 23 studies [33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55]. Regarding the ML algorithms, logistic regression was the most commonly used method for predicting AF recurrence after catheter ablation, accounting for 24 out of 40 studies (60%). The remaining studies utilized other ML methods, including K-nearest neighbor (KNN), logistic regression (LR), Cox proportional hazard model (COX), Cox proportional-hazards deep neural networks (DeepSurv), Adaptive boosting (Adaboost), support vector machine (SVM), convolutional neural networks (CNN), artificial neural network (ANN), extreme gradient boosting (XGBoost), random forest (RF), decision tree (DT), linear discriminant analysis (LDA). The characteristics of the included studies are detailed in Supplementary Table 3.

This study involved 93 predictors, with the top 5 are being age, radiomics features, left atrial diameter, type of AF, and AF duration. The remaining predictors include body mass index (BMI), sex, left ventricular ejection fraction (LVEF), hypertension, diabetes, and estimated glomerular filtration rate (eGFR) (see attachment materials—Supplementary Table 3 for modeling variables in detail).

The risk of bias and the overall applicability was assessed using the PROBAST checklist, which is provided in Supplementary Table 1. Details of the risk of bias and applicability for each model included in the study can be found in online Supplementary Table 4, and a summary of the bias risk is presented in Fig. 2.

Fig. 2.

Risk of Bias Assessment Result Included in the Machine Learning Model.

Out of the 54 models identified of the 40 eligible studies, two models (3.7%) had high and moderate risks of bias in terms of participants and predictors, possibly because their study type, namely case-control design, makes it impossible to determine whether the source of participants is appropriate or whether the predictors were evaluated without knowing outcome data. The risk of bias in outcome was moderate in 42 models (77.8%). Regarding the statistical analysis, the underfitting process resulting from insufficient sample size or failure to overfit the prediction model led to a high risk of bias in 43 models.

This meta-analysis aimed to assess the performance of ML models in predicting AF recurrence following ablation. The pooled C-index results of 54 models demonstrated the high accuracy of ML in predicting and recognizing AF recurrence. As a digital-driven method, ML allows continuous learning from data to refine the model using various statistical probability and optimization techniques. This feature presents significant opportunities for developing risk prediction models in cardiovascular research similar to the well-known Framingham Heart Study [56]. By developing risk models using ML, it becomes possible to classifying ablation-treated AF patients into different risk groups, which in turn, allows the formulation of personalized follow-up protocols based on the specific timing and populations. This approach can minimize overtreatment in low-risk populations and strike a better balance between the risk-benefit and cost-benefit in the screening of AF recurrence. Overall, ML holds promising potential in advancing the field of cardiovascular risk prediction and improving patient care.

We tested many methods of subgroup analysis to predict AF recurrence in patients after catheter ablation treatment. The traditional methods included logistic regression and Cox regression. Additionally, we explored the application of support vector machines, ensemble learning, artificial neural networks, deep learning, and other ML methods. Deep learning proved to be advantageous in image recognition and data processing, as it can convert low-level characteristic data into more abstract high-level characteristic data through layer-by-layer conversion. Based on the subgroup analysis results, the model constructed using the CNN algorithm by Yi-Ting Hwang et al. [52] demonstrated the highest C-index, specificity, and sensitivity. However, due to the limited number of models, it is essential to increase the sample size and conduct external validation to gather more robust risk assessment evidence. After considering the models constructed based on clinical characteristics and radiomics features, logistic regression emerged as the most commonly used method for predicting AF recurrence in patients after ablation treatment. It had the second-highest testing power compared to the CNN model in the training set and displayed the best specificity and sensitivity in the validation set. Given these advantages, logistic regression is expected to be effectively applied in developing nomograms based on clinical characteristics for predicting AF recurrence after ablation treatment.

The selection of variables in prediction models plays a critical role in their performance. Among the 54 models, 29 models included the age of AF patients receiving ablation treatment as a modeling variable. Age has been identified as the most likely risk factor for AF, more so than with sex, BMI, hypertension, and cardiac failure [57]. However, for AF patients receiving ablation treatment at different ages, there were no statistical differences in the AF recurrence rate [58, 59]. Another important modeling variable is radiomics features, formally proposed by Lambin in 2012 [60]. These high-dimensional features that not visible to the naked eye in medical digital images such as ultrasound, computed tomography (CT) and magnetic resonance imaging (MRI). However, they can be analyzed using high throughput programs. By transforming the image data of the region of interest (ROI) into high-resolution, exploitable spatial data using full-automatic or semi-automatic analysis methods, the accuracy of disease prediction, diagnosis, and prognosis estimation can be improved.

The subgroup analysis results showed no significant differences in the pooled C-index between the models constructed based on clinical characteristics and those based on radiomics features in either the training set or the validation set. This lack of difference may be due to data overfitting caused by excessive data extraction and decreased prediction performance resulting from inaccurate image segmentation [17, 30]. Nonetheless, prediction models constructed based on radiomics features exhibited higher sensitivity, which is clinically significant for predicting AF recurrence after ablation.

While several studies have highlighted the significance of genetic variation in AF within the context of genomics [61, 62], none of the studies included in this review used alleles related to AF recurrence after ablation as predictors for model development.

Moreover, most of the predictors in these models were came from the baseline data of AF patients before admission, such as BMI, eGFR and left atrial diameter. However, it’s important to note that these short-term risk factors are subject to change, and AF recurrence may be influenced by healthy habits after discharge. Unfortunately, these factors are rarely considered in the analysis of prediction models. A recent single-center, randomized controlled trial of symptomatic AF in obesity [63] demonstrated that weight control and enhanced management of risk factors in AF patients after discharge improved the long-term success rate of AF ablation.

As cross-disciplinary research in AI-medicine progresses, there is a growing focus on developing and validating prediction models based on ML algorithms for cardiovascular diseases [64, 65]. In this systematic review and meta-analysis, we combined the training set and the validation set (including both randomly acquired internal sampling results and a small number of external validation results) to assess the performance of ML in predicting AF recurrence in patients after ablation. The C-index results demonstrated high accuracy in both the training set (0.760 [0.739–0.781]) and the validation set (0.787 [0.752–0.821]) with similar prediction performance, and without overfitting. Among the top 5 risk predictors for AF recurrence after ablation, age, type of AF, and duration of AF are relatively easy to obtain, show small population differences, and high reproducibility, making them suitable for clinical use and popularization to a certain degree.

This systematic review represents the first attempt to assess the predictive accuracy of ML for AF recurrence after ablation, providing evidence for the promising prediction capacity of ML models in these patients. However, our study does have some limitations.

First, the ML models included in the review suffered from high bias due to the rigid assessment using PROBAST for bias risk. In terms of statistical methods, a model is considered low bias only if the events per variable (EPV) is larger than 20 and it has an independent validation set with more than 100 cases. However, this rule ignores certain rare diseases or particular research fields (radiomics). Therefore, we focused on prediction factors and results for studies with high bias.

Second, an essential aspect of ML is selecting effective modeling variables. To minimize the discrepancy in modeling variables, we conducted subgroup analysis based on clinical characteristics and radiomics features, which reduced the number of models in the analysis process.

Third, radiomics lacks a standardized operating procedure, resulting in multiple approaches for dividing new areas, extracting texture features, screening modeling features, and constructing models. Despite this variability, it is important to acknowledge and recognize its clinical application value.

Finally, it is worth noting that some models in the included studies lacked valid independent validation sets [25, 38, 39, 40]. Overcoming this limitation in systematic reviews of ML can be challenging. To address this issue, we combined the results of both the training set and the validation set to assess the value of ML by comparing their accuracy levels.