{
    "abstract": "COVID-19 remains a complex disease in terms of its trajectory and the diversity of outcomes rendering disease management and clinical resource allocation challenging. Varying symptomatology in older patients as well as limitation of clinical scoring systems have created the need for more objective and consistent methods to aid clinical decision making. In this regard, machine learning methods have been shown to enhance prognostication, while improving consistency. However, current machine learning approaches have been limited by lack of generalisation to diverse patient populations, between patients admitted at different waves and small sample sizes.\nWe sought to investigate whether machine learning models, derived on routinely collected clinical data, can generalise well i) between European countries, ii) between European patients admitted at different COVID-19 waves, and iii) between geographically diverse patients, namely whether a model derived on the European patient cohort can be used to predict outcomes of patients admitted to Asian, African and American ICUs.\nWe compare Logistic Regression, Feed Forward Neural Network and XGBoost algorithms to analyse data from 3,933 older patients with a confirmed COVID-19 diagnosis in predicting three outcomes, namely: ICU mortality, 30-day mortality and patients at low risk of deterioration. The patients were admitted to ICUs located in 37 countries, between January 11, 2020, and April 27, 2021.\nThe XGBoost model derived on the European cohort and externally validated in cohorts of Asian, African, and American patients, achieved AUC of 0.89 (95% CI 0.89-0.89) in predicting ICU mortality, AUC of 0.86 (95% CI 0.86-0.86) for 30-day mortality prediction and AUC of 0.86 (95% CI 0.86-0.86) in predicting low-risk patients. Similar AUC performance was achieved also when predicting outcomes between European countries and between pandemic waves, while the models showed high calibration quality. Furthermore, saliency analysis showed that FiO2 values of up to 40% do not appear to increase the predicted risk of ICU and 30-day mortality, while PaO2 values of 75 mmHg or lower are associated with a sharp increase in the predicted risk of ICU and 30-day mortality. Lastly, increase in SOFA scores also increase the predicted risk, but only up to a value of 8. Beyond these scores the predicted risk remains consistently high.\nThe models captured both the dynamic course of the disease as well as similarities and differences between the diverse patient cohorts, enabling prediction of disease severity, identification of low-risk patients and potentially supporting effective planning of essential clinical resources.\nNCT04321265.",
    "authors": [
        {
            "affiliation": "Digital Health Centre, Fondazione Bruno Kessler Research Institute, Trento, Italy.",
            "firstname": "Behrooz",
            "initials": "B",
            "lastname": "Mamandipoor"
        },
        {
            "affiliation": "Heinrich-Heine-University Duesseldorf, Medical Faculty, Department of Cardiology, Pulmonology and Vascular Medicine, Duesseldorf, Germany.",
            "firstname": "Raphael Romano",
            "initials": "RR",
            "lastname": "Bruno"
        },
        {
            "affiliation": "Department of Internal Medicine, General Hospital Oberndorf, Teaching Hospital of the Paracelsus Medical University Salzburg, 5020 Salzburg, Austria.\nInstitute of General Practice, Family Medicine and Preventive Medicine, Paracelsus Medical University, Salzburg, Austria.",
            "firstname": "Bernhard",
            "initials": "B",
            "lastname": "Wernly"
        },
        {
            "affiliation": "Heinrich-Heine-University Duesseldorf, Medical Faculty, Department of Cardiology, Pulmonology and Vascular Medicine, Duesseldorf, Germany.",
            "firstname": "Georg",
            "initials": "G",
            "lastname": "Wolff"
        },
        {
            "affiliation": "Department of Anaesthesia and Intensive Care, Viborg Regional Hospital, Viborg, Denmark.",
            "firstname": "Jesper",
            "initials": "J",
            "lastname": "Fj\u00f8lner"
        },
        {
            "affiliation": "Department of Intensive Care Medicine, CIBER Enfermedades Respiratorias, Corporacion Sanitaria Universitaria Parc Tauli, Autonomous University of Barcelona, Sabadell, Spain.",
            "firstname": "Antonio",
            "initials": "A",
            "lastname": "Artigas"
        },
        {
            "affiliation": "Department of Acute Medicine, Geneva University Hospitals, Geneva, Switzerland.",
            "firstname": "Bernardo Bollen",
            "initials": "BB",
            "lastname": "Pinto"
        },
        {
            "affiliation": "Department of Intensive Care Medicine, Inselspital, Universit\u00e4tsspital, University of Bern, Bern, Switzerland.",
            "firstname": "Joerg C",
            "initials": "JC",
            "lastname": "Schefold"
        },
        {
            "affiliation": "Heinrich-Heine-University Duesseldorf, Medical Faculty, Department of Cardiology, Pulmonology and Vascular Medicine, Duesseldorf, Germany.",
            "firstname": "Malte",
            "initials": "M",
            "lastname": "Kelm"
        },
        {
            "affiliation": "Dept. of Medical Intensive Care, Hadassah Medical Center and Faculty of Medicine, Hebrew University of Jerusalem, Israel.",
            "firstname": "Michael",
            "initials": "M",
            "lastname": "Beil"
        },
        {
            "affiliation": "Dept. of Medical Intensive Care, Hadassah Medical Center and Faculty of Medicine, Hebrew University of Jerusalem, Israel.",
            "firstname": "Sviri",
            "initials": "S",
            "lastname": "Sigal"
        },
        {
            "affiliation": "General Intensive care, St George's University Hospitals NHS Foundation trust, London, United Kingdom.",
            "firstname": "Susannah",
            "initials": "S",
            "lastname": "Leaver"
        },
        {
            "affiliation": "Department of Intensive Care Medicine, University Medical Center, University Utrecht, the Netherlands.",
            "firstname": "Dylan W",
            "initials": "DW",
            "lastname": "De Lange"
        },
        {
            "affiliation": "Sorbonne Universit\u00e9s, UPMC Univ Paris 06, INSERM, UMR_S 1136, Institut Pierre Louis d'Epid\u00e9miologie et de Sant\u00e9 Publique, Equipe: \u00e9pid\u00e9miologie hospitali\u00e8re qualit\u00e9 et organisation des soins, F-75012, Paris, France.\nAssistance Publique-H\u00f4pitaux de Paris, H\u00f4pital Saint-Antoine, service de r\u00e9animation m\u00e9dicale, Paris, France.",
            "firstname": "Bertrand",
            "initials": "B",
            "lastname": "Guidet"
        },
        {
            "affiliation": "Department of Clinical Medicine, University of Bergen, Department of Anaesthesia and Intensive Care, Haukeland University Hospital, Bergen, Norway.",
            "firstname": "Hans",
            "initials": "H",
            "lastname": "Flaatten"
        },
        {
            "affiliation": "Jagiellonian University Medical College, Center for Intensive Care and Perioperative Medicine, Krakow, Poland.",
            "firstname": "Wojciech",
            "initials": "W",
            "lastname": "Szczeklik"
        },
        {
            "affiliation": "Heinrich-Heine-University Duesseldorf, Medical Faculty, Department of Cardiology, Pulmonology and Vascular Medicine, Duesseldorf, Germany.",
            "firstname": "Christian",
            "initials": "C",
            "lastname": "Jung"
        },
        {
            "affiliation": "Digital Health Centre, Fondazione Bruno Kessler Research Institute, Trento, Italy.",
            "firstname": "Venet",
            "initials": "V",
            "lastname": "Osmani"
        }
    ],
    "conclusions": "The models captured both the dynamic course of the disease as well as similarities and differences between the diverse patient cohorts, enabling prediction of disease severity, identification of low-risk patients and potentially supporting effective planning of essential clinical resources.",
    "copyrights": "Copyright: \u00a9 2022 Mamandipoor et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.",
    "doi": "10.1371/journal.pdig.0000136\n10.3390/jcm9072106\n10.1056/NEJMoa2002032\n10.1016/S0140-6736(20)31189-2\n10.1016/S0140-6736(20)30566-3\n10.1371/journal.pone.0246318\n10.1001/jama.2021.1545\n10.1136/bmj.m1328\n10.1007/s11606-021-06790-w\n10.1136/medethics-2020-106794\n10.1007/s11739-020-02475-0\n10.1038/s41467-020-18684-2\n10.1016/S2589-7500(21)00080-7\n10.1016/j.annemergmed.2020.07.022\n10.1016/j.patter.2020.100092\n10.3390/BIOMED2010002\n10.1038/s42256-021-00307-0\n10.1145/2939672.2939785\n10.1002/wics.175\n10.1016/J.INFFUS.2021.11.011\n10.1186/s12864-019-6413-7\n10.1186/s12911-019-1014-6\n10.1109/TPAMI.2022.3145392\n10.5555/3295222\n10.2196/32949\n10.1038/s41746-020-00343-x\n10.1016/j.chest.2020.12.009\n10.1109/MC.2021.3074263\n10.1186/s40635-019-0286-6\n10.1186/s13613-021-00911-8\n10.1186/s13054-021-03551-3\n10.3233/CH-219202\n10.1093/ehjcvp/pvaa083\n10.1183/13993003.00979-2021",
    "journal": "PLOS digital health",
    "keywords": [],
    "methods": null,
    "publication_date": "2023-02-23",
    "pubmed_id": "36812571\n32635468\n32109013\n32442528\n32171076\n33571301\n33595630\n32265220\n34003427\n33514637\n32812204\n33024092\n34049834\n33012378\n32838344\n31898477\n31906931\n35077357\n35099394\n33083565\n33345948\n31823128\n34417919\n33874987\n34487039\n32645153\n34172464",
    "results": "The XGBoost model derived on the European cohort and externally validated in cohorts of Asian, African, and American patients, achieved AUC of 0.89 (95% CI 0.89-0.89) in predicting ICU mortality, AUC of 0.86 (95% CI 0.86-0.86) for 30-day mortality prediction and AUC of 0.86 (95% CI 0.86-0.86) in predicting low-risk patients. Similar AUC performance was achieved also when predicting outcomes between European countries and between pandemic waves, while the models showed high calibration quality. Furthermore, saliency analysis showed that FiO2 values of up to 40% do not appear to increase the predicted risk of ICU and 30-day mortality, while PaO2 values of 75 mmHg or lower are associated with a sharp increase in the predicted risk of ICU and 30-day mortality. Lastly, increase in SOFA scores also increase the predicted risk, but only up to a value of 8. Beyond these scores the predicted risk remains consistently high.",
    "title": "COVID-19 machine learning model predicts outcomes in older patients from various European countries, between pandemic waves, and in a cohort of Asian, African, and American patients.",
    "xml": "<Element 'PubmedArticle' at 0x788506454090>"
}