The ESM had a hospitalization-level operating characteristic curve, or AUC, of 0.63 -- "substantially worse," than that reported by Epic, they said. [...] It did not identify two-thirds of patients with sepsis -- despite generating alerts on 18% of all hospitalized patients, creating a large burden of alert fatigue. [...] In its statement, Epic argued that the purpose of the model is to identify harder-to-recognize patients who otherwise might have been missed.
Curated from healthcareitnews.com · 22 June 2021 →
Epic's sepsis prediction model was running in hundreds of US hospitals when researchers at Michigan Medicine validated it independently against nearly 40,000 admissions and published in JAMA Internal Medicine in June 2021. It missed two thirds of the sepsis cases while alerting on nearly one patient in five. Epic disputed the framing and pointed to its own studies. The reason it belongs here rather than in a medical journal club is what had to happen for anyone to know: the model was proprietary, it was deployed at scale on the vendor's own internal validation, and it took an outside team with access to one health system's records to check it. That sequence is the norm rather than the exception for clinical AI.
