Research Network Based at DCRI Awarded up to $102.5M for Work on Antibacterial Resistance

December 13, 2019 – The Antibacterial Resistance Leadership Group conducts clinical research to better understand how to diagnose, treat, and prevent antibacterial resistant infections.

 The Antibacterial Resistance Leadership Group (ARLG), a unique research network for which the DCRI acts as a coordinating center, received a federal award worth up to $102.5 million over seven years that will continue the ARLG mission to prioritize, design, and execute clinical research to reduce the public health threat of antibacterial resistance.

The funding, provided by the National Institute of Allergy and Infectious Diseases (NIAID), part of the National Institutes of Health, supports the mission of the ARLG under award number UM1AI104681.

Antibacterial treatments are becoming less effective, resulting in an urgent threat to public health. A 2019 report from the Centers for Disease Control and Prevention estimates that antibiotic-resistant bacteria infect more than 2.8 million people annually, killing at least 35,000 in the U.S.

The ARLG, which received its inaugural funding from NIAID in 2013, is composed of more than 50 leading experts working together to innovate clinical trial design, inform guidelines, and improve clinical practice in infectious diseases.

The ARLG’s research team has collaborations in 19 countries and has initiated more than 40 clinical research studies involving more than 20,000 patients across more than 130 sites. Its three areas of research align with the CDC’s antibiotic resistance threats and include:

  • Infections caused by Gram-negative bacteria, such as Escherichia coli;
  • Infections caused by Gram-positive bacteria, such as Staphylococcus aureus; and
  • Diagnostics such as rapid point-of-care tests to detect drug resistance, guide antibacterial therapy, and support clinical trials.

Vance Fowler, MD, MHS“The renewal support from the NIAID will allow the ARLG to continue its collaborative work to advance science in antibacterial research, and to provide funding opportunities for the next generation of researchers dedicated to addressing this public health threat,” said Vance Fowler, M.D., ARLG co-principal investigator, member of the DCRI, and professor of medicine at the Duke University School of Medicine.

“We are delighted to be able to continue to support efforts to fight antibiotic resistance by generating data that is used to inform dosing guidelines and developing diagnostic testing for better detection and timely treatment,” Fowler said.

To learn more, read the news release from the NIAID.

Working at the Intersection of Expertise to Improve Patient Care

December 11, 2019 – With nine-plus clinical areas under one roof, the DCRI enables faculty members of various specialties to work together to draw deeper insights in clinical research and find better treatments for patients.

Working at the Intersection of Expertise to Improve Patient Care

Traditionally, patient care has taken a somewhat siloed approach, with specialists treating patients for organ-specific complaints. However, advancements in medicine have resulted in the need for a new paradigm: one in which specialists in different areas of medicine come together to address issues at the intersection of their expertise.

“We now have tools that we know are effective in keeping our patients healthier and helping them to live longer,” said Jennifer Green, MD, an endocrinologist at the DCRI. “We all need to partner and be engaged in ensuring our patients’ care aligns with current evidence-based guidelines. This will require communication with other specialty areas, but it is very important to understand that optimization of care is a shared responsibility. We can no longer only focus on our areas of expertise; instead, we have to focus on collaborating to care for the whole patient.”

With 140 faculty members across nine therapeutic areas who often come together to work on clinical trials, the DCRI is particularly well poised to work in this space.

DCRI faculty from various specialty areas discuss ongoing projects and future opportunities for collaboration in the cardiometabolic and cardiorenal spaces.

New Medications, New Opportunity

As health care advances, patients live longer and develop more comorbidities, calling for a new era of health care in which providers must think about clinical areas outside the one in which they work. For instance, a cardiologist now needs to consider how medications he or she prescribes affect other conditions like diabetes or renal disease. Simultaneously, new evidence is emerging that show many therapies can be used for multiple indications.

Some of this evidence was presented at the American Heart Association Scientific Sessions 2019, such as results from DAPA-HF, which was led by John McMurray, MD, of the BHF Cardiovascular Research Centre in Glasgow, Scotland. The study results showed that dapagliflozin, an SGLT2 inhibitor traditionally used to treat diabetes, can also be used to safely and effectively treat heart failure with reduced ejection fraction both in patients with and without diabetes.

“It’s changed how we think of dapagliflozin; we now consider it a foundational therapy for reduced ejection fraction, and I think many of us eagerly await the preserved ejection fraction data, as well,” said Robert Mentz, MD, a heart failure specialist at the DCRI.

Although the DAPA-HF study was not powered to look at renal outcomes, nephrologists were also struck by the results, which Daniel Edmonston, MD, a nephrologist at the DCRI, called “profound.”

“Results like these are exciting for work moving forward from a cardio-renal-metabolic standpoint as we expand the use of these medications in growing indications outside of diabetes,” he said.

Green also looks forward to the opportunities provided by expanded uses for these medications. She is the principal investigator for a trial called EMPA-Kidney, which is examining the effects of empagliflozin, another SGLT2 inhibitor, in patients who have chronic kidney disease. Although empagliflozin, like dapagliflozin, was initially approved to reduce glucose levels in patients with type 2 diabetes, the leaders of EMPA-Kidney expect that it will also reduce cardiovascular and kidney complications in patients with and without diabetes.

“Although these drugs are traditionally viewed as diabetes medications, we need to start thinking of them as drugs with much broader benefits for our patients,” Green said.

Closing the Gap via Implementation Science

Although advancements like DAPA-HF have resulted in more options for treatment of previously difficult-to-treat conditions, implementation of these therapies into clinical practice remains challenging.

A DCRI-led study called COORDINATE-Diabetes is making an effort to tackle this problem by working with cardiology clinics to improve care for patients who have cardiovascular disease and diabetes. Through the intervention arm of the trial, teams from the DCRI are helping clinics identify their challenges and create solutions that would boost prescribing rates of efficacious therapies. The teams that visit the sites are diverse in their expertise—each team includes a cardiologist, an endocrinologist, and a quality improvement specialist—in an effort to help sites coordinate their patients’ care among cardiologists, endocrinologists, and primary care doctors.

DCRI co-chief fellow and cardiologist Adam Nelson, MBBS, PhD, who is working on COORDINATE-Diabetes along with Green and others from the DCRI, contributed to a poster presented at AHA 2019 that outlined data from patients similar to the COORDINATE study illustrating the depth of the problem. In a population of over 150,000 patients with both atherosclerotic cardiovascular disease and diabetes, fewer than 5 percent of all patients were being prescribed all three agents that have proven to be effective including high intensity statins, ACE inhibitors or ARBs, and SGLT2 inhibitors or GLP-1 receptor agonists.

Neha Pagidipati, MD, MPH, a cardiovascular prevention specialist and one of the faculty leads for COORDINATE-Diabetes, noted that there are multiple barriers to prescribing guideline-recommended therapies, both at the provider level and at the system level. Some cardiologists report hesitancy to infringe on the diabetes care providers’ space, while others share they are uncomfortable with some of the metabolic side effects that can accompany the newer medications proven to be effective for cardiovascular conditions. She said she hopes to see a shift as medications move from being defined as primarily diabetes treatments to now being seen as cardiovascular risk reduction medications.

“We have been thinking a lot at the DCRI about how to actually get therapies implemented because we know it takes approximately 17 years for guideline therapies to make it into clinical practice,” Pagidipati said. “When you have therapies that are this effective and populations that are at such high risk, this is not acceptable.”

The AHA 2019 analysis that Nelson contributed to provides further insight into this divide. In the cohort his team examined, less than 30 percent were seen by an endocrinologist, while 70 percent were seen by a cardiologist. Nelson said these findings show that an intervention targeted toward cardiologists has the greatest potential impact to increase prescriptions of guideline-recommended therapies. Through COORDINATE’s intervention, the study team will be looking for ways to empower cardiologists to take a more active role in their patients’ overall health and feel comfortable prescribing medications beyond traditional cardiovascular medications.

Looking Toward a Future of Integrated Care

Although it previously would have been unusual to see cardiologists working alongside a nephrologist to improve patient care, these days it is commonplace at the DCRI, Nelson said.

Edmonston agrees. “There is increasing overlap in the Venn diagram of subspecialty care for our patients,” he said. “It’s exciting to see opportunities where we will be able to work together across differing specialties to provide the best care.”

Because the DCRI contains nine different therapeutic areas under one roof, along with numerous subspecialties within these therapeutic areas, Nelson said the institute provides an ideal environment for a new way of working that lays the groundwork for the future of health care.

“The DCRI is a one-stop shop that has experts in every clinical area running trials and collaborating on research,” Nelson said. “A model in which we can see the common threads in our research helps us to aggregate data, opinions, and ideas to inform more well-rounded patient care.”

Considerations for Assessing Open-Label, Comparative Effectiveness Trials

Matthew Roe, MD, MHS

Deepak Bhatt, MD, MPH

The DCRI’s Matthew Roe, MD, MHS, (pictured left), recently participated in a discussion with his colleague, Deepak Bhatt, MD, MPH, (pictured right), of Brigham and Women’s Hospital and Harvard Medical School. The two cardiologists discussed how to interpret results from head-to-head comparative effectiveness trials involving cardio-metabolic drugs. Listen to an audio recording of the conversation.

 

 

 

 

Conflicts of Interest for Dr. Matthew Roe from July, 2017 – June, 2019:
Research Grant Funding: Sanofi-Aventis, Astra Zeneca, Patient Centered Outcomes Research Institute, Ferring Pharmaceuticals, Myokardia, Familial Hypercholesterolemia Foundation, Bayer.
Consulting or Honoraria: Astra Zeneca, Amgen, Cytokinetics, Eli Lilly, Roche-Genentech, Janssen Pharmaceuticals, Regeneron, Novo Nordisk, Pfizer, Sanofi-Aventis, Signal Path, and Elsevier Publishers.
All conflicts of interest are listed at https://www.dcri.org/about-us/conflict-of-interest.

Dr. Deepak L. Bhatt discloses the following relationships - Advisory Board: Cardax, Cereno Scientific, Elsevier Practice Update Cardiology, Medscape Cardiology, PhaseBio, Regado Biosciences; Board of Directors: Boston VA Research Institute, Society of Cardiovascular Patient Care, TobeSoft; Chair: American Heart Association Quality Oversight Committee; Data Monitoring Committees: Baim Institute for Clinical Research (formerly Harvard Clinical Research Institute, for the PORTICO trial, funded by St. Jude Medical, now Abbott), Cleveland Clinic (including for the ExCEED trial, funded by Edwards), Duke Clinical Research Institute, Mayo Clinic, Mount Sinai School of Medicine (for the ENVISAGE trial, funded by Daiichi Sankyo), Population Health Research Institute; Honoraria: American College of Cardiology (Senior Associate Editor, Clinical Trials and News, ACC.org; Vice-Chair, ACC Accreditation Committee), Baim Institute for Clinical Research (formerly Harvard Clinical Research Institute; RE-DUAL PCI clinical trial steering committee funded by Boehringer Ingelheim; AEGIS-II executive committee funded by CSL Behring), Belvoir Publications (Editor in Chief, Harvard Heart Letter), Duke Clinical Research Institute (clinical trial steering committees, including for the PRONOUNCE trial, funded by Ferring Pharmaceuticals), HMP Global (Editor in Chief, Journal of Invasive Cardiology), Journal of the American College of Cardiology (Guest Editor; Associate Editor), Medtelligence/ReachMD (CME steering committees), Population Health Research Institute (for the COMPASS operations committee, publications committee, steering committee, and USA national co-leader, funded by Bayer), Slack Publications (Chief Medical Editor, Cardiology Today’s Intervention), Society of Cardiovascular Patient Care (Secretary/Treasurer), WebMD (CME steering committees); Other: Clinical Cardiology (Deputy Editor), NCDR-ACTION Registry Steering Committee (Chair), VA CART Research and Publications Committee (Chair); Research Funding: Abbott, Afimmune, Amarin, Amgen, AstraZeneca, Bayer, Boehringer Ingelheim, Bristol-Myers Squibb, Chiesi, CSL Behring, Eisai, Ethicon, Ferring Pharmaceuticals, Forest Laboratories, Fractyl, Idorsia, Ironwood, Ischemix, Lilly, Medtronic, PhaseBio, Pfizer, PLx Pharma, Regeneron, Roche, Sanofi Aventis, Synaptic, The Medicines Company; Royalties: Elsevier (Editor, Cardiovascular Intervention: A Companion to Braunwald’s Heart Disease); Site Co-Investigator: Biotronik, Boston Scientific, CSI, St. Jude Medical (now Abbott), Svelte; Trustee: American College of Cardiology; Unfunded Research: FlowCo, Merck, Novo Nordisk, Takeda.

New Quadruple Therapy Shows Promise, But More Data are Needed

December 4, 2019 – In a recent editorial in Circulation, the DCRI’s Michael Felker, MD, raises questions that the heart failure field should consider in light of new results from the DAPA-HF trial.

Recent results from DAPA-HF may be moving the treatment of heart failure into a new era of “quadruple therapy,” but further studies are needed to understand the benefits of using an SGLT2 inhibitor such as dapagliflozin in addition the guideline-recommended triple therapy, wrote the DCRI’s Michael Felker, MD, in a recent editorial in Circulation.

The editorial, which accompanied new results published from DAPA-HF on quality of life and the impact of patient age, points to the importance of these findings in understanding the role of dapagliflozin in patients with heart failure with reduced ejection fraction regardless of diabetes status. SGLT2 inhibitors had previously been found to prevent heart failure in patients with Type 2 diabetes.

Michael Felker, MD

More studies will need to be conducted, Felker notes, to determine whether these benefits are specific to dapagliflozin or whether they extend to the entire class of SGLT2 inhibitors. Additional study of the specific mechanisms that result in the benefits of the drug is also needed. Implementation of this new quadruple therapy will also be challenging in a landscape in which it is already difficult to get patients on guideline-directed medical therapy, especially in at-risk subsets of patients like older patients.

Felker points to two papers published in the same issue of Circulation that help provide some insight to answer these questions. A paper by Kosiborod et al. found that dapagliflozin resulted in an improvement in health-related quality of life that was sustained over six months. Another paper by Martinez et al. reports on the effects of dapagliflozin stratified by age group. Although adults over 75 were more likely to have adverse events, the study saw similar rates between the intervention group and the control group, suggesting that dapagliflozin is well tolerated even in older patients.

Although the results from DAPA-HF are promising, more data are needed to confirm the benefit of dapagliflozin and other SGLT2 inhibitors in the treatment of heart failure, Felker writes. This is especially important in other subsets of the population, such as patients with heart failure with preserved ejection fraction. In addition, physicians will have to focus on implementing these new findings into clinical practice to provide the best care for patients with heart failure.

Proteins in Plasma Could Provide Information about IPF Disease Severity

December 3, 2019 – A study of proteins in plasma taken from patients in the IPF-PRO registry found that patients with IPF have a distinct circulating set of proteins and that select proteins correlate well with clinical measures of disease severity.

Certain proteins may provide important information about clinical measures of disease severity in patients with idiopathic pulmonary fibrosis (IPF), according to recent research from the DCRI.

IPF is a devastating lung disease characterized by progressive scarring of lung tissue. Although the incidence and prevalence of IPF are increasing, its clinical diagnosis and management remain challenging. The DCRI oversees the IPF-PRO registry, which has enrolled over 1,000 patients with IPF. The DCRI’s Scott Palmer, MD, MHS, serves as the principal investigator for the registry. The prospective, serially collected clinical data and biosamples in IPF-PRO provide a unique and rich infrastructure from which to grow both clinical and translational research initiatives aimed at improving care for patients with IPF.

Jamie Todd, MD, MHSThe most recent study from the IPF-PRO Registry, which was led by DCRI’s Jamie Todd, MD, MHS, and was published in Respiratory Research, included 300 patients in the registry and 100 patients without known lung disease as a control group. The patients without lung disease were drawn from the MURDOCK study, which is led by Kristin Newby, MD. Plasma was collected from patients at enrollment and analyzed using aptamer-based proteomics. Of more than 1,300 proteins assayed, 551 had statistically significantly different expression in patients with IPF as compared with individuals without lung disease—including 47 proteins that appeared to be at least 1.5 fold different in IPF relative to the control group. Notably, several of these proteins also correlated well with clinical measures of IPF severity, including lung function parameters and indicators of the extent of lung fibrosis observed on CT scans of the lungs.

“This research is the first phase of a stepwise approach,” Todd said. “Now that we have a better understanding of protein expression profiles in these patients and their relationship to the clinical measures we use to assess disease severity, we can conduct further studies to consider the value of these proteins as predictors of disease behavior or patient prognosis. We are really just at the tip of the iceberg of what is possible using the rich repository that is IPF-PRO.”

Todd added that she and her team, including Megan Neely, PhD, Robert (AJ) Overton, and Hillary Mulder of DCRI Biostatistics, are now embarking on this effort. Specifically, the team is using multivariable modeling approaches to consider whether sets of these proteins can provide information about clinically relevant outcomes in patients with IPF.

“Early observations indicate that some proteins confer important information about future outcomes in patients with IPF—information that has not been completely captured by considering clinical factors alone,” Todd said. Her team has submitted an abstract with these early observations to the American Thoracic Society meeting.

To Leverage Power of Machine Learning, Focus on Implementation is Needed

November 26, 2019 – With machine learning on the rise, DCRI’s Eric Peterson, MD, MPH, takes a retrospective look at prior efforts to implement risk prediction models and what lessons can be drawn from the past for current and future care.

Machine learning holds great potential for improving patient risk prediction; however, clinicians and health systems still will have challenges getting widespread adoption of these predictive models into clinical care, wrote DCRI’s Eric Peterson, MD, MPH, in a recent JAMA Viewpoint.

The use of automated machine learning algorithms for risk prediction could help physicians and patients make more informed care decisions. However, Peterson notes, machine learning does have limitations; in some instances, machine learning models do not outperform traditional regression models, and clinicians are sometimes frustrated that they cannot identify the factors that influence machine learning models’ predictions.

Eric Peterson, MD, MPHIn order to realize machine learning’s full potential in health care, Peterson writes, implementation is critical. He points to some of the first computer generation models that were developed at Duke in 1968.  While the Duke prediction tools were proven to perform as well or better than skilled clinicians, the models were never fully implemented in clinical care. One challenge was that physicians were not trained in statistics or probabilistic thinking and so did not see the new models’ value. “This failure to adopt predictive analytics into practice would be relived again and again in medicine,” Peterson notes.

The availability of electronic health records (EHRs) could be helpful during the implementation of machine learning models, but challenges still exist in this space, as well, such as developing data standards that can be used across health systems and formatting the data to ensure it is usable.

Multiple initiatives need to be put into place in order to leverage the power of EHRs combined with machine learning to improve patient care. Initiatives should target the entire health care delivery, from practitioners to patients; better quantitative training is needed to empower clinicians to use novel predictive models, while improved communication tools could better educate patients on their risk.

To read more from Peterson on the implementation of machine learning models, read the Viewpoint piece in JAMA.

AHA 2019: DCRI Study Tests Apixaban versus Warfarin in New Patient Population

November 25, 2019 – The exploratory study of patients with atrial fibrillation and on hemodialysis evaluated rates of bleeding when randomized to apixaban versus warfarin.

Apixaban may be a reasonable treatment option for patients with atrial fibrillation who also have renal disease and are receiving hemodialysis, according to late-breaking clinical trial results from RENAL-AF presented Saturday, Nov. 16 at the American Heart Association Scientific Sessions 2019.

Apixaban is one of a class of drugs called oral anticoagulants (OACs), which are used for stroke prevention in patients with atrial fibrillation. Previous trials, such as the DCRI-led ARISTOTLE, had shown apixaban, a newer OAC, to be superior in both safety and efficacy to the traditionally used warfarin; however, these drugs had not previously been tested in patients receiving hemodialysis to see if these results held true in a patient population with end-stage renal disease.

The DCRI’s Sean Pokorney, MD, MBA, was a member of the trial steering committee and presented the results of the study, which aimed to test the hypothesis that apixaban results in less bleeding than warfarin when given to patients on hemodialysis. Although the exploratory study was terminated early due to slower than anticipated enrollment, it revealed important findings that can be built upon in future studies. Notably, minority patients were underrepresented in the previous landmark trials evaluating the newer OACs, while 44.9 percent of the patients included in this study were black.

The study was underpowered, given the challenges with enrolling hemodialysis patients in this cardiovascular clinical trial.  The study found similar rates of major bleeding or clinically relevant non-major bleeding in both groups (25.6 percent on apixaban versus 22.2 percent on warfarin). Ongoing pharmacokinetic analyses may shed light that could help guide apixaban dose selection in this high-risk population.

“Bleeding rates and mortality rates are high in this population, and nearly three-quarters of the clinically relevant non-major bleeding we saw was associated with the hemodialysis access site,” Pokorney said. “Of course, any time a patient is taking an anticoagulant, measures to reduce bleeding, such as avoiding aspirin, should be considered. Although larger trials are needed, the first randomized data show us that apixaban may be comparable to warfarin when treating patients receiving hemodialysis.”

Other DCRI contributors include co-principal investigator Christopher Granger, MD, and steering committee members Hussein Al-Khalidi, PhD; Renato Lopes, MD, PhD; and Kevin Thomas, MD. The DCRI provided statistical support, and statistician Kerry Lee, PhD, served on the data safety monitoring board. The DCRI’s Clinical Events Classification group also made important contributions to the study by adjudicating all clinical events. RENAL-AF was funded by Bristol-Myers Squibb and Pfizer.

DCRI Faculty Recognized on Global “Highly Cited” List

November 22, 2019 – Fifty-four researchers from Duke, including nine from the DCRI, were recognized for their high citation rates.

This year’s “Highly Cited Researchers” recognizes nine researchers from the DCRI as “researchers with broad community influence.”

The list, which is released annually by ISI-Web of Science, calculates how many times authors’ work appear in the citations of other papers. The list also enables a global comparison of academic institutions’ citation rates—Duke University, with 54 researchers recognized, is in a four-way tie for eighth most cited university.

The data to develop this metric is taken from 21 broad research fields defined by sets of journals. This year’s honorees from the DCRI, as well as the fields for which they were recognized, include:

  • Lesley Curtis, PhD – Cross-Field
  • Pamela Douglas, MD – Clinical Medicine
  • Christopher Granger, MD – Clinical Medicine
  • Adrian Hernandez, MD, MHS – Clinical Medicine
  • Magnus Ohman, MBBS – Clinical Medicine
  • Manesh Patel, MD – Clinical Medicine
  • Michael Pencina, PhD – Social Sciences and Clinical Medicine
  • Eric Peterson, MD, MPH – Clinical Medicine
  • Bryce Reeve, PhD – Social Sciences

Several of those recognized have also made the list in previous years. Former DCRI faculty members Christopher O’Connor, MD, Robert Harrington, MD, and Robert Califf, MD, were also recognized in the field of clinical medicine.

Recent statistics compiled by the DCRI show that since 1996, the DCRI’s work has been cited in more than 760,000 scientific articles.

“I am proud to see our faculty being cited by our peers and colleagues,” said Lesley Curtis, PhD, the DCRI’s interim executive director. “By delivering on our mission to share knowledge, we are enabling other researchers to build upon our work to conduct further studies and make more discoveries.”

Data Monitoring Committees Need Complete Data to Protect Patients

November 21, 2019 – Data monitoring committees could be provided more and better data in order to fulfill its role of protecting trial participants.

Data monitoring committees (DMCs) play a critical role in clinical trials by protecting patients—especially in studies involving high-risk populations or potentially harmful treatments—and in order to fulfill this role successfully, these committees should have access to all data at each interim review, wrote the authors of a recent paper published in the Annals of Internal Medicine.

Frank Rockhold, PhDThe DCRI’s Frank Rockhold, PhD, and Robert Bigelow, PhD, served on the writing group for the paper, which provided recommendations for summaries produced by statistical data analysis centers (SDACs) and provided to DMCs, or data and safety monitoring boards.

The authors note that often DMCs only receive the data thought necessary for making decisions about safety. However, they argue, when these groups receive limited data and information, they are lacking important context. “To fulfill its mandate to protect trial participants, the DMC needs timely access to all relevant information, including efficacy data,” the authors write.

The paper also identifies problems with reports currently produced by SDACs, which are often lengthy and difficult to digest. “To foster efficient, informed decision making, reports should be streamlined, concise documents that display important data in optimally informative ways,” the authors recommend.

In addition, the authors advocate for implementation of graphical summaries that integrate benefits and harms. Visuals could help the DMCs make more accurate risk-benefit analyses with less room for bias and misinterpretation of the data. The paper walks through several examples of graphical representations that could provide information that would be helpful to the DMC. This will require investment in resources to develop data visualizations, which can then be reused in future trials.

“In writing this paper, we concluded that several immediate steps can be taken to help DMCs do their jobs more effectively,” Rockhold said. “DMCs will only be equipped to protect patients to the fullest extent when they have complete and easily digestible information to guide their decisions.”

AHA 2019: Machine Learning Not Always the Answer, DCRI Study Finds

November 20, 2019 – A DCRI-led study used two registries to compare three different types of machine learning algorithms with stepwise logistic regression.

Although machine learning is a novel technique that has impressive applications in health care, in some settings, these novel approaches do not improve upon traditional approaches, according to a recent oral abstract DCRI fellow Zak Loring, MD, presented Saturday at the American Heart Association 2019 Scientific Sessions.

The analysis compared three different machine learning techniques—random forests, gradient boosting, and neural networks—with traditional stepwise logistic regression to determine which technique produced the most accurate outcomes model to predict risk for atrial fibrillation patients.

The study team tested the models in two different registries of patients with atrial fibrillation: ORBIT-AF, which includes 23,000 patients, and GARFIELD-AF, which includes 52,000 patients across 35 countries. The team also developed a common data model so that each model could be used across both registries to test external validity.  This is important, Loring said, because often machine learning algorithms are powered for to be highly predictive in one specific patient population.

Zak Loring, MD“Some machine learning algorithms yield impressive results, but may not yield the same results when applied outside the original sample,” Loring said. “Often we build algorithms in clean clinical trial datasets, but when we apply it outside that setting to a wider population that would not have been eligible for the clinical trial, we see weaker performance. It is important to account for generalizability when building these algorithms.”

In comparing the machine learning models to the logistic regression model, the team examined two other measures in addition to external validity: discrimination capacity and calibration. In discrimination capacity, the machine learning method performed as well or slightly worse than traditional regression; in calibration, machine learning performed worse.

In addition, the traditional regression used structured data elements like case report forms, a positive in this scenario because it makes for an interpretable model in which clinicians can identify risk factors.

“One major complaint associated with machine learning models is that they can sometimes be a bit of a black box,” Loring said. “That is, even if they can accurately predict risk, they can’t tell you why that risk is present.”

Loring added that these results show that despite the promise of machine learning, there are likely tasks that are better suited for older techniques. One area that warrants more discussion is the structure of registries. In order to fully harness the power of machine learning, it might benefit researchers to build registries with fewer binary variables and more continuous data.

Other DCRI contributors to this analysis include Jonathan Piccini, MD, MHS; David Carlson, PhD; Eric Peterson, MD, MPH, and former DCRI statistician Karen Pieper, MS.