The study findings suggest that factors unrelated to key surgical outcomes may be driving consumer-directed rankings.
Consumer-directed hospital rankings influence decisions made by patients, physicians, payers, and hospital systems. In cardiac surgery, little is known about how these rankings correlate with clinical registry quality ratings. To examine how a popular consumer-directed ranking is correlated with risk-adjusted surgical outcomes, a research team at the Duke Clinical Research Institute (DCRI) and the Duke University School of Medicine undertook a comparison of the Society of Thoracic Surgeons (STS) Adult Cardiac Surgery Database (ACSD) outcomes with the annual U.S. News & World Report (USNWR) rankings.
After calculating agreement between the ACSD’s outcomes (risk-adjusted major morbidity/mortality) and the USNWR rankings, the research team found no correlation between the two.
Lead author Oliver K. Jawitz, MD, MHS, a former fellow at the DCRI and current resident physician in General Surgery in the Department of Surgery at the Duke University School of Medicine, presented the findings as a featured abstract during the plenary session of the STS 58th Annual Meeting on January 29.
The researchers queried the STS ACSD for isolated coronary artery bypass graft (CABG) or CABG+valve patients at hospitals among the top 100 USNWR Cardiology & Heart Surgery rankings from 2016 to 2020. They grouped hospitals into deciles based on risk-adjusted observed/expected (O/E) ratios for morbidity/mortality using the STS 2018 risk models. Then, they calculated agreement between the STS ACSD and USNWR ranked deciles using Bowker’s symmetry test. Using repeated measures regression analysis, they modeled the association between each center’s annual change in STS O/E ratio and change in USNWR ranking.
The researchers arrived at the following results:
- In total, 524,393 patients from 149 hospitals (ranked in USNWR top 100 at least once during study period) undergoing CABG or CABG+valve procedures met inclusion criteria, 65 of which were consistently ranked in the top 100 during the 5-year study period.
- There was no agreement between hospital USNWR ranking and STS major morbidity/mortality O/E ratio (p>0.50 for all years).
- There was no association between annual change in hospital USNWR ranking and change in STS O/E ratio (p>0.3).
In conclusion, there was minimal agreement between annual USNWR hospital ranking and the corresponding risk-adjusted STS morbidity/mortality. Further, annual changes in USNWR rankings could not be accounted for using clinical outcomes. Ultimately, these results suggest that factors unrelated to key surgical outcomes, the ultimate metric of quality, may be driving consumer-directed rankings.
A full paper is forthcoming. Additional DCRI and Duke contributors to the abstract were Andrew M. Vekstein, MD; Rebecca Young, MS; Sreekanth Vemulapalli, MD; Brittany A. Zwischenberger, MD; Dylan P. Thibault, MS; Sean O’Brien, PhD; and Peter K. Smith, MD.