As a sports fanatic, I always watch the Super Bowl, even if my favorite team isn't playing. I also enjoy watching the television commercials, and there was a time when our family used to rate the quality of each commercial! If you watched Super Bowl LIX a few weeks ago, you may have noticed a commercial by NYU Langone Hospital, which showed several hospital leaders playing football. At the very end, former New York Giants wide receiver Victor Cruz says, "Seriously, you guys are the best...", to which one of the hospital leaders replies, "We sure are." Cruz then says, "At the health system stuff." The advertisement then makes the statement that NYU Langone was ranked #1 for quality care in the United States.
We know that NYU Langone paid at least $8 million on the 30-second advertisement, which is the going rate for television commercials airing during the Super Bowl. One of the physicians who appeared in the ad said that the commercial took almost nine hours to record. The question that is being asked is whether it was appropriate for a not-for-profit hospital to spend over $8 million on a commercial advertisement. Just as important (at least in my opinion), Michael Millenson, a journalist for Forbes magazine, asked whether the claim of being #1 was accurate (see his article, "NYU Langone Super Bowl Ad Claimed 'We're #1,' But Where's Scorecard?").
If you really paid close attention during the commercial, you may have caught the reference shown at the bottom of the video during the final 2 seconds of the commercial, stating "2024 Vizient Quality and Accountability Ranking. Ranked #1 out of 115 participating comprehensive academic medical centers." Vizient, Inc. started out as a group purchasing organization, which allows member hospitals to secure volume discounts on supplies and other purchases, though more recently the organization has started a consulting practice to help hospitals improve the quality of care that they deliver at lower costs. I went to their website (see "Vizient announces 2024 top performers in clinical quality"), which lists NYU Langone Hospital as one of 14 top performers in clinical quality in the comprehensive academic medical center category. While they are listed first, it's not clear that they were ranked first, as no numerical ranking is provided. Vizient states that they based their ratings on their own clinical database, as well as publicly available data from the HCAHPS survey (patient and family experience survey data) and the CDC's National Healthcare Safety Network (patient safety and outcome data).
Notably, NYU Langone Hospital is also listed as a Honor Roll hospital by U.S. News and World Report. Importantly, USNWR stopped providing overall numerical rankings for hospitals last year. I am sure that the recognition that NYU Langone Hospital is well deserved, and they should certainly be proud of both these achievements. Unfortunately, neither the Vizient study nor the USNWR survey tell me with any degree of confidence whether NYU Langone Hospital is the greatest hospital, in terms of providing the best in clinical care. As I've mentioned at least a couple of times in the past (see "Do hospital rankings matter?" and most recently, "The first step is to clearly define the problem..."), It's impossible to clearly define "greatness" by any one or even a set of measures. I wonder if we should even try.
A few years ago, a group of physician scientists and experts in outcomes measurement assigned a grade (think A, B, C, D, or F) to the various hospital rankings and ratings systems that are commonly used (these findings were published in the online journal, NEJM Catalyst in an article titled, "Rating the Raters: An Evaluation of Publicly Reported Hospital Quality Rating Systems"). Importantly, no rating system received an A (the top grade) or F (a failing grade). The highest grade received was a B, by USNWR. The authors of the study concluded, "Each rating system had unique weaknesses that led to potential misclassification of hospital performance, ranging from inclusion of flawed measures, use of proprietary data that are not validated, and methodological decisions. More broadly, there were several issues that limited all rating systems we examined: limited data and measures, lack of robust data audits, composite measure development, measuring diverse hospital types together, and lack of formal peer review of their methods." Notably, the Vizient ranking was not included in this study.
It's clear that these rating systems do actually matter, particularly when patients have a choice of where they can go to seek care. Unfortunately, hospitals spend a lot of money on responding to these surveys, and I do wonder how much "gaming the system" goes on when they do provide their internal quality data. Most of the ratings systems aren't very transparent with their methods, which has also led to further questions on their overall accuracy.
Regardless, I do think that NYU Langone Hospital deserves our congratulations, particularly when it's widely acknowledged that the system was in near collapse in the mid-2000's. I've talked about the hospital's remarkable turnaround in my post "Elevators, M&M's, dust balls, and toilet paper", which is further detailed in a book by William Haseltine, World Class: A Story of Adversity, Transformation, and Success at NYU Langone Health. However, I do agree with some of the critics who've called into question whether it's appropriate for a not-for-profit hospital to spend $8 million on a one-time television advertisement.
Suffice it to say, I don't have a lot of confidence in any rating system, as I don't think it's possible to rank a hospital accurately, even when using a whole set of validated measures. For that matter, I don't think it's possible to define "greatness", at least when it comes to clinical quality. But that's a topic for another post...
No comments:
Post a Comment