Hospital Ratings Are In The Eye Of The Beholder

By Jordan Rau, Kaiser Health News

(Courtesy: Kaiser Health News)

(Courtesy: Kaiser Health News)

How good a hospital is St. Mary Mercy Livonia Hospital in Michigan? It depends on whom you ask.

The Leapfrog Group, a respected nonprofit that promotes patient safety, gave an “A” to the hospital. The company Healthgrades named it one of America’s best 50 hospitals.

But the Joint Commission, a nonprofit organization that accredits hospitals, and U.S. News and World Report omitted St. Mary from their best hospital lists. Consumer Reports gave it an average safety score of 47 points out of 100, citing high numbers of readmissions, poor communication with patients and excessive use of scans. Medicare, which has a new program rewarding hospitals for meeting certain quality measures, is reducing St. Mary’s payments by a fraction this year.

Is your head spinning yet?

As ratings multiply, more and more hospitals have something they can brag about. A third of U.S. hospitals won at least one distinction from a major rating group or company.

Evaluations of hospitals are proliferating, giving patients unprecedented insight into institutions where variations in quality can determine whether they live or die. Many have similar names, such as “Best Hospitals Honor Roll,” “America’s Best Hospitals” and “100 Top Hospitals.” Some states have created their own report cards. In California there are more than a dozen organizations offering assessments on hospital quality.

But those ratings, each using its own methodology, often come to wildly divergent conclusions, sometimes providing as much confusion as clarity for consumers.

“We’ve alternatively been labeled the least safe hospital in Maine and the safest hospital in Maine,” said Dr. Douglas Salvador, vice president of quality at Maine Medical Center in Portland.

And the ratings do not always jibe with the views of authorities who oversee hospitals. For instance, UCSF Medical Center has gotten good grades from multiple safety raters even as California public health officials have fined it $425,000 repeatedly for endangering patients.

As ratings multiply, more and more hospitals have something they can brag about. A third of U.S. hospitals — more than 1,600 — last year won at least one distinction from a major rating group or company, according to a Kaiser Health News analysis. In the greater Fort Lauderdale hospital market, 21 of 24 hospitals were singled out as exemplary by at least one rating source. In the Baltimore region, 19 out of 22 hospitals won an award.

“I worry a lot about these ratings,” said Jerod Loeb, executive vice president for health care quality evaluation at the Joint Commission. “They’re all justifiable efforts to provide information, but at the end of the day every single one of them is flawed in some respect. Rather than enlightening, we may be confusing.”

Not A ‘Complete Picture Of The Care’

There are so many report cards on hospitals that the Informed Patient Institute runs a website that grades the raters. Carol Cronin, its executive director, said most report cards are not easy for consumers to use. “A lot of them don’t help users quickly understand which hospital is better than another,” she said.

But many hospitals are eager to trump these distinctions in their marketing. Healthgrades, U.S. News and Leapfrog not only encourage this but also profit from it by charging licensing fees to hospitals that want to advertise their awards. “A hospital cannot buy an award, they must achieve it,” Healthgrades said in a statement.

Dr. Andrew Brotman, chief clinical officer at NYU Langone Medical Center in Manhattan, said the fees can be substantial. “Healthgrades, which is one we did well on, charges $145,000 to use this even on the website as a logo, so we don’t do that,” he said. “U.S. News is in the $50,000 range. Leapfrog is $12,500.”

Consumer Reports bars hospitals from using its ratings in marketing, but patients must subscribe to read them online. (Others generally provide free access to ratings on their sites.) The Joint Commission does not charge hospitals that make its top quality list.

The calculations that go into these ratings are complex. Most hospital assessments synthesize dozens of pieces of data Medicare publishes on its Hospital Compare website, including death rates and the results of patient satisfaction surveys. They also examine other sources and use private surveys to create user-friendly lists or grades, which they display on their websites.

Here’s a snapshot of what different ratings measure:

  • The Joint Commission looks at how frequently patients received recommended treatments, such as flu shots for those with pneumonia.
  • Consumer Reports examines the numbers of patients who die or are readmitted, infection rates and Medicare patient surveys of their experiences.
  • Leapfrog looks at data from its surveys of hospitals, the consistency with which hospitals followed safe surgical practices and frequencies of infections and some types of patient harm.
  • Healthgrades analyzes detailed Medicare records to find death and complication rates for 27 procedures and conditions.
  • Truven considers profitability along with quality in its assessments.
  • U.S. News surveys physicians about which places they think are the best, and those reputation ratings account for a third of most of its assessments. U.S. News ranks hospitals by geography and singles out hospitals that do well in 16 specialties, including cancer, neurology and orthopedics.

“Ratings and ranking programs certainly offer people information they can use to make their hospital selections, but we don’t recommend relying on any one of them completely,” Jennifer Kennedy, a spokeswoman for St. Mary Mercy, said in an e-mail. “None are able to tell the whole story or paint a complete picture of the care that is delivered.”

Patient safety rankings are based on how frequently correct procedures or errors occur, so hospitals can get good grades even if they have made some egregious errors. Leapfrog gave a “B” to UCSF Medical Center in San Francisco even though California regulators have penalized UCSF eight times for infractions since 2008, most recently for leaving a sponge in one patient and a plastic clip in the skull of another. Those errors cost UCSF $200,000 in fines.

Dr. Josh Adler, the chief medical officer at UCSF, said penalties were partly a result of the hospital’s policy to make sure that all errors are reported to authorities. “The key is that we constantly strive to deliver the highest quality, safest, and most satisfying care, and that we are a learning organization,” he said in an e-mail.

The ratings groups believe the public benefits from the multitude of ratings. Dr. John Santa, who directs Consumer Reports’ health ratings, said consumers benefit from different vantages just as they do for cars or electronic devices, and the competition spurs each rating group to get better. “We think that’s consistent with good science,” Santa said.

Constant Turnover

Much of the quality data is rudimentary, as the science of evaluating hospitals is still in its adolescence. Adding to the confusion is that hospitals can rise and fall from year to year as groups tweak their methods of assessment and as hospital performance shifts, even slightly.

In some cases, hospitals that have won awards are being penalized financially by Medicare for falling short on the government’s quality assessment. Saint Francis Hospital in Tulsa, Okla., which is losing 0.54 percent of its Medicare payments this year under the government’s quality program, is a Healthgrades Distinguished Hospital for Clinical Excellence and was ranked the second best hospital in Oklahoma by U.S. News.

Some of the hospitals that do the best in the rankings have limited respect for them. Advocate Christ Medical Center in Oak Lawn, Ill., last year received praise from Leapfrog, U.S. News, the Joint Commission, Truven and Healthgrades. But Dr. William Adair, vice president for clinical transformation, says the hospital doesn’t license any of the distinctions. “We’re all made a little bit uneasy, to be frank about it,” Adair says. “Some of these organizations are looking for revenue. It blurs the effectiveness of the ratings processes.”

Still, many hospitals are happy to use the praise. Dr. Brotman from NYU said: “Even though there’s not a hospital executive who won’t tell you that they have a great deal of skepticism about a lot of the methodology, there’s not one who will tell you they don’t want to be on the lists.”

This article was produced by Kaiser Health News with support from The SCAN Foundation.

Related