Do-it-yourself

n 1999, a national study showed hospital mistakes and medical accidents were killing 99,000 patients a year — twice the fatalities car wrecks claimed. In 2003, we began publishing our list of the state’s top hospitals, an annual feature that proved popular with both readers and advertisers. Measuring quality of care has become a national crusade, one that, as Senior Contributing Editor Ed Martin points out in the story that begins on page 44, now influences how hospitals are paid.

Through the years, we drew on various sources for our listings, but primary among them had been Health Grades Inc., a Denver-based company that uses Medicare data to report on the performance of 5,000 hospitals nationwide in more than two dozen medical procedures and conditions. As we began laying the groundwork for this, our 10th annual listing, we discovered Health Grades was no longer partnering with publications to publish its findings. So, after a bit of head-scratching and seeing what else was out there, we decided to tackle it ourselves. Ed, whose reporting on medical matters has earned many a national award, would take on a new role: that of data miner and number cruncher. Let him tell you about it.

“How do you fairly gauge quality when dealing with the vagaries of the human body? We tapped the broadest possible measures, ranging from how likely you were to die after being rushed to a hospital with a heart attack, to seemingly minor concerns such as whether noise out in the hall kept you awake all night. The idea was, the broader the view, the more likely it’ll produce a fair assessment.

“We assigned values to various things that Medicare measures, such as how likely you are to wind up back in the hospital within 30 days after a bout with pneumonia. We gave the highest value to patient satisfaction — five points for top hospitals — figuring not only that a patient wouldn’t be satisfied if he or she had a bad outcome but also that high satisfaction reflects a culture of quality and concern.

“We tapped other respected measurements, such as those by U.S. News & World Report, Consumers Union and The Leapfrog Group, a nonprofit founded by large employers such as Boeing and General Motors to research where they get the best return on their health-care dollar. We weighted those results heavily because, in addition to using Medicare data, some relied on patient polling and surveys of top doctors. Blue Cross and Blue Shield of North Carolina was particularly helpful since it’s able to mine more than $11 billion a year in claims to see which hospitals are best at what.
   
“We narrowed the field to approximately 90 of the state’s 135 hospitals — those that have 50 beds or more. At that, about 4,000 individual calculations went into a rating system in which 52 points was the most a hospital could possibly score. When an Excel spreadsheet performed the calculations, the top hospitals had 43 points. Few topped 40.”

Was it fair? Yes, but as Ed admits, not infallible. Choosing what data to use and how to weight it is a subjective process, no matter how fair you try to make it. And, as Ed’s story notes, the Medicare data itself might show a bias against what many would consider our very best hospitals.