Yelp reviews aren’t trustworthy. They could be written by the owner’s brother. They could be written by an obnoxious customer who totally deserved the rude treatment he is ranting about. Most of the time, who cares? The worst consequence from an inaccurate review is that you have a bad dinner. But when it’s your health at stake, the quality of reviews is a more important issue. And many excellent doctors aren’t exactly known for their bedside manner.
Doctor reviews on sites like ZocDoc or RateMDs and Yelp, too, give patients who otherwise are groping in the dark at least something to go on. Whether good reviews actually reflect better physicians and medical results in real life isn’t known.
To date, some research has shown that higher reviews are indeed given to doctors involved in fewer lawsuits and who have more years of experience, but in terms of measuring actual skill, that’s been harder to say. There aren’t many public data sets that objectively evaluate individual physicians’ skills. Also, sicker patients tend to seek out and see highly rated doctors, whereas someone with a more minor problem might just go to the office closest to them, making the question less straight-forward to evaluate.
Now two researchers from the University of Rochester have put together a study that attempts to get around some of these issues in evaluating whether patients do indeed get better care from top-rated doctors. Their result? Go for the five-stars, at least (or especially if) you have heart disease.
The authors, writing in a paper posted on the Social Science Research Network, decided to look at heart surgeons who do bypass surgeries on coronary artery disease patients, one of the more common causes of death in the U.S. They figured people are likely to review their doctors for this serious surgery, and more importantly, the mortality rate after the surgery is a pretty objective and useful measure of the doctor’s skills. They compared online reviews on RateMDs, the oldest physician review site founded in 2004, with surgery mortality rates from Florida’s hospital discharge data in 2012. Using data from the hospitals about the patients’ demographics and statistical modeling, they attempted to account for differences in how severely ill each surgeon’s patients were.
“Our main findings show that patients treated by five-star surgeons have better odds of living than those treated by surgeons rated below five stars, even after controlling for other observed surgeon characteristics such as education and experience,” the researchers write. Because seriously ill patients were seen disproportionately by five-star surgeons, these highly rated doctors actually had higher mortality rates overall–but accounting for this factor really showed they were better at what they did.
There’s a lot of complexity in their analysis, and the researchers don’t know whether their results would hold for other kinds of doctors, such as general practitioners or ob-gyns, or for the treatment of less serious illnesses. Still, something is better than nothing.
Another thing the researchers learned apparently is that patients aren’t dumb. RateMDs allows people to rate based on four categories: helpfulness, knowledge, staff, and punctuality. They found that people’s reviews for staff and punctuality did not correlate with health outcomes, only knowledge and helpfulness. While it might be nice to avoid hours in the waiting room, what was more important was the best doctor for their heart. “To our surprise, patients are smarter than many critics imagine,” the authors write.