When someone arrives at a hospital due to COVID-19 or just about any other issue, pulse oximeters are the go-to gadget to check how much oxygen is in their blood. Once it’s slipped on a patient’s finger and shining light through the skin, it quickly tells a clinician whether or not the person’s heart and lungs are supplying enough oxygen to meet the body’s needs.
But pulse oximeter readings are notoriously racist, routinely overestimating the amount of oxygen inside Black and Hispanic bodies. A study in 2020 found that Black patients often had dangerously overestimated oxygen levels—at three times the rate that white patients did. And now, new research published in JAMA has found that these inaccuracies measurably impacted COVID-19 treatment at hospitals.
Black people in particular waited an hour longer for treatment than white people due to these overestimated readings, while many were erroneously told they weren’t eligible for treatment altogether. What happened to those patients who likely went home, researchers do not know. But they believe it’s likely one of several reasons that people of color have suffered notably worse outcomes from COVID-19 than white people, even when accounting for socioeconomic factors.
The study was led by Ashraf Fawzy and Tianshi David Wu, both practicing pulmonary and critical care physicians in addition to holding assistant professorships at Johns Hopkins and Baylor College of Medicine, respectively. “I think that it’s probably fair to say, the reason we did this study in the first place is that both of us took care of critically ill patients in COVID ICUs, and we often become surprised at the variation and difference of oxygen saturation measured by pulse oximeters versus arterial blood gas,” says Wu, alluding to the two main methods available to check oxygen levels in blood.
Pulse oximeters are easy, slip-on devices with a quick digital readout that can track someone over time. Arterial blood gas measurements are more accurate but they require a patient’s blood be drawn from the wrist—a particularly painful spot—and the results take time to process. Even when that data comes in, it’s just one snapshot of a patient’s well-being. If a doctor wants another peek at a patient’s oxygen saturation later, that requires another blood draw. In other words, pulse oximeters trade some accuracy for a practical UX that’s made them essential.
In the new study, the research team collected patient data from more than 7,000 COVID-19 patients who were admitted to the Johns Hopkins Health System in Baltimore. About 1,000 of those patients received both pulse oximeter readings and blood gas measurement tests. This data allowed researchers to see for themselves exactly how imprecise the pulse oximeter numbers were, and build a model to estimate how far off pulse oximeter measurements were for the other 6,000 patients.
As it turned out, pulse oximeter results were mostly accurate for white patients, but for Black, Latino, and Asian patients they were consistently off by 1% to 2%. Such a small overestimate might not sound like much on a 100-point scale, but Wu says it’s critical. The diagnostic difference between being labeled with COVID and “severe COVID”—and getting drugs like Remdesivir—can come down to the single percentage point between 93% and 94% blood oxygen saturation.
“When our treatment recommendations are based on these thresholds, and the total number of people who had COVID in the U.S. is so high, that tipping point can encompass a lot of people on a population level,” Wu says. “We anticipate that this was pretty wide-ranging in terms of the number of people it affected.”
Digging through the data, it turns out that 451 of the 7,126 patients were eligible for treatment but didn’t have that recognized by doctors at the time. A majority of those unrecognized patients (55%) were Black, followed by Hispanic (27%). (Asian people represented only 0.9%, though Wu notes that the low numbers of Asian people analyzed in this study makes it harder to draw conclusions.) Meanwhile, more than one-third of Black and Hispanic patients experienced delays in getting treatment due to slow diagnosis, while only a fifth of white patients dealt with such delays.
The figures sound shocking, and yet the truth is that pulse oximeters have a finicky design to begin with. They work by shining light through someone’s skin to see their blood, but there are all sorts of variables in this tactic. “Skin pigmentation, nail polish color, anything that can change the absorption characteristics of the wavelengths of light for the pulse emitter seems to affect the accuracy of it,” Wu says.
This inaccuracy is a known problem within the scientific community. The Food and Drug Administration went so far as to publish a warning in 2020, cautioning healthcare and the general public to be wary of pulse oximeter readings. But that’s done nothing to stop just about everyone I know from buying a home pulse oximeter during the pandemic, much less to stop doctors from using them.
“I don’t think that’s percolated to the general public, or even physicians, that there’s this inaccuracy in pulse oximetry,” Wu says. “Not just inaccuracy, but the main, specific inaccuracy is bias—where it systematically tells someone reading it that people of color, ethnic and racial minorities, are healthier than they think they are. So it really masks the underlying severity of illness in this situation.”
Yet, if we know a pulse oximeter is reliably overestimating oxygen levels for certain groups of people, is that something that doctors could simply compensate for, much like they need to study how rashes appear differently on white and Black skin? Wu says that many in the medical community have suggested this idea, but he doesn’t think such a “race correction” should have to exist in a quantitative measurement from a diagnostic tool.
“We feel that the real solution here is the technological one,” he says. “We’ll leave it to the device companies to figure how to engineer the solution, but the issue here is really the issue with the measurement itself.”
In response to this bias, Co.Design reached out to several major pulse oximeter manufacturers (Masimo, Nonin, ICU Medical, and Medtronic) asking what the testing methodologies are for these devices, and whether they’re being validated across races and ethnicities in equal measure. Only Medtronic wrote back by the time of publishing, noting that the FDA requires only 15% of test subjects to have darker skin in order to validate the devices for sale, but for future products, the company is reconsidering its approach.
“In an ongoing study of our next-generation Nellcor pulse oximetry system, we went beyond the threshold of 15% cited in the FDA guidance and included up to 40% dark skin pigmentation enrolled patients to ensure our technology will perform as intended for all patient populations,” wrote Frank Chan, president of patient monitoring at Medtronic. “Medtronic is engaged in ongoing discussions with regulatory agencies and industry organizations about these thresholds to ensure patients are well represented.”
As for the future of Wu’s research, he’s currently working on a follow-up study to ascertain whether people who were delayed COVID-19 treatment or turned away entirely have suffered worse health outcomes as a result.