The one thing that's been missing from the criticism surrounding Theranos, the multibillion-dollar blood-testing startup? Data.
But today a group of biomedical informatics researchers from the Icahn Institute at Mount Sinai published the high-level results of a study they've been working on for about a year—the first peer-reviewed and independent study of Theranos's blood tests. The researchers recruited 60 healthy patients for 22 tests to compare the accuracy of blood tests obtained from a finger prick via Theranos's technology with a standard blood draw from conventional testing companies Quest and LabCorp.
On the face of it, the results don't look good for Theranos. Of the three vendors, it had the highest number of results outside of the normal range. The researchers also found discrepancies in lipid-panel test results between Theranos and other clinical services, which is a particularly critical test to get right.
Nevertheless, the researchers (who frequently test biotech startups) think that Theranos actually performed pretty well, especially given that Theranos's tests cost about a fifth of the price of its competitors. In an interview with Fast Company, Icahn Institute biomedical informatics researcher Joel T. Dudley, who was a part of study team, says the researchers approved the study with "an enthusiasm for disruptive technology" and that Theranos "performed fairly well for a large number of tests."
A Theranos spokesperson did not immediately respond to a request for comment about these results.
This could have been spun as a rare bit of positive news for Theranos, but the company reacted quite negatively to the Icahn study. In the wake of an investigative series by the Wall Street Journal, the company has been strongly criticized. In October of 2015, federal regulators released a heavily redacted report documenting a number of problems with the company's technology. And after an audit of its lab in Newark, California, this January, federal health inspectors found a number of deficiencies that might put patients in jeopardy.
In response to its critics and negative media reports, Theranos's go-to strategy has been to play defense, often referring to news articles as "baseless" and "factually and scientifically erroneous."
In response to the Icahn study, the company sent the Journal of Clinical Investigation a harshly worded letter in an attempt to stop it from publishing the study. The letter referred to the study as "flawed and inaccurate," adding that the company was "disappointed that any journal would accept (it)."
For the researchers, this reaction felt "oddly personal." Dudley says that Theranos critiqued the researchers for not getting in touch with them before publishing their paper—the researchers claim to have contacted Theranos via LinkedIn "out of courtesy."
Theranos questioned the researchers' methods. One particular point of contention was that taking a draw of blood from a vein immediately before a finger prick, which is what the Icahn researchers did, is not standard practice. But Dudley says they took this approach to mimic a real-world scenario, in which a patient might get multiple blood tests. "And if that is such an issue, why wasn't Theranos asking their patients if they'd just had a blood draw?"
Dudley says he was surprised at Theranos's response, particularly in light of the results, which were irregular but not permanently damning. He stresses that his team initiated the study long before Theranos's string of bad press, and that the researchers don't have a vendetta against the company. They were "just curious," he says, as Theranos's technology had not yet been vetted by an independent team of researchers for a peer-reviewed study.
"I'm disappointed by the reaction," Dudley says. "I was expecting them to respond by publishing their own data. After all, data [speak] louder than words."
But Dudley says he is still rooting for startups like Theranos to disrupt the lab-test industry, which he refers to as the "dark corner of medicine." Quest's and LabCorp's methods are rarely questioned, but there were still discrepancies between their results. As we have previously reported, these companies have largely been able to evade regulatory scrutiny due to an obscure loophole.
Dudley says the "last thing" he hopes will result from the study is that innovators like Theranos will "be scared away" from the biotesting world, especially as federal regulators crack down on the industry. As for Theranos, he says that there's still ample opportunity for the company to learn from the results and make improvements to the technology.
"I wouldn't write them off," he says. "These results don't mean that its technology isn't fixable."