When bombs went off at the Boston Marathon on April 15, Beth Israel Deaconess Medical Center (BIDMC) CIO John Halamka found himself dealing with the kind of the emergency few drills could ever prepare you for. As bombing victims were brought into his downtown hospital and the city went into lockdown, Halamka and his team began to parse a nightmare situation.
Then it got worse. Suspect Dzhokhar Tsarnaev was bought to Beth Israel... and Halamka, a prominent figure in the bioinformatics world, had to deal with a very unique challenge: How to make sure the Boston bombers' medical records were not stolen by journalists, leaked by hospital employees looking for a payday, or made catch of the day by hackers or foreign intelligence services. (Some of these records, it's worth noting, have recently been released by court order.)
Halamka came to his position at BIDMC with a unique resume. A practicing emergency room physician, he previously worked as a research assistant to Edward Teller and Milton Friedman. Outside of medicine, Halamka founded a software development firm and is a professor at Harvard Medical School. These days, he maintains the popular Geek Doctor blog and lives on an alpaca-breeding farm in rural Massachusetts.
BIDMC explained their tech challenges following the marathon bombing at the United Summit in Boston, an annual security event sponsored by Metasploit creators Rapid7. It was a unique situation for everyone at the hospital, and IT workers had to jump into crisis mode much like the surgeons and nurses. After all, what happens to the hospital if their computers crash?
After his presentation, Halamka explained to Fast Company how nobody accounted for the possibility that BIDMC's engineers could be detained in the hospital's off-site data center as Boston entered lockdown.
"They drank a lot of coffee," Halamka recalled. The hours and days after the Boston attack caused everyone at the hospital—up to and especially including the IT staff—to spring heroically into action. For Halamka's department, ensuring that systems stayed online and maintaining the privacy of patients was essential. In his prior life as a surgeon in Los Angeles, Halamka saw how journalists would try any trick in the book to get a scoop on a breaking celebrity story. From BIDMC's perspective, there was a real risk someone would attempt to steal the medical records of Tsarnaev or the victims. This would hinder the hospital's ability to provide care and risk exposing it to lawsuits.
Hospitals in Los Angeles, New York, and Washington are well acquainted with putting medical records on triple-secret reverse lockdown, something Beth Israel now found themselves doing. Although both paper and electronic health records in the United States are covered by a set of privacy regulations known as HIPAA, it's also an open secret that doctors and nurses routinely disregard HIPAA protocols. This reporter has personally spoken with doctors who share X-rays and charts with colleagues via Dropbox (a huge HIPAA no-no), and Beth Israel itself had a huge data breach after a doctor's laptop was stolen from an unsecured area of the hospital. In violation of Beth Israel and HIPAA protocol, the doctor's computer contained an unencrypted Excel spreadsheet with names and diagnoses for more than 4,000 patients. Halamka had to make sure nothing like that would happen again, especially after the hospital had to pay more than $500,000 in costs for forensics, lawyers, crisis publicists, and other expenses.
"Medical records are one of the most expensive commodities in organized crime circles. They go for at least a thousand dollars, and who knows what the premium on Tsarnaev information would be," Halamka explained. Putting a strict privacy guard in place was a necessity, but there was resistance from doctors and nurses. "I told doctors their days would be longer (as a result of the security precautions) and their work streams would become less usable. Wasn't that great?"
At the core of Beth Israel's emergency privacy practices were recommendations given to the hospital by Deloitte. The global consulting giant was bought in for an audit after the laptop leak, and a second massive privacy data lapse—a radiology station which mysteriously started sending patient data to an IP address in China for unknown reasons. A technician connected the normally non-Internet connected radiology station in order to install software updates. The technician failed to scrub IP addresses and DNS info and shortly after, the radiology station began sending megabytes of encrypted data over Port 80 to a Chinese IP address. When contacted, the manufacturer denied any role. To this day, Beth Israel has no idea why a piece of hospital equipment suddenly began sending patient records to China.
Deloitte's implementations, as explained to the security conference crowd, centered around siloing the data accessed by Beth Israel's 22,000 employees. Halamka compared the audit to a "public colonoscopy"—their consultants examined every single aspect of how employees use computers. How often do passwords expire? How were doctors' personal iPhone usage tracked? What was the Windows registry info? Anything and everything was grist for the mill.
Once the report was ready, Deloitte recommended a level of security that Halamka compared to the CIA or Mossad. It included the creation of 26 new staff positions and the addition of millions of dollars in new costs to the hospital. His task was to make as many of these implementations workable in real life as possible. The most important thing was doing triage to determine what health information was most important to keep safe. For Beth Israel Deaconess, choices had to be made about which records received the strictest, most costly security. Was it billing information? HIV status? Drug abuse history? Psychiatric history? Or information on what vitamins patients took?
Halmaka used an interesting metaphor. "The environment I'm in calls for creating the safest library ever, which charges $500,000 if you lose a book. But if you do that, you don't actually lend out any books. Obviously we can't do that." Halamka then had the unenviable task of coordinating security precautions like updating the way doctors' permissions are changed when they get new job titles with the sprawling hospital complex's many third-party vendors. The IT expert claimed to be dismayed to find out many of his cloud/SAAS vendors had never even had an external security audit.
But the most important thing was putting the hospital on information lockdown. New messages appeared on all portals reminded employees of the importance of discretion. Medical records regarding bombing victims and the terrorists were put under the highest security precautions the hospital had to offer. Halamka said that one of his biggest worries was medical records in the emergency room being accessed by other doctors curious about the terrorist attack. New restrictions had to be created to prevent this. Even after access to written and electronic medical records was restricted, the hospital began auditing and interviewing everyone who had access to the interview. Every single doctor, nurse, and hospital employee who accessed the restricted records had to explain why they wanted them—and Deaconess had to make sure that no one else accessed them.
Meanwhile, Halamka received blowback from doctors and nurses. "After the marathon, I lived information security. I've dealt with both confrontation and hate mail." Some doctors didn't realize viewing unencrypted patient records on their iPhone was a security breach; nurses used home computers for work purposes. That all had to change.
In the end, Halamka said teaching IT security to doctors depended on how, as he put it, they are "motivated primarily by money, titles, and the need to avoid embarrassment." When the Boston attacks occurred, they forced the hospital into crisis mode. Amazingly, their patient records, through it all, remained safe.
[Image: Wikimedia user Twp]