This AI can see in total darkness

MIT engineers have found a way to use AI to reveal objects in total darkness–even transparent materials like glass and tissue.

Current AI-based “night mode” technology, like the software you’ll find on Google’s Pixel 3, can make nighttime photos remarkably clear by quickly capturing multiple noisy photos and using AI to combine them into a single, noiseless photo. But unlike those techniques that require a lot of light to begin with, MITs method can work in a completely sealed room. In fact, it only requires one photon per pixel.

It’s a breakthrough for imaging, and it could have implications for a broad range of industries. It could reveal invisible details in deep space photography and let doctors see living tissue inside of patients without damaging their cells. As the authors of the research paper, published this week in Physical Review Letters, explain: “When it comes to X-ray imaging, if you expose a patient to X-rays, you increase the danger they may get cancer. What we’re doing here is, you can get the same image quality, but with a lower exposure to the patient.” Coauthor and professor of mechanical engineering George Barbastathis told MIT News, “In biology, you can reduce the damage to biological specimens when you want to sample them.”

The scientists trained a deep learning system by exposing it to photos of 10,000 glass-like etchings that were almost invisible to the eye to begin with. The photos themselves, taken in a sealed room with no light, looked like the static noise you might see on a TV. Then, the team trained the AI on those photos along with the corresponding patterns beneath all that visual noise–which they refer to as “ground-truths.”

The system learned to make sense out of the apparently random black-and-white dots, though it ultimately produced out-of-focus images. To solve the blurriness, the researchers added another layer capable of focusing the output–which worked in a similar way to this depixelization AI–until they got a crystal-clear image. Once the AI was trained, they tested it with the image you can see in this article: a photo of gondolas docked at a pier taken in the dark.

I can’t wait to see this system embedded in imaging systems, from medical devices to telescopes. Perhaps one day, the same computational power could let us take perfect smartphone photos even in the darkest conditions.

Recognize your brand’s excellence by applying to this year’s Brands That Matter Awards before the early-rate deadline, May 3.

Explore Topics