Remember all the times you snickered when some TV character zoomed in on a photo and said “Enhance!” turning a blurred mess into a highly detailed, razor-sharp image? Now there’s a system–an artificial intelligence-powered image-scaling program called EnhanceNet–that will make that film trope impossible to laugh at, because it’s now a technological reality.
Its inventors–Mehdi S. M. Sajjadi, Bernhard Schölkopf, and Michael Hirsch at the Max-Planck Institute for Intelligent Systems in Germany–classify their software as a “Single Image Super-Resolution Through Automated Texture Synthesis.” That’s exactly what it does: It scales up very low-resolution photos by synthesizing textures that introduce new detail into the resulting high-resolution image. The synthetic detail makes the resulting pictures so realistic that they are almost indistinguishable from the actual photos. Perhaps for the first time ever, artificial intelligence can now re-create reality with uncanny accuracy–by using fake details.
The technology puts current high-end photo scaling methods to shame. You can see the huge difference in this comparison between the top state-of-the-art method (PSNR, or “peak signal-to-noise ratio”) on the left and the result obtained by EnhanceNet on the right:
Talking over email, Mehdi S. M. Sajjadi claims that “the algorithm could easily be put in commercial software such as Photoshop.” What’s more interesting is that you could actually make it part of the operating system “on smartphones, enhancing the quality of images as you zoom into them to avoid blurriness” in real time, Sajjadi says. There are additional applications for this, he adds:
From upsampling old movies to 4K-quality, restoring old family photographs that are too blurry when you want to get a large print over to more general applications such as improving object detection. This is something we’ve actually studied in the paper. It turns out that using our algorithm on images makes it easier for other neural networks to detect objects in images, which has wide applications from Google image search to detecting pedestrians in self-driving cars.
Imagine that: One neural network enhances the image, then another one uses the result to detect objects. Of course, that also potentially includes law enforcement applications. Sajjadi told me that even while the “details that are reconstructed are not necessarily the true ones [. . .] the techniques can indeed be used in special cases such as license plate recognition.” That means that police would not have to wait for ultra-high resolution cameras to catch people speeding. They could use this technology to do it right now.
This technology could pose a real threat to privacy. Someone could get the code for this and train a neural network to re-build an image that was blurred or pixelated before. If you’ve ever uploaded amateur porn with pixelated faces, prepare to get unmasked.