We have smartphones, tablets, and laptops. They’re all equipped with an unprecedented number of cameras and even 3D sensors. And yet, if you want to just take a picture of your cat and put her onto a thank-you card, that process is incredibly convoluted. You might take the photo on your phone. Email it to your computer. Then import it into Photoshop.
But a new smartphone and desktop app coming from Cyril Diagne, an artist-in-residence at Google, greatly simplifies the process. Called AR Copy Paste, it can literally take a photo of a plant, mug, person, newspaper—whatever—on your phone, then, through a magical bit of UX, drop that object right onto your canvas in Photoshop on your main computer.
You kind of have to see it to get it. So take a look:
https://twitter.com/cyrildiagne/status/1256916982764646402
“Recently, during confinement, I’ve spent some time every weekend to prototype new ideas on this topic of interaction design + machine learning,” says Diagne. “They are small prototypes that I post on my Twitter, along with technical insights and link to the source code for anyone to try.”
AR Copy Paste began in just this way. Diagne mocked up the project during off hours, then posted it to Twitter . . . where it was promptly liked over 40,000 times. He also shared the technology that made it possible. AR Copy Paste uses an object recognition algorithm called BASNet to capture an object and cut it out from its background. Then Diagne uses another piece of code, called SIFT, to handle the paste.
After seeing the viral popularity of the tweet, Diagne began working to create a standalone smartphone and computer app that will allow anyone to use AR Copy Paste, no coding knowledge required. The smartphone app prototype will be free to test when it comes out. You can sign up to be among the first to try it here.
“It’s not so hard to open the scope of its potential applications. First of all, it’s not limited to objects, and it works equally well with printed materials like books, old photo albums, and brochures,” says Diagne. “Likewise, the principle is not limited to Photoshop but can be applied to any image, document, or video-editing software.”
Diagne even teases that AR Copy Paste could be used for functions other than merely copying and pasting. He imagines that you might use its core software to point to objects around you in a certain sequence to perform a function. I imagine you could build a password tool in such a way, treating objects as a code. Rather than “1, 2, 3, 4″ your passcode could be a series of things you aim your phone’s camera at—”succulent, feet, mouse, lamp.”
Perhaps Diagne’s app will take off, and perhaps it won’t. But at the very minimum, the project is a reminder: The user interface of our world hasn’t been perfected yet. We have the technology to improve it. Now, we just need to keep up the imagination.
Recognize your company's culture of innovation by applying to this year's Best Workplaces for Innovators Awards before the extended deadline, April 12.
