Pinterest is making big changes to its search functionality that involve machine learning and image recognition techniques. In a blog post, engineering manager Kevin Jing unveiled what the company calls their “Crazy-fun new visual search tool,” which lets users of the company’s mobile app search for aspects of individual pictures.
Pinterest users on phones and tablets can click on furniture, tattoos, pieces of clothing, or anything else in a picture, and Pinterest will then search its archives for other objects that resemble it. The feature will also be available on Pinterest’s website for desktop and laptop users.
“When you spot something in a Pin that you want to learn more about, tap the search tool in the corner. Then select the part of the Pin you’re interested in, and we’ll show you Pins just like it. You can even filter your visual search results by topic so you find exactly what you’re looking for,” Jing writes.
The new functionality also significantly ups Pinterest’s search and advertising games. As the Wall Street Journal’s Yoree Koh notes, it gives advertisers a significant database of what users are searching for–by indexing individual objects, it creates a knowledge bank which can be leveraged for later advertising purposes.
Jing previously worked on image searches for Google. This initiative also replicates much of the basic feature set of Google’s underused Goggles mobile app, which searches the web for objects that resemble a picture a user takes. It was later updated to search without actually taking a photo of the item in question. But Pinterest faces a unique challenge: making sure searches for tens of thousands of similar-but-different consumer goods display relevant products for users. This, while staying in step with the likes of Instagram, which is now testing 3D Touch ads to make images more commerce-friendly.