The Instagram Search Engine Wars: Why Machine Vision’s The Next Big Thing For Brands

A wave of new startups can identify brands, location, and people inside Instagram photos.

The Instagram Search Engine Wars: Why Machine Vision’s The Next Big Thing For Brands

A new wave of Instagram search engines are coming… and instead of being operated by parent company Facebook, they’re coming from private vendors. EyeIn, launched today by Israel-based photo app Mobli (and the creator of cult/Silicon Valley-punchline app Yo) is one of the first contestants in a soon-to-be crowded space. The prize? Making hundreds of millions of dollars by aggregating and sorting the endless stream of photo data coming out of Instagram.


EyeIn is a search engine designed to help advertisers find images on Instagram, Twitter, Mobli and other platforms. They’ve signed initial clients up like Huffington Post and Rant Media, who use the service to find images uploaded to social media to include in their stories. Because Instagram images are free–and royalty-free–to include in online content, publishers naturally love them. EyeIn and their forthcoming competitors are betting publishers will generate big bucks for similar Instagram search engines. In EyeIn’s case, they plan to monetize the search engine through advertising and content recommendation in a publisher plug-in appended to the bottom of web pages similar to those offered by two other Israeli firms, Outbrain and Taboola.

Moshe Hogeg, EyeIn’s CEO, explained the way the search engine works in an email to Fast Company:

EyeIn is the first search tool to truly channel the incredible power of user generated content to produce a search engine that is relevant and impactful for visual media. If you search “Cleveland Cavaliers” in Google, you’d receive a page full of information that also understands the context of the inquiry. It doesn’t just have the team’s website, it shows articles from the news and has a boxscore from the most recent game. If you did the same Google search in images, all you get is stock photos without any connection to the major news connected to those images and that search.

The reason is simple: Google is a web crawler and while that works well for text, it is incredibly slow for images and it doesn’t tap into social media effectively. On the other hand, EyeIn would give you access to the myriad of unique perspectives from fans at the game providing a richer and more nuanced view of the experience.

The Dearth of Instagram Search Engines

Surprisingly for such a market-dominant platform–Instagram has 300 million monthly users as of late 2014–Instagram and their corporate overseers at Facebook have been slow to capitalize on the site’s search functions. Despite a recent web makeover, searching through Instagram’s core app is based around either hashtags or username functions. Finding images of a specific place, event, or emotion through Instagram requires considerable work.

Via email, Huffington Post multimedia platform manager Marc Janks explained that “The first time we used the EyeIn plug-in was for an Ed Sheeran concert. At the concert Sheeran brought fans on stage, took selfies, and videos, and all of that media was incorporated into the EyeIn plug-in. One of the best parts about that was that our editors didn’t have to spend time sourcing and finding relevant content, through the millions of social media platforms, from Twitter, Instagram, and Facebook, EyeIn just did it for us.”

A number of third-party search engines such as Iconosquare, FindGram, and have popped up over the past few years which allow web-based searches of Instagram. But the massive firehose of image data that’s constantly posted to Instagram has largely escaped the watchful eyes of marketers and other third parties ranging from intelligence agencies to media organizations to sports teams. However, new advances in machine learning and image recognition are changing that.

Machine Learning, Meet Selfies

EyeIn’s value proposition to the Huffington Post and others is simple: We help you find content, and you generate clicks for us.


When I met with Hogeg several months ago, he was excited about the possibilities that machine learning meant in terms of image recognition (Disclosure: Mobli’s public relations agency was helping to organize a Tel Aviv tech conference I was speaking at). He was fascinated by the idea that algorithms and technology were now at a level where the contents of images–rather than metadata such as the location and the time an image was taken–could be mined by startups. But EyeIn aren’t the only ones doing this.

EyeIn’s model uses visual recognition algorithms and computer vision to mine content for publishers. But another company, mathematician Stephen Wolfram’s Wolfram Alpha, unveiled a reverse image identifier last month that relies on many of the same concepts as EyeIn. The image identifier allows users to upload any photo, and then Wolfram Alpha–with an amazing degree of accuracy–will tell you what is in the picture.

Similarly, Google unveiled their new Google Photos app at the 2015 I/O conference, which also applies machine learning to photos. Type in broad identifiers like “Golden Retriever By Pyramids,” and Google Photos will mine it from the photo archive you upload. But unlike Mobli, Google is sitting on near-infinite cash reserves and has less motivation to monetize their image recognition technology.

Why Advertisers Care About The Logo On Your T-Shirt

Google Photos

A few weeks ago, I had the opportunity to speak with Yann LeCun, one of Facebook’s (and the world’s) top experts on machine learning and “deep learning”–the esoteric field of study which teaches computers how to categorize and comprehend everything from the contents of images to the nuances of a spoken conversation. LeCun was especially interested in image recognition, and the potential of computers to understand everything from X-rays to everyday street photography. He used Facebook’s facial recognition as a case study, but noted that the technique is becoming more and more commonplace.

It’s no surprise that Google unveiled their new Google Photos app, which features robust identification of people and objects within images, this year. The techniques behind image recognition and deep learning are become easier to leverage, and the significant barriers to entry into the sphere keep on becoming easier.

If Hogeg and Mobli play it smart, they’re likely to make a tidy profit on EyeIn. Meanwhile, it’s a safe bet to assume there are a host of competitors offering similar search engines for the contents of pictures on Instagram, Google, and Flickr flanking from the rear. Both Google and Facebook have far easier ways of making money than cracking open true search functions for Instagram and Google-indexed photos, respectively. But for smaller companies with good machine vision algorithms (or the cash to hire machine vision-savvy talent), it’s open season.