A new study offers a peek behind the scenes at how Google structures its search results. The study focuses on how Google’s RankBrain algorithm, which was first announced in October 2015, parses the English language. It’s one of the most detailed efforts to understand the algorithm to date.
“Google improved in about 55% of the queries that they didn’t understand back in July of 2015,” study author Eric Enge told Fast Company. “Honestly, I think that’s pretty amazing.”
Enge, who works for marketing consulting firm Stone Temple, focused on how RankBrain works compared to other Google machine learning products. He then made inferences on RankBrain’s behavior and results, which is something Google has not extensively discussed publicly.
In order to conduct the study, Stone Temple compared a sample set of 1.4 million pre-RankBrain queries to Google’s current search engine. They then analyzed a small remnant of search queries from the older set for which Google didn’t provide appropriate results.
After the launch of RankBrain, 54.6% of search queries that previously returned irrelevant results began returning appropriate results.
Some of these hard-to-resolve searches included queries like “What is low in the army” (Where the searcher is believed to have been searching for “what is low rank in the army”) and “Why are PDFs so weak” (Which, in the older version, first showed PDFs with the word “weak” rather than results about the security of PDF files). Stone Temple also saw what appear to be improvements surrounding specific phrases like “What is,” “Who is,” and “Where is.”
“We also found certain specific classes of phrases they handle better,” Enge said. These certain misspellings, such as when users misspell “Qatar” (the country) as “Cutter.”
One of the things Enge emphasized in the report is that he believes RankBrain has a negligible effect on SEO. He writes that it “simply (does) a better job of matching user queries with your web pages, so you’d arguably be less dependent on having all the words from the user query on your page.” The biggest changes from RankBrain, he added, have to do with increasing search quality and creating a likely framework for Google to apply further machine learning improvements to its search engine.
Meanwhile, there’s one big challenge for developers, engineers, and marketers seeking to understand how RankBrain–and Google’s search engine in general–works: Google doesn’t discuss the topic much in public. Google announced RankBrain through an interview with Bloomberg late last year, with senior research scientist Eric Corrado saying that it became the third most important signal contributing to the result of a search engine query.
The first and second signals, of course, are still unknown.