advertisement
advertisement

Innovation Wednesday: How eBay Searches and Updates 100 Million Listings in 60 Seconds

As I described in the magazine recently, eBay is pursuing a new strategy, catering to buyers instead of sellers. It’s the biggest and most urgent change to the site in years.

As I described in the magazine recently, eBay is pursuing a new strategy, catering to buyers instead of sellers. It’s the biggest and most urgent change to the site in years.

advertisement

Although the number of listings and gross merchandise sales have continued climbing, buyer activity has slowed down dramatically. Despite the mind-blowing number of registered users worldwide – 222 million, or the equivalent of three-quarters of the U.S. population – only about a third bid, bought, or listed an item in the previous 12 months. Those active buyers increased at the smallest rate in four years, a dangerous trend.

eBay has been overhauling its site to make shopping easier and enjoyable, with features such as window-shopping, auction-countdown, and eBay-to-Go. But one of the keys to improving the overall experience remains behind the scenes: the search technology.

Meet Randy Shoup, the main architect for search infrastructure at eBay.

Search infrastructure may not sound exciting, but it certainly is to him. Shoup comes across as someone who can’t believe his good fortune, like a pilot who finds himself in the cockpit of the Boeing 787 Dreamliner. “There are only a handful of e-commerce sites that operate at this level,” he said. “Yahoo, Amazon, Google, and eBay.”

The search challenge for eBay is not only indexing and searching an enormous database, but keeping up with the latest changes to that database – about 1,000 per second. A bid. A purchase. A new listing. The inventory and prices are in constant flux. Now you see what’s exciting about search infrastructure?

advertisement
advertisement

Several years ago, eBay featured 12 million listings on average. Now it’s up to 100 million listings. iPhones. Used cars. The new Madden. DVDs on the 1964 New York World’s Fair (more on that later). In the old days, it would take as long as nine hours for a new item to appear in a search, not the sort of time lag that does sellers or buyers any good. “It was pretty unpleasant,” says Shoup, “but now it takes only a minute or two for a new item to show up in the search engine.”

How do you search 100 million items and 60,000 changes per minute in such a short time? The same way any team tackles a big job: by breaking it down. Instead of searching the entire database at once, eBay looks at nearly two dozen slices. Conducting smaller searches simultaneously at data centers in San Jose and Phoenix and elsewhere is much faster. The trick is then combining those sets of results to look like a single query.

And that is how I did part of my Christmas shipping this year. I searched eBay for DVDs showing historical footage of the 1964 New York World’s Fair, my parents’ first trip to New York. When they opened the DVD, they gave me one of those priceless “Where in the world did you find this?” looks.

I’d love to take the credit. But it wasn’t me. It was Shoup and eBay’s algorithms.

advertisement

About the author

Chuck Salter is a senior editor at Fast Company and a longtime award-winning feature writer for the magazine. In addition to his print, online and video stories, he performs live reported narratives at various conferences, and he edited the Fast Company anthologies Breakthrough Leadership, Hacking Hollywood, and #Unplug.

More