The Search for the Fastest Engine

The Unbranded Brand Alltheweb.com is already a hit among scientists, librarians, and researchers. And it gives FAST a working laboratory where Lervik and his team can test-drive new tools and techniques. But when it comes to building the Alltheweb.com brand, Lervik is clear about his company's course. "Google is trying to leverage its brand and become a portal," he says. "We are so technology focused that even if we tried to be consumer focused, we couldn't be."

The difference for FAST is in its business model. While rivals like Google remain reliant on advertising sales, FAST harvests revenue in two ways: by powering Internet search services on behalf of portals like Lycos, which pays FAST a fee for each query served, and by selling software licenses to enterprise customers such as Dell, Ericsson, FirstGov, IBM, Reed Elsevier, and Reuters, which want FAST search technology for their intranets and e-commerce sites. According to Lervik, Alltheweb.com is deliberately starved of PR so that it will not compete with these partners. In fact, working closely with its partners brings FAST other benefits. "Around half of our product road map comes from working with partners," says Lervik.

FAST Recovery FAST's rapid rise is all the more remarkable given that just a couple of years ago, the company nearly ran off the road, following an ill-fated stab at a Nasdaq listing. When FAST realized that a 2000 IPO was futile and that the company had expanded into too many noncore areas, one-third of FAST's 270 then-employees lost their jobs -- including the CEO who had been recruited to handle the listing. Lervik, who had been FAST's chief technology officer, moved into the role of CEO. Among his key decisions: calling a halt to FAST's peripheral activities in bioinformatics and image compression. Alliances with some of Europe's biggest portals -- Freeserve, Tiscali, and T-Online -- have propelled FAST to leadership as the continent's number-one search engine. But to challenge Google for overall leadership, Lervik says, FAST must snag one of the big-three portals in the United States.

AOL has just signed with Google; MSN seems content with Inktomi. In September 2002, however, Yahoo's contract with Google was up -- but at press time, it was unclear whether FAST was ready to compete for and win that prize. But Lervik is clear about what he has in mind for FAST's overall future: Having grown up in the fjords and mountains of western Norway, Lervik, an expert cross-country skier, is building his company for stamina as well as speed.

Sidebar: Under the Hood

In its race with Google, FAST is betting the future on its own architecture. Like Google, FAST uses patented algorithms and computer linguistics to search and retrieve. But while both search engines may use the same components, the difference is in how they perform when combined. The real difference, says Danny Sullivan, editor of searchenginewatch.com, is that FAST searches are performed on a core architecture that is infinitely scalable to any amount of data or number of users.

While other search engines use large, expensive multiprocessor computers, FAST queries whiz along a distributed network of search-and-dispatch "nodes," making more-efficient use of existing CPUs. The result: FAST can process a larger number of queries on a smaller number of servers.

Not only is FAST's system more scalable, but FAST's unique architecture also delivers a faster search: A typical query on FAST races through 300 million documents in less than one second.