Why Watson Wagered $947, and Other Intel on the Jeopardy Supercomputer

In case you haven't heard, computers inched a bit closer to besting humanity this week when IBM's Watson supercomputer handily beat opponents Ken Jennings and Brad Rutter in a three-night Jeopardy tournament. Last night, Fast Company had the chance to watch the final match with John Prager and Burns Lewis, two of the IBM researchers responsible for the prowess of Watson, a device capable of natural language processing, knowledge representation, information retrieval, and reasoning algorithms. This is what we learned.

  • On night two of the match, Watson incorrectly answered "Toronto" to the Final Jeopardy question of which city is home to airports named for a World War II hero and a famous World War II battle. (Answer: Chicago.) There are multiple explanations for this. Watson had learned in general that the category (in this case, U.S. Cities) is not necessarily a reliable determinant of the answer. There are multiple towns named Toronto in the U.S, the Toronto Blue Jays play in the American league, and Toronto is an "American-style city," explains Prager. In any case, Watson's confidence in its answer was very low. In other words, the computer knew that it might be wrong. That's why Watson bet just $947.
  • Watson used 3,000 parallel IBM Power7 processors in its Jeopardy run—a feature that allowed the supercomputer to respond to queries in three seconds. Before installing the processors, Watson took up to two hours to answer a question. Now Watson takes three seconds to answer a question no matter what it is, so in many cases, humans have the advantage of speed.
  • Watson's mechanical hand takes between six and 10 milliseconds to push down the Jeopardy buzzer. Humans can push down the buzzer in as little as zero to one milliseconds.
  • Watson's quirky Daily Double and Final Jeopardy wagers—$347, $947, $17,973—are a goof from IBM. "The guy programming it thought it would be boring [to end wagers with a 0]," Prager explains. So the programmer ever-so-slightly tweaked Watson's wagers "to give Alex Trebek something to talk about."
  • Watson wasn't connected to the Internet during its Jeopardy run—instead, Watson downloaded  "textual resources" a few weeks beforehand. This ensured that Watson wouldn't be a cultural laggard on the show. "We knew the clues had been written some time ago," Prager says.
  • In one instance during the Jeopardy run, Watson repeated a wrong answer from another contestant. IBM knew that might happen, but thought the chances were statistically insignificant enough to ignore the issue. The fact that Watson came up with the same wrong answer as a human was just bad luck.
  • At certain points during the match, Watson jumped around to different categories in a seemingly random fashion. At other points, the supercomputer stuck with single categories. The reason? Watson was trying to find Daily Doubles.

Follow Fast Company on Twitter. Ariel Schwartz can be reached by email.

Add New Comment


  • Chris Reich

    The article is an interesting peek into how we think we think, don't you think?

    When asked why computer software manuals were so huge (before they quit producing manuals), I used to have people do an exercise of writing out all the instructions necessary to start their car and drive to work. They soon learned that it is difficult enough to express in writing how to simply start a car to someone who had never seen a car.

    Chris Reich

  • William Dais

    Excellent point! If the wager had to be made prior to being fed the question, how could Watson anticipate his odds of knowing the answer?

  • George Bush

    Re: bullet 1, attempting to explain why Watson wagered $947 on Final Jeopardy, a relatively low percentage of his accumulated total score at that point. Low confidence in his answer can not be the reason. All players must lock in the amount of their wager after the Final Jeopardy category is revealed, but before the clue is revealed. Being able to place a wager after the clue is revealed would completely change the strategy. Thus, Watson's low wager must have been for strategic reasons, but is still surprising, as it must have a large calatog of information about the category, "U.S. Cities", so it would be expected to be very confident about its ability to find a correct response, particularly given the longer time period allocated to respond to the Final Jeopardy clue (30 seconds vs. 5 seconds for a "ringing in" clue). In turn, it would be expected to make a much larger wager.

  • aaron@omnificventures.com

    I second that.

    It raises some questions. Similar questions have been raised about IBM's chess match as well. I'd very much like to know what happened, and why it did what it did.

    Did it have the question before wagering? I never watched Jeopardy and I don't know all the rules to this match, but if it did have to have its wager locked in before getting the question... something interesting happened.

    Who really knows though. Maybe it was just lucky tactical programming.

  • This question was supposed to be more complicated, exceptional in some way. Watson is optimized to answer typical questions, so the failure was more probable here. It was rational to minimize risk and just wager something not to spoil the game.