In 2007, IBM scientist David Ferrucci and his team embarked on the challenge of building a computer that could beat the best players of the popular US TV quiz show Jeopardy!, a 25 year old trivia game in which contestants are given clues in categories ranging from academic subjects to pop culture and must ring in with responses that are in the form of questions.
In some sense, the project was a follow-up to Deep Blue, the IBM computer that defeated chess champion Garry Kasparov in 1997. Although a TV quiz show may seem to lack the gravitas of the classic game of chess, the task was in many ways much harder. It wasn’t just that the computer had to master straightforward language, it had to master humor, nuance, puns, allusions, and slang—a verbal complexity well beyond the reach of most computer programs. Meeting that challenge was about much more than just a Jeopardy! championship. The work of Ferrucci and his team illuminates both the great potential and the severe limitations of current computer intelligence—as well as the capacities of the human mind. Although the machine they created was ultimately dubbed “Watson” (in honor of IBM’s founder, Thomas J. Watson), to the team that painstakingly constructed it, the game-playing computer was known as Blue J.
Last month, Watson ultimately proved to be too much for the humans in Jeopardy. Ken Jennings and Brad Rutter, two of the most successful Jeopardy players, put up a spirited fight, but couldn’t handle the IBM supercomputer. The two day totals highlighted the extent of the victory for Watson which had $77,147 while Jennings had $24,000 and Rutter had $21,600. In the end, Watson was a natural on Jeopardy. It (I am tempted to say he!) was even likeable with his quirks, very precise wagers and a robotic voice. “Unlike us, Watson cannot be intimidated. It never gets cocky or discouraged. It plays its game coldly, implacably, always offering a perfectly timed buzz when it's confident about an answer.” Said Ken Jennings, “But there's no shame in losing to silicon, after all I don't have 2,880 processor cores and 15 terabytes of reference works at my disposal. My puny human brain, just a few bucks worth of water, salts, and proteins, hung in there just fine against a jillion-dollar supercomputer.”
But what does Watson’s landslide victory really mean? On the one hand someone might consider it to be an indication that machine is becoming smarter than man. Of course that is not the case, this is a massive machine with thousands of processes and an unbelievable amount of pre arranged and stored data designed by man. On the other hand some might consider this to be just a purpose built piece of technology that won’t go beyond the purpose it was built for which is winning the Jeopardy quiz show. That too isn’t accurate because Ferrcci’s team had to develop and build complex artificial intelligence programs that can and will be reusable in many commercial applications from healthcare to providing legal advice. "We're moving beyond Jeopardy!" said Ferrucci. "With the Watson technology, we're going to look at creating a commercial offering in the next 24 months that will help empower doctors to do higher quality decision making and diagnoses."
Despite the Jeopardy win and the promise of Watson, Ferrucci is careful to point out that his creation is still no substitute for human decision making."With Jeopardy! these are human questions written for humans, whereas all the computer has are words. It can't rely on human context to determine things, when you deconstruct this, and look at the machine, is any part of this really understanding the question? No." Blue J’s literal-mindedness posed the greatest challenge. Finding suitable data for this machine was only the first job. Once Blue J was equipped with its source material the IBM team would have to teach the machine to make sense of those texts: to place names and facts into context, and to come to grips with how they were related to each other. Hamlet, just to pick one example, was related not only to his mother, Gertrude, but also to Shakespeare, Denmark, Elizabethan literature, and themes ranging from mortality to self-doubt, just for starters. Preparing Blue J to navigate all of these connections for virtually every entity on earth, factual or fictional, would be the machine’s true education and the team’s true miracle. The process would involve creating, testing, and fine-tuning thousands of algorithms. The final challenge would be to prepare the machine to play the game itself. Eventually, Blue J would have to come up with and formulate answers it could bet on within three to five seconds and then use its robotic arm operate the buzzer.
How does Watson “Think”
Consider one of the thousands of clues the system grappled with. Under the category Diplomatic Relations, one clue was: “Of the four countries the United States does not have diplomatic relations with, the one that’s farthest north.”
First a wave of grammar specialized algorithms analyzed the sentence identifying the nouns, verbs, direct objects, and prepositional phrases. This analysis helped to clear up doubts about specific words. In this clue concluding “the United States” referred to the country, not the Army, the economy, or the Olympic basketball team. Unlike humans, who can instantly understand a question and pursue a single answer, the computer might hedge, launching searches for hundreds of different possibilities at the same time. In this way and many others, Blue J would battle the efficient human mind with spectacular, flamboyant inefficiency.
Another set of algorithms searched for the focus, or answer type concluding that in this clue about diplomacy, “the one” evidently referred to a country. If this was the case, the universe of Blue J’s possible answers was reduced to a mere 194, the number of countries in the world. This, of course, was assuming that “country” didn’t refer to “Marlboro Country” or “country music.” Blue J had to remain flexible, because these types of exceptions often popped up in Jeopardy!
Once the clue was parsed into a question the machine could understand, the hunt commenced. Expert algorithm looked to match strings of words in the clue with similar strings maybe in some stored Wikipedia entry or in articles about diplomacy. The computer had to come up with the four countries with which the United States had no diplomatic ties. Then it had to figure out which of those four was the farthest north. A group of Blue J’s programmers had recently developed an algorithm that focused on these so-called nested clues, in which one answer lay inside another. It broke the clues into two questions, pursued two hunts for answers, and then pieced them together. It found the four countries (Bhutan, Cuba, Iran, and North Korea), checked their geographical coordinates, and came up with and formulated the answer: “What is North Korea?”
At this point, Blue J had the right answer. But the machine did not yet know that North Korea was correct, or that it even merited enough confidence for a bet. The machine would proceed to check how many of the answers matched the question type: “country.” After ascertaining from various lists that North Korea appeared to be a country, confidence in “What is North Korea?” rose further up the list. For an additional test, it would place the words “North Korea” into a simple sentence generated from the clue: “North Korea has no diplomatic relations with the United States.” Then it would see if similar sentences showed up in its data trove. If so, confidence climbs even higher. In the end, it chose North Korea as the answer to bet on.
But that is not all, elsewhere in IBM’s Almaden Research Center, Dharmendra Modha is building a simulated brain equipped with 700 million electronic neurons. Within years, he hopes to map the brain of a cat, and then a monkey, and, eventually, a human. But mapping the human brain, with its 100 billion neurons and trillions or quadrillions of connections among them, is a long-term project. With time, it might result in a bold new architecture for computing, one that could lead to a new level of computer intelligence. Perhaps then, machines would come up with their own ideas, wrestle with concepts, appreciate irony, and think more like humans.
What really differentiates IBM is not its ability to build the fastest computer or to develop the next version of Tivoli but its deep rooted ability and determination to build its leadership on investing in basic research that may not produce commercial results for many years, the type of research you would normally expect from university research centers. For many years IBM has been registering and receiving more US patents than any other company in the world.
Whether it is the hard disk or the barcode or the PC, IBM's innovations are all around us. Now you know who will make our planet smarter!