Friday, January 14, 2011


Watson and Jeopardy!

Today, the folks at Jeopardy! will be recording the competition, to be aired on 14-16 February, between IBM’s Watson computer and two of the game’s biggest champions, Ken Jennings and Brad Rutter. I’m told that the Watson Research Center lab is closed to employees today, and that employees were asked to work from home or make other working arrangements for the day.

They did a practice round that Watson won, and you can watch some video of that in the ZDNet article. In that round, no one answered any questions wrong — it will be interesting to see how it all works out when the errors start coming in — and it looks like Watson has an edge on the buzzer timing.

Some observers are not impressed by all this. One commenter to the ZDNet article says, This is not progress. I’ve talked with others who think the whole thing is a WOMBAT.[1] And, indeed, one has to wonder about an expenditure of a million dollars on a replica Jeopardy! set (according to CNN Money).

But, of course, they want to make a spectacle of this, just as they did with Deep Blue and Garry Kasparov.

Spectacle aside, though, is this just a silly waste? We’ll have to see what comes of the technology after the Jeopardy! match. It’s not directly clear what IBM did with the technology that went into Deep Blue, but it’s unlikely that the technology that has gone into Watson will languish. If all these projects do is produce machines that can play chess or Jeopardy!, then, indeed, they’re wasteful, no more than novelties.

But, surely, technology that can understand human-language questions and answer them has many practical uses. Such a system could be a useful front-end to many systems that have to direct people to the right experts, diagnose problems, and answer common questions. Of course, on the other side, many of us might find ourselves more frustrated than we are already, when it becomes even harder to get a real human on the phone.

Though, might we be getting closer to passing the Turing test? Perhaps before too long we won’t be able to tell whether we have a real human on the phone or not. And if that computer, Watson XVII perhaps, can answer our questions and give us a smooth and pleasant experience in the process, does it matter?

It’s clear that, while word processors and spreadsheets are useful, it’s games that have really pushed and expanded the limits of technology. 3-D graphics rendering, hand-held motion sensors, and even parts of the underlying network technology are where they are because of games. If we take advantage of where the games move us and use the technology beyond the realm of entertainment — by, say, rendering images of heart scans in 3-D to give doctors diagnostic capabilities that our parents’ doctors couldn’t even dream about, and allowing them to perform surgery with amazing levels of precision — then what we spent on the frivolity of the games was well worth it.

So let’s see what’s next for the Watson technology after Jeopardy!

We’ve come a long way since Eliza.

[1] WOMBAT = Waste Of Money, Brains, And Time


The Ridger, FCD said...

Watson will certainly be able to buzz in faster. I wonder about the questions - often if the question were asked normally (What is the highest mountain in Australia?) I'd have no idea, but Jeopardy! questions tend to be more like "This highest mountain in Australia was named for a Polish general who fought in the American Revolution" and that makes them much easier, if less straightforward. If Watson can learn to deal with that kind of language, it will be a breakthrough.

And lead to fewer actual people with jobs, I fear. Unintended Consequences...

Katharine said...

What intrigues me is finding out which items Watson misses. Perhaps those are key to discovering the difference between between mere memory/grinding through data and higher-level thinking (even puns).

E.g., some of the garbage that Google Translate spews when presented with any but the simplest prose -- especially hilarious when retranslated into the original language -- demonstrates that computers still have a long ways to go to match the human brain.

Katharine said...

Did you or Ridger happen to hear this report on NPR's "All Things Considered" this afternoon?

"Google's Artificial Intelligence Translates Poetry":

I suspect Ridger and I still won't be replaced any time soon :-)

The Ridger, FCD said...

Check out Cuttlefish's take on the poetry translations:

Katharine said...

Barry, thanks so much for the link (which I've already forwarded to an interested party)! Cuttlefish's poetic take on computer translation of poetry is inspired.