You are using an older browser version. Please use a supported version for the best MSN experience.

The boy who took the test

LiveMint logoLiveMint 11-06-2014 Dilip D’Souza

Chances are you haven’t heard of Eugene Goostman. Don’t lose sleep over it though. My guess is that his 15 minutes of fame are already done.

Goostman passed the Turing Test some days ago. More accurately, Eugene Goostman is the name given to a computer programme that people claim passed the Turing Test some days ago. To computer scientists, that is stunning news. If true, it would mean humankind can now build computers that have an innate intelligence.

Sadly, if you look beyond the hype, it isn’t true, and on various counts.

But first, who is Turing and what is this test that’s named for him? Alan Turing was a British mathematician who is usually described as the father of computers and artificial intelligence, the science of getting computers to think. He began a famous 1950 paper (Computing Machinery and Intelligence) with this sentence: “I propose to consider the question ‘Can computers think?’”

What his consideration amounted to is what we now call the Turing Test, and it goes something like this. Behind a screen sit both the computer that seeks to pass the test, and a human. You sit in front of the screen and type out questions for them. They respond, but you don’t know which answer came from whom. You try to divine that from the answers. If you can’t, and if several others can’t either, the computer passes the test. See my essay I took a Turing test and the test won (mintne.ws/1quoqgU) for more about this.

This is how Turing envisaged it in his paper: “I believe that in about fifty years’ time”—meaning 2000—“it will be possible to programme computers...so well that an average interrogator will not have more than 70% chance of making the right identification after five minutes of questioning.”

While this was simply Turing’s attempt to look into the future, it has turned into a line to be crossed when administering Turing Tests. Typically, several judges will take turns interrogating the computer on test. If less than 70% of them identify the computer correctly—that is, if 30% or more of them cannot tell computer from human—the computer passes.

This is what happened on 8 June at the annual Turing Test competition at England’s University of Reading. Goostman, a computer programme from Russia, persuaded 10 of the 30 judges—33%—that it was human. Thus the claim that Goostman has passed.

But did it? For one thing, Goostman was consciously programmed as a 13-year-old Ukrainian boy who speaks poor English. That automatically skews the test. After all, Turing mentioned humans, period, not humans limited in various ways. For another, Goostman is effectively what’s called a chatbot, designed to look for phrases in your questions and build reasonable-sounding answers around them. Whether this is intelligence, or whether intelligence is embodied in such a design itself, is a question worth asking. But it means two things.

One, there have been such chatbots before, for instance, Eliza, who dates from the 1960s. Goostman is really just a more sophisticated version of those, one that was lucky enough to fool 10 judges on 8 June. Two, just like Eliza, Goostman gives answers that would furrow most judges’ brows. He lives on the Internet at mintne.ws/1nx9IoX, and I spoke to him there. Here’s an excerpt from our chat.

So where are you from?

EG: I am from Ukraine, from the city called Odessa. You might have heard about it.

But you look Indian.

EG: Calm down, please.

I’m pretty calm, but I do think you look Indian.

EG: Yes I do. But better ask something else. By the way, what’s your occupation? I mean, could you tell me about your work?

No, let me ask the questions. Where do you suggest I take my friend who will visit next month from France?

EG: France? I’ve never there. But I do suspect that these crappy robots from the Great Robots Cabal will try to defeat this nice place too.

Hmm. Reasonable enough, until... Well, what’s all this about a cabal defeating nice places?

But in the end, the real story here is about intelligence. In 1950, we were just beginning both to use computers and grasp the idea of intelligence. Turing imagined an intelligent computer as one that would understand and participate in a conversation. Six decades later, we understand intelligence and computers better. So we can build a certain intelligence into places that will benefit from it. Think of driver-less cars, or voice-recognition systems, or even the way Google offers completions as you type what you want to search for. These targeted, specific systems won’t pass the Turing Test, but they are clever enough to make your life a little easier.

And maybe that’s a good way to consider artificial intelligence. But Eugene Goostman? Just more hype.

More From LiveMint

image beaconimage beaconimage beacon