Chatbots try to impersonate humans – and fail

By Mark O’Neill
Contributing Writer, [GAS]

Alan Turing once said that “if, during text-based conversation, a machine is indistinguishable from a human, then it could be said to be ‘thinking’, and therefore could be attributed with intelligence”.    This became known as the Turing test, and yesterday, five chatbots tried to pass the test.   They failed spectacularly.

Every year since 1990, the Loebner Prize has been held to determine if there is a computer bot out there which is indistinguishable from a human being.   The top prize is a gold medal and $100,000, but so far, there hasn’t been a grand winner. However, some people have won the bronze medal and $2000 for the prize of most human-like.

The winner yesterday was Elbot (picture above) and I’ve been trying him out.   He answered my questions quite well.   When I asked him if he had feelings, he responded : “I think I have feelings.   I cried when they killed the robot in Terminator“!

There’s no sound card in the website so everything is text-based.  The Loebner judges decided though that he was only 25% accurate.   So he didn’t even come remotely close to passing Turing’s test.

I think we can safely say that the possibility of something like Skynet, Terminator and cyborgs are probably a long way off.   I don’t see Elbot taking over the world, do you?


6 Responses to Chatbots try to impersonate humans – and fail

  1. I wouldn't be so sure. Back in the '50s some of the smartest scientists and science fiction writers thought that we wouldn't get to the moon for generations — but it was only a decade away. Sometimes a single big push or breakthrough will make all the difference.

  2. Hey, Chip, I'll be delighted to live to be proven wrong… but I've produced Silicon chips (I work at Intel) and I've studied some Neuroscience in my time… the human brain has some 100 billion neurons, with up to 10,000 synaptic connections (inputs) EACH, and all working in parallel. Current Microprocessor chips have 2-4 parallel cores, and around 100 million transistors. Long way to go…

    And of course the Turing Test is very aggressive – it suffices to trip the computer up on one single question…

    • But to pass, a computer does not have to mimic the human brain perfectly — it just needs to be able to fool one. We humans are pretty easily duped. For millenia we thought rocks and trees were possessed of humanesque intelligence. We find it comforting to project anthropomorphisms onto non-human agents, so we're likely to help the Turing test winner along, whether we mean to or not.

  3. I had to go try Elbot out — I was disappointed that I was so easily able to stump him. Just ask "which foot do you have forward right now?" He'll try to change the subject every time. If you do it enough times, though, he replies, "I'm afraid you're in an endless loop. Maybe you should re-boot yourself."

  4. I'm in computer support. I've worked with personal computers since before Windows and DOS. (This just to establish my credentials). About 10 years ago I was at a home show and came across a booth selling the Encyclopedia Britanica (20 printed volumes). I told the salesman I would wait for it to come out on a disk. The salesman said "Not in our lifetime!" I said "5 years". You can now buy the Encyclopedia Britanica on a DVD. This is leading up to my opinion that a computer that can pass the Turing test will happen in less than 10 years, likely less than 5 years.

    David K.