When an MIT professor called Joseph Weizenbaum created one of the first chatbots in the mid-1960s, he received two shocks in quick succession. The first was how readily people anthropomorphised the rudimentary program, which he called Eliza. “What I had not realized is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people,” he wrote in his 1976 book Computer Power and Human Reason.
But something else troubled him even more: how swiftly people began to envisage what technology like this could do one day, without pausing to ask what it should do. In 1966, for instance, researchers at Stanford University wrote a paper that envisaged using a computer program similar to Weizenbaum’s as a form of psychotherapy. “Because of the time-sharing capabilities of modern and future computers, several hundred patients an hour could be handled by a computer system designed for this purpose,” they wrote.
Weizenbaum was appalled. He thought there were “some human functions for which computers ought not to be substituted. It has nothing to do with what computers can or cannot be made to do,” he added. “Respect, understanding, and love are not technical problems.”