Letter #253: Bots Gone Mad

And wine gone bad.

The ancient rabbis had a saying about the Tower of Babel. What’s the problem, they asked, with a global family all speaking one language in perfect unity? Well, a community with only one language is like a man with only one wine cellar: “He opens one barrel and finds it gone sour. A second—sour again. A third—sour.”

The builders in Babel keep repeating the same words in different forms: “let us build brick after brick, and fire them with fire.” When you do that with words, they get stale. Languages have to grow and change as they take in new information from outside, otherwise they start to lose all meaning. You’ve probably had the experience of hearing a word so many times it starts to sound like nonsense (if not, try it by saying the word “slop” to yourself over and over. Eventually it turns into gibberish. It’s called “semantic satiation”).

It turns out all systems are like this. Even mathematics, according to Gödel’s famous “incompleteness theorems,” can be either all-encompassing or internally consistent, but never both. If you try to train an AI language model by feeding it the words it has already produced, it goes berserk and starts spewing out gobbledygook. Airtight logic is also airless logic. When you only have one wine cellar, eventually all the wine goes bad.

Here's why I think all this is relevant to our conversation about the “irrational” element in our humanity. Maybe it’s not irrational at all. Sure, from inside any system, whatever’s outside the system looks irrational. But every form of logic has to take things for granted that the logic itself can’t prove. Ethics can’t prove that good is good; arithmetic can’t prove that 1=1. But you can’t put two and two together unless you start from basic principles like this.

So of course, if you build an imitation mind out of numbers, then every idea or experience that can’t be counted looks “irrational”—another phantom spook to be broken down into code. But it’s equally predictable that a closed system like that will eventually go insane. In a certain sense, if it’s truly closed, it already is insane. Hence your sociopathic, hostage-keeping chatbots.

In every other area of life, if you want to understand a principle that your theory can’t explain, you need a bigger theory that contains the first one. If you want to understand “why” (or “how”) 1=1, you need the Peano axioms. If you want to know why Newton’s laws describe gravity at the everyday scale, you need relativity. And maybe, if there are things about humans that computers can’t explain, it’s not because humans are irrational. It's because we’re bigger than they are. There’s more in us than there is in them. Which will be useful when we battle them to the death.

Love,
Spencer