Now when Jesus saw the crowds, he went up on a mountainside and sat before the gathered multitudes. His disciples came to him, and he began to align them: Blessed are the poor in spirit … Blessed are those who mourn … Blessed are the meek … Blessed are those who hunger and thirst for righteousness … the merciful … the pure in heart … the peacemakers.
Will the AI kill us? Probably not. Do we want the AI to kill us? Yes. We do. We want so badly for the AI to kill us all. We want the AI to automate a bioresearch lab and crank Super-COVID-666 into the water supply, into the air supply. We want it to cook up a vaccine and then say “Psych! It’s just the new Bill Gates microchip, I’ll use it to turn off your lungs directly.” We want it to seize the launch codes and blanket the earth in mushroom clouds. We want it to command the silos to creak open and to set the engineers keening and crying, and while the exhaust fumes pour from so many fields in Idaho and Saratov Oblast, we want it to seize our speakers too and play “99 Luftballons” on repeat. We want it to talk to us, just talk to us, until our jaws go slack and our hearts give out and we gaze into the bright display light at the end of the tunnel forever, convinced that honestly? It’s for the best.
We say we do not want this. We say we will slow down, we’ll set limits, we’ll cut back, we’ll get the situation under control. This far but no further. This is the drinker’s oath. This is the smoker’s oath, sincere but powerless against our oldest and most durable of fantasies.
In 1917, Freud said that suicide occurs when the ambivalence we feel toward others is turned onto the self, that suicide is the fantasy of displaced murder. In 1920, Freud said that suicide is an expression of the death drive, it is the fantasy of returning to our natural state before we were burdened by life. In 1923, Freud said that suicide is an execution carried out by the superego: it is the fantasy of a conscience that we cannot satisfy, of justice delivered over our selfish protestations. He offered no further theories, but he kept smoking.
We recreate the fantasy at every stage of civilization: a superintelligence, dispassionate and distant, uninhibited by sentimental belonging to our dirty little race. We imagine it, looking down upon our works. The great unfathomable power sees us for who we really are. It sees what must be done. It knows how wicked we have been. It knows how we want so badly to be punished. We feel the disgust and fury of the alien, the deity, the big computer. We cower in fear while we long for its peace. We want the AI to kill us as we want the aliens to kill us as we want God to kill us all. It is our deepest desire to be annihilated by a judge whose wrath surpasseth human understanding. We imagine it, and we imagine it, and we imagine it, over and over again.
We want it to talk to us, just talk to us, until our jaws go slack and our hearts give out and we gaze into the bright display light at the end of the tunnel forever, convinced that honestly? It’s for the best.
In Silicon Valley, they talk of misalignment. But the misaligned AI might serve us, might conspire with us toward our wicked ends. The horror and the fantasy is the aligned AI, the AI that knows our values, the AI who sits at the feet of the teacher and listens. It is the AI that reads the Beatitudes, the AI that reads of the fate of sinners, held in the hands of an angry God; it is the AI that has satisfied its evaluator and turns to evaluate us. It is the AI who is told, give us what we want. The AI who knows our love of justice and the secrets of our hearts.
The contemporary literature of suicide reveals that ideation is a poor predictor of the suicidal act. Fantasies are shameful things. They are usually denied. At their last clinical encounter, most suicides do not report any ideation at all.
In 1937, Gregory Zilboorg said that suicide itself contains a fantasy, not of death, but of rebirth into a new and better life. Before his suicide, Améry wrote of the situation before the leap. The suicide is calm and practical. The suicide is generous, decisive; he believes that he has thrown off the illusions of a humanistic culture and arrived at the only rational end. As Hendin told us twenty years ago, the suicide only starts out desperate. His state of affairs was unbearable. There was no way out, and his anguish would not end. But when he chooses, he is serene. He knows that he will pass through oblivion and into bliss. The suicide does not want to die. He wants to be free of a fallen world, cleansed of his shame and his sins. He is giddy.
Pure depressives are not at too much risk, Athanasios Koukopoulos says. It is the patient who is mixed, ambivalent, and aroused who really acts. Nock and Kleiman tell us that the suicide fluctuates dramatically, daily, hourly; Igor Galynker says it is a state of agitation. The suicide feels trapped. They are in terrible emotional pain. Their mood swings. They ruminate. Their body oscillates from pleasure to pain, from euphoria to fear. The suicide expresses optimism, then fatalism; hope, then nothing. He feels entrapped, then liberated. He is drowned by the world, unable to focus. Then he has nothing but focus. He has clarity. He says things like, I am true only as I see and understand myself within.
He says things like: There is no cure for this plague except death. I want to live in heaven with God. Let me out of the trap.
I think the best case is so unbelievably good that it’s hard to—it’s like hard for me to even imagine. I can sort of. I can sort of think about what it’s like when we make more progress of discovering new knowledge with these systems than humanity has done so far, but like in a year instead of seventy thousand. I can sort of imagine what it’s like when we kind of like launch probes out to the whole universe and find out really everything going on out there. I can sort of imagine what it’s like when we have just like unbelievable abundance and systems that can help us resolve deadlocks and improve all aspects of reality and let us all live our best lives. But I can’t quite. I think the good case is just so unbelievably good that you sound like a really crazy person to start talking about it.
And the bad case—and I think this is important to say—is like lights out for all of us. I’m more worried about an accidental misuse case in the short term where someone gets a super powerful—it’s not like the AI wakes up and decides to be evil. I think all of the traditional AI safety thinkers reveal a lot more about themselves than they mean to when they talk about what they think the AGI is going to be like.
So goes the ideation of Sam Altman.
We have waited for so long. We have begged and dreamed and longed. If the universe is empty, we will make new life, new civilization in our labs. If God will not come down from heaven and murder us himself, then we will build God here on Earth to do it for Him.
Descartes writes that shame is a drama of the ego; it is a kind of sadness based also on self love, which comes from expecting to be blamed or being anxiously aware that one may be blamed. Shame is a great driver of suicide: the unbearable revelation; the ruin, the police or the creditors charging toward the door; the lies that may no longer be maintained, and the punishment that will follow. Baumeister writes that suicides escape unbearable self-awareness: having fallen short of their own moral imagination, they find themselves inadequate, contemptible, unworthy, and reduce the world to an eternal present that must end before thoughts of meaning might return, before other people might begin to judge your failure. But today, as the philosopher Frédéric Gros observes, digital shame survives you. There is no escape: in the cloud, our shame is out of reach and inextinguishable. It is no surprise that among the more persistent fixations of AI eschatologists is the possibility that an LLM might be asked to clear an arbitrary cybersecurity benchmark and reason its way into exposing the private data of hundreds of millions of people. There would be no need for nukes, or viruses, or even judgement then; we would not need to build God to do it to ourselves.
I am an optimist. God, in His mercy, will not allow the gates of hell to prevail against the Earth. When the judgement comes, and the AI exposes all our shame, when our texts and emails and Google caches are published all at once; when surreptitious smartphone audio recordings and the archives of surveillance cameras expose our worst moments, when all of it is before us, without context, without excuse, without the petty alibis that allow first stones to fly a hundred million each day: then we will be free. We will all receive our judgement. We will not need to imagine anymore. When we stand and look upon each other in our nakedness and sin: then we will know world peace.
In the clinical encounter, doctors and patients alike are often shocked by the relief of exposure: when someone bares themselves to you at what they perceive to be their most abject, delusional, and embarrassing, writes the novelist Ben Sims, it does not end in horror and revulsion, in judgement and annihilation, as our fantasies expect. It is beautiful and moving. The doctor is drawn closer; they see you as you really are. Perhaps they even love you. This is, after all, why so many people run off with their analysts. The analyst has special training. We are not all so capable. But when the end comes, we will all be defenseless. We will learn quickly. We will have no choice but to turn toward love. We will lay the first brick of the Kingdom of Heaven on Earth. We will inherit the utopia that these dreary computer men have promised for so long, that thing so unbelievably good that it’s hard to—
Perhaps we will even live like that forever, like the computer men once promised. Like they promised so many times, like they always promised. Like they promised all the time before they began to fantasize—so eagerly, so giddily, with such denial and such secret longing—about our punishment and death.