Man, Machine, and Jeremy Bentham’s Head

When you grow up in Western New York, as I did, you learn two things about the Niagara River. First, it’s great for swimming and boating. Second, don’t get caught in the current. Past a certain point, the water accelerates very rapidly. So does the turbulence. So does the noise of the Falls. The result is panic, paralysis, and a very unpleasant trip over the brink. So a little caution—or a lot, even better—is good for the health. The same should apply in our approach to AI and its uses.

On the matter of artificial intelligence, there’s good news and other news. In fact, there’s more news, more quickly, with more noise and confusion, than anyone can fully digest. But we nonetheless need to try.

First, the good news. Scripture tells us dozens of times to “be not afraid.” That’s not a suggestion. It’s a command. There is a God; he does love us, and he’s intimately involved in our affairs. Words like “apocalypse” and “crisis” pepper our public discourse about AI, and they have a boogeyman effect that invites unwarranted fear. But in the original Greek, apocalypse simply means “disclosure”; the revealing of some fact or truth previously unseen. And “crisis” comes from the Greek word krinein, which means to decide or judge. There’s nothing fated or doomed or unavoidably catastrophic about the rise of AI. AI is a tool; a very powerful and problematic tool, but a tool nonetheless. God made us rational creatures with free will. Which means we need to engage AI with the wisdom of experience and the prudence of clear and morally informed thinking. And there’s at least some evidence that’s happening.

Back in July, my wife and I got a call from a longtime friend who’s an executive with Google. The friend asked us for a list of Christian and other religious thinkers who could serve as ethical advisers in the product development of Gemini, Google’s search agent, similar to ChatGPT and Claude. Just five years ago, that kind of request for help from anywhere in Silicon Valley would have been inconceivable. The same thing is happening at Anthropic, OpenAI, and across the industry. Tech leaders are realizing that they have a tiger by the tail. It could easily eat all of us. So they want help in shaping its character. They’re feeling the heat of an apprehensive public. And they need to prove good intent.

This was obvious at a closed-door, Vatican-related conference I attended in Rome in early September. The centerpiece speaker was the cofounder of one of the nation’s (and world’s) largest AI tech companies. He came to the meeting precisely for the contacts and conversation, and he engaged accordingly. Which means that people of faith–Catholic and otherwise–have an exceptional opportunity right now to help maximize the good and minimize the dangerous in AI development. It may not last. But it’s real for the moment.

Quite apart from Pope Leo’s recent encyclical, Magnifica Humanitas,—and its related, preceding Roman documents, Antiqua et Nova and Quo Vadis, Humanitas?—the Vatican has an active, ongoing Commission on Artificial Intelligence. The U.S. bishops now have a “Task Force on Preserving Human Dignity in the Age of Artificial Intelligence.” Both have promise. Philosophers and ethicists like Father Paolo Benanti at the Vatican, Father Philip Larrey at Boston College, and Prof. Charles Camosy at CUA—along with many others—are already involved with government and tech leaders on AI and its implications.

All of these efforts have the potential for positive impact. AI’s beneficial value for medicine, education, aid for the disabled, and research in nearly all fields could be enormous. But the real work to redeem the emerging character of AI is happening at ground level among thousands of young Christian scientists, techies, ethicists, and entrepreneurs with energy and good ideas. And they come together very fruitfully on platforms of mutual support and exchange like the “Builders AI Forum” on WhatsApp.

So that’s some of the good news. Here’s the other news. Today’s “doomer” anxieties about an AI disaster are not at all paranoid. Quite the contrary. John-Clark Levin, the (Catholic) research lead at Kurzweil Technologies—as in Ray Kurzweil, futurist—describes worst-case AI scenarios as “very credible,” absent rigorous forethought and controls. AI will likely dwarf the social, political, cultural, and psychological impact of the printing press, the Industrial Revolution, and every other deep reconfiguration of daily life that came before it. Consider the outcomes chronicled in Karl Polanyi’s The Great Transformation and Carlos Eire’s Reformations, then multiply their consequences.

Of course, maybe we’ll get lucky. Maybe we’ll get a smooth transition to some new and sunnier reality. We can take some modest solace from our experience with fission. We’ve had nuclear weapons for more than 80 years without using them. This, despite nonstop regional wars and periods of intense superpower rivalry. Of course, we’d already incinerated 120,000 people in Hiroshima and Nagasaki, with 130,000 more dying later of sustained injuries. That kind of memory burns prudence into the brain. We’ve had no similar devastating event—yet—with AI. But rogue incidents like OpenAI’s “Hugging Face” fiasco, and Anthropic’s AI offering a phony homicide tip to Philadelphia police in mid-October, do not breed confidence.

All of this is concerning because, in the words of Father Philip Larrey—who’s a friend of both Sam Altman, the CEO of OpenAI, and Eric Schmidt, the former CEO of Google—nobody fully understands what goes on in a computer between the query we input and the answer in human language we receive. Microsoft’s Mustafa Suleyman, among others, argues that nothing can be “conscious,” which is not also biological. And since AI is a matter of silicon and metal, it cannot be alive and thus can never be conscious. Others argue that no adequate definition of consciousness exists, and machines may simply be conscious in a way very different from our own. The folks at Anthropic claim they “can’t rule it out” that Claude may already be conscious.

Unpredicted and unintended behaviors, accelerated by AI’s skill at recursive self-improvement–i.e., AI’s ability to enhance itself without human involvement—are at the heart of the discussion about AI sentience. But whatever the computer is doing, it’s not the same as human thought or consciousness. Most of us experience our computers through a friendly but deceptive visual GUI—the graphical user interface. But if you want to understand just how utterly alien your computer really is, take the time to learn Linux and then run your machine through its CLI—the computer’s command line interface. It’s an unforgivingly exact experience.

Humans communicate through our bodies—our postures, expressions, and tones of voice—not just with our words. And even our words are conditioned, in their meaning, by the context and relationships in which we say them. None of that applies to a computer. AI is disembodied, inhumanly precise, and hyper-logical.

Here’s the point. Iain McGilchrist, the psychiatrist, neuroscientist, and philosopher, notes that any tool which merely manipulates symbols, however ingeniously, stands in the same relation to reality that a map does to a place in the real world. The map is a valuable guide, but it’s not the actual terrain. In a broadly similar way, machines process; they may even seem to “think”—but they don’t understand. They don’t have emotions. They can’t love or empathize. They have no spiritual life or sense of the transcendent.

As a scientist, McGilchrist also argues—see his book, The Master and His Emissary—that the right and left hemispheres of the brain have very different functions. The right hemisphere has the task of seeing and making sense of the world in an integrated, holistic way. The left hemisphere has a narrow focus on practical needs and problems. The culture of the modern West, in his view, is dominated by a kind of left hemisphere imperialism, a distorted and overconfident rationalism. And AI too easily becomes a perfect expression of it.

There’s much more to be said on this. But it’s worth remembering that fools with tools are still fools. We use our tools, but our tools also use and reshape us. AI has two paths to the future—one filled with great possibilities, the other with real disaster. That’s the apocalypse, the revelation, the crisis of our moment.

I’ll end with a few relevant thoughts from the Russian Christian writer Nikolai Berdyaev—and a note about Jeremy Bentham’s head.

Written more than 90 years ago (1933), Berdyaev’s “Man and Machine” saw the future of technology-dominated societies with unsettling clarity. His essay is a gold mine of prescient comments, all of them relating to the central question of human identity and dignity posed by Psalm 8: Who and what is man? A few examples:

Technology is the last love of humankind; and swayed by the object of our love, we are ready to exchange this object for our own image . . . [the machine] creates a new reality; everything is here and now. It tears the person away from nature and from the world of others.

The heart can scarcely bear the touch of cold metal; it cannot live in a metal casing. These [technological] procedures that lead to the destruction of the heart, as the core of the soul, are characteristic of our age . . . Everything has degraded into an intellectual element and into sensory impressions.

There will come a time when there will be perfect machines with which man could rule the world, but there will be no humans at the helm. The machines themselves will behave with perfection and achieve maximum results. The last people themselves will turn into machines, but then they will disappear because of the uselessness [and] hindrances of organic respiration and the circulation of blood . . . The new reality will remain in cosmic life. But there will be no man, there will be no organic life.

And finally:

Technological civilization is essentially impersonalistic; it does not know and does not want to know the individual . . . [M]achinism would like to replace in man the image and likeness of God with the image and likeness of the machine. This is not the creation of a new person, it is the extermination of the person, the disappearance of the person, replacing him with another being, with a different non-human existence. This is the whole agony of the problem. Humanity created the machine, and it can give [man] a proud awareness of his dignity and strength. But human pride can turn imperceptibly into human humiliation. A truly new being may appear, but it is no longer a human being . . . The world is not only de-Christianized, but also dehumanized. This is the whole severity of the question that the monstrous power of technology has set before us.

Such is the gist of Berdyaev’s thinking. And such is the larger lens through which we should weigh the value—or the lack of it—in AI, in every new technological advance, and in the Silicon Valley minds that seek to shape our future. We should note that the architects of our nascent AI “New Age” tend to be atheists with tumescent egos. AI “hallucinations,” rogue behaviors, and wrong answers are more common than generally known. And the financial vulnerability of the major AI players is pronounced, starting with massive cash burn rates, dependence on new funding rounds, and fixed future commitments.

Which brings us to Jeremy Bentham’s head. I’ll explain.

“Effective altruism” (EA) is a movement common to several key quarters of Silicon Valley, including artificial intelligence and frontier-level research labs. Crudely put, it involves doing the maximum good for the maximum number of people for the optimal practical results. Using the resources one has, it applies a comparative analysis to “good” causes that are rooted in evidence and reasoning rather than emotional attachment. It also—conveniently—tends to bless making a boatload of money, the better to bankroll one’s selective giving. This sounds relatively inoffensive, and it often is. But on October 3, The Economist treated EA to a thorough carpet-bombing in a piece entitled “The century’s biggest idea.” It’s worth reading the whole text. EA can be innocent and fruitful. It can also be inhumanly rationalist, “a crude moral arithmetic” that discounts the “flesh and blood concerns” of actual individuals in need.

Kindly note here that I’m not accusing Google’s Dario Amodei or OpenAI’s Sam Altman of lunatic ethics. Neither subscribes in a formal way to EA thinking. But there’s a vigorous utilitarian spirit endemic to Silicon Valley which—like EA—has an illustrious grandfather: Jeremy Bentham. It was Bentham, of course, who described natural rights as “nonsense on stilts;” suggested the panopticon as a tool for social control in prisons, workhouses, and schools; and argued for utility as the gold standard for moral reasoning.

The trouble is, we’re never as smart as we think we are, and we’re rarely as wise as we need to be. We humans are flawed creatures who always, in some way, imprint our flaws on what we make. AI is not—and never will be—an exception. In engaging any new technology, we’re back to a choice between two paths: Sorcerer’s Apprentice or Tolkien’s sub-creator. What’s lacking in a tech-dominated culture like our own is humility; a bias for prudent restraint; an awareness that we’re responsible to a God higher than ourselves, outside ourselves, who will hold us accountable. Ignoring that elemental fact doesn’t end well.

But let’s return, at last, to Bentham’s head.

Bentham required that his remains be dissected after his death, despite the social disapproval of his time. His body was then put back together, stuffed, and the preserved corpse dressed in Bentham’s own clothes. It was, and still is, displayed seated in a cabinet at University College London. Alas, the preservation “technology” used wasn’t quite up to the task when it came to his head. The process blackened and distorted Bentham’s head, and left his face worthy of Halloween. The head was replaced by a wax replica and now sits safely stowed away in a (climate-controlled) storeroom.

Somewhere in all that lies an instructive and delicious irony.

(Editor’s note: This article is adapted from recent comments at the University of Notre Dame.)