The Pope Is Not Impressed by Your Bots
Pope Leo’s encyclical is a lucid condemnation of AI’s threats — and an approach to issue-based leadership Washington could learn from.
If I had to guess who would offer the next most effective critique of the AI boom, I would not have predicted the man who lives in a 16th-century palace with a 2000-year-old job. But Pope Leo XIV’s first encyclical, Magnifica Humanitas, is a gimlet-eyed skewering of the value (and values) behind Silicon Valley’s would-be deus ex machina. For policy advocates, it should also serve as a sterling example of what it looks like to own an issue.
In a seemingly inexhaustible cycle of hype and backlash, Leo’s message rings with clarity of intent, equally uninfected by the apocalyptic doomerism, utopian technobabble, and forced logic of inevitability that has made so much AI discourse — and policy-making — by turns alarming and deeply stupid. He is not here to warn us about a paperclip-based sorcerer’s apprentice situation or to herald the transhuman singularity. He is not here to advise us that the de-humanizing of vast swaths of culture is coming whether we like it or not, so we might as well learn to love the bots. He’s not wasting time on “alignment,” a framing of machine ethics so two-dimensional that I genuinely suspect it comes from Dungeons & Dragons.1
Magnifica Humanitas simply gets down to the business of laying out the threats that unregulated AI poses to human welfare in the here and now, including lost and degraded labor, autonomous weapons, and documented risks to mental health. You know, the kind of things the Catholic Church’s constituency — humans — tends to care about. “Artificial intelligence needs to be disarmed,” Leo said in a short speech announcing the text. “The word is strong, I know, but deliberately chosen because this moment needs words capable of attracting attention...” The Pope knows the power of a tagline!
The encyclical has more on what he means by “disarm,” and it’s an elegant statement of purpose:
To disarm means discrediting the assumption that technical power automatically confers the right to govern. To disarm does not mean rejecting technology, but preventing it from dominating humanity. It means freeing technology from monopolistic control and opening it to discussion and debate, therefore making it human-friendly and restoring it to the plurality of human cultures and ways of life. Our task today is not only ethical or technical. It is ecological in the deepest sense, for it concerns a new dimension of our common home.
Everything about the letter’s rollout felt deliberate. The Pope pointedly signed it on an anniversary of the publication of Rerum Novarum (“Of New Things”), an encyclical by the previous Pope Leo celebrated as a landmark expression of the Church’s social gospel. Published at the tail end of the 19th century, Rerum Novarum — subtitle: On Capital and Labor — was a cri de coeur against working conditions during the Industrial Revolution. This Pope Leo, who has said his name is an homage to his predecessor, describes the AI-juiced digital age as a “fourth industrial revolution” in the making. On his second day as pope, he told the College of Cardinals that he wanted the Church to address AI’s threats to “human dignity, justice and labor,” and in the year before the encyclical became his first major theological statement he has spoken about it in forums around the world.
This is how you do a signature issue.
U.S. leadership has a terrible track record of discernment around digital technology. Perhaps you recall the “series of tubes” debacle, or when Congress was a little late to the game in appreciating Facebook’s role in the 2016 election. Yesterday, Trump released an Executive Order that, on its face, softens the administration’s long opposition to AI regulation — by asking manufacturers to (voluntarily) provide new models for a government cybersecurity review — but only after getting cold feet weeks ago for fear any such order would be an innovation “blocker.” As-is, the order is the watered-down product of chaotic equivocation, and pretty rich coming from an administration that has slashed its core cybersecurity agency to the bone.
Whatever Trump is going for at the moment, comprehensive, humane AI regulation is never going to come out of this administration. Why aren’t more politicians with a documented shred of conscience owning this issue?
That the Pope is an unlikely commentator on Silicon Valley is what makes him a useful one. He doesn’t have to care about campaign contributions. He doesn’t have to care about the Dow Jones. As the faith leader of 1.3 billion people, he doesn’t even have to care about America (but given that he’s a Chicago native, let’s assume he does). He definitely doesn’t have to care about the ire of the Trump administration, either their mealy-mouthed snark or their impotent threats.2 In this space, he’s a commentator with the power of an outsider.
AI regulation advocacy has proven to be a laboratory for unlikely coalitions, which is a promising sign. Opposition to data centers, in particular, has unusually strong bipartisan support. How and why this is happening are questions worth answering, at a time when we need the biggest tent we can find going into the midterms and 2028.
That said, there was one false note in the encyclical announcement: the fact that one of the co-founders of Anthropic, Christopher Olah, was invited as a speaker. I respect Leo’s call for dialogue, but I couldn’t help feeling like Olah’s participation was a fundamentally hollow gesture by a company seeking to burnish their “lesser evil” brand.
Olah began his speech by acknowledging that any AI maker “operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing,” and on that we can agree. Anthropic has gotten plaudits this year for rejecting the Pentagon’s demands that it remove internal safeguards, which it deserves — but this strikes me as a necessary bar that has been set too low. Even as Anthropic was apparently standing up to the administration, it announced the removal of its once-central safety promise to “pause training of more powerful models if our AI scaling outstrips our ability to comply with the necessary safety procedures.” Now is the time for growth at all costs, CEO Dario Amodei said — because if a less scrupulous company builds the next powerfully disruptive model first, it “could result in a world that is less safe.”
Incentives indeed. Anthropic’s models have the same vulnerabilities as their competitors, and I will never buy “if we don’t build it, someone else will” as a decent rationale for anything.
The rest of Olah’s speech made its own arguments for how little his ilk should be involved in setting the regulatory course for this technology:
· “There is a real possibility that AI will displace human labor at a very large scale” — inevitability-speak minus accountability
· “AI systems are not engineered the way a bridge or an airplane is engineered…They are grown, on a structure roughly modeled after the brain…” — I simply can’t begin to get into how wrong this is
· “They are not the cold, calculating robots we were promised. They are made from us, from our words…” — they are hot plagiarism robots
· “If it helps, one way I sometimes describe it is as being a little like bringing a fictional character to life.” — don’t patronize me
The Emperor’s tailor was hard at work. What we’re building is so dangerous, but also so mysterious; stand back while we create the dehumanizing robots from classic sci-fi novel Don’t Create the Dehumanizing Robots. Meanwhile, as the National Catholic Reporter pointed out, Anthropic is “notably absent” among corporate signatories on another Vatican document calling for the ethical development of AI.
I hope Leo understands that he doesn’t need Anthropic’s endorsement any more than he needs their product. While Olah and co. scramble around serving their various “incentives,” the Pope has the advantage of answering to just the one boss, who has been good at delegating for at least a couple millennia. Here’s hoping the rest of us take some cues from his style.
No offense to D&D. Prove me wrong, Altman
“ . . . would-be Silicon Valley’s deus ex machina . . . .” This description and others in this essay are spot on. Thank you for the clarity.
Excellent