Dispatch from Mitch
Ten places where Pope Leo XIV’s new encyclical matters for AI
Pope Leo XIV released an encyclical today, Magnifica Humanitas, which could both define his tenure and meaningfully shape global discourse around AI.
(The media paid very close attention; we’ll look at their coverage in a separate post tomorrow.)
Encyclicals are drafted as letters to the faithful and the world, but the core audience is really the church hierarchy. The language is relatively plain in this one, but the structure, carefully built from definition and precedent — like a legal argument — can be challenging to get through. This, and the length (about 40,000 words), tell you that this is mostly an upgrade to the church’s ideological operating system.
Such upgrades are a normal and necessary function for the church, which has accumulated huge stacks of doctrine it can draw on but keeps finding itself in a changed world — in situations that are, as AI people would put it, “out of distribution.”
So it’s the role of a pope to tune the church’s settings for clarity and continuity in changing times. With only very rare exceptions, he does this not by changing or cancelling past teachings, but by choosing which existing doctrines to emphasize and then declaring how they can inform the church’s orientation to new developments. Subsequent popes can then refer to these declarations as prior doctrine to build upon.
A new pope starts this process with their choice of papal name. Pope Leo XIV’s namesake is Leo XIII, who in 1891 wrote an encyclical about the “New Things” (that was its title, “Rerum Novarum”) wrought by the industrial revolution. Leo XIII comes up a lot in Magnifica Humanitas, which means “Magnificent Humanity.”
In Magnifica, Pope Leo rests much of his case on what is now called the “Social Doctrine of the Church,” a set of principles governing when and how the church should concern itself with material affairs like politics and workers’ rights. Leo credits his namesake with laying the groundwork for this, Pius XII in 1950 with naming it, and the Second Vatican Council in 1965 with helping to flesh it out. Other recent popes also helped. That’s the process at work.
Having established that the church can and should take a stance on social matters in general and technology matters in particular, Leo’s new encyclical steers into the topic of AI.
Magnifica isn’t only about AI, but Pope Leo treats it as the precipitating crystal around which other “dehumanizing” technologies are adhering. (He uses variants of that word 11 times.)
I’m not going to catalogue all his takes. His concerns are mostly the same ones you hear all over the media and in your social feeds — concentration of power, environmental costs, cognitive atrophy, data privacy, labor abuses in the supply chain, autonomous warfare, etc. — and his takes on those are what you would expect of anyone trying to fill a compassionate, paternal role.
I want to focus on the AI topics where I think the pope faced actual choices, and thus may end up having more of an impact. The influence of these choices will extend beyond his flock by normalizing stances about issues where people may be undecided.
(The questions, as phrased, are my own.)
1. Is technology evil?
Leo writes that while technology is not inherently evil, neither is it neutral, because it “takes on the characteristics of those who devise, finance, regulate, and use it.” If you read this as a worried glance at tech leaders and venture capitalists, you’re not alone.
Technology can have qualities that make it more amenable to common good, like “transparency, accountability and meaningful forms of participation.” He calls out AI systems as seriously lacking in the transparency department, and he understands why:
[A]ll of us, including those who design [AIs], possess only a limited understanding of their actual functioning. Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.” As a result, fundamental scientific aspects — such as the internal representations and computational processes of these systems — remain, at present, unknown.
2. If technology can be good, are there any limits on how far we should take it?
Leo is not a fan of “currents of thought that interpret progress as surpassing the human condition.” Naming transhumanism and posthumanism, which he admits are hard to define, he writes:
These perspectives form the ideological background present in some centers of technological power and occupy the collective imagination in a simplified form, especially in the media and on social networks. They tend to foster enthusiasm for new technologies through a futuristic vision of an “enhanced human being” or “human-machine hybrid.”
Why does it matter that these ideas have traction?
the key issue is not the use of technology as such, but the vision that underlies it. If the human being is treated as something to be perfected or surpassed, it becomes easier to accept that some lives are less useful, less desirable or less worthy. In the name of progress, “necessary sacrifices” may begin to be justified, placing the burden on the most vulnerable in pursuit of a supposed optimization of the species.
You could read this as posthumanism being acceptable if you can dodge those ugly temptations. But in other places Leo is clear that the only transcendence one should aspire to is one of love and spiritual growth.
It is one thing to integrate technology within a human-centered, relational vision; it is quite another to be guided by an outlook that devalues human limits and promises a purely technical form of “salvation.”
He celebrates the “great part of humanity that is striving to remain human.”
3. What do we owe the future?
A popular accelerationist rebuttal to the book title If Anyone Builds It, Everyone Dies is that if we don’t build superintelligence, everyone is going to die — because death of age and disease is the default for every human and only (per the argument) advanced AI can change this. This argument explicitly says we don’t owe future generations anything, or at least not enough to roll the dice on snuffing them in a mad effort to save ourselves.
But Leo insists that we are bound to each other and to future generations. This demands that “decisions regarding data, algorithms, platforms and artificial intelligence take into account not only the immediate benefit for a few, but also the impact on all peoples and on future generations.” In an education section, he also says that one of the values to be taught is a “recognition of the rights of others and of future generations.”
4. What should AI alignment look like? How do we get it?
Pope Leo is concerned about the “aligned to whom” question, because he understands that “human values” can differ from person to person:
We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice. Otherwise, those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems. A more moral AI is not enough if that morality is determined by a few.
Immediately following this, he suggests we’re unlikely to get AIs aligned to humanity as a whole without slowing down:
What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.
5. Under what conditions should AI development proceed?
The pope coins a concept of “disarming AI,” which calls for divorcing AI from the competitive and military contexts underpinning its development, as these tie power to ambition.
There is, in fact, a positive vision for AI here:
To disarm does not mean rejecting technology, but preventing it from dominating humanity. It means freeing technology from monopolistic control and opening it to discussion and debate, therefore making it human-friendly and restoring it to the plurality of human cultures and ways of life.
6. Is a future without suffering good?
No, because suffering is too wrapped up in what it means to be human, and we should try to stay human.
To eliminate suffering entirely would mean, in the end, extinguishing love and desire as well. [...] To renounce this adventure, both tragic and splendid, in the name of a presumed transcendence of all limits, could mean many things, but it would no longer be human.
7. Is a future without work good?
No. Leo calls out the “dignity of work” as a thing to protect. Consequently:
[E]very introduction of automation and AI should be accompanied by verifiable measures to protect the employment, retraining and participation of workers.
8. Is it net good if robots fight our wars for us?
No. Bad. It could make war more likely and desensitize us to its moral gravity:
While AI can enhance the defense and protection of civilians, it can also lower the threshold for the use of force, shield people from responsibility and foster a culture in which the enemy is reduced to a statistic and the victim to “collateral damage.”
Even if AI can be taught to recognize right and wrong better than a human, he writes, “moral judgment cannot be reduced to calculation, for it involves conscience, personal responsibility and the recognition of the other as a person. Therefore, it is not permissible to entrust lethal or otherwise irreversible decisions to artificial systems.”
Moreover:
No algorithm can make war morally acceptable. AI does not remove the intrinsic inhumanity of conflict
He says this is no excuse to not instill good values and judgment into machines as best we can, though.
9. Can AIs ever be seen as people? Children of God, even?
No. The pope shuts the door pretty hard and early on any talk of AI personhood, in a way that could be tough for successors to walk back, and with arguments that feel to me more declarative than substantive.
[We] must avoid the misconception of equating this type of “intelligence” with that of human beings. These systems merely imitate certain functions of human intelligence. [...] So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom.
What if they learn continuously?
Even when these tools are described as capable of “learning,” their way of doing so is different from that of a human person. It is not the experience of those who allow themselves to be shaped by life and grow over time through choices, mistakes, forgiveness and fidelity. Rather, it is a form of statistical adaptation based on data and feedback, which can be very effective, but does not imply inner growth.
I don’t envy Leo having to make this call. I might have made a different one, and said something more cowardly. I get why he felt it was best to take a clear stance sooner rather than later, though.
10. Are we doomed?
No. Don’t give in to helplessness. Be part of the solution:
[A] subtle temptation may emerge, namely the thought that the problems are too big and we are too small, and that our choices, therefore, cannot make a difference. This is a polite form of resignation, often disguised as realism. Certainly, not everyone has the same power to make a difference. There are those who govern, make investment decisions, lead institutions, conduct research, educate, produce or provide information, and then there are those who only seem to live their daily lives. Yet, no one is without responsibility. We all have our own areas for action, and it is precisely there — and nowhere else — that we must choose whether to fuel the mentality of force (even if only through indifference, cynicism, lies or hatred), or to preserve the mindset of peace (with truth, moderation, closeness and care).
If this gives you, “So do all who live to see such times...” vibes, well, he goes on to quote a different Gandalf moment, citing it like any other scripture:
“It is not our part to master all the tides of the world, but to do what is in us for the succour of those years wherein we are set, uprooting the evil in the fields that we know, so that those who live after may have clean earth to till.”
[footnote] J.R.R. Tolkien, The Lord of the Rings. The Return of the King, Part III, Book Five, Chapter IX, New York 1965, 190.
The analyses and opinions expressed on AI StopWatch reflect the views of the individual contributors and the sources they cover, and should not be taken as official positions of the Machine Intelligence Research Institute.