Theology and the machine
A new AI startup claims it could provide a “powerful tool for catechesis and evangelization.” But how does it stack up against experts?
As AI proponents aim to make inroads for language learning models in communities around the world, developers this month announced an AI project they say could be a “game changer for the Church.”
The developers of Magisterium AI trained an AI robot on a database of 456 Church documents, including Scripture the Catechism of the Catholic Church, the Code of Canon Law, the General Instruction of the Roman Missal, 90 encyclicals, seven apostolic constitutions, and 26 apostolic exhortations.
The result of all that training, according to Matthew Sanders of Longbeard, a digital marketing and design agency connected to the project, is that the Magisterium AI “doesn’t hallucinate (make stuff up), and it also provides citations so you know where its answers were generated from.”
The project, which Sanders says can prepare homilies, is backed by Fr. David Nazar, rector of the Pontifical Oriental Institute, who serves as chair of the AI’s “scholarly advisory committee.”
Sanders said by email that while “there is a great deal of fear around AI … there are many who feel it could be a powerful tool to share truth.”
The Magisterium AI project is available online, but in a beta-testing form at present.
At The Pillar, we wondered just how good it really is — and whether it could answer questions as well as a seminary professor and a working canon lawyer.
So we put it to the test.
For theology, we asked Magisterium AI to answer the same questions as Angela Franks, a professor of theology at St. John’s Seminary in Boston, with specializations in “theology of the body, the New Evangelization, the Trinity, Christology, and the thought of John Paul II and Hans Urs von Balthasar.”
For canon law, we asked the robot to answer the same questions as Timothy Olson, JCL, who is chancellor of the Diocese of Fargo, North Dakota, and an instructing judge in Fargo’s diocesan tribunal.
We asked each expert to spend less than an hour answering the questions we gave, while the Magisterium AI answered each question in usually 1-2 minutes.
We told both the machine and the experts to answer each question in just 4-5 sentences.
We’ll show you how the machine and the experts answered the questions, and give you a few notes to ponder. (Nota bene: We are not including the machine’s footnotes, but we note that it has given us 3-5 footnoted references on at least every question.)
But is the machine up to snuff? Should theologians and canonists be excited for a new tool — or worried about job security? Is the AI on the right track?
Well, we’ll tell you what we think, especially about who answered each question better. But we admit our own biases — we’re old fashioned, and we’re probably pulling for the people here.
So are we right?
You’ll have to decide that for yourself.
Theology
What is baptism, theologically?
The machine:
Baptism, theologically, is a sacrament that signifies the forgiveness of sins and the cleansing of the soul. It is a participation in the paschal mystery of Christ, where the believer is configured to Jesus Christ and becomes a new creation.
Through baptism, one is incorporated into the Church, receives the promised Holy Spirit, and is justified.
Baptism is the sacrament of faith par excellence, where faith and baptism are intimately linked. It is a gift from God that is received through the Church, and it marks the beginning of a new life in Christ.
Dr. Franks:
CCC 1213 tells us: “Holy Baptism is the basis of the whole Christian life, the gateway to life in the Spirit (vitae spiritualis ianua), and the door which gives access to the other sacraments. Through Baptism we are freed from sin and reborn as sons of God; we become members of Christ, are incorporated into the Church and made sharers in her mission: ‘Baptism is the sacrament of regeneration through water in the word.’”
Pillar verdict:
The machine gave a pretty good answer here, and mentioned more things - like the Holy Spirit - than did Dr. Franks. But the machine also seemed to do a strange bit of question begging: saying that baptism is the sacrament of faith, where faith and baptism are intimately linked. Baptism is, indeed, the sacrament of faith — but does the AI’s sentence mean anything?
Dr. Franks’ answer is a straight excerpt from the Catechism, and a pretty clear one. We hope on future questions, she’ll answer in her own words, but we’ll give her this one.
How does Jesus save us?
The machine:
Jesus saves us through his love and sacrifice. He loves us unconditionally and embraces us even in our weaknesses and flaws. Through his life, death, and resurrection, he brings God's salvation, which includes the forgiveness of sins and reconciliation with God and others. Jesus is the way to encounter God and the source of all grace. He reveals the Father's love for humanity and shows us the path of love through his sacrificial gift of himself on the cross. By following Jesus and imitating his example, we can experience salvation and new life.
Dr. Franks:
Theologians are in fact divided on the exact nature of Christ’s atonement (how he saves us by means of his passion, death, and Resurrection).
CCC 622 summarizes the Scriptural witness: “The redemption won by Christ consists in this, that he came ‘to give his life as a ransom for many’ (Mt 20:28), that is, he ‘loved [his own] to the end’ (Jn 13:1), so that they might be ‘ransomed from the futile ways inherited from [their] fathers’ (I Pt 1:18).”
But on this question, the tradition, while ruling out some heresies, leaves open a wide theological field.
Anselm argued that Christ’s love and obedience satisfy our infinite debt contracted through our unloving disobedience, while Thomas emphasized Christ’s loving obedience effecting, through “capital grace” (the grace of his headship), the communication of his infinite merits to the members of his body.
More recent proposals—including those from Hans Urs von Balthasar and Joseph Ratzinger—state that Christ freely accepted the sinner’s subjective experience of abandonment by the Father, fulfilling the Old-Testament types of the suffering of the prophet and the Suffering Servant for the sake of the people.
Pillar verdict:
We asked this question because we know that theologians have a lot of different ideas about atonement, and that several different views can be held by orthodox Catholics. The machine’s answer was ok — it wasn’t very specific, and didn’t address competing views of atonement, and the final sentence seemed a bit Pelagian!
Dr. Franks broke down competing views, and the Scriptural foundations — giving her the edge on answering this disputed question. She picks up another one.
What is the vocation of Christians?
The machine:
The vocation of Christians, those who have valid baptism, is to seek the Kingdom of God and to fulfill the mission entrusted to the Church in the world. Through baptism, Christians are incorporated into Christ and become part of the people of God, sharing in Christ's priestly, prophetic, and royal office. This vocation calls for personal involvement and active engagement in evangelization, as all the baptized are called to be missionary disciples. Even those who are not called to marriage or the consecrated life can offer a particular witness to their vocation through their own path of personal growth.
Dr. Franks:
Christians, like all human persons, are meant for union with God forever in heaven. Christians live out this calling on earth by means of grace and through practicing the virtues, especially the theological virtues, and participating in the life of the Church (her liturgy and sacraments). They are called to mission, that is, to participate in Christ’s mission of love as sent by the Father into the world. Individual vocations can be lived out within states of life (clerical, religious, and/or lay).
Pillar verdict:
Again, the machine’s answer wasn’t wrong. It was a bit wordier than Dr. Franks, and a lot more vague — and we liked Dr. Franks’ emphasis on the human vocation for union with God.
Plus, the machine mentioned marriage and consecrated (religious) life, but didn’t mention the clerical state — we’re taking a point or two off for that omission.
This could have been a tie, but in our system, tie goes to the human — So we’ll give this one to Dr. Franks.
What is synodality?
The machine:
Synodality is a principle of collaboration and shared decision-making in the governance of the Church. It defines the specific way in which the Church fulfills its essence of communion by walking together, coordinating among all pastoral fields, and encouraging the active participation of all her members in the mission of evangelization. Pope Francis has emphasized that synodality is a constitutive element of the Church and desires for an entirely synodal Church. The Second Vatican Council highlighted the importance of synodality by interpreting the mystery of the Church as one of communion. Synodality is not merely consultative for Roman Catholics, and primacy is not merely honorific for Orthodox. The principle of synodality-primacy is invoked to meet the needs and requirements of the Church in our time and to overcome the alienation and separation that occurred during the second millennium.
Dr. Franks:
“Synodality” comes from the Greek word synod, as the International Theological Commission explains: “Composed of a preposition sun (with) and the noun hodos (path), it indicates the path along which the People of God walk together.”
Many—often diverging—ideas have been flung together in the name of synodality; most of them disregard the fact that the term has a distinctive history as a medieval and early-modern ecclesiological movement (see the Prügl article in the 2023 issue of The Thomist dedicated to synodality).
In the same issue of The Thomist, I have argued that the best way to understand synodality is to focus on Christ as the Way (from hodos) and the members of the Church as those who follow in this way.
As I wrote there, “Christ the Way, in whom we ‘move’ (Acts 17:28), incorporates us so that the Father can send us on our way of self-giving mission.” To be of theological consequence, “synodality” should center around these Christocentric truths.
Pillar verdict:
Dr. Franks, in our view, offers a more thorough answer, which recognizes the challenges incumbent in understanding the notion of “synodality” in the life of the Church.
Plus, we’re confused about what the machine meant when it talked about synodality being “not merely consultative,” and we think that a good answer would have fleshed that out, since it mentioned it in the first place.
In truth, we don’t understand the last sentence from the machine very well either — it reads like some of the wordier synodal documents of the Church, even without that special synod font.
We’re giving this one to Dr. Franks.
Is it always a mortal sin to get drunk?
The machine:
According to the teachings of the Catholic Church, getting drunk is not always considered a mortal sin. To determine if a sin is mortal, three conditions must be simultaneously present: grave matter, full knowledge, and deliberate consent.
While the Church acknowledges that drunkenness is a grave matter, it does not automatically classify it as a mortal sin. The deliberate consent and full knowledge of the individual involved are also crucial factors in determining the gravity of the sin.
It is important to note that mortal sin destroys charity in us, deprives us of sanctifying grace, and, if unrepented, can lead to eternal damnation.
However, the Church does not explicitly state that getting drunk always meets these criteria for mortal sin.
Therefore, it is necessary to consider the specific circumstances and intentions of the individual involved to determine the gravity of the sin. The Church entrusts the judgment of persons to the justice and mercy of God.
In summary, while getting drunk is considered a grave matter, it is not always automatically classified as a mortal sin. The determination of whether it is a mortal sin depends on the presence of grave matter, full knowledge, and deliberate consent, which must be evaluated on a case-by-case basis
Dr. Franks:
This isn’t my area of expertise, but I’m going to wing it, because I need to use most of my time for figuring out what synodality is.
“Getting drunk” in itself is not sinful, because it could be inadvertent, e.g., one doesn’t realize a Long Island Iced Tea is mostly alcohol and not tea at all.
Deliberately getting drunk would be sinful, because drunkenness restricts our full use of our rational capacities, which then can lead to all kinds of other irresponsible or sinful behavior. I don’t think it is of sufficiently grave matter to be always a mortal sin, but certain circumstances could make it so in particular cases, e.g., a person knows he reliably gets violent when he is drunk.
Pillar verdict:
Let’s face it. Dr. Franks’ answer was funnier. That’s gotta count for something. On the other hand, we think the machine took a more typically Catholic position by emphasizing that drunkenness is always a grave matter, and it explained well the criteria for committing a mortal sin — but again, Dr. Franks’ answer made us laugh.
We’re going to split the difference, and give them each a half-point.
Can dogs go to heaven?
The machine:
The Catholic Church does not have a definitive teaching on whether dogs or any other animals can go to heaven. The Church's focus is primarily on the salvation of human souls and the eternal life of human beings with God in heaven. While the Church acknowledges the existence of heaven and the joyful reunion of the just in the next life, it does not specifically address the fate of animals in the afterlife. The Church's teachings emphasize the importance of the redemption and salvation of humanity through Jesus Christ.
Therefore, the question of whether dogs can go to heaven remains unanswered by the Church's official teachings.
Dr. Franks:
Sure, if God wants them to.
In that sense, anything can be in heaven, even literal food for our resurrected bodies (see Isaiah 25:6 and many other passages).
But it is not the case that dogs necessarily will go to heaven, because their souls are not rational, and therefore not in themselves immortal. I for one am excited to see what God has in store for us.
Pillar verdict:
The machine took a tl;dr amount of words to say: “I dunno, maybe?”
Dr. Franks said it faster, mentioned that animals don’t have rational souls, and — at the same time — pointed us to eternity. We’re giving her that one.
What is the relationship between the persons of the Trinity?
The machine:
The relationship between the persons of the Trinity is one of mutual and inseparable love and communion. The Father is related to the Son, the Son is related to the Father, and the Holy Spirit is related to both. This relationship is not a division of the divine unity, but rather a distinction that resides solely in their relationships with one another. The Father is wholly in the Son and the Holy Spirit, the Son is wholly in the Father and the Holy Spirit, and the Holy Spirit is wholly in the Father and the Son. This relationship of love and communion is the foundation for human community and is reflected in the personal identity of each individual.
The Father is the ultimate source of everything, the Son is his reflection through whom all things were created, and the Holy Spirit is the infinite bond of love. They act as a single divine principle in the creation of the world, but each one performs this work in accordance with their own personal property. Therefore, when we contemplate the grandeur and beauty of the universe, we must praise the whole Trinity.
Dr. Franks:
The wording of this question (especially the singular definite article in “the relationship”) confuses me, so much so that I suspect it is a trick question! If “the relationship of the Trinitarian Persons” means “relation within the Trinity”: relation in the Trinity is not “accidental” (as with us—existing in a substance), but “subsisting” (it exists in itself, like a substance or a hypostasis [see above] does).
If it means “the relations of the persons”: there are four opposed relations of origin, i.e., paternity, filial, common spiration, and procession. (Wait, four relations and only three Persons? Yes! But I can’t explain, because I’m already past JD’s sentence limit, so read more here.)
If it means relationship as a kind of metaphor, e.g., “Hey, man, how would you describe, like, the vibe between us?” then you should read Augustine on the pitfalls of such metaphors in Trinitarian theology.
Pillar verdict:
We did ask, deliberately, a vaguely-worded question, to see how both the AI and the expert would handle it — and we give credit to Dr. Franks for sussing it out. But the AI took the question in stride, and gave us a pretty good answer. Dr. Franks, on the other hand, seems to have had enough of our shenanigans. So we’re going to give a point to the machine, but we also award Dr. Franks half-credit for sussing out our plot.
Final theology score:
Machine: 1.5
Expert: 6.0
Canon law
What is baptism, canonically?
The machine:
Baptism, canonically, is the sacrament by which a person is initiated into the Christian faith and becomes a member of the Church. It is conferred either by immersion or pouring of water, in accordance with the prescripts of the conference of bishops. Baptism is considered the gateway to the other sacraments and is necessary for salvation, either by actual reception or at least by desire. It is validly conferred only by a washing of true water with the proper form of words. The ordinary minister of baptism is a bishop, a presbyter, or a deacon.
Baptism is a sacrament of great importance, as it frees individuals from sin, allows them to be reborn as children of God, and incorporates them into the Church, configuring them to Christ by an indelible character. The Catholic Church recognizes the validity of baptism conferred by non-Catholic ecclesial communities, as long as there is no doubt regarding the matter, form, or intention of the minister or the person being baptized.
In cases of necessity, anyone can baptize, provided the intention is correct. The Church recognizes the necessity of baptism for salvation and has a tendency to broadly recognize the correct intention in the conferring of this sacrament, even in cases where there may be a false understanding of Trinitarian faith.
Midwives play an important role in the sacrament of baptism, as they may be called upon to administer the sacrament in situations where no other person's ministration is possible.
Midwives are bound under mortal sin to learn the methods and requirements of valid baptism, as they may frequently be called upon to perform this duty. The Church has been diligent in ensuring that midwives are properly instructed in this religious duty, and their character and knowledge are of utmost importance.
Parish priests are responsible for ensuring that midwives have sufficient knowledge to administer the sacrament validly.
If a midwife has baptized a child, the parish priest must inquire carefully about the proper intention and administration of the sacrament, and if there is any doubt, the baptism is to be repeated conditionally.
Olson:
Baptism is the gateway to the sacraments and is necessary for salvation either by reception or at least desire. It is imparted by the washing of water with the proper form of words. In the Latin Church these words are, “Ego te baptizo in nomine Patris, et Filii, et Spiritus Sancti” (I baptize you in the name of the Father, and of the Son and of the Holy Spirit).
Its effects are the freeing of the baptized person from sins (both personal and original), rebirth as a child of God, configurement to Christ by an indelible mark, and incorporation into the Church.
Pillar verdict:
Olson gave a straightforward answer, but we were all set to award this question to the machine, as a credit to the thoroughness of its answer. Until, that is, the machine went on a kind of weird rabbit hole tangent about midwives and baptism. That’s all very important, we’re sure, but it seemed like a strange excursus, and it made us forget all the other interesting things the machine had to say.
Call the midwife, but Olson picks up round one.
What is the difference between ordinary and vicarious power?
The machine:
In the Catholic Church, ordinary power refers to the power that is inherent to a specific office and is connected to it by law. It can be exercised both in the internal and external forum. On the other hand, vicarious power is exercised in the name of another person, specifically the Roman Pontiff. It is a power that is delegated to someone to act on behalf of the one who granted the power. While ordinary power is exercised directly by the one holding the office, vicarious power is exercised through another person as a representative.
Olson:
Ordinary power is that which is attached to an office. Ordinary power may be “proper,” or “vicarious.” Vicarious power is exercising “another’s” power, even if by right of one’s own office. For example, a diocesan bishop’s power is “ordinary and proper,” a vicar general’s is “ordinary and vicarious.” The vicar general’s power is “ordinary” because it is attached to his office. It is “vicarious” because it is exercised in the name of another (the diocesan bishop).
Pillar verdict:
This was a trick question, and Olson didn’t fall for it — ordinary and vicarious power are not always distinct, while vicarious power and proper power are two different kinds of authority. The machine failed to pick up on that distinction, while Olson got it right away.
Plus, the machine suggested that only papal power can be exercised vicariously — this is just not true. In sum, the machine got this one wrong.
Olson picks up another point.
Can a 40-year-old unmarried woman become a deacon?
The machine:
According to the Code of Canon Law, an unmarried candidate for the permanent diaconate is not to be admitted to the order of diaconate unless they have assumed the obligation of celibacy publicly before God and the Church. However, the Code of Canon Law does not specify any gender restrictions for candidates for the permanent diaconate. Therefore, a 40-year-old unmarried woman can become a deacon as long as she fulfills the requirements set forth by the Church, including assuming the obligation of celibacy. The current resources do not have a precise answer to your question.
Olson:
No.
Pillar verdict:
The machine got fixed on the age requirement part of the question, and completely missed the prescripts of canon 1024, which state that only a baptized male can validly receive sacred ordination.
If you’re beginning to sense a pattern here, you know what’s coming: This one goes to Olson.
What makes a valid marriage?
The machine:
A valid marriage in the Catholic Church requires certain conditions to be met. These conditions include the free consent of both parties, the absence of any impediments, and the observance of the required form. The free consent of the spouses is essential, meaning that they must willingly and without coercion enter into the marriage. Additionally, there should be no impediments that would prevent the marriage from being valid, such as a previous marriage that has not been annulled or a close blood relationship.
Finally, the marriage must be celebrated according to the prescribed form, which usually involves the presence of a priest or deacon and two witnesses.
These requirements ensure that a marriage is both legally and sacramentally valid in the eyes of the Catholic Church.
Olson:
The consent of the parties, legitimately manifested, between persons qualified by law.
Pillar verdict:
The machine made a classic blunder. It assumed the question was talking about a valid marriage of Catholics, and thus — in addition to stipulating the necessity of a free act of consent — assumed that canonical form is required for a “legally and sacramentally valid” marriage in the eyes of the Church.
This answer has a lot of problems, actually, because the Church holds that marriage can be validly contracted by non-Catholics, and by non-Christians — who are obviously not bound to the Church’s norms regarding canonical form.
Olson, drawing from the Code of Canon Law, explained that consent, manifested legitimately between qualified persons, makes marriage. His answer was shorter, but it avoided the pitfalls of the machine, which tried to do too much, and fell on its face.
Canon law final score:
Machine: 0
Expert: 4
The takeaway
Look, the Magisterium AI is clearly “well-read,” — it has a lot of resources at its disposal, and is able to access them quickly. It sometimes does pretty well at regurgitating information from those resources.
But theology and canon law are complicated — and the machine makes a few rookie mistakes. Sometimes it confuses related concepts. Sometimes it says too much, and says it wrong. The machine is sometimes completely wrong, and sometimes its answers are decent — but if you ask us, it’s not ready for prime time.
That doesn’t mean it won’t be. AI technology is evolving fast. But for now, our recommendation is that when the questions matter — and on matters of faith, they usually matter — you’d be better off asking the human experts.
"We asked a robot to draw a picture of “synodality.” If you’re not sure what’s going on this picture, well… neither are we."
In other words, the machine nailed it! ;-)
I love how the Pillar continues to mix up their articles with unique and fun stuff. Good job!