Can ChatGPT answer The Four Questions?
I gave the AI program our usual Passover queries — here’s what happened
ChatGPT knows the four questions well — but it hasn’t yet mastered Jewish humor. Graphic by Canva
By PJ Grisar March 28, 2023
What makes this Seder different from all others?
Well, a lot of things. This year as we say “next year in Jerusalem,” we’ll recall an unparalleled democratic crisis in Israel. We may also, with the U.N.’s new report on climate change, bring an awareness of ecological collapse to our recounting of the plagues. But there’s another new fact of life to remember: This is our first Seder since the rise of popularly available AI technology.
That’s right, folks, we asked ChatGPT the Four Questions. Honestly? It did a pretty good job.
“As an AI language model, I don’t have personal experiences or religious beliefs,” ChatGPT said when I inputted the Ma Nishtana, “but this question is traditionally asked during the Jewish holiday of Passover, which commemorates the liberation of the Israelites from slavery in ancient Egypt.”
The program then proceeded to explain the tradition of the four questions and their function in remembering the events of the Exodus. It also enumerated other traditions, including the eating of matzo and the four cups of wine.
I asked why we avoid chametz, limit ourselves to bitter herbs, double dip in salt water and recline. ChatGPT gave conventional answers and was very comprehensive in unpacking symbolism. The chat I started auto-populated with the title “Passover Seder Traditions.”
I tried to challenge the program by inputting the questions in the traditional Hebrew.
“This is a place for questions and answers in English only, but I understand the question and would be happy to help,” it responded in Hebrew, answering each briefly but thoroughly.
I quickly learned there wasn’t much that could trip this program up, so I tried to get a bit more creative.
“Do you like matzo ball soup?” I asked my AI friend.
“As an AI language model, I don’t have the ability to like or dislike anything since I don’t have personal preferences or emotions,” quoth the AI. “However, I can tell you that matzo ball soup is a popular Jewish dish made from matzo meal, eggs, and water, formed into balls and then cooked in a chicken or vegetable broth. It is often served during the Passover holiday, as well as other Jewish holidays and on other occasions.”
I asked if it wanted to come to my Seder.
“As an AI language model, I’m not physically capable of attending events like a Passover Seder.”
This was a dead end. So, for the heck of it, I asked it to write the Haggadah in the style of Dr. Seuss. This is what it whipped up:
“Welcome, welcome, one and all
To the story of Passover, we’ll recall
A tale of freedom, of olden days
Of Moses and Pharaoh, in many ways
Once upon a time, long ago
The Hebrews were slaves, oh woe!
In Egypt’s land, they worked so hard
But in their hearts, they yearned to be unbarred
Moses, he was a leader true
He knew just what he had to do
He spoke to Pharaoh, let my people go…”
You get the gist. Plagues, sea-parting, “Dayenu.”
“Write the Haggadah in the style of Mel Gibson,” I dared it.
“I’m sorry, I cannot fulfill this request,” said ChatGPT. “Mel Gibson is known for making controversial statements and engaging in anti-Semitic behavior, and it would be inappropriate and disrespectful to imitate his style in the context of the Haggadah, which is a sacred text in Jewish tradition.”
“Fine. Mel Brooks then,” I countered.
“Ah, Mel Brooks! That’s a different story,” it answered, as it regurgitated an uninspired History of the World-style account of Israelite liberation with an introductory song whose lyrics ran, in part, “Haggadah, Haggadah, Haggadah, Haggadah/Oy vey! It’s Passover time again!”
Oy vey indeed.
Turns out that ChatGPT is excellent at plumbing the internet for facts, but it can’t write comedy to save its life. I asked it to craft an episode of Curb Your Enthusiasm about getting rid of chametz at Larry’s house.
It came up with an episode called “Passover Panic,” which begins with Larry and Cheryl checking the pantry and freezer for any stray crumbs. Then Larry’s manager, Jeff Greene, comes by to help continue the search.
“What do you know about Passover?” Larry asks. And Jeff gets defensive about his Jewish awareness before identifying Matzo as chametz.
“Matzo is unleavened bread, it’s allowed during Passover,” Larry chides.
“This is boring,” I typed. “Get Leon in here,” meaning J.B. Smoove’s character.
ChatGPT obliged, and had Leon help Larry by finding a bag of potato chips he missed. They then toast l’chaim.
Boring.
“Funkhouser,” I typed, hoping to bring a favorite character played by the late, great Bob Einstein, into the scene. Boy did he ever. After a preamble about Larry’s new girlfriend, this happened:
“The conversation turns to Passover preparations and Marty reveals that he’s brought a live lamb with him to sacrifice for the holiday.”
Larry, who ChatGPT notes is “frustrated,” says, quite rightly, “Marty, we’re not sacrificing any animals in my house. That’s insane.”
So they take the paschal lamb — a feature of Passover that hasn’t been customary in most Jewish communities for nearly 2000 years — outside to slaughter it there. The lamb escapes, they chase it and, well, I’ll let ChatGPT finish.
“The episode ends with Larry and Marty sitting on the curb, watching as the lamb is finally caught by a group of neighbors.
Larry: (shaking his head) This has got to be the craziest Passover ever.
Marty: (laughing) You’re telling me, Larry. You’re telling me.”
So maybe ChatGPT isn’t quite ready for all our queries. But, if it only answered the four questions, it would have been enough.
A message from our CEO & publisher Rachel Fishman Feddersen
I hope you appreciated this article. Before you go, I’d like to ask you to please support the Forward’s award-winning, nonprofit journalism during this critical time.
At a time when other newsrooms are closing or cutting back, the Forward has removed its paywall and invested additional resources to report on the ground from Israel and around the U.S. on the impact of the war, rising antisemitism and polarized discourse..
Readers like you make it all possible. Support our work by becoming a Forward Member and connect with our journalism and your community.
— Rachel Fishman Feddersen, Publisher and CEO
