National Population Survey Postponed — Again

By Nacha Cattan

Published April 11, 2003, issue of April 11, 2003.
  • Print
  • Share Share

The National Jewish Population Survey, the $6 million study whose release was postponed last November so that its methods could be reviewed, has been postponed again for a second review.

The new delay means that the survey, commissioned by United Jewish Communities and originally set for completion in 2000, will not be published for at least two more months, according to investigators called in to review the survey.

The goal of the new review is “not to point fingers” but to determine if “the estimates are as good as they can be,” said the head of the new review panel, Marc Schulman, president of the research firm Schulman, Ronca & Bucuvalas Inc. “Our committee is just a fresh pair of eyes.”

Schulman is also president of the American Association of Public Opinion Research. He was called in after the head of the initial review, sociologist Bernard Shapiro, vice chancellor of McGill University, reportedly recommended a higher level of technical expertise than his panel was able to offer.

The repeated delays appear to reflect the intense polarization of views among Jewish demographic researchers over the validity of methods used by the survey and its 1990 predecessor. The 1990 survey drew worldwide attention with its finding that American Jews were marrying outside the faith at a rate of 52% per year. Critics said the number was inflated as a result of methodological flaws, and that some flaws have been replicated in the new survey, whose early version showed a 4% Jewish population drop. Some critics said that by inflating rates of assimilation, the surveys caused unwarranted panic in parts of the Jewish community.

“If those people weren’t out there, the Marc Schulman group wouldn’t be coming in now,” said a source close to the survey.

Defenders of the survey, for their part, said the criticism was unwarranted and damaging.

“The review simply injects another note of unnecessary concern that all may not be well with this [survey],” said sociologist Egon Mayer, a member of the survey’s National Technical Advisory Committee.

“At the very least it makes the results of decreasing relevance, because of the delay,” said Mayer who objected to the survey being pulled in the first place. Mayer is co-director of the North American Jewish Data Bank, which is co-sponsored by UJC and the City University of New York.

The survey was partially rolled out last October at a press conference where it was announced that America’s Jewish population had declined by some 4% or 300,000 in the previous decade. A second set of findings was to have been released at a UJC assembly in November. However, the second release was canceled after UJC officials disclosed that some of the survey’s backup data had been lost by the polling company, Roper Audits & Surveys Worldwide, making if difficult to verify the survey’s findings. Sources close to the survey told the Forward at the time that the lost data, while relatively minor, were part of a larger pattern of suspected errors.

A six-member investigative committee was assembled under Shapiro to study the survey’s methods, and a new release date was tentatively set for Passover.

Shapiro’s team reported to UJC in January with a list of glitches and “programming errors” to be addressed before the survey could be released. Among them were missing information in the survey’s “screener,” or opening questionnaire, which may have resulted in an undercounting of the overall population by about 1%.

Last week, however, Shapiro called for a second review by a new panel.

The second review is “part of a process of further rebuilding the confidence of the various publics that are awaiting the results of the study,” Shapiro wrote in a memo obtained by the Forward.

Shapiro said the independent review team, headed by Schulman, would examine his own group’s technical investigations. It would check the study’s statistical sample and the weights used to calibrate the results, and would compare the data with similar studies.

Other members of the new panel include Stanley Presser of the University of Maryland, Eugene Ericksen of Temple University and Tom Smith of the National Opinion Research Center in Chicago.

The review will also explore the “strengths and limitations” of the survey data and, for the benefit of future studies, make recommendations about the methods used to gather the data, Schulman said.

Ira Sheskin, director of the Jewish Demography Project at the University of Miami’s Miller Center for Contemporary Jewish Studies, said it was necessary for top researchers in the field to conduct an independent review because Shapiro’s team was not charged with or equipped to perform an in-depth technical analysis of the survey. Sheskin added that a delay was inevitable even without the second review because the firm that conducted the survey and was blamed for many of its glitches, Roper Audits & Surveys Worldwide, took longer implementing the weighting changes than originally anticipated.

When the survey was initially pulled, several members on the advisory committee decried the action. They said that the missing data was relatively minor, and argued that the delay would do more to damage the standing of the already controversial study. Leaders of UJC and a few members of the advisory committee, however, supported the move as a way to free the study from uncertainty. The same debate seems to be reemerging now.

The co-chair of the technical advisory committee, Vivian Klaff, told the Forward that he is not against the latest review but that the study’s results are sound enough to be released before the review is completed.

Even after the re-weighting is completed, “some things are not fixable,” Shapiro said. He said there are irreparable errors in the logging of some questions surrounding Jewish education and he criticized the decision to alter some 2000 survey questions from their 1990 versions, saying it prevented comparisons on important trend issues in the Jewish community.

“It’s not going to be perfect in the end,” Shapiro told the Forward. “And it’s not going to be totally uncontroversial in the end.”

— The JTA contributed to this report.

Find us on Facebook!
  • “You will stomp us into the dirt,” is how her mother responded to Anya Ulinich’s new tragicomic graphic novel. Paul Berger has a more open view of ‘Lena Finkle’s Magic Barrel." What do you think?
  • PHOTOS: Hundreds of protesters marched through lower Manhattan yesterday demanding an end to American support for Israel’s operation in #Gaza.
  • Does #Hamas have to lose for there to be peace? Read the latest analysis by J.J. Goldberg.
  • This is what the rockets over Israel and Gaza look like from space:
  • "Israel should not let captives languish or corpses rot. It should do everything in its power to recover people and bodies. Jewish law places a premium on pidyon shvuyim, “the redemption of captives,” and proper burial. But not when the price will lead to more death and more kidnappings." Do you agree?
  •'s Allison Benedikt wrote that Taglit-Birthright Israel is partly to blame for the death of American IDF volunteer Max Steinberg. This is why she's wrong:
  • Israeli soldiers want you to buy them socks. And snacks. And backpacks. And underwear. And pizza. So claim dozens of fundraising campaigns launched by American Jewish and Israeli charities since the start of the current wave of crisis and conflict in Israel and Gaza.
  • The sign reads: “Dogs are allowed in this establishment but Zionists are not under any circumstances.”
  • Is Twitter Israel's new worst enemy?
  • More than 50 former Israeli soldiers have refused to serve in the current ground operation in #Gaza.
  • "My wife and I are both half-Jewish. Both of us very much felt and feel American first and Jewish second. We are currently debating whether we should send our daughter to a Jewish pre-K and kindergarten program or to a public one. Pros? Give her a Jewish community and identity that she could build on throughout her life. Cons? Costs a lot of money; She will enter school with the idea that being Jewish makes her different somehow instead of something that you do after or in addition to regular school. Maybe a Shabbat sing-along would be enough?"
  • Undeterred by the conflict, 24 Jews participated in the first ever Jewish National Fund— JDate singles trip to Israel. Translation: Jews age 30 to 45 travelled to Israel to get it on in the sun, with a side of hummus.
  • "It pains and shocks me to say this, but here goes: My father was right all along. He always told me, as I spouted liberal talking points at the Shabbos table and challenged his hawkish views on Israel and the Palestinians to his unending chagrin, that I would one day change my tune." Have you had a similar experience?
  • "'What’s this, mommy?' she asked, while pulling at the purple sleeve to unwrap this mysterious little gift mom keeps hidden in the inside pocket of her bag. Oh boy, how do I answer?"
  • "I fear that we are witnessing the end of politics in the Israeli-Palestinian conflict. I see no possibility for resolution right now. I look into the future and see only a void." What do you think?
  • from-cache

Would you like to receive updates about new stories?

We will not share your e-mail address or other personal information.

Already subscribed? Manage your subscription.