WASHINGTON — A recent study that was widely presented as documenting the success of an all-Christian rehabilitation program in a Texas state prison — and serving as proof that the Bush administration-backed programs should be expanded — actually shows that the program is a failure.
The two-year study (2000-2002), published jointly by the University of Pennsylvania and the Manhattan Institute, examined a program run by the evangelical Innerchange Freedom Initiative in a Texas state prison. The IFI program involves 16 to 24 months of in-prison biblical education, work and community service, and six to 12 months of aftercare in which each participant must hold a job and be an active church member for 3 consecutive months following his or her release from prison.
According to an IFI press release issued in late June as the study was published, the study shows a significant linkage between faith-based mentoring and decreased recidivism rates. According to the press release, “when IFI graduates are compared to a similar group of released inmates (controlled for race, age, offense type, and salient risk factor score) that met IFI program criteria but did not enter the program, program graduates were 50% less likely to be arrested, and 60% less likely to become reincarcerated.”
Following the study’s publication, IFI founder Chuck Colson, who was White House counsel under former president Nixon and spent seven months in prison for his part in the Watergate affair, met in the White House with President Bush, and was warmly praised for his apparent success by the director of the White House Office of Faith-Based and Community Initiatives, Jim Towey.
Last week, however, in an article published by the Internet news magazine Slate, UCLA social studies professor Mark A.R. Kleiman pointed out that the study does not exactly live up to this claim. In fact, prisoners who enrolled in the program were not better rehabilitated than those who didn’t; only prisoners who went on to graduate from the program — a minority (75 out of 177) — did better.
“Graduation involved sticking with the program, not only in prison but after release,” Kleiman wrote, “Naturally, the graduates did better than the control group…. the InnerChange cheerleaders simply ignored the other 102 participants who dropped out, were kicked out, or got early parole and didn’t finish. Naturally, the non-graduates did worse than the control group. If you select out the winners, you leave mostly losers.”
In fact, after scrutinizing the study’s results and methodology, Kleiman concluded that “overall, the 177 entrants did a little bit worse than the controls. That result ought to discourage InnerChange’s advocates, but it doesn’t because they have just ignored the failure of the failures and focused on the success of the successes.”
The study’s author, Byron Johnson, director of the Center for Research on Religion and Urban Civil Society at Pennsylvania University, was not available for reaction. IFI also declined to react, explaining that all its staff members were away for their summer convention.
Marc Stern, assistant executive director of the American Jewish Congress and a longtime critic of the Bush administration’s faith-based initiative, said that this is not the first study showing that “faith-based programs don’t have any particular advantage over their secular counterparts.” Despite conclusive evidence to the contrary, “there is a myth out there that faith-based programs are better, and public policy cannot rely on urban legends,” Stern said.
UCLA’s Kleiman, in an interview with the Forward, said that the methodological flaw in the study is not unique. Kleiman, who specializes in rehabilitation research, said that the flaw, known among statisticians as “selection bias,” is commonplace in this science, because researchers often focus on “completers” of rehabilitation programs, discounting the dropouts. “Cherry-picking is one well-known way to make your program look as if it works,” he said.