Can New Number Crunchers Predict Upcoming Genocides?
Crowd-sourcing Mass Killing: Experts in genocide have quantified predictions of mass slaughter. Image by Jay Ulfelder
By Nathan Guttman June 9, 2014
Danger Levels: In recent months, a pool of experts gave these percentages to express the probability of these countries having a ‘mass killing event’ before January 2015. By May 2014, events in South Sudan had already qualified as a mass killing. Image by Getty Images
Where will the world’s next genocide take place?
Up until now, the idea of predicting such an event with near certainty seemed like a grim pipe dream of somber policy wonks.
But if new predictive methodologies now being developed by researchers centered at Dartmouth College bear out, policymakers should be worried right now about Burma.
The research group’s crowd-sourced survey of area experts, which has accurately predicted recent genocides in South Sudan and the Central African Republic, shows the probability for a genocide in Burma over the next year rising for the first time, to slightly more than 50%.
Part of the terrifying nature of many outbreaks of genocide is their suddenness, when years of tension peak abruptly in mass waves of slaughter. Meanwhile, in other societies, simmering tensions merely continue to simmer.
Statistician Nate Silver stunned the political world when he out-predicted all pundits and experts by employing quantitative measures to forecast precisely the results of the 2012 presidential election. And now, the research group, with support from the United States Holocaust Memorial Museum, is developing a similar methodology to forecast outbreaks of mass atrocities and genocide before they are even seen on the horizon.
The power of endless data now available in the Internet era, and the growing field of crowd sourcing, these researchers say, can provide decision-makers with early warnings for such outbreaks. And advance knowledge of these signs, they hope, could put conflict areas on the radar screen of decision-makers before events on the ground deteriorate beyond all remedy.
According to Cameron Hudson, director of the Holocaust Museum’s Center for the Prevention of Genocide, a retrospective test of the researchers’ system provides strong support for its efficacy.
Those working on the project recently ran a retroactive data analysis going back five decades and found, among other things, that the 1994 Rwanda genocide could have been predicted a year before events broke out, with 95% certainty. One of the reasons the United States did not intervene during those horrific events was the perception that by the time the full scope and genocidal nature of the atrocity was clear, it was too late to stop it.
“Looking back, we’ve seen that every case of mass violence from the 1960s was listed as being in the highest risk,” Hudson said.
But would this information, if available, stop the atrocities?
“You have to have early warning, but you also have to have a response,” Hudson acknowledged. “What early warning gives us is more possibilities for effective response.”
Creating an early warning system for genocide prevention was among the key recommendations in a 2008 report of a special task force convened by the Holocaust museum and co-chaired by former Clinton administration Cabinet secretaries Madeleine Albright and William Cohen. “Effective early warning does not guarantee successful prevention, but if warning is absent, slow, inaccurate, or indistinguishable from the ‘noise’ of regular reporting, failure is virtually guaranteed,” the report stated.
More than two years into the process, the team working on developing a mass killing early warning system believes it has made significant progress toward a successful forecast. An initial version of the system is already operational, and the team plans to launch a public website by the end of the year. The project is a partnership between the Holocaust museum and Dartmouth College, and is funded by private donations. Its objective is to employ prediction methods already used in the fields of politics, sports and business for atrocity forecasts.
To achieve this goal, the team has been working on three programs: One, the crowd-sourcing component, pools together a large number of experts to assess the current risks of genocide across the world; another seeks to aggregate data on mass killings of the past to learn which societal elements could be useful in predicting similar events in the future, and the third uses data mining techniques to see how Internet chatter that includes certain references or keywords can serve as a precursor for violent events to come.
An initial progress report published in May shows real success in using expert crowd sourcing to predict mass killings.
The part of the program pools together dozens of experts to produce a combined assessment of the risk of genocide outbreak in specific conflict areas. “We know that individual experts are not very good at predicting rare and complex events, but we also know that groups are pretty good at predicting,” said political scientist Jay Ulfelder, who leads the program. Ulfelder, a specialist in political and social forecasting, explained that “each individual expert has his own bias, but when you aggregate them, the biases tend to cancel out.”
To build a crowd-sourcing community for the project, Ulfelder and his team created a list of more than 100 experts who weigh in collectively. Each has access to a private website in which questions, such as “Will an episode of state-led mass killing occur in Russia’s North Caucuses region?” are posed. The experts fill in their forecast on a scale of 0–100%, and their views are added to those of other experts. They can update their forecast as circumstances change.
“The idea is to take a group of people who spend time every day thinking about a certain conflict and have added insight, and to aggregate their thoughts,” said Daniel Solomon, one of the experts in the pool. Solomon, 23, has been working on genocide prevention issues in college for more than four years. He said his motivation for participating in the program, which does not provide compensation and does not include financial incentives like other market prediction systems, is primarily “an investment in making this prediction system work.”
After seven months of operating, and even before reaching a full pool of experts, the system has already proved accurate. Two of the conflicts assigned with a 75% risk or higher, South Sudan and the Central African Republic, experienced outbreaks of mass killings within seven and two months, respectively, after being assessed as high-risk conflicts. Other regions that were estimated to have a lower than 50% risk of mass killings, such as Iraq, Pakistan and Ethiopia, did not experience genocidal episodes. The definition of a mass killing event, for the purpose of this prediction system, is an episode of sustained violence in which at least 1,000 noncombatant civilians from a discrete group are intentionally killed within a period of one year or less.
Moving forward, the program seeks to recruit more experts, leading to an eventual pool of several hundred. The program’s managers then could also calculate each expert’s success rate and weigh his or her predictions accordingly. The forecasts will be published on an open website by the end of the year, and will be accessible not only to governments, but also to any stakeholder who could help prevent genocides, such as nongovernmental organizations, policy experts and the media.
“For us, success will be if these warnings prove to be accurate and draw the attention of people in the policy and decision communities to take action,” Ulfelder said.
For all its innovative aspects, Ulfelder’s crowd-sourcing project remains based on human assessments, even if taken in aggregate. Other components of the genocide prediction program rely solely on statistical data.
By compiling available data from the past 50 years of known mass killings, the program’s second component seeks to identify categories of information most worth watching when forecasting mass killings. In the relatively young academic field of genocide prevention, there are several theories regarding which elements predict most accurately a genocidal outburst. These include the type of regime, divisions (ethnic, religious, economic) within the country, the role played by the military and the state of the country’s economy. This part of the program attempts to quantify which of these elements or combinations of elements correlate most closely with genocide or mass killings.
With this information in hand, the researchers will use the idea of big data mining to learn about contemporary trends in societies worldwide and identify those that could soon face a genocide. The researchers are just now starting on this phase.
In the program’s third component, researchers hope to use the process of automatically analyzing billions of information pieces, either on social media or in any other news source available on the Web, for keywords to identify trends in various countries and societies. It’s a technique that governments already use to identify terror plots, and one that commercial companies exploit to spot consumer trends. By finding the correlation between certain types of communications and potential mass killings, the system could provide another useful tool for predicting genocide.
While proponents of all these modern-day prediction tools see great promise in them, some caution against relying solely on science and statistics. Simon Adams, executive director of the Global Centre for the Responsibility To Protect, a group that advocates for intervention against mass atrocities, warned against “the idea that we can crack the code, and that we only need the right data set to prevent the next Auschwitz or the next Darfur.”
Genocide prediction is a mixture of science and art, Adams said, and quantitative analysis should not overwhelm qualitative human judgments. Still, he said of the genocide prediction project, “I think the effort should be applauded.”
The team working on the project says that it does not seek to replace experts and qualitative human judgments. It merely hopes to add an effective tool at a low cost. Hudson noted that the entire budget of the genocide prediction project is around $750,000 a year, a mere fraction of the cost of any American military intervention to stop genocide after it breaks out. “Prevention is a lot more affordable,” he said.
Contact Nathan Guttman at [email protected] or on Twitter, @nathanguttman
A message from our Publisher & CEO Rachel Fishman Feddersen
I hope you appreciated this article. Before you go, I’d like to ask you to please support the Forward’s award-winning, nonprofit journalism during this critical time.
We’ve set a goal to raise $260,000 by December 31. That’s an ambitious goal, but one that will give us the resources we need to invest in the high quality news, opinion, analysis and cultural coverage that isn’t available anywhere else.
If you feel inspired to make an impact, now is the time to give something back. Join us as a member at your most generous level.
— Rachel Fishman Feddersen, Publisher and CEO
