Using Open Data To Understand War And Peace

A new project, funded by the U.S. Department of Defense, will make hard-to-find data on war and conflict available to academics–and help crowdsource military tactics.


Academic studies of wars and conflicts have been around for centuries, but a new one funded by the U.S. Defense Department could change our fundamental understanding of war and peace. The massive, publicly accessible conflict data archive called “The Empirical Studies of Conflict (ESOC)” project, headed up by Stanford and Princeton University academics, will also publish working papers and other research showing their findings.

To start, the ESOC project is analyzing conflicts in Iraq, Afghanistan, the Philippines (against the amazingly named MILF separatist group), Colombia, Northern Ireland, and Pakistan. According to project co-director Col. Joseph Felter of Stanford University, the choice of conflicts to be studied primarily reflects the availability of pre-existing data to collect and analyze–with priority given to “the ongoing conflict in Iraq and Afghanistan.”

ESOC is funded by a five-year, $8.6 million Defense Department grant. According to Stanford University literature, the Defense Department’s goal is to make hard-to-find data on conflicts and insurgencies available to the academic community.

Of course, the military isn’t paying to make all this conflict data available for altruistic reasons. By providing non-classified and declassified data sets and other information caches to academics and researchers, the Defense Department can essentially crowdsource research and analysis.

Traditionally, academics in the social sciences (anthropology, economics, political science, etc.) have hesitated to work with the Defense Department directly. Making conflict data publicly available allows the military to sidestep what they perceieve as academic bias.

Usable real-world results have already been obtained from the project. Felter told Fast Company that ESOC discovered a previously unnoticed–and counterintuitive–correlation between unemployment rates and politically motivated violence. According to Felter, “we looked at unemployment rates in Iraq, Afghanistan, and the Philippines and found that higher unemployment was associated with lower measures of politically motivated violence at the province level.”


Other ESOC studies found empirical data to back up military theories on troop size and on the use of impromptu humanitarian relief projects to prevent insurgent violence.

Data from ESOC will be made available to academics and the public through a yet-to-be-created website; access to some data, however, will require vetting by ESOC depending on government restrictions. A white paper on the project obtained by Fast Company notes that data to be made available to researchers will, in some cases, “be extracted, sanitized and compiled in a format suitable for analysis by the academic and policy community.” In other words, the project will be paying close attention to anyone accessing information of particular relevance to current combatants.

Both of the project’s directors come from a military background. Col. Felter was the former director of the Combating Terrorism Center at West Point, while co-director Jacob Shapiro is a Navy veteran and a professor at Princeton’s Woodrow Wilson School of Public and International Affairs.

Felter also notes that new discoveries made by the project could help improve American counter-insurgency doctrine. “Policy informed by anecdotal evidence alone or from experiences drawn from a limited number of cases has inherent risks. ESOC will help provide the quality micro-conflict data needed to conduct more robust analyses.”

The raw data being used by ESOC comes from a variety of sources. Apart from military and government records, the project also makes use of more unorthodox sources–among other things, more than 700 files belonging to Al Qaeda in Iraq and battlefront materials from a variety of conflicts.

[Image: U.S. Army]


For more stories like this, follow @fastcompany on Twitter. Email Neal Ungerleider, the author of this article, here or find him on Twitter and Google+.