Category Archives: Early Warning

How Can Innovative Technology Make Conflict Prevention More Effective?

I’ve been asked to participate in an expert working group in support of a research project launched by the International Peace Institute (IPI) on new technologies for conflict prevention. Both UNDP and USAID are also partners in this effort. To this end, I’ve been invited to make some introductory remarks during our upcoming working group meeting. The purpose of this blog post is to share my preliminary thoughts on this research and provide some initial suggestions.

Before I launch into said thoughts, some context may be in order. I spent several years studying, launching and improving conflict early warning systems for violence prevention. While I haven’t recently blogged about conflict prevention on iRevolution, you’ll find my writings on this topic posted on my other blog, Conflict Early Warning. I have also published and presented several papers on conflict prevention, most of which are available here. The most relevant ones include the following:

  • Meier, Patrick. 2011. Early Warning Systems and the Prevention of Violent Conflict. In Peacebuilding in the Information Age: Sifting Hype from Reality, ed. Daniel Stauffacher et al. GenevaICT4Peace. Available online.
  • Leaning, Jennifer and Patrick Meier. 2009. “The Untapped Potential of Information Communication Technology for Conflict Early Warning and Crisis Mapping,” Working Paper Series, Harvard Humanitarian Initiative (HHI), Harvard University. Available online.
  • Leaning, Jennifer and Patrick Meier. 2008. “Community Based Conflict Early Warning and Response Systems: Opportunities and Challenges.” Working Paper Series, Harvard Humanitarian Initiative (HHI), Harvard University. Available online.
  • Leaning, Jennifer and Patrick Meier. 2008. “Conflict Early Warning and Response: A Critical Reassessment.” Working Paper Series, Harvard Humanitarian Initiative (HHI), Harvard University. Available online.
  • Meier, Patrick. 2008. “Upgrading the Role of Information Communication Technology (ICT) for Tactical Early Warning/Response.” Paper prepared for the 49th Annual Convention of the International Studies Association (ISA) in San Francisco. Available online.
  • Meier, Patrick. 2007. “New Strategies for Effective Early Response: Insights from Complexity Science.” Paper prepared for the 48th Annual Convention of the International Studies Association (ISA) in Chicago.Available online.
  • Campbell, Susanna and Patrick Meier. 2007. “Deciding to Prevent Violent Conflict: Early Warning and Decision-Making at the United Nations.” Paper prepared for the 48th Annual Convention of the International Studies Association (ISA) in Chicago. Available online.
  • Meier, Patrick. 2007. From Disaster to Conflict Early Warning: A People-Centred Approach. Monday Developments 25, no. 4, 12-14. Available online.
  • Meier, Patrick. 2006. “Early Warning for Cowardly Lions: Response in Disaster & Conflict Early Warning Systems.” Unpublished academic paper, The Fletcher SchoolAvailable online.
  • I was also invited to be an official reviewer of this 100+ page workshop summary on “Communication and Technology for Violence Prevention” (PDF), which was just published by the National Academy of Sciences. In addition, I was an official referee for this important OECD report on “Preventing Violence, War and State Collapse: The Future of Conflict Early Warning and Response.”

An obvious first step for IPI’s research would be to identify the conceptual touch-points between the individual functions or components of conflict early warning systems and information & communication technology (ICT). Using this concep-tual framework put forward by ISDR would be a good place to start:

That said, colleagues at IPI should take care not to fall prey to technological determinism. The first order of business should be to understand exactly why previous (and existing) conflict early warning systems are complete failures—a topic I have written extensively about and been particularly vocal on since 2004. Throwing innovative technology at failed systems will not turn them into successful operations. Furthermore, IPI should also take note of the relatively new discourse on people-centered approaches to early warning and distinguish between first, second, third and fourth generation conflict early warning systems.

On this note, IPI ought to focus in particular on third and fourth generation systems vis-a-vis the role of innovative technology. Why? Because first and second generation systems are structured for failure due to constraints explained by organizational theory. They should thus explore the critical importance of conflict preparedness and the role that technology can play in this respect since preparedness is key to the success of third and fourth generation systems. In addition, IPI should consider the implications of crowdsourcing, crisis mapping, Big Data, satellite imagery and the impact that social media analytics might play for the early detection and respons to violent conflict. They should also take care not to ignore critical insights from the field of nonviolent civil resistance vis-a-vis preparedness and tactical approaches to community-based early response. Finally, they should take note of new and experimental initiatives in this space, such as PeaceTXT.

IPI’s plans to write up several case studies on conflict early warning systems to understand how innovative technology might (or already are) making these more effective. I would recommend focusing on specific systems in Kenya, Kyrgyzstan Sri Lanka and Timor-Leste. Note that some community-based systems are too sensitive to make public, such as one in Burma for example. In terms of additional experts worth consulting, I would recommend David Nyheim, Joe Bock, Maria Stephan, Sanjana Hattotuwa, Scott Edwards and Casey Barrs. I would also shy away from inviting too many academics or technology companies. The former tend to focus too much on theory while the latter often have a singular focus on technology.

Many thanks to UNDP for including me in the team of experts. I look forward to the first working group meeting and reviewing IPI’s early drafts. In the meantime, if iRevolution readers have certain examples or questions they’d like me to relay to the working group, please do let me know via the comments section below and I’ll be sure to share.

Behind the Scenes: The Digital Operations Center of the American Red Cross

The Digital Operations Center at the American Red Cross is an important and exciting development. I recently sat down with Wendy Harman to learn more about the initiative and to exchange some lessons learned in this new world of digital  humanitarians. One common challenge in emergency response is scaling. The American Red Cross cannot be everywhere at the same time—and that includes being on social media. More than 4,000 tweets reference the Red Cross on an average day, a figure that skyrockets during disasters. And when crises strike, so does Big Data. The Digital Operations Center is one response to this scaling challenge.

Sponsored by Dell, the Center uses customized software produced by Radian 6 to monitor and analyze social media in real-time. The Center itself sits three people who have access to six customized screens that relate relevant information drawn from various social media channels. The first screen below depicts some of key topical areas that the Red Cross monitors, e.g., references to the American Red Cross, Storms in 2012, and Delivery Services.

Circle sizes in the first screen depict the volume of references related to that topic area. The color coding (red, green and beige) relates to sentiment analysis (beige being neutral). The dashboard with the “speed dials” right underneath the first screen provides more details on the sentiment analysis.

Lets take a closer look at the circles from the first screen. The dots “orbiting” the central icon relate to the categories of key words that the Radian 6 platform parses. You can click on these orbiting dots to “drill down” and view the individual key words that make up that specific category. This circles screen gets updated in near real-time and draws on data from Twitter, Facebook, YouTube, Flickr and blogs. (Note that the distance between the orbiting dots and the center does not represent anything).

An operations center would of course not be complete without a map, so the Red Cross uses two screens to visualize different data on two heat maps. The one below depicts references made on social media platforms vis-a-vis storms that have occurred during the past 3 days.

The screen below the map highlights the bio’s of 50 individual twitter users who have made references to the storms. All this data gets generated from the “Engagement Console” pictured below. The purpose of this web-based tool, which looks a lot like Tweetdeck, is to enable the Red Cross to customize the specific types of information they’re looking form, and to respond accordingly.

Lets look at the Consul more closely. In the Workflow section on the left, users decide what types of tags they’re looking for and can also filter by priority level. They can also specify the type of sentiment they’re looking, e.g., negative feelings vis-a-vis a particular issue. In addition, they can take certain actions in response to each information item. For example, they can reply to a tweet, a Facebook status update, or a blog post; and they can do this directly from the engagement consul. Based on the license that the Red Cross users, up to 25 of their team members can access the Consul and collaborate in real-time when processing the various tweets and Facebook updates.

The Consul also allows users to create customized timelines, charts and wordl graphics to better understand trends changing over time in the social media space. To fully leverage this social media monitoring platform, Wendy and team are also launching a digital volunteers program. The goal is for these volunteers to eventually become the prime users of the Radian platform and to filter the bulk of relevant information in the social media space. This would considerably lighten the load for existing staff. In other words, the volunteer program would help the American Red Cross scale in the social media world we live in.

Wendy plans to set up a dedicated 2-hour training for individuals who want to volunteer online in support of the Digital Operations Center. These trainings will be carried out via Webex and will also be available to existing Red Cross staff.

As  argued in this previous blog post, the launch of this Digital Operations Center is further evidence that the humanitarian space is ready for innovation and that some technology companies are starting to think about how their solutions might be applied for humanitarian purposes. Indeed, it was Dell that first approached the Red Cross with an expressed interest in contributing to the organization’s efforts in disaster response. The initiative also demonstrates that combining automated natural language processing solutions with a digital volunteer net-work seems to be a winning strategy, at least for now.

After listening to Wendy describe the various tools she and her colleagues use as part of the Operations Center, I began to wonder whether these types of tools will eventually become free and easy enough for one person to be her very own operations center. I suppose only time will tell. Until then, I look forward to following the Center’s progress and hope it inspires other emergency response organizations to adopt similar solutions.

Twitcident: Filtering Tweets in Real-Time for Crisis Response

The most recent newcomer to the “tweetsourcing” space comes to us from Delft University of Technology in the Netherlands. Twitcident is a web-based filtering system that extracts crisis information from Twitter in real-time to support emergency response efforts. Dutch emergency services have been testing the platform over the past 10 months and results “show the system to be far more useful than simple keyword searching of a twitter feed” (NewScientist).

Here’s how it works. First the dashboard, which shows current events-of-interest being monitored.

Lets click on “Texas”, which produces the following page. More than 22,000 tweets potentially relate to the actual fire of interest.

This is where the filtering comes in:

The number of relevant tweets is reduced with every applied filter.

Naturally, geo-location is also an optional filter.

Twitcident also allows for various visualization options, including timelines, word clouds and charts.

The system also allows the user to view the filtered tweets on a map. The pictures and videos shared via twitter are also aggregated and viewable on dedicated tabs.

The developers of the platform have not revealed how their algorithms work but will demo the tool at the World Wide Web 2012 conference in France next week. In the meantime, here’s a graphic that summarizes the platform workflow.

I look forward to following Twitcident’s developments. I’d be particularly interested in learning more about how Dutch emergency services have been using the tool and what features they think would improve the platform’s added value.

SMS for Violence Prevention: PeaceTXT International Launches in Kenya

[Cross-posted from my post on the Ushahidi blog]

One of the main reasons I’m in Nairobi this month is to launch PeaceTXT International with PopTech, Praekelt Foundation, Sisi ni Amani and several other key partners. PeaceTXT International is a spin-off from the original PeaceTXT project that several of us began working on with CeaseFire Chicago last year. I began thinking about the many possible international applications of the PeaceTXT project during our very first meeting, which is why I am thrilled and honored to be spearheading the first PeaceTXT International pilot project.

The purpose of PeaceTXT is to leverage mobile messaging to catalyze behavior change around peace and conflict issues. In the context of Chicago, the joint project with CeaseFire aims to leverage SMS reminders to interrupt gun violence in marginalized neighborhoods. Several studies in other fields of public health have already shown the massive impact that SMS reminders can have on behavior change, e.g., improving drug adherence behavior among AIDS and TB patients in Africa, Asia and South America.

Our mobile messaging campaign in Chicago builds on another very successful one in the US: “Friends Don’t Let Friends Drink and Drive.” Inspired by this approach, the PeaceTXT Team is looking to launch a friends-don’t-let-friends-get-killed campaign. Focus groups recently conducted with high-risk individuals have resulted in rich content for several dozen reminder messages (see below) that could be disseminated via SMS. Note that CeaseFire has been directly credited for significantly reducing the number of gun-related killings in Chicago over the past 10 years. In other words, they have a successful and proven methodology; one being applied to several other cities and countries worldwide. PeaceTXT simply seeks to scale this success by introducing SMS.

These messages are user-generated in that the content was developed by high-risk individuals themselves—i.e., those most likely to get involved in gun violence. The messages are not limited to reminders. Some also prompt the community to get engaged by responding to various questions. Indeed, the project seeks to crowdsource community solutions to gun violence and thus greater participation. When high-risk individuals were asked how they’d feel if they were to receive these messages on their phones, they had the following to share: “makes me feel like no one is forgetting about me”; “message me once a day to make a difference.”

Given that both forwarding and saving text messages is very common among the population that CeaseFire works with, the team hopes that the text messages will circulate and recycle widely. Note that the project is still in prototype phase but going into implementation mode as of 2012. So we’ll have to wait and see how the project fares and what the initial impact looks like.

In the meantime, PeaceTXT is partnering with Sisi ni Amani (We are Peace) to launch its first international pilot project. Rachel Brown, who spearheads the initiative, first got in touch with me back in the Fall of 2009 whilst finishing her undergraduate studies at Tufts. Rachel was interested in crowdsourcing a peace map of Kenya, which I blogged about here shortly after our first conversation. Since then, Rachel and her team have set up the Kenyan NGO Sisi ni Amani Kenya (SnA-K) to leverage mobile technology for awareness raising and civic engagement with the aim of preventing possible violence during next year’s Presidential Elections.

SnA-K currently manages a ~10,000 member SMS subscriber list in Baba Dogo and Korogocho, Kamukunji and Narok. SnA-K’s SMS campaigns focus on voter education, community cohesion and rumor prevention. What SnA-K needs, how-ever, is the scalable SMS broadcasting technology, the type of focus that PeaceTXT brought to CeaseFire Chicago and the unique response methodology developed by the CeaseFire team. So I reached out to Rachel early on during the work in Chicago to let her know about PeaceTXT and to gain insights from her projects in Kenya. We set up regular conference calls throughout the year to keep each other informed of our respective progress and findings.

Soon enough, PopTech’s delightful Leetha Filderman asked me to put together a pitch for international applications of PeaceTXT’s work, an initiative I have “code-named” PeaceTXT International. I was absolutely thrilled when she shared the good news at PopTech 2011 that our donor, the Rita Allen Foundation, had provided us with additional funding, some of which could go towards an international pilot project. Naturally, Sisi ni Amani was a perfect fit.

So we organized a half-day brainstorming session at the iHub last week to chart the way forward on PeaceTXT Kenya. For example, what is the key behavioral change variable (like friendship in Chicago) that is most likely to succeed in Kenya? As for interrupting violence, how can the CeaseFire methodology be customized for the SnA-K context? Finally, what kind of SMS broadcasting technology do we need to have in place to provide maximum flexibility and scalability earlier rather than later? Answering these questions and implementing scalable solutions essentially forms the basis of the partnership between SnA-K and PeaceTXT (which also includes Mobile:Medic & Revolution Messaging). We have some exciting leads on next steps and will be sure to blog about them as we move forward to get feedback from the wider community.

Conflicts are often grounded in the stories and narratives that people tell themselves and the emotions that these stories generate. Narratives shape identity and the social construct of reality—we interpret our lives through stories. These have the power to transform relationships and communities. We believe the PeaceTXT model can be applied to catalyze behavior  change vis-a-vis peace and conflict issues at the community level by amplifying new narratives via SMS. There is considerable potential here and still much to learn, which is why I’m thrilled to be working with SnA, PopTech & partners on launching our first international pilot project: PeaceTXT Kenya.

Using Ushahidi Data to Study the Micro-Dynamics of Violent Conflict

The field of conflict analysis has long been handicapped by the country-year straightjacket. This is beginning to change thanks to the increasing availability of subnational and sub-annual conflict data. In the past, one was limited to macro-level data, such as the number of casualties resulting from violent conflict in a given county and year. Today, datasets such as the Armed Conflict Location Event Data (ACLED) provide considerably more temporal and spatial resolution. Another example is this quantitative study: “The Micro-dynamics of Reciprocity in an Asymmetric Conflict: Hamas, Israel, and the 2008-2009 Gaza Conflict,” authored by by NYU PhD Candidate Thomas Zeitzoff.

Picture 5

I’ve done some work on conflict event-data and reciprocity analysis in the past (such as this study of Afghanistan), but Thomas is really breaking new ground here with the hourly temporal resolution of the conflict analysis, which was made possible by Al-Jazeera’s War on Gaza project powered by the Ushahidi platform.


The Gaza Conflict (2008-2009) between Hamas and Israel was de fined the participants’ strategic use of force. Critics of Israel point to the large number of Palestinian casualties compared to Israelis killed as evidence of a disproportionate Israeli response. I investigate Israeli and Hamas response patterns by constructing a unique data set of hourly conflict intensity scores from new social media and news source over the nearly 600 hours of the conflict. Using vector autoregression techniques (VAR), I fi nd that Israel responds about twice as intensely to a Hamas escalation as Hamas responds to an Israeli escalation. Furthermore, I find that both Hamas’ and Israel’s response patterns change once the ground invasion begins and after the UN Security Council votes. (Study available as PDF here).

As Thomas notes, “Ushahidi worked with Al-Jazeera to track events on the ground in Gaza via SMS messages, email, or the web. Events were then sent in by reporters and civilians through the platform and put into a Twitter feed entitled AJGaza, which gave the event a time stamp. By cross-checking with other sources such as Reuters, the UN, and the Israeli newspaper Haaretz, I was able see that the time stamp was usually within a few minutes of event occurrence.”

Key Highlights from the study:

  • Hamas’ cumulative response intensity to an Israeli escalation decreases (by about 17 percent) after the ground invasion begins. Conversely, Israel’s cumulative response intensity after the invasion increases by about three fold.
  • Both Hamas and Israel’s cumulative response drop after the UN Security Council vote on January 8th, 2009 for an immediate cease-fi re, but Israel’s drops more than Hamas (about 30 percent to 20 percent decrease).
  • For the period covering the whole conflict, Hamas would react (on average) to a “surprise” 1 event (15 minute interval) of Israeli misinformation/psy-ops with the equivalent of 1 extra incident of mortar re/endangering civilians.
  • Before the invasion, Hamas would respond to a 1 hour shock of targeted air strikes with 3 incidents of endangering civilians. Comparatively, after the invasion, Hamas would only respond to that same Israeli shock with 3 incidents of psychological warfare.
  • The results con firm my hypotheses that Israel’s reactions were more dependent upon Hamas and that these responses were contextually dependent.
  • Wikipedia’s Timeline of the 2008-2009 Gaza Conflict was particularly helpful in sourcing and targeting events that might have diverging reports (i.e. controversial).

[An earlier version of this blog post appeared on my Early Warning blog]

The Mathematics of War: On Earthquakes and Conflicts

A conversation with my colleague Sinan Aral at PopTech 2011 reminded me of some earlier research I had carried out on the mathematics of war. So this is a good time to share some of the findings from this research. The story begins some 60 years ago, when British physicist Lewis Fry Richardson found that international wars follow what is called a power law distribution. A power law distribution relates the frequency and “magnitude” of events. For example, the Richter scale, relates the size of earthquakes to their frequency. Richardson found that the frequency of international wars and the number of causalities each produced followed a power law.

More recently, my colleague Erik-Lars Cederman sought to explain Richardson’s findings in his 2003 peer-reviewed publication “Modeling the Size of Wars: From Billiard Balls to Sandpiles.” However, Lars used an invalid statistical technique to test for power law distributions. In 2005, I began collaborating with Pro-fessors Neil Johnson and Michael Spagat on related research after I came across their fascinating co-authored study that tested casualty distributions in new wars (internal conflicts) for power laws. Though he was not a co-author on the 2005 study, my colleague Sean Gourely presented this research at TED in 2009.

In any case, I invited Michael to present his research at The Fletcher School in the Fall of 2005 to generate interest here. Shortly after, I suggested to Michael that we test whether conflict events, in addition to casualties, followed a power law distribution. I had access to an otherwise proprietary dataset on conflict events that spanned a longer time period than the casualty datasets that he and Neils were working off. I also suggested we try to test whether casualties from natural disasters follow a power law distribution.

We chose to pursue the latter first and I submitted an abstract to the 2006 American Political Science Association (APSA) conference to present our findings. Soon after, I was accepted to the Santa Fe Institute’s Complex Systems Summer Institute for PhD students and took the opportunity to pursue my original research in testing conflict events for power law distributions with my colleague Dr. Ryan Woodard.

The APSA paper, presented in August 2006, was entitled “Natural Disasters, Casualties and Power Laws:  A Comparative Analysis with Armed Conflict” (PDF). Here is the paper’s abstract and findings:

Power-law relationships, relating events with magnitudes to their frequency, are common in natural disasters and violent conflict. Compared to many statistical distributions, power laws drop off more gradually, i.e. they have “fat tails”. Existing studies on natural disaster power laws are mostly confined to physical measurements, e.g., the Richter scale, and seldom cover casualty distributions. Drawing on the Center for Research on the Epidemiology of Disasters (CRED) International Disaster Database, 1980 to 2005, we find strong evidence for power laws in casualty distributions for all disasters combined, both globally and by continent except for North America and non-EU Europe. This finding is timely and gives useful guidance for disaster preparedness and response since natural catastrophes are increasing in frequency and affecting larger numbers of people.  We also find that the slopes of the disaster casualty power laws are much smaller than those for modern wars and terrorism, raising an open question of how to explain the differences. We show that many standard risk quantification methods fail in the case of natural disasters.


Dr. Woodard and I presented our research on power laws and conflict events at SFI in June 2006. We produced a paper in August of that year entitled “Concerning Critical Correlations in Conflict, Cooperation and Casualties” (PDF). As the title implies, we also tested whether cooperative events followed a power law. As far as I know, we were the first to test conflict events not to mention cooperative events for power laws. In addition, we looked at conflict/cooperation (C/C) events in Western countries.

The abstract and some findings are included below:

Knowing that the number of casualties of war are distributed as a power law and given a rich data set of conflict and cooperation (C/C) events, we ask: Are there correlations among C/C events? Is there a correlation between C/C events and war casualties? Can C/C data be used as proxy for (potentially) less reliable casualty data? Can C/C data be used in conflict early warning systems? To begin to answer these questions we analyze the distribution of C/C event data for the period 1990–2004 in Afghanistan, Colombia, Iran, Iraq, North Korea, Switzerland, UK and USA. We find that the distributions of individual C/C event types scale as power laws, but only over approximately a single decade, leaving open the possibility of a more appropriate fit (for which we have not yet tested). However, the average exponent of the power law (2.5) is the same as that found in recent studies of casualties of war. We find low levels of correlations between C/C events in Iraq and Afghanistan but not in the other countries studied. We find that the distribution of the sum of all conflict or cooperation events scales exponentially. Finally, we find low levels of correlations between a two year time series of casualties in Afghanistan and the corresponding conflict events.


I’m looking to discuss all this further with Sinan and learning more about his fascinating area of research.

Detecting Emerging Conflicts with Web Mining and Crisis Mapping

My colleague Christopher Ahlberg, CEO of Recorded Future, recently got in touch to share some exciting news. We had discussed our shared interests a while back at Harvard University. It was clear then that his ideas and existing technologies were very closely aligned to those we were pursuing with Ushahidi’s Swift River platform. I’m thrilled that he has been able to accomplish a lot since we last spoke. His exciting update is captured in this excellent co-authored study entitled “Detecting Emergent Conflicts Through Web Mining and Visualization” which is available here as a PDF.

The study combines almost all of my core interests: crisis mapping, conflict early warning, conflict analysis, digital activism, pattern recognition, natural language processing, machine learning, data visualization, etc. The study describes a semi-automatic system which automatically collects information from pre-specified sources and then applies linguistic analysis to user-specified extract events and entities, i.e., structured data for quantitative analysis.

Natural Language Processing (NLP) and event-data extraction applied to crisis monitoring and analysis is of course nothing new. Back in 2004-2005, I worked for a company that was at the cutting edge of this field vis-a-vis conflict early warning. (The company subsequently joined the Integrated Conflict Early Warning System (ICEWS) consortium supported by DARPA). Just a year later, Larry Brilliant told TED 2006 how the Global Public Health Information Net-work (GPHIN) had leveraged NLP and machine learning to detect an outbreak of SARS 3 months before the WHO. I blogged about this, Global Incident Map, European Media Monitor (EMM), HavariaHealthMap and Crimson Hexagon back in 2008. Most recently, my colleague Kalev Leetaru showed how applying NLP to historical data could have predicted the Arab Spring. Each of these initiatives represents an important effort in leveraging NLP and machine learning for early detection of events of interest.

The RecordedFuture system works as follows. A user first selects a set of data sources (websites, RSS feeds, etc) and determines the rate at which to update the data. Next, the user chooses one or several existing “extractors” to find specific entities and events (or constructs a new type). Finally, a taxonomy is selected to specify exactly how the data is to be grouped. The data is then automatically harvested and passed through a linguistics analyzer which extracts useful information such as event types, names, dates, and places. Finally, the reports are clustered and visualized on a crisis map, in this case using an Ushahidi platform. This allows for all kinds of other datasets to be imported, compared and analyzed, such as high resolution satellite imagery and crowdsourced data.

A key feature of the RecordedFuture system is that extracts and estimates the time for the event described rather than the publication time of the newspaper article parsed, for example. As such, the harvested data can include both historic and future events.

In sum, the RecordedFuture system is composed of the following five features as described in the study:

1. Harvesting: a process in which text documents are retrieved from various sources and stored in the database. The documents are stored for long-term if permitted by terms of use and IPR legislation, otherwise they are only stored temporarily for the needed analysis.

2. Linguistic analysis: the process in which the retrieved texts are analyzed in order to extract entities, events, time and location, etc. In contrast to other components, the linguistic analysis is language dependent.

3. Refinement: additional information can be obtained in this process by synonym detection, ontology analysis, and sentiment analysis.

4. Data analysis: application of statistical and AI-based models such as Hidden Markov Models (HMMs) and Artificial Neural Networks (ANNs) to generate predictions about the future and detect anomalies in the data.

5. User experience: a web interface for ordinary users to interact with, and an API for interfacing to other systems.

The authors ran a pilot that “manually” integrated the RecordedFuture system with the Ushahidi platform. The result is depicted in the figure below. In the future, the authors plan to automate the creation of reports on the Ushahidi platform via the RecordedFuture system. Intriguingly, the authors chose to focus on protest events to demo their Ushahidi-coupled system. Why is this intriguing? Because my dissertation analyzed whether access to new information and communication technologies (ICTs) are statistically significant predictors of protest events in repressive states. Moreover, the protest data I used in my econometric analysis came from an automated NLP algorithm that parsed Reuters Newswires.

Using RecordedFuture, the authors extracted some 6,000 protest event-data for Quarter 1 of 2011. These events were identified and harvested using a “trained protest extractor” constructed using the system’s event extractor frame-work. Note that many of the 6,000 events are duplicates because they are the same events but reported by different forces. Not surprisingly, Christopher and team plan to develop a duplicate detection algorithm that will also double as a triangulation & veracity scoring feature. I would be particularly interested to see them do this kind of triangulation and validation of crowdsourced data on the fly.

Below are the protest events picked up by RecordedFuture for both Tunisia and Egypt. From these two figures, it is possible to see how the Tunisian protests preceded those in Egypt.

The authors argue that if the platform had been set up earlier this year, a user would have seen the sudden rise in the number of protests in Egypt. However, the authors acknowledge that their data is a function of media interest and attention—the same issue I had with my dissertation. One way to overcome this challenge might be by complementing the harvested reports with crowdsourced data from social media and Crowdmap.

In the future, the authors plan to have the system auto-detect major changes in trends and to add support for the analysis of media in languages beyond English. They also plan to test the reliability and accuracy of their conflict early warning algorithm by comparing their forecasts of historical data with existing conflict data sets. I have several ideas of my own about next steps and look forward to speaking with Christopher’s team about ways to collaborate.