Crowdsourcing and Humanitarian Action: Analysis of the Literature

Raphael Hörler from Zurich’s ETH University has just completed his thesis on the role of crowdsourcing in humanitarian action. His valuable research offers one of the most up-to-date and comprehensive reviews of the principal players and humanitarian technologies in action today. In short, I highly recommend this important resource. Raphael’s full thesis is available here (PDF).

Crowdsourcing Yolanda Response


Computing Research Institutes as an Innovation Pathway for Humanitarian Technology

The World Humanitarian Summit (WHS) is an initiative by United Nations Secretary-General Ban Ki-moon to improve humanitarian action. The Summit, which is to be held in 2016, stands to be one of the most important humanitarian conferences in a decade. One key pillar of WHS is humanitarian innovation. “Transformation through Innovation” is the WHS Working Group dedicated to transforming humanitarian action by focusing explicitly on innovation. I have the pleasure of being a member of this working group where my contribution focuses on the role of new technologies, data science and advanced computing. As such, I’m working on an applied study to explore the role of computing research institutes as an innovation pathway for humanitarian technology. The purpose of this blog post is to invite feedback on the ideas presented below.


I first realized that the humanitarian community faced a “Big Data” challenge in 2010, just months after I had joined Ushahidi as Director of Crisis Mapping, and just months after co-founding CrisisMappers: The Humanitarian Technology Network. The devastating Haiti Earthquake resulted in a massive overflow of information generated via mainstream news, social media, text messages and satellite imagery. I launched and spearheaded the Haiti Crisis Map at the time and together with hundreds of digital volunteers from all around the world went head-to head with Big Data. As noted in my forthcoming book, we realized there and then that crowdsourcing and mapping software alone were no match for Big (Crisis) Data.

Digital Humanitarians: The Book

This explains why I decided to join an advanced computing research institute, namely QCRI. It was clear to me after Haiti that humanitarian organizations had to partner directly with advanced computing experts to manage the new Big Data challenge in disaster response. So I “embedded” myself in an institute with leading experts in Big Data Analytics, Data Science and Social Computing. I believe that computing research institutes (CRI’s) can & must play an important role in fostering innovation in next generation humanitarian technology by partnering with humanitarian organizations on research & development (R&D).

There is already some evidence to support this proposition. We (QCRI) teamed up with the UN Office for the Coordination of Humanitarian Affairs (OCHA) to create the Artificial Intelligence for Disaster Response platform, AIDR as well as MicroMappers. We are now extending AIDR to analyze text messages (SMS) in partnership with UNICEF. We are also spearheading efforts around the use and analysis of aerial imagery (captured via UAVs) for disaster response (see the Humanitarian UAV Network: UAViators). On the subject of UAVs, I believe that this new technology presents us (in the WHS Innovation team) with an ideal opportunity to analyze in “real time” how a new, disruptive technology gets adopted within the humanitarian system. In addition to UAVs, we catalyzed a partnership with Planet Labs and teamed up with Zooniverse to take satellite imagery analysis to the next level with large scale crowd computing. To this end, we are working with humanitarian organizations to enable them to make sense of Big Data generated via social media, SMS, aerial imagery & satellite imagery.

The incentives for humanitarian organizations to collaborate with CRI’s are obvious, especially if the latter (like QCRI) commits to making the resulting prototypes freely accessible and open source. But why should CRI’s collaborate with humanitarian organizations in the first place? Because the latter come with real-world challenges and unique research questions that many computer scientists are very interested in for several reasons. First, carrying out scientific research on real-world problems is of interest to the vast majority of computer scientists I collaborate with, both within QCRI and beyond. These scientists want to apply their skills to make the world a better place. Second, the research questions that humanitarian organizations bring enable computer scientists to differentiate themselves in the publishing world. Third, the resulting research can help advanced the field of computer science and advanced computing.

So why are we see not seeing more collaboration between CRI’s & humanitarian organizations? Because of this cognitive surplus mismatch. It takes a Director of Social Innovation (or related full-time position) to serve as a translational leader between CRI’s and humanitarian organizations. It takes someone (ideally a team) to match the problem owners and problem solvers; to facilitate and manage the collaboration between these two very different types of expertise and organizations. In sum, CRI’s can serve as an innovation pathway if the following three ingredients are in place: 1) Translation Leader; 2) Committed CRI; and 3) Committed Humanitarian Organization. These are necessary but not sufficient conditions for success.

While research institutes have a comparative advantage in R&D, they are not the best place to scale humanitarian technology prototypes. In order to take these prototypes to the next level, make them sustainable and have them develop into enterprise level software, they need to be taken up by for-profit companies. The majority of CRI’s (QCRI included) actually do have a mandate to incubate start-up companies. As such, we plan to spin-off some of the above platforms as independent companies in order to scale the technologies in a robust manner. Note that the software will remain free to use for humanitarian applications; other uses of the platform will require a paid license. Therein lies the end-to-end innovation path that computing research institutes can offer humanitarian organization vis-a-vis next generation humanitarian technologies.

As noted above, part of my involvement with the WHS Innovation Team entails working on an applied study to document and replicate this innovation pathway. As such, I am looking for feedback on the above as well as on the research methodology described below.

I plan to interview Microsoft Research, IBM Research, Yahoo Research, QCRI and other institutes as part of this research. More specifically, the interview questions will include:

  • Have you already partnered with humanitarian organizations? Why/why not?
  • If you have partnered with humanitarian organizations, what was the outcome? What were the biggest challenges? Was the partnership successful? If so, why? If not, why not?
  • If you have not yet partnered with humanitarian organizations, why not? What factors would be conducive to such partnerships and what factors serve as hurdles?
  • What are your biggest concerns vis-a-vis working with humanitarian groups?
  • What funding models did you explore if any?

I also plan to interview humanitarian organizations to better understand the prospects for this potential innovation pathway. More specifically, I plan to interview ICRC, UNHCR, UNICEF and OCHA using the following questions:

  • Have you already partnered with computing research groups? Why/why not?
  • If you have partnered with computing research groups, what was the outcome? What were the biggest challenges? Was the partnership successful? If so, why? If not, why not?
  • If you have not yet partnered with computing research groups, why not? What factors would be conducive to such partnerships and what factors serve as hurdles?
  • What are your biggest concerns vis-a-vis working with computing research groups?
  • What funding models did you explore if any?

My plan is to carry out the above semi-structured interviews in February-March 2015 along with secondary research. My ultimate aim with this deliverable is to develop a model to facilitate greater collaboration between computing research institutes and humanitarian organizations. To this end, I welcome feedback on all of the above (feel free to email me and/or add comments below). Thank you.


See also:

  • Research Framework for Next Generation Humanitarian Technology and Innovation [link]
  • From Gunfire at Sea to Maps of War: Implications for Humanitarian Innovation [link]

Establishing Social Media Hashtag Standards for Disaster Response

The UN Office for the Coordination of Humanitarian Affairs (OCHA) has just published an important, must-read report on the use of social media for disaster response. As noted by OCHA, this document was inspired by conversations with my team and I at QCRI. We jointly recognize that innovation in humanitarian technology is not enough. What is needed—and often lacking—is innovation in policymaking. Only then can humanitarian technology have widespread impact. This new think piece by OCHA seeks to catalyze enlightened policymaking.


I was pleased to provide feedback on earlier drafts of this new study and look forward to discussing the report’s recommendations with policymakers across the humanitarian space. In the meantime, many thanks to Roxanne Moore and Andrej Verity for making this report a reality. As Andrej notes in his blog post on this new study, the Filipino Government has just announced that “twitter will become another source of information for the Philippines official emergency response mechanism,” which will lead to an even more pressing Big (Crisis) Data challenge. The use of standardized hashtags will thus be essential.


The overflow of information generated during disasters can be as paralyzing to disaster response as the absence of information. While information scarcity has long characterized our information landscapes, today’s information-scapes are increasingly marked by an overflow of information—Big Data. To this end, encouraging the proactive standardization of hashtags may be one way to reduce this Big Data challenge. Indeed, standardized hashtags—i.e., more structured information—would enable paid emergency responders (as well as affected communities) to “better leverage crowdsourced information for operational planning and response.” At present, the Government of the Philippines seems to be the few actors that actually endorse the use of specific hashtags during major disasters as evidenced by their official crisis hashtags strategy.

The OCHA report thus proposes three hashtag standards and also encourages social media users to geo-tag their content during disasters. The latter can be done by enabling auto-GPS tagging or by using What3Words. Users should of course be informed of data-privacy considerations when geo-tagging their reports. As for the three hashtag standards:

  1. Early standardization of hashtags designating a specific disaster
  2. Standard, non-changing hashtag for reporting non-emergency needs
  3. Standard, non-changing hashtags for reporting emergency needs

1. As the OCHA think piece rightly notes, “News stations have been remarkably successful in encouraging early standardization of hashtags, especially during political events.” OCHA thus proposes that humanitarian organizations take a “similar approach for emergency response reporting and develop partnerships with Twitter as well as weather and news teams to publicly encourage such standardization. Storm cycles that create hurricanes and cyclones are named prior to the storm. For these events, an official hashtag should be released at the same time as the storm announcement.” For other hazards, “emergency response agencies should monitor the popular hashtag identifying a disaster, while trying to encourage a standard name.”

2. OCHA advocates for the use of #iSee, #iReport or #PublicRep for members of the public to designate tweets that refer to non-emergency needs such as “power lines, road closures, destroyed bridges, large-scale housing damage, population displacement or geographic spread (e.g., fire or flood).” When these hashtags are accompanied with GPS information, “responders can more easily identify and verify the information, therefore supporting more timely response & facilitating recovery.” In addition, responders can more easily create live crisis maps on the fly thanks to this structured, geo-tagged information.

3. As for standard hashtags for emergency reports, OCHA notes emergency calls are starting to give way to emergency SMS’s. Indeed, “Cell phone users will soon be able to send an SMS to a toll-free phone number. For emergency reporting, this new technology could dramatically alter the way the public interacts with nation-based emergency response call centers. It does not take a large imaginary leap to see the potential move from SMS emergency calls to social media emergency calls. Hashtags could be one way to begin reporting emergencies through social media.”

Most if not all countries have national emergency phone numbers already. So OCHA suggests using these existing, well-known numbers as the basis for social media hashtags. More specifically, an emergency hashtag would be composed of the country’s emergency number (such as 911 in the US, 999 in the UK, 133 in Austria, etc) followed by the country’s two-letter code (US, UK, AT respectively). In other words: #911US, #999UK, #133AT. Some countries, like Austria, have different emergency phone numbers for different types of emergencies. So these could also be used accordingly. OCHA recognizes that many “federal agencies fear that such a system would result in people reporting through social media outside of designated monitoring times. This is a valid concern. However, as with the implementation of any new technology in the public service, it will take time and extensive promotion to ensure effective use.”

Digital Humanitarians: The Book

Of course, “no monitoring system will be perfect in terms of low-cost, real-time analysis and high accuracy.” OCHA knows very well that there are a number of important limitations to the system they propose above. To be sure, “significant steps need to be taken to ensure that information flows from the public to response agencies and back to the public through improved efforts.” This is an important theme in my forthcoming book “Digital Humanitarians.”


See also:

  • Social Media & Emergency Management: Supply and Demand [link]
  • Using AIDR to Automatically Classify Disaster Tweets [link]

New: List of Software for UAVs and Aerial Imagery

My research team and I at the Humanitarian UAV Network (UAViators) have compiled a list of more than 30 common software platforms used to operate UAVs and analyze resulting aerial imagery. We carried out this research to provide humanitarian organizations with a single repository where they can review existing software platforms (including free & open source solutions) for their humanitarian UAV missions. The results, available here, provide a brief description of each software platform along with corresponding links for additional information and download. We do realize that this list is not (yet) comprehensive, so we hope you’ll help us fill remaining gaps. This explains why we’ve made our research available as an open, editable Google Doc.

UAV software

Many thanks to my research assistant Peter Mosur for taking the lead on this. We have additional research documents available here on the UAViators website.


See also:

  • Humanitarian UAV Network: Strategy for 2014-2015 [link]
  • Humanitarians in the Sky: Using UAVs for Disaster Response [link]
  • Low-Cost UAV Applications for Post-Disaster Damage Assessments: A Streamlined Workflow [Link]

Humanitarian UAV/Drones in Conflict Zones: Fears, Concerns and Opportunities

My research team and I at the Humanitarian UAV Network (UAViators) have compiled a list of fears and concerns expressed by humanitarians and others on the use of UAVs in humanitarian settings. To do this, we closely reviewed well over 50 different documents, reports, articles, etc., on humanitarian UAVs as part of this applied research project. The motivation behind this research is to better understand the different and overlapping concerns that humanitarian organizations have over the use of non-lethal drones in crises, particularly crises mired by violent conflict.

Resarch Table

The results of this research are available in this open & editable spreadsheet and summarized in the table above. We identified a total of 9 different categories of concerns and tallied the unique instances in which these appear in the official humanitarian reports, articles, papers, studies, etc., that we reviewed. The top 3 concerns are: Military Association, Data Privacy and Consent. We very much welcome feedback, so feel free to get in touch via the comments section below and/or add additional content directly to the spreadsheet. This research will feed into an upcoming workshop that my colleague Kristin Sandvik (on the Advisory Board of UAViators) and I are looking to organize in the Spring of 2015. The workshop will address the most pressing issues around the use of civilian UAVs in conflict zones.

I tend to believe that UAV initiatives like the Syria Airlift Project (video above) can play a positive role in conflict settings. In fact, I wrote about this exact use of UAVs back in 2008 for PeaceWork Magazine (scroll down) and referred to previous (conventional) humanitarian airlift examples from the Berlin Airlift in the 1940’s to the Biafran Airlift in the 1960’s as a basis for remotely piloted aircraft systems. As such, I suggested that UAVs could be used in Burma at the time to transport relief supplies in response to the complex emergency. While fraught with risks, these risks can at times be managed when approached with care, integrity and professionalism, especially if a people-centered, community-based approach is taken, which prioritizes both safety and direct empowerment.

While some humanitarians may be categorically against any and all uses of non-lethal UAVs in conflict zones regardless of the circumstances, the fact is that their opinions won’t prevent affected communities and others from using UAVs anyway. More and more individuals will have access to cheaper and cheaper UAVs in the months and years ahead. As a UN colleague noted with respect to the Syria Airlift Project, initiatives like these may well be a sign of things to come. This sentiment is also shared by my colleague Jules Frost at World Vision. See her recent piece entitled: “Eyes in the Sky are Inevitable: UAVs and Humanitarian Response.”


Acknowledgements: Many thanks to my research assistants Peter Mosur and Jus Mackinnon for taking the lead in this research.

See also:

  • On UAVs for Peacebuilding and Conflict Prevention [link]
  • The Use of Drones for Nonviolent Civil Resistance [link]
  • Drones for Human Rights: Brilliant or Foolish [link]

Using Flash Crowds to Automatically Detect Earthquakes & Impact Before Anyone Else

It is said that our planet has a new nervous system; a digital nervous system comprised of digital veins and intertwined sensors that capture the pulse of our planet in near real-time. Next generation humanitarian technologies seek to leverage this new nervous system to detect and diagnose the impact of disasters within minutes rather than hours. To this end, LastQuake may be one of the most impressive humanitarian technologies that I have recently come across. Spearheaded by the European-Mediterranean Seismological Center (EMSC), the technology combines “Flashsourcing” with social media monitoring to auto-detect earthquakes before they’re picked up by seismometers or anyone else.

Screen Shot 2014-10-23 at 5.08.30 PM

Scientists typically draw on ground-motion prediction algorithms and data on building infrastructure to rapidly assess an earthquake’s potential impact. Alas, ground-motion predictions vary significantly and infrastructure data are rarely available at sufficient resolutions to accurately assess the impact of earthquakes. Moreover, a minimum of three seismometers are needed to calibrate a quake and said seismic data take several minutes to generate. This explains why the EMSC uses human sensors to rapidly collect relevant data on earthquakes as these reduce the uncertainties that come with traditional rapid impact assess-ment methodologies. Indeed, the Center’s important work clearly demonstrates how the Internet coupled with social media are “creating new potential for rapid and massive public involvement by both active and passive means” vis-a-vis earthquake detection and impact assessments. Indeed, the EMSC can automatically detect new quakes within 80-90 seconds of their occurrence while simultaneously publishing tweets with preliminary information on said quakes, like this one:

Screen Shot 2014-10-23 at 5.44.27 PM

In reality, the first human sensors (increases in web traffic) can be detected within 15 seconds (!) of a quake. The EMSC’s system continues to auto-matically tweet relevant information (including documents, photos, videos, etc.), for the first 90 minutes after it first detects an earthquake and is also able to automatically create a customized and relevant hashtag for individual quakes.

Screen Shot 2014-10-23 at 5.51.05 PM

How do they do this? Well, the team draw on two real-time crowdsourcing methods that “indirectly collect information from eyewitnesses on earthquakes’ effects.” The first is TED, which stands for Twitter Earthquake Detection–a system developed by the US Geological Survey (USGS). TED filters tweets by key word, location and time to “rapidly detect sharing events through increases in the number of tweets” related to an earthquake. The second method, called “flashsourcing” was developed by the European-Mediterranean to analyze traffic patterns on its own website, “a popular rapid earthquake information website.” The site gets an average of 1.5 to 2 million visits a month. Flashsourcing allows the Center to detect surges in web traffic that often occur after earthquakes—a detection method named Internet Earthquake Detection (IED). These traffic surges (“flash crowds”) are caused by “eyewitnesses converging on its website to find out the cause of their shaking experience” and can be detected by analyzing the IP locations of website visitors.

It is worth emphasizing that both TED and IED work independently from traditional seismic monitoring systems. Instead, they are “based on real-time statistical analysis of Internet-based information generated by the reaction of the public to the shaking.” As EMSC rightly notes in a forthcoming peer-reviewed scientific study, “Detections of felt earthquakes are typically within 2 minutes for both methods, i.e., considerably faster than seismographic detections in poorly instrumented regions of the world.” TED and IED are highly complementary methods since they are based on two entirely “different types of Internet use that might occur after an earthquake.” TED depends on the popularity of Twitter while IED’s effectiveness depends on how well known the EMSC website is in the area affected by an earthquake. LastQuake automatically publishes real-time information on earthquakes by automatically merging real-time data feeds from both TED and IED as well as non-crowdsourcing feeds.


Lets looks into the methodology that powers IED. Flashsourcing can be used to detect felt earthquakes and provide “rapid information (within 5 minutes) on the local effects of earthquakes. More precisely, it can automatically map the area where shaking was felt by plotting the geographical locations of statistically significant increases in traffic […].” In addition, flashsourcing can also “discriminate localities affected by alarming shaking levels […], and in some cases it can detect and map areas affected by severe damage or network disruption through the concomitant loss of Internet sessions originating from the impacted region.” As such, this “negative space” (where there are no signals) is itself an important signal for damage assessment, as I’ve argued before.

remypicIn the future, EMSC’s flashsourcing system may also be able discriminate power cuts between indoor and outdoor Internet connections at the city level since the system’s analysis of web traffic session will soon be based on web sockets rather than webserver log files. This automatic detection of power failures “is the first step towards a new system capable of detecting Internet interruptions or localized infrastructure damage.” Of course, flashsourcing alone does not “provide a full description of earthquake impact, but within a few minutes, independently of any seismic data, and, at little cost, it can exclude a number of possible damage scenarios, identify localities where no significant damage has occurred and others where damage cannot be excluded.”

Screen Shot 2014-10-23 at 5.59.20 PM

EMSC is complementing their flashsourching methodology with a novel mobile app that quickly enables smartphone users to report about felt earthquakes. Instead of requiring any data entry and written surveys, users simply click on cartoonish-type pictures that best describe the level of intensity they felt when the earthquake (or aftershocks) struck. In addition, EMSC analyzes and manually validates geo-located photos and videos of earthquake effects uploaded to their website (not from social media). The Center’s new app will also make it easier for users to post more pictures more quickly.


What about typical criticisms (by now broken records) that social media is biased and unreliable (and thus useless)? What about the usual theatrics about the digital divide invalidating any kind of crowdsourcing effort given that these will be heavily biased and hardly representative of the overall population? Despite these already well known short-comings and despite the fact that our inchoate digital networks are still evolving into a new nervous system for our planet, the existing nervous system—however imperfect and immature—still adds value. TED and LastQuake demonstrate this empirically beyond any shadow of a doubt. What’s more, the EMSC have found that crowdsourced, user-generated information is highly reliable: “there are very few examples of intentional misuses, errors […].”

My team and I at QCRI are honored to be collaborating with EMSC on integra-ting our AIDR platform to support their good work. AIDR enables uses to automatically detect tweets of interest by using machine learning (artificial intelligence) which is far more effective searching for keywords. I recently spoke with Rémy Bossu, one masterminds behind the EMSC’s LastQuake project about his team’s plans for AIDR:

“For us AIDR could be a way to detect indirect effects of earthquakes, and notably triggered landslides and fires. Landslides can be the main cause of earthquake losses, like during the 2001 Salvador earthquake. But they are very difficult to anticipate, depending among other parameters on the recent rainfalls. One can prepare a susceptibility map but whether there are or nor landslides, where they have struck and their extend is something we cannot detect using geophysical methods. For us AIDR is a tool which could potentially make a difference on this issue of rapid detection of indirect earthquake effects for better situation awareness.”

In other words, as soon as the EMSC system detects an earthquake, the plan is for that detection to automatically launch an AIDR deployment to automatically identify tweets related to landslides. This integration is already completed and being piloted. In sum, EMSC is connecting an impressive ecosystem of smart, digital technologies powered by a variety of methodologies. This explains why their system is one of the most impressive & proven examples of next generation humanitarian technologies that I’ve come across in recent months.


Acknowledgements: Many thanks to Rémy Bossu for providing me with all the material and graphics I needed to write up this blog post.

See also:

  • Social Media: Pulse of the Planet? [link]
  • Taking Pulse of Boston Bombings [link]
  • The World at Night Through the Eyes of the Crowd [link]
  • The Geography of Twitter: Mapping the Global Heartbeat [link]

Code of Conduct: Cyber Crowdsourcing for Good

There is currently no unified code of conduct for digital crowdsourcing efforts in the development, humanitarian or human rights space. As such, we propose the following principles (displayed below) as a way to catalyze a conversation on these issues and to improve and/or expand this Code of Conduct as appropriate.

Screen Shot 2014-10-20 at 5.22.21 PM

This initial draft was put together by Kate ChapmanBrooke Simons and myself. The link above points to this open, editable Google Doc. So please feel free to contribute your thoughts by inserting comments where appropriate. Thank you.

An organization that launches a digital crowdsourcing project must:

  • Provide clear volunteer guidelines on how to participate in the project so that volunteers are able to contribute meaningfully.
  • Test their crowdsourcing platform prior to any project or pilot to ensure that the system will not crash due to obvious bugs.
  • Disclose the purpose of the project, exactly which entities will be using and/or have access to the resulting data, to what end exactly, over what period of time and what the expected impact of the project is likely to be.
  • Disclose whether volunteer contributions to the project will or may be used as training data in subsequent machine learning research.
  • Not ask volunteers to carry out any illegal tasks.
  • Explain any risks (direct and indirect) that may come with volunteer participation in a given project. To this end, carry out a risk assessment and produce corresponding risk mitigation strategies.
  • Clearly communicate if the results of volunteer tasks will or are likely to be sold to partners/clients.
  • Limit the level of duplication required (for data quality assurance) to a reasonable number based on previous research and experience. In sum, do not waste volunteers’ time and do not offer tasks that are not meaningful. When all tasks have been carried, inform volunteers accordingly.
  • Be fully transparent on the results of the project even if the results are poor or unusable.
  • Only launch a full-scale crowdsourcing project if they are not able to analyze the results and deliver the findings within a timeframe that provides added value to end-users of the data.

An organization that launches a digital crowdsourcing project should:

  • Share as much of the resulting data with volunteers as possible without violating data privacy or the principle of Do No Harm.
  • Enable volunteers to opt out of having their tasks contribute to subsequent machine learning research. Provide digital volunteers with the option of having their contributions withheld from subsequent machine learning studies.
  • Assess how many digital volunteers are likely to be needed for a project and recruit appropriately. Using additional volunteers just because they are available is not appropriate. Should recruitment nevertheless exceed need, adjust project to inform volunteers as soon as their inputs are no longer needed, and possibly give them options for redirecting their efforts.
  • Explain that the same crowdsourcing task (microtask) may/will be given to multiple digital volunteers for data control purposes. This often reassures volunteers who initially lack confidence when contributing to a project.