Using Mechanical Turk to Crowdsource Humanitarian Response

I’m increasingly intrigued by the idea of applying Mechanical Turk services to humanitarian response. Mechanical Turk was first developed by Amazon to crowdsource and pay for simple tasks.

An excellent example of a Mechanical Turk service in the field of ICT for Development (ICT4D) is txteagle, a platform that enables mobile phone subscribers in developing countries to earn money and accumulate savings by completing simple SMS-based micro-tasks for large corporate clients. txteagle has been used to translate pieces of text by splitting them into individual words and sending these out by SMS. Subscribers can then reply with the translation and earn some money in the process. This automatic compensation system uses statistical machinery to automatically evaluate the value of submitted work.

In Haiti, Samasource and Crowdflower have partnered with Ushahidi and FrontlineSMS to set up a Mechanical Turk service called “Mission 4636“. The system that Ushahidi and partners originally set up uses the generosity of Haitian volunteers in the US to translate urgent SMS’s from the disaster affected population in near real-time. Mission 4636 will relocate the translation work to Haiti and become an automatic compensation system for Haitian’s in-country.

At Ushahidi, we aggregate and  categorize urgent, actionable information from multiple sources including SMS and geo-tag this information on the Ushahidi’s interactive mapping platform. In the case of Haiti, this work is carried out by volunteers in Boston, Geneva, London and Portland coordinated by the Ushahidi-Haiti Situation Room at Tufts University. Volunteer retention is often a challenge, however. I wonder whether we an automated compensation system could be used to sustain future crisis mapping efforts.

Another challenge of crowdsourcing crisis information is tracking response. We know for a fact that a number of key responders are following our near real-time mapping efforts but knowing which reports they respond to is less than automatic. We have been able to document a number of success stories and continue to receive positive feedback from responders themselves but this information is hard to come by.

In a way, by crisis mapping actionable information in near real-time and in the public domain, we are in effect trying to crowdsource response. This, by nature, is a distributed and decentralized process, hence difficult to track. The tracking challenge is further magnified when the actors in question are relief and aid organizations responding to a large disaster. As anyone who has worked in disaster response knows, communicating who is doing what, where and when is not easy. Responders don’t have the bandwidth to document which reports they’ve responded to on Ushahidi.

This is problematic for several reasons including coordination. Organizations don’t necessarily know who is responding to what and whether this response is efficient. I wonder whether a Mechanical Turk system could be set up to crowdsource discrete response tasks based on individual organizations’ mandates. Sounds a little far out and may not be feasible, but the idea nevertheless intrigues me.

The automatic compensation system could be a public way to compensate response. Incoming SMS’s could be clustered along the UN Cluster system. The Shelter Cluster, for example, would have a dedicated website to which all shelter-related SMS’s would be pushed to. Organizations working in this space would each have access to this password protected website and tag the alerts they can and want to respond to.

In order to “cash in” following a response, a picture (or text based evidence) has to be submitted as proof, by the organization in question e.g., of new shelters being built. The number of completed responses could also be made public and individuals compelled to help, could send donations via SMS to each organization to reward and further fund the responses.

The task of evaluating the evidence of responses can also be crowdsource à la Mechanical Turk and serve as a source of revenue for beneficiaries.

For example, local Haitian subscribers to the system would receive an SMS notifying them that new shelters have been set up near Jacmel. Only subscribers in the Jacmel area would receive the SMS. They would then have a look for themselves to see whether the new shelters were in fact there and text back accordingly. Dozens of individuals could send in SMS’s to describe their observations which would further help triangulate the veracity of the evaluation à la Swift River. Note that the Diaspora could also get involved in this. And like txteagle, statistical machinery could also  be used to automatically evaluate the response and dispense the micro-compensations.

I have no doubt there are a number of other important kinks to be ironed out but I wanted to throw this out there now to get some preliminary feedback. This may ultimately not be feasible or worthwhile. But I do think that a partnership between Ushahidi and Crowdflower makes sense, not only in Haiti but for future deployments as well.

Patrick Philippe Meier