Tag Archives: Imagery

The Best Way to Crowdsource Satellite Imagery Analysis for Disaster Response

My colleague Kirk Morris recently pointed me to this very neat study on iterative versus parallel models of crowdsourcing for the analysis of satellite imagery. The study was carried out by French researcher & engineer Nicolas Maisonneuve for the next GISscience2012 conference.

Nicolas finds that after reaching a certain threshold, adding more volunteers to the parallel model does “not change the representativeness of opinion and thus will not change the consensual output.” His analysis also shows that the value of this threshold has significant impact on the resulting quality of the parallel work and thus should be chosen carefully.  In terms of the iterative approach, Nicolas finds that “the first iterations have a high impact on the final results due to a path dependency effect.” To this end, “stronger commitment during the first steps are thus a primary concern for using such model,” which means that “asking expert/committed users to start,” is important.

Nicolas’s study also reveals that the parellel approach is better able to correct wrong annotations (wrong analysis of the satellite imagery) than the iterative model for images that are fairly straightforward to interpret. In contrast, the iterative model is better suited for handling more ambiguous imagery. But there is a catch: the potential path dependency effect in the iterative model means that  “mistakes could be propagated, generating more easily type I errors as the iterations proceed.” In terms of spatial coverage, the iterative model is more efficient since the parallel model leverages redundancy to ensure data quality. Still, Nicolas concludes that the “parallel model provides an output which is more reliable than that of a basic iterative [because] the latter is sensitive to vandalism or knowledge destruction.”

So the question that naturally follow is this: how can parallel and iterative methodologies be combined to produce a better overall result? Perhaps the parallel approach could be used as the default to begin with. However, images that are considered difficult to interpret would get pushed from the parallel workflow to the iterative workflow. The latter would first be processed by experts in order to create favorable path dependency. Could this hybrid approach be the wining strategy?

Imagery and Humanitarian Assistance: Gems, Errors and Omissions

The Center for Technology and National Security Policy based at National Defense University’s Institute for National Strategic Studies just published an 88-page report entitled “Constructive Convergence: Imagery and Humanitarian Assistance.” As noted by the author, “the goal of this paper is to illustrate to the technical community and interested humanitarian users the breadth of the tools and techniques now available for imagery collection, analysis, and distribution, and to provide brief recommendations with suggestions for next steps.” In addition, the report “presents a brief overview of the growing power of imagery, especially from volunteers and victims in disasters, and its place in emergency response. It also highlights an increasing technical convergence between professional and volunteer responders—and its limits.”

The study contains a number of really interesting gems, just a few errors and some surprising omissions. The point of this blog post is not to criticize but rather to provide constructive-and-hopefully-useful feedback should the report be updated in the future.

Lets begin with the important gems, excerpted below.

“The most serious issues overlooked involve liability protections by both the publishers and sources of imagery and its data. As far as our research shows there is no universally adopted Good Samaritan law that can protect volunteers who translate emergency help messages, map them, and distribute that map to response teams in the field.”

Whether a Good Samaritan law could ever realistically be universally adopted remains to be seen, but the point is that all of the official humanitarian data protection standards that I’ve reviewed thus far simply don’t take into account the rise of new digitally-empowered global volunteer networks (let alone the existence of social media). The good news is that some colleagues and I are working with the International Committee of the Red Cross (ICRC) and a consor-tium of major humanitarian organizations to update existing data protection protocols to take some of these new factors into account. This new document will hopefully be made publicly available in October 2012.

“Mobile devices such as tablets and mobile phones are now the primary mode for both collecting and sharing information in a response effort. A January 2011 report published by the Mobile Computing Promotion Consortium of Japan surveyed users of smart phones. Of those who had smart phones, 55 percent used a map application, the third most common application after Web browsing and email.”

I find this absolutely fascinating and thus read the January 2011 report, which is where I found the graphic below.

“The rapid deployment of Cellular on Wheels [COW] is dramatically improving. The Alcatel-Lucent Light Radio is 300 grams (about 10 ounces) and stackable. It also consumes very little power, eliminating large generation and storage requirements. It is capable of operating by solar, wind and/or battery power. Each cube fits into the size of a human hand and is fully integrated with radio processing, antenna, transmission, and software management of frequency. The device can operate on multiple frequencies simultaneously and work with existing infrastructure.”

-

“In Haiti, USSOUTHCOM found imagery, digital open source maps, and websites that hosted them (such as Ushahidi and OpenStreetMap) to occasionally be of greater value than their own assets.”

-

“It is recommended that clearly defined and restricted use of specialized #hashtags be implemented using a common crisis taxonomy. For example:

#country + location + emergency code + supplemental data

The above example, if located in Washington, DC, U.S.A., would be published as:

#USAWashingtonDC911Trapped

The specialized use of #hashtags could be implemented in the same cultural manner as 911, 999, and other emergency phone number systems. Metadata using these tags would also be given priority when sent over the Internet through communication networks (landline, broadband Internet, or mobile text or data). Abuse of ratified emergency #hashtag’s would be a prosecutable offense. Implementing such as system could reduce the amount of data that crisis mappers and other response organizations need to monitor and improve the quality of data to be filtered. Other forms of #Hashtags syllabus can also be implemented such as:

#country + location + information code (411) + supplemental data
#country + location + water (H20) + supplemental data
#country + location + Fire (FD) + supplemental data”

I found this very interesting and relevant to this earlier blog post: “Calling 911: What Humanitarians Can Learn from 50 Years of Crowdsourcing.” Perhaps a reference to Tweak the Tweet would have been worthwhile.

I also had not come across some of the platforms used in response to the 2011 earthquake in New Zealand. But the report did an excellent job sharing these.

EQviewer.co.nz

Some errors that need correcting:

Open source mapping tools such as Google Earth use imagery as a foundation for layering field data.”

Google Earth is not an open source tool.

CrisisMappers.net, mentioned earlier, is a group of more than 1,600 volunteers that have been brought together by Patrick Meier and Jen Ziemke. It is the core of collaboration efforts that can be deployed anywhere in the world. CrisisMappers has established workshops and steering committees to set guidelines and standardize functions and capabilities for sites that deliver imagery and layered datasets. This group, which today consists of diverse and talented volunteers from all walks of life, might soon evolve into a professional volunteer organization of trusted capabilities and skill sets and they are worth watching.”

CrisisMappers is not a volunteer network or an organization that deploys in any formal sense of the word. The CrisisMappers website explains what the mission and purpose of this informal network is. The initiative has some 3,500 members.

-

“Figure 16. How Ushahidi’s Volunteer Standby Task Force was Structured for Libya. Ushahidi’s platform success stems from its use by organized volunteers, each with skill sets that extract data from multiple sources for publication.”

The Standby Volunteer Task Force (SBTF) does not belong to Ushahidi, nor is the SBTF an Ushahidi project. A link to the SBTF website would have been appropriate. Also, the majority of applications of the Ushahidi platform have nothing to do with crises, or the SBTF, or any other large volunteer networks. The SBTF’s original success stems from organized volunteers who where well versed in the Ushahidi platform.

“Ushahidi accepts KML and KMZ if there is an agreement and technical assistance resources are available. An end user cannot on their own manipulate a Ushahidi portal as an individual, nor can external third party groups unless that group has an arrangement with the principal operators of the site. This offers new collaboration going forward. The majority of Ushahidi disaster portals are operated by volunteer organizations and not government agencies.”

The first sentence is unclear. If someone sets up an Ushahidi platform and they have KML/KMZ files that they want to upload, they can go ahead and do so. An end-user can do some manipulation of an Ushahidi portal and can also pull the Ushahidi data into their own platform (via the GeoRSS feed, for example). Thanks to the ESRI-Ushahidi plugin, they can then perform a range of more advanced GIS analysis. In terms of volunteers vs government agencies, indeed, it appears the former is leading the way vis-a-vis innovation.

Finally, below are some omissions and areas that I would have been very interested to learn more about. For some reason, the section on the Ushahidi deployment in New Zealand makes no reference to Ushahidi.

Staying on the topic of the earthquake in Christchurch, I was surprised to see no reference to the Tomnod deployment:

I had also hoped to read more about the use of drones (UAVs) in disaster response since these were used both in Haiti and Japan. What about the rise of DIY drones and balloon mapping? Finally, the report’s reference to Broadband Global Area Network (BGAN) doesn’t provide information on the range of costs associated with using BGANs in disasters.

In conclusion, the report is definitely an important contribution to the field of crisis mapping and should be required reading.

Crowdsourcing Satellite Imagery Analysis for UNHCR-Somalia: Latest Results


253,711

That is the total number of tags created by 168 volunteers after processing 3,909 satellite images in just five days. A quarter of a million tags in 120 hours; that’s more than 2,000 tags per hour. Wow. As mentioned in this earlier blog post, volunteers specifically tagged three different types of informal shelters to provide UNHCR with an estimate of the IDP population in the Afgooye Corridor. So what happens now?

Our colleagues at Tomnod are going to use their CrowdRank algorithm to triangulate the data. About 85% of 3,000+ images were analyzed by at least 3 volunteers. So the CrowdRank algorithm will determine which tags had the most consensus across volunteers. This built-in quality control mechanism is a distinct advantage of using micro-tasking platforms like Tomnod. The tags with the most consensus will then be pushed to a dedicated UNHCR Ushahidi platform for further analysis. This project represents an applied research & development initiative. In short, we certainly don’t have all the answers. This next phase is where the assessment and analysis begins.

In the meantime, I’ve been in touch with the EC’s Joint Research Center about running their automated shelter detection algorithm on the same set of satellite imagery. The purpose is to compare those results with the crowdsourced tags in order to improve both methodologies. Clearly, none of this would be possible without the imagery and  invaluable support from our colleagues at DigitalGlobe, so huge thanks to them.

And of course, there would be no project at all were it not for our incredible volunteers, the best “Mapsters” on the planet. Indeed, none of those 200,000+ tags would exist were it not for the combined effort between the Standby Volunteer Task Force (SBTF) and students from the American Society for Photogrammetry and Remote Sensing (ASPRS); Columbia University’s New Media Task Force (NMTF) who were joined by students from the New School; the Geography Departments at the University of Wisconsin-Madison, the University of Georgia, and George Mason University, and many other volunteers including humanitarian professionals from the United Nations and beyond.

As many already know, my colleague Shadrock Roberts played a pivotal role in this project. Shadrock is my fellow co-lead on the SBTF Satellite Team and he took the important initiative to draft the feature-key and rule-sets for this mission. He also answered numerous questions from many volunteers throughout past five days. Thank you, Shadrock!

It appears that word about this innovative project has gotten back to UNHCR’s Deputy High Commissioner, Professor Alexander Aleinikoff. Shadrock and I have just been invited to meet with him in Geneva on Monday, just before the 2011 International Conference of Crisis Mappers (ICCM 2011) kicks off. We’ll be sure to share with him how incredible this volunteer network is and we’ll definitely let all volunteers know how the meeting goes. Thanks again for being the best Mapsters around!

 

Syria: Crowdsourcing Satellite Imagery Analysis to Identify Mass Human Rights Violations

Update: See this blog post for the latest. Also, our project was just featured on the UK Guardian Blog!

What if we crowdsourced satellite imagery analysis of key cities in Syria to identify evidence of mass human rights violations? This is precisely the question that my colleagues at Amnesty International USA’s Science for Human Rights Program asked me following this pilot project I coordinated for Somalia. AI-USA has done similar work in the past with their Eyes on Darfur project, which I blogged about here in 2008. But using micro-tasking with backend triangulation to crowdsource the analysis of high resolution satellite imagery for human rights purposes is definitely breaking new ground.

A staggering amount of new satellite imagery is produced every day; millions of square kilometers’ worth according to one knowledgeable colleague. This is a big data problem that needs mass human intervention until the software can catch up. I recently spoke with Professor Ryan Engstrom, the Director of the Spatial Analysis Lab at George Washington University, and he confirmed that automated algorithms for satellite imagery analysis still have a long, long way to go. So the answer for now has to be human-driven analysis.

But professional satellite imagery experts who have plenty of time to volunteer their skills are far and few between. The Satellite Sentinel Project (SSP), which I blogged about here, is composed of a very small team and a few interns. Their focus is limited to the Sudan and they are understandably very busy. My colleagues at AI-USA analyze satellite imagery for several conflicts, but this takes them far longer than they’d like and their small team is still constrained given the number of conflicts and vast amounts of imagery that could be analyzed. This explains why they’re interested in crowdsourcing.

Indeed, crowdsourcing imagery analysis has proven to be a workable solution in several other projects & sectors. The “crowd” can indeed scan and tag vast volumes of satellite imagery data when that imagery is “sliced and diced” for micro-tasking. This is what we did for the Somalia pilot project thanks to the Tomnod platform and the imagery provided by Digital Globe. The yellow triangles below denote the “sliced images” that individual volunteers from the Standby Task Force (SBTF) analyzed and tagged one at a time.

We plan do the same with high resolution satellite imagery of three key cities in Syria selected by the AI-USA team. The specific features we will look for and tag include: “Burnt and/or darkened building features,” “Roofs absent,” “Blocks on access roads,” “Military equipment in residential areas,” “Equipment/persons on top of buildings indicating potential sniper positions,” “Shelters composed of different materials than surrounding structures,” etc. SBTF volunteers will be provided with examples of what these features look like from a bird’s eye view and from ground level.

Like the Somalia project, only when a feature—say a missing roof—is tagged identically  by at least 3 volunteers will that location be sent to the AI-USA team for review. In addition, if volunteers are unsure about a particular feature they’re looking at, they’ll take a screenshot of said feature and share it on a dedicated Google Doc for the AI-USA team and other satellite imagery experts from the SBTF team to review. This feedback mechanism is key to ensure accurate tagging and inter-coder reliability. In addition, the screenshots shared will be used to build a larger library of features, i.e., what a missing roof looks like as well military equipment in residential areas, road blocks, etc. Volunteers will also be in touch with the AI-USA team via a dedicated Skype chat.

There will no doubt be a learning curve, but the sooner we climb that learning curve the better. Democratizing satellite imagery analysis is no easy task and one or two individuals have opined that what we’re trying to do can’t be done. That may be, but we won’t know unless we try. This is how innovation happens. We can hypothesize and talk all we want, but concrete results are what ultimately matters. And results are what can help us climb that learning curve. My hope, of course, is that democratizing satellite imagery analysis enables AI-USA to strengthen their advocacy campaigns and makes it harder for perpetrators to commit mass human rights violations.

SBTF volunteers will be carrying out the pilot project this month in collaboration with AI-USA, Tomnod and Digital Globe. How and when the results are shared publicly will be up to the AI-USA team as this will depend on what exactly is found. In the meantime, a big thanks to Digital Globe, Tomnod and SBTF volunteers for supporting the AI-USA team on this initiative.

If you’re interested in reading more about satellite imagery analysis, the following blog posts may also be of interest:

• Geo-Spatial Technologies for Human Rights
• Tracking Genocide by Remote Sensing
• Human Rights 2.0: Eyes on Darfur
• GIS Technology for Genocide Prevention
• Geo-Spatial Analysis for Global Security
• US Calls for UN Aerial Surveillance to Detect Preparations for Attacks
• Will Using ‘Live’ Satellite Imagery to Prevent War in the Sudan Actually Work?
• Satellite Imagery Analysis of Kenya’s Election Violence: Crisis Mapping by Fire
• Crisis Mapping Uganda: Combining Narratives and GIS to Study Genocide
• Crowdsourcing Satellite Imagery Analysis for Somalia: Results of Trial Run
• Genghis Khan, Borneo & Galaxies: Crowdsourcing Satellite Imagery Analysis
• OpenStreetMap’s New Micro-Tasking Platform for Satellite Imagery Tracing




Crowdsourcing Satellite Imagery Analysis for Somalia: Results of Trial Run

We’ve just completed our very first trial run of the Standby Task Volunteer Force (SBTF) Satellite Team. As mentioned in this blog post last week, the UN approached us a couple weeks ago to explore whether basic satellite imagery analysis for Somalia could be crowdsourced using a distributed mechanical turk approach. I had actually floated the idea in this blog post during the floods in Pakistan a year earlier. In any case, a colleague at Digital Globe (DG) read my post on Somalia and said: “Lets do it.”

So I reached out to Luke Barrington at Tomnod to set up distributed micro-tasking platform for Somalia. To learn more about Tomond’s neat technology, see this previous blog post. Within just a few days we had high resolution satellite imagery from DG and a dedicated crowdsourcing platform for imagery analysis, courtesy of Tomnod . All that was missing were some willing and able “mapsters” from the SBTF to tag the location of shelters in this imagery. So I sent out an email to the group and some 50 mapsters signed up within 48 hours. We ran our pilot from August 26th to August 30th. The idea here was to see what would go wrong (and right!) and thus learn as much as we could before doing this for real in the coming weeks.

It is worth emphasizing that the purpose of this trial run (and entire exercise) is not to replicate the kind of advanced and highly-skilled satellite imagery analysis that professionals already carry out.  This is not just about Somalia over the next few weeks and months. This is about Libya, Syria, Yemen, Afghanistan, Iraq, Pakistan, North Korea, Zimbabwe, Burma, etc. Professional satellite imagery experts who have plenty of time to volunteer their skills are far and few between. Meanwhile, a staggering amount of new satellite imagery is produced  every day; millions of square kilometers’ worth according to one knowledgeable colleague.

This is a big data problem that needs mass human intervention until the software can catch up. Moreover, crowdsourcing has proven to be a workable solution in many other projects and sectors. The “crowd” can indeed scan vast volumes of satellite imagery data and tag features of interest. A number of these crowds-ourcing platforms also have built-in quality assurance mechanisms that take into account the reliability of the taggers and tags. Tomnod’s CrowdRank algorithm, for example, only validates imagery analysis if a certain number of users have tagged the same image in exactly the same way. In our case, only shelters that get tagged identically by three SBTF mapsters get their locations sent to experts for review. The point here is not to replace the experts but to take some of the easier (but time-consuming) tasks off their shoulders so they can focus on applying their skill set to the harder stuff vis-a-vis imagery interpretation and analysis.

The purpose of this initial trial run was simply to give SBTF mapsters the chance to test drive the Tomnod platform and to provide feeback both on the technology and the work flows we put together. They were asked to tag a specific type of shelter in the imagery they received via the web-based Tomnod platform:

There’s much that we would do differently in the future but that was exactly the point of the trial run. We had hoped to receive a “crash course” in satellite imagery analysis from the Satellite Sentinel Project (SSP) team but our colleagues had hardly slept in days because of some very important analysis they were doing on the Sudan. So we did the best we could on our own. We do have several satellite imagery experts on the SBTF team though, so their input throughout the process was very helpful.

Our entire work flow along with comments and feedback on the trial run is available in this open and editable Google Doc. You’ll note the pages (and pages) of comments, questions and answers. This is gold and the entire point of the trial run. We definitely welcome additional feedback on our approach from anyone with experience in satellite imagery interpretation and analysis.

The result? SBTF mapsters analyzed a whopping 3,700+ individual images and tagged more than 9,400 shelters in the green-shaded area below. Known as the “Afgooye corridor,” this area marks the road between Mogadishu and Afgooye which, due to displacement from war and famine in the past year, has become one of the largest urban areas in Somalia. [Note, all screen shots come from Tomnod].

Last year, UNHCR used “satellite imaging both to estimate how many people are living there, and to give the corridor a concrete reality. The images of the camps have led the UN’s refugee agency to estimate that the number of people living in the Afgooye Corridor is a staggering 410,000. Previous estimates, in September 2009, had put the number at 366,000″ (1).

The yellow rectangles depict the 3,700+ individual images that SBTF volunteers individually analyzed for shelters: And here’s the output of 3 days’ worth of shelter tagging, 9,400+ tags:

Thanks to Tomnod’s CrowdRank algorithm, we were able to analyze consensus between mapsters and pull out the triangulated shelter locations. In total, we get 1,423 confirmed locations for the types of shelters described in our work flows. A first cursory glance at a handful (“random sample”) of these confirmed locations indicate they are spot on. As a next step, we could crowdsource (or SBTF-source, rather) the analysis of just these 1,423 images to triple check consensus. Incidentally, these 1,423 locations could easily be added to Google Earth or a password-protected Ushahidi map.

We’ve learned a lot during this trial run and Luke got really good feedback on how to improve their platform moving forward. The data collected should also help us provide targeted feedback to SBTF mapsters in the coming days so they can further refine their skills. On my end, I should have been a lot more specific and detailed on exactly what types of shelters qualified for tagging. As the Q&A section on the Google Doc shows, many mapsters weren’t exactly sure at first because my original guidelines were simply too vague. So moving forward, it’s clear that we’ll need a far more detailed “code book” with many more examples of the features to look for along with features that do not qualify. A colleague of mine suggested that we set up an interactive, online quiz that takes volunteers through a series of examples of what to tag and not to tag. Only when a volunteer answers all questions correctly do they move on to live tagging. I have no doubt whatsoever that this would significantly increase consensus in subsequent imagery analysis.

Please note: the analysis carried out in this trial run is not for humanitarian organizations or to improve situational awareness, it is simply for testing purposes only. The point was to try something new and in the process work out the kinks so when the UN is ready to provide us with official dedicated tasks we don’t have to scramble and climb the steep learning curve there and then.

In related news, the Humanitarian Open Street Map Team (HOT) provided SBTF mapsters with an introductory course on the OSM platform this past weekend. The HOT team has been working hard since the response to Haiti to develop an OSM Tasking Server that would allow them to micro-task the tracing of satellite imagery. They demo’d the platform to me last week and I’m very excited about this new tool in the OSM ecosystem. As soon as the system is ready for prime time, I’ll get access to the backend again and will write up a blog post specifically on the Tasking Server.

On Genghis Khan, Borneo and Galaxies: Using Crowdsourcing to Analyze Satellite Imagery

My colleague Robert Soden was absolutely right: Tomnod is definitely iRevolution material. This is why I reached out to the group a few days ago to explore the possibility of using their technology to crowdsource the analysis of satellite imagery for Somalia. You can read more about that project here. In this blog post, however, is to highlight the amazing work they’ve been doing with National Geographic in search of Genghis Khan’s tomb.

This “Valley of the Khans Project” represents a new approach to archeology. Together with National Geographic, Tomnod has collected thousands of GeoEye satellite images of the valley and designed a  simple user interface to crowdsource the tagging of roads, rivers and modern or ancient structures they. I signed up to give it a whirl and it was a lot of fun. A short video gives a quick guide on how to recognize different structures and then off you go!

You are assigned the rank “In Training” when you first begin. Once you’ve tagged your first 10 images, you progress to the next rank, which is “Novice 1″. The squares at the bottom left represent the number of individual satellite images you’ve tagged and how many are left. This is a neat game-like console and I wonder if there’s a scoreboard with names, listed ranks and images tagged.

In any case, a National Geographic team in Mongolia use the results to identify the most promising archeological sites. The field team also used Unmanned Areal Vehicles (UAVs) to supplement the satellite imagery analysis. You can learn more about the “Valley of the Khans Project” from this TEDx talk by Tomnod’s Albert Lin. Incidentally, Tomnod also offered their technology to map the damage from the devastating earthquake in New Zealand, earlier this year. But the next project I want to highlight focuses on the forests of Borneo.

I literally just found out about the “EarthWatchers: Planet Patrol” project thanks to Edwin Wisse’s comment on my previous blog post. As Edwin noted, EarthWatchers is indeed very similar to the Somalia initiative I blogged about. The project is “developing the (web)tools for students all over the world to monitor rainforests using updated satellite imagery to provide real time intelligence required to halt illegal deforestation.”

This is a really neat project and I’ve just signed up to participate. EarthWatchers has designed a free and open source platform to make it easy for students to volunteer. When you log into the platform, EarthWatchers gives you a hexagon-shaped area of the Borneo rainforest to monitor and protect using the satellite imagery displayed on the interface.

The platform also provides students with a number of contextual layers, such as road and river networks, to add context to the satellite imagery and create heat-maps of the most vulnerable areas. Forests near roads are more threatened since the logs are easier to transport, for example. In addition, volunteers can compare before-and-after images of their hexagon to better identify any changes. If you detect any worrying changes in your hexagon, you can create an alert that notifies all your friends and neighbors.

An especially neat feature about the interface is that it allows students to network online. For example, you can see who your neighbors in nearby hexagons are and even chat with them thanks to a native chat feature. This is neat because it facilitates collaboration mapping in real time and means you don’t feel alone or isolated as a volunteer. The chat feature helps to builds community.

If you’d like to learn more about this project, I recommend the presentation below by Eduardo Dias.

The third and final project I want to highlight is called Galaxy Zoo. I first came across this awesome example of citizen science in MacroWikinomics—an excellent book written by Don Tapscott and Anthony Williams. The purpose of Galaxy Zoo is to crowdsource the tagging and thus classification of galaxies as either spiral or elliptical. In order to participate, users to take a short tutorial on the basics of galaxy morphology.

While this project began as an experiment of sorts, the initiative is thriving with more than 275,000 users participating and 75 million classifications made. In addition, the data generated has resulted in several peer reviewed publica-tions real scientific discoveries. While the project uses imagery of the stars rather than earth, it really qualifies as a major success story in crowdsourcing the analysis of imagery.

Know of other intriguing applications of crowdsourcing for imagery analysis? If so, please do share in the comments section below.

Analyzing Satellite Imagery of the Somali Crisis Using Crowdsourcing

 Update: results of satellite imagery analysis available here.

You gotta love Twitter. Just two hours after I tweeted the above—in reference to this project—a colleague of mine from the UN who just got back from the Horn of Africa called me up: “Saw your tweet, what’s going on?” The last thing I wanted to was talk about the über frustrating day I’d just had. So he said, “Hey, listen, I’ve got an idea.” He reminded me of this blog post I had written a year ago on “Crowdsourcing the Analysis of Satellite for Disaster Response” and said, “Why not try this for Somalia? We could definitely use that kind of information.” I quickly forgot about my frustrating day.

Here’s the plan. He talks to UNOSAT and Google about acquiring high-resolution satellite imagery for those geographic areas for which they need more information on. A colleague of mine in San Diego just launched his own company to develop mechanical turk & micro tasking solutions for disaster response. He takes this satellite imagery and cuts it into say 50×50 kilometers square images for micro-tasking purposes.

We then develop a web-based interface where volunteers from the Standby Volunteer Task Force (SBTF) sign in and get one high resolution 50×50 km image displayed to them at a time. For each image, they answer the question: “Are there any human shelters discernible in this picture? [Yes/No].” If yes, what would you approximate the population of that shelter to be? [1-20; 21-50; 50-100; 100+].” Additional questions could be added. Note that we’d provide them with guidelines on how to identify human shelters and estimate population figures.

No shelters discernible in this image

Each 50×50 image would get rated by at least 3 volunteers for data triangulation and quality assurance purposes. That is, if 3 volunteers each tag an image as depicting a shelter (or more than one shelter) and each of the 3 volunteers approximate the same population range, then that image would get automatically pushed to an Ushahidi map, automatically turned into a geo-tagged incident report and automatically categorized by the population estimate. One could then filter by population range on the Ushahidi map and click on those reports to see the actual image.

If satellite imagery licensing is an issue, then said images need not be pushed to the Ushahidi map. Only the report including the location of where a shelter has been spotted would be mapped along with the associated population estimate. The satellite imagery would never be released in full, only small bits and pieces of that imagery would be shared with a trusted network of SBTF volunteers. In other words, the 50×50 images could not be reconstituted and patched together because volunteers would not get contiguous 50×50 images. Moreover, volunteers would sign a code of conduct whereby they pledge not to share any of the imagery with anyone else. Because we track which volunteers see which 50×50 images, we could easily trace any leaked 50×50 image back to the volunteer responsible.

Note that for security reasons, we could make the Ushahidi map password protected and have a public version of the map with very limited spatial resolution so that the location of individual shelters would not be discernible.

I’d love to get feedback on this idea from iRevolution readers, so if you have thoughts (including constructive criticisms), please do share in the comments section below.