Category Archives: Satellite Imagery

MicroMappers: Towards Next Generation Humanitarian Technology

The MicroMappers platform has come a long way and still has a ways to go. Our vision for MicroMappers is simple: combine human computing (smart crowd-sourcing) with machine computing (artificial intelligence) to filter, fuse and map a variety of different data types such as text, photo, video and satellite/aerial imagery. To do this, we have created a collection of “Clickers” for MicroMappers. Clickers are simply web-based crowdsourcing apps used to make sense of “Big Data”. The “Text Cicker” is used to filter tweets & SMS’s; “Photo Clicker” to filter photos; “Video Clicker” to filter videos and yes the Satellite & Aerial Clickers to filter both satellite and aerial imagery. These are the Data Clickers. We also have a collection of Geo Clickers that digital volunteers use to geo-tag tweets, photos and videos filtered by the”Data Clickers. Note that these Geo Clickers auto-matically display the results of the crowdsourced geo-tagging on our MicroMaps like the one below.

MM Ruby Tweet Map

Thanks to our Artificial Intelligence (AI) engine AIDR, the MicroMappers “Text Clicker” already combines human and machine computing. This means that tweets and text messages can be automatically filtered (classified) after some initial crowdsourced filtering. The filtered tweets are then pushed to the Geo Clickers for geo-tagging purposes. We want to do the same (semi-automation) for photos posted to social media as well as videos; although this is still a very active area of research and development in the field of computer vision.

So we are prioritizing our next hybrid human-machine computing efforts on aerial imagery instead. Just like the “Text Clicker” above, we want to semi-automate feature detection in aerial imagery by adding an AI engine to the “Aerial Clicker”. We’ve just starting to explore this with computer vision experts in Switzerland and Canada. Another development we’re eyeing vis-a-vis UAVs is live video streaming. To be sure, UAVs will increasingly be transmitting live video feeds directly to the web. This means we may eventually need to develop a “Streaming Clicker”, which would in some respects resemble our existing “Video Clicker” except that the video would be broadcasting live rather than play back from YouTube, for example. The “Streaming Clicker” is for later, however, or at least until a prospective partner organization approaches us with an immediate and compelling social innovation use-case.

In the meantime, my team & I at QCRI will continue to improve our maps (data visualizations) along with the human computing component of the Clickers. The MicroMappers smartphone apps, for example, need more work. We also need to find partners to help us develop apps for tablets like the iPad. In addition, we’re hoping to create a “Translate Clicker” with Translators Without Borders (TWB). The purpose of this Clicker would be to rapidly crowdsource the translation of tweets, text messages, etc. This could open up rather interesting possibilities for machine translation, which is certainly an exciting prospect.

MM All Map

Ultimately, we want to have one and only one map to display the data filtered via the Data and Geo Clickers. This map, using (Humanitarian) OpenStreetMap as a base layer, would display filtered tweets, SMS’s, photos, videos and relevant features from satellite and UAV imagery. Each data type would simply be a different layer on this fused “Meta-Data Crisis Map”; and end-users would simply turn individual layers on and off as needed. Note also the mainstream news feeds (CNN and BBC) depicted in the above image. We’re working with our partners at UN/OCHA, GDELT & SBTF to create a “3W Clicker” to complement our MicroMap. As noted in my forthcoming book, GDELT is the ultimate source of data for the world’s digitized news media. The 3Ws refers to Who, What, Where; an important spreadsheet that OCHA puts together and maintains in the aftermath of major disasters to support coordination efforts.

In response to Typhoon Ruby in the Philippines, Andrej Verity (OCHA) and I collaborated with Kalev Leetaru from GDELT to explore how the MicroMappers “3W Clicker” might work. The result is the Google Spreadsheet below (click to enlarge) that is automatically updated every 15 minutes with the latest news reports that refer to one or more humanitarian organizations in the Philippines. GDELT includes the original URL of the news article as well as the list of humanitarian organizations referenced in the article. In addition, GDELT automatically identifies the locations referred to in the articles, key words (tags) and the date of the news article. The spreadsheet below is already live and working. So all we need now is the “3W Clicker” to crowdsource the “What”.

MM GDELT output

The first version of the mock-up we’ve created for the “3W Clicker” is displayed below. Digital volunteers are presented with an interface that includes an news article with the names of humanitarian organizations highlighted in red for easy reference. GDELT auto-populates the URL, the organization name (or names if there are more than one) and the location. Note that both the “Who” & “Where” information can be edited directly by the volunteer incase GDELT’s automated algorithm gets those wrong. The main role of digital volunteers, however, would simply be to identify the “What” by quickly skimming the article.

MM 3W Clicker v2

The output of the “3W Clicker” would simply be another MicroMap layer. As per Andrej’s suggestion, the resulting data could also be automatically pushed to another Google Spreadsheet in HXL format. We’re excited about the possibilities and plan to move forward on this sooner rather than later. In addition to GDELT, pulling in feeds from CrisisNET may be worth exploring. I’m also really keen on exploring ways to link up with the Global Disaster Alert & Coordination System (GDACS) as well as GeoFeedia.

In the meantime, we’re hoping to pilot our “Satellite Clicker” thanks to recent conversations with Planet Labs and SkyBox Imaging. Overlaying user-generated content such as tweets and images on top of both satellite and aerial imagery can go a long way to helping verify (“ground truth”) social media during disasters and other events. This is evidenced by recent empirical studies such as this one in Germany and this one in the US. On this note, as my QCRI colleague Heather Leson recently pointed out, the above vision for MicroMappers is still missing one important data feed; namely sensors—the Internet of Things. She is absolutely spot on, so we’ll be sure to look for potential pilot projects that would allow us to explore this new data source within MicroMappers.

The above vision is a tad ambitious (understatement). We really can’t do this alone. To this end, please do get in touch if you’re interested in joining the team and getting MicroMappers to the next level. Note that MicroMappers is free and open source and in no way limited to disaster response applications. Indeed, we recently used the Aerial Clicker for this wildlife protection project in Namibia. This explains why our friends over at National Geographic have also expressed an interest in potentially piloting the MicroMappers platform for some of their projects. And of course, one need not use all the Clickers for a project, simply the one(s) that make sense. Another advantage of MicroMappers is that the Clickers (and maps) can be deployed very rapidly (since the platform was initially developed for rapid disaster response purposes). In any event, if you’d like to pilot the platform, then do get in touch.

bio

See also: Digital Humanitarians – The Book

Digital Jedis: There Has Been An Awakening…

Zoomanitarians: Using Citizen Science and Next Generation Satellites to Accelerate Disaster Damage Assessments

Zoomanitarians has been in the works for well over a year, so we’re excited to be going fully public for the first time. Zoomanitarians is a joint initiative between Zooniverse (Brook Simmons), Planet Labs (Alex Bakir) and myself at QCRI. The purpose of Zoomanitarians is to accelerate disaster damage assessments by leveraging Planet Labs’ unique constellation of 28 satellites and Zooniverse’s highly scalable microtasking platform. As I noted in this earlier post, digital volunteers from Zooniverse tagged well over 2 million satellite images (of Mars, below) in just 48 hours. So why not invite Zooniverse volunteers to tag millions of images taken by Planet Labs following major disasters (on Earth) to help humanitarians accelerate their damage assessments?

Zooniverse Planet 4

That was the question I posed to Brooke and Alex in early 2013. “Why not indeed?” was our collective answer. So we reached out to several knowledgeable colleagues of mine including Kate Chapman from Humanitarian OpenStreetMap and Lars Bromley from UNOSAT for their feedback and guidance on the idea.

We’ll be able to launch our first pilot project later this year thanks to Kate who kindly provided us with very high-resolution UAV/aerial imagery of downtown Tacloban in the Philippines. Why do we want said imagery when the plan is to use Planet Labs imagery? Because Planet Labs imagery is currently available at 3-5 meter resolution so we’ll be “degrading” the resolution of the aerial imagery to determine just what level and type of damage can be captured at various resolutions as compared to the imagery from Planet Labs. The pilot project will therefore serve to (1) customize & test the Zoomanitarians microtasking platform and (2) determine what level of detail can be captured at various resolutions.

PlanetLabs

We’ll then spend the remainder of the year improving the platform based on the results of the pilot project during which time I will continue to seek input from humanitarian colleagues. Zooniverse’s microtasking platform has already been stress-tested extensively over the years, which is one reason why I approached Zooniverse last year. The other reason is that they have over 1 million digital volunteers on their list-serve. Couple this with Planet Labs’ unique constellation of 28 satellites, and you’ve got the potential for near real-time satellite imagery analysis for disaster response. Our plan is to produce “heat maps” based on the results and to share shape files as well for overlay on other maps.

It took imagery analysts well over 48 hours to acquire and analyze satellite imagery following Typhoon Yolanda. While Planet Labs imagery is not (yet) available at high-resolutions, our hope is that Zoomanitarians will be able to acquire and analyze relevant imagery within 12-24 hours of a request. Several colleagues have confirmed to me that the results of this rapid analysis will also prove invaluable for subsequent, higher-resolution satellite imagery acquisition and analysis. On a related note, I hope that our rapid satellite-based damage assessments will also serve as a triangulation mechanism (ground-truthing) for the rapid social-media-driven damage assessments carried out using the Artificial Intelligence for Disaster Response (AIDR) platform and MicroMappers.

While much work certainly remains, and while Zoomanitairans is still in the early phases of research and development, I’m nevertheless excited and optimistic about the potential impact—as are my colleagues Brooke and Alex. We’ll be announcing the date of the pilot later this summer, so stay tuned for updates!

Results of the Crowdsourced Search for Malaysia Flight 370 (Updated)

Update: More than 3 million volunteers thus far have joined the crowdsourcing efforts to locate the missing Malaysian Airlines plane. These digital volunteers have viewed over a quarter-of-a-billion micro-maps and have tagged almost 3 million features in these satellite maps. Source of update.

Malaysian authorities have now gone on record to confirm that Flight 370 was hijacked, which reportedly explains why contact with the passenger jet abruptly ceased a week ago. The Search & Rescue operations now involve 13 countries around the world and over 100 ships, helicopters and airplanes. The costs of this massive operation must easily be running into the millions of dollars.

FlightSaR

Meanwhile, a free crowdsourcing platform once used by digital volunteers to search for Genghis Khan’s Tomb and displaced populations in Somalia (video below) has been deployed to search high-resolution satellite imagery for signs of the missing airliner. This is not the first time that crowdsourced satellite imagery analysis has been used to find a missing plane but this is certainly the highest profile operation yet, which may explain why the crowdsourcing platform used for the search (Tomnod) reportedly crashed for over a dozen of hours since the online search began. (Note that Zooniverse can easily handle this level of traffic). Click on the video below to learn more about the crowdsourced search for Genghis Khan and displaced peoples in Somalia.

NatGeoVideo

Having current, high-resolution satellite imagery is almost as good as having your own helicopter. So the digital version of these search operations includes tens of thousands of digital helicopters, whose virtual pilots are covering over 2,000 square miles of Thailand’s Gulf right from their own computers. They’re doing this entirely for free, around the clock and across multiple time zones. This is what Digital Humanitarians have been doing ever since the 2010 Haiti Earthquake, and most recently in response to Typhoon Yolanda.

Tomnod has just released the top results of the crowdsourced digital search efforts, which are displayed in the short video below. Like other microtasking platforms, Tomnod uses triangulation to calculate areas of greatest consensus by the crowd. This is explained further here. Note: The example shown in the video is NOT a picture of Flight 370 but perhaps of an airborne Search & Rescue plane.

While looking for evidence of the missing airliner is like looking for the proverbial needle in a massive stack of satellite images, perhaps the biggest value-added of this digital search lays in identifying where the aircraft is most definitely not located—that is, approaching this crowdsourced operation as a process of elimination. Professional imagery analysts can very easily and quickly review images tagged by the crowd, even if they are mistakenly tagged as depicting wreckage. In other words, the crowd can provide the first level filter so that expert analysts don’t waste their time looking at thousands of images of bare oceans. Basically, if the mandate is to leave no stone unturned, then the crowd can do that very well.

In sum, crowdsourcing can reduce the signal to noise ratio so that experts can focus more narrowly on analyzing the potential signals. This process may not be perfect just yet but it can be refined and improved. (Note that professionals also get it wrong, like Chinese analysts did with this satellite image of the supposed Malaysian airliner).

If these digital efforts continue and Flight 370 has indeed been hijacked, then this will certainly be the first time that crowdsourced satellite imagery analysis is used to find a hijacked aircraft. The latest satellite imagery uploaded by Tomnod is no longer focused on bodies of water but rather land. The blue strips below (left) is the area that the new satellite imagery covers.

Tomnod New Imagery 2

Some important questions will need to be addressed if this operation is indeed extended. What if the hijackers make contact and order the cessation of all offline and online Search & Rescue operations? Would volunteers be considered “digital combatants,” potentially embroiled in political conflict in which the lives of 227 hostages are at stake?

bio

Note: The Google Earth containing the top results of the search is available here.

See also: Analyzing Tweets on Malaysia Flight #MH370 [link]

Crowdsourcing the Search for Malaysia Flight 370 (Updated)

Early Results available here!

Update from Tomnod: The response has literally been overwhelming: our servers struggled to keep up all day.  We’ve been hacking hard to make some fixes and I think that the site is working now but I apologize if you have problems connecting: we’re getting up to 100,000 page views every minute! DigitalGlobe satellites are continuing to collect imagery as new reports about the possible crash sites come in so we’ll keep updating the site with new data.

Beijing-bound Flight 370 suddenly disappeared on March 8th without a trace. My colleagues at Tomnod have just deployed their satellite imagery crowdsourcing platform to support the ongoing Search & Rescue efforts. Using high-resolution satellite imagery from DigitalGlobe, Tomnod is inviting digital volunteers from around the world to search for any sign of debris from missing Boeing 777.

MH370

The DigitalGlobe satellite imagery is dated March 9th and covers over 1,000 square miles. What the Tomnod platform does is slice that imagery into many small squares like the one below (click to enlarge). Volunteers then tag one image at a time. This process is known as microtasking (or crowd computing). For quality control purposes, each image is shown to more than one volunteer. This consensus-based approach allows Tomnod to triangulate the tagging.

TomNod

I’ve long advocated for the use of microtasking to support humanitarian efforts. In 2010, I wrote about how volunteers used microtasking to crowdsource the search for Steve Fossett who had disappeared while flying a small single-engine airplane in Nevada. This was back in 2007. In 2011, I spearheaded a partnership with the UN Refugee Agency (UNCHR) in Somalia and used the Tomnod platform to crowdsource the search for internally displaced populations in the drought-stricken Afgooye Corridor. More here. I later launched a collaboration with Amnesty International in Syria to crowdsource the search for evidence of major human rights violations—again with my colleagues from Tomnod. Recently, my team and I at QCRI have been developing MicroMappers to support humanitarian efforts. At the UN’s request, MicroMappers was launched following Typhoon Yolanda to accelerate their rapid damage assessment. I’ve also written on the use of crowd computing for Search & Rescue operations.

TomnodSomalia

I’m still keeping a tiny glimmer of hope that somehow Malaysia Flight 370 was able to land somewhere and that there are survivors. I can only image what families, loved ones and friends must be going through. I’m sure they are desperate for information, one way or another. So please consider spending a few minutes of your time to support these Search and Rescue efforts. Thank you.

Bio

Note: If you don’t see any satellite imagery on the Tomnod platform for Flight 370, this means the team is busy uploading new imagery. So please check in again in a couple hours.

See also: Analyzing Tweets on Malaysia Flight #MH370 [link]

Yes, I’m Writing a Book (on Digital Humanitarians)

I recently signed a book deal with Taylor & Francis Press. The book, which is tentatively titled “Digital Humanitarians: How Big Data is Changing the Face of Disaster Response,” is slated to be published next year. The book will chart the rise of digital humanitarian response from the Haiti Earthquake to 2015, highlighting critical lessons learned and best practices. To this end, the book will draw on real-world examples of digital humanitarians in action to explain how they use new technologies and crowdsourcing to make sense of “Big (Crisis) Data”. In sum, the book will describe how digital humanitarians & humanitarian technologies are together reshaping the humanitarian space and what this means for the future of disaster response. The purpose of this book is to inspire and inform the next generation of (digital) humanitarians while serving as a guide for established humanitarian organizations & emergency management professionals who wish to take advantage of this transformation in humanitarian response.

2025

The book will thus consolidate critical lessons learned in digital humanitarian response (such as the verification of social media during crises) so that members of the public along with professionals in both international humanitarian response and domestic emergency management can improve their own relief efforts in the face of “Big Data” and rapidly evolving technologies. The book will also be of interest to academics and students who wish to better understand methodological issues around the use of social media and user-generated content for disaster response; or how technology is transforming collective action and how “Big Data” is disrupting humanitarian institutions, for example. Finally, this book will also speak to those who want to make a difference; to those who of you who may have little to no experience in humanitarian response but who still wish to help others affected during disasters—even if you happen to be thousands of miles away. You are the next wave of digital humanitarians and this book will explain how you can indeed make a difference.

The book will not be written in a technical or academic writing style. Instead, I’ll be using a more “storytelling” form of writing combined with a conversational tone. This approach is perfectly compatible with the clear documentation of critical lessons emerging from the rapidly evolving digital humanitarian space. This conversational writing style is not at odds with the need to explain the more technical insights being applied to develop next generation humanitarian technologies. Quite on the contrary, I’ll be using intuitive examples & metaphors to make the most technical details not only understandable but entertaining.

While this journey is just beginning, I’d like to express my sincere thanks to my mentors for their invaluable feedback on my book proposal. I’d also like to express my deep gratitude to my point of contact at Taylor & Francis Press for championing this book from the get-go. Last but certainly not least, I’d like to sincerely thank the Rockefeller Foundation for providing me with a residency fellowship this Spring in order to accelerate my writing.

I’ll be sure to provide an update when the publication date has been set. In the meantime, many thanks for being an iRevolution reader!

bio

How UAVs Are Making a Difference in Disaster Response

I visited the University of Torino in 2007 to speak with the team developing UAVs for the World Food Program. Since then, I’ve bought and tested two small UAVs of my own so I can use this new technology to capture aerial imagery during disasters; like the footage below from the Philippines.

UAVs, or drones, have a very strong military connotation for many of us. But so did space satellites before Google Earth brought satellite imagery into our homes and changed our perceptions of said technology. So it stands to reason that UAVs and aerial imagery will follow suit. This explains why I’m a proponent of the Drone Social Innovation Award, which seeks to promote the use of civilian drone technology for the benefit of humanity. I’m on the panel of judges for this award, which is why I reached out to DanOffice IT, a Swiss-based company that deployed two drones in response to Typhoon Yolanda in the Philippines. The drones in question are Huginn X1’s, which have a flight time of 25 minutes with a range of 2 kilometers and maximum altitude of 150 meters.

HUGINN X1

I recently spoke with one of the Huginn pilots who was in Tacloban. He flew the drone to survey shelter damage, identify blocked roads and search for bodies in the debris (using thermal imaging cameras mounted on the drone for the latter). The imagery captured also helped to identify appropriate locations to set up camp. When I asked the pilot whether he was surprised by anything during the operation, he noted that road-clearance support was not a use-case he had expected. I’ll be meeting with him in Switzerland in the next few weeks to test-fly a Huginn and explore possible partnerships.

I’d like to see closer collaboration between the Digital Humanitarian Network (DHN) and groups like DanOffice, for example. Providing DHN-member Humanitarian OpenStreetMap (HOTosm) with up-to-date aerial imagery during disasters would be a major win. This was the concept behind OpenAerialMap, which was first discussed back in 2007. While the initiative has yet to formally launch, PIX4D is a platform that “converts thousands of aerial images, taken by lightweight UAV or aircraft into geo-referenced 2D mosaics and 3D surface models and point clouds.”

Drone Adventures

This platform was used in Haiti with the above drones. The International Organization for Migration (IOM) partnered with Drone Adventures to map over 40 square kilometers of dense urban territory including several shantytowns in Port-au-Prince, which was “used to count the number of tents and organize a ‘door-to-door’ census of the population, the first step in identifying aid requirements and organizing more permanent infrastructure.” This approach could also be applied to IDP and refugee camps in the immediate aftermath of a sudden-onset disaster. All the data generated by Drone Adventures was made freely available through OpenStreetMap.

If you’re interested in giving “drones for social good” a try, I recommend looking at the DJI Phantom and the AR.Drone Parrot. These are priced between $300- $600, which beats the $50,000 price tag of the Huginn X1.

 bio