Tag Archives: Satellite Imagery

Crowdsourcing the Analysis of Satellite Imagery for Disaster Response

I recently got a call from a humanitarian colleague in the field who asked whether it would be possible to crowdsource the basic analysis of satellite imagery.  They wanted to know because their team was sitting on a pile of satellite imagery but did not have the time or  staff to go through the high-resolution pictures. They wanted to use the imagery to identify where IDPs were located in order to know where to send aid via helicopters.

My colleague’s question reminded me of the search for Steve Fossett, a famous adventurer who went missing in September 2007 after taking off from a small airport in Nevada in a small single-engine airplane. The area where Steve went missing is particularly rugged terrain. The search and rescue aircraft were not able to find any sign of wreckage. However, high-resolution satellite imagery from GeoEye enabled Amazon to produce a Help Find Steve Fossett website, allowing volunteers to search small sections of the available imagery.

“This is an approach to more rapidly search a large area of imagery using many eyeballs of people around the world. A similar technique was used to search for Jim Gray, a Microsoft scientist who went missing on his sailboat off the coast of California.”

Micro-tasking the analysis of satellite imagery has already been done.  So why not in the context of disaster response? One could add this feature to a platform like Crowdflower, which is already being used as a plugin to micro-task the processing of text messages from disaster affected areas. Instead of text, volunteers would see a small subsection of satellite imagery. They’d be asked whether they could see any evidence of individuals in the imagery and if so how many approximately they can make out. A simple 5-minute guide on how to identify people and approximate population size using satellite imagery could be put on YouTube for volunteers to watch before getting started.

Like any type of micro-tasking approach (a.k.a. mechanical turk service), one could triangulate answers to maintain some level of quality control. For example, only when 10 volunteers each tag an image as having individuals in it would the picture be processed as such. The same would apply to the population ranged estimated in a given image. This wouldn’t necessarily produce perfect results, but it would take the bulk of the load off the shoulders of humanitarian on the ground. It would act as a first filter.

Of course the obvious question that arises is security and privacy. There are several ways this could be addressed. First, images would be stripped of any GPS coordinates. Second, images would be sliced up in small bits to prevent easy recognition of the territory. Third, a volunteer would not be given contiguous slices so they couldn’t piece together more information from the satellite imagery. These measures won’t provide 100% security and privacy. The only way to achieve that would be to use bounded crowdsourcing, i.e., only have trusted individuals analyze the imagery.

Google’s New Earth Engine for Satellite Imagery Analysis: Applications to Humanitarian Crises

So that’s what they’ve been up to. Google is developing a new computational platform for global-scale analysis of satellite imagery to monitor deforestation. But this is just “the first of many Earth Engine applications that will help scientists, policymakers, and the general public to better monitor and understand the Earth’s ecosystems.”

How about the Earth’s social systems? Humanitarian crises? Armed conflicts? This has been one of the main drivers of the Program on Crisis Mapping and Early Warning (CM&EW) which I co-direct at the Harvard Humanitarian Initiative (HHI) with Dr. Jennifer Leaning. Indeed, we had a meeting with the Google Earth team earlier this year to discuss the development of a computational platform to analyze satellite imagery of humanitarian crises for the purposes of early detection and early response.

In particular, we were interested in determining whether certain spatial patterns could be identified and if so whether we could develop a taxonomy of different spatial patterns of humanitarian crises; something like a library of “crisis finger prints.” As we noted to Google in writing following the conversations,

It is our view that the work of interpretation will be powerfully enhanced by the development of valid patterns relating to issues of importance in specific sets of circumstances that can be reproducibly recognized in satellite imagery. To be sure, the geo-spatial analysis of humanitarian crisis can serve as an important control mechanism for Google’s efforts in extending the functionality of Google Earth and Google’s analytical expertise.

This is something that a consortium of organizations including HHI can get engaged in. Population movement and settlement, shelter options and conditions, environmental threats, access to food and water, are discernible from various elements and resolution levels of satellite imagery.  But much more could be apprehended from these images were patterns assembled and then tested against other information sources and empirical field assessments. For an excellent presentation on this, see my colleague Jennifer Leaning’s excellent Keynote address at ICCM 2009:

The military uses of satellite imagery are far more developed than the humanitarian capacities because the interpretive link between what can be seen in the image and what is actually happening on the ground has been made, in great iterative detail, over a period of many years, encompassing a wide span of geographies and technological deployments. We need to develop a process to explore and validate what can be understood from satellite imagery about key humanitarian concerns by augmenting standard satellite analytics with time-specific and informed assessments of what was concurrently taking place in the location being photographed.

The potential for such applications has just begun to surface in humanitarian circles.  The Darfur Google initiative has demonstrated the force of vivid images of destruction tethered to actual locations of villages across the span of Darfur.  Little further detail is available from the actual images, however, and much of the associated information depicted by clicking on the image is static derived from other sources, somewhat laboriously acquired.  The full power of what might be gleaned simply from the satellite image remains to be explored.

Because systematic and empirical analysis of what a series of satellite images might reveal about humanitarian issues has not yet been undertaken, any effort to draw inferences from current images does not lead far.  The recent coverage of the war in Sri Lanka included satellite photos of the same contested terrain in the northeast, for two time frames, a month apart.  The attempt to determine what had transpired in that interim, relating to population movement, shelter de-construction and reconstruction, and land bombardment, was a matter of conjecture.

Bridging this gap from image to insight will not only be a matter of technological enhancement of satellite imaging. It will require interrogating the satellite images through the filter of questions and concerns that are relevant to humanitarian action and then infusing other kinds of information, gathered through a range of methods, to create visual metrics for understanding what the images project.

There is a lot of exciting work to be done in this space and I do hope that Google will seek to partner with humanitarian organizations and applied research institutes to develop an Earth Engine for Humanitarian Crises. While the technological and analytical breakthroughs are path breaking, let us remember that they can be even more breathtaking by applying them to save lives in humanitarian crises.

Patrick Philippe Meier

New Tech in Emergencies and Conflicts: Role of Information and Social Networks

I had the distinct pleasure of co-authoring this major new United Nations Foundation & Vodafone Foundation Technology Report with my distinguished colleague Diane Coyle. The report looks at innovation in the use of technology along the time line of crisis response, from emergency preparedness and alerts to recovery and rebuilding.

“It profiles organizations whose work is advancing the frontlines of innovation, offers an overview of international efforts to increase sophistication in the use of IT and social networks during emergencies, and provides recommendations for how governments, aid groups, and international organizations can leverage this innovation to improve community resilience.”

Case studies include:

  • Global Impact and Vulnerability Alert System (GIVAS)
  • European Media Monitor (EMM, aka OPTIMA)
  • Emergency Preparedness Information Center (EPIC)
  • Ushahidi Crowdsourcing Crisis Information
  • Télécoms sans Frontières (TSF)
  • Impact of Social Networks in Iran
  • Social Media, Citizen Journalism and Mumbai Terrorist Attacks
  • Global Disaster Alert and Coordination System (GDACS)
  • InSTEDD RIFF
  • UNOSAT
  • AAAS Geospatial Technologies for Human Rights
  • Info Technology for Humanitarian Assistance, Cooperation and Action (ITHACA)
  • Camp Roberts
  • OpenStreetMap and Walking Papers
  • UNDP Threat and Risk Mapping Analysis project (TRMA)
  • Geo-Spatial Info Analysis for Global Security, Stability Program (ISFEREA)
  • FrontlineSMS
  • M-PESA and M-PAISA
  • Souktel

I think this long and diverse list of case studies clearly shows that the field of humanitarian technology is coming into it’s own.  Have a look at the report to learn how all these fit in the ecosystem of humanitarian technologies. And check out the tag #Tech4Dev on Twitter or the UN Foundation’s Facebook page to discuss the report and feel free to add any comments to this blog post below. I’m happy to answer all questions. In the meantime, I salute the UN Foundation for producing a forward looking report on projects that are barely two years old, and some just two months old.

Patrick Philippe Meier

Evolving a Global System of Info Webs

I’ve already blogged about what an ecosystem approach to conflict early warning and response entails. But I have done so with a country focus rather than thinking globally. This blog post applies a global perspective to the ecosystem approach given the proliferation of new platforms with global scalability.

Perhaps the most apt analogy here is one of food webs where the food happens to be information. Organisms in a food web are grouped into primary producers, primary consumers and secondary consumers. Primary producers such as grass harvest an energy source such as sunlight that they turn into biomass. Herbivores are primary consumers of this biomass while carnivores are secondary consumers of herbivores. There is thus a clear relationship known as a food chain.

This is an excellent video visualizing food web dynamics produced by researchers affiliated with the Santa Fe Institute (SFI):

Our information web (or Info Web) is also composed of multiple producers and consumers of information each interlinked by communication technology in increasingly connected ways. Indeed, primary producers, primary consumers and secondary consumers also crawl and dynamically populate the Info Web. But the shock of the information revolution is altering the food chains in our ecosystem. Primary consumers of information can now be primary producers, for example.

At the smallest unit of analysis, individuals are the most primary producers of information. The mainstream media, social media, natural language parsing tools, crowdsourcing platforms, etc, arguably comprise the primary consumers of that information. Secondary consumers are larger organisms such as the global Emergency Information Service (EIS) and the Global Impact and Vulnerability Alert System (GIVAS).

These newly forming platforms are at different stages of evolution. EIS and GIVAS are relatively embryonic while the Global Disaster Alert and Coordination Systems (GDACS) and Google Earth are far more evolved. A relatively new organism in the Info Web is the UAV as exemplified by ITHACA. The BrightEarth Humanitarian Sensor Web (SensorWeb) is further along the information chain while Ushahidi’s Crisis Mapping platform and the Swift River driver are more mature but have not yet deployed as a global instance.

InSTEDD’s GeoChat, Riff and Mesh4X solutions have already iterated through a number of generations. So have ReliefWeb and the Humanitarian Information Unit (HIU). There are of course additional organisms in this ecosystem, but the above list should suffice to demonstrate my point.

What if we connected these various organisms to catalyze a super organism? A Global System of Systems (GSS)? Would the whole—a global system of systems for crisis mapping and early warning—be greater than the sum of its parts? Before we can answer this question in any reasonable way, we need to know the characteristics of each organism in the ecosystem. These organisms represent the threads that may be woven into the GSS, a global web of crisis mapping and early warning systems.

Global System of Systems

Emergency Information Service (EIS) is slated to be a unified communications solution linking citizens, journalists, governments and non-governmental organizations in a seamless flow of timely, accurate and credible information—even when local communication infrastructures are rendered inoperable. This feature will be made possible by utilizing SMS as the communications backbone of the system.

In the event of a crisis, the EIS team would sift, collate, make sense of and verify the myriad of streams of information generated by a large humanitarian intervention. The team would gather information from governments, local media, the military, UN agencies and local NGOs to develop reporting that will be tailored to the specific needs of the affected population and translated into local languages. EIS would work closely with local media to disseminate messages of critical, life saving information.

Global Impact and Vulnerability Alert System (GIVAS) is being designed to closely monitor vulnerabilities and accelerate communication between the time a global crisis hits and when information reaches decision makers through official channels. The system is mandated to provide the international community with early, real-time evidence of how a global crisis is affecting the lives of the poorest and to provide decision-makers with real time information to ensure that decisions take the needs of the most vulnerable into account.

BrightEarth Humanitarian Sensor Web (SensorWeb) is specifically designed for UN field-based agencies to improve real time situational awareness. The dynamic mapping platform enables humanitarians to easily and quickly map infrastructure relevant for humanitarian response such as airstrips, bridges, refugee camps, IDP camps, etc. The SensorWeb is also used to map events of interest such as cholera outbreaks. The platform leverages mobile technology as well as social networking features to encourage collaborative analytics.

Ushahidi integrates web, mobile and dynamic mapping technology to crowdsource crisis information. The platform uses FrontlineSMS and can be deployed quickly as a crisis unfolds. Users can visualize events of interest on a dynamic map that also includes an animation feature to visualize the reported data over time and space.

Swift River is under development but designed to validate crowdsourced information in real time by combining machine learning for predictive tagging with human crowdsourcing for filtering purposes. The purpsose of the platform is to create veracity scores to denote the probability of an event being true when reported across several media such as Twitter, Online news, SMS, Flickr, etc.

GeoChat and Mesh4X could serve as the nodes connecting the above platforms in dynamic ways. Riff could be made interoperable with Swift River.

Can such a global Info Web be catalyzed? The question hinges on several factors the most important of which are probably awareness and impact. The more these individual organisms know about each other, the better picture they will have of the potential synergies between their efforts and then find incentives to collaborate. This is one of the main reasons I am co-organizing the first International Conference on Crisis Mapping (ICCM 2009) next week.

Patrick Philippe Meier

Spying with Maps

Mark Monmonier has written yet another excellent book on maps. I relished and reviewed his earlier book on “How To Lie with Maps” and enjoyed this one even more on “Spying with Maps.” I include below some short excerpts that I found particularly neat and interesting.

Picture 1

“Mapping, it turns out, can reveal quite a bit about what we do and who we are. I say mapping, rather than maps, because cartography is not limited to static maps printed on paper or displayed on computer screens. In the new cartographies of surveillance, the maps one looks at are less important than the spatial data systems that store and integrate facts about where we live and work. Location is a powerful key for relating disparate databanks and unearthing information [...].”

Big Brother is doing most of the watching, at least for now, but corporations, local governments, and other Little Brothers are quickly getting involved.”

“Much depends, of course, on who’s in charge, us or them, and on who ‘them’ is. A police state could exploit geographic technology to round up dissidents—imagine the Nazi SS with a GeoSurveillance Corps. By contrast, a capitalist marketer can exploit locational data by making a cleverly tailored pitch at a time and place when you’re most receptive. Control is control whether it’s blatant or subtle.”

Corrona Satellites

“Spy satellites became a top priority during the Cold War, and Congress generously supported remote sensing. [...] analysts with security clearances pored over images from the CIA’s top-secret Corona satellites at the agency’s clandestine National Reconnaissance Office (NRO).” By 1967, “a massive research and development effort had refined the [resolution] down to an impressive 1.5 meters (5 ft.).” Today’s “intelligence satellites have even sharper eyes: various estimates suggest that pictures from Corona’s most advanced successors have a resolution of roughly 3 inches.”

“Because image intelligence focuses on detecting change, 1-meter satellite imagery is often more informative [then people realize]. A new railway spur or clearing, for instance, could signify a new missile site or weapons factory. And a suspicious accumulation of vehicles might presage an imminent attack. As John Pike observes, ‘if a picture is worth 1,000 words, two pictures are worth 10,000 words.’”

“Washington strongly discourages the sale of high-resolution satellite imagery of Israel, and during the 2001 Middle Eastern campaign, the government thwarted enemy media hopes by buying exclusive rights to Ikonos imagery of Afghanistan.”

I really appreciated Mark’s take on the panopticon. His points below are largely ignored by the mainstream literature on the subject and go a long way to explaining just why satellite imagery has not (yet?) acted a strong deterrent against genocide and crimes against humanity. For more on this, please see this post on geospatial technologies for genocide prevention.

Panopticon

Panopticon

“Although the [panopticon] metaphor seems largely appropriate, I am not convinced  that the similarity between Bentham’s model prison and video surveillance tells us anything that’s not obvious about the watcher’s power over the watched. My hunch is that the prison’s walls and bars as well as the isolation of inmates in individual cells exert far greater control over prisoners’ lives than a ready ability to spy on their actions. [...] What’s relevant [...] is the power of surveillance to intimidate someone already under the watcher’s control, like a prisoner (who can be beaten), an employee (who can be fired), or a motorist who runs red lights (and could be fined or lose his or her license.”

I had come across ShotSpotter a while back but rediscovered the tool in Mark’s book. What is neat is that ShotSpotter combines audio and mapping in a way that may also be applicable to crisis mapping.

Shotspotter

Shotspotter

“[...] police in several California cities rely on ShotSpotter, which its investors describe as an ‘automatic real-time gunshot locator and display system’ [...] a clever marriage of seismic analysis and acoustic filtering. [...] Like an earthquake, a gunshot generates a sharply defined circular pulse, which expands outward at constant speed. [...] ShotSpotter’s microphones detect the wave at slightly different times depending on their distance from the shooter’s location [which the computer can use] to triangulate a location in either two or three dimension. [...] The process pinpoints gunshots within 15 yards [...].”

Patrick Philippe Meier

Crisis Mapping and Health Geographics

Crisis Mapping is by definition a cross-disciplinary field. Crises can be financial, ecological, humanitarian, etc., but these crises all happen in time and space, and necessarily interact with social networks. We may thus want to learn how different fields such as health, environment, biology, etc., visualize and analyze large complex sets of data to detect and amplify or dampen specific patterns.

We can’t all become specialists in each others’ areas of expertise but we can learn from each other, especially if we share a common language. Like the field of complexity science, Crisis Mapping can provide a common but malleable language, taxonomy and conceptual framework to facilitate the exchange of insights driven by innovative thinking in diverse fields.

This explains why I was excited to come across the International Journal of Health Geographics a few days ago. The Journal is an online and open-access resource. This means new ideas can be shared openly, which is conducive to innovation, just like arXiv.

Two of the Journal’s latest articles caught my interest:

1) An Agent-Based Approach for Modeling Dynamics of Contagious Disease Spread

This study developed a spatially explicit epidemiological model of infectious disease to better understand how contagious diseases spatially diffuse through a network of human contacts. To do this, the authors developed an agent-based model (ABM) that integrates geographic information systems (GIS) to simulate the spatial diffusion. (See my previous post on ABM and crisis mapping).

What is very neat about the authors’ approach is that they chose to draw on georeferenced land use data and census data. In other words, they combined the fomalistic rules of ABM with empirical GIS data. This means that the model can actually be tested and different scenarios can be played out by adding or changing some of the parameters. Could we use this model for conflict contageon?

2) Combining Google Earth and GIS Mapping Technologies in a Dengue Surveillance System

This study overlayed georeferenced epidemiological data on a town in Nicaragua with satellite imagery from Google Earth to enable dengue control specialists to prioritize specific neighborhoods for targetted interventions. The authors used ArcGIS to “accurately identify areas with high indices of mosquito infestation and interpret the spatial relationship of these areas with potential larval development sites such as garbage piles and large pools of standing water.”

It’s worth noting that the above Google Earth imagery was not particularly high resolution but the authors were still able to make full use of the imagery.

This approach to mapping for decision-support is particularly relevant for resource-limited settings since. As the authors note, the surveillance project “utilizes readily available technologies that do not rely on Internet access for daily use and can easily be implemented in many developing countries for very little cost.”

While the team had a free copy of ArcGIS thanks to the Global Fund, they plan to consider free and low-cost alternatives such as SaTScan, MapServer and Quantum GIS in the future. (See this post for additional alternatives like GeoCommons). I hope the authors also know about Walking Papers. I’ll email them just in case. Here’s to cross-disciplinary collaboration!

Patrick Philippe Meier


US Calls for UN Aerial Surveillance to Detect Preparations for Attacks

The US President:

“I am planning in the near future to submit to the United Nations a proposal for the creation of a United Nations aerial surveillance to detect preparations for attack. This plan I had intended to place before this conference. This surveillance system would operate in the territories of all nations prepared to accept such inspection. For its part, the United States is prepared not only to accept United Nations aerial surveillance but to do everything in its power to contribute to the rapid organization and successful operation of such international surveillance.”

The conference in question was the US-Soviet Summit meeting held in Paris on May 16th, 1960, and the words above were Dwight Eisenhower’s. Just weeks earlier, the Soviets had shot down an American U-2 CIA spy plane and captured it’s pilot Gary Powers. The Soviet leader Nikita Khrushchev lost no time in lashing out against the US President during the Summit, holding him directly responsible for the collapse of the talks, which many on both sides had hoped would usher in a period of “peaceful coexistence” between the superpowers.

Khrushchev called the espionage sanctioned by Eisenhower a provocative and aggressive act against the Soviet Union.

“We regret that this Meeting has been torpedoed by the reactionary element in the United States as the outcome of provocative flights by American military planes over the Soviet Union. [...] Let the shame and blame for it fall on those who have proclaimed a brigand policy in relation to the Soviet Union…” (1).

Eisenhower, who is said to have been furious at Khrushchev’s public attacks, replied forthwith:

“I have come to Paris to seek agreements with the Soviet Union which would eliminate the necessity for all forms of espionage, including overflights. I see no reason to use this incident to disrupt the conference.”

“Should it prove impossible, because of the Soviet attitude, to come to grips here in Paris with this problem and the other vital issues threatening world peace, I am planning in the near future to submit to the United Nations a proposal for the creation of a United Nations aerial surveillance to detect preparations for attack. This plan I had intended to place before this conference. This surveillance system would operate in the territories of all nations prepared to accept such inspection. For its part, the United States is prepared not only to accept United Nations aerial surveillance but to do everything in its power to contribute to the rapid organization and successful operation of such international surveillance” (2).

I find this all absolutely fascinating, and mentioned the exchange to colleagues at UNOSAT just a few weeks ago at CERN in Geneva. The UN’s Operational Satellite Program was actually created 40 years after Eisenhower’s threats to set up UN aerial surveillance unit. It was equally fascinating to learn about UNOSAT’s analysis of satellite imagery during Sri Lanka’s military attacks in April. The analysis clearly showed that the military shelled areas where civilians were sheltering in a no-fire zone.

As per UNOSAT’s mandate, this analysis was done regardless of whether the Sri Lankan government was prepared to accept such inspection, and rightly so.

Military attacks are not random, they are organized. This by definition means that preparations for military attacks reveal patterns. Heavy equipment, military trucks, jeeps, etc., all need to be mobilized in a coordinated manner. I recently spoke with one of the world’s leading experts on automated change detection of satellite imagery and he confirmed that algorithms could now be developed to detect specific types of traffic patterns, for example.

Will the UN ever be allowed to monitor and detect preparations for attack? After all, the first Article of the Charter commits the UN to “maintain international peace and security, and to that end: to take effective collective measures for the prevention and removal of threats to the peace [...].” Can a US President today commit the UN to a full fledged international aerial surveillance program? There clearly is a strong precedent and it is important we not forget this important piece of history.

UPDATED: Professor Alan Kuperman just sent me an email the Open Skies Proposal that Eisenhower put forward 5 years before the US-Soviet Summit. The Open Skies Treaty actually entered into force in 2002:

The Treaty establishes a regime of unarmed aerial observation flights over the entire territory of its participants. The Treaty is designed to enhance mutual understanding and confidence by giving all participants, regardless of size, a direct role in gathering information about military forces and activities of concern to them. Open Skies is one of the most wide-ranging international efforts to date to promote openness and transparency of military forces and activities.

Absolutely fascinating, thanks Alan!

Patrick Philippe Meier