Tag Archives: Disaster

Humanitarians Using UAVs for Post Disaster Recovery

I recently connected with senseFly’s Adam Klaptocz who founded the non-profit group DroneAdventures to promote humanitarian uses of UAVs. I first came across Adam’s efforts last year when reading about his good work in Haiti, which demonstrated the unique role that UAV technology & imagery can play in post-disaster contexts. DroneAdventures has also been active in Japan and Peru. In the coming months, the team will also be working on “aerial archeology” projects in Turkey and Egypt. When Adam emailed me last week, he and his team had just returned from yet another flying mission, this time in the Philippines. I’ll be meeting up with Adam in a couple weeks to learn more about their recent adventures. In the meantime, here’s a quick recap of what they were up to in the Philippines this month.

MedAir

Adam and team snapped hundreds of aerial images using their “eBee drones” to create a detailed set of 2D maps and 3D terrain models of the disaster-affected areas where partner Medair works. This is the first time that the Swiss humanitarian organization Medair is using UAVs to inform their recovery and rehabilitation programs. They plan to use the UAV maps & models of Tacloban and hard-hit areas in Leyte to assist in assessing “where the greatest need is” and what level of “assistance should be given to affected families as they continue to recover” (1). To this end, having accurate aerial images of these affected areas will allow the Swiss organization to “address the needs of individual households and advocate on their behalf when necessary” (2). 

ebee

An eBee Drone also flew over Dulag, north of Leyte, where more than 80% of the homes and croplands were destroyed following Typhoon Yolanda. Medair is providing both materials and expertise to build new shelters in Dulag. As one Medair representative noted during the UAV flights, “Recovery from a disaster of this magnitude can be complex. The maps produced from the images taken by the drones will give everyone, including community members themselves, an opportunity to better understand not only where the greatest needs are, but also their potential solutions” (3). The partners are also committed to Open Data: “The images will be made public for free online, enabling community leaders and humanitarian organizations to use the information to coordinate reconstruction efforts” (4). The pictures of the Philippines mission below were very kindly shared by Adam who asked that they be credited to DroneAdventures.

Credit: DroneAdventures

At the request of the local Mayor, DroneAdventures and MedAir also took aerial images of a relatively undamaged area some 15 kilometers north of Tacloban, which is where the city government is looking to relocate families displaced by Typhoon Yolanda. During the deployment, Adam noted that “Lightweight drones such as the eBee are safe and easy to operate and can provide crucial imagery at a precision and speed unattainable by satellite imagery. Their relatively low cost of deployment make the technology attainable even by small communities throughout the developing world. Not only can drones be deployed immediately following a disaster in order to assess damage and provide detailed information to first-responders like Medair, but they can also assist community leaders in planning recovery efforts” (5). As the Medair rep added, “You can just push a button or launch them by hand to see them fly, and you don’t need a remote anymore—they are guided by GPS and are inherently safe” (6).

Credit: DroneAdventures

I really look forward to meeting up with Adam and the DroneAdventures team at the senseFly office in Lausanne next month to learn more about their recent work and future plans. I will also be asking the team for their feedback and guidance on the Humanitarian UAV Network (UAViators) that I am launching. So stay tuned for updates!

Bio

See also:

  • Calling All UAV Pilots: Want to Support Humanitarian Efforts? [link]
  • How UAVs are Making a Difference in Disaster Response [link]
  • Grassroots UAVs for Disaster Response (in the Philippines) [link]

 

Grassroots UAVs for Disaster Response

I was recently introduced to a new initiative that seeks to empower grassroots communities to deploy their own low-cost xUAVs. The purpose of this initiative? To support locally-led disaster response efforts and in so doing transfer math, science and engineering skills to local communities. The “x” in xUAV refers to expendable. The initiative is a partnership between California State University (Long Beach), University of Hawaii, Embry Riddle, The Philippine Council for Industry, Energy & Emerging Technology Research & Development, Skyeye, Aklan State University and Ateneo de Manila University in the Philippines. The team is heading back to the Philippines next week for their second field mission. This blog post provides a short overview of the project’s approach and the results from their first mission, which took place during December 2013-February 2014.

xUAV1

The xUAV team is specifically interested in a new category of UAVs, those that are locally available, locally deployable, low-cost, expendable and extremely easy to use. Their first field mission to the Philippines focused on exploring the possibilities. The pictures above/below (click to enlarge) were kindly shared by the Filipinos engaged in the project—I am very grateful to them for allowing me to share these publicly. Please do not reproduce these pictures without their written permission, thank you.

xUAV2

I spoke at length with one of the xUAV team leads, Ted Ralston, who is heading back to the Philippines the second field mission. The purpose of this follow up visit is to shift the xUAV concept from experimental to deployable. One area that his students will be focusing on with the University of Manila is the development of a very user-friendly interface (using a low-cost tablet) to pilot the xUAVs so that local communities can simply tag way-points on a map that the xUAV will then automatically fly to. Indeed, this is where civilian UAVs are headed, full automation. A good example of this trend towards full automation is the new DroidPlanner 2.0 App just released by 3DRobotics. This free app provides powerful features to very easily plan autonomous flights. You can even create new flight plans on the fly and edit them onsite.

DroidPlanner.png

So the xUAV team will focus on developing software for automated take-off and landing as well as automated adjustments for wind conditions when the xUAV is airborne, etc. The software will also automatically adjust the xUAV’s flight parameters for any added payloads. Any captured imagery would then be made easily viewable via touch-screen directly from the low-cost tablet.

xUAV3

One of the team’s top priorities throughout this project is to transfer their skills to young Filipinos, given them hands on training in science, math and engineering. An equally important, related priority, is their focus on developing local partnerships with multiple partners. We’re familiar with ideas behind Public Participatory GIS (PPGIS) vis-a-vis the participatory use of geospatial information systems and technologies. The xUAV team seeks to extend this grassroots approach to Public Participatory UAVs.

xUAV4

I’m supporting this xUAV initiative in a number of ways and will be uploading the team’s UAV imagery (videos & still photos) from their upcoming field mission to MicroMappers for some internal testing. I’m particularly interested in user-generated (aerial) content that is raw and not pre-processed or stitched together, however. Why? Because I expect this type of imagery to grow in volume given the very rapid growth of the personal micro-UAV market. For more professionally produced and stitched-together aerial content, an ideal platform is Humanitarian OpenStreetMap’s Tasking Server, which is tried and tested for satellite imagery and which was recently used to trace processed UAV imagery of Tacloban.

Screen Shot 2014-03-12 at 1.03.20 PM

I look forward to following the xUAV team’s efforts and hope to report on the outcome of their second field mission. The xUAV initiative fits very nicely with the goals of the Humanitarian UAV Network (UAViators). We’ll be learning a lot in the coming weeks and months from our colleagues in the Philippines.

bio

Crisis Mapping without GPS Coordinates

I recently spoke with a UK start-up that is doing away with GPS coordinates even though their company focuses on geographic information and maps. The start-up, What3Words, has divided the globe into 57 trillion squares and given each of these 3-by-3 meter areas a unique three-word code. Goodbye long postal addresses and cryptic GPS coordinates. Hello planet.inches.most. The start-up also offers a service called OneWord, which allows you to customize a one-word name for any square. In addition, the company has expanded to other languages such as Spanish, Swedish and Russian. They’re now working on including Arabic, Chinese, Japanese and others by mid-January 2014. Meanwhile, their API lets anyone build new applications that tap their global map of 57 trillion squares.

Credit: What3Words

When I spoke with CEO Chris Sheldrick, he noted that their very first users were emergency response organizations. One group in Australia, for example, is using What3Words as part of their SMS emergency service. “This will let people identify their homes with just three words, ensuring that emergency vehicles can find them as quickly as possible.” Such an approach provides greater accuracy, which is vital in rural areas. “Our ambulances have a terrible time with street addresses, particularly in The Bush.” Moreover, many places in the world have no addresses at all. So What3Words may also be useful for certain ICT4D projects in addition to crisis mapping. The real key to this service is simplicity, i.e., communicating three words over the phone, via SMS/Twitter or email is far easier (and less error prone) than dictating a postal address or a complicated set of GPS coordinates.

Credit: What3Words

How else do you think this service could be used vis-à-vis disaster response?

Bio

Rapid Disaster Damage Assessments: Reality Check

The Multi-Cluster/Sector Initial Rapid Assessment (MIRA) is the methodology used by UN agencies to assess and analyze humanitarian needs within two weeks of a sudden onset disaster. A detailed overview of the process, methodologies and tools behind MIRA is available here (PDF). These reports are particularly insightful when comparing them with the processes and methodologies used by digital humanitarians to carry out their rapid damage assessments (typically done within 48-72 hours of a disaster).

MIRA PH

Take the November 2013 MIRA report for Typhoon Haiyan in the Philippines. I am really impressed by how transparent the report is vis-à-vis the very real limitations behind the assessment. For example:

  • “The barangays [districts] surveyed do not constitute a represen-tative sample of affected areas. Results are skewed towards more heavily impacted municipalities [...].”
  • “Key informant interviews were predominantly held with baranguay captains or secretaries and they may or may not have included other informants including health workers, teachers, civil and worker group representatives among others.”
  • Barangay captains and local government staff often needed to make their best estimate on a number of questions and therefore there’s considerable risk of potential bias.”
  • Given the number of organizations involved, assessment teams were not trained in how to administrate the questionnaire and there may have been confusion on the use of terms or misrepresentation on the intent of the questions.”
  • “Only in a limited number of questions did the MIRA checklist contain before and after questions. Therefore to correctly interpret the information it would need to be cross-checked with available secondary data.”

In sum: The data collected was not representative; The process of selecting interviewees was biased given that said selection was based on a convenience sample; Interviewees had to estimate (guesstimate?) the answer for several questions, thus introducing additional bias in the data; Since assessment teams were not trained to administrate the questionnaire, this also introduces the problem of limited inter-coder reliability and thus limits the ability to compare survey results; The data still needs to be validated with secondary data.

I do not share the above to criticize, only to relay what the real world of rapid assessments resembles when you look “under the hood”. What is striking is how similar the above challenges are to the those that digital humanitarians have been facing when carrying out rapid damage assessments. And yet, I distinctly recall rather pointed criticisms leveled by professional humanitarians against groups using social media and crowdsourcing for humanitarian response back in 2010 & 2011. These criticisms dismissed social media reports as being unrepresentative, unreliable, fraught with selection bias, etc. (Some myopic criticisms continue to this day). I find it rather interesting that many of the shortcomings attributed to crowdsourcing social media reports are also true of traditional information collection methodologies like MIRA.

The fact is this: no data or methodology is perfect. The real world is messy, both off- and online. Being transparent about these limitations is important, especially for those who seek to combine both off- and online methodologies to create more robust and timely damage assessments.

bio

Video: Humanitarian Response in 2025

I gave a talk on “The future of Humanitarian Response” at UN OCHA’s Global Humanitarian Policy Forum (#aid2025) in New York yesterday. More here for context. A similar version of the talk is available in the video presentation below.

Some of the discussions that ensued during the Forum were frustrating albeit an important reality check. Some policy makers still think that disaster response is about them and their international humanitarian organizations. They are still under the impression that aid does not arrive until they arrive. And yet, empirical research in the disaster literature points to the fact that the vast majority of survivals during disasters is the result of local agency, not external intervention.

In my talk (and video above), I note that local communities will increasingly become tech-enabled first responders, thus taking pressure off the international humanitarian system. These tech savvy local communities already exit. And they already respond to both “natural” (and manmade) disasters as noted in my talk vis-a-vis the information products produced by tech-savvy local Filipino groups. So my point about the rise of tech-enabled self-help was a more diplomatic way of conveying to traditional humanitarian groups that humanitarian response in 2025 will continue to happen with or without them; and perhaps increasingly without them.

This explains why I see OCHA’s Information Management (IM) Team increasingly taking on the role of “Information DJ”, mixing both formal and informal data sources for the purposes of both formal and informal humanitarian response. But OCHA will certainly not be the only DJ in town nor will they be invited to play at all “info events”. So the earlier they learn how to create relevant info mixes, the more likely they’ll still be DJ’ing in 2025.

Bio

How UAVs Are Making a Difference in Disaster Response

I visited the University of Torino in 2007 to speak with the team developing UAVs for the World Food Program. Since then, I’ve bought and tested two small UAVs of my own so I can use this new technology to capture aerial imagery during disasters; like the footage below from the Philippines.

UAVs, or drones, have a very strong military connotation for many of us. But so did space satellites before Google Earth brought satellite imagery into our homes and changed our perceptions of said technology. So it stands to reason that UAVs and aerial imagery will follow suit. This explains why I’m a proponent of the Drone Social Innovation Award, which seeks to promote the use of civilian drone technology for the benefit of humanity. I’m on the panel of judges for this award, which is why I reached out to DanOffice IT, a Swiss-based company that deployed two drones in response to Typhoon Yolanda in the Philippines. The drones in question are Huginn X1′s, which have a flight time of 25 minutes with a range of 2 kilometers and maximum altitude of 150 meters.

HUGINN X1

I recently spoke with one of the Huginn pilots who was in Tacloban. He flew the drone to survey shelter damage, identify blocked roads and search for bodies in the debris (using thermal imaging cameras mounted on the drone for the latter). The imagery captured also helped to identify appropriate locations to set up camp. When I asked the pilot whether he was surprised by anything during the operation, he noted that road-clearance support was not a use-case he had expected. I’ll be meeting with him in Switzerland in the next few weeks to test-fly a Huginn and explore possible partnerships.

I’d like to see closer collaboration between the Digital Humanitarian Network (DHN) and groups like DanOffice, for example. Providing DHN-member Humanitarian OpenStreetMap (HOTosm) with up-to-date aerial imagery during disasters would be a major win. This was the concept behind OpenAerialMap, which was first discussed back in 2007. While the initiative has yet to formally launch, PIX4D is a platform that “converts thousands of aerial images, taken by lightweight UAV or aircraft into geo-referenced 2D mosaics and 3D surface models and point clouds.”

Drone Adventures

This platform was used in Haiti with the above drones. The International Organization for Migration (IOM) partnered with Drone Adventures to map over 40 square kilometers of dense urban territory including several shantytowns in Port-au-Prince, which was “used to count the number of tents and organize a ‘door-to-door’ census of the population, the first step in identifying aid requirements and organizing more permanent infrastructure.” This approach could also be applied to IDP and refugee camps in the immediate aftermath of a sudden-onset disaster. All the data generated by Drone Adventures was made freely available through OpenStreetMap.

If you’re interested in giving “drones for social good” a try, I recommend looking at the DJI Phantom and the AR.Drone Parrot. These are priced between $300- $600, which beats the $50,000 price tag of the Huginn X1.

 bio

Automatically Identifying Eyewitness Reporters on Twitter During Disasters

My colleague Kate Starbird recently shared a very neat study entitled “Learning from the Crowd: Collaborative Filtering Techniques for Identifying On-the-Ground Twitterers during Mass Disruptions” (PDF). As she and her co-authors rightly argue, “most Twitter activity during mass disruption events is generated by the remote crowd.” So can we use advanced computing to rapidly identify Twitter users who are reporting from ground zero? The answer is yes.

twitter-disaster-test

An important indicator of whether or not a Twitter user is reporting from the scene of a crisis is the number of times they are retweeted. During the Egyptian revolution in early 2011, “nearly 30% of highly retweeted Twitter users were physically present at those protest events.” Kate et al. drew on this insight to study tweets posted during the Occupy Wall Street (OWS) protests in September 2011. The authors manually analyzed a sample of more than 2,300 Twitter users to determine which were tweeting from the protests. They found that 4.5% of Twitter users in their sample were actually onsite. Using this dataset as training data, Kate et al. were able to develop a classifier that can automatically identify Twitter users reporting from the protests with an accuracy of just shy of 70%. I expect that more training data could very well help increase this accuracy score. 

In any event, “the information resulting from this or any filtering technique must be further combined with human judgment to assess its accuracy.” As the authors rightly note, “this ‘limitation’ fits well within an information space that is witnessing the rise of digital volunteer communities who monitor multiple data sources, including social media, looking to identify and amplify new information coming from the ground.” To be sure, “For volunteers like these, the use of techniques that increase the signal to noise ratio in the data has the potential to drastically reduce the amount of work they must do. The model that we have outlined does not result in perfect classification, but it does increase this signal-to-noise ratio substantially—tripling it in fact.”

I really hope that someone will leverage Kate’s important work to develop a standalone platform that automatically generates a list of Twitter users who are reporting from disaster-affected areas. This would be a very worthwhile contribution to the ecosystem of next-generation humanitarian technologies. In the meantime, perhaps QCRI’s Artificial Intelligence for Disaster Response (AIDR) platform will help digital humanitarians automatically identify tweets posted by eyewitnesses. I’m optimistic since we were able to create a machine learning classifier with an accuracy of 80%-90% for eyewitness tweets. More on this in our recent study

MOchin - talked to family

One question that remains is how to automatically identify tweets like the one above? This person is not an eyewitness but was likely on the phone with her family who are closer to the action. How do we develop a classifier to catch these “second-hand” eyewitness reports?

bio

World Disaster Report: Next Generation Humanitarian Technology

This year’s World Disaster Report was just released this morning. I had the honor of authoring Chapter 3 on “Strengthening Humanitarian Information: The Role of Technology.” The chapter focuses on the rise of “Digital Humanitarians” and explains how “Next Generation Humanitarian Technology” is used to manage Big (Crisis) Data. The chapter complements the groundbreaking report “Humanitarianism in the Network Age” published by UN OCHA earlier this year.

The key topics addressed in the chapter include:

  • Big (Crisis) Data
  • Self-Organized Disaster Response
  • Crowdsourcing & Bounded Crowdsourcing
  • Verifying Crowdsourced Information
  • Volunteer & Technical Communities
  • Digital Humanitarians
  • Libya Crisis Map
  • Typhoon Pablo Crisis Map
  • Syria Crisis Map
  • Microtasking for Disaster Response
  • MicroMappers
  • Machine Learning for Disaster Response
  • Artificial Intelligence for Disaster Response (AIDR)
  • American Red Cross Digital Operations Center
  • Data Protection and Security
  • Policymaking for Humanitarian Technology

I’m particularly interested in getting feedback on this chapter, so feel free to pose any comments or questions you may have in the comments section below.

bio

See also:

  • What is Big (Crisis) Data? [link]
  • Humanitarianism in the Network Age [link]
  • Predicting Credibility of Disaster Tweets [link]
  • Crowdsourced Verification for Disaster Response [link]
  • MicroMappers: Microtasking for Disaster Response [link]
  • AIDR: Artificial Intelligence for Disaster Response [link]
  • Research Agenda for Next Generation Humanitarian Tech [link]

AIDR: Artificial Intelligence for Disaster Response

Social media platforms are increasingly used to communicate crisis information when major disasters strike. Hence the rise of Big (Crisis) Data. Humanitarian organizations, digital humanitarians and disaster-affected communities know that some of this user-generated content can increase situational awareness. The challenge is to identify relevant and actionable content in near real-time to triangulate with other sources and make more informed decisions on the spot. Finding potentially life-saving information in this growing stack of Big Crisis Data, however, is like looking for the proverbial needle in a giant haystack. This is why my team and I at QCRI are developing AIDR.

haystpic_pic

The free and open source Artificial Intelligence for Disaster Response platform leverages machine learning to automatically identify informative content on Twitter during disasters. Unlike the vast majority of related platforms out there, we go beyond simple keyword search to filter for informative content. Why? Because recent research shows that keyword searches can miss over 50% of relevant content posted on Twitter. This is very far from optimal for emergency response. Furthermore, tweets captured via keyword search may not be relevant since words can have multiple meanings depending on context. Finally, keywords are restricted to one language only. Machine learning overcomes all these limitations, which is why we’re developing AIDR.

So how does AIDR work? There are three components of AIDR: the Collector, Trainer and Tagger. The Collector simply allows you to collect and save a collection of tweets posted during a disaster. You can download these tweets for analysis at any time and also use them to create an automated filter using machine learning, which is where the Trainer and Tagger come in. The Trainer allows one or more users to train the AIDR platform to automatically tag tweets of interest in a given collection of tweets. Tweets of interest could include those that refer to “Needs”, “Infrastructure Damage” or “Rumors” for example.

AIDR_Collector

A user creates a Trainer for tweets-of-interest by: 1) Creating a name for their Trainer, e.g., “My Trainer”; 2) Identifying topics of interest such as “Needs”, “Infrastructure Damage”,  “Rumors” etc. (as many topics as the user wants); and 3) Classifying tweets by topic of interest. This last step simply involves reading collected tweets and classifying them as “Needs”, “Infrastructure Damage”, “Rumor” or “Other,” for example. Any number of users can participate in classifying these tweets. That is, once a user creates a Trainer, she can classify the tweets herself, or invite her organization to help her classify, or ask the crowd to help classify the tweets, or all of the above. She simply shares a link to her training page with whoever she likes. If she choses to crowdsource the classification of tweets, AIDR includes a built-in quality control mechanism to ensure that the crowdsourced classification is accurate.

As noted here, we tested AIDR in response to the Pakistan Earthquake last week. We quickly hacked together the user interface displayed below, so functionality rather than design was our immediate priority. In any event, digital humanitarian volunteers from the Standby Volunteer Task Force (SBTF) tagged over 1,000 tweets based on the different topics (labels) listed below. As far as we know, this was the first time that a machine learning classifier was crowdsourced in the context of a humanitarian disaster. Click here for more on this early test.

AIDR_Trainer

The Tagger component of AIDR analyzes the human-classified tweets from the Trainer to automatically tag new tweets coming in from the Collector. This is where the machine learning kicks in. The Tagger uses the classified tweets to learn what kinds of tweets the user is interested in. When enough tweets have been classified (20 minimum), the Tagger automatically begins to tag new tweets by topic of interest. How many classified tweets is “enough”? This will vary but the more tweets a user classifies, the more accurate the Tagger will be. Note that each automatically tagged tweet includes an accuracy score—i.e., the probability that the tweet was correctly tagged by the automatic Tagger.

The Tagger thus displays a list of automatically tagged tweets updated in real-time. The user can filter this list by topic and/or accuracy score—display all tweets tagged as “Needs” with an accuracy of 90% or more, for example. She can also download the tagged tweets for further analysis. In addition, she can share the data link of her Tagger with developers so the latter can import the tagged tweets directly into to their own platforms, e.g., MicroMappers, Ushahidi, CrisisTracker, etc. (Note that AIDR already powers CrisisTracker by automating the classification of tweets). In addition, the user can share a display link with individuals who wish to embed the live feed into their websites, blogs, etc.

In sum, AIDR is an artificial intelligence engine developed to power consumer applications like MicroMappers. Any number of other tools can also be added to the AIDR platform, like the Credibility Plugin for Twitter that we’re collaborating on with partners in India. Added to AIDR, this plugin will score individual tweets based on the probability that they convey credible information. To this end, we hope AIDR will become a key node in the nascent ecosystem of next-generation humanitarian technologies. We plan to launch a beta version of AIDR at the 2013 CrisisMappers Conference (ICCM 2013) in Nairobi, Kenya this November.

In the meantime, we welcome any feedback you may have on the above. And if you want to help as an alpha tester, please get in touch so I can point you to the Collector tool, which you can start using right away. The other AIDR tools will be open to the same group of alpha tester in the coming weeks. For more on AIDR, see also this article in Wired.

AIDR_logo

The AIDR project is a joint collaboration with the United Nations Office for the Coordination of Humanitarian Affairs (OCHA). Other organizations that have expressed an interest in AIDR include the International Committee of the Red Cross (ICRC), American Red Cross (ARC), Federal Emergency Management Agency (FEMA), New York City’s Office for Emergency Management and their counterpart in the City of San Francisco. 

bio

Note: In the future, AIDR could also be adapted to take in Facebook status updates and text messages (SMS).

Developing MicroFilters for Digital Humanitarian Response

Filtering—or the lack thereof—presented the single biggest challenge when we tested MicroMappers last week in response to the Pakistan Earthquake. As my colleague Clay Shirky notes, the challenge with “Big Data” is not information overload but rather filter failure. We need to make damned sure that we don’t experience filter failure again in future deployments. To ensure this, I’ve decided to launch a stand-alone and fully interoperable platform called MicroFilters. My colleague Andrew Ilyas will lead the technical development of the platform with support from Ji Lucas. Our plan is to launch the first version of MicroFilters before the CrisisMappers conference (ICCM 2013) in November.

MicroFilters

A web-based solution, MicroFilters will allow users to upload their own Twitter data for automatic filtering purposes. Users will have the option of uploading this data using three different formats: text, CSV and JSON. Once uploaded, users can elect to perform one or more automatic filtering tasks from this menu of options:

[   ]  Filter out retweets
[   ]  Filter for unique tweets
[   ]  Filter tweets by language [English | Other | All]
[   ]  Filter for unique image links posted in tweets [Small | Medium | Large | All]
[   ]  Filter for unique video links posted in tweets [Short | Medium | Long | All]
[   ]  Filter for unique image links in news articles posted in tweets  [S | M | L | All]
[   ]  Filter for unique video links in news articles posted in tweets [S | M | L | All]

Note that “unique image and video links” refer to the long URLs not shortened URLs like bit.ly. After selecting the desired filtering option(s), the user simply clicks on the “Filter” button. Once the filtering is completed (a countdown clock is displayed to inform the user of the expected processing time), MicroFilters provides the user with a download link for the filtered results. The link remains live for 10 minutes after which the data is automatically deleted. If a CSV file was uploaded for filtering, the file format for download is also in CSV format; likewise for text and JSON files. Note that filtered tweets will appear in reverse chronological order (assuming time-stamp data was included in the uploaded file) when downloaded. The resulting file of filtered tweets can then be uploaded to MicroMappers within seconds.

In sum, MicroFilters will be invaluable for future deployments of MicroMappers. Solving the “filter failure” problem will enable digital humanitarians to process far more relevant data and in a more timely manner. Since MicroFilters will be a standalone platform, anyone else will also have access to these free and automatic filtering services. In the meantime, however, we very much welcome feedback, suggestions and offers of help, thank you!

bio