Using Big Data for Humanitarian Crisis Mapping



In the wake of natural disasters like Typhoon Haiyan, which brought widespread destruction to the Philippines several weeks ago, data management tools have become a critical component of the post-disaster landscape. Aid groups are monitoring tweets and instant messages where the infrastructure exists to support them, while tracking local news reports on the ground to find the areas suffering the greatest damage, directing resources to those most in need.

Sourcing data can significantly improve the efforts of aid initiatives after a disaster. Big data for development, or data philanthropy, streamlines crisis management and prevention by using data processing tools to anticipate and respond to humanitarian emergencies.

Initiatives like the UN Global Pulse team are using data to find the “digital smoke signals of distress” that can appear months before showing up on official reports. Real-time data monitoring using social networks, cell phones, blogs, and online commerce platforms can alert the team to indicators of social distress or natural disaster. And with the capacity to recognize these trends comes the ability to prepare the right aid or prevention plan that could save lives.

What Big Data Can Do

Big data can create a clear picture of a disaster’s regional effects. A program called Ushahidi sourced eyewitness reports (in person and through social media) of the 2010 earthquake in Haiti. The reports’ data became a live crisis map, showing where victims lay buried under collapsed buildings and where aid was most needed. After Typhoon Bopha in the Philippines last year, the Digital Humanitarian Initiative used over 20,000 social media messages to create a map of the storm’s impact and determine where to send aid first.

Some organizations believe data for development can soothe social discontent. CNN reported that the U.S. State Department has analyzed data to try and prevent conflict from starting or escalating. Its Conflict and Stabilization Operations office analyzes behavioral patterns and semantic trends in social media to anticipate threats to peace while designing strategies to thwart potential outbreaks of violence.

Partnerships For Philanthropy

As the data philanthropy movement grows, the tech industry will be observing which companies and corporations are the first to join this global project. Twitter, Facebook, or Instagram might help us move towards a future where disease or disaster can be instantly monitored and possibly prevented, or where the spread of poverty can be stopped in its tracks. The success of these new ventures will not only depend on the determination of the people who work on them. Small humanitarian initiatives will need to develop partnerships with the larger corporations that control telecommunications and census data. Without access to big data or the proper processing tools, data philanthropy groups will not be able to keep up with the demands of crises happening in real time.

Going Forward

The United Nations Office for the Coordination of Humanitarian Affairs released a report this past June on the importance of big data and humanitarianism. Finding ways to improve humanitarian aid services with data is one of the great challenges and opportunities of our age.

But accessing data is not necessarily straightforward. Negotiating with data providers can be difficult and privacy concerns could make corporations unwilling to participate. And while big data processing can be used to improve lives, it should augment existing data gathering methods, not replace them.

MongoDB has helped several organizations use data mining to augment public service. The city of Chicago used MongoDB to design WindyGrid, a geographic information system providing a unified view of the city’s operations across a map. Including real-time data like 911 and 311 service calls, critical information is geospatially enabled and tracked to help the Chicago’s Emergency Management and Communications Office handle events or crises across the city.

To explore the frontiers of physics, CERN built a Data Aggregation System (DAS) on MongoDB to help physicists search for and aggregate information across complex data landscapes. The data and metadata CERN handles are constantly evolving, but the DAS allows researchers to find information with text based queries, aggregating the results from distributed providers while preserving integrity and security. While these companies haven’t used data mining directly for humanitarian aid, mining data with MongoDB can easily be adapted to philanthropic service.

Data philanthropy has the potential to influence humanitarian efforts and change how we understand the scope of big data. As these aid organizations grow in influence, it will be interesting to see how the industry shifts to make room for this new use of data.