Starting small can lead to startling results
Case in point, the City of Chicago. One person, one laptop, and MongoDB’s technology jumpstarted a project that, with other people joining in, went from prototype to one of the nation’s pioneering projects to analyze and act on municipal data in real time.
Called WindyGrid, this system is an intelligent operations platform built on MongoDB. What makes WindyGrid exceptional isn’t just that it pulls together seven million different pieces of data from city departments every day. It’s that WindyGrid creatively pairs MongoDB-powered analytics with visual maps, giving managers insights they’ve never had before on city operations. In real time. As they unfold.
With MongoDB, WindyGrid created a central nervous system for Chicago, helping improve services, cut costs, and create a more livable city.
WindyGrid’s powerful combination of data, maps, and analytics pinpoints unexpected correlations, identifies potential issues before they develop into bigger problems, and helps coordinate responses among departments to everything from marathons to major snowstorms. Managers can act in the moment, instead of after the fact. Even better, they can anticipate, rather than just react to issues.
The platform makes sense of the millions of pieces of information gathered daily from Chicago’s 15 most crucial departments, including police, transportation, and fire. It’s an ever changing, skyrocketing collection that tracks what makes the city tick – or doesn’t. Roadwork updates, trash pickup delays, 911 health emergencies, 311 complaints about noise, public tweets about the minutia of the city’s workings, bus locations along their route, traffic light patterns, and much more.
A Platform for Innovation
In 2011, Brett Goldstein, an Internet entrepreneur turned beat cop turned Chief Data Officer of Chicago, was scrambling during the first days of his new job. Goldstein had made a name for himself in the Chicago Police Department by launching a new program that uses data to improve policing. Former Mayor Rahm Emanuel had tapped him to meet the city’s new goal of not just using data analysis to improve services, but to open up the city data to the public.
Typing away on his laptop in the mayor’s office, Goldstein started with a single MongoDB node and built from there with his team. Goldstein’s innovation was to see how the data squirreled away in the city’s siloed systems could be linked together using geolocation and analytics to create maps that dished up entirely new real-time perspectives they could act on immediately.
This vision required a different approach to data. Chicago’s new system didn’t just need to combine floods of structured and unstructured data. It needed powerful analytics that could parse that data in real time. And Chicago’s mayor wanted this system in place immediately. Goldstein decided to use MongoDB and created a solution never before possible: a real-time geospatial platform for innovation made possible by MongoDB’s flexible document storage, powerful and fast analytics, and scalable architecture.
WindyGrid in Action
By pulling together 311 and 911 calls, tweets, and bus locations, the city can better manage traffic and incidents during parades and get streets cleaned and opened up more quickly.
Using analytics, city managers can pinpoint the cascading consequences of separate events, such as faulty streetlights or new traffic patterns, to address problems or prevent issues before they occur.
For instance, after statistically predicting that within seven days of a garbage complaint a rodent complaint occurs, the city is able to get ahead of rat problems.
WindyGrid’s Backbone: The Flexible Data Model and Powerful Real-Time Analytics
WindyGrid put Chicago on the path of revolutionizing how it operates not by replacing the administrative systems already in place, but by using MongoDB to bring that data together into a new application. With MongoDB’s flexible data model, WindyGrid doesn’t have to go back and redo the schema for each new piece of data. Instead, it can evolve schemas in real time. Which is crucial as WindyGrid expands and adds predictive analytics, growing by millions of pieces of structured and unstructured data each day.
This flexible, scalable aggregation of data is the backbone for running real-time, predictive analytics. MongoDB can store any kind of data, no matter the structure, format, or source, no matter how often it changes. And it can analyze any kind of data in place within the database, using features such as rich indexing and query support for secondary, geospatial and text search indexes. All while providing horizontal scalability through automatic sharding across widely distributed commodity servers. Which means MongoDB can dish up results in real time and cost effectively.
Making Data Smarter
With MongoDB, Chicago now has the unified view of city operations it needs to analyze data, pull out crucial insights, and build new applications. This single view of city operations on a map of Chicago gives managers access to all the city’s spatial data, historically and in real time. Surprising discoveries can be made when you’re able to consider data in new ways.
The next step for WindyGrid is building an open-source, predictive analytics system called the SmartData Platform to anticipate problems before they occur, and propose solutions. For example, officials will be able to map crime stats and 911 calls against the number of abandoned buildings or liquor permits in a neighborhood to pinpoint correlations. So it can prevent crimes instead of react to them.
This SmartData system will use real-time and historical data to predict the best ways, for instance, to prepare for weather emergencies or cut traffic congestion. Using predictive analytics, city managers will be able to dive into energy usage trends and take preemptive measures to improve efficiency rather than installing more power lines.
Meantime, the city has opened its data to the public, publishing more than 450 sets of data on everything from building permits to food inspections at restaurants. The government’s creative new approach to data sparked further innovation. The public responded by creating new services such as the Sweeparound, an app that taps street sweeping datasets to alert citizens when street sweepers are coming.
