Getting to know Geospatial Indexing on MongoDB with TST
With built-in geospatial indexing, MongoDB is an excellent option for storing and querying location data. But how are these capabilities being used in the real world? We decided to discuss how one of 10gen's partners, Thermopylae Sciences + Technology (TST), have been leveraging MongoDB for customers in the Federal and Commercial markets.
Who is TST?
TST is a 10gen partner that focuses on implementing MongoDB for customers in the Federal and Commercial sector. We are avid open source supporters and have contributed additional tools and capabilities to mongoDB and other OSS projects.
How would you define Geospatial?
Geospatial deals with a few different key areas. The underlying and foundational layer of geospatial is generally known as a map or an image. Managing large amounts of data in this foundational layer has the potential to reach multi-petabyte scale quite quickly. The second element of geospatial is data that helps define the foundational layer - generally this is terrain elevation data or 3D building (urban terrain).
The data on top of the previous layers - the static information - is another component and includes information that does not change frequently such as activities, frequency of activities, reports on events at a certain place and time, etc. Static information includes historic data, such as how many robberies occurred in your neighborhood over the previous 5 years. /p>
The final set of data is dynamic geo data , which deals with things that are moving such as tracking your friends on Google Latitude. The military has Common Operating Picture displays that show the position of all their troops in a region - another example of dynamic data. As geospatial data management capabilities continue to evolve, the dynamic geo data space is growing considerably.
Why MongoDB for Geospatial?
As a document oriented storage system, MongoDB allows you to store complex data types. In contrast to distributed hash tables, key value stores, or relational databases that only permit storage of simple types, MongoDB DOES allow for the complex data types.
The real value for TST is the ability to support operations-focused customers. These folks, usually in our US Military or other Intelligence Community agencies, have geospatial data that number into the tens of millions of objects, including hyperspectral data and spatio-temporal data that changes rapidly. This volume of data requires the scale-out functionality that MongoDB provides.
MongoDB is easy to extend and configure from a developer's perspective, which leads to rapid development. And the technology is schema flexible- with a query language retaining some of the properties of SQL
Additionally, we have found the efficient memory management design that MongoDB provides complements our approach well. This enables the storage of the entire index directly within memory as opposed to extensive memory to disk swapping, which hampers write performance.
Outside of customer application development, how else does TST use MongoDB for geo?
TST has three primary capabilities that currently leverage MongoDB.
iSpatialis a geospatial framework that uses MongoDB as one of its indexes for data. As improvements to MongoDB's geospatial management features are contributed, TST will continuously enhance iSpatial to take advantage of those features.
iHarvest is a tool that uses MongoDB to monitor hundreds of millions of events on a network. Those events could be carried out by system users, sensors on network, or hardware devices on network. iHarvest creates an individual model for each actor and can then conduct a variety of analysis that helps protect against insider threat, promotes collaboration among individuals with common interests, provides organization heads a roll-up of what their organization is working on, and alerts the users within an organization to new data that they might be interested in reviewing.
Ubiquity is TST's mobile software framework that leverages MongoDB through iSpatial. This software enables every day users to drag and drop widgets into a mobile application container to create their own unique mobile apps.
TST has extended MongoDB core codebase with an initial update to allow an abstraction of the current indexing that now supports different indexing structures. This allows a variety of additional spatial capabilities.
Specifically, the TST extensions now allow for MongoDB to support 3D and 4D searching on geospatial data. TST plans to continue contributing to the MongoDB core around spatial applications and will be working on indexing hyperspectral data and enhancing the geo sharding capabilities. TST will also be working internally to rotate our development staff through our open source contribution team in our Applied Sciences Group, so that a significant percentage of our top developer are smart on MongoDB and able to contribute updates to the core software.
To learn more about TST, visit t-sciences.com
Also, check out Nicholas Knize presentation from mongoDC on Geospatial Indexing and MongoDB.