November 26, 2013
In HR and Accounting, keeping track of employee or transactional information is a complex task. Numeric data is frequently considered the domain of the relational database, but attributes like addresses or skills are more difficult to serialize. With data volume growing alongside reliance on complex data structures to improve operational insights, the relational model increasingly falls short. Richard Minney is the co-founder of iBE.net, an Enterprise Resource Planning and Professional Services Automation (ERP/PSA) system. He turned to MongoDB to address this challenge and model his company’s complex data.
iBE.net provides end-to-end applications to help smaller consulting firms and IT service providers run their business in the cloud. Their customers fill the sweet spot between those who are too large to run on Freshbooks or QuickBooks, but can’t afford NetSuite or Microsoft. iBE.net helps these firms manage their projects, clients, leads, employees, invoices, expenses, and more.
iBE.net needed to model the complex and rapidly changing data required for an end-to-end enterprise system, without thousands of SQL tables. iBE.net already has about 300+ MongoDB collections. Several of these have documents embedded within them, containing widely varying information.
“We chose MongoDB for three reasons” says Minney:
MongoDB’s dynamic schema and query power offer significant data management improvements over a relational database, while replication and sharding capabilities provide high availability. Minney was lucky, he says, because he began designing his application in 2011 and was able to use MongoDB from the start. He decided a document database would aptly suit his needs due to his application’s complex and dynamic data structures.
Minney notes “MongoDB helps us because we can create objects in each collection.” Employee data, which might have been stored in many SQL tables, can be modeled into one collection with several nested structures. Employees have “names, emails, phone numbers, emergency contacts, benefits, skills, training, id cards, health data, prior work experience, addresses, and certifications.” Aggregating this data properly is essential for forming a clear picture of each client and what they need.
iBE.net uses a single unsharded cluster with three replica sets to be expanded as it scales. It runs on several servers for redundancy, and maintains separate databases for each client (as well as common databases for operations data, configurations settings, and shared collaborations data). iBE.net uses MongoDB for “all master and settings data across the whole enterprise.” Its flexibility helps developers add their own customer-specific documents to larger collections without regenerating table schema, reducing down time for the site. Rather than having an account spread across many SQL tables, iBE.net can now store all relevant information into one collection. And to support multi-document updates, they implemented their own solution with custom two-phase commits.
Before switching to MongoDB, Minney and his team estimated that 25% of their development effort would be in server APIs and object relational mapping for SQL table. With MongoDB, this was reduced to 5% of the development effort. Query response is now at an average of 100-300 ms, compared to two second query responses using SQL. And the estimated cost for hosting and supporting iBE.net’s set-up was only $1000 a month, compared with the estimated cost of supporting a SQL database with equivalent capacity for about $2500 a month.
Minney’s developers were able to design their application to ingest highly variable, dynamic data from users, creating documents in each collection with deep nested structures. And though iBE.net’s entire technical team came from a relational background, they had no trouble picking up MongoDB. They were able to learn from a number of resources, including support from MongoDB core contributors and informative experiences at MongoDB Days.
MongoDB has helped iBE.net organize and understand its data in a way that would have been impossible using a relational database. Dynamic schema allowed developers to design their applications to serve their business needs. Collections easily connect individual employees and their data (stored as documents) into a comprehensive picture. And the ability to shard will be particularly important as iBE.net continues to grow, allowing it to expand to meet new data processing demands.