MongoDB customers and community members are the people who realize GIANT ideas. We are excited to begin highlighting some of our community members, our MongoDB Giants, who are tackling challenging problems and bringing solutions to life with MongoDB.
March’s Giant of the Month is Mike Grayson, Senior MongoDB DBA at PayChex, a provider of payroll, human resource, and benefits outsourcing solutions for small to medium-sized businesses. Mike has been involved in many aspects of the MongoDB community since he started using the database in 2014. He received numerous internal awards from PayChex for his dedication to operationalizing their new system and educating his teams on the new database technology. In his own words, “with lots of help from Ops Manager, integrating MongoDB in to our ecosystem has been a great and painless process.”
Mike was also instrumental in the MongoDB 3.2 release and participated in Beta Testing for The Encrypted Storage Engine and MongoDB Compass. Paychex is a member of MongoDB’s customer advisory board and through their involvement Michael provides input into MongoDB’s product development.
In his spare time, if he’s not spending his time on the Advocacy Hub or reading about MongoDB and other databases, you can find Mike playing video games, rooting for Bayern Munich, or any of his favorite Philadelphia sports teams with his wife, three girls, and two dogs in the beautiful Finger Lakes Region of New York.
Have your voice heard in the MongoDB community. Join our Advocacy Hub and start getting involved today.
Using MongoDB, Kafka and Spark to Build Infrastructure for India’s First Affordable Smart-Homes Project
By Gautam Rege, Co-Founder of Josh Software and Co-Founder of SimplySmart Solutions In Sheltrex , a growing community about two hours outside of Mumbai, India, we’re part of a project that will put more than 100,000 people in affordable smart homes. To make those homes truly smart we’re building infrastructure that streams data from millions of sensors in near real-time. Citizens can then access the data through a mobile application that allows them to better manage their home. It’s a fantastic example of how technology can improve our lives, but building scalable and fast infrastructure is not simple. In this blog, I want to highlight how my team at Josh Software , one of India’s leading internet of things and web application specialists, is overcoming those challenges by using a stack of interesting data tools like Apache Kafka, Apache Spark and MongoDB . Of the planned 20,000 homes in Sheltrex, more than 1,500 have already been completed. Many people people are already living on site. The pilot is a proving ground for a whole host of smart township technologies. From mobile connected security to smart-meters monitoring power consumption. Along with the mobile application for individual citizens we’ve also built software that will aggregate this data for the entire community. This gives the township the ability to negotiate more competitive rates from India’s electricity providers. Sheltrex affordable home project in Karjat, India To provide homeowners and the community with accurate and timely utility data means processing information from millions of sensors quickly, then storing it in a robust and efficient way. The Smart City Application communicates with our stack APIs to make business sense for residents and the township management. The entire solution is split into two “universes.” Universe One is where we stream all the sensor data that is flooding in from the homes in real time. This could include data points like temperature or energy usage. The sensor and smart-meter data is first ingested into a messaging system powered by Kafka (an open source, high-throughput, distributed, publish-subscribe platform that can quickly process real-time data feeds at a large scale). Through Kafka the data is dropped into Spark , a large-scale data processing engine that is basically a much faster and simpler alternative to MapReduce. It’s in Spark, using Java and Python, that we do the processing and aggregation of the data - before it’s written on to our second “universe.” Universe Two is where the smart home data is stored and accessed by the mobile application. We need something fast, flexible and robust, so we turned to MongoDB. It is the primary database for all storage, analysis and archiving of the smart home data. This includes time-series data like regular temperature information, as well as enriched metadata such as accumulated electricity costs and usage rates. To connect the analytical and operational data sets we use the MongoDB Connector for Hadoop . We’ve found that the three technologies work well in harmony, creating a resilient, scalable and powerful big data pipeline, without the complexity inherent in other distributed streaming and database environments. Both in development, where it’s relatively simple to integrate them, and in production where the data flows smoothly between each stage. Smarter, faster I’ve been using MongoDB since the beginning, in fact, I’ve written a couple of books on the subject . It’s been great to see how the database itself has matured and kept adding the right features at the right time. Another big advantage for us is how much more productive MongoDB makes developers and operations staff. The devops team is continuously delivering code to support new requirements, so they need to make things happen fast. MongoDB’s ease of use means we can accelerate our development process and get new features integrated, tested and deployed quickly. Right now we’re operating across eight Amazon Web Services instances in the same zone. As the project expands and more citizens move into Sheltrex we expect to see huge growth. That’s why it’s been so important for us to leverage technologies that operate efficiently at scale. Sheltrex affordable home project in Karjat, India So far the pilot has been incredibly successful and we’re pleased with how our infrastructure is steadily increasing it’s capacity as thousands of new homes come online. But what we’re doing in Sheltrex is only the beginning. Housing is a volume game, as more people live in smart affordable homes the greater the effect will be for the community and the environment. I believe this type of affordable and intelligent housing should become standard across the world. Minor initial costs lead to massive efficiencies over the lifetime of the building. These are not simply monetary - consider the wasted water and electricity that we could save. To get there it will take political will and, of course, considerable funding, but from my point of view the technology is ready to go today. By building our giant idea on modern and mature technologies like MongoDB, we’re ready to change the world. About Josh Software & SimplySmart Driven by enthusiasm and passion, Josh is India’s leading company in building innovative web applications working exclusively in Ruby On Rails since 2007. The company thrives only on three basic needs - disruption, innovation, and learning. It builds products for customers who are able to fulfil at least two of these needs. Details are available at www.joshsoftware.com . Due to the diverse nature of building smart solutions for townships, Josh has incorporated another company called SimplySmart Solutions that builds and implements these solutions. As the name suggests SimplySmart Technologies relies on simple solutions for making things smarter. Details are available at www.simplysmart.tech . Who else runs on MongoDB? Find out: Who else uses MongoDB?
Revolutionizing Data Storage and Analytics with MongoDB Atlas on Google Cloud and HCL
Every organization requires data they can trust—and access—regardless of its format, size, or location. The rapid pace of change in technology and the shift towards cloud computing is revolutionizing how companies handle, govern and manage their data by freeing them from the heavy operational burden of on-premise deployments. Enterprises are looking for a centralized, cost-effective solution that allows them to scale their storage and analytics so they can ingest data and perform artificial intelligence (AI) and machine learning (ML) operations, ultimately expanding their marketing horizon. This blog post explores why companies should partner with MongoDB Atlas on Google Cloud to begin their data revolution journey, and how HCL Technologies can support customers looking to migrate. MongoDB Atlas as the distributed data platform MongoDB Atlas is the leading database-as-a-service on the market for three main reasons: Unparalleled developer experience - allows organizations to bring new features to market at a high velocity Horizontal scalability - supports hundreds of terabytes of data with sub-second queries Flexibility - stores data to meet various regulatory, operational, and high availability requirements. The versatility offered by MongoDB’s document model makes it ideal for modern data-driven use cases that require support for structured, semi-structured, and unstructured content all within a single platform. Its flexible schema allows changes to support new application features without costly schema migrations typically required with relational databases. MongoDB Atlas extends the core database by offering services like Atlas Search and MongoDB Realm that are a necessity for modern applications. Atlas Search provides a powerful Apache Lucene-based full text search engine that automatically indexes data in your MongoDB database without the need for a separate dedicated search engine or error-prone replication processes. Realm provides edge-to-cloud sync and backend services to accelerate and simplify mobile and web development. Atlas’ distributed architecture supports horizontal scaling for data volume, query latency, and query throughput which offers the scalability benefits of distributed data storage alongside the rich functionality of a fully-featured general purpose database. MongoDB Atlas is unique in its ability to provide the most wanted database as a managed service and is relied on by the world’s largest companies for their mission-critical production applications. Innovation powered by collaboration with HCL Technologies MongoDB’s versatility as a general-purpose database, in addition to its massive scalability, makes it a perfect foundation for analytics, visualization, and AI/ML applications on Google Cloud. As an MSP partner for Google Cloud, HCL Technologies helps enterprises accelerate and risk-mitigate their digital agenda, powered by Google Cloud. We’ve successfully implemented applications leveraging MongoDB Atlas on Google Cloud, building upon MongoDB’s flexible JSON-like data model, rich querying and indexing, and elastic scalability in conjunction with Google Cloud’s class-leading cloud infrastructure, data analytics, and machine learning capabilities. HCL is working with some of the world’s largest enterprises in building secure, performant, and cost-effective solutions with MongoDB and Google. Possessing technical expertise in Google Cloud, MongoDB, machine learning, and data science, our dedicated team developed a reference architecture that ensures high performance and scalability. This is simplified by MongoDB Atlas’ support for Google Cloud services which allows it to essentially operate as a cloud-native solution. Highlighted features include: Integration with Google Cloud Key Management Service Use of Google Cloud’s native storage snapshot for fast backup and restore Ability to create read-only MongoDB nodes in Google Cloud to reduce latency with Google Cloud-native services regardless of where the primary node is located (even other public cloud providers!) Integrated billing with Google Cloud Ability to span a single MongoDB cluster across Google Cloud regions worldwide, and more As represented in Figure 1 below, MongoDB Atlas on Google Cloud can be used as a single database solution for transactional, operational, and analytical workloads across a variety of use cases. Figure 1: MongoDB's core characteristics and features The following architecture in Figure 2 demonstrates the ease of reading and writing data to MongoDB from Google Cloud services. Dataflow, Cloud Data Fusion, and Dataproc can be leveraged to build data pipelines to migrate data from heterogeneous databases to MongoDB and to feed data to create interactive dashboards using Looker. These data pipelines support both batch and real-time ingestion workloads and can be automated and orchestrated using Google Cloud - native services.. Figure 2: MongoDB Atlas' integration with core Google Cloud services A data platform built using MongoDB Atlas and Google Cloud offers an integrated suite of services for storage, analysis, and visualization. Address your business challenges with HCL: Industry use cases Data-driven solutions built with MongoDB Atlas on Google Cloud have multiple applications across industries such as financial services, media and entertainment, healthcare, oil and gas, energy, manufacturing, retail, and the public sector. Every industry can benefit from this highly integrated storage and analytical solution. Use Cases and Benefits Data lake modernization with low cost and high availability for media and entertainment customers: Maintaining high availability and a low-cost data lake is an obstacle for any online entertainment platform that builds mobile or web ticketing applications. However, building on Google App Engine with MongoDB Atlas Clusters in the backend allows for a high-availability, low-cost data platform that seamlessly feeds data to downstream analytics platforms in real time. Unified data platform for retail customers: The retail business frequently requests an agile environment in order to encourage innovation among its engineers. With its agility in scaling and resource management, seamless multi-region clusters, and premium monitoring, running MongoDB Atlas on Google Cloud is a fantastic choice for building a single data platform. This simplifies the management of different data platforms and allows developers to focus on new ideas. High-speed real-time data platform of supply chain system for manufacturing units: By having real-time visibility and distributed data services, supply chain data can become a competitive advantage. MongoDB Atlas on Google Cloud provides a solid foundation for creating distributed data services with a unified, easy-to-maintain architecture. The unrivaled speed of MongoDB Atlas simplifies supply chain operations with real-time data analytics. The way forward Even in just the past decade, organizations have been forced to adapt to the extremely fast pace of innovation in the data analytics landscape: moving from batch to real-time, on-premise to cloud, gigabytes to petabytes, and the increased accessibility of advanced AI/ML models thanks to providers like Google Cloud. With our track record of success in this domain, HCL Technologies is uniquely positioned to help organizations realize the joint benefits of building data analytics applications with best-of-breed solutions from Google Cloud and MongoDB. Visit us to learn more about the HCL Google Ecosystem Business Unit and how we can help you harness the power of MongoDB Atlas and Google Cloud Platform to change the way you store and analyze your data through these solutions.