MongoDB customers and community members are the people who realize GIANT ideas. We are excited to begin highlighting some of our community members, our MongoDB Giants, who are tackling challenging problems and bringing solutions to life with MongoDB.
This month’s MongoDB Giant is Doug Duncan, a DBA at Alteryx, who is making a meaningful impact to the MongoDB community. Alteryx is a leader in data preparation, blending, and advanced analytics.
Doug has worked with RDBMSs for longer than he cares to admit, focusing in both development and administration. Through his experience, Doug grew to embrace new data storage technologies. He began working with MongoDB several years ago (starting with the MongoDB 1.8 release in early 2011) both professionally and recreationally. Doug acted as an online TA the first two years following MongoDB University’s founding, working closely with the team by answering students’ questions about the M101J, M101JS and M202 courses, as well as providing questions used in the courses. For his contributions to the education team Doug was awarded the first ever MongoDB DBA certification back in November of 2013.
In his spare time, if Doug’s not reading up on MongoDB, Hadoop, or other distributed data stores, you can find him walking around the foothills of Colorado with his wife, two boys and two dogs.
Become an important part of the MongoDB community. Join our Advocacy Hub and start getting involved today.
Leaf in the Wild: SilkRoute Chooses MongoDB Over SQL Server for Critical Quality Assurance Platform
Leaf in the Wild posts highlight real world MongoDB deployments. Read other stories about how companies are using MongoDB for their mission-critical projects. MongoDB chosen for development productivity, operational efficiency with Cloud Manager, and “truly outstanding” professional services From manufacturing to retail, every part of the supply chain is starting to see the value of data. Whether it’s developing IoT quality assurance applications in manufacturing to ensure your products are defect-free or building data-driven customer loyalty programs so that brands can connect with and reward their fans, the top companies are working to improve their approach to data. SilkRoute Global is a software-as-a-service company focused on this industry. Its analytics products automate processes and present consumable, useful information to its customers. To understand the benefits they get from MongoDB, I spoke with Devin Duden, CTO of OmniSky (a division of SilkRoute) & Senior Software Engineer, and Amjad Hussain, CEO & Chief Data Scientist. Tell us a little bit about SilkRoute. SilkRoute is a passionate team of designers, machine learning scientists, and software engineers with tremendous industry knowledge of manufacturing, distribution, and retail. We live for solving big problems. Our industry-specific predictive and prescriptive analytics platform creates immense operational and strategic value for our customers. Our customer footprint is global and growing. Applied machine learning, business process automation, and mobility are woven into the fabric of everything we build. We offer a unique risk-free rapid implementation and integration approach for our customers to enjoy our solutions. Please describe how you’re using MongoDB. The application SilkRoute is building is a mobile application performing RFID inspections on industrial manufactured products. The application provides a centralized data store of customers’ products and the inspections associated with a product, and allows those customers to easily share the inspection records with others. MongoDB was chosen for this application based on: Simplified schema design Increased flexibility for modeling complex relationships (e.g., using MongoDB eliminated recursive relationships necessary in a SQL-based solution) Easier capture of user generated data Reduced development timeline Durability, scalability, and disaster recovery SilkRoute Enterprise mobile RFID inspection architecture What were you using before MongoDB? Was this a new project or did you migrate from a different database? The current version of the application is a client-server implementation using SQL Server as a cloud sharing data store and Windows CE on the mobile device. The application is a rewrite. How did you hear about MongoDB? Did you consider other alternatives, like relational or NoSQL databases? I was introduced to MongoDB three years ago when I started working at SilkRoute. We were working on a social network at the time, which was using MongoDB as its primary data store. The RFID mobile application’s technical requirements were originally to use MS SQL Server. This technical requirement was provided by the client. During our working Joint Application Design session with the client, we suggested using MongoDB, but didn’t make headway. When we attended MongoDB World 2015 , we gathered enough details about MongoDB’s capabilities, along with real-world examples of high-volume, transaction-based applications being developed on MongoDB, that we were able to persuade the client to switch from SQL Server to MongoDB. Please describe your MongoDB deployment, technology stack, and the version of MongoDB that you are running. The MongoDB deployment is a 5 node replica set using Cloud Manager for operational management and deployment. The replica set is deployed in the US East AWS region across all availability zones. At this point, we have not implemented sharding. The MongoDB replica set has been deployed in AWS following MongoDB’s best practices using Amazon Linux AMIs. Each production node will be running on EC2 instances with 16 GB memory and 4 core CPUs, with three 100GB provisioned IOPS EBS volumes. Each volume is XFS format. One volume is mapped for “data”, one volume is mapped for “log”, and one volume is mapped for “journal”. The API stack is written in .NET 5 using C# MVC/Web API framework. We are using the MongoDB .NET driver version 2.0. Are you using any tools to monitor, manage and backup your MongoDB deployment? If so what? Do you use Ops Manager / Cloud Manager? The replica set has been deployed and managed using Cloud Manager. Cloud Manager simplified and streamlined replica set deployment and operations. This solution is the first time the majority of team members used MongoDB. To reduce time spent with MongoDB replica set deployment and configuration, Cloud Manager was a great fit. Following Cloud Manager’s directions to create AWS EC2 instances made it very easy for us to create images, and build/tear down replica sets quickly. Streamlining manual tasks allowed the team to focus more time on development than deploying a fully managed MongoDB replica set. In addition to Cloud Manager, the team just started using MongoDB Compass to analyze collections and document sizes. Are you integrating MongoDB with other data analytics, BI or visualization tools? If so, can you share any details? At this point we have not integrated any BI. One of our objectives is to connect with the client’s BI system using the MongoDB Connector for BI and/or extract data from a tagged node to hydrate a SQL-based BI system. We’re planning to perform a POC on the Connector for BI, now that it has been released. How are you measuring the impact of MongoDB on your business? SilkRoute measures MongoDB’s impact by many factors, including ease of use with deployments, a code first approach, increased agile development model, reduced total cost of ownership, and reduced time to market. The ease of deployments reduces or eliminates maintenance windows when spinning up a replica set or upgrading database versions, which means higher uptime for customers and less productive time eaten up for developers. A code first approach adds to increased savings by eliminating daunting DDL script management and aids with better agile development. These factors result in reduced total cost of ownership and faster time to market. Do you use any commercial subscriptions or services to support your MongoDB deployment? SilkRoute is a MongoDB OEM partner. For the RFID application we will be embedding MongoDB Enterprise Server 3.2 and managing the deployment with Cloud Manager. We allocated a budget for MongoDB’s professional services in the early stages of the project. The professional services were tailored to the team’s skill set and agenda. With two separate onsite sessions, we covered topics from deployment, management, and recovery using Cloud Manager, to schema modeling and scaling. The value gained working hands-on directly with a MongoDB consulting engineer was twice the investment. During one session, we encountered a disaster recovery situation in a non-production environment. Unexpected though the situation was, I personally gained the most from the experience of working through the issue with a MongoDB expert in a very collaborative fashion. The professionalism and knowledge of our MongoDB consulting engineer was truly outstanding. Do you have plans to use MongoDB for other applications? If so, which ones? Yes, both internal initiatives and client initiatives. These include BI, a Warehouse Manager SaaS solution, a customer loyalty/couponing app, and client SaaS solutions, which we are not at liberty to disclose at this point. We would prefer to use MongoDB for all application and system development projects. Our preference to use MongoDB for development is based on ease of use, an emphasis on a code first approach for projects going forward, and built-in scalability and durability. Have you upgraded to MongoDB 3.2? What most excites you about this release? We’ve been developing the solution using MongoDB 3.0.x. We are actively migrating the database to version 3.2.1, and the production deployment will use 3.2.1. The most exciting features of MongoDB 3.2 for us are the BI connector, document validation, $lookup, and WiredTiger's in-memory option. We feel the biggest value add to our clients are the BI connector and the in-memory storage engine. The BI connector will allow our clients’ BI environments to integrate directly with the solution we are building, eliminating the need to write ETL processes from MongoDB to a BI environment. The in-memory storage engine will increase performance with read operations, which will reduce latency with API requests. Anything to increase overall performance is a plus. What advice would you give someone who is considering using MongoDB for their next project? I would highly recommend allocating a budget for MongoDB’s professional services to help with operations, deployment, and schema modeling. The value gained with their best practices approach really reduces learning curves and POC time. Coming from a SQL world, prepare ERDs and break the ERDs into schema designs. This approach will help bridge team members from a relational to a non-relational data store. Take a top-down development approach as it will uncover access patterns that may help with schema modeling. Thank you for sharing your MongoDB experiences with us! If you’re comparing MongoDB with relational databases, read our RDBMS to MongoDB Migration Guide to learn more. Read the RDBMS to MongoDB Migration Guide About the Author - Eric Holzhauer Eric is a Product Marketing Manager at MongoDB.
The Rise of the Strategic Developer
The work of developers is sometimes seen as tactical in nature. In other words, developers are not often asked to produce strategy. Rather, they are expected to execute against strategy, manifesting digital experiences that are defined by the “business.” But that is changing. With the automation of many time-consuming tasks -- from database administration to coding itself -- developers are now able to spend more time on higher value work, like understanding market needs or identifying strategic problems to solve. And just as the value of their work increases, so too does the value of their opinions. As a result, many developers are evolving, from coders with their heads-down in the corporate trenches to highly strategic visionaries of the digital experiences that define brands. “I think the very definition of ‘developer’ is expanding,” says Stephen “Stennie” Steneker, an engineering manager on the Developer Relations team at MongoDB. “It’s not just programmers anymore. It’s anyone who builds something.” Stennie notes that the learning curve needed to build something is flattening. Fast. He points to an emerging category of low code tools like Zapier, which allows people to stitch web apps together without having to write scripts or set up APIs. “People with no formal software engineering experience can build complex automated workflows to solve business problems. That’s a strategic developer.” Many other traditional developer tasks are being automated as well. At MongoDB, for example, we pride ourselves on removing the most time-consuming, low-value work of database administration. And of course, services like GitHub Copilot are automating the act of coding itself. So what does this all mean for developers? A few things: First, move to higher ground. In describing one of the potential outcomes of GitHub Copilot, Microsoft CTO Kevin Scott said, ““It may very well be one of those things that makes programming itself more approachable.” When the barriers to entry for a particular line of work start falling, standing still is not an option. It’s time to up your strategic game by offering insight and suggestions on new digital experiences that advance the objectives of the business. Second, accept more responsibility. A strategic developer is someone who can conceive, articulate, and execute an idea. That also means you are accountable for the success or failure of that idea. And as Stennie reminded me, “There are more ways than ever before to measure the success of a developer’s work.” And third, never stop skilling. Developers with narrow or limited skill sets will never add strategic value, and they will always be vulnerable to replacement. Like software itself, developers need to constantly evolve and improve, expanding both hard and soft skills. How do you see the role of the developer evolving? Any advice for those that aspire to more strategic roles within their organizations? Reach out and let me know what you think at @MarkLovesTech .