All Blog Posts

How Accenture’s Data Modernizer Tool Accelerates the Modernizing of Legacy and Mainframe Apps to MongoDB

As companies scale their MongoDB estates and apply them to more and more use cases, opportunities for deeper insights arise—but first come issues around data modeling. The Challenge It’s one thing to start modeling for a green field application where you have the ability to apply modeling patterns . But it’s a different matter entirely to start from an existing relational estate and take into account multiple data sources and their business requirements in an effort to create the ideal MongoDB data model based on explicit schema (table and relations) as well as implicit schema (access patterns and queries). Unfortunately, this is a fairly common obstacle. Perhaps an application has been around for a while and documentation hasn’t kept up with changes. Or maybe a company has lost talent and expertise in some areas of a monolithic app. These are just a couple of examples that demonstrate how time consuming and error-prone this effort can be. The Solution MongoDB and Accenture are excited to announce a modernization tool which enables faster MongoDB adoption by accelerating data modeling. As part of its mainframe offload and cloud modernization initiatives, Accenture’s Java-based data modernizer helps companies arrive at a recommended MongoDB document data model more quickly and efficiently. Using this tool results in a faster operational data layer for offloading mainframes and faster implementation of MongoDB Atlas for migrating applications to the cloud while also transforming them. Accenture and MongoDB are long-standing strategic partners, and this investment is another addition to our joint customers’ toolkits. “This modernizer is the right tool for teams who are focused on changing business requirements as they rethink their applications for the cloud. With on average 50% faster data modeling and 80% data model accuracy straight out of the tool, developers can work on modernizing instead of spending weeks figuring out the right data model to simply meet existing business requirements that have not been explicit.” (Francesco De Cassai) How it Works The modernizer analyzes access logs and relational schema from Oracle and DB2 databases, whether they’re powered by mainframes or not. This allows teams to obtain critical insights before kicking off development activities, thereby better informing the process. Additionally, the modernizer combines the implementation of modeling patterns , development best practices such as document sizing (best, average, worst case), and naming conventions, while also providing confidence levels to predict the accuracy of the model it’s suggested. The tool combines Accenture’s deep expertise and lessons learned from numerous successful projects. Our customers are investing in a new approach to managing data: Data as a Service & Data Decoupling. This strategic initiative focuses on consolidating and organizing enterprise data in one place, most often on the cloud, and then making it available to serve digital projects across the enterprise. MongoDB Atlas unlocks data from legacy systems to drive new applications and digital systems, without the need to disrupt existing backends as companies modernize and migrate to the cloud. With this tool and MongoDB capabilities, MongoDB and Accenture allow organizations to get the most out of their data. Find out more about the MongoDB & Accenture partnership here . Learn more about MongoDB’s Modernization Initiative .

January 21, 2021

How MongoDB’s Technical Services Team Solves Customers’ Complex Problems

I sat down with Blake Deakin , Area Vice President for Technical Services, to get a deeper understanding of the complex and unique customer problems his team solves every day. Here, we explore how the Technical Services team has grown, the challenges they tackle, and what skills make someone successful in this role. Ashley Perez: As the Area Vice President of Technical Services, can you share insight about your team? Blake Deakin: Although our Technical Service team is a global operation, I specifically oversee the Technical Services team for the Americas. This covers the United States, Canada, and our new office in Argentina. Technical Services has been around for more than nine years now. Ultimately, the reason for Technical Services is simple: to give our customers access to “on demand” subject matter expertise to clear blockers and advise on best practices. This makes it easier for customers to fearlessly build important parts of their business on MongoDB, whether it’s a net-new application, feature expansion, or the replatforming of an existing system. We have the flexibility and situational awareness to help our customers rapidly adapt to their changing needs. AP: How quickly has the team grown since you’ve been here, and what’s the culture like? BD: I’ve been here for almost 3 years, during which time the team has basically doubled in size. The people on the team are varied, ranging from those in early stages of their careers to individuals who have worked 20 or 30 years in software in a variety of roles. Some are even former founders of companies — typically CTOs. Our employee retention is unusually high, so there are different tenured Engineers working together, passing along successive knowledge from different “epochs.” Our Engineers continue to grow each other’s skills, building on an extremely strong nucleus of engineering talent. The team is collaborative by necessity. The overall technology landscape is growing in complexity, as is our product portfolio. The result is that there is a vast body of knowledge we need to make available when working with our customers, so accessing the right knowledge within our organization at the right time is critical. Our other defining characteristic is our commitment to technical excellence. When you have customers who are often solving truly novel, world-scale problems, it’s crucial to provide them with the correct answer quickly so they can continue their work unimpeded. The default operating environment of our customers is often one of tight deadlines, high-velocity change, and competing priorities. We seek to help our customers feel confident that MongoDB products are a reliable and indispensable component of their tech stack that helps them adapt and exploit opportunity. COVID-19 obviously has created some unforeseen complexity in terms of how we operate as a team. Interestingly, our team didn’t slow down because of the shift to going fully remote during lockdowns. This situation revealed how well we can work this way even if we’re not face-to-face, at least in the short term. AP: Is it a challenge to keep this consistent team culture despite being scattered across multiple countries? BD: We’re quite lucky in that our core work requires global collaboration. It’s common for a customer issue to “travel around the world,” with engineers across geographies each owning a piece of resolving a customer’s issue. Everyone works together by default and has high expectations of one another, which creates a virtuous cycle that sustains and reinforces how the team operates. Everyone across the globe speaks the same language in terms of how we help make our customers successful. Our team members actually did a fair amount of jet-setting prior to the pandemic to help build a cohesive and collaborative team. We have a significant amount of spiritual adjacency to the Product Development organization and have participated in the engineering offsite over the years, which was an opportunity for the entire Americas team to get together and bond. These events are multiday offsites during which the product roadmap is discussed, there are workshops for acquiring new skills, and there is a lot of opportunity for social interaction. Aside from the offsite, we often hosted regional summits on a specific technology interest that Engineers from our separate teams would travel to. This was especially useful for us to get a handle on up-and-coming technologies, such as Kubernetes. I feel lucky that the team has the initiative and autonomy to do things like this. I think it’s emblematic of how Engineers at MongoDB have the freedom to create and pursue their interests. AP: You mentioned the team deals with a lot of different problems. Can you share some examples? BD: With all the interesting problems we’re constantly faced with, it’s hard to pick. However, during COVID-19, there have been some extremely urgent customer needs we’ve helped address. For example, a video chat app we support basically went to #1 on the European mobile app store charts overnight and ran into a bunch of immediate challenges with lockups and crashing. With the app having gone from 70K concurrent users on average to 1.7 million over the course of a month, that kind of rapid scale put a tremendous amount of pressure on the system, and many technologies simply couldn’t keep up. Even for us, it was a challenge to figure out a non-disruptive approach for scaling up. But this is actually the kind of thing at which we excel: calmly working in high-pressure environments and helping rescue customers from problems they couldn’t predict. Trends such as these are fickle. If this customer had failed the scale-out, its users would have moved on to another platform. Another great example was Sanoma Learning . We actually made a video about it. I won’t spoil the story, but this one was particularly great to share with friends and family. I feel as if a lot of us in tech struggle to explain what exactly we do when talking to the important people in our lives, so stories like this make it real for them. AP: With such a range of customers and problems, what skills are important for team members to have? BD: First and foremost, we need impressive intellectual and experiential horsepower on the team. We’re dealing with applications that have huge numbers of concurrent users, large transactional volumes, and strict latency requirements so users have a responsive experience. To make systems run like that at a global scale, you need people who understand complex problems and who can work comfortably across the tech stack. Not everyone knows everything, but it’s typical for people on the team to bring deep experience in areas such as networking, storage, development patterns, drivers, operating systems, distributed systems, security, and so on. The breadth of knowledge is large, but the operating environment is arguably more difficult; our Engineers often are solving problems in high-stakes situations with time sensitivity and typically reputational or revenue consequences for failure. We need to adopt many different tactics and approaches to drive customer success. We work with everyone, from household name brands to the next big startup, which drives a significant amount of variation in how we engage. Customers often have different goals, expectations, and tolerance for risk. One thing that keeps our job interesting is that although many customers encounter similar issues, those issues rarely present in the same way. A big part of the diagnostic art is figuring out how to come up with a strategy that rules in or rules out causes in the most effective and efficient manner while maintaining trust with the customer that you’re driving their issue to closure quickly and methodically. AP: With the retention of your team being so high, how can someone grow their career at MongoDB? BD: Technical Services provides a ton of career transformation and growth opportunities, whether someone remains with our team for a long tenure (and many do; our average tenure hovers around five years, and a large number of founding members are still with the company) or takes the skills they gain working with us to go on to other things. The type of work we do gives people a crash course in the marketplace’s most important technologies, so our people are extremely well positioned for whatever they decide to do in the future. AP: What skills or tools are team members given to help them transform their careers? BD: We provide everyone in the organization with access to a technical learning platform that includes recorded videos and O’Reilly books. The library is extremely extensive, and it’s one of the preferred ways for people to augment their skills. Our Leadership & Development team also is regularly adding to its overall portfolio of training, which is available on a self-paced learning platform that the learners can manage. The team has an aggressive delivery lifecycle, pushing out lots of valuable foundational learning. Additionally, we build Engineer knowledge by encouraging ongoing cross-training within the team, providing opportunities for people to do “lightning talks” or “deep dives” on topics of interest. We also budget for paid training provided by third parties across several subject areas, from basic professional development to technical skill areas to leadership, as well as stipends to attend technical conferences that offer professional development tracks. AP: MongoDB products help our customers innovate faster, but how does the team innovate internally? BD: We have extremely rapid product delivery lifecycles as a company, so there’s always something new to learn. A crucial part of how we get the job done every day is by developing tools and automations that make diagnosing customer issues easier — everything from visualization tools that help us understand and reason about the vast amount of telemetry we have about our customer environments (which help drive issue identification and resolution) to automated pipelines that produce candidate diagnoses before an Engineer ever looks at the customer’s issue. While it isn’t necessary for everyone on the team to have experience building tools like these, it’s definitely helpful and one of the opportunities we provide our Engineers to keep their development skills sharp. AP: In closing, can you share why someone would be excited to join the Technical Services team? BD: There are two main reasons. One is the opportunity to solve really big, really interesting problems for our customers. All companies are becoming software companies, and there’s a good chance you’ll work on something, see it in the news, and then say, “Hey! I helped make that happen .” For me, that’s one of the most gratifying things about working here. The other is that we’re an organization that celebrates continual skills growth. Everyone is constantly learning, and we have some of the brightest engineering minds working within Technical Services, which means plenty of opportunity for you to learn too. Interested in pursuing a career at MongoDB? We have several open roles on our teams across the globe, and would love for you to build your career with us!

January 21, 2021

Built with MongoDB: Spectro Cloud

Recently named one of the hottest Kubernetes startups , Spectro Cloud has been making waves across the tech ecosystem. An enterprise cloud-native infrastructure founded by three startup veterans, Spectro Cloud makes Kubernetes manageable at scale for enterprises that need manageability and flexibility. Spectro Cloud’s cluster profiles automate cluster deployment and maintenance across the enterprise and help operations prioritize the needs of the applications teams and simplify infrastructure administration. The company has raised $7.5 million in seed funding and has 36 team members. In this edition of #BuiltWithMongoDB, we talk to CTO/Co-founder Saad Malik and Vice President of Product Tina Nolte about Spectro Cloud’s deeply technical team and why they #BuiltWithMongoDB. Siya Raj Purohit: What makes you want to work at Spectro Cloud? Tina Nolte: Most of the team comes from Cisco, where the founders previously worked, so we already had good rapport and were truly friends. We believe that culture is something you build from Day 1, and once it exists, it’s hard to change. For that reason, we have a strict no-jerks policy. How that plays out is that we provide a lot of autonomy to the team and help support goals that individuals have. For example, we encourage everyone on the team to write blog content about things they are interested in to share their knowledge and build their personal brands. It’s something that our engineering managers even nudge junior engineers about: “So you haven’t shared any wisdom with the world recently — why don’t you write a post?” If you—as a developer—experience some kind of issue, or it took you time to understand something, it’s likely that someone else on the planet has experienced that issue too. So we encourage our engineers to help them out, and it’s good for our engineers to have an external presence, too. See Spectro Cloud’s post: Kustomize your way to MongoDB ReplicaSet Finally, we talk about Spectro Cloud as a family. We’re pretty confident that other people have our backs whether it’s personally or professionally, and that kind of connectivity is pretty special. SRP: How did you decide to start using MongoDB? Saad Malik: Our application is not very data heavy in terms of transactions or relations; it’s very document based. Although we are running a SaaS platform, we didn’t want to be in the business of doing backups, managing policy, and storing configurations. We wanted to use a platform that managed these things for us. So we were looking at Amazon’s DocumentDB or MongoDB Atlas. We realized we would have to use an on-premises version of our platform, so obviously if we were to be running an on-premises version of MongoDB, it would make sense to use Atlas. Our team also had experience with MongoDB so it was a clear choice. And it’s been fantastic — we have been very happy with the performance, and so far, it’s been scaling very well for us. SRP: What has it been like to scale with MongoDB? SM: It’s been fantastic — we haven’t had any outages with Atlas. We obviously have our notifications configured so if there are any outages or things going wrong with the MongoDB clusters, we can catch when one of our clusters is misbehaving. That visibility and getting the monitoring upfront is very helpful to us, because we’re able to figure out which of our application issues is causing the problem. When we were getting started, we had a technical advisor from MongoDB provide us with a one-day seminar on best practices for utilizing the platform and how to optimize queries. From then on, the online documentation has been sufficient for us to problem solve and scale. SRP: What does the database infrastructure look like today? SM: On the database side, we have three different environments: dev integration, stage, and production. All three of them run an Atlas version from a database that is completely separate. The stack that sits on top of it is Kubernetes application, all using Golang, DB drivers for MongoDB, and accessing the application on there. SRP: What advice do you have for developers who aspire to someday become CTO? SM: The number one thing I look for is someone who is very curious. When I was an early-career engineer, my mentors would tell me to always be very curious and focused in terms of what you’re doing. Understand how things are working, not just at the library level, but keep on digging down until you understand not just how, but why something works a certain way. If you understand the nuances, you’ll be able to identify true game-changing opportunities. Building something cool with MongoDB? Check out our developer resources , and let us know if you want your startup to be featured in our #BuiltWithMongoDB series.

January 20, 2021

Add Interactivity to Your Embedded Analytics with Click Events

MongoDB Charts’ data visualizations can now become more interactive, so users and stakeholders can dive deeper into the insights they care more about. That’s possible with a new feature currently in beta with support for most Charts types: click events. A click event in the Charts embedding SDK is simply a notification that a user clicked on a chart. That click could be anything: They might have clicked on a bar in a bar chart, a chart’s legend, or even empty white space within the chart. Web developers can subscribe to these events and create different functionality depending on where the user clicked. Why might you want to enhance your app or embedded analytics workflow with click event data? Click-event data opens up a wide range of possibilities. Here are a couple of examples, inspired by various Charts users who’ve been telling us how they’d like to use click-event data. Open up another chart, based on a user clicking on data within a chart: A logistics company has a bar chart that shows pending orders at an aggregate level per region, and they want to see more detail on pending orders for a specific region. They create a click event handler in their application that opens up a new chart with pending orders per supplier, based on the region selected in the aggregate chart. Filtering the other charts on a dashboard when a series or data point on a single chart is clicked: A retail clothing company has a dashboard with various shopping cart information such as sales, orders processed, and returns, for their portfolio of products. The head of outerwear sales only wants to see data for the “outerwear” category of products, so they click on the “outerwear” series within a bar chart. The rest of the dashboard can adapt so that it shows only information relevant to outerwear. The example below is created from one of our sample data sets. We created two charts in a single app, tied to the sample movie data set that every Atlas user can access. On the left is a stacked bar chart with category level data that includes genre and decade. On the right is a table chart that shows each individual movie within a category. Clicking on a specific category in the bar chart updates the movies shown in the table chart. How can you get started with click events of embedded charts? If you haven’t yet used the embedding SDK for MongoDB Charts, you’ll want to familiarize yourself with the docs , consider watching this video tutorial , and access the SDK via the Charts GitHub repository . Regardless if you’re new to using the SDK or have experience with it, it’s important to note that you will need to use the @mongodb-js/charts-embed-dom@beta tagged version of the SDK to have access to the click events functionality while it’s in beta. There are two examples specifically for click events in the repository: click-events-basic and click-events-filtering . If you just want to explore and test out the functionality, you can play around with them in the following sandboxes using Click events basic sandbox Click events filtering sandbox Here’s a snapshot of the data surfaced in a click event that is available for developers to use in their apps. In this example I clicked on the yellow section of the top bar in the Movie Genres chart above. Note how it includes details of the click coordinates, the type and role of the clicked mark, and the corresponding chart data for the mark. { "chartId": "90a8fe84-dd27-4d53-a3fc-0e40392685dd", "event": { "type": "click", "altKey": false, "ctrlKey": false, "shiftKey": false, "metaKey": false, "offsetX": 383, "offsetY": 15, "clientX": 403, "clientY": 99, "pageX": 403, "pageY": 99, "screenX": 756, "screenY": 217 }, "data": { "y": { "label": "Genre", "value": "Drama" }, "x": { "label": "# Movies", "value": 3255 }, "color": { "label": "Decade", "value": "2010 - 2020", "lowerBound": 2010, "upperBound": 2020 } }, "target": { "type": "rect", "role": "mark", "fill": "#F0D175" }, "apiVersion": 1 } Whether you’re an avid user or new to MongoDB Charts, we hope you consider taking advantage of the new click event capability to increase the interactivity of Charts. It’s in beta because there is more functionality still to come. It has yet to be released for a few chart types: geospatial, table, top item, word cloud, and number charts. On that note, we’d love to hear your thoughts through the MongoDB Feedback Engine . If you haven’t tried Charts yet, you can get started for free by signing up for a MongoDB Atlas and deploying a free tier cluster.

January 20, 2021

Flowhub Relies on MongoDB to Meet Changing Regulations and Scale Its Business

The legal landscape for cannabis in the United States is in constant flux. Each year, new states and other jurisdictions legalize or decriminalize it, and the regulations governing how it can be sold and used change even more frequently. For companies in the industry, this affects not only how they do business, but also how they manage their data. Responding to regulatory changes requires speedy updates to software and a database that makes it easy to change the structure of your data as needed – and that’s not to mention the scaling needs of an industry that’s growing incredibly rapidly. Flowhub makes software for the cannabis industry, and the company is leaping these hurdles every day. I recently spoke with Brad Beeler , Lead Architect at Flowhub, about the company, the challenges of working in an industry with complex regulations, and why Flowhub chose MongoDB to power its business. We also discussed how consulting from MongoDB not only improved performance but also saved the company money, generating a return on investment in less than a month. Eric Holzhauer: First, can you tell us a bit about your company? Brad Beeler: Flowhub provides essential technology for cannabis dispensaries. Founded in 2015, Flowhub pioneered the first Metrc API integration to help dispensaries stay compliant. Today, over 1,000 dispensaries trust Flowhub's point of sale, inventory management, business intelligence, and mobile solutions to process $3B+ cannabis sales annually. Flowhub in use in a dispensary EH: How is Flowhub using MongoDB? BB: Essentially all of our applications – point of sale, inventory management, and more – are built on MongoDB, and we’ve been using MongoDB Atlas from the beginning. When I joined two and a half years ago, our main production cluster was on an M40 cluster tier, and we’ve now scaled up to an M80. The business has expanded a lot, both by onboarding new clients with more locations and by increasing sales in our client base. We’re now at $3 billion of customer transactions a year. As we went through that growth, we started by making optimizations at the database level prior to throwing more resources at it, and then went on to scale the cluster. One great thing about Atlas is that it gave us the metrics we needed to understand our growth. After we’d made some optimizations, we could look at CPU and memory utilization, check that there wasn’t a way to further improve query execution with indexes, and then know it was time to scale. It’s really important for usability that we keep latency low and that the application UI is responsive, and scaling in Atlas helps us ensure that performance. We also deploy an extra analytics node in Atlas, which is where we run queries for reporting. Most of our application data access is relatively straightforward CRUD, but we run aggregation pipelines to create reports: day-over-day sales, running financials, and so forth. Those reports are extra intensive at month-end or year-end, when our customers are looking back at the prior period to understand their business trends. It’s very useful to be able to isolate that workload from our core application queries. I’ll also say that MongoDB Compass has been an amazing tool for creating aggregation pipelines. EH: Can you tell us some more about what makes your industry unique, and why MongoDB is a good fit? BB: The regulatory landscape is a major factor. In the U.S., there’s a patchwork of regulation, and it continues to evolve – you may have seen that several new states legalized cannabis in the 2020 election cycle. States are still exploring how they want to regulate this industry, and as they discover what works and what doesn’t, they change the regulations fairly frequently. We have to adapt to those changing variables, and MongoDB facilitates that. We can change the application layer to account for new regulations, and there’s minimal maintenance to change the database layer to match. That makes our development cycles faster and speeds up our time to market. MongoDB’s flexibility is great for moving quickly to meet new data requirements. As a few concrete examples: The state of Oregon wanted to make sure that consumers knew exactly how much cannabis they were purchasing, regardless of format. Since some dispensaries sell prerolled products, they need to record the weight of the paper associated with those products. So now that’s a new data point we have to collect. We updated the application UI to add a form field where the dispensary can input the paper weight, and that data flows right into the database. Dispensaries are also issuing not only purchase receipts, but exit labels like what you’d find on a prescription from a pharmacy. And depending on the state, that exit label might include potency level, percentage of cannabinoids, what batch and package the cannabis came from, and so on. All of that is data we need to be storing, and potentially calculating or reformatting according to specific state requirements. Everything in our industry is tracked from seed to sale. Plants get barcodes very early on, and that identity is tracked all the way through different growth cycles and into packaging. So if there’s a recall, for example, it’s possible to identify all of the products from a specific plant, or plants from a certain origin. Tracking that data and integrating with systems up the supply chain is critical for us. That data is all tracked in a regulatory system. We integrate with Metrc , which is the largest cannabis tracking system in the country. So our systems feed back into Metrc, and we automate the process of reporting all the required information. That’s much easier than a manual alternative – for example, uploading spreadsheets to Metrc, which dispensaries would otherwise need to do. We also pull information down from Metrc. When a store receives a shipment, it will import the package records into our system, and we’ll store them as inventory and get the relevant information from the Metrc API. Flowhub user interface EH: What impact has MongoDB had on your business? BB: MongoDB definitely has improved our time to market in a couple of ways. I mentioned the differences of regulation and data requirements across states; MongoDB’s flexibility makes it easier to launch into a new state and collect the right data or make required calculations based on data. We also improve time to market because of developer productivity. Since we’re a JavaScript shop, JSON is second nature to our developers, and MongoDB’s document structure is very easy to understand and work with. EH: What version of MongoDB are you using? BB: We started out on 3.4, and have since upgraded to MongoDB 4.0. We’re preparing to upgrade to 4.2 to take advantage of some of the additional features in the database and in MongoDB Cloud. One thing we’re excited about is Atlas Search : by running a true search engine close to our data, we think we can get some pretty big performance improvements. Most of our infrastructure is built on Node.js, and we’re using the Node.js driver . A great thing about MongoDB’s replication and the driver is that if there’s a failover and a new primary is elected, the driver keeps chugging, staying connected to the replica sets and retrying reads and writes if needed. That’s prevented any downtime or connectivity issues for us. EH: How are you securing MongoDB? BB: Security is very important to us, and we rely on Atlas’s security controls to protect data. We’ve set up access controls so that our developers can work easily in the development environment, but there are only a few people who can access data in the production environment. IP access lists let us control who and what can access the database, including a few third-party applications that are integrated into Flowhub. We’re looking into implementing VPC Peering for our application connections, which currently go through the IP access list. We’re also interested in Client-Side Field-Level Encryption . We already limit the amount of personally identifiable information (PII) we collect and store, and we’re very careful about securing the PII we do need to store. Client-Side Field-Level Encryption would let us encrypt that at the client level, before it ever reaches the database. EH: You're running on Atlas, so what underlying cloud provider do your use? BB: We’re running everything on Google Cloud. We use Atlas on Google Cloud infrastructure, and our app servers are running in Google Kubernetes Engine. We also use several other Google services. We rely pretty heavily on Google Cloud Pub/Sub as a messaging backbone for an event-driven architecture. Our core applications initially were built with a fairly monolithic architecture, because it was the easiest approach to get going quickly. As we’ve grown, we’re moving more toward microservices. We’ve connected Pub/Sub to MongoDB Atlas, and we’re turning data operations into published events. Microservices can then subscribe to event topics and use the events to take action and maintain or audit local data stores. Our data science team uses Google BigQuery as the backend to most of our own analytics tooling. For most uses, we migrate data from MongoDB Atlas to BigQuery via in-house ETL processes, but for more real-time needs we’re using Google Dataflow to connect to MongoDB’s oplog and stream data into BigQuery. EH: As you grow your business and scale your MongoDB usage, what's been the most important resource for you? BB: MongoDB’s Flex Consulting has been great for optimizing performance and scaling efficiently. Flowhub has been around for a number of years, and as we’ve grown, our database has grown and evolved. Some of the schema, query, and index decisions that we had made years ago weren’t optimized for what we’re doing now, but we hadn’t revisited them comprehensively. Especially when we were scaling our cluster, we knew that we could make more improvements. Our MongoDB Consulting Engineer investigated our data structure and how we were accessing data, performance, what indexes we had, and so on. We even got into the internals of the WiredTiger storage engine and what optimizations we could make there. We learned a ton about MongoDB, and the Consulting Engineer also introduced us to some tools so we could diagnose performance issues ourselves. Based on our Consulting Engineer’s recommendations, we changed the structure of how we stored some data and reworked certain queries to improve performance. We also cleaned up a bunch of unnecessary indexes. We had created a number of indexes over the years for different query patterns, and our Consulting Engineer was able to identify which ones could be removed wholesale, and which indexes could be replaced with a single new one to cover different query patterns. We made some optimizations in Atlas as well, moving to a Low CPU instance based on the shape of our workload and changing to a more efficient backup option. With the optimizations recommended in our consulting engagement, we were able to reduce our spend by more than 35%. MongoDB consulting paid for itself in less than a month, which was incredible. I had to develop a business case internally for investing in consulting, and this level of savings made it an easy sell. The knowledge we picked up during our consulting engagement was invaluable. That’s something we’ll carry forward and that will continue to provide benefits. We’re much better at our indexing strategy, for example. Say you’re introducing a new type of query and thinking about adding an index: now we know what questions to ask. How often is this going to be run? Could you change the query to use an existing index, or change an existing index to cover this query? If we decide we need a new index, should we deprecate an old one? With the optimizations recommended in our consulting engagement, we were able to reduce our spend by more than 35%. MongoDB consulting paid for itself in less than a month, which was incredible. Brad Beeler, Lead Architect, Flowhub EH: What advice would you give to someone who's considering MongoDB for their next project? BB: Take the time upfront to understand your data and how it’s going to be used. That’ll give you a good head start for structuring the data in MongoDB, designing queries, and implementing indexes. Obviously, Flex Consulting was very helpful for us on this front, so give that a look.

January 19, 2021

Imposter Syndrome and Public Speaking: 5 Tips for a Successful Tech Talk

I've done hundreds of presentations at large and small events. I’ve delivered keynotes in front of a thousand people. You probably think I enter the stage each time with full confidence that I’ll give a solid and engaging presentation. You’re wrong. I’m an imposter when it comes to public speaking, and I'm not the only one. Every time I accept a speaking gig I think there must be people in the audience with much more experience and who can deliver my talk more eloquently. I feel like a fraud, and I know lots of other speakers that have the same feeling. That’s imposter syndrome. Studies have shown that actually 70% of us have experienced this kind of anxiety at some stage in our lives. But none of this stops me from speaking, and it shouldn’t stop you. You’ll get so much out of being brave enough to tell your story in front of a small or large audience: people open up and share their experiences with you, you end up helping others by inspiring them with your story, and you meet a lot of interesting people. Here are five tips for how to overcome your imposter syndrome and deliver a successful tech talk with confidence. 1. Be Better It helps to watch a lot of talks from conferences and note down what you like and don’t like about them. When preparing your talk, have that list present and avoid including things you dislike. This will help you develop better practices in delivery, story telling, slide design, or whatever area you have identified as important. When I put together my first presentation for an event in Munich, I watched some talks from the previous year. I was bored when speakers started with five minutes of talking about themselves and what they’ve done, so I decided to always begin my talks with telling a personal relatable story. It helped me to have the impression that my talks had a much better intro than all others (which might not be true, but I felt that way). 2. Be Authentic If you pretend to be someone you’re not, you’ll definitely suffer from the imposter syndrome. Be honest with the audience. Share your experience about a specific topic. Audiences love to hear real stories and want to learn from your successes and mistakes. As I mentioned before, I always start talks by sharing a true and relatable story. That serves a few purposes: The talk has an interesting start, and I have people’s attention from the first minute. When I share my own struggles and successes, it helps make the audience feel that I’m one of them. I don’t have to pretend to be someone that I’m not. It's my story and my experience. Being authentic helps you feel less like a fraud. 3. Grow Confidence Get feedback at every stage of your talk development, from writing your conference submission , putting together your talk , and crafting your slides to practicing and refining. Ask coworkers, friends, or other speakers to give you feedback. I know this is very hard for us imposters. We’re afraid to get caught being a fraud by people we care about. On the other hand, it’s a safe environment for receiving candid and honest feedback. I always test my new talks in front of a couple of coworkers. They know me and don’t want to hurt my feelings, but they also do not hold back if they don’t like parts of my presentation. It always makes my talks much better and gives me confidence that if they like it, the conference audience might like it too. If you’re just starting with public speaking, it might be an option to hold your presentation first at a smaller community gathering. These events normally are held in a much more relaxed atmosphere and have a smaller audience. It’ll boost your confidence if those user groups like your presentation, and you’ll feel more prepared for larger stages. Start small, grow confidence, and then go big (if you want). 4. The Audience is Your Friend Every speaker I know is nervous before the talk, no matter how many presentations they’ve given. Always remember: the audience wants you to succeed. They came to learn something. They don’t want you to fail. The audience is full of friends that want to support you. Bring some friends. It’s always good to see a friendly, familiar face in the audience (even if your talk is being delivered virtually). Talk to your friend right before you go on stage so your mind is not focused too much on the challenge ahead of you. Find friendly faces. Look around in the audience and find some friendly faces. You can use those attendees later in the talk to get visual confirmation that you’re doing great. If you’re giving your talk in-person, talk to people at the entrance or in the first row. Have a casual conversation, or just say hi. Connecting to the audience helps you to not think of them as strangers that are going to raise critical questions. 5. Use the Imposter Syndrome to Your Advantage Imposter syndrome gives you the feeling of being a fraud talking about a specific topic. You think there must be experts at this conference that know much more about the topic than you do. Use that feeling as a catalyst to really double down on learning more about the topic. I once submitted a talk about software development automation to an event in Norway and got accepted. I knew a few concepts but wasn’t an expert. The next day I went into deep-learning mode, read tons of articles on the internet, and scheduled a dozen interviews with people from various organizations who were responsible for running development tools, were tech leads, or just were innovative developers. I dug up so many interesting stories. I was scared as hell when giving the talk for the first time because I still did not feel like an expert on the topic, but people loved the stories I uncovered and as a result, I got invited to give the talk as a keynote at another event in Italy. Investing time in your research helps you to build confidence, so use the imposter syndrome as an accelerator. How can you start putting together your tech talk abstract and getting ready to submit it to your next conference? At MongoDB University we’ve put together a free course on this topic. At MongoDB, we believe that everyone has an interesting story to share, and we want to help you bring it to life. 2021, MongoDB’s biggest conference of the year, is back, and we’re currently looking for speakers to inspire and equip attendees with new technologies, ideas, and solutions. Submit your talk today .

January 19, 2021

Modernize data between siloed data warehouses with Infosys Data Mesh and MongoDB

The Data Challenge in Digital Transformation Enterprises that embark on a Digital transformation often face significant challenges with accessing data in a timely manner—an issue that can quickly impede customer satisfaction. To deliver the best digital experience for customers, companies must create the right engagement strategy. This requires all relevant data available in the enterprise be readily accessible. For example, when a customer contacts an insurance company, it is important that the company has a comprehensive understanding of the customer’s background as well as any prior interactions, so they can orchestrate the best possible experience. Data is available in both BI (Business Intelligence) systems, like Enterprise Data Warehouses, and OI (Operational Intelligence) systems, like policy and claim systems. There is a need to bring these BI and OI systems together to avoid any disruption to the digital functions that may delay synchronization. Data removed from an operational system loses context. Re-establishing this domain context and providing persona-based access to the data requires domain-oriented, decentralized data ownership, as well as architecture. Ultimately, organizations seek to use data as a key to fueling the products and services they provide their customers. This data should minimize the cost of customer research—but the data needs to be trusted and high quality. Companies need access to these siloed sources of data in a seamless self-service approach across various product life cycles. The Challenge of Centralized Data Historically, businesses have handled large amounts of data from various sources by ingesting it all into a centralized database (data warehouse, data lake, or data lake on cloud). They would then feed insight drivers, like reporting tools and dashboards as well as online transaction processing applications, from that central repository. The challenge with this approach is the broken link between analytical systems and transactional systems that impedes the digital experience. Centralized systems, like data warehouses, introduce latency and fail to meet the real time response and performance levels needed to build next-generation digital experiences. What is Infosys Data Mesh? Data Mesh helps organizations bridge the chasm between analytics and application development teams within large enterprises. Data Mesh is an architecture pattern that takes a new approach to domain-driven distributed architecture and the decentralization of data. Its basic philosophy is to encapsulate the data, its relationships, context, and access functionality into a data product with guaranteed quality, trust, and ease of use for business consumption. Data Mesh is best suited for low-latency access to data assets used in digital transformations that are intended to improve experience through rich insights. With its richer domain flavor — distributed ownership, manageability, and low latency access — Data Mesh is best positioned as a bridge between transactional (consuming applications) and analytical systems. This diagram depicts the high-level solution view of Data Mesh: Data Mesh Key Design Principles Domain-first approach. Data as a product. Data Mesh products share the following attributes which maximize usability and minimize friction: Self-described: metadata is precise and accurate Discoverable and addressable: products are uniquely identifiable and easy to find Secure and well-governed: only those who are granted access have it Trustworthy: proper data quality conrtols are applie, SLA/SLOs are maintainted Open standard and interoperable: data formats — XBRL, JSON Build new products easily. Any cross-functional team can build a new, enterprise-level product in an existing domain and/or fro existing products Simplified access for multiple technology stacks. Polygot data and ports, cloud and non-cloud. Common infrastructure and services for all data pipelines and catalogs. Platform Requirements for a Data Mesh To build a data mesh, companies need a database platform that can create domain-driven data products that meet various enterprise needs. This includes: Flexible data structures — to accomodate new behaviors An API-driven construct — to access current data products and build new domain-data ones Support for high-performance query on large-scale data structures A shared, scalable infrastructure Why MongoDB is the Optimal Platform for Infosys Data Mesh MongoDB is the best platform for realizing the Infosys Data Mesh architecture and powering analytics-driven enterprises because it provides: A flexible document model and a poly-cloud infrastructure availability so teams can easily modify and enrich flat or hierarchical data models MongoDB Realm Webhooks to create service APIs which connect data across products and enable consumption needs based on business context A scalable, shared infrastructure and support for high-performance querying of large scale data Service APIs for constructing Infosys Data Mesh Two use cases: table, th, td { border: 1px solid black; border-collapse: collapse; } Case 1: Case 2: A wealth management firm offers a variety of products to its customers — things like checking and savings accounts, trading, credit and debit cards, insurance, and investment vehicles. Challenges: Each product is serviced by a different system and technology infrastructure Internal consumers of this data have different needs: product managers analyze product performance, wealth managers and financial advisors rely on customer-centric analytics, and financial control teams track the firm’s revenue performance Solution: Using the Infosys Data Mesh model, the firm’s data owners create domain-data products categorized by customer and product, and then curate and publish them through a technology-agnostic, API-driven service layer. Consumers can then use this service layer to build the data products they need to carry out their business functions. The Risk and Finance unit of a large global bank has multiple regional data lakes catering to each region’s management information system and analytical needs. This poses multiple challenges for creating global data products: Challeges: Technology varies across regions ETL can becomes less advantageous depending on circumstance Regulations govern cross-regional data transfer policies Solution: To address these challenges, the bank creates an architecture of regional data hubs for region-specific products and, as with Case 1, makes those products available to authorized consumers through a technology-agnostic, API-driven service layer. Next, it implements an enterprise data catalog with an easy-to-use search interface on top of the API layer. The catalog’s query engine executes cross-hub queries, creating a self-service model for users to seamlessly discover and consume data products and to align newer ones with their evolving business needs. Enterprise security platform integration ensures that all regulatory and compliance requirements are fully met. How Businesses Overall Can Benefit Data and Insights become pervasive and consumable across applications and personas Speed-to-insights (including real time) enable newer digital experiences and better engagement leading to superior business results Self-service through trusted data products is enabled Infosysy DNA Assets on MongoDB Accelerates the Creation of Industry-Specific Domain Data Products table, th, td { border: 1px solid black; border-collapse: collapse; } Infosys Genome Infosys Data Prep Infosys Marketplace Creates the foundation for Data Mesh by unifying semantics across industries Guides consumers through the product creation process with a scalable data preparation framework Enables discovery and consumption of domain-data products via an enterprise data catalog Download our Modernization Guide for information about which applications are best suited for modernization and tools to get started.

January 14, 2021

Learn How to Scale Your Startup with this Educational Series

As a startup, you're probably interested in spending more time on shipping features than managing your database. Upskill in the new year with Scaling Your Startup with MongoDB Atlas: A Series . Join us for a three-part educational monthly webinar series kicking off January 26, completely focused on how to do less with more. When? The last Tuesday of every month, January through March, at 9 am PT/noon ET What? Our technical experts will take you through NoSQL, the MongoDB Query Language, MongoDB Atlas (our global cloud database service), and show you how to take the heavy lifting out of database management. View Scaling Your Startup with MongoDB Altas: A Series Why NoSQL for Startups | Lauren Schaefer | January 26 Scale Your Startup with MongoDB Atlas | Mike Lynn | February 23 Reaching Scalability with MongoDB Atlas | Adrienne Tacke | March 30 Where? Zoom How much? Free! Who should attend? Attend if you are a MongoDB beginner, in need of a technical refresher, building a product or service, or part of a startup. Will it be recorded? Of course. Each webinar will be recorded and posted to the website for you to watch anytime, at your convenience. Why? Just check out the takeaways below. Plus at the end of each presentation, you’ll have time to submit queries for Q&A with the technical expert. Key takeaways will include: Mapping terms & concepts from tables to documents Discovering the 4 major advantages of documents for your startup Changing your mindset in 3 key ways Deploying a free tier cluster Adding users & managing user access Connecting to MongoDB Compass, the GUI for MongoDB What's Realm? Third-party integration What is and why Multi-Cloud? How to set-up Multi-Cloud Cluster And much more! Save Your Spot for Scaling Your Startup with MongoDB Atlas: A Series

January 13, 2021

Meet Kaitlin Mahar: How MongoDB’s Drivers Team Makes It Easier for Developers to Innovate

MongoDB is committed to helping both our customers and the tech community innovate quickly. I recently sat down with Kaitlin Mahar, Lead Engineer, to learn more about the Drivers team, how she grew her career at MongoDB, and what her team does to positively impact the open source community. Ashley Perez: What is the Drivers team, and how does it help MongoDB customers? Kaitlin Mahar: Our team builds the programming language-specific client libraries — which we refer to as “drivers” — for MongoDB. Drivers are just as essential to using MongoDB as the database itself. Without a driver, you can't actually connect to the database from your application. Our team prioritizes building drivers that we call “idiomatic” for the programming language and ecosystem in which they’re used. A MongoDB driver should follow language best practices, “feel” like other libraries you’ve used in the same language, and integrate easily with popular frameworks in that language. This makes it easy for our customers and community to get started and ramped up on our drivers. Given that anyone using MongoDB has to use a driver too, our team works with just about every type of customer and use case. Our drivers are fully open source and completely free to use, so many of our users are language community members and not customers at all. In fact, my first introduction to MongoDB was using the community version along with the Node.js driver to build a web application for a college class. What’s interesting about this team is that we’re constantly innovating. We develop drivers for new and upcoming program languages. For example, we just released new drivers for Swift and Rust last year. And we have to keep up to date on the latest trends, technologies, and best practices in our respective programming languages to incorporate into our work. We’re always learning. AP: What’s your role on the team? KM: I’m one of four Lead Engineers on the team. Each of us oversees two or three drivers; I manage the Node.js, Swift, and Rust drivers. My day-to-day is a mix of technical work and managerial work. On the technical side, I work on my own technical projects, which includes coding, writing designs, and so on. I also review the technical work of people I manage. On the management side, I work with our Product Manager and Director to decide which work our team should prioritize and decide which team member(s) should take on what work. Depending on the scope of work, I also coordinate with other departments if needed. However, I think my most important job as a manager is supporting my direct reports by enabling them to do their best work and providing them with opportunities to accomplish both short- and long-term goals so they can grow as engineers. AP: You joined MongoDB as an intern and participated in the rotation program. When selecting a full-time role, why did you choose the Drivers team? KM: I started at MongoDB as a summer intern, and then came back as a new grad and went through the new grad rotation program, which is sort of like three more mini-internships for six weeks each. This program allowed me to learn about the range of technical problems people work on at MongoDB while trying out a variety of team cultures and work styles to see what I liked best. One of my rotations was on the Drivers team. When I rotated, I worked on a major revamp of the Node.js driver’s BSON library. This was a high-impact, user-facing project that I was surprised to be entrusted with as a new grad, and I found I really enjoyed the experience. The chance to have a high level of ownership over what I worked on was motivating. Due to the structure of the department, where small teams of two to four engineers work on each language project, there is a lot of opportunity for ownership on the team, even as a new grad. At the time of the rotation program, I had discussed returning to the team with my mentor, Matt — who later became my manager — and learned there was an opportunity for me to work with him on a brand-new driver written in Swift. I was excited about the prospect of seeing how a new driver is built and learning a new language, so Drivers seemed like a natural choice. The Drivers team gets to think about a wide range of technical problems, ranging from API design to networking to distributed systems. The variety of areas I’d get to work in and learn about on the team was a huge factor in why I decided to pursue this. I also knew there was a lot of opportunity on the team to get involved with open source communities by attending and speaking at conferences, engaging with users, and contributing to open source libraries, all of which I was interested in doing. AP: It turns out you took the chance to work with the community and are doing interesting work. Can you share more about that? KM: I’m a member of the Swift Server Work Group (SSWG), a steering team that promotes the use of Swift for developing and deploying server applications. This is a committee composed of representatives from a few different companies, such as Amazon, Apple, and MongoDB, as well as representatives from a popular open source Swift web framework, Vapor. Our focus is guiding the development of a healthy and robust server-side Swift ecosystem. Since Swift is a fairly new language and the use of Swift on the server is even newer, it’s really exciting to be a part of. There are a lot of important foundational open source libraries being developed and conversations happening about what we want the ecosystem to look like. One set of those foundational server-side Swift libraries being developed is database drivers, such as our Swift driver for MongoDB. I originally got involved with the SSWG by pitching our driver to go through the SSWG’s incubation process. It’s an honor to be a part of the group, and so far it has been a great way to connect with people outside the company, contribute to open source, and keep up to date on the latest developments in the ecosystem. AP: That sounds like a great group that not only helps impact the community, but also allows you to grow as an engineer. How else have you grown professionally and personally at MongoDB? KM: Once I joined the Drivers team full time, I started working on the Swift driver. Over the next few years, I got promoted and eventually became a Senior Engineer. My manager, Matt, gradually handed me more responsibility for the project, such as making big technical decisions, providing input on what we should work on next, presenting what work we plan to do to the CTO and VPs at quarterly planning meetings, and representing the company at Swift conferences. I also had a lot of opportunities to mentor new grads and interns, overseeing their work and developing management skills that are now very important to me as a Lead Engineer. Once I had gotten experience in both technical leadership and leading people, I was promoted to be a Lead Engineer, responsible for both the Swift and Rust drivers and the engineers who work on them. I recently took on managing the Node.js driver as well, which is one of our most popular drivers and is much older than the Rust and Swift Drivers, so it has been an interesting new challenge for me. In terms of personal growth, one of the biggest ways I’ve changed is in my willingness to admit what I don’t know and proactively ask questions, even if they seem simple. I’ve realized that being a good engineer is less about what you know exactly, and more about how you approach solving problems and finding the answers. AP: Well said. How does your team collaborate? KM: Our team was quite distributed even before the pandemic began. Whereas about half of us usually work out of the New York office, my manager is in the San Francisco Bay Area, and two of the other Lead Engineers are located in Boston and Munich. Our Product Manager is in North Carolina, and we have team members scattered around the United States and other countries as well. Effective digital collaboration is essential for getting things done, since we don’t physically sit in the same location or work the same hours. In our day-to-day, we use a number of tools to accomplish this, including Slack, GitHub, Zoom, and Google Docs. Because we’re all solving the same problems, just in different programming languages, it’s beneficial for us to share knowledge with each other, both within our individual language teams and also across the department as a whole. For language-specific projects, we use a collaborative design process, where one individual writes up a proposed design and the rest of the team reviews it and provides feedback to improve the design. In regards to cross-driver projects, the drivers team writes shared specifications for how MongoDB drivers in all languages should work. These cover everything from high-level driver APIs (e.g., CRUD) down to low-level behavior such as connection pooling. These are collaborations between individuals who work on different language teams so we can ensure the specifications will work for drivers written in any programming language. We also often need to work with the Server team. For example, if the server adds a new feature such as transactions, the drivers also need to add APIs to support using that feature. Therefore, we frequently review server scope and design documents, and vice versa. What’s cool is that many of the other MongoDB teams use our drivers. For instance, the Cloud team uses both the Go and Java drivers, Atlas Data Lake uses the Go driver, MongoDB Compass and the new MongoDB shell (mongosh) use the Node.js driver, and Realm uses the Node.js BSON library. We coordinate with those teams to add new features, make changes, and act as necessary support for their use cases. They often make contributions directly to our drivers too. AP: What skills are important for someone to be successful on this team? KM: We are highly collaborative and do a lot of technical writing for our design and specification process, so team members should be strong written and verbal communicators. Since our team’s first and foremost purpose is making it easy to use MongoDB from any programming language, it’s also important that our team members have the ability to empathize with our users. They should have the ability to make pragmatic technical decisions too. As library developers, we have to make a lot of difficult choices — such as what features to include in our APIs — and balance trade-offs such as usability and maintainability. I also mentioned ownership as being a key component of working on our team, and something that attracted me to it: engineers on our team need to be ready to own the driver and projects they work on and live out MongoDB’s core value “own what you do.” AP: After working at MongoDB for four years, why do you stay? And why would someone be excited to join the team? KM: The strong mentorship and the established new grad program was a great way to launch my career, and I’ve seen many others experience the same thing. About three-quarters of the people in my new grad class are still at the company! In general, there are so many growth opportunities here that there’s no shortage of places where you can take your career at MongoDB, and we have both individual-contributor and people-management tracks, depending on your interests. The company has a wide variety of technical topics to work on, ranging from UI design all the way down to low-level C code, so it’s hard to get bored here. Even for more seasoned technologists, the opportunity to engage with the open source community through your work allows you to become an expert or deepen your expertise in your primary language, while learning more about other programming languages. You get to understand how a distributed database and database driver operates by working on a range of problems involving API design, concurrency and parallelism, distributed systems, and performance. You’ll constantly be learning new things here. From a general perspective, I personally love the company size. It’s big enough to have a good amount of structure and rigorous technical processes, but it’s still small enough that you can make an impact and get recognized as an individual. The flexible working arrangements are great too. Even before the COVID-19 pandemic, we had flexible hours, work-from-home options, and flexible time off. The people here are great — very smart, but down to earth and open to collaboration. It makes working at MongoDB really enjoyable. Interested in pursuing a career at MongoDB? We have several open roles on our teams across the globe, and would love for you to build your career with us!

January 12, 2021