MongoDB Blog

Articles, announcements, news, updates and more

Powered by MongoDB, Bliinx is Changing the Way Software is Sold

Regardless of the industry, sales organizations often struggle to determine the best way to identify potential customers. There are many schools of thought as to what the best approach is, and when the most opportune time a sales executive should reach out might be. One startup company aims to make that process as simple and efficient as possible. Bliinx , based in Montreal, Quebec, Canada, was created to help revenue teams focus and act on the most qualified leads and accounts based on product usage, billing, firmographic and marketing engagement data. Bliinx’s mission is to “change the way we sell software.” We spoke with Bliinx co-founders Fred Melanson and John Espinoza about starting the company, their journey, and where they see Bliinx headed in the future. How did you decide to start Bliinx? Melanson: I realized that it’s hard to build quality relationships with a lot of people, especially people that you’re trying to get investments from. I would ask people a lot of questions, and those were around relationship building and the question became how do you manage your clients relationships? Everyone would answer that they do everything manually, across siloed channels, and it’s a pain to manage and scale. So I figured there must be something there, that was really the spark that we created Bliinx on. What does Bliinx do? Melanson: We are a lead prioritization platform for bottom-up B2B SaaS, so we help sales teams - mainly account executives - to know who the best leads are at the best accounts to reach out to, and also to identify when it’s the best time to reach out to them. And the way we do that is by finding signals and insights in their sales conversations, their marketing engagement, and product usage. Our tool will plug into your system and find insights that are worth engaging on and scoring your talents and your leads, so the sales reps are focused on the best customers at the best time, without having to use generic one size fits all automation, which can be great for top of funnel SDRs, but for CSMs, who are really about nurturing, closing, and expanding revenue, it has to be more thoughtful and and more human because it’s getting harder and harder to get people’s attention and retention is immensely valuable for SaaS companies, so our tool helps us just find the best people at the best time to grow revenue faster. What are some tools that Bliinx connects with? Melanson: The basic one will plug into your email and calendar, we also have LinkedIn integration, which is pretty unique to sync your LinkedIn messages and plug into your CRM. It also connects with Slack to receive notifications and right now we are building integrations with Segment, Intercom, Stripe, and Snowflake, so reps can have product insights. We are also building new integrations for LinkedIn and Twitter so that reps can also have content marketing engagement insights to act on. Where are you right now with Bliinx? How has the journey been, have you gone through accelerators and are you funded by VC’s? Melanson: I started working on the project about a year-and-a-half, two years ago, it was really an idea out of college. So after a lot of learning, we raised an angel round really quickly, and a couple of months later we got accepted to 500 Startups. From there we raised a pre seed round and we’ve been iterating on the product, trying to really find our positioning, and find the people that have the problem, and figure out what’s the best version of the problem that we can solve. How did getting accepted into 500 Startups shape Bliinx? Melanson: It’s a game changer. I don’t think we would have been here today if it wasn’t for 500 Startups. It was an amazing experience, you’re surrounded by so many smart people, and have such an expertise that you don’t normally have access to. You get what you take out of it, so I pushed it to the max, every time there was office hours, I would take it, every time there was an investor meeting open, I would take it. I would really, really push and it got us to great results, and it’s through 500 Startups that I’ve met our lead investor. Can you tell us about your tech stack? Espinoza: I want to keep it simple, this is the main rule of the company. We've built our system with microservices, use NodeJS and NoSQL for our back-end and have built a robust back-end infrastructure to build our proprietary engines for data orchestration. The rest of our platform is built on typescript and we use MongoDB to manage our databases. How did you decide to go with MongoDB? Espinoza: My first startup, we used MongoDB, and had a great experience. We use MongoDB, and I really love it. We don’t have to care about backups, or anything to do with the infrastructure. It’s plug and play, so what’s amazing for us is I come from the background where you have to build everything. So going with the NoSQL database is fantastic because you don’t have to maintain all the schema, which can be really messy. Like I said, we try to keep it simple. What excites you now about working with Bliinx? Melanson: With the rise of companies that are product-led or marketing-led, and the fact that people are working remotely, sales is changing, and I think it’s for the better. Tools on the market need to adjust, yes people want to try it out before they buy it, but they don’t want to go through a sales rep, they still want to meaningfully connect with people in sales. And sales reps are a big part of that journey, it’s just that you don’t reach out cold to sell, you have them try it, and then you’re more of a consultant, or the hand holder through that way. So it excites me about figuring out a way for people to build meaningful connections in business, with us being so remote. Espinoza: Everything that we build in here is new for me, and that’s what excites me. Working with a lot of data coming from everywhere, and building something valuable for you, let’s do something valuable with a lot of data. This is the magic box that we build in our building, this is a great opportunity. What advice would you give to someone starting up their own company? Melanson: 99% of people just don’t start, so my main advice is to just start. That’s really what the hurdle is, that’s the toughest part, people think it’s recruiting a technical co-founder, or raising money is the toughest part, but it’s starting. You can go so far validating your idea, without having a single line of code. Espinoza: Don’t start with titles. In the beginning, you’re just people with a project. The other is to go talk to people who are doing the same thing. Finding other people to bounce ideas off of, just to validate ideas, that is something that has helped me a lot. Interested in learning more about MongoDB for Startups? Learn more about us here .

January 26, 2022
Applied

Scale Out Without Fear or Friction: Live Resharding in MongoDB

Live resharding was one of the key enhancements delivered in our MongoDB 5.0 Major Release . With live resharding you can change the shard key for your collection on demand as your application evolves with no database downtime or complex data migrations . In this blog post, we will be covering: Product developments that have made sharding more flexible What you had to do before MongoDB 5.0 to reshard your collection, and how that changed with 5.0 live resharding Guidance on the performance and operational considerations of using live resharding Before that, we should discuss why you should shard at all, and the importance of selecting a good shard key – even though you have the flexibility with live resharding to change it at any time. Go ahead and skip the next couple of sections if you are already familiar with sharding! Why Shard your Database? Sharding enables you to distribute your data across multiple nodes. You do that to: Scale out horizontally — accommodate growing data or application load by sharding once your application starts to get close to the capacity limits of a single replica set. Enforce data locality — for example pinning data to shards that are provisioned in specific regions so that the database delivers low latency local access and maintains data sovereignty for regulatory compliance. Sharding is the best way of scaling databases and MongoDB was developed to support sharding natively. Sharding MongoDB is transparent to your applications and it’s elastic so you can add and remove shards at any time. The Importance of Selecting a Good Shard Key MongoDB’s native sharding has always been highly flexible — you can select any field or combination of fields in your documents to shard on. This means you can select a shard key that is best suited to your application’s requirements. The choice of shard key is important as it defines how data is distributed across the available shards. Ideally you want to select a shard key that: Gives you low latency and high throughput reads and writes by matching data distribution to your application’s data access patterns. Evenly distributes data across the cluster so you avoid any one shard taking most of the load (i.e., a “hot shard”). Provides linear scalability as you add more shards in the future. While you have the flexibility to select any field(s) of your documents as your shard key, it was previously difficult to change the shard key later on. This made some developers fearful of sharding. If you chose a shard key that doesn’t work well, or if application requirements change and the shard key doesn’t work well for its changed access patterns, the impact on performance could be significant. At this point in time, no other mainstream distributed database allows users to change shard keys, but we wanted to give users this ability. Making Shard Keys More Flexible Over the past few releases, MongoDB engineers have been working to provide more sharding flexibility to users: MongoDB 4.2 introduced the ability to modify a shard key’s value . Under the covers the modification process uses a distributed, multi-document ACID transaction to change the placement of a document in a sharded cluster. This is useful when you want to rehome a document to a different geographic region or age data out to a slower storage tier . MongoDB 4.4 went further with the ability to refine the shard key for a collection by adding a suffix to an existing key. Both of these enhancements made sharding more flexible, but they didn’t help if you needed to reshard your collection using an entirely different shard key. Manual Resharding: Before MongoDB 5.0 Resharding a collection was a manual and complex process that could only be achieved through one of two approaches: Dumping the entire collection and then reloading it into a new collection with the new shard key . This is an offline process, and so your application is down until data reloading is complete — for example, it could take several days to dump and reload a 10 TB+ collection on a three-shard cluster. Undergoing a custom migration that involved writing all the data from the old cluster to a new cluster with the resharded collection. You had to write the query routing and migration logic, and then constantly check the migration progress to ensure all data had been successfully migrated. Custom migrations entail less downtime, but they come with a lot of overhead. They are highly complex, labor-intensive, risky, and expensive (as you had to run two clusters side-by-side). It took one MongoDB user three months to complete the live migration of 10 billion documents. How this Changed with MongoDB 5.0: Live Resharding We made manual resharding a thing of the past with MongoDB 5.0. With 5.0 you just run the reshardCollection command from the shell, point at the database and collection you want to reshard, specify the new shard key, and let MongoDB take care of the rest. reshardCollection: "<database>.<collection>", key: <shardkey> When you invoke the reshardCollection command, MongoDB clones your existing collection into a new collection with the new shard key, then starts applying all new oplog updates from the existing collection to the new collection. This enables the database to keep pace with incoming application writes. When all oplog updates have been applied, MongoDB will automatically cut over to the new collection and remove the old collection in the background. Lets walk through an example where live resharding would really help a user: The user has an orders collection. In the past, they needed to scale out and chose the order_id field as the shard key. Now they realize that they have to regularly query each customer’s orders to quickly display order history. This query does not use the order_id field. To return the results for such a query, all shards need to provide data for the query. This is called a scatter-gather query. It would have been more performant and scalable to have orders for each customer localized to a shard, avoiding scatter-gather, cross-shard queries. They realize that the optimal shard key would be "customer_id: 1, order_id: 1" rather than just the order_id . With MongoDB 5.0’s live resharding, the user can just run the reshard command, and MongoDB will reshard the orders collection for them using the new shard key, without having to bring the database and the application down. Watch our short Live Resharding talk from MongoDB.Live 2021 to see a demo with this exact example. Not only can you change the field(s) for a shard key, you can also review your sharding strategy, changing between range, hash, and zones. Live Resharding: Performance and Operational Considerations Even with the flexibility that live resharding gives you, it is still important to properly evaluate the selection of your shard key. Our documentation provides guidance to help you make the best choice of shard key . Of course, live resharding makes it much easier to change that key should your original choice have not been optimal, or if your application changes in a way that you hadn’t previously anticipated. If you find yourself in this situation, it is essential to plan for live resharding. What do you need to be thinking about before resharding Make sure you have sufficient storage capacity available on each node of your cluster. Since MongoDB is temporarily cloning your existing collection, spare storage capacity needs to be at least 1.2x the size of the collection you are going to reshard. This is because we need 20% more storage in order to buffer writes that occur during the resharding process. For example, if the size of the collection you want to reshard is 2 TB compressed, you should have at least 2.4 TB of free storage in the cluster before starting the resharding operation. While the resharding process is efficient, it will still consume additional compute and I/O resources. You should therefore make sure you are not consistently running the database at or close to peak system utilization. If you see CPU usage in excess of 80% or I/O usage above 50%, you should scale up your cluster to larger instance sizes before resharding. Once resharding is done, it's fine to scale back down to regular instance sizes. Before you run resharding, you should update any queries that reference the existing shard key to include both the current shard key and the new shard key. When resharding is complete, you can remove the old shard key from your queries. Review the resharding requirements documentation for a full run down on the key factors to consider before resharding your collection. What should you expect during resharding? Total duration of the resharding process is dependent on the number of shards, the size of your collection, and the write load to your collection. For a constant data size, the more shards the shorter the resharding duration. From a simple POC on MongoDB Atlas, a 100 GB collection took just 2 hours 45 minutes to reshard on a 4-shard cluster and 5 hours 30 minutes on a 2-shard cluster. The process scales up and down linearly with data size and number of shards – so a 1 TB collection will take 10 times longer to reshard than a 100GB collection. Of course your mileage may vary based on the read/write ratio of your application along with the speed and quality of your underlying hardware infrastructure. While resharding is in flight, you should expect the following impacts to application performance: The latency and throughput of reads against the collection that is being resharded will be unaffected . Even though we are writing to the existing collection and then applying oplog entries to both its replicas and to the cloned collection, you should expect to see negligible impact to write latency given enough spare CPU. If your cluster is CPU-bound, expect a latency increase of 5 to 10% during the cloning phase and 20 to 50% during the applying phase (*) . As long as you meet the aforementioned capacity requirements, the latency and throughput of operations to other collections in the database won't be impacted . (*) Note: If you notice unacceptable write latencies to your collection, we recommend you stop resharding, increase your shard instance sizes, and then run resharding again. The abort and cleanup of the cloned collection are instantaneous. If your application has time periods with less traffic, reshard your collection during that time if possible. All of your existing isolation, consistency, and durability guarantees are honored while resharding is running. The process itself is resilient and crash-safe, so if any shard undergoes a replica set election, there is no impact to resharding – it will simply resume when the new primary has been elected. You can monitor the resharding progress with the $currentOp pipeline stage. It will report an estimate of the remaining time to complete the resharding operation. You can also abort the resharding process at any time. What happens after resharding is complete? When resharding is done and the two collections are in sync, MongoDB will automatically cut over to the new collection and remove the old collection for you, reclaiming your storage and returning latency back to normal. By default, cutover takes up to two seconds — during which time the collection will not accept writes, and so your application will see a short spike in write latency. Any writes that timeout are automatically retried by our drivers , so exceptions are not surfaced to your users. The cutover interval is tunable: Resharding will be quicker if you raise the interval above the two second default, with the trade-off that the period of write unavailability will be longer. By dialing it down below two seconds, the interval of write unavailability will be shorter. However, the resharding process will take longer to complete, and the odds of the window ever being short enough to cutover will be diminished. You can block writes early to force resharding to complete by issuing the commitReshardCollection command. This is useful if the current time estimate to complete the resharding operation is an acceptable duration for your collection to block writes. What you Get with Live Resharding Live sharding is available wherever you run MongoDB – whether that’s in our fully managed Atlas application data platform in the cloud , with Enterprise Advanced , or if using the Community Edition of MongoDB. To recap how you benefit from live resharding: Evolve with your apps with simplicity and resilience: As your applications evolve or as you need to improve on the original choice of shard key, a single command kicks off resharding. This process is automated, resilient, and non-disruptive to your application. Compress weeks/months to minutes/hours: Live resharding is fully automated, so you eliminate disruptive and lengthy manual data migrations. To make scaling out even easier, you can evaluate the effectiveness of different shard keys in dev/test environments before committing your choice to production. Even then, you can change your shard key when you want to. Extend flexibility and agility across every layer of your application stack: You have seen how MongoDB’s flexible document data model instantly adapts as you add new features to your app. With live resharding you get that same flexibility when you shard. New features or new requirements? Simply reshard as and when you need to. Summary Live Resharding is a huge step forward in the state of distributed systems, and is just the start of an exciting and fast-paced MongoDB roadmap that will make sharding even easier, more flexible, and automated. If you want to dig deeper, please take a look at the Live Resharding session recording from our developer conference and review the resharding documentation . To learn more about MongoDB 5.0 and our new Rapid Releases, download our guide to what’s new in MongoDB .

January 26, 2022
Developer

10 Signs Your Data Architecture Is Limiting Your Innovation: Part 3

When it comes to your database architecture, complexity can quickly lead to a drag on your productivity, frustration for your developers, and less time to focus on innovation while your team maintains the status quo. New feature rollouts take longer than they should, while your resources are consumed up by tedious tasks that allow your app to survive, but not truly thrive. This complexity manifests in many different ways; as they accumulate, they can become a serious hindrance to your ability to bring innovative ideas to market. We think of the effect as a kind of tax — a tax that is directly rooted in the complexity of your data architecture. We call it DIRT — the Data and Innovation Recurring Tax . We have identified ten symptoms that can indicate your business is paying DIRT. For an in-depth view, read our white paper 10 Signs Your Data Infrastructure is Holding You Back . Sign #5: New features are rolled out in months, not days With a complex data architecture, your developers have to switch constantly between languages and think in different frameworks. They may use one language to work directly with a database, another to use the object-relational mapping (ORM) layer built on top of it, and yet another to access search functionality. That becomes a major drag on productivity. That slows down your individual developers, but it also has consequences for how they work as a team. If every application architecture is bespoke, it’s almost impossible for developers’ skills to be shared and put to use across an organization. Development slows down. When a key person leaves, there is no one who can effectively fill in and you end up hiring for very specific skills. That’s hard enough, but you also don’t know if you’ll still need those skills in a year or three. Sign #6: It takes longer to roll out schema changes than to build new features If you’re rolling out application changes frequently — or trying to — and you’re using a relational database, then schema changes are hard to avoid. One survey found that 60% of application changes require modifications to existing schema, and, worse, those database changes take longer to deploy than the application changes they are supposed to support. Legacy relational databases require developers to choose a schema at the outset of a project, before they understand the entirety of the data they need or the ways in which their applications will be used. Over time, and with user feedback, the application takes shape — but often it’s not the shape that was originally anticipated. At that point, a fixed schema makes it very hard to iterate, leaving teams with a tough choice: try to achieve your new goals within the context of a schema that isn’t really suitable or go through the painful process of changing it. Learn more about the innovation tax and how to lessen it in our white paper DIRT and the High Cost of Complexity .

January 21, 2022
Developer

Faster Migrations to MongoDB Atlas on Google Cloud with migVisor by EPAM

As the needs of Google Cloud customers evolve and shift towards new user expectations, more and more customers are choosing the MongoDB Application Data Platform as an ideal alternative to legacy databases. Together, we’ve partnered with users looking to digitize and grow their businesses (such as Forbes ), or meet increased demand due to COVID (such as our work with Boxed , the online grocer) by scaling up infrastructure and data processing within a condensed time frame. As a fully-managed service within the Google Cloud Marketplace , MongoDB Atlas enables our joint customers to quickly deploy applications on Google Cloud with a unified user experience and an integrated billing model. Migrations to managed cloud database services vary in complexity, but even under the most straightforward circumstances, careful evaluation and planning is required. Customer database environments often leverage database technologies from multiple vendors, across different versions, and can run into thousands of deployments. This makes manual assessment cumbersome and error prone. This is where EPAM Systems , a provider with strategic specialization in database and application modernization solutions, comes in. EPAM’s database migration assessment tool, migVisor , is a first-of-its-kind cloud database migration assessment product that helps companies analyze database workloads, configuration, and structure to generate a visual cloud migration roadmap that identifies potential quick wins as well as challenge areas. migVisor identifies the best migration path for databases using sophisticated scoring logic to rank the complexity of migrating them to a cloud-centric technology stack. Previously applicable only to migrations from RDBMS to cloud-based RDBMS, migVisor is now available for MongoDB to MongoDB Atlas migrations. migVisor helps you: Analyze migration decisions objectively by providing a secure assessment of source and target databases that’s independent of deployed environments Accelerate time to migration by automating the discovery and assessment process, which reduces development cycles from a few weeks to a few days Easily understand tech insights by providing a visual overview of your entire journey, enabling better planning and improving stakeholder visibility Reduce database licensing costs by giving you intelligent insights on the target environment and recommended migration paths Key features of migVisor for MongoDB For several years, migVisor by EPAM has delivered automated assessments that have helped hundreds of customers migrate their relational databases to cloud-based or cloud-native databases. Now, migVisor adds support for the world’s leading modern data platform: MongoDB. As part of the initial release, migVisor will support self-managed MongoDB to MongoDB Atlas migration assessments. We plan to support TCO for MongoDB migrations, application modernization, migration assessment, and relational MongoDB migration assessments in future releases. MongoDB is also a natural fit for Google Cloud’s Open Cloud strategy of providing customers a broad set of fully managed database services, as Google Cloud's own GM and VP of Engineering & Databases, Andi Gutmans, notes: We are always looking for ways to simplify migrations for our customers. Now, with EPAM's database migration assessment tool, migVisor, supporting MongoDB Atlas, our customers can easily complete database assessments—including TCO analyses and migration complexity assessments, and generate comprehensive migration plans. A simplified migration experience combined with our joint Marketplace success enables customers to consolidate their data workloads into the cloud while making the development and procurement process simple&#8212;so users can focus more on innovation. How the migVisor assessment works migVisor analyzes source databases (on-prem or in any cloud environment) for migration assessment to a new target. The assessment includes the following steps: The simple-to-use migVisor Metadata Collector (mMC) collects metadata from the source database, including: featureCompatibilityVersion value, journaling status for data bearing nodes, MongoDB storage size used, replica set configuration, and more. Figure 1: mMC GUI Edit Connection Screen On the migVisor Analysis Dashboard you can select the source/target pair (e.g., MongoDB to MongoDB Atlas on Google Cloud). Figure 2: Source and Target Selection In the migVisor console, you can then view the automated assessment output that was created from migVisor’s migration complexity scoring engine, including classification of the migration into high/medium/low complexity and identification of potential migration challenges and incompatibilities. Figure 3: Source Cluster Features Finally, you can also export the assessment output in CSV format for further analysis in your preferred data analysis/reporting tool. Conclusion Together, Google Cloud and MongoDB have successfully worked with many organizations to streamline cloud migrations and modernize their legacy landscape. To build on the foundation of providing our customers with the best-in-class experience, we’ve closely worked with Google Cloud and EPAM Systems to integrate MongoDB Atlas with migVisor. Because of this, customers will now be able to better plan migrations, reduce risk and avoid missteps, identify quick wins for TCO reduction, review migration complexities, and appropriately plan migration phases for the best outcomes. Learn more about how you can deploy, manage, and grow MongoDB on Google Cloud on our partner page . If you’d like guidance and migration advice, please reach out to mdb-gcp-marketplace@mongodb.com to get in touch with the Google, MongoDB, and EPAM Sales teams.

January 21, 2022
Developer

Starting a Career as a Solutions Architect in MongoDB’s Remote Presales Centre

MongoDB’s Remote Presales Centre is kickstarting presales careers and helping customers unlock the value of MongoDB technology. I spoke with Chris Dowling and Snehal Bhatia to learn more about the Remote Presales Centre Solutions Architect role, how they’re making an impact, and why this is an exciting opportunity for those interested in understanding the intersection of business and technology. Jackie Denner: Hi, Chris and Snehal. Thanks for sitting down with me today to discuss the Remote Presales Centre. What is MongoDB’s Remote Presales Centre team? Chris Dowling: The Remote Presales Centre Solutions Architect is an introductory Solutions Architect (SA) role. Our global team is spread across the Americas, EMEA, and APAC, and we are actively growing. We currently have SAs in EMEA covering French, German, Italian, Spanish, and English speaking customers. By joining the team, you’ll essentially be in an “incubation” period to gain experience in a presales role and exposure to sales cycles. Snehal Bhatia: Yes, this Solutions Architect role is for people who are earlier in their career and might not necessarily come from a presales background. We’re not dedicated to particular customers or accounts, rather we cover a wider perspective to help a larger volume of customers across various regions and Sales teams. Not only do we gain valuable experience, but we’re able to add value to the sales cycle by way of customer education through enablement sessions and workshops, along with engaging with customers at an earlier stage to bring technical value from the get-go. We’re also brought in to help qualify opportunities during discovery meetings. Overall, the biggest gap we see is that customers often have a difficult time understanding MongoDB technology, so we’re there to provide clarity, answer questions, and showcase the value of MongoDB. JD: So, what is a typical week like in your Solutions Architect role? CD: I’ve had 15 customer contacts this week. If you’re looking at strictly one-on-one sessions, the maximum number of customers someone on our team would handle per week is around 20. If you take into account some of the wider marketing events we help run as well, it could be as many as 100 customers, it really depends on the day. We don’t just do account-based activities, we also run wider campaigns like workshops and webinars. Snehal and I also had the opportunity to speak at MongoDB.local London in November 2021 on the topics of read and write concerns and how to set up your database for the tradeoffs you need and how ethical concerns need to be factored into technology and IoT design. We also get the chance to do things outside of core responsibilities and are able to work on side projects if we’d like. For example, I really enjoy management and education so I do a lot with sales reps to help them understand MongoDB technology. We really do a mixture of things. In a typical week, we’ll have one or two webinars, a few security questionnaires which is part of the end of a deal cycle and includes some technical questions that we need to respond to, then we have discovery meetings and prep calls with different reps, and we also have a day dedicated to enablement. SB: Yes, we have all of these customer engagements but the core of it is the prep that comes beforehand. We end up working with Marketing, Sales, Sales Managers, Product Owners, Professional Services - we work with a lot of different teams to get their insight so that we’re able to provide a complete view or solution to the customer. The internal prep meetings are a big part of that execution. JD: Why would someone move from an implementation role into a Remote Presales Centre role? CD: Snehal and I both come from an implementation background. I think you should join the Remote Presales Centre team if you’re interested in the architecture of how businesses are running their systems and want to see how the sales process works. In this role, we’re uncovering the answers to “What is motivating the customer to do this? Why would they buy MongoDB? Does MongoDB work for them?” Every day is different for us. In an implementation role, you end up working on the same system and use cases day in and day out, whereas in our role we get to see everything under the sun of what customers might want to do and get to go in and explore a new piece of technology. It’s exciting to see the newest things in tech. SB: In my previous implementation role the goal was to become an expert on just one of the products, which didn’t really help with broadening my skillset. When I came here, I had the opportunity to work with customers from financial services, telecom, banking, IoT, startups, big enterprises, you name an industry or company size and we’ve done something for them, or you name a technology and we’ve likely worked with it. That variety is not something you’d get in an implementation role. Not to mention, in implementation roles you’re often told what to do. The requirements are already made up and you just have to meet them. In our roles as SAs, we’re really influencing the direction of things and understanding the bigger picture and business implications of utilizing the technology. We have the ability to influence customers in a positive way and provide value. JD: Can you describe the learning curve for someone moving into the Remote Presales Centre from a more delivery-focused role? SB: I would say that the biggest mindset shift is instead of immediately answering questions, you need to stop and ask why. If someone says “We want to do this” your first instinct may be to respond and say “Yes we have the capabilities to meet that”, but really you should stop and ask “Why do you want to do this? What value is it going to bring for you? How is this going to influence your business direction?” You need curiosity to understand what the customer is trying to achieve instead of focusing on solving specific issues and pain points, which is very much the focus in an implementation role. CD: It’s also learning the sales cycle and how sales operates, along with figuring out what drives reps and what they want out of the Remote Presales Centre. Sometimes reps need us to explain the technology and sometimes we’re just there for credibility. It’s getting in the mindset of partnering with sales not working for sales. There is obviously a technology learning curve as well since MongoDB products are vast and often complex. SB: I think that extends to the customers we work with as well. Every call you go into you’ll be meeting with a different “customer persona”. Sometimes you’re talking to very technical people like developers and DBAs, so you need to be able to tailor the conversation as per their priorities. But, if you’re meeting with the CTO, you need to contextualize it in business terms to relay what the business needs. It’s all about understanding your audience and tailoring the conversation. JD: Aside from databases, what other technologies do you need to be familiar with or are you exposed to? SB: Everything! When you think of a database, you will never use a database by itself, you have to build an application on top of it. A lot of our role is understanding how the database is contributing to the whole software development lifecycle and overall project. At the end of the day, it’s a part of the tech stack, so you have to understand the whole tech stack, the underlying infrastructure, and the application that’s built on top of the database. It’s not just MongoDB that we talk or learn about, but it’s every other database in the market and every technology that the customer is working with. Every customer we talk to is working with a different tool, programming language, or software development methodology, and you need to be able to communicate with them. JD: How do you stay connected with your colleagues when you are all working remote? CD: If we’re running a workshop it’s a team event, so we end up working closely for that. We also have weekly syncs where we talk about what we’re working on and talk through challenges, and we have things like enablement sessions and coffee chats. SB: These sessions are also on a global level so we have the opportunity to work with the team in the Americas. Since we operate on a volume basis, we’ll discuss workload distribution and try to prioritize tasks based on people’s interests. CD: Yes, for example, I really like time series and search, so I’ll handle a lot of time series and search requests. There’s someone else on the team who loves Realm, our mobile database, so we give him all the Realm requests. JD: Often people are reluctant to move into presales as they don’t consider themselves sales-oriented. How would you respond to that? CD: Stop thinking of it as sales! Think of it as you get to talk to tons of customers about what they think the best technological solution is, and then you can provide insight into MongoDB and how our technology can improve what they’re trying to do. It’s a really technical job in the sense that you’re looking at organizations’ architectures and you’re figuring out why customers are doing what they do. You get to ask a lot of questions and see a lot of new technology. You could end up building proof of values out of that which means you then get to play around with this new technology. SB: I think presales is the best of both worlds. You get to interact with a lot of people in various scenarios, but you are the trusted advisor for the customer. You’re there to help them and are on their side, which means customers trust and confide in you. JD: What learning and growth opportunities are there for someone on the Remote Presales Centre team? CD: You start off doing simple things like learning about MongoDB products, getting ground knowledge, learning customer stories, and understanding why customers use MongoDB. Then you move on to discovery calls with customers and learning how to scope things out for yourself. From there, as you spend more time in the Service Centre, you slowly get further and further through the deal cycle. For example, a few months ago I was in a workshop to determine the technical feasibility of MongoDB’s solution after we had already worked with the customer to determine business objectives and requirements. You eventually go through the whole sales cycle with the goal being that you can execute the whole sales cycle by the time you leave to go into the field. SB: Since the Service Centre is a somewhat new team for MongoDB, you’re also part of discussing processes and helping determine what makes the team most efficient. You get to contribute to building a whole new team and company right now, which is not something you would get in a mature team with defined processes. CD: As the team grows there are a lot of mentorship opportunities as well. MongoDB is growing so quickly that new sales reps come in and are great at selling, but they don’t always have a technical background or understand MongoDB’s value proposition. We are that technical backup for them, and this allows the field SAs more time to do the really deep technical things that we’ll eventually get to do once we move into a more senior position. JD: Why should someone join your team? CD: You have the opportunity to learn so much about MongoDB’s technology and sales cycle, and you get to meet anyone and everyone. I could be talking to a Product Manager in the morning about the newest release and a Customer Success Manager in the afternoon. You really get to meet the whole organization. You’ll have a lot of internal visibility which is great because it also provides pathways to transfer internally if you want to. SB: You don’t get this visibility in most other roles because you’re usually aligned to a region or team. Here, we get to meet everyone in Europe. Chris and I put together a spreadsheet of all of the sales reps in Europe and there’s only 12 we haven’t had the chance to work with yet. Not only do we get to work with all the reps, but we also work with Product Managers, Customer Success, Marketing, Information Security, plus all of their managers. It’s a great way to get introduced to the company. Interested in a Presales career at MongoDB? We have several open roles on our teams across the globe and would love for you to transform your career with us!

January 20, 2022
Culture

Optimize Atlas Spending with the Billing Dashboard in Atlas Charts

Managing infrastructure spend across cloud services is a universal challenge for organizations around the world. Teams want to know that their investments in services are driving business value. How much are we spending every month? Where are we spending the most within a given product or service? Are there outliers in our spending we need to take a closer look at? Are there changes that could reduce costs without a negative impact? These are just a few of the many questions you can answer about your Atlas services with the new billing dashboard built on Atlas Charts. Most modern software services give you the basic tools needed to track spending. Information such as: monthly cost, next payment date, billing period, cost by product and/or sub product, past invoice access, and saved payment methods are all common. Atlas itself offers all of these options and more – you can read all about this in our billing page documentation . For digging in even further, that’s where the billing dashboard on Atlas Charts becomes useful. The billing dashboard provides a dedicated space for additional insights into your Atlas spending and it requires minimal setup. If you’re ready to dive in, check out our Github repository for setup instructions, or read on below to learn more about how the dashboard works and how it can help your team find new insights into your Atlas spending. Visualizing your billing data with Atlas Charts The billing dashboard is available as an open source Github project , containing a Realm app to ingest billing data from the Atlas Billing API, an Atlas Charts dashboard to visualize the data, and scripts to help you set this up within your own Atlas organization. If you’re not familiar, Realm is our fully managed solution for edge-to-cloud sync and backend services for quickly building web and mobile apps. Atlas Charts is our native data visualization tool. Charts gives teams the ability to build dashboards from cluster data to quickly answer general questions about your business, to investigate specific questions or issues in your application data, to share dashboards with stakeholders on your team, or even to embed visualizations into your internal or external applications. Take a look at the video below to learn more about setup and see some of the built-in functionality and benefits: The solution comes with a ready-built dashboard that will answer many common questions about your cloud spend. Aggregate metrics such as current monthly spend, your previous month’s spend, and yearly spend provide an overview of your data. Detailed charts break down your projects, clusters, and product categories (i.e. instance, backup, and data transfer) across different time horizons to take your analytics a level deeper. Many Atlas customers further customize charts, and build new charts and dashboards leveraging the same underlying billing data. The dashboard gives you complete flexibility to track the metrics that are most important to your business. For organizations with many projects and clusters inside of Atlas, this flexibility can be invaluable in identifying opportunities to optimize the use of MongoDB. The set up process was recently updated to greatly simplify the steps required to get started. Check out the Github repo for step-by-step instructions. Upleveling your billing insights, for free Gaining additional insight into your Atlas billing will make your team confident that you are doing all you can to best manage your infrastructure spending with MongoDB Atlas. We want you to get the most out of MongoDB, while spending the least! If you need help getting started, feel free to reach out to your customer success manager (CSM). MongoDB’s customer success team works with customers every day to ensure they get the most of the Atlas platform. If you don’t have a CSM and would like to learn more about support at MongoDB, get in touch and we can talk more. Interested in using Atlas Charts for other insights? Get started today by logging into or signing up for MongoDB Atlas , deploying or selecting a cluster, and activating Charts for free.

January 20, 2022
Updates

Introducing the MongoDB 5.2 Rapid Release

MongoDB allows you to address a wide variety of data workloads using a single API. Our latest rapid release — MongoDB 5.2 — builds upon this vision with improvements to query ergonomics, enhancements to time series collections (introduced in MongoDB 5.0), scaling, operational resilience, and new capabilities that allow teams to execute more sophisticated analytics in-place. Columnar Compression for Time Series Collections Introduced in MongoDB 5.0, time series collections allow you to easily ingest and work with time series data alongside your operational or transactional data, without the need to integrate a separate single-purpose database into your environment. The 5.2 Rapid Release introduces columnar compression for time series collections in MongoDB. Time series use cases — whether it’s device monitoring, trendspotting, or forecasting — require that new data be inserted into the database for every measurement. In cases where data is being continuously created, the sheer amount of data can be staggering, making it difficult to manage an ever growing storage footprint. To help teams achieve high performance while maintaining resource efficiency, we’ve introduced a few capabilities to time series collections. New columnar compression for time series collections will help teams dramatically reduce their database storage footprint by as much as 70% with best-in-class compression algorithms such as delta, delta-of-delta encoding, simple-8b, run-length encoding, and more. For teams using MongoDB Atlas, Atlas Online Archive support for time series collections (introduced with the 5.1 Rapid Release) allows them to define archiving policies to automatically move aged data out of the database and into lower-cost, fully managed cloud object storage. Better Query Ergonomics and Point in Time Queries for Operational Analytics More efficient queries make developers’ lives easier. With the MongoDB 5.2 Rapid Release, we are introducing new operators and enhancements that will increase productivity, query performance and reduce the number of queries needed to unlock insights. This also allows teams to push more work down to the database, reducing the amount of code developers need to write and maintain while limiting the amount of data that has to be pushed back and manipulated in applications. New accumulators & expression to sort arrays MongoDB 5.2 brings new operators that streamline your queries. The $top and $bottom operators allow you to compute the top and bottom elements of a data set and return related fields within the same query without complex logic. For example, let’s say that you were analyzing sales performance and wanted the top salesperson for every region, including their sales. These new operators can help you retrieve the results in a single dataset, including any additional fields from the original dataset. {$group: { _id: "$region", person: { $top: { output: ["$name", "$sales"], sortBy: {"sales": -1} } } }} Result: { {_id:’amer’, person: [‘Daniel LaRousse’, 100000]}, {_id:’emea’, person: [‘John Snow’, 1]}, {_id:’latam’, person: [‘Frida Kahlo’, 99]} } We are also introducing $maxN , $minN , and accumulators such as $firstN , $lastN , which return elements while taking into account the current order of documents in a dataset. A highly requested feature, the new $sortArray expression allows you to sort the elements in an array directly in your aggregation pipeline in an intuitive, optimized way. The input array can be as simple as an array of scalars or as complex as an array of documents with embedded subdocuments. Let’s say you had previously sorted product reviews based on timestamp but now want to sort based on user rating. You can now easily do this using the $sortArray operator to change the sorting criteria with no additional code required. Sorting an array of integers $sortArray: { input: [3, 1, 4, 1, 5, 9], sortBy: 1 } Result: [1, 1, 3, 4, 5, 9] Sorting arrays of documents { "team": [ { "name": "kyle", "age": 28, "address": { "street": "12 Baker St", "city": "London" } }, { "name": "bill", "age": 42, "address": { "street": "12 Blaker St", "city": "Boston" } } ] A simple sort: "name" ascending {$project: { _id: 0, result: { $sortArray: { input: "$team", sortBy: {name: 1} } } } Output: { "result": [ { "name": "bill", "age": 42, "address": { "street": "12 Blaker St", "city": "Boston" } }, { "name": "kyle", "age": 28, "address": { "street": "12 Baker St", "city": "London" } } ] } Long-running snapshot queries now generally available Your applications can now execute complex analytical queries against a globally and transactionally consistent snapshot of your live, operational data. Even as data changes beneath you, MongoDB preserves point-in-time consistency of the query results returned to your users without you having to implement complex reconciliation controls back in your code. The default for long-running snapshot queries in MongoDB Atlas is 5 minutes but can be changed with the help of our support team. Queries can span multiple shards, unlocking analytics against large, distributed data sets. By routing long-running queries to secondaries, you can isolate analytics from transactional queries with both workloads served by the same cluster, avoiding slow, complex, and expensive ETL to data warehouses. Query results can be returned directly to the application or cached in a materialized view, providing your users with low latency access to deep analytics. Typical uses include end-of-day reconciliation and reporting, along with ad-hoc data exploration and mining. All of these use-cases can now be served directly from your transactional data layer, dramatically simplifying the data infrastructure you need to serve multiple classes of workloads. Improving Resilience with Faster Initial Sync via File Copy Initial sync is how a replica set member in MongoDB loads a full copy of data from an existing member. This process occurs when users are adding new nodes to replica sets to improve resilience, or to reduce read latency or improve read scalability with secondary reads. Initial sync is also commonly used to recover replica set members that have fallen too far behind the other members in a cluster. Prior to 5.2, logical initial sync was the only option available for performing an initial sync. With logical initial sync, every collection in the source node is scanned and all documents are then inserted into matching collections in the target node (with indexes being built at the time of document insertion). However, users and customers leveraging logical initial sync, especially those trying to synchronize large data sizes, have reported frustratingly long initial sync times. Starting with the 5.2 Rapid Release, we have added the option of initial sync via file copy to significantly improve the performance of initial syncs. With this method, MongoDB will copy files from the file system of the source node to the file system of the target node. This process can be faster than a logical initial sync, especially at larger data sizes. In our testing with a 630 GB dataset, initial sync via file copy was nearly four times (4X) faster than a logical initial sync on the same dataset. This new capability builds upon the continuous enhancements we’ve made to improve resilience and scalability, including the ability for initial sync to automatically resume after a network failure, and allowing users to specify their preferred initial sync source – both introduced with MongoDB 4.4. For more information, see the documentation on initial sync . Enhanced Developer Experience with MongoDB Analyzer for .NET And finally, we’re pleased to announce the release of the MongoDB Analyzer for .NET , which enables C# developers to more easily troubleshoot queries and aggregations, and prevent errors from cropping up at runtime. The MongoDB Analyzer builds on earlier releases of the MongoDB .NET driver. It makes it easier and faster for developers to use MongoDB with C#, including a fully redesigned LINQ interface. Previously, C# developers were able to interact with MongoDB idiomatically using Builders or LINQ expressions, but there was no easy way to see before running their code if those mapped correctly to the MongoDB Query API. Downloadable as a NuGet package, the MongoDB Analyzer allows developers to easily see if their queries and aggregations correspond to expressions supported by the Query API. By surfacing unsupported expressions during code development, the MongoDB Analyzer ultimately improves developer productivity and reduces the pain of debugging. Getting Started with MongoDB 5.2 MongoDB 5.2 is available now. If you are running Atlas Serverless instances or have opted in to receive Rapid Releases in your dedicated Atlas cluster, then your deployment will be automatically updated to 5.2 starting today. For a short period after upgrade, the Feature Compatibility Version (FCV) will be set to 5.1; certain 5.2 features will not be available until we increment the FCV. MongoDB 5.2 is also available as a Development Release for evaluation purposes only from the MongoDB Download Center. Consistent with our new release cadence announced last year, the functionality available in 5.2 and the subsequent Rapid Releases will all roll up into MongoDB 6.0, our next Major Release scheduled for delivery later this year. Safe Harbour Statement The development, release, and timing of any features or functionality described for our products remains at our sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality.

January 19, 2022
Updates

Utilizing AI and MongoDB to Power Content Personalization

Content is everywhere. Whatever it is a consumer is looking for, and whatever industry that person might be in, it’s not hard to find content. The problem, however, is trying to find the right content. That’s where Concured comes into play. Concured , an AI-startup based in Montreal, helps marketing teams align their website and sales content to audiences, as well as content marketing teams who need to differentiate and accelerate insight-driven content personalization. Concured was founded in 2015 by CEO Tom Salvat and built for content marketers to better understand their audience's interests in order to deliver more impactful content. Built with MongoDB spoke with CTO Tom Wilson , who joined Concured roughly a year after it was founded. We discussed Concured’s use of artificial intelligence, how Wilson joined Concured, as well as what he sees as the future for the company. Built with MongoDB: What does Concured do? Tom Wilson: Concured is a software company that leverages artificial intelligence techniques developed in the last five to 10 years. It is designed to help marketers know what to write about on a per-topic basis, to see what is working well from their own content, as well as that of their competitors, and within their industry. Finally, it maximizes the return on investment they make on any content that they create by personalizing the web visitor’s journey through the client’s website. Concured has developed a recommender system, which is personalized to the individual visitor. It is a privacy friendly approach that doesn’t use third-party cookies or spy on the user, it is purely based on the web visitor’s behavior on the site, and it builds up an interest profile on that visitor as they click through successive web pages. As it builds a more precise picture of the interests and intent of the user, it tries to recommend content to them to read next, whether it’s a blog post, a product description, or any other kind of content. Built with MongoDB: You mentioned artificial intelligence. How does Concured use artificial intelligence behind the scenes? Wilson: We use artificial intelligence in a number of places. One of our unique selling propositions to our clients, is that, unlike other personalized systems, Concured doesn’t require a lengthy integration period, nor does it require ongoing maintenance on the part of the domain. Our approach is to scrape the content of the clients’ websites using AI powered bots to find the content that is relevant and to extract the text, the title, and any other relevant metadata, and to index that in an automatic way. Our system leverages recent advances in natural language processing (NLP) in order to generate semantic metadata for each document, corresponding to a unique point within a multi-dimensional space. Another aspect is to understand the key entities that are in it, and also to understand the relationship of that particular article to the other ones within the same website. We create a lot of metadata automatically on each piece of content that we find using AI powered web crawlers and scrapers. Built with MongoDB: Since AI isn’t always 100% accurate, what is the accuracy for the NLP that you’ve created at Concured? Wilson: With recommender systems, it’s very hard to say what the best recommendation is, because it could be very different for the same person depending on the day, or the web session. If you think of some of the most famous recommender systems, such as Netflix, Amazon, or Spotify, they try to show us what we want to see next, but there’s actually no single right answer. Because of that, it’s very hard to measure the performance, so the approach that we use is not to say that there is a 100% correct answer, but rather, to say, if we make changes to the algorithms, do the visitors on these websites click on more articles, and do they reach the target pages defined by the website’s owner, e.g. a product page or a sign-up form. The higher proportion of people performing that action at the end of the journey on the website relative to the number of the people who arrived on the website, the more successful the recommender system is, and we can compare the success rate that our clients have on their websites before and after they activate the personalization system from Concured. So far we’re looking at a two to three times uplift, and the algorithms are improving all the time. Built with MongoDB: When did you join the team at Concured? Wilson: At a time when the company had raised its first serious money from external investors, and one of the requirements was that they brought on a professional CTO, this often happens in early stage companies, the investors want to impose some structure, they want to know that their money is going to be spent wisely, and do things a little less shooting from the hip. And so, in some companies they joke about bringing in adult supervision. I don’t know if I’d say that about my role, since the team was already nicely established, but I was able to provide a lot more structure to ensure that we would meet our next milestones, as well as longer-term strategic planning and a technical vision. Built with MongoDB: How did your team decide to build with MongoDB? Wilson: The team that I joined was already using MongoDB at the time. Within a few months of joining, there was some discussion about whether to move to a structured database. That was a decision that had to be made, so that’s where I got involved and it became a conscious decision not to move away from MongoDB. This was the right fit for what we wanted going forward, and we absolutely made the right decision. We are also going to move away from our Community edition hosted on Google Cloud Platform to MongoDB Atlas Serverless. We’ll be happy not to manage machines any more, thanks to the serverless aspect, and we’re also excited to try out the text search features available on Atlas, since this could potentially simplify our tech stack. For where we are, as a business today, and where we want to be in five years, MongoDB is and will continue to be the right choice. Built with MongoDB: What does the future of Concured look like? Wilson: The future is being written as we speak. It’s bringing on more clients with similar needs to those of our largest clients at the moment, namely enterprises that have a huge amount of content, already in the archives, that they would like to continue to squeeze value out of, as well as publishing a lot; whether it’s big companies, such as consultancies or financial services industries, or traditional publishers, you want to make sure that they’re promoting the right content that’s going to have the biggest uplift in terms of whatever KPIs they’re measuring. Built with MongoDB: What is the best piece of feedback that you’ve received? Wilson: One nice piece of feedback I’ve received from my team is that I’ve always been there for them. If they have a problem, I will either fix it or remove obstacles for them so they can do the best that they can. It’s part of my philosophy that if you take care of the team, a lot of other things will take care of themselves. For any business that invests in content as part of its marketing strategy, it makes only good business sense to try to maximize the return on that. Learn more about how to turn readers into customers via Content Personalization on Concured’s website . Interested in learning more about MongoDB for Startups? Learn more about us here .

January 19, 2022
Applied

Manufacturing at Scale: MongoDB & IIoT

In recent years, we’ve seen a massive shift in digital transformation. As people, we’re all “connected”, by our smart devices, smart homes, smart cars, smart cities, and so on. We interact with smart devices because of the convenience it offers us- automating daily tasks and giving insight into daily movements, from how much pressure is in each car tire to whether we left our stove on before leaving the house. The reasons we use smart devices is mirrored in why businesses are adopting IoT and IIoT (Industrial IoT) on a much larger scale- for convenience, insight, predictive maintenance, automation, and productive efficiency. IoT is becoming increasingly more critical in manufacturing and engineering, connecting thousands of sensors and actors of devices in the processes before, during, and after fabrication. The implementation of IoT within manufacturing processes, from raw materials to the product or smart products has only just begun and is destined to evolve into a key differentiator for successful manufacturing companies throughout the entire supply chain. The digital transformation in IIoT comes down to data at its core and how data is generated, stored and analyzed. IIoT requires data to be collected and processed at massive volumes in real/ near real time to provide accurate and live business insights for better decision making. Shop floors are continuously optimized as new components, sensors and actors are introduced to improve OEE (Overall Equipment Effectiveness), increase quality and reduce waste. With almost every new device, additional data variety is introduced and thus requires a flexible, highly available and scalable data platform to store and analyze this data. Furthermore, with the increasing convergence of IT and OT, even more complexity and diverse data needs to be integrated and processed, which adds higher complexity to the picture. MongoDB’s general purpose data platform, allows manufacturers to store OT / time series data sets in MongoDB together with recipes or digital twins for a complete real time end-to-end picture from edge to cloud and onto mobile devices for insights and control anytime and anywhere, online or offline. A connected factory model The Industry Solutions team at MongoDB set out to demonstrate how easily MongoDB can be integrated to solve the digital transformation challenges of IIoT in manufacturing with its flexible, scalable, application data platform. Using the small scale model of a smart fabrication factory, from Fischertechnik , the team collects and sends data via MQTT and processes it in MongoDB Atlas and Realm . Similar to a full scale physical factory, the smart factory model demonstrates how easily IIoT use cases can be built on top of MongoDB’s application data platform to enable and accelerate digitalization of manufacturing processes in any industry. The Fischertechnik model factory itself is often used to train engineering students in the field of what a manufacturing fabrication would look like, as well as a model for manufacturing companies to plan for the very real setup, building, and investment of their factories . So, what initially looks like a toy in robotics gets serious quite quickly. The model factory operates as the foundation of an IIoT use case. It is made up of several components- a warehouse, multiprocessing station, and sorting area. The warehouse is where raw material is stacked and stored, and when triggered, the raw material is retrieved and moved to processing by a mobile crane. From there, the items are sorted by color (i.e. red, white, or blue), to be sent to the correct outbound destination. The process covers ordering and storing of raw material to ordering and manufacturing of end products. Throughout these processes, there are multiple sensors detecting the type and color of the items, as well as environmental aspects like temperature and how much inventory is in stock. A surveillance camera detects motion and sends alerts including photos via MQTT. This simulates the wide variety of data a smart factory would emit in real time for track and trace, monitoring and visualization, alerts and as input for machine learning algorithms. The factory's infrastructure Out of the box, the Fischertechnik factory comes with a few dashboards connected to the Fischertechnik cloud, established via a WLAN router integrated in the factory. These dashboards include a: Customer View: A webshop interface, where a product can be ordered to trigger the supply chain processing Supplier View: A visualization and display of the ordering process of raw material Production View: A visualization of the factory status, production process, and sensor values from the camera and NFC/RFID readers. To emphasize and explain how MongoDB can be leveraged in this picture, the MongoDB team developed additional apps, using JavaScript, ReactJS, and Realm, to integrate and streamline data flows and processes on top of the MongoDB data platform. This included a: MongoDB Realm Order Portal: A ReactJS web application to order new products and track the process of orders. Data Visualization: A visualization of the different data types collected in MongoDB and visualized via MongoDB Charts for insights. Alert Management App: A mobile app leveraging MongoDB Realm and Realm Sync for alert notification and management offline and online. The machines of the factory are controlled by TXT controllers, Linux-based computers which use MQTT to communicate between each other and also with the cloud based applications. There are basically two types of data sent and received via MQTT- commands to trigger an action and streaming of event and time series sensor data. The main TXT controller runs a MQTT broker and replicates selected topics to a HiveMQ MQTT broker in the HiveMQ cloud. From there a redpanda kafka container collects the data streams and inserts them into MongoDB. The data persisted in MongoDB is then visualized via MongoDB Charts for real-time insights. Factory layout connected to data infrastructure The MongoDB Order Portal uses the Realm Web SDK and the serverless GraphQL API. GraphQL is used to pull data from MongoDB Atlas and the web SDK is used to add new orders (insert new documents) into a MongoDB cluster. When a new order is inserted into the Atlas database, an Atlas trigger is executed, which then sends a MQTT message directly to the HiveMQ MQTT broker, alerting the factory to process the order. The HiveMQ broker then replicates the order to the factory for processing. Sending data to the factory Receiving data from the factory is just as simple. The Factory provides a large amount of live data that can be streamed from the factory. In order to receive the data, HiveMQ and Kafka are used. The factory has an MQTT broker, which is bridged to a cloud HiveMQ broker. From the HiveMQ broker Kafka Connect with an MQTT source and a MongoDB sink connector the data is moved into MongoDB Atlas. Receiving data from the factory MongoDB & IIoT Digitalization in manufacturing means connecting IT and OT, mixing and meshing data from both domains and providing access to people and algorithms for higher levels of automation, increased efficiency and less waste. MongoDB’s application data platform is the data platform optimized for large varieties and amounts of data with a powerful query language for better decision making across a large volume of data. All easier said than done. However, Atlas helps solve these complex requirements with its ecosystem of functions, including: Real Time Analytics: As IIoT continues to boom with a surge of connected devices, limits are pushed each day with increased volumes of data. Atlas scales seamlessly, capable of ingesting enormous amounts of sensor and event data to support real time analysis for catching any critical events or changes as they happen. Dynamic Scalability: MongoDB Atlas and Realm provide automated scalability allowing you to start small and dynamically adapt your clusters with increasing/decreasing demand. Especially as sensor data gets colder over time, you can automatically offload cold data into object stores, such as S3, while maintaining the ability to query the hot and cold data through a single API. Time-Series: MongoDB 5.0 supports Time-Series Data natively through optimized storage with clustered indexes and optimized Time-Series query operators to analyze trends and identify anomalies quickly. The combination of time series data with other data structures such as digital twin models within the same data platform dramatically reduces complexity, development efforts and costs by avoiding additional technologies, ETL processes and data duplication. The MongoDB database can also be deployed next to the shop floor for data collection and analysis, making the shopfloor independent of the cloud. Pre Aggregated or raw data can then be seamlessly replicated or streamed into the public cloud for global views across factories. Additionally Realm, the serverless backend on top of MongoDB Atlas provides easy application and system integrations through REST / MongoDB Data and GraphQL APIs as well as synchronizing data with mobile devices for offline first use cases such as workforce enablement use cases. Atlas, Realm, and IIoT IIOT is an exciting realm (no pun intended) right now, with a massive opportunity for growth and innovation. The next level of innovation requires a resilient multi-functional data platform that reduces complexity, increases developer efficiency and reduces data duplication/integration while scaling elastically with your demand. What the MongoDB team scratched by quickly syncing the smart model factory with Atlas and Realm and iterating on top of that is just a fraction of the innovation we can support within manufacturing use cases. Learn more MongoDB Atlas and Realm, and how major enterprises are using these solutions for their manufacturing and IIoT needs here . This is the first of an IIoT series from MongoDB’s Industry Solutions team. Stay tuned for more to come!

January 19, 2022
Applied

Ready to get Started with MongoDB Atlas?

Start Free