Start on Your Journey to Operationalize AI-Enhanced Real-Time Applications with MongoDB and Databricks
MongoDB and Databricks have succeeded in two complementary worlds: For MongoDB , the focus is making the world of data easy for developers building applications. For Databricks, the focus is helping enterprises to unify their data, analytics, and AI by combining a data lake's flexibility with the openness, performance, and governance of a data warehouse. Traditionally, these operational and analytical functions have existed in separate domains built by different teams and serving different audiences. Though some will pretend a data warehouse can unify such disparate data and systems, the reality is this approach leaves you making false trade-offs where your developers, your data scientists, and, ultimately, your applications and customers suffer. Data warehouses are not designed to serve consumer-facing applications at scale and process machine learning in real time. It takes the unique application-serving layer of a MongoDB database, combined with the scale and real-time capabilities of a lakehouse, such as Databricks, to automate and operationalize complex and AI-enhanced applications at scale. We observed that a large and growing population of joint customers has for years enabled the flow of data between our two platforms to run real-time businesses and enable a world of application-driven analytics, using MongoDB Connector for Apache Spark . So we asked ourselves: How could we make that a more seamless and elegant experience for these customers? Today we're announcing that Databricks now features MongoDB as a data source within a Databricks notebook , thereby enabling data practitioners with an easier, more curated experience for connecting Databricks with MongoDB Atlas data. This notebook experience makes it simpler for enterprises to deliver real-time analytics, handle complex data warehouse/BI workloads, and to operationalize AI/ML pipelines using the MongoDB Spark Connector . In turn, developer and data teams can collaborate more closely on building a new generation of app-driven intelligence. MongoDB and Databricks are committed to further improve our integration in the coming months. In this post, we'll explain how Databricks can be used as a real-time processing layer for data on MongoDB Atlas using the Spark Connector, extending MongoDB's built-in data processing capabilities like our aggregation framework . We'll also cover how to use Databricks' MongoDB notebook to make this even easier. In future posts we'll outline how to use MongoDB Atlas and Databricks Delta Lake to build sophisticated AI/ML pipelines. Live application data plus the data lakehouse MongoDB Atlas is a fully-managed developer data platform that powers a wide variety of workloads - supporting everything from simple CRUD operations to sophisticated data processing pipelines for analytics and transformation - all with a common query interface. With MongoDB Atlas you can isolate operational and analytical workloads using dedicated analytical nodes. Analytics nodes are read-only nodes that can be exclusively targeted by your queries Let's look at an example. Assume you have long-running analytical queries that you want to run against your cluster and your operations team does not want these queries competing for resources with your regular operational workload. To address this, you add an analytics node to your cluster and then target it in your connection string using an Atlas replica set tag. You can connect to the analytical nodes to run sophisticated aggregation queries, BI and reporting workloads using the Atlas SQL interface , visualize your data using MongoDB Charts , or run Spark jobs using MongoDB’s Spark Connector. For more complex data science and warehousing analytical queries, many enterprises choose the Databricks Lakehouse Platform . Enterprises can also benefit from enriching MongoDB data with data from other internal or external sources in the Databricks Lakehouse. The Databricks Lakehouse Platform combines the best elements of data lakes and data warehouses to deliver the reliability, strong governance, and performance of data warehouses with the openness, flexibility, and machine learning support of data lakes. This unified approach simplifies your modern data stack by eliminating the data silos that traditionally separate and complicate data engineering, analytics, BI, data science, and machine learning. With Databricks notebooks, developers and analytics teams can collaboratively write code in Python, R, Scala, and SQL, plus explore data with interactive visualizations and discover new insights. You can confidently and securely share code with co-authoring, commenting, automatic versioning, Git integrations, and role-based access controls. As good as MongoDB and Databricks are on their own, together we offer enterprises the unmatched ability to work with live application data across traditionally separate domains. This ability allows your teams to deliver what we call application-driven analytics . How does this work? Using MongoDB and Databricks together MongoDB and Databricks offer several ways to integrate the two systems, but the primary means is MongoDB’s Spark Connector. The Spark connector can be used within Databricks notebooks to directly query live application data managed in MongoDB collections and then loaded into data frames for further processing. You can also transform and/or enrich this data with data ingested from other sources using SparkSQL. Queries issued by the Spark Connector can be pushed down to MongoDB's aggregation framework and indexes for pre-processing, significantly improving query efficiency (measured in milliseconds not seconds or minutes). Result sets generated from the Databricks notebooks can then be inserted back into MongoDB collections or can be pushed into Delta Lake for long-running analytics and machine learning. Easier integration using Databricks' MongoDB Notebook A Databricks notebook is a web-based interface that contains runnable code, visualizations, and explanatory text in the form of paragraphs. It lets personas, such as data scientists and data engineers, build linked sets of code in different languages and visualize results in a format in which they are used to working. Notebooks are great for collaboration and can be easily iterated on and improved. MongoDB and Databricks created an example notebook that has sample code for: Reading the data from MongoDB Atlas collections as is into Spark dataframes. Pre-processing and filtering the data from Atlas collections using the aggregation framework, before passing into Spark dataframes. Enriching/transforming the data using SparkSQL Writing the enriched data back to the MongoDB Atlas collection. Figure 1: Screenshot of data sources in a Databricks notebook. This notebook can help as an initial template for developers to start building complex transformation jobs on MongoDB data with Databricks platform. Interested in a practical example of how this works? Let's demonstrate how you can run analytics on a sample sales dataset using MongoDB's aggregation framework and visualize it with Charts. The example also explains how you can enrich this data using our Databricks notebook and load that back to MongoDB. Refer to the GitHub repo for the same. Figure 2: Ways to integrate MongoDB and the Databricks Lakehouse Platform. In addition to Spark, MongoDB and Databricks provide seamless integration through shared Cloud Object stores to enable a more traditional data exchange using analytics-optimized formats such as Parquet, as well as event streaming integration using Apache Kafka. Together, MongoDB and Databricks offer unparalleled abilities to unify and process data from disparate systems in real-time. And now with the newly announced Databricks notebooks integration, data engineers and data scientists have an even easier and more intuitive interface to harness MongoDB data for their most sophisticated analytics and AI processing, making real-time applications more intelligent. Conclusion MongoDB Atlas along with Databricks Platform together will help organizations handle the increasing convergence between operational and analytical workloads. This convergence enables application-driven analytics and will help you build smarter applications and derive the right insights in real-time. Reach out to email@example.com to learn more.
MongoDB for Startups Is Now an Exclusive Offer in AWS Activate
Starting a new business isn’t easy. MongoDB and Amazon Web Services (AWS) have worked together for more than a decade to create developer tools that simplify the process of building software, while helping startups get going with minimal resources. MongoDB Atlas launched on AWS in 2016 and is now available in more than 20 regions across the world. By providing an integrated set of database and data services and a unified developer experience, MongoDB Atlas on AWS lets companies at all stages build applications that are highly available, performant at global scale, and compliant with the most demanding security and privacy standards. Now we’re taking our startup commitment to the next level by offering MongoDB for Startups as an exclusive offer in AWS’s Activate startup program. MongoDB for Startups MongoDB for Startups helps early-stage companies get started with MongoDB by providing access to MongoDB’s cloud developer data platform, including credits and support on MongoDB Atlas , Atlas Search , MongoDB Realm , and more. Additionally, MongoDB provides dedicated technical advice, shared marketing opportunities, and partnership opportunities to help startups grow their customer base. AWS Activate and exclusive offers AWS Activate also provides startups with a host of benefits, including AWS credits to purchase services, AWS support plan credits to use toward technical help from an expert, and architecture guidance to help grow your business. These Activate benefits offer the right mix of tools, resources, and expert support to help you succeed with AWS, while also optimizing performance, managing risk, and keeping your costs under control. Additionally, within the AWS Activate Console , startups will see member-only exclusive offers and find direct access to MongoDB for Startups. Next steps Learn more about the MongoDB for Startups program, and, if you’re already an AWS Activate startup participant, be sure to check out exclusive offers in the Activate Console for the specific benefits of this partnership. You can try MongoDB Atlas (Pay as You Go) on AWS Marketplace while we review your startup application. To ask us questions in real time, register for our joint webinar covering best startup practices with MongoDB and AWS.
A Guide to Freeing Yourself from Legacy RDBMS
Oracle introduced the first commercial relational database (RDBMS) to the market in 1979 — more than a decade before the World Wide Web. Now, digital transformation is reshaping every industry at an accelerating pace. In an increasingly digital economy, this means a company's competitive advantage is defined by how well they build software around their most critical asset — data. MongoDB and Palisade Compliance have helped some of the largest and most complex Oracle customers transform their architecture and shift to a cloud-first world. Although every client is unique, we have identified three important steps to moving away from Oracle software, reducing costs, and achieving their digital transformation goals: Understand your business and technical requirements for today and tomorrow, and identify the technical solution and company that will be by your side to help future-proof your organization. Decipher your Oracle contracts and compliance positions to maximize cost reduction initiatives and minimize any risks from Oracle audits and non-compliance that may derail your ultimate goals. Mobilize internal momentum and traction to make the move. MongoDB can help with #1, Palisade Compliance assists with #2, and you have to supply #3. This is a guide to getting started, as outlined by the main pillars of success above. 1. Understand your requirements and find the right partner — MongoDB The most common requirements we hear from organizations are that they need to move faster, increase developer productivity, and improve application performance and scale -- all while reducing cost and breaking free from vendor lock-in. For example , to keep pace with demands from the business, Travelers Insurance modernized its development processes with a microservices architecture supported by agile and DevOps methodologies. But the rigidity of its existing Oracle and SQL Server databases created blockers to move at the speed they needed. The solution was MongoDB and its flexible data model. They eliminated the three-day wait to make any database changes, creating a software development pipeline supporting continuous delivery of new business functionality. Similarly, Telefonica migrated its customer personalization service from Oracle to MongoDB. Using Oracle, it took 7 developers, multiple iterations and 14 months to build a system that just didn't perform. Using MongoDB, a team of 3 developers built its new personalization service in 3 months, which now powers both legacy and new products across the globe. MongoDB helps Telefonica be more agile, save money and drive new revenue streams. While some organizations try to innovate by allowing siloed, modern databases to coexist with their legacy relational systems, many organizations are moving to fully replace RDBMS. Otherwise, a level of complexity remains that creates significant additional work for developers because separate databases are required for search, additional technologies are needed for local data storage on mobile devices, and data often needs to be moved to dedicated analytics systems. As a result, development teams move slowly, create fewer new features, and cost the organization more capital. MongoDB provides the industry’s first data platform that allows you to accelerate and simplify how you build with data for any application. Developers love working with MongoDB’s document model because it aligns with how they think and code. The summarized functional requirements that we typically hear from leading companies and development teams regarding what they require from a data platform include: A data structure that is both natural and flexible for developers to work with Auto-scaling and multi-node replication Distributed multi-document transactions that are fully ACID compliant Fully integrated full-text search that eliminates the need for separate search engines Flexible local datastore with seamless edge to cloud sync In-place, real-time analytics with workload isolation and native data visualization Ability to run federated queries across your operational/ transactional databases and cloud object storage Turnkey global data distribution for data sovereignty and fast access to Data Lake Industry-leading data privacy controls with client-side, field level encryption Freedom to run anywhere, including the major clouds across many regions MongoDB delivers everything you need from a modern data platform. But it’s not just about being the right data platform; we’re also the right modernization partner. Through our Modernization Program we have built and perfected modernization guides that help you select and prioritize applications, review best practices, and design best-in-class, production-grade, migration frameworks. We’ve built an ecosystem around accelerating and simplifying your journey that includes: deployment on the leading cloud providers to enable the latest innovations technology companies that help with data modeling, migration, and machine learning, and expert System Integrators to provide you with tools, processes and support to accelerate your projects. We are proud to be empowering development teams to create faster and develop new features and capabilities, all with a lower total cost of ownership. 2. Manage Oracle as you move away — Palisade Compliance Oracle’s restrictive contracts, unclear licensing rules, and the threat of an audit can severely impact a company’s ability to transform and adopt new technologies that are required in a cloud-first world. To move away from Oracle and adopt new solutions, companies must be sure they can actually reduce their costs while staying in compliance and avoiding the risks associated with an audit. There will be a time when you are running your new solution and your legacy Oracle software at the same time. This is a critical phase in your digital transformation as you do not want to be tripped up by Oracle’s tactics and forced to stay with them. It may seem counterintuitive, but as you spend less with Oracle you must be even more careful with your licensing. As long as you keep spending money with Oracle and renewing those expensive contracts, the threat of an audit and non-compliance will remain low. Oracle is unlikely to audit a company that keeps giving it money. However, the moment you begin to move to newer technologies, your risk of an audit significantly increases. As a result, you must be especially vigilant to prevent Oracle from punishing you as you move away from them. Even if you’ve found a technical partner and managed your Oracle licenses and compliance to ensure no surprises, you still have to find a way to reduce your costs. It’s not as simple as terminating Oracle licenses and seeing your support costs go down. As stated above, Oracle contracts are designed to lock in customers and make it nearly impossible to actually reduce costs. Palisade Compliance has identified eleven ways to manage your Oracle licenses and reduce your Oracle support. It is critical that you understand and identify the options that work for your firm, and then build and execute on a plan that ensures your success. 3. Mobilize internal momentum and traction to make the move Legacy technology companies excel at seeding doubt into organization and preventing moves that threaten their antiquated solutions. Unfortunately, too many companies succumb to these tactics and are paralyzed into a competitive disadvantage in the market. In software, as in life, it’s easier to stay the course than to follow through with change. But when it comes to technical and business decisions that impact the overall success and direction of an organization, innovation and change aren’t just helpful, they’re necessary to survive--especially in a world with high customer demands and easy market entry. Ensuring you have the right technical partner and Oracle advisor is the best way to build the confidence and momentum needed to make your move. Creating that momentum is easier with MongoDB’s Database Platform, consisting of a fully managed service across 80+ regions, and Palisade’s expertise in Oracle licensing and contracts. Technical Alternative (MongoDB) + Independent Oracle Advisors (Palisade) ⇒ Momentum Parting thoughts To schedule a preliminary health check review and begin building the right strategy for your needs, fill out your information here . And to learn more about MongoDB’s Modernization Program, visit this page . About Palisade Compliance With over 400 clients in 30 countries around the world, Palisade is the leading provider of Oracle-independent licensing, contracting, and cost reduction services. Visit the website to learn more. To schedule a complementary one-hour Oracle consultation send an email to firstname.lastname@example.org.
How Legacy Modernization with WeKan and MongoDB Atlas Helps Meet Evolving Consumer Demands
COVID-19 has accelerated the growth and adoption of digital economies across the globe, and the businesses best positioned to keep pace with related changes in consumer behavior and demand will continue to gain a competitive advantage in the marketplace. According to a consumer study by FIS Global that surveyed participants to understand changes in recent buying behaviour and patterns, consumers have spent 58% more money online since the pandemic started. What’s more, 42% of respondents stated an increase in purchases from local/independent small businesses, and 27% of consumers have subscribed to one new online streaming platform. Large institutions and household brands can not risk complacency if they want to maintain market share. The customer loyalty of today will be captured by the companies that act with agility and optimize data to deliver the most seamless, custom experience for consumers. Unsurprisingly, business models that have prioritized and directed resources towards aligning their processes with digital transformation are better placed to deal with customer behaviour shifting into the digital realm. And yet, innumerable businesses are plagued by the limitations of their legacy IT systems when trying to modernize their digital experience. For many organizations, legacy systems are seen as holding back the business initiatives and business processes that rely on them...when a tipping point is reached, application leaders must look to application modernization to help remove the obstacles Stefan Van Der Zijden, VP Analyst, Gartner Continued use of these systems holds back businesses’ potential for revenue generation and building customer-facing credibility; but modernizing them reaps worthy rewards. Luckily, you don’t have to go it alone. This post will detail how organizations can undertake this modernization process, often termed “legacy modernization,” so as to leverage the speed, agility, and responsiveness required to succeed in a digital-first marketplace. What is legacy modernization? Legacy modernization refers to the process of updating an organization’s antiquated IT stack to align with new-age business goals and workflows. To drive innovation, business leaders need to be supported by technology that can help implement their goals in the real world. They need fast-paced, highly connected systems with minimal-to-zero downtime, and platforms or dashboards that provide cohesive and easily comprehensible views of the entire ecosystem. Generally, legacy IT stacks are incapable of meeting these standards which is where legacy modernization comes in. Defining legacy systems and 4 major drawbacks Essentially, a legacy system is any software or technological system that slows down an organization’s business growth and its ability to shift and adapt to changing market forces. If a software setup is unable to integrate with newer systems, workflows or processes, it qualifies as “legacy.” Generally the incompatibility of legacy technologies, and the bottlenecks that come with them, lead to major issues related to maintenance, support, updates, integration and overall user experience. Think of it this way: using a legacy system in 2021 is comparable to driving a Prius with an engine made in 2000. Legacy solutions lack flexibility and carry a significant technology debt due to dated languages, databases, architectures, and a limited supply of aging baby-boomer programmers. a Deloitte Study on Legacy Systems and Modernization The business impact of legacy systems are varied, but often adverse. They include: 1. Inability to act with agility and meet demand Generally, legacy systems can only be accessed from office computers. But in a digital-first world, mobile devices are at the core of digital transformation. If employees cannot access necessary software from anywhere at any time, their productivity and operational capacity is severely limited. The link between software and employee performance has, in fact, been well-documented . For instance, in 2015 , a computer running a 23 year-old operating system (Windows 3.1) caused planes to be grounded at Paris’ Orly airport for several hours. Needless to say, customers were not happy. 2. Decreased employee productivity and customer satisfaction Everyday people are at the heart of digital transformation. If a business wants to attract and retain customers who are increasingly reliant on their internet-powered mobile devices for day-to-day activities and transactions, they have to meet them online. And if they want to attract top talent, they need to equip their employees with the tools and agility needed to innovate. Being saddled with legacy systems will prevent companies from using newer apps and providing the best possible customer service, support and experience. Additionally, sub-par employee performance and customer service will inevitably cause financial loss due to unsatisfied customers and missed opportunities for expansion. 3. Scalability issues and security risks Legacy software is usually incapable of scaling up, which poses major obstacles to business growth. In a competitive marketplace, businesses must be able to shift strategy and optimize according to market forces, for which they need the support of their IT stack. An excellent example of this is how companies have had to adapt to remote work becoming the ‘new normal’ due to the global pandemic. The IBM 2020 Cost of a Data Breach Report puts the average cost of a data breach at USD 3.92 million. Legacy software almost always has glaring flaws in its security mechanisms for multiple reasons: withdrawal of manufacturer support, lack of updates and regular maintenance, difficulties in fixing vulnerabilities within outdated systems. Issues like security breaches will significantly harm brand credibility and repel customers from entrusting the business with their data. 4. Higher costs Administrative, support and maintenance costs are unnecessarily high when companies have to work with legacy software. Additionally, hiring and training new employees, especially developers, is difficult since there is a shortage of coders trained in legacy languages like COBOL and Natural. Most legacy systems are hosted on premise, which translates to enormous and unnecessary overhead related to maintenance and upgrades. These costs are easily eliminated by leveraging cloud computing platforms like AWS, Google Cloud, or Microsoft Azure. Despite these glaring inadequacies, the pandemic has revealed how far too many organizations continue to rely on aging IT systems. In a 2020 AppDynamics Report , 66% of technologists say “the pandemic has exposed weaknesses in their digital strategy, driving an urgent need to push through initiatives which were once a part of multi-year digital transformation programs.” A roadmap for legacy modernization The journey to legacy modernization can be an intensive one, but there are proven best practices and expert guidance to help you get started. Galvanize the key players in your organization and get started by asking the right questions: What resources can be assigned to the modernization endeavour? Do your employees possess the skills to operate the new systems? What are the specific competitive advantages that modernization needs to provide for your organization? Is there a separate support and retirement schedule in place for your legacy system? Should modernization occur in a single shift or in phases? How will this affect our business? Escaping the pressures imposed by unwieldy tech stacks has become possible with microservices and cloud-based application development and/or usage. The trick lies in decentralizing business tech offerings by migrating them from Relational Database Management Systems (RDBMS) to the Cloud via scalable solutions like MongoDB Atlas , MongoDB's hosted database-as-a-service offering. Moving from monolith to microservices architecture can be complex, but offers multiple long-term advantages across multiple parameters. Refactoring monolithic systems requires carefully constructed strategies, the most successful of which are drawn from the Strangler Pattern approach . How do we modernize from existing legacy systems? Initiate new functionalities as microservices: Every time a business has to implement a new functionality or feature, they can incorporate it as a microservice instead of adding it to the existing monolith architecture. Not only does this prevent the legacy stack from expanding, but allows stakeholders to become acquainted with the advantages of microservice ecosystems. Dismantle the monolith: Once microservices have been introduced into an organization’s ecosystem, monolith structures need to be deconstructed for eventual elimination. Companies like FedEx and CitiBank have attested to the success of a microservices-based strategy with real world implementation. To quote FedEx CIO Rob Carter , “We began to build out the services and microservices that represent the less complex, more flexible, faster-to-market capabilities that we have today.” CitiBank, too, opted for migrating its monolith system to a microservices-based architecture so as to accelerate digital transformation. How WeKan and MongoDB Atlas can help Implementing successful, sustainable and scalable legacy modernization requires expertise in executing on the process itself, as well as the right tools that can understand and adapt to an organization’s unique needs and business goals. Databases and platforms like MongoDB and its tool suite help address the challenges of replatforming from monolith to microservice. MongoDB Atlas is the leading choice of general purpose databases for modernization. As a document-based, distributed database, MongoDB reduces time spent on development cycles and empowers developers with flexible schema and the tools they need to maintain productivity. A leap forward from traditional RDBMS, MongoDB Atlas's smart infrastructure helps organizations scale effortlessly and maintain business-critical reliability while driving lower TCO, reducing security risk, and remaining ACID compliant. Complementary to MongoDB, WeKan’s Modernization process is composed of 5 phases that aim to scope an optimal modernization journey for any business operating on legacy systems and looking for a better return on their technology investment: Diagnosis phase – The first step is to understand the current state of the business, its most critical pain points and identify major inefficiencies that can be solved through technology modernization. Prescription phase – With a good understanding of the business’ state, we propose reference solution architectures that can address most critical pain points and enhance overall performance of their technology ecosystem with a focus on always reducing the total cost of ownership (TCO) and increasing ROI on their technology spend. Validation phase – After gathering potential solutions, we then validate through POCs their tech viability, expected outcomes and leverage results from these efforts to narrow down and select the option that is best suited to the business’ needs. Requirements definition phase – With a target solution in hand, we work on defining the technical requirements and specifications of the proposed solution to ensure seamless integration to the overall technology ecosystem. Execution and Implementation phase – With the right solution architecture, technical requirements in place, and a proposed modernization plan, our modernization consultants work hand-in-hand with internal stakeholders on the development, testing, delivery and implementation of the proposed modernized solution. According to the World Economic Forum, digital transformation could generate more than $100 trillion by 2025 . Without legacy modernization, businesses will miss out on tapping into revenue streams offered by the digital economy. It is integral for organizations to leverage the many advantages of modernization so that they may gain and retain a competitive edge in a constantly connected and perpetually online marketplace. To learn more about WeKan and MongoDB Atlas's efficacy in organization-centric digital transformation, refer to our case study with RideKleen. After migrating operations to AWS, WeKan chose MongoDB Atlas, Atlas Data Lake and MongoDB Realm as their central data platform. Atlas offers a fully managed cloud database service with built-in automation, Atlas Data Lake provides federated query capabiliites to natively data query across MongoDB and AWS S3, while MongoDB Realm simplifies the critical edge-to-cloud sync and provides backend services to speed development work, including triggers, functions, and GraphQL. RideKleen case study Watch how MongoDB’s industry-best modernization services helped OTTO, Germany’s #2 global e-commerce provider and #1 site for e-commerce, fashion and lifestyle. Learn more about our Modernization Program Learn more about WeKan
Build Better Mobile Apps -- Running MongoDB Realm and Google Cloud
MongoDB Atlas Connector for Apache Spark is Certified for Azure Databricks
We are happy to announce that the MongoDB Connector for Apache Spark is now officially certified for Microsoft Azure Databricks . Databricks, founded by the original creators of Apache Spark, provides the Databricks Unified Analytics platform . MongoDB Atlas users can integrate Spark and MongoDB in the cloud for advanced analytics and machine learning workloads by using the MongoDB Connector for Apache Spark which is fully supported and maintained by MongoDB. The MongoDB Connector for Apache Spark exposes all of Spark’s libraries, including Scala, Java, Python, and R. MongoDB data is materialized as DataFrames and Datasets for analysis with machine learning, graph, streaming, and SQL APIs. The MongoDB Connector for Apache Spark can take advantage of MongoDB’s aggregation pipeline and rich secondary indexes to extract, filter, and process only the range of data it needs – for example, analyzing all customers located in a specific geography. This is very different from simple NoSQL data stores that do not offer secondary indexes or in-database aggregations and require the extraction of all data based on a simple primary key, even if only a subset of that data is needed for the Spark process. This results in more processing overhead, more hardware, and longer time-to-insight for data scientists and engineers. Additionally, MongoDB’s workload isolation makes it easy for users to efficiently process data drawn from multiple sources into a single database with zero impact on other business-critical database operations. Running Spark on MongoDB reduces operational overhead as well by greatly simplifying your architecture and increasing the speed at which analytics can be executed. MongoDB Atlas, our on-demand, fully-managed cloud database service for MongoDB, makes it even easier to run sophisticated analytics processing by eliminating the operational overhead of managing database clusters directly. By combining Azure Databricks and MongoDB, Atlas users can make benefit of a fully managed analytics platform, freeing engineering resources to focus on their core business domain and deliver actionable insights quickly. What's Next? Get started now with MongoDB Atlas on Azure Download the connector from Maven Central Take a deep dive into the project on GitHub Learn more about the Spark Connector in our documentation
MongoDB Radio: Our New Podcast Project
This is for our previous Podcast series. We've launched a new one since this post went live. Find out more information here: https://www.mongodb.com/blog/post/mongodb-podcast-has-launched . Welcome to the inaugural post of MongoDB Radio, our new podcast project. We’re very excited to bring you great content about MongoDB, the people who build it, and the people who use it. Throughout this series we will feature interviews with MongoDB engineers, experts in the field of distributed computing and databases, stories from our community and trends in technology, and much more. The world of distributed systems and next generation applications is a fascinating place, and we can’t wait to share it with you. In episode one we spent time with Luke Lovett, a software engineer on the driver’s integration team at MongoDB. Among many things, Luke is responsible for maintaining one of our most popular projects – the Hadoop connector for MongoDB. The connector allows you to plug MongoDB into the Hadoop ecosystem of tools and perform sophisticated processing against the data within MongoDB. We spoke with Luke during our developer conference in San Jose, where he was delivering a talk on some of the new features available on the connector. We discussed the connector in depth, what it’s like to work on an open source project with the community, and how he got started at MongoDB. Join us for two days of GIANT thinking. Learn more about MongoDB World About the Author - Bryan Reinero Bryan is US Developer Advocate at MongoDB fostering understanding and engagement in the community. Previously Bryan was a Senior Consulting Engineer at MongoDB, helping users optimize MongoDB for scale and performance and a contributor to the Java Driver for MongoDB. Earlier, Bryan was Software Engineering Manager at Valueclick, building and managing large scale marketing applications for advertising, retargeting, real-time bidding and campaign optimization. Earlier still, Bryan specialized in software for embedded systems at Ricoh Corporation and developed data analysis and signal processing software at the Experimental Physics Branch of Ames Research Center.
Parse Shutdown: How to Seamlessly Continue Operations with Azure and MongoDB Cloud Manager
It has now been a few months since Parse announced their shutdown, and we at MongoDB are working to develop resources and options for users interested in migrating to other environments. We previously announced our guide to migrating from Parse to AWS and MongoDB Cloud Manager and are now adding a comprehensive guide to migrating a Parse backend to Azure with Cloud Manager . Along with Parse’s moving on announcement , they released Parse Server, an open source replacement for their hosted backend. Along with Parse Server, they have provided a practical migration path from their hosted solution to a private one. To use this solution, a team needs to provision a private MongoDB database, migrate their Parse data to that MongoDB, deploy the Parse Server into a hosting environment of their choosing, and update their client to issue API calls to their new Parse Server. MongoDB Cloud Manager’s tight integration with Azure makes it easy to setup a MongoDB deployment. This guide is for mobile developers and does not assume any prior experience with MongoDB, Node.js, or Azure. Each section provides a complete, high-level checklist with detailed, step-by-step directions. In addition, there are great jumping-off points to the specific MongoDB documentation relevant to those migrating from Parse, such as how to manage indexes and set up monitoring. Migrate from Parse to MongoDB and Azure
Announcing MongoDB's Giant of the Month, Doug Duncan of Alteryx
MongoDB customers and community members are the people who realize GIANT ideas. We are excited to begin highlighting some of our community members, our MongoDB Giants, who are tackling challenging problems and bringing solutions to life with MongoDB. This month’s MongoDB Giant is Doug Duncan, a DBA at Alteryx , who is making a meaningful impact to the MongoDB community. Alteryx is a leader in data preparation, blending, and advanced analytics. Doug has worked with RDBMSs for longer than he cares to admit, focusing in both development and administration. Through his experience, Doug grew to embrace new data storage technologies. He began working with MongoDB several years ago (starting with the MongoDB 1.8 release in early 2011) both professionally and recreationally. Doug acted as an online TA the first two years following MongoDB University’s founding, working closely with the team by answering students’ questions about the M101J , M101JS and M202 courses, as well as providing questions used in the courses. For his contributions to the education team Doug was awarded the first ever MongoDB DBA certification back in November of 2013. Doug also periodically answers questions on the MongoDB Google group . This year he will increase his participation in the community by delivering talks at the Denver MongoDB User Group . In his spare time, if Doug’s not reading up on MongoDB, Hadoop, or other distributed data stores, you can find him walking around the foothills of Colorado with his wife, two boys and two dogs. Become an important part of the MongoDB community. Join our Advocacy Hub and start getting involved today. Join the Advocacy Hub