MongoDB 3.4.1 is out and is ready for production deployment. This release contains only fixes since 3.4.0, and is a recommended upgrade for all 3.4 users.
Fixed in this release:
- SERVER-27124 Disallow readConcern:majority reads on pv0
- SERVER-27201 $graphLookup triggers null pointer dereference
- SERVER-27207 Find on view with sort through mongos may incorrectly return empty result set
- SERVER-27213 Two $match stages combine incorrectly, yielding incorrect results.
- SERVER-27300 Disallow indexing of BSONType::Symbol with a non-simple collation
- SERVER-27210 3.4.0 mongo shell unable to connect using MongoURI with "ssl=true"
- SERVER-27271 rolesInfo command raises System.InvalidOperationException : Duplicate element name 'roles'.
- SERVER-26870 Sometimes collection data file is not removed even though collection is dropped
- TOOLS-1541 Support exporting views
As always, please let us know of any issues.
-- The MongoDB Team
What’s New in MongoDB 3.4, Part 1: Multimodel Done Right
MongoDB 3.4 is now Generally Available (GA) and ready for production deployment ! MongoDB 3.4 is the latest release of the industry’s fastest growing database. It offers a major evolution in capabilities and enhancements that enable you to address emerging opportunities and use cases. This 3-part blog series aims to help you navigate everything that is new, and provides the most important resources to get you started: Part 1 covers the extended multimodel capabilities of MongoDB 3.4, including native graph processing, faceted navigation, rich real-time analytics, and powerful connectors for BI and Apache Spark. Part 2 discusses enhanced capabilities to support mission-critical applications, including geo-distributed MongoDB zones, elastic clustering, tunable consistency, and enhanced security controls. Part 3 concludes with modernized DBA and Ops tooling available in the new release. If you want to get the detail now on everything the new release offers, download the What’s New in MongoDB 3.4 white paper . Why Multimodel? Rather than the monolithic codebases of the past, today’s applications are increasingly being decomposed into loosely coupled suites of microservices, each implementing specific functionality within an application. Different services can place very different demands on the database used – from simple key-value lookups to complex analytics, aggregations, and graph traversals, through to rich search queries. Some data may need to be stored only in-memory for predictable low latency, while other data sets may need to be encrypted on disk for regulatory compliance. Data sets may vary from billions of small records, each just several KBs in size, to the management of large, multi-MB objects. To try and tame the complexity that would come from using a multitude of storage technologies, the industry is moving towards the concept of “multimodel” databases. Such designs are based on the premise of presenting multiple data models within the same platform, thereby serving diverse application requirements. However, many self-described multimodel database are little more than a collection of discrete technologies for data storage, search, and analytics, each with its own domain specific language, API and deployment requirements, and working on its own copy of the data. This approach to multimodel fails to offer much of an improvement over just running multiple independent databases, imposing high complexity, overhead, friction, and cost for developers and operations teams. MongoDB Takes a Different Approach How does MongoDB differ? MongoDB’s flexible document data model presents a superset of other database models. It allows data be represented as simple key-value pairs and flat, table-like structures, through to rich documents and objects with deeply nested arrays and sub-documents. With an expressive query language, documents can be queried in many ways – from simple lookups to creating sophisticated processing pipelines for data analytics and transformations, through to faceted search, JOINs and graph traversals. With a flexible storage architecture, application owners can deploy storage engines optimized for different workload and operational requirements. MongoDB’s approach to delivering multimodel significantly reduces developer and operational complexity, compared to running multiple, separate technologies to satisfy diverse application requirements. Users can leverage the same MongoDB query language, data model, scaling, security, and operational tooling across different applications, all within a single, integrated database platform. MongoDB 3.4 introduces new native graph processing, faceted navigation, multi-language collations, additional aggregation pipeline operators, a new decimal data type, along with enhanced connectors for BI and Apache Spark integration. Native Graph Processing Applications storing data in MongoDB frequently contain data that represents graph or tree type hierarchies. These connections can be as simple as a management reporting chain in a HR application, or as complex as multi-directional, deeply nested relationships maintained by social networks, master data management, recommendation engines, disease taxonomy, fraud detection, and more. While special purpose graph databases are effective at storing and querying graph data, it’s often desirable to store and traverse graph data directly in MongoDB. Here it can be processed, queried, and analyzed alongside all other operational data in real time, without the complexity of duplicating data across two separate databases. Graph and hierarchical data is commonly queried to uncover indirect or transitive relationships. For example, if company “A” is owned by company “B”, and “B” is owned by company “C”, then “C” indirectly owns company “A”. MongoDB 3.4 offers this functionality via a new aggregation stage called $graphLookup to recursively lookup a set of documents with a specific defined relationship to a starting document. Developers can specify the maximum depth for the recursion, and apply additional filters to only search nodes that meet specific query predicates. $graphLookup can recursively query within a single collection, or across multiple collections. Review the documentation to learn more about the MongoDB $graphLookup operator for graph processing . Faceted Navigation Faceting is a popular analytics and search capability that allows an application to group information into related categories by applying multiple filters to query results. Facets allow users to narrow their search results by selecting a facet value as a filter criteria. Facets also provide an intuitive interface to exploring a data set, and allow convenient navigation to data that is of most interest. Most databases need to execute multiple GROUP_BY statements to render facets, resulting in long running queries and poor user experience. MongoDB 3.4 introduces new aggregation pipeline stages for the bucketing, grouping and counting of one or more facets in a single round trip to the database. As a result, developers can generate richer and intuitive experiences to help users navigate complex data sets. Review the documentation to learn more about MongoDB faceted navigation . Multi-Language Collations Applications addressing global audiences require handling content that spans many languages. Each language has different rules governing the comparison and sorting of data. In order to create intuitive, localized user experiences, applications must handle non-English text with the appropriate rules for that language. For example, French has detailed rules for sorting names with accents on them. German phonebooks order words differently than the German dictionary. MongoDB 3.4 significantly expands language support capabilities to allow users to build applications that adhere to language-specific comparison rules. Support for collations – the rules governing text comparisons and sorting – has been added throughout the MongoDB Query Language and indexes for over 100 different languages and locales. Each collation can also be further customized to provide precise control over case sensitivity, numeric ordering, whitespace handling, and more. Developers can specify collation for a collection, an index, a view, or for specific operations that support collation (i.e. find, aggregate, update). You can learn more about collation in MongoDB from the documentation . Aggregation Pipeline Enhancements MongoDB developers and data engineers rely on the aggregation pipeline due to its power and flexibility in enabling sophisticated processing and manipulation demanded by real-time analytics and data transformations. MongoDB 3.4 continues to extend the aggregation pipeline by adding new capabilities within the database that simplify application-side code, as well as optimizer enhancements that improve performance. In addition to the graph and facet features described earlier, many other expressions are added in MongoDB 3.4. These expressions address string manipulation, array handling, type handling, and schema detection and transformation: String handling includes expressions for splitting and manipulating strings either in bytes or code points (a code point can represent a single component of the string, e.g, a character, emoji, or formatting instruction). Array expressions allow more sophisticated manipulation and computations on arrays, including parallel array processing. New expressions allow determining types of fields Case/switch expressions for branching Support for ISO week expressions MongoDB 3.4 also brings additional performance optimizations to the aggregation pipeline. Where possible, the query optimizer automatically moves the $match stage earlier in the pipeline, and combines it with other stages, to increase cases where indexes can be used to filter results sets. In most cases, no modifications of existing queries need to be made. You can learn more about the many MongoDB 3.4 aggregation pipeline enhancements from the documentation . Decimal Data Type Decimal128 is a 16 byte decimal floating-point number format. It is intended for calculations on decimal numbers where high levels of precision are required, such as financial (i.e. tax calculations, currency conversions) and scientific computations. Decimal128 supports 34 decimal digits of significance and an exponent range of −6143 to +6144. MongoDB 3.4 adds support for the decimal data type which represents decimal128 values. Unlike the double data type, which only stores approximations of decimal values, the decimal data type stores the exact value. For example, a decimal type ("9.99") has a precise value of 9.99, while 9.99 represented as a double would have an actual value of 9.99000000000000021316282072803, thus creating the potential for rounding errors when it is used in calculations. Decimal type values are treated like any other numeric type, and compare and sort correctly with other types based on actual numeric value. Operations on decimals are implemented in accordance with the decimal128 standard, so a value of 0.10 will retain its trailing zeros while comparing equal to 0.1, 0.10000 and so on. Review the documentation to learn more about the new MongoDB decimal data type . Visualizing MongoDB Data The MongoDB Connector for BI was introduced in November 2015. For the first time analysts, data scientists, and business users were able to seamlessly visualize semi-structured and unstructured data managed in MongoDB, alongside traditional data from their SQL databases, using the same BI tools deployed within millions of enterprises. Building on its initial release, the Connector for BI has been reengineered to improve performance, simplify installation and configuration, and support Windows. Figure 1: Uncover new insights with powerful visualizations generated from MongoDB Performance and scalability has been improved by moving more query execution down to the MongoDB processes themselves. Queries and complex aggregations are executed natively within the database, thus reducing latency and bandwidth consumption. In addition, installation and authentication has been simplified. Users now authenticate as an existing user already declared within MongoDB, no longer needing to create separate username and password credentials within the connector. The Connector for BI is part of the Advanced Analytics suite available with MongoDB Enterprise Advanced . Review the MongoDB Connector for BI documentation to learn more. MongoDB Connector for Apache Spark Following general availability in June 2016, the MongoDB Connector for Apache Spark has been updated to support the latest Spark 2.0 release. Spark 2.0 support in the connector provides access to the new SparkSession entry point, the unified DataFrame and Dataset API, enhanced SparkSQL and SparkR functionality, and the experimental Structured Streaming feature. The connector exposes all of Spark’s libraries, including Scala, Java, Python, and R. MongoDB data is materialized as DataFrames and Datasets for analysis through machine learning, graph, streaming, and SQL APIs. Already powering sophisticated analytics at organizations including China Eastern Airlines , Black Swan, and x.ai, the MongoDB Connector for Apache Spark takes advantage of MongoDB’s aggregation framework, rich queries, and secondary indexes to extract, filter, and process only the range of data it needs – for example, all customers located in a specific geography. To maximize performance across large, distributed data sets, the connector can co-locate Resilient Distributed Datasets (RDDs) with the source MongoDB node, thereby minimizing data movement across the cluster and reducing latency. You can download the MongoDB Connector for Apache Spark from GitHub, and sign up for a free Spark course from MongoDB University. Next Steps That wraps up the first part of our 3-part blog series. Remember, you can get the detail now on everything packed into the new release by downloading the What’s New in MongoDB 3.4 white paper . Alternatively, if you’d had enough of reading about it and want to get started now, then: Download MongoDB 3.4 Alternatively, spin up your own MongoDB 3.4 cluster on the MongoDB Atlas database service Sign up for our free 3.4 training from the MongoDB University
Revolutionizing Data Storage and Analytics with MongoDB Atlas on Google Cloud and HCL
Every organization requires data they can trust—and access—regardless of its format, size, or location. The rapid pace of change in technology and the shift towards cloud computing is revolutionizing how companies handle, govern and manage their data by freeing them from the heavy operational burden of on-premise deployments. Enterprises are looking for a centralized, cost-effective solution that allows them to scale their storage and analytics so they can ingest data and perform artificial intelligence (AI) and machine learning (ML) operations, ultimately expanding their marketing horizon. This blog post explores why companies should partner with MongoDB Atlas on Google Cloud to begin their data revolution journey, and how HCL Technologies can support customers looking to migrate. MongoDB Atlas as the distributed data platform MongoDB Atlas is the leading database-as-a-service on the market for three main reasons: Unparalleled developer experience - allows organizations to bring new features to market at a high velocity Horizontal scalability - supports hundreds of terabytes of data with sub-second queries Flexibility - stores data to meet various regulatory, operational, and high availability requirements. The versatility offered by MongoDB’s document model makes it ideal for modern data-driven use cases that require support for structured, semi-structured, and unstructured content all within a single platform. Its flexible schema allows changes to support new application features without costly schema migrations typically required with relational databases. MongoDB Atlas extends the core database by offering services like Atlas Search and MongoDB Realm that are a necessity for modern applications. Atlas Search provides a powerful Apache Lucene-based full text search engine that automatically indexes data in your MongoDB database without the need for a separate dedicated search engine or error-prone replication processes. Realm provides edge-to-cloud sync and backend services to accelerate and simplify mobile and web development. Atlas’ distributed architecture supports horizontal scaling for data volume, query latency, and query throughput which offers the scalability benefits of distributed data storage alongside the rich functionality of a fully-featured general purpose database. MongoDB Atlas is unique in its ability to provide the most wanted database as a managed service and is relied on by the world’s largest companies for their mission-critical production applications. Innovation powered by collaboration with HCL Technologies MongoDB’s versatility as a general-purpose database, in addition to its massive scalability, makes it a perfect foundation for analytics, visualization, and AI/ML applications on Google Cloud. As an MSP partner for Google Cloud, HCL Technologies helps enterprises accelerate and risk-mitigate their digital agenda, powered by Google Cloud. We’ve successfully implemented applications leveraging MongoDB Atlas on Google Cloud, building upon MongoDB’s flexible JSON-like data model, rich querying and indexing, and elastic scalability in conjunction with Google Cloud’s class-leading cloud infrastructure, data analytics, and machine learning capabilities. HCL is working with some of the world’s largest enterprises in building secure, performant, and cost-effective solutions with MongoDB and Google. Possessing technical expertise in Google Cloud, MongoDB, machine learning, and data science, our dedicated team developed a reference architecture that ensures high performance and scalability. This is simplified by MongoDB Atlas’ support for Google Cloud services which allows it to essentially operate as a cloud-native solution. Highlighted features include: Integration with Google Cloud Key Management Service Use of Google Cloud’s native storage snapshot for fast backup and restore Ability to create read-only MongoDB nodes in Google Cloud to reduce latency with Google Cloud-native services regardless of where the primary node is located (even other public cloud providers!) Integrated billing with Google Cloud Ability to span a single MongoDB cluster across Google Cloud regions worldwide, and more As represented in Figure 1 below, MongoDB Atlas on Google Cloud can be used as a single database solution for transactional, operational, and analytical workloads across a variety of use cases. Figure 1: MongoDB's core characteristics and features The following architecture in Figure 2 demonstrates the ease of reading and writing data to MongoDB from Google Cloud services. Dataflow, Cloud Data Fusion, and Dataproc can be leveraged to build data pipelines to migrate data from heterogeneous databases to MongoDB and to feed data to create interactive dashboards using Looker. These data pipelines support both batch and real-time ingestion workloads and can be automated and orchestrated using Google Cloud - native services.. Figure 2: MongoDB Atlas' integration with core Google Cloud services A data platform built using MongoDB Atlas and Google Cloud offers an integrated suite of services for storage, analysis, and visualization. Address your business challenges with HCL: Industry use cases Data-driven solutions built with MongoDB Atlas on Google Cloud have multiple applications across industries such as financial services, media and entertainment, healthcare, oil and gas, energy, manufacturing, retail, and the public sector. Every industry can benefit from this highly integrated storage and analytical solution. Use Cases and Benefits Data lake modernization with low cost and high availability for media and entertainment customers: Maintaining high availability and a low-cost data lake is an obstacle for any online entertainment platform that builds mobile or web ticketing applications. However, building on Google App Engine with MongoDB Atlas Clusters in the backend allows for a high-availability, low-cost data platform that seamlessly feeds data to downstream analytics platforms in real time. Unified data platform for retail customers: The retail business frequently requests an agile environment in order to encourage innovation among its engineers. With its agility in scaling and resource management, seamless multi-region clusters, and premium monitoring, running MongoDB Atlas on Google Cloud is a fantastic choice for building a single data platform. This simplifies the management of different data platforms and allows developers to focus on new ideas. High-speed real-time data platform of supply chain system for manufacturing units: By having real-time visibility and distributed data services, supply chain data can become a competitive advantage. MongoDB Atlas on Google Cloud provides a solid foundation for creating distributed data services with a unified, easy-to-maintain architecture. The unrivaled speed of MongoDB Atlas simplifies supply chain operations with real-time data analytics. The way forward Even in just the past decade, organizations have been forced to adapt to the extremely fast pace of innovation in the data analytics landscape: moving from batch to real-time, on-premise to cloud, gigabytes to petabytes, and the increased accessibility of advanced AI/ML models thanks to providers like Google Cloud. With our track record of success in this domain, HCL Technologies is uniquely positioned to help organizations realize the joint benefits of building data analytics applications with best-of-breed solutions from Google Cloud and MongoDB. Visit us to learn more about the HCL Google Ecosystem Business Unit and how we can help you harness the power of MongoDB Atlas and Google Cloud Platform to change the way you store and analyze your data through these solutions.