We are very excited to announce that Cloud Manager Backup now supports the following command-line options:
smallfiles(this is an option for MMAPv1 only)
Cloud Manager Backup will now take into account the command line options of the primary during an initial sync. If your primary uses one of these options, and you want your backup to as well, just resync your backups. If you’ve been holding off using Cloud Manager Backup because of our lack of support of these options, you need wait no more.
Please note that existing snapshots will not be converted, only snapshots created for jobs that were resynced after noon, EDT on May 16 will have these options enabled.
Unlocking Operational Intelligence from the Data Lake: Part 2 - Operationalizing the Data Lake
As we discussed in part 1 , Hadoop-based data lakes excel at generating new forms of insight from diverse data sets, but are not designed to provide real-time access to operational applications. Users need to make analytic outputs from Hadoop available to their online, operational apps. These applications have specific access demands that cannot be met by HDFS, including: Millisecond latency query responsiveness. Random access to indexed subsets of data. Supporting expressive ad-hoc queries and aggregations against the data, making online applications smarter and contextual. Updating fast-changing data in real time as users interact with online applications, without having to rewrite the entire data set. Bringing together operational and analytical processing across high volumes of variably structured data in a single database requires capabilities unique to MongoDB: Workload isolation. MongoDB replica sets can be provisioned with dedicated analytic nodes. This allows users to simultaneously run real-time analytics and reporting queries against live data, without impacting nodes servicing the operational application, and avoiding lengthy ETL cycles. Dynamic schema, coupled with data governance. MongoDB's document data model makes it easy for users to store and combine data of any structure, without giving up sophisticated validation rules, data access and rich indexing functionality. If new attributes need to be added – for example enriching user profiles with geo-location data – the schema can be modified without application downtime, and without having to update all existing records. Expressive queries. The MongoDB query language enables developers to build applications that can query and analyze the data in multiple ways – by single keys, ranges, text search, and geospatial queries through to complex aggregations and MapReduce jobs, returning responses in milliseconds. Complex queries are executed natively in the database without having to use additional analytics frameworks or tools, and avoiding the latency that comes from moving data between operational and analytical engines. Rich secondary indexes. Providing fast filtering and access to data by any attribute, MongoDB supports compound, unique, array, partial, TTL, geospatial, sparse, and text indexes to optimize for multiple query patterns, data types and application requirements. Indexes are essential when operating across slices of the data, for example updating the churn analysis of a subset of high net worth customers, without having to scan all customer data. BI & analytics integration. The MongoDB Connector for BI enables industry leading analytical and visualization tools such as Tableau to efficiently access data stored in MongoDB using standard SQL. Robust security controls. Extensive access controls, auditing for forensic analysis and encryption of data both in-flight and at-rest enables MongoDB to protect valuable information and meet the demands of big data workloads in regulated industries. Scale-out on commodity hardware. MongoDB can be scaled within and across geographically distributed data centers, providing extreme levels of availability and scalability. As your data lake grows, MongoDB scales easily with no downtime and no application changes. Advanced management and cloud platform. To reduce data lake TCO and risk of application downtime, MongoDB Ops Manager provides powerful tooling to automate database deployment, scaling, monitoring and alerting, and disaster recovery. Further simplifying operations, MongoDB Atlas delivers MongoDB as a service, providing all of the features of the database, without the operational heavy lifting required for any application. MongoDB Atlas is a great choice if you want the database run for you, or if your data lake and apps are also running on a public cloud platform. MongoDB Atlas is available on-demand through a pay-as-you-go model and billed on an hourly basis. High skills availability. With availability of Hadoop skills cited by Gartner analysts as a top challenge, it is essential you choose an operational database with a large available talent pool. This enables you to find staff who can rapidly build differentiated big data applications. Across multiple measures, including DB Engines Rankings , The 451 Group NoSQL Skills Index and the Gartner Magic Quadrant for Operational Databases , MongoDB is the leading non-relational database. In addition, the ability to apply the same distributed processing frameworks such as Apache Spark, MapReduce and Hive to data stored in both HDFS and MongoDB allows developers to converge analytics of both real time, rapidly changing data sets with the models created by batch Hadoop jobs. Through sophisticated connectors, Spark and Hadoop can pass queries as filters and take advantage of MongoDB’s rich secondary indexes to extract and process only the range of data it needs – for example, retrieving all customers located in a specific geography. This is very different from less featured datastores that do not support a rich query language or secondary indexes. In these cases, Spark and Hadoop jobs are limited to extracting all data based on a simple primary key, even if only a subset of that data is required for the query. This means more data movement between the data lake and the database, more processing overhead, more hardware, and longer time-to-insight for the user. Table 1: How MongoDB stacks up for operational intelligence As demonstrated in Table 1, operational intelligence requires a fully-featured database serving as a System of Record for online applications. These requirements exceed the capabilities of simple key-value or column-oriented datastores that are typically used for short lived, transient data, or legacy relational databases structured around rigid row and column table formats and scale-up architectures. Figure 1: Design pattern for operationalizing the data lake Figure 1 presents a design pattern for integrating MongoDB with a data lake: Data streams are ingested to a pub/sub message queue, which routes all raw data into HDFS. Processed events that drive real-time actions, such as personalizing an offer to a user browsing a product page, or alarms for vehicle telemetry, are routed to MongoDB for immediate consumption by operational applications. Distributed processing frameworks such as Spark or MapReduce jobs materialize batch views from the raw data stored in the Hadoop data lake. MongoDB exposes these models to the operational processes, serving queries and updates against them with real-time responsiveness. The distributed processing frameworks can re-compute analytics models, against data stored in either HDFS or MongoDB, continuously flowing updates from the operational database to analytics views. In part 3, we’ll demonstrate how leading companies are using the design pattern discussed above to operationalize their data lakes. Learn more by reading the Operational Data Lake white paper. Unlocking Operational Intelligence from the Data Lake About the Author - Mat Keep Mat is director of product and market analysis at MongoDB. He is responsible for building the vision, positioning and content for MongoDB’s products and services, including the analysis of market trends and customer requirements. Prior to MongoDB, Mat was director of product management at Oracle Corp. with responsibility for the MySQL database in web, telecoms, cloud and big data workloads. This followed a series of sales, business development and analyst / programmer positions with both technology vendors and end-user companies.
How Thoughtful Illustration Is Setting MongoDB Apart: Meet Champa Lo
I sat down with Champa Lo, Technical Illustrator based in our New York headquarters, to learn more about her role as the first full-time illustrator at MongoDB. We talked about her passion for illustration, what she does, and how she’s shaping the future of design within the company. Ashley Perez: Welcome to the team! Can you tell me about your role? Champa Lo: Sure. I joined MongoDB right before COVID-19 hit. I came into the headquarters twice for an interview but ended up being one of the first new hires who had to start at home, on top of being the first person in a brand-new role. Technical Illustration is a first for MongoDB. The company has never had an illustrator on hand. Although we have talented designers who can illustrate within a design, that’s not their main focus: the overall design is. The difference with my role is that I work specifically on illustration. I also work to define the illustration style and help create a style guide. The most important aspect of my job is building good relationships with people throughout the company. I need to understand their goals and what they’re looking for so I can tell a purely visual story. AP: How did you get into illustration? CL: I guess you can say I fell into it (at least the illustration part). I always knew I wanted to be a graphic designer early on. I was a mentee for a graphic designer in high school and absolutely fell in love with the profession. I even have a cute clipping from my senior year high school paper where I talk about my dreams of being a designer. Interview excerpt from Champa's senior-year high school newspaper After high school, I studied graphic design at the University of Colorado Denver. When I was in the design program, I always found ways to incorporate fun illustrations in my projects. A year after I graduated, I moved to New York City because there were more jobs in design there and landed a job that allowed me to put my illustrating skills to good use. My first job was working with an incredible Creative Director at a small startup who built an amazing brand using illustrations to convey the company’s goals and messages. This was a part-time job: for four hours a day, I would concentrate on illustrating bespoke email banners for marketing prompts the team created that morning. With her guidance, I saw my illustration skills grow. It showed me the possibility of being a full-time illustrator. Here’s an example of a design I did while I was there: Email banner Champa created for ThinkEco during her first job as illustrator I love to illustrate (especially this type of illustration) because I’m a designer by trade, and the core of designing is to problem-solve. Illustration is no different. As a Technical Illustrator, I simplify and visualize complicated theories and concepts. Also, it’s fun! If I’m not having fun while illustrating, I’m very unmotivated. My creativity relies on avoiding boredom. I’m always working to improve my artistic skills. I’m a lover of learning, so I subscribe to tutorial sites such as Skillshare; follow artists on YouTube who share tutorials; and subscribe to a monthly art box that sends paints, brushes, pens, and so forth so I can try new mediums. Champa's illustration for a Google Local Guides social media post AP: How do you make your illustrations purposeful, engaging, and memorable? CL: Having thoughtful conversations about the subject matter is how you get good designs and illustration. If you don’t understand the subject to the best of your ability, how can you be successful at visualizing it? In school, I was taught to always research your subject matter and not design blindly. Putting in the extra work makes a huge difference. That’s also why 1:1 meetings are so important. It’s a time for me to learn, and it’s also a creative process for the stakeholders, because they find creative ways to help me understand. GIF Champa created for a MongoDB University Page We want to understand the goal. For example, should the illustration be futuristic or nostalgic? Recently, we had a conversation about cars and how we wanted to present them for a project. We decided to design the cars as compact or electric to show MongoDB as forward thinking and environmentally conscious, because those are the kinds of people we want to hire and work with. Or take COVID-19, for instance. The pandemic has changed the way people illustrate office environments. No longer do you have teams sitting in conference rooms. Instead, you have people working at home. So, I had to think of things to illustrate such as a sofa, home desk, and desk lamp. Maybe even a dog or a child. We thought about how we could incorporate this into the Zoom interface. Before, we didn’t have to think about it. Now, Zoom can be a way to add some personality to everyone’s digital space as we work remotely. That’s what I’m here for. To have those conversations and get deeper behind the meaning of everything we create. AP: Let’s talk a little more about your role at MongoDB. What projects do you work on? CL: I’m part of the Visual Design Team, which supports the whole company. It’s fun to meet and talk to many different people at MongoDB. It gives us a lot of diversity in the projects we work on. Along with illustrations, I also work on diagrams and small animations. Projects include campaigns, web illustrations, and events. Because I’ve joined the team, we’re able to have fuller discussions about illustration. Our designers work in a fast-paced world, but my process is slower because I make more bespoke illustrations and have to talk to people to understand the technicalities so we can go beyond generic illustrations. I have to be more thoughtful of what we’re presenting to the audience. Even though by having these conversations I slow down how quickly the designers move, I'm striving to build stronger relationships on the team through this practice. Top left: Champa’s illustration for MongoDB's new multi-cloud feature. Bottom right: An illustration for MongoDB's vendors page. I have found that by showing and explaining my illustration process and inviting them into it, people seem to trust me more. For example, I always share my sketches with stakeholders before digitizing the work. My sketches aren’t perfect, but by showing them not-so-perfect work, we can build the relationship and align on ideas. My hope is that the sketches allow people to see I’m open for collaboration and conversation. Example of a project working with MongoDB's Web Design team from initial sketch through final illustration AP: How does having these conversations help your design? CL: Great question! Working with such a diversity of people and projects helps me gain an immense amount of knowledge and insight. Past conversations and concerns help inform my design decisions. I’m almost like a liaison for all these different departments, and it's nice to transfer the information so we’re all aligned. For example, I’ve been working closely with Product Marketing on diagrams, and soon I’ll be working on diagrams with a member from the Docs team, too. Each team has taken its own paths for diagrams, but I would love to eventually create a holistic style that works for all teams beyond just these two. I believe having a good process to follow leads to meaningful and engaging illustrations. However, it’s important to find balance. You can’t overengineer it, because that can easily turn unproductive and formulaic. I always want an open dialogue and strive to show there’s room to collaborate. The process we have created has been successful so far, but it’s not set in stone. Further along we can add another step, or we may find certain things aren’t needed. AP: What’s your creative vision for MongoDB? CL: My goal for illustration is that we are inclusive, diverse, and thoughtful. What I’ve seen here is a global company full of people who are very passionate and kind. As designers, we have the power to show who and what MongoDB is. For me, that’s showing off who we are. One of our company’s values is “Own What You Do.” I think it’s such an important one for designers, because we should always add our personal experiences and perspectives to our work and translate the rest of the company’s perspectives and experiences, too. For the team, my goal is to continue streamlining a process so we’re transparent and support a collaborative spirit when it comes to working with us. Champa’s illustration for the MongoDB Atlas onboarding experience My goal is to create a unified vision between our two audiences: developer and enterprise customers. My hope is the illustrations bring joy and delight, and that our audiences see MongoDB has a personality. A really effective illustration system is memorable, and our research is starting to show that our audiences are beginning to remember our visuals. This is a huge brand lift, creating a personal experience versus the cold one people may experience with other tech brands. Interested in pursuing a career at MongoDB? We have several open roles on our teams across the globe , and would love for you to build your career with us!