Democratizing Data Access With Plain English, Cogram, and MongoDB
Access to information is important for our world and the applications that power it. Today, the disparity in access has narrowed, but not enough. The proliferation of the document database and its flexible data model has unlocked a velocity and freedom that SQL systems will never unlock. It’s time to remove the burden of query languages for users early in their journey. That’s why we are launching an early pilot with Cogram, the startup building the world’s first Mongo Query Language (MQL) generation platform for natural language. The early-stage company is on a mission to democratize data access by allowing users to query data irrespective of their technical background. Cogram is working toward a future in which humans can access and manipulate any data through natural language. Cogram is partnering with MongoDB to enable language-driven data exploration. While it is still early days, users can log in to the tool, enter an English-language question for five collections from our sample dataset, and execute the generated MQL to see the results. Here are some questions to get you started on the Sample Supply Store dataset: Find one sale. What's the average customer satisfaction in Denver? What's the percentage of customers in Denver who used a coupon? How many users have an email address that contains ‘em.uy’? Find two sales. Get the last inserted sale. Show all store locations. Show all unique tags. What are the top five tags? Find all customers who use gmail. Show me all customer email addresses that end on ‘.sh’ What are the three most common store locations? What's the average price of all items? How many customers used a coupon? How many times was each item sold? Additionally, we’re excited to have Cogram included in MongoDB for Startups , our program for early-stage, high-growth startups. Designed to help founders and CTOs build and grow data-driven organizations with MongoDB, the program provides a wide range of resources, including MongoDB’s best-in-class data platform, personalized technical advice, and access to our robust developer community. Get a first look by signing up for a Cogram account and testing it out.
Freedom from the Throes of Tables and Rows and Legacy Product Management
When I suggest that I’m your typical employee at MongoDB, I don’t mean to imply that anyone at the company is ordinary. In fact, practically everyone I have worked with has brought an extraordinary talent or talents to the company, and it’s part of what makes me so excited every day. “Embrace the Power of Difference” is an important value at MongoDB that’s more than a poster on the wall. Our diverse life experiences are an important part of what makes our company and our products special. In my own case, I suffered a life-changing brain injury when I was a teenager. It was so severe that I was wheeling around a hospital in a wheelchair, unable to write my name with a writing implement, and required 24/7 supervision for months to follow. Today, I’m a Staff Product Manager working to challenge one of the least innovative and slowest moving areas of the tech industry with a paradigm shifting and fast-paced team at MongoDB. Before jumping into day-to-day life, let me talk about when I first fell in love with the database. I was 23 when I thought about building an IoT security device for high-value employees working from home. The heterogeneity of home network topologies meant that the flexible data model of MongoDB was the only option for us. The ability to move fast and iterate was our operating model, and MongoDB enables velocity in ways that other databases cannot because of the lost time at the beginning of the development process. At that time, there were also far fewer open source databases. With no money and coding mostly from my mom’s house back in Detroit, the ability to get started on my own and for free was critical. Starting that company eventually changed my life, and I owe MongoDB in many ways. Product Management at MongoDB Product Management at MongoDB varies quite a bit based on the age of the product, the deployment model, and the target persona. I work on a newer product offering in Atlas Search that requires me to be flexible and manage a variety of different tasks. Perhaps you can tell from the name of the product that it’s a cloud product, which unlocks lots of time for the engineering teams and I to focus on quickly delivering world-class features and value for our customers. Like every Product Manager at MongoDB, the primary persona I focus on is developers. It may be the most important point about this post because Product Managers should focus on their customers. If you have trouble remembering who matters most to me, watch this clip from the former CEO of one of MongoDB’s largest partners. As a Product Manager at MongoDB, I am empowered to influence the operations of various functions, I articulate the value and use of planned features, and I talk to customers constantly to echo their voice in our business where they cannot speak for themselves. What's it like to work on a new product? When I joined MongoDB, practically everything around Atlas Search was limited. We had not finalized messaging of the product, we weren’t totally certain which use cases or customers would be good for us, and we had not yet found product-market fit. Even knowledge on the various technical teams was limited, like Technical Support, Consulting, and Database Engineering. I want to emphasize that I am tied to the hip with our engineers and work continuously to have their trust and understand their perspective because without them I am nothing. These are all common problems for a Product Manager of a new product, and it’s our job to solve them. Even if you have answers, if a business does not have processes in place or a culture that empowers product managers to be influential thought leaders, you can find yourself spinning your wheels. At MongoDB, leadership empowered me to solve product issues. As the organization is very large, and there are many competing priorities, I had to lean on a written strategy and lots of data. I write strategy documents based on internal metrics, conversations with team members from other functions, and my expertise (I have worked on Search and databases for several years). While my field was a bit more greenfield than some of the other products, we have the space and the people for anyone with a vision to come in and show us a better way. If you do, be prepared to work with a variety of teams who view the world differently than you. Having a data-driven justification for them to meet you halfway and drawing a map that shows them how to get there will get the job done. It’s a hallmark of a great company with a great future. Even as our on-premise continues to grow, most of our revenue will come from the cloud, so Cloud Product Managers typically have 3 hands full. Atlas Search, as the first purely database product only in the cloud at MongoDB, needs to incentivize customers who are using a different search engine or running MongoDB on-prem strong enough that they change their deployment models. This strong call for incentives requires that I work very closely with our customers on what they need and what they want in a new search product. I talk to executives about their and our product visions, I talk to architects and dev ops engineers about the current architecture, and I talk to developers and data scientists about our API. I have MongoDB Compass and VS Code fired up and ready to go for whatever programming language our customers are using! What's it like to work on a cloud product? The biggest difference for me from many of my colleagues is the fact that I work on a cloud-only product, which means things change very fast for both our team and our customers. MongoDB the database can be run anywhere. The MongoDB server teams need to maintain a comprehensive test suite for various platforms whereas I am targeting AWS, GCP, and Azure in my work. I wouldn’t say they move slowly because MongoDB is still a rapidly evolving database with a very ambitious roadmap. However, one thing that commonly happens in the cloud that does not happen as much on-premise is rapidly scaling self-service growth. Back in 2020, an ecommerce retailer using Atlas Search saw their queries explode in volume. The change required several levels of engagement. Our product was shifting to be unusable by that team for their use case. In turn, we needed to change a lot about some of our fundamental operations. Over the course of a year, a product maturation occurred that would’ve taken many years in a less scalable business model. As a product manager, I have to keep up with all the moving parts, the teams, the docs, the risks, and the customer feedback across a variety of channels. While I know many customers wish we offered a local version of the product, many more customers benefit from our ability to move at a breakneck pace thanks to the single deployment model. It’s a tradeoff that we’ve committed to for the continued success of our customers. With the deployment advantages we have, the roadmap I work on has to meet the expectations of our customers. If we don’t stay at the cutting edge, our customers won’t tolerate our lack of flexibility. I need to keep a pulse on a variety of media, ranging from academia and Medium to GitHub and server logs. Cloud customers on average tend to be using a newer set of tools for managing their operations and development workflows, so there is an entire ecosystem to stay abreast of as well. The work is very demanding and technical. That makes me happy. Our customers’ sophistication is rising daily and their questions need answers. If ever I don’t have an answer right away, I commit time in my week to learn more about my customers’ use cases and aspirations. Who are the customers? As a Product Manager, nothing is more important than empowering our customers. If I ever have the thought that I’m out of ideas, it means I have failed to be sufficiently empathetic. The roadmap that I am working on is driven by those customers. MongoDB has increasingly reached executives as we became a widely adopted data platform, but at the core, our customers are builders, specifically the application developers, dev ops engineers, data scientists, and data engineers who are shaping our digitally native world. I spend countless hours every week talking to customers, reading as many support cases as possible — even those that aren’t directly tied to my domain – on Stack Overflow, our MongoDB user forum , and various message boards and blogs. Other media like YouTube and podcasts provide another window into the hearts and minds of the people that put us in business. There are lots of people to please, and lots of decisions to make. I try my best not to be hasty and base every decision on a combination of intuition and data. My goal is to build the most widely adopted full-text search solution in the world to support tens of thousands of Googles across a number of domains. I have been working closely with another team at MongoDB to build a very close relationship with customers with very demanding workloads. It’s important to set those customers up for success early to support their ambitions. They are companies ranging from household names in the United States in video streaming to Indian cybersecurity startups experiencing hyper growth. I watch these customers live coding and even doing production deployments because I want to know how it feels to work with our product at every step of the way, the entire development lifecycle. I also want our customers to know that while I do not have the bandwidth to do this with every customer or even all the time, I care immensely about empowering our customers to move fast with world-class performance, reliability, and stability with or without me. It's bigger than Atlas Search At the beginning of this blog, I mentioned that I am different. It’s true. I come from a place that is perceived as rough, and I was educated in public schools there. I live with a disability impacting my brain that is invisible to most. I also speak loudly and frequently, and I’m an underrepresented minority in a country where opportunity is unfortunately unequally distributed. MongoDB inculcates an environment that is safe for underrepresented and neurodivergent team members. It’s incredible to work at a company where the managers and the executives do not simply claim to care about the employees but empower their ideas based on the quality of their work. I hope that every customer knows that MongoDB is a unique organization in its inclusivity and diversity. Our values drive our product’s excellence — and my happiness at work. I lead an extension of the business-at-large. I view myself as the CEO of the mini-company that is the MongoDB Atlas Search product. I work hard to listen to everyone. It's impossible to know who the smartest person in the room is, as it changes by the minute and it’s not about “the smartest” at MongoDB. You do not need to be an expert in the fields of Distributed Systems, Natural Language Processing, User Experience, Linear Algebra, or Statistics to build a world-class search experience in 2022. You can trust that MongoDB is the best place to surround yourself with the various experts you need to grow your expertise. Being intellectually honest, I am none of those things. Despite my expertise and experience working on a variety of open source search applications, I approach every day as a beginner with our newest customers in mind. I am an expert in my customer’s needs and by extension my product’s priorities. If I ever got tired of working toward that end, though unlikely, MongoDB is a place where I could spend time training to do something different and make the switch. It’s the type of company from which you could join another company later and have great success because of the skills you develop at MongoDB, start your own company to spread our love for developers even further, or stay until you retire because there is never a dull moment. If you are a bit different, bring your super power to MongoDB! Learn more about Atlas Search by listening to my episode of The MongoDB Podcast Follow me on GitHub and Twitter
100x Faster Facets and Counts with MongoDB Atlas Search: Public Preview
Today we’ve released one of the most powerful features of Atlas Search in public preview, and ready for your evaluation: lightning fast facets and counts over large data sets. Faceted search allows users to filter and quickly navigate search results by categories and see the total number of results per category for at-a-glance statistics. With the new facet operator , facet and count operations are pushed down into Atlas Search’s embedded Lucene index and processed locally – taking advantage of 20+ years of Lucene optimizations – before returning the faceted result set back to the application. What this means is that now facet-heavy workloads such as ecommerce product catalogs, content libraries, and counts run up to 100x faster . The power of facets and counts in full-text search Faceting is a popular search and analytics capability that allows an application to group information into related categories by applying filters to query results. Users can narrow their search results by simply selecting a facet value as a filter criteria. They can intuitively explore complex data sets, providing fast and convenient navigation to quickly drill into the data that is of most interest. A common use of faceting is navigating product catalogs. With travel starting to reopen, let's take a travel site as an example. By using faceted search, the site can present vacation options by destination region, trip type (i.e. hotel, self-catering, beach, ski, city break), price band, season, and more, enabling users to quickly navigate to the category that is most relevant to them. Facets also enable fast results counting. Extending our travel site example, business analysts can use facets to quickly compare sales statistics by counting the number of trips sold by region and season. Prior to the new facet operator, the only way Atlas Search could facet and count data was to retrieve the entire result set back to MongoDB’s internal $facet aggregation pipeline stage . While that was OK for smaller data sets, it became slow when the result set exceeded tens of thousands of documents. This all changes as now operations are pushed down to Atlas Search’s embedded and optimized Lucene library in a single $search pipeline stage. From our internal testing of a collection with one million documents, the new Atlas Search faceting improves performance by 100x. How to use faceting in Atlas Search Our new Atlas Search facets tutorial will help you get started. It describes how to: Create an index with a facet definition on string, date, and numeric fields in the sample_mflix.movies collection. Then run an Atlas Search query against those fields for results grouped by values for the string field and by ranges for the date and numeric fields, including the count for each of those groups. To use Atlas Search facets, you must be running your Atlas cluster on MongoDB 4.4.11 and above or MongoDB 5.0.4 and above. These clusters must be running on the M10 tier or higher. Facets and counts currently work on non-sharded collections. Support for sharded collections is scheduled for next year. The power of Atlas Search in a unified data platform in the cloud MongoDB Atlas Search makes it easy to build fast, relevant full-text search on top of your data in the cloud. A couple of API calls or clicks in the Atlas UI, and you instantly expose your data to sophisticated search experiences that boost engagement and improve satisfaction with your applications. Your data is immediately more discoverable, usable, and valuable. By embedding the Apache Lucene library directly alongside your database, data is automatically synchronized with the search index; developers get to work with a single API; there is no separate system to run and pay for; and everything is fully-managed for you on any cloud you choose. Figure 1: Rather than bolting-on a separate search engine to your database, Atlas Search provides a fully integrated platform. Atlas Search provides the power you get with Lucene — including faceted navigation, autocomplete, fuzzy search, built-in analyzers, highlighting, custom scoring, and synonyms — combining it with the productivity you get fromMongoDB. As a result, developers can ship search applications and new features 30%+ faster. Next steps You can try out Atlas Search with the public preview of lightning-fast facets and counts today: If you are new to Atlas Search, simply spin up a cluster (M10 tier or above) and get started with our Atlas Search facets tutorial . If you are already using Atlas Search on M10 tiers and above then update your indexes to use the facet field mapping , and then start querying ! Your data remains searchable while it is being re-indexed. If you want to dig into the use cases you can serve with Atlas Search — along with users who are already taking advantage of it today — download our new Atlas Search whitepaper . Safe Harbor The development, release, and timing of any features or functionality described for our products remains at our sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality.