MongoDB Blog

Articles, announcements, news, updates and more

Built With MongoDB: Queenly

Queenly founders Trisha Bantigue and Kathy Zhou grew up in low-income immigrant families, trying to balance their cultural upbringing with their desire to fit into their American lifestyle. To earn scholarships to pay for college, they both started participating in beauty pageants. “Beauty pageants provide young women with the opportunity to kickstart their careers,” says Queenly Co-founder & CTO Kathy. “And one of the core parts of the pageant system is the evening gown — that was the spark of inspiration for us wanting to tackle the whole fashion industry.” According to Kathy, women in America — especially outside the coastal cities — end up attending around 15 special occasions a year. “Whether it’s prom, beauty pageants, or other formal occasions, women need cost-effective formal outfits,” she says. After working across leading Silicon Valley companies, Trisha and Kathy teamed up to build Queenly , a marketplace and search engine for the formalwear industry. “No one has created a robust search engine for formal dresses,” says Kathy. “People are picky about formal attire — there’s so much consideration that goes into it, from neckline to hemline, silhouettes, colors, and fabrics. We’re trying to build a marketplace, do complex queries, and provide personalized recommendations.” Queenly has 80,000 registered users and 50,000 dresses listed. The team of five ( which is hiring! ) is backed by Y-Combinator. We recently sat down with Kathy to learn more about how her pageant experience has informed her career, her experience with MongoDB, and the challenges in building a formalwear business. MongoDB: What was your first pageant experience like? Kathy: It was really eye-opening: I have always been a shy person, and my number one fear is public speaking. What people don’t realize about pageants is that along with having to learn how to dress well, you also have to be able to speak well. You have to learn to speak from your heart and to communicate well. Gaining the confidence and soft skills to answer those pageant questions has also helped me in my career, helping me grow from an engineer to engineering leadership. One of the most memorable questions from an early pageant was about what’s the most important thing you want to do in your pageant regime. I talked about how it’s okay for young women to both be nerdy and girly — you should be able to embrace all these different sides of yourself, and not fear falling into one box of being. I wish someone had told me that when I was younger. Now, I’m honored to be able to embrace both sides as a CTO, a Y-Combinator female founder, and a beauty pageant contestant. MongoDB: Building a two-sided marketplace is a challenge. What did the minimum viable product look like? Kathy: The MVP was very rough — I started by coding an iOS app part-time and during the weekends while I was still employed at Pinterest. The goal was to tackle the supply-side of the marketplace first to get people to upload dresses, so I optimized for creating a really easy dress-upload experience. You could only search for one size and one color at a time. Now, we’re using natural language processing query for search, and also a larger combination of different dress-type attributes. We’re also including reverse image search, and I’ve been working on tailored user recommendations. MongoDB: How did you make decisions for your technical back end? Kathy: Initially, we had very basic search and exploration using Google’s Firebase. It was very easy to set up and has a fairly good UI tooling, but its query capacity was something we were quickly outgrowing. At our stage of company, non-relational storages are a really great decision for the sake of speed and adaptability. As we’re working towards product-market fit, we need to move quickly in launching new user experiences and reworking old ones, so it’s important to have that flexibility in restructuring and reshaping our data. That’s when I went to MongoDB and realized that it was a really quick migration and had all the capacities and flexibility we need. MongoDB is great for JavaScript developers. I started with a background in front end, with a foundation in HTML / CSS and JavaScript, and it was very easy to pick up MongoDB. It’s also going to help a lot of emerging developers, and those coming out of coding bootcamps, get started on the back end more quickly. As people say, we were building the airplane as we were flying. We needed to move fast so people could access and search for dresses quickly. Many of our users are women who live in the Midwest and the South where they may not have amazing internet access, so speed and performance are pretty important. MongoDB: Are there specific features of MongoDB you're using, aside from Atlas? Kathy: The most important aspects are the core functionality and the monitoring toolings and dashboards. Those are useful and come right out of the box. I’ve been meaning to take a look at search capabilities — I think it’s cool that there are indexes right out of the box. We’re trying to adapt our product as it goes, and figure out how to tag and enable different attributes on a dress. MongoDB: What was the last good technical book or article you read? Kathy: I really enjoy reading the Towards Data Science publication on Medium. They do a good job of covering different use cases as well as making different fields algorithms and data science/machine learning concepts more approachable. Beyond that, I read several fashion magazines and pageant blogs because I think CTOs — and the technical side of the business — should really understand the users. I try to keep up with trends in fashion and retail to better understand the opportunity, and use that to influence how our product functions. Looking to build something cool? Get started with the MongoDB for Startups program.

April 20, 2021
Applied

The Difference Between R and D

I used to believe that Research and Development (R&D) departments should work in lockstep with Product teams so that they can stay focused on commercially-viable innovations. After all, not every innovation has a market, and not every business has resources to bet on future markets. All of that changed when I met Dr. Michael Cahill, the head of MongoDB Labs in Australia. Michael came to MongoDB through our acquisition of WiredTiger back in 2014, an open source storage engine company he co-founded with Keith Bostic. He holds a PhD in Computer Science and a history of breakthrough innovation. He also has an enlightened point of view on the role of research in any technology company. “Researchers need time and space to pursue the theoretical,” he told me. “We want them to come up with crazy ideas, with much longer time horizons.” Michael is referring to the fundamentally different mindsets required of researchers versus developers. Our developers are focused on new products or features that can make an impact in the next 3-4 quarters. Our researchers are thinking about solving problems that have the potential to reshape entire markets for decades. Big difference. Funding this kind of innovation is challenging for the MBA set, and measuring the ROI of basic research is notoriously difficult. Progress can seem slow and difficult to quantify. Our researchers occupy a space that straddles art and science, industry and academia. They spend a lot of time reading, thinking, and tinkering. Ideas are freely shared, cultivated, iterated, and sometimes abandoned. This is the price of disruptive innovation. In fact, MongoDB would never exist if our founders had set out to simply improve upon relational databases. Instead, they wanted to invent an entirely new way to manage data. It was an ambitious idea that required long-term thinking. An idea that despite MongoDB’s current success, is still only in its infancy. An idea so humongous, Michael Cahill may have even called it “crazy.” Don’t get me wrong. The work of MongoDB Labs is firmly grounded in MongoDB’s core strategy: to constantly improve the way data is stored, accessed, secured and processed. Document databases are only the first act of this play. And I’m certain the next act is being written as we speak, by Michael and his team. Have a different approach to R&D? Think my ideas are “crazy”? Let me hear about it on Twitter at @MarkLovesTech

April 19, 2021
Mark Loves Tech

Accelerate Data Modernization with Infosys Data Model Converter

Are you in the process of migrating applications from a relational database to MongoDB? If so, you’re likely trying to best understand and decide how your enterprise data needs to be modeled. Our previous blog discussed how Infosys Data Services Suite can help enterprises move data seamlessly from legacy relational databases to MongoDB. But moving data is only one part of the puzzle. The more significant step is choosing the target data model, or schema design, a process that usually requires several hours of highly skilled talent. That’s why we created this follow-up blog to help you get started. Rethinking Schema Design Ultimately, schema design can be the difference between an inefficient, disorganized database and a strategic one that empowers the entire company. Schema design in MongoDB requires a change in perspective for data architects, developers, and database administrators. They have to: Rethink the legacy relational data model. This model flattens data into rigid two-dimensional tabular structures of rows and columns. The new data model is a rich and dynamic one with embedded sub-documents and arrays Rethink how the data platform works. In relational databases, it is extremely difficult to change the data platform as the application evolves. However, in MongoDB, the apps and APIs come first and the data platform dynamically accommodates the data Getting Schema Design Right Begin the schema design process by considering the application’s requirements. You’ll want to model the data in a way that leverages the flexibility of the document model. In schema migrations, it may seem easy at first to simply mirror the flat schema of the relational database in the document model. However, this negates the advantages enabled by the rich and embedded data structures of the document model. For example, data that belongs to a parent-child relationship in two RDBMS tables can be collapsed (embedded) into a single document in MongoDB. The application data access patterns should also drive schema design with a specific focus on: The read/write ratio of database operations and whether it is more important to optimize the performance of one operation over another The types of queries and updates performed by the databases The lifecycle of the data and growth rate of documents Simplifying Schema Design with Infosys Data Model Converter Infosys has developed a solution called Infosys Data Model Convertor that processes source relational schema and the above-mentioned signals as inputs and automatically provides target MongoDB schema suggestions. Infosys Data Model Converter is available as part of Infosys Modernization Suite which accelerates enterprises’ modernization journey. Each schema suggestion is accompanied by a detailed analysis report. The data modeler can use this as a starting point and iterate over the schema to arrive at the final MongoDB schema. The Infosys Data Model Converter reduces 50-60% of the effort typically spent in schema design. Key Features Boosts productivity by augmenting the migration of RDBMS to NoSQL database Saves time by automatically extracting schema, query and data patterns from an existing RDBMS Comprehensively analyzes the RDBMS entity relations, data and read-and-write patterns Applies a rich set of rules and generates a fully compliant NoSQL target state data model Offers flexibility by externalizing the rules for organization-specific customizations Connects and deploys the model to the target NoSQL platform with sample data Discover more ways in which Infosys can help you unlock value from modernization. Contact us for any modernization questions.

April 15, 2021
Developer

A Field Marketer Adapts to a New COVID-19 Landscape: Meet Amy Rosenberg

I sat down with Amy Rosenberg, a Senior Manager for MongoDB Field Marketing based in New York, to gain insight into how her role transformed when the COVID-19 pandemic started, her newfound love for data, and the ways in which MongoDB helped her adapt to a new working environment. We also spoke about the initial hardships of the COVID-19 outbreak in New York, scouring for toilet paper and Clorox, and how she envisions Field Marketing in a post-pandemic landscape. Jackie Denner: Thanks for sharing your story with us, Amy. Can you tell me about your journey into field marketing? Amy Rosenberg: I started my first “grown-up job” three weeks before graduating college. I was a one-woman marketing department for a 10-person startup. Over the course of my five years there, I had the chance to try my hand at every part of marketing: content, product, demand generation, social media, communications, advocacy, and events. I realized early on that what I love and what I am good at is being in the field interacting with and building community for customers. About three years into my career, someone told me that what I did sounded a lot like field marketing. I’d never heard that term before, but after reading some job descriptions, I decided it sounded fitting. One of the perks of working for a startup was the ability to change my title and team’s name. We became Field Marketing, and I officially became a field marketer. JD: How would you describe field marketing to those who aren't familiar? AR: When people hear the words field marketing, they think of events. They picture the team searching for venues, coordinating with A/V, running promotional campaigns, and handing their sales team leads to follow up with. This is definitely a part of field marketing, but to me, it’s not the full picture. Maybe it’s because I started my career wearing all of the marketing hats, but I’ve always seen myself as responsible for understanding when and why to do these events, how to ensure the leads make it through the sales funnel and become new customers, and how to track and analyze the ROI. This is one of the main reasons I joined MongoDB. In my first interview, my future boss discussed how MongoDB Field Marketing was part of a larger account-based marketing (ABM) strategy. I wouldn’t be an event planner; I would be the CMO of my region and a business partner to my regional Sales team. Events would be one of many tools I could use to support driving new leads and accelerating deals. I’d never heard field marketing described as such a strategic and impactful function, and I jumped at the opportunity to join the team. After a year without live events, the scope of my role feels even more true today. JD: What was a day in your role like prior to COVID-19? AR: Before COVID-19, I was always on the move, jetting around the world to host various events. This gave me the opportunity to get to know my Sales teams, talk to our customers, and visit dozens of incredible places such as Montreal, Beijing, and San Francisco, to name a few. My suitcase was always packed, and I got pretty used to spending only two or three nights in my New York City apartment each week. I like to describe MongoDB as the perfect mix between startup and established company. The company is doing very well and has the structure, leadership, and product to succeed. However, we still have my favorite parts of a startup culture: transparency from leadership, fun perks such as surprise swag gifts, unique benefits such as Headspace memberships and Carrot Fertility, and — my favorite part — the ability to make a meaningful contribution no matter what your level of seniority is. I hosted my favorite event about six months after joining — a C-level dinner at Classic Car Club Manhattan. Our CEO gave the opening talk, and our Chief Product Officer hosted a customer panel. No one questioned whether a new manager should own something so big, and my leaders gave me full autonomy to make it what I wanted. It ended up being a huge success and still gets brought up two years later. Hosting two or three events a month was exciting and made a huge impact on my region, but it was also exhausting. By the time I was back at my desk in New York, I hardly had the energy to analyze whether or not my projects were yielding the best results. I knew hundreds of customers and potential customers attended my events each month, but I rarely had the opportunity to think about things such as whether or not those were the customers with the greatest potential to buy, if the Sales Development Representatives were following up with the right materials to ensure conversion, or if the leads were being accurately routed to the right people. I knew I wanted to be even more strategic in my role. JD: How did the COVID-19 pandemic affect your personal and professional life? AR: In March 2020, my entire world turned upside down and inside out. I went from spending a few nights a week in my one-bedroom apartment to not leaving it for weeks on end. New York City was one of the hardest-hit cities. I remember in early April I accidentally missed a meeting with one of my Sales teams because someone told me a store nearby finally had toilet paper and Clorox wipes in stock, and there was no way I was missing an opportunity like that. I felt comfortable putting myself before my work because from day one of the pandemic, MongoDB’s leaders encouraged us to do exactly that. They gave us companywide mental health days, hosted biweekly all-hands to keep us informed, and offered forums to discuss the current social and political environment. I then started hearing how companies were taking down job postings for open field marketing positions and some were even laying off existing field marketers. Field marketing equaled live events, and no one knew when live events would happen again. Naturally, this scared my team and me. Pretty quickly, our leadership sat us down and told us there were no planned layoffs, but we needed to get creative and find new ways to support the Sales team. Sales goals hadn’t changed, which meant we were still responsible for driving new leads and accelerating opportunities. I picked this job because I thrived on the social interactions it gave me. Suddenly, I was left with Zoom calls and an empty apartment. My fiancé is a doctor and was sent to battle COVID-19, working 24-hour shifts with limited PPE. I am extremely grateful that neither of us has gotten sick. It’s been a very lonely and stressful time, but having a team that jumped in to help when I needed a day off or scheduled a random Zoom happy hour just to chat made it much easier. JD: How have you pivoted in your role since the COVID-19 outbreak began? AR: When my job was producing live events, they had to be run based on location, meaning the same event could take place in five or more markets in a given month. Even if we could reuse the content, the hours spent on promotion to each location, traveling to and from, and managing the spreadsheets full of tasks to make it a success ate away all of my time. With the transition to virtual, this duplication became irrelevant. A webinar can reach hundreds of customers across the globe at once. As a Field Marketing team, we began to talk about and better understand what each region needed and where we could find overlap. We found that we could split up the work and build a marketing program targeted to different industries and use cases. Then, we could all take advantage of a single program for our relevant accounts. This sounded as if we were doing less work, but it really just gave each of us time back to focus on improving the work we did, rather than rushing to do more. The webinars became much higher quality, since each field marketer was producing less and could spend more time improving an individual program. An unexpected benefit of this sharing of work was our team becoming much more collaborative: team brainstorming sessions, asking for and providing feedback on work we could all take advantage of, sharing resources anytime we saw a success from our work, and bringing half a dozen brains together to create a much stronger program. JD: How has MongoDB helped you transition during this time? AR: Before COVID-19, the lines differentiating teams within Marketing were mainly based on the types of activities we owned: Field Marketing hosted in-person regional events, Demand Generation ran digital ads and webinars, and Strategic Events managed our large-scale global events. With no live events, these lines became blurry. With a lot of help and guidance from our Marketing leadership, we created lines based more on goals than on activity type. For Field Marketing, our goal is to source new and accelerate existing deals for our specific sales region. Events (now virtual) were just one of the tools in our toolbelt, along with customer stories, digital ads, executive engagement, direct mail, and even sales enablement to improve conversions on the inbound leads from Marketing. I told my manager that my new motto was “avoid doing work.” Naturally, they got very concerned. But, what I really meant was to take advantage of what is already being done by others, instead of duplicating efforts, and then reallocate my time to things such as lead flow handoff improvements, data hygiene, advising other teams on customer stories, and educating my sales reps on the self-serve tools we provide. This has been a very scary change of mindset for me, because I always equated success to the programs I owned. I’m insanely grateful I have such amazing leaders who completely supported my new mentality. This change also helped me finally realize what it means to be the CMO of my region: working collaboratively with the entire Marketing organization to ensure my region has everything it needs to hit its numbers. Not only has this made me much more strategic in my actions, but it also gave me the opportunity to meet and become friends with people outside of my direct team. I can say with full confidence that I work on the best Marketing team out there because of the people. JD: You've fallen in love with data during quarantine. How did that happen, and how do you envision it playing a role in your approach moving forward? AR: Some people baked sourdough bread. Others completed puzzles. I learned Tableau. We were given access to new data dashboards right around the time lockdown started, and maybe I just needed an escape from staring at my own face on Zoom, but I began spending a lot of time in these reports. Going back to the concept of being the CMO of my region and all the time I saved by “avoiding work,” I wanted to have a clear and deep understanding of what programs, messaging, promotion strategies, and content worked best in my region, so I could double down on what works and either stop or change the things that didn’t work. I’d never been trained on using Tableau or looking at data this way. When I expressed my interest in this analysis, my manager gave me the time to learn and asked our Marketing Ops team to help. I spent hours building new reports, asking Marketing Ops questions, and then discussing my findings with other stakeholders on the team. I began making changes and improvements to the programs I ran as well as to the ways all inbound leads for my region were handled. Without adding more events, I saw our conversions to new deals increase. On a personal level, I’ve found that I’m actually pretty good at this kind of analysis. My team and leadership now come to me with questions, and my manager actually helped take other work off of my plate so I could focus on this. I was even given the opportunity to present to our global Sales leaders on the lead flow process I helped improve. I absolutely love finding new insights and uncovering challenges I get to fix. JD: What do you think MongoDB Field Marketing will look like in the future? AR: I’m not going to lie: I really miss live events. Although we still achieved our goals this year, there is something special about how events foster relationships and community between a company and its customers. But whatever the world of events looks like in the future, I don’t expect Field Marketing to go back to being solely event planners. This past year made us learn how to work much more collaboratively and efficiently with the entire Marketing organization. We built better cross-functional relationships, learned the tools to help us analyze what our regions needed, expanded our use of digital marketing, and got extremely creative with our virtual events. We also demonstrated the importance of a strong partnership between Sales and Marketing by getting involved in enablement and lead conversion improvements — areas I’d never even thought to investigate before. We’ve shown the value Marketing can bring to Sales and the entire company when given the time, and it isn’t just more leads. When we slow down, think strategically, and become experts on our region’s needs, the impact has nothing to do with events. Calling this past year extremely challenging is the understatement of the century, but I always try to find the silver lining in every situation. In my experience, the pandemic gave field marketers the chance to become stronger business partners to our Sales leaders and own the role of CMO of our region. Interested in pursuing a career in Marketing at MongoDB? We have several open roles on our teams across the globe , and we would love for you to build your career with us!

April 15, 2021
Culture

Introducing: Atlas Operator for Kubernetes

The MongoDB Enterprise Operator serves to automate and manage MongoDB clusters on self-managed infrastructure. While this integration has provided complete control over self-managed MongoDB deployments from a single Kubernetes control plane, we’re taking it a step further by extending this functionality to our fully-managed database—MongoDB Atlas. We’re excited to introduce the trial version of the Atlas Operator for Kubernetes. The Atlas Operator will allow you to manage all your MongoDB Atlas clusters without ever having to leave Kubernetes. Keep your workflow as seamless and optimized as possible by managing the lifecycle of your cloud-native applications from where you want most. With the trial version of this Atlas Operator, you can provision and deploy fully-managed MongoDB Atlas clusters on the cloud provider of your choice through Kubernetes. This provider is especially important for those seeking to unlock the power of multi-cloud with unique tools and services native to AWS, Google Cloud, and Azure without any added complexity to the data management experience. With this new Atlas Operator, you get the best of all clouds with multi-cloud clusters on Atlas , coupled with the freedom to run your entire stack anywhere, all while managed in one central location. The “trial version” simply means it has all the core functionality to provision fully-managed Atlas clusters, but the bells and whistles are yet to come. In addition to encapsulating core Atlas functionality, it ensures Kubernetes Secrets are created for each database user which allows for easier management of sensitive data. The Atlas Operator also allows you to create IP Bindings so your applications can securely access clusters. If you’re interested in using the trial version of the Atlas Operator today, follow our quickstart guide below to get started! Quickstart Below you’ll find the steps to create your first cluster in Atlas using the Atlas Operator. Note that you need to have a running Kubernetes cluster before deploying the Atlas Operator. Register/Login to Atlas and create API Keys for your Organization. This information together with the Organization ID will be used to configure the Atlas Operator access to Atlas. Deploy the Atlas Operator kubectl apply -f \ https://raw.githubusercontent.com/mongodb/mongodb-atlas-kubernetes/main/deploy/all-in-one.yaml Create a Secret containing connection information from step one. This Secret will be used by the Atlas Operator to connect to Atlas: kubectl create secret generic mongodb-atlas-operator-api-key \ --from-literal="orgId=<the_atlas_organization_id>" \ --from-literal="publicApiKey=<the_atlas_api_public_key>" \ --from-literal="privateApiKey=<the_atlas_api_private_key>" \ -n mongodb-atlas-system Create AtlasProject Custom Resource: cat <<EOF | kubectl apply -f - apiVersion: atlas.mongodb.com/v1 kind: AtlasProject metadata: name: my-project spec: name: Test Atlas Operator Project projectIpAccessList: - ipAddress: "0.0.0.0/0" comment: "Allowing access to database from everywhere (only for Demo!)" EOF Create AtlasCluster Custom Resource cat <<EOF | kubectl apply -f - apiVersion: atlas.mongodb.com/v1 kind: AtlasCluster metadata: name: my-atlas-cluster spec: name: "Test-cluster" projectRef: name: my-project providerSettings: instanceSizeName: M10 providerName: AWS regionName: US_EAST_1 EOF (You'll have to wait until the cluster is ready - "status" field shows "ready:true":) kubectl get atlasclusters my-atlas-cluster -o=jsonpath='{.status.conditions[?(@.type=="Ready")].status}' True Create a Secret for the password that will be used to log into Atlas Cluster Database kubectl create secret generic the-user-password \ --from-literal="password=P@@sword%" Create AtlasDatabaseUser Custom Resource (references the password Secret) cat <<EOF | kubectl apply -f - apiVersion: atlas.mongodb.com/v1 kind: AtlasDatabaseUser metadata: name: my-database-user spec: roles: - roleName: "readWriteAnyDatabase" databaseName: "admin" projectRef: name: my-project username: theuser passwordSecretRef: name: the-user-password EOF Shortly the Secret will be created by the Atlas Operator containing the data necessary to connect to the Atlas Cluster. You can mount it into your application Pod and read the connection strings from the file or from the environment variable. kubectl get secrets/test-atlas-operator-project-test-cluster-theuser \ -o=jsonpath="{.data.connectionString.standardSrv}} | base64 -d mongodb+srv://theuser:P%40%40sword%25@test-cluster.peqtm.mongodb.net Stay Tuned for More Be on the lookout for updates in future blog posts! The trial version of the MongoDB Atlas Operator is currently available on multiple marketplaces, but we’ll be looking to make enhancements in the near future. For more information, check out our MongoDB Atlas & Kubernetes GitHub page and our documentation .

April 8, 2021
Developer

MongoDB Connector for Apache Kafka 1.5 Available Now

Today, MongoDB has released version 1.5 of the MongoDB Connector for Apache Kafka! This article highlights some of the key features of this new release in addition to continuing to improve the overall quality & stability of the Connector . DeleteOne write model strategy When messages arrive on Kafka topics, the MongoDB Sink Connector reads them and by default will upsert them into the MongoDB cluster specified in the sink configuration. However, what if you didn’t want to always upsert them? This is where write strategies come in and provide you with the flexibility to define what you want to do with the document. While the concept of write strategies is not new to the connector, in this release there is a new write strategy available called DeleteOneBusinessKeyStrategy . This is useful for when a topic contains records identifying data that should be removed from a collection in the MongoDB sink. Consider the following: You run an online store selling fashionable face masks. As part of your architecture, the website sends orders to a Kafka topic, “web-orders” which upon message arrival kicks off a series of actions such as sending an email confirmation, and inserting the order details into an “Orders” collection in a MongoDB cluster. A sample Orders document: { _id: ObjectId("6053684f2fe69a6ad3fed028"), 'customer-id': 123, 'order-id': 100, order: { lineitem: 1, SKU: 'FACE1', quantity: 1 } } This process works great, however, when a customer cancels an order, we need to have another business process to update our inventory, send the cancellation, email and remove the order from our MongoDB sink. In this scenario a cancellation message is sent to another Kafka topic, “canceled-orders”. For messages in this topic, we don’t just want to upsert this into a collection, we want to read the message from the topic and use a field within the document to identify the documents to delete in the sink. For this example, let’s use the order-id key field and define a sink connector using the DeleteOneBusinessKeyStrategy as follows: "connector.class": "com.mongodb.kafka.connect.MongoSinkConnector", "topics":"FaceMaskWeb.OrderCancel", "connection.uri":"mongodb://mdb1", "database":"FaceMaskWeb", "collection":"Orders", "writemodel.strategy": "com.mongodb.kafka.connect.sink.writemodel.strategy.DeleteOneBusinessKeyStrategy", "document.id.strategy": "com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy", "document.id.strategy.partial.value.projection.type": "AllowList", "document.id.strategy.partial.value.projection.list": "order-id", "value.converter":"org.apache.kafka.connect.json.JsonConverter", "value.converter.schemas.enable":false, "document.id.strategy.overwrite.existing": true Now when messages arrive in the “FakeMaskWeb.OrderCancel” topic, the “order-id” field is used to delete documents in the Orders collection. For example, using the sample document above, if we put this value into the OrderCancel topic { “order-id”: 100 } It would cause the document in the Orders collection with order-id and value 100 to be deleted. For a complete list of write model strategies check out the MongoDB Kafka Connector Sink documentation . Qlik Replicate Qlik Replicate is recognized as an industry leader in data replication and ingestion. With this new release of the Connector, you can now replicate and stream heterogeneous data from data sources like Oracle, MySQL, PostGres and others to MongoDB via Kafka and the Qlik Replicate CDC handler . To configure the MongoDB Connector for Apache Kafka to consume Qlik Replicate CDC events, use “com.mongodb.kafka.connect.sink.cdc.qlik.rdbms.RdbmsHandler” as the value for the change data capture handler configuration parameter. The handler supports, insert, refresh, read, update and delete events. Errant Record Reporting Kafka Connect, the service which manages connectors that integrate with a Kafka deployment, has the ability to write records to a dead letter queue (DLQ) topic if those records could not be serialized or deserialized. Starting with Apache Kafka version 2.6, there was added support for error reporting within the sink connectors. This gives sink connectors the ability to send individual records to the DLQ if the connector deems the records to be invalid or problematic. For example, if you are projecting fields in the sink that do not exist in the kafka message or if your sink is expecting a JSON document and the message arrives in a different format. In these cases an error is written to the DLQ versus failing the connector. Various Improvements As with every release of the connector, we are constantly improving the quality and functionality. This release is no different. You’ll also see pipeline errors now showing up in the connect logs, and the sink connector can now be configured to write to the dead letter queue! Next Steps Download the latest MongoDB Connector for Apache Kafka 1.5 from the Confluent Hub ! Read the MongoDB Connector for Apache Kafka documentation . Questions/Need help with the connector? Ask the Community . Have a feature request? Provide Feedback or a file a JIRA .

April 7, 2021
Developer

Built With MongoDB: Gryphon Online Safety

As friends and coworkers at an IoT company, John Wu and Arup Bhattacharya used to commiserate about the perils the internet posed for their children. It’s a problem most parents can relate to—especially now, when some children spend more than seven hours a day online. One day, John’s daughter saw something online that horrified him, and he and Arup decided they wanted to help bring the internet back into the hands of parents so they could curate online content for their children. With that, Gryphon Online Safety was formed. Gryphon is a cloud-managed network protection platform for homes and small businesses that blocks viruses, malware, and hackers while giving parents the chance to filter content and monitor what their children are doing. With $5.4 million in seed funding, more than 30,000 customers, and a team of 30 employees across three countries, Gryphon is growing quickly. The COVID-19 pandemic further accelerated adoption of Gryphon’s products; with children spending more time on devices at home and hacking activity increasing online, the company has seen a significant boost in users. In this edition of #BuiltWithMongoDB, we talk to CTO and Co-Founder Arup Bhattacharya and Senior Cloud Solutions Architect Sandip Das about the future of internet security and their experience building Gryphon with MongoDB. MongoDB: How has your business changed during COVID-19, given that families have been spending more time at home and online? Arup Bhattacharya: Our business has thrived during COVID. Although we typically add a thousand customers every month, during the pandemic that number has skyrocketed. More people are working from home and more children are attending virtual classes, which has caused families to think more about security and parental controls. Although we typically see two main cycles with our business, one in August and the other around the holiday season, our product isn’t that cyclical. People upgrade their hardware at different times, and when they look for high-performance mesh WiFi routers and security, we are an obvious solution. What’s funny is that while parents deeply appreciate our solution and the security it provides, children often hate us. I stumbled across a Reddit post in which a child wondered how he could get past the access filters his father had set up via Gryphon. Someone responded: “There’s nothing you can do but grow up and buy your own router.” With that said, there’s so much bad content out there—from bullying to games that hurt children—that it’s crucial we allow an easy way for parents to control the experience their children have online. MongoDB: At what point did you implement MongoDB, and what decision framework and criteria led to that decision? Sandip Das: We compared the big databases in terms of what solutions were available. We wanted something freely available for rapid prototyping and that made integration easy. For the back end, we use JavaScript with Node.js runtime, which is easily compatible with MongoDB. In fact, it’s the default choice for database integration. MongoDB owns its library, and combined with how simple the integration was, this made MongoDB a good choice for us. Another big factor was the storage. With MongoDB Atlas, you can have any number of servers, and you can quickly scale up to whatever your demands are. We developed the service from the beginning, and we were managing it ourselves. However, as the load has increased and more customers came on board, we thought it was time to seek out a better and more scalable solution that’s also easy to manage. That’s how we found MongoDB Atlas. With MongoDb Atlas autoscaling, we were able to achieve the flexibility we always wanted, along with automated backup solutions. MongoDB: Arup, you've held several senior engineering positions before becoming Co-founder and CTO of Gryphon. What advice would you give to others looking to follow that path? AB: The CTO position is very critical because it is the bridge between technology and business. The first thing you should think about when starting a company is the pain point you are solving. We started by first asking ourselves how our product will help society. How will it help people improve their lives? The starting point of a company shouldn’t just be to make money overnight. What will keep you motivated through the difficulty of building a business is thinking deeply about how your product will make a positive impact on people’s lives. Second, there inevitably will be low times and high times. At several points in the founder’s journey, you will experience real doubt and wonder whether you can really achieve your goals. The best thing to do is to keep on pushing for the highest-quality product possible. If your product is the best on the market and you are solving a genuine problem, the customers will find and appreciate you. Looking to build something cool? Get started with the MongoDB for Startups program.

April 6, 2021
Applied

Dive Deeper into Chart Data with New Drill-Down Capability

With the latest release of MongoDB Charts, you’re now able to dive deeper into the data that’s aggregated in your visualizations. At a high level, we generally create charts, graphs and visualizations of our data to answer questions about our business or products. Oftentimes, we need to “double click” on those visualizations to get insight into each individual data point that makes up the line, bar, column, etc. How the drill-down functionality works: Step 1: Right click on the data point you are interested in drilling down into Step 2: Click "show data for this item" Step 3: View the data in tabular or document format Each view can be better for different circumstances. For data without too many fields or no nested arrays, it might be quicker and more easily viewed in a table. On the other hand, the JSON view allows you to explore the structure of documents and click into arrays. Scenarios where more detailed information can help: Data visualization use cases are relatively broad spanning, but oftentimes they fall into 3 main categories: monitoring data, finding insights, and embedding analytics into applications. I’ll be focusing on the first two of these three as there are many different ways you could potentially build drilling-down into data via embedded charts. (Read more about our click events and embedded analytics ). For data or performance monitoring purposes , we're not speaking so much about the performance of your actual database and its underlying infrastructure, but the performance of the application or system built on top of the database. Imagine I have an application or website that takes reviews, if I build a chart like the one below where I want to easily see when an interaction hits a threshold that I want to dive deeper into, I now have the ability to quickly see the document that created that data point. This chart shows app ratings given after a user session in an app. For this example, we want to dive into any rating that was below a 3 (out of 5). This scatter plot shows I have two such ratings that cross that threshold. With the drill-down capability, I can easily see all the details captured in that user session. For finding new insights, let’s imagine I’m tracking how many transactions happen on my ecommerce site over time. In the column chart below, you can see I have purchases by month for the last year and a half (note, there’s a gap because this example is for a seasonal business!). Just by glancing at the chart, I can quickly see purchases have increased over time, and my in-app purchases have increased my overall sales. However, I want to see more about the documents that were aggregated to create those columns, so I can quickly see details about the transaction amount and location without needing to create another chart or dashboard filter. In both examples, I was able to answer a deeper level question that the original chart couldn’t answer on it’s own. We hope this new feature helps you and your stakeholders get more out of MongoDB Charts, regardless if you’re new to it or have been visualizing your Atlas data with it for months, if not years! If you haven’t tried Charts yet, you can get started for free by signing up for a MongoDB Atlas and deploying a free tier cluster.

April 6, 2021
Developer

How Three College Friends Became MongoDB Coworkers

Siya Raj Purohit, Chaitanya Varanasi, and Sohail Shaikh first met while attending the University of Texas at Austin (UT Austin) as undergraduate students. Five years after graduating, they found themselves brought together again — this time by MongoDB. I recently sat down with Siya, Chai, and Sohail to talk about this friendship that has been sustained through divergent career paths and continues to grow alongside their roles at MongoDB. Jackie Denner: Tell us about your story leading up to MongoDB. How did the three of you meet and begin to grow your careers? Siya Raj Purohit: I studied electrical and computer engineering at UT Austin from 2010 to 2013. Although Chai, Sohail, and I weren’t in the same year, we became friends from hanging out and working through the rigorous engineering curriculum in the same study lounge. Outside of the engineering building, Austin’s tech scene was exploding; some of my favorite memories with Chai and Sohail are going to tech events together. We met Stephen Wolfram (from WolframAlpha), briefly hung out with Mark Cuban, and crashed many SXSW tech events. Since graduating from college, I’ve lived in four states and worked across startups and venture capital firms. At MongoDB, I help provide founders with the resources they need to push the tech industry forward. Chaitanya (Chai) Varanasi: I am an electrical and computer engineering major from UT Austin, class of 2015 (Hook ‘Em!). Electrical and computer engineering is a fairly small cohort of students who all share a building and sit in the same hall for introductory classes. It is always said that the hottest fires forge the strongest metal. In our situation, we all had to go through grueling labs and coding assignments that would keep us up all night and unite us toward a common goal of passing that class. What started as collaboration on class materials very quickly transitioned into late-night frozen yogurt hangouts, playing Catan, and discovering Austin together. Sohail and I used to travel across the country for various hackathons, which was how we started our careers in software engineering. One of my favorite memories is of Siya taking us to the meetup of a lifetime at the Capital Factory, a startup incubator in Austin; we even got a picture with Stephen Wolfram! After graduating, I joined a large financial institution in Dallas as a software engineer, and then I began my presales journey in the performance space. After realizing the potential of data and understanding the value companies gain from data insights, I joined MongoDB. Sohail Shaikh: My journey in tech began when I was 12 years old and built my first computer. Since then, I have always been fascinated with new technologies and learning more about them. I was a math major at UT Austin, class of 2015. I actually can’t remember the first time I met Siya or Chai, because it seems as if I have known them forever, and I felt an immediate bond with both of them from the start. I have vivid memories of our times at UT together: attending hackathons, collaborating on ideas, and spending a lot of time talking about the future and how we could bring change. In the five-and-a-half years since graduating, I have worked in Palo Alto and Dallas — at a startup, at AppDynamics, and now at MongoDB. I’m excited to be reunited with Chai and Siya; we are all very passionate about making a positive impact in this world, and we are all doing that today at MongoDB! JD: What is your role at MongoDB? SRP: I’m helping the next generation of developers to build great companies. There is so much great talent coming out of universities and startup accelerator programs, and MongoDB for Startups works with developers to ensure they have the right products and services to transform their ideas into innovative companies. More than 1,500 companies have #BuiltWithMongoDB so far — and we’re super excited to continue growing the ecosystem. CV: I am a Senior Solutions Architect. My day-to-day job consists of being a technical partner to our rock-star sales team and performing proof of concepts with our customers to continually grow our MongoDB presence. SS: I am a Solutions Architect at MongoDB for the South Central region. My day-to-day job is working with customers in the presales organization and showcasing why MongoDB is so amazing. JD: How did you maintain your friendship after college? SRP: After college, I lost touch with Chai and Sohail for a couple of years. I moved to Silicon Valley, and although we periodically caught up through mutual friends, we didn’t really reconnect until we all joined MongoDB. I joined a few weeks before Chai (mostly to be part of his welcoming crew) and was ecstatic when Sohail told us he was joining MongoDB too. Now, we have a private Slack channel (named after one of our favorite Bollywood films) where we talk about our jobs and lives and also share cute memes and gifs. CV: Sohail and I both lived in Dallas and worked on the same team at a previous company. We have done multiple trips together and spent way too many nights eating sushi and Whataburger! Siya and I lost touch for a little because of the distance, but we were able to make up for lost time after joining MongoDB. SS: I am horrible at maintaining relationships, but Chai and Siya keep me in check (it’s just the type of people they truly are). I would meet Chai once a year on a group trip, and one day I called him to learn more about his new role at AppDynamics; he didn’t hesitate to refer me in. Next thing I knew, I was working with him on his team. Two-and-a-half years later, Chai decided to move to MongoDB, and I couldn’t resist. After working with Chai, I am now convinced I talk to him more than his wife does. Siya and I reconnected during the pandemic through a socially distanced meetup at a park while I was visiting San Francisco. Now that we both work for MongoDB, our friendship has picked up right where we left off. JD: All three of you joined MongoDB during the COVID-19 pandemic. How was the remote onboarding experience? SRP: Honestly, I was sort of nervous about joining remotely. I had left a company where I had really strong relationships with my coworkers, and it was daunting to imagine building new connections while being entirely remote. During my interview process, I asked for advice on how to best onboard. I was recommended the book The First 90 Days , which provided a great framework and onboarding roadmap. The MongoDB onboarding week itself was awesome — I met many people across the company, joined a few employee affinity groups (MongoDB Women is my favorite!), and learned about the lives of my coworkers beyond work — I even virtually met some of their babies and pets! I’m really excited to spend time with coworkers in person once it’s safer to do so. CV: I had a phenomenal experience with onboarding. Everyone at MongoDB has been nothing short of helpful. This was the first time in my life that I got to meet an entire executive team in a small group setting within the first month of joining the company. Each MongoDB executive hosts a coffee chat once a quarter, which is a great way to get to know them more personally. That kind of exposure is unparalleled, and it truly showed me how a great culture was supported from both bottom up and top down. SS: Onboarding at MongoDB is the best I have ever seen! Training and role clarity have been phenomenal, even in a remote setting. The material is organized and easy to grasp, and I don’t feel as if I have been left to figure everything out on my own. The team is extremely helpful in answering all of my questions and helping me grow. In Sales, there is also boot camp, which is divided up into two parts for my role. Boot camp lasted for a month to avoid any Zoom fatigue (given that we are all virtual), which also gave us more time to work on our assignments and properly learn the lay of the land. JD: What are you most excited about? SRP: I am so excited about Chai moving to NYC so we can work out of the same office when it reopens. I’ve already mapped out the top 10 bubble tea shops in NYC for us to visit. CV: I am ready to explore New York with Siya and have future MongoDB lunches together. Sohail and I are ready to tackle our Sales Kickoff and have fun when we return to normal situations after the pandemic. We are all career-driven individuals, and I am excited to see how we can uplift each other as a family. SS: I am most excited to be learning about the database space and contributing to growing the business. I am also super excited to see where MongoDB goes in the future. As one of the world’s fastest-growing databases, it feels as if we are on a rocket ship. JD: What advice would you give to others who are looking for a new role? SRP: Recruiting is always hard. Find unique ways to showcase why you’re a fit for a certain role or company — passion is seen and rewarded. CV: Always keep your connections and networks alive. Keep interacting with the folks you care about. I am nothing without my work friends and my work family. MongoDB is on a rocket ship right now, and you will absolutely love working here. SS: Don’t be afraid to take a risk in your careers, and put in an application to MongoDB today! We love working with talented, hard-working folks, and the grass is truly green on this side! Interested in pursuing a career at MongoDB? We have several open roles on our teams across the globe and would love for you to build your career with us!

April 1, 2021
Culture

Ready to get Started with MongoDB Atlas?

Start Free