Take Advantage of Low-Latency Innovation with MongoDB Atlas, Realm, and AWS Wavelength
Build a Single View of Your Customers with MongoDB Atlas and Cogniflare's Customer 360
The key to successful, long-lasting commerce is knowing your customers. If you truly know your customers, then you understand their needs and wants and can identify the right product to deliver to them—at the right time and in the right way. However, for most B2C enterprises, building a single view of the customer poses a major hurdle due to copious amounts of fragmented data. Businesses gather data from their customers in multiple locations, such as ecommerce platforms, CRM, ERP, loyalty programs, payment portals, web apps, mobile apps and more. Each data set can be structured, semi-structured or unstructured, delivered as stream or require batch processing, which makes compiling already fragmented customer data even more complex. This has led some organizations to bespoke solutions, which still only provide a partial view of the customer. Siloed data sets make running operations like customer service, targeted marketing and advanced analytics—such as churn prediction and recommendations—highly challenging. Only with a 360 degree view of the customer can an organization deeply understand their needs, wants and requirements, as well as how to satisfy them. A single view of that 360 data is therefore vital for a lasting relationship. In this blog, we’ll walk through how to build a single view of the customer using MongoDB’s database and Cogniflare’s Calledio Customer 360 tool. We’ll also explore a real-world use case focused on sentiment analysis. Building a single view with Calleido's Customer 360 With a Customer 360 database, organizations can access and analyze various individual interactions and touchpoints to build a holistic view of the customer. This is achieved by acquiring data from a number of disparate sources. However, routing and transforming this data is a complex and time-consuming process. Many of the existing Big Data tools often aren’t compatible with cloud environments. These challenges inspired Cogniflare to create Calleido . Figure 1: Calleido Customer 360 Use Case Architecture Calleido is a data processing platform built on top of battle-tested open source tools such as Apache NiFi. Calleido comes with over 300 processors to move structured and unstructured data from and to anywhere. It facilitates batch and real-time updates, and handles simple data transformations. Critically, Calleido seamlessly integrates with Google Cloud and offers one-click deployment. It uses Google Kubernetes Engine to scale up and down based on the demand, and provides an intuitive and slick low-code development environment. Figure 2: Calleido Data Pipeline to Copy Customers From PostgreSQL to MongoDB A real-world use case: Sentiment analysis of customer emails To demonstrate the power of Cogniflare’s Calleido , MongoDB Atlas , and the Customer 360 view, consider the use case of conducting a sentiment analysis on customer emails. To streamline the build of a Customer 360 database, the team at Cogniflare created flow templates for implementing data pipelines in seconds. In the upcoming sections, we’ll walk through some of the most common data movement patterns for this Customer 360 use case and showcase a sample dashboard. Figure 3: Sample Customer Dashboard The flow commences with a processor pulling IMAP messages from an email server (ConsumeIMAP). Each new email that arrives into the chosen inbox (e.g. customer service), triggers an event. Next, the process extracts email headers to determine topline details about the email content (ExtractEmailHeaders). Using the sender's email, Calleido identifies the customer (UpdateAttribute) and extracts the full email body by executing a script (ExecuteScript). Now, with all the data collected, a message payload is prepared and published through Google Cloud Platform (GCP) Pub/Sub (Kafka can also be used) for consumption by downstream flows and other services. Figure 4: Translating Emails to Cloud PubSub Messages The GCP Pub/Sub messages from the previous flow are then consumed (ConsumeGCPPubSub). This is where the power of MongoDB Atlas integration comes in as we verify each sender in the MongoDB database (GetMongo). If a customer exists in our system, we pass the email data to the next flow. Other emails are ignored. Figure 5: Validating Customer Email with MongoDB and Calleido Analysis of the email body copy is then conducted. For this flow, we use a processor to prepare a request body, which is then sent to Google Cloud Natural Language AI to assess the tone and sentiment of the message. The results from the Language Processing API then go straight to MongoDB Atlas so they can be pulled through into the dashboard. Figure 6: Making Cloud AutoML Call with Calleido End result in the dashboard: The Customer 360 database can be used in internal back-office systems to supplement and inform customer support. With a single view, it’s quicker and more effective to troubleshoot issues, handle returns and resolve complaints. Leveraging information from previous client conversations ensures each customer is given the most appropriate and effective response. These data sets can then be fed into analytics systems to generate learnings and optimizations, such as associating negative sentiment with churn rate. How MongoDB's document database helps In the example above, Calleido takes care of copying and routing data from the business source system into MongoDB Atlas, the operational data store (ODS). Thanks to MongoDB’s flexible data structure, we can transfer data in its original format, and subsequently implement necessary schema transformations in an iterative manner. There is no need to run complex schema migrations. This allows for the quick delivery of a single view database. Figures 7 & 8: Calleido Data Pipelines to Copy Products and Orders From PostgreSQL to MongoDB Atlas Calleido allows us to make this transition in just a few simple steps. The tool runs a custom SQL query (ExecuteSQL) that will join all the required data from outer tables and compile the results in order to parallelize the processing. The data arrives in Avro format, then Calleido converts it into JSON (ConvertAvroToJSON) and transforms it to the schema designed for MongoDB (JoltTransformJSON). End result in the Customer 360 dashboard: MongoDB Atlas is the market-leading choice for the Customer 360 database. Here are the core reasons for its world-class standard: MongoDB can efficiently handle non-standardized schema coming from legacy systems and efficiently store any custom attributes. Data models can include all the related data as nested documents. Unlike SQL databases, MongoDB avoids complicated join queries, which are difficult to write and not performant. MongoDB is rapid. The current view of a customer can be served in milliseconds without the need to introduce a caching layer. The MongoDB flexible schema model enables agility with an iterative approach. In the initial extraction, the data can be copied nearly exactly as its original shape. This drastically reduces latency. In subsequent phases, the schema can be standardized and the quality of the data can be improved without complex SQL migrations. MongoDB can store dozens of terabytes of data across multiple data centers and easily scale horizontally. Data can be shared across multiple regions to help navigate compliance requirements. Separate analytics nodes can be set up to avoid impacting performance of production systems. MongoDB has a proven record of acting as a single view database, with legacy and large organizations up and running with prototypes in two weeks and into production within a business quarter. MongoDB Atlas can autoscale out of the box, reducing costs and handling traffic peaks. The data can be encrypted both in transit and at rest, helping to accomplish compliance with security and privacy standards, including GDPR, HIPAA, PCI-DSS, and FERPA. Upselling the customer: Product recommendations Upselling customers is a key part of modern business, but the secret to doing it successfully is that it’s less about selling and more about educating. It’s about using data to identify where the customer is in the customer journey, what they may need, and which product or service can meet that need. Using a customer's purchase history, Calleido can help prepare product recommendations by routing data to the appropriate tools such as BigQuery ML. These recommendations can then be promoted through the call center and marketing teams for both online or mobile app recommendations. There are two flows to achieve this: preparing training data and generating recommendations: Preparing training data First, appropriate data from PostgreSQL to BigQuery is transferred using the ExecuteSQL processor. The data pipeline is scheduled to execute periodically. In the next step, appropriate data is fetched from PostgreSQL, dividing it to 1K row chunks with the ExecuteSQLRecord processor. These files are then passed to the next processor which uses load balancing enabled to utilize all available nodes. All that data then gets inserted to a BigQuery table using the PutBigQueryStreaming processor. Figure 9: Copying Data from PostgreSQL to BigQuery with Calleido Generating product recommendations Next, we move on to generating product recommendations. First, you must purchase Big Query capacity slots, which offer the most affordable way to take advantage of BigQuery ML features. Here, Calleido invokes an SQL procedure with the ExecuteSQL processor, then ensures that the requested BigQuery capacity is ready to use. The next processor (ExecuteSQL) executes an SQL query responsible for creating and training the Matrix Factorization ML model using the data copied by the first flow. Next in the queue, Calleido uses the ExecuteSQL processor to query our trained model to acquire all the predictions and store them in a dedicated BigQuery table. Finally, the Wait processor waits for both capacity slots to be removed, as they are no longer required. Figure 10 & 11: Generating Product Recommendations with Calleido Then, we remove old recommendations through the power of two processors. First, the ReplaceText processor updates the content of incoming flow files, setting the query body. This is then used by the DeleteMongo processor to perform the removal action. Figure 12: Remove Old Recommendations The whole flow ends with copying Recommendations to MongoDB. The ExecuteSQL processor fetches and aggregates the top 10 recommendations per user, all in chunks of 1k rows. Then, the following two processors (ConvertAvroToJSON and ExecuteScript) prepare data to be inserted into the MongoDB collection, by the PutMongoRecord processor. Figure 13: Copy Recommendations to MongoDB End result in the Customer 360 dashboard (the data used here in this example is autogenerated): Benefits of Calleido's 360 customer database on MongoDB Atlas Once the data is available in a centralized operational data store like MongoDB, Calleido can be used to sync it with an analytics data store such as Google BigQuery. Thanks to the Customer 360 database, internal stakeholders can then use the data to: Improve customer satisfaction through segmentation and targeted marketing Accurately and easily access compliance audits Build demand planning forecasts and analyses of market trends Reward customer loyalty and reduce churn Ultimately, a single view of the customer enables organizations to deliver the right message to prospective buyers, funneling those at the brand awareness stage into the conversion stage and ensures retention and post sales mechanics are working effectively. Historically, a 360 view of the customer was a complex and fragmented process, but with Cogniflare’s Calleido and MongoDB Atlas, a Customer 360 database has become the most powerful and cost efficient data management stack that an organization can harness.
Real-Time Analytics with MongoDB Change Streams
Querying over a million data points to compute analytics takes time — which is a problem when you need real-time updates. With MongoDB, you can meet the challenges of modern analytics by delivering insights directly on fresh, operational data without time-consuming and fragile ETL. In this 20-minute presentation, you’ll learn how organizations like Rent the Runway, Verizon, and Pitney Bowes are using MongoDB for analytics to help them visualize unstructured data, deliver real-time intelligence, scale diverse workloads in the cloud, and harness the power of AI. Shrey Batra, MongoDB User Group lead, will discuss how MongoDB creates patterns of data modelling and application architecture, so any new use case can be solved in a plug-and-play type architecture, without affecting your application code. Topics include: What is Real-Time Analytics? Modelling High Volume Metrics MongoDB Change Streams [Plug-n-Play] MongoDB Atlas Triggers Solving a Real Life Example
TECH REMIX WITH MONGODB
MongoDB hosted a virtual session, specifically for 17LIVE. MongoDB product experts and your dedicated Account Manager will share best practices for working on Replica Set, Sharing and provide examples of how you can innovate through MongoDB. What we will cover during the session: Replica Set What is ReplicaSet Write and writeConcern Read and readConcern Sharding What is Sharding The Components - mongos, config server Shard Key Strategy Advanced Topic about Sharding
MongoDB’s Application Data Platform via Edge through Cloud to Mobile
Industry of Things World 2021 This session provides an update on why MongoDB powers some of the most popular IoT platforms today and how our application data platform just got even better with native support for time series data. Watch the recording of this presentation by Felix Reichenbach, Princ. Industry Solutions at MongoDB, speaking at this year's Industry of Things World! What you'll learn: How to reduce complexity and data duplication with MongoDB’s application data platform Learn about our new native time series support Popular IoT Use Cases with MongoDB
Transforming Customer Experience With MongoDB Atlas Search
Download Full-text search is essential for modern applications, but building it is hard. Developers either try to: Contort their database to handle search queries, quickly running up against feature limitations and performance overhead. Bolting on specialized search engines to their database, dramatically increasing the complexity of their application infrastructure. MongoDB Atlas Search gives you a much better way. It combines the power of Apache Lucene — the same technology underpinning the world’s most popular search engines — with the developer productivity, scale, and resilience of the MongoDB Atlas database. A couple of API calls or clicks in the Atlas UI and you instantly expose your data to sophisticated, relevance-based search experiences that boost engagement and improve customer satisfaction. Download this whitepaper to learn more about challenges of implementing search today and how that’s transformed with Atlas Search. We’ll discuss the ideal use cases for Atlas Search, along with those requirements where you may be better served considering alternative approaches. We’ll wrap up with how you can get started with Atlas Search today.
DIRT and the High Cost of Complexity: How Data Architecture Becomes a Tax on Innovation
As the data requirements for applications become increasingly sophisticated, organizations often find that their data architecture, and in particular their relational databases, can’t keep up. In response, developers cobble together niche databases, with cumbersome pipelines to move data between them. Or, organizations move some of their applications and data to the cloud, grabbing a bunch of off-the-shelf software that doesn’t work or play well together. Either way, you’re paying DIRT: The data and innovation recurring tax. That’s when your team is stuck supporting a sprawling, creaky architecture when they could be working on new features that the business needs and customers will love. In this paper, you’ll learn: How organizations accumulate DIRT The essential requirements for a modern data architecture How an application data platform can help boost innovation How to start eliminating your organization’s DIRT, and help your developers become more productive Fill out the form to read more!
MongoDB Employees Share Their Coming Out Stories: (Inter)National Coming Out Day 2021
National Coming Out Day is celebrated annually on October 11 and is widely recognized in the United States. MongoDB proudly supports and embraces the LGBTQIA+ community across the globe, so we’ve reimagined this celebration as (Inter)National Coming Out Day. In our yearly tradition of honoring (Inter)National Coming Out Day, we asked employees who are members of the LGBTQIA+ community to share their coming out experiences. These are their stories. Jamie Ivanov , Escalation Manager For as long as I can remember, I always wanted to play with dolls and felt closer to my female cousins. This was rather difficult for someone who is a male at birth being brought up in a fairly conservative family. At a young age, I knew that I was different but lacked a way to describe it. I certainly didn't have the support I needed, so I was brought up as a male. My father went out of his way to “make a man out of me” and toughen me up in ways that weren't exactly the most productive. Going through school, I still knew that I was different because I kept feeling attracted to both genders, but I was too afraid to admit to it. I found a youth group for LGBT teenagers that gave me a safe place to be myself and admit to people who I really was. Outside of that group was still pretty scary; I knew that I had to be straight or I would risk being beaten up or harassed, so I tried to push my queerness aside. In my 30s, after going through the Army and having three children, I realized that I couldn't keep pretending anymore -- who I was wasn't the true me. I started telling people that I was bisexual and hoping that they wouldn't see me as less of a person. Most of the responses I received were "yeah, we kinda figured.” Having that weight off of my shoulders was immensely relieving but something still wasn't quite right; while admitting that helped explain who I was interested in, it still didn't explain who I was. Through a series of fortunate unfortunate events, a lot of the facade I had built up for so many years came down, and I realized that who I was didn't match the body that I was given. It was terrifying to talk to anyone about how I was feeling or who I was, but I finally told people that I am a transgender woman. It was one of the scariest things that I have ever done. Some people didn't understand, and I did lose some family over it, but most people accepted me for who I am with open arms! Since being true to myself, more weight has been lifted off of me, and my only regret is not having the resources and courage to admit who I really was years and years ago. Since I've come out as bi/pansexual and a transgender woman, I've built stronger relationships and felt much more comfortable with myself, even to the point of liking photos of myself (which is something I've always hated and realized it was because it wasn't the real me). When a MongoDB recruiter reached out to me, I asked him the same question I asked other recruiters: "How LGBT friendly is MongoDB (with an emphasis on the transgender part)?" The response I got back from my technical recruiter Bryan Spears was the best response I had received from ANY recruiter, or company, and was the deciding factor in why I chose to work at MongoDB. Here’s what he said: “MongoDB is a company that truly does its best to follow our values like embracing the power of differences; we have many employees who identify as LGBTQ+ or are allies of the LGBTQ+ community. We also have two ERGs, MongoDB Queeries and UGT (Underrepresented Genders in Tech), which both aim to create and maintain a safe environment for those identifying as LGBTQ+ or questioning. From a benefits standpoint, we have expanded the amount of WPATH Standards of Care services available for people who identify as Transgender, Gender Nonconforming, or Transsexual through Cigna. While I know none of the information I have shared tells you what life is like at MongoDB, I hope that it shows we are doing our best to make sure that everyone feels respected and welcome here.” I didn't always have the support I needed to be myself at some previous jobs but MongoDB has raised the bar to a level that is hard to compete with. I'm happy to finally find a place that truly accepts me for who I am. Ryan Francis , VP of Global Demand Generation & Field Marketing Growing up in the 90s in what I used to call “the buckle of the Bible Belt,” I did not believe coming out was in the cards. In fact, I would sit up at night to devise my grand escape to New York City after being disowned (how I planned on paying for said escape remains unknown). I was, however, out to my best friend, Maha. During the summer between my Sophomore and Junior years of high school, I spent time with her family in Egypt. On the return trip, I bought a copy of The Advocate to learn about the big gay life that awaited me after my great escape. Later that month, my mother stumbled upon that magazine when she was cleaning the house. She waited six months to bring it up, but one day in January sat me down in the living and asked, “Are you gay?” I paused for a moment and said… “yup.” She started crying and thanked me for being honest with her. A month later, she picked up a rainbow coffee mug at a yard sale and has been Mrs. PFLAG ever since, organizing pride rallies in our little Indiana hometown and sitting on the Episcopal church vestry this year in order to push through our parish’s blessing of same-sex marriage. Needless to say, I didn’t have to escape. My father was also unequivocally accepting. This is a good thing because my sister Lindsay is a Lesbian, so they sure would have had a tough time given 100% of their kids turned out gay. Lindsay is the real hero here who stayed in our homeland to raise her children with her wife, changing minds every day so that, hopefully, there will be fewer and fewer kids who actually have to make that great escape. Angie Byron , Principal Community Manager Growing up in the Midwest in the 80s and 90s, I was always a “tomboy;” as a young kid, I gravitated to toys like Transformers and He-Man and refused to wear pink or dresses. Since we tended to have a lot in common, most of my best friends growing up were boys; I tended to feel awkward and shy around girls and didn’t really understand why at the time. I was also raised both Catholic and Bahá’í, which led to a very interesting mix of perspectives. While both religions have vastly different belief and value systems, the one thing they could agree on was that homosexuality was wrong (“intrinsically immoral and contrary to the natural law” in the case of Catholicism, and “an affliction that should be overcome” in the case of Bahá’í). Additionally, being “out” as queer at that time in that part of the United States would generally get you made fun of, if not the everlasting crap kicked out of you, so finding other queer people felt nearly impossible. As a result, I was in strong denial about who I was for most of my childhood and gave several valiant but ultimately failed attempts at the whole “trying to date guys” thing as a teenager (I liked guys just fine as friends, but when it came to kissing and stuff it was just, er… no.). In the end, I came to the reluctant realization that I must be a lesbian. I knew no other queer people in my life, and so was grappling with this reality alone, feeling very isolated and depressed. So, I threw myself into music and started to find progressively more and more feminist/queer punk bands whose songs resonated with my experiences and what I was feeling: Bikini Kill, Team Dresch, The Need, Sleater-Kinney, and so on. I came out to my parents toward the end of junior high, quite by accident. Even though I had no concrete plan for doing so, I always figured Mom would be the more accepting one, given that she was Bahá’i (a religion whose basic premise is the unity of religions and equality of humanity), and I’d have to work on Dad for a bit, since he was raised Catholic and came from a family with more conservative values from an even smaller town in the midwest. Imagine my surprise when one day, Mom and I were watching Ricky Lake or Sally Jesse Raphael or one of those daytime talk shows. The topic was something like “HELP! I think my son might be gay!” My mom said something off-handed like “Wow, I don’t know what I would do if one of you came out to me as gay...” And, in true 15-year old angsty fashion, I said, “Oh YEAH? Well you better FIGURE IT OUT because I AM!” and ran into my room and slammed the door. I remember Mom being devastated, wondering what she did wrong as a parent, and so on. I told her, truly, nothing. My parents were both great parents; home was my sanctuary from bullying at school, and my siblings and I were otherwise accepted exactly as we were, tomboys or otherwise. After we’d finished talking, she told me that I had better go tell my father, so I begrudgingly went downstairs. “Dad… I’m gay.” Instead of a lecture or expressing disdain, he just said, “Oh really? I run a gay support group at your Junior High!” and I was totally mind blown. Bizarro world. He was the social worker at my school, so this makes sense, but it was the exact opposite reaction that I was expecting. An important life lesson in not prejudging people. When I moved onto high school, we got… drumroll ... the Internet. Here things take a much happier turn. Through my music, I was able to find a small community of fellow queers (known as Chainsaw), including a ton of us from various places in the Midwest. I was able to learn that I was NOT a freak, I was NOT alone, there were SO many other folks who felt the exact same way, and they were all super rad! We would have long talks into the night, support each other through hardships, and more than a few of us met each other in person and hung out in “real life.” Finding that community truly saved my life, and the lives of so many others. (Side-note: This is also how I got into tech because the chat room was essentially one gaping XSS vulnerability, and I taught myself HTML by typing various tags in and seeing how they rendered.) I never explicitly came out to anyone in my hometown. I was too scared to lose important relationships (it turns out I chose my friends well, and they were all completely fine with it, but the prospect of further isolating myself as a teenager was too terrifying at the time). Because of that, when I moved to a whole new country (Canada) and went to college, the very first thing I did on my first day was introduce myself as “Hi, I’m Angie. I’ve been building websites for fun for a couple of years. Also, I’m queer, so if you’re gonna have a problem with that, it’s probably best we get it out of the way now so we don’t waste each others’ time.” Flash forward to today, my Mom is my biggest supporter, has rainbow stickers all over her car, and has gone to dozens of Pride events. Hacking together HTML snippets in a chat room led to a full-blown career in tech. I gleaned a bit more specificity around my identity and now identify as a homoromantic asexual . Many of those folks I met online as a teenager have become life-long friends. And, I work for a company that embraces people for who they are and celebrates our differences. Life is good. Learn more about Diversity & Inclusion at MongoDB Interested in joining MongoDB? We have several open roles on our teams across the globe and would love for you to transform your career with us!
MongoDB 101 Back to Basics Webinar
28 October 2021, 1PM AEDT Does NoSQL feel like a bunch of NoSense to you? If so, join us for this webinar! In Back to Basics, we’ll teach you the fundamentals of the world’s most popular NoSQL database, MongoDB. We start with a brief overview of how data is stored in MongoDB and compare that to the legacy table-based (relational) structure you may be used to. After that, we dive deep into a demo! The demo will include: How to create a MongoDB database using MongoDB Atlas (for free!) A walk through the basic CRUD (create, read, update, and delete) operations Some tips and tricks for better efficiency/productivity After viewing this webinar, you'll be able to confidently use MongoDB to build your next app. Who will get the most out of this webinar? Developers, DBAs, or students.