Transforming Credit Management with Credisense and MongoDB
Credit Management is a grind -- clunky, time consuming and laden with risk.
It requires millions of dollars to capture consumer attention and nurture through the sales cycle. And then comes the arduous credit assessment, throwing a wrench into the promise of a seamless digital customer experience. In fact, up to 90% of bank new customer applications drop out due to slow onboarding 1.
Even once a deal closes, there is still plenty of work to do. The next hurdle is invoice collections, with default and delinquency rates averaging anywhere between 0.2-54.5% internationally 2.
In today’s digitally-driven market, consumers are demanding quicker turnaround times and instant approvals. This makes simplifying and streamlining the entire credit management process more critical than ever.
As MongoDB’s OEM business continues to rapidly expand, it’s a personal priority to work with organizations who are solving serious market needs with the most innovative technology. Credisense, our newest OEM partner, and their MongoDB- powered, full end-to-end origination and credit decisioning solution is a phenomenal example of this.
They’re already making splashes worldwide. For example: CTOS Data Systems (Malaysia’s largest credit reporting agency) is enabling banks, utilities, non-bank credit issuers, fintechs and lenders in the P2P lending space to make real time credit decisions using the Credisense platform, enabling things like instant loan approvals for credit cards and auto loans!
I had the opportunity to discuss the Credisense platform and the data technology behind it with Richard Brooks, Co-founder and Director.
Tell us a bit about yourself and the genesis of the company?
Our three co-founders have different, but complementary backgrounds. I have worked for bureau and data companies my entire career and been involved in the automation side quite extensively. Our second co-founder and CEO, Sean Hywood, is a software expert having built up several software companies over his career focusing on low-code technologies. We combined our knowledge with the technical expertise of our third co-founder and CTO, Waylon Turney-Mizen, with the vision of providing enterprise grade functionality to organizations of all sizes. The aim is to allow all businesses to make smarter decisions, faster.
For anyone that isn’t familiar with Credisense yet, could you describe why you set out to build this and the problem it’s solving?
Credit is a highly regulated, complex, often manual and costly process. McKinsey 3 rightly points out there are five key pressures on credit providers currently:
- Changing customer expectations, specifically digital and the customer experience
- Tighter regulatory controls such as AML/CFT and GDPR
- Data management, increasing reliance on clean data for analysis and decisions
- Market disruptor such as P2P lenders and digital banks
- Cost pressures driving down returns
There are some sobering stats that show how important these are, such as over $200 billion dollars 4 of regulatory fines in the US alone since the GFC, to the fact that traditional lenders have lost over 30 percent of personal loan market share 5 to agile financial technology companies. All these add up to some serious issue for business, some that even threaten their very existence. Our aim when creating Credisense was to tackle these issues, both by assisting traditional corporates to embrace this digital strategy, to providing this same technology and expertise to smaller businesses so they can compete and level the playing field.
How would you describe the platform and the unique advantages that Credisense gives its customers?
Our platform is born in the cloud and offers a “no-code” build capability allowing organizations to build out the functionality internally and grow the solution with their business. We have a unique graphical interface, and this coupled with the “no-code” technology allows business people -- not IT -- to build, own and manage the system.
The platform itself revolves around the decision and scoring engine which powers the advanced assessment and risk decisions for organizations.
Our MongoDB backend means we can confidently scale to handle millions of credit applications and still support real-time workflow and decision making in seconds.
How did you land on MongoDB to help you solve these challenges?
We needed a database to support a minimum of 100,000 transaction a day across a cloud platform. There are only a handful of NoSQL databases that can support the level of transaction with the ability to further scale if required. MongoDB ticked all the boxes. Add that to MongoDB’s great documentation security, tooling, support and APIs, and it made MongoDB the right choice for our development teams.
What advice would you give someone who is considering using MongoDB for their next project?
MongoDB offered us extensibility to be on-premises, which is something other cloud database platforms would not offer. It made sense to go with a database platform that offered both so that we could in turn offer this to our customers that require data to be held within their own environments for security reasons. Also, reach out and talk to MongoDB early in your process. The support they give you up front will help ensure you’re making the best decisions.
How are you securing MongoDB?
We utilize MongoDB Atlas for our Continuous Integration and Testing environment and will have a managed service offering. This is secured with an IP whitelist, secure password and SSL connection which was easy with Atlas and Atlas Professional.
We also have a customer-managed deployment secured out of the box behind a VPN that connects the app server to the MongoDB server. It also utilizes a strong username/password combination with minimum length and character requirements.
Through our OEM arrangement with MongoDB, we package MongoDB Enterprise as part of our product to ensure our customers have highly secure and enterprise-grade solutions.
Where have you deployed MongoDB? On-premises, in the cloud, via MongoDB Atlas? What tools are you using to deploy, monitor MongoDB?
All! The requirement for extensibility across platforms without any changes to the code was one of the key reasons for MongoDB selection.
MongoDB Atlas removes operational overhead and mitigates risk through automating many of the manual processes (configuring operating system, upgrades, backups and restores). This means we can focus on ensuring our customers have the robust platform they need to provide instant loan approvals.
We also have a production environment on-premises. Soon, we will introduce the use of MongoDB Cloud Manager for monitoring and alerts of on-premises production environments. With over 100 metrics and proactive alerting, we’ll be able to catch issues before they arise.
Fraud Detection at FICO with MongoDB and Microservices
FICO is more than just the FICO credit score. Founded in 1956, FICO also offers analytics applications for customer acquisition, service, and security, plus tools for decision management.
One of those applications is the Falcon Assurance Navigator (FAN), a fraud detection system that monitors purchasing and expenses through the full procurement to pay cycle. Consider an expense report: the entities involved include the reporter, the approver, the vendor, the department or business unit, the expense line items, and more. A single report has multiple line items, where each line may be broken into different expense codes, different budget sources, and so on. This translates into a complicated data model that can be nested 6 or 7 layers deep – a great match for MongoDB’s document model, but quite hard to represent in the tabular model of relational databases.
FAN Architecture Overview
The fraud detection engine consists of a series of microservices that operate on transactions in queues that are persisted in MongoDB:
- Each transaction arrives in a receiver service, which places it into a queue.
- An attachment processor service checks for an attachment; if one exists, it sends it to an OCR service and stores the transaction enriched with the OCR data.
- A context creator service analyzes it and associates it with any past transactions that are related to it.
- A decision execution engine runs the rules that have been set up by the client and identifies violations.
- One or more analytics engines review transactions and flag outliers.
- Now decorated with a score, the transaction goes to a case manager service, which decides whether to create a case for human follow-up based on any identified issues.
- At the same time, a notification manager passes updates on the processing of each transaction back to the client’s expense/procurement system.
To learn more, watch FICO’s presentation at MongoDB World 2018.
How Planable Uses Mongodb Atlas to Help Social Media Teams Move Their Creative Processes Forward
In 2015, Nicolae Gudumac was working at a social media agency managing hundreds of campaigns for clients. However, with each campaign requiring multiple rounds of review, he needed a way to streamline the disjointed feedback loop and bring everyone onto the same page. Along with his co-founders, Xenia Muntean and Vlad Calus, he began building a platform that would streamline this time-consuming process for social media managers, agencies, and their clients.
The team created Planable, a platform that simplifies planning, visualizing, and approving social media posts. The tool feels like a live mock-up of the social feed, making it easy for teams to collaborate and give real-time feedback in a familiar format.
Prioritizing Developer Velocity From the Beginning
As the newly minted CTO, Nicolae decided to use MongoDB’s document data model to help his team innovate quickly to stay ahead of the ever-changing social media landscape. “With social media content, requirements are constantly changing as platforms evolve and introduce new formats. Having a flexible data model has been a boon to our productivity.”
He also needed to build a foundation that could scale with them as the business grew. Since Planable is a collaboration tool that needs to keep users synced in real-time, Nicolae built the tool on top of Node.js and websockets. The team harnessed MongoDB’s oplog tailing functionality to send real-time updates to all connected users when relevant data changed and recently started to leverage MongoDB change streams to make this process more simple and scalable. To easily scale their app servers at peak times, Nicolae chose to run Planable on AWS container service.
Migrating to MongoDB Atlas
For managing their MongoDB deployment, the team started off using Compose.com but were having issues with restoring backups and only had access to a very limited set of configuration options. Compose also charged a premium for upgrading their storage engine to WiredTiger. MongoDB Atlas, with its queryable backup snapshots, automated upgrades, and configuration flexibility — especially the ease at which clusters can be scaled horizontally — looked very appealing.
We needed to be able to provide our clients with a platform that is as reliable as we are. Atlas’ native cloud first and scale-out architecture aligned well with our increasing performance demands and usage growth.Nicolae Gudumac, CTO, Planable
When Planable was accepted into the MongoDB Startup Accelerator, the timing was right to make the move to MongoDB Atlas. The team had customers all over the world and couldn’t afford any downtime with the migration. They used the Atlas Live Migration Service to move over their data from Compose with no downtime.
Currently, the team of 6 is split between engineering and business. With a small engineering team, they’re hyper-focused on product improvements and new features that will drive the business forward. Atlas features such as the Real-Time Performance panel and the Performance Advisor, which monitors clusters for slow queries and automatically suggests indexes to improve performance, allow the team to dedicate more of their attention to application improvements. According to Nicolae, “The Performance Advisor has made indexing the database and optimizing queries a no-brainer."
Queryable backups have also helped the team quickly address customer questions and as Nicolae recalls, “It’s saved us numerous times when someone accidentally dropped/updated a few documents and they needed to be restored. We’ve managed to quickly inspect and restore data in just a few clicks.”
The team is looking forward to the upcoming release of MongoDB Charts which is currently available in beta. “Charts will enable our marketing and business team to gain insights from our database, without resorting to sophisticated and expensive BI tools” says Nicolae.
Planable is bringing thousands onto their platform each month and are well on their way to becoming the default tool for social media collaboration.
MongoDB Atlas has helped this fast-growing startup focus on helping social media teams move their process forward and allows Nicolae to give valuable time back to his team. “There’s no doubt in my mind that moving to MongoDB Atlas has increased our team’s productivity.”
Millions of Users and a Developer-Led Culture: How Blinkist Powers its Berlin Startup on MongoDB Atlas
Not unlike other startups, Blinkist grew its roots in a college dorm. Only, its creators didn’t know it at the time. It took years before the founders decided to build a business on their college study tricks. Blinkist condenses nonfiction books into pithy, but accessible 15-minute summaries which you can read or listen to via its app.
“It all started with four friends,” says Sebastian Schleicher, Director of Engineering at Blinkist. “After leaving university, they found jobs and built lifestyles that kept them fully occupied—but they were pretty frustrated because their packed schedules left them no time for reading and learning new things.”
Rather than resign themselves to a life without learning, they racked their brains as to how they could find a way to satisfy their craving for knowledge. They decided to revive their old study habits from university where they would write up key ideas from material that they’d read and then share it with each other. It didn’t take long for them to realise that they could build a business on this model of creating valuable easily accessible content to inspire people to keep learning. In 2012, Blinkist was born.
Six years later, the Berlin-based outfit has nearly 100 employees, but instead of writers and editors, they have Tea Masters and Content Ninjas. Blinkist has no formal hierarchical management structure, having replaced bosses with BOS, the Blinkist Operating System. The app has over five million users and, at its foundation, it has MongoDB Atlas, the fully managed service for MongoDB, running on AWS. But it didn’t always.
“In four years, we had a million users and 2,500 books,” says Schleicher. “We’d introduced audiobooks and seen them become the most important delivery channel. We tripled our revenue, doubled our team, moved into a larger, open-plan office, and even got a dog. Things were good.”
Running into trouble with 3rd party MongoDB as a Service
Then came an unwelcome plot twist. Blinkist had built its service on Compose, a third-party database as a service, based on MongoDB. MongoDB had been an obvious choice as the document model provided Blinkist with the flexibility needed to iterate quickly, but the team was too lean to spend time on infrastructure management
In 2016, Compose unexpectedly decided to change the architecture of its database, creating major obstacles for Blinkist as they would become locked in to an old version of MongoDB. “They left us alone,” says Schleicher. “They said, ‘Here’s a tool, migrate your data.’ I asked if they’d help. No dice. I offered them money. Not interested, no support. After being a customer for all those years? I said goodbye.”
After years of issues, it became clear last year that Blinkist would need to leave Compose, which meant choosing a new database provider. “We looked at migrating to MySQL, we were that desperate. That would have meant freezing development and concentrating on the move ourselves. On a live service. It was bleak.”
Discovering MongoDB Atlas
By this time, MongoDB’s managed cloud Atlas service was well established and seemed to be the logical solution. “We downloaded MongoDB’s free
mongomirror service to make the transition,” says Schleicher, “but we hit a brick wall. Compose had locked us into a very old version of the database and who knows what else, and we couldn’t work it out.”
At that point, Schleicher made a call to MongoDB. MongoDB didn’t say, ‘Do it yourself.’ Instead, they sent their own data ninja—or, in more conventional, business-card wording, a principal consulting engineer. “It was the easiest thing in the world,” Schleicher remembers. “In one day, he implemented four feature requests, got the migration done and our databases were in live sync. Such a great experience.”
Now that Blinkist is on Atlas, Schleicher feels like they have a very solid base for the future. “Performance is terrific. Our mobile app developers accidentally coded in a distributed denial of service attack on our own systems. Every day at midnight, in each time zone, our mobile apps all simultaneously sync. This pushes the requests load up from a normal peak of 7,500 requests a minute to 40,000 continuous. That would have slaughtered the old system, with real business impacts—killing sign-ups and user interactions. This time, nobody noticed anything was wrong."
Right now it feels like we have a big tech advantage. With MongoDB Atlas and AWS, we’re on the shoulders of people who can scale the world. I know for the foreseeable future I have partners I can really rely on.Sebastian Schleicher, Director of Engineering, Blinkist
Schleicher adds: “We’re building our future through microarchitecture with all the frills. Developers know they don’t have to worry about what’s going on behind the API in MongoDB. It just works. We’re free to look at data analytics and AI—whatever techniques and tools we believe will help us grow—and not spend all our time maintaining a monolithic slab of code.”
With Blinkist’s global ambitions, scaling isn’t just a technical challenge; it tests company culture—no matter how modern—to the limits. MongoDB’s own customer focused culture, it turns out, is proving as compatible as MongoDB’s data platform.
“Talking to MongoDB isn’t like being exposed to relentless sales pressure. It’s cooperative, it’s reassuring. There are lots of good technical people on tap. It’s holistic, no silos, whatever it takes to help us.”
This partnership is helping make Blinkist a great place to be a developer.
“A new colleague we hired last year told me we’ve created an island of happiness for engineers. Once you have an understanding of the business needs and vision, you get to drive your own projects. We believe in super transparency. Everyone is empowered.”
“Oh, and did I mention we have a dog?”
Leading digital cryptocurrency exchange cuts developer time by two-thirds and overcomes scaling challenges with MongoDB Atlas
Cryptocurrency investing is a wild ride. And while many have contemplated the lucrative enterprise of building an exchange, few have the technical know-how, robust engineering, and nerves of steel to succeed. Discidium Internet Labs decided it qualified and launched Koinex, a multi-cryptocurrency trading platform, in India in August 2017. By the end of the year, it was the largest digital asset exchange by volume in the country.
“India loves cryptocurrencies like Bitcoin and Ripple,” says Rakesh Yadav, Co-Founder & CTO Koinex. “We wanted to be the first local exchange to operate in accordance with global best practices. But we needed to provide a great user experience fast with a small development team. Transaction speed is one thing but developer bandwidth is the real limiting factor. “
Those who follow the cryptocurrency markets will know that challenges come fast and furious, with huge swings in prices and trading volumes driven by unpredictable developments, all in an environment of rapidly changing regulations. For an exchange with the scope and ambition of Koinex – it currently offers trading in more than 50 pairs of cryptocurrencies – that leads to a lot of exposure to market volatility.
For example, Rakesh says, three months after the launch, Koinex saw a huge spike in Ripple (XRP) transactions. “It was 50 times the volume we’d seen before,” he says. A group of Japanese credit card companies had just announced Ripple support, giving it a huge increase in trustworthiness. The trading volumes ramped up and stayed up. But the PostgreSQL deployment (a tabular database) underpinning the Koinex platform couldn’t keep pace with surging demand.
“Everything was stored in PostgreSQL, and it wasn’t keeping up. We had an overwhelming growth in data with read and write times slowing because of large indexes, and CPUs spiking. Moving our deployment to Aurora RDS gave us two times improvement. But it wasn’t enough as we could not scale beyond a single instance for writes. We were seeing just one thousand transactions per second, and we wanted to aim for 100,000.” If one spike on one cryptocurrency could cause such problems, what would a really busy market look like? Time to aim high.
“We decided to move the 80 percent of data that needed real-time responses to MongoDB’s fully managed database as a service Atlas, and run it all on AWS.”
The MongoDB Atlas experience
The move was started in January as part of the development of a new trading engine and, as Ankush Sharma Senior Platform Engineer at Koinex explains, MongoDB Atlas looked like a good fit for a number of reasons. “It had sharding out of the box, which we saw as essential as this gave us the ability to distribute write loads out across multiple servers, without any application changes at all. Atlas also meant fewer code changes, less frequent resizing or cluster changes, and as little operational input from us as possible.”
Other aspects of the database seemed promising. “Its flexible data model made it a great fit for blockchain RPC-based communications as it meant we could handle any cryptocurrency regardless of its data structure. MongoDB Atlas is fully managed so it’s zero DevOps resources to run, and it’s got an easy learning curve.”
That last aspect was as important as the technical suitability. “Allocating developer bandwidth is completely crucial,” says Ankush. “If we’d stuck with Postgres, creating the new Trading Engine would have been three to four months. That wouldn’t have been survivable. With MongoDB we did it in 30 to 40 days.” And although he initially wasn’t sure MongoDB Atlas would be a long-term solution, its performance convinced him otherwise.
“It scaled out as we needed, and it scaled back so gracefully. There are times when the market is slower, so it lets us track costs to market liquidity. It’s working really well for us.”
It’s continued to free up developer bandwidth, too. “We started off with just the one product on MongoDB but we have eight or nine on it now. We wouldn’t have been able to concentrate on the mobile app, or provide historical data on demand to traders if our DevOps team didn’t find the database so easy to work with and with so many features.”
And long lead times on new products aren’t an option in the cryptocurrency market. Over just 17 days in July, Koinex built out and launched a new service called Loop – a novel peer-to-peer digital token exchange system designed to deal with controversial regulatory moves by the Indian central bank. “Digital currencies are complex. Policies and technologies are changing all the time so our business often depends on being able build new features quickly, sometimes in just a few weeks. Not only does it have to be done fast but it has to be tested, robust and at scale. It’s a financial platform – you can’t compromise. Time we don’t spend managing the database is time we can spend on new features and products, and that’s a huge payback.”
MongoDB also has the right security features to fit in with a financial exchange, says Ankush: “We have solid protocols limiting who in the company can see what data, with strong access controls, encryption and proper separation of production and development environments. We look to global best practices, and these are all implemented by default in the MongoDB Atlas service.”
For a company barely a year old, Koinex has big plans for the future. “Koinex has been leading the digital asset revolution in India,” says Ankush. “We give users a world-class experience. The long-term plan is to have multiple digital asset management products available, not just cryptocurrencies. Whole new ecosystems are going to develop. With MongoDB Atlas, we’re going to be able to do all the things that other top exchanges do as well as add in our own extras and features.”
AO.com Turns to MongoDB to Build a Single Customer View
Transforms customer experience, fights fraud, and meets the demands of the GDPR with MongoDB Atlas and Apache Kafka running on AWS
Adopting a Serverless Approach at Bazaarvoice with MongoDB Atlas and AWS Lambda
I recently had the pleasure of welcoming Ani Hammond, Senior Staff Software Engineer from Bazaarvoice, to the MongoDB World stage. To a completely packed room, Ani chronicled her team’s journey as they replatformed Bazaarvoice’s Curations service from a runaway monolith architecture to a completely serverless architecture backed by MongoDB Atlas.
Even if you’ve never heard of Bazaarvoice, it’s almost impossible that you’ve never interacted with their services. To use Ani’s own description, “If you're shopping online and you’re reading a review, it's probably powered by us.”
Bazaarvoice strives to connect brands and retailers with consumers through the gathering, curation, and display of user-generated content—anything from pictures on Instagram to an online product review—during a potential customer’s buying journey.
To give you a sense of the scale of this task, Bazaarvoice clocked over a billion total page views between Thanksgiving Day and Cyber Monday in 2017, peaking at around 6,000 page views per second!
Even if you’ve never heard of Bazaarvoice, it’s almost impossible that you’ve never interacted with their services.
One of the technologies behind this herculean task is the Curations platform. To understand how this platform works, let’s look at an example:
An Instagram user posts a cute photo of their child wearing a particular brand’s rain boots. Using Curations, that brand is watching for specific content that mentions their products, so the social collection service picks up that post and shows it to the client team in the Curations application. The post can then be enriched in various manual and automatic ways. For example, a member of the client team can append metadata describing the product contained in the image or automatic rules can filter content for potentially offensive material. The Curations platform then automates the process of securing the original poster’s permission for the client to use their content. Now, this user-generated content is able to be displayed in real time on the brand’s homepage or product pages to potential customers considering similar products.
In a nutshell, this is what Curations does for hundreds of clients and hundreds of thousands of individual content pieces.
The technology behind Curations was previously a monolithic Python/Django-based stack on Amazon EC2 instances on top of a MySQL datastore deployed via RDS.
The technology behind Curations was previously a monolithic Python/Django-based stack on Amazon EC2 instances on top of a MySQL datastore deployed via RDS.
This platform was effective in allowing Bazaarvoice to scale to hundreds of new clients. However, this architecture did have an Achilles heel: each additional client onboarded to Bazaarvoice’s platform represented an additional Python/Django/MySQL cluster to manage. Not only was this configuration expensive (approximately $60,000/month), the operational overhead generated by each additional cluster made debugging, patching, releases, and general data management an ever-growing challenge. As Ani put it, “Most of our solutions were basically to throw more hardware/money at the problem and have a designated DevOps person to manage these clusters.”
One of the primary factors in selecting MongoDB for the new Curations platform was its support for a variety of different access patterns. For example, the part of the platform responsible for sourcing new social content had to support high write volume whereas the mechanism for displaying the content to consumers is read-intensive with strict availability requirements.
Diving into the specifics of why the Bazaarvoice team opted to move from a MySQL-based stack to one built on MongoDB is a blog post for another day. (Though, if you’d like to see what motivated other teams to do so, I recommend How DevOps, Microservices, and MongoDB are Making HSBC “Simpler, Better, and Faster” and Breuninger delivers omnichannel shopping experience for thousands of daily online users.)
That is to say, the focus of this particular post is the paradigm shift the Curations team made from a linearly-scaling monolith to a completely serverless approach, underpinned by MongoDB Atlas.
The new Curations platform is broken into three distinct services for content collection, enrichment, and display. The collections service is powered by a series of AWS Lambda functions triggered by an Amazon Kinesis stream written in Node.js whereas the enrichment and display services are built on autoscaling AWS Elastic Beanstalk instances. All three services making up the new Curations platform are backed by MongoDB Atlas.
Not only did this approach address the cluster-per-customer challenges of the old system, but the monthly costs were reduced by nearly 90% to approximately $6,500/month. The results are, again, best captured by Ani’s own words:
Massive cost savings, huge performance gains, strong consistency, and a handful of services rather than hundreds of clusters.
MongoDB Atlas was a natural fit in this new serverless paradigm as the team is fully able to focus on developing their product rather than on infrastructure management. In fact, the team had originally opted to manage the MongoDB instances on AWS themselves. After a couple of iterations of manual deployment and management, a desire to gain even more operational efficiency and increased insight into database performance prompted their move to Atlas. According to Ani, the cost of migrating to and leveraging a fully managed service was, "Way cheaper than having dedicated DevOps engineers.” Atlas’ support for direct VPC peering also made the transition to a hosted solution straightforward for the team.
Speaking of DevOps, one of the first operational benefits Ani and her team experienced was the ability to easily optimize their index usage in MongoDB. Previously, their approach to indexing was “build stuff that makes sense at the time and is easy to iterate on.” After getting up and running on Atlas, they were able to use the built-in Performance Advisor to make informed decisions on indexes to add and unused ones to remove. As Ani puts it:
An index killed is as valuable as an index added. This ensures all your indexes to fit into memory and a bad index doesn't push out the good ones.
Ani’s team also used the Atlas Performance Advisor to diagnose and correct inefficient queries. According to her, the built-in tools helped keep the team honest, "[People] say, ‘My database isn't scaling. It's not able to perform complex queries in real time...it doesn't work.’ Fix your code. The hardware is great, the tools are great but they can only carry you so far. I think sometimes we tend to get sloppy with how we write our code because of how cheap and how easy hardware is but we have to write code responsibly too.”
In another incident, a different Atlas feature, the Real Time Performance Panel, was key to identifying an issue with high load times in the display service. Some client’s displays were taking more than 6 seconds to load. (For context, content delivery network provider, Akamai, found that a two-second delay in web page load time can cause bounce rates to double!) High-level metrics in Datadog reported 5+ seconds query response times, while Atlas reported less than 100 ms response times for the same query. The team used both data points to triangulate and soon realized the discrepancy was a result of the time it took for Lambda to connect to MongoDB for each new operation. Switching from standard Lambda functions to a dockerized service ensured each operation could leverage an open connection rather than initiating a “cold start.”
I know a lot of the cool things that Atlas does can be done by hand but unless this is your full-time job, you're just not going to do it and you’re not going to do it as well.
Ani’s team also used the Atlas Performance Advisor to diagnose and correct inefficient queries.
Before wrapping up her presentation, Ani shared an improvement over the old system that the team wasn’t expecting. Using Atlas, they were able to provide the customer support and services teams read-only views into the database. This afforded them deeper insight into the data and allowed them to perform ad-hoc queries directly. The result was a more proactive approach to issue management, leading to an 80% reduction in inbound support tickets.
By re-architecting their Curations platform, Bazaarvoice is well-positioned to bring on hundreds of new clients without a proportional increase in operations work for the team. But once again, Ani summarized it best:
As the old commercial goes… ‘Old platform: $60,000. New platform: $6,000. Getting to focus all of my time on development: priceless.'
Thank you very much to Ani Hammond and the rest of the Curations team at Bazaarvoice for putting together the presentation that inspired this post. Be sure to check out Ani’s full presentation in addition to dozens of other high-quality talks from MongoDB World on our YouTube channel.
High-end retailer in Germany delivers omni-channel shopping experience on MongoDB Atlas for thousands of daily online users
The importance of delivering an optimized customer experience cannot be overstated, especially if your business is high-end retail. For Breuninger, the customer-first approach has been in their DNA for more than 130 years.
When the top German retailer set out to build a new e-commerce platform, they wanted the online experience to match that of walking in to one of Breuninger’s premium department stores. Accomplishing this goal required a feature-rich, high-performance, and reliable database capable of supporting complex data sets across multiple categories.
“Today, our development teams have a lot of independence. We only have a handful of rules about how they design and build applications within their respective business units,” says Benedikt Stemmildt, Lead Software Architect of E. Breuninger GmbH & Co. “It’s not quite a rule that you have to use MongoDB, but you do have to explain yourself if you don’t.”
However, it wasn’t always this way. Breuninger’s previous platform was built on one of the industry-standard product content management (PCM) platforms, which Stemmildt felt was “monolithic and difficult to code for.” Code freezes were common and the underlying architecture was a frequent cause of frustration for an organization striving to adopt more agile processes.
A new development and feature roll-out approach was needed to execute the company’s aggressive omni-channel integration plans, and time to market for new online features became a top priority. Breuninger decided to build a technology group in response, going from 10 to 30 in-house developers in just a year.
“We broke down our monolithic architecture and split our application into separate microservices that reflect how our customers shop in the physical stores,” Stemmildt says. “It’s the customer journey — they search, discover, evaluate, and buy not just individual products, but complete outfits.”
“To reflect this architectural change, we split our development teams by different steps of the customer journey and kept dependencies to an absolute minimum,” Stemmildt continues. “One key to making this work is a high-performance database capable of working easily with data in lots of different ways. The document model of MongoDB means we can deliver data with the quality and detail that reflects our products and shopping experience.”
The result? Much faster time to market. Breuninger was able to build their omni-channel platform in months rather than years by enabling teams to decide on important architectural components for their own sections, without having to ask the permission of other teams.
As a seven-year veteran of MongoDB, Stemmildt was confident in recommending the database to his organization. “There are a lot of good databases,” he says. “However, many of them require developers to have a deep knowledge about how they work before getting any benefit. MongoDB is not like that. It’s very quick to learn and start getting results. Our teams are able to deliver features straight away. Once users do expand their use of the database, it’s so feature-rich that you never get a sense of having to push it beyond what it was designed for.”
And agile wouldn’t be agile without automation. “Everything we deploy is automated, and with MongoDB Atlas on AWS, the deployment and management of our databases fit neatly into our processes. After a period of operating MongoDB ourselves on EC2, it’s great not having to worry about the details and not having to spend time setting up, configuring, and managing database[s]. You free up a lot of opportunities to add value to your service by not running things yourself.”
AWS offers a healthy mix of other tools for the teams at Breuninger to leverage, such as a managed Kubernetes service and serverless Lambda functions. MongoDB Atlas and AWS also help Breuninger stay on the right side of the regulators. “We need to comply with GDPR so we keep everything running within our borders. MongoDB Atlas’s built-in security features have helped us satisfy these requirements.”
The finished platform might look different to someone who is used to traditional architectures, but to Stemmildt, not being restrained by legacy approaches makes a lot of sense. “Each of our teams owns one or more sections of the customer journey. The search team updates its own database, pulling data in from the product data producer via a feed and re-populating its own database as needed. We don’t have to ripple refreshes out across the system as they happen. That means each team is free to add new features without changing some core database component and affecting other teams. Self-contained systems are an important design rule.”
And although there are some 25 different and largely independent systems, the customers see just one website. A front-end proxy uses server-side includes to marshal data as required from a mix of micro-frontends before delivering the final composite to the shopper. Product data, product availability, outfit data, price information, navigation metadata — these are all woven together from separate MongoDB databases as the customer goes through the shopping experience online.
Comparing a microservices architecture to a monolithic one revealed to Breuninger that some metrics don’t matter as much as they once did, while others matter more. “With multiple teams developing things so rapidly, I don’t know exactly how much total data is in play. But we are a very metrics-driven company, not just in the technical infrastructure but across the business. We know when a component is and is not working well from both a technical and business perspective, if it needs optimizing for performance, or whether it is delivering value to the business or we need to revisit that aspect of the system architecture.”
While Stemmildt couldn’t comment too much on future plans, he’s enthusiastic about MongoDB’s part in whatever they may be. “We wanted high performance, but most importantly we wanted to be able to add more features. We’re not using MongoDB’s graph database feature yet, but we may be by the end of the year. There are a lot of things we could do with text search, too.”
Other new features — such as multi-document transaction support in MongoDB 4.0 — may also be useful, but in unorthodox ways. “I don’t actually think transactions are needed anymore for our platform,” he laughs, “But there are some teams, like the customer data team, who don’t agree with me yet and won’t use MongoDB because of that. So the release of MongoDB 4.0 will help me to help them make the transition.”
While customers won’t see the nuts and bolts of Breuninger’s transformation to a data-driven enterprise, they will benefit from the company’s newly integrated omni-channel platform, which delivers an improved customer experience and more ways to get inspired.
And to anyone thinking about using MongoDB on their next project, Stemmildt has just one piece of advice: “Use it. Get a MongoDB Atlas account, create a cluster, and play with it. The way we see it, after the majority of our teams have naturally adopted MongoDB, if you can’t say why you should use another database, then you should just use MongoDB.”
2018 MongoDB Innovation Award Winners
We received an overwhelming number of nominations for the fifth annual MongoDB Innovation Awards, recognizing companies who are using MongoDB to dream big and deliver incredibly bold, innovative solutions that are moving forward industries and changing lives for the better.
We are thrilled to announce our 12 winners who will be honored at MongoDB World, New York, June 26 and 27.
See the full list and read a bit more about how they are disrupting the status quo here:
Global Go to Market Partner of the Year: Accenture
Accenture is a leading global professional services company, providing a broad range of services and solutions strategy, consulting, digital, technology and operations. Combining unmatched experience and specialized skills across more than 40 industries and all business functions, Accenture works at the intersection of business and technology to help clients improve their performance and create sustainable value for their stakeholders. The company partners with more than three-quarters of the Fortune Global 500, driving innovation to improve the way the world works and lives. Accenture and MongoDB have worked together to help organizations leverage the power of data to gain a competitive edge.
The Enterprise: Charles Schwab
Charles Schwab is one of the largest financial services firms in the United States. To improve customer experience, speed up development cycles, and prepare for cloud portability, Charles Schwab is modernizing a significant portion of its applications by migrating to MongoDB-powered microservices. Multiple applications are built on MongoDB, including an authentication app leveraged by retail customers as well as a portfolio management solution utilized by registered investment advisors.
Launch Fast: Coinbase
Coinbase is dedicated to creating an open financial system for the world and defining what the future of finance will look like. To do this, they built the most trusted and regulatory compliant global cryptocurrency trading platform to broker exchanges of Bitcoin, Bitcoin Cash, Ethereum and Litecoin as well as pioneering cryptocurrency indexes and institutional cryptocurrency trading. In 2017, they experienced exponential growth with over 20M+ users and $150B+ being traded on their platform in over 190 countries. The Coinbase engineering team scaled and optimized MongoDB to respond to this unprecedented volume of traffic and to prepare for future waves of cryptocurrency enthusiasm.
Scale: Epic Games
Epic Games develops cutting-edge games and cross-platform gaming engine technology. Their massively popular, multi-platform game, Fortnite, has been played by more than 125 million gamers around the globe. The Epic team has implemented a number of best practices and performance improvements to get the best scaling and availability characteristics out of MongoDB.
Data-Driven Business: Freddie Mac
Freddie Mac set out to modernize a number of applications that were previously built on legacy relational databases. One mission-critical application, a property appraisal tool, held massive amounts of property and loan information, but was increasingly expensive and time consuming to update. Turning to MongoDB, Freddie Mac was able to collect information from a variety of different sources in a variety of formats to build a single view of all the information needed to accurately appraise a property. In the months since using MongoDB, Freddie Mac has seen an increase in developer productivity.
Customer Experience: Fresenius Medical Care North America
Fresenius Medical Care North America is the premier health care company focused on providing the highest quality care to people with renal and other chronic conditions. Through its industry-leading network of dialysis facilities, outpatient cardiac and vascular labs, and urgent care centers Fresenius Medical Care North America (FMCNA) provides coordinated health care services at pivotal care points for hundreds of thousands of chronically ill customers throughout the continent.
Since 2015, FMCNA has used MongoDB Enterprise Advanced for a variety of projects to help support their mission to deliver superior care that improves the quality of life of every patient. These projects have included analytics platforms, a data lake and the FHIR platform (a healthcare standard for exchanging medical records securely and at scale). However, the most impactful application has been a single view of the patient platform built on MongoDB. This platform brings together a variety of data sources to ensure the patient, doctors and other caregivers all have a complete understanding of the treatments required and can make adjustments with confidence.
Healthcare: Genomics England
Genomics England, a company owned by the UK government's Department of Health and Social Care, is working with the NHS to sequence 100,000 genomes from patients with rare diseases and their families, as well as patients with common cancer. In the future, there may be a diagnosis where there wasn't one before and, in time, there is the potential of new and more effective personalized treatments for patients.
On average, 1,000 genomes are sequenced per week, which amounts to around 10 terabytes of data per day. To manage this immense and sensitive data set as well as power the data science that makes it all possible, Genomics England used MongoDB Enterprise Advanced. The partnership with MongoDB allows the processing time for complex queries to be reduced from hours to milliseconds, which means scientists can discover new insights more quickly.
Internet of Things: Humana Inc.
With a variety of applications built on MongoDB, Humana is changing healthcare for the better. One of their IoT applications called Go365 is a corporate wellness and rewards program which helps employees live healthier lives, which in turn increases productivity and reduces overall health claims costs for employers. Go365 features a personalized program that inspires, supports, and rewards members for taking steps to improve and continue healthy behavior. Users are able to compete in challenges, connect their fitness devices and mobile apps to log healthy activities and earn points, reward themselves through the Go365 Mall, and track their progress. In fact, by year 3, people who engaged with the program saw that the cost of their health claims were reduced by over 10%, relative to those of unengaged members.
Delivery Partner of the Year: Infosys
A perennial winner, this is the third year in a row Infosys has won a MongoDB Innovation Award. As a global leader in consulting, technology and next-generation services, this year Infosys has been working closely with MongoDB to accelerate application modernization for client organizations. A key part of this is the joint delivery of single view and mainframe modernization offerings to migrate and digitize business-critical applications away from rigid tabular databases and on to next-generation technology. In this long standing partnership, Infosys and MongoDB are already helping many large enterprises with renewing and modernizing their IT landscape.
The William Zola Award for Community Excellence: Ken W. Alger
Ken Alger is one of the most prolific bloggers on MongoDB's technology with dozens of posts in the past two years. He is a self-taught programmer and a teacher at Treehouse. An avid follower of open-source, he has previously sat on the board of directors of the Django Software Foundation. He is delighted to share his extensive MongoDB knowledge via his blog, Twitter, and his GitHub account. He exemplifies the true community spirit of MongoDB and The William Zola Award for Community Excellence.
Savvy Startup: Radar
Radar, a seed-stage startup and member of the MongoDB Startup Accelerator program, has built iOS and Android SDKs on MongoDB Atlas and AWS. As the location platform for modern apps, Radar allows developers to easily add location context and tracking to their applications. Radar currently runs on more than 25 million devices around the globe, processing billions of locations each week.
7-Eleven is continuing to redefine what convenience is. By leveraging MongoDB Atlas on AWS and a microservices architecture, 7-Eleven has built an e-commerce application called 7-Now which allows consumers to browse a product catalog connected to their local store’s inventory, make purchases on their mobile phones, and schedule in-store pick up or delivery through services like Postmates. This application not only streamlines the consumer’s experience, but also gives the 7-Eleven team extensive analytics capabilities allowing them to improve the overall customer experience. This is sure to have a major impact in their 10,000 stores in the US and Canada, and with 60% of the US population living within one mile of a 7-Eleven.
Stratifyd & MongoDB: AI-Driven Business Insights to Keep Customers Happy
2017 was a banner year for MongoDB's partner ecosystem. We remain strategic about engaging with our channels, and the results are validating our approach. Our strong network of ISVs, global system integrators, cloud, resellers, and technology partners is a competitive differentiator that helps us scale.
We are especially excited about the innovation and growth in store for our ISV business in 2018. It's already off to a great start. Our newest ISV partner Stratifyd is a fantastic example of how platforms built around MongoDB address serious market needs with the most cutting-edge, innovative technology.
Stratifyd is an end-to-end customer analytics platform powered by AI. The platform provides competitive advantages to some of the most recognized brands in the world. LivePerson, Etsy, MASCO, Kimberly-Clark, and many more rely on Stratifyd for a 360-degree view of their end customers.
Stratifyd analyzes customer interactions such as online reviews, social media posts, phone calls, emails, chats, surveys, CRM data, and more to turn them into actionable business insights which increase customer acquisition and retention, which is critical to the continued success of Stratifyd’s clients. In addition to these benefits, Stratifyd is just a flat-out cool implementation of AI.
I caught up with Stratifyd's CTO, Kevin O'Dell, to discuss the data technology behind the platform, and how MongoDB drives value for their customers.
For anyone that isn’t familiar with Stratifyd yet, how do you describe the platform?
Stratifyd uses human generated data to analyze, categorize, and understand intent with the purpose of changing human behavior. This changes the way brands interact with their customers, but also the way customers interact with brands, increasing customer acquisition and raising retention rates.
What was the genesis of the company? Why did you set out to build this?
Stratifyd was a result of postdoctoral research done at the University of North Carolina, Charlotte. Our founders were researching how AI could analyze unstructured data. During their research, they discovered strong business and government use cases. The founding team was working with numerous three letter agencies on predicting terrorist and disease movements globally. They were able to raise millions of dollars in funding from these agencies. The demand for a product organically grew from there, which led to the development of the Stratifyd platform.
I love the insights Stratifyd can provide – how would you describe the unique advantages that Stratifyd gives its customers?
Stratifyd provides near real-time business intelligence for contact center, marketing, product, and customer experience teams, all based on customer interactions. These insights enable businesses to be proactive rather than reactive in regard to business strategy. Stratifyd customers are able to respond to customer requests, complaints, or general feedback in near real time, changing the way companies interact with their end users. For example, we have empowered a customer with the knowledge to launch a new product line. Another customer gained insights that fundamentally changed how they are rolling out a 700+ million-dollar brand in a new continent.
What kind of feedback are you getting from customers that have deployed Stratifyd in their businesses?
Our customers love using our platform. They are surprised at how simple it is to use, and how powerful it is. They really appreciate how Stratifyd is making AI and machine learning meaningful for them in their day-to day-lives. Stratifyd helps ensure measurable results from day one. Speaking of implementation, they REALLY love that we don’t just use the term day one figuratively – customers are up and running in less than a day.
Talk to me about how you landed on MongoDB. What were you looking for in a database, and what problems were you having before moving to MongoDB?
That's an easy one to answer: speed and flexibility. Stratifyd ingests data from hundreds of sources. We needed a database that could keep up with high read and write request rates while handling a flexible schema. The hardest problem we were trying to solve was lack of secondary indexes; with those, MongoDB accelerated our query response times by at least 100x.
Can you share any best practices for scaling your MongoDB infrastructure? Any impressive metrics around the number of interactions, the volume of reads / writes per second, response times?
As a SaaS-first platform, being always-on is a HUGE best practice for us. MongoDB’s innate replication and failover abilities ensured less than 17 minutes of total downtime last year! Using MongoDB as our backend system, our AI can process a quarter of a million words in less than a minute.
How do you measure the impact of MongoDB on your business?
Our business wouldn’t be able to succeed without MongoDB. The uptime, failover, query response times, secondary indexes, and dynamic schemas have empowered most of Stratifyd’s key differentiators.
What advice would you give someone who is considering using MongoDB for their next project?
With all projects, I recommend truly understanding the requirements for the end results. There is a ton of excellent technology out there, but picking the wrong one can be detrimental to project success. Always run numerous tests, comparing different stacks to make sure you find the right one and fail fast on the wrong technology stack.
Stratifyd has some impressive customers, from the Fortune 500 to some really innovative startups: what’s next for the company?
We have some pretty big things planned for 2018. We're now providing more than just actionable intelligence; we are now streamlining and automating workflows. We are closing the customer feedback loop, which enables us to plug Stratifyd into any business process quickly to deliver measurable results.