Fraud Detection at FICO with MongoDB and Microservices
FICO is more than just the FICO credit score. Founded in 1956, FICO also offers analytics applications for customer acquisition, service, and security, plus tools for decision management.
One of those applications is the Falcon Assurance Navigator (FAN), a fraud detection system that monitors purchasing and expenses through the full procurement to pay cycle. Consider an expense report: the entities involved include the reporter, the approver, the vendor, the department or business unit, the expense line items, and more. A single report has multiple line items, where each line may be broken into different expense codes, different budget sources, and so on. This translates into a complicated data model that can be nested 6 or 7 layers deep – a great match for MongoDB’s document model, but quite hard to represent in the tabular model of relational databases.
FAN Architecture Overview
The fraud detection engine consists of a series of microservices that operate on transactions in queues that are persisted in MongoDB:
- Each transaction arrives in a receiver service, which places it into a queue.
- An attachment processor service checks for an attachment; if one exists, it sends it to an OCR service and stores the transaction enriched with the OCR data.
- A context creator service analyzes it and associates it with any past transactions that are related to it.
- A decision execution engine runs the rules that have been set up by the client and identifies violations.
- One or more analytics engines review transactions and flag outliers.
- Now decorated with a score, the transaction goes to a case manager service, which decides whether to create a case for human follow-up based on any identified issues.
- At the same time, a notification manager passes updates on the processing of each transaction back to the client’s expense/procurement system.
To learn more, watch FICO’s presentation at MongoDB World 2018.
How Planable Uses Mongodb Atlas to Help Social Media Teams Move Their Creative Processes Forward
In 2015, Nicolae Gudumac was working at a social media agency managing hundreds of campaigns for clients. However, with each campaign requiring multiple rounds of review, he needed a way to streamline the disjointed feedback loop and bring everyone onto the same page. Along with his co-founders, Xenia Muntean and Vlad Calus, he began building a platform that would streamline this time-consuming process for social media managers, agencies, and their clients.
The team created Planable, a platform that simplifies planning, visualizing, and approving social media posts. The tool feels like a live mock-up of the social feed, making it easy for teams to collaborate and give real-time feedback in a familiar format.
Prioritizing Developer Velocity From the Beginning
As the newly minted CTO, Nicolae decided to use MongoDB’s document data model to help his team innovate quickly to stay ahead of the ever-changing social media landscape. “With social media content, requirements are constantly changing as platforms evolve and introduce new formats. Having a flexible data model has been a boon to our productivity.”
He also needed to build a foundation that could scale with them as the business grew. Since Planable is a collaboration tool that needs to keep users synced in real-time, Nicolae built the tool on top of Node.js and websockets. The team harnessed MongoDB’s oplog tailing functionality to send real-time updates to all connected users when relevant data changed and recently started to leverage MongoDB change streams to make this process more simple and scalable. To easily scale their app servers at peak times, Nicolae chose to run Planable on AWS container service.
Migrating to MongoDB Atlas
For managing their MongoDB deployment, the team started off using Compose.com but were having issues with restoring backups and only had access to a very limited set of configuration options. Compose also charged a premium for upgrading their storage engine to WiredTiger. MongoDB Atlas, with its queryable backup snapshots, automated upgrades, and configuration flexibility — especially the ease at which clusters can be scaled horizontally — looked very appealing.
We needed to be able to provide our clients with a platform that is as reliable as we are. Atlas’ native cloud first and scale-out architecture aligned well with our increasing performance demands and usage growth.Nicolae Gudumac, CTO, Planable
When Planable was accepted into the MongoDB Startup Accelerator, the timing was right to make the move to MongoDB Atlas. The team had customers all over the world and couldn’t afford any downtime with the migration. They used the Atlas Live Migration Service to move over their data from Compose with no downtime.
Currently, the team of 6 is split between engineering and business. With a small engineering team, they’re hyper-focused on product improvements and new features that will drive the business forward. Atlas features such as the Real-Time Performance panel and the Performance Advisor, which monitors clusters for slow queries and automatically suggests indexes to improve performance, allow the team to dedicate more of their attention to application improvements. According to Nicolae, “The Performance Advisor has made indexing the database and optimizing queries a no-brainer."
Queryable backups have also helped the team quickly address customer questions and as Nicolae recalls, “It’s saved us numerous times when someone accidentally dropped/updated a few documents and they needed to be restored. We’ve managed to quickly inspect and restore data in just a few clicks.”
The team is looking forward to the upcoming release of MongoDB Charts which is currently available in beta. “Charts will enable our marketing and business team to gain insights from our database, without resorting to sophisticated and expensive BI tools” says Nicolae.
Planable is bringing thousands onto their platform each month and are well on their way to becoming the default tool for social media collaboration.
MongoDB Atlas has helped this fast-growing startup focus on helping social media teams move their process forward and allows Nicolae to give valuable time back to his team. “There’s no doubt in my mind that moving to MongoDB Atlas has increased our team’s productivity.”
Millions of Users and a Developer-Led Culture: How Blinkist Powers its Berlin Startup on MongoDB Atlas
Not unlike other startups, Blinkist grew its roots in a college dorm. Only, its creators didn’t know it at the time. It took years before the founders decided to build a business on their college study tricks. Blinkist condenses nonfiction books into pithy, but accessible 15-minute summaries which you can read or listen to via its app.
“It all started with four friends,” says Sebastian Schleicher, Director of Engineering at Blinkist. “After leaving university, they found jobs and built lifestyles that kept them fully occupied—but they were pretty frustrated because their packed schedules left them no time for reading and learning new things.”
Rather than resign themselves to a life without learning, they racked their brains as to how they could find a way to satisfy their craving for knowledge. They decided to revive their old study habits from university where they would write up key ideas from material that they’d read and then share it with each other. It didn’t take long for them to realise that they could build a business on this model of creating valuable easily accessible content to inspire people to keep learning. In 2012, Blinkist was born.
Six years later, the Berlin-based outfit has nearly 100 employees, but instead of writers and editors, they have Tea Masters and Content Ninjas. Blinkist has no formal hierarchical management structure, having replaced bosses with BOS, the Blinkist Operating System. The app has over five million users and, at its foundation, it has MongoDB Atlas, the fully managed service for MongoDB, running on AWS. But it didn’t always.
“In four years, we had a million users and 2,500 books,” says Schleicher. “We’d introduced audiobooks and seen them become the most important delivery channel. We tripled our revenue, doubled our team, moved into a larger, open-plan office, and even got a dog. Things were good.”
Running into trouble with 3rd party MongoDB as a Service
Then came an unwelcome plot twist. Blinkist had built its service on Compose, a third-party database as a service, based on MongoDB. MongoDB had been an obvious choice as the document model provided Blinkist with the flexibility needed to iterate quickly, but the team was too lean to spend time on infrastructure management
In 2016, Compose unexpectedly decided to change the architecture of its database, creating major obstacles for Blinkist as they would become locked in to an old version of MongoDB. “They left us alone,” says Schleicher. “They said, ‘Here’s a tool, migrate your data.’ I asked if they’d help. No dice. I offered them money. Not interested, no support. After being a customer for all those years? I said goodbye.”
After years of issues, it became clear last year that Blinkist would need to leave Compose, which meant choosing a new database provider. “We looked at migrating to MySQL, we were that desperate. That would have meant freezing development and concentrating on the move ourselves. On a live service. It was bleak.”
Discovering MongoDB Atlas
By this time, MongoDB’s managed cloud Atlas service was well established and seemed to be the logical solution. “We downloaded MongoDB’s free
mongomirror service to make the transition,” says Schleicher, “but we hit a brick wall. Compose had locked us into a very old version of the database and who knows what else, and we couldn’t work it out.”
At that point, Schleicher made a call to MongoDB. MongoDB didn’t say, ‘Do it yourself.’ Instead, they sent their own data ninja—or, in more conventional, business-card wording, a principal consulting engineer. “It was the easiest thing in the world,” Schleicher remembers. “In one day, he implemented four feature requests, got the migration done and our databases were in live sync. Such a great experience.”
Now that Blinkist is on Atlas, Schleicher feels like they have a very solid base for the future. “Performance is terrific. Our mobile app developers accidentally coded in a distributed denial of service attack on our own systems. Every day at midnight, in each time zone, our mobile apps all simultaneously sync. This pushes the requests load up from a normal peak of 7,500 requests a minute to 40,000 continuous. That would have slaughtered the old system, with real business impacts—killing sign-ups and user interactions. This time, nobody noticed anything was wrong."
Right now it feels like we have a big tech advantage. With MongoDB Atlas and AWS, we’re on the shoulders of people who can scale the world. I know for the foreseeable future I have partners I can really rely on.Sebastian Schleicher, Director of Engineering, Blinkist
Schleicher adds: “We’re building our future through microarchitecture with all the frills. Developers know they don’t have to worry about what’s going on behind the API in MongoDB. It just works. We’re free to look at data analytics and AI—whatever techniques and tools we believe will help us grow—and not spend all our time maintaining a monolithic slab of code.”
With Blinkist’s global ambitions, scaling isn’t just a technical challenge; it tests company culture—no matter how modern—to the limits. MongoDB’s own customer focused culture, it turns out, is proving as compatible as MongoDB’s data platform.
“Talking to MongoDB isn’t like being exposed to relentless sales pressure. It’s cooperative, it’s reassuring. There are lots of good technical people on tap. It’s holistic, no silos, whatever it takes to help us.”
This partnership is helping make Blinkist a great place to be a developer.
“A new colleague we hired last year told me we’ve created an island of happiness for engineers. Once you have an understanding of the business needs and vision, you get to drive your own projects. We believe in super transparency. Everyone is empowered.”
“Oh, and did I mention we have a dog?”
Leading digital cryptocurrency exchange cuts developer time by two-thirds and overcomes scaling challenges with MongoDB Atlas
Cryptocurrency investing is a wild ride. And while many have contemplated the lucrative enterprise of building an exchange, few have the technical know-how, robust engineering, and nerves of steel to succeed. Discidium Internet Labs decided it qualified and launched Koinex, a multi-cryptocurrency trading platform, in India in August 2017. By the end of the year, it was the largest digital asset exchange by volume in the country.
“India loves cryptocurrencies like Bitcoin and Ripple,” says Rakesh Yadav, Co-Founder & CTO Koinex. “We wanted to be the first local exchange to operate in accordance with global best practices. But we needed to provide a great user experience fast with a small development team. Transaction speed is one thing but developer bandwidth is the real limiting factor. “
Those who follow the cryptocurrency markets will know that challenges come fast and furious, with huge swings in prices and trading volumes driven by unpredictable developments, all in an environment of rapidly changing regulations. For an exchange with the scope and ambition of Koinex – it currently offers trading in more than 50 pairs of cryptocurrencies – that leads to a lot of exposure to market volatility.
For example, Rakesh says, three months after the launch, Koinex saw a huge spike in Ripple (XRP) transactions. “It was 50 times the volume we’d seen before,” he says. A group of Japanese credit card companies had just announced Ripple support, giving it a huge increase in trustworthiness. The trading volumes ramped up and stayed up. But the PostgreSQL deployment (a tabular database) underpinning the Koinex platform couldn’t keep pace with surging demand.
“Everything was stored in PostgreSQL, and it wasn’t keeping up. We had an overwhelming growth in data with read and write times slowing because of large indexes, and CPUs spiking. Moving our deployment to Aurora RDS gave us two times improvement. But it wasn’t enough as we could not scale beyond a single instance for writes. We were seeing just one thousand transactions per second, and we wanted to aim for 100,000.” If one spike on one cryptocurrency could cause such problems, what would a really busy market look like? Time to aim high.
“We decided to move the 80 percent of data that needed real-time responses to MongoDB’s fully managed database as a service Atlas, and run it all on AWS.”
The MongoDB Atlas experience
The move was started in January as part of the development of a new trading engine and, as Ankush Sharma Senior Platform Engineer at Koinex explains, MongoDB Atlas looked like a good fit for a number of reasons. “It had sharding out of the box, which we saw as essential as this gave us the ability to distribute write loads out across multiple servers, without any application changes at all. Atlas also meant fewer code changes, less frequent resizing or cluster changes, and as little operational input from us as possible.”
Other aspects of the database seemed promising. “Its flexible data model made it a great fit for blockchain RPC-based communications as it meant we could handle any cryptocurrency regardless of its data structure. MongoDB Atlas is fully managed so it’s zero DevOps resources to run, and it’s got an easy learning curve.”
That last aspect was as important as the technical suitability. “Allocating developer bandwidth is completely crucial,” says Ankush. “If we’d stuck with Postgres, creating the new Trading Engine would have been three to four months. That wouldn’t have been survivable. With MongoDB we did it in 30 to 40 days.” And although he initially wasn’t sure MongoDB Atlas would be a long-term solution, its performance convinced him otherwise.
“It scaled out as we needed, and it scaled back so gracefully. There are times when the market is slower, so it lets us track costs to market liquidity. It’s working really well for us.”
It’s continued to free up developer bandwidth, too. “We started off with just the one product on MongoDB but we have eight or nine on it now. We wouldn’t have been able to concentrate on the mobile app, or provide historical data on demand to traders if our DevOps team didn’t find the database so easy to work with and with so many features.”
And long lead times on new products aren’t an option in the cryptocurrency market. Over just 17 days in July, Koinex built out and launched a new service called Loop – a novel peer-to-peer digital token exchange system designed to deal with controversial regulatory moves by the Indian central bank. “Digital currencies are complex. Policies and technologies are changing all the time so our business often depends on being able build new features quickly, sometimes in just a few weeks. Not only does it have to be done fast but it has to be tested, robust and at scale. It’s a financial platform – you can’t compromise. Time we don’t spend managing the database is time we can spend on new features and products, and that’s a huge payback.”
MongoDB also has the right security features to fit in with a financial exchange, says Ankush: “We have solid protocols limiting who in the company can see what data, with strong access controls, encryption and proper separation of production and development environments. We look to global best practices, and these are all implemented by default in the MongoDB Atlas service.”
For a company barely a year old, Koinex has big plans for the future. “Koinex has been leading the digital asset revolution in India,” says Ankush. “We give users a world-class experience. The long-term plan is to have multiple digital asset management products available, not just cryptocurrencies. Whole new ecosystems are going to develop. With MongoDB Atlas, we’re going to be able to do all the things that other top exchanges do as well as add in our own extras and features.”
AO.com Turns to MongoDB to Build a Single Customer View
Transforms customer experience, fights fraud, and meets the demands of the GDPR with MongoDB Atlas and Apache Kafka running on AWS
Adopting a Serverless Approach at Bazaarvoice with MongoDB Atlas and AWS Lambda
I recently had the pleasure of welcoming Ani Hammond, Senior Staff Software Engineer from Bazaarvoice, to the MongoDB World stage. To a completely packed room, Ani chronicled her team’s journey as they replatformed Bazaarvoice’s Curations service from a runaway monolith architecture to a completely serverless architecture backed by MongoDB Atlas.
Even if you’ve never heard of Bazaarvoice, it’s almost impossible that you’ve never interacted with their services. To use Ani’s own description, “If you're shopping online and you’re reading a review, it's probably powered by us.”
Bazaarvoice strives to connect brands and retailers with consumers through the gathering, curation, and display of user-generated content—anything from pictures on Instagram to an online product review—during a potential customer’s buying journey.
To give you a sense of the scale of this task, Bazaarvoice clocked over a billion total page views between Thanksgiving Day and Cyber Monday in 2017, peaking at around 6,000 page views per second!
Even if you’ve never heard of Bazaarvoice, it’s almost impossible that you’ve never interacted with their services.
One of the technologies behind this herculean task is the Curations platform. To understand how this platform works, let’s look at an example:
An Instagram user posts a cute photo of their child wearing a particular brand’s rain boots. Using Curations, that brand is watching for specific content that mentions their products, so the social collection service picks up that post and shows it to the client team in the Curations application. The post can then be enriched in various manual and automatic ways. For example, a member of the client team can append metadata describing the product contained in the image or automatic rules can filter content for potentially offensive material. The Curations platform then automates the process of securing the original poster’s permission for the client to use their content. Now, this user-generated content is able to be displayed in real time on the brand’s homepage or product pages to potential customers considering similar products.
In a nutshell, this is what Curations does for hundreds of clients and hundreds of thousands of individual content pieces.
The technology behind Curations was previously a monolithic Python/Django-based stack on Amazon EC2 instances on top of a MySQL datastore deployed via RDS.
The technology behind Curations was previously a monolithic Python/Django-based stack on Amazon EC2 instances on top of a MySQL datastore deployed via RDS.
This platform was effective in allowing Bazaarvoice to scale to hundreds of new clients. However, this architecture did have an Achilles heel: each additional client onboarded to Bazaarvoice’s platform represented an additional Python/Django/MySQL cluster to manage. Not only was this configuration expensive (approximately $60,000/month), the operational overhead generated by each additional cluster made debugging, patching, releases, and general data management an ever-growing challenge. As Ani put it, “Most of our solutions were basically to throw more hardware/money at the problem and have a designated DevOps person to manage these clusters.”
One of the primary factors in selecting MongoDB for the new Curations platform was its support for a variety of different access patterns. For example, the part of the platform responsible for sourcing new social content had to support high write volume whereas the mechanism for displaying the content to consumers is read-intensive with strict availability requirements.
Diving into the specifics of why the Bazaarvoice team opted to move from a MySQL-based stack to one built on MongoDB is a blog post for another day. (Though, if you’d like to see what motivated other teams to do so, I recommend How DevOps, Microservices, and MongoDB are Making HSBC “Simpler, Better, and Faster” and Breuninger delivers omnichannel shopping experience for thousands of daily online users.)
That is to say, the focus of this particular post is the paradigm shift the Curations team made from a linearly-scaling monolith to a completely serverless approach, underpinned by MongoDB Atlas.
The new Curations platform is broken into three distinct services for content collection, enrichment, and display. The collections service is powered by a series of AWS Lambda functions triggered by an Amazon Kinesis stream written in Node.js whereas the enrichment and display services are built on autoscaling AWS Elastic Beanstalk instances. All three services making up the new Curations platform are backed by MongoDB Atlas.
Not only did this approach address the cluster-per-customer challenges of the old system, but the monthly costs were reduced by nearly 90% to approximately $6,500/month. The results are, again, best captured by Ani’s own words:
Massive cost savings, huge performance gains, strong consistency, and a handful of services rather than hundreds of clusters.
MongoDB Atlas was a natural fit in this new serverless paradigm as the team is fully able to focus on developing their product rather than on infrastructure management. In fact, the team had originally opted to manage the MongoDB instances on AWS themselves. After a couple of iterations of manual deployment and management, a desire to gain even more operational efficiency and increased insight into database performance prompted their move to Atlas. According to Ani, the cost of migrating to and leveraging a fully managed service was, "Way cheaper than having dedicated DevOps engineers.” Atlas’ support for direct VPC peering also made the transition to a hosted solution straightforward for the team.
Speaking of DevOps, one of the first operational benefits Ani and her team experienced was the ability to easily optimize their index usage in MongoDB. Previously, their approach to indexing was “build stuff that makes sense at the time and is easy to iterate on.” After getting up and running on Atlas, they were able to use the built-in Performance Advisor to make informed decisions on indexes to add and unused ones to remove. As Ani puts it:
An index killed is as valuable as an index added. This ensures all your indexes to fit into memory and a bad index doesn't push out the good ones.
Ani’s team also used the Atlas Performance Advisor to diagnose and correct inefficient queries. According to her, the built-in tools helped keep the team honest, "[People] say, ‘My database isn't scaling. It's not able to perform complex queries in real time...it doesn't work.’ Fix your code. The hardware is great, the tools are great but they can only carry you so far. I think sometimes we tend to get sloppy with how we write our code because of how cheap and how easy hardware is but we have to write code responsibly too.”
In another incident, a different Atlas feature, the Real Time Performance Panel, was key to identifying an issue with high load times in the display service. Some client’s displays were taking more than 6 seconds to load. (For context, content delivery network provider, Akamai, found that a two-second delay in web page load time can cause bounce rates to double!) High-level metrics in Datadog reported 5+ seconds query response times, while Atlas reported less than 100 ms response times for the same query. The team used both data points to triangulate and soon realized the discrepancy was a result of the time it took for Lambda to connect to MongoDB for each new operation. Switching from standard Lambda functions to a dockerized service ensured each operation could leverage an open connection rather than initiating a “cold start.”
I know a lot of the cool things that Atlas does can be done by hand but unless this is your full-time job, you're just not going to do it and you’re not going to do it as well.
Ani’s team also used the Atlas Performance Advisor to diagnose and correct inefficient queries.
Before wrapping up her presentation, Ani shared an improvement over the old system that the team wasn’t expecting. Using Atlas, they were able to provide the customer support and services teams read-only views into the database. This afforded them deeper insight into the data and allowed them to perform ad-hoc queries directly. The result was a more proactive approach to issue management, leading to an 80% reduction in inbound support tickets.
By re-architecting their Curations platform, Bazaarvoice is well-positioned to bring on hundreds of new clients without a proportional increase in operations work for the team. But once again, Ani summarized it best:
As the old commercial goes… ‘Old platform: $60,000. New platform: $6,000. Getting to focus all of my time on development: priceless.'
Thank you very much to Ani Hammond and the rest of the Curations team at Bazaarvoice for putting together the presentation that inspired this post. Be sure to check out Ani’s full presentation in addition to dozens of other high-quality talks from MongoDB World on our YouTube channel.
High-end retailer in Germany delivers omni-channel shopping experience on MongoDB Atlas for thousands of daily online users
The importance of delivering an optimized customer experience cannot be overstated, especially if your business is high-end retail. For Breuninger, the customer-first approach has been in their DNA for more than 130 years.
When the top German retailer set out to build a new e-commerce platform, they wanted the online experience to match that of walking in to one of Breuninger’s premium department stores. Accomplishing this goal required a feature-rich, high-performance, and reliable database capable of supporting complex data sets across multiple categories.
“Today, our development teams have a lot of independence. We only have a handful of rules about how they design and build applications within their respective business units,” says Benedikt Stemmildt, Lead Software Architect of E. Breuninger GmbH & Co. “It’s not quite a rule that you have to use MongoDB, but you do have to explain yourself if you don’t.”
However, it wasn’t always this way. Breuninger’s previous platform was built on one of the industry-standard product content management (PCM) platforms, which Stemmildt felt was “monolithic and difficult to code for.” Code freezes were common and the underlying architecture was a frequent cause of frustration for an organization striving to adopt more agile processes.
A new development and feature roll-out approach was needed to execute the company’s aggressive omni-channel integration plans, and time to market for new online features became a top priority. Breuninger decided to build a technology group in response, going from 10 to 30 in-house developers in just a year.
“We broke down our monolithic architecture and split our application into separate microservices that reflect how our customers shop in the physical stores,” Stemmildt says. “It’s the customer journey — they search, discover, evaluate, and buy not just individual products, but complete outfits.”
“To reflect this architectural change, we split our development teams by different steps of the customer journey and kept dependencies to an absolute minimum,” Stemmildt continues. “One key to making this work is a high-performance database capable of working easily with data in lots of different ways. The document model of MongoDB means we can deliver data with the quality and detail that reflects our products and shopping experience.”
The result? Much faster time to market. Breuninger was able to build their omni-channel platform in months rather than years by enabling teams to decide on important architectural components for their own sections, without having to ask the permission of other teams.
As a seven-year veteran of MongoDB, Stemmildt was confident in recommending the database to his organization. “There are a lot of good databases,” he says. “However, many of them require developers to have a deep knowledge about how they work before getting any benefit. MongoDB is not like that. It’s very quick to learn and start getting results. Our teams are able to deliver features straight away. Once users do expand their use of the database, it’s so feature-rich that you never get a sense of having to push it beyond what it was designed for.”
And agile wouldn’t be agile without automation. “Everything we deploy is automated, and with MongoDB Atlas on AWS, the deployment and management of our databases fit neatly into our processes. After a period of operating MongoDB ourselves on EC2, it’s great not having to worry about the details and not having to spend time setting up, configuring, and managing database[s]. You free up a lot of opportunities to add value to your service by not running things yourself.”
AWS offers a healthy mix of other tools for the teams at Breuninger to leverage, such as a managed Kubernetes service and serverless Lambda functions. MongoDB Atlas and AWS also help Breuninger stay on the right side of the regulators. “We need to comply with GDPR so we keep everything running within our borders. MongoDB Atlas’s built-in security features have helped us satisfy these requirements.”
The finished platform might look different to someone who is used to traditional architectures, but to Stemmildt, not being restrained by legacy approaches makes a lot of sense. “Each of our teams owns one or more sections of the customer journey. The search team updates its own database, pulling data in from the product data producer via a feed and re-populating its own database as needed. We don’t have to ripple refreshes out across the system as they happen. That means each team is free to add new features without changing some core database component and affecting other teams. Self-contained systems are an important design rule.”
And although there are some 25 different and largely independent systems, the customers see just one website. A front-end proxy uses server-side includes to marshal data as required from a mix of micro-frontends before delivering the final composite to the shopper. Product data, product availability, outfit data, price information, navigation metadata — these are all woven together from separate MongoDB databases as the customer goes through the shopping experience online.
Comparing a microservices architecture to a monolithic one revealed to Breuninger that some metrics don’t matter as much as they once did, while others matter more. “With multiple teams developing things so rapidly, I don’t know exactly how much total data is in play. But we are a very metrics-driven company, not just in the technical infrastructure but across the business. We know when a component is and is not working well from both a technical and business perspective, if it needs optimizing for performance, or whether it is delivering value to the business or we need to revisit that aspect of the system architecture.”
While Stemmildt couldn’t comment too much on future plans, he’s enthusiastic about MongoDB’s part in whatever they may be. “We wanted high performance, but most importantly we wanted to be able to add more features. We’re not using MongoDB’s graph database feature yet, but we may be by the end of the year. There are a lot of things we could do with text search, too.”
Other new features — such as multi-document transaction support in MongoDB 4.0 — may also be useful, but in unorthodox ways. “I don’t actually think transactions are needed anymore for our platform,” he laughs, “But there are some teams, like the customer data team, who don’t agree with me yet and won’t use MongoDB because of that. So the release of MongoDB 4.0 will help me to help them make the transition.”
While customers won’t see the nuts and bolts of Breuninger’s transformation to a data-driven enterprise, they will benefit from the company’s newly integrated omni-channel platform, which delivers an improved customer experience and more ways to get inspired.
And to anyone thinking about using MongoDB on their next project, Stemmildt has just one piece of advice: “Use it. Get a MongoDB Atlas account, create a cluster, and play with it. The way we see it, after the majority of our teams have naturally adopted MongoDB, if you can’t say why you should use another database, then you should just use MongoDB.”
2018 MongoDB Innovation Award Winners
We received an overwhelming number of nominations for the fifth annual MongoDB Innovation Awards, recognizing companies who are using MongoDB to dream big and deliver incredibly bold, innovative solutions that are moving forward industries and changing lives for the better.
We are thrilled to announce our 12 winners who will be honored at MongoDB World, New York, June 26 and 27.
See the full list and read a bit more about how they are disrupting the status quo here:
Global Go to Market Partner of the Year: Accenture
Accenture is a leading global professional services company, providing a broad range of services and solutions strategy, consulting, digital, technology and operations. Combining unmatched experience and specialized skills across more than 40 industries and all business functions, Accenture works at the intersection of business and technology to help clients improve their performance and create sustainable value for their stakeholders. The company partners with more than three-quarters of the Fortune Global 500, driving innovation to improve the way the world works and lives. Accenture and MongoDB have worked together to help organizations leverage the power of data to gain a competitive edge.
The Enterprise: Charles Schwab
Charles Schwab is one of the largest financial services firms in the United States. To improve customer experience, speed up development cycles, and prepare for cloud portability, Charles Schwab is modernizing a significant portion of its applications by migrating to MongoDB-powered microservices. Multiple applications are built on MongoDB, including an authentication app leveraged by retail customers as well as a portfolio management solution utilized by registered investment advisors.
Launch Fast: Coinbase
Coinbase is dedicated to creating an open financial system for the world and defining what the future of finance will look like. To do this, they built the most trusted and regulatory compliant global cryptocurrency trading platform to broker exchanges of Bitcoin, Bitcoin Cash, Ethereum and Litecoin as well as pioneering cryptocurrency indexes and institutional cryptocurrency trading. In 2017, they experienced exponential growth with over 20M+ users and $150B+ being traded on their platform in over 190 countries. The Coinbase engineering team scaled and optimized MongoDB to respond to this unprecedented volume of traffic and to prepare for future waves of cryptocurrency enthusiasm.
Scale: Epic Games
Epic Games develops cutting-edge games and cross-platform gaming engine technology. Their massively popular, multi-platform game, Fortnite, has been played by more than 125 million gamers around the globe. The Epic team has implemented a number of best practices and performance improvements to get the best scaling and availability characteristics out of MongoDB.
Data-Driven Business: Freddie Mac
Freddie Mac set out to modernize a number of applications that were previously built on legacy relational databases. One mission-critical application, a property appraisal tool, held massive amounts of property and loan information, but was increasingly expensive and time consuming to update. Turning to MongoDB, Freddie Mac was able to collect information from a variety of different sources in a variety of formats to build a single view of all the information needed to accurately appraise a property. In the months since using MongoDB, Freddie Mac has seen an increase in developer productivity.
Customer Experience: Fresenius Medical Care North America
Fresenius Medical Care North America is the premier health care company focused on providing the highest quality care to people with renal and other chronic conditions. Through its industry-leading network of dialysis facilities, outpatient cardiac and vascular labs, and urgent care centers Fresenius Medical Care North America (FMCNA) provides coordinated health care services at pivotal care points for hundreds of thousands of chronically ill customers throughout the continent.
Since 2015, FMCNA has used MongoDB Enterprise Advanced for a variety of projects to help support their mission to deliver superior care that improves the quality of life of every patient. These projects have included analytics platforms, a data lake and the FHIR platform (a healthcare standard for exchanging medical records securely and at scale). However, the most impactful application has been a single view of the patient platform built on MongoDB. This platform brings together a variety of data sources to ensure the patient, doctors and other caregivers all have a complete understanding of the treatments required and can make adjustments with confidence.
Healthcare: Genomics England
Genomics England, a company owned by the UK government's Department of Health and Social Care, is working with the NHS to sequence 100,000 genomes from patients with rare diseases and their families, as well as patients with common cancer. In the future, there may be a diagnosis where there wasn't one before and, in time, there is the potential of new and more effective personalized treatments for patients.
On average, 1,000 genomes are sequenced per week, which amounts to around 10 terabytes of data per day. To manage this immense and sensitive data set as well as power the data science that makes it all possible, Genomics England used MongoDB Enterprise Advanced. The partnership with MongoDB allows the processing time for complex queries to be reduced from hours to milliseconds, which means scientists can discover new insights more quickly.
Internet of Things: Humana Inc.
With a variety of applications built on MongoDB, Humana is changing healthcare for the better. One of their IoT applications called Go365 is a corporate wellness and rewards program which helps employees live healthier lives, which in turn increases productivity and reduces overall health claims costs for employers. Go365 features a personalized program that inspires, supports, and rewards members for taking steps to improve and continue healthy behavior. Users are able to compete in challenges, connect their fitness devices and mobile apps to log healthy activities and earn points, reward themselves through the Go365 Mall, and track their progress. In fact, by year 3, people who engaged with the program saw that the cost of their health claims were reduced by over 10%, relative to those of unengaged members.
Delivery Partner of the Year: Infosys
A perennial winner, this is the third year in a row Infosys has won a MongoDB Innovation Award. As a global leader in consulting, technology and next-generation services, this year Infosys has been working closely with MongoDB to accelerate application modernization for client organizations. A key part of this is the joint delivery of single view and mainframe modernization offerings to migrate and digitize business-critical applications away from rigid tabular databases and on to next-generation technology. In this long standing partnership, Infosys and MongoDB are already helping many large enterprises with renewing and modernizing their IT landscape.
The William Zola Award for Community Excellence: Ken W. Alger
Ken Alger is one of the most prolific bloggers on MongoDB's technology with dozens of posts in the past two years. He is a self-taught programmer and a teacher at Treehouse. An avid follower of open-source, he has previously sat on the board of directors of the Django Software Foundation. He is delighted to share his extensive MongoDB knowledge via his blog, Twitter, and his GitHub account. He exemplifies the true community spirit of MongoDB and The William Zola Award for Community Excellence.
Savvy Startup: Radar
Radar, a seed-stage startup and member of the MongoDB Startup Accelerator program, has built iOS and Android SDKs on MongoDB Atlas and AWS. As the location platform for modern apps, Radar allows developers to easily add location context and tracking to their applications. Radar currently runs on more than 25 million devices around the globe, processing billions of locations each week.
7-Eleven is continuing to redefine what convenience is. By leveraging MongoDB Atlas on AWS and a microservices architecture, 7-Eleven has built an e-commerce application called 7-Now which allows consumers to browse a product catalog connected to their local store’s inventory, make purchases on their mobile phones, and schedule in-store pick up or delivery through services like Postmates. This application not only streamlines the consumer’s experience, but also gives the 7-Eleven team extensive analytics capabilities allowing them to improve the overall customer experience. This is sure to have a major impact in their 10,000 stores in the US and Canada, and with 60% of the US population living within one mile of a 7-Eleven.
Stratifyd & MongoDB: AI-Driven Business Insights to Keep Customers Happy
2017 was a banner year for MongoDB's partner ecosystem. We remain strategic about engaging with our channels, and the results are validating our approach. Our strong network of ISVs, global system integrators, cloud, resellers, and technology partners is a competitive differentiator that helps us scale.
We are especially excited about the innovation and growth in store for our ISV business in 2018. It's already off to a great start. Our newest ISV partner Stratifyd is a fantastic example of how platforms built around MongoDB address serious market needs with the most cutting-edge, innovative technology.
Stratifyd is an end-to-end customer analytics platform powered by AI. The platform provides competitive advantages to some of the most recognized brands in the world. LivePerson, Etsy, MASCO, Kimberly-Clark, and many more rely on Stratifyd for a 360-degree view of their end customers.
Stratifyd analyzes customer interactions such as online reviews, social media posts, phone calls, emails, chats, surveys, CRM data, and more to turn them into actionable business insights which increase customer acquisition and retention, which is critical to the continued success of Stratifyd’s clients. In addition to these benefits, Stratifyd is just a flat-out cool implementation of AI.
I caught up with Stratifyd's CTO, Kevin O'Dell, to discuss the data technology behind the platform, and how MongoDB drives value for their customers.
For anyone that isn’t familiar with Stratifyd yet, how do you describe the platform?
Stratifyd uses human generated data to analyze, categorize, and understand intent with the purpose of changing human behavior. This changes the way brands interact with their customers, but also the way customers interact with brands, increasing customer acquisition and raising retention rates.
What was the genesis of the company? Why did you set out to build this?
Stratifyd was a result of postdoctoral research done at the University of North Carolina, Charlotte. Our founders were researching how AI could analyze unstructured data. During their research, they discovered strong business and government use cases. The founding team was working with numerous three letter agencies on predicting terrorist and disease movements globally. They were able to raise millions of dollars in funding from these agencies. The demand for a product organically grew from there, which led to the development of the Stratifyd platform.
I love the insights Stratifyd can provide – how would you describe the unique advantages that Stratifyd gives its customers?
Stratifyd provides near real-time business intelligence for contact center, marketing, product, and customer experience teams, all based on customer interactions. These insights enable businesses to be proactive rather than reactive in regard to business strategy. Stratifyd customers are able to respond to customer requests, complaints, or general feedback in near real time, changing the way companies interact with their end users. For example, we have empowered a customer with the knowledge to launch a new product line. Another customer gained insights that fundamentally changed how they are rolling out a 700+ million-dollar brand in a new continent.
What kind of feedback are you getting from customers that have deployed Stratifyd in their businesses?
Our customers love using our platform. They are surprised at how simple it is to use, and how powerful it is. They really appreciate how Stratifyd is making AI and machine learning meaningful for them in their day-to day-lives. Stratifyd helps ensure measurable results from day one. Speaking of implementation, they REALLY love that we don’t just use the term day one figuratively – customers are up and running in less than a day.
Talk to me about how you landed on MongoDB. What were you looking for in a database, and what problems were you having before moving to MongoDB?
That's an easy one to answer: speed and flexibility. Stratifyd ingests data from hundreds of sources. We needed a database that could keep up with high read and write request rates while handling a flexible schema. The hardest problem we were trying to solve was lack of secondary indexes; with those, MongoDB accelerated our query response times by at least 100x.
Can you share any best practices for scaling your MongoDB infrastructure? Any impressive metrics around the number of interactions, the volume of reads / writes per second, response times?
As a SaaS-first platform, being always-on is a HUGE best practice for us. MongoDB’s innate replication and failover abilities ensured less than 17 minutes of total downtime last year! Using MongoDB as our backend system, our AI can process a quarter of a million words in less than a minute.
How do you measure the impact of MongoDB on your business?
Our business wouldn’t be able to succeed without MongoDB. The uptime, failover, query response times, secondary indexes, and dynamic schemas have empowered most of Stratifyd’s key differentiators.
What advice would you give someone who is considering using MongoDB for their next project?
With all projects, I recommend truly understanding the requirements for the end results. There is a ton of excellent technology out there, but picking the wrong one can be detrimental to project success. Always run numerous tests, comparing different stacks to make sure you find the right one and fail fast on the wrong technology stack.
Stratifyd has some impressive customers, from the Fortune 500 to some really innovative startups: what’s next for the company?
We have some pretty big things planned for 2018. We're now providing more than just actionable intelligence; we are now streamlining and automating workflows. We are closing the customer feedback loop, which enables us to plug Stratifyd into any business process quickly to deliver measurable results.
DarwinBox Evolves HR SaaS Platform and Prepares for 10x Growth with MongoDB Atlas
Evolution favors those that find ways to thrive in changing environments. DarwinBox has done just that, providing a full spectrum of HR services online and going from a standing start to a top-four sector brand in the Indian market in just two years. From 40 enterprise clients in its first year to more than 80 in its second, it now supports over 200,000 employees, and is hungrily eyeing expansion in new territories.
“We’re expecting 10x growth in the next two years,” says Peddi. “That means aggressive scaling for our platform and MongoDB Atlas will play a big role."
Starting from a blank sheet of paper
The company’s key business insight is that employees have grown accustomed to the user experience of online services they access in their personal lives. However, the same ease of use is simply not found at work, especially in HR solutions that address holiday booking, managing benefits, and appraisals. DarwinBox’s approach is to deliver a unified platform of user-friendly HR services to replace a jumble of disparate offerings, and to do so in a way that supports its own aggressive growth plans. The company aims to support nearly every employee interaction with corporate HR, such as recruitment, employee engagement, expense management, separation, and more.
“We started in 2015 from a blank sheet of paper,” Peddi says. “It became very clear very quickly that for most of our use cases, only a non-relational database would work. Not only did we want to provide an exceptionally broad set of integrated services, but we also had clients with a large number of customization requirements. This meant we needed a very flexible data model. We looked at a lot of options. We wanted an open source technology to avoid lock-in and our developers pushed for MongoDB, which fit all our requirements and was a pleasure to work with. Our databases are now 90 percent MongoDB. We expect that to be at 100 percent soon.”
Reducing costs and future-proofing database management
When DarwinBox launched, it ran its databases in-house, which wasn’t ideal. “We have a team of 40+ developers, QA and testers, and three running infrastructure, and suddenly we’re growing much faster than we expected. It’s a good problem to have, but we couldn’t afford to offer anything less than excellent service.” Peddi emphaszied that of all the things they wanted to do to succeed, becoming database management experts wasn’t high on the list.
This wasn’t the only reason that MongoDB Atlas looked like the next logical step for the company when it became available, says Peddi, “We were rapidly developing our services and our customer base, but our strategies for backing up the databases, for scaling, for high availability, and for monitoring performance weren’t keeping up. In the end, we decided that we’d migrate to Atlas for a few major reasons.”
The first reason was the most obvious. “The costs of managing the databases, infrastructure, and backups were increasing. In addition, it became increasingly difficult to self-manage everything as requirements became more sophisticated and change requests became more frequent. Scaling up and down to match demand and launching new clusters consumed precious man hours. Monitoring performance and issue resolution was taking up more time than we wanted. We had built custom scripts, but they weren’t really up to the task.”
With MongoDB Atlas on AWS, Peddi says, all these issues are greatly reduced. “We’re able to do everything we need with our fully managed database very quickly – scale according to business need at the press of a button, for example. There are other benefits. With MongoDB technical engineers a phone call away, we’re able to fix issues far quicker than we could in the past. MongoDB Compass, the GUI for the database, is proving helpful in letting our teams visually explore our data and tune things accordingly.”
Migrating to Atlas has also helped Darwinbox dramatically reduce costs.
We’ve optimized our database infrastructure and how we manage backups. Not only did we bring down costs by 40%, but by leveraging the queryable snapshot feature, we’re able to restore the data we actually need 80% faster.Chaitanya Peddi, Co-founder and Head of Product, DarwinBox
The increased availability and data resilience from the switch to MongoDB Atlas on AWS eases the responsibility in managing the details of 200,000 employees’ working lives. “Data is the most sensitive part of our business, the number one thing that we care about,” says Peddi, “We can’t lose even 0.00001 percent of our data. We used to take snapshots of the database, but that was costly and difficult to manage. Now, it’s more a live copy process. We can guarantee data retention for over a year, and it only takes a few moments to find what you need with MongoDB Atlas.”
For DarwinBox to achieve its target of 10x growth in two years, it has to – and plans to – go international.
“We had that in mind from the outset. We’ve designed our architecture to cope with a much larger scale, both in total employee numbers and client numbers, and to handle different regulatory regimes.” According to Peddi, that means moving to microservices, developing data analytics, maybe even looking at other cloud providers to host the DarwinBox HR Platform. He added: “If we were to do this on AWS and self-manage the database with our current resources, we would have to invest a significant amount of effort into orchestrating and maintaining a globally distributed database. MongoDB Atlas with its cross-region capabilities makes this all much easier.”
Darwinbox is confident that MongoDB Atlas will help the organization achieve its product plans.
“MongoDB Atlas will be able to support the business needs that we've planned out for the next two years.” says Peddi, “We’re happy to see how rapidly the Atlas product roadmap is evolving.”