Nuxeo Achieves 11-Billion-Object Benchmark on AWS in Partnership with MongoDB Atlas
Pushing the Limits to Demonstrate Nuxeo's Ability to Manage the Largest, Most Complex Content Management Requirements Case Study Executive Summary Having already proven that their Content Services Platform could handle nearly unlimited workloads, Nuxeo set out to smash their established benchmark of 1 billion documents. The challenge: load 11 billion objects while maintaining the highest levels of system performance. MongoDB, Nuxeo’s data platform partner, helped make it possible with powerful tools capable of managing hundreds of metadata tags per object across billions of objects. In addition, Nuxeo leveraged the AWS cloud infrastructure and Amazon Elasticsearch Service – both key components of its Nuxeo Cloud offering – to achieve elastic scalability and the highest levels of indexing and search performance. Customers Expect Content Managment Without Limitations Global companies use Nuxeo to build applications that manage enormous volumes of digital content including scanned images, documents, PDFs, and even rich-media assets like high-resolution photos and video. Nuxeo supports high levels of complexity including customization, layers of security, and sophisticated metadata as well as complicated workflows and business processes. This enables Nuxeo customers to put their content to work solving complex business problems and delivering unique content-enabled solutions. David Woolston, VP Business Development at Nuxeo, describes Nuxeo as a content services platform on steroids. “One thing that drives Nuxeo’s success is our ability to scale up to handle the most complex workloads and largest repositories of documents,” explained Woolston. The secret is their relationship with AWS and MongoDB Atlas, the global cloud database service. As a disruptive player in the content services market, Nuxeo has differentiated itself through technology innovation. The Nuxeo Platform operates on the AWS Cloud, which allows for unparalleled flexibility and scalability. Nuxeo also uses the managed database service, MongoDB Atlas. Atlas is capable of managing hundreds of metadata tags across literally billions of Nuxeo objects, storing them securely and making them easily digestible and queryable in JSON-like documents. This means the Nuxeo team can focus on building new content services and platform capabilities rather than managing a database. Challenge Accepted “The biggest companies in the world are coming to us with their largest workloads and saying, ‘we know you can handle this.’ We believe our technology can scale almost endlessly with MongoDB Atlas and AWS,” said Woolston. “With a 1-billion-object benchmark already completed, we really wanted to push the limits and prove it out with a 11-billion-object benchmark.” The idea was to test Nuxeo from an optimal application and configuration perspective, not to simply throw money at more hardware. “In order to solve our customers’ complex challenges, Nuxeo provides an extremely robust platform. The deployment includes Elasticsearch, MongoDB Atlas, and all the bells and whistles of the Nuxeo Platform itself,” said Joe Quinto, Senior Program Manager for Nuxeo Cloud. “We wanted to push the boundaries of every element – stress as many components as we could. Effectively managing 11 billion documents was our yardstick. The goal was not just to hit the ceiling, but to break through it.” The biggest companies in the world are coming to us with their largest workloads and saying, ‘we know you can handle this.’ We believe our technology can scale almost endlessly with MongoDB Atlas and AWS. David Woolston, VP Business Development, Nuxeo Dynamic Testing with a Two-Phase Approach Nuxeo adopted a two-phase approach for its benchmarking exercise. In the first phase, the Nuxeo team used a single Nuxeo repository configured with MongoDB Atlas and Elasticsearch. The point of the exercise was to test the practical limits of a single-repository approach and also to illustrate the inherent advantages of a NoSQL solution like MongoDB. In the first phase, the team was able to successfully scale a single Nuxeo repository to 3 billion objects with no database sharding, a feat that’s virtually impossible with SQL-based technologies. In the second phase of the project, the Nuxeo team employed a multirepository approach and made use of multiple instances of MongoDB Atlas and the Amazon Elasticsearch Service as well as MongoDB sharding to efficiently scale to over 11 billion objects. For both phases of the benchmark the team used an actual Nuxeo Cloud deployment. This was not a highly orchestrated lab exercise. In both phases of the benchmarking project, the team continuously tested to ensure that the Nuxeo Platform and underlying technologies would scale and perform at a level that would meet enterprise customers’ high expectations. The team monitored metrics to determine if, as the repository grew, users could continue to import new objects and metadata at an extremely high rate of ingestion. Since this was a real-world exercise, Nuxeo employed its default Ingestion Pipeline throughout the benchmarking project, complete with metadata import and full-text indexing. Nuxeo also employed automated testing to address common user activities for content management. This included search (both database queries and full-text searches) and navigation as well as create, read, update, and delete (CRUD) actions. Over three months, as Nuxeo was progressing toward their 11-billion-object benchmark and continuing to load new objects and data into the system, MongoDB Atlas was managing the database. While the test was running, the MongoDB team monitored data access patterns and gave advice on best practices for such a large number of documents. It was important to identify points where they needed to increase capacity on the platform before moving to higher tiers. Using more than 100 metrics, MongoDB was able to identify tipping points and suggest ways to improve the environment before going on to the next level. “The Nuxeo team was prepared with a solid plan and they let us know what they wanted to achieve,” said Diego Burstyn, Sr. Solutions Architect at MongoDB. “Our role was to correlate that with what we were seeing under the hood.” Pushing the Limits While Maintaining Response Times and Throughput As previously mentioned, the key success indicators the Nuxeo team was looking for were response times and throughput. It was important to see that the application was responding as expected, and enormous amounts of data could be loaded in a reasonable amount of time. A critical outcome for the project was to be able to provide Nuxeo customers with real-world guidance and best practices for scaling up the Nuxeo Platform along with key services, like MongoDB Atlas. The team categorized its learnings in three specific areas: elasticity, steps, and real-life usability. Bragging Rights and Lessons Learned The 11-billion-object benchmark test proved Nuxeo’s ability to scale with AWS Cloud elasticity. Flexible scalability allowed the team to expand volume when a capacity limit was reached, or for a temporary need like re-indexing data. By implementing the test with deliberate steps, the team learned about infrastructure and configuration adjustments that needed to be made to maintain optimal performance at extreme high volumes. Testing in an actual environment, with real documents, demonstrated how the Nuxeo Platform can handle enormous document volumes in real-life applications. Traditional content management systems can’t scale the way we can,” said Woolston. “But beyond that, the power of MongoDB Atlas combined with the complete toolset and support we get from AWS allows us to scale in the most efficient and intelligent way possible. Hitting the 11-billion-object benchmark was more than a matter of bragging rights. It provided tangible proof of the unique value proposition Nuxeo offers through its partnership with MongoDB Atlas and AWS. This exercise delivered an understanding of how these different components scale and the best practices to do this efficiently. With these learnings and meaningful data, Nuxeo customers are better prepared to scale up, either in Nuxeo Cloud or in their own cloud environments. Get Started Today Try MongoDB Atlas on AWS: Redeem promo code NuxeoAtlas100 for $100 in Atlas Credits Schedule a demo of the Nuxeo Platform
Transforming Credit Management with Credisense and MongoDB
Credit Management is a grind -- clunky, time consuming and laden with risk. It requires millions of dollars to capture consumer attention and nurture through the sales cycle. And then comes the arduous credit assessment, throwing a wrench into the promise of a seamless digital customer experience. In fact, up to 90% of bank new customer applications drop out due to slow onboarding 1 . Even once a deal closes, there is still plenty of work to do. The next hurdle is invoice collections, with default and delinquency rates averaging anywhere between 0.2-54.5% internationally 2 . In today’s digitally-driven market, consumers are demanding quicker turnaround times and instant approvals. This makes simplifying and streamlining the entire credit management process more critical than ever. As MongoDB’s OEM business continues to rapidly expand, it’s a personal priority to work with organizations who are solving serious market needs with the most innovative technology. Credisense, our newest OEM partner, and their MongoDB- powered, full end-to-end origination and credit decisioning solution is a phenomenal example of this. They’re already making splashes worldwide. For example: CTOS Data Systems (Malaysia’s largest credit reporting agency) is enabling banks, utilities, non-bank credit issuers, fintechs and lenders in the P2P lending space to make real time credit decisions using the Credisense platform, enabling things like instant loan approvals for credit cards and auto loans! I had the opportunity to discuss the Credisense platform and the data technology behind it with Richard Brooks, Co-founder and Director. Tell us a bit about yourself and the genesis of the company? Our three co-founders have different, but complementary backgrounds. I have worked for bureau and data companies my entire career and been involved in the automation side quite extensively. Our second co-founder and CEO, Sean Hywood, is a software expert having built up several software companies over his career focusing on low-code technologies. We combined our knowledge with the technical expertise of our third co-founder and CTO, Waylon Turney-Mizen, with the vision of providing enterprise grade functionality to organizations of all sizes. The aim is to allow all businesses to make smarter decisions, faster. For anyone that isn’t familiar with Credisense yet, could you describe why you set out to build this and the problem it’s solving? Credit is a highly regulated, complex, often manual and costly process. McKinsey 3 rightly points out there are five key pressures on credit providers currently: Changing customer expectations, specifically digital and the customer experience Tighter regulatory controls such as AML/CFT and GDPR Data management, increasing reliance on clean data for analysis and decisions Market disruptor such as P2P lenders and digital banks Cost pressures driving down returns There are some sobering stats that show how important these are, such as over $200 billion dollars 4 of regulatory fines in the US alone since the GFC, to the fact that traditional lenders have lost over 30 percent of personal loan market share 5 to agile financial technology companies. All these add up to some serious issue for business, some that even threaten their very existence. Our aim when creating Credisense was to tackle these issues, both by assisting traditional corporates to embrace this digital strategy, to providing this same technology and expertise to smaller businesses so they can compete and level the playing field. How would you describe the platform and the unique advantages that Credisense gives its customers? Our platform is born in the cloud and offers a “no-code” build capability allowing organizations to build out the functionality internally and grow the solution with their business. We have a unique graphical interface, and this coupled with the “no-code” technology allows business people -- not IT -- to build, own and manage the system. The platform itself revolves around the decision and scoring engine which powers the advanced assessment and risk decisions for organizations. Our MongoDB backend means we can confidently scale to handle millions of credit applications and still support real-time workflow and decision making in seconds. How did you land on MongoDB to help you solve these challenges? We needed a database to support a minimum of 100,000 transaction a day across a cloud platform. There are only a handful of NoSQL databases that can support the level of transaction with the ability to further scale if required. MongoDB ticked all the boxes. Add that to MongoDB’s great documentation security, tooling, support and APIs, and it made MongoDB the right choice for our development teams. What advice would you give someone who is considering using MongoDB for their next project? MongoDB offered us extensibility to be on-premises, which is something other cloud database platforms would not offer. It made sense to go with a database platform that offered both so that we could in turn offer this to our customers that require data to be held within their own environments for security reasons. Also, reach out and talk to MongoDB early in your process. The support they give you up front will help ensure you’re making the best decisions. How are you securing MongoDB? We utilize MongoDB Atlas for our Continuous Integration and Testing environment and will have a managed service offering. This is secured with an IP whitelist, secure password and SSL connection which was easy with Atlas and Atlas Professional. We also have a customer-managed deployment secured out of the box behind a VPN that connects the app server to the MongoDB server. It also utilizes a strong username/password combination with minimum length and character requirements. Through our OEM arrangement with MongoDB, we package MongoDB Enterprise as part of our product to ensure our customers have highly secure and enterprise-grade solutions. Where have you deployed MongoDB? On-premises, in the cloud, via MongoDB Atlas? What tools are you using to deploy, monitor MongoDB? All! The requirement for extensibility across platforms without any changes to the code was one of the key reasons for MongoDB selection. MongoDB Atlas removes operational overhead and mitigates risk through automating many of the manual processes (configuring operating system, upgrades, backups and restores). This means we can focus on ensuring our customers have the robust platform they need to provide instant loan approvals. We also have a production environment on-premises. Soon, we will introduce the use of MongoDB Cloud Manager for monitoring and alerts of on-premises production environments. With over 100 metrics and proactive alerting, we’ll be able to catch issues before they arise. References: https://thefinancialbrand.com/66143/7-steps-to-improved-customer-onboarding/ https://data.worldbank.org/indicator/FB.AST.NPER.ZS?year_high_desc=true https://www.mckinsey.com/business-functions/risk/our-insights/the-value-in-digitally-transforming-credit-risk-management https://www.mckinsey.com/business-functions/risk/our-insights/the-value-in-digitally-transforming-credit-risk-management https://qz.com/1334899/personal-loans-are-surging-in-the-us-fueled-by-fintech-startups/
MongoDB wins 2018 Customer Impact award at SpringOne Platform!
MongoDB has won the Independent Software Vendor 2018 Pivotal Partner Award for Customer Impact , at the PivotalSpringOne Platform summit. This award recognizes partners that have delivered technology contributing to notable customer success. We are humbled and honored! In a world with endless options, the most sophisticated and demanding organizations choose to run their business on MongoDB. This is not a coincidence. MongoDB constantly pushes the pace of innovation to addresses the most challenging problems for our customers and strategically partners with leaders like Pivotal to deliver robust solutions to accelerate development and success. We launched MongoDB for Pivotal Cloud FoundryⓇ (PCF) earlier this year and already have amazing and very public success. For example, Merrill Corporate launched a new category for M&A professionals with MongoDB, Pivotal and Microsoft . Our joint solution yielded 20x faster Deployment, client-identified bugs fixed in hours, 25% increase in sales and a true transformation into a product led technology organization! Staying close to customer requirements and focused on customer success - that’s why it’s now easier than ever to rapidly deploy MongoDB powered applications on Pivotal Cloud Foundry by abstracting complexities around ensuring a consistent, predictable, and secure underlying infrastructure that can scale. MongoDB will continue with frequent updates and releases as we evolve our product inline with customer needs. Customers can download a BOSH deployed tile for Pivotal Application ServiceⓇ (PAS) and we are thrilled to announce the beta of MongoDB for the Pivotal Container ServiceⓇ (PKS) as well! Learn more about the tile in these great posts by my colleagues: On Demand MongoDB Enterprise Server on Pivotal Cloud Foundry and MongoDB Enterprise Server for Pivotal Cloud Foundry goes GA For those with us in Washington D.C. for the SpringOne Platform conference, we have two amazing sessions for you: Join MongoDB’s Jeff Yemin (Lead Engineer, Database Engineering) and Pivotal’s Christoph Strobl (Software Engineer) for ‘Next Generation MongoDB: Sessions, Streams, Transactions’ and Diana Esteves (MongoDB Senior Engineer) for MongoDB + CredHub = Secure By Default Data Services on PCF and stop by our meeting room to say hello! We’re honored to have received this prestigious award from Pivotal and look forward to continued success for our joint customers as MongoDB and Pivotal help tackle their biggest challenges!