I recently spoke on an interesting panel at Red Hat Summit entitled "Big Data and Traditional Databases," along with representatives from the MySQL, PostgreSQL and Sybase/SAP communities. At one point, the moderator asked, "What features are your adding to your roadmap to make your database ready for Big Data."
I was stumped. Unlike the relational databases, MongoDB was born as a Big Data database.
This isn't a secret. Whether measured by how IT professionals talk about MongoDB on Twitter, Big Data-related job postings or other means, MongoDB is always rated one of the industry's top-two Big Data technologies.
Which isn't to suggest that 10gen and the MongoDB community are sitting still. We've been adding Big Data-relevant functionality like full-text indexing and continue to improve the already great performance, among other things. But the essential ability to handle a wide variety of data types and sources, in real time, at significant scale? That comes ready out of the box.
10gen has never intended for MongoDB to be solely a "Big Data database." That would be too narrow. MongoDB is a general-purpose database designed for and comfortable managing a broad and growing array of applications. Some will argue that all applications will be Big Data applications going forward. Perhaps. Regardless, MongoDB has it covered.
This Week in MongoDB June 10-16
Here's what's going on in the MongoDB community this week: Learn More Database shakedown: Five reasons why there’s a revolution Protect your data and reduce operational risk with MongoDB Backup Service - Now in Limited Release NoSQL Vs. Hadoop IBM and 10gen are collaborating to fashion a new standard for mobile computing within enterprise networks Considerations for rolling out a NoSQL strategy in the enterprise ComputerWeekly: IBM and 10gen collaborate on database standard for enterprise mobile New MongoDB User Group in Krakow, Poland Free Online Class, MongoDB for Developers, Begins June 17 Fill out the MongoDB Developer Survey Upcoming MongoDB Days June 16: MongoDB Israel June 21: MongoDB New York City July 2: An Evening with MongoDB Sao Paulo Upcoming Webinars: June 11: German Webinar: Was is neu in MongoDB 2.4 June 12: How Partners Can Benefit from our New Program (EMEA) June 13: General Technical Overview of MongoDB for Dev Teams June 18: MongoDB Migration Patterns - How Customers Start Using MongoDB June 20: Replication and Replica Sets Online Education: M101P: MongoDB for Developers (Python) begins June 17 (next Monday) M102: MongoDB for DBAs begins July 17 -- registration now open M101J: MongoDB for Java Developers begins July 29 -- registration now open User Groups and Events this Week June 11: Matt Asay will be giving his talk "NoSQL is the New Normal" at the Boston MUG June 12: Matt Asay will discuss Big Data & Traditional Databases at RedHat Summit & JBoss World June 12: Stockholm MUG June 12-14: Christian Kvalheim will present at the Norwegian Developers Conference Have something you'd like to share? Let us know
Splitit & MongoDB Atlas: Racing to Capture a Global Opportunity
Splitit is a global payment solution that allows businesses to offer installment plans for their customers. Unlike with other buy now, pay later (BNPL) solutions, Splitit shoppers can split their online purchases into monthly installments by using their existing credit, without the need for registration, application, or approval. “We have a very different proposition than others in this space,” says Splitit’s CTO, Ran Landau. “We’re not a financing company. We utilize the customer’s existing credit card arrangement, which allows us to accommodate smaller average deal values and a broader range of installment schedules.” Splitit works with online retailers across all market sectors and diverse price points, and recently raised $71.5 million in investment to fund global expansion. Following its IPO in January 2019, the business had seen strong growth as more consumers moved from brick and mortar to ecommerce. Then COVID-19 hit, and online shopping boomed. Landau recognized that the company needed to quickly scale its infrastructure in order to capture this large opportunity. The Need for Speed Landau joined Splitit in May 2019 and worked to modernize the company’s infrastructure. At the time, the team was using a traditional relational database. “As tech leaders, we need to make the right decision,” he says. “When I came to Splitit, I knew I needed a powerful NoSQL server so that my developers could develop faster and so that we could scale – both things that our relational databases were failing to deliver.” In the interest of getting up and running quickly, Ran’s team thought that they could move faster using a cloud-provider database that mimicked MongoDB functionality. He had used MongoDB before and saw that this solution offered the same drivers he was familiar with and claimed compatibility with MongoDB 3.6. Initially, the new solution seemed fine. But as the team started to migrate more data into the database, however, Landau noticed a few missing features. Scripts for moving documents from one collection to another were failing, and overall performance was deteriorating. The application became slow and unresponsive even though the load on the database was normal. “We were having issues with small things, like renaming collections. I couldn’t search or navigate through documents easily,” recalls Landau. Offline Database: A Breaking Point Then one day, the application was unable to communicate with the database for 20 minutes, and when the database finally came back online, something wasn’t right. Landau contacted support, but the experience was not very helpful. “We were not pleased with the response from the database vendor,” he explains. “They insisted that the issue was on our side. It wasn’t so collaborative.” Fortunately, he had taken a snapshot of the data so Splitit was able to revert back to an earlier point in time. But the incident was troubling. Other teams also had been complaining about how difficult it was to debug problems and connect to the database successfully. Landau knew he needed to find a better solution as soon as possible. MongoDB Atlas: A Reliable, Scalable Solution Landau believed that MongoDB was still the right choice for Splitit, and investigated whether the company offered a cloud solution. He discovered MongoDB Atlas and decided to give it a try. “The migration to MongoDB Atlas was so simple. I exported whatever data I had, then imported it into the new cluster. I changed the connection strings and set up VPC peering in all of my environments,” says Landau. “It was incredibly easy.” Not only was MongoDB Atlas built on actual MongoDB database software, but it was also secure, easy to use, and offered valuable features such as Performance Advisor . “It can tell you which indexes need to be built to increase speed. It’s such a powerful tool — you don’t need to think; it analyzes everything for you,” explains Landau. Another great feature was auto-scaling. “My biggest concern as I scale is that things keep working. I don’t have to stop, evaluate, and maintain the components in my system,” says Landau. “If we go back to doing database operations, we can’t build new features to grow the business.” Auto-archival Made Easy with Online Archive As a business in the financial services industry, Splitit needs to comply with various regulations, including PCI DSS . A key requirement is logging every transaction and storing it for auditing purposes. For Splitit, that adds up to millions of logs per day. Landau knew that storing this data in the operational database was not a cost-effective, long-term solution, so he initially used an AWS Lambda function to move batches of logs older than 30 days from one collection to another periodically. A few months ago, he discovered Online Archive , a new feature released at MongoDB.live in June 2020. With it, Landau was able to define a simple rule for archiving data from a cluster into a more cost-effective storage layer and let Atlas automatically handle the data movement. “The gem of our transition to Atlas was finding Online Archive,” says Landau. “There’s no scripting involved and I don’t have to worry about my aging data. I can store years of logs and know that it’s always available if I need it.” Online Archive gives me the flexibility to store all of my data without incurring high costs, and feel safe that I won't lose it. It's the perfect solution. Ran Landau, CTO, Splitit With federated queries, the team can also easily analyze the data stored in both the cluster and the Online Archive for a variety of use cases. Ready for Hypergrowth and Beyond Looking back, Landau admits that he learned his lesson. In trying to move quickly, he selected a solution that appeared to work like MongoDB, but ultimately paid the price in reliability, features, and scalability. You wouldn't buy a fake shirt. You wouldn't buy fake shoes. Why buy a fake database? MongoDB Atlas is the real thing. Ran Landau, CTO, Splitit Landau is confident that his investment in MongoDB puts in place a core building block for the business’ continued success. With a fully managed solution, his team can focus on building features that differentiate Splitit from competitors to capture more of the market. “We saw our growth triple in March due to COVID-19, but the sector as a whole is expanding,” he says. “Our technology is patent protected. Everything we build moving forward will be on MongoDB. As a company that’s scaling rapidly, the most important thing is not having to worry about my scaling. MongoDB Atlas takes care of everything.”