As you may have read previously on the blog, the MongoDB team is adopting the Go language for a variety of projects. At last month’s New York MongoDB User Group, Sam Helman presented on how MongoDB is using Go for new and existing cloud tools. In Sam’s talk, you’ll learn how MongoDB is using Go for the backup capabilities in MongoDB Management Service and a new continuous integration tool.
Why go with Go? Between the lightweight syntax, the first-class concurrency and the well documented, idiomatic libraries such as mgo, Go is a great choice for writing anything from small scripts to large distributed applications.
Thanks to g33ktalk for recording Sam’s talk.
Open Source Software in Business
Open Source software (OSS) has made significant strides in enterprises. From the New York Stock Exchange to the smallest mom-and-pop store, the Linux operating system is powering today's business servers. Its growing popularity is due in no small part to the fact that thousands of IT professionals around the world recognize that producing and distributing this type of open standard software has tremendous benefits over traditional commercial products. Unlike off-the-shelf software, open source gives users access to the underlying source code of the application that programmers write and how they instruct a computer to perform certain tasks. Commercial software is considered proprietary to the company that produces it. “It works well, saves money, and is more secure. SMB or large enterprise, there's no down-side, says Steven Vaughn-Nichols, a long-time Linux and open source journalist. “OSS tends to be more secure -- its code is open so anyone can find, report and fix the bugs. You have no idea what may or may not be hiding in proprietary code.” OSS also tends to be cheaper than its proprietary brothers, he adds. “Better security and cheaper to boot? What's not to like?” Previously, security and licensing acted as traditional barriers to adoption. Now, OSS is driving change from the bottom up, according to the seventh annual “Future of Open Source Survey,” of more than 800 respondents, sponsored by North Bridge Venture Partners and Black Duck Software. Survey results indicate OSS is experiencing a growing influence in organizations. This is due to a cultural shift supported by executives' openness to work with active and strong communities to influence projects and spur innovation, the study found. Additionally, “open source has reached a depth and maturity where quality, access to code and costs are no longer barriers to adoption. “This trend is reinforced by thousands of developers working to reduce defects in code, improve its security and innovate with new features and enhancements that get closer to what users want - because those users can have a hand in making it so,” the study found. The biggest factor in OSS adoption, respondents said, is quality, a ranking that increased from third place in 2012. Freedom from vendor lock-in dropped to second place this year, from first place in 2012. Lower costs, big data, and systems integration are the top three business problems open source is solving. In terms of sector growth, the study found that government, as well as health care and media/entertainment are moving toward adoption of open source. Technical capabilities and features were cited by 45 percent of respondents as reasons for why they choose OSS over proprietary solutions. Only 12 percent chose commercial vendor support as being an important factor, according to the study. Linux has taken over some verticals, notes Vaughn-Nichols. “Most stock markets’ core servers, for example, now run Linux. It really has been moving everywhere in other businesses.” As for the types of applications, he says it started with edge servers, file and Web servers, and over the years Linux is now being used for a large variety of apps, including CRM. Sears, for example, used open source software to build a private cloud platform. According to published reports, the company opted for OSS because it helped reduce costs and boost flexibility. And Sears and Chevron are using Hadoop, an open source database to analyze large amounts of data. Because the code is reusable, OSS is also being used by MasterCard to build prototypes of mobile apps. This gives developers the ability to create apps quickly, the company said. While the energy industry typically hasn’t shown an interest in OSS and has generally only used Linux application servers, Chevron is using Hadoop to find oil in areas including the Gulf of Mexico. As for smaller businesses, Vaughn-Nichols says many are probably not aware that their Network-Attached Storage (NAS) device and Internet router “are almost certainly running Linux.” The OS is everywhere, he adds. “It's in your pocket with your Android smartphone and it's what powers your Google Web searches. The only place it really isn't is the conventional desktop.” Better quality was the number one factor for open source adoption in business cited by respondents in the Future of Open Source survey. Other factors in order of importance were: freedom from vendor lock-in; flexibility/access to libraries of software, extensions and add-ons; elasticity/ability to scale at little cost or penalty; superior security; pace of innovation; lower costs; and access to source code. Increasingly, open source software is being chosen over proprietary alternatives. Industry observers say it’s a viable choice for businesses because it provides: Control – Unlike commercial software, OSS lets you make decisions about how to run your business. Flexibility – OSS is licensed in a way that lets you modify it yourself or hire a third party to tailor the software to meet the needs of your business. Reliability – OSS typically has fewer bugs and is more reliable than software developed using a standard commercial development process. Cost – There are little to no upfront costs with OSS. Users only pay for the support they need and most importantly, when they need it. Longevity – If a commercial software company goes out of business, you lose all of your support, bug fixes, security patches and possibility of future upgrades. Contrast that to a mission-critical software application a business is using: all it has to do is find a consulting firm, programmer or another third-party provider. “Enterprises see [open source software] as leading innovation, delivering higher quality and driving growth rather than being just a free or low-cost alternative,’’ says Michael Skok, general partner at North Bridge Venture Partners. “Going forward, as broader adoption creates a virtuous cycle of innovation and investment, we can expect more disruption from open source, new business models and many more exciting new projects and companies." Vaughn-Nichols also sees OSS becoming more ubiquitous in business. “It's becoming invisible. People tend to think of software as what they see in front of them. For most people that means Windows or Macs. Behind all of them everything -- and I mean everything -- has been switching over to OSS.”
Splitit & MongoDB Atlas: Racing to Capture a Global Opportunity
Splitit is a global payment solution that allows businesses to offer installment plans for their customers. Unlike with other buy now, pay later (BNPL) solutions, Splitit shoppers can split their online purchases into monthly installments by using their existing credit, without the need for registration, application, or approval. “We have a very different proposition than others in this space,” says Splitit’s CTO, Ran Landau. “We’re not a financing company. We utilize the customer’s existing credit card arrangement, which allows us to accommodate smaller average deal values and a broader range of installment schedules.” Splitit works with online retailers across all market sectors and diverse price points, and recently raised $71.5 million in investment to fund global expansion. Following its IPO in January 2019, the business had seen strong growth as more consumers moved from brick and mortar to ecommerce. Then COVID-19 hit, and online shopping boomed. Landau recognized that the company needed to quickly scale its infrastructure in order to capture this large opportunity. The Need for Speed Landau joined Splitit in May 2019 and worked to modernize the company’s infrastructure. At the time, the team was using a traditional relational database. “As tech leaders, we need to make the right decision,” he says. “When I came to Splitit, I knew I needed a powerful NoSQL server so that my developers could develop faster and so that we could scale – both things that our relational databases were failing to deliver.” In the interest of getting up and running quickly, Ran’s team thought that they could move faster using a cloud-provider database that mimicked MongoDB functionality. He had used MongoDB before and saw that this solution offered the same drivers he was familiar with and claimed compatibility with MongoDB 3.6. Initially, the new solution seemed fine. But as the team started to migrate more data into the database, however, Landau noticed a few missing features. Scripts for moving documents from one collection to another were failing, and overall performance was deteriorating. The application became slow and unresponsive even though the load on the database was normal. “We were having issues with small things, like renaming collections. I couldn’t search or navigate through documents easily,” recalls Landau. Offline Database: A Breaking Point Then one day, the application was unable to communicate with the database for 20 minutes, and when the database finally came back online, something wasn’t right. Landau contacted support, but the experience was not very helpful. “We were not pleased with the response from the database vendor,” he explains. “They insisted that the issue was on our side. It wasn’t so collaborative.” Fortunately, he had taken a snapshot of the data so Splitit was able to revert back to an earlier point in time. But the incident was troubling. Other teams also had been complaining about how difficult it was to debug problems and connect to the database successfully. Landau knew he needed to find a better solution as soon as possible. MongoDB Atlas: A Reliable, Scalable Solution Landau believed that MongoDB was still the right choice for Splitit, and investigated whether the company offered a cloud solution. He discovered MongoDB Atlas and decided to give it a try. “The migration to MongoDB Atlas was so simple. I exported whatever data I had, then imported it into the new cluster. I changed the connection strings and set up VPC peering in all of my environments,” says Landau. “It was incredibly easy.” Not only was MongoDB Atlas built on actual MongoDB database software, but it was also secure, easy to use, and offered valuable features such as Performance Advisor . “It can tell you which indexes need to be built to increase speed. It’s such a powerful tool — you don’t need to think; it analyzes everything for you,” explains Landau. Another great feature was auto-scaling. “My biggest concern as I scale is that things keep working. I don’t have to stop, evaluate, and maintain the components in my system,” says Landau. “If we go back to doing database operations, we can’t build new features to grow the business.” Auto-archival Made Easy with Online Archive As a business in the financial services industry, Splitit needs to comply with various regulations, including PCI DSS . A key requirement is logging every transaction and storing it for auditing purposes. For Splitit, that adds up to millions of logs per day. Landau knew that storing this data in the operational database was not a cost-effective, long-term solution, so he initially used an AWS Lambda function to move batches of logs older than 30 days from one collection to another periodically. A few months ago, he discovered Online Archive , a new feature released at MongoDB.live in June 2020. With it, Landau was able to define a simple rule for archiving data from a cluster into a more cost-effective storage layer and let Atlas automatically handle the data movement. “The gem of our transition to Atlas was finding Online Archive,” says Landau. “There’s no scripting involved and I don’t have to worry about my aging data. I can store years of logs and know that it’s always available if I need it.” Online Archive gives me the flexibility to store all of my data without incurring high costs, and feel safe that I won't lose it. It's the perfect solution. Ran Landau, CTO, Splitit With federated queries, the team can also easily analyze the data stored in both the cluster and the Online Archive for a variety of use cases. Ready for Hypergrowth and Beyond Looking back, Landau admits that he learned his lesson. In trying to move quickly, he selected a solution that appeared to work like MongoDB, but ultimately paid the price in reliability, features, and scalability. You wouldn't buy a fake shirt. You wouldn't buy fake shoes. Why buy a fake database? MongoDB Atlas is the real thing. Ran Landau, CTO, Splitit Landau is confident that his investment in MongoDB puts in place a core building block for the business’ continued success. With a fully managed solution, his team can focus on building features that differentiate Splitit from competitors to capture more of the market. “We saw our growth triple in March due to COVID-19, but the sector as a whole is expanding,” he says. “Our technology is patent protected. Everything we build moving forward will be on MongoDB. As a company that’s scaling rapidly, the most important thing is not having to worry about my scaling. MongoDB Atlas takes care of everything.”