Ken Hoff

STREAM: How MongoDB Atlas and AWS help make it easier to build, scale, and personalize feeds that reach millions of users

This is a guest post by Ken Hoff of Stream ( getstream.io ). Stream is a platform designed for building, personalizing, and scaling activity feeds that reach over 200 million users. We offer an alternative to building app feed functionality from scratch by simplifying implementation and maintenance so companies can stay focused on what makes their products unique. Today our feed-as-a-service platform helps personalize user experiences for some of the most engaging applications and websites. For example, Product Hunt , which surfaces new products daily and allows enthusiasts to share and geek out about the latest mobile apps, websites, and tech creations, uses our API to do so. We’ve recently been working on an application called Winds , an open source RSS and podcast application powered by Stream, that provides a new and personalized way to listen, read, and share content. We chose MongoDB to support the first iteration of Winds as our developers found the database very easy to work with. I personally feel that the mix of data model flexibility, scalability, and rich functionality that you get with MongoDB makes it superior to what you would get out of the box with other NoSQL databases or tabular databases such as MySQL and PostgreSQL. Our initial MongoDB deployment was managed by a vendor called Compose but that ultimately didn’t work out due to issues with availability and cost. We migrated off Compose and built our own self-managed deployment on AWS. When MongoDB’s own database as a service, MongoDB Atlas , was introduced to us, we were very interested. We wanted to reduce the operational work that our team was doing and found Atlas’s pricing much more predictable than what we had experienced with our previous MongoDB service provider. We also needed a database service that would be highly available out of the box. The fact that MongoDB Atlas sets a minimum replica set member count and automatically distributes each cluster across AWS availability zones had us sold. The great thing about managing or scaling MongoDB with MongoDB Atlas is that pretty much almost all of the time, we don’t have to worry about it. We run our application on a deployment using the M30 size instances with the auto-expanding storage option enabled. When our disk utilization approaches 90%, Atlas automatically provisions us more with no impact to availability. And if we experience spikes in traffic like we have in the past, we can easily scale up or out using MongoDB Atlas by either clicking a few buttons in the UI or triggering a scaling event using the API. Another benefit that MongoDB Atlas has provided us is on the cost savings side. With Atlas, we no longer need a dedicated person to worry about operations or maintaining uptime. Instead, that person can work on the projects that we’d rather have them working on. In addition, our team is able to move much faster. Not only can we make changes on the fly to our application leveraging MongoDB’s flexible data model, but we can deploy any downstream database changes on the fly or easily spin up new clusters to test new ideas. All of these can happen without impacting things in production; no worrying about provisioning infrastructure, setting up backups, monitoring, etc. It’s a real thing of beauty. In the near future, we plan to look into utilizing change streams from MongoDB 3.6 for our Winds application, which is already undergoing some major upgrades (users can sign up for the beta here ). This may eliminate the need to maintain separate Redis instances, which would further increase our savings and reduce architectural complexity. We’re also looking into migrating more applications onto MongoDB Atlas as its built-in high availability, automation, fully managed backups, and performance optimization tools make it a no-brainer. While there are other MongoDB as a service providers out there (Compose, mLab, etc.) available, no other solution comes close to what MongoDB Atlas can provide. --- Interested in reducing costs and faster time to market? Get started today with a free 512 MB database managed by MongoDB Atlas. Be a part of the largest gathering of the MongoDB community. Join us at MongoDB World .