This is a guest post by Oren Rubin, CEO @ Testim
Over the last three months we have migrated our service database from Firebase to MongoDB. We have been using Firebase for the last three years, it served us well when we were just getting started, but it came to a point where it started holding us back and jeopardized our business. The transition was quite smooth and allowed us to scale our business and enhance our product development efforts. In this post we will explore the reasons why we initially selected Firebase, the challenges we experienced as our needs evolved, and the reasons why we decided to make the transition.
Starting with Google Firebase
Three years ago when we started Testim, our MVP was just a little more than a proof of concept. At the time, our team was made up of only front end developers. We did not have the resources to hire an experienced backend developer. Additionally, the serverless paradigm was quite strong back then (not objective apparently) and Firebase fitted well into this mindset. We were building an Angular 1.x app which required a lot of frequent data structure/schema changes so a JSON-like database seemed just right.
The top benefits we experienced
- Real time updates - This is one of the neat features in Firebase. You can see changes in specific subtree paths inside your project with a single SDK call. No socket configuration is required, performance is instant and we didn’t notice any data glitches while using it. This allowed us to connect our views to real time data changes. It also allowed us to create cross-entity messaging between our web client, mobile client and backend, which enriched our service experience.
- Extra Services
- Database service - Firebase is also packed with user authentication APIs (including social OAuth 2.0) which saved us a lot of time in implementing signin/signup functionality.
- Data permissions - This mechanism allowed us to define access to specific parts of the project to certain users. This enabled us to create a subtree per customer with restricted access.
- Website and Single Page Application (SPA) hosting - This capability was great while we were just started. At some point we even hosted our website and SPA at Firebase.
Reasons why we switched
Messy coding & architecture
Soon we noticed that the ability to access your data in a single SDK call from anywhere in the client will force you to write some very bad code where business flow and data manipulation are mixed too often. One of the first things we did (too late apparently) was to limit Firebase calls to a specific module in our client so it hides behind an abstraction, and once someone develops a new feature it is done through a set of complicated APIs.
You can do some very basic filtering and pagination with Firebase, but for more complex queries, you will have to write code to do it yourself or manipulate the way you store your data so you don’t need to query the database. Implementing queries with code will force you to download large amounts of data and start processing it in the client, which is not great for performance.
No Local or Staging environment
Developers need more than just production to build a real product. Of course we could have created multiple projects, set them exactly as we set our production, and then have the our client code point to the right project. This of course defeats the purpose why we choose to work with Firebase - it’s simplicity and the ability to make a lot changes quickly. Pushing changes frequently to multiple projects was not an option.
Data Control and Backups
Some time into our startup existence, where our customers usage began to increase, we started to notice our data was growing into the 40-100 gigabyte space. At this point exporting our data through the UI was either not working, or it disrupted the WHOLE service for a while. We had to contact firebase by mail to get a database copy as a zip file. When our database reached 700GB it took 2.5 days to generate this file, which really hurt our ability to move quickly.
Performance & Monitoring
As long as Firebase was working perfectly we didn’t care that there was no way to easily monitor what was going on under the hood. But one day we started noticing a slowdown which soon evolved into endless latency to read data. Since our app loads data on startup it meant a complete service outage!! And we had no way to root cause why it had happened, or to figure out a way to fix or workaround it.
Once we had our service outage due to the Firebase performance issues we couldn’t get decent support, although we were paying a few thousand dollars a month at that point. We opened a high priority ticket, the Firebase support team did respond instantly, but no real resolution for the problem’s root cause was provided. This continued for more than 4 days until the issue was finally resolved.
Firebase is a great tool for small projects that require real time data synchronization between your front and backend. It’s very flexible and easy to learn and doesn’t require any special setup or DevOps effort. It will fit perfectly for a small client web or mobile project. However, if your goal is to build a SaaS product where data relations, queries, performance, backups, etc. matter, we determined that this was not the ideal tool for our needs.
The migration to MongoDB succeeded and was smooth. MongoDB offers a more mature and feature-rich environment. Now we can fetch specific data using complex queries. In addition, it enables us to run the database locally and support on-premises environments, which is very important for our enterprise customers.
Moreover, MongoDB reduces our costs and provides enhanced monitoring and backup services. We also use the MongoDB Atlas cloud database service for some of our customers. And so with MongoDB, we can have complete platform portability, providing the freedom to run wherever our developers, and our customers, need.