Evaluating Amazon DocumentDB Compatibility with MongoDB

Amazon claims that migrating from MongoDB to DocumentDB is “as easy as changing the database endpoint to the new Amazon DocumentDB cluster”.

We assessed DocumentDB’s compatibility claims by running 6 MongoDB test suites, totaling over 1,000 tests, against DocumentDB’s API emulation. This is the suite that we use to test MongoDB’s conformity and correctness on every database release, and is the best representation of what the full MongoDB API is.

The tests comprise:

jsCore: over 800 tests of MongoDB CRUD operations and database commands

aggregation: over 150 tests of the MongoDB aggregation pipeline

jsCore_decimal: evaluates correct behavior of applications using the decimal data type for high precision, fractional numeric data typical in financial and scientific workloads

jsonSchema: tests the data governance controls provided by MongoDB

change_streams: tests MongoDB change streams, used by developers to build event-driven data pipelines that react in real time to database changes

At the time of testing, DocumentDB emulates a subset of the MongoDB 3.6 protocol. Unlike DocumentDB, MongoDB Atlas — the fully managed MongoDB service — supports the latest MongoDB 4.0 release.

Test Results

61% of all of the correctness tests failed when running against the DocumentDB API emulation. For developers, this means that:

  1. Any existing MongoDB apps relying on this functionality would need to be re-engineered if they are to be migrated to DocumentDB

  2. Any new apps written against the DocumentDB API only support a subset of MongoDB’s functionality

Number of TestsSucceededFailed
jsCore843339504
aggregation1505694
jsCore_Decimal1376
change_streams*10010
jsonSchema*24024
Totals1,040402638

*Feature not supported by the DocumentDB API emulation

In terms of functionality, DocumentDB most closely resembles MongoDB 2.4, which was released in 2013. As a result, developers will need to either:

  1. Reimplement required database functionality back in the application tier, slowing down the pace of application development

  2. Move multiple copies of the data into adjacent AWS technologies, with the associated increases in development and operational costs and platform complexity

In contrast, Atlas is updated as soon as each new database release is declared as Generally Available, meaning developers don’t have to wait months or years to access the latest platform enhancements. The service is backed by thousands of support engineers, consultants, and solutions architects from MongoDB and the partner ecosystem who offer the benefits of collective MongoDB knowledge acquired supporting tens of thousands of MongoDB customers over the past decade.