Migration tool for Python

We are using MongoDB in our project (with a Python backend) and every once in awhile there’s a need to run migrations on the production environment. As far as I can see there are a few migration tools for NodeJS and other frameworks/languages, but there lacks a good solution for Python. The existing libraries are not well supported and lack a few key functionalities.

I’m wondering if anyone here has experience with migrating on Python. What libraries did you use? Any best practices?

I’m also thinking about creating an open-source tool for it. Is it worth the effort?

Hi Henry! Welcome to the MongoDB Community Forums, and thank you for your question. To help us better assist you, can you please share what Driver/ODM/ORM you are using to access MongoDB from Python?

By ‘migration’ I assume here that you mean a tool like alembic (Welcome to Alembic’s documentation! — Alembic 1.9.4 documentation)? As far as we know, no such tool exists for MongoDB’s Python ODMs though people have tried to roll their own as you are planning (e.g. Bitbucket).

In general, migrations aren’t something that have much importance in the MongoDB ecosystem because of the inherently schema-free nature of the database. The use of migrations implies that your app is written in a way that assumes a rigid schema which means it cannot take advantage of MongoDB’s flexible data model going forward. Instead of a migration, you might want to consider using the schema versioning in your data-model - Building with Patterns: The Schema Versioning Pattern | MongoDB Blog. When using this kind of a pattern, your application can lazily migrate documents i.e. when a document with an outdated schema version is retrieved, it can be passed through ‘migration code’ that migrates it to the newest schema version before carrying on with applying the application logic.

2 Likes

Following up on this answer, we have been looking for a client ODM tool (preferably in Python) that can actually implement the schema versioning pattern. So far, we have not found one. Neither Mongoengine nor PyMODM seem to support it naturally and the one we use (Mongoengine) makes this pattern especially difficult.

Does anyone have any recommendations here?

We have same problem and found pymongo-migrate thats looks good. Going to use it in prod.