Community

Muhammed Ali

Software Developer and Technical Writer

Migrating a Flask application from SQL to MongoDB involves several steps, as the two database systems have different paradigms. This article will guide you through converting an existing Flask blog application that uses SQLAlchemy to [Flask-PyMongo](https://flask-pymongo.readthedocs.io/en/latest/) to integrate MongoDB with the Flask application. We will explain the changes made to each file and the reasons behind them.

## **Understanding MongoDB**

MongoDB is a NoSQL database known for its document-style kind of structure and ease of use.

**Key concepts**

- **Collections**: Analogous to tables in SQL databases, collections in MongoDB store documents. However, unlike tables, collections do not enforce a schema, meaning each document can have different fields.
- **Documents**: This is the primary data structure in MongoDB, similar to rows in SQL databases, but more complex. Documents are JSON-like objects that can contain arrays and nested subdocuments.
- **BSON**: Binary representation of JSON is used internally by MongoDB. It extends JSON with additional data types, such as Date and Binary.

## Differences between MongoDB and SQL databases

|                    |                                                                       |                                                                             |
| ------------------ | --------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| **Feature**        | **SQL**                                                               | **MongoDB**                                                                 |
| **Schema**         | Enforces a fixed schema with predefined tables and columns            | Schema-less, allowing for flexible and dynamic data models                  |
| **Data Structure** | Stores data in tables with rows and columns                           | Stores data in collections of documents                                     |
| **Relationships**  | Uses foreign keys and joins to establish relationships between tables | Embeds documents within other documents or references them, but lacks joins |
| **Query Language** | Uses Structured Query Language (SQL)                                  | Uses a rich, JSON-based query language                                      |

## **Planning the migration**

Before migrating the Flask application from SQL to MongoDB, thorough planning is necessary to ensure a smooth transition and preserve data integrity.

Proper planning helps identify potential blockers and develop strategies to address them.

### Steps required for migration

- **Set up MongoDB environment**
- **Assess current SQL schema and relationships**: Review the existing database schema, including tables, relationships, and data types. Identify complex queries and relationships that might require special handling in MongoDB.
- **Define the MongoDB schema**: Design the MongoDB schema considering the document-oriented nature of the database. Map SQL tables and relationships to MongoDB collections and documents.
- **Backup existing data**: Create a complete backup of the current SQL database to prevent data loss. Ensure backup processes are reliable and can be restored if needed.
- **Data transformation and migration strategy**: Choose a migration tool or write custom scripts to handle data migration, as we will do in this article.
- **Update Flask application code**: Modify Flask models to use MongoDB instead of SQLAlchemy. Update routes, forms, and templates to work with the new data structure. Test each module of the application to ensure compatibility with MongoDB.
- **Post-migration testing**: Thoroughly test the application to ensure all functionalities are working as expected. Monitor performance and optimize queries and indexes in MongoDB.

## Application features and structure

The application to be migrated is a Flask blog application with the following features:

- Creating, editing, and viewing blog posts
- Searching for blog posts

## **Setting up MongoDB for a Flask application**

Before integrating MongoDB with a Flask application, the first step is to install MongoDB on your local machine. MongoDB provides detailed [installation guides](https://www.mongodb.com/docs/manual/installation/) for various operating system versions.

## **Configuring MongoDB for Flask**

After installing MongoDB, the next step is to configure it to work on your Flask application. This involves installing the `Flask-PyMongo` package and setting up the connection with your local mongo db server.

[Flask-PyMongo](https://flask-pymongo.readthedocs.io/en/latest/) is a Flask extension that makes it easy to use MongoDB instances in your Flask app.

Install it using pip:

```bash
pip install Flask-PyMongo Flask
```

You can also install Flask if you haven’t already.

### Original SQL-based Flask application

Here’s a brief overview of the files in our [original Flask application](https://github.com/khabdrick/flask-mongo/tree/main/flask-sql):

- **models.py**: Defines the **Post** model using SQLAlchemy with attributes for id, title, content, and date_posted
- **\_\_init\_\_.py**: Initializes the Flask application and SQLAlchemy
- **forms.py**: Contains form definitions using Flask-WTF
- **manage.py**: Manages database migrations using Flask-Script and Flask-Migrate
- **routes.py**: Contains the route definitions and logic for handling requests

What you will do is make a copy of the folder containing the old project. We will use the current file structure and just replace the current code with some new code.

### Update models.py

Go to models.py and paste in the following code:

```python
from datetime import datetime
from bson.objectid import ObjectId
class Post:
    def __init__(self, title, content, date_posted=None, _id=None):
        self.title = title
        self.content = content
        self.date_posted = date_posted if date_posted else datetime.utcnow()
        self._id = _id if _id else ObjectId()

    def to_dict(self):
        return {
            "title": self.title,
            "content": self.content,
            "date_posted": self.date_posted,
            "_id": self._id
        }

    @staticmethod
    def from_dict(data):
        return Post(
            title=data.get('title'),
            content=data.get('content'),
            date_posted=data.get('date_posted'),
            _id=data.get('_id')
        )

    def __repr__(self):
        return f"Post('{self.title}', '{self.date_posted}')"
```

This code defines a `Post` class that models a blog post for use with a MongoDB database, utilizing PyMongo's BSON types. The class includes an initializer (`__init__` method) that sets the *title*, *content*, *date posted*, and a unique identifier for each post. If the date posted or identifier is not provided during instantiation, it defaults to the current UTC time and a new `ObjectId`, respectively. The class also provides a `to_dict` method to convert an instance into a dictionary format suitable for MongoDB storage, and a `from_dict` static method to create a Post instance from a dictionary. Additionally, the `__repr__` method is defined to offer a clear string representation of the post, useful for debugging and logging.

The use of this is for the handling of blog post data within a Flask web application that uses MongoDB for its database. Converting the `Post` instances to and from dictionary format allows easy interaction with MongoDB's document-oriented storage.

### Update \_\_init\_\_.py

Paste the following code into your *\_\_init\_\_.py* file. The Flask application is initialized to use MongoDB through the Flask-PyMongo extension. The PyMongo instance (mongo) is set up with the Flask application which enables MongoDB support.  The application routes are imported to define the URL endpoints and their associated handlers.

```python
from flask import Flask
from flask_pymongo import PyMongo
from config import Config


app = Flask(__name__)
app.config.from_object(Config)
mongo = PyMongo(app)


from app import routes
```

Additionally, update the `Config` class in your config.py to include MongoDB settings:

```python
import os
class Config:
    SECRET_KEY = os.urandom(24)
    MONGO_URI = 'mongodb://localhost:27017/blogdb'
```

### Update manage.py

*manage.py* is used for database migrations in SQL, which isn't directly applicable to MongoDB so you can delete the file. If you need migrations, the [Beanie ODM](https://beanie-odm.dev/) provides some support for migrations with MongoDB.

### Update routes.py

```python
from flask import render_template, url_for, flash, redirect, request
from app import app, mongo
from bson.objectid import ObjectId
from app.models import Post
from app.forms import PostForm

@app.route("/")
@app.route("/home")
def home():
    """
    Render the home page with a list of all posts.
    """
    posts_data = mongo.db.posts.find()
    posts = [Post.from_dict(post) for post in posts_data]
    return render_template("index.html", posts=posts)


@app.route("/post/new", methods=["GET", "POST"])
def new_post():
    """
    Handle the creation of a new post. Render the new post form and process the form submission.

    """
    form = PostForm()
    if form.validate_on_submit():
        post = Post(title=form.title.data, content=form.content.data)
        mongo.db.posts.insert_one(post.to_dict())
        flash("Your post has been created!", "success")
        return redirect(url_for("home"))
    return render_template("post.html", title="New Post", form=form)


@app.route("/post/<post_id>")
def post(post_id):
    """
    Display a single post by its ID.
    """
    post_data = mongo.db.posts.find_one_or_404({"_id": ObjectId(post_id)})
    post = Post.from_dict(post_data)
    return render_template("detail.html", post=post)


@app.route("/post/<post_id>/edit", methods=["GET", "POST"])
def edit_post(post_id):
    """
    Handle the editing of an existing post. Render the edit post form and process the form submission.
    """
    post_data = mongo.db.posts.find_one_or_404({"_id": ObjectId(post_id)})
    post = Post.from_dict(post_data)
    form = PostForm()
    if form.validate_on_submit():
        updated_post = {
            "$set": {"title": form.title.data, "content": form.content.data}
        }
        mongo.db.posts.update_one({"_id": ObjectId(post_id)}, updated_post)
        flash("Your post has been updated!", "success")
        return redirect(url_for("post", post_id=post_id))
    elif request.method == "GET":
        form.title.data = post.title
        form.content.data = post.content
    return render_template("edit_post.html", title="Edit Post", form=form)


@app.route("/post/<post_id>/delete", methods=["POST"])
def delete_post(post_id):
    """
    Handle the deletion of a post by its ID.
    """
    mongo.db.posts.delete_one({"_id": ObjectId(post_id)})
    flash("Your post has been deleted!", "success")
    return redirect(url_for("home"))


@app.route("/search", methods=["GET", "POST"])
def search():
    """
    Handle the search functionality for posts. Search for posts by title or content.
    """
    query = request.args.get("query")
    posts_data = mongo.db.posts.find(
        {
            "$or": [
                {"title": {"$regex": query, "$options": "i"}},
                {"content": {"$regex": query, "$options": "i"}},
            ]
        }
    )
    posts = [Post.from_dict(post) for post in posts_data]
    return render_template("index.html", posts=posts)
```

To make the Flask application compatible with MongoDB, several modifications were made to the routes to interact with MongoDB instead of SQLAlchemy. Firstly, the Post class instances are now converted to dictionaries using the to_dict method for storage in MongoDB. The home route retrieves all posts from the MongoDB collection using `mongo.db.posts.find()`, converts them to Post instances using the `from_dict` method, and then renders them.

For creating new posts in the new_post route, the post data is inserted into MongoDB with `mongo.db.posts.insert_one(post.to_dict())`. For retrieving, editing, and deleting posts, routes, post, `edit_post`, and `delete_post` use MongoDB's query methods (`find_one_or_404`, `update_one`, and `delete_one`) with `ObjectId` to handle the unique identifiers. The search route performs a text search on titles and content using MongoDB's regex capabilities, ensuring posts matching the query are fetched and rendered.

## **Migrating the data**

The migration process involves exporting data from the existing SQL database and importing it into MongoDB. We will start by exporting the data from SQLite. On GitHub, you will find [the db file](https://github.com/khabdrick/flask-mongo/blob/main/flask-sql/instance/site.db) we will use in this tutorial.

If you are using my db file, run the following code to query the database to list all tables and determine their names.

```python
import sqlite3


# Connect to the SQLite database
db_path = 'relative/path/to/site.db'
conn = sqlite3.connect(db_path)
cursor = conn.cursor()

# Query to get all table names
cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
tables = cursor.fetchall()

# Close the connection
conn.close()

print(tables)
```

This Python code above connects to your existing SQLite database specified in the path `db_path and `retrieves the names of all tables within it, then closes the connection. The SQL query `SELECT name FROM sqlite_master WHERE type='table';` is executed to fetch the names of all tables in the database. You should have *post* as your output.

After that, run the following code to export the data to a JSON file.

```python
import sqlite3
import json

# Connect to the SQLite database
db_path = 'relative/path/to/site.db'
conn = sqlite3.connect(db_path)
cursor = conn.cursor()

# Export data from the 'post' table
cursor.execute("SELECT * FROM post")
rows = cursor.fetchall()

# Get the column names
column_names = [description[0] for description in cursor.description]

# Convert to list of dictionaries
data = [dict(zip(column_names, row)) for row in rows]

# Save to a JSON file
with open('post.json', 'w') as f:
    json.dump(data, f, indent=4)

# Close the connection
conn.close()
```

This Python code connects to your SQLite database, extracts all data from the *post* table, converts it to a JSON format, and saves it to a file.  A cursor object is created from this connection to execute SQL commands. The SQL query `SELECT * FROM post` is used to fetch all rows from the post table, and the results are stored in the rows variable. The variable `column_names` is used to extract the column names from the cursor's description attribute. Each row is then combined with the column names to create a list of dictionaries, where each dictionary represents a row of the table. And finally, the list is saved as a JSON file named *post.json* using the `json.dump()` function.

After running, you will then be provided with a JSON file (*post.json*) that looks like this:

```json
[
    {
        "id": 1,
        "title": " Superworm (Zophobas morio)",
        "content": "Superworms, scientifically known as Zophobas morio, are the larval stage of a darkling beetle species...",
        "date_posted": "2024-05-14 18:03:43.219562"
    },
    {
        "id": 2,
        "title": "MongoDB article",
        "content": "MongoDB is a widely-used NoSQL database known for its flexibility...",
        "date_posted": "2024-05-14 18:04:57.884181"
    },
    {
        "id": 3,
        "title": "Unique Techniques in Farming",
        "content": "Modern farming has evolved in various ways…",
        "date_posted": "2024-05-14 18:07:12.665001"
    },
    {
        "id": 4,
        "title": " Examples of NoSQL Databases",
        "content": "NoSQL databases have improved the way...",
        "date_posted": "2024-05-14 18:08:28.759601"
    },
    {
        "id": 5,
        "title": "d",
        "content": "d",
        "date_posted": "2024-05-18 18:06:59.767954"
    }
]
```

## Importing data into MongoDB

You can now import this JSON file into your MongoDB database. Here's a script to help you with that:

```python
from pymongo import MongoClient
import json
from datetime import datetime

def import_json_to_mongo(db_name, collection_name, json_path):
    client = MongoClient("mongodb://localhost:27017/")
    db = client[db_name]

    def convert_date(data):
        # Update 'date_posted' field to datetime object
        for item in data:
            if 'date_posted' in item:
                item['date_posted'] = datetime.strptime(item['date_posted'], '%Y-%m-%d %H:%M:%S.%f')
        return data

    with open(json_path, 'r') as f:
        data = json.load(f)
        data = convert_date(data)
        db[collection_name].insert_many(data)

    client.close()

if __name__ == "__main__":
    db_name = 'blogdb'
    collection_name = 'posts'
    json_path = 'flask-mongo/post.json'
    import_json_to_mongo(db_name, collection_name, json_path)
```

The connection string `mongodb://localhost:27017/blogdb` connects to your MongoDB instance running on localhost at port 27017 and uses the "blogdb" database. If the specified database does not exist, MongoDB will create it automatically when you first store data.

## Update the templates

For the templates, you only need to change the variable for *id*. You will change all instances of `post_id=post.id` to `post_id=post._id`.

You can now run the app with the following command:

```bash
python run.py
```

Your application will be running on <http://127.0.0.1:5000/home>.

![the new app with MongoDB][1]

You can find the complete application on [GitHub](https://github.com/khabdrick/flask-mongo).

## **Conclusion**

Migrating a Flask application from SQL to MongoDB involves a deep understanding of the differences between both database systems and then planning to ensure a smooth transition. This article provided a step-by-step guide on how to convert an existing Flask blog application from using SQLAlchemy to using Flask-PyMongo to integrate MongoDB with the Flask application. We focused mainly on covering the necessary changes to each file. In a real-world scenario, there would be a need to construct a document schema from a tabular schema with joins and several queries. Even though this is a simple application, this tutorial covers the necessary steps to have a successful migration.

Want to continue the conversation? Head to the [MongoDB Developer Community](https://www.mongodb.com/community/forums/) next.


  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltbece8af5bc146963/667e8dcfb1132450803a2f21/image1.gif

Learn how to migrate a Flask application from SQLAlchemy to MongoDB.

Muhammed Ali

Content by Muhammed Ali

How to Migrate Your Flask App From SQL To MongoDB