How to Use PyMongo to Connect MongoDB Atlas with AWS Lambda
Rate this tutorial
Armed with AWS Lambda’s pay-per-execution structure and MongoDB Atlas’ unparalleled scalability, developers will truly understand what it means for their applications to thrive without the hardware limitations they might be used to.
Please ensure that the cloud provider picked is AWS. It’s also necessary to pick a secure username and password so that we will have the proper authorization later on in this tutorial, along with proper IP address access.
Once your cluster is up and running, click the ellipses next to the Browse Collections button and download the
sample dataset. Your finished cluster will look like this:
Once our cluster is provisioned, let’s set up our AWS Lambda function.
Sign into your AWS account and search for “Lambda” in the search bar. Hit the orange “Create function” button at the top right side of the screen, and you’ll be taken to the image below. Here, make sure to first select the “Author from scratch” option. Then, we want to select a name for our function (AWSLambdaDemo), the runtime (3.8), and our architecture (x86_64).
Hit the orange “Create function” button on the bottom right to continue. Once your function is created, you’ll see a page with your function overview above and your code source right below.
Now, we are ready to set up our connection from AWS Lambda to our MongoDB cluster.
To make things easier for ourselves because we are going to be using Pymongo, a dependency, instead of editing directly in the code source, we will be using Visual Studio Code. AWS Lambda has a limited amount of pre-installed libraries and dependencies, so in order to get around this and incorporate Pymongo, we will need to package our code in a special way. Due to this “workaround,” this will not be a typical tutorial with testing at every step. We will first have to download our dependencies and upload our code to Lambda prior to ensuring our code works instead of using a typical
requirements.txtfile. More on that below.
Now we are ready to establish a connection between AWS Lambda and our MongoDB cluster!
Create a new directory on your local machine and name it
Create a new directory where your dependencies will live:
pymongodirectly in your
Open Visual Studio Code, open the
awslambda-demodirectory, and create a new Python file named
lambda_function.py. This is where the heart of our connection will be.
Insert the code below in our
lambda_function.py. Here, we are setting up our console to check that we are able to connect to our Atlas cluster. Please keep in mind that since we are incorporating our environment variables in a later step, you will not be able to connect just yet. We have copied the
lambda_handlerdefinition from our Lambda code source and have edited it to insert one document stating my full name into a new “test” database and “test” collection. It is best practice to incorporate our MongoClient outside of our
lambda_handlerbecause to establish a connection and performing authentication is reactively expensive, and Lambda will re-use this instance.
If this is properly inserted in AWS Lambda, we will see “Document inserted successfully” and in MongoDB Atlas, we will see the creation of our “test” database and collection along with the single document holding the name “Anaiya Raisinghani.” Please keep in mind we will not be seeing this yet since we haven’t configured our environment variables and will be doing this a couple steps down.
Now, we need to create a .zip file, so we can upload it in our Lambda function and execute our code. Create a .zip file at the root:
This creates a
deployment.zipfile in your project directory.
Now, we need to add in our
lambda_function.pyfile to the root of our .zip file:
Once you have your .zip file, access your AWS Lambda function screen, click the “Upload from” button, and select “.zip file” on the right hand side of the page:
Upload your .zip file and you should see the code from your
lambda_function.pyin your “Code Source”:
Once you have your Environment Variables in place, we are ready to run our code and see if our connection works. Hit the “Test” button. If it’s the first time you’re hitting it, you’ll need to name your event. Keep everything else on the default settings. You should see this page with our “Execution results.” Our document has been inserted!
When we double-check in Atlas, we can see that our new database “test” and collection “test” have been created, along with our document with “Anaiya Raisinghani.”
For our pipeline, let’s change our code to connect to our
restaurantscollection. We are going to be incorporating our aggregation pipeline to find a sample size of five American cuisine restaurants that are located in Brooklyn, New York. Let’s dive right in!
Since we have our
pymongodependency downloaded, we can directly incorporate our aggregation pipeline into our code source. Change your
lambda_function.pyto look like this:
Here, we are using
$matchto find all the American cuisine restaurants located in Brooklyn. We are then using
$limitto only five documents out of our database. Next, we are using
$projectto only show the fields we want. We are going to include “borough”, “cuisine”, and the “name” of the restaurant. Then, we are executing our pipeline and printing out our results.
Click on “Deploy” to ensure our changes have been deployed to the code environment. After the changes are deployed, hit “Test.” We will get a sample size of five Brooklyn American restaurants as the result in our console:
Our aggregation pipeline was successful!
This tutorial provided you with hands-on experience to connect a MongoDB Atlas database to AWS Lambda. We also got an inside look on how to write to a cluster from Lambda, how to read back information from an aggregation pipeline, and how to properly configure our dependencies when using Lambda. Hopefully now, you are ready to take advantage of AWS Lambda and MongoDB to create the best applications without worrying about external infrastructure.