Creating an API with the AWS API Lambda and the Atlas Data API
Rate this tutorial
This article will walk through creating an API using the Amazon API Gateway in front of the MongoDB Atlas Data API. When integrating with the Amazon API Gateway, it is possible but undesirable to use a driver, as drivers are designed to be long-lived and maintain connection pooling. Using serverless functions with a driver can result in either a performance hit – if the driver is instantiated on each call and must authenticate – or excessive connection numbers if the underlying mechanism persists between calls, as you have no control over when code containers are reused or created.
TheMongoDB Atlas Data API is an HTTPS-based API that allows us to read and write data in Atlas where a MongoDB driver library is either not available or not desirable. For example, when creating serverless microservices with MongoDB.
AWS (Amazon Web Services) describe their API Gateway as:
"A fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. APIs act as the "front door" for applications to access data, business logic, or functionality from your backend services. Using API Gateway, you can create RESTful APIs and WebSocket APIs that enable real-time two-way communication applications. API Gateway supports containerized and serverless workloads, as well as web applications.
API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management. API Gateway has no minimum fees or startup costs. You pay for the API calls you receive and the amount of data transferred out and, with the API Gateway tiered pricing model, you can reduce your cost as your API usage scales."
A core requirement for this walkthrough is to have an Amazon Web Services account, the API Gateway is available as part of the AWS free tier, allowing up to 1 million API calls per month, at no charge, in your first 12 months with AWS.
A common use of Atlas with the Amazon API Gateway might be to provide a managed API to a restricted subset of data in our cluster, which is a common need for a microservice architecture. To demonstrate this, we first need to have some data available in MongoDB Atlas. This can be added by selecting the three dots next to our cluster name and choosing "Load Sample Dataset", or following instructions here.
The instructions here are an extended variation from Amazon's own "Getting Started with the API Gateway" tutorial. I do not presume to teach you how best to use Amazon's API Gateway as Amazon itself has many fine resources for this, what we will do here is use it to get a basic Public API enabled that uses the Data API.
The Data API itself is currently in an early preview with a flat security model allowing all users who have an API key to query or update any database or collection. Future versions will have more granular security. We would not want to simply expose the current data API as a 'Public' API but we can use it on the back-end to create more restricted and specific access to our data.
We are going to create an API which allows users to GET the ten films for any given year which received the most awards - a notional "Best Films of the Year". We will restrict this API to performing only that operation and supply the year as part of the URL
We will first create the API, then analyze the code we used for it.
- Choose Create function.
- For Function name, enter top-movies-for-year.
- Choose Create function.
When you see the Javascript editor that looks like this
Replace the code with the following, changing the API-KEY and APP-ID to the values for your Atlas cluster. Save and click Deploy (In a production application you might look to store these in AWS Secrets manager , I have simplified by putting them in the code here).
Alternatively, if you are familiar with working with packages and Lambda, you could upload an HTTP package like Axios to Lambda as a zipfile, allowing you to use the following simplified code.
We now need to route an HTTP endpoint to our Lambda function using the HTTP API.
The HTTP API provides an HTTP endpoint for your Lambda function. API Gateway routes requests to your Lambda function, and then returns the function's response to clients.
- Go to the API Gateway console at https://console.aws.amazon.com/apigateway.
- Do one of the following: To create your first API, for HTTP API, choose Build. If you've created an API before, choose Create API, and then choose Build for HTTP API.
- For Integrations, choose Add integration.
- Choose Lambda.
- For Lambda function, enter top-movies-for-year.
- For API name, enter movie-api.
- Choose Next.
- Review the route that API Gateway creates for you, and then choose Next.
- Review the stage that API Gateway creates for you, and then choose Next.
- Choose Create.
Now you've created an HTTP API with a Lambda integration and the Atlas Data API that's ready to receive requests from clients.
You should now be looking at API Gateway details that look like this, if not you can get to it by going tohttps://console.aws.amazon.com/apigatewayand clicking on movie-api
Take a note of the Invoke URL, this is the base URL for your API
Now, in a new browser tab, browse to
<Invoke URL>/top-movies-for-year?year=2001 . Changing <Invoke URL> to the Invoke URL shown in AWS. You should see the results of your API call - JSON listing the top 10 "Best" films of 2001.
We start by importing the Standard node.js https library - the Data API needs no special libraries to call it. We also define our API Key and the path to our find endpoint, You get both of these from the Data API tab in Atlas.
Now we check that the API call included a parameter for year and that it's a number - we need to convert it to a number as in MongoDB, "2001" and 2001 are different values, and searching for one will not find the other. The collection uses a number for the movie release year.
Then we construct our payload - the parameters for the Atlas API Call, we are querying for year = year, projecting just the title and the number of awards, sorting by the numbers of awards descending and limiting to 10.
We then construct the options for the HTTPS POST request to the Data API - here we pass the Data API API-KEY as a header.
Finally we use some fairly standard code to call the API and handle errors. We can get Request errors - such as being unable to contact the server - or Response errors where we get any Response code other than 200 OK - In both cases we return a 500 Internal error from our simplified API to not leak any details of the internals to a potential hacker.
Our Axios verison is just the same functionality as above but simplified by the use of a library.
As we can see, calling the Atlas Data API from AWS Lambda function is incredibly simple, especially if making use of a library like Axios. The Data API is also stateless, so there are no concerns about connection setup times or maintaining long lived connections as there would be using a Driver.