Interested in speaking at MongoDB World 2022? Click here to become a speaker.
HomeLearnQuickstartCreating an API with the AWS API Gateway and the Atlas Data API

Creating an API with the AWS API Gateway and the Atlas Data API

Updated: Jan 11, 2022 |

Published: Dec 15, 2021

By John Page

Rate this article

#Introduction

This article will walk through creating an API using the Amazon API Gateway in front of the MongoDB Atlas Data API. When integrating with the Amazon API Gateway, it is possible but undesirable to use a driver, as drivers are designed to be long-lived and maintain connection pooling. Using serverless functions with a driver can result in either a performance hit – if the driver is instantiated on each call and must authenticate – or excessive connection numbers if the underlying mechanism persists between calls, as you have no control over when code containers are reused or created.

TheMongoDB Atlas Data API is an HTTPS-based API that allows us to read and write data in Atlas where a MongoDB driver library is either not available or not desirable. For example, when creating serverless microservices with MongoDB.

AWS (Amazon Web Services) describe their API Gateway as:

"A fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. APIs act as the "front door" for applications to access data, business logic, or functionality from your backend services. Using API Gateway, you can create RESTful APIs and WebSocket APIs that enable real-time two-way communication applications. API Gateway supports containerized and serverless workloads, as well as web applications. API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management. API Gateway has no minimum fees or startup costs. You pay for the API calls you receive and the amount of data transferred out and, with the API Gateway tiered pricing model, you can reduce your cost as your API usage scales."

#Prerequisites.

A core requirement for this walkthrough is to have an Amazon Web Services account, the API Gateway is available as part of the AWS free tier, allowing up to 1 million API calls per month, at no charge, in your first 12 months with AWS.

We will also need an Atlas Cluster for which we have enabled the Data API – and our endpoint URL and API Key. You can learn how to get these in this Article or this Video if you do not have them already.

A common use of Atlas with the Amazon API Gateway might be to provide a managed API to a restricted subset of data in our cluster, which is a common need for a microservice architecture. To demonstrate this, we first need to have some data available in MongoDB Atlas. This can be added by selecting the three dots next to our cluster name and choosing "Load Sample Dataset", or following instructions here. https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Screenshot_2021_11_29_at_09_50_02_6532382c89.png

#Creating an API with the Amazon API Gateway and the Atlas Data API

#

The instructions here are an extended variation from Amazon's own "Getting Started with the API Gateway" tutorial. I do not presume to teach you how best to use Amazon's API Gateway as Amazon itself has many fine resources for this, what we will do here is use it to get a basic Public API enabled that uses the Data API.

The Data API itself is currently in an early preview with a flat security model allowing all users who have an API key to query or update any database or collection. Future versions will have more granular security. We would not want to simply expose the current data API as a 'Public' API but we can use it on the back-end to create more restricted and specific access to our data.

We are going to create an API which allows users to GET the ten films for any given year which received the most awards - a notional "Best Films of the Year". We will restrict this API to performing only that operation and supply the year as part of the URL

We will first create the API, then analyze the code we used for it.

#Create a AWS Lambda Function to retrieve data with the Data API

  1. Sign in to the Lambda console athttps://console.aws.amazon.com/lambda.
  2. Choose Create function.
  3. For Function name, enter top-movies-for-year.
  4. Choose Create function.

https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Screenshot_2021_12_07_at_15_24_30_c38ba7b6cb.png

When you see the Javascript editor that looks like this

https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Screenshot_2021_12_07_at_15_25_59_2ba3959057.png Replace the code with the following, changing the API-KEY and APP-ID to the values for your Atlas cluster. Save and click Deploy (In a production application you might look to store these in AWS Secrets manager , I have simplified by putting them in the code here).

1const https = require('https');
2
3const atlasEndpoint = "/app/APP-ID/endpoint/data/beta/action/find";
4const atlasAPIKey = "API-KEY";
5
6
7exports.handler = async(event) => {
8
9 if (!event.queryStringParameters || !event.queryStringParameters.year) {
10 return { statusCode: 400, body: 'Year not specified' };
11 }
12
13 //Year is a number but the argument is a string so we need to convert as MongoDB is typed
14
15
16 let year = parseInt(event.queryStringParameters.year, 10);
17 console.log(`Year = ${year}`)
18 if (Number.isNaN(year)) { return { statusCode: 400, body: 'Year incorrectly specified' }; }
19
20
21 const payload = JSON.stringify({
22 dataSource: "Cluster0",
23 database: "sample_mflix",
24 collection: "movies",
25 filter: { year },
26 projection: { _id: 0, title: 1, awards: "$awards.wins" },
27 sort: { "awards.wins": -1 },
28 limit: 10
29 });
30
31
32 const options = {
33 hostname: 'data.mongodb-api.com',
34 port: 443,
35 path: atlasEndpoint,
36 method: 'POST',
37 headers: {
38 'Content-Type': 'application/json',
39 'Content-Length': payload.length,
40 'api-key': atlasAPIKey
41 }
42 };
43
44 let results = '';
45
46 const response = await new Promise((resolve, reject) => {
47 const req = https.request(options, res => {
48 res.on('data', d => {
49 results += d;
50 });
51 res.on('end', () => {
52 console.log(`end() status code = ${res.statusCode}`);
53 if (res.statusCode == 200) {
54 let resultsObj = JSON.parse(results)
55 resolve({ statusCode: 200, body: JSON.stringify(resultsObj.documents, null, 4) });
56 }
57 else {
58 reject({ statusCode: 500, body: 'Your request could not be completed, Sorry' }); //Backend Problem like 404 or wrong API key
59 }
60 });
61 });
62 //Do not give the user clues about backend issues for security reasons
63 req.on('error', error => {
64 reject({ statusCode: 500, body: 'Your request could not be completed, Sorry' }); //Issue like host unavailable
65 });
66
67 req.write(payload);
68 req.end();
69 });
70 return response;
71
72};

Alternatively, if you are familiar with working with packages and Lambda, you could upload an HTTP package like Axios to Lambda as a zipfile, allowing you to use the following simplified code.

1const axios = require('axios');
2
3const atlasEndpoint = "https://data.mongodb-api.com/app/APP-ID/endpoint/data/beta/action/find";
4const atlasAPIKey = "API-KEY";
5
6
7exports.handler = async(event) => {
8
9 if (!event.queryStringParameters || !event.queryStringParameters.year) {
10 return { statusCode: 400, body: 'Year not specified' };
11 }
12
13 //Year is a number but the argument is a string so we need to convert as MongoDB is typed
14
15
16 let year = parseInt(event.queryStringParameters.year, 10);
17 console.log(`Year = ${year}`)
18 if (Number.isNaN(year)) { return { statusCode: 400, body: 'Year incorrectly specified' }; }
19
20
21 const payload = {
22 dataSource: "Cluster0",
23 database: "sample_mflix",
24 collection: "movies",
25 filter: { year },
26 projection: { _id: 0, title: 1, awards: "$awards.wins" },
27 sort: { "awards.wins": -1 },
28 limit: 10
29 };
30
31 try {
32 const response = await axios.post(atlasEndpoint, payload, { headers: { 'api-key': atlasAPIKey } });
33 return response.data.documents;
34 }
35 catch (e) {
36 return { statusCode: 500, body: 'Unable to service request' }
37 }
38};

#Create an HTTP endpoint for our custom API function

#

We now need to route an HTTP endpoint to our Lambda function using the HTTP API.

The HTTP API provides an HTTP endpoint for your Lambda function. API Gateway routes requests to your Lambda function, and then returns the function's response to clients.

  1. Go to the API Gateway console athttps://console.aws.amazon.com/apigateway.
  2. Do one of the following:

To create your first API, for HTTP API, choose Build. If you've created an API before, choose Create API, and then choose Build for HTTP API. 3. For Integrations, choose Add integration. 4. Choose Lambda. 5. For Lambda function, enter top-movies-for-year. 6. For API name, enter movie-api. https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Screenshot_2021_12_07_at_15_38_55_cfcd82c333.png 8. Choose Next.

  1. Review the route that API Gateway creates for you, and then choose Next.

https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Screenshot_2021_12_07_at_15_42_20_23bd84911f.png 9. Review the stage that API Gateway creates for you, and then choose Next. https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Screenshot_2021_12_07_at_15_43_43_ebab329b15.png 10. Choose Create.

Now you've created an HTTP API with a Lambda integration and the Atlas Data API that's ready to receive requests from clients.

#Test your API

You should now be looking at API Gateway details that look like this, if not you can get to it by going tohttps://console.aws.amazon.com/apigatewayand clicking on movie-api

https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Screenshot_2021_12_07_at_15_46_17_06301a223e.png

Take a note of the Invoke URL, this is the base URL for your API

Now, in a new browser tab, browse to <Invoke URL>/top-movies-for-year?year=2001 . Changing <Invoke URL> to the Invoke URL shown in AWS. You should see the results of your API call - JSON listing the top 10 "Best" films of 2001.

https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/unnamed_33c3be7d0f.png

#Reviewing our Function.

#

We start by importing the Standard node.js https library - the Data API needs no special libraries to call it. We also define our API Key and the path to our find endpoint, You get both of these from the Data API tab in Atlas.

1const https = require('https');
2
3const atlasEndpoint = "/app/data-amzuu/endpoint/data/beta/action/find";
4const atlasAPIKey = "YOUR-API-KEY";

Now we check that the API call included a parameter for year and that it's a number - we need to convert it to a number as in MongoDB, "2001" and 2001 are different values, and searching for one will not find the other. The collection uses a number for the movie release year.

1exports.handler = async (event) => {
2
3 if (!event.queryStringParameters || !event.queryStringParameters.year) {
4 return { statusCode: 400, body: 'Year not specified' };
5 }
6 //Year is a number but the argument is a string so we need to convert as MongoDB is typed
7 let year = parseInt(event.queryStringParameters.year, 10);
8 console.log(`Year = ${year}`)
9 if (Number.isNaN(year)) { return { statusCode: 400, body: 'Year incorrectly specified' }; }
10
11
12 const payload = JSON.stringify({
13 dataSource: "Cluster0", database: "sample_mflix", collection: "movies",
14 filter: { year }, projection: { _id: 0, title: 1, awards: "$awards.wins" }, sort: { "awards.wins": -1 }, limit: 10
15 });

THen we construct our payload - the parameters for the Atlas API Call, we are querying for year = year, projecting just the title and the number of awards, sorting by the numbers of awards descending and limiting to 10.

1 const payload = JSON.stringify({
2 dataSource: "Cluster0", database: "sample_mflix", collection: "movies",
3 filter: { year }, projection: { _id: 0, title: 1, awards: "$awards.wins" },
4 sort: { "awards.wins": -1 }, limit: 10
5 });

We then construct the options for the HTTPS POST request to the Data API - here we pass the Data API API-KEY as a header.

1 const options = {
2 hostname: 'data.mongodb-api.com',
3 port: 443,
4 path: atlasEndpoint,
5 method: 'POST',
6 headers: {
7 'Content-Type': 'application/json',
8 'Content-Length': payload.length,
9 'api-key': atlasAPIKey
10 }
11 };

Finally we use some fairly standard code to call the API and handle errors. We can get Request errors - such as being unable to contact the server - or Response errors where we get any Response code other than 200 OK - In both cases we return a 500 Internal error from our simplified API to not leak any details of the internals to a potential hacker.

1 let results = '';
2
3 const response = await new Promise((resolve, reject) => {
4 const req = https.request(options, res => {
5 res.on('data', d => {
6 results += d;
7 });
8 res.on('end', () => {
9 console.log(`end() status code = ${res.statusCode}`);
10 if (res.statusCode == 200) {
11 let resultsObj = JSON.parse(results)
12 resolve({ statusCode: 200, body: JSON.stringify(resultsObj.documents, null, 4) });
13 } else {
14 reject({ statusCode: 500, body: 'Your request could not be completed, Sorry' }); //Backend Problem like 404 or wrong API key
15 }
16 });
17 });
18 //Do not give the user clues about backend issues for security reasons
19 req.on('error', error => {
20 reject({ statusCode: 500, body: 'Your request could not be completed, Sorry' }); //Issue like host unavailable
21 });
22
23 req.write(payload);
24 req.end();
25 });
26 return response;
27
28};

Our Axios verison is just the same functionality as above but simplified by the use of a library.

#Conclusion

As we can see, calling the Atlas Data API from AWS Lambda function is incredibly simple, especially if making use of a library like Axios. The Data API is also stateless, so there are no concerns about connection setup times or maintaining long lived connections as there would be using a Driver.

Rate this article
MongoDB logo
© 2021 MongoDB, Inc.

About

  • Careers
  • Investor Relations
  • Legal Notices
  • Privacy Notices
  • Security Information
  • Trust Center
© 2021 MongoDB, Inc.