This is the third in a series of blog posts examining the technologies that are driving the development of modern web and mobile applications.
Part 1: Introducing The MEAN Stack (and the young MERN upstart) introduced the technologies making up the MEAN (MongoDB, Express, Angular, Node.js) and MERN (MongoDB, Express, React, Node.js) Stacks, why you might want to use them, and how to combine them to build your web application (or your native mobile or desktop app).
The remainder of the series is focused on working through the end to end steps of building a real (albeit simple) application. – MongoPop. Part 2: Using MongoDB With Node.js created an environment where we could work with a MongoDB database from Node.js; it also created a simplified interface to the MongoDB Node.js Driver.
This post builds on from those first posts by using Express to build a REST API so that a remote client can work with MongoDB. You will be missing a lot of context if you have skipped those posts – it's recommended to follow through those first.
The REST API
A Representational State Transfer (REST) interface provides a set of operations that can be invoked by a remote client (which could be another service) over a network, using the HTTP protocol. The client will typically provide parameters such as a string to search for or the name of a resource to be deleted.
Many services provide a REST API so that clients (their own and those of 3rd parties) and other services can use the service in a well defined, loosely coupled manner. As an example, the Google Places API can be used to search for information about a specific location:
Breaking down the URI used in that
- No method is specified and so the
curl command defaults to a HTTP
maps.googleapis.com is the address of the Google APIs service.
/maps/api/place/details/json is the route path to the specific operation that's being requested.
placeid=ChIJKxSwWSZgAUgR0tWM0zAkZBc is a parameter (passed to the function bound to this route path), identifying which place we want to read the data for.
key=AIzaSyC53qhhXAmPOsxc34WManoorp7SVN_Qezo is a parameter indicating the Google API key, verifying that it's a registered application making the request (Google will also cap, or bill for, the number of requests made using this key).
There's a convention as to which HTTP method should be used for which types of operation:
- GET: Fetches data
- POST: Adds new data
- PUT: Updates data
- DELETE: Removes data
Mongopop's REST API breaks this convention and uses
POST for some read requests (as it's simpler passing arguments than with
These are the REST operations that will be implemented in Express for Mongopop:
Express routes implemented for the Mongopop REST API
|Returns the version of the API.
|Fetches the IP Address of the server running the Mongopop backend.
|Fetches client-side defaults from the back-end config file.
numberDocs batches of documents, using documents fetched from
|Read a sample of the documents from a collection.
|Counts the number of documents in the collection.
|Apply an update to all documents in a collection
which match a given pattern
Express can handle the routing of requests to the right functions within your application (or to different apps running in the same environment).
You can run the app's full business logic within Express and even use an optional view engine to generate the final HTML to be rendered by the user's browser. At the other extreme, Express can be used to simply provide a REST API – giving the front-end app access to the resources it needs e.g., the database.
The Mongopop application uses Express to perform two functions:
- Send the front-end application code to the remote client when the user browses to our app
- Provide a REST API that the front-end can access using HTTP network calls, in order to access the database
Downloading, running, and using the application
The application's Express code is included as part of the Mongopop package installed in Part 2: Using MongoDB With Node.js.
What are all of these files?
A reminder of the files described in Part 2:
package.json: Instructs the Node.js package manager (
npm) on what it needs to do; including which dependency packages should be installed
node_modues: Directory where
npm will install packages
node_modues/mongodb: The MongoDB driver for Node.js
node_modues/mongodb-core: Low-level MongoDB driver library; available for framework developers (application developers should avoid using it directly)
Other files and directories that are relevant to our Express application:
config.js: Contains the application–specific configuration options
bin/www: The script that starts an Express application; this is invoked by the
npm start script within the
package.json file. Starts the HTTP server, pointing it to the
app module in
app.js: Defines the main application module (
- That the application will be run by Express
- Which routes there will be & where they are located in the file system (
- What view engine to use (Jade in this case)
- Where to find the /views/ to be used by the view engine (
- What middleware to use (e.g. to parse the JSON received in requests)
- Where the static files (which can be read by the remote client) are located (
- Error handler for queries sent to an undefined route
views: Directory containing the templates that will be used by the Jade view engine to create the HTML for any pages generated by the Express application (for this application, this is just the error page that's used in cases such as mistyped routes ("404 Page not found"))
routes/pop.js: Contains the Express application for the
/pop route; this is the implementation of the Mongopop REST API. This defines methods for all of the supported route paths.
public: Contains all of the static files that must be accessible by a remote client (e.g., our Angular to React apps). This is not used for the REST API and so can be ignored until Parts 4 and 5.
The rest of the files and directories can be ignored for now – they will be covered in later posts in this series.
The new REST API (implemented in
routes/pop.js) uses the
curl command-line tool to manually test the REST API.
config module can be imported by other parts of the application so that your preferences can be taken into account.
expressPort is used by
bin/www to decide what port the web server should listen on; change this if that port is already in use.
client contains defaults to be used by the client (Angular or React). It's important to create your own schema at Mockaroo.com and replace
client.mockarooUrl with your custom URL (the one included here will fail if used too often).
This is mostly boiler-plate code to start Express with your application. This code ensures that it is our application,
app.js, that is run by the Express server:
This code uses the
config.js as the port for the server to listen on; it will be overruled if the user sets the
PORT environment variable:
This file defines the
app module ; much of the contents are boilerplate (and covered by comments in the code) but we look here at a few of the lines that are particular to this application.
Make this an Express application:
Define where the views (templates used by the Jade view engine to generate the HTML code) and static files (files that must be accessible by a remote client) are located:
/pop route and associate it with the file containing its code (
This file implements each of the operations provided by the Mongopop REST API. Because of the the
/pop route defined in
app.js Express will direct any URL of the form
http://<mongopop-server>:3000/pop/X here. Within this file a route handler is created in order direct incoming requests to
http://<mongopop-server>:3000/pop/X to the appropriate function:
/pop route is only intended for the REST API, end users shouldn't be browsing here but we create a top level handler for the
GET method in case they do:
This is the first time that we see how to send a response to a request;
testObject into a JSON document and sends it back to the requesting client as part of the response message.
The simplest useful route path is for the
GET method on
/pop/ip which sends a response containing the IP address of the back-end server. This is useful to the Mongopop client as it means the user can see it and add it to the MongoDB Atlas whitelist. The code to determine and store
publicIP is left out here but can be found in the full source file for
We've seen that it's possible to test
GET methods from a browser's address bar; that isn't possible for
POST methods and so it's useful to be able to test using the
curl command-line command:
GET method for
/pop/config is just as simple – responding with the client-specific configuration data:
The results of the request are still very simple but the output from
curl is already starting to get messy; piping it through
python -mjson.tool makes it easier to read:
The simplest operation that actually accesses the database is the
POST method for the
/pop/countDocs route path:
database is an instance of the object prototype defined in
- Connect to the database (using the address of the MongoDB server provided in the request body). The results from the promise returned by
database.connect is passed to the function(s) in the first
.then clause. Refer back to Part 2: Using MongoDB With Node.js if you need a recap on using promises.
- The function in the
.then clause handles the case where the
database.connect promise is resolved (success). This function requests a count of the documents – the database connection information is now stored within the
database object and so only the collection name needs to be passed. The promise returned by
database.countDocuments is passed to the next
.then clause. Note that there is no second (error) function provided, and so if the promise from
database.connect is rejected, then that failure passes through to the next
.then clause in the chain.
- The second
.then clause has two functions:
- The first is invoked if and when the promise is resolved (success) and it returns a success response (which is automatically converted into a resolved promise that it passed to the final
.then clause in the chain).
count is the value returned when the promise from the call to
database.countDocuments was resolved.
- The second function handles the failure case (could be from either
database.countDocuments) by returning an error response object (which is converted to a resolved promise).
- The final
.then clause closes the database connection and then sends the HTTP response back to the client; the response is built by converting the
resultObject (which could represent success or failure) to a JSON string.
curl can be used from the command-line to test this operation; as this is a
POST request, the
--data option is used to pass the JSON document to be included in the request:
curl can also be used to test the error paths. Cause the database connection to fail by using the wrong port number in the MongoDB URI:
Cause the count to fail by using the name of a non-existent collection:
POST method for the
pop/sampleDocs route path works in a very similar way:
Testing this new operation:
POST method for
pop/updateDocs is a little more complex as the caller can request multiple update operations be performed. The simplest way to process multiple asynchronous, promise-returning function calls in parallel is to build an array of the tasks and pass it to the
Promise.all method which returns a promise that either resolves after all of the tasks have succeeded or is rejected if any of the tasks fail:
The final method uses example data from a service such as Mockaroo to populate a MongoDB collection. A helper function is created that makes the call to that external service:
That function is then used in the
POST method for
This method is longer than the previous ones – mostly because there are two paths:
- In the first path, the client has requested that a fresh set of 1,000 example documents be used for each pass at adding a batch of documents. This path is much slower and will eat through your Mockaroo quota much faster.
- In the second path, just one batch of 1,000 example documents is fetched from Mockaroo and then those same documents are repeatedly added. This path is faster but it results in duplicate documents (apart from a MongoDB-created
_id field). This path cannot be used if the
_id is part of the example documents generated by Mockaroo.
So far, we've used the Chrome browser and the
curl command-line tool to test the REST API. A third approach is to use the Postman Chrome app:
One way to debug a Node.js application is to liberally sprinkle
console.log messages throughout your code but that takes extra effort and bloats your code base. Every time you want to understand something new, you must add extra logging to your code and then restart your application.
- Add breakpoints
- View & alter contents of variables
- View and modify css styles
- View network messages
- Access the console (view output and issue commands)
- Check security details
- Audit memory use, CPU, etc.
You open the Chrome DevTools within the Chrome browser using "View/Developer/Developer Tools".
Fortunately, you can use the
node-debug command of
node-inspector to get a very similar experience for Node.js back-end applications. To install
node-inspector can be used to debug the Mongopop Express application by starting it with
node-debug via the
express-debug script in
To run the Mongopop REST API with
node-debug, kill the Express app if it's already running and then execute:
Note that this automatically adds a breakpoint at the start of the app and so you will need to skip over that to run the application.
Depending on your version of Node.js, you may see this error:
If you do, apply this patch to
Summary & what's next in the series
Part 1: Introducing The MEAN Stack provided an overview of the technologies that are used by modern application developers – in particular, the MERN and MEAN stacks. Part 2: Using MongoDB With Node.js set up Node.js and the MongoDB Driver and then used them to build a new Node.js module to provide a simplified interface to the database.
This post built upon the first two of the series by stepping through how to implement a REST API using Express. We also looked at three different ways to test this API and how to debug Node.js applications. This REST API is required by both the Angular (Part 4) and React (Part 5) web app clients, as well as by the alternative UIs explored in Part 6.
The next part of this series implements the Angular client that makes use of the REST API – at the end of that post, you will understand the end-to-end steps required to implement an application using the MEAN stack.
Continue to follow this blog series to step through building the remaining stages of the MongoPop application:
If you're interested in learning everything you need to know to get started building a MongoDB-based app you can sign up for one of our free online MongoDB University courses.
Sign up for M101JS: MongoDB for
Node.js Developers today!