Tobias Quante

## About Ghost

[Ghost](https://ghost.org/) is an open-source blogging platform. Unlike other content management systems like WordPress, its focus lies on professional publishing.

This ensures the core of the system remains lean. To integrate third-party applications, you don't even need to install plugins. Instead, Ghost offers a feature called [Webhooks](https://ghost.org/docs/webhooks/) which runs while you work on your publication.

These webhooks send particular items, such as a [post](https://ghost.org/docs/publishing/) or a [page](https://ghost.org/docs/themes/contexts/page/), to an HTTP endpoint defined by you and thereby provide an excellent base for our real-time service.

## Server-sent events

You are likely familiar with the concept of [an HTTP session](https://developer.mozilla.org/en-US/docs/Web/HTTP/Session). A client sends a request, the server responds and then closes the connection. When using server-sent events (SSEs), said connection remains open. This allows the server to continue writing messages into the response.

![A default HTTP lifecycle consists of a client request and a server response](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt189f6f60c53c4516/64daa9d91abd0f66b695a444/image2.png)

Like [Websockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API) (WS), apps and websites use SSEs for real-time communication. Where WSs use a dedicated protocol and work in both directions, SSEs are unidirectional. They use plain HTTP endpoints to write a message whenever an event occurs on the server side.

![Server Sent events are written into an already existing connection response](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt526d1df07c94e9f4/64daaa10ab46d299692f2409/image3.png)

Clients can subscribe to these endpoints using the [EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) browser API:

```javascript
const subscription = new EventSource("https://example.io/subscribe")
```

## MongoDB Change Streams

Now that we’ve looked at the periphery of our application, it's time to present its core. We'll use MongoDB to store a subset of the received Ghost Webhook data. On top of that, we'll use [MongoDB Change Streams](https://www.mongodb.com/basics/change-streams) to watch our webhook collection.

In a nutshell, Change Streams register data flowing into our database. We can subscribe to this data stream and react to it. Reacting means sending out SSE messages to connected clients whenever a new webhook is received and stored.

The following Javascript code showcases a simple Change Stream subscription.

```javascript
import {MongoClient} from 'mongodb';

const client = new MongoClient("<mongodb-url>");
const ghostDb = client.db('ghost');
const ghostCollection = ghostDb.collection('webhooks');
const ghostChangeStrem = ghostCollection.watch();

ghostChangeStream.on('change', document => {
    /* document is the MongoDB collection entry, e.g. our webhook */
});
```

Its event-based nature matches perfectly with webhooks and SSEs. We can react to newly received webhooks where the data is created, ensuring data integrity over our whole application.

## Build a real-time endpoint

We need an extra application layer to propagate these changes to connected clients. I've decided to use Typescript and Express.js, but you can use any other server-side framework. You will also need a dedicated MongoDB instance*. For a quick start, you can sign up for [MongoDB Atlas](https://www.mongodb.com/atlas). Then, [create a free cluster](https://www.mongodb.com/docs/atlas/tutorial/create-atlas-account/) and [connect to it](https://www.mongodb.com/docs/atlas/tutorial/connect-to-your-cluster/).

Let's get started by cloning the `1-get-started` branch from this Github repository:

```bash
# ssh
$ git clone git@github.com:tq-bit/mongodb-article-mongo-changestreams.git

# HTTP(s)
$ git clone https://github.com/tq-bit/mongodb-article-mongo-changestreams.git

# Change to the starting branch
$ git checkout 1-get-started

# Install NPM dependencies
$ npm install

# Make a copy of .env.example
$ cp .env.example .env
```

> Make sure to fill out the MONGO_HOST environment variable with your connection string!

Express and the database client are already implemented. So in the following, we'll focus on adding MongoDB change streams and server-sent events.

Once everything is set up, you can start the server on `http://localhost:3000` by typing

```bash
npm run dev
```

The application uses two important endpoints which we will extend in the next sections: 

- `/api/notification/subscribe` <- Used by EventSource to receive event messages
- `/api/notification/article/create` <- Used as a webhook target by Ghost


\* If you are not using MongoDB Atlas, make sure to have [Replication Sets](https://www.mongodb.com/docs/manual/tutorial/deploy-replica-set/) enabled.

## Add server-sent events

Open the cloned project in your favorite code editor. We'll add our SSE logic under `src/components/notification/notification.listener.ts`.

In a nutshell, implementing SSE requires three steps:

- Write out an HTTP status 200 header.
- Write out an opening message.
- Add event-based response message handlers.

We’ll start sending a static message and revisit this module after adding ChangeStreams.

> You can also `git checkout 2-add-sse` to see the final result.

### Write the HTTP header

Writing the HTTP header informs clients of a successful connection. It also propagates the response's content type and makes sure events are not cached.

Add the following code to the function `subscribeToArticleNotification` inside:

```javascript
// Replace
// TODO: Add function to write the head
// with
console.log('Step 1: Write the response head and keep the connection open');
res.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    Connection: 'keep-alive'
});
```

### Write an opening message

The first message sent should have an event type of 'open'. It is not mandatory but helps to determine whether the subscription was successful.

Append the following code to the function `subscribeToArticleNotification`:

```javascript
// Replace
// TODO: Add functionality to write the opening message
// with
console.log('Step 2: Write the opening event message');
res.write('event: open\n');
res.write('data: Connection opened!\n'); // Data can be any string
res.write(`id: ${crypto.randomUUID()}\n\n`);
```

### Add response message handlers

We can customize the content and timing of all further messages sent. Let's add a placeholder function that sends messages out every five seconds for now. And while we’re at it, let’s also add a handler to close the client connection:

Append the following code to the function `subscribeToArticleNotification`:

```javascript
setInterval(() => {
    console.log('Step 3: Send a message every five seconds');
    res.write(`event: message\n`);
    res.write(`data: ${JSON.stringify({ message: 'Five seconds have passed' })}\n`);
    res.write(`id: ${crypto.randomUUID()}\n\n`);
}, 5000);


// Step 4: Handle request events such as client disconnect
// Clean up the Change Stream connection and close the connection stream to the client
req.on('close', () => {
    console.log('Step 4: Handle request events such as client disconnect');
    res.end();
});
```

To check if everything works, visit `http://localhost:3000/api/notification/subscribe`.

![server sent event data in a browser screen](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt2724881e861bb1c8/64daaa538cd995c129389110/image1.png)

## Add a POST endpoint for Ghost

Let's visit `src/components/notification/notification.model.ts` next. We'll add a simple `insert` command for our database into the function `createNotificiation`:

> You can also `git checkout 3-webhook-handler` to see the final result.

```javascript
// Replace
// TODO: Add insert one functionality for DB
// with
return notificationCollection.insertOne(notification);
```

And on to `src/components/notification/notification.controller.ts`. To process incoming webhooks, we'll add a handler function into `handleArticleCreationNotification`:

```javascript
// Replace
// TODO: ADD handleArticleCreationNotification
// with
const incomingWebhook: GhostWebhook = req.body;
await NotificationModel.createNotificiation({
    id: crypto.randomUUID(),
    ghostId: incomingWebhook.post?.current?.id,
    ghostOriginalUrl: incomingWebhook.post?.current?.url,
    ghostTitle: incomingWebhook.post?.current?.title,
    ghostVisibility: incomingWebhook.post?.current?.visibility,
    type: NotificationEventType.PostPublished,
});

res.status(200).send('OK');
```

This handler will pick data from the incoming webhook and insert a new notification.

```bash
curl -X POST -d '{
	"post": {
		"current": {
            "id": "sj7dj-lnhd1-kabah9-107gh-6hypo",
            "url": "http://localhost:2368/how-to-create-realtime-notifications",
            "title": "How to create realtime notifications",
            "visibility": "public"
		}
	}
}' http://localhost:3000/api/notification/article/create
```

You can also test the insert functionality by using Postman or [VSCode REST client](https://marketplace.visualstudio.com/items?itemName=humao.rest-client) and then check your MongoDB collection. There is an example request under `/test/notification.rest` in the project's directory, for your convenience.

## Trigger MongoDB Change Streams

So far, we can send SSEs and insert Ghost notifications. Let's put these two features together now.

Earlier, we added a static server message sent every five seconds. Let's revisit `src/components/notification/notification.listener.ts` and make it more dynamic.

First, let's get rid of the whole `setInterval` and its callback. Instead, we'll use our `notificationCollection` and its built-in method `watch`. This method returns a `ChangeStream`.

You can create a change stream by adding the following code above the `export default` code segment:

```javascript
const notificationStream = notificationCollection.watch();
```

The stream fires an event whenever its related collection changes. This includes the `insert` event from the previous section.

We can register callback functions for each of these. The event that fires when a document inside the collection changes is 'change':

```javascript
notificationStream.on('change', (next) => {
    console.log('Step 3.1: Change in Database detected!');
});
```

The variable passed into the callback function is a change stream document. It includes two important information for us:

- The document that's inserted, updated, or deleted.
- The type of operation on the collection.

Let's assign them to one variable each inside the callback:

```javascript
notificationStream.on('change', (next) => {
    // ... previous code
    const {
        // @ts-ignore, fullDocument is not part of the next type (yet)
        fullDocument /* The newly inserted fullDocument */,
        operationType /* The MongoDB operation Type, e.g. insert */,
    } = next;
});
```

Let's write the notification to the client. We can do this by repeating the method we used for the opening message.

```javascript
notificationStream.on('change', (next) => {
    // ... previous code
    console.log('Step 3.2: Writing out response to connected clients');
    res.write(`event: ${operationType}\n`);
    res.write(`data: ${JSON.stringify(fullDocument)}\n`);
    res.write(`id: ${crypto.randomUUID()}\n\n`);
});
```

And that's it! You can test if everything is functional by:

1. Opening your browser under `http://localhost:3000/api/notification/subscribe`.
2. Using the file under `test/notification.rest` with VSCode's HTTP client.
3. Checking if your browser includes an opening and a Ghost Notification.

![server sent event data in a browser screen after fully implementing change streams](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt28fad804140cd0cc/64daaa965572446fc6e22d3c/image6.png)

For an [HTTP webhook implementation](https://ghost.org/docs/webhooks/), you will need a running Ghost instance. I have added a [dockerfile](https://github.com/tq-bit/mongodb-article-mongo-changestreams/blob/master/docker-compose.yml) to this repo for your convenience. [You could also install Ghost yourself locally](https://ghost.org/docs/install/local/).

To start Ghost with the dockerfile, make sure you have [Docker Engine](https://docs.docker.com/engine/install/) or [Docker Desktop](https://docs.docker.com/desktop/) with support for `docker compose` installed.

For a local installation and the first-time setup, you should follow the [official Ghost installation guide](https://ghost.org/docs/install/).

After your Ghost instance is up and running, open your browser at `http://localhost:2368/ghost`. You can set up your site however you like, give it a name, enter details, and so on.

![Configure the Ghost deployment](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltfe21f1c4ad80772e/64daac4c2b55eb944840c8e4/gif1.gif)

In order to create a webhook, you must first create a custom integration. To do so, navigate into your site’s settings and click on the “Integrations” menu point. Click on “Add Webhook,” enter a name, and click on “Create.”

![Ghost third-party custom integrations](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt12432f5aed316b08/64daacb28ef021688773fbf1/gif2.gif)

Inside the newly created integration, you can now configure a webhook to point at your application under `http://<host>:<port>/api/notification/article/create`*.

\* This URL might vary based on your local Ghost setup. For example, if you run Ghost in a container, you can find your machine's local IP using the terminal and `ifconfig` on Linux or `ipconfig` on Windows.

![Local ghost-realtime application](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltca1701eede410cc3/64daad17ab46d294592f241b/gif3.gif)

And that’s it. Now, whenever a post is published, its contents will be sent to our real-time endpoint. After being inserted into MongoDB, an event message will be sent to all connected clients.

## Subscribe to Change Streams from your Ghost theme

There are a few ways to add real-time notifications to your Ghost theme. Going into detail is beyond the scope of this article. I have prepared two files, a `plugin.js` and a `plugin.css` file [you can inject into the default Casper theme](https://ghost.org/tutorials/use-code-injection-in-ghost/).

Try this out by starting a local Ghost instance using the provided dockerfile.

You must then instruct your application to serve the JS and CSS assets. Add the following to your `index.ts` file:

```javascript
// ... other app.use hooks
app.use(express.static('public'));
// ... app.listen()
```

Finally, navigate to [Code Injection](http://localhost:2368/ghost/#/settings/code-injection) and add the following two entries in the 'Site Header':

```html
<script src="http://localhost:3000/js/plugin.js"></script>
<link href="http://localhost:3000/css/plugin.css" rel="stylesheet">
```

> The core piece of the plugin is the [EventSource browser API](https://developer.mozilla.org/en-US/docs/Web/API/EventSource). You will want to use it when integrating this application with other themes.

When going back into your Ghost publication, you should now see a small bell icon on the upper right side.

![subscription notifications in Ghost](https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt3fbfeb0c37c5ed78/64daab2bd7ccb0bad5e43892/image4.png)

## Moving ahead

If you’ve followed along, congratulations! You now have a working real-time notification service for your Ghost blog. And if you haven’t, what are you waiting for? Sign up for a free account on MongoDB Atlas and start building. You can use the [final branch of this repository](https://github.com/tq-bit/mongodb-article-mongo-changestreams/tree/5-add-frontend-example) to get started and explore the full power of MongoDB’s toolkit.

Learn how to work with MongoDB change streams and develop a server-sent event application that integrates with Ghost CMS.

Tobias Quante

Content by Tobias Quante

Adding Real-Time Notifications to Ghost CMS Using MongoDB and Server-Sent Events