Igor Alekseev

Partner Solutions Architect, Data & Analytics

Robert Walters

Senior Product Manager

[Amazon Managed Streaming for Apache Kafka](https://aws.amazon.com/msk/features/) (MSK) is a fully managed, highly available Apache Kafka service. MSK makes it easy to ingest and process streaming data in real time and leverage that data easily within the AWS ecosystem. By being able to quickly stand up a Kafka solution, you spend less time managing infrastructure and more time solving your business problems, dramatically increasing productivity. MSK also supports integration of data sources such as MongoDB via the [AWS MSK Connect](https://aws.amazon.com/msk/features/msk-connect/) (Connect) service. This Connect service works with the [MongoDB Connector for Apache Kafka](https://www.mongodb.com/kafka-connector), enabling you to easily integrate MongoDB data.  

In this blog post, we will walk through how to set up MSK, configure the MongoDB Connector for Apache Kafka, and create a secured VPC Peered connection with MSK and a MongoDB Atlas cluster. The high-level process is as follows:

* Configure Amazon Managed Streaming for Apache Kafka
* Configure EC2 client 
* Configure a MongoDB Atlas Cluster
* Configure Atlas Private Link including VPC and subnet of the MSK
* Configure plugin in MSK for MongoDB Connector
* Create topic on MSK Cluster
* Install MongoSH command line tool on client
* Configure MongoDB Connector as a source or sink

In this example, we will have two collections in the same MongoDB cluster—the “source” and the “sink.” We will insert sample data into the source collection from the client, and this data will be consumed by MSK via the MongoDB Connector for Apache Kafka running as an MSK connector. As data arrives in the MSK topic, another instance of the MongoDB Connector for Apache Kafka will write the data to the MongoDB Atlas cluster “sink” collection. To align with best practices for secure configuration, we will set up an AWS Network Peered connection between the MongoDB Atlas cluster and the VPC containing MSK and the client EC2 instance.  

![Figure 1. Data flow between AWS MSK and MongoDB Atlas ](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/data_flow_mongodb_atlas_177d1bca15.png)

## Configure AWS Managed Service for Kafka

To create an Amazon MSK cluster using the AWS Management Console, sign in to the AWS Management Console, and open the [Amazon MSK console](https://console.aws.amazon.com/msk/home).

* Choose Create cluster and select Quick create.

For Cluster name, enter MongoDBMSKCluster.

For Apache Kafka version, select one that is 2.6.2 or above.

For broker type, select kafla.t3.small.

From the table under All cluster settings, copy the values of the following settings and save them because you will need them later in this blog:

* VPC
* Subnets
* Security groups associated with VPC

![Figure 2. Cluster settings showing VPC and Subnets](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/cluster_settings_95438aaef0.png)

* Choose “Create cluster.”

## Configure an EC2 client

Next, let's configure an EC2 instance to create a topic. This is where the MongoDB Atlas source collection will write to. This client can also be used to query the MSK topic and monitor the flow of messages from the source to the sink.

To create a client machine, open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/).

* Choose Launch instances.
* Choose Select to create an instance of Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type.
* Choose the t2.micro instance type by selecting the check box.
* Choose Next: Configure Instance Details.
* Navigate to the Network list and choose the VPC whose ID you saved in the previous step.
* Go to Auto-assign Public IP list and choose Enable.
* In the menu near the top, select Add Tags.
* Enter Name for the Key and MongoDBMSKCluster for the Value.
* Choose Review and Launch, and then click Launch.
* Choose Create a new key pair, enter MongoDBMSKKeyPair for Key pair name, and then choose Download Key Pair. Alternatively, you can use an existing key pair if you prefer.
* Start the new instance by pressing Launch Instances.

Next, we will need to configure the networking to allow connectivity between the client instance and the MSK cluster.

* Select View Instances. Then, in the Security Groups column, choose the security group that is associated with the MSKTutorialClient instance.
* Copy the name of the security group, and save it for later.
* Open the Amazon VPC console at [https://console.aws.amazon.com/vpc/](https://console.aws.amazon.com/vpc/).
* In the navigation pane, click on Security Groups. Find the security group whose ID you saved in Step 1 (Create an Amazon MSK Cluster). Choose this row by selecting the check box in the first column.
* In the Inbound Rules tab, choose Edit inbound rules.
* Choose Add rule.
* In the new rule, choose All traffic in the Type column. In the second field in the Source column, select the security group of the client machine. This is the group whose name you saved earlier in this step.
* Click Save rules.

The cluster's security group can now accept traffic that comes from the client machine's security group.

## Create MongoDB Atlas Cluster

To create a MongoDB Atlas Cluster, follow the Getting Started with Atlas tutorial. Note that in this blog, you will need to create an M30 Atlas cluster or above—as VPC peering is not available for M0, M2, and M5 clusters.

![Figure 3: Create new cluster dialog](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/create_new_cluster_144e85016b.png)

Once the cluster is created, configure an AWS private endpoint in the Atlas Network Access UI supplying the same subnets and VPC.  

* Click on Network Access.
* Click on Private Endpoint, and then the Add Private Endpoint button.

![Figure 4. Add private endpoint](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/creating_private_endpoint_ba8f2ce2b8.png)

* Fill out the VPC and subnet IDs from the previous section.

![Figure 5. Create private endpoint](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/create_private_endpoint_interface_dialogue_58645ad28f.png)

* SSH into the client machine created earlier and issue the following command in the Atlas portal: **aws ec2 create-vpc-endpoint **
    * Note that you may have to first configure the AWS CLI command using **aws configure** before you can create the VPC through this tool. See [Configuration Basics](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) for more information.

## Configure MSK plugin

Next, we need to create a custom plugin for MSK. This custom plugin will be the MongoDB Connector for Apache Kafka. For reference, note that the connector will need to be uploaded to an S3 repository **before** you can create the plugin. You can download the MongoDB Connector for Apache Kafka from [Confluent Hub](https://www.confluent.io/hub/mongodb/kafka-connect-mongodb).

* Select “Create custom plugin” from the Custom Plugins menu within MSK.
* Fill out the custom plugin form, including the S3 location of the downloaded connector, and click “Create custom plugin.”

![Figure 6. Create custom plugin](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/create_custom_plugin_7514327ccf.png)

## Create topic on MSK cluster

When we start reading the data from MongoDB, we also need to create a topic in MSK to accept the data. On the client EC2 instance, let’s install Apache Kafka, which includes some basic tools. 

To begin, run the following command to install Java:

**sudo yum install java-1.8.0**

Next, run the command below to download Apache Kafka.

**wget https://archive.apache.org/dist/kafka/2.6.2/kafka_2.12-2.6.2.tgz**

Building off the previous step, run this command in the directory where you downloaded the TAR file:

**tar -xzf kafka_2.12-2.6.2.tgz**

The distribution of Kafka includes a **bin** folder with tools that can be used to manage topics. Go to the **kafka_2.12-2.6.2** directory.

To create the topic that will be used to write MongoDB events, issue this command:

`bin/kafka-topics.sh --create --zookeeper (INSERT YOUR ZOOKEEPER INFO HERE)--replication-factor 1 --partitions 1 --topic MongoDBMSKDemo.Source`

Also, remember that you can copy the Zookeeper server endpoint from the “View Client Information” page on your MSK Cluster. In this example, we are using plaintext.

![Figure 7. Apache ZooKeeper connection](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/apache_zookeeper_connection_6e23beafd6.png)

## Configure source connector

Once the plugin is created, we can create an instance of the MongoDB connector by selecting “Create connector” from the Connectors menu.

![Figure 8. Custom plugin dialog](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/choose_custom_plugin_f50bd58c30.png)

* Select the MongoDB plug in and click “Next.”
* Fill out the form as follows:

Conector name: **MongoDB Source Connector**

Cluster Type: **MSK Connector**

Select the MSK cluster that was created previously, and select “None” under the authentication drop down menu.

Enter your connector configuration (shown below) in the configuration settings text area.

`connector.class=com.mongodb.kafka.connect.MongoSourceConnector
database=MongoDBMSKDemo
collection=Source
tasks.max=1
connection.uri=(MONGODB CONNECTION STRING HERE)
value.converter=org.apache.kafka.connect.storage.StringConverter
key.converter=org.apache.kafka.connect.storage.StringConverter`

**Note**: You can find your Atlas connection string by clicking on the Connect button on your Atlas cluster. Select “Private Endpoint” if you have already configured the Private Endpoint above, then press “Choose a connection method.” Next, select “Connect your application” and copy the **mongodb+srv** connection string.

![Figure 9. Access permissions](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/access_permission_error_403a7a5e48.png)

In the “Access Permissions” section, you will need to create an IAM role with the [required trust policy](https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect-service-execution-role.html).

![Figure 10. Log delivery dialog](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/aws_msk_log_delivery_0a4ab236a0.png)

Once this is done, click “Next.” The last section will offer you the ability to use logs—which we highly recommend, as it will simplify the troubleshooting process. 

## Configure sink connector

Now that we have the source connector up and running, let’s configure a sink connector to complete the round trip. Create another instance of the MongoDB connector by selecting “Create connector” from the Connectors menu.

![Figure 8. Custom plugin dialog](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/choose_custom_plugin_f50bd58c30.png)

Select the same plugin that was created previously, and fill out the form as follows: 

Connector name: **MongoDB Sink Connector**
Cluster type: **MSK Connector**
Select the MSK cluster that was created previously and select “None” under the authentication drop down menu.
Enter your connector configuration (shown below) in the Configuration Settings text area.

`connector.class=com.mongodb.kafka.connect.MongoSinkConnector
database=MongoDBMSKDemo
collection=Sink
tasks.max=1
topics=MongoDBMSKDemo.Source
connection.uri=(MongoDB Atlas Connection String Gos Here)
value.converter=org.apache.kafka.connect.storage.StringConverter
key.converter=org.apache.kafka.connect.storage.StringConverter`

In the Access Permissions section, select the IAM role created earlier that has the [required trust policy](https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect-service-execution-role.html). As with the previous connector, be sure to leverage a log service like CloudWatch. 

Once the connector is successfully configured, we can test the round trip by writing to the Source collection and seeing the same data in the Sink collection.  

We can insert data in one of two ways: either through the intuitive Atlas UI, or with the new MongoSH (mongoshell) command line tool. Using [MongoSH](https://docs.mongodb.com/mongodb-shell/), you can interact directly with a MongoDB cluster to test queries, perform ad hoc database operations, and more. 

For your reference, we’ve added a section on how to use the mongoshell on your client EC2 instance below. 

## Install MongoDB shell on client

On the client EC2 instance, create a **/etc/yum.repos.d/mongodb-org-5.0.repo** file by typing:

`sudo nano /etc/yum.repos.d/mongodb-org-5.0.repo`

Paste in the following:

`[mongodb-org-5.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/amazon/2/mongodb-org/5.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=[https://www.mongodb.org/static/pgp/server-5.0.asc](https://www.mongodb.org/static/pgp/server-5.0.asc)`

Next, install the MongoSH shell with this command:

`sudo yum install -y mongodb-mongosh`

Use the template below to connect to your MongoDB cluster via mongoshell: 

`mongosh “ (paste in your Atlas connection string here) “`

Once connected, type:
`Use MongoDBMSKDemo
db.Source.insertOne({“Testing”:123})`

To check the data on the sink collection, use this command:
`db.Sink.find({})`

If you run into any issues, be sure to check the log files. In this example, we used CloudWatch to read the events that were generated from MSK and the MongoDB Connector for Apache Kafka.

![Figure 12. Log events  ](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/cloud_watch_log_events_e5a0045c3d.png)

## Summary

Amazon Managed Streaming for Apache Kafka ([MSK](https://aws.amazon.com/msk/)) is a fully managed, secure, and highly available Apache Kafka service that makes it easy to ingest and process streaming data in real time. MSK allows you to import Kafka connectors such as the MongoDB Connector for Apache Kafka. These connectors make working with data sources seamless within MSK. In this article, you learned how to set up MSK, MSK Connect, and the MongoDB Connector for Apache Kafka.  You also learned how to set up a MongoDB Atlas cluster and configure it to use AWS network peering. To continue your learning, check out the following resources:

[MongoDB Connector for Apache Kafka Documentation](https://docs.mongodb.com/kafka-connector/current/)
[Amazon MSK Getting Started](https://docs.aws.amazon.com/msk/latest/developerguide/getting-started.html)
[Amazon MSK Connect Getting Started](https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect-getting-started.html)

In this article, learn how to set up Amazon MSK, configure the MongoDB Connector for Apache Kafka, and how it can be used as both a source and sink for data integration with MongoDB Atlas running in AWS.

Integrating MongoDB with Amazon Managed Streaming for Apache Kafka (MSK)

Andrew Morgan

Staff Developer Advocate

>
>
>This post was developed with the help of AWS.
>
>

[Realm](https://www.mongodb.com/realm) makes it easy to develop compelling mobile applications backed by a serverless MongoDB Realm back end and the [MongoDB Atlas database service](https://www.mongodb.com/cloud/atlas). You can enrich those applications by integrating with AWS's broad ecosystem of services. In this article, we'll show you how to configure Realm and AWS to turn Atlas database changes into [Amazon EventBridge](https://aws.amazon.com/eventbridge/) events – all without adding a single line of code. Once in EventBridge, you can route events to other services which can act on them.

We'll use an existing mobile chat application ([RChat](https://github.com/realm/RChat)). RChat creates new `ChatMessage` objects which [Realm Sync](https://www.mongodb.com/realm/mobile/sync) writes to the ChatMessage Atlas collection. Realm also syncs the chat message with all other members of the chat room.

!["Realm EventBridge Integration"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Realm_EventBridge_Integration.png)

This post details how to add a new feature to the RChat application – forwarding messages to a Slack channel.

We'll add a [Realm Trigger](https://docs.mongodb.com/realm/triggers) that forwards any new `ChatMessage` documents to EventBridge.  EventBridge stores those events in an event bus, and a rule will route it to a Lambda function. The Lambda function will use the [Slack SDK](https://slack.dev/node-slack-sdk/) (using credentials we'll store in [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/)).

>
>
>[Amazon EventBridge](https://aws.amazon.com/eventbridge/) is a serverless event bus that makes it easier to connect applications together using data from your applications, integrated software as a service (SaaS) applications, and AWS services. It does so by delivering a stream of real-time data from various event sources. You can set up routing rules to send data to targets like [AWS Lambda](https://aws.amazon.com/lambda/) and build loosely coupled application architectures that react in near-real time to data sources.
>
>

## Prerequisites

If you want to build and run the app for yourself, this is what you'll
need:

-   Mac OS 11+
-   [Node.js 12.x+](https://nodejs.org/en/download/)
-   [Xcode 12.3+](https://developer.apple.com/xcode/)
-   iOS 14.2+ (a real device, or the simulator built into Xcode)
-   [git command-line tool](https://git-scm.com/downloads)
-   [realm-cli command-line tool](https://docs.mongodb.com/realm/deploy/realm-cli-reference)
-   [AWS account](https://console.aws.amazon.com/)
-   (Free) [MongoDB account](https://cloud.mongodb.com/)
-   [Slack account](https://slack.com/)

If you're not interested in running the mobile app (or don't have access to a Mac), the article includes instructions on manually adding a document that will trigger an event being sent to EventBridge.

## Walkthrough

This walkthrough shows you how to:

-   [Set Up the RChat Back End Realm App](#set-up-the-rchat-back-end-realm-app)
-   [Create a Slack App](#create-a-slack-app)
-   [Receive MongoDB Events in Amazon EventBridge](#receive-mongodb-events-in-amazon-eventbridge)
-   [Store Slack Credentials in AWS Secrets Manager](#store-slack-credentials-in-aws-secrets-manager)
-   [Write and Configure the AWS Lambda Function](#write-and-configure-the-aws-lambda-function)
-   [Link the Lambda Function to the MongoDB Partner Event Bus](#link-the-lambda-function-to-the-mongodb-partner-event-bus)
-   [Run the RChat iOS App](#run-the-rchat-ios-app)
-   [Test the End-to-End Integration (With or Without the iOS App)](#test-the-end-to-end-integration--with-or-without-the-ios-app-)

### Set Up the RChat Back End Realm App

If you don't already have a MongoDB cloud account, [create one](https://cloud.mongodb.com/). You'll also create an Atlas organization and project as you work through the wizard. For this walkthrough, you can use the free tier. Stick with the defaults (e.g., "Cluster Name" = "Cluster0") but set the version to MongoDB 4.4.

While your database cluster is starting, select "Project Access" under "Access Manager." Create an API key with "Project Owner" permissions.  Add your current IP address to the access list. Make a note of the API keys; they're needed when using [realm-cli](https://docs.mongodb.com/realm/deploy/realm-cli-reference).

Wait until the Atlas cluster is running.

From a terminal, import the back end Realm application (substituting in your Atlas project's API keys) using [realm-cli](https://docs.mongodb.com/realm/deploy/realm-cli-reference):

``` bash
git clone https://github.com/realm/RChat.git
cd RChat/RChat-Realm/RChat
realm-cli login --api-key <your new public key> --private-api-key <your new private key>
realm-cli import # Then answer prompts, naming the app "RChat"
```

From the Atlas UI, click on the Realm logo and you will see the RChat app. Open it and make a note of the Realm "App Id":

!["Realm App ID"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Realm_App_ID.png)

Optionally, create database indexes by using [mongorestore](https://docs.mongodb.com/manual/reference/program/mongorestore/) to import the empty database from the `dump` folder.

### Create a Slack App

The Slack app simply allows us to send a message to a Slack channel.

Navigate to the [Slack API page](https://api.slack.com/apps). (You'll need to log in or register a new account if you don't have one.)

Click on the button to create a new Slack app, name it "RChat," and select one of your Slack workspaces. (If using your company's account, you may want or need to create a new workspace.)

Give your app a short description and then click "Save Changes."

After creating your Slack app, select the "OAuth & Permissions" link.  Scroll down to "Bot Token Scopes" and add the `chat.write` and `channels:read` scopes.

Click on "Install to Workspace" and then "Allow."

Take a note of the new "Bot User OAuth Access Token."

From your Slack client, create a new channel named "rchat-notifications." Invite your Slack app bot to the channel (i.e., send a message from the channel to "@RChat Messenger" or whatever Slack name you gave to your app):

!["Slack invite"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Slack_Invite.png)

You now need to find its channel ID from a terminal window (substituting in your Slack OAuth access token):

``` bash
curl --location --request GET 'slack.com/api/conversations.list' \
--header 'Authorization: Bearer xoxb-XXXXXXXXXXXXXXX-XXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXX'
```

In the results, you'll find an entry for your new "rchat-notifications" channel. Take a note of its `id`; it will be stored in AWS Secrets Manager and then used from the Lambda function when calling the Slack SDK:

``` json
{
   "name" : "rchat-notifications",
   "is_pending_ext_shared" : false,
   "is_ext_shared" : false,
   "is_general" : false,
   "is_private" : false,
   "is_member" : false,
   "name_normalized" : "rchat-notifications",
   "is_archived" : false,
   "is_channel" : true,
   "topic" : {
   "last_set" : 0,
   "creator" : "",
   "value" : ""
   },
   "unlinked" : 0,
   "is_org_shared" : false,
   "is_group" : false,
   "shared_team_ids" : [
   "T01JUGHQXXX"
   ],
   "is_shared" : false,
   "is_mpim" : false,
   "is_im" : false,
   "pending_connected_team_ids" : [],
   "purpose" : {
   "last_set" : 1610987122,
   "creator" : "U01K7ET1XXX",
   "value" : "This is for testing the RChat app"
   },
   "creator" : "U01K7ET1XXX",
   "created" : 1610987121,
   "parent_conversation" : null,
   "id" : "C01K1NYXXXX",
   "pending_shared" : [],
   "num_members" : 3,
   "previous_names" : []
}
```

### Receive MongoDB Events in Amazon EventBridge

EventBridge supports MongoDB as a partner event source; this makes it very easy to receive change events from Realm Triggers.

From the [EventBridge console](https://console.aws.amazon.com/events), select "Partner event sources." Search for the "MongoDB" partner and click "Set up":

!["EventBridge Partner Source"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/EventBridge_partner_source.png)

Take a note of your AWS account ID.

Return to the Realm UI navigate to "Triggers" and click "Add a trigger." Configure the trigger as shown here:

!["Realm Trigger"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Realm_Trigger.png)

Rather than sticking with the default "Function" event type (which is
Realm Function, not to be confused with Lambda), select "EventBridge,"
add your AWS Account ID from the previous section, and click "Save"
followed by "REVIEW & DEPLOY":

!["EventBridge Trigger"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/EventBridge_Trigger.png)

Return to the AWS "Partner event sources" page, select the new source, and click "Associate with event bus":

!["Event Bus"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Event_Bus.png)

On the next screen, leave the "Resource-based policy" empty.

Returning to the "Event buses" page, you'll find the new MongoDB partner bus.

### Store Slack Credentials in AWS Secrets Manager

We need a new Lambda function to be invoked on any MongoDB change events added to the event bus. That function will use the Slack API to send messages to our channel. The Lambda function must provide the OAuth token and channel ID to use the Slack SDK. Rather than storing that private information in the function, it's more secure to hold them in AWS Secrets Manager.

Navigate to the [Secrets Manager console](https://console.aws.amazon.com/secretsmanager) and click "Store a new secret." Add the values you took a note of when creating the Slack app:

!["New Secret"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/New_Secret.png)

Click through the wizard, and apart from assigning a unique name to the secret (and take a note of it as it's needed when configuring the Lambda function), leave the other fields as they are. Take a note of the ARN for the new secret as it's required when configuring the Lambda function.

### Write and Configure the AWS Lambda Function

From the [Lambda console](https://console.aws.amazon.com/lambda), click "Create Function." Name the function "sendToSlack" and set the runtime to "Node.js 12.x."

After creating the Lambda function, navigate to the "Permissions" tab and click on the "Execution role" role name. On the new page, click on the "Policy name" and then "Edit policy."

Click "Add additional permissions" and select the "Secrets Manager" service:

!["Lambda Permissions"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Lambda_Permissions.png)

Select the "ListSecrets" action. This permission allows the Lambda function to see what secrets are available, but not to read our specific Slack secret. To remedy that, click "Add additional permissions" again.  Once more, select the "Secrets Manager" service, but this time select the "Read" access level and specify your secret's ARN in the resources section:

!["Secrets ARN"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Secrets_ARN.png)

Review and save the new permissions.

Returning to the Lambda function, select the "Configuration" tab and add an environment variable to set the "secretName" to the name you chose when creating the secret (the function will use this to access Secret Manager):

!["Lambda Environment Variable"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Lambda_Enivornment_Variable.png)

It can take some time for the function to fetch the secret for the first time, so set the timeout to 30 seconds in the "Basic settings" section.

Finally, we can write the actual Lambda function.

From a terminal, bootstrap the function definition:

``` bash
mkdir lambda
cd lambda
npm install '@slack/web-api'
```

In the same lambda directory, create a file called `index.js`:

``` javascript
const {WebClient} = require('@slack/web-api');
const AWS = require('aws-sdk');

const secretName = process.env.secretName;

let slackToken = "";
let channelId = "";
let secretsManager = new AWS.SecretsManager();

const initPromise = new Promise((resolve, reject) => {
   secretsManager.getSecretValue(
      { SecretId: secretName },
      function(err, data) {
            if(err) {
               console.error(`Failed to fetch secrets: ${err}`);
               reject();
            } else {
               const secrets = JSON.parse(data.SecretString);
               slackToken = secrets.slackToken;
               channelId = secrets.channelId;
               resolve()
            }
      }
   )
});

exports.handler = async (event) => {
   await initPromise;
   const client = new WebClient({ token: slackToken });
   const blocks = [
      {
            "type": "section",
            "text": {
               "type": "mrkdwn",
               "text": `*${event.detail.fullDocument.author} said...*\n\n${event.detail.fullDocument.text}`
            },
            "accessory": {
               "type": "image",
               "image_url": "https://cdn.dribbble.com/users/27903/screenshots/4327112/69chat.png?compress=1&resize=800x600",
               "alt_text": "Chat logo"
            }
      },
      {
            "type": "section",
            "text": {
               "type": "mrkdwn",
               "text": `Sent from <https://github.com/realm/RChat|RChat>`
            }
      },
      {
            "type": "divider"
      }
   ]

   await publishMessage(
      channelId, `Sent from RChat: ${event.detail.fullDocument.author} said "${event.detail.fullDocument.text}"`,
      blocks);

   const response = {
      statusCode: 200,
      body: JSON.stringify('Slack message sent')
   };
   return response;

   async function publishMessage(id, text, blocks) {
      try {
            const result = await client.chat.postMessage({
               token: slackToken,
               channel: id,
               text: text,
               blocks: blocks
            });
      }
      catch (error) {
            console.error(error);
      }
   }
};
```

There are a couple of things to call out in that code.

This is how the Slack credentials are fetched from Secret Manager:

``` javascript
const secretName = process.env.secretName;
var MyPromise = new AWS.SecretsManager();
const secret = await MyPromise.getSecretValue({ SecretId: secretName}).promise();
const openSecret = JSON.parse(secret.SecretString);
const slackToken = openSecret.slackToken;
const channelId = openSecret.channelId;
```

`event` is passed in as a parameter, and the function retrieves the original MongoDB document's contents from `event.detail.fullDocument`.

`blocks` is optional, and if omitted, the SDK uses text as the body of the Slack message.

Package up the Lambda function:

``` bash
zip -r ../lambda.zip .
```

From the Lambda console, upload the zip file and then deploy:

!["Upload Lambda Function"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Upload_Lambda_Function.png)

The Lambda function is now complete, and the next section will start routing events from the EventBridge partner message bus to it.

### Link the Lambda Function to the MongoDB Partner Event Bus

The final step to integrate our Realm app with the new Lambda function is to have that function consume the events from the event bus. We do that by adding a new EventBridge rule.

Return to the [EventBridge console](https://console.aws.amazon.com/events) and click the "Rules" link. Select the "aws.partner/mongodb.com/stitch.trigger/xxx" event bus and click "Create rule."

The "Name" can be anything. You should use an "Event pattern," set "Pre-defined pattern by service," search for "Service partner" "MongoDB," and leave the "Event pattern" as is. This rule matches all bus events linked to our AWS account (i.e., it will cover everything sent from our Realm function):

!["EventBridge Pattern"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/EventBridge_Pattern.png)

Select the new Lambda function as the target and click "Create":

!["EventBridge Rule Lambda"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/EventBridge_Rule_Lambda.png)

### Run the RChat iOS App

After creating the back end Realm app, open the RChat iOS app in Xcode:

``` bash
cd ../../RChat-iOS
open RChat.xcodeproj
```

Navigate to `RChatApp.swift`. Replace `rchat-xxxxx` with your Realm App Id:

!["Set Realm ID Xcode"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Set_Realm_ID_Xcode.png)

Select your target device (a connected iPhone/iPad or one of the built-in simulators) and build and run the app with `⌘r`.

### Test the End-to-End Integration (With or Without the iOS App)

To test a chat app, you need at least two users and two instances of the chat app running.

From Xcode, run (`⌘r`) the RChat app in one simulator, and then again in a second simulator after changing the target device. On each device, register a new user. As one user, create a new chat room (inviting the second user). Send messages to the chat room from either user, and observe that message also appearing in Slack:

!["RChat Demo"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/RChat_Demo.gif)

#### If You Don't Want to Use the iOS App

Suppose you're not interested in using the iOS app or don't have access to a Mac. In that case, you can take a shortcut by manually adding documents to the `ChatMessage` collection within the `RChat` database.  Do this from the "Collections" tab in the Atlas UI. Click on "INSERT DOCUMENT" and then ensure that you include fields for "author" and "text":

!["Insert Document"](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/how-to/realm-eventbridge-slack/Insert_Document.png)

## Summary

This post stepped through how to get your data changes from MongoDB into your AWS ecosystem with no new code needed. Once your EventBridge bus has received the change events, you can route them to one or more services. Here we took a common approach by sending them to a Lambda function which then has the freedom to import external libraries and work with other AWS or external services.

To understand more about the Realm chat app that was the source of the messages, read [Building a Mobile Chat App Using Realm – Data Architecture](/how-to/realm-swiftui-ios-chat-app).

## References

-   [RChat GitHub repo](https://github.com/realm/RChat)
-   [Building a Mobile Chat App Using Realm – Data Architecture](/how-to/realm-swiftui-ios-chat-app)
-   [Slack SDK](https://slack.dev/node-slack-sdk/)
-   [Sending Trigger Events to AWS EventBridge](https://docs.mongodb.com/realm/triggers/eventbridge)

>
>
>If you have questions, please head to our [developer community website](https://community.mongodb.com/) where the MongoDB engineers and the MongoDB community will help you build your next big idea with MongoDB.
>
>


Step through extending a Realm chat app to send messages to a Slack channel using Amazon EventBridge

Igor Alekseev

Content by Igor Alekseev

Integrating MongoDB with Amazon Managed Streaming for Apache Kafka (MSK)

Integrate Your Realm App with Amazon EventBridge