Docs Menu

Kafka Connector Tutorial Setup

The tutorials in this section run on a development environment using Docker to package the dependencies and configurations you need to run the MongoDB Connector for Apache Kafka. Make sure you complete the development environment setup steps before proceeding to the tutorials.

  • Create or sign into your Docker account to download the Docker containers you need for this tutorial. To learn how to sign up for an account and install Docker Desktop, read the signup and download steps in the Docker Hub Quickstart.
  • A terminal app and shell. For MacOS users, use Terminal or a similar app. For Windows users, use PowerShell.
  • Optional. Install git to download the setup files. To learn how to install git, read the Git Downloads page.
1

Start Docker Desktop.

After it completes startup, download the tutorial Docker image which contains helper scripts to simplify the commands you need to run in the tutorials.

Log into Docker on the command line, providing your credentials when prompted, by running the following command:

docker login

Run the following command in your terminal to download the image from Docker Hub:

docker pull robwma/mongokafkatutorial

When the command successfully completes, you should see the following output:

Status: Downloaded newer image for robwma/mongokafkatutorial:latest
docker.io/robwma/mongokafkatutorial:latest

If you already have the Docker image in your local repository, you should see the following message:

Status: Image is up to date for robwma/mongokafkatutorial:latest
docker.io/robwma/mongokafkatutorial:latest
2

Next, clone the tutorial git repository with the following command:

git clone https://github.com/mongodb-university/kafka-edu.git

If you do not have git installed, you can download the zip archive instead.

3

Select the tab that matches your OS for instructions on how to run the commands in this guide:

Note

If the script exits with the message "Please terminate the local mongod on 27017", you must terminate the process that is listening on port 27017 before you can continue. After you terminate the process, run the script again.

If you previously started the image in Docker Desktop and have not stopped it, the image automatically starts when you launch Docker Desktop. If the image is already running, you can proceed to the next step.

The "mongo-kafka-base" image creates a Docker container that includes all the services you need in the tutorial and runs them on a shared network called "mongodb-kafka-base_localnet" as shown in the following diagram:

Diagram that shows the Docker compose containers in mongo-kafka-base

When the script completes successfully, it outputs the following information:

The following services are running:
MongoDB on port 27017
Kafka Broker on 9092
Kafka Zookeeper on 2181
Kafka Connect on 8083
4

Confirm the development environment started normally by running the following command from the "mongodb-kafka-base" directory:

This command should output the following information if the Docker development environment was set up successfully:

Kafka topics:
[
<list of kafka topics>
]
The status of the connectors:
Currently configured connectors
[]
Version of MongoDB Connector for Apache Kafka installed:
{"class":"com.mongodb.kafka.connect.MongoSinkConnector","type":"sink","version":"<version>"}
{"class":"com.mongodb.kafka.connect.MongoSourceConnector","type":"source","version":"<version>"}

Since you have not started the connectors, the status and configured list are empty.

Your development environment setup is complete and you can proceed to the next step of the tutorial.

←  TutorialsExplore MongoDB Change Streams →
Give Feedback
© 2022 MongoDB, Inc.

About

  • Careers
  • Investor Relations
  • Legal Notices
  • Privacy Notices
  • Security Information
  • Trust Center
© 2022 MongoDB, Inc.