Comprehensive document how to contribute/develop MongoDB Kafka Source Connector?

Working with Data Connectors & Integrations
#1

Hi experts!!

I’m looking for a comprehensive document that explains details how to contribute/develop MongoDB Kafka Source Connector.

  • how to setup the local development environment
  • any rule or convention for code
  • how to run tests
  • how to compile
  • etc.

Gatsby

#2

Hi @Gatsby_Lee ,

I’m not aware of a comprehensive guide for contributing to the Kafka Connector, but I would start by reviewing the source repo: GitHub - mongodb/mongo-kafka: MongoDB Kafka Connector

It looks like you need Java 8+ and gradle installed as well as a replica set for running tests.

Before contributing to an open source project, I would start by creating an issue to discuss the bug fix or feature you plan to contribute so the maintainers can give you some feedback on how to approach this.

If there aren’t any documented code conventions, I would follow the style of the existing code and tests.

Regards,
Stennie

#3

Hi @Gatsby_Lee,

Thanks for your interest in the connector! Thats a great question, the Readme has a couple of sections; the Bugs and Feature Requests section for information regarding the process of reporting bugs or adding new features to the connector. We have a triage process, so it make take time for a new feature requests to be fully developed and coded.

The build section covers how to run the test suite. The project uses gradle so dependencies are downloaded and managed by the gradle build. To get up and running it really should be as easy as:

$ git clone https://github.com/mongodb/mongo-kafka.git
$ cd mongo-kafka
$ ./gradlew check -Dorg.mongodb.test.uri=mongodb://localhost:27017

The test suite requires mongod to be running. Note: the source connector requires a replicaSet.

Coding convention should be sympathetic to the existing code. The project uses Spotless and has various spotless rules that will auto format the code when running the check command. The -Dorg.mongodb.test.uri flag lets you pass in the location of the MongoDB to run the test suite against and it defaults to localhost:27017.

For writing your own custom post processors I have an example repo which should be of assistance. Inspired by your question, I have added KAFKA-276 to see if we can add that into the main repo and provide documentation on how to use and deploy any custom post processes - such as a new write strategy.

I hope that all helps :slight_smile:

Ross

2 Likes
#4

HI Ross,

Thank you for your reply.
I will try it out.

Gatsby

#5

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.