Connection.uri validation error when using a config provider

Working with Data Connectors & Integrations
,
1

Hello,

I am following this article to create a source connector in the AWS Ecosystem (AWS MSK and AWS Kafka Connect). I am following this article which details how we can externalize the secrets (username and password) from the mongo connection string with a config provider: Stream data with Amazon DocumentDB, Amazon MSK Serverless, and Amazon MSK Connect | AWS Database Blog

However, I get the following validation error on the connection.uri property when using the config provider:

Code: **InvalidInput.InvalidConnectorConfiguration**

Message: **The connector configuration is invalid. Message: Connector configuration is invalid and contains the following 1 error(s): Invalid value mongodb://${ssm::/order/db/order/username}:${ssm::/order/db/order/password}@redactedurlformongo:27017/?ssl=true&replicaSet=rs0&retryWrites=false for configuration connection.uri: The connection string contains an invalid host '${ssm::'. Reserved characters such as ':' must be escaped according RFC 2396. Any IPv6 address literal must be enclosed in '[' and ']' according to RFC 2732.**

Here is an example of my configuration:

connector.class=com.mongodb.kafka.connect.MongoSourceConnector
connection.ssl.truststorePassword=${ssm::/platform/db/master/cacert/truststore/password}
tasks.max=1
change.stream.full.document=updateLookup
config.providers.ssm.class=com.amazonaws.kafka.config.providers.SsmParamStoreConfigProvider
config.providers=s3import,ssm
collection=orders
connection.ssl.truststore=${s3import:us-east-2:ole-poc-kafka-connectors/rds-truststore.jks}
config.providers.s3import.param.region=us-east-2
database=order
topic.namespace.map={"order.orders": "orders"}
connection.uri=mongodb://${ssm::/order/db/order/username}:${ssm::/order/db/order/password}@redactedurlformongo:27017/?ssl=true&replicaSet=rs0&retryWrites=false
errors.tolerance=all
config.providers.ssm.param.region=us-east-2
config.providers.s3import.class=com.amazonaws.kafka.config.providers.S3ImportConfigProvider

This issue is also reported here: MongoDB Source Connector - configuration validation runs before replacement when using a Config Provider · Issue #1319 · confluentinc/kafka-connect-jdbc · GitHub

2

Hi @Dan_Hvidding,

Welcome to the MongoDB Community!

I’m not very familiar with Kafka connectors and the AWS Ecosystem, but based on the error message, it seems like there is an issue parsing the URI of the MongoDB due to a syntax problem.

  • May I ask, does it work without externalizing the credentials?
  • It is possible to connect and try doing it one by one, first with the password and then with the username, to see where the issue actually occurs.

The error indicates that the MongoDB connection URI is not being parsed correctly due to the ${ssm::} placeholders used for the username and password. I guess the SSM placeholders are not being resolved even before the connection URI is validated.

Based on the error message, it seems that the appropriate way to fix the issue is to escape the : and $ characters in the URI and use [ ] around the IPv6 address so that it becomes valid even before parameter resolution.

Hope the above helps!

Regards,
Kushagra

3

Thanks Kushagra,

I was able to resolve this by putting the config provider configuration within the Worker config and not the Connector config.

I found the description here: https://jira.mongodb.org/browse/KAFKA-361

1 Like
4