> For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt # Add a Kafka Private Link Connection Atlas Stream Processing supports creating Private Link connections to various hosted [Apache Kafka](https://kafka.apache.org/) services. Select an option to display the procedure to add the connection for your service using your preferred interface: ## Add an AWS (Amazon Web Services) Confluent Private Link Connection through the Atlas Administration API **Important:** You can't have more than one Private Link connection to a given Confluent cluster per Atlas project. Before you begin this procedure, call the [Return All Private Link Connections](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-listgroupstreamprivatelinkconnections) endpoint. If you have an existing Private Link connection to your Confluent cluster within Atlas but not within your Confluent account, only perform those steps that configure your Confluent-side networking. ### Create the Private Endpoint ### Dedicated To create a connection to an AWS (Amazon Web Services) Confluent Dedicated cluster for use in your Atlas Stream Processing project, follow these steps: 1. Get the AWS (Amazon Web Services) account ID for your Atlas project. You must configure your Confluent cluster to accept incoming connections from your Atlas project. IMPORTANT: Confluent accepts incoming connections only from AWS (Amazon Web Services). To use a Confluent Private Link connection, you must host your stream processing workspaces on AWS (Amazon Web Services). Call the [Return Account ID and VPC ID for group and region](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-getgroupstreamaccountdetails) Atlas Administration API endpoint. Save the value of `awsAccountId` for further use in this procedure. 2. Go to your Confluent cluster networking details. In your Confluent account: Navigate to the cluster you want to connect to. In your cluster networking interface, navigate to your cluster networking details. 3. Add Private Link Access to your Confluent cluster. [Add Private Link Access](https://docs.confluent.io/cloud/current/networking/private-links/aws-privatelink.html#aws-privatelink-register) to your cluster. Provide a name of your choice. For the AWS (Amazon Web Services) account number, provide the value of the `awsAccountId` field that you saved previously. Save the value of the VPC Endpoint service name for further use in this procedure. 4. Create an Atlas Private Endpoint. The Atlas Administration API provides an endpoint for requesting a Private Link connection configured for Atlas Stream Processing. [Create One Private Link](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-creategroupstreamprivatelinkconnection) For an AWS (Amazon Web Services) Confluent Dedicated cluster Private Link connection, you must set the following key-value pairs: | Key | Value | | --- | --- | | `serviceEndpointId` | Your cluster's VPC Endpoint service name. | | `dnsDomain` | Fully qualified domain name of the bootstrap server on your Confluent cluster. | | `dnsSubDomain` | If your cluster doesn't use subdomains, you must set this to the empty array `[]`.; If your cluster uses subdomains, you must set this to an array containing one fully qualified subdomain name for each of your cluster's subdomains. | You can find these values in your Confluent cluster's networking details. The following example command requests a connection to your Confluent cluster and illustrates a typical response: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```sh curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/privateLinkConnections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "vendor": "Confluent", "provider": "AWS", "region": "us_east_1", "serviceEndpointId": "com.amazonaws.vpce.us-east-1.vpce-svc-93da685022ee702a9", "dnsDomain": "sample.us-east-1.aws.confluent.cloud", "dnsSubDomain: [ "use1-az1.sample.us-east-1.aws.confluent.cloud", "use1-az2.sample.us-east-1.aws.confluent.cloud", "use1-az4.sample.us-east-1.aws.confluent.cloud" ] }' ``` **Output:** ```bash {"_id":"6aa12e7ccd660d4b2380b1c1","dnsDomain":"sample.us-east-1.aws.confluent.cloud.","vendor":"Confluent","provider":"AWS","region":"us_east_1","serviceEndpointId":"com.amazonaws.vpce.us-east-1.vpce-svc-93da685022ee702a9"} ``` After you send the request, save the value of the `_id` field in the response body for further use in this procedure. Atlas generates a private endpoint ID and returns it as the `_id` field in the response body. ### Create the Atlas-Side Connection [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). AWS (Amazon Web Services) Confluent PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ## Add an AWS (Amazon Web Services) Confluent Private Link Connection through the Atlas CLI **Important:** You can't have more than one Private Link connection to a given Confluent cluster per Atlas project. Before you begin this procedure, call the [Return All Private Link Connections](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-listgroupstreamprivatelinkconnections) endpoint. If you have an existing Private Link connection to your Confluent cluster within Atlas but not within your Confluent account, only perform those steps that configure your Confluent-side networking. ### Create the Private Endpoint ### Dedicated To create a connection to an AWS (Amazon Web Services) Confluent Dedicated cluster for use in your Atlas Stream Processing project, follow these steps: 1. Get the AWS (Amazon Web Services) account ID for your Atlas project. You must configure your Confluent cluster to accept incoming connections from your Atlas project. IMPORTANT: Confluent accepts incoming connections only from AWS (Amazon Web Services). To use a Confluent Private Link connection, you must host your stream processing workspaces on AWS (Amazon Web Services). Call the [atlas api streams getAccountDetails](https://www.mongodb.com/docs/atlas/cli/command/atlas-api-streams-getAccountDetails.md#std-label-atlas-api-streams-getAccountDetails) command. Save the value of `awsAccountId` for further use in this procedure. 2. Go to your Confluent cluster networking details. In your Confluent account: Navigate to the cluster you want to connect to. In your cluster networking interface, navigate to your cluster networking details. 3. Add Private Link Access to your Confluent cluster. [Add Private Link Access](https://docs.confluent.io/cloud/current/networking/private-links/aws-privatelink.html#aws-privatelink-register) to your cluster. Provide a name of your choice. For the AWS (Amazon Web Services) account number, provide the value of the `awsAccountId` field that you saved previously. Save the value of the VPC Endpoint service name for further use in this procedure. 4. Create an Atlas Private Endpoint. Call the [atlas api streams createPrivateLinkConnection](https://www.mongodb.com/docs/atlas/cli/command/atlas-api-streams-createPrivateLinkConnection.md#std-label-atlas-api-streams-createPrivateLinkConnection) command. For an AWS (Amazon Web Services) Confluent Dedicated cluster Private Link connection, you must set the following key-value pairs: | Key | Value | | --- | --- | | `serviceEndpointId` | Your cluster's VPC Endpoint service name. | | `dnsDomain` | Fully qualified domain name of the bootstrap server on your Confluent cluster. | | `dnsSubDomain` | If your cluster doesn't use subdomains, you must set this to the empty array `[]`.; If your cluster uses subdomains, you must set this to an array containing one fully qualified subdomain name for each of your cluster's subdomains. | You can find these values in your Confluent cluster's networking details. The following example command requests a connection to your Confluent cluster and illustrates a typical response: After you run the command, save the value of the `_id` field in the response body for further use in this procedure. Atlas generates a private endpoint ID and returns it as the `_id` field in the response body. ### Create the Atlas-Side Connection [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). AWS (Amazon Web Services) Confluent PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. ## Add an AWS (Amazon Web Services) Confluent Private Link Connection through the Atlas UI To add an AWS (Amazon Web Services) Confluent Private Link connection to your stream processing workspace through the Atlas UI, first create the private endpoint, then create the Atlas-side connection. ### Create the Private Endpoint ### Dedicated 1. Go to your Confluent cluster networking details. In your Confluent account: Navigate to the cluster you want to connect to. In your cluster networking interface, navigate to your cluster networking details. 2. Add Private Link Access to your Confluent cluster. [Add Private Link Access](https://docs.confluent.io/cloud/current/networking/private-links/aws-privatelink.html#aws-privatelink-register) to your cluster. Provide a name of your choice. For the AWS (Amazon Web Services) account number, provide the value of the `awsAccountId` field that you saved previously. Save the value of the VPC Endpoint service name for further use in this procedure. 3. In Atlas, go to the Database & Network Access page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Database & Network Access under the Security heading. The [Database & Network Access](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2Fsecurity%2Fnetwork%2FaccessList) page displays. 4. Go to the Atlas Stream Processing private endpoint interface. In the sidebar, click Private Endpoint. Click the Atlas Stream Processing tab. Click Create endpoint. 5. Select your cloud provider and vendor. Set Cloud Provider to AWS. Set Vendor to Confluent Cloud. Click Next, enter service details. 6. Provide your service details. Select the Dedicated Cluster Type. Set Service Endpoint ID to the VPC Endpoint service name you noted earlier. Set the Endpoint region to a region in which you have an Atlas cluster. Set the DNS Domain to the fully qualified domain name of the bootstrap server on your Confluent cluster. Select the DNS Subdomain(s) option suitable to your use case and apply any subdomain entries as necessary. - If your cluster doesn't use subdomains, select a Single availability zone. - If your cluster uses subdomains, select Multi-availability zones and provide the fully-qualified subdomain name of each subdomain. 7. Click Next, generate endpoint ID. Atlas generates a private endpoint ID. You may now view your AWS (Amazon Web Services) Confluent private endpoint's details in the Network Access interface under the Atlas Stream Processing tab by clicking the View button in the corresponding row. ### Create the Atlas-Side Connection 1. In Atlas, go to the Stream Processing page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Stream Processing under the Streaming Data heading. The [Stream Processing](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2FstreamProcessing) page displays. 2. Add a new connection. Locate the overview panel of the stream processing workspace you want to modify and click Manage. Select the Connection Registry tab. Click + Add Connection. Select a Kafka connection. 3. Provide your connection details. AWS (Amazon Web Services) Confluent PrivateLink connections support either the `SASL_SSL` or the `SSL` Security Protocol Method. Select the tab for the method you want to use and provide the following values: ### SASL\_SSL | Field | Necessity | Description | | --- | --- | --- | | Connection Name | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. This is the name used to reference the connection in Atlas Stream Processing [aggregations.](https://www.mongodb.com/docs/atlas/atlas-stream-processing/stream-aggregation-stages.md#std-label-atlas-sp-aggregation) | | Network Access | Required | Select PrivateLink, toggle Enable PrivateLink networking on, and select the private endpoint you created earlier from the dropdown menu. | | Bootstrap Servers | Required | One or more [bootstrap servers](https://kafka.apache.org/documentation/#streamsconfigs_bootstrap.servers) for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | Security Protocol Method | Required | Set to `SASL_SSL`. | | SASL Mechanism | Required | Authentication mechanism to use. Atlas Stream Processing supports: `PLAIN`; `SCRAM-SHA-256`; `SCRAM-SHA-512`; `OAUTHBEARER`; `AWS MSK IAM` If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `SCRAM-SHA-256` and `SCRAM-SHA-512` mechanisms aren't available. For AWS MSK private endpoints, Atlas Stream Processing also supports the `AWS MSK IAM` mechanism. | | Certificate Authority PEM file | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. Upload your certificate if you aren't using the default one. | | Username | Conditional | Username for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Password | Conditional | Password for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Client ID | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Client secret | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Token endpoint URL | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Scope | Optional | Scope of the access request to the broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. Use commas to separate multiple scopes. | | Extensions | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. | | IAM Role ARN | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set SASL Mechanism to `AWS MSK IAM`. Applies only to AWS MSK connections. | | Configuration File | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | 4. Click Add connection. ## Add a Microsoft Azure Confluent Private Link Connection through the Atlas Administration API To add a Microsoft Azure Confluent Private Link connection to your stream processing workspace through the Atlas Administration API, follow these steps: 1. Configure Confluent cluster. Call the `streams/accountDetails` endpoint to get your Atlas project's Azure (Microsoft Azure) subscription ID: ```bash curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups//streams/accountDetails?cloudProvider=azure®ionName=' \ --header 'Accept: application/vnd.atlas.2024-11-13+json' { "azureSubscriptionId": "f1a2b3c4-d5e6-87a8-a9b0-c1d2e3f4a5b6", "cidrBlock": "192.168.123.0/21", "virtualNetworkName": "vnet_a1b2c3d4e5f6a7b8c9d0e1f2_xyz987ab", "cloudProvider": "azure" } ``` 2. Navigate to the cluster you want to connect to. In your Confluent account, navigate to the cluster you want to connect to. 3. Navigate to your cluster networking details. In your cluster networking interface, navigate to your cluster networking details. 4. Add PrivateLink Acess. Follow the procedure provided in the [Confluent documentation](https://docs.confluent.io/cloud/current/networking/private-links/azure-privatelink.html#add-a-private-link-access-in-ccloud) to add PrivateLink access. **Note:** You need to provide your `azureSubscriptionId`. 5. Create an Atlas Private Endpoint. | Key | Value | | --- | --- | | region | Region of the Confluent cluster | | dnsDomain | The DNS domain of your cluster's network. Eg: `abcxyz12345.eastus2.azure.confluent.cloud` | | azureResourceIds | The resource ID for the Confluent Cloud Private Link service endpoint in each Availability Zone (AZ) used by your cluster's network. Multi-AZ Clusters: Include the unique Resource ID corresponding to each Availability Zone where your cluster has Private Link enabled.; Single-AZ Clusters: Provide only the single Resource ID for the specific Availability Zone used. | **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```text curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/privateLinkConnections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2024-11-13+json' \ --data '{ "vendor": "Confluent", "provider": "Azure", "region": "US_EAST_2", "dnsDomain": "abcxyz12345.eastus2.azure.confluent.cloud", "azureResourceIds: [ "/subscriptions/a1b2c3d4-e5f6-7890-abcd-ef1234567890/resourceGroups/d-xyz98/providers/Microsoft.Network/privateLinkServices/d-xyz98-privatelink-1", "/subscriptions/a1b2c3d4-e5f6-7890-abcd-ef1234567890/resourceGroups/d-xyz98/providers/Microsoft.Network/privateLinkServices/d-xyz98-privatelink-2", "/subscriptions/a1b2c3d4-e5f6-7890-abcd-ef1234567890/resourceGroups/d-xyz98/providers/Microsoft.Network/privateLinkServices/d-xyz98-privatelink-3" ] }' ``` **Output:** ```text { "_id": "65f8a3b4c5d6e7f8a9b0c1d2", "azureResourceIds": [ "/subscriptions/a1b2c3d4-e5f6-7890-abcd-ef1234567890/resourceGroups/d-xyz98/providers/Microsoft.Network/privateLinkServices/d-xyz98-privatelink-1", "/subscriptions/a1b2c3d4-e5f6-7890-abcd-ef1234567890/resourceGroups/d-xyz98/providers/Microsoft.Network/privateLinkServices/d-xyz98-privatelink-2", "/subscriptions/a1b2c3d4-e5f6-7890-abcd-ef1234567890/resourceGroups/d-xyz98/providers/Microsoft.Network/privateLinkServices/d-xyz98-privatelink-3" ], "dnsDomain": "abcxyz12345.eastus2.azure.confluent.cloud", "provider": "Azure", "region": "US_EAST_2", "vendor": "Confluent" } ``` 6. Create the Atlas-side connection. [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. The following example command creates a [Apache Kafka](https://kafka.apache.org/) connection in Atlas using `SASL_SSL` with the `SCRAM-512` mechanism: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```sh curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/spinstance/connections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "name": "confluent_demo", "bootstrapServers": "slr-ntgrbn.sample.us-east-1.aws.confluent.cloud:9092", "security": { "protocol": "SASL_SSL" }, "authentication": { "mechanism": "SCRAM-512", "password": "apiSecretDemo", "username": "apiUserDemo" }, "type": "Kafka", "networking": { "access": { "type": "PRIVATE_LINK", "connectionId": "38972b0cbe9c2aa40a30a246" } } }' ``` ## Add a Microsoft Azure Confluent Private Link Connection through the Atlas CLI To add a Microsoft Azure Confluent Private Link connection to your stream processing workspace through the Atlas CLI, follow these steps: 1. Get the Microsoft Azure subscription ID for your project. You must configure your Confluent cluster to accept incoming connections from your Atlas project. Call the [atlas api streams getAccountDetails](https://www.mongodb.com/docs/atlas/cli/command/atlas-api-streams-getAccountDetails.md#std-label-atlas-api-streams-getAccountDetails) command. Save the value of `azureSubscriptionId` for further use in this procedure. 2. Go to your Confluent cluster networking details. In your Confluent account: Navigate to the cluster you want to connect to. In your cluster networking interface, navigate to your cluster networking details. 3. Add PrivateLink access to your Confluent cluster. Follow the procedure provided in the [Confluent documentation](https://docs.confluent.io/cloud/current/networking/private-links/azure-privatelink.html#add-a-private-link-access-in-ccloud) to add PrivateLink access. Provide the value of the `azureSubscriptionId` field that you saved previously. Save the DNS domain of your cluster's network and the resource ID of the Private Link service endpoint in each availability zone for further use in this procedure. 4. Create an Atlas Private Endpoint. Call the [atlas api streams createPrivateLinkConnection](https://www.mongodb.com/docs/atlas/cli/command/atlas-api-streams-createPrivateLinkConnection.md#std-label-atlas-api-streams-createPrivateLinkConnection) command. For a Microsoft Azure Confluent Private Link connection, you must set the following key-value pairs: | Key | Value | | --- | --- | | `vendor` | `"Confluent"` | | `provider` | `"Azure"` | | `region` | Region of the Confluent cluster. | | `dnsDomain` | The DNS domain of your cluster's network that you saved previously. | | `azureResourceIds` | The resource ID for the Confluent Cloud Private Link service endpoint in each Availability Zone (AZ) used by your cluster's network. Multi-AZ Clusters: Include the unique Resource ID corresponding to each Availability Zone where your cluster has Private Link enabled.; Single-AZ Clusters: Provide only the single Resource ID for the specific Availability Zone used. | You can find these values in your Confluent cluster's networking details. After you run the command, save the value of the `_id` field in the response body for further use in this procedure. 5. Create the Atlas-side connection. [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. ## Add a Microsoft Azure Confluent Private Link Connection through the Atlas UI To add a Microsoft Azure Confluent Private Link connection to your stream processing workspace through the Atlas UI, first create the private endpoint, then create the Atlas-side connection. ### Create the Private Endpoint 1. Configure your Confluent cluster. In your Confluent account: Navigate to the cluster you want to connect to. In your cluster networking interface, navigate to your cluster networking details. 2. Add PrivateLink access to your Confluent cluster. Follow the procedure provided in the [Confluent documentation](https://docs.confluent.io/cloud/current/networking/private-links/azure-privatelink.html#add-a-private-link-access-in-ccloud) to add PrivateLink access. Note the following values for further use in this procedure: - The DNS domain of your cluster's network. - The resource ID of the Confluent Cloud Private Link service endpoint in each availability zone used by your cluster's network. 3. In Atlas, go to the Database & Network Access page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Database & Network Access under the Security heading. The [Database & Network Access](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2Fsecurity%2Fnetwork%2FaccessList) page displays. 4. Go to the Atlas Stream Processing private endpoint interface. In the sidebar, click Private Endpoint. Click the Atlas Stream Processing tab. Click Create endpoint. 5. Select your cloud provider and vendor. Set Cloud Provider to Azure. Set Vendor to Confluent Cloud. Click Next, enter service details. 6. Provide your service details. Set the Endpoint region to the region that hosts your Confluent cluster. Set the DNS Domain to the DNS domain of your cluster's network that you noted earlier. Select the Cluster availability zones option suitable to your use case. - If your cluster uses a single availability zone, select Single availability zone. - If your cluster uses multiple availability zones, select Multi-availability zones. Set Private Link Service resource ID(s) to the resource IDs that you noted earlier. For a multi-availability zone cluster, provide the resource ID for each availability zone. For a single availability zone cluster, provide only the single resource ID. 7. Click Next, generate endpoint ID. Atlas generates a private endpoint ID. You may now view your Microsoft Azure Confluent private endpoint's details in the Network Access interface under the Atlas Stream Processing tab by clicking the View button in the corresponding row. ### Create the Atlas-Side Connection 1. In Atlas, go to the Stream Processing page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Stream Processing under the Streaming Data heading. The [Stream Processing](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2FstreamProcessing) page displays. 2. Add a new connection. Locate the overview panel of the stream processing workspace you want to modify and click Manage. Select the Connection Registry tab. Click + Add Connection. Select a Kafka connection. 3. Provide your connection details. [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` Security Protocol Method. Select the tab for the method you want to use and provide the following values: ### SASL\_SSL | Field | Necessity | Description | | --- | --- | --- | | Connection Name | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. This is the name used to reference the connection in Atlas Stream Processing [aggregations.](https://www.mongodb.com/docs/atlas/atlas-stream-processing/stream-aggregation-stages.md#std-label-atlas-sp-aggregation) | | Network Access | Required | Select PrivateLink, toggle Enable PrivateLink networking on, and select the private endpoint you created earlier from the dropdown menu. | | Bootstrap Servers | Required | One or more [bootstrap servers](https://kafka.apache.org/documentation/#streamsconfigs_bootstrap.servers) for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | Security Protocol Method | Required | Set to `SASL_SSL`. | | SASL Mechanism | Required | Authentication mechanism to use. Atlas Stream Processing supports: `PLAIN`; `SCRAM-SHA-256`; `SCRAM-SHA-512`; `OAUTHBEARER`; `AWS MSK IAM` If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `SCRAM-SHA-256` and `SCRAM-SHA-512` mechanisms aren't available. For AWS MSK private endpoints, Atlas Stream Processing also supports the `AWS MSK IAM` mechanism. | | Certificate Authority PEM file | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. Upload your certificate if you aren't using the default one. | | Username | Conditional | Username for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Password | Conditional | Password for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Client ID | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Client secret | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Token endpoint URL | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Scope | Optional | Scope of the access request to the broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. Use commas to separate multiple scopes. | | Extensions | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. | | IAM Role ARN | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set SASL Mechanism to `AWS MSK IAM`. Applies only to AWS MSK connections. | | Configuration File | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | 4. Click Add connection. ## Add a Google Cloud Confluent Private Service Connect Connection through the Atlas Administration API To add a Google Cloud Confluent Private Service Connect connection to your stream processing workspace through the Atlas Administration API, follow these steps: 1. Configure Confluent cluster. Call the `streams/accountDetails` endpoint to get your Atlas project's Google Cloud project ID: ```bash curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups//streams/accountDetails?cloudProvider=gcp®ionName=' \ --header 'Accept: application/vnd.atlas.2024-11-13+json' { "cidrBlock": "192.168.123.0/21", "gcpProjectId": "f1a2b3c4d5e687a8a9b0c1d2e3f4a5b6", "vpcNetworkName": "nt-a1b2c3d4e5f6a7b8c9d0e1f2-xyz987ab", "cloudProvider": "gcp" } ``` Note the value of `gcpProjectId` for a later step. 2. Retrieve the Service Attachment URIs from Confluent. 1. In your Confluent account, [provision a Google Cloud Private Service Connect network.](https://docs.confluent.io/cloud/current/networking/private-links/gcp-private-service-connect.html) 2. Navigate to the Ingress connections tab for your network. 3. Note the value of DNS domain. You will need this value for a later step. 4. Click the + Private Service Connect Access button. 5. Provide a name for the connection. 6. In the GCP Project ID field, provide the `gcpProjectId` which you noted in the `getAccountDetails` response. 7. Under Step 2, Note the service attachment URIs for a later step. For multi-zone clusters, note all three URIs. For a single-zone cluster, navigate to the Cluster settings page for your cluster. Under Cloud details, note the value of Zones. Note the value of the service attachment URI belonging to that zone. 3. Click Add. 4. Create an Atlas Private Endpoint. | Key | Value | | --- | --- | | `region` | Region of the Confluent cluster | | `dnsDomain` | The DNS domain of your cluster's network. Eg: `abcxyz12345.us-central1.gcp.confluent.cloud` | | `gcpServiceAttachmentUris` | The service attachment URIs provided by Confluent during Private Service Connect configuration Multi-Zone Clusters: Include the unique URI corresponding to each Availability Zone where your cluster has Private Link enabled.; Single-AZ Clusters: Provide only the single URI for the specific Availability Zone where your cluster is deployed. | The following example illustrates a Multi-Zone cluster. **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```text curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/privateLinkConnections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2024-11-13+json' \ --data '{ "dnsDomain": "abcxyz12345.us-central1.gcp.confluent.cloud", "gcpServiceAttachmentUris: [ "projects/demo-proj/regions/us-central1/serviceAttachments/l-ipsum-service-attachment-us-central1-a", "projects/demo-proj/regions/us-central1/serviceAttachments/l-ipsum-service-attachment-us-central1-b", "projects/demo-proj/regions/us-central1/serviceAttachments/l-ipsum-service-attachment-us-central1-c" ] "provider": "gcp", "region": "us-central1", "vendor": "CONFLUENT", }' ``` **Output:** ```text { "_id": "65f8a3b4c5d6e7f8a9b0c1d2", "dnsDomain": "abcxyz12345.us-central1.gcp.confluent.cloud", "gcpServiceAttachmentUris: [ "projects/demo-proj/regions/us-central1/serviceAttachments/l-ipsum-service-attachment-us-central1-a", "projects/demo-proj/regions/us-central1/serviceAttachments/l-ipsum-service-attachment-us-central1-b", "projects/demo-proj/regions/us-central1/serviceAttachments/l-ipsum-service-attachment-us-central1-c" ] "provider": "gcp", "region": "us-central1", "vendor": "CONFLUENT", } ``` 5. Create the Atlas-side connection. [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. The following example command creates a [Apache Kafka](https://kafka.apache.org/) connection in Atlas using `SASL_SSL` with the `SCRAM-512` mechanism: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```sh curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/spinstance/connections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "name": "confluent_demo", "bootstrapServers": "slr-ntgrbn.sample.us-east-1.aws.confluent.cloud:9092", "security": { "protocol": "SASL_SSL" }, "authentication": { "mechanism": "SCRAM-512", "password": "apiSecretDemo", "username": "apiUserDemo" }, "type": "Kafka", "networking": { "access": { "type": "PRIVATE_LINK", "connectionId": "38972b0cbe9c2aa40a30a246" } } }' ``` ## Add a Google Cloud Confluent Private Service Connect Connection through the Atlas CLI To add a Google Cloud Confluent Private Service Connect connection to your stream processing workspace through the Atlas CLI, follow these steps: 1. Get the Google Cloud project ID for your Atlas project. You must configure your Confluent cluster to accept incoming connections from your Atlas project. Call the [atlas api streams getAccountDetails](https://www.mongodb.com/docs/atlas/cli/command/atlas-api-streams-getAccountDetails.md#std-label-atlas-api-streams-getAccountDetails) command. Save the value of `gcpProjectId` for further use in this procedure. 2. Retrieve the Service Attachment URIs from Confluent. 1. In your Confluent account, [provision a Google Cloud Private Service Connect network.](https://docs.confluent.io/cloud/current/networking/private-links/gcp-private-service-connect.html) 2. Navigate to the Ingress connections tab for your network. 3. Note the value of DNS domain. You will need this value for a later step. 4. Click the + Private Service Connect Access button. 5. Provide a name for the connection. 6. In the GCP Project ID field, provide the `gcpProjectId` value that you saved previously. 7. Under Step 2, note the service attachment URIs for a later step. For multi-zone clusters, note all three URIs. For a single-zone cluster, navigate to the Cluster settings page for your cluster. Under Cloud details, note the value of Zones. Note the value of the service attachment URI belonging to that zone. 8. Click Add. 3. Create an Atlas Private Endpoint. Call the [atlas api streams createPrivateLinkConnection](https://www.mongodb.com/docs/atlas/cli/command/atlas-api-streams-createPrivateLinkConnection.md#std-label-atlas-api-streams-createPrivateLinkConnection) command. For a Google Cloud Confluent Private Service Connect connection, you must set the following key-value pairs: | Key | Value | | --- | --- | | `vendor` | `"CONFLUENT"` | | `provider` | `"gcp"` | | `region` | Region of the Confluent cluster. | | `dnsDomain` | The DNS domain of your cluster's network that you noted earlier. | | `gcpServiceAttachmentUris` | The service attachment URIs provided by Confluent during Private Service Connect configuration. Multi-Zone Clusters: Include the unique URI corresponding to each Availability Zone where your cluster has Private Link enabled.; Single-Zone Clusters: Provide only the single URI for the specific Availability Zone where your cluster is deployed. | You can find these values in your Confluent cluster's networking details. After you run the command, save the value of the `_id` field in the response body for further use in this procedure. 4. Create the Atlas-side connection. [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. ## Add a Google Cloud Confluent Private Service Connect Connection through the Atlas UI To add a Google Cloud Confluent Private Service Connect connection to your stream processing workspace through the Atlas UI, first create the private endpoint, then create the Atlas-side connection. ### Create the Private Endpoint 1. In Atlas, go to the Database & Network Access page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Database & Network Access under the Security heading. The [Database & Network Access](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2Fsecurity%2Fnetwork%2FaccessList) page displays. 2. Go to the Atlas Stream Processing private endpoint interface. In the sidebar, click Private Endpoint. Click the Atlas Stream Processing tab. Click Create endpoint. 3. Select your cloud provider and vendor. Set Cloud Provider to GCP. Set Vendor to Confluent Cloud. Click Next, enter service details. 4. Select your Google Cloud region. Set GCP Region to the region that hosts your Confluent cluster. Atlas uses this region to generate a GCP Project ID. You must have an Atlas cluster in the region you select. 5. Note your GCP Project ID. Atlas displays a GCP Project ID. Save this value for further use in this procedure. 6. Retrieve the Service Attachment URIs from Confluent. 1. In your Confluent account, [provision a Google Cloud Private Service Connect network.](https://docs.confluent.io/cloud/current/networking/private-links/gcp-private-service-connect.html) 2. Navigate to the Ingress connections tab for your network. 3. Note the value of DNS domain. You will need this value for a later step. 4. Click the + Private Service Connect Access button. 5. Provide a name for the connection. 6. In the GCP Project ID field, provide the GCP Project ID that you saved previously. 7. Under Step 2, note the service attachment URIs for a later step. For multi-zone clusters, note all three URIs. For a single-zone cluster, navigate to the Cluster settings page for your cluster. Under Cloud details, note the value of Zones. Note the value of the service attachment URI belonging to that zone. 8. Click Add. 7. Return to Atlas and provide your service details. Select the cluster type that matches your Confluent cluster. Atlas Stream Processing supports Single-zone cluster, Multi-zone cluster, and Serverless. Set Service Attachment URI(s) to the service attachment URIs that you noted earlier. For a multi-zone cluster, provide all three URIs. For a single-zone or serverless cluster, provide the single URI. Set the DNS domain to the DNS domain of your cluster's network that you noted earlier. 8. Click Next, generate endpoint ID. Atlas generates a private endpoint ID. You may now view your Google Cloud Confluent private endpoint's details in the Network Access interface under the Atlas Stream Processing tab by clicking the View button in the corresponding row. ### Create the Atlas-Side Connection 1. In Atlas, go to the Stream Processing page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Stream Processing under the Streaming Data heading. The [Stream Processing](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2FstreamProcessing) page displays. 2. Add a new connection. Locate the overview panel of the stream processing workspace you want to modify and click Manage. Select the Connection Registry tab. Click + Add Connection. Select a Kafka connection. 3. Provide your connection details. [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` Security Protocol Method. Select the tab for the method you want to use and provide the following values: ### SASL\_SSL | Field | Necessity | Description | | --- | --- | --- | | Connection Name | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. This is the name used to reference the connection in Atlas Stream Processing [aggregations.](https://www.mongodb.com/docs/atlas/atlas-stream-processing/stream-aggregation-stages.md#std-label-atlas-sp-aggregation) | | Network Access | Required | Select PrivateLink, toggle Enable PrivateLink networking on, and select the private endpoint you created earlier from the dropdown menu. | | Bootstrap Servers | Required | One or more [bootstrap servers](https://kafka.apache.org/documentation/#streamsconfigs_bootstrap.servers) for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | Security Protocol Method | Required | Set to `SASL_SSL`. | | SASL Mechanism | Required | Authentication mechanism to use. Atlas Stream Processing supports: `PLAIN`; `SCRAM-SHA-256`; `SCRAM-SHA-512`; `OAUTHBEARER`; `AWS MSK IAM` If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `SCRAM-SHA-256` and `SCRAM-SHA-512` mechanisms aren't available. For AWS MSK private endpoints, Atlas Stream Processing also supports the `AWS MSK IAM` mechanism. | | Certificate Authority PEM file | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. Upload your certificate if you aren't using the default one. | | Username | Conditional | Username for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Password | Conditional | Password for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Client ID | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Client secret | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Token endpoint URL | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Scope | Optional | Scope of the access request to the broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. Use commas to separate multiple scopes. | | Extensions | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. | | IAM Role ARN | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set SASL Mechanism to `AWS MSK IAM`. Applies only to AWS MSK connections. | | Configuration File | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | 4. Click Add connection. ## Add an AWS MSK Private Link Connection through the Atlas Administration API To add an AWS MSK Private Link connection to your stream processing workspace through the Atlas Administration API, follow these steps: 1. Configure your AWS MSK cluster. You must configure your AWS MSK cluster to accept incoming connections from your Atlas project. **Important:** AWS MSK accepts incoming connections from AWS (Amazon Web Services) only. To use an AWS MSK Private Link connection, you must host your stream processing workspaces on AWS (Amazon Web Services). Use the [Get Account Details](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-getgroupstreamaccountdetails) endpoint to retrieve the AWS (Amazon Web Services) account ID. You will need this value for your AWS MSK cluster policy. Sign in to the AWS (Amazon Web Services) Management Console and navigate to the AWS MSK console. Ensure that `multi-VPC connectivity` is enabled on the cluster to which you want to connect. Click Properties, Security Settings, and Edit cluster policy. Use the AWS (Amazon Web Services) account ID you retrieved earlier to construct the principal ARN, in the form `arn:aws:iam:::root`. Provide this ARN as the value of `Statement.Principal.AWS` and ensure the policy takes the following form: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:root" }, "Action": [ "kafka:CreateVpcConnection", "kafka:GetBootstrapBrokers", "kafka:DescribeCluster", "kafka:DescribeClusterV2" ], "Resource": "arn:aws:kafka:us-east-1:123456789012:cluster/testing/de8982fa-8222-4e87-8b20-9bf3cdfa1521-2" } ] } ``` 2. Create an Atlas Private Endpoint. The Atlas Administration API provides an endpoint for requesting a Private Link connection configured for Atlas Stream Processing. [Create One Private Link](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-creategroupstreamprivatelinkconnection) **Tip:** Alternatively, you can use the Atlas CLI to request a Private Link connection. To a Private Link endpoint for a stream processor connection using the Atlas CLI, run the following command: ```sh atlas streams privateLinks create [options] ``` To learn more about the command syntax and parameters, see the Atlas CLI documentation for [atlas streams privateLinks create.](https://www.mongodb.com/docs/atlas/cli/current/command/atlas-streams-privateLinks-create/) For an AWS MSK Private Link connection, you must set the following key-value pairs: | Key | Value | | --- | --- | | `vendor` | Must be set to `"msk"` | | `provider` | Must be set to `"aws"` | | `arn` | String representing the Amazon Resource Number of your AWS MSK cluster. | | `authenticationScheme` | Must be set to one of `"IAM"`, `"SASL_SCRAM"`, or `"TLS"`, depending on the authentication mechanism you plan to use for the connection. | You can find the ARN (Amazon Resource Name) in your AWS MSK cluster's details. The following example command requests a connection to your AWS MSK cluster and illustrates a typical response: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```sh curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/privateLinkConnections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "vendor": "msk", "provider": "AWS", "arn": "1235711"}' ``` **Output:** ```bash {"_id":"6aa12e7ccd660d4b2380b1c1","dnsDomain":"scram.sample.us-east-1.amazonaws.com","vendor":"msk","provider":"AWS","region":"us_east_1","serviceEndpointId":"com.amazonaws.vpce.us-east-1.vpce-svc-93da685022ee702a9"} ``` After you send the request, note the value of the `_id` field in the response body. You will need this in a later step. 3. Create the Atlas-side connection. [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. The following example command creates a [Apache Kafka](https://kafka.apache.org/) connection in Atlas using `SASL_SSL` with the `SCRAM-512` mechanism: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```sh curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/spinstance/connections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "name": "msk_demo", "bootstrapServers": "slr-ntgrbn.sample.kafka.us-east-1.amazonaws.com:9092", "security": { "protocol": "SASL_SSL" }, "authentication": { "mechanism": "SCRAM-512", "password": "apiSecretDemo", "username": "apiUserDemo" }, "type": "Kafka", "networking": { "access": { "type": "PRIVATE_LINK", "connectionId": "38972b0cbe9c2aa40a30a246" } } }' ``` ## Add an AWS MSK Private Link Connection through the Atlas CLI To add an AWS MSK Private Link connection to your stream processing workspace through the Atlas CLI, follow these steps: 1. Configure your AWS MSK cluster. You must configure your AWS MSK cluster to accept incoming connections from your Atlas project. **Important:** AWS MSK accepts incoming connections from AWS (Amazon Web Services) only. To use an AWS MSK Private Link connection, you must host your stream processing workspaces on AWS (Amazon Web Services). Use the [Get Account Details](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-getgroupstreamaccountdetails) endpoint to retrieve the AWS (Amazon Web Services) account ID. You will need this value for your AWS MSK cluster policy. Sign in to the AWS (Amazon Web Services) Management Console and navigate to the AWS MSK console. Ensure that `multi-VPC connectivity` is enabled on the cluster to which you want to connect. Click Properties, Security Settings, and Edit cluster policy. Use the AWS (Amazon Web Services) account ID you retrieved earlier to construct the principal ARN, in the form `arn:aws:iam:::root`. Provide this ARN as the value of `Statement.Principal.AWS` and ensure the policy takes the following form: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:root" }, "Action": [ "kafka:CreateVpcConnection", "kafka:GetBootstrapBrokers", "kafka:DescribeCluster", "kafka:DescribeClusterV2" ], "Resource": "arn:aws:kafka:us-east-1:123456789012:cluster/testing/de8982fa-8222-4e87-8b20-9bf3cdfa1521-2" } ] } ``` 2. Create an Atlas Private Endpoint. The Atlas Administration API provides an endpoint for requesting a Private Link connection configured for Atlas Stream Processing. [Create One Private Link](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-creategroupstreamprivatelinkconnection) **Tip:** Alternatively, you can use the Atlas CLI to request a Private Link connection. To a Private Link endpoint for a stream processor connection using the Atlas CLI, run the following command: ```sh atlas streams privateLinks create [options] ``` To learn more about the command syntax and parameters, see the Atlas CLI documentation for [atlas streams privateLinks create.](https://www.mongodb.com/docs/atlas/cli/current/command/atlas-streams-privateLinks-create/) For an AWS MSK Private Link connection, you must set the following key-value pairs: | Key | Value | | --- | --- | | `vendor` | Must be set to `"msk"` | | `provider` | Must be set to `"aws"` | | `arn` | String representing the Amazon Resource Number of your AWS MSK cluster. | | `authenticationScheme` | Must be set to one of `"IAM"`, `"SASL_SCRAM"`, or `"TLS"`, depending on the authentication mechanism you plan to use for the connection. | You can find the ARN (Amazon Resource Name) in your AWS MSK cluster's details. The following example command requests a connection to your AWS MSK cluster and illustrates a typical response: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```sh curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/privateLinkConnections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "vendor": "msk", "provider": "AWS", "arn": "1235711"}' ``` **Output:** ```bash {"_id":"6aa12e7ccd660d4b2380b1c1","dnsDomain":"scram.sample.us-east-1.amazonaws.com","vendor":"msk","provider":"AWS","region":"us_east_1","serviceEndpointId":"com.amazonaws.vpce.us-east-1.vpce-svc-93da685022ee702a9"} ``` After you send the request, note the value of the `_id` field in the response body. You will need this in a later step. 3. Create the Atlas-side connection. [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. The following example command creates a [Apache Kafka](https://kafka.apache.org/) connection in Atlas using `SASL_SSL` with the `SCRAM-512` mechanism: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```sh curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/spinstance/connections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "name": "msk_demo", "bootstrapServers": "slr-ntgrbn.sample.kafka.us-east-1.amazonaws.com:9092", "security": { "protocol": "SASL_SSL" }, "authentication": { "mechanism": "SCRAM-512", "password": "apiSecretDemo", "username": "apiUserDemo" }, "type": "Kafka", "networking": { "access": { "type": "PRIVATE_LINK", "connectionId": "38972b0cbe9c2aa40a30a246" } } }' ``` ## Add an AWS MSK Private Link Connection through the Atlas UI To add an AWS MSK Private Link connection to your stream processing workspace through the Atlas UI, first create the private endpoint, then create the Atlas-side connection. ### Create the Private Endpoint 1. In Atlas, go to the Database & Network Access page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Database & Network Access under the Security heading. The [Database & Network Access](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2Fsecurity%2Fnetwork%2FaccessList) page displays. 2. Go to the Atlas Stream Processing private endpoint interface. In the sidebar, click Private Endpoint. Click the Atlas Stream Processing tab. Click Create endpoint. 3. Select your cloud provider and vendor. Set Cloud Provider to AWS. Set Vendor to MSK. Click Next, enter service details. 4. Provide your service details. In the Authentication Scheme select, choose the option that you'd like to use to authenticate with your AWS MSK cluster. Provide your [AWS MSK cluster ARN.](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-account-settings.html#ecs-resource-ids) 5. Click Next, generate endpoint ID. Atlas generates a private endpoint ID. You may now view your AWS (Amazon Web Services) MSK private endpoint's details in the Network Access interface under the Atlas Stream Processing tab by clicking the View button in the corresponding row. ### Create the Atlas-Side Connection 1. In Atlas, go to the Stream Processing page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Stream Processing under the Streaming Data heading. The [Stream Processing](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2FstreamProcessing) page displays. 2. Add a new connection. Locate the overview panel of the stream processing workspace you want to modify and click Manage. Select the Connection Registry tab. Click + Add Connection. Select a Kafka connection. 3. Provide your connection details. [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` Security Protocol Method. Select the tab for the method you want to use and provide the following values: ### SASL\_SSL | Field | Necessity | Description | | --- | --- | --- | | Connection Name | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. This is the name used to reference the connection in Atlas Stream Processing [aggregations.](https://www.mongodb.com/docs/atlas/atlas-stream-processing/stream-aggregation-stages.md#std-label-atlas-sp-aggregation) | | Network Access | Required | Select PrivateLink, toggle Enable PrivateLink networking on, and select the private endpoint you created earlier from the dropdown menu. | | Bootstrap Servers | Required | One or more [bootstrap servers](https://kafka.apache.org/documentation/#streamsconfigs_bootstrap.servers) for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | Security Protocol Method | Required | Set to `SASL_SSL`. | | SASL Mechanism | Required | Authentication mechanism to use. Atlas Stream Processing supports: `PLAIN`; `SCRAM-SHA-256`; `SCRAM-SHA-512`; `OAUTHBEARER`; `AWS MSK IAM` If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `SCRAM-SHA-256` and `SCRAM-SHA-512` mechanisms aren't available. For AWS MSK private endpoints, Atlas Stream Processing also supports the `AWS MSK IAM` mechanism. | | Certificate Authority PEM file | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. Upload your certificate if you aren't using the default one. | | Username | Conditional | Username for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Password | Conditional | Password for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Client ID | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Client secret | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Token endpoint URL | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Scope | Optional | Scope of the access request to the broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. Use commas to separate multiple scopes. | | Extensions | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. | | IAM Role ARN | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set SASL Mechanism to `AWS MSK IAM`. Applies only to AWS MSK connections. | | Configuration File | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | 4. Click Add connection. ## Add an Azure (Microsoft Azure) EventHub Private Link Connection through the Atlas Administration API To add an Azure (Microsoft Azure) EventHub Private Link connection to your stream processing workspace through the Atlas Administration API, perform these steps: 1. Create an Atlas Private Endpoint. The Atlas Administration API provides an endpoint for requesting a Private Link connection configured for Atlas Stream Processing. [Create One Private Link](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/operation/operation-creategroupstreamprivatelinkconnection) For an Azure (Microsoft Azure) Private Link connection, you must set the following key-value pairs: | Key | Value | | --- | --- | | `serviceEndpointId` | Your EventHub namespace [endpoint.](https://learn.microsoft.com/en-us/rest/api/eventhub/namespaces/get?view=rest-eventhub-2024-01-01&tabs=HTTP) Note that this value must be the Azure Resource Manager (ARM) ID of the Event Hub namespace, not the ARM ID of an individual Event Hub. | | `dnsDomain` | Fully qualified domain name, with port number, of the bootstrap server in your Azure (Microsoft Azure) Event Hub namespace. This domain name conforms to the format described [here.](https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs?tabs=passwordless) | The following example command requests a connection to your Azure (Microsoft Azure) Event Hub and illustrates a typical response: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```bash curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/privateLinkConnections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "provider": "AZURE", "region": "US_EAST_2", "serviceEndpointId": "/subscriptions/b82d6aa0-0b0a-ffa3-7c22-e167dc44f5b0/resourceGroups/asp/providers/Microsoft.EventHub/namespaces/sample", "dnsDomain": "sample.servicebus.windows.net" }' ``` **Output:** ```sh {"_id":"6aa12e7ccd660d4b2380b1c1","dnsDomain":"sample.servicebus.windows.net","provider":"AZURE","region":"US_EAST_2","serviceEndpointId":"/subscriptions/b82d6aa0-0b0a-ffa3-7c22-e167dc44f5b0/resourceGroups/asp/providers/Microsoft.EventHub/namespaces/sample"} ``` After you send the request, note the value of the `_id` field in the response body. You will need this in a later step. 2. Accept the requested connection within your cloud provider account. For Private Link connections to Azure (Microsoft Azure), navigate to your Event Hub networking page and select the Private endpoint connections tab. In the table of connections, identify your newly requested connection and approve it. 3. Create the Atlas-side connection. [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. The following example command creates a [Apache Kafka](https://kafka.apache.org/) connection in Atlas using `SASL_SSL` with the `SCRAM-512` mechanism: **Note:** This `curl` command uses a service account access token ([OAuth 2.0](https://oauth.net/2/)) to authenticate instead of API keys. To learn more, see [Get Started with the Atlas Administration API.](https://www.mongodb.com/docs/atlas/configure-api-access.md#std-label-atlas-admin-api-access) ```sh curl --location 'https://cloud.mongodb.com/api/atlas/v2/groups/8358217d3abb5c76c3434648/streams/spinstance/connections' \ --header "Authorization: Bearer {ACCESS-TOKEN}" \ --header 'Content-Type: application/json' \ --header 'Accept: application/vnd.atlas.2023-02-01+json' \ --data '{ "name": "confluent_demo", "bootstrapServers": "slr-ntgrbn.sample.us-east-1.aws.confluent.cloud:9092", "security": { "protocol": "SASL_SSL" }, "authentication": { "mechanism": "SCRAM-512", "password": "apiSecretDemo", "username": "apiUserDemo" }, "type": "Kafka", "networking": { "access": { "type": "PRIVATE_LINK", "connectionId": "38972b0cbe9c2aa40a30a246" } } }' ``` ## Add an Azure (Microsoft Azure) EventHub Private Link Connection through the Atlas CLI To add an Azure (Microsoft Azure) EventHub Private Link connection to your stream processing workspace through the Atlas CLI, follow these steps: 1. Create an Atlas Private Endpoint. Call the [atlas api streams createPrivateLinkConnection](https://www.mongodb.com/docs/atlas/cli/command/atlas-api-streams-createPrivateLinkConnection.md#std-label-atlas-api-streams-createPrivateLinkConnection) command. For an Azure (Microsoft Azure) EventHub Private Link connection, you must set the following key-value pairs: | Key | Value | | --- | --- | | `provider` | `"AZURE"` | | `region` | Region of your Azure (Microsoft Azure) Event Hub namespace. | | `serviceEndpointId` | Your EventHub namespace [endpoint.](https://learn.microsoft.com/en-us/rest/api/eventhub/namespaces/get?view=rest-eventhub-2024-01-01&tabs=HTTP) Note that this value must be the Azure Resource Manager (ARM) ID of the Event Hub namespace, not the ARM ID of an individual Event Hub. | | `dnsDomain` | Fully qualified domain name, with port number, of the bootstrap server in your Azure (Microsoft Azure) Event Hub namespace. This domain name conforms to the format described [here.](https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs?tabs=passwordless) | After you run the command, save the value of the `_id` field in the response body for further use in this procedure. 2. Accept the requested connection within your cloud provider account. For Private Link connections to Azure (Microsoft Azure), navigate to your Event Hub networking page and select the Private endpoint connections tab. In the table of connections, identify your newly requested connection and approve it. 3. Create the Atlas-side connection. [Add a connection](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-connection-registry.md#std-label-atlas-sp-manage-connection-add). [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` security protocol. Select the tab for the protocol you want to use and provide the following key-value pairs: ### SASL\_SSL | Key | Necessity | Value | | --- | --- | --- | | `name` | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. | | `type` | Required | `"Kafka"` | | `bootstrapServers` | Required | One or more bootstrap servers for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | `networking.access.type` | Required | `"PRIVATE_LINK"` | | `networking.access.connectionId` | Required | `_id` value from your Private Link request response. | | `security.protocol` | Required | `"SASL_SSL"` | | `security.brokerPublicCertificate` | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. | | `authentication.mechanism` | Required | One of `"PLAIN"`, `"SCRAM-256"`, `"SCRAM-512"`, `"OAUTHBEARER"`, `"AWS_MSK_IAM"`, or `""`. If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `"SCRAM-256"` and `"SCRAM-512"` mechanisms aren't available. To enable `mTLS`, set this field to an empty string `""`. For AWS MSK private endpoints, Atlas Stream Processing also supports the `"AWS_MSK_IAM"` mechanism. | | `authentication.username` | Conditional | Username for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.password` | Conditional | Password for authentication. Required if you set `authentication.mechanism` to `"PLAIN"`, `"SCRAM-256"`, or `"SCRAM-512"`. | | `authentication.method` | Conditional | `"OIDC"`. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientId` | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.clientSecret` | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.tokenEndpointUrl` | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.scope` | Optional | Scope of the access request to the broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.saslOauthbearerExtensions` | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set `authentication.mechanism` to `"OAUTHBEARER"`. | | `authentication.aws.roleArn` | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set `authentication.mechanism` to `"AWS_MSK_IAM"`. Applies only to AWS MSK connections. | | `config` | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | Set all other values as necessary. ## Add an Azure (Microsoft Azure) EventHub Private Link Connection through the Atlas UI To add an Azure (Microsoft Azure) EventHub Private Link connection to your stream processing workspace through the Atlas UI, first create the private endpoint, then create the Atlas-side connection. ### Create the Private Endpoint 1. In Atlas, go to the Database & Network Access page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Database & Network Access under the Security heading. The [Database & Network Access](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2Fsecurity%2Fnetwork%2FaccessList) page displays. 2. Navigate to the Atlas Stream Processing private endpoint interface. In the sidebar, click Private Endpoint. Click the Atlas Stream Processing tab. If you have not created an Atlas Stream Processing private endpoint previously, click Create endpoint. If you have, click Add ASP Endpoint. 3. Select your cloud provider and vendor. Set Cloud Provider to Azure. Set Vendor to EventHub. Click Next, enter service details 4. Provide your Azure EventHub endpoint details. Provide your [Azure service endpoint ID.](https://learn.microsoft.com/en-us/rest/api/eventhub/namespaces/get?view=rest-eventhub-2024-01-01\&tabs=HTTP) Select your Endpoint region. Select your Host name. Click Next, generate endpoint ID 5. Approve the connection request within EventHub. Log in to your Azure (Microsoft Azure) account. Go to your EventHub namespace. Click Settings, Networking, Private endpoint connections. Select and approve your incoming request. The request ID value appears in the Private endpoint column. You may now view your Azure (Microsoft Azure) EventHub private endpoint in the Network Access interface under the Atlas Stream Processing tab by clicking the View button in its row. ### Create the Atlas-Side Connection 1. In Atlas, go to the Stream Processing page for your project. If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar. If it's not already displayed, select your project from the Projects menu in the navigation bar. In the sidebar, click Stream Processing under the Streaming Data heading. The [Stream Processing](https://cloud.mongodb.com/go?l=https%3A%2F%2Fcloud.mongodb.com%2Fv2%2F%3Cproject%3E%23%2FstreamProcessing) page displays. 2. Add a new connection. Locate the overview panel of the stream processing workspace you want to modify and click Manage. Select the Connection Registry tab. Click + Add Connection. Select a Kafka connection. 3. Provide your connection details. [Apache Kafka](https://kafka.apache.org/) PrivateLink connections support either the `SASL_SSL` or the `SSL` Security Protocol Method. Select the tab for the method you want to use and provide the following values: ### SASL\_SSL | Field | Necessity | Description | | --- | --- | --- | | Connection Name | Required | Name for the connection. Each connection name must be unique within a stream processing workspace. This is the name used to reference the connection in Atlas Stream Processing [aggregations.](https://www.mongodb.com/docs/atlas/atlas-stream-processing/stream-aggregation-stages.md#std-label-atlas-sp-aggregation) | | Network Access | Required | Select PrivateLink, toggle Enable PrivateLink networking on, and select the private endpoint you created earlier from the dropdown menu. | | Bootstrap Servers | Required | One or more [bootstrap servers](https://kafka.apache.org/documentation/#streamsconfigs_bootstrap.servers) for your [Apache Kafka](https://kafka.apache.org/) system. Each server must include a port, as in `broker.example.com:9092`. To specify multiple servers, separate them with commas. | | Security Protocol Method | Required | Set to `SASL_SSL`. | | SASL Mechanism | Required | Authentication mechanism to use. Atlas Stream Processing supports: `PLAIN`; `SCRAM-SHA-256`; `SCRAM-SHA-512`; `OAUTHBEARER`; `AWS MSK IAM` If your AWS MSK private endpoint uses the `TLS` authentication scheme, the `SCRAM-SHA-256` and `SCRAM-SHA-512` mechanisms aren't available. For AWS MSK private endpoints, Atlas Stream Processing also supports the `AWS MSK IAM` mechanism. | | Certificate Authority PEM file | Optional | Trusted public x509 certificate for connecting to [Apache Kafka](https://kafka.apache.org/) over SSL. Upload your certificate if you aren't using the default one. | | Username | Conditional | Username for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Password | Conditional | Password for authentication. Required if you set SASL Mechanism to `PLAIN`, `SCRAM-SHA-256`, or `SCRAM-SHA-512`. | | Client ID | Conditional | Public identifier for the [Apache Kafka](https://kafka.apache.org/) client. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Client secret | Conditional | Secret known only to the [Apache Kafka](https://kafka.apache.org/) client and the authorization server. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Token endpoint URL | Conditional | OAUTH issuer token endpoint URI used to retrieve the token. Required if you set SASL Mechanism to `OAUTHBEARER`. | | Scope | Optional | Scope of the access request to the broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. Use commas to separate multiple scopes. | | Extensions | Optional | Additional information to provide to the [Apache Kafka](https://kafka.apache.org/) broker. Applies only when you set SASL Mechanism to `OAUTHBEARER`. | | IAM Role ARN | Conditional | ARN of the AWS (Amazon Web Services) IAM role that Atlas assumes to authenticate with your MSK cluster. Required if you set SASL Mechanism to `AWS MSK IAM`. Applies only to AWS MSK connections. | | Configuration File | Optional | Additional [Apache Kafka](https://kafka.apache.org/) configuration properties as key-value pairs. | 4. Click Add connection.