Docs Menu
Docs Home
/
MongoDB Atlas
/
MongoDB Atlas Kubernetes Operator

Manage Atlas Stream Processing

On this page

  • Stream Processing Instances and Connections
  • Prerequisites
  • Procedure

You can use Atlas Kubernetes Operator to manage stream processing instances and connections for Atlas Stream Processing. Atlas Stream Processing enables you to process streams of complex data using the same MongoDB Query API that Atlas databases use. Atlas Stream Processing allows you to do the following tasks:

Atlas Stream Processing components belong directly to Atlas projects and operate independent of Atlas clusters. To learn more, see Atlas Stream Processing Overview.

Stream Processing Instances and Connections

Atlas Stream Processing instances provide the context for all of your operations on streaming data. You can configure a sample connection, a connection to an Atlas change stream, or a connection to an Apache Kafka system. Then you can add the connection to the Connection Registry for your stream processing instance. To learn more, see Manage Stream Processing Instances.

Prerequisites

To use Atlas Kubernetes Operator to manage stream processing instances and connections, you must:

Procedure

To use Atlas Kubernetes Operator to manage a stream processing instance and its connections, do the following steps:

1

Configure the AtlasStreamInstance Custom Resource.

Example:

cat <<EOF | kubectl apply -f -
apiVersion: atlas.mongodb.com/v1
kind: AtlasStreamInstance
metadata:
  name: my-stream-instance
spec:
  name: my-stream-instance
  clusterConfig:
    provider: AWS
    region: VIRGINIA_USA
    tier: SP30
  projectRef:
    name: my-project
EOF

To learn more about the available parameters, see the AtlasStreamInstance Custom Resource.

Note

Currently, Atlas Kubernetes Operator supports only the AWS provider and the VIRGINIA_USA region for this custom resource.

2

Configure the AtlasStreamConnection Custom Resource.

You can configure a sample connection, a connection to an Atlas change stream, or a connection to an Apache Kafka system.

Examples:

apiVersion: atlas.mongodb.com/v1
kind: AtlasStreamConnection
metadata:
  name: my-stream-connection
spec:
  name: sample_stream_solar
  type: Sample

Note

If you specify Sample for the spec.type parameter, the spec.name parameter must match the sample collection name. Currently, Atlas Kubernetes Operator supports only the sample_stream_solar sample collection for this custom resource.

apiVersion: atlas.mongodb.com/v1
kind: AtlasStreamConnection
metadata:
  name: my-stream-connection
spec:
  name: my-stream-connection
  type: Cluster
  clusterConfig:
    name: my-cluster
  role:
    name: my-db-role
    type: CUSTOM
apiVersion: atlas.mongodb.com/v1
kind: AtlasStreamConnection
metadata:
  name: my-stream-connection
spec:
  name: my-stream-connection
  type: Kafka
  kafkaConfig:
    bootstrapServers: "comma,separated,list,of,server,addresses"
    authentication:
      mechanism: SCRAM-512
      credentials:
        name: ref-to-creds-secret
        namespace: default
    security:
      protocol: SSL
      certificate:
        name: ref-to-certificate-secret
        namespace: default

To learn more about the available parameters, see the AtlasStreamConnection Custom Resource.

3

Add your connections to the Connection Registry in the AtlasStreamInstance Custom Resource.

Example:

cat <<EOF | kubectl apply -f -
apiVersion: atlas.mongodb.com/v1
kind: AtlasStreamInstance
metadata:
   name: my-stream-instance
spec:
  name: my-stream-instance
  clusterConfig:
    provider: AWS
    region: VIRGINIA_USA
    tier: SP30
  projectRef:
    name: my-project
  connectionRegistry:
    - name: ref-my-connection-1
      namespace: my-namespace1
    - name: ref-my-connection-2
      namespace: my-namespace2
    - name: ref-my-connection-3
      namespace: my-namespace1
EOF

Note

Currently, Atlas Kubernetes Operator supports only the AWS provider and the VIRGINIA_USA region for this custom resource.

To learn more about the available parameters, see the AtlasStreamInstance Custom Resource.

Back

Atlas Vector Search