Change Streams

Availability

Change streams are available for replica sets and sharded clusters:

• Storage Engine.

  The replica sets and sharded clusters must use the WiredTiger storage engine. Change streams can also be used on deployments that employ MongoDB's encryption-at-rest feature.

• Replica Set Protocol Version.

  The replica sets and sharded clusters must use replica set protocol version 1 (pv1).

• Read Concern "majority" Enablement.

  Starting in MongoDB 4.2, change streams are available regardless of the "majority" read concern support; that is, read concern majority support can be either enabled (default) or disabled to use change streams.

  In MongoDB 4.0 and earlier, change streams are available only if "majority" read concern support is enabled (default).

Connect

Connections for a change stream can either use DNS seed lists with the +srv connection option or by listing the servers individually in the connection string.

If the driver loses the connection to a change stream or the connection goes down, it attempts to reestablish a connection to the change stream through another node in the cluster that has a matching read preference. If the driver cannot find a node with the correct read preference, it throws an exception.

For more information, see Connection String URI Format.

Watch a Collection, Database, or Deployment

You can open change streams against:

Change Stream Performance Considerations

If the amount of active change streams opened against a database exceeds the connection pool size, you may experience notification latency. Each change stream uses a connection and a getMore operation on the change stream for the period of time that it waits for the next event. To avoid any latency issues, you should ensure that the pool size is greater than the number of opened change streams. For details see the maxPoolSize setting.

Open A Change Stream

To open a change stream:

• For a replica set, you can issue the open change stream operation from any of the data-bearing members.

• For a sharded cluster, you must issue the open change stream operation from the mongos.

The following example opens a change stream for a collection and iterates over the cursor to retrieve the change stream documents. [1]

➤ Use the Select your language drop-down menu in the upper-right to set the language of the examples on this page.

To retrieve the data change event from the cursor, iterate the change stream cursor. For information on the change stream event, see Change Events.

The change stream cursor remains open until one of the following occurs:

• The cursor is explicitly closed.

• An invalidate event occurs; for example, a collection drop or rename.

• The connection to the MongoDB deployment closes or times out. See Cursor Behaviors for more information.

• If the deployment is a sharded cluster, a shard removal may cause an open change stream cursor to close, and the closed change stream cursor may not be fully resumable.

Modify Change Stream Output

➤ Use the Select your language drop-down menu in the upper-right to set the language of the examples on this page.

Lookup Full Document for Update Operations

By default, change streams only return the delta of fields during the update operation. However, you can configure the change stream to return the most current majority-committed version of the updated document.

➤ Use the Select your language drop-down menu in the upper-right to set the language of the examples on this page.

Resume a Change Stream

Change streams are resumable by specifying a resume token to either resumeAfter or startAfter when opening the cursor.

{
   "_id": {
      "_data": "82635019A0000000012B042C0100296E5A1004AB1154ACACD849A48C61756D70D3B21F463C6F7065726174696F6E54797065003C696E736572740046646F63756D656E744B65790046645F69640064635019A078BE67426D7CF4D2000004"
    },
    "operationType": "insert",
    "clusterTime": Timestamp({ "t": 1666193824, "i": 1 }),
    "collectionUUID": new UUID("ab1154ac-acd8-49a4-8c61-756d70d3b21f"),
    "fullDocument": {
       "_id": ObjectId("635019a078be67426d7cf4d2"'),
       "name": "Giovanni Verga"
    },
    "ns": {
       "db": "test",
       "coll": "names"
    },
    "documentKey": {
       "_id": ObjectId("635019a078be67426d7cf4d2")
    }
}

{
   "cursor": {
      "firstBatch": [],
      "postBatchResumeToken": {
         "_data": "8263515EAC000000022B0429296E1404"
      },
      "id": Long("4309380460777152828"),
      "ns": "test.names"
   },
   "ok": 1,
   "$clusterTime": {
      "clusterTime": Timestamp({ "t": 1666277036, "i": 1 }),
      "signature": {
         "hash": Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
         "keyId": Long("0")
      }
   },
   "operationTime": Timestamp({ "t": 1666277036, "i": 1 })
}

{
   "cursor": {
      "nextBatch": [],
      "postBatchResumeToken": {
         "_data": "8263515979000000022B0429296E1404"
      },
      "id": Long("7049907285270685005"),
      "ns": "test.names"
   },
   "ok": 1,
   "$clusterTime": {
      "clusterTime": Timestamp( { "t": 1666275705, "i": 1 } ),
      "signature": {
         "hash": Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
         "keyId": Long("0")
      }
   },
   "operationTime": Timestamp({ "t": 1666275705, "i": 1 })
}

Use Cases

Change streams can benefit architectures with reliant business systems, informing downstream systems once data changes are durable. For example, change streams can save time for developers when implementing Extract, Transform, and Load (ETL) services, cross-platform synchronization, collaboration functionality, and notification services.

Access Control

For deployments enforcing Authentication and authorization:

• To open a change stream against specific collection, applications must have privileges that grant changeStream and find actions on the corresponding collection.

  { resource: { db: <dbname>, collection: <collection> }, actions: [ "find", "changeStream" ] }

• To open a change stream on a single database, applications must have privileges that grant changeStream and find actions on all non-system collections in the database.

  { resource: { db: <dbname>, collection: "" }, actions: [ "find", "changeStream" ] }

• To open a change stream on an entire deployment, applications must have privileges that grant changeStream and find actions on all non-system collections for all databases in the deployment.

  { resource: { db: "", collection: "" }, actions: [ "find", "changeStream" ] }

Event Notification

Change streams only notify on data changes that have persisted to a majority of data-bearing members in the replica set. This ensures that notifications are triggered only by majority-committed changes that are durable in failure scenarios.

For example, consider a 3-member replica set with a change stream cursor opened against the primary. If a client issues an insert operation, the change stream only notifies the application of the data change once that insert has persisted to a majority of data-bearing members.

If an operation is associated with a transaction, the change event document includes the txnNumber and the lsid.

Collation

Starting in MongoDB 4.2, change streams use simple binary comparisons unless an explicit collation is provided. In earlier versions, change streams opened on a single collection (db.collection.watch()) would inherit that collection's default collation.