Subscribe to MongoDB Change Streams Via WebSockets
Rate this tutorial
Change streams allow applications to access real-time data changes without the complexity and risk of tailing the oplog. Applications can use change streams to subscribe to all data changes on a single collection, a database, or an entire deployment, and immediately react to them. Because change streams use the aggregation framework, applications can also filter for specific changes or transform the notifications at will.
The example code in this article will assume you have a cluster running in MongoDB Atlas, but it will work with any MongoDB version from 3.6 onwards.
To allow clients to subscribe to your change stream via WebSockets, you must first create a WebSocket server. This WebSocket server, written in Python and using Tornado, proxies any new data from the change stream to your connected clients.
As clients connect and disconnect from the WebSocket server, they trigger the
open and on_close methods.When a client connects, your server stores a reference to it in the
connected_clients set. This allows it to push new data to the client when it is received from the change stream. Likewise, when a client disconnects from the server, it is removed from the set of connected clients, so your server does not try to push updates on a connection which no longer exists.It is worth noting that the server does not have a
on_message handler. As WebSockets are bi-directional, typically a WebSocket server has a on_message method. When the client sends data to the server, it invokes this method to handle the incoming message. However, as you are only using the WebSocket connection to push change stream data to the connected clients, your WebSocket connection is essentially mono-directional, and your server does not have a method for handling inbound data.When you have new data from the change stream, you pass it to the WebSocket server using the
on_change method. This method formats the change stream data into a string ready to be pushed out to the connected clients. This push occurs in the send_updates method. Within this method, you loop over all clients in the connected_clients set and use the write_message action to write the data to the client's WebSocket connection.In Motor, MongoDB Collections have a
watch() method which you can use to monitor the collection for any changes. Then, whenever there is a new change on the stream, you can use the WebSocket server's on_change method to proxy the data to the WebSocket clients.This
watch function is attached to your Tornado loop as a callback.For this example, your WebSocket client is a simple web page, and it logs any WebSocket messages to the browser's JavaScript console.
You can use Tornado to serve this web page as well.
To try the example for yourself:
- install the requirements.
- set the required environmental variables.
- run the Python script.
Once the WebSocket server is running in your terminal, open a browser window to localhost:8000 and view your JavaScript console. Then, make some changes to your Collection via Compass or the MongoDB Shell.
In this article, you have subscribed to all changes on a single collection. However, you can use change streams to subscribe to all data changes on a single collection, a database, or even an entire deployment. And, because change streams use the aggregation framework, applications can also filter for specific changes or transform the notifications.
Rate this tutorial
Related
Tutorial
Analyze Time-Series Data with Python and MongoDB Using PyMongoArrow and Pandas
Sep 21, 2023 | 6 min read
Article
The Six Principles for Building Robust Yet Flexible Shared Data Applications
Sep 23, 2022 | 3 min read