BlogAnnounced at MongoDB.local NYC 2024: A recap of all announcements and updatesLearn more >>
MongoDB Developer
Atlas
plus
Sign in to follow topics
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Productschevron-right
Atlaschevron-right

Exploring Window Operators in Atlas Stream Processing

Robert Walters4 min read • Published May 02, 2024 • Updated May 02, 2024
Stream ProcessingAtlas
Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Atlas Stream Processing is now available. Learn more about it here.
In our previous post on windowing, we introduced window operators available in Atlas Stream Processing. Window operators are one of the most commonly used operations to effectively process streaming data. Atlas Stream Processing provides two window operators: $tumblingWindow and $hoppingWindow. In this tutorial, we will explore both of these operators using the sample solar data generator provided within Atlas Stream Processing.

Getting started

Before we begin creating stream processors, make sure you have a database user who has “atlasAdmin” access to the Atlas Project. Also, if you do not already have a Stream Processing Instance created with a connection to the sample_stream_solar data generator, please follow the instructions in Get Started with Atlas Stream Processing: Creating Your First Stream Processor and then continue on.

View the solar stream sample data

For this tutorial, we will be using the MongoDB shell.
First, confirm sample_stream_solar is added as a connection by issuing sp.listConnections().
Next, let’s define a $source stage to describe where Atlas Stream Processing will read the stream data from.
Then, issue a .process command to view the contents of the stream on the console.
You will see the stream of solar data printed on the console. A sample of this data is as follows:

Create a tumbling window query

A tumbling window is a fixed-size window that moves forward in time at regular intervals. In Atlas Stream Processing, you use the $tumblingWindow operator. In this example, let’s use the operator to compute the average watts over one-minute intervals.
Refer back to the schema from the sample stream solar data. To create a tumbling window, let’s create a variable and define our tumbling window stage.
We are calculating the maximum value and average over the span of one-minute, non-overlapping intervals. Let’s use the .process command to run the streaming query in the foreground and view our results in the console.
Here is an example output of the statement:

Exploring the window operator pipeline

The pipeline that is used within a window function can include blocking stages and non-blocking stages.
Accumulator operators such as $avg, $count, $sort, and $limit can be used within blocking stages. Meaningful data returned from these operators are obtained when run over a series of data versus a single data point. This is why they are considered blocking.
Non-blocking stages do not require multiple data points to be meaningful, and they include operators such as $addFields, $match, $project, $set, $unset, and $unwind, to name a few. You can use non-blocking before, after, or within the blocking stages. To illustrate this, let’s create a query that shows the average, maximum, and delta (the difference between the maximum and average). We will use a non-blocking $match to show only the results from device_1, calculate the tumblingWindow showing maximum and average, and then include another non-blocking $addFields.
Now we can use the .process command to run the stream processor in the foreground and view our results in the console.
The results of this query will be similar to the following:
Notice the time segments and how they align on the minute.
Time segments aligned on the minute
Additionally, notice that the output includes the difference between the calculated values of maximum and average for each window.

Create a hopping window

A hopping window, sometimes referred to as a sliding window, is a fixed-size window that moves forward in time at overlapping intervals. In Atlas Stream Processing, you use the $hoppingWindow operator. In this example, let’s use the operator to see the average.
To help illustrate the start and end time segments, let's create a filter to only return device_1.
Now let’s issue the .process command to view the results in the console.
An example result is as follows:
Notice the time segments.
Overlapping time segments
The time segments are overlapping by 30 seconds as was defined by the hopSize option. Hopping windows are useful to capture short-term patterns in data.

Summary

By continuously processing data within time windows, you can generate real-time insights and metrics, which can be crucial for applications like monitoring, fraud detection, and operational analytics. Atlas Stream Processing provides both tumbling and hopping window operators. Together these operators enable you to perform various aggregation operations such as sum, average, min, and max over a specific window of data. In this tutorial, you learned how to use both of these operators with solar sample data.

Learn more about MongoDB Atlas Stream Processing

Check out the MongoDB Atlas Stream Processing announcement blog post. For more on window operators in Atlas Stream Processing, learn more in our documentation.
Log in today to get started. Atlas Stream Processing is available to all developers in Atlas. Give it a try today!

Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Tutorial

Synchronize Your Mobile Application With MongoDB Atlas and Google Cloud MySQL


Feb 08, 2024 | 6 min read
Tutorial

Stream Data Into MongoDB Atlas Using AWS Glue


Apr 16, 2024 | 6 min read
Tutorial

No Connectivity? No Problem! Enable Offline Inventory with Atlas Edge Server


May 02, 2024 | 8 min read
Tutorial

How to Use Cohere Embeddings and Rerank Modules with MongoDB Atlas


Apr 04, 2024 | 10 min read
Table of Contents