BlogAnnounced at MongoDB.local NYC 2024: A recap of all announcements and updatesLearn more >>
MongoDB Developer
C++
plus
Sign in to follow topics
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Languageschevron-right
C++chevron-right

Red Mosquitto: Implement a noise sensor with an MQTT client in an ESP32

Jorge D. Ortiz-Fuentes25 min read • Published Mar 18, 2024 • Updated Mar 18, 2024
RaspberryPiC++Rust
Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Welcome to another article of the "Adventures in IoT" series. So far, we have defined an end-to-end project, written the firmware for a Raspberry Pi Pico MCU board to measure the temperature and send the value via Bluetooth Low Energy, learned how to use Bluez and D-Bus, and implemented a collecting station that was able to read the BLE data. If you haven't had the time yet, you can read them or watch the videos.
In this article, we are going to write the firmware for a different board: an ESP32-C6-DevKitC-1. ESP32 boards are very popular among the DIY community and for IoT in general. The creator of these boards, Espressif, is putting a good amount of effort into supporting Rust as a first-class developer language for them. I am thankful for that and I will take advantage of the tools they have created for us.
We can write code for the ESP32 that talks to the bare metal, a.k.a. core, or use an operating system that allows us to take full advantage of the capabilities provided by std library. ESP-IDF –i.e., ESPressif IoT Development Framework– is created to simplify that development and is not only available in C/C++ but also in Rust, which we will be using for the rest of this article. By using ESP-IDF through the corresponding crates, we can use threads, mutexes, and other synchronization primitives, collections, random number generation, sockets, etc. enter image description here My board is an ESP32-C6 that uses a RISC-V architecture. It doesn't have any built-in sensor, but it features an RGB LED and is capable of communicating wirelessly with the rest of the world in several ways: WiFi, Bluetooth LE, Zigbee, and Thread. Let's see how we can use these features.

Concepts

Embedded HAL

On January 9th, 2024 –i.e., a few days after I had started preparing this tutorial– embedded-hal v1.0 was released. It provides an abstraction to create drivers that are independent from the MCU. This is very useful for us developers because it allows us to develop and maintain the driver once and use it for the many different MCU boards that honor that abstraction.
This development board kit has a neopixel LED –i.e., an RGB LED controlled by a WS2812– which we will use for our "Hello World!" iteration and then to inform the user about the state of the device. The WS2812 requires sending sequences of high and low voltages that use the duration of those high and low values to specify the bits that define the RGB color components of the LED. The ESP32 has a Remote Control Transceiver (RMT) that was conceived as an infrared transceiver but can be repurposed to generate the signals required for the single-line serial protocol used by the WS1812. Neither the RMT nor the timers are available in the just released version of the embedded-hal, but the ESP-IDF provided by Expressif does implement the full embedded-hal abstraction, and the WS2812 driver uses the available abstractions.

Setup

The tools

There are some tools that you will need to have installed in your computer to be able to follow along and compile and install the firmware on your board. I have installed them on my computer, but before spending time on this setup, consider using the container provided by Espressif if you prefer that choice.
The first thing that might be different for you is that we need the bleeding edge version of the Rust toolchain. We will be using the nightly version of it:
As for the tools, you may already have some of these tools on your computer, but double-check that you have installed all of them:
  • Git (in macOS installed with Code)
  • Some tools to assist on the building process (brew install cmake ninja dfu-util python3 –This works on macOS, but if you use a different OS, please check the list here)
  • A tool to forward linker arguments to the actual linker (cargo install ldproxy)
  • A utility to write the firmware to the board (cargo install espflash)
  • A tool that is used to produce a new project from a template (cargo install cargo-generate)

Project creation using a template

We can then create a project using the template for stdlib projects (esp-idf-template):
And we fill in this data:
  • Project name: mosquitto-bzzz
  • MCU to target: esp32c6
  • Configure advanced template options: false
cargo b produces the build. Target is riscv32imac-esp-espidf (RISC-V architecture with support for atomics), so the binary is generated in target/riscv32imac-esp-espidf/debug/mosquitto-bzzz. And it can be run on the device using this command:
And at the end of the output log, you can find these lines:
Let's understand the project that has been created so we can take advantage of all the pieces:
  • Cargo.toml: It is main the configuration file for the project. Besides what a regular cargo new would do, we will see that:
    • It defines some features available that modify the configuration of some of the dependencies.
    • It includes a couple of dependencies: one for the logging API and another for using the ESP-IDF.
    • It adds a build dependency that provides utilities for building applications for embedded systems.
    • It adjusts the profile settings that modify some compiler options, optimization level, and debug symbols, for debug and release.
  • build.rs: A build script that doesn't belong to the application but is executed as part of the build process.
  • rust-toolchain.toml: A configuration file to enforce the usage of the nightly toolchain as well as a local copy of the Rust standard library source code.
  • sdkconfig.defaults: A file with some configuration parameters for the esp-idf.
  • .cargo/config.toml: A configuration file for Cargo itself, where we have the architecture, the tools, and the unstable flags of the compiler used in the build process, and the environment variables used in the process.
  • src/main.rs: The seed for our code with the minimal skeleton.

Foundations of our firmware

The idea is to create firmware similar to the one we wrote for the Raspberry Pi Pico but exposing the sensor data using MQTT instead of Bluetooth Low Energy. That means that we have to connect to the WiFi, then to the MQTT broker, and start publishing data. We will use the RGB LED to show the status of our sensor and use a sound sensor to obtain the desired data.

Control the LED

Making an LED blink is considered the hello world of embedded programming. We can take it a little bit further and use colors rather than just blink.
  1. According to the documentation of the board, the LED is controlled by the GPIO8 pin. We can get access to that pin using the Peripherals module of the esp-idf-svc, which exposes the hal adding use esp_idf_svc::hal::peripherals::Peripherals;:
  2. Also using the Peripherals singleton, we can access the RMT channel that will produce the desired waveform signal required to set each of the three color components of the LED:
  3. We could do the RGB color encoding manually, but there is a crate that will help us talk to the built-in WS2812 (neopixel) controller that drives the RGB LED. The create smart-leds could be used on top of it if we had several LEDs, but we don't need it for this board.
  4. We create an instance that talks to the WS2812 in pin 8 and uses the Remote Control Transceiver – a.k.a. RMT – peripheral in channel 0. We add the symbol use ws2812_esp32_rmt_driver::Ws2812Esp32RmtDriver; and:
  5. Then, we define the data for a pixel and write it with the instance of the driver so it gets used in the LED. It is important to not only import the type for the 24bi pixel color but also get the trait with use ws2812_esp32_rmt_driver::driver::color::{LedPixelColor,LedPixelColorGrb24};:
  6. At this moment, you can run it with cargo r and expect the LED to be on with a yellow color.
  7. Let's add a loop and some changes to complete our "hello world." First, we define a second color:
  8. Then, we add a loop at the end where we switch back and forth between these two colors:
  9. If we don't introduce any delays, we won't be able to perceive the colors changing, so we add use std::{time::Duration, thread}; and wait for half a second before every change:
  10. We run and watch the LED changing color from purple to yellow and back every half a second.

Use the LED to communicate with the user

We are going to encapsulate the usage of the LED in its own thread. That thread needs to be aware of any changes in the status of the device and use the current one to decide how to use the LED accordingly.
  1. First, we are going to need an enum with all of the possible states. Initially, it will contain one variant for no error, one variant for WiFi error, and another one for MQTT error:
  2. And we can add an implementation to convert from eight-bit unsigned integers into a variant of this enum:
  3. We would like to use the DeviceStatus variants by name where a number is required. We achieve the inverse conversion by adding an annotation to the enum:
  4. Next, I am going to do something that will be considered naïve by anybody that has developed anything in Rust, beyond the simplest "hello world!" However, I want to highlight one of the advantages of using Rust, instead of most other languages, to write firmware (and software in general). I am going to define a variable in the main function that will hold the current status of the device and share it among the threads.
  5. We are going to define two threads. The first one is meant for reporting back to the user the status of the device. The second one is just needed for testing purposes, and we will replace it with some real functionality in a short while. We will be using sequences of colors in the LED to report the status of the sensor. So, let's start by defining each of the steps in those color sequences:
  6. We also define a constructor as an associated function for our own convenience:
  7. We can then use those steps to transform each status into a different sequence that we can display in the LED:
  8. We start the thread by initializing the WS2812 that controls the LED:
  9. We can keep track of the previous status and the current sequence, so we don't have to regenerate it after displaying it once. This is not required, but it is more efficient:
  10. We then get into an infinite loop, in which we update the status, if it has changed, and the sequence accordingly. In any case, we use each of the steps of the sequence to display it in the LED:
  11. Notice that the status cannot be compared until we implement PartialEq, and assigning it requires Clone and Copy, so we derive them:
  12. Now, we are going to implement the function that is run in the other thread. This function will change the status every 10 seconds. Since this is for the sake of testing the reporting capability, we won't be doing anything fancy to change the status, just moving from one status to the next and back to the beginning:
  13. With the two functions in place, we just need to spawn two threads, one with each one of them. We will use a thread scope that will take care of joining the threads that we spawn:
  14. Compiling this code will result in errors. It is the blessing/curse of the borrow checker, which is capable of figuring out that we are sharing memory in an unsafe way. The status can be changed in one thread while being read by the other. We could use a mutex, as we did in the previous C++ code, and wrap it in an Arc to be able to use a reference in each thread, but there is an easier way to achieve the same goal: We can use an atomic type. (use std::sync::atomic::AtomicU8;)
  15. We modify report_status() to use the reference to the atomic type and add use std::sync::atomic::Ordering::Relaxed;:
  16. And change_status(). Notice that in this case, thanks to the interior mutability, we don't need a mutable reference but a regular one. Also, we need to specify the guaranties in terms of how multiple operations will be ordered. Since we don't have any other atomic operations in the code, we can go with the weakest level – i.e., Relaxed:
  17. Finally, we have to change the lines in which we spawn the threads to reflect the changes that we have introduced:
  18. You can use cargo r to compile the code and run it on your board. The lights should be displaying the sequences, which should change every 10 seconds.

Getting the noise level

It is time to interact with a temperature sensor… Just kidding. This time, we are going to use a sound sensor. No more temperature measurements in this project. Promise.
The sensor I am going to use is an OSEPP Sound-01 that claims to be "the perfect sensor to detect environmental variations in noise." It supports an input voltage from 3V to 5V and provides an analog signal. We are going to connect the signal to pin 0 of the GPIO, which is also the pin for the first channel of the analog-to-digital converter (ADC1_CH0). The other two pins are connected to 5V and GND (+ and -, respectively). enter image description here You don't have to use this particular sensor. There are many other options on the market. Some of them have pins for digital output, instead of just an analog one as in this one. Some sensors also have a potentiometer that allows you to adjust the sensitivity of the microphone.

Read from the sensor

  1. We are going to perform this task in a new function:
  2. We want to use the ADC on the pin that we have connected the signal. We can get access to the ADC1 using the peripherals singleton in the main function.
  3. And also to the pin that will receive the signal from the sensor:
  4. We modify the signature of our new function to accept the parameters we need:
  5. Now, we use those two parameters to attach a driver that can be used to read from the ADC. Notice that the AdcDriver needs a configuration, which we create with the default value. Also, AdcChannelDriver requires a generic const parameter that is used to define the attenuation level. I am going to go with maximum attenuation initially to have more sensibility in the mic, but we can change it later if needed. We add use esp_idf_svc::hal::adc::{attenuation, AdcChannelDriver};:
  6. With the required pieces in place, we can use the adc_channel to sample in an infinite loop. A delay of 10ms means that we will be sampling at ~100Hz:
  7. Lastly, we spawn a thread with this function in the same scope that we were using before:

Compute noise levels (Sorta!)

In order to get an estimation of the noise level, I am going to compute the Root Mean Square (RMS) of a buffer of 50ms, i.e., five samples at our current sampling rate. Yes, I know this isn't exactly how decibels are measured, but it will be good enough for us and the data that we want to gather.
  1. Let's start by creating that buffer where we will be putting the samples:
  2. Inside the infinite loop, we are going to have a for-loop that goes through the buffer:
  3. We modify the sampling that we were doing before, so a zero value is used if the ADC fails to get a sample:
  4. Before starting with the iterations of the for loop, we are going to define a variable to hold the addition of the squares of the samples:
  5. And each sample is squared and added to the sum. We could do the conversion into floats after the square, but then, the square value might not fit into a u16:
  6. And we compute the decibels (or something close enough to that) after the for loop:
  7. We compile and run with cargo r and should get some output similar to:

MQTT

Concepts

When we wrote our previous firmware, we used Bluetooth Low Energy to make the data from the sensor available to the rest of the world. That was an interesting experiment, but it had some limitations. Some of those limitations were introduced by the hardware we were using, like the fact that we were getting some interferences in the Bluetooth signal from the WiFi communications in the Raspberry Pi. But others are inherent to the Bluetooth technology, like the maximum distance from the sensor to the collecting station.
For this firmware, we have decided to take a different approach. We will be using WiFi for the communications from the sensors to the collecting station. WiFi will allow us to spread the sensors through a much greater area, especially if we have several access points. However, it comes with a price: The sensors will consume more energy and their batteries will last less.
Using WiFi practically implies that our communications will be TCP/IP-based. And that opens a wide range of possibilities, which we can summarize with this list in increasing order of likelihood:
  • Implement a custom TCP or UDP protocol.
  • Use an existing protocol that is commonly used for writing APIs. There are other options, but HTTP is the main one here.
  • Use an existing protocol that is more tailored for the purpose of sending event data that contains values.
Creating a custom protocol is expensive, time-consuming, and error-prone, especially without previous experience. It''s probably the worst idea for a proof of concept unless you have a very specific requirement that cannot be accomplished otherwise.
HTTP comes to mind as an excellent solution to exchange data. REST APIs are an example of that. However, it has some limitations, like the unidirectional flow of data, the overhead –both in terms of the protocol itself and on using a new connection for every new request– and even the lack of provision to notify selected clients when the data they are interested in changes.
If we want to go with a protocol that was designed for this, MQTT is the natural choice. Besides overcoming the limitations of HTTP for this type of communication, it has been tested in the field with many sensors that change very often and out of the box, can do fancy things like storing the last known good value or having specific client commands that allow them to receive updates on specific values or a set of them. MQTT is designed as a protocol for publish/subscribe (pub/sub) in the scenarios that are common for IoT. The server that controls all the communications is commonly referred to as a broker, and our sensors will be its clients.

Connect to the WiFi

Now that we have a better understanding of why we are using MQTT, we are going to connect to our broker and send the data that we obtain from our sensor so it gets published there.
However, before being able to do that, we need to connect to the WiFi.
It is important to keep in mind that the board we are using has support for WiFi but only on the 2.4GHz band. It won't be able to connect to your router using the 5GHz band, no matter how kindly you ask it to do it.
Also, unless you are a wealthy millionaire and you've got yourself a nice island to focus on following along with this content, it would be wise to use a fairly strong password to keep unauthorized users out of your network.
  1. We are going to begin by setting some structure for holding the authentication data to access the network:
  2. We could set the values in the code, but I like better the approach suggested by Ferrous Systems. We will be using the toml_cfg crate. We will have default values (useless in this case other than to get an error) that we will be overriding by using a toml file with the desired values. First things first: Let's add the crate:
  3. Let's now annotate the struct with some macros:
  4. We can now add a cfg.toml file with the actual values of these parameters.
  5. Please, remember to add that filename to the .gitignore configuration, so it doesn't end up in our repository with our dearest secrets:
  6. The code for connecting to the WiFi is a little bit tedious. It makes sense to do it in a different function:
  7. This function should have a way to let us know if there has been a problem, but we want to simplify error handling, so we add the anyhow crate:
  8. We can now use the Result type provided by anyhow (import anyhow::Result;). This way, we don't need to be bored with creating and using a custom error type.
  9. If the function doesn't get an SSID, it won't be able to connect to the WiFi, so it's better to stop here and return an error (import anyhow::bail;):
  10. If the function gets a password, we will assume that authentication uses WPA2. Otherwise, no authentication will be used (use esp_idf_svc::wifi::AuthMethod;):
  11. We will need an instance of the system loop to maintain the connection to the WiFi alive and kicking, so we access the system event loop singleton (use esp_idf_svc::eventloop::EspSystemEventLoop; and use anyhow::Context).
  12. Although it is not required, the esp32 stores some data from previous network connections in the non-volatile storage, so getting access to it will simplify and accelerate the connection process (use esp_idf_svc::nvs::EspDefaultNvsPartition;).
  13. The connection to the WiFi is done through the modem, which can be accessed via the peripherals of the board. We pass the peripherals, obtain the modem, and use it to first wrap it with a WiFi driver and then get an instance that we will use to manage the WiFi connection (use esp_idf_svc::wifi::{EspWifi, BlockingWifi};):
  14. Then, we add a configuration to the WiFi (use esp_idf_svc::wifi;):
  15. With the configuration in place, we start the WiFi radio, connect to the WiFi network, and wait to have the connection completed. Any errors will bubble up:
  16. It is useful at this point to display the data of the connection.
  17. We also want to return the variable that holds the connection. Otherwise, the connection will be closed when it goes out of scope at the end of this function. We change the signature to be able to do it:
  18. And return that value:
  19. We are going to initialize the connection to the WiFi from our function to read the noise, so let's add the modem as a parameter:
  20. This new parameter has to be initialized in the main function:
  21. And passed it onto the function when we spawn the thread:
  22. Inside the function where we plan to use these parameters, we retrieve the configuration. The CONFIGURATION constant is generated automatically by the cfg-toml crate using the type of the struct:
  23. Next, we try to connect to the WiFi using those parameters:
  24. And, when dealing with the error case, we change the value of the status:
  25. This function doesn't take the state as an argument, so we add it to its signature:
  26. That argument is provided when the thread is spawned:
  27. We don't want the status to be changed sequentially anymore, so we remove that thread and the function that was implementing that change.
  28. We run this code with cargo r to verify that we can connect to the network. However, this version is going to crash. 😱 Our function is going to exceed the default stack size for a thread, which, by default, is 4Kbytes.
  29. We can use a thread builder, instead of the spawn function, to change the stack size:
  30. After performing this change, we run it again cargo r and it should work as expected.

Set up the MQTT broker

The next step after connecting to the WiFi is to connect to the MQTT broker as a client, but we don't have an MQTT broker yet. In this section, I will show you how to install Mosquitto, which is an open-source project of the Eclipse Foundation.
  1. For this section, we need to have an MQTT broker. In my case, I will be installing Mosquitto, which implements versions 3.1.1 and 5.0 of the MQTT protocol. It will run in the same Raspberry Pi that I am using as a collecting station.
  2. We modify the Mosquitto configuration to enable clients to connect from outside of the localhost. We need some credentials and a configuration that enforces authentication:
  3. Let's test that we can subscribe and publish to a topic. The naming convention tends to use lowercase letters, numbers, and dashes only and reserves dashes for separating topics hierarchically. On one terminal, subscribe to the testTopic:
  4. And on another terminal, publish something to it:
  5. You should see the message that we wrote on the second terminal appear on the first one. This means that Mosquitto is running as expected.

Publish to MQTT from the sensor

With the MQTT broker installed and ready, we can write the code to connect our sensor to it as an MQTT client and publish its data.
  1. We are going to need the credentials that we have just created to publish data to the MQTT broker, so we add them to the Configuration structure:
  2. You have to remember to add the values that make sense to the cfg.toml file for your environment. Don't expect to get them from my repo, because we have asked Git to ignore this file. At the very least, you need the hostname or IP address of your MQTT broker. Copy the user name and password that we created previously:
  3. Coming back to the function that we have created to read the noise sensor, we can now initialize an MQTT client after connecting to the WiFi (use mqtt::client::{EspMqttClient, MqttClientConfiguration, QoS},):
  4. The first parameter is a URL to the MQTT server that will include the user and password, if defined:
  5. The second parameter is the configuration. Let's add them to the creation of the MQTT client:
  6. In order to publish, we need to define the topic:
  7. And a variable that will be used to contain the message that we will publish:
  8. Inside the loop, we will format the noise value because it is sent as a string:
  9. We publish this value using the MQTT client:
  10. As we did when we were publishing from the command line, we need to subscribe, in an independent terminal, to the topic that we plan to publish to. In this case, we are going to start with home/noise sensor/01. Notice that we represent a hierarchy, i.e., there are noise sensors at home and each of the sensors has an identifier. Also, notice that levels of the hierarchy are separated by slashes and can include spaces in their names.
  11. Finally, we compile and run the firmware with cargo r and will be able to see those values appearing on the terminal that is subscribed to the topic.

Use a unique ID for each sensor

I would like to finish this firmware solving a problem that won't show up until we have two sensors or more. Our firmware uses a constant topic. That means that two sensors with the same firmware will use the same topic and we won't have a way to know which value corresponds to which sensor. A better option is to use a unique identifier that will be different for every ESP32-C6 board. We can use the MAC address for that.
  1. Let's start by creating a function that returns that identifier:
  2. Our function is going to use an unsafe function from ESP-IDF, and format the result as a String (use esp_idf_svc::sys::{esp_base_mac_addr_get, ESP_OK}; and use std::fmt::Write). The function that returns the MAC address uses a pointer and, having been written in C++, couldn't care less about the safety rules that Rust code must obey. That function is considered unsafe and, as such, Rust requires us to use it within an unsafe scope. It is their way to tell us, "Here be dragons… and you know about it":
  3. Then, we use the function before defining the topic and use its result with it:
  4. And we slightly change the way we publish the data to use the topic:
  5. We also need to change the subscription so we listen to all the topics that start with home/sensor/ and have one more level:
  6. We compile and run with cargo r and the values start showing up on the terminal where the subscription was initiated.

Recap and future work

In this article, we have used Rust to write the firmware for an ESP32-C6-DevKitC-1 board from beginning to end. Although we can agree that Python was an easier approach for our first firmware, I believe that Rust is a more robust, approachable, and useful language for this purpose.
The firmware that we have created can inform the user of any problems using an RGB LED, measure noise in something close enough to deciBels, connect our board to the WiFi and then to our MQTT broker as a client, and publish the measurements of our noise sensor. Not bad for a single tutorial.
We have even gotten ahead of ourselves and added some code to ensure that different sensors with the same firmware publish their values to different topics. And to do so, we have done a very brief incursion in the universe of unsafe Rust and survived the wilderness. Now you can go to a bar and tell your friends, "I wrote unsafe Rust." Well done!
In our next article, we will be writing C++ code again to collect the data from the MQTT broker and then send it to our instance of MongoDB Atlas in the Cloud. So get ready!

Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Tutorial

Me and the Devil BlueZ: Reading BLE sensors from C++


Mar 18, 2024 | 16 min read
Tutorial

Turn BLE: Implementing BLE Sensors with MCU Devkits


Apr 02, 2024 | 13 min read
Tutorial

Me and the Devil BlueZ: Implementing a BLE Central in Linux - Part 1


Dec 14, 2023 | 10 min read
Tutorial

MongoDB Time Series with C++


Apr 03, 2024 | 6 min read
Table of Contents