Connecting to Mongodb with an ESP32 based board

Hello experts,

Does anyone work with arduinos/ESP32 boards and HTTP POSTING to a MongoDB?

I’m a Newbie to MongoDB, I’m struggling to make a connection from an ESP32-based board to an Atlas-Mongo db. I’m developing in the Arduino IDE.

I managed to create a serverless endpoint in Mongdb and I have successfully used an HTTP POST JSON request to add data to MongoDB using both the online “Postman” tool and a local “Postman” app on my machine. I’ve made sure I whitelisted my Router’s external IP address. So in theory, the endpoint works. I would think that success also validates the API key.

Coding that up in the Arduino IDE world is a different story for me. In searching for solutions, I’ve found three examples that I tried to stay as close to as possible but all are either slightly out of date or just different enough not to lead to an answer. I’ve never seen an Arduino example using the newest form of serverless Mongo-Atlas endpoints for example. Doing an HTTP POST seems so simple, I’ve done it many times in other situations, but I’m stymied by the connection to MongoDB.

I’m posting the code below. I’m moving from using a MYSQL database to a MongoDB. The Arduino code posted here chooses between them with a simple #define and it works fine with the MYSQL path.

For this forum post, I’ve only obscured a few personal details. I left the Mongodb details (that I can change later) open because those details might be key. I left in the comments some of the minor adjustments I’ve tried, I hope that isn’t too confusing.

//Simple canned sensor reading to test POST

#include <Arduino.h>
#include <WiFi.h>
//#include <esp_wifi.h>
#include <HTTPClient.h>

#define xMYSQL 0  //set to 1 to enable sending readings to a MYSQL database...0 implies MongoDB

//------------Server pointers---------------------
#define MySQL_DBASE "http://192.168.1.86/UR_do_solar/urpost-esp-data3.php"  //pointer to php files that reads the data into MYSQL
char serverAddressMY[] = "192.168.1.86";
const char resourceMY[] = "/UR_do_solar/urpost-esp-data3.php";


//#define MyMONGO_DBASE "https://us-east-2.aws.data.mongodb-api.com"
#define MyMONGO_DBASE "https://us-east-2.aws.data.mongodb-api.com/app/data-gwtpz/endpoint/ur_sensor/do"
//#define MyMONGO_DBASE "https://us-east-2.aws.data.mongodb-api.com/app/data-gwtpz/endpoint/ur_sensor/do/app/application-0-astev/endpoint"

//char serverAddressMD[] = "https://us-east-1.aws.data.mongodb-api.com";
char serverAddressMD[] = "https://us-east-2.aws.data.mongodb-api.com/app/data-gwtpz/endpoint/ur_sensor/do";

//const char resourceMD[] = "";
const char resourceMD[] = "/app/application-0-astev/endpoint";
//const char resourceMD[] = "/ur_sensor/do/app/application-0-astev/endpoint";

const int port = 80;
const int mdport = 27017;

String DBASE;

// Wifi network credentials---------------------
const char* ssid = "************";
const char* password = "************";

//----------Networking variables if you need it, or to assign static IP-----------------------------------------------------

IPAddress Server_ip(192, 168, 1, 208);  // IP address of this box
IPAddress gateway(192, 168, 1, 254);    // gateway of your network
IPAddress subnet(255, 255, 255, 0);     // subnet mask of your network
IPAddress dns(192, 168, 1, 254);        //dns address needed to help get to internet AND to ntp site below

//message buffering
String postData;  //String to post status data

//Client and Server starts
WiFiClient client;
HTTPClient http;

/******************************  Setup *************************************/
/***************************************************************************/
/***************************************************************************/


void setup() {
  Serial.begin(9600);
  WiFi.disconnect(true);
  //esp_wifi_start();
  Serial.println("POST test program");
  WiFi.mode(WIFI_STA);

  //WiFi.config(Server_ip, dns, gateway, subnet);  // forces to use the fixed IP
  WiFi.begin(ssid, password);
  delay(1000);
  while (WiFi.status() != WL_CONNECTED) {
    delay(100);
    Serial.print(".");
  }

  Serial.print("Connecting to:   ");
  Serial.print(WiFi.SSID());
  Serial.print("\n");
  Serial.print("This device IP address: ");
  Serial.println(WiFi.localIP());


  //choose the database target
  if (xMYSQL) {
    DBASE = MySQL_DBASE;
  } else {
    DBASE = MyMONGO_DBASE;
  }
}
/******************************  Close Setup *************************************/
/******************************  Close Setup *************************************/
/******************************  Close Setup *************************************/
/*********************************************************************************/

void loop() {

  String Bstr;  //string received from sensor

  //Bstr= "{\"Temp\":\"22.22\",\"DO\":\"11111.00\",\"HWBRD\":\"DO Sensor Solar2 sw:0.0.2\",\"Battery_Voltage_Monitor\":\"4.01\",\"VoltageRange\":\"MONGOTRY\",\"Charging\":\"15:48\",\"ChargeDone\":\"0\"}\r\n\r\n";
  Bstr = "{\"Temp\":\"22.22\",\"DO\":\"11111.00\",\"HWBRD\":\"DO Sensor Solar2 sw:0.0.2\"}\r\n\r\n";
  SendSensorData(Bstr);

  delay(40000);
}  //end loop
/******************************  Close Loop *************************************/
/******************************  Close Loop *************************************/
/******************************  Close Loop *************************************/
/*********************************************************************************/


//*****************************   SENSOR  DATA TRANSMITTING   *****************************//
void SendSensorData(String sensor_message) {
  //HTTPClient http;  //Declare object of class HTTPClient
  //HttpClient http(client, serverAddressMD, mdport);  /* changed *******************************/////////

  postData = sensor_message;

  Serial.println(postData);

  //http.begin(client, serverAddressMY, port, resourceMY);/* changed *******************************/////////
  http.begin(client, DBASE);

  http.addHeader("Content-Type", "application/json");      // SPECIFY JSON
  http.addHeader("firstkey", "6466305061dc2db2354b04c4");  //specify API key
  // String contentType = "application/json" ;                     /* changed *******************************/////////
  //int httpCode = http.post(resourceMD, contentType, postData);  /* changed *******************************/////////


  int httpCode = http.POST(postData);  //Send the request
  delay(2000);                         //do we need to wait longer?
  String payload = http.getString();   //Get the response payload

  Serial.println(httpCode);  //Print HTTP return code
  Serial.println(payload);   //Print request response payload


  http.end();  //Close connection
}

I have the debug compile option on and get the following error results. My interpretation of this error is a connection is made to the MongoDB website but it disconnects. I can’t figure out why, I get lost tracing thru the libraries to find the reason. One example mentioned the need for a secure certificate but the other two examples did not so I am not sure if that is the relevant debugging path to take.

18:45:42.910 -> {"Temp":"22.22","DO":"11111.00","HWBRD":"DO Sensor Solar2 sw:0.0.2"}
18:45:42.973 ->
18:45:42.973 ->
18:45:43.005 -> [ 45607][V][HTTPClient.cpp:252] beginInternal(): url: https://us-east-2.aws.data.mongodb-api.com/app/data-gwtpz/endpoint/ur_sensor/do
18:45:43.133 -> [ 45690][D][HTTPClient.cpp:303] beginInternal(): protocol: https, host: us-east-2.aws.data.mongodb-api.com port: 443 url: /app/data-gwtpz/endpoint/ur_sensor/do
18:45:43.292 -> [ 45857][D][HTTPClient.cpp:598] sendRequest(): request type: 'POST' redirCount: 0
18:45:43.388 ->
18:45:43.388 -> [ 46090][D][HTTPClient.cpp:1170] connect(): connected to us-east-2.aws.data.mongodb-api.com:443
18:45:43.484 -> [ 46143][D][WiFiClient.cpp:546] connected(): Disconnected: RES: -1, ERR: 104
18:45:43.580 -> [ 46143][D][HTTPClient.cpp:642] sendRequest(): sendRequest code=-5
18:45:43.645 ->
18:45:43.645 -> [ 46211][W][HTTPClient.cpp:1483] returnError(): error(-5): connection lost
18:45:45.597 -> [ 48291][W][HTTPClient.cpp:1483] returnError(): error(-4): not connected
18:45:45.693 -> -5

I suspect it’s how I’m specifying the endpoint in some manner, maybe a router/network issue, or confusion with using HTTPS that I don’t understand.

Can anyone give some guidance?

~
kurt h.

1 Like

I’m solving my own problem….documenting for others if their search brings them here. I found two related solutions. Atlas-Mongodb does indeed require a more secure communication link above just an API key.
First the Solution(s):
Referencing the original code I posted
a. I needed to change the WifIClient library to the WiFiClientSecure. (.h)
b. I then added the client.setInsecure() function before calling http.begin(…).

That was it!

Alternatively, I found a security certificate on the Mongdb webite…….I set the certificate to a char constant named rootCACertificate and included the function client.setCACert(rootCACertificate) INSTEAD of client.setInsecure() , again before http.begin.
Why did it take so long?:
I was using the following examples as a guide for doing a POST to Mongdb but only ONE mentioned the added security and commented they didn’t understand why it worked. (When I tried that example out originally, it didn’t work…forcing me to try other things. I’m guessing the original failure was only because of an expired certificate). For the examples that do not mention certificates or TSL or anything but an API key, I’m guessing Mongodb security has evolved over time too.
Here were the examples:
//Christmas Lights and Webcams with the MongoDB Data API | MongoDB (mentioned the need for WifIClientSecure.h)
//https://jimb-cc.medium.com/esp-and-mongodb-sitting-in-a-tree-7d043fb1a4d (no mention of certificate need or WiFiClientSecure.h)
//https://www.donskytech.com/control-your-arduino-iot-projects-with-a-mongodb-database/ (no mention of certificate need or WiFiClientSecure.h)

My success with “Postman” helped pull me off track too. I’m now guessing that the “postman” app handles this security in a hidden manner explaining it’s success and my failure.

Further comments are welcomed….especially my conjectures on my original failures.

2 Likes

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.