$exists query database with incoming payload value and respond

Hi

I’m testing out a Realm app working on an incoming webhook post request.

I have an incoming webhook with a JSON payload which has a particular field that needs to be defined (or filtered?) and used to query the mongodb database collection I created. I want to see if this particular incoming value exists in the database and then return one of two responses.

As a starting point for the webhook function I used a mongodb tutorial based on an incoming webhook.

Then I set the find() method with the $exists: true query selector, and define the incoming payload field which is name.

For the response I have the response object method of setBody(body) with one of two responses.

When running the function on realm I get an error: error transpiling function source (Syntax error: exit status 1)

Please help, I’m a newbie still learning) and would love to make use of mongodb apps properly!

Many thanks, Andi

Here is the function I’m working on:

exports = async function(payload, response) {

    const mongodb = context.services.get("mongodb-atlas");

    const eventsdb = mongodb.db("mydatabase");

    const eventscoll = eventsdb.collection("mycollection”);

    const result= await eventscoll.find({“payload.name": {$exists: true}});

response.setBody("{"ok": true,"details”:”Found”}");

  } else {

response.setBody("{"ok": false,"details":"Not found”}");

    }

    return { text: `searched` };

 }

Hi @a_Jn,

Looks like you have some bad quotes :

 const eventscoll = eventsdb.collection("mycollection”);

    const result= await eventscoll.find({“payload.name

See

Additionally I do not see an if before. Not sure what you are trying to achieve but if you want to get the attribute name from the payload you first need to extract it:

var body = JSON.parse(payload.body.text())
var name = body.name

Now do you need to look it under a collection document or look for a Field with the passed value?

Replace those with correct ones they are probably a copy paste error.

Thanks
Pavel

Hi Pavel

Thank you for your reply – to try clarify:

to handle a request and send a response,

I want to query “mycollection" in “mydatabase" on mongoldb-atlas

and I need to use the value of a specific field extracted from an incoming webhook to query “mycollection"

(this specific field called “name" will always have a different value to match to “mycollection” – so yes I need to look for the incoming webhook’s extracted/passed value in the collection document)

then I need to return a response,

if this extracted value matches a value in “mycollection”

it will return a response of {“ok”: true,"details”:”Found”}

if this extracted value does not exist

it will return a response of {“ok”: false,“details”:"Not found”}

Sorry I left out the payload extract and if – I’ve updated the script below, although it is not showing the syntax error when run it still shows: error transpiling function source

Many thanks Andi

exports = async function(payload, response) {

    const mongodb = context.services.get("mongodb-atlas");

    const eventsdb = mongodb.db("mydatabase");

    const eventscoll = eventsdb.collection("mycollection");

    var body = JSON.parse(payload.body.text());

    var name = body.name;

    const result= await eventscoll.find({ "payload.name": { $exists: true } });

    if(result) {

        response.setBody( "{"ok": true,"details”:”Found”}" );    

      } else {

        response.setBody( "{"ok": false,"details":"Not found”}" );

    }

Just a quick note after looking around some more –

does async await work with find() to query/loop through the database collection and return a result?

or how can this query work to match a result (or not) and return the response whenever the webhook with the value to query comes in?

Many thanks Andi

Hi @a_Jn,

Ok I got you requirment. I assume that you document have a field named “name” correct?

{
 "name" : "..."
}

And you need to take a payload “name” field and search it. If the above is correct and you have a “POST” webhook, please use the following function

exports = async function(payload, response) {

    const mongodb = context.services.get("mongodb-atlas");

    const eventsdb = mongodb.db("mydatabase");

    const eventscoll = eventsdb.collection("mycollection");

    var body = JSON.parse(payload.body.text());

    var name = body.name;

    const result= await eventscoll.count({ "name": name });

    if(result.count > 0 ) {

        response.setBody( JSON.stringify({"ok": true,"details":"Found"}) );    

      } else {

        response.setBody(  JSON.stringify({"ok": false,"details":"Not Found"}) );

    }
}

To find if value exists you need to search it with a count operator and test if the count is more than 0.

Then you need to setBody with a stringfied JSON.

Now you can call the webhook:

curl \
-H "Content-Type: application/json" \
-d '{"name":"bar"}' \
https://webhooks.mongodb-realm.com/api/client/v2.0/app/<APP_ID>/service/<SRV_NAME>/incoming_webhook/<WEBHOOK_NAME>

My test resulted in:

{"details":"Not Found","ok":false}

Consider indexing the search field for best performance.

Best regards,
Pavel

Hi Pavel

Great thank you – really good to see how the count() is the solution instead of $exists with async await here.

Regarding the payload value –

below is an example of the incoming payload, it has an array of values, and I want the “name” value “Jonty22” to search with.

To get to this do we need to loop through the _embedded[‘ff:items’] object ?

How would this specific name value of “Jonty22” be extracted?

Many thanks Andi

{
    "_links": {
    },
    "_embedded": {
        "ff:items": [
            {
                "_links": {
                },
                "_embedded": {
                    "ff:options": [
                        {
                            "_links": {
                            },
                            "name": "Eyecolor",
                            "value": "brown",
                        }
                    ],
                    "ff:category": {
                        "_links": {
                        },
                        "name": "Number",
                        "code": "101",
                        "send_email": false,
                    }
                },
                "group": "B",
                "name": "Jonty22",
                "date_created": null,
                "date_modified": "2020-06-20T06:11:08-0900"
            },
            {
                "_links": {
                },
                "_embedded": {
                    "ff:item_category": {
                        "_links": {
                        },
                        "email": "",
                        "type": "live",
                        "weight": 65,
                        "count_name": "",
                    }
        ],
        "ff:counts": [
            {
                "code": 22,
                "name": "Default count",
                "display": "22",
                "future_count": false
            }
        ],
        "ff:custom_field": [
            {
                "name": "custom_note",
                "value": "Happy Jonty",
                "is_hidden": 0
            }],
        "ff:area": {
            "city": "London",
            "country": "UK",
        },
        "ff:people": {
            "email": false,
            "is_anonymous": "0",
            "_embedded": {
                "ff:transfers": [
                    {
                        "type": "all",
                        "reception": null
                    }
                ],
                "ff:default": {
                    "code": "",
                    "phone": ""
                }
            }
        }
    },
    "language": "",
    "session_id": "jjks2312md9sw8ee3kljd3",
}

Hi @a_Jn,

Well when you parse this object you need to use a “.” Notation to access the value of the desired “name” level.

var body = JSON.parse(payload.body.text());
 var items = body._embedded["ff:items"];
var name= items[0].name;

I rather access special characters field names with a [].

Pavel

Great thank you Pavel,

I’m getting a response now, except – it is the same "Not Found” response for both when the name is in the collection and also when it is not.

Is there a way to see what the “name” value extracted from the incoming payload is? so to make sure it is extracting the correct “Jonty22” value to search with?

Then is there a way to test the search and see where the problem is and why it is responding with “Not found” in both instances?

Here is the database collection with the “Jonty22” id value to match and return the ”Found” response

https://webhooks.mongodb-stitch.com/api/client/v2.0/app/app1-ifvea/service/httpGET/incoming_webhook/webhookhttpGET

Many thanks Andi

And the function:

exports = async function(payload, response) {

    const mongodb = context.services.get("mongodb-atlas");

    const eventsdb = mongodb.db("mydatabase");

    const eventscoll = eventsdb.collection("mycollection");

    var body = JSON.parse(payload.body.text());

    var items = body._embedded["ff:items"];
 
    var name= items[0].name;

    const result= await eventscoll.count({ "name": name });

    if(result.count > 0 ) {

        response.setBody( JSON.stringify({"ok": true,"details":"Found"}) );    

      } else {

        response.setBody(  JSON.stringify({"ok": false,"details":"Not Found"}) );

    }
}

Hi @a_Jn,

Wait I see that the field in the collection is “id” and not “name”.

If you need to search a field name “id” for that value you need it to be specified in the count query:

const result= await eventscoll.count({ "id": name });

To debug functions you can use console.log() and print any of the parameters , the output will be shown on screen/logs.

Please consider doing sopme of our query tutorials to grasp you head around MongoDB queries.

Best regards,
Pavel[

Great thank you Pavel

yes the database collection key is “id", and the search key from the payload is “name" (looping through the _embedded[‘ff:items’] object)

I was just about to ask how I can log this to try root out why it’s not getting the results expected,

like how to print the "name” I’m getting from the payload to a log so I can see what I’m getting

this will be super helpful for the future

Many thanks Andi

with the:

const result= await eventscoll.count({ "id": name });

I still get the same result:

{
  "mydatabase.mycollection": {
    "no_matching_role": 0
  }
}

need to see what what the “name” value is it extracts to search with

Hi @a_Jn,

I think your rules/authetication method for the webhook does not allow you to see the documents.

Can you change the authentication for webhook to SYSTEM?

Please share the application main page link and I can review.

Best regards,
Pavel

Thank you Pavel,

yes strange because when it receives the payload that does have the “Jonty22” name field it returns the “Not found” when searching that database which does have the “Jonty22”

so I’m not sure if it actually extracts/uses this “Jonty22” name value from the incoming payload.

(The webhook Authentication is set to SYSTEM, and I have not created any rules.)

Below is another, maybe better, example of the incoming payload that has the “Jonty22” to search the database collection with.

Please let me know which link you need to review?

Many thanks Andi

{
  "_links": {
    "curies": [
      {
        "name": "ff",
        "href": "https://api.test.com/",
        "templated": true
      }
    ],
    "self": {
      "href": "https://api.test.com/",
      "title": "This"
    },
    "ff:attributes": {
      "href": "https://api.test.com/",
      "title": "Attributes"
    },
    "ff:counts": {
      "href": "https://api.test.com/",
      "title": "counts"
    }
  },
  "_embedded": {
    "ff:items": [
      {
        "_links": {
          "curies": [
            {
              "name": "ff",
              "href": "https://api.test.com/",
              "templated": true
            }
          ],
          "self": {
            "href": "https://api.test.com/",
            "title": "Idea"
          },
          "ff:item_category": {
            "href": "https://api.test.com/",
            "title": "Idea Category"
          },
          "ff:item_options": {
            "href": "https://api.test.com/",
            "title": "Idea Options"
          },
          "ff:attributes": {
            "href": "https://api.test.com/",
            "title": "Idea Attributes"
          }
        },
        "_embedded": {
          "ff:item_category": {
            "_links": {
              "curies": [
                {
                  "name": "ff",
                  "href": "https://api.test.com/",
                  "templated": true
                }
              ],
              "self": {
                "href": "https://api.test.com/",
                "title": "DEFAULT"
              },
              "ff:email_templates": {
                "href": "https://api.test.com/",
                "title": "Email Templates"
              }
            },
            "code": "DEFAULT",
            "name": "Default for all",
            "default_weight": 65,
            "count_type": "",
            "date_created": "2020-07-09T12:03:06-0700",
          }
        },
        "category_uri": "",
        "name": "Jonty22",
        "weight": 65,
        "code": "",
        "count_type": "",
        "url": "",
        "date_modified": "2020-07-27T10:02:16-0700"
      }
    ],
    "ff:counts": [],
    "ff:custom_fields": [
      {
        "name": "total",
        "value": "20",
      },
      {
        "name": "total_future",
        "value": "0",
      },
      {
        "name": "agree",
        "value": "No",
      }
    ],
    "ff:ament": {
      "city": "London",
      "region": "",
      "country": "GB",
    },
    "ff:ament_results": [],
    "ff:cuter": {
      "id": "0",
      "first_name": "",
      "last_name": "",
      "email": "test@api.test.com",
      "is_anonymous": 1,
      "_embedded": {
        "ff:aments": [
          {
            "year": "",
            "age": null,
          }
        ],
        "ff:faults": {
          "country": "",
          "city": "",
        }
      }
    }
  },
  "customer_uri": "",
  "template_set_uri": "",
  "language": "",
  "date_created": null,
  "date_modified": "2020-07-27T09:06:24-0700",
  "ip": "90.252.101.84",
  "session_name": "fcsid",
  "session_id": "bs4mpdbd3hjlc1e5reumeaptd0",
}

Hi @a_Jn,

When you are on the webhook page in the UI please copy the browser URL and post it here so I can view the code.

In general the second document also have “name” looking for on the same level under '_embedded. “ff:items”.name`

Best regards,
Pavel

Is there a way to see in the logs what the extracted value is that this webhook uses to search with? I cant find/see the value, only

Headers:

{
“X-Forwarded-Proto-Stitch”: [
“https”
],
“Sslclientcertstatus”: [
“NoClientCert”
],
“Content-Length”: [
“6301”
],

etc…

Many thanks Andi

Sorry here is the link

https://realm.mongodb.com/groups/5e80b666924a0058a7e2e79d/apps/5e80be3b308f9d66390b6f83/services/5f1bf380a6ac7caa8b53020a/incomingWebhooks/5f1bf3abfb55eba1d0c15fb5

Many thanks Andi

https://realm.mongodb.com/groups/5e80b666924a0058a7e2e79d/apps/5e80be3b308f9d66390b6f83/services/5f1bf380a6ac7caa8b53020a/incomingWebhooks/5f1bf3abfb55eba1d0c15fb5

Add the following lines and rerun:

    var name= items[0].name;
 console.log("Name : " + name);

Thanks!

Great that shows up! so it is extracting the correct field value to search with

Logs:

[

"Name : Jonty22"

]

Function Call Location:

US-VA

Query Arguments:

{}