Hi! First post here!
I need help with how to query a .find() in my nodeJs backend
I search for project_x and intent_y and sort responseTime where i want to get back the document after if a ‘conversation_id’ is the same as the one im searching for.
And where can I find some good reading / videos about learning more advanced queries? 
Please publish sample documents from your collections.
Course at https://university.mongodb.com/ helped me a lot.
Read Formatting code and log snippets in posts before posting your documents so that we can cut-n-paste them easily in our system.
Thanks.
_id: 61f06a83afbf51c80b9a9b4b
project_id: "sorteringshatt-dgwo"
name: "2022-01-25 22:23:41.315399"
inputContexts: Array
originalRequestSource: "SORTERINGSHATT_MAI"
v2Response: Object
responseTime: "2022-01-25 22:24:19.697221"
logType: "TESTING"
updated: false
__v: 0
where name is current conversation start, so next document will have the same name if a user talked to chatbot more.
It looks like this now, so i get ALOT of returns, then i start checking for if(i and i+1) is the same and do logic on the server…
const history = await History
.find({ project_id: project_id })
.find({ updated: false })
.sort({ responseTime: 1 });
So i would like something like:
const history = await History
.find({ project_id: project_id })
.find({ 'v2Response.queryResult.intent.displayName': intent_name })
.find({ updated: false })
.sort({ responseTime: 1 });
find right project, then right intent, then get back next document in responseTime if it also has the same name (not intentName).
You sample document is not valid JSON that we can cut-n-paste.
The major issue with your query is the chaining of find() calls.
You need to put all conditions in one find() call like:
await History
.find( { project_id: project_id ,
'v2Response.queryResult.intent.displayName': intent_name ,
updated: false } )
.sort( ... ) ;
Got it, thanks!
So right now with your help im finding the startpoint(s), now I need to check if next document has the same name value and then only get those. If possible.
If you want to get all the documents with same name value, you can just filter by name in addition.
History.find({
"project_id": project_id,
"v2Response.queryResult.intent.displayName": intent_name,
"updated": false,
"name": name
}).sort({
"responseTime": 1
})
Hi! Its kinda hard this query i think.
I need to find current project and intent and sort on responseTime
Thats the startpoint. It will be lots of documents. I AM HERE.
But I want nothing of that.
I want the document after that (only one for each of the above), sorted on responseTime, if it exists with the same name. This only cares about if its the next one and if name is the same.
That can also be alot of documents.
You won’t be able to do that with a simple find() query.
You will need to use the aggregation framework using $push within a $group or $setWindowFields.
But without
it is hard to help further.
Thanks for letting me know what i need to learn. I will add some samples here.
For this five documents, two should be returned.
{"_id":{"$oid":"622f2cd4075c226bde0ba966"},"project_id":"sorteringshatt-dgwo","name":"2022-03-14 12:53:52.712155","inputContexts":[],"originalRequestSource":"SORTERINGSHATT_MAI","v2Response":{"responseId":"d317ddda-fbe5-4511-bf2b-0a985a61ecf1-53cb9be6","queryResult":{"queryText":"phase_1_name_mai","allRequiredParamsPresent":true,"fulfillmentText":"Hej, Välkommen till denna upplevelse. Tillsammans ska vi undersöka ditt innersta väsen. Mitt namn är MAI, vad heter du?","fulfillmentMessages":[{"text":{"text":["Hej, Välkommen till denna upplevelse. Tillsammans ska vi undersöka ditt innersta väsen. Mitt namn är MAI, vad heter du?"]},"platform":0}],"outputContexts":[{"name":"projects/sorteringshatt-dgwo/agent/sessions/2022-03-14 12:53:52.712155/contexts/phase1intro-followup","lifespanCount":2},{"name":"projects/sorteringshatt-dgwo/agent/sessions/2022-03-14 12:53:52.712155/contexts/phase1name-followup","lifespanCount":2}],"intent":{"name":"projects/sorteringshatt-dgwo/agent/intents/47119b7f-827c-4f4f-a304-5bbf23d53f39","displayName":"Phase 1: Intro MAI","webhookState":0,"priority":0,"isFallback":false,"mlDisabled":false,"liveAgentHandoff":false,"endInteraction":false,"inputContextNames":[],"events":[],"trainingPhrases":[],"action":"","outputContexts":[],"resetContexts":false,"parameters":[],"messages":[],"defaultResponsePlatforms":[],"rootFollowupIntentName":"","parentFollowupIntentName":"","followupIntentInfo":[]},"intentDetectionConfidence":1,"languageCode":"sv","speechRecognitionConfidence":0,"action":"","webhookSource":""},"webhookStatus":{"code":0,"message":"","details":[]},"outputAudioConfig":{"audioEncoding":1,"sampleRateHertz":44100,"synthesizeSpeechConfig":{"speakingRate":1,"voice":{"name":"sv-SE-Wavenet-D","ssmlGender":0},"pitch":0,"volumeGainDb":0,"effectsProfileId":[]}},"outputAudio":""},"responseTime":"2022-03-14 12:53:56.138199","logType":"PRODUCTION","updated":false,"__v":0}
{"_id":{"$oid":"622f2ce0075c226bde0ba96a"},"project_id":"sorteringshatt-dgwo","name":"2022-03-14 12:53:52.712155","inputContexts":[],"originalRequestSource":"SORTERINGSHATT_MAI","v2Response":{"responseId":"1faf96ba-7fb1-4741-87a1-d67f4ccaeefe-53cb9be6","queryResult":{"queryText":"Fredrik","action":"Phase1Intro.Phase1Intro-custom","parameters":{"person":{"name":"Fredrik"}},"allRequiredParamsPresent":true,"fulfillmentText":"Hejsan Fredrik. Angenämt! Framför oss har vi ett litet personlighetstest för att avgöra vilket värdeord du tillhör. Testet kommer ta några minuter att genomföra.","fulfillmentMessages":[{"text":{"text":["Hejsan Fredrik. Angenämt! Framför oss har vi ett litet personlighetstest för att avgöra vilket värdeord du tillhör. Testet kommer ta några minuter att genomföra."]},"platform":0}],"outputContexts":[{"name":"projects/sorteringshatt-dgwo/agent/sessions/2022-03-14 12:53:52.712155/contexts/phase1name-followup","lifespanCount":1,"parameters":{"person.original":"Fredrik","person":{"name":"Fredrik"}}},{"name":"projects/sorteringshatt-dgwo/agent/sessions/2022-03-14 12:53:52.712155/contexts/phase1intro-followup","lifespanCount":1,"parameters":{"person":{"name":"Fredrik"},"person.original":"Fredrik"}}],"intent":{"name":"projects/sorteringshatt-dgwo/agent/intents/63425b74-2641-498e-8435-23b9b462439b","displayName":"Phase 1: Intro - myNameIs","endInteraction":true,"webhookState":0,"priority":0,"isFallback":false,"mlDisabled":false,"liveAgentHandoff":false,"inputContextNames":[],"events":[],"trainingPhrases":[],"action":"","outputContexts":[],"resetContexts":false,"parameters":[],"messages":[],"defaultResponsePlatforms":[],"rootFollowupIntentName":"","parentFollowupIntentName":"","followupIntentInfo":[]},"intentDetectionConfidence":1,"diagnosticInfo":{"end_conversation":true},"languageCode":"sv","speechRecognitionConfidence":0,"webhookSource":""},"webhookStatus":{"code":0,"message":"","details":[]},"outputAudioConfig":{"audioEncoding":1,"sampleRateHertz":44100,"synthesizeSpeechConfig":{"speakingRate":1,"voice":{"name":"sv-SE-Wavenet-D","ssmlGender":0},"pitch":0,"volumeGainDb":0,"effectsProfileId":[]}},"outputAudio":""},"responseTime":"2022-03-14 12:54:08.498651","logType":"PRODUCTION","updated":false,"__v":0}
{"_id":{"$oid":"6228bb45075c226bde0ba1d1"},"project_id":"sorteringshatt-dgwo","name":"2022-03-09 15:34:42.939426","inputContexts":[],"originalRequestSource":"SORTERINGSHATT_KAI","v2Response":{"responseId":"b907a90a-1432-4fd1-b146-3a92e2a7e163-53cb9be6","queryResult":{"queryText":"phase_2_perfectsaturday","allRequiredParamsPresent":true,"fulfillmentText":"Hur skulle du beskriva den perfekta lördagen?","fulfillmentMessages":[{"text":{"text":["Hur skulle du beskriva den perfekta lördagen?"]},"platform":0}],"outputContexts":[{"name":"projects/sorteringshatt-dgwo/agent/sessions/2022-03-09 15:34:42.939426/contexts/phase2simplequestion3-perfectsaturday-followup","lifespanCount":2}],"intent":{"name":"projects/sorteringshatt-dgwo/agent/intents/239938ed-0d10-429b-9802-ae868dc37a09","displayName":"Phase 2: Simple Question 3 - Perfect Saturday","webhookState":0,"priority":0,"isFallback":false,"mlDisabled":false,"liveAgentHandoff":false,"endInteraction":false,"inputContextNames":[],"events":[],"trainingPhrases":[],"action":"","outputContexts":[],"resetContexts":false,"parameters":[],"messages":[],"defaultResponsePlatforms":[],"rootFollowupIntentName":"","parentFollowupIntentName":"","followupIntentInfo":[]},"intentDetectionConfidence":1,"languageCode":"sv","speechRecognitionConfidence":0,"action":"","cancelsSlotFilling":false,"webhookSource":""},"webhookStatus":{"code":0,"message":"","details":[]},"outputAudioConfig":{"audioEncoding":1,"sampleRateHertz":44100,"synthesizeSpeechConfig":{"speakingRate":1,"voice":{"name":"sv-SE-Wavenet-E","ssmlGender":0},"pitch":0,"volumeGainDb":0,"effectsProfileId":[]}},"outputAudio":""},"responseTime":"2022-03-09 15:35:49.121453","logType":"PRODUCTION","updated":false,"__v":0}
{"_id":{"$oid":"6228bb4d075c226bde0ba1d3"},"project_id":"sorteringshatt-dgwo","name":"2022-03-09 15:34:42.939426","inputContexts":[],"originalRequestSource":"SORTERINGSHATT_KAI","v2Response":{"responseId":"413e5d0b-fdfd-4a3b-8b4b-544e07417032-53cb9be6","queryResult":{"queryText":"utan planer","action":"Phase2SimpleQuestion3-PerfectSaturday.Phase2SimpleQuestion3-PerfectSaturday-fallback","allRequiredParamsPresent":true,"fulfillmentText":"Vilken lördag!","fulfillmentMessages":[{"text":{"text":["Vilken lördag!"]},"platform":0}],"outputContexts":[{"name":"projects/sorteringshatt-dgwo/agent/sessions/2022-03-09 15:34:42.939426/contexts/phase2simplequestion3-perfectsaturday-followup","lifespanCount":1},{"name":"projects/sorteringshatt-dgwo/agent/sessions/2022-03-09 15:34:42.939426/contexts/__system_counters__","lifespanCount":1,"parameters":{"no-input":0,"no-match":1}}],"intent":{"name":"projects/sorteringshatt-dgwo/agent/intents/d1eda7c3-cf96-459c-abab-b937690aa911","displayName":"Phase 2: Simple Question 3 - Perfect Saturday - fallback","isFallback":true,"endInteraction":true,"webhookState":0,"priority":0,"mlDisabled":false,"liveAgentHandoff":false,"inputContextNames":[],"events":[],"trainingPhrases":[],"action":"","outputContexts":[],"resetContexts":false,"parameters":[],"messages":[],"defaultResponsePlatforms":[],"rootFollowupIntentName":"","parentFollowupIntentName":"","followupIntentInfo":[]},"intentDetectionConfidence":1,"diagnosticInfo":{"end_conversation":true},"languageCode":"sv","speechRecognitionConfidence":0,"cancelsSlotFilling":false,"webhookSource":""},"webhookStatus":{"code":0,"message":"","details":[]},"outputAudioConfig":{"audioEncoding":1,"sampleRateHertz":44100,"synthesizeSpeechConfig":{"speakingRate":1,"voice":{"name":"sv-SE-Wavenet-E","ssmlGender":0},"pitch":0,"volumeGainDb":0,"effectsProfileId":[]}},"outputAudio":""},"responseTime":"2022-03-09 15:35:57.307794","logType":"PRODUCTION","updated":false,"__v":0}
{"_id":{"$oid":"62289ea1075c226bde0ba1bb"},"project_id":"sorteringshatt-dgwo","name":"2022-03-09 13:31:16.723215","inputContexts":[],"originalRequestSource":"SORTERINGSHATT_KAI","v2Response":{"responseId":"2d9530a4-31ba-4ab3-a6b7-f101618b909a-53cb9be6","queryResult":{"queryText":"phase_3_outro","allRequiredParamsPresent":true,"fulfillmentText":"Jag tycker att vi tar och går vidare till den sista fasen av testet!","fulfillmentMessages":[{"text":{"text":["Jag tycker att vi tar och går vidare till den sista fasen av testet!"]},"platform":0}],"intent":{"name":"projects/sorteringshatt-dgwo/agent/intents/9f93a04f-4868-45d5-8824-dae8b81e3d7e","displayName":"Phase 3: Outro","endInteraction":true,"webhookState":0,"priority":0,"isFallback":false,"mlDisabled":false,"liveAgentHandoff":false,"inputContextNames":[],"events":[],"trainingPhrases":[],"action":"","outputContexts":[],"resetContexts":false,"parameters":[],"messages":[],"defaultResponsePlatforms":[],"rootFollowupIntentName":"","parentFollowupIntentName":"","followupIntentInfo":[]},"intentDetectionConfidence":1,"diagnosticInfo":{"end_conversation":true},"languageCode":"sv","speechRecognitionConfidence":0,"action":"","cancelsSlotFilling":false,"webhookSource":"","outputContexts":[]},"webhookStatus":{"code":0,"message":"","details":[]},"outputAudioConfig":{"audioEncoding":1,"sampleRateHertz":44100,"synthesizeSpeechConfig":{"speakingRate":1,"voice":{"name":"sv-SE-Wavenet-E","ssmlGender":0},"pitch":0,"volumeGainDb":0,"effectsProfileId":[]}},"outputAudio":""},"responseTime":"2022-03-09 13:33:36.821703","logType":"PRODUCTION","updated":false,"__v":0}
I hope I did it correctly.
Check out this series of blog post as well (see at the bottom of the page the list of blog posts). Great way to learn by doing.
Which two of the five? And for which query?
typo, only one should be returned!
await History
.find( { project_id: “sorteringshatt-dgwo” ,
‘v2Response.queryResult.intent.displayName’: “phase_1_name_mai” ,
updated: false } )
.sort( … ) ;
So here my startpoint / query is #1 and #2 is the one I want back. Since it has the same “name” property and is first in line on the “responseTime” property.
I was able to import your documents.
But none match your query.
Always use triple back ticks when publishing code as mentioned in one of the link above. HTML changes the quotes to fancy back and forward single and double quotes and we cannot cut-n-paste it without editing.
But none match your query.*
Here are the v2Response…displayName of your documents:
{ _id: ObjectId("622f2cd4075c226bde0ba966"),
v2Response: { queryResult: { intent: { displayName: 'Phase 1: Intro MAI' } } } }
{ _id: ObjectId("622f2ce0075c226bde0ba96a"),
v2Response: { queryResult: { intent: { displayName: 'Phase 1: Intro - myNameIs' } } } }
{ _id: ObjectId("6228bb45075c226bde0ba1d1"),
v2Response: { queryResult: { intent: { displayName: 'Phase 2: Simple Question 3 - Perfect Saturday' } } } }
{ _id: ObjectId("6228bb4d075c226bde0ba1d3"),
v2Response: { queryResult: { intent: { displayName: 'Phase 2: Simple Question 3 - Perfect Saturday - fallback' } } } }
{ _id: ObjectId("62289ea1075c226bde0ba1bb"),
v2Response: { queryResult: { intent: { displayName: 'Phase 3: Outro' } } } }
I think it helps if I paint a picture of what I need and why.
We have created a chatbot that stands at some hotels in Stockholm. They are kinda stupid but thats why we are trying with methods like this to make it better.
I’ve made an app that find missMatches and sort them in amount so if alot of people ask where u can find a good restaurant and the chatbot goes to fallback we see that and can prioritize that answer before others.
When i search for a project_id and then an intent, the point is to have that as a start-point and then look at all the next documents that also had that intent in the same conversation. I can now do this, but it is kinda hard on the server to look for everything (and then check for intent, and then check for if next blabla is the same conversation, and do that for everything). Maybe because its a free atlas cluster? Anyway i made a new index 2 days ago sorting project then intents and some magic happened. From 20-30 seconds to 0.5-2 seconds responses.
My query is not complete, its what I use right now because I don’t know better, and that gives me the start-point. After that i use my server to find the answers i need.
So the perfect query would need the projectId and intent, then give me back the documents that has the same projectId and the first name sorted in responseTime (name is conversation, kinda strange but whatever)
Kinda hard! I think. Im learning as fast as i can!
Here is an idea using the aggregation framework.
You start with a $match stage using the query you have in your find().
Then you do $graphLookup from the same collection startingWith:$name, with a connectFromField:name and connectToField:name and maxDepth:0. You also want to restrictSearchWithMatch with an expression that ensure you do not pickup your starting point.
After the $graphLookup, you end up with an array of 1 element that should be what you want. You could then use $replaceRoot to only output that resulting document.
Thank you, that is probably the correct answer.
We will upgrade, I noticed $graphLookup was not available on free atlas tier.
I would be surprised if that is the case. Any links to documentation to that effect?
This is the console.logged catch on the server
MongoServerError: $$graphLookup is not allowed in this atlas tier
at lotsOfText {
ok: 0,
code: 8000,
codeName: 'AtlasError'
}
but yes, you might be correct again. I might just have written the query wrong on my testQuery I did here.
The best way to find out is to share the query with us.