How to pass variable to mongo pipeline from python

Working with Data
,
1

Please provide samples to pass variable from python to mongo aggregate

2

Please see the following thread.

3

My python code is as below:

pipeline= … “_id” : “$var_id”

collection.aggregate(pipeline)

Here, how to pass value for $var_id?

Pipeline query comes from mongo…it has got more stages… i want to pass value for 5 variables at different srages…

Please advise…

4

If var_id is a python variable, then you just use it as a python variable.

With not quotes and no dollar sign:

pipeline= … "_id" : var_id
5

This works only if i pass mongo query directly to aggregate…

Whereas my requirement is i am fetching query from mongo…and it is string…when i pass string to aggregate it throws error as aggregate accept only list…so i am using json.loads. and it expects double quotes for variable…

So how to pass variable from python to mongo pipeline…
My pipeline varrable has big mongo query with many parameters to be psses

pipeline= ($match: var_id), ($match: date: todasdate)
collection.aggregate(pipeline)

6

Could you please me solution for this?

7

@Krishnamoorthy_Kalidoss I assume you are using pymongo as your MongoDB driver.

pymongo shows a few aggregation examples in their documentation that may be helpful.

Without more details about your python code, etc., it’s difficult to be fully confident answering you, but perhaps the pipeline assignment you are looking for is:

pipeline = [{"$match": {"_id": var_id, "date": todaysdate}}]
8

Share a few examples of the strings you get from reading the query.

Share the results you get running json.loads on your sample query strings.

Share the variables you need to inject into your queries.

9

i have pipeline in mongodb as below
pipeline = [{"$match": {"_id": “${var_id}”, “date”: ${“todaysdate}” }]

in python i have retrieved the above pipeline from mongo and stored in variable

Now, srcQuerry=[{"$match": {"_id": “${var_id}”, “date”: ${“todaysdate}” }]

now i am using aggreagate

result = collection.aggregate(srcQuery)

how to pass value to var_id and date from srcQuery??

10

Please read Formatting code and log snippets in posts and reformat your code appropriately.

Check your quotes, they are not consistents.

It is not clear if srcQuery is an array with an object or if it is still a string. Share the code you use tk read the string from mongo and that parse it into an array.

Is it

you want to inject or var_id and todaysdate?

Where did you find that you could use ${var_id} to substitute variables in a string? Any reference to python API that mentioned that? The dollar syntax is more a shell thing rather than python. May be what you want is more like

11

i have tried the below steps:

  1. stored the below pipeline in mongo db
    [{"$match": {"_id": var_id, “date”: todaysdate}}]

  2. retrieved the pipeline from mongo using python and stored in variable as below
    pipeline= [($match: var_id), ($match: date: todasdate)]
    here pipeline is string.

collection.aggregate(pipeline) // This line throws error as aggregate accepts only list not string.

so i have used json.loads(pipeline)

again now i got error as json expects double quotes for var_id… if i use double quote for var_id , value will not get assigned…

Hope this helps …Please help here

12

Your code is still not formatted as recommended in the code snippet link I provided.

Your stored query is still not using the % syntax as shown in the stackoverflow post I shared.

You have to replace the variables in the string using the % syntax before you call json.loads().

13

Thanks Steeve!!

I have tried as below:

in Mongo DB i have below pipeline:


      "_id" : %s,
    "my_dt" : ISODate(%s)

in Python i have as below:

_id and my_dt from mongo db substituted with id and myDate variables in python. srcQuery has pipeline fetched from Mongodb.


    srcQueryToAggregate = srcQuery % ('"'+id+'"','"'+myDate+'"')
    print(srcQueryToAggregate)    
    pipeline = json.loads(srcQueryToAggregate)

It prints as below:

```

          "_id" : "XYZ-1"
	     "my_dt" :  ISODate("2022-10-31T04:00:00Z") 
```

Although printing is good, i m getting below error for date at json.loads … ("_id" working fine.)

file “C:\Program Files\Python38\lib\json\decoder.py” line 2, in raw_decode
raise JSONDecodeError(“Expecting Value”, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value : line 2 column 24(char)

i have date value… but still it throws error as Expecting value for my_dt. … Please advise

14

Rather than

you need to use

"my_dt" : %s

and evaluate ISODate before doing the substitution.

Also since _id is a string, it needs to be quoted in the substitution. It has to be something like

"_id" : "%s" ,

rather than

This is outside the scope of this forum since it is strictly Python knowledge. And I am not a Python programmer, I actually do not like Python, since it uses a non-visible character for indentation, the same error make’s authors did a long time ago. Stackoverflow might be better for a followup unless a user of this forum more fluent with Python takes over.

15

Hi Krishna,

You may want to checkout this library GitHub - VianneyMI/monggregate: Open souce project to make MongoDB aggregation framework easy to use in python to ease the process of building new pipelines. It will guide you along the way.