Sava a list with many dictionary to mongoDB

Working with Data Developer Tools
1

Hello everyone,
I would like to fill my mangoDB, but I don’t have an exact plan how to do it, maybe someone of the Python nerds here can give me some tips or show me the way how to do it.

Output situation.

I have a pickle file that was saved as a list with a lot of dictionaries. About 230 pieces plus minu more.

how exactly do i get the dictionaries from the list into mongoDB,

def load_pickle_raw():

    # Lade das
    with open(savefile + 'pickle_raw.txt', 'rb') as l_file:
        load_raw = pickle.load(l_file)

    return load_raw

load_raw = load_pickle_raw()

for lo in range(0, (len(load_raw))):
    print(load_raw[lo])

{'resort': 'Lorem', 'headline1': '"Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis 
parturient montes, nascetur ridiculus mus."', 'headline2': 'Lorem ipsum dolor sit amet, consectetuer adipiscing elit.', 'snippet': 'ddd', 'link': 'https://www.webseite.com', 
'linkhash': '1e5e6b27bde47251aa01c590368cff430d1dd3d2', 'timestamp': '2023-03-29 20:02:20', 'status_create': 'False', 'status_release': 'False'}
many many Dict.....

it is important to know that I have created a HASH and a dict which is already saved in the DB should not be saved again. I have already found a solution for this on https://stackoverflow.com/, but my question is this really the best and most effective method?

userID = ctx.message.author.id
if conn.mydb.mycol.count_documents({ 'userID': userID }):
    await ctx.send("**Error: You're already in the database**")
else:
    await ctx.send("**Adding new inventory to the database**")
    mydict = { "userID": userID, "coin": "0", "inv": {"inv1": "", "inv2": "", "inv3": "", "inv4": "", "inv5": ""} }
    y = conn.mydb.mycol.insert_one(mydict)

I decided to save with pickle because I didn’t like the handling of json and the pickle files are much smaller.

2

This is super easy, try not to over think this friend,

So remember any and all Document DBs or NoSQL DBs essentially devour JSON.

The easiest way to do this, instead of figuring out how to get the dictionaries into MongoDB, figure out how to convert the dictionaries to JSON instead. And then use the import function to import the JSON into MongoDB.

We can accomplish the Conversion from dictionaries to JSON by doing the following in Python:

import json 
# importing the module to show pickles

Pickles_dictionary ={ 
  "pickles_id": "1", 
  "pickles_yeah": "Rick", 
  "eatspicypickle": "sweet"
} 
 

# Dictionaries output
print("The dictionary is as: \n", pickles, "\n")
json_object = json.dumps(pickles,  indent = 4) 

# JSON output for inspection
print("Conversion Completed:", json_object)
3

Basically the main goal is just to convert the dictionaries to JSON and then just import the JSON.

There are like 20 ways to convert entire dictionaries with even thousands of entries into JSON and accomplish this.

This is the better way to do things, and then in your application you match the fields to your outputs.

Then you don’t care about the format stored in MongoDB at all.

4

I have now already managed, here mail my code

def load_pickle_raw():

    with open(savefile + 'pickle_raw.txt', 'rb') as l_file:
        load_raw = pickle.load(l_file)

    return load_raw

load_raw = load_pickle_raw()

res = myCollection.insert_many(load_raw)

I will stay away from JSON, because the data I store with the Pickle Libery is much more compact. and I have now found another way to synchronize the already stored data.

I still thank you for the quick and friendly response here in the forum.

5

All good, if you feel that works better for you, then all good.