Hello Everybody!
How are you doing today? I hope that everybody is safe and sound! I have a necessity to interact over a large database, we are talking about 16 MM records. I’m using python with mongoengine to interact over inside my application. I’m using a simple queryset interacting using a for loop.
But it is costing many hours to process all queryset and it is causing a bottleneck in my system
I would like know if mongo (through) pymongo has alternatives to interact more faster where it can be reduced from hours to minutes.
This is the code:
contacts = Contact.objects.filter(__raw__=query).only('id', 'email', 'lead_score').no_cache().batch_size(10000)
# query is a simple dictionary where it is place the search criteria
Estou iterando pelo for abaixo:
for index, contact in enumerate(contacts, 1):
print contact.name
Thank you so much for your help guys!
Rogério Carrasqueira