Push collection to S3 in csv

Working with Data Connectors & Integrations
1

Kindly I need a way to push data from mongodb to S3 bucket in csv format, the issue is that the data has a huge number of objects and arrays, so it is not recommended to have a script to take it field by field, and when the code is like

         "$out": {
               "s3": {
                  "bucket": "testmdbbucket",
                  "region": "eu-central-1",
                  "filename": "secairbnb",
                  "format": {
                        "name": "csv"       
                  }
               }
            }

it gives an error in S3 any help with that will be really appreciated, thanks in advance

2

Hi @yousef_osama and welcome in the MongoDB Community :muscle: !

Sharing the exact error you get might help :upside_down_face:.

There are a few other options that you can use to resolve your problem: https://www.mongodb.com/docs/datalake/reference/pipeline/out/#syntax

errorMode could be one solution but if the size is the problem then probably maxFileSize can help?

Cheers,
Maxime.

1 Like