CSV export by a scheduled trigger - MongoDB Stitch

[Omur_Guner (ömür Güner)]

Hi,
We´d like to export an existing collection data (most probably after a simple filter query) to csv and upload the file to an AWS S3 bucket, every night.It is around 40000 documents(ending in ~=15KB csv file) but should scale well as the data is expected grow.

From initial investigation I see we can do this by MongoDB Stitch ; configuring a scheduled trigger connected to function doing the export. But I don´t see any examples/discussions about “export” specifically.
Is this good idea?

[Drew_DiPalma]

Hi Omur – This should definitely be accomplishable with a Trigger. Generally, I think you will want to write a function that follows these steps –

1. Pull the relevant documents from MongoDB
2. Construct a CSV (you will need to include an NPM package via dependency resolution)
3. Upload the CSV to the relevant S3 bucket (using the AWS Service or by importing the AWS SDK)

One caveat is that Stitch currently will only pull up to 50k documents in a single MongoDB request. So, as your data grows you may need to make multiple requests to construct your CSV.