Duplicate Key Using Data Federation $out to S3

MongoDB Atlas App Services & Realm Atlas Triggers & Functions
,
1

Hi,

We recently started using Data Federation and Triggers in order to export data as Parquet files to an S3 bucket. It’s working perfectly fine for the most part, however, for one of our collections we are getting the following error:

(DuplicateKey) cannot write document with duplicate key: “countryIso” to parquet file: cannot create BSONSchema from document with duplicate keys: countryIso

The documents in that collection contains a “deliveryAddress” field which contains an array of embedded documents, each containing fields such as “countryIso”, “city”, and so on. Is this what’s causing it? I’m fairly sure that I read that embedded documents should be supported. If not, how do we get around this?

Thanks in advance,
Tom

2

Were you able to solve this issue? I am facing the same problem.

3

Unfortunately not, but I also haven’t really looked into it any further. We could do without that particular data, so we ended up using a $project stage to strip those fields.