Inserting documents into a collection from another collection

Working with Data
1

I have a mongo collection collection1 with ~ 1000 million records. Now I want to use value of certain fields from the documents of this collection, apply some transformation and insert a new document into another collection collection2.

What is the recommended way to do this (considering the scale) while having some kind of observability to the process? If it helps, this is on mongo atlas.

3

You write a aggregation pipeline that

  1. $match the documents you want to transform and insert into the other collection
  2. $project/$set/$addFields to make the transformation
  3. you $merge or $out into the other collection
2 Likes