I am trying to write a pipeline to export our current time series data that is stored in a regular collection to a time series collection. I can do it as a one-off using $out for testing say one sensor at a time but I can’t do it for all the data at once as $out overwrites the output collection. I tried using $merge instead, but I get the error “lock for timeseries is not allowed”. Are there any alternatives? If not I will just do it by pulling into Python first then writing, which will be slower.
That’s interesting! I wasn’t aware of this limitation, but there is a lot of stuff going on with timeseries collections, so it’s definitely possible.
Let me check internally and see if I can get you a more authoritative answer.
Currently, we don’t support $merge on time-series collections. We are aware of this limitation and have it on our roadmap as something we want to support in the future. $out is currently the recommended way or custom script might make sense in your case.
For $out, you would have to transform the data in your regular collection into new desired TS schema in a temporary collection and then mongodump/restore can be used on the transformed collection created from the $out
output.
You can find more information on the steps here – https://www.mongodb.com/docs/v5.3/core/timeseries/timeseries-migrate-data-into-timeseries-collection/.