Okay, so I have some (TB sized) collections (observation data) that I’d like to copy over to equivalent time-series collections (currently I’m using 5.X, self-managed). The current collection has a proper UTC date-time as a “string” and time-series of course requires that to be a legit UTC “date”.
Option A, do I introduce a new UTC date field (based on the string that exists) into the existing collection that permeates the entire collection? First of all, is that bad? I’m not 100% sure what Mongo would do at the locking level if a command like that was issued (on a production system).
Option B: Pre-create the Time-Series collection, then attempt to populate it (from the original) while copy/converting that “string” UTC date to the actual UTC date required by time-series? Perhaps there is an option C that I haven’t thought of (other than "just retire already!)? I’d greatly appreciate a nudge in the right direction.
Could you please share the specific use case and the rationale behind opting for the time-series collections? Moreover, it’s essential to note that in time series collections, data insertion occurs in accordance with the timestamp, which ultimately results in the creation of meta-bucket collections.
I recommend you consider inserting data in a sorted manner based on the timestamp. By doing so, you can fully leverage the advantages of time series data, ensuring efficient querying and analysis.
Great, thank you for responding. The types of use cases is 100% on par with the concept of weather sensors that repeatedly give new readings at well defined intervals. The rationale is that we were using a pre 5.X version of MongoDB that didn’t support Time Series Collections and thought that a handful of our collections fit the mold of Time Series. I will look at inserting/experimenting with sorted order with some of the smaller collections and see how that goes. Thanks again!
Thanks Kushagra, I’ll have a second look at Kafka. At first glance I thought it was too complicated for something I should be able to do with NoSQLBooster or export/import features. I’m totally old school when it comes to doing things like this, so I’ll take your advice and see how we go. Thank again.