Hi everyone,
I’m working with MongoDB 7.0 and time series collections.
I have an use case for which I need to delete some documents based on their metadata field and their timestamp.
My time-series collection is pretty big, more than 150GB of data and ~ 131846032 documents.
The granularity of the collection is hourly
I noticed that the deletions are very slow, even if there is an index on the metadata and timestamp field and even when my deletion filter matches only 1 or few documents.
More specifically my document structure is the following:
{ ts: <value> // this is the timestamp field
meta: {metafield1: <value1>, metafield2: <value2>, metafield3: <value3>}, // this is the metadata field
metrics: <array of measurements> // this is the value containing the measurements for the represented datapoint
}
The collection has two indexes:
1 ) meta.metafield1_1_ts_1
2) meta_1_ts_1
A typical deletion operation that I’m running is the following:
db.my_time_series_collection.delete_one({meta.metafield1: <value1>, meta.metafield2: <value2>, meta.metafield3: <value3>, ts: <some_timestamp>})
What I’m observing is that the deletion is taking always more than 1 seconds to be performed and it’s often taking more than 10 seconds to delete a single record.
Is there something that I’m doing wrong or is this related to the way MongoDB organizes time-series data?
Thanks in advance