Timeseries - deleting data - golang

Working with Data Drivers
, ,
1

Long story short:
I need to delete records from a Timeseries using golang. After about 10 iterations of trying it using different ways I got the closest to this.

I have a pipeline:

years := bson.A{2018, 2019, 2020, 2021, 2022, 2023, 2024}
pipeline := mongo.Pipeline{
	{{Key: "$addFields", Value: bson.M{"year": bson.M{"$year": "$t"}}}},
	{{Key: "$match", Value: bson.M{"year": bson.M{"$nin": years}}}},
}

Using it I get the list of all _ids I want to delete. But then the DeleteMany refuses with:

write exception: write errors: [Cannot perform an update or delete on a time-series collection when querying on a field that is not the metaField 'c']
exit status 1

The code to delete is:

deleteFilter := bson.M{"_id": bson.M{"$in": idsToDelete}}
deleteResult, err := collection.DeleteMany(context.TODO(), deleteFilter)

All the records are the same:

{
  "t": {
    "$date": {
      "$numberLong": "1662152400000000"
    }
  },
  "c": "921e00fe0a6a306",
  "_id": {
    "$oid": "635e7222afc17c0064ff796e"
  },
  "v": {
    "$numberLong": "908698"
  }
}

There is an index t_1_c_1 (yes, t ASC, c ASC).

Is this solvable?

NOTE: I need to use the filter expression as is. Cannot use anything like year > xxxx and year < yyyy!

Any idea? Thanks in advance!

2

@Tomas_Kucera thanks for the question!

When the timeseries collection was introduced in MongoDB 5.0, there were many limitations on what delete operations you could perform, including that deletes can only filter on the metaField. Most of those limitations were resolved in MongoDB 7.0. Check out the Deletes section in the timeseries documentation for more information.

For example, using MongoDB 5.x and 6.x, and given the record structure you described, you can only filter deletes using the metaField "c":

deleteFilter := bson.M{"c": "some c value"}
deleteResult, err := collection.DeleteMany(context.TODO(), deleteFilter)

However, starting in MongoDB 7.0, you’re able to filter deletes on any value.

Some questions:

  • What version of MongoDB are you currently using?
  • Are you able to upgrade to MongoDB 7.0?
1 Like