Point in time restore in Atlas

MongoDB Atlas
1

Hello,
I’m new to MongoDB and I have a question regarding the Point in Time Recovery feature in Atlas.
What timestamp I need to provide to the fields “Timestamp” and “Increment”?

My understanding is that I need to provide the timestamp of the operation, until which I want to restore, excluding the operation itself.

For example, I am executing several updates in a collection, followed by a delete. During the point in time restore, I am providing the timestamp of the delete operation. I expect data to be restored until the delete operation, excluding the delete itself. However the restore replays also the delete operation.

According to the documentation, the oplog replay will include all operations occurring up to the critical timestamp, but will not include the operation with the critical timestamp. However in my case the replay includes the bad transaction.

In the Atlas web interface I think the field Oplog timestamp -> Timestamp & Increment is the parameter --oplogLimit ? Seems this is not the case.
"–oplogLimit: Prevents mongorestore from applying oplog entries with timestamp newer than or equal to "

2

Hi… The timestamp we need for the purposes of PIT restore is documented here → https://docs.mongodb.com/manual/reference/bson-types/#timestamps

Let me know if this makes.

3

Hi Benjamin, thank you for the answer.

Well, I still have the same question - see one of my scenarios attached. I still don’t understand if the PITR timestamp has to be the last known good transaction timestamp or the critical(bad) transaction timestamp.

I think the critical transaction should not be replayed during a PITR.

I will try to illustrate:

My oplog looks like:

/* 1 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a1a757767a99114d71"),
        "DBAName" : "StudentBE-EU",
        "MongoDBDateCourse" : ISODate("2020-11-05T10:44:50.107Z")
    },
    "ts" : Timestamp(1604573089, 1),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T10:44:49.998Z"),
    "v" : NumberLong(2)
}
 
/* 2 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114d7c"),
        "DBAName" : "StudentNL-EU",
        "MongoDBDateCourse" : ISODate("2020-11-05T10:44:50.131Z")
    },
    "ts" : Timestamp(1604573090, 1),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T10:44:50.022Z"),
    "v" : NumberLong(2)
}
 
/* 3 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114d87"),
        "DBAName" : "StudentLU-EU",
        "MongoDBDateCourse" : ISODate("2020-11-05T10:44:50.154Z")
    },
    "ts" : Timestamp(1604573090, 2),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T10:44:50.045Z"),
    "v" : NumberLong(2)
}
 
/* 4 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114d93"),
        "DBAName" : "StudentUS-NA",
        "MongoDBDateCourse" : ISODate("2020-11-05T10:44:50.178Z")
    },
    "ts" : Timestamp(1604573090, 3),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T10:44:50.069Z"),
    "v" : NumberLong(2)
}
 
/* 5 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114d9e"),
        "DBAName" : "StudentCA-NA",
        "MongoDBDateCourse" : ISODate("2020-11-05T10:44:50.202Z")
    },
    "ts" : Timestamp(1604573090, 4),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T10:44:50.093Z"),
    "v" : NumberLong(2)
}
 
/* 6 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114da9"),
        "DBAName" : "StudentMX-LA",
        "MongoDBDateCourse" : ISODate("2020-11-05T10:44:50.230Z")
    },
    "ts" : Timestamp(1604573090, 5),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T10:44:50.122Z"),
    "v" : NumberLong(2)
}
 
/* 7 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114db4"),
        "DBAName" : "StudentBR-LA",
        "MongoDBDateCourse" : ISODate("2020-11-05T10:44:50.256Z")
    },
    "ts" : Timestamp(1604573090, 6),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T10:44:50.148Z"),
    "v" : NumberLong(2)
}
 
/* 8 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114dbf"),
        "DBAName" : "StudentAR-LA",
        "MongoDBDateCourse" : ISODate("2020-11-05T10:44:50.282Z")
    },
    "ts" : Timestamp(1604573090, 7),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T10:44:50.174Z"),
    "v" : NumberLong(2)
}
 
/* 9 */
{
    "op" : "d",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114d93")
    },
    "ts" : Timestamp(1604574293, 1),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T11:04:53.510Z"),
    "v" : NumberLong(2)
}
 
/* 10 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3dc7ea757767a99118a1c"),
        "DBAName" : "StudentBH-ME",
        "MongoDBDateCourse" : ISODate("2020-11-05T11:05:34.534Z")
    },
    "ts" : Timestamp(1604574334, 1),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T11:05:34.423Z"),
    "v" : NumberLong(2)
}
 
/* 11 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3dc7ea757767a99118a28"),
        "DBAName" : "StudentSA-ME",
        "MongoDBDateCourse" : ISODate("2020-11-05T11:05:34.561Z")
    },
    "ts" : Timestamp(1604574334, 2),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T11:05:34.449Z"),
    "v" : NumberLong(2)
}
 
/* 12 */
{
    "op" : "i",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3dc7ea757767a99118a34"),
        "DBAName" : "StudentOM-ME",
        "MongoDBDateCourse" : ISODate("2020-11-05T11:05:34.588Z")
    },
    "ts" : Timestamp(1604574334, 3),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T11:05:34.477Z"),
    "v" : NumberLong(2)
}

Critical bad transaction - the delete -

/* 9 */
{
    "op" : "d",
    "ns" : "students.from",
    "ui" : UUID("f69dcfbc-90a0-4cd2-b83d-55716b8e6742"),
    "o" : {
        "_id" : ObjectId("5fa3d7a2a757767a99114d93")
    },
    "ts" : Timestamp(1604574293, 1),
    "t" : NumberLong(12),
    "wall" : ISODate("2020-11-05T11:04:53.510Z"),
    "v" : NumberLong(2)
}

Restore:

Option 1 - provide a date - RPO 1 minute → 11:03 – works as expected

“wall” : ISODate(“2020-11-05T11:04:53.510Z”)

Result - good - as expected
************ 8 documents, without the 3 last updates

Option 2 - provide the critical timestamp – doesn’t work as expected

“ts” : Timestamp(1604574293, 1)

Result - not good
************ only 7 documents – the delete operation is replayed as well.

I’m investigating why the PITR timestamp has to be the last know good and not the last critical (bad) transaction timestamp.

4

You would want to use the timestamp before the delete operation… not the timestamp of the delete operation, as it is expected to apply the timestamp your restoring too

1 Like
5

Thank you, Benjamin, for the confirmation. So PITR requires the last known good timestamp.
We can close the topic as resolved.

6

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.