Date format Handling from JSON to BSON using GO

Working with Data Drivers & ODMs
, , ,
#1

Hi

We have to import several different JSON Exports into MonogDB using Go. What we do is fetching the JSON from a database, unmarshalling it into a struct and save the struct into a Mongo collection. This works. The problem I struggle is the handling of the dates. The dates are stored in the format “YYYY-MM-DDTHH:MM:SS” in the incoming JSON. When using the normal unmarshal function it claims with a parsing time error “parsing time \"\\\"2023-01-10T08:56:02\\\"\" as \"\\\"2006-01-02T15:04:05Z07:00\\\"\": cannot parse \"\\\"\" as \"Z07:00\"”.

For this reason, I created then an own type

type MyDate time.time

and a implmented the UnmarshalJSON interface.

func (v *MyDate) UnmarshalJSON(b []byte) error {
   parsedTime, err := time.Parse("\"2006-01-02T15:04:05\"", string(b))
   if err != nil {
      return err
   }
   *v = MyDate(parsedTime)
   return nil
}

So far so good, I got my date in my Go struct

type MyData struct {
   FieldS string
   FieldI int64
   FieldD *MyDate
}

When I now try to save this struct to MongoDB, it creates an empty Object for MyDate. Therefore I implmented the MarshalBSON interface:

func (v FcsDate) MarshalBSON() ([]byte, error) {
   return bson.Marshal(map[string]map[string]interface{}{
      "$date": {"$numberLong": fmt.Sprintf("%v", time.Time(v).UnixMilli())},
   })
}

This works, and my date is stored in the collection, but as it seems, it’s not really a date. When checking with Compass, it’s shown as an object, when checking with Atlas it’s shown as a date. When trying to project it using an aggregation pipeline it claims with “can’t convert from BSON type object to Date”. When I look into another collection from another running application holding a date, then the JSON representation is the same {$date:{$numberLong: nnn}}

What am I doing wrong? I could use string instead of a date, and then everything works technically fine, but my date is still not a date, it’s a string. What solution would you suggest to solve this issue?

#2

@Meinrad_Hermanek thanks for posting and welcome!

The implementation you posted has two issues:

  1. The string that looks like {"$date": ... } is actually the Extended JSON representation for a BSON “UTC datetime” field, not the BSON representation. By default, a Go time.Time is marshaled to a BSON “UTC datetime” field, but the current MarshalBSON function is actually returning a nested document with a single field called $date.
  2. Implementations of the bson.Marshaler interface (i.e. the MarshalBSON function) must return an entire BSON document. However, what you want to do is override the encoding for a field, not create a nested document. To encode an individual field in a BSON document, you actually want to implement the bson.ValueMarshaler interface instead.

To resolve those two issues, replace MarshalBSON with MarshalBSONValue and return the default BSON field encoding for a Go time.Time value:

func (v FcsDate) MarshalBSONValue() (bsontype.Type, []byte, error) {
	return bson.MarshalValue(time.Time(v))
}

See an example on the Go Playground here.

1 Like
#3

Many thanks, this solves my issue!

#4

@Matt_Dale thanks again for your hint.

Now, when I try to Unmarshal this bson value back into my FcsDate, I’m struggling again. I tried following:

func (v *FcsDate) UnmarshalBSONValue(t bsontype.Type, b []byte) (err error) {
	log.Printf("UnmarshalBSONValue: %v, %v", t, b)
	ts := int64(binary.BigEndian.Uint64(b))
	// ts is in nanoseconds, so we convert it to seconds
	*v = FcsDate(time.Unix(ts/1000/1000/1000, (ts%1000)*1000000).UTC())
	log.Printf("FcsSate is: %v", *v)
	return nil
}

This returns a date, but it’s wrong. For example it returns “1970-06-06T09:59:31.880Z” instead of “1997-05-23T00:00:00.000+00:00”.

Do you have any advise?

Regards

Meinrad

#5

Hey @Meinrad_Hermanek thanks for the follow-up question! To unmarshal the BSON value, create a bson.RawValue and use that to unmarshal the bytes into a Go time.Time value:

func (v *MyDate) UnmarshalBSONValue(t bsontype.Type, b []byte) error {
	rv := bson.RawValue{
		Type:  t,
		Value: b,
	}

	var res time.Time
	if err := rv.Unmarshal(&res); err != nil {
		return err
	}
	*v = MyDate(res)

	return nil
}

See an example on the Go Playground here.

P.S. I’m surprised that there isn’t a corresponding UnmarshalValue function in the bson package. That seems like an oversight and definitely makes satisfying the ValueUnmarshaler interface a lot less intuitive. There is an open Jira ticket for adding an UnmarshalValue function (see GODRIVER-1892) that I will suggest the Go driver team prioritize (I work on the Go driver team).

1 Like
#6

Hey @Matt_Dale many thanks for this solution. If I could vote for the Jira you mentioned, I would.