MongoDB Go Driver - Capture data transfer metrics

[Clark_McCauley]

I’m trying to measure the number of bytes transferred by the MongoDB Go client. We’re seeing higher than expected data transfer between MongoDB and some of our services, and we’d like to understand which services are moving the most data. Measuring the size of the objects after decoding isn’t a viable option since we could theoretically decode less data into a struct than was passed back from the database to the client.

[Matt_Dale]

@Clark_McCauley thanks for the question and sorry for the slow reply!

You can use a CommandMonitor to get access to all of the request and response messages sent to the database.

For example, to print the number of bytes recorded by the CommandMonitor every 5 seconds:

var bytesMonitored uint64

func main() {
	cmdMonitor := &event.CommandMonitor{
		Started: func(_ context.Context, evt *event.CommandStartedEvent) {
			atomic.AddUint64(&bytesMonitored, uint64(len(evt.Command)))
		},
		Succeeded: func(_ context.Context, evt *event.CommandSucceededEvent) {
			atomic.AddUint64(&bytesMonitored, uint64(len(evt.Reply)))
		},
	}

	client, err := mongo.Connect(
		context.Background(),
		options.Client().ApplyURI("mongodb://myURI").SetMonitor(cmdMonitor))
	if err != nil {
		panic(err)
	}
	defer client.Disconnect(context.Background())

	go func() {
		for range time.Tick(5 * time.Second) {
			log.Print("Current bytes monitored:", atomic.LoadUint64(&bytesMonitored))
		}
	}()

	// Use the client to run operations.
}

A few important caveats:

• The bytes counted by the CommandMonitor will not exactly match the bytes sent over the network. Encryption, compression, MongoDB wire format encapsulation, and other factors may significantly impact the actual number of bytes sent over the network. However, the bytes counted by the CommandMonitor should provide an order-of-magnitude approximation that is comparable across services.
• The CommandMonitor doesn’t give direct access to the raw BSON from failure events. If the database is sending back a lot of failure messages, those will not be counted.