ZStandard Compression not working in MongoDB v4.2.7

[Asmaa_Hamdi]

I’m trying to utilize ZStandard compression but for some reason whenever I add a new collection, its block_compression is set to “snappy”.
I’ve edited the mongod.cfg file and set the block_compression field under WiredTiger to “zstd” and I specified zstd compression in my MongoSettings while creating the MongoClient in my c# code.
Is there something else I’m supposed to do for Zstd compression to work?

This is my C# code:

var mongoClientSettings = new MongoClientSettings();

mongoClientSettings.Compressors = new List<CompressorConfiguration>() {
       new CompressorConfiguration(CompressorType.ZStandard)
    };

var client = new MongoClient(mongoClientSettings);

IMongoDatabase testdb = client.GetDatabase("testdb");

var eaiRequestLogsCollection = testdb.GetCollection<EAIRequestsLogMDB>("EAIRequestsLogs");

eaiRequestLogsCollection.InsertMany(eAIRequestsLogMDBs);

This is the part I edited in the mongo.cfg file:

storage:
  dbPath: C:\Program Files\MongoDB\Server\4.2\data
  journal:
    enabled: true
  engine: "wiredTiger"
  wiredTiger:
      collectionConfig:
         blockCompressor: "zstd"

Note: I’m using MongoDB v4.2.7 along with the .Net MongoDB driver v2.11.0(beta v) on a windows 10 machine.

[Stennie_X (Stennie)]

Welcome to the community @Asmaa_Hamdi!

@Asmaa_Hamdi

mongoClientSettings.Compressors = new List<CompressorConfiguration>() {
       new CompressorConfiguration(CompressorType.ZStandard)
    };

This option is for network compression, so isn’t directly related to the compression used for collections on disk. Network compression and on-disk compression can use different compressors.

Can you confirm if the collection you are referencing was created before changing the blockCompressor option in your MongoDB config file? The blockCompressor option sets the default compressor for newly created collections but does not affect existing collections.

If you want to compare the effect of different compressors for your data, you can also pass block_compressor as a storage engine option when creating a collection.

Here’s a quick example in the mongo shell:

// Create new collection using zstd compression
> db.createCollection("zlogs", {storageEngine: {wiredTiger: {configString: "block_compressor=zstd"}}})
{ "ok" : 1 }

// Check which block compressor is being used for a collection
> var wt_options = db.zlogs.stats().wiredTiger.creationString.split(',')
> wt_options.filter((wt_options) => wt_options.startsWith('block_compressor'))
[ "block_compressor=zstd" ]

The .NET driver has an analogous CreateCollection command which supports setting the block compressor via the StorageEngine property in the CreateCollectionOptions class.

Regards,
Stennie

[Asmaa_Hamdi]

The collection was created after I edited the config but I was setting the Network compression instead of block compression as you explained in your reply.

I used CreateCollectionOptions class as suggested and it worked!
This is the C# code in case anyone needs it later on:

           IMongoDatabase testdb = client.GetDatabase("testdb");

           testdb.CreateCollection("Hamsters", new CreateCollectionOptions()
           {
               StorageEngine = new BsonDocument {
                   { "wiredTiger", new BsonDocument {
                       {  "configString" , "block_compressor=zstd"  }
                   } }
               }
           });```

Thank you

[Stennie_X (Stennie)]

@Asmaa_Hamdi

The collection was created after I edited the config but I was setting the Network compression instead of block compression as you explained in your reply.

I used CreateCollectionOptions class as suggested and it worked!

Hi @Asmaa_Hamdi,

Great to hear you have a solution.

FYI, the CreateCollectionOptions approach should only be needed if you explicitly want to set a compressor.

If you change the server’s blockCompressor default, all new collections created should use the current default compressor without having to specify any additional collection options. Make sure you restart the mongod service after changing any server configuration defaults to have changes take effect.

Regards,
Stennie

[Asmaa_Hamdi]

I set the block_compressor setting to zstd in the mongod.cfg file and restarted the server but it still defaults to snappy compression unless I explicitly specify zstd compression via the CreateCollection Options class.

Regards,
Asmaa