Encryption related exception: A timeout occurred after 10000ms selecting a server using CompositeServerSelector

Working with Data Drivers & ODMs
1

Hi,

I am using CSFLE with .NET core web app hosted on App Service. When I try to run a find query, I am running into this Encryption related exception.

Following are the details of exception:

Note the inner exception messages which reads but makes no sense “System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (10013): An attempt was made to access a socket in a way forbidden by its access permissions”

Source : “MongoDB.Driver”
TargetSite : “EncryptFields”
ErrorType : “MongoDB.Driver.Encryption.MongoEncryptionException”

A timeout occurred after 10000ms selecting a server using CompositeServerSelector{ Selectors = MongoDB.Driver.MongoClient+AreSessionsSupportedServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 }, OperationsCountServerSelector }.

{
ClusterId: “3”,
Type: “Unknown”,
State: “Disconnected”,
Servers: [
{
ServerId: {
“ClusterId”: 3,
“EndPoint”: “Unspecified/localhost:27020”
},
EndPoint: “Unspecified/localhost:27020”,
ReasonChanged: “Heartbeat”,
State: “Disconnected”,
ServerVersion:,
TopologyVersion:,
Type: “Unknown”,
HeartbeatException: “MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server.\r\n —> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (10013): An attempt was made to access a socket in a way forbidden by its access permissions. [::1]:27020\r\n at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)\r\n at System.Net.Sockets.Socket.<>c.b__274_0(IAsyncResult iar)\r\n— End of stack trace from previous location where exception was thrown —\r\n at MongoDB.Driver.Core.Connections.TcpStreamFactory.ConnectAsync(Socket socket, EndPoint endPoint, CancellationToken cancellationToken)\r\n at MongoDB.Driver.Core.Connections.TcpStreamFactory.CreateStreamAsync(EndPoint endPoint, CancellationToken cancellationToken)\r\n at MongoDB.Driver.Core.Connections.BinaryConnection.OpenHelperAsync(CancellationToken cancellationToken)\r\n — End of inner exception stack trace —\r\n at MongoDB.Driver.Core.Connections.BinaryConnection.OpenHelperAsync(CancellationToken cancellationToken)\r\n at MongoDB.Driver.Core.Servers.ServerMonitor.InitializeConnectionAsync(CancellationToken cancellationToken)\r\n at MongoDB.Driver.Core.Servers.ServerMonitor.HeartbeatAsync(CancellationToken cancellationToken)”,
LastHeartbeatTimestamp: “2021-08-30T08:05:59.8615815Z”,
LastUpdateTimestamp: “2021-08-30T08:05:59.8615818Z”
}
]
}

Stack Tarce

MongoDB.Driver.Encryption.AutoEncryptionLibMongoCryptController.EncryptFields(String databaseName, Byte[] unencryptedCommandBytes, CancellationToken cancellationToken)
at MongoDB.Driver.Core.WireProtocol.CommandMessageFieldEncryptor.EncryptFields(String databaseName, CommandRequestMessage unencryptedRequestMessage, CancellationToken cancellationToken)
at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol1.AutoEncryptFieldsIfNecessary(CommandRequestMessage unencryptedRequestMessage, IConnection connection, CancellationToken cancellationToken) at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol1.Execute(IConnection connection, CancellationToken cancellationToken)
at MongoDB.Driver.Core.WireProtocol.CommandWireProtocol1.Execute(IConnection connection, CancellationToken cancellationToken) at MongoDB.Driver.Core.Servers.Server.ServerChannel.ExecuteProtocol[TResult](IWireProtocol1 protocol, ICoreSession session, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Servers.Server.ServerChannel.Command[TResult](ICoreSession session, ReadPreference readPreference, DatabaseNamespace databaseNamespace, BsonDocument command, IEnumerable1 commandPayloads, IElementNameValidator commandValidator, BsonDocument additionalOptions, Action1 postWriteAction, CommandResponseHandling responseHandling, IBsonSerializer1 resultSerializer, MessageEncoderSettings messageEncoderSettings, CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.CommandOperationBase1.ExecuteProtocol(IChannelHandle channel, ICoreSessionHandle session, ReadPreference readPreference, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.ReadCommandOperation1.ExecuteAttempt(RetryableReadContext context, Int32 attempt, Nullable1 transactionNumber, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.RetryableReadOperationExecutor.Execute[TResult](IRetryableReadOperation1 operation, RetryableReadContext context, CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.ReadCommandOperation1.Execute(RetryableReadContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.FindCommandOperation1.Execute(RetryableReadContext context, CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.FindOperation1.Execute(RetryableReadContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.FindOperation1.Execute(IReadBinding binding, CancellationToken cancellationToken) at MongoDB.Driver.OperationExecutor.ExecuteReadOperation[TResult](IReadBinding binding, IReadOperation1 operation, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.ExecuteReadOperation[TResult](IClientSessionHandle session, IReadOperation1 operation, ReadPreference readPreference, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.ExecuteReadOperation[TResult](IClientSessionHandle session, IReadOperation1 operation, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.FindSync[TProjection](IClientSessionHandle session, FilterDefinition1 filter, FindOptions2 options, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.<>c__DisplayClass45_01.<FindSync>b__0(IClientSessionHandle session) at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSession[TResult](Func2 func, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.FindSync[TProjection](FilterDefinition1 filter, FindOptions2 options, CancellationToken cancellationToken)
at MongoDB.Driver.FindFluent2.ToCursor(CancellationToken cancellationToken) at MongoDB.Driver.IAsyncCursorSourceExtensions.FirstOrDefault[TDocument](IAsyncCursorSource1 source, CancellationToken cancellationToken)
at MongoDB.Driver.IFindFluentExtensions.FirstOrDefault[TDocument,TProjection](IFindFluent`2 find, CancellationToken cancellationToken)

2

Also note that I can connect to this mongodb server from my app service, perform curd operations on collections which are not part of automic CSFLE.

Important factor here is I am able to run this find query from my laptop without any issue but same code fails when it is hosted on app service.

3

@Arvind_Bhairat did you get any resolution? I am also facing issue after deployment.

4

@Arvind_Bhairat I am able to fix this issue by installing Mongodb Enterprise on web server. You also have to spawn the mongocryptd to listen on 27020.

5

Good to here that you managed to resolve the issue. In my case, the app is hosted in app service environment (serverless) and mongodb is hosted on atlas (managed cloud service by mongodb). I had to switch to explicit CSFLE in order to make it work.

6

I had received following response from mongodb support team which I haven’t really tried myself. If someone is still looking for an answer, this might help.

The error message indicates, .NET/C# driver is attempting to connect to mongocryptd.exe on port 27020 (default mongocryptd port) over IPv6. ::1 is the IPv6 localhost address whereas 127.0.0.1 is the IPv4 localhost address.

an attempt was made to access a socket in a way forbidden by its access permissions. [::1]:27020

Running mongocryptd --help, we can see:

–ipv6 Enable IPv6 support (disabled by default)

mongocryptd doesn’t listen on IPv6 addresses by default.

You can enable IPv6 support via:

var extraOptions = new Dictionary<string, object>()

{

    { "mongocryptdSpawnArgs", new [] { "--ipv6" } },

};
autoEncryptionOptions.With(extraOptions: extraOptions);

Alternatively you can override mongocryptdURI to explicitly specify 127.0.01:27020:

var extraOptions = new Dictionary<string, object>()

{

    { "mongocryptdURI", "mongodb://127.0.0.1:27020" } },

};

autoEncryptionOptions.With(extraOptions: extraOptions);

Next Steps:

  • Please test either of the above changes and confirm if application is able to connect to mongod when using CSFLE.

Note: You have to either enable IPv6 support OR connect to mongocryptd.exe via IPv4 . You don’t have to do both.

In the meantime, please let us know if you have any further questions.