Exception due to Socket Connection Closed or TimeOut or Reset

I’m currently working on a backend application using
Spring: 2.7.3
Java: 1.8
Database: MongoDB Serverless

I’m using MongoDB default URI to connect to the database. I believe the default connection pool size is 100 and the idle time before close for each connection is 10 minutes.

From time to time I keep getting the following issue. For a quick fix, I have used spring @Retryable annotation to retry the methods if the given exception is encountered. By the way, I have also enabled a scheduled job (that calls a simple find query every 2 minutes) to avoid MongoDB serverless cold start issues.

What can be done or can configuration changes be made to avoid the given exception? Please suggest.

May 22 05:00:00 ip-172-31-4-152 web: org.springframework.dao.DataAccessResourceFailureException: Exception sending message; nested exception is com.mongodb.MongoSocketWriteException: Exception sending message
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:85) ~[spring-data-mongodb-3.4.2.jar!/:3.4.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:3044) ~[spring-data-mongodb-3.4.2.jar!/:3.4.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2980) ~[spring-data-mongodb-3.4.2.jar!/:3.4.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2667) ~[spring-data-mongodb-3.4.2.jar!/:3.4.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2649) ~[spring-data-mongodb-3.4.2.jar!/:3.4.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:902) ~[spring-data-mongodb-3.4.2.jar!/:3.4.2]
May 22 05:00:00 ip-172-31-4-152 web: at com.onthecase.config.database.InheritanceAwareSimpleMongoRepository.findAll(InheritanceAwareSimpleMongoRepository.java:51) ~[classes!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.onthecase.config.database.InheritanceAwareSimpleMongoRepository.findAll(InheritanceAwareSimpleMongoRepository.java:16) ~[classes!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.repository.core.support.RepositoryMethodInvoker$RepositoryFragmentMethodInvoker.lambda$new$0(RepositoryMethodInvoker.java:289) ~[spring-data-commons-2.7.2.jar!/:2.7.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:137) ~[spring-data-commons-2.7.2.jar!/:2.7.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:121) ~[spring-data-commons-2.7.2.jar!/:2.7.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:530) ~[spring-data-commons-2.7.2.jar!/:2.7.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:286) ~[spring-data-commons-2.7.2.jar!/:2.7.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:640) ~[spring-data-commons-2.7.2.jar!/:2.7.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:164) ~[spring-data-commons-2.7.2.jar!/:2.7.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:139) ~[spring-data-commons-2.7.2.jar!/:2.7.2]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at com.sun.proxy.$Proxy129.findAll(Unknown Source) ~[na:na]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) ~[spring-tx-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at com.sun.proxy.$Proxy129.findAll(Unknown Source) ~[na:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.onthecase.repository.InvestigatorRepository$findAll$0.call(Unknown Source) ~[na:na]
May 22 05:00:00 ip-172-31-4-152 web: at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-3.0.12.jar!/:3.0.12]
May 22 05:00:00 ip-172-31-4-152 web: at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) ~[groovy-3.0.12.jar!/:3.0.12]
May 22 05:00:00 ip-172-31-4-152 web: at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:130) ~[groovy-3.0.12.jar!/:3.0.12]
May 22 05:00:00 ip-172-31-4-152 web: at com.onthecase.schedule.DeleteUnpaidUsers.softDeleteInvestigator(DeleteUnpaidUsers.groovy:37) ~[classes!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:95) [spring-context-5.3.22.jar!/:5.3.22]
May 22 05:00:00 ip-172-31-4-152 web: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.lang.Thread.run(Thread.java:750) [na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: Caused by: com.mongodb.MongoSocketWriteException: Exception sending message
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:684) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:555) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:381) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:329) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:116) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:644) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:71) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.LoadBalancedServer$LoadBalancedServerProtocolExecutor.execute(LoadBalancedServer.java:159) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:226) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:126) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:116) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.CommandOperationHelper.createReadCommandAndExecute(CommandOperationHelper.java:232) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.FindOperation.lambda$execute$1(FindOperation.java:695) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.OperationHelper.lambda$withSourceAndConnection$2(OperationHelper.java:575) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.OperationHelper.withSuppliedResource(OperationHelper.java:600) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.OperationHelper.lambda$withSourceAndConnection$3(OperationHelper.java:574) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.OperationHelper.withSuppliedResource(OperationHelper.java:600) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.OperationHelper.withSourceAndConnection(OperationHelper.java:573) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.FindOperation.lambda$execute$2(FindOperation.java:690) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.async.function.RetryingSyncSupplier.get(RetryingSyncSupplier.java:65) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:722) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:86) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:191) ~[mongodb-driver-sync-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:135) ~[mongodb-driver-sync-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:92) ~[mongodb-driver-sync-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2968) ~[spring-data-mongodb-3.4.2.jar!/:3.4.2]
May 22 05:00:00 ip-172-31-4-152 web: ... 53 common frames omitted
May 22 05:00:00 ip-172-31-4-152 web: Caused by: java.net.SocketException: Connection reset
May 22 05:00:00 ip-172-31-4-152 web: at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:319) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1193) ~[na:1.8.0_352]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:99) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:552) ~[mongodb-driver-core-4.6.1.jar!/:na]
May 22 05:00:00 ip-172-31-4-152 web: ... 77 common frames omitted

TCP Connection is reset by the peer. it can happen for many reasons, to name a few, host crash, server process crash…

What you mean by Mongodb serverless? If the server hosts are only available for a short amount of time, then the connections will obviously be “reset” after the server hosts are gone.

@Kobe_W The ‘serverless’ means the Atlas Serverless plan. If you go to Pricing | MongoDB it is the first kinda free option. Actually, I have the same problem the connection is closed periodically. What is the correct config should be to avoid it? Thank you.