RuntimeError: can't create new thread at interpreter shutdown in Pymongo

MongoDB Atlas
1

I’m using pymongo for building a RAG application. Until now I have only defined a couple of functions in my code and on running my .py file from the VS Code terminal, I’m getting a ‘Exception in thread pymongo_server_monitor_thread:’. The full traceback is given below:

python helper.py 
Exception in thread pymongo_server_monitor_thread:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1052, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 989, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/periodic_executor.py", line 133, in _run
    if not self._target():
           ^^^^^^^^^^^^^^
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 61, in target
    monitor._run()  # type:ignore[attr-defined]
    ^^^^^^^^^^^^^^
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 213, in _run
    self._start_rtt_monitor()
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 167, in _start_rtt_monitor
    self._rtt_monitor.open()
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 87, in open
    self._executor.open()
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/periodic_executor.py", line 95, in open
    thread.start()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 971, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown
Exception in thread pymongo_server_monitor_thread:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1052, in _bootstrap_inner
Exception in thread pymongo_server_monitor_thread:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1052, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 989, in run
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 989, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/periodic_executor.py", line 133, in _run
    if not self._target():
    self._target(*self._args, **self._kwargs)
           ^^^^^^^^^^^^^^
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/periodic_executor.py", line 133, in _run
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 61, in target
    if not self._target():
           ^^^^^^^^^^^^^^
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 61, in target
    monitor._run()  # type:ignore[attr-defined]
    ^^^^^^^^^^^^^^
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 213, in _run
    monitor._run()  # type:ignore[attr-defined]
    ^^^^^^^^^^^^^^
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 213, in _run
    self._start_rtt_monitor()
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 167, in _start_rtt_monitor
    self._rtt_monitor.open()
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 87, in open
    self._start_rtt_monitor()
    self._executor.open()
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 167, in _start_rtt_monitor
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/periodic_executor.py", line 95, in open
    self._rtt_monitor.open()
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/monitor.py", line 87, in open
    thread.start()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 971, in start
    self._executor.open()
  File "/Users/zuhaib/Documents/ast-bot-mongoDB/venv/lib/python3.12/site-packages/pymongo/periodic_executor.py", line 95, in open
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown
    thread.start()
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 971, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't create new thread at interpreter shutdown
2

Hi, thanks for raising this as it’s been a new ticket reported to us.

As our engineer provided on the ticket,

…the cause of this issue is that MongoClient starts a new thread for each monitored server in the MongoDB cluster. If the client is created directly before the application exits, then threads will attempt to start up during the Python interpreter shutdown. This did not used to be an issue in Python but it was changed in Python 3.12 here: gh-104690 Disallow thread creation and fork at interpreter finalizati… · python/cpython@ce558e6 · GitHub

As an additional note, this should not affect your overall application behavior, but please let us know if your application is failing to execute overall as a result of this issue.

Please see PYTHON-4147 for more information.

Thanks again!

3

Hi Adegunloye. I am facing a similar issue and in my case my application doesn’t start up at all. This is the full traceback below:

`````` `` Exception in thread pymongo_server_monitor_thread: Traceback (most recent call last): File "C:\Users\Afua Ayiku\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1073, in _bootstrap_inner self.run() File "C:\Users\Afua Ayiku\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1010, in run self._target(*self._args, **self._kwargs) File "C:\Users\Afua Ayiku\AppData\Local\Programs\Python\Python312\Lib\site-packages\pymongo\periodic_executor.py", line 133, in _run if not self._target(): ^^^^^^^^^^^^^^ File "C:\Users\Afua Ayiku\AppData\Local\Programs\Python\Python312\Lib\site-packages\pymongo\monitor.py", line 61, in target monitor._run() # type:ignore[attr-defined] \Local\Programs\Python\Python312\Lib\site-packages\pymongo\periodic_executor.py", line 95, in open thread.start() File "C:\Users\Afua Ayiku\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 992, in start _start_new_thread(self._bootstrap, ()) RuntimeError: can't create new thread at interpreter shutdown

4

Hi @Afua_Ayiku,

This error only appears as the python interpreter is shutting down, so it should not stop the application from starting up; I believe the error you’re receiving is most likely unrelated.

Could you provide a little more context on what your code is doing?

Specifically,

  • What does your application do?
  • When do you make a call to the MongoClient()
  • Could you check for any other area that could be leading to an early termination?

What happens when you run this code against Python 3.11? (If you are able to at all)

5

I found the same error. For my case using Python 3.12

The error occur during unit test (I don’t know about production since it don’t have any effect)
I use pytest to run test and the error appear after the test report

MongoClient has been call at the beginning of the test via fixture, the fixture function is here

@pytest.fixture(scope="session", autouse=True)
def mongo_connect(settings: Settings) :
    client = settings.get_mongo_client()         # MongoConnect here
    set_mongo_database(settings.get_mongo_db())  # set db to environment
    yield 
    client.close()
6

Hey @Chet_Chetchaiyan,

Thanks for providing your context. As an update, we have provided a solution and the fix will be a part of our next patch release.

1 Like
7

We have released 4.6.2 with a fix.

1 Like