Issue python#121: Fix a reference leak, if Python runs Python code while clearing a thread state

If Python clears a thread state and a destructor or a weakref-callback runs Python code, Python used to create a new initial stub without clearing it later. This change reorders PyThreadState_Clear() to clear the PyStacklessState sub-structure last. A new test case for this problem uses thread local storage to execute a __del__-method.

https://bitbucket.org/stackless-dev/stackless/issues/121
(grafted from ede2ad2b32b917d364f523318a82c75d0df2a0e6)

Author: Anselm Kruis

Python/pystate.c

@@ -337,7 +337,7 @@ void
 PyThreadState_Clear(PyThreadState *tstate)
 {
 #ifdef STACKLESS
-    STACKLESS_PYSTATE_CLEAR;
+    slp_kill_tasks_with_stacks(tstate);
 #endif
     if (Py_VerboseFlag && tstate->frame != NULL)
         fprintf(stderr,
@@ -360,6 +360,9 @@ PyThreadState_Clear(PyThreadState *tstate)
     tstate->c_tracefunc = NULL;
     Py_CLEAR(tstate->c_profileobj);
     Py_CLEAR(tstate->c_traceobj);
+#ifdef STACKLESS
+    STACKLESS_PYSTATE_CLEAR;
+#endif
 }
 
 

Stackless/changelog.txt

@@ -9,6 +9,11 @@ What's New in Stackless 3.X.X?
 
 *Release date: 20XX-XX-XX*
 
+- https://bitbucket.org/stackless-dev/stackless/issues/121
+  Fix a reference leak. If Python clears a thread state and a destructor or a
+  weakref-callback runs Python code, Python used to leak a reference to a
+  C-stack object.
+
 - https://bitbucket.org/stackless-dev/stackless/issues/119
   Fix a rare bug in the stack unwinding mechanism, that caused a SystemError
   exception or an assertion violation, if a __del__()-method or a weakref

Stackless/core/stackless_tstate.h

@@ -114,7 +114,6 @@ struct _ts; /* Forward */
 void slp_kill_tasks_with_stacks(struct _ts *tstate);
 
 #define __STACKLESS_PYSTATE_CLEAR \
-    slp_kill_tasks_with_stacks(tstate); \
     Py_CLEAR(tstate->st.initial_stub); \
     Py_CLEAR(tstate->st.del_post_switch); \
     Py_CLEAR(tstate->st.interrupted); \

Stackless/unittests/test_thread.py

@@ -616,6 +616,33 @@ def test_setup_from_other_thread(self):
         theThread.join()
 
 
+@unittest.skipUnless(withThreads, "requires thread support")
+class TestThreadLocalStorage(StacklessTestCase):
+    class ObjectWithDestructor(object):
+        def __init__(self, event):
+            self.event = event
+
+        def __del__(self):
+            self.event.set()
+
+    def test_destructor_at_end_of_thread(self):
+        # Test case for issue #121 https://bitbucket.org/stackless-dev/stackless/issue/121
+        # Run a destructor during clean up of thread local storage
+        # Until issue #121 got fixed, this caused a reference leak
+        tls = threading.local()
+        deleted = threading.Event()
+
+        def other_thread():
+            tls.owd = self.ObjectWithDestructor(deleted)
+
+        self.assertFalse(deleted.is_set())
+        t = threading.Thread(target=other_thread, name="other thread")
+        t.start()
+        t.join()
+        time.sleep(0.1)  # give the thread time to clean up
+        self.assertTrue(deleted.is_set())
+
+
 if __name__ == '__main__':
     if not sys.argv[1:]:
         sys.argv.append('-v')