This is on top of #2225. The goal is to be able to skip specific tests. Many core test sets have just one test that uses subprocess or threads. Others have a few tests that are failing due to upstream problems in emscripten. I went through and switched to skipping specific tests in as many cases as I could. In modules where most or all tests use threading or subprocess, I gave up and put a comment saying something like 35/50 tests fork and I didn't want to bother to salvage the remaining ones.

In order to accommodate using patterns to skip specific tests (and also extra comments) I switched the format of `python_tests` to a yaml file. This gives us significantly more flexibility in the structure of the file and allows us to use `ruamel.yaml` to update it. We gain a lot flexibility while reducing the length of `make_test_lists.py` modestly from 108 to 78 loc.

In the future, hopefully we should be able to make fixes by deleting one skip pattern and looking into why that particular test fails.

Rework core tests. This is on top of #2225. The goal is to be able to skip specific tests. Many core test sets have just one test that uses subprocess or threads. Others have a few tests that are failing due to upstream problems in emscripten. I went through and switched to skipping specific tests in as many cases as I could. In modules where most or all tests use threading or subprocess, I gave up and put a comment saying something like 35/50 tests fork and I didn't want to bother to salvage the remaining ones.

In order to accommodate using patterns to skip specific tests (and also extra comments) I switched the format of `python_tests` to a yaml file. This gives us significantly more flexibility in the structure of the file and allows us to use `ruamel.yaml` to update it. We gain a lot flexibility while reducing the length of `make_test_lists.py` modestly from 108 to 78 loc.

In the future, hopefully we should be able to make fixes by deleting one skip pattern and looking into why that particular test fails.

From 16abd462459b36c73db23b776f1eb5ab2c0c7813 Mon Sep 17 00:00:00 2001
From: Hood <hood@mit.edu>
Date: Thu, 24 Jun 2021 14:55:10 -0700
Subject: [PATCH 09/10] xfail core ctypes tests that don't work

PyCode_SetExtra doesn't work and ctypes doesn't seem to work with variadic functions including PyUnicode_FromFormat/
---
 Lib/test/test_code.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/Lib/test/test_code.py b/Lib/test/test_code.py
index b9c4f8bdcb..31cb8876c8 100644
--- a/Lib/test/test_code.py
+++ b/Lib/test/test_code.py
@@ -396,6 +396,7 @@ def test_bad_index(self):
                               ctypes.c_voidp(100)), 0)
 
         def test_free_called(self):
+            raise unittest.SkipTest("PyCode_SetExtra is broken")
             # Verify that the provided free function gets invoked
             # when the code object is cleaned up.
             f = self.get_func()
@@ -405,6 +406,7 @@ def test_free_called(self):
             self.assertEqual(LAST_FREED, 100)
 
         def test_get_set(self):
+            raise unittest.SkipTest("PyCode_SetExtra is broken")
             # Test basic get/set round tripping.
             f = self.get_func()
 
@@ -421,6 +423,7 @@ def test_get_set(self):
             del f
 
         def test_free_different_thread(self):
+            raise unittest.SkipTest("Requires threading")
             # Freeing a code object on a different thread then
             # where the co_extra was set should be safe.
             f = self.get_func()
@@ -445,8 +448,8 @@ def test_main(verbose=None):
     from test import test_code
     run_doctest(test_code, verbose)
     tests = [CodeTest, CodeConstsTest, CodeWeakRefTest]
-    if check_impl_detail(cpython=True) and ctypes is not None:
-        tests.append(CoExtra)
+    # if check_impl_detail(cpython=True) and ctypes is not None:
+    #     tests.append(CoExtra)
     run_unittest(*tests)
 
 if __name__ == "__main__":
-- 
2.25.1


From 16abd462459b36c73db23b776f1eb5ab2c0c7813 Mon Sep 17 00:00:00 2001
From: Hood <hood@mit.edu>
Date: Thu, 24 Jun 2021 14:55:10 -0700
Subject: [PATCH 09/10] xfail core ctypes tests that don't work

PyCode_SetExtra doesn't work and ctypes doesn't seem to work with variadic functions including PyUnicode_FromFormat/
---
 Lib/test/test_code.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/Lib/test/test_code.py b/Lib/test/test_code.py
index b9c4f8bdcb..31cb8876c8 100644
--- a/Lib/test/test_code.py
+++ b/Lib/test/test_code.py
@@ -396,6 +396,7 @@ def test_bad_index(self):
                               ctypes.c_voidp(100)), 0)
 
         def test_free_called(self):
+            raise unittest.SkipTest("PyCode_SetExtra is broken")
             # Verify that the provided free function gets invoked
             # when the code object is cleaned up.
             f = self.get_func()
@@ -405,6 +406,7 @@ def test_free_called(self):
             self.assertEqual(LAST_FREED, 100)
 
         def test_get_set(self):
+            raise unittest.SkipTest("PyCode_SetExtra is broken")
             # Test basic get/set round tripping.
             f = self.get_func()
 
@@ -421,6 +423,7 @@ def test_get_set(self):
             del f
 
         def test_free_different_thread(self):
+            raise unittest.SkipTest("Requires threading")
             # Freeing a code object on a different thread then
             # where the co_extra was set should be safe.
             f = self.get_func()
@@ -445,8 +448,8 @@ def test_main(verbose=None):
     from test import test_code
     run_doctest(test_code, verbose)
     tests = [CodeTest, CodeConstsTest, CodeWeakRefTest]
-    if check_impl_detail(cpython=True) and ctypes is not None:
-        tests.append(CoExtra)
+    # if check_impl_detail(cpython=True) and ctypes is not None:
+    #     tests.append(CoExtra)
     run_unittest(*tests)
 
 if __name__ == "__main__":
-- 
2.25.1

