<!-- Thank you for contributing to Pyodide! All improvements are welcome,
     so don't be afraid to make a PR. -->

<!-- [IMPORTANT] Note on CI failures:
     Currently, we are having issues with selenium-based tests.
     Don't panic if the CI fails on your PR because of timeouts.
     It's probably not your fault. We will investigate :) -->

### Description

<!-- Please explain what your PR is about:
     - reasoning for the change
     - some details of updated code
     - any noteworthy choices to be aware of
	Please refer to any related issues by #<issue_id> -->

Bumping MPL version - it turns out that Hist (which uses boost-histogram) depends on mplhep, which in turn requires matplotlib 3.4+. So I migrated the patches and bumped MPL.

I noticed micropip doesn't seem to respect extras, that is, `await micropip.install("hist[plot]")` doesn't seem to work. (will investigate later)

* Updated matplotlib to 3.5
    * Added fonttools
* Updated yt to 4
    * Added `cmyt` and `unyt`, which I am pretty sure are yt specific. At least by looking at the name. 😆 
    * Also `colorspacious`
    * Also `tdqm`
    * Also `tomli` and `tomli-w` (backported from the next yt release to avoid adding a dead dependency `toml`)

Closes #2088 too. Also 3.5 was the first version to officially support 3.10, I think.
chore: matplotlib 3.5.1, yt 4. <!-- Thank you for contributing to Pyodide! All improvements are welcome,
     so don't be afraid to make a PR. -->

<!-- [IMPORTANT] Note on CI failures:
     Currently, we are having issues with selenium-based tests.
     Don't panic if the CI fails on your PR because of timeouts.
     It's probably not your fault. We will investigate :) -->

### Description

<!-- Please explain what your PR is about:
     - reasoning for the change
     - some details of updated code
     - any noteworthy choices to be aware of
	Please refer to any related issues by #<issue_id> -->

Bumping MPL version - it turns out that Hist (which uses boost-histogram) depends on mplhep, which in turn requires matplotlib 3.4+. So I migrated the patches and bumped MPL.

I noticed micropip doesn't seem to respect extras, that is, `await micropip.install("hist[plot]")` doesn't seem to work. (will investigate later)

* Updated matplotlib to 3.5
    * Added fonttools
* Updated yt to 4
    * Added `cmyt` and `unyt`, which I am pretty sure are yt specific. At least by looking at the name. 😆 
    * Also `colorspacious`
    * Also `tdqm`
    * Also `tomli` and `tomli-w` (backported from the next yt release to avoid adding a dead dependency `toml`)

Closes #2088 too. Also 3.5 was the first version to officially support 3.10, I think.
From 320cc07c0a30436759cde8f4c926e76ed8dc918a Mon Sep 17 00:00:00 2001
From: Hood <hood@mit.edu>
Date: Fri, 3 Sep 2021 18:08:26 -0700
Subject: [PATCH] Patch in keyboard interrupt handling

---
 Include/cpython/ceval.h   |  1 +
 Include/cpython/pystate.h |  3 +++
 Modules/signalmodule.c    |  3 +++
 Python/ceval.c            | 13 ++++++++++++-
 Python/pystate.c          |  2 ++
 5 files changed, 21 insertions(+), 1 deletion(-)

This patch adds a callback called pyodide_callback with signature 
`int callback(void)` to the main loop in `ceval.c`. This function gets called once 
per opcode except after opcodes `SETUP_FINALLY`, `SETUP_WITH`, `BEFORE_ASYNC_WITH`,
and `YIELD_FROM`. The main loop normally prevents signal handling, etc from happening
after these instructions, so I figured this should apply to my callback too.
(There is an extra layer of protection here though because in the callback we use 
`PyErr_SetInterrupt` which simulates a SIGINT signal and triggers the KeyboardInterrupt
via the standard mechanism.)

Note that we call the callback outside of the normal 
`if (_Py_atomic_load_relaxed(eval_breaker))` 
block where most of the "periodic things" happen. This is because normally when a 
"periodic thing" is queued, the `eval_breaker` flag is set signalling the need for
handling. The whole point of this patch though is that we need to be able to set an
interrupt from a remote thread and the `eval_breaker` flag lives on the WASM heap.
Unless the whole WASM heap is made into a `SharedArrayBuffer` and shared between 
webworkers, we have no way to set the `eval_breaker` flag. We still want to skip
the callback after the sensitive opcodes `SETUP_FINALLY`, `SETUP_WITH`, 
`BEFORE_ASYNC_WITH`, and `YIELD_FROM`, so we hoisted the check for that condition
out of the `eval_breaker` conditional.

We also patched the `threadstate` struct to include a `pyodide_callback` field, patch
`new_threadstate` to initialize the `pyodide_callback` field to `NULL`, and add a new
API `PyPyodide_SetPyodideCallback` to set `pyodide_callback`.

Lastly, we patched PyErr_CheckSignals to check our custom source of keyboard interrupts
so that C extensions that use PyErr_CheckSignals to allow keyboard interrupts will be 
interruptable in pyodide too.

diff --git a/Include/cpython/ceval.h b/Include/cpython/ceval.h
index e1922a6..a4acd9b 100644
--- a/Include/cpython/ceval.h
+++ b/Include/cpython/ceval.h
@@ -10,6 +10,7 @@ PyAPI_FUNC(void) PyEval_SetProfile(Py_tracefunc, PyObject *);
 PyAPI_DATA(int) _PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg);
 PyAPI_FUNC(void) PyEval_SetTrace(Py_tracefunc, PyObject *);
 PyAPI_FUNC(int) _PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg);
+PyAPI_FUNC(void) PyPyodide_SetPyodideCallback(PyPyodide_callback);
 PyAPI_FUNC(int) _PyEval_GetCoroutineOriginTrackingDepth(void);
 PyAPI_FUNC(int) _PyEval_SetAsyncGenFirstiter(PyObject *);
 PyAPI_FUNC(PyObject *) _PyEval_GetAsyncGenFirstiter(void);
diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h
index f292da1..3577ac8 100644
--- a/Include/cpython/pystate.h
+++ b/Include/cpython/pystate.h
@@ -17,6 +17,7 @@ PyAPI_FUNC(PyObject *) _PyInterpreterState_GetMainModule(PyInterpreterState *);
 
 /* Py_tracefunc return -1 when raising an exception, or 0 for success. */
 typedef int (*Py_tracefunc)(PyObject *, PyFrameObject *, int, PyObject *);
+typedef int (*PyPyodide_callback)(void);
 
 /* The following values are used for 'what' for tracefunc functions
  *
@@ -75,6 +76,8 @@ struct _ts {
     PyObject *c_profileobj;
     PyObject *c_traceobj;
 
+    PyPyodide_callback pyodide_callback;
+
     /* The exception currently being raised */
     PyObject *curexc_type;
     PyObject *curexc_value;
diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
index de564c2..bfe2fd1 100644
--- a/Modules/signalmodule.c
+++ b/Modules/signalmodule.c
@@ -1683,6 +1683,9 @@ PyErr_CheckSignals(void)
 int
 _PyErr_CheckSignalsTstate(PyThreadState *tstate)
 {
+    if (tstate->pyodide_callback != NULL && tstate->pyodide_callback() == -1)
+        return -1;
+
     if (!_Py_atomic_load(&is_tripped)) {
         return 0;
     }
diff --git a/Python/ceval.c b/Python/ceval.c
index 91e879e..f8d424d 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1374,7 +1374,6 @@ main_loop:
            async I/O handler); see Py_AddPendingCall() and
            Py_MakePendingCalls() above. */
 
-        if (_Py_atomic_load_relaxed(eval_breaker)) {
             opcode = _Py_OPCODE(*next_instr);
             if (opcode == SETUP_FINALLY ||
                 opcode == SETUP_WITH ||
@@ -1399,11 +1398,18 @@ main_loop:
                 goto fast_next_opcode;
             }
 
+        if (_Py_atomic_load_relaxed(eval_breaker)) {
             if (eval_frame_handle_pending(tstate) != 0) {
                 goto error;
             }
         }
 
+        if(tstate->pyodide_callback != NULL){
+            if(tstate->pyodide_callback() != 0){
+                goto error;
+            }
+        }
+
     fast_next_opcode:
         f->f_lasti = INSTR_OFFSET();
 
@@ -4816,6 +4822,11 @@ PyEval_SetTrace(Py_tracefunc func, PyObject *arg)
     }
 }
 
+void
+PyPyodide_SetPyodideCallback(PyPyodide_callback pyodide_callback){
+    PyThreadState *tstate = _PyThreadState_GET();
+    tstate->pyodide_callback = pyodide_callback;
+}
 
 void
 _PyEval_SetCoroutineOriginTrackingDepth(PyThreadState *tstate, int new_depth)
diff --git a/Python/pystate.c b/Python/pystate.c
index 9beefa8..408b7ce 100644
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -602,6 +602,8 @@ new_threadstate(PyInterpreterState *interp, int init)
     tstate->c_profileobj = NULL;
     tstate->c_traceobj = NULL;
 
+    tstate->pyodide_callback = NULL;
+
     tstate->trash_delete_nesting = 0;
     tstate->trash_delete_later = NULL;
     tstate->on_delete = NULL;
-- 
2.17.1


From 320cc07c0a30436759cde8f4c926e76ed8dc918a Mon Sep 17 00:00:00 2001
From: Hood <hood@mit.edu>
Date: Fri, 3 Sep 2021 18:08:26 -0700
Subject: [PATCH] Patch in keyboard interrupt handling

---
 Include/cpython/ceval.h   |  1 +
 Include/cpython/pystate.h |  3 +++
 Modules/signalmodule.c    |  3 +++
 Python/ceval.c            | 13 ++++++++++++-
 Python/pystate.c          |  2 ++
 5 files changed, 21 insertions(+), 1 deletion(-)

This patch adds a callback called pyodide_callback with signature 
`int callback(void)` to the main loop in `ceval.c`. This function gets called once 
per opcode except after opcodes `SETUP_FINALLY`, `SETUP_WITH`, `BEFORE_ASYNC_WITH`,
and `YIELD_FROM`. The main loop normally prevents signal handling, etc from happening
after these instructions, so I figured this should apply to my callback too.
(There is an extra layer of protection here though because in the callback we use 
`PyErr_SetInterrupt` which simulates a SIGINT signal and triggers the KeyboardInterrupt
via the standard mechanism.)

Note that we call the callback outside of the normal 
`if (_Py_atomic_load_relaxed(eval_breaker))` 
block where most of the "periodic things" happen. This is because normally when a 
"periodic thing" is queued, the `eval_breaker` flag is set signalling the need for
handling. The whole point of this patch though is that we need to be able to set an
interrupt from a remote thread and the `eval_breaker` flag lives on the WASM heap.
Unless the whole WASM heap is made into a `SharedArrayBuffer` and shared between 
webworkers, we have no way to set the `eval_breaker` flag. We still want to skip
the callback after the sensitive opcodes `SETUP_FINALLY`, `SETUP_WITH`, 
`BEFORE_ASYNC_WITH`, and `YIELD_FROM`, so we hoisted the check for that condition
out of the `eval_breaker` conditional.

We also patched the `threadstate` struct to include a `pyodide_callback` field, patch
`new_threadstate` to initialize the `pyodide_callback` field to `NULL`, and add a new
API `PyPyodide_SetPyodideCallback` to set `pyodide_callback`.

Lastly, we patched PyErr_CheckSignals to check our custom source of keyboard interrupts
so that C extensions that use PyErr_CheckSignals to allow keyboard interrupts will be 
interruptable in pyodide too.

diff --git a/Include/cpython/ceval.h b/Include/cpython/ceval.h
index e1922a6..a4acd9b 100644
--- a/Include/cpython/ceval.h
+++ b/Include/cpython/ceval.h
@@ -10,6 +10,7 @@ PyAPI_FUNC(void) PyEval_SetProfile(Py_tracefunc, PyObject *);
 PyAPI_DATA(int) _PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg);
 PyAPI_FUNC(void) PyEval_SetTrace(Py_tracefunc, PyObject *);
 PyAPI_FUNC(int) _PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg);
+PyAPI_FUNC(void) PyPyodide_SetPyodideCallback(PyPyodide_callback);
 PyAPI_FUNC(int) _PyEval_GetCoroutineOriginTrackingDepth(void);
 PyAPI_FUNC(int) _PyEval_SetAsyncGenFirstiter(PyObject *);
 PyAPI_FUNC(PyObject *) _PyEval_GetAsyncGenFirstiter(void);
diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h
index f292da1..3577ac8 100644
--- a/Include/cpython/pystate.h
+++ b/Include/cpython/pystate.h
@@ -17,6 +17,7 @@ PyAPI_FUNC(PyObject *) _PyInterpreterState_GetMainModule(PyInterpreterState *);
 
 /* Py_tracefunc return -1 when raising an exception, or 0 for success. */
 typedef int (*Py_tracefunc)(PyObject *, PyFrameObject *, int, PyObject *);
+typedef int (*PyPyodide_callback)(void);
 
 /* The following values are used for 'what' for tracefunc functions
  *
@@ -75,6 +76,8 @@ struct _ts {
     PyObject *c_profileobj;
     PyObject *c_traceobj;
 
+    PyPyodide_callback pyodide_callback;
+
     /* The exception currently being raised */
     PyObject *curexc_type;
     PyObject *curexc_value;
diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
index de564c2..bfe2fd1 100644
--- a/Modules/signalmodule.c
+++ b/Modules/signalmodule.c
@@ -1683,6 +1683,9 @@ PyErr_CheckSignals(void)
 int
 _PyErr_CheckSignalsTstate(PyThreadState *tstate)
 {
+    if (tstate->pyodide_callback != NULL && tstate->pyodide_callback() == -1)
+        return -1;
+
     if (!_Py_atomic_load(&is_tripped)) {
         return 0;
     }
diff --git a/Python/ceval.c b/Python/ceval.c
index 91e879e..f8d424d 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1374,7 +1374,6 @@ main_loop:
            async I/O handler); see Py_AddPendingCall() and
            Py_MakePendingCalls() above. */
 
-        if (_Py_atomic_load_relaxed(eval_breaker)) {
             opcode = _Py_OPCODE(*next_instr);
             if (opcode == SETUP_FINALLY ||
                 opcode == SETUP_WITH ||
@@ -1399,11 +1398,18 @@ main_loop:
                 goto fast_next_opcode;
             }
 
+        if (_Py_atomic_load_relaxed(eval_breaker)) {
             if (eval_frame_handle_pending(tstate) != 0) {
                 goto error;
             }
         }
 
+        if(tstate->pyodide_callback != NULL){
+            if(tstate->pyodide_callback() != 0){
+                goto error;
+            }
+        }
+
     fast_next_opcode:
         f->f_lasti = INSTR_OFFSET();
 
@@ -4816,6 +4822,11 @@ PyEval_SetTrace(Py_tracefunc func, PyObject *arg)
     }
 }
 
+void
+PyPyodide_SetPyodideCallback(PyPyodide_callback pyodide_callback){
+    PyThreadState *tstate = _PyThreadState_GET();
+    tstate->pyodide_callback = pyodide_callback;
+}
 
 void
 _PyEval_SetCoroutineOriginTrackingDepth(PyThreadState *tstate, int new_depth)
diff --git a/Python/pystate.c b/Python/pystate.c
index 9beefa8..408b7ce 100644
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -602,6 +602,8 @@ new_threadstate(PyInterpreterState *interp, int init)
     tstate->c_profileobj = NULL;
     tstate->c_traceobj = NULL;
 
+    tstate->pyodide_callback = NULL;
+
     tstate->trash_delete_nesting = 0;
     tstate->trash_delete_later = NULL;
     tstate->on_delete = NULL;
-- 
2.17.1

