ENH Use dist-info + `importlib.metadata` to store and retrieve package metadata. This switches to using the file system to store the information about packages
and using `importlib.metadata` to retrieve it.
This has two related benefits:

1. We don't have to separately maintain our own separate state about what
we've installed.

2. We are guaranteed to agree with the Python stdlib about whether or not a
package is installed, and if so which version is installed. Since the state is 
maintained in only one place, there is no chance of it diverging.

According to the packaging specs, if the package is from a url we should put 
that into a file here:
https://packaging.python.org/en/latest/specifications/direct-url/#direct-url
Other than that, we should set the `INSTALLER` to `pyodide.loadPackage` 
if the package was loaded with `pyodide.loadPackage` either directly or 
indirectly via `micropip`. Otherwise set `INSTALLER` to `micropip`. That
way we can maintain the current behavior of `micropip.list`:
1. if `direct_url.json` is present, then `source` is the url in `direct_url.json`
2. otherwise, set `source` to `pyodide` if `INSTALLER` is `pyodide.loadPackage`
and set it to `micropip` if `INSTALLER` is `micropip`.

Oddly enough, the packaging specs suggest no place to put the source
index, so it seems that according to the specs if a package was installed
by name from some custom index, this info should not be recorded.

### Checklists

- [x] Add a [CHANGELOG](https://github.com/pyodide/pyodide/blob/main/docs/project/changelog.md) entry
- [x] Add / update tests

(interrupting_execution)=

# Interrupting execution

The native Python interrupt system is based on preemptive multitasking but Web
Assembly has no support for preemptive multitasking. Because of this,
interrupting execution in Pyodide must be achieved via a different mechanism
which takes some effort to set up.

## Setting up interrupts

In order to use interrupts you must be using Pyodide in a webworker.
You also will need to use a `SharedArrayBuffer`, which means that your server
must set appropriate security headers. See [the MDN
docs](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer#security_requirements)
for more information.

To use the interrupt system, you should create a `SharedArrayBuffer` on either
the main thread or the worker thread and share it with the other thread. You
should use {any}`pyodide.setInterruptBuffer` to set the interrupt buffer on the
Pyodide thread. When you want to indicate an interrupt, write a `2` into the
interrupt buffer. When the interrupt signal is processed, Pyodide will set the
value of the interrupt buffer back to `0`.

By default, when the interrupt fires, a `KeyboardInterrupt` is raised. [Using
the `signal`
module](https://docs.python.org/3/library/signal.html#signal.signal), it is
possible to register a custom Python function to handle `SIGINT`. If you
register a custom handler function it will be called instead.

Here is a very basic example. Main thread code:

```js
let pyodideWorker = new Worker("pyodideWorker.js");
let interruptBuffer = new Uint8Array(new SharedArrayBuffer(1));
pyodideWorker.postMessage({ cmd: "setInterruptBuffer", interruptBuffer });
function interruptExecution() {
  // 2 stands for SIGINT.
  interruptBuffer[0] = 2;
}
// imagine that interruptButton is a button we want to trigger an interrupt.
interruptButton.addEventListener("click", interruptExecution);
async function runCode(code) {
  // Clear interruptBuffer in case it was accidentally left set after previous code completed.
  interruptBuffer[0] = 0;
  pyodideWorker.postMessage({ cmd: "runCode", code });
}
```

Worker code:

```js
self.addEventListener("message", (msg) => {
  if (msg.data.cmd === "setInterruptBuffer") {
    pyodide.setInterruptBuffer(msg.data.interruptBuffer);
    return;
  }
  if (msg.data.cmd === "runCode") {
    pyodide.runPython(msg.data.code);
    return;
  }
});
```

## Allowing JavaScript code to be interrupted

The interrupt system above allows interruption of Python code and also of C code
that opts to allow itself to be interrupted by periodically calling
`PyErr_CheckSignals`. There is also a function {any}`pyodide.checkInterrupt` that
allows JavasSript functions called from Python to check for an interrupt. As a
simple example, we can implement an interruptible sleep function using
`Atomics.wait`:

```js
let blockingSleepBuffer = new Int32Array(new SharedArrayBuffer(4));
function blockingSleep(t) {
  for (let i = 0; i < t * 20; i++) {
    // This Atomics.wait call blocks the thread until the buffer changes or a 50ms timeout ellapses.
    // Since we won't change the value in the buffer, this blocks for 50ms.
    Atomics.wait(blockingSleepBuffer, 0, 0, 50);
    // Periodically check for an interrupt to allow a KeyboardInterrupt.
    pyodide.checkInterrupt();
  }
}
```

(interrupting_execution)=

# Interrupting execution

The native Python interrupt system is based on preemptive multitasking but Web
Assembly has no support for preemptive multitasking. Because of this,
interrupting execution in Pyodide must be achieved via a different mechanism
which takes some effort to set up.

## Setting up interrupts

In order to use interrupts you must be using Pyodide in a webworker.
You also will need to use a `SharedArrayBuffer`, which means that your server
must set appropriate security headers. See [the MDN
docs](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer#security_requirements)
for more information.

To use the interrupt system, you should create a `SharedArrayBuffer` on either
the main thread or the worker thread and share it with the other thread. You
should use {any}`pyodide.setInterruptBuffer` to set the interrupt buffer on the
Pyodide thread. When you want to indicate an interrupt, write a `2` into the
interrupt buffer. When the interrupt signal is processed, Pyodide will set the
value of the interrupt buffer back to `0`.

By default, when the interrupt fires, a `KeyboardInterrupt` is raised. [Using
the `signal`
module](https://docs.python.org/3/library/signal.html#signal.signal), it is
possible to register a custom Python function to handle `SIGINT`. If you
register a custom handler function it will be called instead.

Here is a very basic example. Main thread code:

```js
let pyodideWorker = new Worker("pyodideWorker.js");
let interruptBuffer = new Uint8Array(new SharedArrayBuffer(1));
pyodideWorker.postMessage({ cmd: "setInterruptBuffer", interruptBuffer });
function interruptExecution() {
  // 2 stands for SIGINT.
  interruptBuffer[0] = 2;
}
// imagine that interruptButton is a button we want to trigger an interrupt.
interruptButton.addEventListener("click", interruptExecution);
async function runCode(code) {
  // Clear interruptBuffer in case it was accidentally left set after previous code completed.
  interruptBuffer[0] = 0;
  pyodideWorker.postMessage({ cmd: "runCode", code });
}
```

Worker code:

```js
self.addEventListener("message", (msg) => {
  if (msg.data.cmd === "setInterruptBuffer") {
    pyodide.setInterruptBuffer(msg.data.interruptBuffer);
    return;
  }
  if (msg.data.cmd === "runCode") {
    pyodide.runPython(msg.data.code);
    return;
  }
});
```

## Allowing JavaScript code to be interrupted

The interrupt system above allows interruption of Python code and also of C code
that opts to allow itself to be interrupted by periodically calling
`PyErr_CheckSignals`. There is also a function {any}`pyodide.checkInterrupt` that
allows JavasSript functions called from Python to check for an interrupt. As a
simple example, we can implement an interruptible sleep function using
`Atomics.wait`:

```js
let blockingSleepBuffer = new Int32Array(new SharedArrayBuffer(4));
function blockingSleep(t) {
  for (let i = 0; i < t * 20; i++) {
    // This Atomics.wait call blocks the thread until the buffer changes or a 50ms timeout ellapses.
    // Since we won't change the value in the buffer, this blocks for 50ms.
    Atomics.wait(blockingSleepBuffer, 0, 0, 50);
    // Periodically check for an interrupt to allow a KeyboardInterrupt.
    pyodide.checkInterrupt();
  }
}
```
