This resolves #2189.

> build isolation would be a bit difficult to use in our case, as for instance when building scipy we need the patched numpy on the host and not the numpy version specified in pyproject.toml (which would be unpatched)

This is indeed the case, certain packages cannot be isolated. My strategy is to make a list of packages that shouldn't be isolated and add symlinks from the isolated build environment into the `.artifacts` directory to "unisolate" them. Then we remove the unisolated package requirements from the list of packages to install, in case pesky constraints aren't satisfied. In particular, packages that expect to be used with `pypa/build` often feel free to put very specific constraints on their build dependencies (often asking them to be == to a particular version). Specific version constraints is good for build reproducibility and with build isolation doesn't cost anything. So we just ignore the constraints. Hopefully nothing goes wrong.

In particular, any package that does stuff both at build time and at runtime and requires synchronization between the build time and run time environments needs the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy and scipy. pycparser needs to be unisolated because it is a dependency of cffi.

Currently I have also unisolated pythran and cython, though these are build time only tools and do not really need to be unisolated. Cython I unisolated specifically because numcodecs needs it but it isn't in the numcodecs build dependencies. Pythran I unisolated because of a problem with the scipy build which I don't fully understand (some problem with long double feature detection). I believe both of these are removable, particularly once #2132 is finished.
Use pypa/build. This resolves #2189.

> build isolation would be a bit difficult to use in our case, as for instance when building scipy we need the patched numpy on the host and not the numpy version specified in pyproject.toml (which would be unpatched)

This is indeed the case, certain packages cannot be isolated. My strategy is to make a list of packages that shouldn't be isolated and add symlinks from the isolated build environment into the `.artifacts` directory to "unisolate" them. Then we remove the unisolated package requirements from the list of packages to install, in case pesky constraints aren't satisfied. In particular, packages that expect to be used with `pypa/build` often feel free to put very specific constraints on their build dependencies (often asking them to be == to a particular version). Specific version constraints is good for build reproducibility and with build isolation doesn't cost anything. So we just ignore the constraints. Hopefully nothing goes wrong.

In particular, any package that does stuff both at build time and at runtime and requires synchronization between the build time and run time environments needs the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy and scipy. pycparser needs to be unisolated because it is a dependency of cffi.

Currently I have also unisolated pythran and cython, though these are build time only tools and do not really need to be unisolated. Cython I unisolated specifically because numcodecs needs it but it isn't in the numcodecs build dependencies. Pythran I unisolated because of a problem with the scipy build which I don't fully understand (some problem with long double feature detection). I believe both of these are removable, particularly once #2132 is finished.
(debugging)=

# Interactive Debugging

See [Emscripten's page about
debugging](https://emscripten.org/docs/porting/Debugging.html) which has
extensive info about the various debugging options available.

[Chromium has support for DWARF
info](https://developer.chrome.com/blog/wasm-debugging-2020/) which can be very helpful
for debugging in certain circumstances. To build Pyodide with DWARF, you should
set `EXTRA_CFLAGS="-g4"` and `EXTRA_LD_FLAGS="-g4 --source-map-base "http://localhost:<port>/"` (substitute in your favorite port). Make sure to
rebuild CPython with these flags set (it isn't necessary to rebuild emsdk).

Once you have done this, when you load Pyodide, you will see 404 errors for
requests for the various source maps. In order to load the source maps
correctly, you will need to run a custom server that "fixes" the source map
urls. The following debugging server seems to work for both CPython and numpy.
Run it in the Pyodide root directory. If you need to debug other C extensions,
you will need to update the server. It should be clear what to do based by
looking at the 404s generated by the server and locating those files in the file
tree, perhaps by using `find`.

```py
import socket
import socketserver
from http.server import SimpleHTTPRequestHandler
import pathlib

alternative_bases=["cpython/build/Python-3.10.2/","src/", "build/"]
def fixup_url(path):
    if pathlib.Path("." + path).exists():
        return path
    for base in alternative_bases:
        q = pathlib.Path(base + path)
        if q.exists():
            return str(q)
    # Numpy source maps can be in a bunch of different places inside of the
    # directory tree, so we need to glob for them.
    dir = list(
        pathlib.Path("packages/numpy/build/numpy-1.17.5/").glob("**" + path)
    )
    if dir:
        return str(dir[0])
    return path


class MyTCPServer(socketserver.TCPServer):
    def server_bind(self):
        """Use socket.SO_REUSEADDR to allow faster restart"""
        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socket.bind(self.server_address)

class Handler(SimpleHTTPRequestHandler):
    def end_headers(self):
        # Enable Cross-Origin Resource Sharing (CORS)
        self.send_header('Access-Control-Allow-Origin', '*')
        super().end_headers()

    def do_GET(self):
        self.path = fixup_url(self.path)
        super().do_GET()


if __name__ == '__main__':
    port = 8000
    with MyTCPServer(("", port), Handler) as httpd:
        print("Serving at: http://127.0.0.1:{}".format(port))
        httpd.serve_forever()
```

(debugging)=

# Interactive Debugging

See [Emscripten's page about
debugging](https://emscripten.org/docs/porting/Debugging.html) which has
extensive info about the various debugging options available.

[Chromium has support for DWARF
info](https://developer.chrome.com/blog/wasm-debugging-2020/) which can be very helpful
for debugging in certain circumstances. To build Pyodide with DWARF, you should
set `EXTRA_CFLAGS="-g4"` and `EXTRA_LD_FLAGS="-g4 --source-map-base "http://localhost:<port>/"` (substitute in your favorite port). Make sure to
rebuild CPython with these flags set (it isn't necessary to rebuild emsdk).

Once you have done this, when you load Pyodide, you will see 404 errors for
requests for the various source maps. In order to load the source maps
correctly, you will need to run a custom server that "fixes" the source map
urls. The following debugging server seems to work for both CPython and numpy.
Run it in the Pyodide root directory. If you need to debug other C extensions,
you will need to update the server. It should be clear what to do based by
looking at the 404s generated by the server and locating those files in the file
tree, perhaps by using `find`.

```py
import socket
import socketserver
from http.server import SimpleHTTPRequestHandler
import pathlib

alternative_bases=["cpython/build/Python-3.10.2/","src/", "build/"]
def fixup_url(path):
    if pathlib.Path("." + path).exists():
        return path
    for base in alternative_bases:
        q = pathlib.Path(base + path)
        if q.exists():
            return str(q)
    # Numpy source maps can be in a bunch of different places inside of the
    # directory tree, so we need to glob for them.
    dir = list(
        pathlib.Path("packages/numpy/build/numpy-1.17.5/").glob("**" + path)
    )
    if dir:
        return str(dir[0])
    return path


class MyTCPServer(socketserver.TCPServer):
    def server_bind(self):
        """Use socket.SO_REUSEADDR to allow faster restart"""
        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socket.bind(self.server_address)

class Handler(SimpleHTTPRequestHandler):
    def end_headers(self):
        # Enable Cross-Origin Resource Sharing (CORS)
        self.send_header('Access-Control-Allow-Origin', '*')
        super().end_headers()

    def do_GET(self):
        self.path = fixup_url(self.path)
        super().do_GET()


if __name__ == '__main__':
    port = 8000
    with MyTCPServer(("", port), Handler) as httpd:
        print("Serving at: http://127.0.0.1:{}".format(port))
        httpd.serve_forever()
```
