This resolves #2189.

> build isolation would be a bit difficult to use in our case, as for instance when building scipy we need the patched numpy on the host and not the numpy version specified in pyproject.toml (which would be unpatched)

This is indeed the case, certain packages cannot be isolated. My strategy is to make a list of packages that shouldn't be isolated and add symlinks from the isolated build environment into the `.artifacts` directory to "unisolate" them. Then we remove the unisolated package requirements from the list of packages to install, in case pesky constraints aren't satisfied. In particular, packages that expect to be used with `pypa/build` often feel free to put very specific constraints on their build dependencies (often asking them to be == to a particular version). Specific version constraints is good for build reproducibility and with build isolation doesn't cost anything. So we just ignore the constraints. Hopefully nothing goes wrong.

In particular, any package that does stuff both at build time and at runtime and requires synchronization between the build time and run time environments needs the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy and scipy. pycparser needs to be unisolated because it is a dependency of cffi.

Currently I have also unisolated pythran and cython, though these are build time only tools and do not really need to be unisolated. Cython I unisolated specifically because numcodecs needs it but it isn't in the numcodecs build dependencies. Pythran I unisolated because of a problem with the scipy build which I don't fully understand (some problem with long double feature detection). I believe both of these are removable, particularly once #2132 is finished.
Use pypa/build. This resolves #2189.

> build isolation would be a bit difficult to use in our case, as for instance when building scipy we need the patched numpy on the host and not the numpy version specified in pyproject.toml (which would be unpatched)

This is indeed the case, certain packages cannot be isolated. My strategy is to make a list of packages that shouldn't be isolated and add symlinks from the isolated build environment into the `.artifacts` directory to "unisolate" them. Then we remove the unisolated package requirements from the list of packages to install, in case pesky constraints aren't satisfied. In particular, packages that expect to be used with `pypa/build` often feel free to put very specific constraints on their build dependencies (often asking them to be == to a particular version). Specific version constraints is good for build reproducibility and with build isolation doesn't cost anything. So we just ignore the constraints. Hopefully nothing goes wrong.

In particular, any package that does stuff both at build time and at runtime and requires synchronization between the build time and run time environments needs the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy and scipy. pycparser needs to be unisolated because it is a dependency of cffi.

Currently I have also unisolated pythran and cython, though these are build time only tools and do not really need to be unisolated. Cython I unisolated specifically because numcodecs needs it but it isn't in the numcodecs build dependencies. Pythran I unisolated because of a problem with the scipy build which I don't fully understand (some problem with long double feature detection). I believe both of these are removable, particularly once #2132 is finished.
package:
  name: scipy
  version: 1.7.3 # when updating version make sure to update pip install scipy==1.7.3 below too.

# See extra explanation in info.md
#
# For future reference: if you see the following errors:
#   Declaration error: adjustable dimension on non-argument
# or:
#   nonconstant array size
# you are trying to compile code that isn't written to the fortran 77 standard.
# The line number in the error points to the last line of the problematic
# subroutine. Try deleting it.

source:
  url: https://files.pythonhosted.org/packages/61/67/1a654b96309c991762ee9bc39c363fc618076b155fe52d295211cf2536c7/scipy-1.7.3.tar.gz
  sha256: ab5875facfdef77e0a47d5fd39ea178b58e60e454a4c85aa1e52fcb80db7babf

  patches:
    - patches/add-lapack_extras-to-flapack.patch
    - patches/convert-xerrwv-message-arg-from-int-to-str.patch
    - patches/disable-blas-detection.patch
    - patches/fix-fortran-files-minpack.patch
    - patches/gemm_-no-const.patch
    - patches/make-int-return-values.patch
    - patches/remove-cuncsd-dorcsd-sorcsd-zuncsd.patch
    - patches/remove-mvnun-not-fortran-77-compliant.patch
    - patches/sasum-returns-double-not-float.patch
    - patches/skip-fortran-fails-to-link.patch
    - patches/rename-_page_trend_test.patch
    - patches/USE_CPP14-branch-doesn-t-work-for-wasm.patch

build:
  cflags: |
    -I$(PYODIDE_ROOT)/packages/numpy/config
    -I$(PYODIDE_ROOT)/packages/CLAPACK/build/CLAPACK-3.2.1/INCLUDE
    -I$(PYODIDE_ROOT)/packages/.artifacts/include/
    -I$(PYODIDE_ROOT)/packages/.artifacts/pythran/
    -Wno-return-type
    -DUNDERSCORE_G77
  ldflags: |
    -L$(NUMPY_LIB)
  # IMPORTANT: Other locations important in scipy build process:
  # There are two files built in the "capture" pass that need patching:
  #    _blas_subroutines.h, and _cython
  # Scipy has a bunch of custom logic implemented in
  # pyodide-build/pyodide_build/_f2c_fixes.py.
  script: |
    pip install scipy==1.7.3 pybind11[global] pythran
    pip install --upgrade pybind11[global] pythran -t $PYODIDE_ROOT/packages/.artifacts/
    # We get linker errors because the following 36 functions are missing
    # Copying them from a more recent LAPACK seems to work fine.
    wget https://github.com/Reference-LAPACK/lapack/archive/refs/tags/v3.10.0.tar.gz
    tar xzf v3.10.0.tar.gz
    cd lapack-3.10.0/SRC
    cat \
      cgemqrt.f cgeqrfp.f cgeqrt.f clahqr.f csyconv.f csyconvf.f csyconvf_rook.f ctpmqrt.f ctpqrt.f \
      dgemqrt.f dgeqrfp.f dgeqrt.f dlahqr.f dsyconv.f dsyconvf.f dsyconvf_rook.f dtpmqrt.f dtpqrt.f \
      sgemqrt.f sgeqrfp.f sgeqrt.f slahqr.f ssyconv.f ssyconvf.f ssyconvf_rook.f stpmqrt.f stpqrt.f \
      zgemqrt.f zgeqrfp.f zgeqrt.f zlahqr.f zsyconv.f zsyconvf.f zsyconvf_rook.f ztpmqrt.f ztpqrt.f \
    >>  ../../scipy/linalg/lapack_extras.f
    cd ../..
    # The additional four functions cuncsd, dorcsd, sorcsd, and zuncsd are also
    # missing but they use features of Fortran that aren't Fortran 77 compatible
    # so f2c can't handle them. We stub them with C definitions that do nothing.
    # These stubs come from f2cpatches/wrap_dummy_g77_abi.patch

    # Change many functions that return void into functions that return int
    find scipy -name "*.c*" | xargs sed -i 's/extern void F_FUNC/extern int F_FUNC/g'
    sed -i 's/void F_FUNC/int F_FUNC/g' scipy/odr/__odrpack.c
    sed -i 's/extern void/extern int/g' scipy/optimize/__minpack.h
    sed -i 's/^void/int/g' scipy/interpolate/src/_fitpackmodule.c
    sed -i 's/void/int/g' scipy/linalg/cython_blas_signatures.txt

    # Missing declaration from cython_lapack_signatures.txt
    echo "void ilaenv(int *ispec, char *name, char *opts, int *n1, int *n2, int *n3, int *n4)" \
      >>  scipy/linalg/cython_lapack_signatures.txt

    # Input error causes "duplicate symbol" linker errors. Empty out the file.
    echo "" > scipy/sparse/linalg/dsolve/SuperLU/SRC/input_error.c
    echo 'import sys' >> scipy/__init__.py
    echo 'if "pyodide_js" in sys.modules:'  >> scipy/__init__.py
    echo '   from pyodide_js._module import loadDynamicLibrary' >> scipy/__init__.py
    echo "   loadDynamicLibrary('/lib/python$PYMAJOR.$PYMINOR/site-packages/scipy/linalg/_flapack.so')" >> scipy/__init__.py

requirements:
  run:
    - numpy
    - CLAPACK

test:
  imports:
    - scipy
    - scipy.cluster
    - scipy.cluster.vq
    - scipy.cluster.hierarchy
    - scipy.constants
    - scipy.fft
    - scipy.fftpack
    - scipy.integrate
    - scipy.interpolate
    - scipy.io
    - scipy.io.arff
    - scipy.io.matlab
    - scipy.io.wavfile
    - scipy.linalg
    - scipy.linalg.blas
    - scipy.linalg.cython_blas
    - scipy.linalg.lapack
    - scipy.linalg.cython_lapack
    - scipy.linalg.interpolative
    - scipy.misc
    - scipy.ndimage
    - scipy.odr
    - scipy.optimize
    - scipy.signal
    - scipy.signal.windows
    - scipy.sparse
    - scipy.sparse.linalg
    - scipy.sparse.csgraph
    - scipy.spatial
    - scipy.spatial.distance
    - scipy.spatial.transform
    - scipy.special
    - scipy.stats
    - scipy.stats.contingency
    - scipy.stats.distributions
    - scipy.stats.mstats
    - scipy.stats.qmc

package:
  name: scipy
  version: 1.7.3

# See extra explanation in info.md
#
# For future reference: if you see the following errors:
#   Declaration error: adjustable dimension on non-argument
# or:
#   nonconstant array size
# you are trying to compile code that isn't written to the fortran 77 standard.
# The line number in the error points to the last line of the problematic
# subroutine. Try deleting it.

source:
  url: https://files.pythonhosted.org/packages/61/67/1a654b96309c991762ee9bc39c363fc618076b155fe52d295211cf2536c7/scipy-1.7.3.tar.gz
  sha256: ab5875facfdef77e0a47d5fd39ea178b58e60e454a4c85aa1e52fcb80db7babf

  patches:
    - patches/add-lapack_extras-to-flapack.patch
    - patches/convert-xerrwv-message-arg-from-int-to-str.patch
    - patches/disable-blas-detection.patch
    - patches/fix-fortran-files-minpack.patch
    - patches/gemm_-no-const.patch
    - patches/make-int-return-values.patch
    - patches/remove-cuncsd-dorcsd-sorcsd-zuncsd.patch
    - patches/remove-mvnun-not-fortran-77-compliant.patch
    - patches/sasum-returns-double-not-float.patch
    - patches/skip-fortran-fails-to-link.patch
    - patches/rename-_page_trend_test.patch
    - patches/USE_CPP14-branch-doesn-t-work-for-wasm.patch

build:
  cflags: |
    -I$(PYODIDE_ROOT)/packages/CLAPACK/build/CLAPACK-3.2.1/INCLUDE
    -I$(HOSTSITEPACKAGES)/pythran/
    -Wno-return-type
    -DUNDERSCORE_G77
  ldflags: |
    -L$(NUMPY_LIB)
  # IMPORTANT: Other locations important in scipy build process:
  # There are two files built in the "capture" pass that need patching:
  #    _blas_subroutines.h, and _cython
  # Scipy has a bunch of custom logic implemented in
  # pyodide-build/pyodide_build/_f2c_fixes.py.
  script: |
    pip install -t $HOSTSITEPACKAGES pythran scipy==$PKG_VERSION
    # We get linker errors because the following 36 functions are missing
    # Copying them from a more recent LAPACK seems to work fine.
    wget https://github.com/Reference-LAPACK/lapack/archive/refs/tags/v3.10.0.tar.gz
    tar xzf v3.10.0.tar.gz
    cd lapack-3.10.0/SRC
    cat \
      cgemqrt.f cgeqrfp.f cgeqrt.f clahqr.f csyconv.f csyconvf.f csyconvf_rook.f ctpmqrt.f ctpqrt.f \
      dgemqrt.f dgeqrfp.f dgeqrt.f dlahqr.f dsyconv.f dsyconvf.f dsyconvf_rook.f dtpmqrt.f dtpqrt.f \
      sgemqrt.f sgeqrfp.f sgeqrt.f slahqr.f ssyconv.f ssyconvf.f ssyconvf_rook.f stpmqrt.f stpqrt.f \
      zgemqrt.f zgeqrfp.f zgeqrt.f zlahqr.f zsyconv.f zsyconvf.f zsyconvf_rook.f ztpmqrt.f ztpqrt.f \
    >>  ../../scipy/linalg/lapack_extras.f
    cd ../..
    # The additional four functions cuncsd, dorcsd, sorcsd, and zuncsd are also
    # missing but they use features of Fortran that aren't Fortran 77 compatible
    # so f2c can't handle them. We stub them with C definitions that do nothing.
    # These stubs come from f2cpatches/wrap_dummy_g77_abi.patch

    # Change many functions that return void into functions that return int
    find scipy -name "*.c*" | xargs sed -i 's/extern void F_FUNC/extern int F_FUNC/g'
    sed -i 's/void F_FUNC/int F_FUNC/g' scipy/odr/__odrpack.c
    sed -i 's/extern void/extern int/g' scipy/optimize/__minpack.h
    sed -i 's/^void/int/g' scipy/interpolate/src/_fitpackmodule.c
    sed -i 's/void/int/g' scipy/linalg/cython_blas_signatures.txt

    # Missing declaration from cython_lapack_signatures.txt
    echo "void ilaenv(int *ispec, char *name, char *opts, int *n1, int *n2, int *n3, int *n4)" \
      >>  scipy/linalg/cython_lapack_signatures.txt

    # Input error causes "duplicate symbol" linker errors. Empty out the file.
    echo "" > scipy/sparse/linalg/dsolve/SuperLU/SRC/input_error.c
    echo 'import sys' >> scipy/__init__.py
    echo 'if "pyodide_js" in sys.modules:'  >> scipy/__init__.py
    echo '   from pyodide_js._module import loadDynamicLibrary' >> scipy/__init__.py
    echo "   loadDynamicLibrary('/lib/python$PYMAJOR.$PYMINOR/site-packages/scipy/linalg/_flapack.so')" >> scipy/__init__.py

requirements:
  run:
    - numpy
    - CLAPACK

test:
  imports:
    - scipy
    - scipy.cluster
    - scipy.cluster.vq
    - scipy.cluster.hierarchy
    - scipy.constants
    - scipy.fft
    - scipy.fftpack
    - scipy.integrate
    - scipy.interpolate
    - scipy.io
    - scipy.io.arff
    - scipy.io.matlab
    - scipy.io.wavfile
    - scipy.linalg
    - scipy.linalg.blas
    - scipy.linalg.cython_blas
    - scipy.linalg.lapack
    - scipy.linalg.cython_lapack
    - scipy.linalg.interpolative
    - scipy.misc
    - scipy.ndimage
    - scipy.odr
    - scipy.optimize
    - scipy.signal
    - scipy.signal.windows
    - scipy.sparse
    - scipy.sparse.linalg
    - scipy.sparse.csgraph
    - scipy.spatial
    - scipy.spatial.distance
    - scipy.spatial.transform
    - scipy.special
    - scipy.stats
    - scipy.stats.contingency
    - scipy.stats.distributions
    - scipy.stats.mstats
    - scipy.stats.qmc
