Our package build process currently has a significant flaw: we first run `setup.py`, recording all compilation commands, then we rewrite these compilation commands to invoke `emcc` and replay them, and then we pray that the cross compiled executables ended up in the right place to go into the wheel. This is not a good strategy because the build script is allowed to implement arbitrary logic, and if it moves, renames, etc any of the output files then we lose track of them. This has repeatedly caused difficulty for us.

However, we also make no particularly significant use of the two pass approach. We can just do the simpler thing: capture the compiler commands as they occur, modify them as needed, and then run the fixed command.

I also added a patch to fix the numpy feature detection for wasm so that we don't have to include `_npyconfig.h` and `config.h`, numpy can generate them in the way it would for a native build. I opened a numpy PR that would fix the detection for us upstream:
https://github.com/numpy/numpy/pull/21154

This clears the way for us to switch to using pypa/build (as @henryiii has suggested) by removing our dependence on specific setuptools behavior.

This is on top of #2238.
No replay. Our package build process currently has a significant flaw: we first run `setup.py`, recording all compilation commands, then we rewrite these compilation commands to invoke `emcc` and replay them, and then we pray that the cross compiled executables ended up in the right place to go into the wheel. This is not a good strategy because the build script is allowed to implement arbitrary logic, and if it moves, renames, etc any of the output files then we lose track of them. This has repeatedly caused difficulty for us.

However, we also make no particularly significant use of the two pass approach. We can just do the simpler thing: capture the compiler commands as they occur, modify them as needed, and then run the fixed command.

I also added a patch to fix the numpy feature detection for wasm so that we don't have to include `_npyconfig.h` and `config.h`, numpy can generate them in the way it would for a native build. I opened a numpy PR that would fix the detection for us upstream:
https://github.com/numpy/numpy/pull/21154

This clears the way for us to switch to using pypa/build (as @henryiii has suggested) by removing our dependence on specific setuptools behavior.

This is on top of #2238.
package:
  name: scipy
  version: 1.7.3 # when updating version make sure to update pip install scipy==1.7.3 below too.

# See extra explanation in info.md
#
# For future reference: if you see the following errors:
#   Declaration error: adjustable dimension on non-argument
# or:
#   nonconstant array size
# you are trying to compile code that isn't written to the fortran 77 standard.
# The line number in the error points to the last line of the problematic
# subroutine. Try deleting it.

source:
  url: https://files.pythonhosted.org/packages/61/67/1a654b96309c991762ee9bc39c363fc618076b155fe52d295211cf2536c7/scipy-1.7.3.tar.gz
  sha256: ab5875facfdef77e0a47d5fd39ea178b58e60e454a4c85aa1e52fcb80db7babf

  patches:
    - patches/add-lapack_extras-to-flapack.patch
    - patches/convert-xerrwv-message-arg-from-int-to-str.patch
    - patches/disable-blas-detection.patch
    - patches/fix-fortran-files-minpack.patch
    - patches/gemm_-no-const.patch
    - patches/make-int-return-values.patch
    - patches/remove-cuncsd-dorcsd-sorcsd-zuncsd.patch
    - patches/remove-mvnun-not-fortran-77-compliant.patch
    - patches/sasum-returns-double-not-float.patch
    - patches/skip-fortran-fails-to-link.patch
    - patches/rename-_page_trend_test.patch
    - patches/USE_CPP14-branch-doesn-t-work-for-wasm.patch

build:
  # set linker and C flags to error on anything to do with function declarations being wrong.
  # In webassembly, any conflicts mean that a randomly selected 50% of calls to the function
  # will fail. Better to fail at compile or link time.
  cflags: |
    -I$(PYODIDE_ROOT)/packages/numpy/config
    -I$(PYODIDE_ROOT)/packages/CLAPACK/build/CLAPACK-3.2.1/INCLUDE
    -I$(PYODIDE_ROOT)/packages/.artifacts/include/
    -I$(PYODIDE_ROOT)/packages/.artifacts/pythran/
    -DUNDERSCORE_G77
    -Werror=implicit-function-declaration
    -Werror=mismatched-parameter-types
    -Werror=mismatched-return-types
    -Wno-return-type
  ldflags: |
    -L$(NUMPY_LIB)
    -Wl,--fatal-warnings
  # IMPORTANT: Other locations important in scipy build process:
  # There are two files built in the "capture" pass that need patching:
  #    _blas_subroutines.h, and _cython
  # Scipy has a bunch of custom logic implemented in
  # pyodide-build/pyodide_build/_f2c_fixes.py.
  script: |
    pip install scipy==1.7.3 pybind11[global] pythran
    pip install --upgrade pybind11[global] pythran -t $PYODIDE_ROOT/packages/.artifacts/
    # We get linker errors because the following 36 functions are missing
    # Copying them from a more recent LAPACK seems to work fine.
    wget https://github.com/Reference-LAPACK/lapack/archive/refs/tags/v3.10.0.tar.gz
    tar xzf v3.10.0.tar.gz
    cd lapack-3.10.0/SRC
    cat \
      cgemqrt.f cgeqrfp.f cgeqrt.f clahqr.f csyconv.f csyconvf.f csyconvf_rook.f ctpmqrt.f ctpqrt.f \
      dgemqrt.f dgeqrfp.f dgeqrt.f dlahqr.f dsyconv.f dsyconvf.f dsyconvf_rook.f dtpmqrt.f dtpqrt.f \
      sgemqrt.f sgeqrfp.f sgeqrt.f slahqr.f ssyconv.f ssyconvf.f ssyconvf_rook.f stpmqrt.f stpqrt.f \
      zgemqrt.f zgeqrfp.f zgeqrt.f zlahqr.f zsyconv.f zsyconvf.f zsyconvf_rook.f ztpmqrt.f ztpqrt.f \
    >>  ../../scipy/linalg/lapack_extras.f
    cd ../..
    # The additional four functions cuncsd, dorcsd, sorcsd, and zuncsd are also
    # missing but they use features of Fortran that aren't Fortran 77 compatible
    # so f2c can't handle them. We stub them with C definitions that do nothing.
    # These stubs come from f2cpatches/wrap_dummy_g77_abi.patch

    # Change many functions that return void into functions that return int
    find scipy -name "*.c*" | xargs sed -i 's/extern void F_FUNC/extern int F_FUNC/g'
    sed -i 's/void F_FUNC/int F_FUNC/g' scipy/odr/__odrpack.c
    sed -i 's/extern void/extern int/g' scipy/optimize/__minpack.h
    sed -i 's/^void/int/g' scipy/interpolate/src/_fitpackmodule.c
    sed -i 's/void/int/g' scipy/linalg/cython_blas_signatures.txt

    # Missing declaration from cython_lapack_signatures.txt
    echo "void ilaenv(int *ispec, char *name, char *opts, int *n1, int *n2, int *n3, int *n4)" \
      >>  scipy/linalg/cython_lapack_signatures.txt

    # Input error causes "duplicate symbol" linker errors. Empty out the file.
    echo "" > scipy/sparse/linalg/dsolve/SuperLU/SRC/input_error.c
    echo 'import sys' >> scipy/__init__.py
    echo 'if "pyodide_js" in sys.modules:'  >> scipy/__init__.py
    echo '   from pyodide_js._module import loadDynamicLibrary' >> scipy/__init__.py
    echo '   loadDynamicLibrary("/lib/python3.9/site-packages/scipy/linalg/_flapack.so")' >> scipy/__init__.py

  prereplay: |
    sed -i 's/void F_FUNC/int F_FUNC/g' scipy/linalg/_lapack_subroutines.h
    sed -i 's/void F_FUNC/int F_FUNC/g' scipy/linalg/_blas_subroutines.h
    find -name '*.c' | xargs sed -i 's/extern void F_WRAPPEDFUNC/extern int F_WRAPPEDFUNC/g'
    find -name '*.c' | xargs sed -i 's/void (\*f2py_func)/int (*f2py_func)/g'
    find -name '*.c' | xargs sed -i 's/static void cb_/static int cb_/g'
    find -name '*.c' | xargs sed -i 's/typedef void(\*cb_/typedef int(*cb_/g'
    cd build/src.linux-x86_64-$PYMAJOR.$PYMINOR/build/src.linux-x86_64-$PYMAJOR.$PYMINOR/scipy/linalg
    sed -i 's/,size_t//' _flapackmodule.c
    sed -i 's/,slen([a-z]*))/)/g' _flapackmodule.c
    sed -i 's/ float (\*f2py_func)/ double (\*f2py_func)/g' _fblasmodule.c

requirements:
  run:
    - numpy
    - CLAPACK

test:
  imports:
    - scipy
    - scipy.cluster
    - scipy.cluster.vq
    - scipy.cluster.hierarchy
    - scipy.constants
    - scipy.fft
    - scipy.fftpack
    - scipy.integrate
    - scipy.interpolate
    - scipy.io
    - scipy.io.arff
    - scipy.io.matlab
    - scipy.io.wavfile
    - scipy.linalg
    - scipy.linalg.blas
    - scipy.linalg.cython_blas
    - scipy.linalg.lapack
    - scipy.linalg.cython_lapack
    - scipy.linalg.interpolative
    - scipy.misc
    - scipy.ndimage
    - scipy.odr
    - scipy.optimize
    - scipy.signal
    - scipy.signal.windows
    - scipy.sparse
    - scipy.sparse.linalg
    - scipy.sparse.csgraph
    - scipy.spatial
    - scipy.spatial.distance
    - scipy.spatial.transform
    - scipy.special
    - scipy.stats
    - scipy.stats.contingency
    - scipy.stats.distributions
    - scipy.stats.mstats
    - scipy.stats.qmc

package:
  name: scipy
  version: 1.7.3 # when updating version make sure to update pip install scipy==1.7.3 below too.

# See extra explanation in info.md
#
# For future reference: if you see the following errors:
#   Declaration error: adjustable dimension on non-argument
# or:
#   nonconstant array size
# you are trying to compile code that isn't written to the fortran 77 standard.
# The line number in the error points to the last line of the problematic
# subroutine. Try deleting it.

source:
  url: https://files.pythonhosted.org/packages/61/67/1a654b96309c991762ee9bc39c363fc618076b155fe52d295211cf2536c7/scipy-1.7.3.tar.gz
  sha256: ab5875facfdef77e0a47d5fd39ea178b58e60e454a4c85aa1e52fcb80db7babf

  patches:
    - patches/add-lapack_extras-to-flapack.patch
    - patches/convert-xerrwv-message-arg-from-int-to-str.patch
    - patches/disable-blas-detection.patch
    - patches/fix-fortran-files-minpack.patch
    - patches/gemm_-no-const.patch
    - patches/make-int-return-values.patch
    - patches/remove-cuncsd-dorcsd-sorcsd-zuncsd.patch
    - patches/remove-mvnun-not-fortran-77-compliant.patch
    - patches/sasum-returns-double-not-float.patch
    - patches/skip-fortran-fails-to-link.patch
    - patches/rename-_page_trend_test.patch
    - patches/USE_CPP14-branch-doesn-t-work-for-wasm.patch

build:
  cflags: |
    -I$(PYODIDE_ROOT)/packages/numpy/config
    -I$(PYODIDE_ROOT)/packages/CLAPACK/build/CLAPACK-3.2.1/INCLUDE
    -I$(PYODIDE_ROOT)/packages/.artifacts/include/
    -I$(PYODIDE_ROOT)/packages/.artifacts/pythran/
    -Wno-return-type
    -DUNDERSCORE_G77
  ldflags: |
    -L$(NUMPY_LIB)
  # IMPORTANT: Other locations important in scipy build process:
  # There are two files built in the "capture" pass that need patching:
  #    _blas_subroutines.h, and _cython
  # Scipy has a bunch of custom logic implemented in
  # pyodide-build/pyodide_build/_f2c_fixes.py.
  script: |
    pip install scipy==1.7.3 pybind11[global] pythran
    pip install --upgrade pybind11[global] pythran -t $PYODIDE_ROOT/packages/.artifacts/
    # We get linker errors because the following 36 functions are missing
    # Copying them from a more recent LAPACK seems to work fine.
    wget https://github.com/Reference-LAPACK/lapack/archive/refs/tags/v3.10.0.tar.gz
    tar xzf v3.10.0.tar.gz
    cd lapack-3.10.0/SRC
    cat \
      cgemqrt.f cgeqrfp.f cgeqrt.f clahqr.f csyconv.f csyconvf.f csyconvf_rook.f ctpmqrt.f ctpqrt.f \
      dgemqrt.f dgeqrfp.f dgeqrt.f dlahqr.f dsyconv.f dsyconvf.f dsyconvf_rook.f dtpmqrt.f dtpqrt.f \
      sgemqrt.f sgeqrfp.f sgeqrt.f slahqr.f ssyconv.f ssyconvf.f ssyconvf_rook.f stpmqrt.f stpqrt.f \
      zgemqrt.f zgeqrfp.f zgeqrt.f zlahqr.f zsyconv.f zsyconvf.f zsyconvf_rook.f ztpmqrt.f ztpqrt.f \
    >>  ../../scipy/linalg/lapack_extras.f
    cd ../..
    # The additional four functions cuncsd, dorcsd, sorcsd, and zuncsd are also
    # missing but they use features of Fortran that aren't Fortran 77 compatible
    # so f2c can't handle them. We stub them with C definitions that do nothing.
    # These stubs come from f2cpatches/wrap_dummy_g77_abi.patch

    # Change many functions that return void into functions that return int
    find scipy -name "*.c*" | xargs sed -i 's/extern void F_FUNC/extern int F_FUNC/g'
    sed -i 's/void F_FUNC/int F_FUNC/g' scipy/odr/__odrpack.c
    sed -i 's/extern void/extern int/g' scipy/optimize/__minpack.h
    sed -i 's/^void/int/g' scipy/interpolate/src/_fitpackmodule.c
    sed -i 's/void/int/g' scipy/linalg/cython_blas_signatures.txt

    # Missing declaration from cython_lapack_signatures.txt
    echo "void ilaenv(int *ispec, char *name, char *opts, int *n1, int *n2, int *n3, int *n4)" \
      >>  scipy/linalg/cython_lapack_signatures.txt

    # Input error causes "duplicate symbol" linker errors. Empty out the file.
    echo "" > scipy/sparse/linalg/dsolve/SuperLU/SRC/input_error.c
    echo 'import sys' >> scipy/__init__.py
    echo 'if "pyodide_js" in sys.modules:'  >> scipy/__init__.py
    echo '   from pyodide_js._module import loadDynamicLibrary' >> scipy/__init__.py
    echo '   loadDynamicLibrary("/lib/python3.9/site-packages/scipy/linalg/_flapack.so")' >> scipy/__init__.py

requirements:
  run:
    - numpy
    - CLAPACK

test:
  imports:
    - scipy
    - scipy.cluster
    - scipy.cluster.vq
    - scipy.cluster.hierarchy
    - scipy.constants
    - scipy.fft
    - scipy.fftpack
    - scipy.integrate
    - scipy.interpolate
    - scipy.io
    - scipy.io.arff
    - scipy.io.matlab
    - scipy.io.wavfile
    - scipy.linalg
    - scipy.linalg.blas
    - scipy.linalg.cython_blas
    - scipy.linalg.lapack
    - scipy.linalg.cython_lapack
    - scipy.linalg.interpolative
    - scipy.misc
    - scipy.ndimage
    - scipy.odr
    - scipy.optimize
    - scipy.signal
    - scipy.signal.windows
    - scipy.sparse
    - scipy.sparse.linalg
    - scipy.sparse.csgraph
    - scipy.spatial
    - scipy.spatial.distance
    - scipy.spatial.transform
    - scipy.special
    - scipy.stats
    - scipy.stats.contingency
    - scipy.stats.distributions
    - scipy.stats.mstats
    - scipy.stats.qmc
