Instead of trying to convince setup.py to do what we want, the wheel
package empowers us to unpack and directly manipulate the wheel.
In particular this is important because `--skip-build` really does
not work.

First we build a native wheel. If skip_host is True, then this
"native" wheel has defective empty files as binaries, if it's False,
we build a real native wheel. If skip_host is False, then we use
pip to install our native wheel into host directory.

Then we unpack the wheel, replace the platform tag with
wasm32_emscripten, remove the non-metadata files, and copy in the
new files with the cross compiled binaries.

Of course, this is all only necessary for binary extensions.


Hopefully this will help get us towards using alternative build
backends. The one concern is that this approach depends on
implementation details of where the build backend puts the files
into the directory tree before copying them into the wheel. If
the build backend moved the linked binaries after building them,
we would be in trouble. Or worse if the build backend deleted the
files after packing them into the wheel, we'd be in trouble.

Basically, it seems clear that we can only support alternative
build backends on a case-by-case basis.

@henryiii I'm curious what your thoughts are cross compilation + arbitrary build backends.

Better package cross compilation. Instead of trying to convince setup.py to do what we want, the wheel
package empowers us to unpack and directly manipulate the wheel.
In particular this is important because `--skip-build` really does
not work.

First we build a native wheel. If skip_host is True, then this
"native" wheel has defective empty files as binaries, if it's False,
we build a real native wheel. If skip_host is False, then we use
pip to install our native wheel into host directory.

Then we unpack the wheel, replace the platform tag with
wasm32_emscripten, remove the non-metadata files, and copy in the
new files with the cross compiled binaries.

Of course, this is all only necessary for binary extensions.


Hopefully this will help get us towards using alternative build
backends. The one concern is that this approach depends on
implementation details of where the build backend puts the files
into the directory tree before copying them into the wheel. If
the build backend moved the linked binaries after building them,
we would be in trouble. Or worse if the build backend deleted the
files after packing them into the wheel, we'd be in trouble.

Basically, it seems clear that we can only support alternative
build backends on a case-by-case basis.

@henryiii I'm curious what your thoughts are cross compilation + arbitrary build backends.

export PYVERSION ?= 3.9.5
export PYODIDE_EMSCRIPTEN_VERSION ?= 2.0.16

# BASH_ENV tells bash to run pyodide_env.sh on startup, which sets various
# environment variables. The next line instructs make to use bash to run each
# command.
export BASH_ENV := $(PYODIDE_ROOT)/pyodide_env.sh
SHELL := /bin/bash

export TOOLSDIR=$(PYODIDE_ROOT)/tools

version_tuple := $(subst ., ,$(PYVERSION:v%=%))
export PYMAJOR=$(word 1,$(version_tuple))
export PYMINOR=$(word 2,$(version_tuple))
export PYMICRO=$(word 3,$(version_tuple))

export HOSTPYTHONROOT=$(shell python${PYMAJOR}.${PYMINOR} -c "import sys; print(sys.prefix)")
export HOSTPYTHON=$(HOSTPYTHONROOT)/bin/python$(PYMAJOR).$(PYMINOR)

export CPYTHONROOT=$(PYODIDE_ROOT)/cpython
export CPYTHONLIB=$(CPYTHONROOT)/installs/python-$(PYVERSION)/lib/python$(PYMAJOR).$(PYMINOR)

export TARGETINSTALLDIR=$(PYODIDE_ROOT)/cpython/installs/python-$(PYVERSION)
export HOSTINSTALLDIR=$(PYODIDE_ROOT)/packages/.artifacts

export PYTHONINCLUDE=$(PYODIDE_ROOT)/cpython/installs/python-$(PYVERSION)/include/python$(PYMAJOR).$(PYMINOR)

# Use env variable if defined, otherwise fallback to './'
export PYODIDE_BASE_URL?=./

# For packages that depend on numpy.
# TODO: maybe move this somewhere else?
export NUMPY_LIB=$(PYODIDE_ROOT)/packages/numpy/build/numpy-1.21.4/build/temp.emscripten_wasm32-$(PYMAJOR).$(PYMINOR)

# This environment variable is used for packages to detect if they are built
# for pyodide during build time
export PYODIDE=1
# This is the legacy environment variable used for the aforementioned purpose
export PYODIDE_PACKAGE_ABI=1

# export DBGFLAGS=-g3 -gseparate-dwarf -sSEPARATE_DWARF_URL=http://localhost:8001/
export DBGFLAGS=-g0

export OPTFLAGS=-O2
export CFLAGS_BASE=\
	$(OPTFLAGS) \
	$(DBGFLAGS) \
	-fPIC \
	$(EXTRA_CFLAGS)

export LDFLAGS_BASE=\
	$(OPTFLAGS) \
	$(DBGFLAGS) \
	-s MODULARIZE=1 \
	-s LINKABLE=1 \
	-s EXPORT_ALL=1 \
	-s WASM=1 \
	-std=c++14 \
	-s LZ4=1 \
	-L $(CPYTHONROOT)/installs/python-$(PYVERSION)/lib/ \
	$(EXTRA_LDFLAGS)

export CXXFLAGS_BASE=

export SIDE_MODULE_LDFLAGS=	$(LDFLAGS_BASE) -s SIDE_MODULE=1
export MAIN_MODULE_LDFLAGS= $(LDFLAGS_BASE) \
	-s MAIN_MODULE=1 \
	-s EXPORT_NAME="'_createPyodideModule'" \
	-s EXPORTED_FUNCTIONS='["___cxa_guard_acquire", "__ZNSt3__28ios_base4initEPv", "_main"]' \
	-s EXCEPTION_CATCHING_ALLOWED=['we only want to allow exception handling in side modules'] \
	-s DEMANGLE_SUPPORT=1 \
	-s USE_FREETYPE=1 \
	-s USE_LIBPNG=1 \
	-s FORCE_FILESYSTEM=1 \
	-s TOTAL_MEMORY=20971520 \
	-s ALLOW_MEMORY_GROWTH=1 \
	\
	-lpython$(PYMAJOR).$(PYMINOR) \
	-lffi \
	-lsqlite3 \
	-lbz2 \
	-lstdc++ \
	-lidbfs.js \
	-lnodefs.js \
	-lproxyfs.js \
	-lworkerfs.js \
	\
	--use-preload-plugins \
	--preload-file $(CPYTHONLIB)@/lib/python$(PYMAJOR).$(PYMINOR) \
	--preload-file src/py/lib@/lib/python$(PYMAJOR).$(PYMINOR)/\
	--exclude-file "*__pycache__*" \
	--exclude-file "*/test/*" \
	--exclude-file "*/tests/*" \
	--exclude-file "*/distutils/*" \
	--pre-js src/core/pre.js


export SIDE_MODULE_CXXFLAGS = $(CXXFLAGS_BASE)

export SIDE_MODULE_CFLAGS= $(CFLAGS_BASE)
export MAIN_MODULE_CFLAGS= $(CFLAGS_BASE) \
	-Wall \
	-Wno-warn-absolute-paths \
	-Werror=unused-variable \
	-Werror=sometimes-uninitialized \
	-Werror=int-conversion \
	-Werror=incompatible-pointer-types \
	-Werror=unused-result \
	-I$(PYTHONINCLUDE) \
	-s EXCEPTION_CATCHING_ALLOWED=['we only want to allow exception handling in side modules']


.output_vars:
	set

export PYVERSION ?= 3.9.5
export PYODIDE_EMSCRIPTEN_VERSION ?= 2.0.16

# BASH_ENV tells bash to run pyodide_env.sh on startup, which sets various
# environment variables. The next line instructs make to use bash to run each
# command.
export BASH_ENV := $(PYODIDE_ROOT)/pyodide_env.sh
SHELL := /bin/bash

export TOOLSDIR=$(PYODIDE_ROOT)/tools

version_tuple := $(subst ., ,$(PYVERSION:v%=%))
export PYMAJOR=$(word 1,$(version_tuple))
export PYMINOR=$(word 2,$(version_tuple))
export PYMICRO=$(word 3,$(version_tuple))

export HOSTPYTHONROOT=$(shell python${PYMAJOR}.${PYMINOR} -c "import sys; print(sys.prefix)")
export HOSTPYTHON=$(HOSTPYTHONROOT)/bin/python$(PYMAJOR).$(PYMINOR)

export CPYTHONROOT=$(PYODIDE_ROOT)/cpython
export CPYTHONLIB=$(CPYTHONROOT)/installs/python-$(PYVERSION)/lib/python$(PYMAJOR).$(PYMINOR)

export TARGETINSTALLDIR=$(PYODIDE_ROOT)/cpython/installs/python-$(PYVERSION)
export HOSTINSTALLDIR=$(PYODIDE_ROOT)/packages/.artifacts

export PYTHONINCLUDE=$(PYODIDE_ROOT)/cpython/installs/python-$(PYVERSION)/include/python$(PYMAJOR).$(PYMINOR)

# Use env variable if defined, otherwise fallback to './'
export PYODIDE_BASE_URL?=./

# For packages that depend on numpy.
# TODO: maybe move this somewhere else?
export NUMPY_LIB=$(HOSTINSTALLDIR)/numpy-wasm-libs

# This environment variable is used for packages to detect if they are built
# for pyodide during build time
export PYODIDE=1
# This is the legacy environment variable used for the aforementioned purpose
export PYODIDE_PACKAGE_ABI=1

# export DBGFLAGS=-g3 -gseparate-dwarf -sSEPARATE_DWARF_URL=http://localhost:8001/
export DBGFLAGS=-g0

export OPTFLAGS=-O2
export CFLAGS_BASE=\
	$(OPTFLAGS) \
	$(DBGFLAGS) \
	-fPIC \
	$(EXTRA_CFLAGS)

export LDFLAGS_BASE=\
	$(OPTFLAGS) \
	$(DBGFLAGS) \
	-s MODULARIZE=1 \
	-s LINKABLE=1 \
	-s EXPORT_ALL=1 \
	-s WASM=1 \
	-std=c++14 \
	-s LZ4=1 \
	-L $(CPYTHONROOT)/installs/python-$(PYVERSION)/lib/ \
	$(EXTRA_LDFLAGS)

export CXXFLAGS_BASE=

export SIDE_MODULE_LDFLAGS=	$(LDFLAGS_BASE) -s SIDE_MODULE=1
export MAIN_MODULE_LDFLAGS= $(LDFLAGS_BASE) \
	-s MAIN_MODULE=1 \
	-s EXPORT_NAME="'_createPyodideModule'" \
	-s EXPORTED_FUNCTIONS='["___cxa_guard_acquire", "__ZNSt3__28ios_base4initEPv", "_main"]' \
	-s EXCEPTION_CATCHING_ALLOWED=['we only want to allow exception handling in side modules'] \
	-s DEMANGLE_SUPPORT=1 \
	-s USE_FREETYPE=1 \
	-s USE_LIBPNG=1 \
	-s FORCE_FILESYSTEM=1 \
	-s TOTAL_MEMORY=20971520 \
	-s ALLOW_MEMORY_GROWTH=1 \
	\
	-lpython$(PYMAJOR).$(PYMINOR) \
	-lffi \
	-lsqlite3 \
	-lbz2 \
	-lstdc++ \
	-lidbfs.js \
	-lnodefs.js \
	-lproxyfs.js \
	-lworkerfs.js \
	\
	--use-preload-plugins \
	--preload-file $(CPYTHONLIB)@/lib/python$(PYMAJOR).$(PYMINOR) \
	--preload-file src/py/lib@/lib/python$(PYMAJOR).$(PYMINOR)/\
	--exclude-file "*__pycache__*" \
	--exclude-file "*/test/*" \
	--exclude-file "*/tests/*" \
	--exclude-file "*/distutils/*" \
	--pre-js src/core/pre.js


export SIDE_MODULE_CXXFLAGS = $(CXXFLAGS_BASE)

export SIDE_MODULE_CFLAGS= $(CFLAGS_BASE)
export MAIN_MODULE_CFLAGS= $(CFLAGS_BASE) \
	-Wall \
	-Wno-warn-absolute-paths \
	-Werror=unused-variable \
	-Werror=sometimes-uninitialized \
	-Werror=int-conversion \
	-Werror=incompatible-pointer-types \
	-Werror=unused-result \
	-I$(PYTHONINCLUDE) \
	-s EXCEPTION_CATCHING_ALLOWED=['we only want to allow exception handling in side modules']


.output_vars:
	set
