Instead of trying to convince setup.py to do what we want, the wheel
package empowers us to unpack and directly manipulate the wheel.
In particular this is important because `--skip-build` really does
not work.

First we build a native wheel. If skip_host is True, then this
"native" wheel has defective empty files as binaries, if it's False,
we build a real native wheel. If skip_host is False, then we use
pip to install our native wheel into host directory.

Then we unpack the wheel, replace the platform tag with
wasm32_emscripten, remove the non-metadata files, and copy in the
new files with the cross compiled binaries.

Of course, this is all only necessary for binary extensions.


Hopefully this will help get us towards using alternative build
backends. The one concern is that this approach depends on
implementation details of where the build backend puts the files
into the directory tree before copying them into the wheel. If
the build backend moved the linked binaries after building them,
we would be in trouble. Or worse if the build backend deleted the
files after packing them into the wheel, we'd be in trouble.

Basically, it seems clear that we can only support alternative
build backends on a case-by-case basis.

@henryiii I'm curious what your thoughts are cross compilation + arbitrary build backends.

Better package cross compilation. Instead of trying to convince setup.py to do what we want, the wheel
package empowers us to unpack and directly manipulate the wheel.
In particular this is important because `--skip-build` really does
not work.

First we build a native wheel. If skip_host is True, then this
"native" wheel has defective empty files as binaries, if it's False,
we build a real native wheel. If skip_host is False, then we use
pip to install our native wheel into host directory.

Then we unpack the wheel, replace the platform tag with
wasm32_emscripten, remove the non-metadata files, and copy in the
new files with the cross compiled binaries.

Of course, this is all only necessary for binary extensions.


Hopefully this will help get us towards using alternative build
backends. The one concern is that this approach depends on
implementation details of where the build backend puts the files
into the directory tree before copying them into the wheel. If
the build backend moved the linked binaries after building them,
we would be in trouble. Or worse if the build backend deleted the
files after packing them into the wheel, we'd be in trouble.

Basically, it seems clear that we can only support alternative
build backends on a case-by-case basis.

@henryiii I'm curious what your thoughts are cross compilation + arbitrary build backends.

#!/usr/bin/perl

use strict;
use warnings;
use IPC::Open2;

# An example hook script to integrate Watchman
# (https://facebook.github.io/watchman/) with git to speed up detecting
# new and modified files.
#
# The hook is passed a version (currently 1) and a time in nanoseconds
# formatted as a string and outputs to stdout all files that have been
# modified since the given time. Paths must be relative to the root of
# the working tree and separated by a single NUL.
#
# To enable this hook, rename this file to "query-watchman" and set
# 'git config core.fsmonitor .git/hooks/query-watchman'
#
my ($version, $time) = @ARGV;

# Check the hook interface version

if ($version == 1) {
	# convert nanoseconds to seconds
	# subtract one second to make sure watchman will return all changes
	$time = int ($time / 1000000000) - 1;
} else {
	die "Unsupported query-fsmonitor hook version '$version'.\n" .
	    "Falling back to scanning...\n";
}

my $git_work_tree;
if ($^O =~ 'msys' || $^O =~ 'cygwin') {
	$git_work_tree = Win32::GetCwd();
	$git_work_tree =~ tr/\\/\//;
} else {
	require Cwd;
	$git_work_tree = Cwd::cwd();
}

my $retry = 1;

launch_watchman();

sub launch_watchman {

	my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')
	    or die "open2() failed: $!\n" .
	    "Falling back to scanning...\n";

	# In the query expression below we're asking for names of files that
	# changed since $time but were not transient (ie created after
	# $time but no longer exist).
	#
	# To accomplish this, we're using the "since" generator to use the
	# recency index to select candidate nodes and "fields" to limit the
	# output to file names only.

	my $query = <<"	END";
		["query", "$git_work_tree", {
			"since": $time,
			"fields": ["name"]
		}]
	END

	print CHLD_IN $query;
	close CHLD_IN;
	my $response = do {local $/; <CHLD_OUT>};

	die "Watchman: command returned no output.\n" .
	    "Falling back to scanning...\n" if $response eq "";
	die "Watchman: command returned invalid output: $response\n" .
	    "Falling back to scanning...\n" unless $response =~ /^\{/;

	my $json_pkg;
	eval {
		require JSON::XS;
		$json_pkg = "JSON::XS";
		1;
	} or do {
		require JSON::PP;
		$json_pkg = "JSON::PP";
	};

	my $o = $json_pkg->new->utf8->decode($response);

	if ($retry > 0 and $o->{error} and $o->{error} =~ m/unable to resolve root .* directory (.*) is not watched/) {
		print STDERR "Adding '$git_work_tree' to watchman's watch list.\n";
		$retry--;
		qx/watchman watch "$git_work_tree"/;
		die "Failed to make watchman watch '$git_work_tree'.\n" .
		    "Falling back to scanning...\n" if $? != 0;

		# Watchman will always return all files on the first query so
		# return the fast "everything is dirty" flag to git and do the
		# Watchman query just to get it over with now so we won't pay
		# the cost in git to look up each individual file.
		print "/\0";
		eval { launch_watchman() };
		exit 0;
	}

	die "Watchman: $o->{error}.\n" .
	    "Falling back to scanning...\n" if $o->{error};

	binmode STDOUT, ":utf8";
	local $, = "\0";
	print @{$o->{files}};
}

#!/usr/bin/perl

use strict;
use warnings;
use IPC::Open2;

# An example hook script to integrate Watchman
# (https://facebook.github.io/watchman/) with git to speed up detecting
# new and modified files.
#
# The hook is passed a version (currently 1) and a time in nanoseconds
# formatted as a string and outputs to stdout all files that have been
# modified since the given time. Paths must be relative to the root of
# the working tree and separated by a single NUL.
#
# To enable this hook, rename this file to "query-watchman" and set
# 'git config core.fsmonitor .git/hooks/query-watchman'
#
my ($version, $time) = @ARGV;

# Check the hook interface version

if ($version == 1) {
	# convert nanoseconds to seconds
	# subtract one second to make sure watchman will return all changes
	$time = int ($time / 1000000000) - 1;
} else {
	die "Unsupported query-fsmonitor hook version '$version'.\n" .
	    "Falling back to scanning...\n";
}

my $git_work_tree;
if ($^O =~ 'msys' || $^O =~ 'cygwin') {
	$git_work_tree = Win32::GetCwd();
	$git_work_tree =~ tr/\\/\//;
} else {
	require Cwd;
	$git_work_tree = Cwd::cwd();
}

my $retry = 1;

launch_watchman();

sub launch_watchman {

	my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')
	    or die "open2() failed: $!\n" .
	    "Falling back to scanning...\n";

	# In the query expression below we're asking for names of files that
	# changed since $time but were not transient (ie created after
	# $time but no longer exist).
	#
	# To accomplish this, we're using the "since" generator to use the
	# recency index to select candidate nodes and "fields" to limit the
	# output to file names only.

	my $query = <<"	END";
		["query", "$git_work_tree", {
			"since": $time,
			"fields": ["name"]
		}]
	END

	print CHLD_IN $query;
	close CHLD_IN;
	my $response = do {local $/; <CHLD_OUT>};

	die "Watchman: command returned no output.\n" .
	    "Falling back to scanning...\n" if $response eq "";
	die "Watchman: command returned invalid output: $response\n" .
	    "Falling back to scanning...\n" unless $response =~ /^\{/;

	my $json_pkg;
	eval {
		require JSON::XS;
		$json_pkg = "JSON::XS";
		1;
	} or do {
		require JSON::PP;
		$json_pkg = "JSON::PP";
	};

	my $o = $json_pkg->new->utf8->decode($response);

	if ($retry > 0 and $o->{error} and $o->{error} =~ m/unable to resolve root .* directory (.*) is not watched/) {
		print STDERR "Adding '$git_work_tree' to watchman's watch list.\n";
		$retry--;
		qx/watchman watch "$git_work_tree"/;
		die "Failed to make watchman watch '$git_work_tree'.\n" .
		    "Falling back to scanning...\n" if $? != 0;

		# Watchman will always return all files on the first query so
		# return the fast "everything is dirty" flag to git and do the
		# Watchman query just to get it over with now so we won't pay
		# the cost in git to look up each individual file.
		print "/\0";
		eval { launch_watchman() };
		exit 0;
	}

	die "Watchman: $o->{error}.\n" .
	    "Falling back to scanning...\n" if $o->{error};

	binmode STDOUT, ":utf8";
	local $, = "\0";
	print @{$o->{files}};
}
