c8d: Just enough Windows support to run the test suite
<!--
Please make sure you've read and understood our contributing guidelines;
https://github.com/moby/moby/blob/master/CONTRIBUTING.md

** Make sure all your commits include a signature generated with `git commit -s` **

For additional information on our contributing process, read our contributing
guide https://docs.docker.com/opensource/code/

If this is a bug fix, make sure your description includes "fixes #xxxx", or
"closes #xxxx"

Please provide the following information:
-->

**- What I did**

Building upon ~~#46402~~ #46572 to bring the containerd image-store support on Windows up to something that is worth having tests enabled for.

**- How I did it**

Firstly, cherry-picked #46402 onto the mainline branch, once containerd 1.7.6 had been vendored by #45966. Update: Currently based directly on #46402 until it merges, unless something important lands in mainline first.

Secondly, got the snapshotter tests functional on Windows matching the Linux version. (Two-liner, most of the work was done by @rumpl in #46402).

~~Violently beat my head against~~ Iteratively worked through failures observed in the integration test suite setup, building the busybox image. As it happens, this step does most of the things we seem to care about, so it was a good test-case.

Rebased the results of the above after #46572 landed, which integrated the outcomes of the second paragraph already.

Remaining TODOs to undraft this PR:

- [x] Label this PR with `containerd-integration` (someone needs to do that for me) and drop the last (0d5edcdca184c8bd3c50b8b25d75544fa9e7d190) commit
- [x] Rebase once ~~#46402~~ #46572 is merged, dropping the first couple of commits
- [x] Make and implement a decision about hack 1 (53777db68e5172cc50568a21959dff704565110d)
- [x] Make and implement a decision about hack 2 (https://github.com/moby/moby/pull/46539/commits/6e48f48df593fb689bb26894a36949632f61bd24 or [`b099f42`](https://github.com/moby/moby/pull/46539/files/53777db68e5172cc50568a21959dff704565110d..b099f42423c30eeaa26c77de32b0e169103c9e82))

----

*- Hack 1 (53777db68e5172cc50568a21959dff704565110d)*

builder-next specifies the `host` network provider, but [that fails on Windows](https://github.com/moby/buildkit/blob/v0.12.2/util/network/netproviders/network_windows.go#L11-L13), blocking daemon startup even though we don't currently support builder-next.

My current hack is to use `auto`, since that means [`cni` if configured](https://github.com/moby/buildkit/blob/v0.12.2/util/network/netproviders/network.go#L36-L46) and if not, [on Windows it falls back to `null`](https://github.com/moby/buildkit/blob/v0.12.2/util/network/netproviders/network_windows.go#L15-L18), and [on Linux it falls back to `host`](https://github.com/moby/buildkit/blob/v0.12.2/util/network/netproviders/network_unix.go#L15-L18).

I've currently only done this for Windows with a `runtime.GOOS` test, but I'm inclined to think we should use `auto` always, since: it does the right thing on Linux, and if we ever do need a more-complex network setup, CNI is the obvious path. If we ever do get builder-next working for Windows, it'll need to use CNI anyway, even for the equivalent of host networking.

----

*- Hack 2 (https://github.com/moby/moby/pull/46539/commits/6e48f48df593fb689bb26894a36949632f61bd24 or [`b099f42`](https://github.com/moby/moby/pull/46539/files/53777db68e5172cc50568a21959dff704565110d..b099f42423c30eeaa26c77de32b0e169103c9e82))*

The windowfilter graphdriver is a bit weird: When you "mount" an image from it, it actually [only does the volume setup (Prepare and Activate in HCS terms)](https://github.com/moby/moby/blob/51f0e7b0e49c709729d67559c9c07616c1da019b/daemon/graphdriver/windows/windows.go#L385-L407) and then [returns the `\\?\Volume{GUID}\` formatted volume name](https://github.com/moby/moby/blob/51f0e7b0e49c709729d67559c9c07616c1da019b/daemon/graphdriver/windows/windows.go#L414-L416), where other graphdrivers (and all the containerd snapshotters) actually [mount the image to a host directory](https://github.com/moby/moby/blob/51f0e7b0e49c709729d67559c9c07616c1da019b/daemon/graphdriver/overlay2/overlay.go#L596-L598) and [return that directory](https://github.com/moby/moby/blob/51f0e7b0e49c709729d67559c9c07616c1da019b/daemon/graphdriver/overlay2/overlay.go#L608).

However, starting a container as we do now [requires us to pass that `\\?\Volume{GUID}\`-formatted volume name in the OCI spec as `Root.Path`](https://github.com/opencontainers/runtime-spec/blob/main/config.md#root).

Normally, containerd doesn't expose that volume name to callers, but we can use the underlying Windows API to get the volume name from the mount point, which relies on [internal knowledge of the containerd/mount implementation](https://github.com/moby/moby/pull/46539/files#diff-6688f4342adf127b206582942bc147a0efab01c2e376b8a1a81e62c4bfee3ce1R296-R337) ([earlier implementation of the same idea that I'm less confident in](https://github.com/moby/moby/pull/46539/files/6e48f48df593fb689bb26894a36949632f61bd24#diff-6688f4342adf127b206582942bc147a0efab01c2e376b8a1a81e62c4bfee3ce1R296-R322)).

The BindFilter approach was pretty messy though, and I didn't like it very much. (There's probably a bunch of overlooked edge cases there too. I have no idea why I see four copies of the bind-mount there, which is another vote against the older BindFilter version.) The `GetLayerMountPath` isn't too bad, it's still poking at internal details, but they're details that containerd's mount package could trivially provide an API to expose, if we go this way.

Beyond cleaning it up, an alternative approach would be to not populate `Windows.LayerFolders` nor `Root.Path` at all (removing the need for 1ad566e21dee685a0c10b01054bd40c5de7a0e5b as well) and instead use [containerd's `WithSnapshot` and `WithSnapshotter` config-functor-things](https://github.com/containerd/containerd/blob/v1.7.6/container_opts.go#L178-L207) when setting up the container.

However, this would, at least on Windows, require us to unmount the directory before starting the container, as this flow makes hcsshim responsible for both [mounting of the root layer](https://github.com/microsoft/hcsshim/blob/v0.11.1/internal/hcsoci/resources_wcow.go#L60-L73) and [deriving `Windows.LayerFolders` from the snapshot](https://github.com/microsoft/hcsshim/blob/v0.11.1/cmd/containerd-shim-runhcs-v1/service_internal.go#L118-L158). Note that in the latter case, [there's a bug where hcsshim'll *append* to any `Windows.LayerFolders` passed in](https://github.com/opencontainers/runtime-spec/issues/1185#issuecomment-1474549056) ([0.12 onwards rejects this case explicitly instead](https://github.com/microsoft/hcsshim/blob/main/cmd/containerd-shim-runhcs-v1/rootfs.go#L15-L45)), so we *must not* populate it in this case. `nil` is fine, even though it's [out-of-spec](https://github.com/opencontainers/runtime-spec/issues/1185).

This should be _feasible_ as we already have the `conditionalMountOnStart` hook, but code flows after that is called need to be checked to ensure they are correctly handling the case of not mounting the container filesystem locally. For example, I noticed code on the Windows side that checks the same conditions as the current `conditionalMountOnStart` check, and if they match, does a temporary mount of the image; clearly there's some scope for refactoring that to not duplicate the condition around.

If it was just Windows, I'd have tried to go with this approach. However, I think it'd be cleaner if we use `WithSnapshot` for both Windows and Linux when executing containers. This would be a larger change, and probably has repercussions I'm unaware of at this time, since I haven't really worked to understand the overarching flow and its rationales. (Most other containerd-calling code [appears to use `WithNewSnapshot`](https://github.com/containerd/nerdctl/blob/v1.6.0/pkg/cmd/container/create.go#L316), but moby currently has a pile of stuff it wants to do to the image first, so I'm not considering that here.)

----

**- How to verify it**

CI.

This PR has the `containerd-integration` label applied, so the new snapshotter test suite introduced in ~~#46402~~ #46572 will run for this PR (and any PR with that label), giving us both Windows and Linux coverage. As long as the failures in the snapshotter test suite are consistent across platforms (and I didn't break anything), we're probably fine.

All of the changes made here (except an interface change while the compiler will validate) are only in the "use snapshotter" code-path, so verification should be low-risk.

If someone wants to take the time and try this out with the WASM runtime, that'd be awesome. I have no particular plans to do so, even though that's what started me on this piece of work, months ago.

**- Description for the changelog**
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->

Containerd image store on Windows is slightly usable.

**- A picture of a cute animal (not mandatory but encouraged)**

![image](https://github.com/moby/moby/assets/138397/149b0bcc-f13b-4b02-be07-a49dcfbee82b)

daemon/image_service.go
package daemon

import (
	"context"
	"io"

	"github.com/distribution/reference"
	"github.com/docker/docker/api/types"
	"github.com/docker/docker/api/types/backend"
	"github.com/docker/docker/api/types/events"
	"github.com/docker/docker/api/types/filters"
	imagetype "github.com/docker/docker/api/types/image"
	"github.com/docker/docker/api/types/registry"
	"github.com/docker/docker/builder"
	"github.com/docker/docker/container"
	"github.com/docker/docker/daemon/images"
	"github.com/docker/docker/image"
	"github.com/docker/docker/layer"
	"github.com/docker/docker/pkg/archive"
	"github.com/opencontainers/go-digest"
	ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)

// ImageService is a temporary interface to assist in the migration to the
// containerd image-store. This interface should not be considered stable,
// and may change over time.
type ImageService interface {
	// Images

	PullImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
	PushImage(ctx context.Context, ref reference.Named, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
	CreateImage(ctx context.Context, config []byte, parent string, contentStoreDigest digest.Digest) (builder.Image, error)
	ImageDelete(ctx context.Context, imageRef string, force, prune bool) ([]imagetype.DeleteResponse, error)
	ExportImage(ctx context.Context, names []string, outStream io.Writer) error
	PerformWithBaseFS(ctx context.Context, c *container.Container, fn func(string) error) error
	LoadImage(ctx context.Context, inTar io.ReadCloser, outStream io.Writer, quiet bool) error
	Images(ctx context.Context, opts types.ImageListOptions) ([]*imagetype.Summary, error)
	LogImageEvent(imageID, refName string, action events.Action)
	CountImages(ctx context.Context) int
	ImagesPrune(ctx context.Context, pruneFilters filters.Args) (*types.ImagesPruneReport, error)
	ImportImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, msg string, layerReader io.Reader, changes []string) (image.ID, error)
	TagImage(ctx context.Context, imageID image.ID, newTag reference.Named) error
	GetImage(ctx context.Context, refOrID string, options imagetype.GetImageOpts) (*image.Image, error)
	ImageHistory(ctx context.Context, name string) ([]*imagetype.HistoryResponseItem, error)
	CommitImage(ctx context.Context, c backend.CommitConfig) (image.ID, error)
	SquashImage(id, parent string) (string, error)

	// Containerd related methods

	PrepareSnapshot(ctx context.Context, id string, parentImage string, platform *ocispec.Platform, setupInit func(string) error) error
	GetImageManifest(ctx context.Context, refOrID string, options imagetype.GetImageOpts) (*ocispec.Descriptor, error)

	// Layers

	GetImageAndReleasableLayer(ctx context.Context, refOrID string, opts backend.GetImageAndLayerOptions) (builder.Image, builder.ROLayer, error)
	CreateLayer(container *container.Container, initFunc layer.MountInit) (layer.RWLayer, error)
	LayerStoreStatus() [][2]string
	GetLayerMountID(cid string) (string, error)
	ReleaseLayer(rwlayer layer.RWLayer) error
	LayerDiskUsage(ctx context.Context) (int64, error)
	GetContainerLayerSize(ctx context.Context, containerID string) (int64, int64, error)
	Mount(ctx context.Context, container *container.Container) error
	Unmount(ctx context.Context, container *container.Container) error
	Changes(ctx context.Context, container *container.Container) ([]archive.Change, error)

	// Windows specific

	GetLayerFolders(img *image.Image, rwLayer layer.RWLayer) ([]string, error)

	// Build

	MakeImageCache(ctx context.Context, cacheFrom []string) (builder.ImageCache, error)
	CommitBuildStep(ctx context.Context, c backend.CommitConfig) (image.ID, error)

	// Other

	DistributionServices() images.DistributionServices
	Children(ctx context.Context, id image.ID) ([]image.ID, error)
	Cleanup() error
	StorageDriver() string
	UpdateConfig(maxDownloads, maxUploads int)
}

package daemon

import (
	"context"
	"io"

	"github.com/distribution/reference"
	"github.com/docker/docker/api/types"
	"github.com/docker/docker/api/types/backend"
	"github.com/docker/docker/api/types/events"
	"github.com/docker/docker/api/types/filters"
	imagetype "github.com/docker/docker/api/types/image"
	"github.com/docker/docker/api/types/registry"
	"github.com/docker/docker/builder"
	"github.com/docker/docker/container"
	"github.com/docker/docker/daemon/images"
	"github.com/docker/docker/image"
	"github.com/docker/docker/layer"
	"github.com/docker/docker/pkg/archive"
	"github.com/opencontainers/go-digest"
	ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)

// ImageService is a temporary interface to assist in the migration to the
// containerd image-store. This interface should not be considered stable,
// and may change over time.
type ImageService interface {
	// Images

	PullImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
	PushImage(ctx context.Context, ref reference.Named, metaHeaders map[string][]string, authConfig *registry.AuthConfig, outStream io.Writer) error
	CreateImage(ctx context.Context, config []byte, parent string, contentStoreDigest digest.Digest) (builder.Image, error)
	ImageDelete(ctx context.Context, imageRef string, force, prune bool) ([]imagetype.DeleteResponse, error)
	ExportImage(ctx context.Context, names []string, outStream io.Writer) error
	PerformWithBaseFS(ctx context.Context, c *container.Container, fn func(string) error) error
	LoadImage(ctx context.Context, inTar io.ReadCloser, outStream io.Writer, quiet bool) error
	Images(ctx context.Context, opts types.ImageListOptions) ([]*imagetype.Summary, error)
	LogImageEvent(imageID, refName string, action events.Action)
	CountImages(ctx context.Context) int
	ImagesPrune(ctx context.Context, pruneFilters filters.Args) (*types.ImagesPruneReport, error)
	ImportImage(ctx context.Context, ref reference.Named, platform *ocispec.Platform, msg string, layerReader io.Reader, changes []string) (image.ID, error)
	TagImage(ctx context.Context, imageID image.ID, newTag reference.Named) error
	GetImage(ctx context.Context, refOrID string, options imagetype.GetImageOpts) (*image.Image, error)
	ImageHistory(ctx context.Context, name string) ([]*imagetype.HistoryResponseItem, error)
	CommitImage(ctx context.Context, c backend.CommitConfig) (image.ID, error)
	SquashImage(id, parent string) (string, error)

	// Containerd related methods

	PrepareSnapshot(ctx context.Context, id string, parentImage string, platform *ocispec.Platform, setupInit func(string) error) error
	GetImageManifest(ctx context.Context, refOrID string, options imagetype.GetImageOpts) (*ocispec.Descriptor, error)

	// Layers

	GetImageAndReleasableLayer(ctx context.Context, refOrID string, opts backend.GetImageAndLayerOptions) (builder.Image, builder.ROLayer, error)
	CreateLayer(container *container.Container, initFunc layer.MountInit) (layer.RWLayer, error)
	LayerStoreStatus() [][2]string
	GetLayerMountID(cid string) (string, error)
	ReleaseLayer(rwlayer layer.RWLayer) error
	LayerDiskUsage(ctx context.Context) (int64, error)
	GetContainerLayerSize(ctx context.Context, containerID string) (int64, int64, error)
	Mount(ctx context.Context, container *container.Container) error
	Unmount(ctx context.Context, container *container.Container) error
	Changes(ctx context.Context, container *container.Container) ([]archive.Change, error)

	// Windows specific

	GetLayerFolders(img *image.Image, rwLayer layer.RWLayer, containerID string) ([]string, error)

	// Build

	MakeImageCache(ctx context.Context, cacheFrom []string) (builder.ImageCache, error)
	CommitBuildStep(ctx context.Context, c backend.CommitConfig) (image.ID, error)

	// Other

	DistributionServices() images.DistributionServices
	Children(ctx context.Context, id image.ID) ([]image.ID, error)
	Cleanup() error
	StorageDriver() string
	UpdateConfig(maxDownloads, maxUploads int)
}

I think passing the containerID makes sense here. I also did the same for [removing the RWLayer field from container struct](https://github.com/moby/moby/pull/46771/files#diff-f3327983b4be118aa13da9d57d9238c81e2a12332a64af846d8a0f281fa1de66R20) (that's only a draft though).