Allow overlapping change in bridge's IPv6 network.
**- What I did**

The original ticket reports that the daemon won't start if the `fixed-cidr-v6` address is changed from `fe80::/64` to, for example, `fe80::/80` ... those are link local addresses, which turn out to be a special case. But, the same applies to any IPv6 subnet - if the daemon is started with IPv6 enabled on the default bridge, then the `fixed-cidr-v6` is changed to an overlapping subnet, the daemon fails to start. (For example, `2001:db8::/64` to `2001:db8::/80`.)

This change makes it possible to change the configured address for the default bridge to an overlapping subnet (without manually deleting the `docker0` bridge).

Link-local addresses were a special case because they were not released when a container stopped. So, for the default bridge where IP addresses are based on MAC addresses - and MAC addresses were recycled - with `fixed-cidr-v6=fe80::/64`, once a container was stopped new containers could not be started.  For non-default bridge networks with a link-local subnet, addresses leaked when containers stopped.

Linux always uses the `fe80::/64` network, the 'veth' devices get addresses in that subnet. So, overlapping LL subnets are not allowed by this change, other LL subnets are ok.

**- How I did it**

The code was doing this:
```
setupBridgeIPv6():
   - programIPv6Address(<default link local address>)
   - programIPv6Address(<configured address>)
if !config.InhibitIPv4 then setupVerifyAndReconcile():
  - some IPv4 stuff
  - remove global-unicast IPv6 addresses, apart from the configured on
```

So, it was trying to add the new configured address before deleting the old one. Nowhere in the code has a complete picture of the IPv6 addresses the bridge needs, so it's not able to reconcile existing/required config properly.

This change calculates the IPv6 addresses needed on the bridge, then reconciles them with the addresses on an existing bridge by deleting then adding addresses as required - all in `setupBridgeIPv6()`.

If the configured subnet is link-local, it replaces the default link local address/network.

Link local addresses are returned to their address pool when containers are stopped.

Also...

The only remaining calls of `bridgeInterface.addresses()` want either IPv4 or IPv6, but not both. So, made the address family a param.

Because the IPv6 addresses are now calculated and applied in one go, there's no need for `setupVerifyAndReconcile()` to check the set of IPv6 addresses on the bridge. And, that code was guarded by `!config.InhibitIPv4`, which can't have been right. So, removed its IPv6 parts, and added IPv4 to the function's name.

Fixes #46829

**- How to verify it**

New integration test, and updated unit tests.

Overlapping change to `--fixed-cidr-v6`:
```
# dockerd  --ipv6 --experimental --ip6tables --fixed-cidr-v6=fc00:1234::/64
...
# ip a
[...]
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:b8:f5:44:23 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fc00:1234::1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::1/64 scope link
       valid_lft forever preferred_lft forever

# dockerd  --ipv6 --experimental --ip6tables --fixed-cidr-v6=fc00:1234::/80
...
# ip a
[...]
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:b8:f5:44:23 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fc00:1234::1/80 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::1/64 scope link
       valid_lft forever preferred_lft forever
```

Explicit errors if `--fixed-cidr-v6` is link local and overlaps with (but isn't equal to) `fe80::/64`:

```
# dockerd --ipv6 --experimental --ip6tables --fixed-cidr-v6=fe80::/65 --validate
unable to configure the Docker daemon with file /etc/docker/daemon.json: merged configuration validation from file and command line flags failed: invalid fixed-cidr-v6: clash with the Link-Local prefix 'fe80::/64'

# docker network create --subnet=fe80::/65 testnet
Error response from daemon: clash with the Link-Local prefix 'fe80::/64'
```

**- Description for the changelog**

Allow overlapping change in bridge's IPv6 subnet, support link-local subnets.
daemon/config/config_linux.go
package config // import "github.com/docker/docker/daemon/config"

import (
	"fmt"
	"net"
	"os/exec"
	"path/filepath"

	"github.com/containerd/cgroups/v3"
	"github.com/docker/docker/api/types/container"
	"github.com/docker/docker/api/types/system"
	"github.com/docker/docker/opts"
	"github.com/docker/docker/pkg/homedir"
	"github.com/docker/docker/pkg/rootless"
	units "github.com/docker/go-units"
	"github.com/pkg/errors"
)

const (
	// DefaultIpcMode is default for container's IpcMode, if not set otherwise
	DefaultIpcMode = container.IPCModePrivate

	// DefaultCgroupNamespaceMode is the default mode for containers cgroup namespace when using cgroups v2.
	DefaultCgroupNamespaceMode = container.CgroupnsModePrivate

	// DefaultCgroupV1NamespaceMode is the default mode for containers cgroup namespace when using cgroups v1.
	DefaultCgroupV1NamespaceMode = container.CgroupnsModeHost

	// StockRuntimeName is the reserved name/alias used to represent the
	// OCI runtime being shipped with the docker daemon package.
	StockRuntimeName = "runc"

	// minAPIVersion represents Minimum REST API version supported
	minAPIVersion = "1.12"
)

// BridgeConfig stores all the parameters for both the bridge driver and the default bridge network.
type BridgeConfig struct {
	DefaultBridgeConfig

	EnableIPTables      bool   `json:"iptables,omitempty"`
	EnableIP6Tables     bool   `json:"ip6tables,omitempty"`
	EnableIPForward     bool   `json:"ip-forward,omitempty"`
	EnableIPMasq        bool   `json:"ip-masq,omitempty"`
	EnableUserlandProxy bool   `json:"userland-proxy,omitempty"`
	UserlandProxyPath   string `json:"userland-proxy-path,omitempty"`
}

// DefaultBridgeConfig stores all the parameters for the default bridge network.
type DefaultBridgeConfig struct {
	commonBridgeConfig

	// Fields below here are platform specific.
	EnableIPv6                  bool   `json:"ipv6,omitempty"`
	FixedCIDRv6                 string `json:"fixed-cidr-v6,omitempty"`
	MTU                         int    `json:"mtu,omitempty"`
	DefaultIP                   net.IP `json:"ip,omitempty"`
	IP                          string `json:"bip,omitempty"`
	DefaultGatewayIPv4          net.IP `json:"default-gateway,omitempty"`
	DefaultGatewayIPv6          net.IP `json:"default-gateway-v6,omitempty"`
	InterContainerCommunication bool   `json:"icc,omitempty"`
}

// Config defines the configuration of a docker daemon.
// It includes json tags to deserialize configuration from a file
// using the same names that the flags in the command line uses.
type Config struct {
	CommonConfig

	// Fields below here are platform specific.
	Runtimes             map[string]system.Runtime `json:"runtimes,omitempty"`
	DefaultInitBinary    string                    `json:"default-init,omitempty"`
	CgroupParent         string                    `json:"cgroup-parent,omitempty"`
	EnableSelinuxSupport bool                      `json:"selinux-enabled,omitempty"`
	RemappedRoot         string                    `json:"userns-remap,omitempty"`
	Ulimits              map[string]*units.Ulimit  `json:"default-ulimits,omitempty"`
	CPURealtimePeriod    int64                     `json:"cpu-rt-period,omitempty"`
	CPURealtimeRuntime   int64                     `json:"cpu-rt-runtime,omitempty"`
	OOMScoreAdjust       int                       `json:"oom-score-adjust,omitempty"` // Deprecated: configure the daemon's oom-score-adjust using a process manager instead.
	Init                 bool                      `json:"init,omitempty"`
	InitPath             string                    `json:"init-path,omitempty"`
	SeccompProfile       string                    `json:"seccomp-profile,omitempty"`
	ShmSize              opts.MemBytes             `json:"default-shm-size,omitempty"`
	NoNewPrivileges      bool                      `json:"no-new-privileges,omitempty"`
	IpcMode              string                    `json:"default-ipc-mode,omitempty"`
	CgroupNamespaceMode  string                    `json:"default-cgroupns-mode,omitempty"`
	// ResolvConf is the path to the configuration of the host resolver
	ResolvConf string `json:"resolv-conf,omitempty"`
	Rootless   bool   `json:"rootless,omitempty"`
}

// GetExecRoot returns the user configured Exec-root
func (conf *Config) GetExecRoot() string {
	return conf.ExecRoot
}

// GetInitPath returns the configured docker-init path
func (conf *Config) GetInitPath() string {
	if conf.InitPath != "" {
		return conf.InitPath
	}
	if conf.DefaultInitBinary != "" {
		return conf.DefaultInitBinary
	}
	return DefaultInitBinary
}

// LookupInitPath returns an absolute path to the "docker-init" binary by searching relevant "libexec" directories (per FHS 3.0 & 2.3) followed by PATH
func (conf *Config) LookupInitPath() (string, error) {
	binary := conf.GetInitPath()
	if filepath.IsAbs(binary) {
		return binary, nil
	}

	for _, dir := range []string{
		// FHS 3.0: "/usr/libexec includes internal binaries that are not intended to be executed directly by users or shell scripts. Applications may use a single subdirectory under /usr/libexec."
		// https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html
		"/usr/local/libexec/docker",
		"/usr/libexec/docker",

		// FHS 2.3: "/usr/lib includes object files, libraries, and internal binaries that are not intended to be executed directly by users or shell scripts."
		// https://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA
		"/usr/local/lib/docker",
		"/usr/lib/docker",
	} {
		// exec.LookPath has a fast-path short-circuit for paths that contain "/" (skipping the PATH lookup) that then verifies whether the given path is likely to be an actual executable binary (so we invoke that instead of reimplementing the same checks)
		if file, err := exec.LookPath(filepath.Join(dir, binary)); err == nil {
			return file, nil
		}
	}

	// if we checked all the "libexec" directories and found no matches, fall back to PATH
	return exec.LookPath(binary)
}

// GetResolvConf returns the appropriate resolv.conf
// Check setupResolvConf on how this is selected
func (conf *Config) GetResolvConf() string {
	return conf.ResolvConf
}

// IsSwarmCompatible defines if swarm mode can be enabled in this config
func (conf *Config) IsSwarmCompatible() error {
	if conf.LiveRestoreEnabled {
		return fmt.Errorf("--live-restore daemon configuration is incompatible with swarm mode")
	}
	return nil
}

func verifyDefaultIpcMode(mode string) error {
	const hint = `use "shareable" or "private"`

	dm := container.IpcMode(mode)
	if !dm.Valid() {
		return fmt.Errorf("default IPC mode setting (%v) is invalid; "+hint, dm)
	}
	if dm != "" && !dm.IsPrivate() && !dm.IsShareable() {
		return fmt.Errorf(`IPC mode "%v" is not supported as default value; `+hint, dm)
	}
	return nil
}

func verifyDefaultCgroupNsMode(mode string) error {
	cm := container.CgroupnsMode(mode)
	if !cm.Valid() {
		return fmt.Errorf(`default cgroup namespace mode (%v) is invalid; use "host" or "private"`, cm)
	}

	return nil
}

// ValidatePlatformConfig checks if any platform-specific configuration settings are invalid.
func (conf *Config) ValidatePlatformConfig() error {
	if conf.OOMScoreAdjust != 0 {
		return errors.New(`DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.`)
	}
	if err := verifyDefaultIpcMode(conf.IpcMode); err != nil {
		return err
	}

	return verifyDefaultCgroupNsMode(conf.CgroupNamespaceMode)
}

// IsRootless returns conf.Rootless on Linux but false on Windows
func (conf *Config) IsRootless() bool {
	return conf.Rootless
}

func setPlatformDefaults(cfg *Config) error {
	cfg.Ulimits = make(map[string]*units.Ulimit)
	cfg.ShmSize = opts.MemBytes(DefaultShmSize)
	cfg.SeccompProfile = SeccompProfileDefault
	cfg.IpcMode = string(DefaultIpcMode)
	cfg.Runtimes = make(map[string]system.Runtime)

	if cgroups.Mode() != cgroups.Unified {
		cfg.CgroupNamespaceMode = string(DefaultCgroupV1NamespaceMode)
	} else {
		cfg.CgroupNamespaceMode = string(DefaultCgroupNamespaceMode)
	}

	if rootless.RunningWithRootlessKit() {
		cfg.Rootless = true

		var err error
		// use rootlesskit-docker-proxy for exposing the ports in RootlessKit netns to the initial namespace.
		cfg.BridgeConfig.UserlandProxyPath, err = exec.LookPath(rootless.RootlessKitDockerProxyBinary)
		if err != nil {
			return errors.Wrapf(err, "running with RootlessKit, but %s not installed", rootless.RootlessKitDockerProxyBinary)
		}

		dataHome, err := homedir.GetDataHome()
		if err != nil {
			return err
		}
		runtimeDir, err := homedir.GetRuntimeDir()
		if err != nil {
			return err
		}

		cfg.Root = filepath.Join(dataHome, "docker")
		cfg.ExecRoot = filepath.Join(runtimeDir, "docker")
		cfg.Pidfile = filepath.Join(runtimeDir, "docker.pid")
	} else {
		cfg.Root = "/var/lib/docker"
		cfg.ExecRoot = "/var/run/docker"
		cfg.Pidfile = "/var/run/docker.pid"
	}

	return nil
}

package config // import "github.com/docker/docker/daemon/config"

import (
	"fmt"
	"net"
	"os/exec"
	"path/filepath"

	"github.com/containerd/cgroups/v3"
	"github.com/docker/docker/api/types/container"
	"github.com/docker/docker/api/types/system"
	"github.com/docker/docker/libnetwork/drivers/bridge"
	"github.com/docker/docker/opts"
	"github.com/docker/docker/pkg/homedir"
	"github.com/docker/docker/pkg/rootless"
	units "github.com/docker/go-units"
	"github.com/pkg/errors"
)

const (
	// DefaultIpcMode is default for container's IpcMode, if not set otherwise
	DefaultIpcMode = container.IPCModePrivate

	// DefaultCgroupNamespaceMode is the default mode for containers cgroup namespace when using cgroups v2.
	DefaultCgroupNamespaceMode = container.CgroupnsModePrivate

	// DefaultCgroupV1NamespaceMode is the default mode for containers cgroup namespace when using cgroups v1.
	DefaultCgroupV1NamespaceMode = container.CgroupnsModeHost

	// StockRuntimeName is the reserved name/alias used to represent the
	// OCI runtime being shipped with the docker daemon package.
	StockRuntimeName = "runc"

	// minAPIVersion represents Minimum REST API version supported
	minAPIVersion = "1.12"
)

// BridgeConfig stores all the parameters for both the bridge driver and the default bridge network.
type BridgeConfig struct {
	DefaultBridgeConfig

	EnableIPTables      bool   `json:"iptables,omitempty"`
	EnableIP6Tables     bool   `json:"ip6tables,omitempty"`
	EnableIPForward     bool   `json:"ip-forward,omitempty"`
	EnableIPMasq        bool   `json:"ip-masq,omitempty"`
	EnableUserlandProxy bool   `json:"userland-proxy,omitempty"`
	UserlandProxyPath   string `json:"userland-proxy-path,omitempty"`
}

// DefaultBridgeConfig stores all the parameters for the default bridge network.
type DefaultBridgeConfig struct {
	commonBridgeConfig

	// Fields below here are platform specific.
	EnableIPv6                  bool   `json:"ipv6,omitempty"`
	FixedCIDRv6                 string `json:"fixed-cidr-v6,omitempty"`
	MTU                         int    `json:"mtu,omitempty"`
	DefaultIP                   net.IP `json:"ip,omitempty"`
	IP                          string `json:"bip,omitempty"`
	DefaultGatewayIPv4          net.IP `json:"default-gateway,omitempty"`
	DefaultGatewayIPv6          net.IP `json:"default-gateway-v6,omitempty"`
	InterContainerCommunication bool   `json:"icc,omitempty"`
}

// Config defines the configuration of a docker daemon.
// It includes json tags to deserialize configuration from a file
// using the same names that the flags in the command line uses.
type Config struct {
	CommonConfig

	// Fields below here are platform specific.
	Runtimes             map[string]system.Runtime `json:"runtimes,omitempty"`
	DefaultInitBinary    string                    `json:"default-init,omitempty"`
	CgroupParent         string                    `json:"cgroup-parent,omitempty"`
	EnableSelinuxSupport bool                      `json:"selinux-enabled,omitempty"`
	RemappedRoot         string                    `json:"userns-remap,omitempty"`
	Ulimits              map[string]*units.Ulimit  `json:"default-ulimits,omitempty"`
	CPURealtimePeriod    int64                     `json:"cpu-rt-period,omitempty"`
	CPURealtimeRuntime   int64                     `json:"cpu-rt-runtime,omitempty"`
	OOMScoreAdjust       int                       `json:"oom-score-adjust,omitempty"` // Deprecated: configure the daemon's oom-score-adjust using a process manager instead.
	Init                 bool                      `json:"init,omitempty"`
	InitPath             string                    `json:"init-path,omitempty"`
	SeccompProfile       string                    `json:"seccomp-profile,omitempty"`
	ShmSize              opts.MemBytes             `json:"default-shm-size,omitempty"`
	NoNewPrivileges      bool                      `json:"no-new-privileges,omitempty"`
	IpcMode              string                    `json:"default-ipc-mode,omitempty"`
	CgroupNamespaceMode  string                    `json:"default-cgroupns-mode,omitempty"`
	// ResolvConf is the path to the configuration of the host resolver
	ResolvConf string `json:"resolv-conf,omitempty"`
	Rootless   bool   `json:"rootless,omitempty"`
}

// GetExecRoot returns the user configured Exec-root
func (conf *Config) GetExecRoot() string {
	return conf.ExecRoot
}

// GetInitPath returns the configured docker-init path
func (conf *Config) GetInitPath() string {
	if conf.InitPath != "" {
		return conf.InitPath
	}
	if conf.DefaultInitBinary != "" {
		return conf.DefaultInitBinary
	}
	return DefaultInitBinary
}

// LookupInitPath returns an absolute path to the "docker-init" binary by searching relevant "libexec" directories (per FHS 3.0 & 2.3) followed by PATH
func (conf *Config) LookupInitPath() (string, error) {
	binary := conf.GetInitPath()
	if filepath.IsAbs(binary) {
		return binary, nil
	}

	for _, dir := range []string{
		// FHS 3.0: "/usr/libexec includes internal binaries that are not intended to be executed directly by users or shell scripts. Applications may use a single subdirectory under /usr/libexec."
		// https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html
		"/usr/local/libexec/docker",
		"/usr/libexec/docker",

		// FHS 2.3: "/usr/lib includes object files, libraries, and internal binaries that are not intended to be executed directly by users or shell scripts."
		// https://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA
		"/usr/local/lib/docker",
		"/usr/lib/docker",
	} {
		// exec.LookPath has a fast-path short-circuit for paths that contain "/" (skipping the PATH lookup) that then verifies whether the given path is likely to be an actual executable binary (so we invoke that instead of reimplementing the same checks)
		if file, err := exec.LookPath(filepath.Join(dir, binary)); err == nil {
			return file, nil
		}
	}

	// if we checked all the "libexec" directories and found no matches, fall back to PATH
	return exec.LookPath(binary)
}

// GetResolvConf returns the appropriate resolv.conf
// Check setupResolvConf on how this is selected
func (conf *Config) GetResolvConf() string {
	return conf.ResolvConf
}

// IsSwarmCompatible defines if swarm mode can be enabled in this config
func (conf *Config) IsSwarmCompatible() error {
	if conf.LiveRestoreEnabled {
		return fmt.Errorf("--live-restore daemon configuration is incompatible with swarm mode")
	}
	return nil
}

func verifyDefaultIpcMode(mode string) error {
	const hint = `use "shareable" or "private"`

	dm := container.IpcMode(mode)
	if !dm.Valid() {
		return fmt.Errorf("default IPC mode setting (%v) is invalid; "+hint, dm)
	}
	if dm != "" && !dm.IsPrivate() && !dm.IsShareable() {
		return fmt.Errorf(`IPC mode "%v" is not supported as default value; `+hint, dm)
	}
	return nil
}

func verifyDefaultCgroupNsMode(mode string) error {
	cm := container.CgroupnsMode(mode)
	if !cm.Valid() {
		return fmt.Errorf(`default cgroup namespace mode (%v) is invalid; use "host" or "private"`, cm)
	}

	return nil
}

// ValidatePlatformConfig checks if any platform-specific configuration settings are invalid.
func (conf *Config) ValidatePlatformConfig() error {
	if conf.OOMScoreAdjust != 0 {
		return errors.New(`DEPRECATED: The "oom-score-adjust" config parameter and the dockerd "--oom-score-adjust" options have been removed.`)
	}
	if err := verifyDefaultIpcMode(conf.IpcMode); err != nil {
		return err
	}

	if err := bridge.ValidateFixedCIDRV6(conf.FixedCIDRv6); err != nil {
		return errors.Wrap(err, "invalid fixed-cidr-v6")
	}

	return verifyDefaultCgroupNsMode(conf.CgroupNamespaceMode)
}

// IsRootless returns conf.Rootless on Linux but false on Windows
func (conf *Config) IsRootless() bool {
	return conf.Rootless
}

func setPlatformDefaults(cfg *Config) error {
	cfg.Ulimits = make(map[string]*units.Ulimit)
	cfg.ShmSize = opts.MemBytes(DefaultShmSize)
	cfg.SeccompProfile = SeccompProfileDefault
	cfg.IpcMode = string(DefaultIpcMode)
	cfg.Runtimes = make(map[string]system.Runtime)

	if cgroups.Mode() != cgroups.Unified {
		cfg.CgroupNamespaceMode = string(DefaultCgroupV1NamespaceMode)
	} else {
		cfg.CgroupNamespaceMode = string(DefaultCgroupNamespaceMode)
	}

	if rootless.RunningWithRootlessKit() {
		cfg.Rootless = true

		var err error
		// use rootlesskit-docker-proxy for exposing the ports in RootlessKit netns to the initial namespace.
		cfg.BridgeConfig.UserlandProxyPath, err = exec.LookPath(rootless.RootlessKitDockerProxyBinary)
		if err != nil {
			return errors.Wrapf(err, "running with RootlessKit, but %s not installed", rootless.RootlessKitDockerProxyBinary)
		}

		dataHome, err := homedir.GetDataHome()
		if err != nil {
			return err
		}
		runtimeDir, err := homedir.GetRuntimeDir()
		if err != nil {
			return err
		}

		cfg.Root = filepath.Join(dataHome, "docker")
		cfg.ExecRoot = filepath.Join(runtimeDir, "docker")
		cfg.Pidfile = filepath.Join(runtimeDir, "docker.pid")
	} else {
		cfg.Root = "/var/lib/docker"
		cfg.ExecRoot = "/var/run/docker"
		cfg.Pidfile = "/var/run/docker.pid"
	}

	return nil
}

> Is bridge.ValidateFixedCIDRV6 only applicable to this specific option, or can it be considered a more "generic" validation of IPv6 CIDRs?

It's not generic IPV6 CIDR validation, because it checks that the network doesn't overlap with the default link-local subnet `fe80::/64`.

But wrapping is clearly better than what I did with `IPv6ReservedLinkLocalError` to make it work for both `fixed-cidr-v6` and the `--subnet` given to a non-default bridge network... I'll sort that out, thank you.

> I was also considering if we need to "type" the error with the correct errdefs type

Yes - the `IPv6ReservedLinkLocalError` that's returned if the address is v6 and overlaps with `fe80::/64` has the `InvalidParameter`. I'll make sure the other errors returned by `bridge.ValidateFixedCIDRV6()` do too.