<commit_before>---
title:  "Docker containers and Macvlan"
date:   2018-01-05 08:00:00 -0400
categories: ansible docker container 
excerpt: "Configuring Macvlan to enable host to container networking"
---

<br>
![Bluebox](/images/docker/network.jpeg)
<br>
<br>

A [previous post]({% post_url 2018-01-02-ansible-docker-service %}) describes how to use Ansible to create and manage docker services under systemd. After a few days it became clear that Ansible, systemd and Docker are working well together, but the network isn't scaling.

# Docker networking
The docker networking strategy that is most often used is a type called 'bridged'. While named bridged, it could be more accuratly called PAT (Port Address Translation). In other words, to access the docker container you access the IP of the host the container a specific port that is mapped back tot he container. This worked well for one of two services, but in my experience scales poorly when running multiple different types of containers that often utilize the same ports. One then needs to remember that their Unifi controller is on port 9443 instead of the default 8443, etc. After about 4 containers this was becoming a struggle and I needed to find a better solution.

# Macvlan
Recent versions Docker and linux kernels support another type of networking called Macvlan. Docker [describes Macvlan](https://docs.docker.com/engine/userguide/networking/get-started-macvlan/ )

> Macvlan is a new twist on the tried and true network virtualization technique. The Linux implementations are extremely lightweight because rather than using the traditional Linux bridge for isolation, they are simply associated to a Linux Ethernet interface or sub-interface to enforce separation between networks and connectivity to the physical network.
> 
> Macvlan offers a number of unique features and plenty of room for further innovations with the various modes. Two high level advantages of these approaches are, the positive performance implications of bypassing the Linux bridge and the simplicity of having less moving parts. Removing the bridge that traditionally resides in between the Docker host NIC and container interface leaves a very simple setup consisting of container interfaces, attached directly to the Docker host interface. This result is easy access for external facing services as there is no port mappings in these scenarios.

This sounds great!  Every container gets its own IP, no more tedious port mappings, no cheat sheet describing which service utilizes which port. 

# What is the catch?
The catch is that the host the containers are running on, by default, cannot talk to any of the containers. The architectrual reasons for this are not really clear to me, the docker documentation says this:
> That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security.

It certainly does provide isolation! An isolation that is not very useful for me. I need my host to talk to the containers for multple reasons. I have some services in containers and others are running on the host and they need to commmunicate. 
 
# How do you fix this failure to communicate?

This is where it got tricky.  Nothing I could find worked for me. The docker Macvlan documentation says:<commit_msg>fixed typo<commit_after>---
title:  "Docker containers and Macvlan"
date:   2018-01-05 08:00:00 -0400
categories: ansible docker container 
excerpt: "Configuring Macvlan to enable host to container networking"
---

<br>
![Bluebox](/images/docker/network.jpeg)
<br>
<br>

A [previous post]({% post_url 2018-01-02-ansible-docker-service %}) describes how to use Ansible to create and manage docker services under systemd. After a few days it became clear that Ansible, systemd and Docker are working well together, but the network isn't scaling.

# Docker networking
The docker networking strategy that is most often used is a type called 'bridged'. While named bridged, it could be more accuratly called PAT (Port Address Translation). In other words, to access the docker container you access the IP of the host the container a specific port that is mapped back tot he container. This worked well for one of two services, but in my experience scales poorly when running multiple different types of containers that often utilize the same ports. One then needs to remember that their Unifi controller is on port 9443 instead of the default 8443, etc. After about 4 containers this was becoming a struggle and I needed to find a better solution.

# Macvlan
Recent versions Docker and linux kernels support another type of networking called Macvlan. Docker [describes Macvlan](https://docs.docker.com/engine/userguide/networking/get-started-macvlan/ )

> Macvlan is a new twist on the tried and true network virtualization technique. The Linux implementations are extremely lightweight because rather than using the traditional Linux bridge for isolation, they are simply associated to a Linux Ethernet interface or sub-interface to enforce separation between networks and connectivity to the physical network.
> 
> Macvlan offers a number of unique features and plenty of room for further innovations with the various modes. Two high level advantages of these approaches are, the positive performance implications of bypassing the Linux bridge and the simplicity of having less moving parts. Removing the bridge that traditionally resides in between the Docker host NIC and container interface leaves a very simple setup consisting of container interfaces, attached directly to the Docker host interface. This result is easy access for external facing services as there is no port mappings in these scenarios.

This sounds great!  Every container gets its own IP, no more tedious port mappings, no cheat sheet describing which service utilizes which port. 

# What is the catch?
The catch is that the host the containers are running on, by default, cannot talk to any of the containers. The architectural reasons for this are not really clear to me, the docker documentation says this:
> That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security.

It certainly does provide isolation! An isolation that is not very useful for me. I need my host to talk to the containers for multple reasons. I have some services in containers and others are running on the host and they need to commmunicate. 
 
# How do you fix this failure to communicate?

This is where it got tricky.  Nothing I could find worked for me. The docker Macvlan documentation says:
{'fraction_non_alphanumeric': 0.03717696841468943, 'fraction_numerical': 0.010578812150521383, 'mean_word_length': 4.715025906735751, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 17, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6326091', 'n_tokens_mistral': 1596, 'n_tokens_neox': 1450, 'n_words': 992}