Docker Swarm has many cool features like load-balancing with an overlay network and puts special emphasis on this one:
- Secure by default: Each node in the swarm enforces TLS mutual authentication and encryption to secure communications between itself and all other nodes.
Well what does that mean specifically?
All swarm service management traffic is encrypted by default, using the AES algorithm in GCM mode. Manager nodes in the swarm rotate the key used to encrypt gossip data every 12 hours.
That leaves the overlay networks un-encrypted.
There is an option for networks --opt encrypted
that enables IPSec for application data but docker does not recommend it:
To encrypt application data as well, add --opt encrypted when creating the overlay network. This enables IPSEC encryption at the level of the vxlan. This encryption imposes a non-negligible performance penalty, so you should test this option before using it in production.
Source: https://docs.docker.com/network/drivers/overlay/#encrypt-traffic-on-an-overlay-network
What does it look like on the network?
Observing a browser like Firefox that has connected to one machine in the swarm who's data gets proxied to the machine actually running the container:
In this example we can see the JSON in the WebSocket connection in plain text.
Encrypting machine to machine traffic with Wireguard
Wireguard has a very low performance impact and can run alongside a production, high performance cluster that is inside one datacenter or distributed over multiple datacenters.
It is important that we:
- do not use overlapping ranges, docker uses anything from
10.0.0.0/8
,172.17.0.0/16
, and172.18.0.0/16
by default - use private ranges inside Wireguard so we notice broken routes
- have direct connections between all nodes in case one goes down
This can be generated with something like ansible so that there is one list of all the public and private keys and then using a template generating the specific configurations for all servers.