Introduction
This tutorial will help you setup Docker with IPv6 in a secure way.
Objectives
- Automatic provisioning of IPv6
- Each Docker container will receive an IPv6 automatically
- Routed IPv6 traffic between containers in different hosts
- Each host will have a
/80that will be routed through pfSense
- Each host will have a
- Secured public access to our containers via IPv6
- They will be reachable publicly so we want to be able to whitelist open ports in a secure way
- Out-of-band Firewall
- Having an out-of-band firewall, meaning outside of the VM, will increase the security of the system
We will need a way to provision VMs, for this article we selected Proxmox. More information on Proxmox can be found on their website: Proxmox Virtual Environment
Prerequisites
The tutorial was tested with Proxmox VE 9, pfSense CE 2.7.2 and Docker Engine 29.
-
A dedicated server with a hypervisor already installed. See the tutorial Install and Configure Proxmox VE.
-
A main public IPv4 address for the hypervisor (Proxmox)
-
A second public IPv4 address for the firewall/router (pfSense) VM running on Proxmox
» This IP must have its own, separate MAC address -
A public IPv6 subnet
» Must point at MAC of second public IPv4 address
Example terminology
For this article, we will use a /64 IPv6 network because that is what many hosting providers give, and we will use the example 2001:db8:1234:321d::/64. This will be divided in 4096 /76s.
| Example value | Assigned to | Purpose |
|---|---|---|
| <your_main_public_IPv4> | Proxmox server | Public IP (WAN) |
| <your_second_public_IPv4> | VM pfSense (firewall/router) |
Public IP (WAN) |
| 2001:db8:1234:321d:10::1/76 | VM pfSense (firewall/router) |
Public IP (WAN) |
| 2001:db8:1234:321d:20::1/76 | VM pfSense (firewall/router) |
Gateway IP (WAN6) for VM1 |
| 2001:db8:1234:321d:20::2/76 | VM1 Linux (Docker host) |
Public IP (WAN6) |
| 2001:db8:1234:321d:21::/80 | VM1 Docker containers | Subnet to the bridge network |
| 2001:db8:1234:321d:30::1/76 | VM pfSense (firewall/router) |
Gateway IP (WAN6) for VM2 |
| 2001:db8:1234:321d:30::2/76 | VM2 Linux (Docker host) |
Public IP (WAN6) |
| 2001:db8:1234:321d:31::/80 | VM2 Docker containers | Subnet to the bridge network |
| 2001:db8:1234:321d:3e80::1/76 | VM pfSense (firewall/router) |
Gateway IP (WAN6) for VMx |
| 2001:db8:1234:321d:3e80::2/76 | VMx Linux (Docker host) |
Public IP (WAN6) |
| 2001:db8:1234:321d:3e81::/80 | VMx Docker containers | Subnet to the bridge network |
Every VM therefore uses the ::1 address of its own /76 as its default
gateway and keeps the ::2 address for itself. pfSense terminates the other
end of that link: each ::2 address is what pfSense reaches the VM's container
subnet through (Step 2.4).
This setup needs a dedicated server: the whole /64 is routed to a single MAC
address, so it has to land on a router that can split it up. A cloud server
gets a single address instead, so the IPv6 prefix cannot be subdivided in the
same way.
Step 1 - Create Bridges
We will create 3 bridges:
- vmbr0: It will connect to the internet and receive both IPv4 and IPv6 traffic but will only have the main IPv4 assigned
<your_main_public_ipv4>/26 - vmbr1: The internal network shared between VMs,
10.10.10.1/24 - vmbr2: This bridge will hold the IPv6 network for our VMs, in this case a
/64
In Proxmox 8 and later, bridges are managed in the web interface under
Datacenter → Your Node → System → Network. The screenshot below shows the result on the host used
for this tutorial: vmbr0 is the only bridge Proxmox creates by itself, the
other two were added manually.
Replace
<your_main_public_ipv4>with your main public IPv4, and<your_gateway_public_ipv4>with your gateway IP address.
The resulting /etc/network/interfaces looks like this:
auto lo
iface lo inet loopback
iface eth0 inet manual
auto vmbr0
iface vmbr0 inet static
address <your_main_public_ipv4>/26
gateway <your_gateway_public_ipv4>
bridge-ports eth0
bridge-stp off
bridge-fd 0
auto vmbr1
iface vmbr1 inet static
address 10.10.10.1/24
bridge-ports none
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
auto vmbr2
iface vmbr2 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0Two things to watch out for here:
-
Replace
eth0with the name of your own network card. You can find it withip -br link. Do not keep a stanza for a card that does not exist on your host: the bridge will not come up and the host ends up without a network. -
Replace
<your_gateway_public_ipv4>with the IPv4 of your gateway. You can find it withip -c r:root@kvm:~# ip -c r default via <your-gateway-ip> dev eth0 -
vmbr2deliberately has no address. It only exists to carry the IPv6 traffic between the VMs and pfSense.
If you edit the file manually, you can apply the changes with:
systemctl restart networkingStep 2 - Setup Firewall (pfSense)
We will need an out-of-band firewall to be able to whitelist open ports and for this, we are going to use pfSense.
More information on using pfSense with Proxmox can be found in the Netgate documentation.
Quickstart:
-
Download pfSense. Netgate publishes the CE installer images at https://atxfiles.netgate.com/mirror/downloads/. The last version offered there as a direct download is 2.7.2:
cd /var/lib/vz/template/iso/ wget https://atxfiles.netgate.com/mirror/downloads/pfSense-CE-2.7.2-RELEASE-amd64.iso.gz gunzip pfSense-CE-2.7.2-RELEASE-amd64.iso.gzNewer CE releases (2.8 and later) are no longer published as direct downloads. To use one of those, request the free installer image from the Netgate Store and copy the resulting ISO into
/var/lib/vz/template/iso/yourself. Everything below works the same way, only the ISO file name changes.The ISO should now also show in Proxmox.
-
Create the pfSense VM
First, check which storages and network bridge you have:
pvesm status # storage name ip -br link # network bridgeNow create the pfSense VM:
Replace
local-zfs,<mac_of_second_ipv4>,vmbr0,vmbr1,vmbr2, and the ISO name with the actual names on your system.qm create 200 \ --name pfsense \ --memory 1024 \ --cores 2 \ --scsihw virtio-scsi-pci \ --scsi0 local-zfs:8 \ --ide2 local:iso/pfSense-CE-2.7.2-RELEASE-amd64.iso,media=cdrom \ --boot order=ide2\;scsi0 \ --net0 virtio=<mac_of_second_ipv4>,bridge=vmbr0 \ --net1 virtio,bridge=vmbr1 \ --net2 virtio,bridge=vmbr2This will add 3 network cards and configure each one to one of the Bridges we created before.
-
Start the pfSense VM
qm start 200
-
Install pfSense
In the Proxmox web interface, go to "Datacenter » your node » 200 (pfsense) » Console". Follow the steps to install pfSense. When it's done, select reboot.
After reboot, run these commands on the Proxmox server to remove the ISO from the CD-ROM drive so future reboots don't re-enter the installer:
qm stop 200 qm set 200 --ide2 none qm set 200 --boot order=scsi0 qm start 200
-
Configure IPs
Go back to "Datacenter » your node » 200 (pfsense) » Console".
When pfSense is booted, there is a dialog:
Option Should VLANs be configured. Type in nand press enter.WAN interface Use vtnet0LAN interface Leave this empty. LAN interface gets set up later. Confirm the changes with
y, if WAN isvtnet0. Once the menu shows up, select2) Set interface(s) IP address:Option 1 — WAN (vtnet0 - static) 1 Configure IPv4 address WAN interface via DHCP? n Enter the new WAN IPv4 address. <your_second_public_ip>Enter the new WAN IPv4 subnet bit count 26 For a WAN, enter the new WAN IPv4 upstream gateway address. <your_gateway_public_ipv4>Once everything is set, you should see a line like this:
WAN (wan) -> vtnet0 -> v4: <your_second_public_ip>/26
Open Proxmox again and navigate to "Datacenter » your node » 200 (pfsense) » Hardware".
Double-click on
Network Device (net0)and make sure the MAC address is the MAC address of your second public IP. If it isn't, change it now.
Now that pfSense is set up with the public IPv4 address and the respective MAC address, you can access the web interface in a browser:
https://<your_second_public_ip>Step 2.1 - MAC assignment for IPv6
Many hosters require that IPv6 traffic comes from the same MAC address associated with one of the IPv4. In this case it is the extra NAT IPv4.
-
Select the symbol next to your public IPv6 subnet
-
Change the MAC from the main IPv4 to the second IPv4
Select the MAC address of the additional IPv4 (the one that is used as the WAN address of pfSense). Hetzner will then route the whole
/64to that MAC, which means the prefix arrives at pfSense and can be split up there.
Remember to save your changes.
Step 2.2 - Interface assignments
In the pfSense web interface, follow the installation steps.
Once set up, go to "Interfaces » Assignments", and assign the three virtual network cards to the interfaces of pfSense.
Use the following settings for each interface:
-
WAN
Example value Description IPv4 Configuration Type Static IPv4 IPv6 Configuration Type Static IPv6 MAC Address <mac_of_second_ipv4>MAC address of your own second IPv4 IPv4 Address <your_second_public_ipv4>/26Your own second IPv4 IPv4 Upstream gateway <your_gateway_public_ipv4>The IPv4 shown for the default route with ip -c rIPv6 Address 2001:db8:1234:321d:10::1/76IPv6 from your own IPv6 subnet IPv6 Upstream gateway fe80::1Do not replace this value. Use as is. See official docs.
-
LAN
Example value Description IPv4 Configuration Type Static IPv4 IPv6 Configuration Type None IPv4 Address 10.10.10.2/24Private IP address. You don't need to change this value. Use this address.
-
WAN6
Example value Description IPv4 Configuration Type None IPv6 Configuration Type Static IPv6 MAC Address <mac_of_second_ipv4>MAC address of your own second IPv4 IPv6 Address 2001:db8:1234:321d:20::1/76pfSense's own address on the VM1 link IPv6 Upstream gateway 2001:db8:1234:321d:20::2IP of VM1, the other end of the link
Once the LAN interface is added, you might lose connection. You can now access the pfSense web interface via the private IP on the Proxmox server. You can use an SSH tunnel to access it on your local machine:
ssh -L 8443:10.10.10.2:443 root@<your_main_public_ipv4>
# In a local browser
https://localhost:8443After assigning them, the interfaces show the addresses we are going to use. You can find them in the pfSense dashboard:
We have 2 IPv4 assigned, one to access the Host KVM, and the other to use as NAT and for the internal LAN. The NAT IPv4 is an additional address that Hetzner routes to the MAC address of the pfSense WAN interface (see Step 2.1).
The default gateway of the VM hosts is the VIP that pfSense answers on:
Use IPs from your own public IPv6 subnet.
| Gateway (pfSense) | Address of the VM host | |
|---|---|---|
| VM1 | 2001:db8:1234:321d:20::1 |
2001:db8:1234:321d:20::2 |
| VM2 | 2001:db8:1234:321d:30::1 |
2001:db8:1234:321d:30::2 |
| VMx | 2001:db8:1234:321d:3e80::1 |
2001:db8:1234:321d:3e80::2 |
The two columns are the two ends of the same link, which is why the pfSense WAN6
page and the VM network configuration in Step 3
show the pair reversed: pfSense holds ::1 and points at ::2, the VM holds
::2 and points at ::1.
Step 2.3 - Bridge WAN and WAN6
This step is the one that makes the rest work. Go to "Interfaces » Assignments » Bridges", and create a bridge that contains both the WAN and the WAN6 interface.
Both interfaces are attached to vmbr0, which is the bridge that has the
connection to the internet. Because they share the same layer 2 network,
pfSense can answer on the VIPs of every VM without the traffic taking an
asymmetric path, and the IPv6 prefix that Hetzner routes to the server reaches
pfSense through the very same interface.
Step 2.4 - Routing
The default GW for IPv6 is fe80::1 through the WAN interface (not WAN6). This is because this interface is connected to the KVM bridge that has access to the connection to the internet.
Each Docker network in the VM host gets a static route so they can communicate between each other.
For this we need to define a Gateway as 2001:db8:1234:321d:20::2 through the WAN6 interface called "vm1" in the picture.
"System » Routing » Gateways"
Finally we add a static route saying that our container subnet 21::/80 can be reached through the "vm1" gateway, that means through the interface WAN6 via the host at 20::2.
"System » Routing » Static Routes"
Replace
2001:db8:1234:321d:21::and the gateway with your own public IPv6 subnet.
Repeat the static route for every VM that has containers, for example
2001:db8:1234:321d:31::/80 and 2001:db8:1234:321d:3e81::/80 for VM2 and VMx.
Step 2.5 - Firewall Rules
We need to create the following firewall rules:
-
WAN (incoming):
- Allow ICMPv6 types 1 to 4 (destination unreachable, packet too big, time exceeded, parameter problem) to the container networks. Without them Path MTU Discovery silently breaks and connections stall on large transfers.
- Allow the ports you want to publish, for example TCP 443 to
2001:db8:1234:321d:21::/80 - Deny everything else
-
WAN6 (incoming):
- Allow ICMPv6 to the VM uplinks (
2001:db8:1234:321d:20::/76and friends) - Deny everything else
- Allow ICMPv6 to the VM uplinks (
-
Outgoing:
- From WAN6 to ANY (outgoing traffic)
The destination of the rules is the address of the container, not the
address of the Docker bridge. Docker does not translate the destination in
routed mode (see Step 4.2), so a request to 2001:db8:1234:321d:21:242:ac11:2
reaches pfSense with that very address as its destination.
Remember that Docker has a firewall of its own. Rules in pfSense are not enough on their own, the traffic also has to pass the filter of the Docker host. Step 4.2 explains how to check it.
If you want a filter that is really out of band, the Hetzner Robot firewall is applied on the switch port of the server and therefore also covers the traffic of the VMs. It filters IPv4 only by default, the IPv6 filter has to be enabled with the Filter IPv6 packets option.
Step 3 - VM configuration
The virtual network cards of the VM have to be attached to the right bridges.
In this example the VM has one card on vmbr1 (internal) and one on vmbr2
(IPv6).
In this example we use systemd-networkd but the same idea can be used on any network manager. Basic information on systemd-networkd can be found in the Hetzner documentation.
Notice we use ::2 for our VM IPv6 and WAN6 interface IP as a default route.
Replace
2001:db8:1234:321d:20::2/76and2001:db8:1234:321d:20::1with IPs from your own public IPv6 subnet.
[Match]
Name=eth1
[Network]
Address=2001:db8:1234:321d:20::2/76
Gateway=2001:db8:1234:321d:20::1
IPv6AcceptRA=noIPv6AcceptRA=no is needed because the address is static. It also makes sure
that the kernel does not accept a Router Advertisement for the default route
and overwrite the gateway we just configured.
This will give us:
# ip -6 addr show dev eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master docker0 state UP group default qlen 1000
inet6 2001:db8:1234:321d:20::2/76 scope global
valid_lft forever preferred_lft forever
inet6 fe80::e470:c4ff:fe34:1491/64 scope link
valid_lft forever preferred_lft foreverTest we have IPv6 connectivity:
# ping -6 www.google.com
PING www.google.com(ams16s29-in-x04.1e100.net (2a00:1450:400e:804::2004)) 56 data bytes
64 bytes from ams16s29-in-x04.1e100.net (2a00:1450:400e:804::2004): icmp_seq=1 ttl=55 time=32.2 ms
64 bytes from ams16s29-in-x04.1e100.net (2a00:1450:400e:804::2004): icmp_seq=2 ttl=55 time=32.5 ms
64 bytes from ams16s29-in-x04.1e100.net (2a00:1450:400e:804::2004): icmp_seq=3 ttl=55 time=32.4 msStep 4 - Docker configuration
Basic information on configuring a routed Docker environment with IPv6 can be found in the Docker documentation.
We configure Docker to use our /80 subnets under 21::/80.
Replace
2001:db8:1234:321d:21::/80with a subnet within your own public IPv6 subnet.
{
"ipv6": true,
"ip6tables": true,
"allow-direct-routing": true,
"fixed-cidr-v6": "2001:db8:1234:321d:21::/80"
}ipv6enables IPv6 on the default bridge network.ip6tablesmakes Docker manage the IPv6 packet filter. It is enabled by default since Docker 27. Before that, the IPv6 traffic of the containers was not part of the rules of the Docker host at all.allow-direct-routingis required since Docker 28.2. See Step 4.2.fixed-cidr-v6assigns the subnet to the default bridge network. It only applies to the default bridge, user defined networks need--ipv6 --subnet.
Restart the daemon for the changes to take effect:
# systemctl restart dockerTest it by using an alpine container:
# docker run -it alpine ash
/ # ip -6 route
2001:db8:1234:321d:21::/80 dev eth0 metric 256
fe80::/64 dev eth0 metric 256
default via 2001:db8:1234:321d:21::1 dev eth0 metric 1024
ff00::/8 dev eth0 metric 256
/ # ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
43: eth0@if44: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 state UP
inet6 2001:db8:1234:321d:21:242:ac11:2/80 scope global flags 02
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe11:2/64 scope link
valid_lft forever preferred_lft forever
/ # ping -6 www.google.com
PING www.google.com (2a00:1450:400e:807::2004): 56 data bytes
64 bytes from 2a00:1450:400e:807::2004: seq=0 ttl=55 time=26.828 ms
64 bytes from 2a00:1450:400e:807::2004: seq=1 ttl=55 time=26.904 msFrom here we can see we got 2001:db8:1234:321d:21:242:ac11:2/80 and ping to Google works.
Step 4.1 - Publish the ports
The container now has a public IPv6, but that alone does not make it reachable. The port has to be published, and the published port is the one that the firewall rules of Step 2.5 refer to.
# docker run -d --name web -p 443:443 nginxThe container is now reachable on its own address, for example
https://2001:db8:1234:321d:21:242:ac11:2/.
Step 4.2 - Allow direct access to the containers
Since Docker 28.2, the bridge driver drops every packet that is addressed
directly to a container and does not come from the bridge itself. The rule
lives in the raw table and is applied before the routing decision:
# ip6tables -t raw -L PREROUTING -n -v --line-numbersWithout the allow-direct-routing option of Step 4, the packets that pfSense
forwards never reach the container: they are dropped by the Docker host
itself. There are three ways to deal with it:
-
allow-direct-routingin/etc/docker/daemon.jsonallows direct access to the published ports of every container. This is what the configuration above uses. -
A user defined network in routed mode is not filtered at all. This is the recommended setup, because it also gives you a network you can attach containers to by name and control on a per network basis:
# docker network create --ipv6 --subnet 2001:db8:1234:321d:22::/80 \ # -o com.docker.network.bridge.gateway_mode_ipv6=routed docker-ipv6 # docker run -d --network docker-ipv6 -p 443:443 nginxIn routed mode no NAT or masquerading rules are set up, and the published port is reachable directly on the address of the container. Remember to add the matching static route for
2001:db8:1234:321d:22::/80in pfSense. -
com.docker.network.bridge.trusted_host_interfacesallows direct access from specific interfaces of the host only, for exampleeth1.
The default bridge cannot be put into routed mode, so if you use it, the first option is the one you need.
IPv6 forwarding: Docker enables
net.ipv6.conf.all.forwarding=1on the host. A kernel that has forwarding enabled ignores Router Advertisements unlessnet.ipv6.conf.all.accept_ra=2is set. If the address of the host comes from SLAAC, set it explicitly.
Step 4.3 - Static IP
Ideally we would want to have static IPv6 for some of the Docker containers that run some kind of server. This can be achieved by passing the correct arguments at the time of creation.
--ip6 only works on a user defined network that was created with a subnet,
so create the network first:
# docker network create --ipv6 --subnet 2001:db8:1234:321d:22::/80 \
# -o com.docker.network.bridge.gateway_mode_ipv6=routed docker-ipv6And then assign the address to the container:
# docker run -d --network docker-ipv6 --ip6 2001:db8:1234:321d:22::1234 -p 443:443 nginxConclusion
If you have followed this tutorial, then every Docker container that you create will have an IPv6 provisioned automatically that can be accessed from the public internet, and you can control which ports are accessible through an out-of-band firewall (pfSense).
Next steps:
- Install Docker CE on Debian with Dual stack and Firewall Support if you also need IPv4 port forwarding
- Distributing IPv6 over libvirt guests for a different way of handing an IPv6 prefix to a guest