How to route cloud server over private network using pfSense and Hetzner Cloud Networks

Introduction

This tutorial explains how to route cloud servers over a private network using pfSense and "Hetzner Cloud Networks".

hcloud-CLI will be used in this tutorial. It can also be done with the Cloud Console.

Why should a server route the traffic over a private network?
The advantage to route the traffic over private networking is that the client servers can be protected by a firewall and are only accessible through the internal IP address. If the public interface of the client servers is disabled, the only access is via the router. It uses NAT (...) to hide the internal network from the public Internet.

Prerequisites

Following things are required:

2 cloud servers (one as pfSense, the other one as client)
A private network range
hcloud-CLI
Cloud Console

Example terminology

pfSense server: router / 10.0.0.2
client server: private-client
Hetzner Network: nat-network / 10.0.0.0/16
Hetzner subnet: 10.0.0.0/24
Hetzner gateway: 10.0.0.1

Please replace those example names and IPs with your own names and IPs in all example commands.

Step 1 - Set up the environment

Before anything can be done, the router and the client have to be created. For this purpose, a CX11 as router is enough.

Create the servers

To create the servers, you have to execute the following commands (it does not matter which image you use on the router):
```
hcloud server create --name router --type cx11 --image ubuntu-22.04
hcloud server create --name private-client --type cx11 --image ubuntu-22.04
```
When this is done, you can create the private network.

Create the private network

hcloud network create --name nat-network --ip-range 10.0.0.0/16

The network needs a subnet. This tutorial will use the IP range 10.0.0.0/24 in zone eu-central, which includes hel1, fsn1 and nbg1.

hcloud network add-subnet nat-network --network-zone eu-central --type server --ip-range 10.0.0.0/24

Attach the servers to the network

You have to attach the servers to the network. You should specify the IP address of the router because the traffic will be routed over this server. You cannot assign 10.0.0.1 because this IP is already used by the Hetzner gateway.
```
hcloud server attach-to-network router --network nat-network --ip 10.0.0.2
hcloud server attach-to-network private-client --network nat-network
```
Add a route to the network

Traffic between servers within the same Hetzner Cloud Network is always routed via the Hetzner gateway. For more information on Hetzner Network gateways, see this FAQ entry.

This means that traffic is always routed like this: server-1➔gateway➔server-2

To achieve this, you need two routes:
- A route that tells the private server (client) to forward traffic to the gateway (10.0.0.1)
- A route that tells the gateway (10.0.0.1) to forward traffic to the pfSense server (10.0.0.2)
In order to route public traffic from the server to the gateway, you have to add a route on the server itself. This is explained in Step 3 - Set up the client server.

In order to route public traffic from the gateway to the pfSense server, you have to add a route to your Cloud Network. You can add the neccessary route with this command:
```
hcloud network add-route nat-network --destination 0.0.0.0/0 --gateway 10.0.0.2
```
This route will tell the Hetzner gateway to forward the whole traffic 0.0.0.0/0 (every IP in every subnet) to your pfSense server 10.0.0.2.

Click here if you are using a dedicated server and vSwitch

If you are using a dedicated server and vSwitch, make sure you configured the dedicated server correctly (see "Configure networking on your dedicated root servers"). You can "Expose routes to vSwitch". As long as the routes are exposed to the vSwitch, they also become reachable for all dedicated servers that are assigned to the vSwitch.

Step 2 - Set up the pfSense server

Mount the pfSense image and restart the server

To install pfSense on the router server, mount the image first.
```
hcloud server attach-iso router pfSense-CE-2.7.2-RELEASE-amd64.iso
```
The ISO name depends on the version. It's possible that the ISO may be a newer version.

When the ISO is mounted, a reboot/reset of the server is needed. To perform the installation, you need the VNC console.
```
hcloud server reset router
```
Perform the installation

You can open the VNC console in the Cloud Console.

Perform the installation with the guide.

When the installation is done, you can detach the ISO.
```
hcloud server detach-iso router
```
You can now reboot the server.
After reboot, configure pfSense using the VNC console in the Cloud Console

When pfSense is booted, there is a dialog if VLANs should be configured. Type in n and press enter.

For the WAN interface use vtnet0. Leave LAN interface empty. LAN interface gets set up later.

Confirm the changes with y, if WAN is vtnet0.

The router is now accessible in browser. Log in with the following user credentials:

User name: admin
Password: pfsense

Follow the installation guide. In step 4, un-check the option Block bogon networks.

Set up the interface and routing:

Go to Interfaces ➔ Assignments

Add the interface vtnet1.

Save the change.
Then, go to Interfaces ➔ LAN

Check Enable interface and set the IPv4 Configuration Type to DHCP.

Save and apply the changes!

Go to System ➔ Routing ➔ Static Routes

Create a new route:

Destination network: 10.0.0.0/16
Gateway: 10.0.0.1
Description: Make private network reachable for pfSense

Save and apply the changes.

Go to System ➔ Advanced ➔ Networking

Under Network Interfaces, enable Disable hardware checksum offload and click Save.

Set up a firewall:

Go to Firewall ➔ NAT ➔ Outbound

Set the Outbound NAT Mode to Hybrid Outbound NAT rule generation.

Create the following mapping rule:

Edit Advanced Outbound NAT Entry
  Interface: WAN
  Address Family: IPv4+IPv6
  Protocol: any
  Source: Network or Alias » 10.0.0.0/16
  Destination: any
Translation
  Address: WAN Address

Save and do not apply the changes!

Go to Firewall ➔ Rules ➔ LAN

Edit the rule with description Default allow LAN to any rule.
- Change the source from LAN subnets to any.
Save and apply the changes.

Go to Firewall ➔ Rules ➔ WAN and create a new rule.

Action: Block
Interface: WAN
Address Family: IPv4+IPv6
Protocol: TCP/UDP
Source: any
Destination: This firewall (self)
Destination Port Range: From "HTTP (80)" to "HTTPS (443)"

Save and apply the changes.

Now go back to Firewall ➔ NAT ➔ Outbound and also apply the changes.

It may happen that the browser is caching the session. In this case, close the browser and re-open it. The pfSense UI should no longer be reachable via the public IP address.

Step 3 - Set up the client server

On the client, check its private networking interfaces. You can use the following command:

ip l

You should get an output like this:

root@private-client:~# ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
3: ens10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether ff:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff

The private network interface has the mtu 1450. In this case, it is ens10.

If your servers also has a public network, you can optionally disable this public interface.

Optional step

To disable the public interface, you can choose from those two options:

Configure route for private networking

Edit the file /etc/network/interfaces:

Add a default route to the network gateway

Private networking by Hetzner Cloud works on Layer 3 so you need to add a default route to the network gateway. The network gateway will then forward the request to your pfSense server (client➔gateway➔pfsense) because of the network rule you added in step 1.

Add the following configuration to /etc/network/interfaces:
```
auto ens10
iface ens10 inet dhcp
	post-up ip route add default via 10.0.0.1
```
Replace 10.0.0.1 with the IP of your own Hetzner gateway. The gateway's IP address is always the first IP address of the network's IP range.
Add the DNS servers

Add the DNS servers by adding the line dns-nameservers 185.12.64.2 185.12.64.1 to /etc/network/interfaces under the post-up line.

If DNS is still not working, systemd-resolved ignores this parameter.

In this case, you need to configure DNS in the file /etc/systemd/resolved.conf. There should be a line like #DNS under the line [Resolve]. Un-comment the DNS line by removing the # and type in some DNS servers or use the DNS servers by Hetzner:
```
DNS=185.12.64.2 185.12.64.1
```

Save the file and restart the server.

If the public interface is disabled, you can no longer access the server via it's external IP.

You can login either via SSH over the router server or via the VNC console.

Try to ping a domain (e.g. fsn.icmp.hetzner.com). If this works, use mtr to check the tracepath and see if the traffic is routed via the pfSense server.

The MTR should look like this (mtr to fsn.icmp.hetzner.com on private-client):

HOST: private-client              Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- _gateway                   0.0%     1    4.0   4.0   4.0   4.0   0.0
  2.|-- 10.0.0.2                   0.0%     1    1.0   1.0   1.0   1.0   0.0
  3.|-- 172.31.1.1                 0.0%     1    1.3   1.3   1.3   1.3   0.0
  4.|-- x.your-cloud.host          0.0%     1    0.2   0.2   0.2   0.2   0.0
  5.|-- static.1.97.69.159.client  0.0%     1   18.9  18.9  18.9  18.9   0.0
  6.|-- static.213-239-231-65.cli  0.0%     1    0.9   0.9   0.9   0.9   0.0
  7.|-- core21.fsn1.hetzner.com    0.0%     1    0.6   0.6   0.6   0.6   0.0
  8.|-- ex9k2.dc1.fsn1.hetzner.co  0.0%     1    0.4   0.4   0.4   0.4   0.0
  9.|-- fsn.icmp.hetzner.com       0.0%     1    0.5   0.5   0.5   0.5   0.0

Conclusion

This article shows the steps necessary for setting up a pfSense server to route the traffic from other servers over pfSense.

How to route cloud server over private network using pfSense and Hetzner Cloud Networks

Author

Published

Time to read

Table of Contents