Installing FreeBSD with OpenZFS via the Linux rescue system

Introduction

Hetzner no longer offers a FreeBSD rescue system but it is possible to install and manage FreeBSD with OpenZFS from the Linux rescue system on a dedicated server with UEFI boot. The installation is done on a mirrored OpenZFS pool consisting of two drives.

Prerequisites

• Hetzner dedicated server
  • UEFI boot enabled. To request UEFI boot, you need to file a free text support request for your dedicated server (Select the "Support" tab, and then the "Other" button)
  • Booted in Linux rescue mode with working SSH access (root password or authorized key)
• Network configuration parameters - IP address, gateway, IPV6 address (any from the assigned subnet)

Example terminology

• IPv4: 192.168.0.35
• IPv6: 2a01:4f8:0:0::2
• Drive 1: /dev/nvme0
• Drive 2: /dev/nvme1

To work with SATA drives, you need to replace /dev/nvme0n1 with /dev/sda, and /dev/nvme1n1 with /dev/sdb.

We will install FreeBSD 14.0-RELEASE in our example.

Step 1 - Activate OpenZFS support on the rescue system

Type zfs in the command line and press "y". This will install OpenZFS support.

zfs

View the version to check if it was installed successfully: zfs version

Step 2 - Download the FreeBSD distribution files

Download base.txz and kernel.txz from a FreeBSD mirror:

curl -O http://ftp2.de.freebsd.org/pub/FreeBSD/releases/amd64/14.0-RELEASE/base.txz
curl -O http://ftp2.de.freebsd.org/pub/FreeBSD/releases/amd64/14.0-RELEASE/kernel.txz

You can use ls to check if the files are now available.

Step 3 - Create the partitions

You need to create three partitions on both drives:

Device                Start        End          Sectors              Size Type
/dev/<partition1>     <start1>     <end1>       <total_sectors1>     EFI System
/dev/<partition2>     <start2>     <end2>       <total_sectors2>     FreeBSD swap
/dev/<partition3>     <start3>     <start3>     <total_sectors3>     FreeBSD ZFS

Now check the names of your drives and create the partitions:

1. Check the names of your drives

   lsblk

2. Run cfdisk with the first drive

   cfdisk /dev/nvme0n1

3. Select gpt as partition type
4. Set up the "EFI System" partition
   • Press N (or select New) and type 100M as partition size
   • Press T (or select Type) and select EFI system (very top)
5. Set up the "FreeBSD swap" partition
   • Use the down arrow to navigate to "Free Space"
   • Press N (or select New) and type e.g. 32G as partition size for the swap partition
   • Press T (or select Type) and select FreeBSD swap (below all Linux partitions)
6. Set up the "FreeBSD ZFS" partition
   • Use the down arrow to navigate to "Free Space"
   • Press N (or select New) and choose any desired size for the OpenZFS partition (or use all space)
   • Press T (or select Type) and select FreeBSD ZFS (below all Linux partitions)
7. Press Shift+W (or select Write) to write out the partition table. Type yes to confirm the write.

                                  Disk: /dev/nvme0n1
               Size: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
             Label: gpt, identifier: D3309DB5-355F-C249-9B5E-E7C6FCE73FC8
   
       Device                Start          End     Sectors     Size Type
       /dev/nvme0n1p1         2048       206847      204800     100M EFI System
       /dev/nvme0n1p2       206848     67315711    67108864      32G FreeBSD swap
   >>  /dev/nvme0n1p3     67315712   1000214527   932898816   444.8G FreeBSD ZFS
    ┌────────────────────────────────────────────────────────────────────────────┐
    │Partition UUID: 49473438-E7AA-EF43-ADBA-F5E4F29287BB                        │
    │Partition type: FreeBSD ZFS (516E7CBA-6ECF-11D6-8FF8-00022D09712B)          │
    └────────────────────────────────────────────────────────────────────────────┘
   
        [ Delete ]  [ Resize ]  [  Quit  ]  [  Type  ]  [  Help  ]  [  Write ]
        [  Dump  ]
                        The partition table has been altered.

8. Press Q (or select Quit)
9. Run cfdisk with the second drive and repeat steps 3-8.

   root@rescue ~ # cfdisk /dev/nvme1n1

Step 4 - Setup the dataset

• Create an OpenZFS pool and root dataset

  We are going to create a mirrored OpenZFS pool and on this pool the dataset ROOT/FreeBSD. We configure this dataset as the boot dataset.
  
  

  If you want to install FreeBSD 13 instead of 14, you need to use the flag -o compatibility=openzfs-2.1-freebsd with zpool create.

  zpool create -m none -O atime=off -O compression=on rpool mirror /dev/nvme0n1p3 /dev/nvme1n1p3
  zfs create rpool/ROOT
  zfs create -o mountpoint=legacy rpool/ROOT/FreeBSD
  zpool set bootfs=rpool/ROOT/FreeBSD rpool

• Mount the OpenZFS dataset and extract the FreeBSD distribution

  mount -t zfs rpool/ROOT/FreeBSD /mnt
  bsdtar -x -C /mnt -f base.txz
  bsdtar -x -C /mnt -f kernel.txz

  If bsdtar is not installed, install with apt -y install libarchive-tools.

Step 5 - Format the EFI partitions

Format the EFI partitions and copy the FreeBSD EFI boot loader.

mkfs.fat -s 1 -F 32 /dev/nvme0n1p1
mkfs.fat -s 1 -F 32 /dev/nvme1n1p1
mkdir -p /efi /efi2
mount -t vfat /dev/nvme0n1p1 /efi
mount -t vfat /dev/nvme1n1p1 /efi2
mkdir -p /efi/EFI/BOOT /efi2/EFI/BOOT
cp /mnt/boot/loader.efi /efi/EFI/BOOT/BOOTX64.efi
cp /mnt/boot/loader.efi /efi2/EFI/BOOT/BOOTX64.efi

Step 6 - Configure the FreeBSD installation

Step 6.1 Create /mnt/boot/loader.conf

Create the file /mnt/boot/loader.conf with the following content:

cat << EOF > /mnt/boot/loader.conf
zfs_load="YES"
vfs.root.mountfrom="zfs:rpool/ROOT/FreeBSD"
autoboot_delay="3"
hw.nvme.use_nvd="0"
kern.geom.label.disk_ident.enable="0"
EOF

If you used a different path, replace rpool/ROOT/FreeBSD accordingly.

Step 6.2 Create /etc/rc.d/autodhcp

If you don't know the name of the network interface yet, you need to configure all interfaces to DHCP. If you know the network interface name (e.g. igb0), you can skip this step and "Step 6.3" and directly use the rc.conf code from "Step 7".

Create the file /mnt/etc/rc.d/autodhcp with the following content:

cat << EOF > /mnt/etc/rc.d/autodhcp
#!/bin/sh

# PROVIDE: autodhcp
# BEFORE: NETWORKING netif routing hostname
# REQUIRE: mountcritlocal mdinit
# KEYWORD: FreeBSD

. /etc/rc.subr

name=autodhcp
rcvar=autodhcp_enable

load_rc_config $name

: \${autodhcp_enable:="NO"}

start_cmd="autodhcp_start"
stop_cmd=":"

autodhcp_start()
{
        _dif=\$(/sbin/ifconfig -l | /usr/bin/sed -E 's/lo[0-9]+//g')
        for i in \$_dif; do
                echo "ifconfig_\$i=\"DHCP\"" >> /etc/rc.conf.d/network
        done
}

load_rc_config \$name
run_rc_command "\$1"
EOF

Now make the file executable:

chmod +x /mnt/etc/rc.d/autodhcp

Step 6.3 Edit /mnt/etc/rc.conf

We need to set the hostname, and enable sshd & our newly created autodhcpd script.

Create the file /mnt/etc/rc.conf with the following content:

cat << EOF > /mnt/etc/rc.conf
hostname="myhost.mydomain"
sshd_enable="YES"
autodhcp_enable="YES"
EOF

Step 6.4 - Setup root login via SSH key

• Configure the root SSH key

  As we cannot create the password database from Linux, we have to login with an SSH key.

  mkdir -p /mnt/root/.ssh
  chmod 0400 /mnt/root/.ssh

  Now you have to copy your public key into /mnt/root/.ssh/authorized_keys.

• Enable root login via SSH

  echo "PermitRootLogin without-password" >> /mnt/etc/ssh/sshd_config

Step 6.5 - Reboot the server

When everything is setup, you can reboot the server into the FreeBSD distribution:

reboot

Step 7 - Post-configuration

SSH to the system as root using its IPv4 address and the correct SSH key.

• Delete /etc/rc.conf.d/network that was generated by autodhcp

  rm /etc/rc.conf.d/network

• Reconfigure /etc/rc.conf

  From the netstat output we can see the default gateway and the network interface:

  root@myhost:~ # netstat -rn | grep default
  default            192.168.0.1         UGS        igb0

  In this example, the network interface is igb0 and the default gateway is 192.168.0.1.

  Now you must rewrite /etc/rc.conf. Please use your interface instead of igb0 (don't forget to replace in ipv6_defaultrouter), your public IPv4 address instead of 192.168.0.35, the gateway you discovered instead of 192.168.0.1 and your IPv6 address instead of 2a01:4f8:0:0::2 (you should see the IPv6 address scope in your Hetzner Robot server details).

  hostname="myhost.mydomain"
  sshd_enable="YES"
  ifconfig_igb0="inet 192.168.0.35/32"
  ifconfig_igb0_ipv6="inet6 2a01:4f8:0:0::2"
  gateway_if="igb0"
  gateway_ip4="192.168.0.1"
  static_routes="gateway default"
  route_gateway="-host $gateway_ip4 -interface $gateway_if"
  route_default="default $gateway_ip4"
  ipv6_defaultrouter="fe80::1%igb0"

• Set the root password

  If you want to set the root password, you can do it now with the passwd command:

  passwd

Step 8 - Setup mirrored swap drive (Optional)

You can now setup a gmirror-ed swap drive:

echo geom_mirror_load=\"YES\" >> /boot/loader.conf
gmirror label -F swap0 nda0p2 nda1p2
echo /dev/mirror/swap0 none swap sw 0 0 >> /etc/fstab
swapon -a

Conclusion

Congratulations! You now have a working FreeBSD installation with root on a (mirrored) OpenZFS pool.

Additional note:

If your installation gets broken and doesn't boot anymore, you can use the Linux rescue system to access it:

1. Boot into the Linux rescue system
2. Install OpenZFS (using zfs like in "Step 1" of this tutorial)
3. Import your root pool without mounting its datasets

   zpool import -o cachefile=none -f -N rpool

4. Mount your root dataset

   mount -t zfs rpool/ROOT/FreeBSD /mnt

5. Make any modifications you need and reboot

   reboot

License: MIT