Proxmox on Ampere Altra (Arm)

Introduction

This article explains how to install Proxmox on an Arm64 server and run several VMs behind the router VM. This requires an additional IP with virtual MAC.

Step 1 - Installation

There is a fork of Proxmox that aims to work with all available architectures. At the moment, Armv8 (aarch64) is also available (see GitHub repository). The latest Arm64 version is available via the following mirror: Mirror. The installation works the same as it does with any other normal Proxmox. Using ZFS is also possible without any issues. The latest tested version is 8.3.2.

To get a working environment with this fork, you have to ming several things. You can find more information in the official wiki of the project: Wiki.

Step 2 - Network configuration

Now, you should setup the bridge for internal communication between the VMs. You can do this via the web interface of Proxmox or via the terminal.

• Via the web interface

  You can setup the bridge on the respective host under "Network". Remember to apply your changes.

• Via the terminal

  Add the following lines in the interface file:

  File: /etc/network/interfaces
  [...]
  
  auto vmbr1
  iface vmbr1 inet static
          bridge-ports none
          bridge-stp off
          bridge-fd 0

The vmbr0 is automatically setup during the installation via ISO and you don't have to create it yourself.

Step 3 - Create router VM

During the VM creation, you should note the following:

• Settings

  	Description
  OS	In this tab, select Do not use any media because the RAW image is directly mounted later. CDROM and Cloud-Init Here, you should also just use SCSI. Other options are not supported.
  System	Graphic Card The default is virtio-gpu. That's what you should use. In some cases, the output does not work properly. In this case, you should later use serial console. BIOS It is recommended to create the VM with an EFI BIOS. For this, you have to select OVMF. Machine type You should always leave the type as virt and the version as Latest. This prevents future issues. SCSI controller Only VirtIO SCSI, VirtIO SCSI single and virtio-blk are supported. No SATA or IDE.
  Disks	On the router VM, you can simply remove the disk because it is added later via RAW image. During the installation of the guest, you should only use SCSI for the disk.
  CPU	You can freely select the number of cores. CPU type For the CPU type, you should select Host or Max. The default is Host. If you want to use specific images, you can emulate a different Arm CPU.
  RAM	You can freely allocate RAM as well.
  Network	Add vmbr1 as the first network. This is the network for internal communication between VMs. Netdev virtio is recommended. You should not use the emulation of other network cards.

• Network Interfaces

  The network device for internal communication should be net0 and the device with the virtual MAC of the additional net1. Only then will openWRT configure the first interface directly as LAN and the second as WAN.

  The virtual MAC that you get in Hetzner Robot for the additional IP is added to the second network device (net1). The bridge interface is vmbr0. This is the bridge on which the main IP is also configured, and it later functions as the WAN port.

  This second network interface is now added to the VM. Under "Hardware", simply enter the MAC address and select vmbr0.

  • First network device net0 -> vmbr1 (internal network)
  • Second network device net1 -> vmbr0 (public network with virtual MAC)

• Serial console

  To configure the router VM without the user interface, you should add a serial console to the hardware of the VM. You can do this in "Hardware" as well.

• Router image

  At the time of writing, the options for a router OS that is available on Armv8 is limited. In this case, the router VM is used to make managing DHCP, port forwarding and firewall rules of the VMs in a private network easier. With smaller setups, this is not always necessary because Proxmox also supports NAT. You can find more information in the following tutorial: Install and Configure Proxmox VE #Masquerading (NAT)

  We use openWRT for the router VM. This mirror provides a version for Armv8. Because the VM is running in the UEFI mode, we also need an EFI image. You can find an appropriate image here (openwrt-24.10.0-rc7-armsr-armv8-generic-ext4-combined-efi.img.gz). This is a system image that can be booted directly.

  The following command downloads the above-mentioned image, unzips it, and imports the VM with the ID 100. local-lvm is the storage on which the disk image should be stored. On setups with zfs, this is usually called local-zfs.

  wget https://archive.openwrt.org/releases/24.10.0-rc7/targets/armsr/armv8/openwrt-24.10.0-rc7-armsr-armv8-generic-ext4-combined-efi.img.gz
  
  gunzip openwrt-24.10.0-rc7-armsr-armv8-generic-ext4-combined-efi.img.gz
  
  qm importdisk 100 openwrt-24.10.0-rc7-armsr-armv8-generic-ext4-combined-efi.img local-lvm

  Afterwards, you have to mount the imported disk into the VM and expand the usable size. To add the disk, double-click on it and select "Add". To adjust the size, select the disk and click on resize under "Disk Action" at the top. There, you can enter the storage that should be added. You might need to adjust the boot order in "Options".

  

  Now you can start the VM for the first time.

Step 4 - First login

By default, the WAN interface does not provide a web GUI. This is good because the image doesn't use a password. First, set a password. You can use one of the options below:

• Via the VM console

  Run passwd in the VM console. Warning: Take the keyboard layout into account

• Via the serial console

  Connect to the host via SSH and start the serial console by running:

  qm terminal <vm id>

  The serial console doesn't show any output at the beginning. Press "Enter" for a prompt. Now you can run passwd.

After you set a password, you can activate the web GUI on the WAN port.

Connecting via WAN (Public IP)

If necessary, it is also possible to access the GUI directly via the WAN IP of the VM. For this, you have to allow it in the firewall. You have to run the commands below on the VM. You can run them either in the console within the Proxmox Web GUI, or in the serial console in the Proxmox CLI. To use the serial console, you have to add a serial device to the VM.

Proxmox CLI

With this command, you can connect to the VM console:

qm terminal <vm id>

uci add firewall rule
uci set firewall.@rule[-1].name='Allow-Admin'
uci set firewall.@rule[-1].enabled='true'
uci set firewall.@rule[-1].src='wan'
uci set firewall.@rule[-1].proto='tcp'
uci set firewall.@rule[-1].dest_port='80 443'
uci set firewall.@rule[-1].target='ACCEPT'
uci commit firewall
service firewall restart

Step 5 - First guest VM

As soon as the openWRT VM runs and the web GUI is available, you ca create the first guest VM. You can do this in a similar way as you would with a router VM. You can adjust the settings as explained above. In general, there are quite a few images for Arm available. Here, a few examples:

• Debian
• Ubuntu
• Fedora
• Arch Server

vmbr1 is now always used as network interface. The router assigns VM IPs and routes traffic to the Internet via that bridge.

Conclusion

For more information about the configuration of Proxmox within the network of Hetzner, see the community article Install and Configure Proxmox VE

License: MIT