Setup custom domain for S3-compatible object storage via reverse proxy

Introduction

This tutorial will guide you to setup a custom domain for S3-compatible object storage using reverse proxy. The advantages of a custom domain are to enable seamless integration with existing infrastructure or services under a unified domain.

There are different ways to configure a custom domain, such as using a CNAME record or a reverse proxy. This tutorial focuses on configuring a custom domain using a reverse proxy.

Prerequisites

• A server (e.g. with Hetzner Cloud)
• An S3-compatible bucket (e.g. with Hetzner)
• A domain you want to use (e.g. storage.example.com).

Step 1 - Create Object Storage Bucket

Create an S3-compatible bucket. With Hetzner, see the getting started "Creating a Bucket". Make sure it is set to public access permissions. Not much benefit to using a custom domain for private buckets.

Create S3 credentials to access your bucket. With Hetzner, see the getting started "Generating S3 keys".

Step 2 - Create Server

Create a new server. With Hetzner, see the getting started "Creating a Server". To install Docker and Docker Compose, follow the official Docker documentation.

Step 3 - Deploy Caddy

SSH to your server ssh root@<server-ip>.

Create a directory for your Docker Compose files and folders for the persistent storage of the Caddy container:

sudo mkdir -p /opt/caddy/data

Step 3.1 - Create Docker deployment and configuration files

• Add a Docker compose file

  sudo vim /opt/caddy/compose.yaml

  Add the following content:

  services:
    caddy:
      container_name: caddy
      image: caddy:latest
      restart: unless-stopped
      ports:
        - 80:80
        - 443:443
      volumes:
         - ./data/Caddyfile:/etc/caddy/Caddyfile
         - ./data/certs:/certs
         - ./data/config:/config
         - ./data/data:/data
         - ./data/sites:/srv

• Add a Caddyfile

  sudo vim /opt/caddy/data/Caddyfile

  Add the following content:

  Replace storage.example.com with your own domain.
  Replace fsn1.your-objectstorage.com with the endpoint of your object storage bucket. If the bucket name comes after the endpoint (e.g. https://s3-endpoint.example.org/<bucket_name>) add your endpoint without the bucket name.

  storage.example.com {
  
      tls {
          issuer acme {
              dir https://acme-v02.api.letsencrypt.org/directory
          }
      }
  
      reverse_proxy https://<bucket_name>.fsn1.your-objectstorage.com {
      #reverse_proxy https://s3-endpoint.example.org {
        header_up Host {http.reverse_proxy.upstream.hostport}
        header_up X-Forwarded-Host {host}
      }
  }

Step 3.2 - Start Caddy

cd /opt/caddy
docker compose up -d
docker ps

After the Docker container started, you can access your files via storage.example.com.

If your bucket name comes after the endpoint, note:

The request URL would be https://storage.example.com/<bucket_name>/object.txt.
It is equivalent to https://s3-endpoint.example.org/<bucket_name>/object.txt.

Step 3.3 - Create Kubernetes deployment and configuration files (Optional)

Assuming you already have configured Kubernetes, gateway API.

Replace storage.example.com with your own domain.
Replace fsn1.your-objectstorage.com with the endpoint of your object storage bucket. If the bucket name comes after the endpoint (e.g. https://s3-endpoint.example.org/<bucket_name>) add your endpoint without the bucket name.

apiVersion: v1
kind: Service
metadata:
  name: caddy-storage
  namespace: caddy
spec:
  type: ClusterIP
  selector:
    service: caddy-storage
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: caddy-storage
  namespace: caddy
spec:
  replicas: 1
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      service: caddy-storage
  template:
    metadata:
      labels:
        service: caddy-storage
    spec:
      containers:
        - image: "caddy:latest"
          name: caddy
          ports:
            - name: http
              protocol: TCP
              containerPort: 80
          volumeMounts:
            - name: config-volume
              mountPath: /etc/caddy
      volumes:
        - name: config-volume
          configMap:
            name: caddy-storage-config

---

apiVersion: v1
kind: ConfigMap
metadata:
  name: caddy-storage-config
  namespace: caddy
data:
  Caddyfile: |
    storage.example.com:80 {
      reverse_proxy https://<bucket_name>.fsn1.your-objectstorage.com {
      #reverse_proxy https://s3-endpoint.example.org {
        header_up Host {http.reverse_proxy.upstream.hostport}
        header_up X-Forwarded-Host {host}
      }
    }

---

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: caddy-storage-route
  namespace: caddy
spec:
  parentRefs:
    - name: kubernetes-gateway
      namespace: istio-gateway
  hostnames:
    - "storage.example.com"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: caddy-storage
          port: 80
          weight: 100

Conclusion

You should now be able to access the contents of your S3-compatible object storage via a custom domain.

License: MIT