Larvitz Blog

FreeBSD, Linux, all things cleanly engineered

FreeBSD Dual-Stack Jails on Hetzner Cloud


Hetzner Cloud networking has a few quirks that complicate otherwise standard FreeBSD setups:

  • IPv4 arrives as a /32 with a pseudo on-link next-hop at 172.31.1.1.
  • You typically get exactly one routed IPv6 /64 per server.
  • There’s no shared L2 domain; your VM doesn’t ARP/NDP for other tenants.
  • NIC offload features can misbehave with VNET + bridges on FreeBSD.

This guide shows a reproducible dual-stack configuration for a FreeBSD 14 host with VNET jails managed by Bastille. The pattern:

  • IPv4: RFC1918 on the jail bridge; NAT to host’s public IPv4.
  • IPv6: Split the single routed /64 into two /65s: one for the host, one routed to the jails (no ULA, no NAT66).
  • PF for IPv4 NAT and port forwards; pure routing for IPv6.

All addresses below use documentation ranges:

  • Host IPv4: 203.0.113.10/32
  • Hetzner pseudo-gateway (IPv4): 172.31.1.1
  • Assigned IPv6 /64: 2001:db8:abcd:1000::/64
  • Host half (/65): 2001:db8:abcd:1000::/65
  • Jails half (/65): 2001:db8:abcd:1000:8000::/65
  • Bastille bridge IPv4: 10.100.0.1/24
  • Jail example IPv4: 10.100.0.100/24
  • Host IPv6 (example): 2001:db8:abcd:1000::10/65
  • Bridge IPv6 (gateway for jails): 2001:db8:abcd:1000:8000::1/65
  • Jail example IPv6: 2001:db8:abcd:1000:8000::100/65

Why split the /64 into two /65s?

Hetzner Cloud gives you a single routed /64. To put global IPv6 addresses in jails without NAT66/ULA, split that /64: - Keep the lower /65 on the host interface. - Route the upper /65 to your jail bridge and use it for jails. This mirrors the “host + routed downstream” model and keeps IPv6 fully end-to-end.


0) Design Philosophy

This guide follows modern networking principles:

  • IPv6 is IP. IPv4 is the legacy compatibility layer.
  • No NAT66. NAT was an IPv4 workaround for address exhaustion. IPv6 has 340 undecillion addresses - use them.
  • End-to-end connectivity. Jails get real, globally routable IPv6 addresses. Firewalling provides security, not address translation.
  • IPv4 best-effort. NAT is acceptable for IPv4 because we have no choice. It’s temporary infrastructure.

If this offends you, you’re reading the wrong guide. 🙂


1) Host /etc/rc.conf

Configure the /32 IPv4 with the on-link host route to the pseudo gateway, enable forwarding, define the Bastille bridge, and assign both v4 and v6 (using the /65 split).

hostname="freebsd-hcloud"
keymap="us.kbd"

# Forwarding for jails
gateway_enable="YES"
ipv6_gateway_enable="YES"

# Hetzner Cloud NIC: /32 IPv4 and routed IPv6
# Disable LRO/TSO to avoid VNET/bridge issues
ifconfig_vtnet0="inet 203.0.113.10 netmask 255.255.255.255 -tso -lro"
ifconfig_vtnet0_ipv6="inet6 2001:db8:abcd:1000::10/65"

# IPv4 pseudo-gateway and default route
static_routes="hcloud default"
route_hcloud="-host 172.31.1.1 -interface vtnet0"
route_default="default 172.31.1.1"

# IPv6 default router (Hetzner uses link-local on your NIC)
ipv6_defaultrouter="fe80::1%vtnet0"

# Bastille VNET bridge
cloned_interfaces="bridge0"
ifconfig_bridge0_name="bastille0"
ifconfig_bastille0="inet 10.100.0.1/24"
# Routed upper /65 goes to the bridge (acts as gateway for jails)
ifconfig_bastille0_ipv6="inet6 2001:db8:abcd:1000:8000::1/65"

# Services
pf_enable="YES"
pflog_enable="YES"
sshd_enable="YES"
zfs_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"

Notes: - -tso -lro on vtnet0 avoids checksum/segmentation oddities with bridged epairs. - The host’s IPv6 address lives in the lower /65; the bridge gets a gateway address in the upper /65.


2) /etc/sysctl.conf

Ensure forwarding is on and (optionally) disable redirects.

net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1
net.inet.ip.redirect=0
net.inet6.ip6.redirect=0

3) PF: /etc/pf.conf

NAT IPv4 from the jail network to the host’s public IPv4. IPv6 is routed (no NAT). Forward HTTP/HTTPS to a specific jail as an example. Lock down SSH to trusted sources.

# --- Macros ---
ext_if = "vtnet0"

# Jail networks
jail_net_v4 = "10.100.0.0/24"
jail_net_v6 = "2001:db8:abcd:1000:8000::/65"

# Host IPv6 on the external interface (lower /65)
host_ipv6 = "2001:db8:abcd:1000::10"

# Example trusted admin sources
table <trusted_v4> persist { 198.51.100.22, 203.0.113.50 }
table <trusted_v6> persist { 2001:db8:ffff:1::/64 }

# Brute-force tracking
table <bruteforce> persist

# --- Options ---
set skip on lo0
set block-policy drop
set loginterface $ext_if

# --- Scrub ---
scrub in all fragment reassemble
scrub out all random-id max-mss 1500

# --- NAT (IPv4 only) ---
nat on $ext_if inet from $jail_net_v4 to any -> ($ext_if)

# --- RDR for public services to a "web" jail ---
# v4: to host IPv4, forward 80/443 to the jail v4
rdr pass on $ext_if inet  proto tcp to ($ext_if) port {80, 443} -> 10.100.0.100
# v6: to the host's IPv6, forward 80/443 to the jail v6
rdr pass on $ext_if inet6 proto tcp to $host_ipv6 port {80, 443} -> 2001:db8:abcd:1000:8000::100

# --- Policy ---
block in log all
block out log all

# Allow established and all outbound by default
pass out quick all keep state

# Anti-spoof
antispoof quick for { $ext_if, bastille0 }

# SSH allowlist (example nonstandard port 22222)
pass in quick on $ext_if inet  proto tcp from <trusted_v4> to ($ext_if) port 22222 \
    flags S/SA keep state (max-src-conn 5, max-src-conn-rate 3/30, overload <bruteforce> flush global)
pass in quick on $ext_if inet6 proto tcp from <trusted_v6> to ($ext_if) port 22222 \
    flags S/SA keep state (max-src-conn 5, max-src-conn-rate 3/30, overload <bruteforce> flush global)

# Log and drop other SSH attempts
block in log quick on $ext_if proto tcp to ($ext_if) port 22222 label "ssh_blocked"

# Essential ICMPv6 (ND, pMTU, echo)
pass in quick inet6 proto ipv6-icmp icmp6-type { echoreq, echorep, neighbrsol, neighbradv, toobig, timex, paramprob }

# Useful ICMPv4 (echo, unreach for pMTU)
pass in inet proto icmp icmp-type { echoreq, unreach }

# Allow from jails bridge (host-facing side)
pass in quick on bastille0 all keep state

# Public HTTP/HTTPS to forwarded jail
pass in quick on $ext_if proto tcp to ($ext_if) port {80, 443} keep state

4) Bastille/Jail networking

VNET jails get an epair; the host end attaches to bastille0. Inside the jail, rename the epair to vnet0 and assign addresses. The bridge addresses act as the default gateways.

Example jail rc.conf (inside the jail):

ifconfig_e0b_webjail_name="vnet0"
ifconfig_vnet0="inet 10.100.0.100 netmask 255.255.255.0"
ifconfig_vnet0_ipv6="inet6 2001:db8:abcd:1000:8000::100/65"
defaultrouter="10.100.0.1"
ipv6_defaultrouter="2001:db8:abcd:1000:8000::1"

# Optional: quiet base daemons
syslogd_flags="-ss"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

If templating with Bastille, a minimal jail.conf fragment:

vnet="on";
vnet.interface="e0a_${name}";

exec.prestart="ifconfig e0a_${name} create up";
exec.prestart="ifconfig bastille0 addm e0a_${name}";

exec.start="/bin/sh /etc/rc";
exec.stop="/bin/sh /etc/rc.shutdown";

mount.devfs;

Bastille will hand the jail end as e0b_${name}, which you rename to vnet0 in the jail’s rc.conf.


5) Bring-up and verification

  • Reload PF and confirm rules:
  • service pf restart
  • pfctl -sr -v
  • Verify routes on the host:
  • IPv4 default via 172.31.1.1 on vtnet0
  • IPv6 default via fe80::1%vtnet0
  • Local routes to 10.100.0.0/24 and 2001:db8:abcd:1000:8000::/65 on bastille0
  • Start Bastille and your jail:
  • service bastille start
  • bastille start webjail
  • Inside the jail:
  • IPv4: ping -c 3 1.1.1.1
  • IPv6: ping -c 3 2001:4860:4860::8888

From the internet: - HTTP/HTTPS should land on the jail via rdr. - SSH should only accept from your trusted sources.


6) Hetzner Cloud quirks and fixes

  • IPv4 /32 with pseudo next-hop:
  • You must add a host route to 172.31.1.1 on vtnet0 and set it as default.
  • Offload features:
  • -tso -lro on vtnet0 prevents oddities with bridged epairs and IPv6 checksums.
  • IPv6 routing model:
  • IPv6 is routed to your VM. Don’t expect SLAAC for jails. Assign static v6 from your routed half (/65) and route via the host.
  • If IPv6 works on the host but not in jails:
  • Ensure net.inet6.ip6.forwarding=1
  • Make sure jails use 2001:db8:abcd:1000:8000::1 as their default router
  • Confirm PF isn’t blocking NDP or pMTU (ipv6-icmp types are allowed above)
  • If IPv4 NAT seems flaky:
  • Check that no later PF rules override pass out
  • Verify MTU/MSS (scrub max-mss 1500 is set)
  • Inspect states: pfctl -ss | grep 10.100.0.

7) Security notes

  • Restrict SSH with allowlists and non-default ports; consider WireGuard for admin.
  • Enable pflog and inspect with tcpdump:
  • tcpdump -n -e -ttt -r /var/log/pflog
  • Rate-limit public services at your reverse proxy jail if needed.

8) Quick reference: full files

/etc/rc.conf

hostname="freebsd-hcloud"
keymap="us.kbd"

gateway_enable="YES"
ipv6_gateway_enable="YES"

ifconfig_vtnet0="inet 203.0.113.10 netmask 255.255.255.255 -tso -lro"
ifconfig_vtnet0_ipv6="inet6 2001:db8:abcd:1000::10/65"

static_routes="hcloud default"
route_hcloud="-host 172.31.1.1 -interface vtnet0"
route_default="default 172.31.1.1"
ipv6_defaultrouter="fe80::1%vtnet0"

cloned_interfaces="bridge0"
ifconfig_bridge0_name="bastille0"
ifconfig_bastille0="inet 10.100.0.1/24"
ifconfig_bastille0_ipv6="inet6 2001:db8:abcd:1000:8000::1/65"

pf_enable="YES"
pflog_enable="YES"
sshd_enable="YES"
zfs_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"

/etc/sysctl.conf

net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1
net.inet.ip.redirect=0
net.inet6.ip6.redirect=0

/etc/pf.conf

ext_if = "vtnet0"

jail_net_v4 = "10.100.0.0/24"
jail_net_v6 = "2001:db8:abcd:1000:8000::/65"

host_ipv6 = "2001:db8:abcd:1000::10"

table <trusted_v4> persist { 198.51.100.22, 203.0.113.50 }
table <trusted_v6> persist { 2001:db8:ffff:1::/64 }
table <bruteforce> persist

set skip on lo0
set block-policy drop
set loginterface $ext_if

scrub in all fragment reassemble
scrub out all random-id max-mss 1500

nat on $ext_if inet from $jail_net_v4 to any -> ($ext_if)

rdr pass on $ext_if inet  proto tcp to ($ext_if) port {80, 443} -> 10.100.0.100
rdr pass on $ext_if inet6 proto tcp to $host_ipv6 port {80, 443} -> 2001:db8:abcd:1000:8000::100

block in log all
block out log all

pass out quick all keep state

antispoof quick for { $ext_if, bastille0 }

pass in quick on $ext_if inet  proto tcp from <trusted_v4> to ($ext_if) port 22222 \
    flags S/SA keep state (max-src-conn 5, max-src-conn-rate 3/30, overload <bruteforce> flush global)
pass in quick on $ext_if inet6 proto tcp from <trusted_v6> to ($ext_if) port 22222 \
    flags S/SA keep state (max-src-conn 5, max-src-conn-rate 3/30, overload <bruteforce> flush global)

block in log quick on $ext_if proto tcp to ($ext_if) port 22222 label "ssh_blocked"

pass in quick inet6 proto ipv6-icmp icmp6-type { echoreq, echorep, neighbrsol, neighbradv, toobig, timex, paramprob }
pass in inet proto icmp icmp-type { echoreq, unreach }

pass in quick on bastille0 all keep state

pass in quick on $ext_if proto tcp to ($ext_if) port { 80, 443 } keep state

Jail rc.conf (inside the jail)

ifconfig_e0b_webjail_name="vnet0"
ifconfig_vnet0="inet 10.100.0.100 netmask 255.255.255.0"
ifconfig_vnet0_ipv6="inet6 2001:db8:abcd:1000:8000::100/65"
defaultrouter="10.100.0.1"
ipv6_defaultrouter="2001:db8:abcd:1000:8000::1"

syslogd_flags="-ss"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

9) Troubleshooting checklist

  • Host routing:
  • netstat -rn shows default v4 via 172.31.1.1; default v6 via fe80::1%vtnet0
  • Bridge membership:
  • ifconfig bastille0 lists e0a_ interfaces for running jails
  • IPv6 neighbors and pMTU:
  • ndp -an on host shows upstream neighbors; ensure ipv6-icmp is allowed
  • NAT state:
  • pfctl -ss | grep 10.100.0. confirms IPv4 NAT states
  • Throughput/stalls:
  • Verify -tso -lro on vtnet0; scrub out max-mss 1500 is present

Wrap-up

This pattern—IPv4 NATed jails, IPv6 routed using a /64 split into two /65s—fits Hetzner Cloud’s model and avoids NAT66/ULA while keeping end-to-end IPv6. It’s simple, robust, and friendly to PF.

Further Reading