HTTP/3 on FreeBSD: Getting QUIC Working with nginx in a Bastille Jail

What looks like a simple nginx config change turns out to involve SSL library compatibility, firewall rules for a new protocol, and a subtle multi-process routing problem that only surfaces under real traffic. This documents what it actually took to get HTTP/3 (QUIC) working on nginx 1.28 inside a FreeBSD 15.0 Bastille jail - serving the Mastodon instance at burningboard.net.

If you’re running nginx on FreeBSD and want HTTP/3, this should save you several hours of troubleshooting.

Prerequisites

• FreeBSD 12+ (this guide uses FreeBSD 15.0-RELEASE)
• nginx 1.25+ with HTTPV3=on and HTTPV3_BORING=on (built from ports)
• A valid TLS certificate (Let’s Encrypt or similar)
• UDP port 443 open in your firewall
• DNS A and AAAA records pointing to your server

Step 1: The Listen Directives

The most common first mistake: replacing listen 443 ssl with listen 443 quic. QUIC doesn’t replace TCP - it runs alongside it. Browsers always connect via regular HTTPS (TCP) first, then upgrade to QUIC/HTTP3 only after seeing an Alt-Svc header announcing that HTTP/3 is available.

Wrong:

server {
    listen 443 quic;
    listen [::]:443 quic;
    http2 on;
}

This makes nginx listen only on UDP. No browser can connect at all because the initial connection is always TCP.

Correct:

server {
    listen 443 ssl;
    listen 443 quic;
    listen [::]:443 ssl;
    listen [::]:443 quic;
    http2 on;
}

Both TCP (ssl) and UDP (quic) listeners must be present in the same server block.

The reason TCP can never be skipped is that the HTTP/3 upgrade is self-announcing: the server tells the browser about QUIC support inside an HTTP response, which first requires a working TCP connection. Here is the full sequence:

 Client                              nginx (:443)
   │                                     │
   │   ① TCP + TLS handshake             │
   ├────────────────────────────────────>┤  \
   │<────────────────────────────────────┤   TCP (always required for first contact)
   │                                     │  /
   │   ② First request - still over TCP  │
   ├─── GET / ──────────────────────────>┤  \
   │<── 200 OK ──────────────────────────┤   TCP
   │    Alt-Svc: h3=":443"; ma=86400     │  /  ← browser learns HTTP/3 is available
   │                                     │
   │   (browser caches Alt-Svc)          │
   │                                     │
   │   ③ Subsequent requests via QUIC    │
   ├═══ QUIC handshake (UDP) ═══════════>┤  \
   │<════════════════════════════════════┤   UDP (HTTP/3)
   ├═══ GET /page ═══════════════════════>┤   lower latency, multiplexed
   │<═══ 200 OK ══════════════════════════┤  /

This is why both the ssl and quic listen directives are always required together: one to serve the first response that announces HTTP/3, the other to handle all subsequent QUIC connections.

Step 2: The Alt-Svc Header

Browsers discover HTTP/3 support through the Alt-Svc response header. Without it, no browser will ever attempt a QUIC connection:

add_header Alt-Svc 'h3=":443"; ma=86400' always;

The always keyword ensures the header is sent even on error responses and redirects, not just successful 200s.

The add_header Inheritance Trap

This is a subtle but critical nginx behavior: add_header directives inside a location block completely override all add_header directives from the parent server block. If you set Alt-Svc at the server level but have any location block with its own add_header (for Cache-Control, Access-Control-Allow-Origin, etc.), the Alt-Svc header silently disappears for those locations.

The fix: add Alt-Svc to every location block that already has any add_header directive.

location /assets {
    add_header Cache-Control public;
    add_header 'Access-Control-Allow-Origin' '*';
    add_header X-Cache-Status $upstream_cache_status;
    add_header Alt-Svc 'h3=":443"; ma=86400' always;  # Must be here too!
}

Verify coverage with curl:

# Check the root path
curl -sI https://yourdomain.com/ | grep -i alt-svc

# Check a subpath - if this is empty, you have the inheritance problem
curl -sI https://yourdomain.com/some/path | grep -i alt-svc

Step 3: The SSL Library Problem

With a correct config and firewall in place, browsers may still refuse HTTP/3. Capturing UDP traffic on port 443 inside the nginx jail reveals the problem:

tcpdump -i vnet0 udp port 443 -c 20

With stock OpenSSL, the server responds with tiny 51-byte packets to every QUIC connection attempt. These are QUIC version negotiation failures - the handshake is breaking at the HTTP/3 framing layer, not the TLS layer.

The Misleading openssl s_client Test

A common false positive when diagnosing this:

echo | openssl s_client -connect yourdomain.com:443 -quic -alpn h3
# Shows: CONNECTED

This only validates TLS-level QUIC support. The actual HTTP/3 protocol layer on top can still be broken - and with stock OpenSSL on FreeBSD, it is.

Why Stock OpenSSL Doesn’t Work

nginx’s HTTP/3 implementation was originally built against BoringSSL. While OpenSSL 3.2+ added QUIC API support, there are known compatibility issues at the HTTP/3 framing layer. The TLS handshake may succeed, but actual HTTP/3 data transfer fails.

Rebuilding nginx with BoringSSL

The FreeBSD nginx port offers several SSL backends for HTTP/3. Check the available options:

make -C /usr/ports/www/nginx showconfig | grep HTTPV3

In nginx 1.28.2 the relevant options are:

HTTPV3=on: Enable HTTP/3 protocol support
HTTPV3_BORING=on: Use security/boringssl
HTTPV3_LSSL=off: Use security/libressl-devel
HTTPV3_QTLS=off: Use security/openssl33-quictls

Enable HTTPV3 and HTTPV3_BORING, then rebuild:

cd /usr/ports/www/nginx
make config          # Enable HTTPV3 and HTTPV3_BORING
pkg unlock nginx     # If the package is locked
make clean
make deinstall
make && make install
pkg lock nginx       # Re-lock if desired
service nginx restart

Verify the new binary links against BoringSSL:

nginx -V 2>&1 | head -3

nginx version: nginx/1.28.2
built with OpenSSL 1.1.1 (compatible; BoringSSL) (running with BoringSSL)
TLS SNI support enabled

After rebuilding, tcpdump shows proper-sized QUIC handshake packets (1200, 892, 223 bytes) instead of the 51-byte rejections.

Step 4: The Worker Process Affinity Problem

With BoringSSL in place, the first HTTP/3 request succeeds - but subsequent requests on the same connection fail with ERR_QUIC_PROTOCOL_ERROR_QUIC_PUBLIC_RESET in Chrome.

Setting worker_processes 1 in nginx.conf makes HTTP/3 work perfectly. This confirms the diagnosis: QUIC packets from the same connection are being routed to different worker processes. A worker that didn’t establish the connection responds with a “public reset” packet, killing it.

The contrast between the broken and fixed behaviour:

WITHOUT reuseport - one shared UDP socket, OS picks any worker per packet:

  pkt 1 (conn A) ──┐
  pkt 2 (conn A) ──┼──> [ :443 UDP ] ──> OS ──> Worker 1  ✓ (owns conn A)
  pkt 3 (conn A) ──┘                       └──> Worker 2  ✗ (no context)
                                                     │
                                               sends RST → connection killed


WITH reuseport + quic_retry - one socket per worker, OS hashes by connection:

  pkt 1 (conn A) ──┐                    ┌──> Worker 1 socket ──> Worker 1  ✓
  pkt 2 (conn A) ──┼──> SO_REUSEPORT_LB ┤
  pkt 3 (conn A) ──┘  (hashes 4-tuple)  ├──> Worker 2 socket ──> Worker 2
                                         └──> Worker 3 socket ──> Worker 3

  All packets for the same connection always reach the same worker.
  quic_retry ensures the correct worker is pinned from the very first packet.

The Linux Solution (Not Available on FreeBSD)

On Linux, nginx solves this with quic_bpf on, which uses eBPF to route QUIC packets to the correct worker based on the connection ID. FreeBSD doesn’t have eBPF - this option doesn’t exist.

The FreeBSD Solution: reuseport + quic_retry

Two directives work together to solve this on FreeBSD:

reuseport on the QUIC listeners creates per-worker listening sockets. FreeBSD 12+ supports SO_REUSEPORT_LB, which load-balances incoming UDP packets across workers while keeping each connection pinned to one worker socket. Add it to one server block only - typically the primary vhost - to avoid duplicate binding errors:

# Primary server block only:
listen 443 ssl;
listen 443 quic reuseport;
listen [::]:443 ssl;
listen [::]:443 quic reuseport;

# All other server blocks:
listen 443 ssl;
listen 443 quic;
listen [::]:443 ssl;
listen [::]:443 quic;

quic_retry on forces a QUIC retry handshake (address validation) before establishing the connection. This extra round-trip ensures the correct worker process handles the entire QUIC session. Add it to the http block in nginx.conf:

http {
    quic_retry on;
    # ...
}

Verify reuseport is active by checking for multiple UDP sockets:

sockstat -4 -6 -l | grep nginx | grep 443

You should see multiple UDP entries on port 443 - one per worker process.

Step 5: Firewall Configuration

QUIC runs over UDP, so you need explicit UDP rules for port 443 alongside the existing TCP rules. The exact configuration depends on your network topology.

In this setup, nginx runs inside a Bastille jail. IPv6 is directly routed - the jail holds a public IPv6 address, so no RDR or NAT is needed for IPv6 QUIC. For IPv4, the host’s pf forwards traffic from the public IP to the jail’s private address using rdr.

The important detail: QUIC (UDP) needs its own rdr rule on the host, separate from the TCP one. A pass rule alone is not enough - without rdr, UDP packets arriving at the public IP are still addressed to the public IP when the filter rules run, so a pass ... to $frontend_v4 (jail IP) rule will never match them.

# --- RDR ---
# Redirect TCP 80/443 to the nginx jail
rdr on $ext_if inet proto tcp to ($ext_if) port {80,443} -> $frontend_v4

# Redirect UDP 443 (QUIC) to the nginx jail - separate rule required
rdr on $ext_if inet proto udp to ($ext_if) port 443 -> $frontend_v4

# --- Pass rules ---
# HTTP/HTTPS (TCP)
pass in quick on $ext_if inet  proto tcp from any to $frontend_v4 port {80,443} flags S/SA keep state
pass in quick on $ext_if inet6 proto tcp from any to $frontend_v6 port {80,443} flags S/SA keep state

# QUIC (UDP) - IPv4 via NAT/rdr above, IPv6 direct
pass in quick on $ext_if inet  proto udp from any to $frontend_v4 port 443 keep state
pass in quick on $ext_if inet6 proto udp from any to $frontend_v6 port 443 keep state

Don’t forget both IPv4 and IPv6 if your server is dual-stack. For a server without a Bastille jail - nginx running directly on the host - the rdr rules are not needed; only the pass rules for both TCP and UDP.

Step 6: Performance Tuning

TLS 1.3 Early Data (0-RTT)

ssl_early_data on;

Speeds up reconnections by allowing data in the first TLS flight. Early data is vulnerable to replay attacks, so pass the status to your backend:

proxy_set_header Early-Data $ssl_early_data;

Add this to all proxy_pass locations so your application can reject replayed requests on sensitive endpoints (POST, DELETE, etc.).

Session Tickets

ssl_session_tickets on;

Required for QUIC 0-RTT resumption. Unlike the common advice to disable session tickets for TLS 1.2 security, they’re important for QUIC performance.

Complete Example Configuration

nginx.conf

worker_processes auto;
error_log /var/log/nginx/error.log;

events {
    worker_connections 1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     on;
    keepalive_timeout 65;

    # QUIC settings
    quic_retry on;
    ssl_early_data on;

    # SSL global settings
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets on;

    # gzip compression
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript
               text/xml application/xml application/xml+rss text/javascript
               image/svg+xml image/x-icon;

    include /usr/local/etc/nginx/vhosts/*.conf;
}

Primary vhost (with reuseport)

server {
    listen 443 ssl;
    listen 443 quic reuseport;
    listen [::]:443 ssl;
    listen [::]:443 quic reuseport;
    http2 on;

    server_name yourdomain.com;

    ssl_certificate     /path/to/fullchain.pem;
    ssl_certificate_key /path/to/privkey.pem;

    add_header Alt-Svc 'h3=":443"; ma=86400' always;

    location / {
        add_header Alt-Svc 'h3=":443"; ma=86400' always;
        # ... your config ...
    }

    location @proxy {
        add_header Alt-Svc 'h3=":443"; ma=86400' always;
        proxy_set_header Early-Data $ssl_early_data;
        # ... your proxy config ...
    }
}

Additional vhosts (without reuseport)

server {
    listen 443 ssl;
    listen 443 quic;
    listen [::]:443 ssl;
    listen [::]:443 quic;
    http2 on;

    server_name other.yourdomain.com;

    ssl_certificate     /path/to/fullchain.pem;
    ssl_certificate_key /path/to/privkey.pem;

    add_header Alt-Svc 'h3=":443"; ma=86400' always;

    location / {
        add_header Alt-Svc 'h3=":443"; ma=86400' always;
        # ... your config ...
    }
}

Testing and Verification

Server-side

# Verify QUIC TLS handshake (note: only tests TLS layer, not full HTTP/3 framing)
echo | openssl s_client -connect yourdomain.com:443 -quic -alpn h3

# Check Alt-Svc is present on all paths
curl -sI https://yourdomain.com/ | grep -i alt-svc
curl -sI https://yourdomain.com/some/path | grep -i alt-svc

# Monitor live QUIC traffic (inside the nginx jail)
tcpdump -i vnet0 udp port 443 -c 20

# Verify reuseport sockets - expect multiple UDP entries on 443
sockstat -4 -6 -l | grep nginx | grep 443

Client-side

• Online checker: http3check.net - verifies both Alt-Svc and QUIC connectivity
• Browser DevTools: Network tab → right-click column headers → enable “Protocol”. Look for h3 (may require a second page load after Alt-Svc is cached)
• Chrome QUIC internals: chrome://net-export/ captures a netlog, analyzable at netlog-viewer.appspot.com

Debugging Tips

• If Chrome marks QUIC as is_broken: true, clear cached data via Settings → Clear browsing data → Cached images and files
• Use an incognito window for clean tests without cached Alt-Svc state
• Check nginx error logs: grep -i quic /var/log/nginx/error.log
• For deep debugging, temporarily set error_log /var/log/nginx/quic-debug.log debug; - remove it again promptly, it fills up fast

Summary of Pitfalls

Conclusion

Getting HTTP/3 working on FreeBSD with nginx requires more legwork than on Linux, mostly because of the SSL library situation and the absence of quic_bpf. The four things that actually matter:

1. BoringSSL - rebuild from ports with HTTPV3=on and HTTPV3_BORING=on
2. quic_retry on - ensures correct worker routing during session establishment
3. reuseport on QUIC listeners - per-worker UDP sockets for reliable packet distribution
4. Alt-Svc in every location block - the silent add_header inheritance gotcha will bite you

In a jail setup with IPv4 NAT, add a dedicated rdr rule for UDP 443 - a pass rule alone isn’t enough.

Once all of this is in place, HTTP/3 works reliably and delivers noticeably lower latency, especially for connection-heavy workloads like a Mastodon instance.

References

• nginx HTTP/3 module documentation
• BoringSSL
• http3check.net - online HTTP/3 availability checker
• Chrome netlog viewer
• FreeBSD Handbook – Jails