Running Your Own AS: Direct Hetzner Peering, a Fourth Edge, and Bringing the Home LAN into the Fabric

Thu 16 April 2026
26 min read
Networking
#freebsd, #bgp, #networking, #ipv6, #frr, #pf, #ibgp, #ixp, #mikrotik, #routeros

Logo

Part 1 set up a single FreeBSD BGP router with two upstream providers. Part 2 added a Vultr edge with native peering and tied both routers together with iBGP. Part 3 joined LocIX Düsseldorf with a dedicated third edge router. This is Part 4.

A few months of operating a multi-PoP BGP network produces a shopping list. I wanted direct peering with networks that move real traffic, a fourth edge in a new PoP, and my own IPv6 space on the home LAN instead of ISP-assigned addressing. This article covers the changes that made that happen.

The headline, if I had to pick one, is two mtr traces. First, from a nettest jail on my home LAN to Hetzner’s network:

root@nettest:~ # mtr -rwz hetzner.com
HOST: nettest                                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. AS201379      2a06:9801:1c:6000::1       0.0%    10    0.2   0.2   0.1   0.2   0.0
  2. AS201379      2a06:9801:1c:fff0::1       0.0%    10    3.2   3.2   2.9   3.4   0.2
  3. AS201379      ixbgp.edge.hofstede.it     0.0%    10    6.9   7.0   6.8   7.3   0.2
  4. AS???         2001:7f8:ca:1:0:2:4940:1   0.0%    10    7.6   7.6   7.3   7.9   0.2
  5. AS24940       core11.nbg1.hetzner.com    0.0%    10   10.9  10.7  10.5  10.9   0.1

Hop 1 is the home LAN gateway - a MikroTik announcing a /64 from my /48 via iBGP. Hop 2 is the iBGP tunnel to the core router. Hop 3 is the new fourth edge (ixbgp) at the iFog datacenter in Zürich. Hop 4 is Hetzner’s peering-LAN address on FogIXP in Frankfurt. That’s a direct BGP session between AS201379 and AS24940. Hop 5 is Hetzner’s backbone. Five hops, 10 ms, and everything in the middle is either mine or Hetzner.

The return path, from a Hetzner machine back to blog.hofstede.it, completes the picture:

[root@krypton ~]# mtr -rwz blog.hofstede.it
HOST: krypton                              Loss%   Snt   Last   Avg  Best  Wrst StDev
  4. AS???    2001:7f8:ca:1:0:20:1379:1     0.0%    10    5.1   5.1   5.1   5.2   0.0
  5. AS201379 ifog-gw.core.hofstede.it      0.0%    10    9.2  14.3   8.9  61.1  16.4
  6. AS201379 radon.server.hofstede.it      0.0%    10    9.9  10.2   9.7  10.9   0.3
  7. AS201379 hofstede.it                   0.0%    10   10.0  10.3   9.9  11.1   0.4

Hop 4 is AS201379’s own peering-LAN address on FogIXP - Hetzner hands traffic for my prefix directly to my router. From there it’s three hops inside AS201379 to the jail serving the blog. No intermediate transit AS, no DE-CIX hop, no shared upstream - just a direct handoff at the exchange.

Note on addresses: AS201379’s prefix 2a06:9801:1c::/48 and all hofstede.it hostnames are shown as-is. Public peering-LAN addresses from PeeringDB (FogIXP’s 2001:7f8:ca:1::/64, LocIX’s 185.1.155.0/24 and 2a0c:b641:701::/64) are also shown as-is. Everything else - provider-assigned addresses, tunnel endpoints, trusted-management IPs - has been replaced with RFC 5737 / RFC 3849 documentation ranges. Upstream and peer AS numbers are visible in public routing tables.

At a Glance

Part 4 adds three things to AS201379:

a fourth FreeBSD edge router at iFog Zürich (ixbgp)
a direct BGP peering session with Hetzner on FogIXP
a MikroTik home router speaking iBGP, bringing the home LAN into the /48

The result is lower-latency paths to Hetzner, more control over outbound policy, and stable provider-independent IPv6 at home.

Architecture: Four Edges and a Home Router

The topology grew in two dimensions. A fourth FreeBSD edge router (ixbgp) joins the existing three at a new PoP. Below hobgp, a new downstream site appears: the home LAN, reachable via an iBGP-speaking MikroTik.

                    ┌──────────────────────────────────────────────────┐
                    │                Default-Free Zone                 │
                    └──┬──────────┬──────────────┬──────────────┬──────┘
                       │          │              │              │
                   AS209533   AS209735       AS212895         AS34927
                   (iFog free) (Lagrange)    (route64)       (iFog paid)
                       │          │              │              │
                    GRE          GRE            GRE            eBGP over vtnet0
                       │          │              │              │      +
                       │          │              │              │  FogIXP RS1/2/3 (AS47498)
                       │          │              │              │      +
                       │          │              │              │  AS24940 (Hetzner, direct)
                  ┌────┴──────────┴──────────────┴────┐   ┌─────┴────────────────────────┐
                  │             hobgp (Core)          │   │       ixbgp (Edge - iFog)    │
                  │       FreeBSD + FRR, AS201379     │◄──┤  FreeBSD + FRR, AS201379     │
                  │       2a06:9801:1c::/48           │iBGP  FogIXP peering LAN          │
                  └─┬───────────┬─────────────┬───────┘   └──────────────────────────────┘
                    │           │             │
               iBGP GIF   iBGP GIF       iBGP 6to4
                    │           │             │
            ┌───────┴───────┐ ┌─┴──────────┐ ┌┴────────────────────────────────┐
            │ vtbgp (Vultr) │ │lobgp(LocIX)│ │ MikroTik (Home LAN)             │
            │ Native AS64515│ │Servperso   │ │ RouterOS 7.20, AS201379         │
            │               │ │+ LocIX RS  │ │ Announces 2a06:9801:1c:6000::/64│
            └───────────────┘ └────────────┘ └─────────────────────────────────┘

Downstream tunnels from hobgp (provider-independent IPv6 for services):
   :1000::/64  radon      (blog, DNS, Gemini, fedi comments)
   :2000::/64  colo       (Hetzner colo OPNsense)
   :3000::/64  mail       (mail.linuxserver.pro)
   :4000::/64  bb         (burningboard.net)
   :5000::/64  road       (road-warrior network)
   :6000::/64  home       (home LAN via MikroTik)

hobgp at Hetzner Nuremberg remains the hub from Part 2 - the core router that every edge speaks iBGP to, and every downstream site tunnels through. ixbgp is the new announcement point at iFog Zürich with a direct link to FogIXP Frankfurt. The home network is a downstream site on the same tunnel pattern as radon, but runs iBGP instead of static routing.

Downstream /64s ride GIF or 6to4 tunnels rather than native transport because most hosting providers don’t let customers announce foreign prefixes on their infrastructure. A tunnel gives each site a direct layer-3 path back to a router that can originate and route the prefix.

The Fourth Edge: ixbgp at iFog and FogIXP

Why a Fourth Edge

The free BGPTunnel service I used in Parts 1-3 is a great on-ramp, but it’s operated as a best-effort community service. Pairing it with a paid iFog BGP port gives me a real transit contract and something the free tunnel does not: a VM with a second NIC directly on FogIXP’s peering LAN. FogIXP is a smaller IXP than LocIX but has a different participant mix - notably, a direct Hetzner presence on that fabric.

The architecture mirrors lobgp from Part 3: two physical interfaces (internet-facing and peering-LAN-facing), an iBGP GIF tunnel back to hobgp, route-maps per peer. The main addition is the direct Hetzner session.

Network Configuration

/etc/rc.conf on ixbgp:

hostname="ixbgp"
kern_securelevel_enable="YES"
kern_securelevel="2"
kld_list="if_gif"

# Internet side (iFog)
ifconfig_vtnet0="inet 198.51.100.176 netmask 255.255.255.0 -lro -tso"
defaultrouter="198.51.100.1"
ifconfig_vtnet0_ipv6="inet6 2001:db8:1030::1229 prefixlen 48 -rxcsum6 -tso6"
ipv6_defaultrouter="2001:db8:1030::1"

# FogIXP peering LAN (direct L2)
ifconfig_vtnet1="up -lro -tso"
ifconfig_vtnet1_ipv6="inet6 2001:7f8:ca:1::20:1379:1 prefixlen 64 -rxcsum6 -tso6"

# iBGP tunnel to core
cloned_interfaces="gif0"
ifconfig_gif0="tunnel 198.51.100.176 198.51.100.10 mtu 1480"
ifconfig_gif0_ipv6="inet6 2a06:9801:1c:fffc::2 2a06:9801:1c:fffc::1 prefixlen 128"
ifconfig_gif0_descr="Uplink-to-hobgp"

ipv6_gateway_enable="YES"
ipv6_static_routes="myblock"
ipv6_route_myblock="2a06:9801:1c::/48 -interface gif0"

pf_enable="YES"
frr_enable="YES"
frr_daemons="mgmtd zebra bgpd bfdd staticd"

The peering-LAN address 2001:7f8:ca:1::20:1379:1 encodes my ASN in the interface ID (20:1379 → AS201379), which is a common convention on IXP LANs. FogIXP assigns these statically; the /64 is the peering-LAN subnet shared by every participant.

ipv6_route_myblock="2a06:9801:1c::/48 -interface gif0" is the same trick as on lobgp: any traffic arriving at ixbgp for the /48 gets forwarded through the iBGP tunnel to hobgp. ixbgp originates the route, but forwarding still happens through hobgp.

FRR Configuration

Five BGP neighbors: three FogIXP route servers, the Hetzner direct peer, the paid iFog transit, and iBGP back to the core. The excerpt below omits the full bogon and hygiene policy for brevity (see Part 1 for the complete PL-BOGONS list); only the per-peer local-preference differences matter here.

frr version 10.5.1
hostname ixbgp
!
ipv6 prefix-list PL-MY-NET seq 5 permit 2a06:9801:1c::/48
!
! [PL-BOGONS: same as Part 1, trimmed for brevity]
!
route-map RM-IXP-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 300
exit
!
route-map RM-HETZNER-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 301
exit
!
route-map RM-IFOG-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 100
exit
!
route-map RM-IXP-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-HETZNER-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-IFOG-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-IBGP-OUT permit 10
exit
!
router bgp 201379
 bgp router-id 198.51.100.176
 no bgp default ipv4-unicast
 no bgp enforce-first-as
 neighbor 2001:db8:1030::1 remote-as 34927
 neighbor 2001:db8:1030::1 description Transit-iFog-Paid
 neighbor 2a06:9801:1c:fffc::1 remote-as 201379
 neighbor 2a06:9801:1c:fffc::1 description Core-Hetzner
 neighbor 2a06:9801:1c:fffc::1 update-source 2a06:9801:1c:fffc::2
 neighbor 2001:7f8:ca:1::111 remote-as 47498
 neighbor 2001:7f8:ca:1::111 description FogIXP-RS1
 neighbor 2001:7f8:ca:1::222 remote-as 47498
 neighbor 2001:7f8:ca:1::222 description FogIXP-RS2
 neighbor 2001:7f8:ca:1::333 remote-as 47498
 neighbor 2001:7f8:ca:1::333 description FogIXP-RS3
 neighbor 2001:7f8:ca:1:0:2:4940:1 remote-as 24940
 neighbor 2001:7f8:ca:1:0:2:4940:1 description Hetzner-Peer
 !
 address-family ipv6 unicast
  neighbor 2001:db8:1030::1 activate
  neighbor 2001:db8:1030::1 soft-reconfiguration inbound
  neighbor 2001:db8:1030::1 route-map RM-IFOG-IN in
  neighbor 2001:db8:1030::1 route-map RM-IFOG-OUT out
  neighbor 2a06:9801:1c:fffc::1 activate
  neighbor 2a06:9801:1c:fffc::1 next-hop-self
  neighbor 2a06:9801:1c:fffc::1 route-map RM-IBGP-OUT out
  neighbor 2001:7f8:ca:1::111 activate
  neighbor 2001:7f8:ca:1::111 soft-reconfiguration inbound
  neighbor 2001:7f8:ca:1::111 route-map RM-IXP-IN in
  neighbor 2001:7f8:ca:1::111 route-map RM-IXP-OUT out
  neighbor 2001:7f8:ca:1::222 activate
  neighbor 2001:7f8:ca:1::222 soft-reconfiguration inbound
  neighbor 2001:7f8:ca:1::222 route-map RM-IXP-IN in
  neighbor 2001:7f8:ca:1::222 route-map RM-IXP-OUT out
  neighbor 2001:7f8:ca:1::333 activate
  neighbor 2001:7f8:ca:1::333 soft-reconfiguration inbound
  neighbor 2001:7f8:ca:1::333 route-map RM-IXP-IN in
  neighbor 2001:7f8:ca:1::333 route-map RM-IXP-OUT out
  neighbor 2001:7f8:ca:1:0:2:4940:1 activate
  neighbor 2001:7f8:ca:1:0:2:4940:1 soft-reconfiguration inbound
  neighbor 2001:7f8:ca:1:0:2:4940:1 route-map RM-HETZNER-IN in
  neighbor 2001:7f8:ca:1:0:2:4940:1 route-map RM-HETZNER-OUT out
 exit-address-family

The local-preference scheme is deliberate:

Hetzner direct (LP 301) - highest. A direct session with a network the size of Hetzner is preferable to transit.
FogIXP route servers (LP 300) - next. Routes learned via the exchange are free to use and generally short.
iFog paid transit (LP 100) - lowest. Transit is the fallback, so it gets the lowest local preference.

The one-point gap is deliberate: if I learn a Hetzner route both directly and via the route servers, the direct session wins. That keeps path selection predictable and avoids relying on route-server propagation behavior.

FogIXP’s route servers are “transparent” - they don’t prepend their own AS onto forwarded routes, so the first AS in the path is the actual origin, not the route server itself. FRR checks for this by default, so route-server sessions usually need no bgp enforce-first-as.

PF: Stateless Transit, Stateful Control Plane

The PF configuration on ixbgp follows the same pattern established in Parts 2 and 3. The tunnel encapsulation must be stateless (otherwise PF’s state table interferes with asymmetric paths), while control-plane TCP sessions must be stateful (TCP needs state to track three-way handshakes). The key bits:

ext_if    = "vtnet0"
ixp_if    = "vtnet1"
hobgp_tun = "gif0"

my_network_v6 = "2a06:9801:1c::/48"
my_router_ip  = "2a06:9801:1c:fffc::2"

# Outer tunnel encapsulation: stateless
pass in  quick on $ext_if proto { 41, ipencap } from 198.51.100.10 to ($ext_if) no state
pass out quick on $ext_if proto { 41, ipencap } from ($ext_if) to 198.51.100.10 no state

# eBGP sessions: stateful
pass in quick on $ext_if proto tcp from 2001:db8:1030::1 to ($ext_if) port 179 keep state
pass in quick on $ixp_if proto tcp from 2001:7f8:ca:1::/64 to ($ixp_if) port 179 keep state

# Transit data plane: stateless (essential for DSR and inner path asymmetry)
pass in  quick inet6 from any to $my_network_v6 no state
pass out quick inet6 from any to $my_network_v6 no state
pass in  quick inet6 from $my_network_v6 to any no state
pass out quick inet6 from $my_network_v6 to any no state

# IXP best practice: don't leak multicast onto the peering LAN
block out quick on $ixp_if to { 224.0.0.0/4, 255.255.255.255, ff02::/16 }
block out quick on $ixp_if proto { igmp, ospf, pim, vrrp, gre }

# Monitoring: allow-list from trusted scrapers only
pass  in quick inet6 proto tcp from { 2a06:9801:1c:2000::21, 2a06:9801:1c:2000::25 } \
    to $my_router_ip port { 9100, 9342 } keep state
block in quick proto tcp from any to any port { 9100, 9342 }

The routing-protocol block rule is IXP-specific: OSPF hellos and the like have no business leaking onto a shared peering LAN, and most IXPs explicitly require participants to block them. The multicast block rule is the same principle - anything that’s not unicast peering traffic gets dropped before it hits the fabric.

Direct Peering with Hetzner

The Path Before

Before the FogIXP session, traffic from my home network to a Hetzner server went:

home MikroTik → hobgp → iFog BGPTunnel → iFog's upstream → DE-CIX fabric
  → AS24940 Hetzner core → target

Four transit hops after leaving my network, round-trip time in the 30-40 ms range despite Frankfurt and Nuremberg being ~180 km apart.

The Path Now

Post-FogIXP:

home MikroTik → hobgp → ixbgp (Zürich) → FogIXP fabric → Hetzner backbone → target

One exchange point, one peering session, one handoff. The mtr at the top of this article shows ~10 ms end-to-end - not because the physical distance changed, but because the routing got shorter.

What Hetzner Sees

Hetzner’s routers now learn 2a06:9801:1c::/48 directly from AS201379, with no transit AS in between. From any Hetzner-hosted machine, traffic to my prefix exits through the Hetzner-side peering session at FogIXP and lands directly on ixbgp, which forwards over iBGP to hobgp.

bgp.tools shows the AS path diversity: the prefix is still visible via iFog free, Lagrange, route64, Vultr, and the LocIX route servers, and now via FogIXP including the Hetzner direct session. Traffic originating inside Hetzner generally prefers the direct path; everyone else picks based on their own policy.

What This Buys in Practice

The practical benefit is not just lower latency to one VPS. Because the /48 is mine and announced by AS201379, every service behind it inherits the direct peering automatically.

Bringing the Home LAN into AS201379

The Gap in Parts 1-3

Parts 1-3 built an AS that serves my public-facing infrastructure. Downstream servers like radon or the colo OPNsense get their own /64 routed natively via GIF. My home network, however, used Deutsche Telekom’s provider-assigned IPv6 - a /56 from DTAG that changes on every reconnect and is useless for inbound connectivity.

The fix is conceptually identical to what radon already does: allocate a /64 from my /48, tunnel it to a router at the endpoint, announce it from there. The difference is that the home router is a MikroTik, not FreeBSD, and MikroTik’s tunnel implementation is 6to4 (interface 6to4) rather than GIF. The iBGP session plugs the home router into the existing core router.

MikroTik Configuration

RouterOS 7.20 has a usable BGP implementation. The tunnel to hobgp is a 6to4 interface (IPv6-in-IPv4, protocol 41), and iBGP runs over it:

/interface 6to4
add local-address=203.0.113.86 mtu=1480 name=tun-hobgp \
    remote-address=198.51.100.10

/ipv6 address
add address=2a06:9801:1c:fff0::2/128 advertise=no interface=tun-hobgp
add address=2a06:9801:1c:6000::1 interface=lan

/ipv6 route
add dst-address=2a06:9801:1c:fff0::1/128 gateway=tun-hobgp
add blackhole dst-address=2a06:9801:1c:6000::/64

/routing bgp instance
add as=201379 name=default router-id=203.0.113.86

/routing bgp template
add as=201379 name=ibgp-hobgp

/routing bgp connection
add afi=ipv6 disabled=no instance=default \
    local.address=2a06:9801:1c:fff0::2 .role=ibgp \
    remote.address=2a06:9801:1c:fff0::1 .as=201379 \
    name=hobgp templates=ibgp-hobgp \
    output.filter-chain=bgp-out .redistribute=connected,static

/routing filter rule
add chain=bgp-out rule="if (dst == 2a06:9801:1c:6000::/64) { accept }"
add chain=bgp-out rule=reject

Three things are worth calling out:

The blackhole route for the /64. Same idea as the -reject route on hobgp for the /48: if the MikroTik gets traffic for an address in the /64 that isn’t assigned, it drops it locally rather than following the default route back out (which would cause a loop). On RouterOS this is expressed as a specific blackhole route.

The output filter. bgp-out only permits the /64 that belongs to this site. Even though redistribute=connected,static would otherwise announce every connected subnet (including LAN RFC 1918 ranges), the filter strips everything that isn’t 2a06:9801:1c:6000::/64. This is the MikroTik equivalent of the PL-MY-NET prefix lists on the FreeBSD edges.

advertise=no on the tunnel address. The tunnel link address 2a06:9801:1c:fff0::2 is a /128 point-to-point - it should not be announced to the LAN as something hosts can SLAAC from.

The Core-Side Configuration

On hobgp, the MikroTik is a downstream iBGP peer with a different routing policy than the other edges. The other three edges receive the full table via RM-IBGP-OUT permit 10 | match PL-MY-NET - they only get our prefix, since that’s the only thing they need to advertise. The home router gets a default route and nothing else:

ipv6 prefix-list PL-MIKROTIK-NET seq 5 permit 2a06:9801:1c:6000::/64
!
route-map RM-DOWNSTREAM-IN permit 10
 match ipv6 address prefix-list PL-MIKROTIK-NET
exit
route-map RM-DOWNSTREAM-IN deny 20
exit
!
route-map RM-DEFAULT-OUT deny 10
exit
!
router bgp 201379
 ...
 neighbor 2a06:9801:1c:fff0::2 remote-as 201379
 neighbor 2a06:9801:1c:fff0::2 description Downstream-MikroTik-Lab
 neighbor 2a06:9801:1c:fff0::2 update-source 2a06:9801:1c:fff0::1
 !
 address-family ipv6 unicast
  neighbor 2a06:9801:1c:fff0::2 activate
  neighbor 2a06:9801:1c:fff0::2 default-originate
  neighbor 2a06:9801:1c:fff0::2 soft-reconfiguration inbound
  neighbor 2a06:9801:1c:fff0::2 route-map RM-DOWNSTREAM-IN in
  neighbor 2a06:9801:1c:fff0::2 route-map RM-DEFAULT-OUT out

default-originate generates a synthetic ::/0 route and sends it to the MikroTik. RM-DEFAULT-OUT deny 10 prevents any other prefix from being advertised. The inbound filter (RM-DOWNSTREAM-IN) only accepts 2a06:9801:1c:6000::/64 - a safety measure against the home router accidentally announcing something it shouldn’t. Combined with the MikroTik’s own outbound filter, this means the worst-case scenario of a misconfigured home lab is a correctly-announced /64 or nothing - never route leakage.

The result is that a PC on the home LAN now SLAAC-configures from 2a06:9801:1c:6000::/64, keeps a stable globally routable address across ISP reconnects, and exits through the AS like any other downstream site. The home LAN is now just another site in AS201379, indistinguishable from radon’s /64 as far as the rest of the internet is concerned.

Traffic Engineering: Steering DTAG via Vultr

The Observation

Deutsche Telekom (AS3320) is the largest German eyeball network. A large share of blog readers reach my services over DTAG’s network, so the return path matters.

With four edge routers, I have four potential ways to reach DTAG. Watching the AS paths that show up on bgp.tools and in show ipv6 bgp on hobgp, the Vultr edge consistently learned shorter paths to DTAG than the others - Vultr’s upstream connectivity into DTAG is short and direct in ways that the transit providers aren’t.

The Implementation

The tool is a BGP AS-path access list combined with a route-map that matches on both the AS-path and the learning peer. On hobgp:

bgp as-path access-list AS-DTAG seq 5 permit _3320$

route-map RM-IBGP-IN permit 5
 match as-path AS-DTAG
 match peer 2a06:9801:1c:fffe::2
 set local-preference 260
exit
route-map RM-IBGP-IN permit 10
exit

Reading this top to bottom:

bgp as-path access-list AS-DTAG matches any AS path that ends in 3320 (the $ anchors to the end of the AS path). That means “routes originated by DTAG” - DTAG is the terminal AS, not a transit.
route-map RM-IBGP-IN permit 5 has two match conditions: the path must match AS-DTAG, and the update must have come from 2a06:9801:1c:fffe::2 - the Vultr edge router. Both must be true.
Matching routes get local-preference 260. Higher local-pref wins in BGP’s selection algorithm, so these routes are preferred over the same prefix learned via iFog, Lagrange, route64, LocIX, or FogIXP direct.
route-map RM-IBGP-IN permit 10 is the catch-all - any route that didn’t match clause 5 falls through and gets accepted with default attributes.

The match peer clause is essential. Without it, the rule would match routes to DTAG learned from any iBGP peer, which isn’t what I want. Vultr happens to have the short path; the other edges don’t. The policy is specifically “prefer Vultr’s version of DTAG routes, if it has one.”

The Effect

show ipv6 bgp X:X:X::/32 for a DTAG-customer prefix shows multiple paths - from iFog, LocIX, Vultr, and so on. Before the route-map, FRR’s tiebreaker (AS path length, then MED, then BGP origin) picked whichever was shortest at that moment. After the route-map, if Vultr has a path for a DTAG destination, Vultr wins - because 260 > 100 (default).

Inbound traffic is out of my control (that’s determined by the announcing side), but outbound traffic from my servers to DTAG customers now consistently exits through Vultr Frankfurt. Latency improvement varies by destination but averages around 5 ms lower than the transit paths, and jitter drops because the path no longer changes based on transient BGP convergence.

This kind of traffic engineering was technically possible before, but not especially useful - with one or two edges, there’s nothing to steer between. With four, “which path goes where” becomes a legitimate question, and AS-path regex gives a precise answer.

Hub Hygiene: One IP for Traceroutes

hobgp has many interfaces. When a packet transits the router, the source address used for any router-originated reply (ICMP time-exceeded from a traceroute, for instance) depends on which interface is closest in the routing table. The result was that hobgp appeared under different addresses in mtr output depending on the flow direction.

The fix is a PF match rule that rewrites router-originated IPv6 traffic to a stable loopback alias from the /48:

ifconfig_lo0_alias0="inet6 2a06:9801:1c::1 prefixlen 64"

match out on { gre0, gre1, gre2, gif0, gif1, gif2, gif3, gif4, gif5, gif6 } \
    inet6 from 2001:db8:1c19:3ee1::1 to any nat-to 2a06:9801:1c::1

match is a non-terminating PF action - it modifies packets without making a pass/block decision. Now traceroutes through hobgp always show 2a06:9801:1c::1 (which resolves to hofstede.it) regardless of which tunnel the forward packet took, and the Hetzner-assigned address stays off public traceroutes.

Downstream Sites: PI Addressing for Friends and Services

A side effect of having a /48 and a working BGP infrastructure is that you can offer stable, provider-independent IPv6 addressing to services you run for friends or that live in separate failure domains. The current allocation:

Prefix	Site	Purpose
`2a06:9801:1c::/64`	loopback on hobgp	router identity
`2a06:9801:1c:1000::/64`	radon	blog, DNS, Gemini, fediverse comments
`2a06:9801:1c:2000::/64`	colo OPNsense	Hetzner colo test network
`2a06:9801:1c:3000::/64`	mail.linuxserver.pro	mail server
`2a06:9801:1c:4000::/64`	burningboard.net	Mastodon instance
`2a06:9801:1c:5000::/64`	road-warrior	laptop on the move
`2a06:9801:1c:6000::/64`	home LAN	residential network
`2a06:9801:1c:fff*::/64`	infra	point-to-point tunnel link addresses

The high :fff* ranges are reserved for infrastructure and point-to-point links: :ffff::/64 for the radon GIF, :fffe::/64 for the Vultr iBGP link, :fffd::/64 for the LocIX edge, :fffc::/64 for the FogIXP edge, :fff0::/64 for the MikroTik. Keeping link addresses out of the customer-facing ranges means the useful space (:1000:: through :6000::) is contiguous and easy to reason about.

Every site runs its own services, has its own firewall, and is operationally independent. What they share is an address prefix that doesn’t change if their hoster does - and that, more than anything else, is why I did all of this.

Lessons Learned

Direct peering is worth the setup effort. A peering session with a large network replaces an entire transit path with a single handoff. The operational cost is one more BGP neighbor and a handful of route-map entries; the benefit is shorter, more predictable paths for a large chunk of real traffic.

RouterOS BGP is good enough for a downstream site. The MikroTik iBGP session has been stable since it was brought up. The configuration is verbose compared to FRR but not surprising - prefix-list equivalents, route-maps, the blackhole for the /64. If the home router is what you already have, BGP on it is a reasonable answer.

Two-condition route-maps are a precision instrument. match as-path plus match peer lets you say exactly what you mean: “prefer this specific path for this specific destination class.” It’s conceptually a small jump from single-condition filters, but it unlocks meaningful traffic engineering the moment you have more than two edges.

A consistent router identity helps more than it should. Rewriting hobgp‘s source address to the /48 loopback is five lines of PF, but having a single hostname appear for the hub in every traceroute - regardless of interface - makes debugging paths significantly easier. It also keeps provider addresses out of public traceroutes.

The “announce at the edge, forward at the core” pattern scales. Each new edge is the same recipe: two interfaces or one plus a tunnel, FRR with per-peer route-maps, PF with stateless transit rules, an iBGP tunnel to hobgp. The core router does the work. Adding a fourth edge was faster than adding the second one, because the pattern was already established.

Interested in peering with AS201379? More details about the network, including upstreams, IXPs, and policy, live at hofstede.it/as201379.html.

If you’d like to establish a peering session - either over a tunnel or on the shared fabric at LocIX Düsseldorf or FogIXP Frankfurt - send a mail to peering@hofstede.it. Hobbyists, small ASNs, and experimental setups are explicitly welcome.

Conclusion

Four months in, AS201379 has started to feel less like a project and more like a very small ISP: a core router, multiple edges in different PoPs, direct peering, traffic engineering, and downstream sites with stable addressing. It is still, by any real metric, tiny. But the difference between “tiny” and “zero” is categorical - Hetzner now learns my prefix directly over a peering session, and a PC on my home LAN now introduces itself to the internet with an address from my own prefix.

Whether there is a Part 5 depends on what breaks, what improves, or what new opportunity shows up next. So far, every version of this network has felt complete right up until the moment it wasn’t.

References

FogIXP - Frankfurt Open Exchange - the IXP used for this expansion
LocIX Düsseldorf - the IXP from Part 3
bgp.tools - looking glass and AS-path visibility
FRR Documentation: BGP - route-maps, AS-path access-lists, match peer
MikroTik RouterOS BGP - filter chains, connection templates
MANRS Routing Security - the filtering and peering-LAN hygiene principles applied on every edge
PeeringDB - where FogIXP and LocIX peering-LAN addresses come from

BGP is one of those protocols where you never really finish. There’s always one more peer to add, one more route-map clause to tune, one more site to bring into the fabric. The satisfying part is that each addition compounds: every new edge improves paths for every existing service, and every new downstream inherits the entire backbone’s routing decisions for free. Four edges and six downstream sites ago, this was one router and one VPS. The architecture was the hard part. Everything since has been variations on the same theme.