FreeBSD Foundationals: Jails - From Chroot on Steroids to Full Virtual Networks

FreeBSD Jails have been around since FreeBSD 4.0, released in the year 2000. That makes them older than Linux cgroups, older than LXC, older than Docker, and older than most people’s understanding of what “containers” even means. Yet they remain one of the most elegant and underappreciated isolation mechanisms available on any operating system.

This article is the first in a series called FreeBSD Foundationals - covering core FreeBSD concepts that deserve more than a man page skim. We start with Jails because they’re central to how FreeBSD is deployed in practice: from hosting providers to Netflix’s CDN to small mail servers on rented hardware.

If you’ve used Docker or LXC on Linux, some concepts will feel familiar. But Jails are not Linux containers with a different name. The design philosophy, the networking model, and the security boundaries are fundamentally different. Understanding those differences is the point.

What Problem Do Jails Solve?

Unix has always had chroot, which changes the apparent root directory for a process. A process inside a chroot sees / as whatever directory you pointed it at. It can’t traverse above that point in the filesystem. Useful, but limited: chroot only isolates the filesystem view. A chrooted process still shares the network stack, the process table, IPC, and - crucially - the ability to escape the chroot if it has root privileges.

Jails extend this concept into a proper isolation boundary. A jailed process gets:

• Its own filesystem root (like chroot, but enforced at the kernel level)
• Its own process space - processes inside a jail can only see other processes in the same jail
• Its own hostname and network identity
• Restricted system calls - no loading kernel modules, no mounting filesystems (unless explicitly allowed), no modifying the host’s network configuration

The key insight: Jails are a kernel-enforced boundary, not a userspace trick. A root user inside a jail is not equivalent to root on the host. The kernel itself refuses operations that would break the isolation. This is a fundamentally different security model from “root in a Docker container” on Linux, where escaping to the host has historically been a much shorter path.

Classic Jails vs VNET Jails

There are two distinct approaches to jail networking, and understanding the difference matters for every design decision that follows.

Classic (IP-Based) Jails

In a classic jail, you assign one or more IP addresses to the jail. The jail shares the host’s network stack - it uses the host’s interfaces, the host’s routing table, the host’s firewall. The kernel simply restricts which addresses the jailed processes can bind to and communicate from.

Classic Jail networking:

  ┌───────────────────────────────────────────────┐
  │                  Host Kernel                  │
  │                                               │
  │   Network Stack (shared)                      │
  │   ┌───────────┐                               │
  │   │  vtnet0   │  203.0.113.50                 │
  │   │           │  10.0.0.11 (bound to jail A)  │
  │   │           │  10.0.0.12 (bound to jail B)  │
  │   └───────────┘                               │
  │                                               │
  │   ┌─────────────┐    ┌─────────────┐          │
  │   │   Jail A    │    │   Jail B    │          │
  │   │ can only    │    │ can only    │          │
  │   │ bind to     │    │ bind to     │          │
  │   │ 10.0.0.11   │    │ 10.0.0.12   │          │
  │   └─────────────┘    └─────────────┘          │
  └───────────────────────────────────────────────┘

This is simple and lightweight. No extra interfaces, no bridges, no routing between host and jail. The jail just sees “its” IP on the host’s interface. Classic jails work well when you need basic process isolation and don’t need the jail to run its own firewall, its own routing, or anything that requires full control over a network stack.

The limitation: because the jail shares the host’s network stack, it cannot run services that need raw socket access, can’t configure its own routes, can’t run its own DHCP client, and can’t have its own firewall rules. The jail has an IP address, but it doesn’t have a network.

VNET (Virtual Network) Jails

VNET jails get their own complete network stack. Not just an IP - a full, independent network stack with its own interfaces, its own routing table, its own ARP cache, and the ability to run network services that would be impossible in a classic jail.

VNET Jail networking:

  ┌────────────────────────────────────────────────────────────────┐
  │                        Host Kernel                             │
  │                                                                │
  │  Host Network Stack            Jail A Network Stack            │
  │  ┌──────────┐                  ┌──────────┐                    │
  │  │ vtnet0   │                  │  vnet0   │                    │
  │  │ bridge0  │                  │  lo0     │                    │
  │  │ e0a_jail │                  │ (own routing table)           │
  │  └──────────┘                  └──────────┘                    │
  │       │                              │                         │
  │       └──────── epair ───────────────┘                         │
  │            (virtual ethernet cable)                            │
  └────────────────────────────────────────────────────────────────┘

This is what you want for production workloads. A VNET jail can run its own DNS resolver, its own firewall (yes, pf inside a jail), handle its own IPv6 router advertisements, and generally behave like a separate machine. From the network’s perspective, it is a separate machine - just one that happens to share a kernel with the host.

The trade-off is complexity. VNET jails need virtual interfaces, bridges, and explicit routing between the host and jail networks. That complexity is what the rest of this article explains.

When to Use Which

For anything resembling a real server workload - a mail server, a web server with its own TLS stack, a database - VNET is the right choice. Classic jails still make sense for batch jobs, build environments, or situations where the jail is purely a filesystem and process boundary.

The VNET Networking Model

This is where most people get lost, so let’s build the mental model piece by piece.

Epair Interfaces: Virtual Ethernet Cables

An epair is a pair of virtual Ethernet interfaces connected back-to-back. Think of it as a virtual Ethernet cable with a plug on each end. Creating one gives you two interfaces: epair0a and epair0b. Whatever goes into one end comes out the other.

# Create an epair - this gives you epair0a and epair0b
ifconfig epair create
# Output: epair0a

The a side stays on the host. The b side gets moved into the jail. They’re connected at the kernel level - traffic sent into epair0a appears on epair0b, and vice versa. No routing, no forwarding decision - it’s a direct Layer 2 link, like plugging an Ethernet cable between two physical machines.

  Host                          Jail
  ┌──────────┐                  ┌──────────┐
  │          │    epair "wire"  │          │
  │  e0a_web ├──────────────────┤ e0b_web  │
  │          │                  │ (→vnet0) │
  └──────────┘                  └──────────┘
     stays                        moves
     on host                      into jail

In practice, you rename both sides to something meaningful (like e0a_mailserver and e0b_mailserver) so you can tell which pair belongs to which jail when you have a dozen of them.

The Bridge: Connecting Jails Together

Each epair connects exactly one jail to the host. But jails usually need to talk to each other and to the outside world. That’s where bridge0 comes in.

A bridge is a virtual network switch. You add the host-side (a) end of each epair to the bridge, assign the bridge an IP address (which becomes the jails’ default gateway), and now all jails are on the same Layer 2 network segment:

                        ┌─────────────────────────────────────────┐
                        │              bridge0                    │
                        │          10.0.0.1/24                    │
                        │    2001:db8:1234:5678::1/64             │
  Internet              │                                         │
     │                  │  ┌──────────┐  ┌──────────┐  ┌────────┐ │
     │   NAT (pf)       │  │e0a_mail  │  │e0a_web   │  │e0a_mon │ │
  ┌──┴──────┐           │  └────┬─────┘  └────┬─────┘  └───┬────┘ │
  │ vtnet0  │           └───────┼─────────────┼────────────┼──────┘
  │ (host)  │                   │             │            │
  └─────────┘             ┌─────┴─────┐ ┌─────┴─────┐ ┌────┴──────┐
                          │ mailserver│ │  webmail  │ │ monitor   │
                          │ 10.0.0.11 │ │ 10.0.0.12 │ │ 10.0.0.13 │
                          └───────────┘ └───────────┘ └───────────┘

This topology is the same as plugging physical servers into a physical switch. The bridge does MAC learning, forwards frames between ports, and the jails communicate at Layer 2. The host is the gateway - it has an IP on the bridge and runs pf to NAT the jail traffic out to the internet (for IPv4) and to filter what comes in.

For IPv6, if you have a large enough allocation, you can route a subnet directly to the bridge and give each jail a public IPv6 address. No NAT needed. This is exactly how dual-stack jail hosting should work.

Putting It Together: The Lifecycle of a VNET Jail

When a VNET jail starts, several things happen in sequence. The exec.prestart commands in jail.conf run before the jail is created, on the host:

1. Create the epair: ifconfig epair create produces epair0a and epair0b
2. Rename and bring up both ends: ifconfig epair0a up name e0a_mailserver, same for the b side
3. Add the host side to the bridge: ifconfig bridge0 addm e0a_mailserver

Then the jail starts. The vnet.interface directive moves the b-side interface into the jail’s network stack:

1. Move e0b_mailserver into the jail - it disappears from the host’s interface list
2. Inside the jail’s rc.conf: rename e0b_mailserver to vnet0, assign IP addresses, set the default route to the bridge IP

When the jail stops, exec.poststop cleans up:

1. Destroy e0a_mailserver - this automatically destroys the paired e0b_mailserver too

An important detail about the exec.prestart lines: each line runs in its own shell invocation. The $epair0 variable from the ifconfig epair create call only exists within the same line, which is why the create, rename and bring-up are chained with && into a single line in the config below.

Here’s what this looks like in jail.conf:

mailserver {
    # Filesystem and process isolation
    path = "/jails/mailserver";
    host.hostname = "mailserver";
    enforce_statfs = 2;
    mount.devfs;
    devfs_ruleset = 4;
    exec.clean;              # reset environment before executing jail commands

    # Standard start/stop
    exec.start = '/bin/sh /etc/rc';
    exec.stop = '/bin/sh /etc/rc.shutdown';
    exec.consolelog = /var/log/jails/mailserver_console.log;

    # VNET networking
    vnet;
    vnet.interface = "e0b_mailserver";

    # Create epair, rename, bridge - runs on the host before jail starts
    exec.prestart += "epair0=\$(ifconfig epair create) && ifconfig \${epair0} up name e0a_mailserver && ifconfig \${epair0%a}b up name e0b_mailserver";
    exec.prestart += "ifconfig bridge0 addm e0a_mailserver";
    exec.prestart += "ifconfig e0a_mailserver description \"vnet0 host interface for Jail mailserver\"";

    # Clean up - runs on the host after jail stops
    exec.poststop += "ifconfig e0a_mailserver destroy";
}

And the corresponding rc.conf inside the jail at /jails/mailserver/etc/rc.conf:

# Rename the interface that was moved in by vnet.interface
ifconfig_e0b_mailserver_name="vnet0"

# IPv4
ifconfig_vnet0="inet 10.0.0.11 netmask 255.255.255.0"
defaultrouter="10.0.0.1"

# IPv6 (public address, no NAT needed)
ifconfig_vnet0_ipv6="inet6 2001:db8:1234:5678::11/64"
ipv6_defaultrouter="2001:db8:1234:5678::1"

# Description for ifconfig output
ifconfig_vnet0_descr="jail interface for bridge0"

# Standard jail housekeeping
syslogd_flags="-ss"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
cron_flags="-J 60"
moused_nondefault_enable="NO"
dumpdev="NO"

# Actual services
postfix_enable="YES"
dovecot_enable="YES"
rspamd_enable="YES"
local_unbound_enable="YES"

A few things to note about the rc.conf pattern:

• syslogd_flags="-ss" disables syslogd’s network socket. Without this, syslogd inside the jail tries to open a socket it shouldn’t need.
• sendmail_*="NO" disables all sendmail components. Even if you’re running Postfix, the base system’s sendmail cron jobs will try to start otherwise.
• cron_flags="-J 60" tells cron to stagger job execution with a random delay of up to 60 seconds. Prevents all jails from firing cron jobs at the exact same second.

Host Configuration: Enabling the Infrastructure

The host needs a few things configured in /etc/rc.conf before jails can use VNET networking:

# Create the bridge at boot
cloned_interfaces="bridge0"

# Bridge gets the gateway IP for the jail network
ifconfig_bridge0="inet 10.0.0.1 netmask 255.255.255.0"
ifconfig_bridge0_ipv6="inet6 2001:db8:1234:5678::1/64"

# Enable IP forwarding (the host is the jails' router)
gateway_enable="YES"
ipv6_gateway_enable="YES"

# Enable the jail subsystem and packet filter
jail_enable="YES"
pf_enable="YES"

The gateway_enable lines are critical. Without them, the kernel won’t forward packets between the bridge (jail network) and the external interface. Your jails would have IP addresses but no connectivity beyond the bridge.

Firewalling with pf

With VNET jails, the host is the router. All traffic between jails and the internet passes through the host’s network stack, which means pf on the host controls everything.

NAT for IPv4

Unless you have a large IPv4 allocation (unlikely and expensive), your jails will use private addresses on the bridge and NAT through the host’s public IP:

ext_if = "vtnet0"
jail_net = "10.0.0.0/24"

# NAT jail traffic going to the internet
nat on $ext_if inet from $jail_net to any -> ($ext_if)

The parentheses around ($ext_if) tell pf to dynamically resolve the interface’s current IP. If the host’s IP changes (DHCP), the NAT rule adapts automatically.

Redirecting Inbound Traffic

For services that need to be reachable from the internet over IPv4, use rdr to forward incoming connections to the jail’s private address:

mail_ipv4 = "10.0.0.11"

# SMTP from anywhere
rdr pass on $ext_if inet proto tcp from any to ($ext_if) port 25 -> $mail_ipv4

# IMAP and submission
rdr pass on $ext_if inet proto tcp from any to ($ext_if) \
    port {143, 587, 4190} -> $mail_ipv4

IPv6: No NAT Needed

With IPv6, each jail has a globally routable address. No NAT, no rdr - just pass rules:

mail_ipv6 = "2001:db8:1234:5678::11"

pass in quick on $ext_if inet6 proto tcp from any to $mail_ipv6 \
    port 25 flags S/SA keep state
pass in quick on $ext_if inet6 proto tcp from any to $mail_ipv6 \
    port {143, 587, 4190} flags S/SA keep state

This is one of the practical advantages of deploying jails on a host with IPv6: the networking model becomes dramatically simpler. Each jail is a first-class citizen on the internet, directly addressable, with filtering handled by pass/block rules rather than NAT and redirection gymnastics.

Jail Egress

Jails also need to initiate outbound connections (DNS queries, package downloads, sending mail). Allow this with a broad egress rule on the bridge:

jail_net = "10.0.0.0/24"
jail_net6 = "2001:db8:1234:5678::/64"

# Allow jails to reach the internet, but not each other's private range
pass in quick on bridge0 from $jail_net to ! $jail_net keep state
pass in quick on bridge0 inet6 from $jail_net6 to ! $jail_net6 keep state

The ! $jail_net exclusion means this rule only matches traffic leaving the jail network towards the internet. Traffic between jails on the same bridge is switched at Layer 2 and never passes through pf in the default FreeBSD configuration (controlled by net.link.bridge.pfil_member and net.link.bridge.pfil_bridge sysctls, both off by default). If you need to restrict inter-jail communication, that requires either separate bridges per jail or enabling pfil_member and adding explicit block rules on the bridge member interfaces.

devfs Rules: Controlling Device Access

When a jail has mount.devfs enabled, it gets a /dev filesystem. But you don’t want a jail to see the host’s disk devices, USB devices, or hardware random number generators it shouldn’t touch. devfs rulesets control which device nodes appear inside the jail.

The relevant rulesets are defined in /etc/devfs.rules on the host. FreeBSD ships with a default set, and ruleset 4 (devfsrules_jail) is specifically designed for jails:

# View the built-in rulesets
devfs rule -s 4 show

Ruleset 4 typically hides everything and then unhides only what a jail needs:

• /dev/null, /dev/zero, /dev/random, /dev/urandom - standard Unix devices
• /dev/fd/* - file descriptors
• /dev/stdin, /dev/stdout, /dev/stderr - standard streams

What it doesn’t include: /dev/ad*, /dev/da* (disk devices), /dev/mem (physical memory), /dev/kmem (kernel memory), /dev/io (I/O port access). A jailed process cannot read raw disk data or probe kernel memory, even as root.

Notably, ruleset 4 also does not unhide /dev/bpf* (Berkeley Packet Filter devices). This matters for VNET jails: without bpf, tools like tcpdump and dhclient inside the jail won’t work. If you need either, you’ll want a custom ruleset that adds bpf access - shown below.

You reference the ruleset in jail.conf:

devfs_ruleset = 4;

If you need to customize device access (for example, giving a jail access to /dev/pf for its own packet filter, or /dev/zvol/* for ZFS volume access), you create a custom ruleset in /etc/devfs.rules:

[devfsrules_jail_with_pf=100]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'bpf*' unhide
add path pf unhide

Then reference it as devfs_ruleset = 100 in the jail’s configuration.

Security: What Jails Actually Prevent

Jails enforce security at the kernel level. Here’s what a root user inside a jail cannot do, regardless of their privilege level:

• Load or unload kernel modules - no kldload, no kldunload
• Mount or unmount filesystems - unless enforce_statfs and allow.mount* are explicitly relaxed
• Modify the host’s network - can’t touch the host’s interfaces, routes, or firewall
• See processes outside the jail - ps aux shows only jail-local processes
• Access raw disk devices - blocked by devfs rules
• Modify the running kernel - no access to /dev/mem, /dev/kmem
• Create device nodes - no mknod
• Modify the system clock - no adjtime, no settimeofday

enforce_statfs

The enforce_statfs directive controls what the jail can see about mounted filesystems:

• enforce_statfs = 0 - jail sees all mount points (don’t use this)
• enforce_statfs = 1 - jail sees mount points beneath its root
• enforce_statfs = 2 - jail sees only its own mount points (recommended)

At level 2, df inside the jail shows only the jail’s own filesystems. The jail can’t discover the host’s ZFS pool layout, other jails’ mount points, or the overall disk topology.

Managing Jails

Starting and Stopping

# Start all jails
service jail start

# Start a specific jail
service jail start mailserver

# Stop a specific jail
service jail stop webmail

# Restart (stop + start) a specific jail
service jail restart mailserver

Entering a Running Jail

# Get a shell inside a jail
jexec mailserver /bin/sh

# Run a specific command
jexec mailserver pkg update

# See running jails
jls

jls shows all active jails with their JID (Jail ID), IP addresses, hostname, and path. It’s the FreeBSD equivalent of docker ps.

Filesystem: ZFS Datasets

While you can use any directory as a jail root, ZFS datasets are the standard approach for production:

# Create a dataset for your jail infrastructure
zfs create zroot/jails

# Create a dataset for a specific jail
zfs create zroot/jails/mailserver

# Install a fresh userland into it
bsdinstall jail /jails/mailserver

ZFS gives you snapshots (zfs snapshot zroot/jails/mailserver@before-upgrade), cloning (create a new jail from an existing snapshot in seconds), quotas, compression, and send/receive for backup and migration. This is comparable to Docker’s layered images, but without the overlay filesystem complexity.

Nullfs Mounts: Sharing Data Between Host and Jail

Sometimes a jail needs access to data that lives outside its filesystem root. The nullfs mount (a loopback mount, similar to mount --bind on Linux) makes a host directory visible inside the jail:

# In jail.conf
mount += "/var/vmail /jails/mailserver/var/vmail nullfs rw 0 0";

This mounts the host’s /var/vmail at /jails/mailserver/var/vmail. The mail jail sees it as a local directory. This is useful for shared mail spools, database directories on fast storage, or any data that should persist independently of the jail’s lifecycle.

A Complete Example: Three-Jail Architecture

Here’s a complete topology for a generic server running three VNET jails:

                                  Internet
                                     │
                            ┌────────┴────────┐
                            │     vtnet0      │
                            │   203.0.113.50  │
                            │  (public IPv4)  │
                            └────────┬────────┘
                                     │
                                 pf (NAT/RDR)
                                     │
     ┌───────────────────────────────┴──────────────────────────────┐
     │                          bridge0                             │
     │                      10.0.0.1/24                             │
     │               2001:db8:1234:5678::1/64                       │
     │                                                              │
     │  ┌───────────────┐  ┌──────────────┐  ┌───────────────────┐  │
     │  │e0a_mailserver │  │ e0a_webmail  │  │  e0a_monitor      │  │
     │  └───────┬───────┘  └──────┬───────┘  └─────────┬─────────┘  │
     └──────────┼─────────────────┼────────────────────┼────────────┘
                │ epair           │ epair              │ epair
     ┌──────────┴──────┐  ┌───────┴───────┐  ┌─────────┴──────────┐
     │   mailserver    │  │    webmail    │  │      monitor       │
     │   10.0.0.11     │  │   10.0.0.12   │  │     10.0.0.13      │
     │                 │  │               │  │                    │
     │   Postfix       │  │    nginx      │  │   Prometheus       │
     │   Dovecot       │  │    php-fpm    │  │   Grafana          │
     │   Rspamd        │  │               │  │                    │
     │   Unbound       │  │               │  │                    │
     └─────────────────┘  └───────────────┘  └────────────────────┘

Each jail gets its own epair, its own IP, and runs only the services it needs. The mailserver jail handles SMTP, IMAP, and spam filtering. The webmail jail runs a web interface. The monitor jail runs Prometheus and Grafana for metrics collection.

This separation means a vulnerability in the webmail application (PHP, web framework) doesn’t automatically grant access to the mail spool, the SMTP relay, or the host system. Each blast radius is contained.

Common Pitfalls

Forgetting gateway_enable: Your jails will get IPs, the bridge will be up, but nothing can reach the internet. The host kernel silently drops forwarded packets.

Not creating /var/log/jails/: If the exec.consolelog directory doesn’t exist, jail startup fails silently. Create it ahead of time:

mkdir -p /var/log/jails

Interface naming conflicts: If two jails try to create epairs simultaneously without unique names, you get collisions. The naming convention in jail.conf (e0a_<jailname> / e0b_<jailname>) avoids this.

DNS resolution inside jails: Jails don’t automatically inherit the host’s /etc/resolv.conf. Either copy it into the jail’s root, run a local resolver like unbound inside the jail, or point resolv.conf at the bridge IP if the host runs a resolver there.

Forgetting to enable jail services: After bsdinstall jail, the base system is minimal. Services need to be installed (pkg -j mailserver install postfix) and enabled in the jail’s rc.conf. Don’t forget sshd_enable="YES" if you want to SSH into the jail directly (though jexec from the host is often sufficient).

Conclusion

Jails are not a historical curiosity. They’re a production-grade isolation mechanism backed by over 25 years of kernel-level enforcement. VNET gives each jail a real network identity, epairs provide the plumbing, bridges create the fabric, and pf ties it all together with filtering and NAT.

The mental model to carry away: a VNET jail is a separate machine that shares a kernel with the host. It has its own network stack, its own filesystem view, and its own process space. What it doesn’t have is the ability to affect the host - and that’s enforced by the kernel, not by convention.

The next article in this series will cover ZFS - the filesystem that makes jail management, snapshots, and backups actually practical at scale.

References

• FreeBSD Handbook - Jails
• jail(8) man page
• jail.conf(5) man page
• devfs.rules(5) man page
• epair(4) man page
• if_bridge(4) man page
• pf.conf(5) man page
• Poul-Henning Kamp, Robert N. M. Watson - Jails: Confining the Omnipotent Root - the original 2000 paper introducing Jails