# Understanding FreeBSD Jails

2019-01-29

## What is a Jail?

Jails are one of the most useful features offered by FreeBSD. This technology, introduced with FreeBSD 4.0, allows system administrators, developers or any other kind of users to create multiple user-space instances that are fully isolated from the rest of the operating system. Jails are considered the standard way to obtain OS-level virtualization on BSD systems. OS-level virtualization is a particular kind of virtualization technology that operate on the OS-level; in that way the host's kernel allows the execution of multiple isolated user-space instances (called containers, Zones or Jails) which they all share host's resources, such as network stack, disk and the kernel. In other words, this virtualization mechanism provides the operating system a way to replicate kernel's functionalities into these user-space instances (Jails).

## Differences between Jails and VMs

Even if they are related topics, Jails and Virtual Machines differ in how they work under the hood. As stated in the previous section, a Jail is nothing more than a user-space instance of the host's OS; this means that every Jail must use the host's kernel and so, it is impossible to execute a different operating system using this approach. Just like you cannot run FreeBSD with Docker (under Linux), you cannot execute a Linux instance using Jails (on FreeBSD). To understand why this is not practical possible, let us suppose that you want to execute a FreeBSD container using Docker under a GNU/Linux distribution. Since the container's system uses the host's kernel, it would attempt to make FreeBSD system calls into the Linux kernel. This is obviously not possible, and so nor using OS-level virtualization with different operating systems (a more comprehensive explanation here).

A virtual machine, on the other hand, do not use the host's kernel to provide functionalities to the guest system. In fact, a VM provides a whole operating system, which is completely isolated from the rest of the system (and from any other virtual machine). In this approach, the Hypervisor (i.e., the software in charge of executing virtual machines) will be responsible for the guest's supervisor (i.e., the kernel) and so, to virtualize the needed hardware resources (for instance, the CPU, the RAM, the disk, etc.).

Another important aspect of Jails is the small overhead compared to virtual machines: since container technology does not have to pass through another level of abstraction (i.e., the hypervisor), the overall overhead is very small. Even if with today's hardware we probably do not have to worry about this detail anymore, it still is a good advance.

## Security of a Jail

Another crucial aspect of Jails is their security. In fact, Jail's processes are isolated to the resources assigned to it. This means that a user can do whatever they want inside the Jail without worrying about damaging other resources or without interfering with other network activities. This also means that if someone hacks a Jail, the intruder cannot see anything outside the Jail. The other security aspect to keep in mind is that Jails are very easy to save/restore; a system administrator could write a script that just copy the Jail's directory and enclose it in a tar archive. Restoring a Jail it is just the reverse process.

## Host base configuration

Now that we have learned enough about how does a Jail work, their security aspects and how they differ from a virtual machine, let us focus on the practical aspect of this guide. In this first section, we will discuss how to prepare the base system to host Jails. At the time of writing this guide, I am using FreeBSD 12.2-RELEASE, but any other version should be fine. By default, FreeBSD does not impose you any default path for Jails, this means that you can store your Jails everywhere you like, but for the sake of simplicity I will use /jail.

## Networking setup

There are two ways to set up networking on a Jail: inheriting the IP from the host or assigning a dedicated one. The first method allows you to use the host's IP address; with this approach you can avoid configuring multiple IP address (or multiples NIC) per each Jail. Note that though, since each Jail has complete control over the IP address, you cannot run multiple instances of the same service. For instance, if the host network has a web server running at port 80, you cannot start the same service on the same port on the Jail. The service just will not start since it cannot bind that port. Keep in mind that both approaches are completely fine and that you can choose which one to use according to the task you are trying to achieve, there is no general rule here.

## Jail configuration

By default, FreeBSD will configure each Jail using the /etc/jail.conf file. Although this configuration file has many options (that cannot be listed in a single article) I will show only the most important, the ones you really need to make your Jail work.

In this first example, let us try to configure a Jail for a NGINX web server. To do that, we first need to install the base system, configure it using the /etc/jail.conf file and then install the NGINX package trough pkg.

## Installing FreeBSD

In order to install a FreeBSD instance, we can use the bsdinstall(8) command. But first, we need to create the base directory of the Jail. Let us name it www:
                    
$> mkdir -p /jail/www$> bsdinstall jail /jail/www


The last command will walk you through the usual BSD installer, for the sake of the completeness, I have included some step-by-step screenshots below.

## Configuring the Jail

Now we need to configure the jail from the host. Open /etc/jail.conf with your favorite text editor and add the following parameters:
                    
www {
mount.devfs; # Mount dev file system
exec.clean; # Reset environment variables
exec.start="sh /etc/rc"; # Init script
exec.stop="sh /etc/rc.shutdown"; # Stop script
path="/jail/www"; # Where this Jail is located
allow.raw_sockets=1; # Enable raw socket support
#ip4=inherit; # Use host's IP address
host.hostname="www";
}


A great aspect of the Jail's configuration file is that it allows you to define global options i.e., options that are shared with every Jail definition. You can use this feature to define smaller Jail entries moving all the general parameters outside the curly brackets. With this new approach, the previous configuration file becomes something like this:
                    
# Get path for each Jail
$jail_path="/jail"; path="$jail_path/$name"; # General settings mount.devfs; # Mount dev file system exec.clean; # Reset environment variables exec.start="sh /etc/rc"; # Init script exec.stop="sh /etc/rc.shutdown"; # Stop script allow.raw_sockets=1; # Enable raw socket support #ip4=inherit; # Use host's IP address www { ip4.addr="10.0.2.16"; # Specify a dedicated IP address host.hostname="www"; }   This cuts down the size of each Jail to two lines(one line if all your Jails use host's IP address)! ## Starting the Jail We are now ready to boot up the Jail. To do that we can use the service(8) command, but before that, we need to configure the /etc/rc.conf file:   […] jail_list="www" # List which jails you want to start at boot jail_enable="YES"   Now start the www Jail using service jail start www. The Jail should be now up and running. To assure that, run jls, you should get something like this:   root@fbsdvm01:/jail> jls JID IP Address Hostname Path 1 10.0.2.16 www /jail/www   ## Opening a shell You now may need to open a shell inside your Jail to perform administration tasks, such as installing packages, configuring files and so on. To do that, you can use the jexec(8) command. In our example:   jexec www /bin/sh   Inside the jail you can try to ping something. It should work. To detach from the Jail, simply hit ^C+d. ## Installing packages We saw in the previous section how to get a shell into an active Jail using jexec. While it is possible to install packages manually logging in into the Jail using this command, it can be very tricky if you have a lot of Jails to administrate. To avoid this, pkg(8) can be used to install packages into a Jail directly from the host. To do that, you just have to specify the name of the jail you want to install the package after the -j flag. In our example we would type:   root@fbsdvm01:/jail> pkg -j www install nginx Updating FreeBSD repository catalogue... [www] Fetching meta.conf: 100% 163 B 0.2kB/s 00:01 [www] Fetching packagesite.txz: 100% 6 MiB 3.2MB/s 00:02 Processing entries: 100% FreeBSD repository update completed. 30177 packages processed. All repositories are up to date. The following 2 package(s) will be affected (of 0 checked): New packages to be INSTALLED: nginx: 1.18.0_45,2 pcre: 8.44 Number of packages to be installed: 2 The process will require 8 MiB more space. 2 MiB to be downloaded. Proceed with this action? [y/N]: y   ## Starting services Just like before, you can use the service(8) command to manage service inside a Jail. Use the -j flag.   root@fbsdvm01:/jail> service -j www nginx onestart Performing sanity check on nginx configuration: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful Starting nginx.   You should be able to reach the Jail at port 80. ## Real world example: Ghost on Jails In the last part of this guide, I want to give an example of a real-world usage of FreeBSD Jails: we will use Jails to deploy a Ghost blog. Ghost is a free and open-source blogging platform written in JavaScript. It aims to be an alternative to WordPress and even if it is not officially supported on FreeBSD, it is really simple to install, configure and maintain. Ghost can be installed using an SQLite or a MariaDB/PostgreSQL database. While for the majority of blogs SQLite just works fine(furthermore, it is very handy for backups operations), I will use a MariaDB database for the sake of this tutorial. Before getting into the details, let us talk about how we are going to configure the Jails. ## Networking To keep things as simple as possible, each Jail will inherit the host's IP address. There will be no network conflicts since each Jail will run a dedicated service(ghost, MariaDB, NGINX) and the host will run nothing but Jails(i.e., no other service will bind port from the host). ## Ghost jail The first Jail we are going to create is the Jail that will host Ghost. This Jail will just need NodeJS(I will use node12-12.20.1); additionally, it will bind port 2368. You can adjust this value, though. As seen in the previous part of this guide, install the base system using bsdinstall(8) and then, configure this jail by editing the /etc/jail.conf file:   # Get path for each Jail$jail_path="/jail";
path="$jail_path/$name";

# General settings
mount.devfs; # Mount dev file system
exec.clean; # Reset environment variables
exec.start="sh /etc/rc"; # Init script
exec.stop="sh /etc/rc.shutdown"; # Stop script
allow.raw_sockets=1; # Enable raw socket support
ip4=inherit; # Use host's IP address

ghost {
host.hostname="ghost";
}


Now we can start the jail and then install NodeJS version 12 using the following command:
                    
$pkg -j ghost install node12 npm-node12   ## NGINX Jail This second Jail will serve as a reverse proxy for the Ghost backend. I will use NGINX for the web server/reverse proxy, but you can use whatever you want. In any case, this Jail will bind port 80(and 443 if you use TLS). Add this Jail to the configuration file:   […] nginx { host.hostname="nginx"; }   Start it, and then install NGINX using the following command:  $ pkg -j nginx install nginx



The last Jail will host the database. I have chosen MariaDB here, but you can use pretty much any other DBMS of your choice. This Jail will bind port 3306. As we have done before, let us add this Jail to the configuration file:
                    
[…]
}


Start it using # service jail start mariadb and then install the MariaDB server using:
                    
$pkg -j mariadb install mariadb105-client mariadb105-server   ## Setup Now we need to configure each Jail. We can start from MariaDB's Jail. Let us start by configuring the database using mysql_secure_installation(1)   (mariadb)$ sysrc mysql_enable=YES
(mariadb)$service mysql-server start (mariadb)$ mysql_secure_installation


                    
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB

haven't set the root password yet, you should just press enter here.

Enter current password for root (enter for none):
OK, successfully used password, moving on...

Setting the root password or using the unix_socket ensures that nobody

Switch to unix_socket authentication [Y/n] y
Enabled successfully!
... Success!

Change the root password? [Y/n] y
... Success!

By default, a MariaDB installation has an anonymous user, allowing anyone
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] y
... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] y
... Success!

By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!

will take effect immediately.

Reload privilege tables now? [Y/n] y
... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.



Now we need to create a low privileged user and a database dedicated to Ghost.
                    
(mariadb)$mysql -uroot -p root@localhost [(none)]> CREATE USER 'ghost'@'localhost' IDENTIFIED BY 'badpw'; Query OK, 0 rows affected (0.003 sec) root@localhost [(none)]> GRANT ALL PRIVILEGES ON ghostcms.* TO 'ghost'@'localhost'; Query OK, 0 rows affected (0.007 sec) root@localhost [(none)]> CREATE DATABASE ghostcms; Query OK, 1 row affected (0.001 sec)   Now that the database is ready, we can configure Ghost. We first install ghost-cli(a tool aimed to easily install Ghost) and then we create a low privileged user for this purpose.   (ghost)$ npm install -g ghost-cli@latest
(ghost)$adduser # Add a new user called 'ghost'   Now we need to create the installation directory, and we also need to fix user permissions.   (ghost)$ mkdir -p /var/www/ghost
(ghost)$chown ghost:ghost /var/www/ghost (ghost)$ chmod 775 /var/www/ghost


Now move to ghost's root directory and install it:
                    
(ghost)$su - ghost # Switch to ghost account (ghost)$ cd /var/www/ghost
(ghost)$ghost install (ghost)$ ghost install --db=mysql \
--dbhost=127.0.0.1 \
--dbuser=ghost \
--dbname=ghostcms


The installer will now download Ghost from the official website and install the needed dependencies. It will also ask you some questions, ask according to your preferences. Ignore any error regarding the service command; it is basically a problem related to the init system(FreeBSD is not officially supported). After that, we need to edit the config.production.json file. Inside that, remove the following line:
                    
“process”: “systemd”


As I said before, Ghost officially supports systemd(but we are in FreeBSD, remember?).

The last thing to do is to create a service file to easily start, stop and restart Ghost. Create file called /usr/local/etc/rc.d/ghost and put the following content inside it:

After that, give execution permission to the file:
                    
(ghost)$chmod +x /usr/local/etc/rc.d/ghost   Finally, enable ghost at boot and start it   (ghost)$ sysrc ghost_enable=YES
(ghost)$service ghost start   To retrieve Ghost status, you can use:   (ghost)$ service ghost status


We are ready to configure the last Jail: NGINX! The following configuration is only a way to create a reverse proxy through a virtual host, you can use a different setting, of course. Put the following into /usr/local/etc/nginx/nginx.conf:
                    
#user  nobody;
worker_processes  1;

events {
worker_connections  1024;
}

http {
include       mime.types;
default_type  application/octet-stream;

sendfile        on;
keepalive_timeout  65;
server_names_hash_bucket_size 64;

include /usr/local/etc/nginx/sites-enabled/*;
}


Now create two directories called sites-available and sites-enabled inside /usr/local/etc/nginx and put this inside /usr/local/etc/nginx/sites-available/ghost:
                    
server {
listen 80 default_server;

server_name 127.0.0.1;
root /var/www/ghost;

location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto$scheme;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header Host$host;
proxy_pass http://127.0.0.1:2368;
}

client_max_body_size 50m;
}


After that create a symbolic link of this file inside site-enabled:
                    
(nginx)$ln -s /usr/local/etc/nginx/sites-available/ghost /usr/local/etc/nginx/sites-enabled/ghost   Finally, enable nginx and start it:   (nginx)$ sysrc nginx_enable=YES
(nginx)$service nginx start   Now exit from this Jail and check that Ghost, NGINX, MariaDB ports(2368, 80, 3306, respectively) are being bounded:   root@fbsdvm01:/jail$ sockstat -46l
www      nginx      8646  6  tcp4   *:80                  *:*
root     nginx      8645  6  tcp4   *:80                  *:*
marco    node       8486  24 tcp4   127.0.0.1:8080        *:*
marco    node       8135  34 tcp4   127.0.0.1:2368        *:*
mysql    mariadbd   7715  19 tcp4   127.0.0.1:3306        *:*
root     sendmail   747   3  tcp4   127.0.0.1:25          *:*
root     sshd       744   3  tcp6   *:22                  *:*
root     sshd       744   4  tcp4   *:22                  *:*
root     syslogd    609   6  udp6   *:514                 *:*
root     syslogd    609   7  udp4   *:514                 *:*


Now you can reach ghost at http://<server_ip_address>/ghost and complete the installation process. You basically have to set up a user account, and you are ready.

## Conclusions

Although we saw how to install, configure and use Jails, there are a lot more to say about Jails. For instance, we did not spend a single word about firewalling or ZFS templates(i.e., a way to deploy a Jail following a precise schema) which come handy when you have to deploy multiple instances of the same type(for example, multiple instances of a LEMP stack). In some cases, you also want to use tools that aims to simplify the configuration process, such as ezjail. For all these advanced topics I suggest you to reach the official documentation, it is a great place to know more about advanced topics.