UNDERSTANDING FREEBSD JAILS
2019-01-29
What is a Jail?
Jails are one of the most useful features offered by FreeBSD. This technology, introduced with FreeBSD 4.0, allows system administrators, developers or any other kind of users to create multiple user-space instances that are fully isolated from the rest of the operating system. Jails are considered the standard way to obtain OS-level virtualization on BSD systems. OS-level virtualization is a particular kind of virtualization technology that operate on the OS-level; in that way the host's kernel allows the execution of multiple isolated user-space instances (called containers, Zones or Jails) which they all share host's resources, such as network stack, disk and the kernel. In other words, this virtualization mechanism provides the operating system a way to replicate kernel's functionalities into these user-space instances (Jails).Differences between Jails and VMs
Even if they are related topics, Jails and Virtual Machines differ in how they work under the hood. As stated in the previous section, a Jail is nothing more than a user-space instance of the host's OS; this means that every Jail must use the host's kernel and so, it is impossible to execute a different operating system using this approach. Just like you cannot run FreeBSD with Docker (under Linux), you cannot execute a Linux instance using Jails (on FreeBSD). To understand why this is not practical possible, let us suppose that you want to execute a FreeBSD container using Docker under a GNU/Linux distribution. Since the container's system uses the host's kernel, it would attempt to make FreeBSD system calls into the Linux kernel. This is obviously not possible, and so nor using OS-level virtualization with different operating systems (a more comprehensive explanation here).A virtual machine, on the other hand, do not use the host's kernel to provide functionalities to the guest system. In fact, a VM provides a whole operating system, which is completely isolated from the rest of the system (and from any other virtual machine). In this approach, the Hypervisor (i.e., the software in charge of executing virtual machines) will be responsible for the guest's supervisor (i.e., the kernel) and so, to virtualize the needed hardware resources (for instance, the CPU, the RAM, the disk, etc.).
Another important aspect of Jails is the small overhead compared to virtual machines: since container technology does not have to pass through another level of abstraction (i.e., the hypervisor), the overall overhead is very small. Even if with today's hardware we probably do not have to worry about this detail anymore, it still is a good advance.
Security of a Jail
Another crucial aspect of Jails is their security. In fact, Jail's processes are isolated to the resources assigned to it. This means that a user can do whatever they want inside the Jail without worrying about damaging other resources or without interfering with other network activities. This also means that if someone hacks a Jail, the intruder cannot see anything outside the Jail. The other security aspect to keep in mind is that Jails are very easy to save/restore; a system administrator could write a script that just copy the Jail's directory and enclose it in a tar archive. Restoring a Jail it is just the reverse process.Host base configuration
Now that we have learned enough about how does a Jail work, their security aspects and how they differ from a virtual machine, let us focus on the practical aspect of this guide. In this first section, we will discuss how to prepare the base system to host Jails. At the time of writing this guide, I am usingFreeBSD 12.2-RELEASE, but any other version should be fine.
By default, FreeBSD does not impose you any default path for Jails, this means that you can store your
Jails everywhere you like, but for the sake of simplicity I will use /jail.
Networking setup
There are two ways to set up networking on a Jail: inheriting the IP from the host or assigning a dedicated one. The first method allows you to use the host's IP address; with this approach you can avoid configuring multiple IP address (or multiples NIC) per each Jail. Note that though, since each Jail has complete control over the IP address, you cannot run multiple instances of the same service. For instance, if the host network has a web server running at port 80, you cannot start the same service on the same port on the Jail. The service just will not start since it cannot bind that port. Keep in mind that both approaches are completely fine and that you can choose which one to use according to the task you are trying to achieve, there is no general rule here.Jail configuration
By default, FreeBSD will configure each Jail using the/etc/jail.conf file. Although this configuration file
has many options (that cannot be listed in a single article) I will show only the most important,
the ones you really need to make your Jail work.
Deploy your first Jail
In this first example, let us try to configure a Jail for a NGINX web server. To do that, we first need to install the base system, configure it using the/etc/jail.conf file and then install the NGINX package trough
pkg.
Installing FreeBSD
In order to install a FreeBSD instance, we can use thebsdinstall(8) command.
But first, we need to create the base directory of the Jail. Let us name it www:
$> mkdir -p /jail/www
$> bsdinstall jail /jail/www
The last command will walk you through the usual BSD installer,
for the sake of the completeness, I have included some step-by-step screenshots below.
Configuring the Jail
Now we need to configure the jail from the host. Open/etc/jail.conf with your favorite text
editor and add the following parameters:
www {
mount.devfs; # Mount dev file system
exec.clean; # Reset environment variables
exec.start="sh /etc/rc"; # Init script
exec.stop="sh /etc/rc.shutdown"; # Stop script
path="/jail/www"; # Where this Jail is located
ip4.addr="10.0.2.16"; # Specify a dedicated IP address
allow.raw_sockets=1; # Enable raw socket support
#ip4=inherit; # Use host's IP address
host.hostname="www";
}
A great aspect of the Jail's configuration file is that it allows you to
define global options i.e., options that are shared with every Jail definition.
You can use this feature to define smaller Jail entries moving all the general parameters
outside the curly brackets. With this new approach, the previous configuration file becomes something like this:
# Get path for each Jail
$jail_path="/jail";
path="$jail_path/$name";
# General settings
mount.devfs; # Mount dev file system
exec.clean; # Reset environment variables
exec.start="sh /etc/rc"; # Init script
exec.stop="sh /etc/rc.shutdown"; # Stop script
allow.raw_sockets=1; # Enable raw socket support
#ip4=inherit; # Use host's IP address
www {
ip4.addr="10.0.2.16"; # Specify a dedicated IP address
host.hostname="www";
}
This cuts down the size of each Jail to two lines(one line if all your Jails use host's IP address)!
Starting the Jail
We are now ready to boot up the Jail. To do that we can use theservice(8) command,
but before that, we need to configure the /etc/rc.conf file:
[…]
jail_list="www" # List which jails you want to start at boot
jail_enable="YES"
Now start the www Jail using service jail start www.
The Jail should be now up and running.
To assure that, run jls, you should get something like this:
root@fbsdvm01:/jail> jls
JID IP Address Hostname Path
1 10.0.2.16 www /jail/www
Opening a shell
You now may need to open a shell inside your Jail to perform administration tasks, such as installing packages, configuring files and so on. To do that, you can use thejexec(8) command. In our example:
jexec www /bin/sh
Inside the jail you can try to ping something. It should work. To detach from the Jail, simply hit
^C+d.
Installing packages
We saw in the previous section how to get a shell into an active Jail usingjexec.
While it is possible to install packages manually logging in into the Jail using this command, it
can be very tricky if you have a lot of Jails to administrate.
To avoid this, pkg(8) can be used to install packages into a Jail directly from the host.
To do that, you just have to specify the name of the jail you want to install
the package after the -j flag. In our example we would type:
root@fbsdvm01:/jail> pkg -j www install nginx
Updating FreeBSD repository catalogue...
[www] Fetching meta.conf: 100% 163 B 0.2kB/s 00:01
[www] Fetching packagesite.txz: 100% 6 MiB 3.2MB/s 00:02
Processing entries: 100%
FreeBSD repository update completed. 30177 packages processed.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
nginx: 1.18.0_45,2
pcre: 8.44
Number of packages to be installed: 2
The process will require 8 MiB more space.
2 MiB to be downloaded.
Proceed with this action? [y/N]: y
Starting services
Just like before, you can use theservice(8) command to manage service inside a Jail.
Use the -j flag.
root@fbsdvm01:/jail> service -j www nginx onestart
Performing sanity check on nginx configuration:
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Starting nginx.
You should be able to reach the Jail at port 80.
Real world example: Ghost on Jails
In the last part of this guide, I want to give an example of a real-world usage
of FreeBSD Jails: we will use Jails to deploy a Ghost blog.
Ghost
is a free and open-source blogging platform written in JavaScript.
It aims to be an alternative to WordPress and even if it is not officially supported on FreeBSD,
it is really simple to install, configure and maintain.
Ghost can be installed using an SQLite or a MariaDB/PostgreSQL database.
While for the majority of blogs SQLite just works fine(furthermore, it is very handy for backups operations),
I will use a MariaDB database for the sake of this tutorial.
Before getting into the details, let us talk about how we are going to configure the Jails.
Networking
To keep things as simple as possible, each Jail will inherit the host's IP address. There will be no network conflicts since each Jail will run a dedicated service(ghost, MariaDB, NGINX) and the host will run nothing but Jails(i.e., no other service will bind port from the host).Ghost jail
The first Jail we are going to create is the Jail that will host Ghost. This Jail will just need NodeJS(I will usenode12-12.20.1); additionally, it will bind port 2368.
You can adjust this value, though. As seen in the previous part of this guide,
install the base system using bsdinstall(8) and then, configure this jail by editing
the /etc/jail.conf file:
# Get path for each Jail
$jail_path="/jail";
path="$jail_path/$name";
# General settings
mount.devfs; # Mount dev file system
exec.clean; # Reset environment variables
exec.start="sh /etc/rc"; # Init script
exec.stop="sh /etc/rc.shutdown"; # Stop script
allow.raw_sockets=1; # Enable raw socket support
ip4=inherit; # Use host's IP address
ghost {
host.hostname="ghost";
depend=mariadb;
}
Now we can start the jail and then install NodeJS version 12 using the following command:
$ pkg -j ghost install node12 npm-node12
NGINX Jail
This second Jail will serve as a reverse proxy for the Ghost backend. I will use NGINX for the web server/reverse proxy, but you can use whatever you want. In any case, this Jail will bind port 80(and 443 if you use TLS). Add this Jail to the configuration file:
[…]
nginx {
host.hostname="nginx";
}
Start it, and then install NGINX using the following command:
$ pkg -j nginx install nginx
MariaDB Jail
The last Jail will host the database. I have chosen MariaDB here, but you can use pretty much any other DBMS of your choice. This Jail will bind port 3306. As we have done before, let us add this Jail to the configuration file:
[…]
mariadb {
host.hostname="mariadb";
}
Start it using # service jail start mariadb
and then install the MariaDB server using:
$ pkg -j mariadb install mariadb105-client mariadb105-server
Setup
Now we need to configure each Jail. We can start from MariaDB's Jail. Let us start by configuring the database usingmysql_secure_installation(1)
(mariadb)$ sysrc mysql_enable=YES
(mariadb)$ service mysql-server start
(mariadb)$ mysql_secure_installation
The setup will now ask you some questions, answer them carefully.
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MariaDB to secure it, we'll need the current
password for the root user. If you've just installed MariaDB, and
haven't set the root password yet, you should just press enter here.
Enter current password for root (enter for none):
OK, successfully used password, moving on...
Setting the root password or using the unix_socket ensures that nobody
can log into the MariaDB root user without the proper authorisation.
You already have your root account protected, so you can safely answer 'n'.
Switch to unix_socket authentication [Y/n] y
Enabled successfully!
Reloading privilege tables..
... Success!
You already have your root account protected, so you can safely answer 'n'.
Change the root password? [Y/n] y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..
... Success!
By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] y
... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] y
... Success!
By default, MariaDB comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] y
- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] y
... Success!
Cleaning up...
All done! If you've completed all of the above steps, your MariaDB
installation should now be secure.
Thanks for using MariaDB!
Now we need to create a low privileged user and a database dedicated to Ghost.
(mariadb)$ mysql -uroot -p
root@localhost [(none)]> CREATE USER 'ghost'@'localhost' IDENTIFIED BY 'badpw';
Query OK, 0 rows affected (0.003 sec)
root@localhost [(none)]> GRANT ALL PRIVILEGES ON ghostcms.* TO 'ghost'@'localhost';
Query OK, 0 rows affected (0.007 sec)
root@localhost [(none)]> CREATE DATABASE ghostcms;
Query OK, 1 row affected (0.001 sec)
Now that the database is ready, we can configure Ghost.
We first install ghost-cli(a tool aimed to easily install Ghost)
and then we create a low privileged user for this purpose.
(ghost)$ npm install -g ghost-cli@latest
(ghost)$ adduser # Add a new user called 'ghost'
Now we need to create the installation directory, and we also need to fix user permissions.
(ghost)$ mkdir -p /var/www/ghost
(ghost)$ chown ghost:ghost /var/www/ghost
(ghost)$ chmod 775 /var/www/ghost
Now move to ghost's root directory and install it:
(ghost)$ su - ghost # Switch to ghost account
(ghost)$ cd /var/www/ghost
(ghost)$ ghost install
(ghost)$ ghost install --db=mysql \
--dbhost=127.0.0.1 \
--dbuser=ghost \
--dbpass=badpw \
--dbname=ghostcms
The installer will now download Ghost from the official website and install the needed dependencies.
It will also ask you some questions, ask according to your preferences.
Ignore any error regarding the service command; it is basically a problem related to the init
system(FreeBSD is not officially supported).
After that, we need to edit the config.production.json file.
Inside that, remove the following line:
"process": "systemd"
As I said before, Ghost officially supports systemd(but we are in FreeBSD, remember?).
The last thing to do is to create a service file to easily start, stop and restart Ghost. Create file called
/usr/local/etc/rc.d/ghost and put the following content inside it:
After that, give execution permission to the file:
(ghost)$ chmod +x /usr/local/etc/rc.d/ghost
Finally, enable ghost at boot and start it
(ghost)$ sysrc ghost_enable=YES
(ghost)$ service ghost start
To retrieve Ghost status, you can use:
(ghost)$ service ghost status
We are ready to configure the last Jail: NGINX!
The following configuration is only a way to create a reverse proxy through a virtual host,
you can use a different setting, of course. Put the following into /usr/local/etc/nginx/nginx.conf:
#user nobody;
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server_names_hash_bucket_size 64;
include /usr/local/etc/nginx/sites-enabled/*;
}
Now create two directories called sites-available
and sites-enabled inside /usr/local/etc/nginx
and put this inside
/usr/local/etc/nginx/sites-available/ghost:
server {
listen 80 default_server;
server_name 127.0.0.1;
root /var/www/ghost;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:2368;
}
client_max_body_size 50m;
}
After that create a symbolic link of this file inside site-enabled:
(nginx)$ ln -s /usr/local/etc/nginx/sites-available/ghost /usr/local/etc/nginx/sites-enabled/ghost
Finally, enable nginx and start it:
(nginx)$ sysrc nginx_enable=YES
(nginx)$ service nginx start
Now exit from this Jail and check that Ghost, NGINX, MariaDB ports(2368, 80, 3306, respectively) are being bounded:
root@fbsdvm01:/jail $ sockstat -46l
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
www nginx 8646 6 tcp4 *:80 *:*
root nginx 8645 6 tcp4 *:80 *:*
marco node 8486 24 tcp4 127.0.0.1:8080 *:*
marco node 8135 34 tcp4 127.0.0.1:2368 *:*
mysql mariadbd 7715 19 tcp4 127.0.0.1:3306 *:*
root sendmail 747 3 tcp4 127.0.0.1:25 *:*
root sshd 744 3 tcp6 *:22 *:*
root sshd 744 4 tcp4 *:22 *:*
root syslogd 609 6 udp6 *:514 *:*
root syslogd 609 7 udp4 *:514 *:*
Now you can reach ghost at
http://<server_ip_address>/ghost
and complete the installation process. You basically have to set up a user account, and you are ready.
