Marco Cetica
Home / About / Contact

CREATE AN EMBEDDED LINUX SYSTEM USING BUILDROOT

2019-06-01
This tutorial will walk you through the process of creating a small Linux distribution for ARM-based embedded devices(e.g. Raspberry Pi) using Buildroot. At the end of the article you will be able to run your own Linux system into a Raspberry Pi 3/4/ZeroW.

What's Buildroot?

Buildroot is a set of tools that will help you to easily generate an embedded Linux distribution for your IoT device. This project is driven by the community, and it's fully open source.

Features of Buildroot

Buildroot is the perfect choice for the majority of your embedded applications. It makes the process of creating and embedded system very easy thanks to the kernel like menuconfig, the interactive download scripts and the overlay system.

Buildroot is designed in such a way to handle everything by itself, you just have to choose what to include in your system and Buildroot will download, configure, compile and install everything without any additional configuration.

With just one command(and about ~30 minutes of your time) you will get a flashable image! Amazing, isn't? Despite my enthusiasm for Buildroot, I also need to mention the main alternative to this project: Yocto.

Differences between Buildroot and Yocto

Both Buildroot and Yocto are building systems for embedded devices, this means that both of them will generate a root filesystem, a bootloader, a kernel and a basic toolchain for your target architecture. These two projects, therefore, differ on the approach of how things are handled: Buildroot focuses on simplicity and minimalism while Yocto focuses on versatility and the ability of creating a real Linux distribution(for instance with a dedicated package manager).

On the other hand, Buildroot creates a plain and simple firmware that has to be re-compiled every time you want to update something, this approach could lead to problems in some large-scale embedded applications where you need to deliver updates very frequently.

Apart from that, Yocto supports a much wider set of packages(about 8000 for Yocto vs about 1800 for Buildroot), meaning that, for a lot of applications, Buildroot will force you to write Kconfig files to integrate missing packages into the target system.

To summarize, if you need to build a complex embedded network which will requires remote updates without the need to rebuild the whole system from scratch, you should look at Yocto. However, if you just want to build a firmware for a small set of embedded devices that probably won't receive updates in the nearly future, you should consider Buildroot.

Last but not least, Yocto has a very steep learning curve compared to Buildroot. If you want to read more about the differences between these two building systems, visit this reousrce.

Prerequisites

In order to build an embedded Linux image, you will need: For a full list of updated requirements, visit this resource. You should also be a proficient UNIX/Linux user, also you should be able to configure a development environment by yourself.

Downloading Buildroot

You can obtain Buildroot through three different methods: At the time of writing this article, I'm using the stable release (2020.08.2) but you can use whatever you want. Buildroot can be downloaded from this link.

Set up your target architecture

Once you've downloaded one of the available releases, extract the archive file and then search your architecture. By default, Buildroot store supported architecture into the configs/ directory. For instance, since we are building a Linux image for a raspberry Pi Zero, we should use one of the following config files: 

$> ls -lh configs | grep raspberry
.rw-r--r-- marco marco  1.1 KB Mon Jun 16 23:13:14 2019 raspberrypi0_defconfig
.rw-r--r-- marco marco    1 KB Mon Jun 16 23:13:14 2019 raspberrypi0w_defconfig
.rw-r--r-- marco marco  1.1 KB Mon Jun 16 23:13:14 2019 raspberrypi2_defconfig
.rw-r--r-- marco marco  1.2 KB Mon Jun 16 23:13:14 2019 raspberrypi3_64_defconfig
.rw-r--r-- marco marco  1.1 KB Mon Jun 16 23:13:14 2019 raspberrypi3_defconfig
.rw-r--r-- marco marco  1.5 KB Mon Jun 16 23:13:14 2019 raspberrypi3_qt5we_defconfig
.rw-r--r-- marco marco  1.1 KB Mon Jun 16 23:13:14 2019 raspberrypi4_64_defconfig
.rw-r--r-- marco marco  1.1 KB Mon Jun 16 23:13:14 2019 raspberrypi4_defconfig
.rw-r--r-- marco marco  1.1 KB Mon Jun 16 23:13:14 2019 raspberrypi_defconfig
To select raspberrypi0w_defconfig, type make raspberrypi0w_defconfig.

Apart from that, the other relevant folders are:

System setup

Now that we have chosen the target platform, it's time to configure our system. But before that, we need to set up a cross-compiler(your computer is probably an x86_64 machine, while Raspberry Pi uses an ARM CPU) and a compatible version of binutils. That seems difficult and time-consuming, right? Well, not for Buildroot! You just have to run >make all and this fantastic tool will do everything by itself. Depending on your hardware configuration, this should take not more than 20-30 minutes. Also, note that Buildroot will automatically detect how many cores your CPU has, so there is no need to pass the -j flag to the make all command. Once done that, you can run make menuconfig to open the interactive Buildroot's menu. You should see something like this: menu config home As you can see, this is the usual configuration menu used to configure the Linux kernel. If you have already seen this menu before, you should know how useful and well organized is in contrast to manual file editing. Moving the cursor with your keyboard arrows, you can enter the various sub-menus and select what you want to include. But since this could be your first time using Buildroot and you might be afraid by the amount of options available, I've made a list of the minimum required by a Raspberry Pi Zero to be usable(for instance Wi-Fi support, an OpenSSH server, basic POSIX utilities, etc.)

Build options

Inside the build options menu I suggest you to enable the compiler cache to speed up(by a lot) the compiling process: 

Build options -> Enable compiler cache (BR2_CCACHE)

Toolchain

Inside toolchain enable WCHAR support, this is need by some packages such as bash or vim: 

Toolchain -> Enable WCHAR support (BR2_TOOLCHAIN_BUILDROOT_WCHAR)

System configuration

Inside system configuration sub menu you can customize your Linux system by changing the default hostname, the system banner, the default root password and the default shell(I suggest you to replace the raw busybox default shell into something more polished such as bash): 

System configuration -> System hostname (BR2_TARGET_GENERIC_HOSTNAME)
System configuration -> System banner (BR2_TARGET_GENERIC_ISSUE)
System configuration -> Root password (BR2_TARGET_GENERIC_ROOT_PASSWD)
System configuration -> /bin/sh (busybox' default shell)  ---> (x) Bash
Note that, in order to change the default shell, you may need to enable the BR2_PACKAGE_BUSYBOX_SHOW_OTHERS flag inside Target packages sub menu.

Target packages

The target packages menu allows you to install all the packages supported by Buildroot. Below, there is a list of some basic packages that you should find in any Raspberry-oriented distribution; you can however exclude anything that seems unnecessary for your project. 

Target packages -> Shell and utilities -> bash completion (BR2_PACKAGE_BASH_COMPLETION)
Target packages -> Shell and utilities -> sudo (BR2_PACKAGE_SUDO)
Target packages -> Shell and utilities -> time (BR2_PACKAGE_TIME)

Target packages -> Hardware handling -> Firmware -> rpi-wifi-firmware (BR2_PACKAGE_RPI_WIFI_FIRMWARE)

Target packages -> Interpreter languages and scripting -> Python3 (BR2_PACKAGE_PYTHON3)

Target packages -> Networking applications -> wget (BR2_PACKAGE_WGET)
Target packages -> Networking applications -> wpa_supplicant (BR2_PACKAGE_WPA_SUPPLICANT)
Target packages -> Networking applications -> wpa_supplicant -> Enable nl80211 support (BR2_PACKAGE_WPA_SUPPLICANT_NL80211)
Target packages -> Networking applications -> wpa_supplicant -> Enable AP mode (BR2_PACKAGE_WPA_SUPPLICANT_AP_SUPPORT)
Target packages -> Networking applications -> wpa_supplicant -> Enable autoscan (BR2_PACKAGE_WPA_SUPPLICANT_AUTOSCAN)
Target packages -> Networking applications -> wpa_supplicant -> Enable WPA3 support (BR2_PACKAGE_WPA_SUPPLICANT_WPA3)
Target packages -> Networking applications -> wpa_supplicant -> Install wpa_cli binary (BR2_PACKAGE_WPA_SUPPLICANT_CLI)
Target packages -> Networking applications -> wpa_supplicant -> Install wpa_client shared library (BR2_PACKAGE_WPA_SUPPLICANT_WPA_CLIENT_SO)
Target packages -> Networking applications -> wpa_supplicant -> Install wpa_passphrase binary (BR2_PACKAGE_WPA_SUPPLICANT_PASSPHRASE)
Target packages -> Networking applications -> iptables (BR2_PACKAGE_IPTABLES)
Target packages -> Networking applications -> ntp (BR2_PACKAGE_NTP)
Target packages -> Networking applications -> ntp -> sntp (BR2_PACKAGE_NTP_SNTP)

Target packages -> Libraries -> Networking -> libcurl -> curl binary (BR2_PACKAGE_LIBCURL_CURL)

Target packages -> System Tools -> htop (BR2_PACKAGE_HTOP)

Target packages -> Text editors and viewers -> vim (BR2_PACKAGE_VIM)

Target packages -> Hardware handling -> rng-tools (BR2_PACKAGE_RNG_TOOLS)
Target packages -> Hardware handling -> NIST Entropy Beacon support (BR2_PACKAGE_RNG_TOOLS_NISTBEACON)

Target packages -> Libraries -> Crypto -> gnutls (BR2_PACKAGE_GNUTLS)
Target packages -> Libraries -> Crypto -> gnutls -> OpenSSL compatibility library (BR2_PACKAGE_GNUTLS_OPENSSL)
Target packages -> Libraries -> Crypto -> gnutls -> install tools (BR2_PACKAGE_GNUTLS_TOOLS)
Target packages -> Libraries -> Crypto -> CA Certificates
(BR2_PACKAGE_CA_CERTIFICATES)

Framebuffer logo

framebuffer cover Have you ever noticed the logo at the top of every tty that represent the number of cores of a CPU? This is called framebuffer logo and while you can ignore it and leave the default one(or disable it), it's cool to have a custom one. Furthermore, you may need to replace the Raspbian's logo with that one from your company. Doing this manually, was usually a tedious task to do, thanks to Buildroot, however, this process is very quick and do not require weird format conversion. You just need to place a 64x64 PNG or JPEG picture with black background into the root of your Buildroot directory and to edit BR2_LINUX_KERNEL_CUSTOM_LOGO_PATH flag's value from Buildroot menuconfig with the absolute path of the picture. That's it!

Once you are satisfied with your configuration, save the configuration file and then exit from menuconfig by hitting ctrl+c.

Kernel configuration

Another cool thing about Buildroot is that you can manually configure the Linux kernel to include(or remove) anything not enabled by default. To do that, you just have to run make linux-menuconfig to open another menuconfig menu. It's useful to mention that for a lot of applications, there's no need to touch the kernel: it will just work with the default configuration. But just for the purpose of this tutorial, let's see how we can remove something; for instance, since I do not plan to use audio in my Raspberry Pi Zero, let's remove ALSA support (CONFIG_SND): ALSA support Once done that, run again make all and wait while Buildroot download, compile and install everything we selected so far.

Init system

As you may have noticed, during the whole system configuration, we had not talked about one of the most crucial part of an operating system: the init system. In particular, I said nothing about one of the most common init system available for GNU/Linux: systemd. While you can easily change the default init system from System configuration -> Init system, I think that the fastest and simplest init system for an embedded system with little performances is the one shipped with Busybox. Don't get me wrong, this is not the usual no-systemd war that goes on since its release; I like systemd and many of its features, but for an embedded application, nothing beat the minimalism and simplicity of Busybox'ss init. Furthermore, systemd requires a different file system skeleton(which is different from Buildroot's default) and some large dependencies such as dbus and udev. Therefore, my advice is to stick to the Busybox default init system and to not use systemd unless your embedded project relies on some of its features(such as cgroups, namespaces or SELinux).

Configure the image: overlay filesystem

Even if you have selected all the packages needed by your embedded project, you still haven't configured a single thing. You can indeed configure it once installed, but what if your clients ask you to ship your Linux system with a particular configuration of SSH or with a particular netfilter set of rules? To achieve this, Buildroot offers a brilliant way to handle configuration files and startup scripts: overlays! An overlay is nothing more than a simple directory that acts like a root for your system. Inside this directory you can put all the files you want to be copied at building time inside your embedded distro. For instance to copy a file inside the /etc/ssh directory of your distribution, you just have to create the following structure: 

overlayfs/
`-- etc/
`-- ssh/
`-- sshd_config

2 directories, 1 file
Buildroot will copy this file into the image according to the directory structure you have created. The only thing you have to do, is to enable it; search for the BR2_ROOTFS_OVERLAY flag inside the Buildroot menu and specify the absolute path of your overlay folder(it can be placed everywhere you want, but I'm suggesting you to place inside the same folder of Buildroot). Now that Buildroot has register the path of the overlay, let's configure our system a bit.

OpenSSH server

By default, OpenSSH do not allow remote root login, but if you need it at the first boot for any reason, you can enable it by specifying PermitRootLogin yes on the sshd_config file. You can also configure the listening port and other security settings here. 

mkdir -p $OVERLAY_FS/etc/ssh
cp -R output/target/etc/ssh/sshd_config $OVERLAY_FS/etc/ssh
vim $OVERLAY_FS/etc/ssh/sshd_config # Edit ssh server
Default SSH init script kills active connection during restarts actions (i.e. /etc/init.d/S50sshd restart), to avoid this, let us create the following folders inside the overlay 

mkdir -p $OVERLAY_FS/etc/init.d
cp -R output/target/etc/init.d/S50sshd $OVERLAY_FS/etc/init.d
And then edit the SSH configuration file(vim $OVERLAY_FS/etc/init.d/S50sshd) by editing the following lines: 

...
20 stop() {
21      printf "Stopping sshd: "
22      killall sshd
23      rm -f /var/lock/sshd
24      echo "OK"
25 }
26 restart() {
27      kill -HUP $(cat /var/run/sshd.pid)
28      echo "OK"
29 }
In that way, SSH daemon will be restarted without killing out active sessions.

Wi-Fi connection

By default, not only Wi-Fi will not work(since no network has been configured yet), but you won't also be able to identify the onboard Raspberry Pi's Wi-Fi card. To fix this problem we first need to load the kernel module at boot time: 

cp -R output/target/etc/inittab $OVERLAY_FS/etc
vim $OVERLAY_FS/etc/inittab # Edit inittab file
Add this: 

16 # Startup the system
17 ::sysinit:/bin/mount -t proc proc /proc
18 ::sysinit:/bin/mount -o remount,rw /
19 ::sysinit:/bin/mkdir -p /dev/pts /dev/shm
20 ::sysinit:/bin/mount -a
21 ::sysinit:/sbin/swapon -a
22 null::sysinit:/bin/ln -sf /proc/self/fd /dev/fd
23 null::sysinit:/bin/ln -sf /proc/self/fd/0 /dev/stdin
24 null::sysinit:/bin/ln -sf /proc/self/fd/1 /dev/stdout
25 null::sysinit:/bin/ln -sf /proc/self/fd/2 /dev/stderr
26 ::sysinit:/bin/hostname -F /etc/hostname
27 ::sysinit:/sbin/modprobe brcmfmac
28 # now run any rc scripts
29 ::sysinit:/etc/init.d/rcS
and then, configure the network interface: 

mkdir -p $OVERLAY_FS/etc/network
cp -R output/target/etc/network/interfaces $OVERLAY_FS/etc/network
vim $OVERLAY_FS/etc/network/interfaces # Edit the interfaces
Add this: 

# interface file auto-generated by buildroot

auto lo
iface lo inet loopback

auto wlan0
iface wlan0 inet dhcp
pre-up wpa_supplicant -B -Dnl80211 -iwlan0 -c/etc/wpa_supplicant.conf
post-down killall -q wpa_supplicant
wait-delay 15
udhcpc_opts -t 10
iface default inet dhcp
That's it. At the next boot, the system will load the Wi-Fi module, configure the interface, execute the wpa_supplicant daemon using /etc/wpa_supplicant.conf as a configuration file and, finally, ask for an IP address. These are the settings I thought were essentials but if you need anything else, just include it. It'ss not difficult after all.

Serial communication

Debugging an embedded system can be very tricky. When you have to figure out which part of your system does not work, you cannot also deal with keyboard, video output, ethernet/wireless connection and so on. In such cases, you must have an UART(Universal Asynchronous Receiver-Transmitter) device: a little piece of hardware that allows you to communicate to a digital device. UART is a really simple and old protocol so, if you have already debugged a microcontroller before, chances are that you already own one of these thing. By the way, if you have never used one of them before, I suggest you to look to FT232H, the FT232H chip also supports a wide range of serial protocols, such as SPI, I2C, JTAG and many more. Whatever you choose to use, you will need to connect the RX and the DX pins and the usual VCC and GND to power on your Raspberry. If you plan to use an external power supply, you do not need to connect the VCC pin. At the end, you should have the following circuit: PCB scheme Now that the circuit is configured, connect the UART device to your computer using the USB port, open a terminal and run the following command: 

$> screen /dev/ttyUSBX 115200
on GNU/Linux systems 

$> screen /dev/tty.usbserial-XXXXX 115200
on macOS and *BSD In both cases, replace the Xs with the correct device(just use the tab key). If you're on Windows, you can use GNU screen through PuTTY or by using WSL. The second parameter of the command is the Baud rate(the number of symbols per seconds) and, if you have not changed it from the Buildroot's menu, should be fixed at 115200.

Initial configuration

Once serial communication is ready and the device is powered on, you should see a bunch of system logs starting to fall down from the top of your terminal. booting GIF Once Linux has booted, and it has opened a getty on the serial port for you, you should be able to log in as root with the password you have set in Buildroot. Note that, the first time you boot up a fresh image, it may take a bit longer. This is normal, in fact the system has to generate the SSH keys using data from the random pool; also, it must fail the Wi-Fi connection since no network has been configured yet(in our settings, udhcpc will try to obtain an IP address for no more than 10 seconds).

Network setup

The Raspberry Pi Zero does not have an Ethernet port, the only way to connect it to the network is through Wi-Fi. Since we configured the system to automatically load the Wireless driver, you should be able to see the network interface using ip a. To add a new network using wpa_passphrase utility, use the following command: 

wpa_passphrase "SSID_NAME" "NETWORK_PW" >> /etc/wpa_supplicant.conf
Be sure to delete any other occurrence of the network entry inside the wpa_supplicant.conf file.At the end you should get something like this: 

ctrl_interface=/var/run/wpa_supplicant
ap_scan=1

network={
     ssid="Router1"
     #psk="a_bad_pw"
     psk=XXX
}
After a reboot, the Raspberry should be able to connect to the network by itself. To retrieve its local IP address you can use this ugly chain of commands: 

ip a | grep wlan0 | grep inet | awk '{print $2}' | sed '$s/...$//'
At this point you can reach your Raspberry using SSH!

Adding a new user

It is not recommended to use the root account for common system usage, you are advised to add a new account: 

mkdir -p /home
adduser -G users -s /bin/bash marco
Changing password for marco
New password:
Retype password:
passwd: password for marco changed by root
Let's give him superuser privileges by adding his name to sudoers(visudo): 

##
## Runas alias specification
##

##
## User privilege specification
##
root ALL=(ALL) ALL
marco ALL=(ALL) ALL
## Uncomment to allow members of group wheel to execute any command
# %wheel ALL=(ALL) ALL

## Same thing without a password
# %wheel ALL=(ALL) NOPASSWD: ALL

## Uncomment to allow members of group sudo to execute any command
%sudo   ALL=(ALL) ALL

## Uncomment to allow any user to run sudo if they know the password
## of the user they are running the command as (root by default).
-- INSERT --                                                  80,20         91%

SSH Server(Again)

The default SSH configuration we made at the beginning allowed remote root login, however, for security purposes, you should disable it. To do that, just change PermitRootLogin back to no in /etc/ssh/sshd_config. After that, restart the daemon by running /etc/init.d/S50sshd restart, your active connection should not be killed.

Conclusion

Final result photo Now your brand new embedded Linux system should be ready! Keep in mind that this tutorial should be seen as an introduction to Buildroot, in fact there are a lot more things to do in order to create something usable and secure. However, I think that this should be a pretty solid starting point. You now might be wondering why you should craft your ARM Linux by hand when there are plenty of pre-compiled solution available for free with many more packages and a much wider community. To be fair, if you are asking yourself this question, then you probably should use Arch Linux ARM, Raspbian or Fedora ARM. There is no need to build something like this for the majority of your Raspberry projects, however there are at least two reason of why you should definitely try it out: