Updating a Raspberry Pi home automation system
Monday, 24 July 2017
This weekend I carried out an overhaul of my Raspberry Pi-based home automation system. It has been more than four years since the system was first installed in March 2013. At that time, the Pi acted as a central heating controller linked via WiFi to the PCs in the house. Shortly afterwards I added the capability to control lights via a 433MHz transmitter. Later, a system was added to allow the heating system to be controlled from outside the house using a smartphone, the original control panel webpage being redesigned for mobile-friendliness.
However, the system was "stuck" on an old Linux kernel version which could not be upgraded easily. I worried about the problems that have hit similar network-connected systems - the "Internet of Things" - where security problems with smart lightbulbs, fridges and suchlike have serious consequences for their owners.
This weekend's update added the capability to update the kernel via package management. The device now runs the most recent official kernel from raspberrypi.org. This is a significant advance on building from source, since it's very easy to "apt upgrade" and automatically receive a newer kernel. A custom driver for the 433MHz hardware is built from source whenever the Raspberry Pi boots, and so it will always match the current kernel, even after an update.
It took me a while to figure out how to arrange this, so I thought I'd write down what I learned while doing it.
Raspbian kernels
My Raspberry Pi (1st generation, model A) runs Debian, more specifically Raspbian. Raspbian provides various Linux kernels for the Raspberry Pi 1, but all of them are hard to install, because all of them are very large - too large for the default size of the /boot partition, which tends to be 64Mb in most of the Pi installer images.
Even if there is space for the kernel and its "initrd" file, the process of building the "initrd" file consumes a lot of temporary space too. If it runs out of space in /boot, the package manager fails to install the new kernel. I hit this problem in particular with "linux-image-4.9.0-2-rpi" and with "linux-image-rpi-rpfv", both of which refer to the same official Raspbian kernel for the Pi 1.
This roadblock was puzzling, because the original Raspbian installation had not required any "initrd". Instead the kernel had been stored in a single file, "/boot/kernel.img", which was significantly smaller than any of the newer packages. Why would installing a new kernel suddenly create this new issue?
rpi-update kernel
It turns out that some Raspbian users aren't actually using APT to install and update the kernels at all. Instead they may use "rpi-update", a completely unrelated package manager, which gets the kernel and other firmware files from https://github.com/Hexxeh/rpi-firmware.
But I really wanted to use APT. I like having a single package manager for everything: a single update process for all files. Also, in order to build a custom kernel module, I need the header files which exactly match the kernel version. In practical terms this means they must come from the same place, with not only the same version number, but also the same configuration.
Raspberrypi.org kernel
I solved the problem by adding a second source of packages to /etc/apt/sources.list:
deb http://archive.raspberrypi.org/debian/ jessie main ui
This goes alongside the usual source for Raspbian packages:
deb http://mirrordirector.raspbian.org/raspbian/ jessie main
contrib non-free rpi
Having done this, you can "apt install raspberrypi-kernel".
The raspberrypi.org packages are compatible with Raspbian, but they're not part of it. The packages here seem to be very similar to those installed by rpi-update, but managed by APT, which makes it very simple to keep the kernel updated along with the other software.
You can also request "raspberrypi-kernel-headers", which installs the packages required to build custom kernel modules. This was the component I particularly needed in order to build the driver for my 433MHz transmitter.
Hardware upgrade
My original plan for overhauling this RPi involved a hardware upgrade to an RPi 2 or RPi 3. Both of these are more powerful than the RPi 1, with four CPU cores and more RAM, and with an RPi 3, you get built-in WiFi and Bluetooth! You also get the benefits of several years of design improvements. The board is smaller and neater. The RPi 1 looks quite old now.
Three things put me off this upgrade:
- The GPIO header is a different size (40 pins rather than 26), which would force me to physically rewire parts of my system, or remove some pins from the header.
- The RPi 3 comes with heatsinks(!) and cannot be underclocked. I don't know if the heatsinks are really required, but certainly my application does not need even 1% of the computing power of the original RPi, and there is no need for a more powerful system.
- If it isn't broken...
It is a little annoying to think that these problems will bite me in the future if the RPi 1 breaks down and has to be replaced. A "new" RPi 1 is already hard to find.
Software upgrade
Installing an OS onto a Raspberry Pi is simple in principle. Generally you download an image file and write it to an SD card with a program that can write to the SD card in raw mode. However, many distributions including Raspbian will require you to connect a screen and keyboard to the Pi in order to configure networking and get the Pi online. This is frustrating, particularly if your Pi is part of an embedded system and the ports are not readily accessible. Even when the Pi is on your desk, it's annoying to have to set up an additional monitor and a keyboard.
I found that the DietPi distribution was able to immediately boot into a working configuration complete with networking. It connects to your LAN via the Ethernet port on the Pi (if one is present) and can also be configured to connect via WiFi if you set the name and key for some access point, using a text file. This is good thinking and impressive. Shortly after booting, you can connect via SSH to finish setting up, changing the default root password as soon as possible! But DietPi did not make it easy to install the correct kernel headers, even though it seemed that the kernel was going to be automatically updated frequently, and so I gave up on it.
I had more success with the excellent Raspbian Unattended Network Installer. This might be the only Linux distribution for RPi that can be installed just from a zip file, with no need to write an image file to the SD card. You only need to create the boot partition (DOS format) and copy the right files into it. You can set your WiFi details using a text file or you can use an Ethernet cable. Once you've done this, you plug in the Pi and wait for the installer to finish the setup. It downloads the Raspbian software just like the netinst version of Debian on a PC. You can watch the lights on the network interface to see if it's actually doing anything. If it succeeds, you can connect via SSH. Of course, if it fails, it's very hard to know what's happened! The manual advises you to cross your fingers.
In the end I did not make use of either DietPi or the unattended installer. I just SSH'ed to my existing system, backed up the SD card, and did an in-place upgrade with APT. The only tricky part was transitioning to the new kernel, as user space files upgraded from Debian 7 (wheezy) to Debian 8 (jessie) without incident. For the kernel, it was a bit more difficult. I had to figure out that the Raspbian kernels weren't the right answer, and use "raspberrypi-kernel" instead. I also had to update the firmware files in the boot partition to support the new kernel version, and then add "dtparam=i2c_arm=1" to the config.txt file to enable I2C. None of this was particularly intuitive. I was caught out by the need for a firmware upgrade, and ended up having to connect a monitor in order to discover where the boot process had halted. As the machine was stuck on the "rainbow screen", I knew it was failing to start the kernel.
Closing thoughts
In the time since I set up my homebrew home automation system, many of its features have become parts of commercial products. Heating systems may be controlled by Hive or Nest, while WiFi-connected lightbulbs have proliferated. In the 90s, nobody outside of a CS department would try to connect a fridge to the Internet. Now it's totally normal. It is the "Internet of Things", now notorious for (1) security issues, and (2) dependence on Internet connections for basic functionality. My own system does not have the second problem - the whole thing works offline - and I hope that the first problem will be minimised by regular software updates. I don't have any plans to replace it with a commercial product. The flexibility and extensibility are just too useful, and I think no commercial product could possibly come close to that.