Sunday, 15 October 2017

Notifications, from the command line to Android

Some time ago I started writing a list of software tools that I would like. It is said that one of the nice things about working in the software business is that you get to make your own tools. But you have to remember what problems you hit, and what tool would have helped you to solve them. So, I have taken to keeping a sort of wish list.

I found that a few wish list items involved a single feature, which was the ability to send a notification from a command-line program to me, wherever I happened to be. For instance, I might like to know that "make" has completed on some computer. So I'd like to be able to enter:

$ make ; notify_jack 'Make completed'

When the "make" program completes I want to see the message "Make completed" as a desktop notification on my PC, and as an Android notification on my phone. I want the phone to ping or vibrate, I want the PC to bleep. The general idea is that the machines tell me that something has happened and needs my attention.

After a prototype implementation that used email, I settled on a version that uses Trello, the cloud todo list system. Normally Trello is used through an app or the website but the system also has an API which allows programs to control the todo lists. For instance a program can move items from one list, e.g. "Running", to another list, e.g. "Done". Trello also allows you to subscribe to an item and receive a notification when it moves. This notification goes to every mobile device running Trello, and every desktop PC which happens to have the website open.

Android notification
Windows 10 notification

These two features combine to provide everything I need. I used a Python interface to the Trello API to build a small Python program which runs on the command line. It connects to the service and carries out a few simple tasks, which can trigger notifications wherever I happen to be.

In Trello terminology, items are called "cards" and have some associated metadata, e.g. a text description. A collection of lists is a "board". My Python program is able to do the following:

  1. Move a card to a specific list.
  2. Move a card to a "Running" list, run another program, and then move the card to a "Done" list, editing the text on the card to reflect the results of running the program.
  3. Scan a list for cards that haven't moved within a time interval written on the card, and move them to another list.

Each of these three features corresponds to a wish list item. The first feature allows the Python program to notify me of a specific event. This is the "make" use case illustrated above. Various events in my house will also trigger notifications of this sort - for instance, when the washing machine has finished work, the "Washing" card moves to the "Home" list, telling me to unload it.

The second feature is useful for running automated builds and tests, since it indicates both that the process has started and that the process has finished. I can check the result (exit code) without looking at the PC. As the script is written in Python it is portable across all of the systems I regularly use.

The third feature allows me to set up recurring reminders. For instance a repeat prescription becomes due for reordering every 8 weeks; so, every 8 weeks, the card representing the prescription is moved automatically to the "Home" list, telling me to go through the manual process of reordering a repeat prescription*.

Here is part of the board:

In this screenshot I have a build in progress (in the "Running" list). I have some jobs to do at "Home". But I have already "Done" some of them. Off to the right, further lists exist for jobs at "Work".

If you like, you can download my program here. It requires the Python "requests" package (which you need to install yourself) and the Trello API package, which I have bundled with the source code (it is a snapshot from here). To use it, you need to edit the Python script to set your Trello credentials - there are some instructions for doing this.

The script can create new cards, but it's best to create cards yourself with the Trello website or app, because this allows you to subscribe to them and receive notifications when they move. By default, notifications are sent by email - it may be worth turning this off, so you just get them as notifications on your desktop and phone.

Before I hit on the idea of using Trello, I tried to build the same system using email. New emails already result in notifications, and it's already possible to send email from the command line, e.g. with the "mail" program which has been part of the Unix toolbox for decades. But actually, it usually isn't possible to send email from the command line, because this requires a lot of setup to configure how the email is sent out onto the Internet, which is hard because virtually all Internet mail services will reject mail that they suspect is spam. Emails sent from a Linux PC are likely to be flagged as spam. They come from a machine on a private network, whose hostname does not match its IP address. It is also hard to hook into an office email system, because that system requires authentication. Would you hardcode your email server password into a configuration file? Would you beg the IT administrator for a special exception allowing your machine to send email without authentication? This is not practical.

I found that the email approach would work if I sent the messages via a service running on a system which is connected to the Internet, has a non-private IP address and correctly configured DNS entries. So, the first generation of the notification process would log into my jwhitham.org server and run "mail" there. However, I found that email wasn't really an appropriate way to send notifications. My inbox filled up with tiny messages and it was annoying to have to clear them out.

I considered writing an Android app to display notifications, but I thought of all the work required to do this - it is not just about the app, but also the way to get notifications to it: lots of work is needed on a server too. Reusing existing technology was clearly preferable to all of that work. (I still have relapses into "not invented here"-ism, but there is not much fun to be had in recreating something that someone else already created.)

I find that the Trello system is well suited to the problem. The Python interface makes it easy to write programs to control the cards, all of the server-side components are already in place, and the desktop and Android apps already exist. I find it particularly useful that I only get a message if a card actually moves, and then only if I'm subscribed to it. I can arrange cards and lists however I like, because there's no fundamental incompatibility between the way that the script uses the board and the way that I use it. The script doesn't do much - it just finds one card and moves it. Turns out, that's all that's needed.

* Footnote: an electronic repeat prescription system was introduced by the NHS for this purpose, but it was extremely poor, and mistakes were frequent - orders were sometimes not placed, and when they were, you would often find that medication was missing and you would then have to chase up the problem. Generally the pharmacy would blame the doctor's surgery. I jumped at the chance to go back to the paper system which allows me to manually check each stage of the process and correct mistakes rapidly.