81. Building the grandma videoconf

Grandma wants to talk to her grandkids and also see them. The problem is that grandma cannot interact with modern technology at all. No keyboard, no mouse, no touchscreen—the solution has to be fully automated. Let’s build this.

Front view

Back view

Hardware discussion

Like everything else with this project, the hardware has to be outwardly simple. The ideal would be a tablet, except that modern tablets require far too much user interaction.

For example, the previous solution was an Android tablet with Signal installed. Grandma’s main problem was that she could not press the smallish “answer” button on incoming calls. But even if we fixed that with an app that’s more user-friendly than Signal, the other problems would still remain:

• The tablet sometimes pops ups notification dialogs about updates and other things. Grandma cannot read them and cannot even see the “Ok” button to dismiss them.
• The tablet sometimes has a lag in turning on its screen. This leads to grandma pressing on the power button for too long until the tablet turns off. Then the tablet takes minutes to start back up.
• Grandma sometimes presses the volume buttons when picking up the tablet and mutes it. The buttons are then too small for her to see or feel out easily.

I considered using a tablet and replacing the stock Android with something else, but everything in the mobile space seems to be very “special” for lack of a better word. I would rather use the same tooling I use for everything else, so the device we’re building has to a PC.

The canonical “small PC” is a Raspberry Pi. The problem with Raspberry Pis is that they’re mobile-adjacent so special in their own ways. For example, here is the NixOS Wiki page on the RaspberryPi 4. It lists a lot of little things you need to configure to get everything working. It’s a lot of friction.

The bigger problem with Raspberry Pis is that they run off of SD cards by default. If you leave one turned on for long enough, you discover that the SD card degrades over about 2-3 months to the point where the system won’t boot. It turns out ext4 on an SD card is a bad idea. You can try using something like F2FS, but now you have to figure out how to make a bootable SD card with that, so again, friction.

Raspberry Pi 5’s are much better at not being special. You can even boot them from NVMe flash drives! But you have to buy a separate NVMe shield, and an NVMe drive, not to mention the power adapter. Oh, and you probably want a heatsink and a fan too.

Hardware choices

If I were building something for myself, a Raspberry Pi 5 is probably what I would use. However, what we’re building here needs to work without intervention for months at a time, so a “stick” form-factor PC is the way to go. There aren’t many choices if we want a recent-ish processor, but this DreamQuest one fits the bill. It features an Intel N95 released in 2023, 12 GB of RAM, and a SATA SSD. It has 4 USB type-A ports, 2 type-C ports (one of which is the power port), an Ethernet port, 2 HDMI ports, and even a headphone jack. Weight-wise, it feels a bit heavier than a mobile phone.

DreamQuest stick PC compared to a mobile phone

For the monitor, we want something small-ish so that it fits on grandma’s coffee table. We also want it to be light enough that she can move it around. I went with the Raspberry Pi monitor. It’s 15.6’ which is a bit big, but it’s USB powered so it doesn’t need an extra power cable, and weighs only about 1 kg. It has built-in loudspeakers, so that’s one less component to worry about. It has a flap on the back which means it doesn’t need a dedicated stand.

Raspberry Pi monitor

The remaining components are any USB webcam and a bunch of cables. I used some I had lying around. The PC cost £200, the monitor was £100, a new webcam would’ve cost about £50, and let’s say the cables were £50. This brings the cost of the whole setup to £400 which is about the same as a mid-range tablet.

Base system

With the hardware settled, we now turn to software. As a reminder, the primary goal is for the setup to be completely hands free—grandma can’t interact with the device in any meaningful way, so there must never be a “System update ready. Install?” prompt. In other words, we need everything to be remotely manageable.

Since I expect I’ll have to build this several times, a secondary goal of mine is for the setup to be repeatable. As such, we’re using NixOS and Home Manager because this allows us to configure everything through files. We do remote deployments with Colmena because it’s what I’m familiar with (not that it matters much since all the NixOS deployment tools are essentially interchangeable).

The full configuration is available here. Below, we’re only going to look at the bits that make this device different from a regular desktop.

First, we need to auto-login into a graphical environment. We do this trivially with getty:

services.getty = {
  autologinUser = "auto";
  autologinOnce = false;
};
environment.loginShellInit = ''
  [[ "$(tty)" == /dev/tty1 ]] && exec dbus-run-session sway
'';

For the desktop manager, I picked Sway because we can configure it through a single file, it’s well supported, and it’s Wayland so we don’t have to deal with any X11 jank. The configuration is basically the default one with the status bar commented out, window borders turned off, and some programs started automatically.

programs.sway.enable = true;
home-manager.users.auto =
  { ... }:
  {
    # …
    home.file.sway-config = {
      source = ./dor-qws-vid1/sway-config;
      target = ".config/sway/config";
    };
  };

# … default config

# bar {
# }

default_border none

output * bg /home/auto/Documents/wallpapers/forest-1.jpg fill
exec /run/current-system/sw/bin/wpaperd -d

exec wayvnc 127.0.0.1 5900

include /etc/sway/config.d/*

We set the background to a nice photo of a forest and we start wpaperd to change it every few minutes.

environment.systemPackages = with pkgs; [
  # …
  wpaperd
];
home-manager.users.auto =
  { ... }:
  {
    # …
    home.file.wpaperd-config = {
      source = ./dor-qws-vid1/wpaperd-config.toml;
      target = ".config/wpaperd/config.toml";
    };
  };

[default]
path = "/home/auto/Documents/wallpapers"
duration = "5m"
transition-time = 500

Next, we want to be able to control screen brightness programmatically so that grandma doesn’t have to find the physical buttons on the back of the monitor. The way to do this is with ddcutil and it’s a bit arcane, but it does work consistently.

environment.systemPackages = with pkgs; [
  # …
  ddcutil
];

# Brightness control
# https://wiki.nixos.org/wiki/Backlight#Via_ddcutil
# Min: `ddcutil --bus=0 setvcp 10 0`
# Max: `ddcutil --bus=0 setvcp 10 60`
hardware.i2c.enable = true;

A command like ddcutil --bus=0 setvcp 10 60 sets the brightness to 60%. The --bus parameter has to be determined experimentally, but ddcutil detect can probably guess it. The setvcp 10 bit is the magic code to control brightness. The last number is the brightness value and it only seems to do anything between 0 and 60 on this screen.

Finally, we want to turn off the screen at night so that it doesn’t disturb grandma’s sleep. We can do this by sending Sway a command every morning and every evening.

let
  swaymsgService =
    { time, cmd }:
    {
      serviceConfig = {
        Type = "oneshot";
        WorkingDirectory = "/home/auto/";
      };
      startAt = "*-*-* ${time}";
      script = ''
        if [[ "$(whoami)" = "auto" ]]; then
          export SWAYSOCK="$(ls /run/user/1000/sway-*)"
          ${pkgs.sway}/bin/swaymsg "${cmd}"
        else
          echo "Not auto user"
        fi
      '';
    };
in
{
  # …
  # Turn off screen during the night
  #
  # IMPORTANT: The timers need to be manually `systemctl enable`'d for
  # them to actually start.
  systemd.user.services.turn-off-screen = swaymsgService {
    time = "20:00:00";
    cmd = "output * power off";
  };
  systemd.user.services.turn-on-screen = swaymsgService {
    time = "08:00:00";
    cmd = "output * power on";
  };
}

Networking

With the base system in place, we can focus on networking. The most important requirement here is for the machine to connect to the Internet out of the box and for us to be able to ssh into it regardless of what NAT it’s placed behind.

First, we configure networkd to use the wired connection if it’s available.

networking.useNetworkd = true;
systemd.network.enable = true;
systemd.network.networks."10-lan" = {
  matchConfig.Name = "enp1s0";
  networkConfig.DHCP = "yes";
};
# Don't stall the boot if a cable isn't connected.
systemd.network.wait-online.enable = false;

Next, we enable NetworkManager for wireless connectivity. We have to use nmtui to pre-configure the WiFi password. This could be done with flat files, but since this part is going to be different for every iteration of this device, I’m not going to bother.

# Use `nmtui` to configure the Wi-Fi networks.
networking.networkmanager = {
  enable = true;
  wifi = {
    powersave = false;
  };
};

SSH

Now comes the important bit: we need a way to ssh into this machine. We could setup a VPN, but in my experience, they tend to break unexpectedly when combined with residential networking. Instead, we’ll use plain old ssh reverse tunnels.

The way this works is that our videoconf device ssh’s into a “middleman” host. When we want to ssh into the videoconf, we first ssh into the middleman, then ssh from there into the videoconf.

The OpenSSH option for reverse tunnels is -R. We basically want the videoconf to be running ssh -N -R 32022:localhost:2022 middleman at all times. In this command, 2022 is the videoconf’s ssh port and 32022 is a random port on the middleman host.

The key part of the above is “at all times”, so we need several more configuration options:

• We want to connect even if the middleman host gets rebuilt with a new SSH keypair. So, we set UserKnownHostsFile to /dev/null and StrictHostKeyChecking to no.
• If the videoconf’s connection to the middleman is interrupted in any way, we want to close the tunnel and then recreate it. The videoconf can detect this happening with the ServerAliveInterval and ServerAliveCountMax options.
• We also want the middleman host to detect if the connection drops, so we add ClientAliveInterval and ClientAliveCountMax to its OpenSSH config.
• We want the ssh command to exit if the tunnel fails somehow (e.g. if the port is already bound on the middleman host). This is what the ExitOnForwardFailure option does.

So, the full command looks like:

# ssh \
    -o "UserKnownHostsFile /dev/null" \
    -o "StrictHostKeyChecking no" \
    -o "ServerAliveInterval 30" \
    -o "ServerAliveCountMax 3" \
    -o "ExitOnForwardFailure yes" \
    -o "ControlMaster no" \
    -N \
    -i /root/.ssh/id_ed25519 \
    -p 2022 \
    -R 32022:localhost:2022 \
    mid@mid.abstractbinary.org

Putting this all together, we first add a new mid user to the middleman host:

users.users.mid = {
  isSystemUser = true;
  createHome = true;
  home = "/home/mid";
  group = "mid";
  openssh.authorizedKeys.keyFiles =
    config.users.users.root.openssh.authorizedKeys.keyFiles ++ cfg.extraKeyFiles;

  # Unless set, the default shell is `nologin` which allows tunnels
  # to be formed, but doesn't allow the user to login or run commands.
  # shell = "${pkgs.coreutils}/bin/true";
};
users.groups.mid = { };
services.openssh.settings = {
  ClientAliveInterval = 30;
  ClientAliveCountMax = 3;
};

Then, we configure services.autossh-ng on the videoconf to create the tunnel at boot and recreate it when it fails:

services.autossh-ng.sessions = {
  # …
  forward-ssh = {
    user = "root";
    destination = "mid@mid.abstractbinary.org";
    extraArguments = "-i /root/.ssh/id_ed25519 -p 2022 -o \"ControlMaster no\" -R 32022:localhost:2022";
    hostKeyChecking = false;
    knownHostsFile = "/dev/null";
  };
};

Finally, we use the ProxyJump setting in our ~/.ssh/config to tell SSH to go via the middleman:

Host via-mid-dor-qws-vid1
  ProxyJump mid.abstractbinary.org:2022
  HostName localhost
  Port 32022
  User root

Now, we can just use ssh via-mid-dor-qws-vid1 to connect to the videoconf. The fact that this is going through an intermediate host is abstracted from us.

VNC

With the above in place, we can reliably ssh into the videoconf. Strictly speaking, this is enough, but I’ve done enough tech support for older relatives to know that it is very useful to be able to see their screens.

Our two options are VNC and Remote Desktop. The later is the more modern protocol and generally works better over bad connections, but all the RDP servers available in NixOS seem like a hassle to setup. So, we just use wayvnc.

It’s so easy to setup that we’ve already started it in the Sway section:

exec wayvnc 127.0.0.1 5900

With the above, wayvnc is listening on port 5900 on localhost on the videoconf. To access this port, we setup another SSH reverse tunnel. It’s the same code as in the previous section, except with 5900 instead of 2022:

services.autossh-ng.sessions = {
  # …
  forward-vnc = {
    user = "root";
    destination = "mid@mid.abstractbinary.org";
    extraArguments = "-i /root/.ssh/id_ed25519 -p 2022 -o \"ControlMaster no\" -R 35900:localhost:5900";
    hostKeyChecking = false;
    knownHostsFile = "/dev/null";
  };
};

Now, we can just use a VNC client like KRDC to connect. The important KRDC settings are “Connect via SSH tunnel” and “Tunnel via loopback address”.

KRDC settings

Video conferencing

Finally, we get to the actual video conferencing bit. We use Jitsi Meet because it has never let me down and it runs in a web browser.

As a reminder, the solution we’re building has to be fully automatic. The flow “we call grandma and grandma presses button to answer” doesn’t work because grandma can’t reliably press a button.

Instead, the plan is to ssh into the videoconf and run a Selenium script that starts up Firefox, goes to Jitsi Meet, and joins a specific meeting.

At the system level, we need Firefox, geckodriver, and Selenium installed:

environment.systemPackages = with pkgs; [
  # …
  geckodriver
  (python3.withPackages (
    python-pkgs: with python-pkgs; [
      selenium
    ]
  ))
];
programs.firefox.enable = true;

Then, we write a simple script to join a meeting. The only trickyness is that we need WAYLAND_DISPLAY set for Firefox to actually start, and we need to configure some permissions in the profile to allow webcam and microphone access.

#!/usr/bin/env python3

from selenium import webdriver
from selenium.webdriver.common.by import By
import os
import argparse

def main():
    parser = argparse.ArgumentParser("join-jitsi-meeting")
    parser.add_argument("meeting", help="Meeting code to join")
    args = parser.parse_args()
    print("Connecting to jitsi meeting '%s'…" % (args.meeting,))

    os.environ["WAYLAND_DISPLAY"] = "wayland-1"
    options = webdriver.FirefoxOptions()
    options.args = ["-profile", "/home/auto/.config/mozilla/firefox/zfegmmhu.default/"]
    options.preferences["media.navigator.enabled"] = True
    options.preferences["permissions.default.microphone"] = 1
    options.preferences["permissions.default.camera"] = 1
    # options.binary_location = "/run/current-system/sw/bin/firefox"
    driver = webdriver.Firefox(options=options)

    driver.get("https://meet.jit.si")

    # https://www.selenium.dev/documentation/webdriver/elements/

    driver.implicitly_wait(1.0)
    room_text_box = driver.find_element(by=By.ID, value="enter_room_field")
    enter_room_button = driver.find_element(by=By.ID, value="enter_room_button")
    room_text_box.send_keys(args.meeting)
    enter_room_button.click()

    driver.implicitly_wait(1.0)
    name_text_box = driver.find_element(by=By.ID, value="premeeting-name-input")
    join_button = driver.find_element(by=By.XPATH, value="//div[@aria-label='Join meeting']")
    name_text_box.send_keys("Petra")
    join_button.click()

if __name__ == "__main__":
    main()

This script will likely break whenever Jitsi change their website, but we have it deployed through Home Manager, so it will be easy to fix:

home-manager.users.auto =
  { ... }:
  {
    # …
    home.file.join_jitsi_meeting = {
      source = ./dor-qws-vid1/join-jitsi-meeting.py;
      target = "scripts/join-jitsi-meeting.py";
    };
  };

The last piece of the puzzle is to make the videoconf ring. I just grabbed a phone ring sound file from Pixabay and we can play it with paplay telephone-ring.ogg.

Looking back

That’s it—we built a video conferencing device for grandma. We can manage it remotely, see its screen, and do video calls, all without grandma having to press any buttons.