Packaging with PyInstaller

Introduction

Not to long ago I needed to deploy a simple Python app I had written on a Linux server. It was a single file and my first thought was to drop the file on the server, but it needed some dependencies. Not a big deal, or so I thought.

Putting a single file Python app on a server is typically fine. Just install the deps from the requirements.txt either through the distro package manager or into a venv. Building out a whl file makes this whole thing even easier since it’s a single package and pip will install all dependencies automatically.

However, in this case the app had dependencies that use C libraries that needed to be installed on the system separately. Complicating matters, the distro I was deploying to had an older version of the C library and it was older than what the Python module supported.

So, I needed a solution that would bundle everything into a single deployable package with all dependencies. Including all the C libraries. This is where PyInstaller comes in.

PyInstaller is a fantastic application that freezes a Python app and creates a distributable package that has everything. It even bundles Python so it is truly a standalone package.

Environment

You’ll want to setup your venv as usual and install all your dependencies. Additionally, you want to install PyInstaller into your venv too. Doing so will allow PyInstaller to detect and include all the dependencies from your venv in the package it makes

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install pyinstaller

Making the package

App directory

By default PyInstaller will create a directory structure with everything included. Just run the binary in the directory and it will use all of the dependencies from within the directory.

pyinstaller my_app.py

Distribution and installation are as easy as zipping the directory, and unzipping to wherever you want the package to live.

Single file

With this method, you get a single file instead of a directory. This makes distribution a bit easier but underneath it’s hiding the directory it normally creates.

Basically, when you run the single file app, the app extracts the directory contents to a temporary location, runs the app, and finally cleans up by deleting the temporary directory.

To make a single file or a single file package use the --onefile flag.

pyinstaller --onefile my_app.py

Possible Linux Errors

I mentioned earlier that I was deploying to a Linux server and there are some issues I run into. These are pretty common with distros that use and backport security patches to older libraries version. Only the first one is specific to PyInstaller and only applies if using the single file package.

Segment mapping

error while loading shared libraries: libz.so.1: failed to map segment from shared object

This can happen if you’re using the single file output. The single file extracts the package to a temporary directory in order to run the app. On Red Hat based systems the tmp directory does not allow execution as a security measure.

You’ll need to create a directory to override default tmp directory. Then run the app. This way it will be able to extract to a directory that allows execute.

mkdir mytmp
export TMPDIR=`pwd`/mytmp

GLIBC version

PyInstaller does not cross compile and if you’re not building on the same distro and release as you’re going to deploy, you can run into GLIBC version incapabilities.

Failed to load Python shared library '/tmp/_MEIxxxxxx/libpython3.12.so': dlopen: /lib64/libm.so.6: version `GLIBC_2.38' not found (required by /tmp/_MEIxxxxxx/libpython3.12.so)

This indicates the GLIBC version on the system running the application is older than the system that built the application. GLIBC is only forward compatible. Meaning, you have to build on a system with the same or an older version of GLIBC. An application built against a newer GLIBC cannot run on a system with an older version.

The solution is to either install an older distro release or the same exact one you’re going to deploy on. It doesn’t matter if they’re different distro’s only the GLIBC version on the build and deployment systems matter.

Conclusion

Normally I distribute my python apps as whl packages. This works fine for the most part. A quick pip install of the whl file and I’m good. Maybe install some libraries but otherwise it’s pretty standard. Except in this one instance where I needed a real stand alone package.

PyInstaller is simple to use and worked very well; I can see myself using it more often. It would make it easier to release packaged binaries on GitHub. I use macOS and Linux so I don’t usually think about Windows but I know Python whl files and Windows don’t mesh very well.

I’m confident making Windows stand alone binaries could be easily automated with GitHub actions. macOS .app packages wouldn’t be a bad idea either. Something for me to think about in the future.