Bug report
Bug description:
Unfortunately, the Android testbed is not currently buildable for 32-bit targets because the 32-bit ABI names are missing from its build system.
The first error that occurs is this:
usage: android.py configure-host [-h] [-v] [--clean] HOST [args ...]
android.py configure-host: error: argument HOST: invalid choice: 'arm-linux-androideabi' (choose from 'aarch64-linux-android', 'x86_64-linux-android')
Another error that occurs is this:
Traceback (most recent call last):
File "<frozen runpy>", line 196, in _run_module_as_main
File "<frozen runpy>", line 87, in _run_code
File "/home/ubuntu/cpython/cross-build/arm-linux-androideabi/../../Lib/sysconfig/__main__.py", line 248, in <module>
_main()
~~~~~^^
File "/home/ubuntu/cpython/cross-build/arm-linux-androideabi/../../Lib/sysconfig/__main__.py", line 235, in _main
_generate_posix_vars()
~~~~~~~~~~~~~~~~~~~~^^
File "/home/ubuntu/cpython/cross-build/arm-linux-androideabi/../../Lib/sysconfig/__main__.py", line 197, in _generate_posix_vars
pybuilddir = _get_pybuilddir()
File "/home/ubuntu/cpython/cross-build/arm-linux-androideabi/../../Lib/sysconfig/__main__.py", line 138, in _get_pybuilddir
pybuilddir = f'build/lib.{get_platform()}-{get_python_version()}'
~~~~~~~~~~~~^^
File "/home/ubuntu/cpython/cross-build/arm-linux-androideabi/../../Lib/sysconfig/__init__.py", line 703, in get_platform
machine = {
~
...<4 lines>...
"armv8l": "armeabi_v7a",
~~~~~~~~~~~~~~~~~~~~~~~~
}[machine]
~^^^^^^^^^
KeyError: 'arm'
Applying several changes allows building the Android testbed for 32-bit targets and running it on them.
This Dockerfile can build the Android testbed for 32-bit ARM and 32-bit x86.
FROM ubuntu:24.04 AS build # based on # https://github.com/thunder-coding/containerTube/blob/93062c010527382b0eb5372e9fd0842cb9f9a3f5/python-android-builder/Dockerfile # Install necessary packages we need to build Python RUN <<DOCKEREOF apt-get update && apt-get full-upgrade -y # Needed for getting the sources and SDK apt-get install -y git curl unzip # For actually building Python apt-get install -y build-essential python-is-python3 autoconf autoconf-archive m4 autotools-dev libffi-dev # Needed for Android command line tools apt-get install -y openjdk-21-jdk # Needed for building zstd in cpython build system apt-get install -y wget DOCKEREOF ADD https://dl.google.com/android/repository/commandlinetools-linux-14742923_latest.zip /root/cmdline-tools.zip ADD https://github.com/python/cpython/archive/5992238986df094e890a89376970aab6058a0759.tar.gz /root/cpython.tar.gz ADD https://github.com/beeware/cpython-android-source-deps/archive/refs/tags/zstd-1.5.7-1.tar.gz /root/cpython-android-source-deps.tar.gz ENV ANDROID_HOME=/root/.android WORKDIR /root RUN <<DOCKEREOF mkdir -p "$ANDROID_HOME/cmdline-tools" unzip cmdline-tools.zip -d "$ANDROID_HOME/cmdline-tools" mv "$ANDROID_HOME/cmdline-tools/cmdline-tools" "$ANDROID_HOME/cmdline-tools/latest" mkdir -p cpython cpython-android-source-deps tar xf cpython.tar.gz --strip-components=1 -C cpython tar xf cpython-android-source-deps.tar.gz --strip-components=1 -C cpython-android-source-deps DOCKEREOF WORKDIR /root/cpython-android-source-deps RUN <<DOCKEREOF git apply -v << 'PATCHEOF' --- a/android-env.sh +++ b/android-env.sh @@ -76,6 +76,13 @@ if [ "$HOST" = "arm-linux-androideabi" ]; then CFLAGS="$CFLAGS -march=armv7-a -mthumb" fi +# prevents this error when linking to the zstd static library built for 32-bit x86: +# ld.lld: error: relocation R_386_PC32 cannot be used +# against symbol 'HUF_writeCTable_wksp'; recompile with -fPIC +if [ "$HOST" = "i686-linux-android" ]; then + CFLAGS="$CFLAGS -fPIC" +fi + if [ -n "${PREFIX:-}" ]; then abs_prefix="$(realpath "$PREFIX")" CFLAGS="$CFLAGS -I$abs_prefix/include" PATCHEOF DOCKEREOF RUN ./build.sh zstd 1.5.7 2 i686-linux-android WORKDIR /root/cpython RUN <<DOCKEREOF cat > /root/git-ls-files.log << 'EOF' README.md android-env.sh android.py testbed/.gitignore testbed/.idea/inspectionProfiles/Project_Default.xml testbed/app/.gitignore testbed/app/build.gradle.kts testbed/app/src/androidTest/java/org/python/testbed/PythonSuite.kt testbed/app/src/main/AndroidManifest.xml testbed/app/src/main/c/CMakeLists.txt testbed/app/src/main/c/main_activity.c testbed/app/src/main/java/org/python/testbed/MainActivity.kt testbed/app/src/main/res/drawable-xxhdpi/ic_launcher.png testbed/app/src/main/res/layout/activity_main.xml testbed/app/src/main/res/values/strings.xml testbed/build.gradle.kts testbed/gradle.properties testbed/gradle/wrapper/gradle-wrapper.properties testbed/settings.gradle.kts EOF git apply -v << 'PATCHEOF' --- a/Android/android.py +++ b/Android/android.py @@ -34,7 +34,7 @@ TESTBED_DIR = ANDROID_DIR / "testbed" CROSS_BUILD_DIR = PYTHON_DIR / "cross-build" -HOSTS = ["aarch64-linux-android", "x86_64-linux-android"] +HOSTS = ["aarch64-linux-android", "x86_64-linux-android", "arm-linux-androideabi", "i686-linux-android"] APP_ID = "org.python.testbed" DECODE_ARGS = ("UTF-8", "backslashreplace") @@ -743,7 +743,7 @@ def package(context): # Include all tracked files from the Android directory. for line in run( - ["git", "ls-files"], + ["cat", "/root/git-ls-files.log"], cwd=ANDROID_DIR, capture_output=True, text=True, log=False, ).stdout.splitlines(): src = ANDROID_DIR / line --- a/Android/testbed/app/build.gradle.kts +++ b/Android/testbed/app/build.gradle.kts @@ -16,6 +16,8 @@ val inSourceTree = ( val KNOWN_ABIS = mapOf( "aarch64-linux-android" to "arm64-v8a", "x86_64-linux-android" to "x86_64", + "arm-linux-androideabi" to "armeabi-v7a", + "i686-linux-android" to "x86", ) // Discover prefixes. --- a/Lib/sysconfig/__init__.py +++ b/Lib/sysconfig/__init__.py @@ -697,12 +697,16 @@ def get_platform(): # When Python is running on 32-bit ARM Android on a 64-bit ARM kernel, # 'os.uname().machine' is 'armv8l'. Such devices run the same userspace # code as 'armv7l' devices. + # During the build process of the Android testbed when targeting 32-bit ARM, + # '_PYTHON_HOST_PLATFORM' is 'arm-linux-androideabi', so 'machine' becomes + # 'arm'. machine = { "x86_64": "x86_64", "i686": "x86", "aarch64": "arm64_v8a", "armv7l": "armeabi_v7a", "armv8l": "armeabi_v7a", + "arm": "armeabi_v7a", }[machine] elif osname == "linux": # At least on Linux/Intel, 'machine' is the processor -- --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -376,6 +376,7 @@ def test_get_platform(self): 'aarch64': 'arm64_v8a', 'armv7l': 'armeabi_v7a', 'armv8l': 'armeabi_v7a', + 'arm': 'armeabi_v7a', }.items(): with self.subTest(machine): self._set_uname(('Linux', 'localhost', '3.18.91+', @@ -585,6 +586,7 @@ def test_android_ext_suffix(self): "aarch64": "aarch64-linux-android", "armv7l": "arm-linux-androideabi", "armv8l": "arm-linux-androideabi", + "arm": "arm-linux-androideabi", }[machine] self.assertEndsWith(suffix, f"-{expected_triplet}.so") PATCHEOF DOCKEREOF WORKDIR /root/cpython/Android RUN <<DOCKEREOF ./android.py configure-build ./android.py make-build DOCKEREOF RUN <<DOCKEREOF ./android.py configure-host arm-linux-androideabi ./android.py make-host arm-linux-androideabi ./android.py package arm-linux-androideabi mv ../cross-build/arm-linux-androideabi/dist/python-*-arm-linux-androideabi.tar.gz \ /root/python-arm-linux-androideabi.tar.gz DOCKEREOF WORKDIR /root/cpython RUN <<DOCKEREOF git apply -v << 'PATCHEOF' --- a/Android/android.py +++ b/Android/android.py @@ -214,6 +214,8 @@ def unpack_deps(host, prefix_dir): download(f"{deps_url}/{name_ver}/{filename}") shutil.unpack_archive(filename) os.remove(filename) + shutil.copyfile("/root/cpython-android-source-deps/zstd/build/1.5.7/i686-linux-android/prefix/lib/libzstd.a", + "/root/cpython/cross-build/i686-linux-android/prefix/lib/libzstd.a") def download(url, target_dir="."): PATCHEOF DOCKEREOF WORKDIR /root/cpython/Android RUN <<DOCKEREOF ./android.py configure-host i686-linux-android ./android.py make-host i686-linux-android ./android.py package i686-linux-android mv ../cross-build/i686-linux-android/dist/python-*-i686-linux-android.tar.gz \ /root/python-i686-linux-android.tar.gz DOCKEREOF FROM scratch COPY --from=build /root/python-arm-linux-androideabi.tar.gz / COPY --from=build /root/python-i686-linux-android.tar.gz /
There is also an error specific to 32-bit x86 that occurs because of the https://github.com/beeware/cpython-android-source-deps repository compiling libzstd.a for 32-bit x86 without applying the -fPIC argument to the compiler:
ld.lld: error: relocation R_386_PC32 cannot be used against symbol 'HUF_writeCTable_wksp'; recompile with -fPIC
>>> defined in /root/cpython/cross-build/i686-linux-android/prefix/lib/libzstd.a(huf_compress.o)
>>> referenced by huf_compress.c
>>> huf_compress.o:(HUF_optimalTableLog) in archive /root/cpython/cross-build/i686-linux-android/prefix/lib/libzstd.a
The way to fix that error would be in the cpython-android-source-deps repository, so I have opened a separate issue and PR for that here:
Then, I was able to use these commands to build, extract and run the python-arm-linux-androideabi.tar.gz on my 32-bit ARM device connected to ADB, with ID 4251484430323498.
docker build --output=. . tar xf python-arm-linux-androideabi.tar.gz ./android.py test --connected 4251484430323498
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux