Merged
Merged
Conversation
Contributor Author
Hi @lupyuen can you run a test on real macOS devices ?
Member
@simbit18 I don't have an Intel Mac though. This will work on M1 Mac?
Member
Thanks @simbit18! On my Mac Mini M2 Pro, it fails while building gperf:
g++ -g -O2 -I. -I./../lib -c ./main.cc
g++ -g -O2 -o gperf version.o positions.o options.o keyword.o keyword-list.o input.o bool-array.o hash-table.o search.o output.o main.o ../lib/libgp.a -lm
ld: archive member '/' not a mach-o file in '/Users/luppy/simbit18-macos-arm64/tools/gperf-3.1/lib/libgp.a'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
https://gist.github.com/lupyuen/9e49ad8f91e2f2e9b76c41eb583664d9
Member
I just restarted the job so it will pick up the fix from apache/nuttx-apps#2829. And it won't run all 6 hours :-)
Member
OK making good progress! I changed this in tools/ci/platforms/darwin.sh
add_path() { PATH=$1:${PATH} ## TODO: Remove Homebrew ar from PATH PATH=$( echo $PATH \ | tr ':' '\n' \ | grep -v "/opt/homebrew/opt/make/libexec/gnubin" \ | grep -v "/opt/homebrew/opt/coreutils/libexec/gnubin" \ | grep -v "/opt/homebrew/opt/binutils/bin" \ | tr '\n' ':' ) echo "**** PATH=$PATH" | tr ':' '\n' ## Should show /usr/bin/ar which ar }
Now it fails while installing Cython:
+ python3 -m venv --system-site-packages /opt/homebrew
+ pip3 install 'Cython<3.0'
[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: python3.13 -m pip install --upgrade pip
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
https://gist.github.com/lupyuen/012515d912b8524d29a01793a256b485
Contributor Author
Now it fails while installing Cython:
+ python3 -m venv --system-site-packages /opt/homebrew + pip3 install 'Cython<3.0' [notice] A new release of pip is available: 24.2 -> 24.3.1 [notice] To update, run: python3.13 -m pip install --upgrade pip error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try brew install
@lupyuen for this problem i try to test on runner macos-14
Contributor Author
this is what I use locally (msys2 and Alpine) for create a virtual environment in Python
python3 -m venv --system-site-packages /usr/local
the problem is the path if you change it it should work
on macos-14
Error: [Errno 13] Permission denied: '/usr/local/include'
Member
I don't quite understand what the code below is doing. PIP_USER and PYTHONUSERBASE seem to be interfering with the venv? This runs OK:
python_tools() { #### TODO: Previously failed to install Cython # # Python User Env # export PIP_USER=yes # export PYTHONUSERBASE=${NUTTXTOOLS}/pylocal # echo "export PIP_USER=yes" >> "${NUTTXTOOLS}"/env.sh # echo "export PYTHONUSERBASE=${NUTTXTOOLS}/pylocal" >> "${NUTTXTOOLS}"/env.sh # add_path "${PYTHONUSERBASE}"/bin # if [ "X$osarch" == "Xarm64" ]; then # python3 -m venv --system-site-packages /opt/homebrew # fi #### TODO: Now it installs Cython OK! python3 -m venv .venv source .venv/bin/activate # workaround for Cython issue # https://github.com/yaml/pyyaml/pull/702#issuecomment-1638930830 pip3 install "Cython<3.0"
Member
CI Build on Mac M2 Pro works OK yay! (with the above fixes)
$ ./cibuild.sh -i -c -A -R testlist/macos.dat ... ==================================================================================== Configuration/Tool: stm32f4discovery/nsh,CONFIG_ARM_TOOLCHAIN_GNU_EABI 2024-11-08 19:04:16 ------------------------------------------------------------------------------------ Cleaning... Configuring... Disabling CONFIG_ARM_TOOLCHAIN_GNU_EABI Enabling CONFIG_ARM_TOOLCHAIN_GNU_EABI Building NuttX...
https://gist.github.com/lupyuen/6d25cfa16b8920b515dcc14835e7a479
Here's the modified script: darwin.sh.txt
Contributor Author
I don't quite understand what the code below is doing. PIP_USER and PYTHONUSERBASE seem to be interfering with the venv? This runs OK:
python_tools() { #### TODO: Previously failed to install Cython # # Python User Env # export PIP_USER=yes # export PYTHONUSERBASE=${NUTTXTOOLS}/pylocal # echo "export PIP_USER=yes" >> "${NUTTXTOOLS}"/env.sh # echo "export PYTHONUSERBASE=${NUTTXTOOLS}/pylocal" >> "${NUTTXTOOLS}"/env.sh # add_path "${PYTHONUSERBASE}"/bin # if [ "X$osarch" == "Xarm64" ]; then # python3 -m venv --system-site-packages /opt/homebrew # fi #### TODO: Now it installs Cython OK! python3 -m venv .venv source .venv/bin/activate # workaround for Cython issue # https://github.com/yaml/pyyaml/pull/702#issuecomment-1638930830 pip3 install "Cython<3.0"
This should be used for caches on github
https://github.com/simbit18/nuttx/blob/5609c3d4166905b224020ef70c5847367616889a/.github/workflows/build.yml#L227C7-L234C103
- name: Restore Tools Cache
id: cache-tools
uses: actions/cache@v4
env:
cache-name: ${{ runner.os }}-cache-tools
with:
path: ./sources/tools
key: ${{ runner.os }}-tools-${{ hashFiles('./sources/nuttx/tools/ci/platforms/darwin.sh') }}
Member
I'm super impressed: arm-01 will actually build on Mac M2 Pro!
$ ./cibuild.sh -i -c -A -R testlist/arm-01.dat Enabling CONFIG_ARM_TOOLCHAIN_GNU_EABI Building NuttX... Normalize pcduino-a10/nsh
https://gist.github.com/lupyuen/e2babbcad9a102fcb002ca6dcf361895
Build for arm-01 completes in 1 hour. That's the same speed as GitHub Actions!
Closed
1 task
Member
FYI I'm running the script below to patch darwin.sh for my Mac Mini M2 Pro:
- Change the toolchains from x86_64 macOS to Arm64 macOS
- Remove Homebrew
arfrom PATH - Fix the Python Environment
The patched darwin.sh looks like this: darwin.sh.txt
When we change the Toolchains from x86_64 macOS to Arm64 macOS:
- Build for arm-01 completes in 1 hour, same as earlier (See the log)
- CPU Utilisation drops a little (compare with above)
https://github.com/lupyuen/nuttx-build-farm/blob/main/patch-ci-macos.sh
## How to run CI Build: ## rm -rf NUTTX_PARENT_FOLDER/tools ## ./cibuild.sh -i -c -A -R testlist/macos.dat ## ./cibuild.sh -i -c -A -R testlist/arm-01.dat set -e # Exit when any command fails set -x # Echo commands ## Create the Temp Folder tmp_dir=/tmp/macos-build-farm rm -rf $tmp_dir mkdir $tmp_dir ## We shall rewrite darwin.sh file=tools/ci/platforms/darwin.sh tmp_file=$tmp_dir/darwin.yml ## Search and replace in the file function rewrite_file { cat $file \ | sed "s/$search/$replace/g" \ >$tmp_file mv $tmp_file $file } ## Change: basefile=arm-gnu-toolchain-13.2.rel1-darwin-x86_64-arm-none-eabi ## To: basefile=arm-gnu-toolchain-13.2.rel1-darwin-arm64-arm-none-eabi #### TODO search='-darwin-x86_64-arm-none-eabi' replace='-darwin-arm64-arm-none-eabi #### TODO: We changed to Arm64 macOS' rewrite_file ## Change: basefile=arm-gnu-toolchain-13.2.Rel1-darwin-x86_64-aarch64-none-elf ## To: basefile=arm-gnu-toolchain-13.2.Rel1-darwin-arm64-aarch64-none-elf #### TODO search='-darwin-x86_64-aarch64-none-elf' replace='-darwin-arm64-aarch64-none-elf #### TODO: We changed to Arm64 macOS' rewrite_file ## Change: basefile=xpack-riscv-none-elf-gcc-13.2.0-2-darwin-x64 ## To: basefile=xpack-riscv-none-elf-gcc-13.2.0-2-darwin-arm64 #### TODO search='-darwin-x64' replace='-darwin-arm64 #### TODO: We changed to Arm64 macOS' rewrite_file ## Change: basefile=xtensa-esp32-elf-12.2.0_20230208-x86_64-apple-darwin ## To: basefile=xtensa-esp32-elf-12.2.0_20230208-aarch64-apple-darwin #### TODO search='-x86_64-apple-darwin' replace='-aarch64-apple-darwin #### TODO: We changed to Arm64 macOS' rewrite_file ## Change: add_path() { ## To: add_path() { ... } \r NOTUSED_add_path() { search='^add_path() {' replace=$( cat <<'EOF' | tr '\n' '\r' set -x # Echo commands add_path() { PATH=$1:${PATH} #### TODO: We removed Homebrew ar from PATH PATH=$( echo $PATH \\ | tr ':' '\\n' \\ | grep -v "\/opt\/homebrew\/opt\/make\/libexec\/gnubin" \\ | grep -v "\/opt\/homebrew\/opt\/coreutils\/libexec\/gnubin" \\ | grep -v "\/opt\/homebrew\/opt\/binutils\/bin" \\ | tr '\\n' ':' ) echo "**** PATH=$PATH" | tr ':' '\\n' which ar ## Should show \/usr\/bin\/ar } NOTUSED_add_path() { EOF ) rewrite_file ## Change: python_tools() { ## To: NOTUSED_python_tools() { search='^python_tools() {' replace='NOTUSED_python_tools() {' rewrite_file ## Change: # workaround for Cython issue ## To: } \r python_tools() { \r ... \r # workaround for Cython issue search='^ # workaround for Cython issue' replace=$( cat <<'EOF' | tr '\n' '\r' } python_tools() { #### TODO: We fixed the Python Environment python3 -m venv .venv source .venv\/bin\/activate # workaround for Cython issue EOF ) rewrite_file ## Change \r back to \n cat $file \ | tr '\r' '\n' \ >$tmp_file mv $tmp_file $file chmod +x $file
(Called by run-job-macos.sh and run-ci-macos.sh)
Member
Hmmm macOS fails to build arm-05, arm-08 ... arm-13 because of clang. I'll have to disable the clang builds for our macOS Build Farm:
Configuration/Tool: mps3-an547/clang,CONFIG_ARM_TOOLCHAIN_CLANG
clang++: error: configuration file 'armv8.1m.main_hard_fp.cfg' cannot be found
clang++: note: was searched for in the directory: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
clang++: error: unsupported argument '-mthumb' to option '-Wa,'
clang++: error: unsupported argument '-mimplicit-it=always' to option '-Wa,'
clang++: error: unsupported argument 'armv8.1-m.main+mve.fp+fp.dp' to option '-march='
clang++: error: configuration file 'armv8.1m.main_hard_fp.cfg' cannot be found
https://gist.github.com/lupyuen/f5eb30774e7359b0bfd07100a07eb252#file-gistfile1-txt-L1374
Member
Our macOS Build Farm is up! 🎉 We skipped these CI Jobs for macOS: run-ci-macos.sh
## Repeat forever for All CI Jobs, excluding: ## arm-05: "nrf5340-dk/rpmsghci_nimble_cpuapp: apps/wireless/bluetooth/nimble/mynewt-nimble/nimble/host/services/gatt/src/ble_svc_gatt.c:174:9: error: variable 'rc' set but not used" ## arm-07: "ucans32k146/se05x: mv: illegal option -- T" ## xtensa-02: "xtensa-esp32s2-elf-gcc: Command not found" ## arm64-01: "imx93-evk/bootloader: ld: library not found for -lcrt0.o" ## sim-01, 02, 03: "clang: error: invalid argument 'medium' to -mcmodel=" ## other: "micropendous3/hello: make: avr-objcopy: Bad CPU type in executable" for (( ; ; )); do for job in \ arm-01 arm-02 arm-03 arm-04 \ arm-06 arm-08 arm-09 arm-10 \ arm-11 arm-12 arm-13 arm-14 \ risc-v-01 risc-v-02 risc-v-03 \ risc-v-04 risc-v-05 risc-v-06 \ xtensa-01 \ x86_64-01 do ## Run the CI Job and find errors / warnings run_job $job clean_log find_messages
And we skipped all clang targets: run-job-macos.sh
## Exclude clang Targets from macOS Build, because they will fail due to unknown arch ## "/arm/lpc54xx,CONFIG_ARM_TOOLCHAIN_CLANG" ## https://github.com/apache/nuttx/pull/14691#issuecomment-2466518544 tmp_file=$tmp_dir/rewrite-testlist.dat for file in nuttx-patched/tools/ci/testlist/*.dat; do grep -v "CLANG" \ $file \ >$tmp_file mv $tmp_file $file done ## Run the CI Job in "nuttx-patched" ## ./cibuild.sh -i -c -A -R testlist/macos.dat ## ./cibuild.sh -i -c -A -R testlist/arm-01.dat pushd nuttx-patched/tools/ci ( ./cibuild.sh -i -c -A -R testlist/$job.dat \ || echo '***** BUILD FAILED' ) popd
Member
Thanks @simbit18, yep just go ahead and create the PR :-)
Merged


