macos-latest and universal2 build error: cpython-312-darwin.so is not a fat binary!

[dmitry-mightydevops]

Description:

🆘 Unable to produce universal2 build (x86 + arm64 combined into one) with pyinstaller on macos-latest runner

I'm getting the following error:

poetry run pyinstaller   --target-architecture universal2 --add-data "ksso/success_message.html:ksso" --onefile ksso/main.py --name ksso-$os_name-$os_arch-$version

produces the error:

PyInstaller.utils.osx.IncompatibleBinaryArchError: /Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/markupsafe/_speedups.cpython-312-darwin.so is not a fat binary! For details about this error message, see: https://pyinstaller.org/en/stable/feature-notes.html#macos-multi-arch-support

Action version:
ctions/setup-python@v4

Platform:

• Ubuntu
• macOS
• Windows

Runner type:

• Hosted
• Self-hosted

Tools version:

• Python 3.12
• macos-latest runner

Repro steps:
A description with steps to reproduce the issue. If your have a public example or repo to share, please provide the link.

Pipeline code:

---
name: KSSO
on:
  push:
    branches:
      - main
  pull_request:
jobs:

  # ┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
  # │   BUILD JOB                                                                                                        │
  # │   ---                                                                                                              │
  # │                                                                                                                    │
  # │   The 'build' job compiles and packages the KSSO py application for different OS and Python versions.              │
  # │   It uses a matrix strategy to run builds.                                                                         │
  # │   The reason we need a matrix build, because we are using pyinstaller to produce executable binary                 │
  # │   and in order to do so, it should be executed in the target OS.                                                   │
  # │                                                                                                                    │
  # │   Outputs:                                                                                                         │
  # │   - `version`: Extracted from the project version and passed to the `release` job.                                 │
  # │                                                                                                                    │
  # └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
  build:
    name: Build
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os:
          - macos-latest
          - ubuntu-latest
          - windows-latest
          - macos-13
        python-version:
          - 3.12
    timeout-minutes: 30
    outputs:
      version: ${{ steps.version.outputs.version }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Set up python ${{ matrix.python-version }}
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}
      - name: Cache poetry dependencies
        id: cache
        uses: actions/cache@v3
        with:
          path: |
            ~/.cache/pypoetry
            ~/Library/Caches/pypoetry
            C:\Users\runneradmin\AppData\Local\pypoetry\Cache
          key: ${{ runner.os }}-${{ runner.arch }}-poetry-${{ hashFiles('**/poetry.lock')
            }}
          restore-keys: |
            ${{ runner.os }}-${{ runner.arch }}-poetry-
      - name: Install poetry
        run: |
          python -m pip install poetry
      - name: Install dependencies
        if: steps.cache.outputs.cache-hit != 'true'
        run: poetry install --no-interaction --no-ansi
      - name: Determine version
        id: version
        run: |
          version=$(poetry version --short)
          echo "version=$version" >> $GITHUB_ENV
          echo "::set-output name=version::$version"
        shell: bash
      - name: Build binary with pyinstaller
        run: |
          source $GITHUB_ENV
          os_name=$(echo ${{ runner.os }}   | tr '[:upper:]' '[:lower:]')
          os_arch=$(echo ${{ runner.arch }} | tr '[:upper:]' '[:lower:]')
          if [[ "${{ matrix.os }}" == "ubuntu-latest" ]]; then
            poetry run pyinstaller --add-data "ksso/success_message.html:ksso" --onefile ksso/main.py --name ksso-$os_name-$os_arch-$version
          elif [[ "${{ matrix.os }}" == "macos-latest" ]]; then
            poetry run pyinstaller   --target-architecture universal2 --add-data "ksso/success_message.html:ksso" --onefile ksso/main.py --name ksso-$os_name-$os_arch-$version
          elif [[ "${{ matrix.os }}" == "macos-13" ]]; then
            poetry run pyinstaller   --target-architecture x86_64 --add-data "ksso/success_message.html:ksso" --onefile ksso/main.py --name ksso-$os_name-$os_arch-$version
          elif [[ "${{ matrix.os }}" == "windows-latest" ]]; then
            poetry run pyinstaller --add-data "ksso/success_message.html;ksso" --onefile ksso/main.py --name ksso-$os_name-$os_arch-$version
          fi
        shell: bash
      - name: Upload artifact
        uses: actions/upload-artifact@v4
        with:
          name: ksso-${{ matrix.os }}-${{ env.version }}
          path: dist/ksso*

Expected behavior:

A universal2 build to be produced without errors on macos-latest runner

Actual behavior:

Build fails with the error

Error log

58 INFO: PyInstaller: 6.11.1, contrib hooks: 2024.11
58 INFO: Python: 3.12.8
71 INFO: Platform: macOS-14.7.2-arm64-arm-64bit
71 INFO: Python environment: /Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12
72 INFO: wrote /Users/runner/work/ksso/ksso/ksso-macos-arm64-0.1.0.spec
75 INFO: Module search paths (PYTHONPATH):
['/Library/Frameworks/Python.framework/Versions/3.12/lib/python312.zip',
 '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12',
 '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload',
 '/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages',
 '/Users/runner/work/ksso/ksso',
 '/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/setuptools/_vendor',
 '/Users/runner/work/ksso/ksso']
194 INFO: Appending 'datas' from .spec
194 INFO: checking Analysis
194 INFO: Building Analysis because Analysis-00.toc is non existent
194 INFO: Running Analysis Analysis-00.toc
194 INFO: Target bytecode optimization level: 0
194 INFO: Initializing module dependency graph...
195 INFO: Initializing module graph hook caches...
200 INFO: Analyzing base_library.zip ...
649 INFO: Processing standard module hook 'hook-heapq.py' from '/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/hooks'
705 INFO: Processing standard module hook 'hook-encodings.py' from '/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/hooks'
1663 INFO: Processing standard module hook 'hook-pickle.py' from '/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/hooks'
2438 INFO: Caching module dependency graph...
2482 INFO: Looking for Python shared library...
2485 INFO: Using Python shared library: /Library/Frameworks/Python.framework/Versions/3.12/Python
2485 INFO: Analyzing /Users/runner/work/ksso/ksso/ksso/main.py
10620 INFO: Including run-time hook 'pyi_rth_pkgutil.py' from '/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/hooks/rthooks'
10621 INFO: Including run-time hook 'pyi_rth_multiprocessing.py' from '/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/hooks/rthooks'
10622 INFO: Including run-time hook 'pyi_rth_pkgres.py' from '/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/hooks/rthooks'
10664 INFO: Looking for dynamic libraries
10822 INFO: Warnings written to /Users/runner/work/ksso/ksso/build/ksso-macos-arm64-0.1.0/warn-ksso-macos-arm64-0.1.0.txt
10845 INFO: Graph cross-reference written to /Users/runner/work/ksso/ksso/build/ksso-macos-arm64-0.1.0/xref-ksso-macos-arm64-0.1.0.html
10945 INFO: checking PYZ
10945 INFO: Building PYZ because PYZ-00.toc is non existent
10945 INFO: Building PYZ (ZlibArchive) /Users/runner/work/ksso/ksso/build/ksso-macos-arm64-0.1.0/PYZ-00.pyz
11522 INFO: Building PYZ (ZlibArchive) /Users/runner/work/ksso/ksso/build/ksso-macos-arm64-0.1.0/PYZ-00.pyz completed successfully.
11529 INFO: EXE target arch: universal2
11529 INFO: Code signing identity: None
11563 INFO: checking PKG
11563 INFO: Building PKG because PKG-00.toc is non existent
11563 INFO: Building PKG (CArchive) ksso-macos-arm64-0.1.0.pkg
Traceback (most recent call last):
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/bin/pyinstaller", line 8, in <module>
    sys.exit(_console_script_run())
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/__main__.py", line [23](https://github.com/saritasa-nest/ksso/actions/runs/12577738363/job/35055540388?pr=3#step:8:24)1, in _console_script_run
    run()
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/__main__.py", line 215, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/__main__.py", line 70, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/building/build_main.py", line 1[25](https://github.com/saritasa-nest/ksso/actions/runs/12577738363/job/35055540388?pr=3#step:8:26)2, in main
    build(specfile, distpath, workpath, clean_build)
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/building/build_main.py", line 1192, in build
    exec(code, spec_namespace)
  File "/Users/runner/work/ksso/ksso/ksso-macos-arm64-0.1.0.spec", line 19, in <module>
    exe = EXE(
          ^^^^
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/building/api.py", line 639, in __init__
    self.pkg = PKG(
               ^^^^
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/building/api.py", line 254, in __init__
    self.__postinit__()
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/building/api.py", line 314, in assemble
    src_name = process_collected_binary(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/building/utils.py", line [30](https://github.com/saritasa-nest/ksso/actions/runs/12577738363/job/35055540388?pr=3#step:8:31)9, in process_collected_binary
    osxutils.binary_to_target_arch(cached_name, target_arch, display_name=src_name)
  File "/Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/PyInstaller/utils/osx.py", line 342, in binary_to_target_arch
    raise IncompatibleBinaryArchError(f"{display_name} is not a fat binary!")
PyInstaller.utils.osx.IncompatibleBinaryArchError: /Users/runner/Library/Caches/pypoetry/virtualenvs/ksso-REv962lH-py3.12/lib/python3.12/site-packages/markupsafe/_speedups.cpython-[31](https://github.com/saritasa-nest/ksso/actions/runs/12577738363/job/35055540388?pr=3#step:8:32)2-darwin.so is not a fat binary! For details about this error message, see: https://pyinstaller.org/en/stable/feature-notes.html#macos-multi-arch-support
Error: Process completed with exit code 1.

[suyashgaonkar]

Hello @dmitry-mightydevops ,
Thank you for creating this issue. We will investigate it and provide feedback as soon as we have some updates.