mirror of
https://github.com/official-stockfish/Stockfish.git
synced 2026-07-22 12:47:08 +00:00
anematodeandJoost VandeVondele
c3bcf74925
We maintain support for quite a few x86 ISA extensions but relied on the end
user to select the binary that is best for their system. We can detect at
runtime which architecture to use, and ship a single binary (per OS/ISA
combination). This PR does so, maintaining performance.
This is what I've landed on after quite a few iterations. Basically, we build
each arch separately, use `cpuid` to select one, and jump to that arch's main
function. Some details:
- The preprocessor macro `UNIVERSAL_BINARY` is defined when building for the universal binary.
- The Makefile target `universal-object-[no]pgo` is added, which produces a `stockfish.o` in each arch's build directory.
- To prevent symbol collisions between the multiple builds, we `#define Stockfish Stockfish_[arch]`. Furthermore we wrap the `main` function in `namespace Stockfish { }`.
- We can still get PGO data by linking to a per-arch binary, with `Stockfish_x86_64_avx2::main` as `main`, and running `bench`.
- For arches not supported by the host we can use Intel SDE – this is what we do in CI.
- When embedding the NNUE, we use C++26/C23 `#embed` instead of `INCBIN`. The issue with `INCBIN` is that it injects assembly directives that we have no control over.
- To ensure there's only one copy of the networks in the final binary, we define the network data as weak symbols in each per-arch build. Then, in the final link, we add a special nnue_embed.cpp which provides strong symbols.
- This does require GCC 15. Statically linking libstdc++ allows the binary to still run on older OSes though. And latest msys2 already does static linking + is based off GCC 15.2.0
building can be as simple as
make -j profile-build ARCH=x86-64-universal
or using the sde if the host doesn't support all architectures supported in the
universal binary, and the default host compiler is not recent enough.
make -j profile-build ARCH=x86-64-universal RUN_PREFIX="/path/to/sde -future --" CXX=g++-15
This change is also integrated in CI, so universal binaries are available as
downloadable artifacts.
Next we could also do ARM64, especially Android (all Apple silicon users use
the same binary). It also might be worth getting this to work with clang.
closes https://github.com/official-stockfish/Stockfish/pull/6740
No functional change
194 lines
6.1 KiB
YAML
194 lines
6.1 KiB
YAML
name: Compilation
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
matrix:
|
|
type: string
|
|
required: true
|
|
jobs:
|
|
Compilation:
|
|
name: ${{ matrix.config.name }} ${{ matrix.binaries }}
|
|
runs-on: ${{ matrix.config.os }}
|
|
env:
|
|
COMPCXX: ${{ matrix.config.compiler }}
|
|
COMP: ${{ matrix.config.comp }}
|
|
EXT: ${{ matrix.config.ext }}
|
|
NAME: ${{ matrix.config.simple_name }}
|
|
BINARY: ${{ matrix.binaries }}
|
|
SDE: ${{ matrix.config.sde }}
|
|
strategy:
|
|
fail-fast: false
|
|
matrix: ${{ fromJson(inputs.matrix) }}
|
|
defaults:
|
|
run:
|
|
working-directory: src
|
|
shell: ${{ matrix.config.shell }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- name: Install fixed GCC on Linux
|
|
if: runner.os == 'Linux'
|
|
uses: egor-tensin/setup-gcc@eaa888eb19115a521fa72b65cd94fe1f25bbcaac # @v1.3
|
|
with:
|
|
version: 11
|
|
|
|
- name: Setup msys and install required packages
|
|
if: runner.os == 'Windows'
|
|
uses: msys2/setup-msys2@v2
|
|
with:
|
|
msystem: ${{ matrix.config.msys_sys }}
|
|
install: mingw-w64-${{ matrix.config.msys_env }} make git zip
|
|
|
|
- name: Download SDE package
|
|
if: runner.os == 'Linux' || runner.os == 'Windows'
|
|
uses: petarpetrovt/setup-sde@f0fa5971dc275704531e94264dd23250c442aa41 # @v2.4
|
|
with:
|
|
environmentVariableName: SDE_DIR
|
|
sdeVersion: 9.33.0
|
|
|
|
- name: Download the used network from the fishtest framework
|
|
run: make net
|
|
|
|
- name: Check compiler
|
|
run: $COMPCXX -v
|
|
|
|
- name: Test help target
|
|
run: make help
|
|
|
|
- name: Check git
|
|
run: git --version
|
|
|
|
- name: Check compiler
|
|
run: $COMPCXX -v
|
|
|
|
- name: Show compiler cpu info
|
|
run: |
|
|
if [[ "$COMPCXX" == clang* ]]; then
|
|
$COMPCXX -E - -march=native -###
|
|
else
|
|
$COMPCXX -Q -march=native --help=target
|
|
fi
|
|
|
|
# x86-64 with newer extensions tests
|
|
|
|
- name: Compile ${{ matrix.config.binaries }} build
|
|
run: |
|
|
make clean
|
|
make -j4 profile-build ARCH=$BINARY COMP=$COMP RUN_PREFIX="$SDE"
|
|
make strip ARCH=$BINARY COMP=$COMP
|
|
RUN_PREFIX="$SDE" ../tests/signature.sh $benchref
|
|
mv ./stockfish$EXT ../stockfish-$NAME-$BINARY$EXT
|
|
|
|
- name: Remove non src files
|
|
run: git clean -fx
|
|
|
|
- name: Upload artifact for (pre)-release
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: ${{ matrix.config.simple_name }} ${{ matrix.binaries }}
|
|
path: |
|
|
.
|
|
!.git
|
|
!.output
|
|
|
|
UniversalLinux:
|
|
name: Ubuntu x86-64 Universal
|
|
runs-on: ubuntu-22.04
|
|
defaults:
|
|
run:
|
|
shell: bash
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
persist-credentials: false
|
|
fetch-depth: 0
|
|
|
|
- name: Install GCC 15
|
|
uses: egor-tensin/setup-gcc@eaa888eb19115a521fa72b65cd94fe1f25bbcaac # @v1.3
|
|
with:
|
|
version: 15
|
|
|
|
- name: Download SDE package
|
|
uses: petarpetrovt/setup-sde@f0fa5971dc275704531e94264dd23250c442aa41 # @v2.4
|
|
with:
|
|
environmentVariableName: SDE_DIR
|
|
sdeVersion: 9.33.0
|
|
|
|
- name: Build universal binary
|
|
run: make -j4 -C src ARCH=x86-64-universal profile-build RUN_PREFIX="$SDE_DIR/sde -future --"
|
|
|
|
- name: Strip binary
|
|
run: strip src/stockfish
|
|
|
|
- name: Rename binary
|
|
run: mv src/stockfish stockfish-ubuntu-x86-64-universal
|
|
|
|
- name: Extract the bench number from the commit history
|
|
run: |
|
|
for hash in $(git rev-list -100 HEAD); do
|
|
benchref=$(git show -s $hash | tac | grep -m 1 -o -x '[[:space:]]*\b[Bb]ench[ :]\+[1-9][0-9]\{5,7\}\b[[:space:]]*' | sed 's/[^0-9]//g') && break || true
|
|
done
|
|
[[ -n "$benchref" ]] && echo "benchref=$benchref" >> $GITHUB_ENV && echo "From commit: $hash" && echo "Reference bench: $benchref" || echo "No bench found"
|
|
|
|
- name: Check arch selection under SDE
|
|
run: ./scripts/check_universal.sh ./stockfish-ubuntu-x86-64-universal "$SDE_DIR/sde" "$benchref"
|
|
|
|
- name: Upload artifact for (pre)-release
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: ubuntu x86-64-universal
|
|
path: stockfish-ubuntu-x86-64-universal
|
|
|
|
UniversalWindows:
|
|
name: Windows x86-64 Universal
|
|
runs-on: windows-2022
|
|
defaults:
|
|
run:
|
|
shell: msys2 {0}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
persist-credentials: false
|
|
fetch-depth: 0
|
|
|
|
- name: Setup msys and install required packages
|
|
uses: msys2/setup-msys2@v2
|
|
with:
|
|
msystem: mingw64
|
|
install: mingw-w64-x86_64-gcc make git zip
|
|
|
|
- name: Download SDE package
|
|
uses: petarpetrovt/setup-sde@f0fa5971dc275704531e94264dd23250c442aa41 # @v2.4
|
|
with:
|
|
environmentVariableName: SDE_DIR
|
|
sdeVersion: 9.33.0
|
|
|
|
- name: Build universal binary
|
|
run: |
|
|
SDE_PATH=$(cygpath "$SDE_DIR")
|
|
make -j4 -C src ARCH=x86-64-universal profile-build RUN_PREFIX="$SDE_PATH/sde.exe -future --"
|
|
|
|
- name: Strip binary
|
|
run: strip src/stockfish.exe
|
|
|
|
- name: Rename binary
|
|
run: mv src/stockfish.exe stockfish-windows-x86-64-universal.exe
|
|
|
|
- name: Extract the bench number from the commit history
|
|
run: |
|
|
for hash in $(git rev-list -100 HEAD); do
|
|
benchref=$(git show -s $hash | tac | grep -m 1 -o -x '[[:space:]]*\b[Bb]ench[ :]\+[1-9][0-9]\{5,7\}\b[[:space:]]*' | sed 's/[^0-9]//g') && break || true
|
|
done
|
|
[[ -n "$benchref" ]] && echo "benchref=$benchref" >> $GITHUB_ENV && echo "From commit: $hash" && echo "Reference bench: $benchref" || echo "No bench found"
|
|
|
|
- name: Check arch selection under SDE
|
|
run: ./scripts/check_universal.sh ./stockfish-windows-x86-64-universal.exe "$SDE_DIR/sde" "$benchref"
|
|
|
|
- name: Upload artifact for (pre)-release
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: windows x86-64-universal
|
|
path: stockfish-windows-x86-64-universal.exe
|