We maintain support for quite a few x86 ISA extensions but relied on the end
user to select the binary that is best for their system. We can detect at
runtime which architecture to use, and ship a single binary (per OS/ISA
combination). This PR does so, maintaining performance.
This is what I've landed on after quite a few iterations. Basically, we build
each arch separately, use `cpuid` to select one, and jump to that arch's main
function. Some details:
- The preprocessor macro `UNIVERSAL_BINARY` is defined when building for the universal binary.
- The Makefile target `universal-object-[no]pgo` is added, which produces a `stockfish.o` in each arch's build directory.
- To prevent symbol collisions between the multiple builds, we `#define Stockfish Stockfish_[arch]`. Furthermore we wrap the `main` function in `namespace Stockfish { }`.
- We can still get PGO data by linking to a per-arch binary, with `Stockfish_x86_64_avx2::main` as `main`, and running `bench`.
- For arches not supported by the host we can use Intel SDE – this is what we do in CI.
- When embedding the NNUE, we use C++26/C23 `#embed` instead of `INCBIN`. The issue with `INCBIN` is that it injects assembly directives that we have no control over.
- To ensure there's only one copy of the networks in the final binary, we define the network data as weak symbols in each per-arch build. Then, in the final link, we add a special nnue_embed.cpp which provides strong symbols.
- This does require GCC 15. Statically linking libstdc++ allows the binary to still run on older OSes though. And latest msys2 already does static linking + is based off GCC 15.2.0
building can be as simple as
make -j profile-build ARCH=x86-64-universal
or using the sde if the host doesn't support all architectures supported in the
universal binary, and the default host compiler is not recent enough.
make -j profile-build ARCH=x86-64-universal RUN_PREFIX="/path/to/sde -future --" CXX=g++-15
This change is also integrated in CI, so universal binaries are available as
downloadable artifacts.
Next we could also do ARM64, especially Android (all Apple silicon users use
the same binary). It also might be worth getting this to work with clang.
closes https://github.com/official-stockfish/Stockfish/pull/6740
No functional change
Improve `get_native_properties.sh` with a refreshed implementation.
The new version covers all Makefile ARCH variants and keeps the
script interface stable while improving portability and CPU feature
detection across supported platforms. Drop the asset file name to
avoid coupling outputs to a specific artifact naming scheme.
Refs: https://github.com/ppigazzini/get-native-properties (includes a
testing framework)
closes https://github.com/official-stockfish/Stockfish/pull/6552
No functional change.
When vnni256 was first introduced #3038 it was very slightly faster than vnni512
on some machines. We have since sped up vnni512 significantly (#4796 over 10%
alone, #6139, and probably others I'm forgetting). Since any machine that can
run vnni256 can run vnni512 this arch is no longer useful.
Note, x86-64-avxvnni still covers targets that don't have AVX512 but do have
VNNI on 128- and 256-bit vectors.
closes https://github.com/official-stockfish/Stockfish/pull/6340
No functional change
Passed STC
LLR: 2.96 (-2.94,2.94) <0.00,2.00>
Total: 166720 W: 43191 L: 42701 D: 80828
Ptnml(0-2): 348, 18567, 45069, 18999, 377
https://tests.stockfishchess.org/tests/view/686ae98dfe0f2fe354c0c867
Refactor movegen to emit to a vector with 16-bit elements, which enables a
speedup with AVX512-VBMI2 when writing moves to the move list.
Very crude timing measurements of perft via timing ./stockfish "go perft 7"
demonstrates approximately 17% perft speedup:
Summary
./Stockfish-dev/src/stockfish 'go perft 7' ran
1.17 ± 0.04 times faster than ./Stockfish-base/src/stockfish 'go perft 7'
Estimated overall nps increase of 0.4% via speedtest: 33605229 -> 33749825
(many thanks JonathanHallstrom).
The corresponding arch is avx512icl as it is a good baseline for consumer
avx-512 feature set support; Intel Ice Lake was the first consumer AVX-512 CPU
and it is a decent subset of what AMD Zen 4 supports.
closes https://github.com/official-stockfish/Stockfish/pull/6153
No functional change
That allows 'make -j profile-build' work on ppc64 architectures, setting the use of
the appropriate SIMD extension, Altivec or VSX.
For VSX, gcc allows to map SSE2 intrinsics and get benefit of the existing SIMD code.
On PowerMac G5, using altivec provides a performance improvement of 30%.
On Talos 2, using vsx provides a performance improvement of 120%.
closes https://github.com/official-stockfish/Stockfish/pull/5624
No functional change
The recent commit introduced a bug in the net downloading script that
the file is not downloaded correctly and the content is redirected to
stdout.
closes https://github.com/official-stockfish/Stockfish/pull/5585
No functional change
Fixes https://github.com/official-stockfish/Stockfish/issues/5564
This patch extracts the net downloading script in Makefile into an
external script file. Also the script is moderately rewritten for
improved readability and speed.
* Use wget preferentially over curl, as curl is known to have slight
overhead.
* Use command instead of hash to check if command exists. Reportedly,
hash always returns zero in some POSIX shells even when the command
fails.
* Command existence checks (wget/curl, sha256sum) are performed only
once at the beginning.
* Each of common patterns is encapsulated in a function
(get_nnue_filename, validate_network).
* Print out error/warning messages to stderr.
closes https://github.com/official-stockfish/Stockfish/pull/5563
No functional change
Co-authored-by: Disservin <disservin.social@gmail.com>
