fix Node.js version dependency, update SDE.
This upgrades to 9.58, upgrade to 10.8 failed with:
Invalid file name and base dir combination: Pin root DLL: pincrt4.dll
Also adjusts Makefile to show PGO bench output on failure
closes https://github.com/official-stockfish/Stockfish/pull/6781
No functional change
Add workflow-level concurrency to the direct GitHub Actions entrypoints so a
newer run for the same pull request or repository ref cancels the older
in-progress run.
Keep the reusable workflow graph, release guards, triggers, and permissions
unchanged while documenting the concurrency policy and the official GitHub
references used for the change.
closes https://github.com/official-stockfish/Stockfish/pull/6773
No functional change.
Rename Linux universal release package from
`stockfish-ubuntu-x86-64-universal.tar.gz` to
`stockfish-linux-x86-64-universal.tar.gz` (and similar for the binary), as the
bundled binary should run on any Linux distro with glibc >= 2.35 (which is the
glibc version of the Ubuntu 22.04 build server).
Note that the Linux universal binary is built with static libstdc++ linkage, so
this adds no extra dependency.
This change will hopefully make it less confusing for users, and in line with
how we show it on stockfishchess.org (as Linux).
closes https://github.com/official-stockfish/Stockfish/pull/6760
No functional change
`stockfish-ubuntu-x86-64-universal.tar` goes from 94.8MiB to 67.4MiB (tar.gz)
Note that the CI code is written such that it easy to select other compression
schemes in the future (e.g. tar.xz) by simply changing the `archive_ext`
parameter.
closes https://github.com/official-stockfish/Stockfish/pull/6746
No functional change
Pin every external action used by the root GitHub workflow set to a full-length
commit SHA and keep the selected release tag as a same-line comment.
Update the root workflows to the latest stable tagged action versions found
during the audit while preserving existing release guards, permissions, and
reusable workflow wiring.
closes https://github.com/official-stockfish/Stockfish/pull/6754
No functional change.
We maintain support for quite a few x86 ISA extensions but relied on the end
user to select the binary that is best for their system. We can detect at
runtime which architecture to use, and ship a single binary (per OS/ISA
combination). This PR does so, maintaining performance.
This is what I've landed on after quite a few iterations. Basically, we build
each arch separately, use `cpuid` to select one, and jump to that arch's main
function. Some details:
- The preprocessor macro `UNIVERSAL_BINARY` is defined when building for the universal binary.
- The Makefile target `universal-object-[no]pgo` is added, which produces a `stockfish.o` in each arch's build directory.
- To prevent symbol collisions between the multiple builds, we `#define Stockfish Stockfish_[arch]`. Furthermore we wrap the `main` function in `namespace Stockfish { }`.
- We can still get PGO data by linking to a per-arch binary, with `Stockfish_x86_64_avx2::main` as `main`, and running `bench`.
- For arches not supported by the host we can use Intel SDE – this is what we do in CI.
- When embedding the NNUE, we use C++26/C23 `#embed` instead of `INCBIN`. The issue with `INCBIN` is that it injects assembly directives that we have no control over.
- To ensure there's only one copy of the networks in the final binary, we define the network data as weak symbols in each per-arch build. Then, in the final link, we add a special nnue_embed.cpp which provides strong symbols.
- This does require GCC 15. Statically linking libstdc++ allows the binary to still run on older OSes though. And latest msys2 already does static linking + is based off GCC 15.2.0
building can be as simple as
make -j profile-build ARCH=x86-64-universal
or using the sde if the host doesn't support all architectures supported in the
universal binary, and the default host compiler is not recent enough.
make -j profile-build ARCH=x86-64-universal RUN_PREFIX="/path/to/sde -future --" CXX=g++-15
This change is also integrated in CI, so universal binaries are available as
downloadable artifacts.
Next we could also do ARM64, especially Android (all Apple silicon users use
the same binary). It also might be worth getting this to work with clang.
closes https://github.com/official-stockfish/Stockfish/pull/6740
No functional change
Ever since #5094 master only lets the main thread terminate `go mate`
searches, reverting the earlier improvement #1215.
This PR restores the old logic. So any thread that found a "mate in x"
can now stop the search.
To make this work robustly, we need to guard against inexact mate scores
in the best thread selection. In addition, in contrast to time limits,
the main thread may now not complete a d1 search for a mated-in
position.
In master an aborted d1 search may have a PV beginning with
`Move::none()`, in which case no thread selection is performed. See
#623. We fix this bug here by checking if `lastBestPV` is empty or not.
For interrupted d1 searches we now label mated-in scores as inexact.
While at it, we also simplify the logic for detecting if we can
terminate a go mate x search, using the fact that threads.stop can only
be false if we have a completed iteration with a valid score.
The PR has no effect on game play, but should slightly improve general
mate finding and speed up multi-threaded `go mate` searches.
We also add a corresponding matecheck run to the CI. This only involves
61 mates up to mate-in-2. Test runs with the first 50 or 100 mates from
`mates2000.epd` did at times not finish within 30 minutes on my fork or
in local tests, possibly due to search explosions for some mate-in-3
positions.
closes https://github.com/official-stockfish/Stockfish/pull/6668
No functional change
WINE_PATH started as a Wine-specific knob, but it’s now used more generally
as a command prefix to run the built engine under wrappers
like Intel SDE, qemu-user, etc.
- Add RUN_PREFIX as the supported “run wrapper/prefix” variable in Makefile
- Set WINE_PATH as a deprecated alias
- Update CI and scripts to use RUN_PREFIX
closes https://github.com/official-stockfish/Stockfish/pull/6500
No functional change
When vnni256 was first introduced #3038 it was very slightly faster than vnni512
on some machines. We have since sped up vnni512 significantly (#4796 over 10%
alone, #6139, and probably others I'm forgetting). Since any machine that can
run vnni256 can run vnni512 this arch is no longer useful.
Note, x86-64-avxvnni still covers targets that don't have AVX512 but do have
VNNI on 128- and 256-bit vectors.
closes https://github.com/official-stockfish/Stockfish/pull/6340
No functional change
Add runs with --threads 4 to our matetrack CI. These won't be deterministic, of
course. But they may catch early some multithreading bugs in our mate
reporting.
Motivated by #6293 and #6296.
https://github.com/official-stockfish/Stockfish/pull/6297
No functional change.
Previously the upload step used the os of the artifact to create the upload and
used an extra msys2 step. This is in fact not needed and we can do all required
changes on ubuntu instead.
closes https://github.com/official-stockfish/Stockfish/pull/6294
No functional change
Since an unknown amount of time the instrumented CI has been a bit
flawed, explained here
https://github.com/official-stockfish/Stockfish/issues/5185. It also
experiences random timeout issues where restarting the workflow fixes it
or very long run times (more than other workflows) and is not very
portable.
The intention of this commit is to port the instrumented.sh to python
which also works on other operating systems. It should also be
relatively easy for beginners to add new tests to assert stockfish's
output and to run it.
From the source directory the following command can be run.
`python3 ../tests/instrumented.py --none ./stockfish`
A test runner will go over the test suites and run the test cases.
All instrumented tests should have been ported over.
The required python version for this is should be 3.7 (untested) + the
requests package, testing.py includes some infrastructure code which
setups the testing.
fixes https://github.com/official-stockfish/Stockfish/issues/5185
closes https://github.com/official-stockfish/Stockfish/pull/5583
No functional change
Fixes https://github.com/official-stockfish/Stockfish/issues/5564
This patch extracts the net downloading script in Makefile into an
external script file. Also the script is moderately rewritten for
improved readability and speed.
* Use wget preferentially over curl, as curl is known to have slight
overhead.
* Use command instead of hash to check if command exists. Reportedly,
hash always returns zero in some POSIX shells even when the command
fails.
* Command existence checks (wget/curl, sha256sum) are performed only
once at the beginning.
* Each of common patterns is encapsulated in a function
(get_nnue_filename, validate_network).
* Print out error/warning messages to stderr.
closes https://github.com/official-stockfish/Stockfish/pull/5563
No functional change
Co-authored-by: Disservin <disservin.social@gmail.com>
now checks correctness of PV lines with TB score.
uses 3-4-5 man table bases, downloaded from lichess,
which are cached with the appropriate action.
closes https://github.com/official-stockfish/Stockfish/pull/5500
No functional change
do not upload some unneeded intermediate directories,
disable running authenticated git commands with the checkout action.
Thanks to Yaron A for the report.
closes https://github.com/official-stockfish/Stockfish/pull/5435
No functional change
this action plays games under fast-chess with a `debug=yes` compiled binary.
It checks for triggered asserts in the code, or generally for engine disconnects.
closes https://github.com/official-stockfish/Stockfish/pull/5403
No functional change
Allow for NUMA memory replication for NNUE weights. Bind threads to ensure execution on a specific NUMA node.
This patch introduces NUMA memory replication, currently only utilized for the NNUE weights. Along with it comes all machinery required to identify NUMA nodes and bind threads to specific processors/nodes. It also comes with small changes to Thread and ThreadPool to allow easier execution of custom functions on the designated thread. Old thread binding (WinProcGroup) machinery is removed because it's incompatible with this patch. Small changes to unrelated parts of the code were made to ensure correctness, like some classes being made unmovable, raw pointers replaced with unique_ptr. etc.
Windows 7 and Windows 10 is partially supported. Windows 11 is fully supported. Linux is fully supported, with explicit exclusion of Android. No additional dependencies.
-----------------
A new UCI option `NumaPolicy` is introduced. It can take the following values:
```
system - gathers NUMA node information from the system (lscpu or windows api), for each threads binds it to a single NUMA node
none - assumes there is 1 NUMA node, never binds threads
auto - this is the default value, depends on the number of set threads and NUMA nodes, will only enable binding on multinode systems and when the number of threads reaches a threshold (dependent on node size and count)
[[custom]] -
// ':'-separated numa nodes
// ','-separated cpu indices
// supports "first-last" range syntax for cpu indices,
for example '0-15,32-47:16-31,48-63'
```
Setting `NumaPolicy` forces recreation of the threads in the ThreadPool, which in turn forces the recreation of the TT.
The threads are distributed among NUMA nodes in a round-robin fashion based on fill percentage (i.e. it will strive to fill all NUMA nodes evenly). Threads are bound to NUMA nodes, not specific processors, because that's our only requirement and the OS can schedule them better.
Special care is made that maximum memory usage on systems that do not require memory replication stays as previously, that is, unnecessary copies are avoided.
On linux the process' processor affinity is respected. This means that if you for example use taskset to restrict Stockfish to a single NUMA node then the `system` and `auto` settings will only see a single NUMA node (more precisely, the processors included in the current affinity mask) and act accordingly.
-----------------
We can't ensure that a memory allocation takes place on a given NUMA node without using libnuma on linux, or using appropriate custom allocators on windows (https://learn.microsoft.com/en-us/windows/win32/memory/allocating-memory-from-a-numa-node), so to avoid complications the current implementation relies on first-touch policy. Due to this we also rely on the memory allocator to give us a new chunk of untouched memory from the system. This appears to work reliably on linux, but results may vary.
MacOS is not supported, because AFAIK it's not affected, and implementation would be problematic anyway.
Windows is supported since Windows 7 (https://learn.microsoft.com/en-us/windows/win32/api/processtopologyapi/nf-processtopologyapi-setthreadgroupaffinity). Until Windows 11/Server 2022 NUMA nodes are split such that they cannot span processor groups. This is because before Windows 11/Server 2022 it's not possible to set thread affinity spanning processor groups. The splitting is done manually in some cases (required after Windows 10 Build 20348). Since Windows 11/Server 2022 we can set affinites spanning processor group so this splitting is not done, so the behaviour is pretty much like on linux.
Linux is supported, **without** libnuma requirement. `lscpu` is expected.
-----------------
Passed 60+1 @ 256t 16000MB hash: https://tests.stockfishchess.org/tests/view/6654e443a86388d5e27db0d8
```
LLR: 2.95 (-2.94,2.94) <0.00,10.00>
Total: 278 W: 110 L: 29 D: 139
Ptnml(0-2): 0, 1, 56, 82, 0
```
Passed SMP STC: https://tests.stockfishchess.org/tests/view/6654fc74a86388d5e27db1cd
```
LLR: 2.95 (-2.94,2.94) <-1.75,0.25>
Total: 67152 W: 17354 L: 17177 D: 32621
Ptnml(0-2): 64, 7428, 18408, 7619, 57
```
Passed STC: https://tests.stockfishchess.org/tests/view/6654fb27a86388d5e27db15c
```
LLR: 2.94 (-2.94,2.94) <-1.75,0.25>
Total: 131648 W: 34155 L: 34045 D: 63448
Ptnml(0-2): 426, 13878, 37096, 14008, 416
```
fixes#5253
closes https://github.com/official-stockfish/Stockfish/pull/5285
No functional change
