This PR removes the variable threads.abortedSearch introduced in #4990 and
relies on threads.stop and completedDepth instead.
The logic in master is only guaranteed to work in single threaded search. For
multiple threads, as soon as threads.stop is set, some of the threads may abort
their search, and so their mated-in scores are not reliable.
Compared to master, this patch prevents unproven mated-in scores in these
multi-threaded scenarios:
in game play (when time management is used)
when the uci commands quit or stop are received
when maximum depth is reached
for go mate commands
The patch will change the bestmove in those very rare situations where the bug
occurs in master. This is because bestmove is taken from rootMoves[0].pv[0],
which will now no longer contain unproven mated-in PVs. In that sense the patch
is "functional". But of course, it does not change bench.
While at it, we also fix a bug in master that suppressed PV output for inexact
mated-in scores on receiving threads.stop. In cases where uciScore < score with
score not being a proven loss, and where the new bestmove was different from
the last completed iteration, this led to a bestmove output that did not match
the final PV (which was for an exact score).
Fixes#6293.
Fixes#6626.
closes https://github.com/official-stockfish/Stockfish/pull/6636
No functional change
Passed STC (first run):
https://tests.stockfishchess.org/tests/view/698030ee6362aee5c8a552c7
LLR: 2.96 (-2.94,2.94) <0.00,2.00>
Total: 120800 W: 31336 L: 30912 D: 58552
Ptnml(0-2): 330, 13084, 33172, 13460, 354
Passed LTC:
https://tests.stockfishchess.org/tests/view/6983946d473df9d1d24a916d
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 86850 W: 22352 L: 21951 D: 42547
Ptnml(0-2): 45, 8506, 25934, 8883, 57
Passed STC SMP (after changes made after opening the PR):
https://tests.stockfishchess.org/tests/view/6987b4c1b0f3ca5200aaf9da
LLR: 2.95 (-2.94,2.94) <0.00,2.00>
Total: 84480 W: 21994 L: 21627 D: 40859
Ptnml(0-2): 98, 9378, 22932, 9723, 109
Passed STC (second run, as a sanity check made after changes made after opening the PR -- these changed, however, didn't change assembly code produced by the compiler):
https://tests.stockfishchess.org/tests/view/6987ad21b0f3ca5200aaf9c9
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 132192 W: 34526 L: 34083 D: 63583
Ptnml(0-2): 421, 14511, 35759, 15014, 391
GCC maps __builtin_prefetch locality=1 to PREFETCHT2 instruction, targeting L2
cache. To make things clear, added template-based approach as suggested by
Disservin.
PREFETCHT2 should be the appropriate choice for the 78 MB threat weight table:
avoids L1d pollution, parks data in L2 where it is promoted to L1d on actual
load.
With 200-500 cycle lead time from make_index, data has time to arrive in L2.
suggestions by Disservin and anematode included
closes https://github.com/official-stockfish/Stockfish/pull/6602
No functional change
Replace FNV-1a hash algorithm by public domain MurmurHash64A, by Austin
Appleby, which is faster and has better hash properties. This is also
the default hash algorithm of GNU C++ Standard Library.
Startup time on a given system (with x86-64-avx2 binaries) reduces from
378ms to 332ms (46ms faster). Hashing of the largest data input (46.1MB)
goes from 44.2ms to 6.2ms, so about 7x faster.
No functional change
closes https://github.com/official-stockfish/Stockfish/pull/6579
Bench: 2570943
Even though this failed with non regression bounds on fishtest, merging
it as is to improve the current situation.
Failed Non Regression:
https://tests.stockfishchess.org/tests/view/6988814db0f3ca5200aafb50
LLR: -2.94 (-2.94,2.94) <-1.75,0.25>
Total: 185792 W: 48109 L: 48565 D: 89118
Ptnml(0-2): 553, 21228, 49811, 20730, 574
Previously we had code similar to this
```
alignas(64) std::uint8_t foo[1024];
const auto* int_ptr = reinterpret_cast<const std::int32_t*>(foo);
```
Which is a strict aliasing violation once we dereference int_ptr, which
invokes UB and causes newer gcc version to emit incorrect code.
fixes https://github.com/official-stockfish/Stockfish/issues/6598
fixes https://github.com/official-stockfish/Stockfish/issues/4932
This violation was introduced in PR #3287 and merged in commit 23c385e.
Due to aggressive compiler optimizations, users may need to include
CXXFLAGS=-fno-strict-aliasing as a build argument to ensure a correctly
functioning binary.
Also thanks to @anematode for writing parts of this and realizing this
strict aliasing violation.
closes https://github.com/official-stockfish/Stockfish/pull/6599
Bench: 2668754
Co-Authored-By: Timothy Herchen <timothy.herchen@gmail.com>
Improve `get_native_properties.sh` with a refreshed implementation.
The new version covers all Makefile ARCH variants and keeps the
script interface stable while improving portability and CPU feature
detection across supported platforms. Drop the asset file name to
avoid coupling outputs to a specific artifact naming scheme.
Refs: https://github.com/ppigazzini/get-native-properties (includes a
testing framework)
closes https://github.com/official-stockfish/Stockfish/pull/6552
No functional change.
Official release version of Stockfish 18
Bench: 2050811
---
Stockfish 18
Today, we have the pleasure of announcing Stockfish 18, a new major release. As
always, you can freely download it at [stockfishchess.org/download][1] and use
it as a drop-in replacement in the [GUI of your choice][2] to benefit from
stronger play and more accurate analysis.
Whether you can spare hours or days of CPU time, your help matters for the
ongoing development of Stockfish. Find out how you can help at
[stockfishchess.org/get-involved][3]. Join our [Discord server][4] to get in
touch with the community of developers and users of the project!
Quality of chess play
In tests against Stockfish 17, this new release brings an Elo gain of up to [46
points][5], and wins [four times as many game pairs][6] as it loses. Quality
improved throughout, including in [Fischer Random Chess][7].
Stockfish is stronger than any human, even when running on older or low-end
hardware. On the highest-end hardware, where Stockfish can search over [500
million positions per second][8], it continues to [dominate chess engine
tournaments][9].
Update highlights
Next-Generation Evaluation
This release introduces the SFNNv10 network architecture. The network’s input
layer has been augmented with 'Threat Inputs' features as part of a massive
community effort. These features allow the engine to 'see' which pieces are
threatened more naturally, resulting in more accurate evaluations.
Hardware and Performance Optimizations
A key highlight is the new 'Shared Memory' implementation, which allows
different Stockfish processes to share the same memory space for neural network
weights. This makes it the most efficient version for cloud analysis and
high-concurrency testing.
Significant efforts have also been made to utilize hardware more effectively by
adapting the code to make use of modern CPU instructions and refining how
threads interact during a search.
Search Improvements
Stockfish 18 features a heavily refined search, utilizing 'Correction History'
to dynamically adjust evaluations based on patterns found during the search
itself. These and other refinements allow the engine to detect stalemates and
evaluate fortresses significantly better than previous versions. A particularly
rare issue, involving threefold repetition detection, en passant, and pins, was
also fixed.
Refactored Training Workflow
The training of Stockfish's neural networks has transitioned to an automated
and reproducible model. This new framework allows the project to employ
standardized recipes to chain complex training stages together. This transition
facilitates the training of networks using over 100 billion positions of [Lc0
evaluation data][10].
Thank you
In this release in particular, we are deeply grateful to the contributors who
shared their research and ideas to help develop the new threat-input network
architecture.
The Stockfish project builds on a thriving community of enthusiasts (thanks to
everybody!) who contribute their expertise, time, and resources to build a free
and open-source chess engine that is robust, widely available, and very strong.
We would like to express our gratitude for the [14.6k stars][11] that light up
our GitHub project. Thank you for your support and encouragement – your
recognition means a lot to us. Programmers can contribute to the project either
directly to [Stockfish][12] (C++), to [Fishtest][13] (HTML, CSS, JavaScript,
and Python), to our trainer [nnue-pytorch][14] (C++ and Python), or to our
[website][15] (HTML, CSS/SCSS, and JavaScript).
The Stockfish team
[1]: https://stockfishchess.org/download
[2]: https://official-stockfish.github.io/docs/stockfish-wiki/Download-and-usage.html#download-a-chess-gui
[3]: https://stockfishchess.org/get-involved/
[4]: https://discord.gg/GWDRS3kU6R
[5]: https://tests.stockfishchess.org/tests/view/696a9e15cec152c6220c1d19
[6]: https://tests.stockfishchess.org/tests/view/696a9e4dcec152c6220c1d1b
[7]: https://tests.stockfishchess.org/tests/view/696a9e83cec152c6220c1d1d
[8]: https://openbenchmarking.org/test/pts/stockfish
[9]: https://en.wikipedia.org/wiki/Stockfish_(chess)#Competition_results
[10]: https://lczero.org/blog/2021/06/the-importance-of-open-data/
[11]: https://github.com/official-stockfish/Stockfish/stargazers
[12]: https://github.com/official-stockfish/Stockfish
[13]: https://github.com/official-stockfish/fishtest
[14]: https://github.com/official-stockfish/nnue-pytorch
[15]: https://github.com/official-stockfish/stockfish-web
some environment dependent execution (e.g. pid) were being std::hash'ed, and
net filenames put in unordered maps. Also uses sprintf instead of
std::to_string. Depending on precise content, this could lead to different
PGO'ed binaries. This is mitigated by using a basic hash function.
This also fixes a potential issue in net filename generation, in cases where
std::hash would use invocation dependent salt, which is not the cases today for
typical std libraries.
Closes https://github.com/official-stockfish/Stockfish/pull/6562
No functional change
With the current setup, on Linux, SIGILL (and SIGSEGV/SIGBUS etc., in the case
of a program bug) lead to no feedback if they occur after the signal handlers
are installed, they just exit silently. By invoking the default signal handler,
we can still get the appropriate feedback. This is particularly important for
feedback if someone downloads the wrong SF binary and runs it on Linux.
closes https://github.com/official-stockfish/Stockfish/pull/6554
No functional change
This PR updates the internal WDL model, using data from 3.1M games played by
the revisions since 44d5467. Note that the normalizing constant increases only
moderately from 377 to 385.
closes https://github.com/official-stockfish/Stockfish/pull/6549
No functional change
avoid munmap of memory when exiting via signal, which avoids side effects such
as triggering asserts or (caught) segfaults while the process exists.
closes https://github.com/official-stockfish/Stockfish/pull/6542
No functional change
Windows 10 is missing the GroupMasks and GroupCount members, this breaks compiles on Windows 10.
Windows 11 builds, including the official ones, run fine on Windows 10/11.
To support developers/testers on Windows 10, fallback conditionally to the Windows 10 struct definition.
closes https://github.com/official-stockfish/Stockfish/pull/6538
No functional change
Although shared history has been successful overall, it led to some speed
issues with large numbers of threads. Originally we just split by NUMA node,
but on systems with non-unified L3 caches (most AMD workstation and server
CPUs, and some Intel E-core based server CPUs), this can still lead to a speed
penalty at the default config. Thus, we decided to further subdivide the shared
history based on the L3 cache structure.
Based on this test, the original SPRTs, and speed experiments, we decided that
grouping L3 domains to reach 32 threads per SharedHistories was a reasonable
balance for affected systems – but we may revisit this in the future. See the
PR for full details.
In an extreme case, a single-socket EPYC 9755 configured with 1 numa domain per socket,
the nps increases from:
Nodes/second : 182827480
to
Nodes/second : 229118365
In many cases, when L3 caches are shared between many threads, or when several
numa nodes are already configured per socket, this patch does not influence the
default. This default setting can adjusted with the existing NumaPolicy option.
closes https://github.com/official-stockfish/Stockfish/pull/6526
No functional change.
