Thanks to MinetaS for discovering this and finding the culprit commit,
specific to this architecture. 9f42980 was subtly incorrect, leading to
spurious wrong benches on avx512icl. Because this piece of code has twice been
the source of wrong benches, I also added a brief comment explaining why it
needs to be this way, and added a sample position to bench.
closes https://github.com/official-stockfish/Stockfish/pull/6654
Bench: 2288704
Currently, the engine updates each level of the continuation history stack (ply -1, -2, -3, etc.) with fixed weights that are independent of one another. The idea behind CMHC is to scale the magnitude of these updates based on the consistency of the move's performance across the stack.
By tracking how often a move has historically performed well (positive history) within the current continuation context, we can adjust the update magnitude dynamically. Moves that demonstrate "continuity" (appearing as strong candidates across multiple historical contexts) receive more significant updates, while unproven or inconsistent moves are dampened.
Passed STC:
LLR: 2.94 (-2.94,2.94) <0.00,2.00>
Total: 103264 W: 26954 L: 26541 D: 49769
Ptnml(0-2): 338, 11910, 26728, 12313, 343
https://tests.stockfishchess.org/tests/view/69a81702fac54339cf435a87
Passed LTC:
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 47406 W: 12259 L: 11920 D: 23227
Ptnml(0-2): 22, 5052, 13237, 5349, 43
https://tests.stockfishchess.org/tests/view/69aab480bd35724e4b290498
closes https://github.com/official-stockfish/Stockfish/pull/6653
Bench: 2356418
Co-Authored-By: Daniel Samek <samekdaniel@email.cz>
This PR removes the variable threads.abortedSearch introduced in #4990 and
relies on threads.stop and completedDepth instead.
The logic in master is only guaranteed to work in single threaded search. For
multiple threads, as soon as threads.stop is set, some of the threads may abort
their search, and so their mated-in scores are not reliable.
Compared to master, this patch prevents unproven mated-in scores in these
multi-threaded scenarios:
in game play (when time management is used)
when the uci commands quit or stop are received
when maximum depth is reached
for go mate commands
The patch will change the bestmove in those very rare situations where the bug
occurs in master. This is because bestmove is taken from rootMoves[0].pv[0],
which will now no longer contain unproven mated-in PVs. In that sense the patch
is "functional". But of course, it does not change bench.
While at it, we also fix a bug in master that suppressed PV output for inexact
mated-in scores on receiving threads.stop. In cases where uciScore < score with
score not being a proven loss, and where the new bestmove was different from
the last completed iteration, this led to a bestmove output that did not match
the final PV (which was for an exact score).
Fixes#6293.
Fixes#6626.
closes https://github.com/official-stockfish/Stockfish/pull/6636
No functional change
Passed STC (first run):
https://tests.stockfishchess.org/tests/view/698030ee6362aee5c8a552c7
LLR: 2.96 (-2.94,2.94) <0.00,2.00>
Total: 120800 W: 31336 L: 30912 D: 58552
Ptnml(0-2): 330, 13084, 33172, 13460, 354
Passed LTC:
https://tests.stockfishchess.org/tests/view/6983946d473df9d1d24a916d
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 86850 W: 22352 L: 21951 D: 42547
Ptnml(0-2): 45, 8506, 25934, 8883, 57
Passed STC SMP (after changes made after opening the PR):
https://tests.stockfishchess.org/tests/view/6987b4c1b0f3ca5200aaf9da
LLR: 2.95 (-2.94,2.94) <0.00,2.00>
Total: 84480 W: 21994 L: 21627 D: 40859
Ptnml(0-2): 98, 9378, 22932, 9723, 109
Passed STC (second run, as a sanity check made after changes made after opening the PR -- these changed, however, didn't change assembly code produced by the compiler):
https://tests.stockfishchess.org/tests/view/6987ad21b0f3ca5200aaf9c9
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 132192 W: 34526 L: 34083 D: 63583
Ptnml(0-2): 421, 14511, 35759, 15014, 391
GCC maps __builtin_prefetch locality=1 to PREFETCHT2 instruction, targeting L2
cache. To make things clear, added template-based approach as suggested by
Disservin.
PREFETCHT2 should be the appropriate choice for the 78 MB threat weight table:
avoids L1d pollution, parks data in L2 where it is promoted to L1d on actual
load.
With 200-500 cycle lead time from make_index, data has time to arrive in L2.
suggestions by Disservin and anematode included
closes https://github.com/official-stockfish/Stockfish/pull/6602
No functional change
Replace FNV-1a hash algorithm by public domain MurmurHash64A, by Austin
Appleby, which is faster and has better hash properties. This is also
the default hash algorithm of GNU C++ Standard Library.
Startup time on a given system (with x86-64-avx2 binaries) reduces from
378ms to 332ms (46ms faster). Hashing of the largest data input (46.1MB)
goes from 44.2ms to 6.2ms, so about 7x faster.
No functional change
closes https://github.com/official-stockfish/Stockfish/pull/6579
Bench: 2570943
Even though this failed with non regression bounds on fishtest, merging
it as is to improve the current situation.
Failed Non Regression:
https://tests.stockfishchess.org/tests/view/6988814db0f3ca5200aafb50
LLR: -2.94 (-2.94,2.94) <-1.75,0.25>
Total: 185792 W: 48109 L: 48565 D: 89118
Ptnml(0-2): 553, 21228, 49811, 20730, 574
Previously we had code similar to this
```
alignas(64) std::uint8_t foo[1024];
const auto* int_ptr = reinterpret_cast<const std::int32_t*>(foo);
```
Which is a strict aliasing violation once we dereference int_ptr, which
invokes UB and causes newer gcc version to emit incorrect code.
fixes https://github.com/official-stockfish/Stockfish/issues/6598
fixes https://github.com/official-stockfish/Stockfish/issues/4932
This violation was introduced in PR #3287 and merged in commit 23c385e.
Due to aggressive compiler optimizations, users may need to include
CXXFLAGS=-fno-strict-aliasing as a build argument to ensure a correctly
functioning binary.
Also thanks to @anematode for writing parts of this and realizing this
strict aliasing violation.
closes https://github.com/official-stockfish/Stockfish/pull/6599
Bench: 2668754
Co-Authored-By: Timothy Herchen <timothy.herchen@gmail.com>
Improve `get_native_properties.sh` with a refreshed implementation.
The new version covers all Makefile ARCH variants and keeps the
script interface stable while improving portability and CPU feature
detection across supported platforms. Drop the asset file name to
avoid coupling outputs to a specific artifact naming scheme.
Refs: https://github.com/ppigazzini/get-native-properties (includes a
testing framework)
closes https://github.com/official-stockfish/Stockfish/pull/6552
No functional change.
Official release version of Stockfish 18
Bench: 2050811
---
Stockfish 18
Today, we have the pleasure of announcing Stockfish 18, a new major release. As
always, you can freely download it at [stockfishchess.org/download][1] and use
it as a drop-in replacement in the [GUI of your choice][2] to benefit from
stronger play and more accurate analysis.
Whether you can spare hours or days of CPU time, your help matters for the
ongoing development of Stockfish. Find out how you can help at
[stockfishchess.org/get-involved][3]. Join our [Discord server][4] to get in
touch with the community of developers and users of the project!
Quality of chess play
In tests against Stockfish 17, this new release brings an Elo gain of up to [46
points][5], and wins [four times as many game pairs][6] as it loses. Quality
improved throughout, including in [Fischer Random Chess][7].
Stockfish is stronger than any human, even when running on older or low-end
hardware. On the highest-end hardware, where Stockfish can search over [500
million positions per second][8], it continues to [dominate chess engine
tournaments][9].
Update highlights
Next-Generation Evaluation
This release introduces the SFNNv10 network architecture. The network’s input
layer has been augmented with 'Threat Inputs' features as part of a massive
community effort. These features allow the engine to 'see' which pieces are
threatened more naturally, resulting in more accurate evaluations.
Hardware and Performance Optimizations
A key highlight is the new 'Shared Memory' implementation, which allows
different Stockfish processes to share the same memory space for neural network
weights. This makes it the most efficient version for cloud analysis and
high-concurrency testing.
Significant efforts have also been made to utilize hardware more effectively by
adapting the code to make use of modern CPU instructions and refining how
threads interact during a search.
Search Improvements
Stockfish 18 features a heavily refined search, utilizing 'Correction History'
to dynamically adjust evaluations based on patterns found during the search
itself. These and other refinements allow the engine to detect stalemates and
evaluate fortresses significantly better than previous versions. A particularly
rare issue, involving threefold repetition detection, en passant, and pins, was
also fixed.
Refactored Training Workflow
The training of Stockfish's neural networks has transitioned to an automated
and reproducible model. This new framework allows the project to employ
standardized recipes to chain complex training stages together. This transition
facilitates the training of networks using over 100 billion positions of [Lc0
evaluation data][10].
Thank you
In this release in particular, we are deeply grateful to the contributors who
shared their research and ideas to help develop the new threat-input network
architecture.
The Stockfish project builds on a thriving community of enthusiasts (thanks to
everybody!) who contribute their expertise, time, and resources to build a free
and open-source chess engine that is robust, widely available, and very strong.
We would like to express our gratitude for the [14.6k stars][11] that light up
our GitHub project. Thank you for your support and encouragement – your
recognition means a lot to us. Programmers can contribute to the project either
directly to [Stockfish][12] (C++), to [Fishtest][13] (HTML, CSS, JavaScript,
and Python), to our trainer [nnue-pytorch][14] (C++ and Python), or to our
[website][15] (HTML, CSS/SCSS, and JavaScript).
The Stockfish team
[1]: https://stockfishchess.org/download
[2]: https://official-stockfish.github.io/docs/stockfish-wiki/Download-and-usage.html#download-a-chess-gui
[3]: https://stockfishchess.org/get-involved/
[4]: https://discord.gg/GWDRS3kU6R
[5]: https://tests.stockfishchess.org/tests/view/696a9e15cec152c6220c1d19
[6]: https://tests.stockfishchess.org/tests/view/696a9e4dcec152c6220c1d1b
[7]: https://tests.stockfishchess.org/tests/view/696a9e83cec152c6220c1d1d
[8]: https://openbenchmarking.org/test/pts/stockfish
[9]: https://en.wikipedia.org/wiki/Stockfish_(chess)#Competition_results
[10]: https://lczero.org/blog/2021/06/the-importance-of-open-data/
[11]: https://github.com/official-stockfish/Stockfish/stargazers
[12]: https://github.com/official-stockfish/Stockfish
[13]: https://github.com/official-stockfish/fishtest
[14]: https://github.com/official-stockfish/nnue-pytorch
[15]: https://github.com/official-stockfish/stockfish-web
some environment dependent execution (e.g. pid) were being std::hash'ed, and
net filenames put in unordered maps. Also uses sprintf instead of
std::to_string. Depending on precise content, this could lead to different
PGO'ed binaries. This is mitigated by using a basic hash function.
This also fixes a potential issue in net filename generation, in cases where
std::hash would use invocation dependent salt, which is not the cases today for
typical std libraries.
Closes https://github.com/official-stockfish/Stockfish/pull/6562
No functional change
With the current setup, on Linux, SIGILL (and SIGSEGV/SIGBUS etc., in the case
of a program bug) lead to no feedback if they occur after the signal handlers
are installed, they just exit silently. By invoking the default signal handler,
we can still get the appropriate feedback. This is particularly important for
feedback if someone downloads the wrong SF binary and runs it on Linux.
closes https://github.com/official-stockfish/Stockfish/pull/6554
No functional change
