Improve `get_native_properties.sh` with a refreshed implementation.
The new version covers all Makefile ARCH variants and keeps the
script interface stable while improving portability and CPU feature
detection across supported platforms. Drop the asset file name to
avoid coupling outputs to a specific artifact naming scheme.
Refs: https://github.com/ppigazzini/get-native-properties (includes a
testing framework)
closes https://github.com/official-stockfish/Stockfish/pull/6552
No functional change.
Official release version of Stockfish 18
Bench: 2050811
---
Stockfish 18
Today, we have the pleasure of announcing Stockfish 18, a new major release. As
always, you can freely download it at [stockfishchess.org/download][1] and use
it as a drop-in replacement in the [GUI of your choice][2] to benefit from
stronger play and more accurate analysis.
Whether you can spare hours or days of CPU time, your help matters for the
ongoing development of Stockfish. Find out how you can help at
[stockfishchess.org/get-involved][3]. Join our [Discord server][4] to get in
touch with the community of developers and users of the project!
Quality of chess play
In tests against Stockfish 17, this new release brings an Elo gain of up to [46
points][5], and wins [four times as many game pairs][6] as it loses. Quality
improved throughout, including in [Fischer Random Chess][7].
Stockfish is stronger than any human, even when running on older or low-end
hardware. On the highest-end hardware, where Stockfish can search over [500
million positions per second][8], it continues to [dominate chess engine
tournaments][9].
Update highlights
Next-Generation Evaluation
This release introduces the SFNNv10 network architecture. The network’s input
layer has been augmented with 'Threat Inputs' features as part of a massive
community effort. These features allow the engine to 'see' which pieces are
threatened more naturally, resulting in more accurate evaluations.
Hardware and Performance Optimizations
A key highlight is the new 'Shared Memory' implementation, which allows
different Stockfish processes to share the same memory space for neural network
weights. This makes it the most efficient version for cloud analysis and
high-concurrency testing.
Significant efforts have also been made to utilize hardware more effectively by
adapting the code to make use of modern CPU instructions and refining how
threads interact during a search.
Search Improvements
Stockfish 18 features a heavily refined search, utilizing 'Correction History'
to dynamically adjust evaluations based on patterns found during the search
itself. These and other refinements allow the engine to detect stalemates and
evaluate fortresses significantly better than previous versions. A particularly
rare issue, involving threefold repetition detection, en passant, and pins, was
also fixed.
Refactored Training Workflow
The training of Stockfish's neural networks has transitioned to an automated
and reproducible model. This new framework allows the project to employ
standardized recipes to chain complex training stages together. This transition
facilitates the training of networks using over 100 billion positions of [Lc0
evaluation data][10].
Thank you
In this release in particular, we are deeply grateful to the contributors who
shared their research and ideas to help develop the new threat-input network
architecture.
The Stockfish project builds on a thriving community of enthusiasts (thanks to
everybody!) who contribute their expertise, time, and resources to build a free
and open-source chess engine that is robust, widely available, and very strong.
We would like to express our gratitude for the [14.6k stars][11] that light up
our GitHub project. Thank you for your support and encouragement – your
recognition means a lot to us. Programmers can contribute to the project either
directly to [Stockfish][12] (C++), to [Fishtest][13] (HTML, CSS, JavaScript,
and Python), to our trainer [nnue-pytorch][14] (C++ and Python), or to our
[website][15] (HTML, CSS/SCSS, and JavaScript).
The Stockfish team
[1]: https://stockfishchess.org/download
[2]: https://official-stockfish.github.io/docs/stockfish-wiki/Download-and-usage.html#download-a-chess-gui
[3]: https://stockfishchess.org/get-involved/
[4]: https://discord.gg/GWDRS3kU6R
[5]: https://tests.stockfishchess.org/tests/view/696a9e15cec152c6220c1d19
[6]: https://tests.stockfishchess.org/tests/view/696a9e4dcec152c6220c1d1b
[7]: https://tests.stockfishchess.org/tests/view/696a9e83cec152c6220c1d1d
[8]: https://openbenchmarking.org/test/pts/stockfish
[9]: https://en.wikipedia.org/wiki/Stockfish_(chess)#Competition_results
[10]: https://lczero.org/blog/2021/06/the-importance-of-open-data/
[11]: https://github.com/official-stockfish/Stockfish/stargazers
[12]: https://github.com/official-stockfish/Stockfish
[13]: https://github.com/official-stockfish/fishtest
[14]: https://github.com/official-stockfish/nnue-pytorch
[15]: https://github.com/official-stockfish/stockfish-web
some environment dependent execution (e.g. pid) were being std::hash'ed, and
net filenames put in unordered maps. Also uses sprintf instead of
std::to_string. Depending on precise content, this could lead to different
PGO'ed binaries. This is mitigated by using a basic hash function.
This also fixes a potential issue in net filename generation, in cases where
std::hash would use invocation dependent salt, which is not the cases today for
typical std libraries.
Closes https://github.com/official-stockfish/Stockfish/pull/6562
No functional change
With the current setup, on Linux, SIGILL (and SIGSEGV/SIGBUS etc., in the case
of a program bug) lead to no feedback if they occur after the signal handlers
are installed, they just exit silently. By invoking the default signal handler,
we can still get the appropriate feedback. This is particularly important for
feedback if someone downloads the wrong SF binary and runs it on Linux.
closes https://github.com/official-stockfish/Stockfish/pull/6554
No functional change
This PR updates the internal WDL model, using data from 3.1M games played by
the revisions since 44d5467. Note that the normalizing constant increases only
moderately from 377 to 385.
closes https://github.com/official-stockfish/Stockfish/pull/6549
No functional change
avoid munmap of memory when exiting via signal, which avoids side effects such
as triggering asserts or (caught) segfaults while the process exists.
closes https://github.com/official-stockfish/Stockfish/pull/6542
No functional change
Windows 10 is missing the GroupMasks and GroupCount members, this breaks compiles on Windows 10.
Windows 11 builds, including the official ones, run fine on Windows 10/11.
To support developers/testers on Windows 10, fallback conditionally to the Windows 10 struct definition.
closes https://github.com/official-stockfish/Stockfish/pull/6538
No functional change
Although shared history has been successful overall, it led to some speed
issues with large numbers of threads. Originally we just split by NUMA node,
but on systems with non-unified L3 caches (most AMD workstation and server
CPUs, and some Intel E-core based server CPUs), this can still lead to a speed
penalty at the default config. Thus, we decided to further subdivide the shared
history based on the L3 cache structure.
Based on this test, the original SPRTs, and speed experiments, we decided that
grouping L3 domains to reach 32 threads per SharedHistories was a reasonable
balance for affected systems – but we may revisit this in the future. See the
PR for full details.
In an extreme case, a single-socket EPYC 9755 configured with 1 numa domain per socket,
the nps increases from:
Nodes/second : 182827480
to
Nodes/second : 229118365
In many cases, when L3 caches are shared between many threads, or when several
numa nodes are already configured per socket, this patch does not influence the
default. This default setting can adjusted with the existing NumaPolicy option.
closes https://github.com/official-stockfish/Stockfish/pull/6526
No functional change.
Only the one on line 158 is actually required but doesn't hurt to add constexpr where applicable here.
Warning was
"comparison of unsigned expression in '< 0' is always false"
closes https://github.com/official-stockfish/Stockfish/pull/6530
No functional change
Recent changes to the Square enum (reducing it from int32_t to int8_t)
now allow the compiler to vectorize loops that were previously too wide
for targets below AVX-512. However, this vectorization which Clang
performs is not correct and causes a miscompilation.
Disable this vectorization.
This particular issue was noticable with Clang 15 and Clang 19,
on avx2 as well as applie-silicon.
Ref: #6063
Original Clang Issue: llvm/llvm-project#80494
First reported by #6528, though misinterpreted.
closes#6529
No functional change
WINE_PATH started as a Wine-specific knob, but it’s now used more generally
as a command prefix to run the built engine under wrappers
like Intel SDE, qemu-user, etc.
- Add RUN_PREFIX as the supported “run wrapper/prefix” variable in Makefile
- Set WINE_PATH as a deprecated alias
- Update CI and scripts to use RUN_PREFIX
closes https://github.com/official-stockfish/Stockfish/pull/6500
No functional change
[Passed STC SMP](https://tests.stockfishchess.org/tests/view/694e506c572093c1986d7276):
```
LLR: 2.97 (-2.94,2.94) <0.00,2.00>
Total: 14992 W: 3924 L: 3653 D: 7415
Ptnml(0-2): 20, 1547, 4090, 1820, 19
```
[Passed LTC SMP](https://tests.stockfishchess.org/tests/live_elo/694ead61572093c1986d7365):
```
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 41146 W: 10654 L: 10342 D: 20150
Ptnml(0-2): 17, 3999, 12225, 4319, 13
```
[Passed a sanity check STC SMP post-refactoring](https://tests.stockfishchess.org/tests/view/69503997572093c1986d763a):
```
LLR: 2.94 (-2.94,2.94) <0.00,2.00>
Total: 46728 W: 12178 L: 11863 D: 22687
Ptnml(0-2): 82, 5093, 12685, 5436, 68
```
(The large gain of the first STC was probably a fluke, and this result
is more reasonable!)
After shared correction history, Viz suggested we try sharing other
histories, especially `pawnHistory`. As far as we're aware, sharing
history besides correction history (like Caissa does) is novel. The
implementation follows the same pattern as shared correction history –
the size of the history table is scaled with
`next_power_of_two(threadsInNumaNode)` and the entry is prefetched in
`do_move`.
A bit of refactoring was done to accommodate this new history. Note that
we prefetch `&history->pawn_entry(*this)[pc][to]` rather than
`&history->pawn_entry(*this)` because unlike the other entries, each
entry contains multiple cache lines.
closes https://github.com/official-stockfish/Stockfish/pull/6498
Bench: 2503391
Co-authored-by: Michael Chaly <Vizvezdenec@gmail.com>
We did quite a few tests because this is a pretty involved change with
unknown scaling behavior, but results are decent.
[STC 10+0.1 1th, non-regression](https://tests.stockfishchess.org/tests/live_elo/6941ce3b46f342e1ec210180)
```
LLR: 2.93 (-2.94,2.94) <-1.75,0.25>
Total: 83200 W: 21615 L: 21452 D: 40133
Ptnml(0-2): 247, 9064, 22844, 9169, 276
```
[STC 5+0.05 8th](https://tests.stockfishchess.org/tests/live_elo/693dc38346f342e1ec20f555)
```
LLR: 3.48 (-2.94,2.94) <0.00,2.00>
Total: 58536 W: 15067 L: 14688 D: 28781
Ptnml(0-2): 87, 6474, 15781, 6825, 101
```
[LTC 20+0.2 8th](https://tests.stockfishchess.org/tests/live_elo/693f2afb46f342e1ec20f847)
```
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 27716 W: 7211 L: 6925 D: 13580
Ptnml(0-2): 8, 2674, 8207, 2962, 7
```
[LTC 10+0.1 64th](https://tests.stockfishchess.org/tests/live_elo/694003aa46f342e1ec20fac4):
```
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 16918 W: 4439 L: 4182 D: 8297
Ptnml(0-2): 3, 1493, 5213, 1744, 6
```
[NUMA test, 5+0.05 256th](https://tests.stockfishchess.org/tests/view/6941ee4e46f342e1ec210203)
```
LLR: 2.95 (-2.94,2.94) <0.00,2.00>
Total: 7124 W: 1910 L: 1678 D: 3536
Ptnml(0-2): 0, 560, 2211, 790, 1
```
[LTC 60+0.6 64th](https://tests.stockfishchess.org/tests/live_elo/6940a85346f342e1ec20fcde):
```
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 15504 W: 4045 L: 3826 D: 7633
Ptnml(0-2): 0, 1002, 5530, 1219, 1
```
Bonus (courtesy of Viz): The 1 double kill in this last test was master
blundering a cool mate in 3: https://lichess.org/jyNZuRl4
Basically the idea here is to share correction history between threads.
That way, T1 can use the correction values produced by T2, which already
searched positions with that pawn structure etc., so that T1 can search
more efficiently. The table size per thread is about the same, so we
shouldn't get a large increase in hash collisions; in fact, I'd expect a
lower collision rate overall.
Although I came up with and implemented the idea independently,
[Caissa](https://github.com/Witek902/Caissa) was the first engine to
implement corrhist sharing (and corrhist in the first place) – this idea
is not completely novel.
The table size is rounded to a power of two. In particular, it's `65536
* nextPowerOfTwo(threadCount)`. That way, the indexing operation becomes
an AND of the key bits with a mask, rather than something more expensive
(e.g., a `mul_hi64`-style approach or a modulo).
The updates are racy, like the TT, but because `entry` is hoisted into a
register, there's no risk of writing back a value that's out of the
designated range `[-D, D]`. Various attempts at rewriting using atomics
led to substantial slowdowns, so we begrudgingly ignored the functions
in thread sanitizer, but at some point we'd like to make this better.
We allocate one shared correction history per NUMA node, because the
penalty associated with crossing nodes is substantial – I get a 40% hit
with NPS=4 and 256 threads, which is intolerable. With separate tables
per NUMA node I get a 6% penalty for nodes per second, which isn't ideal
but apparently compensated for.
closes https://github.com/official-stockfish/Stockfish/pull/6478
Bench: 2690604
Co-authored-by: Disservin <disservin.social@gmail.com>
