Windows 10 is missing the GroupMasks and GroupCount members, this breaks compiles on Windows 10.
Windows 11 builds, including the official ones, run fine on Windows 10/11.
To support developers/testers on Windows 10, fallback conditionally to the Windows 10 struct definition.
closes https://github.com/official-stockfish/Stockfish/pull/6538
No functional change
Although shared history has been successful overall, it led to some speed
issues with large numbers of threads. Originally we just split by NUMA node,
but on systems with non-unified L3 caches (most AMD workstation and server
CPUs, and some Intel E-core based server CPUs), this can still lead to a speed
penalty at the default config. Thus, we decided to further subdivide the shared
history based on the L3 cache structure.
Based on this test, the original SPRTs, and speed experiments, we decided that
grouping L3 domains to reach 32 threads per SharedHistories was a reasonable
balance for affected systems – but we may revisit this in the future. See the
PR for full details.
In an extreme case, a single-socket EPYC 9755 configured with 1 numa domain per socket,
the nps increases from:
Nodes/second : 182827480
to
Nodes/second : 229118365
In many cases, when L3 caches are shared between many threads, or when several
numa nodes are already configured per socket, this patch does not influence the
default. This default setting can adjusted with the existing NumaPolicy option.
closes https://github.com/official-stockfish/Stockfish/pull/6526
No functional change.
This enables different Stockfish processes that use the same weights to use the
same memory. The approach establishes equivalence by memory content, and is
compatible with NUMA replication. The benefit of sharing is reduced memory usage
and a speedup thanks to improved (inter-process) caching of the network in the
CPUs cache, and thus reduced bandwidth usage to main memory. Even though this
change doesn't benefit a user running a single process, this helps on fishtest
or e.g. for Lichess, when multiple games run concurrently, or multiple
positions are analyzed in parallel.
This concept was probably first introduced in the Monty engine
(https://github.com/official-monty/Monty/pull/62), after a discussion in
https://github.com/official-stockfish/fishtest/issues/2077 on the issue of
memory pressure. Measurements based on Torch
(https://github.com/user-attachments/files/21386224/verbatim.pdf) further
suggested that large gains were possible. Multiple other engines have
adopted this 'verbatim' format as well.
The implementation here adds the flexibility needed for SF, for example, retains
the ability to bundle compressed networks with the binary, to load nets by uci
option, and to distribute the shared nets to the proper NUMA region. This
flexibility comes with a fair amount of complexity in the implementation, such
as OS specific code, and fallback code.
For most users this should be transparent. However, for example, those running
docker containers should ensure the `--ipc` flag is set correctly, and
`--shm-size` is sufficiently large.
The benefits of this patch significantly depend on hardware, with systems with
many cores and a large (O(150MB), the net size) L3 cache benefitting typically
most. On such systems SF speedups (as measured via nps playing games with
large concurrency but just 1 thread) can be 38%, which results in master vs.
patch Elo which gains about 25 Elo.
```
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 shared_memoryPR : 24.8 1.9 39432.0 73728 53
2 master : 0.0 ---- 34296.0 73728 47
```
In a multithreaded setup, where weights are already shared, that benefit is smaller,
for example on the same HW as above, but with 8t for each side.
```
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 shared_memoryPR : 5.2 3.5 9351.0 18432 51
2 master : 0.0 ---- 9081.0 18432 49
```
On fishtest with a typical hardware mix of our contributors, the following was measured:
STC, 60k games
https://tests.stockfishchess.org/tests/view/69074a49ea4b268f1fac236c
Elo: 4.69 ± 1.4 (95%) LOS: 100.0%
Total: 60000 W: 16085 L: 15275 D: 28640
Ptnml(0-2): 154, 6440, 16053, 7148, 205
nElo: 9.38 ± 2.8 (95%) PairsRatio: 1.12
To verify correctness with a single process on a NUMA architecture,
speedtest was used, confirming near equivalence:
```
master: Average (over 10): 296236186
shared_memory: Average (over 10): 295769332
```
Currently, using large pages for the shared network weights is not always possible,
which can lead to a small slowdown (1-2%), in case a single process is run.
closes https://github.com/official-stockfish/Stockfish/pull/6173
No functional change
Co-authored-by: disservin <disservin.social@gmail.com>
Co-authored-by: Joost VandeVondele <Joost.VandeVondele@gmail.com>
`speedtest [threads] [hash_MiB] [time_s]`. `threads` default to system concurrency. `hash_MiB` defaults to `threads*128`. `time_s` defaults to 150.
Intended to be used with default parameters, as a stable hardware benchmark.
Example:
```
C:\dev\stockfish-master\src>stockfish.exe speedtest
Stockfish dev-20240928-nogit by the Stockfish developers (see AUTHORS file)
info string Using 16 threads
Warmup position 3/3
Position 258/258
===========================
Version : Stockfish dev-20240928-nogit
Compiled by : g++ (GNUC) 13.2.0 on MinGW64
Compilation architecture : x86-64-vnni256
Compilation settings : 64bit VNNI BMI2 AVX2 SSE41 SSSE3 SSE2 POPCNT
Compiler __VERSION__ macro : 13.2.0
Large pages : yes
User invocation : speedtest
Filled invocation : speedtest 16 2048 150
Available processors : 0-15
Thread count : 16
Thread binding : none
TT size [MiB] : 2048
Hash max, avg [per mille] :
single search : 40, 21
single game : 631, 428
Total nodes searched : 2099917842
Total search time [s] : 153.937
Nodes/second : 13641410
```
-------------------------------
Small unrelated tweaks:
- Network verification output is now handled as a callback.
- TT hashfull queries allow specifying maximum entry age.
closes https://github.com/official-stockfish/Stockfish/pull/5354
No functional change
- Capitalize comments
- Reformat multi-lines comments to equalize the widths of the lines
- Try to keep the width of comments around 85 characters
- Remove periods at the end of single-line comments
closes https://github.com/official-stockfish/Stockfish/pull/5469
No functional change
1. Fix GetProcessGroupAffinity still not getting properly aligned memory
sometimes.
2. Fix a very theoretically possible heap corruption if
GetActiveProcessorGroupCount changes between calls.
3. Fully determine affinity on Windows 11 and Windows Server 2022. It
should only ever be indeterminate in case of an error.
4. Separate isDeterminate for old and new API, as they are &'d together
we still can end up with a subset of processors even if one API is
indeterminate.
5. likely_used_old_api() that is based on actual affinity that's been
detected
6. IMPORTANT: Gather affinities at startup, so that we only later use
the affinites set at startup. Not only does this prevent us from our
own calls interfering with detection but it also means subsequent
setoption NumaPolicy calls should behave as expected.
7. Fix ERROR_INSUFFICIENT_BUFFER from GetThreadSelectedCpuSetMasks being
treated like an error.
Should resolve
https://github.com/vondele/Stockfish/commit/02ff76630b358e5f958793cc93df0009d2da65a5#commitcomment-142790025
closes https://github.com/official-stockfish/Stockfish/pull/5372
Bench: 1231853
Can be used in case a GUI (e.g. ChessBase 17 see #5307) sets affinity to a
single processor group, but the user would like to use the full capabilities of
the hardware. Improves affinity handling on Windows in case of multiple
available APIs and existing affinities.
closes https://github.com/official-stockfish/Stockfish/pull/5353
No functional change
Previously, we had two type aliases, LargePagePtr and AlignedPtr, which
required manually initializing the aligned memory for the pointer.
The new helpers:
- make_unique_aligned
- make_unique_large_page
are now available for allocating aligned memory (with large pages). They
behave similarly to std::make_unique, ensuring objects allocated with
these functions follow RAII.
The old approach had issues with initializing non-trivial types or
arrays of objects. The evaluation function of the network is now a
unique pointer to an array instead of an array of unique pointers.
Memory related functions have been moved into memory.h
Passed High Hash Pressure Test Non-Regression STC:
https://tests.stockfishchess.org/tests/view/665b2b36586058766677cfd2
LLR: 2.93 (-2.94,2.94) <-1.75,0.25>
Total: 476992 W: 122426 L: 122677 D: 231889
Ptnml(0-2): 1145, 51027, 134419, 50744, 1161
Failed Normal Non-Regression STC:
https://tests.stockfishchess.org/tests/view/665b2997586058766677cfc8
LLR: -2.94 (-2.94,2.94) <-1.75,0.25>
Total: 877312 W: 225233 L: 226395 D: 425684
Ptnml(0-2): 2110, 94642, 246239, 93630, 2035
Probably a fluke since there shouldn't be a real slowndown and it has also
passed the high hash pressure test.
closes https://github.com/official-stockfish/Stockfish/pull/5332
No functional change
Specialize and privatize NumaConfig::get_process_affinity.
Only enable NUMA capability for 64-bit Windows.
Following #5307 and some more testing it was determined that the way affinity
was being determined on Windows was incorrect, based on incorrect assumptions
about GetNumaProcessorNodeEx.
This patch fixes the issue by attempting to retrieve the actual process'
processor affinity using Windows API. However one issue persists that is not
addressable due to limitations of Windows, and will have to be considered a
limitation. If affinities were set using SetThreadAffinityMask instead of
SetThreadSelectedCpuSetMasks and GetProcessGroupAffinity returns more than 1
group it is NOT POSSIBLE to determine the affinity programmatically on Windows.
In such case the implementation assumes no affinites are set and will consider
all processors available for execution.
closes https://github.com/official-stockfish/Stockfish/pull/5312
No functional change
Allow for NUMA memory replication for NNUE weights. Bind threads to ensure execution on a specific NUMA node.
This patch introduces NUMA memory replication, currently only utilized for the NNUE weights. Along with it comes all machinery required to identify NUMA nodes and bind threads to specific processors/nodes. It also comes with small changes to Thread and ThreadPool to allow easier execution of custom functions on the designated thread. Old thread binding (WinProcGroup) machinery is removed because it's incompatible with this patch. Small changes to unrelated parts of the code were made to ensure correctness, like some classes being made unmovable, raw pointers replaced with unique_ptr. etc.
Windows 7 and Windows 10 is partially supported. Windows 11 is fully supported. Linux is fully supported, with explicit exclusion of Android. No additional dependencies.
-----------------
A new UCI option `NumaPolicy` is introduced. It can take the following values:
```
system - gathers NUMA node information from the system (lscpu or windows api), for each threads binds it to a single NUMA node
none - assumes there is 1 NUMA node, never binds threads
auto - this is the default value, depends on the number of set threads and NUMA nodes, will only enable binding on multinode systems and when the number of threads reaches a threshold (dependent on node size and count)
[[custom]] -
// ':'-separated numa nodes
// ','-separated cpu indices
// supports "first-last" range syntax for cpu indices,
for example '0-15,32-47:16-31,48-63'
```
Setting `NumaPolicy` forces recreation of the threads in the ThreadPool, which in turn forces the recreation of the TT.
The threads are distributed among NUMA nodes in a round-robin fashion based on fill percentage (i.e. it will strive to fill all NUMA nodes evenly). Threads are bound to NUMA nodes, not specific processors, because that's our only requirement and the OS can schedule them better.
Special care is made that maximum memory usage on systems that do not require memory replication stays as previously, that is, unnecessary copies are avoided.
On linux the process' processor affinity is respected. This means that if you for example use taskset to restrict Stockfish to a single NUMA node then the `system` and `auto` settings will only see a single NUMA node (more precisely, the processors included in the current affinity mask) and act accordingly.
-----------------
We can't ensure that a memory allocation takes place on a given NUMA node without using libnuma on linux, or using appropriate custom allocators on windows (https://learn.microsoft.com/en-us/windows/win32/memory/allocating-memory-from-a-numa-node), so to avoid complications the current implementation relies on first-touch policy. Due to this we also rely on the memory allocator to give us a new chunk of untouched memory from the system. This appears to work reliably on linux, but results may vary.
MacOS is not supported, because AFAIK it's not affected, and implementation would be problematic anyway.
Windows is supported since Windows 7 (https://learn.microsoft.com/en-us/windows/win32/api/processtopologyapi/nf-processtopologyapi-setthreadgroupaffinity). Until Windows 11/Server 2022 NUMA nodes are split such that they cannot span processor groups. This is because before Windows 11/Server 2022 it's not possible to set thread affinity spanning processor groups. The splitting is done manually in some cases (required after Windows 10 Build 20348). Since Windows 11/Server 2022 we can set affinites spanning processor group so this splitting is not done, so the behaviour is pretty much like on linux.
Linux is supported, **without** libnuma requirement. `lscpu` is expected.
-----------------
Passed 60+1 @ 256t 16000MB hash: https://tests.stockfishchess.org/tests/view/6654e443a86388d5e27db0d8
```
LLR: 2.95 (-2.94,2.94) <0.00,10.00>
Total: 278 W: 110 L: 29 D: 139
Ptnml(0-2): 0, 1, 56, 82, 0
```
Passed SMP STC: https://tests.stockfishchess.org/tests/view/6654fc74a86388d5e27db1cd
```
LLR: 2.95 (-2.94,2.94) <-1.75,0.25>
Total: 67152 W: 17354 L: 17177 D: 32621
Ptnml(0-2): 64, 7428, 18408, 7619, 57
```
Passed STC: https://tests.stockfishchess.org/tests/view/6654fb27a86388d5e27db15c
```
LLR: 2.94 (-2.94,2.94) <-1.75,0.25>
Total: 131648 W: 34155 L: 34045 D: 63448
Ptnml(0-2): 426, 13878, 37096, 14008, 416
```
fixes#5253
closes https://github.com/official-stockfish/Stockfish/pull/5285
No functional change
