mirror of
https://github.com/official-stockfish/Stockfish.git
synced 2026-07-22 12:47:08 +00:00
Use shared memory for network weights
This enables different Stockfish processes that use the same weights to use the same memory. The approach establishes equivalence by memory content, and is compatible with NUMA replication. The benefit of sharing is reduced memory usage and a speedup thanks to improved (inter-process) caching of the network in the CPUs cache, and thus reduced bandwidth usage to main memory. Even though this change doesn't benefit a user running a single process, this helps on fishtest or e.g. for Lichess, when multiple games run concurrently, or multiple positions are analyzed in parallel. This concept was probably first introduced in the Monty engine (https://github.com/official-monty/Monty/pull/62), after a discussion in https://github.com/official-stockfish/fishtest/issues/2077 on the issue of memory pressure. Measurements based on Torch (https://github.com/user-attachments/files/21386224/verbatim.pdf) further suggested that large gains were possible. Multiple other engines have adopted this 'verbatim' format as well. The implementation here adds the flexibility needed for SF, for example, retains the ability to bundle compressed networks with the binary, to load nets by uci option, and to distribute the shared nets to the proper NUMA region. This flexibility comes with a fair amount of complexity in the implementation, such as OS specific code, and fallback code. For most users this should be transparent. However, for example, those running docker containers should ensure the `--ipc` flag is set correctly, and `--shm-size` is sufficiently large. The benefits of this patch significantly depend on hardware, with systems with many cores and a large (O(150MB), the net size) L3 cache benefitting typically most. On such systems SF speedups (as measured via nps playing games with large concurrency but just 1 thread) can be 38%, which results in master vs. patch Elo which gains about 25 Elo. ``` # PLAYER : RATING ERROR POINTS PLAYED (%) 1 shared_memoryPR : 24.8 1.9 39432.0 73728 53 2 master : 0.0 ---- 34296.0 73728 47 ``` In a multithreaded setup, where weights are already shared, that benefit is smaller, for example on the same HW as above, but with 8t for each side. ``` # PLAYER : RATING ERROR POINTS PLAYED (%) 1 shared_memoryPR : 5.2 3.5 9351.0 18432 51 2 master : 0.0 ---- 9081.0 18432 49 ``` On fishtest with a typical hardware mix of our contributors, the following was measured: STC, 60k games https://tests.stockfishchess.org/tests/view/69074a49ea4b268f1fac236c Elo: 4.69 ± 1.4 (95%) LOS: 100.0% Total: 60000 W: 16085 L: 15275 D: 28640 Ptnml(0-2): 154, 6440, 16053, 7148, 205 nElo: 9.38 ± 2.8 (95%) PairsRatio: 1.12 To verify correctness with a single process on a NUMA architecture, speedtest was used, confirming near equivalence: ``` master: Average (over 10): 296236186 shared_memory: Average (over 10): 295769332 ``` Currently, using large pages for the shared network weights is not always possible, which can lead to a small slowdown (1-2%), in case a single process is run. closes https://github.com/official-stockfish/Stockfish/pull/6173 No functional change Co-authored-by: disservin <disservin.social@gmail.com> Co-authored-by: Joost VandeVondele <Joost.VandeVondele@gmail.com>
This commit is contained in:
Tomasz Sobczyk
committed by
Joost VandeVondele
Joost VandeVondele
co-authored by
disservin
Joost VandeVondele
disservin
Joost VandeVondele
parent
bc9f08731f
commit
69a01b88f3
+8
-77
@@ -55,12 +55,6 @@
|
||||
// the calls at compile time), try to load them at runtime. To do this we need
|
||||
// first to define the corresponding function pointers.
|
||||
|
||||
extern "C" {
|
||||
using OpenProcessToken_t = bool (*)(HANDLE, DWORD, PHANDLE);
|
||||
using LookupPrivilegeValueA_t = bool (*)(LPCSTR, LPCSTR, PLUID);
|
||||
using AdjustTokenPrivileges_t =
|
||||
bool (*)(HANDLE, BOOL, PTOKEN_PRIVILEGES, DWORD, PTOKEN_PRIVILEGES, PDWORD);
|
||||
}
|
||||
#endif
|
||||
|
||||
|
||||
@@ -106,77 +100,14 @@ void std_aligned_free(void* ptr) {
|
||||
|
||||
static void* aligned_large_pages_alloc_windows([[maybe_unused]] size_t allocSize) {
|
||||
|
||||
#if !defined(_WIN64)
|
||||
return nullptr;
|
||||
#else
|
||||
|
||||
HANDLE hProcessToken{};
|
||||
LUID luid{};
|
||||
void* mem = nullptr;
|
||||
|
||||
const size_t largePageSize = GetLargePageMinimum();
|
||||
if (!largePageSize)
|
||||
return nullptr;
|
||||
|
||||
// Dynamically link OpenProcessToken, LookupPrivilegeValue and AdjustTokenPrivileges
|
||||
|
||||
HMODULE hAdvapi32 = GetModuleHandle(TEXT("advapi32.dll"));
|
||||
|
||||
if (!hAdvapi32)
|
||||
hAdvapi32 = LoadLibrary(TEXT("advapi32.dll"));
|
||||
|
||||
auto OpenProcessToken_f =
|
||||
OpenProcessToken_t((void (*)()) GetProcAddress(hAdvapi32, "OpenProcessToken"));
|
||||
if (!OpenProcessToken_f)
|
||||
return nullptr;
|
||||
auto LookupPrivilegeValueA_f =
|
||||
LookupPrivilegeValueA_t((void (*)()) GetProcAddress(hAdvapi32, "LookupPrivilegeValueA"));
|
||||
if (!LookupPrivilegeValueA_f)
|
||||
return nullptr;
|
||||
auto AdjustTokenPrivileges_f =
|
||||
AdjustTokenPrivileges_t((void (*)()) GetProcAddress(hAdvapi32, "AdjustTokenPrivileges"));
|
||||
if (!AdjustTokenPrivileges_f)
|
||||
return nullptr;
|
||||
|
||||
// We need SeLockMemoryPrivilege, so try to enable it for the process
|
||||
|
||||
if (!OpenProcessToken_f( // OpenProcessToken()
|
||||
GetCurrentProcess(), TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY, &hProcessToken))
|
||||
return nullptr;
|
||||
|
||||
if (LookupPrivilegeValueA_f(nullptr, "SeLockMemoryPrivilege", &luid))
|
||||
{
|
||||
TOKEN_PRIVILEGES tp{};
|
||||
TOKEN_PRIVILEGES prevTp{};
|
||||
DWORD prevTpLen = 0;
|
||||
|
||||
tp.PrivilegeCount = 1;
|
||||
tp.Privileges[0].Luid = luid;
|
||||
tp.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED;
|
||||
|
||||
// Try to enable SeLockMemoryPrivilege. Note that even if AdjustTokenPrivileges()
|
||||
// succeeds, we still need to query GetLastError() to ensure that the privileges
|
||||
// were actually obtained.
|
||||
|
||||
if (AdjustTokenPrivileges_f(hProcessToken, FALSE, &tp, sizeof(TOKEN_PRIVILEGES), &prevTp,
|
||||
&prevTpLen)
|
||||
&& GetLastError() == ERROR_SUCCESS)
|
||||
{
|
||||
// Round up size to full pages and allocate
|
||||
allocSize = (allocSize + largePageSize - 1) & ~size_t(largePageSize - 1);
|
||||
mem = VirtualAlloc(nullptr, allocSize, MEM_RESERVE | MEM_COMMIT | MEM_LARGE_PAGES,
|
||||
PAGE_READWRITE);
|
||||
|
||||
// Privilege no longer needed, restore previous state
|
||||
AdjustTokenPrivileges_f(hProcessToken, FALSE, &prevTp, 0, nullptr, nullptr);
|
||||
}
|
||||
}
|
||||
|
||||
CloseHandle(hProcessToken);
|
||||
|
||||
return mem;
|
||||
|
||||
#endif
|
||||
return windows_try_with_large_page_priviliges(
|
||||
[&](size_t largePageSize) {
|
||||
// Round up size to full pages and allocate
|
||||
allocSize = (allocSize + largePageSize - 1) & ~size_t(largePageSize - 1);
|
||||
return VirtualAlloc(nullptr, allocSize, MEM_RESERVE | MEM_COMMIT | MEM_LARGE_PAGES,
|
||||
PAGE_READWRITE);
|
||||
},
|
||||
[]() { return (void*) nullptr; });
|
||||
}
|
||||
|
||||
void* aligned_large_pages_alloc(size_t allocSize) {
|
||||
|
||||
Reference in New Issue
Block a user