Files
stockfish/src/engine.h
T
69a01b88f3 Use shared memory for network weights
This enables different Stockfish processes that use the same weights to use the
same memory. The approach establishes equivalence by memory content, and is
compatible with NUMA replication.  The benefit of sharing is reduced memory usage
and a speedup thanks to improved (inter-process) caching of the network in the
CPUs cache, and thus reduced bandwidth usage to main memory. Even though this
change doesn't benefit a user running a single process, this helps on fishtest
or e.g.  for Lichess, when multiple games run concurrently, or multiple
positions are analyzed in parallel.

This concept was probably first introduced in the Monty engine
(https://github.com/official-monty/Monty/pull/62), after a discussion in
https://github.com/official-stockfish/fishtest/issues/2077 on the issue of
memory pressure. Measurements based on Torch
(https://github.com/user-attachments/files/21386224/verbatim.pdf) further
suggested that large gains were possible. Multiple other engines have
adopted this 'verbatim' format as well.

The implementation here adds the flexibility needed for SF, for example, retains
the ability to bundle compressed networks with the binary, to load nets by uci
option, and to distribute the shared nets to the proper NUMA region. This
flexibility comes with a fair amount of complexity in the implementation, such
as OS specific code, and fallback code.

For most users this should be transparent. However, for example, those running
docker containers should ensure the `--ipc` flag is set correctly, and
`--shm-size` is sufficiently large.

The benefits of this patch significantly depend on hardware, with systems with
many cores and a large (O(150MB), the net size) L3 cache benefitting typically
most.  On such systems SF speedups (as measured via nps playing games with
large concurrency but just 1 thread) can be 38%, which results in master vs.
patch Elo which gains about 25 Elo.

```
   # PLAYER             :  RATING  ERROR   POINTS  PLAYED   (%)
   1 shared_memoryPR    :    24.8    1.9  39432.0   73728    53
   2 master             :     0.0   ----  34296.0   73728    47
```

In a multithreaded setup, where weights are already shared, that benefit is smaller,
for example on the same HW as above, but with 8t for each side.
```
   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)
   1 shared_memoryPR    :     5.2    3.5  9351.0   18432    51
   2 master             :     0.0   ----  9081.0   18432    49
```

On fishtest with a typical hardware mix of our contributors, the following was measured:

STC, 60k games
https://tests.stockfishchess.org/tests/view/69074a49ea4b268f1fac236c
Elo: 4.69 ± 1.4 (95%) LOS: 100.0%
Total: 60000 W: 16085 L: 15275 D: 28640
Ptnml(0-2): 154, 6440, 16053, 7148, 205
nElo: 9.38 ± 2.8 (95%) PairsRatio: 1.12

To verify correctness with a single process on a NUMA architecture,
speedtest was used, confirming near equivalence:
```
master:        Average (over 10):  296236186
shared_memory: Average (over 10):  295769332
```
Currently, using large pages for the shared network weights is not always possible,
which can lead to a small slowdown (1-2%), in case a single process is run.

closes https://github.com/official-stockfish/Stockfish/pull/6173

No functional change

Co-authored-by: disservin <disservin.social@gmail.com>
Co-authored-by: Joost VandeVondele <Joost.VandeVondele@gmail.com>
2025-11-02 16:04:09 +01:00

131 lines
4.3 KiB
C++

/*
Stockfish, a UCI chess playing engine derived from Glaurung 2.1
Copyright (C) 2004-2025 The Stockfish developers (see AUTHORS file)
Stockfish is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
Stockfish is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef ENGINE_H_INCLUDED
#define ENGINE_H_INCLUDED
#include <cstddef>
#include <cstdint>
#include <functional>
#include <optional>
#include <string>
#include <string_view>
#include <utility>
#include <vector>
#include "nnue/network.h"
#include "numa.h"
#include "position.h"
#include "search.h"
#include "syzygy/tbprobe.h" // for Stockfish::Depth
#include "thread.h"
#include "tt.h"
#include "ucioption.h"
namespace Stockfish {
class Engine {
public:
using InfoShort = Search::InfoShort;
using InfoFull = Search::InfoFull;
using InfoIter = Search::InfoIteration;
Engine(std::optional<std::string> path = std::nullopt);
// Cannot be movable due to components holding backreferences to fields
Engine(const Engine&) = delete;
Engine(Engine&&) = delete;
Engine& operator=(const Engine&) = delete;
Engine& operator=(Engine&&) = delete;
~Engine() { wait_for_search_finished(); }
std::uint64_t perft(const std::string& fen, Depth depth, bool isChess960);
// non blocking call to start searching
void go(Search::LimitsType&);
// non blocking call to stop searching
void stop();
// blocking call to wait for search to finish
void wait_for_search_finished();
// set a new position, moves are in UCI format
void set_position(const std::string& fen, const std::vector<std::string>& moves);
// modifiers
void set_numa_config_from_option(const std::string& o);
void resize_threads();
void set_tt_size(size_t mb);
void set_ponderhit(bool);
void search_clear();
void set_on_update_no_moves(std::function<void(const InfoShort&)>&&);
void set_on_update_full(std::function<void(const InfoFull&)>&&);
void set_on_iter(std::function<void(const InfoIter&)>&&);
void set_on_bestmove(std::function<void(std::string_view, std::string_view)>&&);
void set_on_verify_networks(std::function<void(std::string_view)>&&);
// network related
void verify_networks() const;
void load_networks();
void load_big_network(const std::string& file);
void load_small_network(const std::string& file);
void save_network(const std::pair<std::optional<std::string>, std::string> files[2]);
// utility functions
void trace_eval() const;
const OptionsMap& get_options() const;
OptionsMap& get_options();
int get_hashfull(int maxAge = 0) const;
std::string fen() const;
void flip();
std::string visualize() const;
std::vector<std::pair<size_t, size_t>> get_bound_thread_count_by_numa_node() const;
std::string get_numa_config_as_string() const;
std::string numa_config_information_as_string() const;
std::string thread_allocation_information_as_string() const;
std::string thread_binding_information_as_string() const;
private:
const std::string binaryDirectory;
NumaReplicationContext numaContext;
Position pos;
StateListPtr states;
OptionsMap options;
ThreadPool threads;
TranspositionTable tt;
LazyNumaReplicatedSystemWide<Eval::NNUE::Networks> networks;
Search::SearchManager::UpdateContext updateContext;
std::function<void(std::string_view)> onVerifyNetworks;
};
} // namespace Stockfish
#endif // #ifndef ENGINE_H_INCLUDED