jirayu_sri/stockfish
1
0
Fork 0
mirror of https://github.com/official-stockfish/Stockfish.git synced 2026-07-22 20:57:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,702 Commits 6 Branches 60 Tags
5fdb48a7cb4c8b34232c99530b862a3e1a672309
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Tomasz Sobczykandnodchip 5fdb48a7cb Change some learn parameter naming. Update docs. 2021-03-14 22:15:16 +09:00
Tomasz Sobczykandnodchip 591609c262 Fix relation between halfmove and fullmove clocks. 2021-03-14 22:01:01 +09:00
Tomasz Sobczykandnodchip 0ddad45ab2 Add gather_statistics command that allows gathering statistics from a .bin or .binpack file. Initially only support position count. 2021-03-01 00:36:45 +09:00
Tomasz Sobczykandnodchip 74774c36e1 Fix wrong multipv depth range. Fixes #291 2021-01-25 21:39:22 +09:00
Tomasz Sobczykandnodchip 1f7e5d3861 Add thread sanitized run for instrumented_learn and fix races. 2020-12-28 16:08:34 +09:00
Tomasz Sobczykandnodchip acf95c7c98 Accumulate clipping statistics to a 64 bit integer to prevent overflow for larger batch sizes. 2020-12-25 10:04:28 +09:00
Tomasz Sobczykandnodchip 1b560efabd Correctly handle the last batch of data in sfen_reader 2020-12-25 10:03:24 +09:00
Tomasz Sobczykandnodchip 6d28d97a91 Don't unload evalfile on set nnue false 2020-12-25 09:58:24 +09:00
Tomasz Sobczykandnodchip c1e69f450e Prevent q_ in loss calculation from reaching values that would produce NaN 2020-12-25 00:41:31 +09:00
Tomasz Sobczykandnodchip 4f6fdca31f Reduce the amount of sfens buffered for the validation step. 
Used to be 10M, now we bound it by a multiple of validation_count, and at most 1M. This reduces the RAM usage greatly.
 2020-12-25 00:17:35 +09:00
Tomasz Sobczykandnodchip 7636bcccd1 Correctly account for factors when computing the average absolute weight of the feature transformer. 2020-12-25 00:08:51 +09:00
Tomasz Sobczykandnodchip 2061be4730 smart_fen_skipping at gensfen_nonpv level 2020-12-24 21:37:30 +09:00
Tomasz Sobczykandnodchip 868b4e9421 add gensfen_nonpv docs 2020-12-24 21:37:30 +09:00
Tomasz Sobczykandnodchip 96b377a90a Add gensfen_nonpv 2020-12-24 21:37:30 +09:00
Tomasz Sobczykandnodchip 3f73c40412 More deterministic move accuracy validation. 2020-12-24 10:16:59 +09:00
Tomasz Sobczykandnodchip 8ca82646a9 Use plain nnue eval for validation loss calculation instead of first performing qsearch 2020-12-22 10:35:19 +09:00
Tomasz Sobczykandnodchip 6853b4aac2 Simple filtering for validation data. 2020-12-22 09:40:25 +09:00
Tomasz Sobczykandnodchip 50df3a7389 fix annoying warning 2020-12-22 09:24:26 +09:00
Tomasz Sobczykandnodchip 994eb5e183 rescore_fen -> rescore. Make it work on .bin and .binpack inputs. 2020-12-21 10:48:20 +09:00
Tomasz Sobczykandnodchip ffae19b5a1 Add docs for rescore_fen 2020-12-21 10:48:20 +09:00
Tomasz Sobczykandnodchip a9cfaa4d98 Add a tool for rescoring fens from an epd file with fixed depth search 2020-12-21 10:48:20 +09:00
Tomasz Sobczykandnodchip f56613ebf6 Add 'validation_count' option for 'learn' that specifies how many positions to use for validation 2020-12-20 09:47:30 +09:00
Tomasz Sobczykandnodchip a7378f3249 Make next_fen in opening_book a critical section 2020-12-14 09:03:04 +09:00
Tomasz Sobczykandnodchip d99ba07b81 Fix incorrect enpassant flag for moves read from uci format in the binpack lib 2020-12-11 00:31:32 +09:00
Tomasz Sobczykandnodchip 3a1bd1185f Add binpack coarse shuffle tool. 2020-12-06 19:08:52 +09:00
Tomasz Sobczykandnodchip 28d6d7cb03 Avoid computing gradient for validation loss. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip fafb9557a8 Get train loss from update_parameters. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 4eb0e77a2a Store references instead of copying the results of intermediate autograd computations. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 6cd0b03098 Add some comments regarding the current state of autograd loss computation. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 99cb869db3 Reintroduce use_wdl. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip cf6bc7ecaf Cleanup around get_loss 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 256c4b55ec Properly apply gradient norm clipping after it's scaled in the update_parameters. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip de675e3503 Reintroduce optional scaling of the teacher signal. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 01ae7b1e2c Simplify passing constants that may vary between calls. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip cbd973fdaa Detect constant expressions in autograd and return 0 grad early. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip e975889132 Move cross_entropy calculation to a separate function. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 891abf5511 Make the autograd loss expression chain thread_local. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 8adf00ae6e Identify a single evalation chain by ID in autograd to prevent cache reuse for subsequent evaluations of the same expression tree. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip cb812c742c Add [[nodiscard]] attributes to autograd functions. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 26f19e1429 Make automatic differentiation node types constexpr. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip aec6017195 When forming an autograd expression only copy parts that are rvalue references, store references to lvalues. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip a5c20bee5b Apply gradient clipping. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip d103867558 Add memoization to the autograd expression evaluator. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip aa55692b97 Cross entropy loss. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 539bd2d1c8 Replace the old loss/grad calculation completely. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip b71d1e8620 Pass the new loss function to update_parameters 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 5a58eb803a Loss func with autograd 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 541fb8177a More utility in autograd. 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 6ce0245787 Basic autograd 2020-12-02 08:56:20 +09:00
Tomasz Sobczykandnodchip 1322a9a5fd Prevent false sharing of num_calls counter in the shared input trainer. Fix current_operation not being local to the executing thread. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 2aa7f5290e Fix comparison of integers with different signedness. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip a97b65eaef Fix compilation error with USE_BLAS 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 622e0b14c2 Remove superfluous example shuffling. Shuffling now only happens on reading. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 34510dd08a Remove used examples asyncronously. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 0bee8fef64 Don't unnecessarily copy the batch part. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip e954b14196 Prefetch weights for feature transformer backprop to shared cache. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 8009973381 Special case for alpha=1 in saxpy, slight performance increase. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 49b2dcb1f3 Preallocate memory for unique_features. Keep the training_features temporary buffer as a thread_local so we reuse the storage. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 1c8495b54b Remove handwritten saxpy because compilers optimize the second look anyway. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 15c528ca7b Prepare feature transformer learner. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip a3c78691a2 Prepare input slice trainer. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 401fc0fbab Prepare clipped relu trainer. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 774b023641 Add chunked for each with workers. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip cc11375f6d Skeleton for new evaluate learner 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 0d4b803b08 Prepare trainer affine transform. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 4ea8572b6d Add single threaded sgemm. 2020-11-30 08:54:53 +09:00
Tomasz Sobczykandnodchip 92b14a5ba2 Add docs for transform. 2020-11-27 09:16:22 +09:00
Tomasz Sobczykandnodchip 89294e2e4f Add transform command. Add transform nudged_static subcommand. 2020-11-27 09:16:22 +09:00
Tomasz Sobczykandnodchip 45e3335ee8 Add missing docs. 2020-11-23 19:22:11 +09:00
Tomasz Sobczykandnodchip 9030020a85 Add smart_fen_skipping option to learn. 2020-11-23 19:22:11 +09:00
Tomasz Sobczykandnodchip ee13cfce67 Fix result assigned for a psvector when the positions are not continuous. 2020-11-23 08:32:08 +09:00
Tomasz Sobczykandnodchip 3cee6881ee Move the terminal position check to after qsearch, otherwise qsearch may end up in a terminal position. 2020-11-23 08:29:38 +09:00
Tomasz Sobczykandnodchip 5f18c88b3d Docs for book in gensfen. 2020-11-17 09:43:23 +09:00
Tomasz Sobczykandnodchip e1dbad47ce Add support for opening book to gensfen. 2020-11-17 09:43:23 +09:00
Tomasz Sobczykandnodchip d4350a16f3 Add representation of an opening book. 2020-11-17 09:43:23 +09:00
Tomasz Sobczykandnodchip d793663188 Add docs for max_grad option for learn 2020-11-16 10:08:56 +09:00
Tomasz Sobczykandnodchip 3dbc45bdfc Add gradient clipping. 2020-11-16 10:08:56 +09:00
Tomasz Sobczykandnodchip 50358e26c7 Fix searching terminal nodes in gensfen. 2020-11-15 22:18:13 +09:00
Tomasz Sobczykandnodchip 00bc80c3c4 Add assume_quiet option to the learner. 2020-11-15 22:18:13 +09:00
Tomasz Sobczykandnodchip 00797a3d86 add option ensure_quiet for gensfen that makes the generated position quiet 2020-11-15 22:18:13 +09:00
Tomasz Sobczykandnodchip 691da3bdad Add more information for factorizers at the start of training. 2020-11-14 18:47:22 +09:00
Tomasz Sobczykandnodchip 4e1653d53a Fix reliance on transitive includes for factorizers in trainer feature transformer. Add a file that includes all factorizers. 2020-11-14 12:35:12 +09:00
Tomasz Sobczykandnodchip 69bc3ef9be Output loss more often. 2020-11-14 12:33:25 +09:00
Tomasz Sobczykandnodchip a71623f74c Add explicit read head seek to the start of the binpack file. Otherwise on MACOS the read head is placed at the end when app is specified. 2020-11-13 19:56:36 +09:00
Tomasz Sobczykandnodchip 2a8576b804 Fix compilation issues. 2020-11-10 10:21:09 +09:00
Tomasz Sobczykandnodchip 8069963c56 Update convert docs. 2020-11-10 10:21:09 +09:00
Tomasz Sobczykandnodchip 5d88e7bce8 Add optional move validation to training data conversion. No longer rely on static initialization order for magics initialization. 2020-11-10 10:21:09 +09:00
Tomasz SobczykandJoost VandeVondele ba35c88ab8 AVX-512 for smaller affine and feature transforms. 
For the feature transformer the code is analogical to AVX2 since there was room for easy adaptation of wider simd registers.

For the smaller affine transforms that have 32 byte stride we keep 2 columns in one zmm register. We also unroll more aggressively so that in the end we have to do 16 parallel horizontal additions on ymm slices each consisting of 4 32-bit integers. The slices are embedded in 8 zmm registers.

These changes provide about 1.5% speedup for AVX-512 builds.

Closes https://github.com/official-stockfish/Stockfish/pull/3218

No functional change.
 2020-11-07 16:49:49 +01:00
Tomasz SobczykandJoost VandeVondele 3f6451eff7 Manually align arrays on the stack 
as a workaround to issues with overaligned alignas() on stack variables in gcc < 9.3 on windows.

closes https://github.com/official-stockfish/Stockfish/pull/3217

fixes #3216

No functional change
 2020-11-04 19:52:42 +01:00
Tomasz SobczykandJoost VandeVondele 75e06a1c89 Optimize affine transform for SSSE3 and higher targets. 
A non-functional speedup. Unroll the loops going over
the output dimensions in the affine transform layers by
a factor of 4 and perform 4 horizontal additions at a time.
Instead of doing naive horizontal additions on each vector
separately use hadd and shuffling between vectors to reduce
the number of instructions by using all lanes for all stages
of the horizontal adds.

passed STC of the initial version:
LLR: 2.95 (-2.94,2.94) {-0.25,1.25}
Total: 17808 W: 1914 L: 1756 D: 14138
Ptnml(0-2): 76, 1330, 5948, 1460, 90
https://tests.stockfishchess.org/tests/view/5f9d516f6a2c112b60691da3

passed STC of the final version after cleanup:
LLR: 2.95 (-2.94,2.94) {-0.25,1.25}
Total: 16296 W: 1750 L: 1595 D: 12951
Ptnml(0-2): 72, 1192, 5479, 1319, 86
https://tests.stockfishchess.org/tests/view/5f9df5776a2c112b60691de3

closes https://github.com/official-stockfish/Stockfish/pull/3203

No functional change
 2020-11-02 19:41:17 +01:00
Tomasz Sobczykandnodchip 987b6c98d4 Move the observed feature collection to the threaded part now that it can be done safely. 2020-11-01 11:02:44 +09:00
Tomasz Sobczykandnodchip c53be1b23f Add specialized bitset for use in the trainer for observed features tracking. 2020-11-01 11:02:44 +09:00
Tomasz Sobczykandnodchip e8907bcfc4 Replace omp in trainer_feature_transformer 2020-10-31 11:54:03 +09:00
Tomasz Sobczykandnodchip db1b33d4ac Optimize trainer clipped relu propagate 2020-10-31 11:52:51 +09:00
Tomasz Sobczykandnodchip b5714c4084 Parallelize input slice trainer backprop. 2020-10-31 11:52:26 +09:00
Tomasz Sobczykandnodchip 941897ff2c Optimize trainer clipped relu backpropagate. 2020-10-31 11:50:12 +09:00
Tomasz Sobczykandnodchip c96743c5bd Optimize feature transformer backpropagation stats. 2020-10-31 11:49:29 +09:00
Tomasz Sobczykandnodchip 2c10b1babc Optimize feature transformer clipped relu. 2020-10-31 11:48:02 +09:00
Tomasz Sobczykandnodchip 7bedf6c5ab Specify the whole evalsave message because otherwise the first evalsave/0 triggers it. 2020-10-31 08:36:58 +09:00
Tomasz Sobczykandnodchip 8c81bbd3db Fix the counter in for_each_index_with_workers going out of scope before workers finish. 2020-10-31 08:36:58 +09:00