Tomasz Sobczyk and nodchip
a97b65eaef
Fix compilation error with USE_BLAS
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
622e0b14c2
Remove superfluous example shuffling. Shuffling now only happens on reading.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
34510dd08a
Remove used examples asyncronously.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
0bee8fef64
Don't unnecessarily copy the batch part.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
e954b14196
Prefetch weights for feature transformer backprop to shared cache.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
8009973381
Special case for alpha=1 in saxpy, slight performance increase.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
49b2dcb1f3
Preallocate memory for unique_features. Keep the training_features temporary buffer as a thread_local so we reuse the storage.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
1c8495b54b
Remove handwritten saxpy because compilers optimize the second look anyway.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
15c528ca7b
Prepare feature transformer learner.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
a3c78691a2
Prepare input slice trainer.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
401fc0fbab
Prepare clipped relu trainer.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
774b023641
Add chunked for each with workers.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
cc11375f6d
Skeleton for new evaluate learner
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
0d4b803b08
Prepare trainer affine transform.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
4ea8572b6d
Add single threaded sgemm.
2020-11-30 08:54:53 +09:00
Tomasz Sobczyk and nodchip
92b14a5ba2
Add docs for transform.
2020-11-27 09:16:22 +09:00
Tomasz Sobczyk and nodchip
89294e2e4f
Add transform command. Add transform nudged_static subcommand.
2020-11-27 09:16:22 +09:00
Tomasz Sobczyk and nodchip
45e3335ee8
Add missing docs.
2020-11-23 19:22:11 +09:00
Tomasz Sobczyk and nodchip
9030020a85
Add smart_fen_skipping option to learn.
2020-11-23 19:22:11 +09:00
Tomasz Sobczyk and nodchip
ee13cfce67
Fix result assigned for a psvector when the positions are not continuous.
2020-11-23 08:32:08 +09:00
Tomasz Sobczyk and nodchip
3cee6881ee
Move the terminal position check to after qsearch, otherwise qsearch may end up in a terminal position.
2020-11-23 08:29:38 +09:00
Tomasz Sobczyk and nodchip
5f18c88b3d
Docs for book in gensfen.
2020-11-17 09:43:23 +09:00
Tomasz Sobczyk and nodchip
e1dbad47ce
Add support for opening book to gensfen.
2020-11-17 09:43:23 +09:00
Tomasz Sobczyk and nodchip
d4350a16f3
Add representation of an opening book.
2020-11-17 09:43:23 +09:00
Tomasz Sobczyk and nodchip
d793663188
Add docs for max_grad option for learn
2020-11-16 10:08:56 +09:00
Tomasz Sobczyk and nodchip
3dbc45bdfc
Add gradient clipping.
2020-11-16 10:08:56 +09:00
Tomasz Sobczyk and nodchip
50358e26c7
Fix searching terminal nodes in gensfen.
2020-11-15 22:18:13 +09:00
Tomasz Sobczyk and nodchip
00bc80c3c4
Add assume_quiet option to the learner.
2020-11-15 22:18:13 +09:00
Tomasz Sobczyk and nodchip
00797a3d86
add option ensure_quiet for gensfen that makes the generated position quiet
2020-11-15 22:18:13 +09:00
Tomasz Sobczyk and nodchip
691da3bdad
Add more information for factorizers at the start of training.
2020-11-14 18:47:22 +09:00
Tomasz Sobczyk and nodchip
4e1653d53a
Fix reliance on transitive includes for factorizers in trainer feature transformer. Add a file that includes all factorizers.
2020-11-14 12:35:12 +09:00
Tomasz Sobczyk and nodchip
69bc3ef9be
Output loss more often.
2020-11-14 12:33:25 +09:00
Tomasz Sobczyk and nodchip
a71623f74c
Add explicit read head seek to the start of the binpack file. Otherwise on MACOS the read head is placed at the end when app is specified.
2020-11-13 19:56:36 +09:00
Tomasz Sobczyk and nodchip
2a8576b804
Fix compilation issues.
2020-11-10 10:21:09 +09:00
Tomasz Sobczyk and nodchip
8069963c56
Update convert docs.
2020-11-10 10:21:09 +09:00
Tomasz Sobczyk and nodchip
5d88e7bce8
Add optional move validation to training data conversion. No longer rely on static initialization order for magics initialization.
2020-11-10 10:21:09 +09:00
Tomasz Sobczyk and Joost VandeVondele
ba35c88ab8
AVX-512 for smaller affine and feature transforms.
...
For the feature transformer the code is analogical to AVX2 since there was room for easy adaptation of wider simd registers.
For the smaller affine transforms that have 32 byte stride we keep 2 columns in one zmm register. We also unroll more aggressively so that in the end we have to do 16 parallel horizontal additions on ymm slices each consisting of 4 32-bit integers. The slices are embedded in 8 zmm registers.
These changes provide about 1.5% speedup for AVX-512 builds.
Closes https://github.com/official-stockfish/Stockfish/pull/3218
No functional change.
2020-11-07 16:49:49 +01:00
Tomasz Sobczyk and Joost VandeVondele
3f6451eff7
Manually align arrays on the stack
...
as a workaround to issues with overaligned alignas() on stack variables in gcc < 9.3 on windows.
closes https://github.com/official-stockfish/Stockfish/pull/3217
fixes #3216
No functional change
2020-11-04 19:52:42 +01:00
Tomasz Sobczyk and Joost VandeVondele
75e06a1c89
Optimize affine transform for SSSE3 and higher targets.
...
A non-functional speedup. Unroll the loops going over
the output dimensions in the affine transform layers by
a factor of 4 and perform 4 horizontal additions at a time.
Instead of doing naive horizontal additions on each vector
separately use hadd and shuffling between vectors to reduce
the number of instructions by using all lanes for all stages
of the horizontal adds.
passed STC of the initial version:
LLR: 2.95 (-2.94,2.94) {-0.25,1.25}
Total: 17808 W: 1914 L: 1756 D: 14138
Ptnml(0-2): 76, 1330, 5948, 1460, 90
https://tests.stockfishchess.org/tests/view/5f9d516f6a2c112b60691da3
passed STC of the final version after cleanup:
LLR: 2.95 (-2.94,2.94) {-0.25,1.25}
Total: 16296 W: 1750 L: 1595 D: 12951
Ptnml(0-2): 72, 1192, 5479, 1319, 86
https://tests.stockfishchess.org/tests/view/5f9df5776a2c112b60691de3
closes https://github.com/official-stockfish/Stockfish/pull/3203
No functional change
2020-11-02 19:41:17 +01:00
Tomasz Sobczyk and nodchip
987b6c98d4
Move the observed feature collection to the threaded part now that it can be done safely.
2020-11-01 11:02:44 +09:00
Tomasz Sobczyk and nodchip
c53be1b23f
Add specialized bitset for use in the trainer for observed features tracking.
2020-11-01 11:02:44 +09:00
Tomasz Sobczyk and nodchip
e8907bcfc4
Replace omp in trainer_feature_transformer
2020-10-31 11:54:03 +09:00
Tomasz Sobczyk and nodchip
db1b33d4ac
Optimize trainer clipped relu propagate
2020-10-31 11:52:51 +09:00
Tomasz Sobczyk and nodchip
b5714c4084
Parallelize input slice trainer backprop.
2020-10-31 11:52:26 +09:00
Tomasz Sobczyk and nodchip
941897ff2c
Optimize trainer clipped relu backpropagate.
2020-10-31 11:50:12 +09:00
Tomasz Sobczyk and nodchip
c96743c5bd
Optimize feature transformer backpropagation stats.
2020-10-31 11:49:29 +09:00
Tomasz Sobczyk and nodchip
2c10b1babc
Optimize feature transformer clipped relu.
2020-10-31 11:48:02 +09:00
Tomasz Sobczyk and nodchip
7bedf6c5ab
Specify the whole evalsave message because otherwise the first evalsave/0 triggers it.
2020-10-31 08:36:58 +09:00
Tomasz Sobczyk and nodchip
8c81bbd3db
Fix the counter in for_each_index_with_workers going out of scope before workers finish.
2020-10-31 08:36:58 +09:00
Tomasz Sobczyk and nodchip
a56d8124d8
Replace non-blas parts of trainers with our own blas-like routines.
2020-10-31 08:36:58 +09:00
Tomasz Sobczyk and nodchip
c56a4a36eb
Add our own blas-like routines that use stockfish's thread pool for parallelization.
2020-10-29 23:57:51 +09:00
Tomasz Sobczyk and nodchip
ee0917a345
Pass ThreadPool to update_parameters, propagate, and backpropagate.
2020-10-29 09:21:19 +09:00
Tomasz Sobczyk and nodchip
f1e96cab55
Align trainer arrays to cache line.
2020-10-29 09:12:50 +09:00
Tomasz Sobczyk and nodchip
8fac468259
Add a cache line aligned allocator.
2020-10-29 09:12:50 +09:00
Tomasz Sobczyk and nodchip
ec9e49e875
Add a HalfKA architecture (a product of K - king, and A - any piece) along with all required infrastructure. HalfKA doesn't discriminate kings compared to HalfKP. Keep old architecture as the default one.
2020-10-29 09:10:01 +09:00
Tomasz Sobczyk and nodchip
317fda2516
Cleanup eval saving and lr scheduling.
2020-10-28 23:08:05 +09:00
Tomasz Sobczyk and nodchip
680654b254
Add dots to output every epoch for progress visualization.
2020-10-28 09:36:43 +09:00
Tomasz Sobczyk and nodchip
f81fa3d712
Replace global_learning_rate with learning_rate local to the learner and passed to update_parameters as a parameter.
2020-10-28 09:36:07 +09:00
Tomasz Sobczyk and nodchip
cde6ec2bf2
Make all grad related functions in learn static. Pass calc_grad as a parameter.
2020-10-27 14:47:50 +09:00
Tomasz Sobczyk and nodchip
e4868cb59e
Move setting learn search limits to learner.
2020-10-27 14:47:07 +09:00
Tomasz Sobczyk and nodchip
c229929d26
Remove the position parameter from learn.
2020-10-27 00:35:43 +09:00
Tomasz Sobczyk and nodchip
a8066cd4a9
Rename elmo lambdas
2020-10-27 00:33:58 +09:00
Tomasz Sobczyk and nodchip
f7de49eb66
Create a collective parameter struct for learner.
2020-10-27 00:33:58 +09:00
Tomasz Sobczyk and nodchip
ba390a7f9a
Print the used factorizer when intializing training.
2020-10-27 00:32:39 +09:00
Tomasz Sobczyk and nodchip
e01397c674
Remove multi_think
2020-10-26 19:40:40 +09:00
Tomasz Sobczyk and nodchip
e515f1f61f
Move SfenWriter to a separate file
2020-10-26 19:39:58 +09:00
Tomasz Sobczyk and nodchip
65e443954a
Update expected gensfen finished responses.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
03abfae41f
Reorder members, renaming.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
6d4d20c4be
Cleaner printing and some renaming.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
d77b3d176e
Always flush sfen writer at the end of gensfen and when it is destroyed.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
21fac7c53c
A collective struct for gensfen parameters.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
cb61dc9c9b
Make sfen writer a part of gensfen.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
3f289546da
Make some gensfen members private.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
821b655bc6
Move gensfen progress reporting from sfen writer to gensfen
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
af238fe132
Rewrite gensfen to use stockfish's thread pool.
2020-10-26 09:37:59 +09:00
Tomasz Sobczyk and nodchip
0e528995c2
Print avg bias/weight for affine trasform and feature transformer during training.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
fe766f4f42
Additional output from layers during training.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
2c477d76ec
Cleaner and more outputs during training initialization.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
b882423005
Bring back info for finished evalsave. Update tests with the new message.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
4b72658409
Synchronize printed info regions in the learner and sfen reader.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
d824bd8ec5
Add an overload for io manip in the logger.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
54dd6a2407
Add logger with synchronized regions.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
cf3edfed82
Improve info messages.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
c49ae541c4
Add layer info for check_health. Print subsequent infos from the same scope with "-->" instead of "INFO:" for clarity.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
8ddef320e6
Print an additional new line before calc_loss progress instead of after check_health in the feature transformer layer.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
d70408f204
Add docs entry for the verbose flag.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
a351c1d65e
Add verbose flag to learn. Only print update parameters info when vebose=true
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
ec436d3dfd
Print some weight update stats
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
be3937c37b
Print layers and their indices during training initialization.
2020-10-25 22:18:28 +09:00
Tomasz Sobczyk and nodchip
3bf397a569
Update instrumented_learn for the current codebase.
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
47a82bfc91
Document new options.
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
371acaa0b5
Allow changing sfen reader buffer sizes for the learn command.
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
d31169bab5
Update CI to use epochs instead of loops.
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
8fb208598b
pass shuffle flag in the constructor
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
31f94a18b3
Update readme and docs after change from loop to epochs.
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
fc3788f630
Use cyclic sfen reader for learning, change loop option to epochs.
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
ad3d1b42e4
Make sfen reader only stop when it's destroyed. Now it is fully RAII.
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
c58aa9696a
Start sfen reader worker thread in the constructor.
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
0636e1256d
Add cyclic mode to the sfen reader. Make sfen reader take all files at construction
2020-10-25 19:22:56 +09:00
Tomasz Sobczyk and nodchip
e4a38c18dd
Don't test syzygi
2020-10-24 08:52:42 +09:00
