Louis Zulli reported that Stockfish suffers from very occasional hangs with his 20 cores machine.
Careful SMP debugging revealed that this was caused by "a ghost split point slave", where thread
was marked as a split point slave, but wasn't actually working on it.
The only logical explanation for this was double booking, where due to SMP race, the same thread
is booked for two different split points simultaneously.
Due to very intermittent nature of the problem, we can't say exactly how this happens.
The current handling of Thread specific variables is risky though. Volatile variables are in some
cases changed without spinlock being hold. In this case standard doesn't give us any kind of
guarantees about how the updated values are propagated to other threads.
We resolve the situation by enforcing very strict locking rules:
- Values for key thread variables (splitPointsSize, activeSplitPoint, searching)
can only be changed when the thread specific spinlock is held.
- Structural changes for splitPoints[] are only allowed when the thread specific spinlock is held.
- Thread booking decisions (per split point) can only be done when the thread specific spinlock is held.
With these changes hangs didn't occur anymore during 2 days torture testing on Zulli's machine.
We probably have a slight performance penalty in SMP mode due to more locking.
STC (7 threads):
ELO: -1.00 +-2.2 (95%) LOS: 18.4%
Total: 30000 W: 4538 L: 4624 D: 20838
However stability is worth more than 1-2 ELO points in this case.
No functional change
Resolves#422
Instead of using hard coded Min and Max values for history,
always adjust the old value slightly downwards before adding a new value.
The adjustment acts like gravity that prevents the value escaping too
far from zero.
Bench: 8020484
Resolves#407
During the search, do not block on condition variable, but instead use std::this_thread::yield().
Clear gain with 16 threads. Again results vary highly depending on hardware, but on average it's a clear gain.
ELO: 12.17 +-4.3 (95%) LOS: 100.0%
Total: 7998 W: 1407 L: 1127 D: 5464
There is no functional change in single thread mode
Resolves#294
Idea and original implementation by Stephane Nicolet
7 threads 15+0.05
ELO: 3.54 +-2.9 (95%) LOS: 99.2%
Total: 17971 W: 2976 L: 2793 D: 12202
There is no functional change in single thread mode
Balance threads between split points.
There are huge differences between different machines and autopurging makes it very difficult to measure the improvement in fishtest, but the following was recorded for 16 threads at 15+0.05:
For Bravone (1000 games): 0 ELO
For Glinscott (1000 games): +20 ELO
For bKingUs (1000 games): +50 ELO
For fastGM (1500 games): +50 ELO
The change was regression for no one, and a big improvement for some, so it should be fine to commit it.
Also for 8 threads at 15+0.05 we measured a statistically significant improvement:
ELO: 6.19 +-3.9 (95%) LOS: 99.9%
Total: 10325 W: 1824 L: 1640 D: 6861
Finally it was verified that there was no (significant) regression for
4 threads:
ELO: 0.09 +-2.8 (95%) LOS: 52.4%
Total: 19908 W: 3422 L: 3417 D: 13069
2 threads:
ELO: 0.38 +-3.0 (95%) LOS: 60.0%
Total: 19044 W: 3480 L: 3459 D: 12105
1 thread:
ELO: -1.27 +-2.1 (95%) LOS: 12.3%
Total: 40000 W: 7829 L: 7975 D: 24196
Resolves#258
- Fix a skill level problem: Don't allow move pruning at root node
- Revert "Fix profile build for gcc on Mac OSX". Results for a faster binary in x86-64.
- Fix a MSVC warning
Bench: 8918745
Seems to be a performance regression for standard build.
For SF6 people compiling on Mac OSX using profile-build option
just need to make necessary adjustments manually...
No functional change
Resolves#223
- Fix a compilation issue related to BMI2 PEXT instruction
- Retrieve a ponder move from TT if PV is only one move long
Bench: 8080602
No functional change
In some UNIX systems "rm" prompts user for confirmation.
However "rm -f" is always a guaranteed forced deletion.
Also move gcc profiling hack under the correct target
No Functional change
Resolves#168
- Currently broken
- Never been really useful
- Does not work well with new splitting model
Verified for no regression at STC with 3 threads:
LLR: 2.96 (-2.94,2.94) [-6.00,0.00]
Total: 81905 W: 12122 L: 12381 D: 57402
No functional change
Instead of waiting to be allocated, actively search
for another split point to join when finishes its
search. Also modify split conditions.
This patch has been tested with 7 threads SMP and
passed both STC:
LLR: 2.97 (-2.94,2.94) [-1.50,4.50]
Total: 2885 W: 519 L: 410 D: 1956
And a reduced-LTC at 25+0.05
LLR: 2.95 (-2.94,2.94) [0.00,6.00]
Total: 4401 W: 684 L: 566 D: 3151
Was then retested against regression in 3 thread case
at standard LTC of 60+0.05:
LLR: 2.96 (-2.94,2.94) [-4.00,0.00]
Total: 40809 W: 5446 L: 5406 D: 29957
bench: 8802105
When time remaining is less than Emergency Move Time,
we won't even complete one iteration and engine reports
a stale +M0 score.
To reproduce run "go wtime 10"
info depth 1 seldepth 1 score mate 0 upperbound nodes 2 nps 500 time 4 multipv 1 pv a2a3
info nodes 2 time 4
bestmove a2a3 ponder (none)
This patch fixes the issue.
Tested by Binky at very short TC: 0.05+0.05
ELO: 5.96 +-12.9 (95%) LOS: 81.7%
Total: 1458 W: 394 L: 369 D: 695
And at a bit longer TC:
ELO: 1.56 +-3.7 (95%) LOS: 79.8%
Total: 16511 W: 3983 L: 3909 D: 8619
bench: 7804908
First tested with 50K games at very short TC of 5+0.05
ELO: 3.11 +-2.0 (95%) LOS: 99.9%
Total: 49665 W: 10941 L: 10497 D: 28227
Then retested with usual SPRT at short TC
LLR: 2.96 (-2.94,2.94) [-1.50,4.50]
Total: 16875 W: 3198 L: 3049 D: 10628
And at long TC
LLR: 2.95 (-2.94,2.94) [0.00,6.00]
Total: 5890 W: 985 L: 857 D: 4048
bench: 7800379
An old idea retested at SPRT(0, 3) with 60+0.05 TC:
LLR: 2.95 (-2.94,2.94) [0.00,3.00]
Total: 98872 W: 15549 L: 15123 D: 68200
This is a very small elo increase patch so it really
stresses the limits of fishtest.
bench: 8596156
This seems a die hard idea :-)
Passed both short TC
LLR: 2.97 (-2.94,2.94) [-1.50,4.50]
Total: 17485 W: 3307 L: 3156 D: 11022
And long TC
LLR: 2.97 (-2.94,2.94) [0.00,6.00]
Total: 38181 W: 6002 L: 5729 D: 26450
bench: 8659830
In case we find a very good move after a
troubled start, we don't return immediately
anymore.
Tested directly at long TC where it passed:
LLR: 2.95 (-2.94,2.94) [0.00,6.00]
Total: 13910 W: 2397 L: 2228 D: 9285
bench: 7995098
The ideal setting for super-blitz might be something like:
"Emergency Base Time" = 50
"Emergency Move Time" = 5
This would give a total emergency time buffer of:
50 + 40 * 5 = 250 ms
This setup replaces the previous half cooked hack
"Don't blunder under extreme time pressure".
Test results are very good at super blitz, but keep good even
at 60 secs.
At 5+0.05
ELO: 24.30 +-2.4 (95%) LOS: 100.0%
Total: 37802 W: 10060 L: 7420 D: 20322
At 15+0.05
ELO: 13.41 +-2.9 (95%) LOS: 100.0%
Total: 22271 W: 4853 L: 3994 D: 13424
At 60+0.05
ELO: 5.30 +-3.2 (95%) LOS: 99.9%
Total: 16000 W: 2897 L: 2653 D: 10450
No functional change.
Set threads number always to 1 at startup and let the
user explicitly to chose the number of threads.
Also preserve the useful behavior of automatically set
"Min Split Depth" according to the requested threads,
indeed this parameter is too technical for a casual user,
so, when left to zero, we set it on a sensible value.
No functional change
Very good at long 60"+0.05 TC
LLR: 2.95 (-2.94,2.94)
Total: 5954 W: 1151 L: 1016 D: 3787
[edit: slightly changed form original patch to avoid useless loop
across killers when killer is MOVE_NONE]
bench: 4327405
Very good result both at short TC 15+0.05
LLR: 2.95 (-2.94,2.94)
Total: 2803 W: 596 L: 483 D: 1724
And at long TC 60+0.05
LLR: 2.95 (-2.94,2.94)
Total: 2862 W: 548 L: 431 D: 1883
bench: 4329221
It seems stronger both at fast 15+0.05 TC with fixed game number test:
ELO: 2.74 +-2.7 (95%) LOS: 97.6%
Total: 24000 W: 4698 L: 4509 D: 14793
And also at long 60+0.05 TC with SPRT
LLR: 3.05 (-2.94,2.94)
Total: 38986 W: 6845 L: 6547 D: 25594
bench: 5157061
Rationale:
- Current settings seem to make engine *significantly* weaker in analysis mode.
- In practice this setting only has effect when king safety scores are high.
- Even in analysis mode its far more important to know if one side is getting mated,
rather than get evaluation correct with 1cp accuracy.
No functional change
Always before pruning the move, it's important to check that:
bestValue > VALUE_MATED_IN_MAX_PLY
See example position:
8/2p1p3/P1NpP3/3k4/1P1BN3/2P1P3/2Q5/6K1 w - - 0 1
http://support.stockfishchess.org/discussions/problems/268-wrong-declaring-a-forced-mate-in-3-moves
This problem was present in 2.3.1, then it was fixed by my patch.
After 24000 games at 15+0.05
ELO: 2.40 +-4.4 (95%) LOS: 95.7%
Total: 24000 W: 4774 L: 4608 D: 14618
bench: 4465997
Another great success by Joona !
After 5876 games at 10"+0.1
Mod vs Orig: 1073 - 849 - 3954 ELO +13 (+- 5.2)
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
This patch temporarily breaks MultiPV and searchmove
features, but they will be re-implemented in future
patches.
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
We missed to set chess960 flag into the std::stringstream used to
setup the PV line.
Bug introduced with commit f803f33e63
of 30/12/2010 when we started to print PV line into a std::stringstream
instead of directly into cout, where the chess960 flag is correctly set.
No functional change.
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
Fixes the reported KNNK ending problem:
http://talkchess.com/forum/viewtopic.php?t=39347
Joona says:
Now I finally had a time to take a look at on this issue.
I've reproduced the problem starting from this position:
1B6/1B2k3/P7/1P3p2/1K6/8/4b3/4b3 w - - 6 85
I made Stockfish play as white and Fruit as black.
I repeated test ten times and once SF was not able to deliver mate.
But I observed several times that SF had reported on last something like mate in 10.
However next time it played move with score mate in 15.
Easiest way to solve the problem is attached as a patch. I tested it several times and SF always
ended up playing the optimal move. Of course the downside is that now delivering mate
takes a bit longer, but IMO it's better to lose once in a while by time in sudden death
game than not being able to deliver simple mate with long time controls.
No functional change.
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
The idea is to try a shallow search with reduced beta
on bad captures so to quickly prune them out in case
are really bad.
After 5529 games 966 - 868 - 3695 ELO +6 (+- 5.4) LOS 91%
Tested also version without upper limitation to 8 plies:
After 8780 games 1537 - 1398 - 5850 ELO +5 (+- 4.3) LOS 93%
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
Respin this old idea. Earlier we tried only
with < 1000 games and result was inconclusive.
After 5845 games
Mod vs Orig: 935 - 936 - 3974 ELO (+-3.6)
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
I got report from Werner that Shredder Gui has problems with
UCI values which maximum value is greater than 30000.
Of course it's stupid to change engine to fix a GUI's bug,
but on the other hand 30000 ms as maximum value is clearly enough,
so why not to be merciful
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
We never reach a position where rule50 > 100.
When rule50 == 100, it's either draw or mate and
there is no way search could go deeper.
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
This time we try very hard to avoid false positives.
The obvious downside is that we also miss many true
winning positions.
After 10544 games on RC
Mod- Orig: 1744 - 1646 - 7154 ELO +3 (+- 2.7) LOS 83%
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
After 2036 games Mod- Orig:
381 - 278 - 1377 ELO +17 (+- 6.2) LOS 99%
One of the biggest increases ever !
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
This patch from Joona greatly reduces move count pruning,
below is the old and new move count limits starting from
ONE_PLY with half-play increment:
Old: 4,5,5,5, 7, 7,11,11,11,19,19,19,35,35
New: 4,5,7,9,12,15,19,23,28,33,39,45,52,59
Surprisingly results are even a bit better at a quite
fast time control.
After 5260 games at 30"+0.1
Mod - Orig: 864 - 806 - 3590 ELO +3 (+- 3.8)
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
More then 100 lines of almost useless evaluations. Prefer
code semplification to a very small and dubious advantage.
After 7457 games on russian cluster:
Mod - Orig: 1285 - 1334 - 4838 ELO -2 (+- 3.2)
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
After testing on our russian cluster: +3 elo after 4200 games
So keep it becuase it allows a good semplification.
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
I created three different systems, tested them all separately and
attached one did best:
1/40: Orig - Mod: 841 - 850 (+2 elo)
1+1 : Orig - Mod: 474 - 498 (+9 elo)
1+0 : Orig - Mod: 455 - 495 (+15 elo)
Because such testing system is not statistically reliable, I made a
confirmation test:
1/40: Orig - Mod: 502 - 543 (+14 elo)
1+1: Orig - Mod: 447 - 489 (+16 elo)
1+0: Orig - Mod: 641 - 656 (+4 elo)
All tests show positive score :-)
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
After 'Randomness' is retired, this is no longer necessary.
NOTE: Possibly some extra care is needed when tuning branch is synced
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
Using multiple threads and good opening book is
much better and more reliable source of randomness than
spoiling psqt-tables
Signed-off-by: Marco Costalba <mcostalba@gmail.com>
Joona Kiiski