Files
stockfish/.github/workflows/matetrack.yml
T
Robert Nurnberg @ elitebookandJoost VandeVondele 1eff8b0389 Guarantee complete mate PVs in multiPV analysis
This PR introduces the additional `RootMove` attribute `previousPV` so that scores and PVs we send to the GUI in MultiPV analysis always match. This allows us in particular to extend our guarantee of exact mate (and TB win/loss) scores having a complete PV (leading to checkmate in the correct number of plies) to all PV lines. Recall that master fails here, since partially searched root moves may send to the GUI the previous score with the current/modified PV. See #6784.

The PR also uses the new attribute to extend the followPV logic to the analysis of sidelines, building on the idea in #6813 by @joergoster.

Passed non-reg STC:
LLR: 2.95 (-2.94,2.94) <-1.75,0.25>
Total: 166880 W: 42357 L: 42282 D: 82241
Ptnml(0-2): 394, 18685, 45177, 18820, 364
https://tests.stockfishchess.org/tests/view/6a0dea55818cacc1db0abb6a

Failed non-reg LTC:
LLR: -2.97 (-2.94,2.94) <-1.75,0.25>
Total: 890520 W: 224168 L: 225282 D: 441070
Ptnml(0-2): 390, 91902, 261789, 90790, 389
https://tests.stockfishchess.org/tests/view/6a1143ad818cacc1db0ac14c

Opening as draft for discussion on how to proceed. In SinglePV analysis, the patch is completely nonfunctional. But it is maybe a (small?) slowdown because of the increased size of `RootMove`. I am not sure if there as an elegant way to enrich the class only for MultiPV analysis (but the switch can happen at any time through the UCI interface), or to mitigate the speed penalty in some other way.

A local speedup test shows only a small slowdown on my system (but still high error bars):
```
sf_base =  1156928 +/-   1459 (95%)
sf_test =  1155885 +/-   1283 (95%)
diff    =    -1043 +/-   1777 (95%)
speedup = -0.09021% +/- 0.154% (95%)
```

The PR also adds the new MultiPV mate PV correctness check to the CI.

closes https://github.com/official-stockfish/Stockfish/pull/6886

No functional change
2026-06-09 19:36:14 +02:00

121 lines
5.7 KiB
YAML

# This workflow will run matetrack on the PR
name: Matetrack
on:
workflow_call:
jobs:
Matetrack:
name: Matetrack
runs-on: ubuntu-22.04
steps:
- name: Checkout SF repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.pull_request.head.sha }}
path: Stockfish
persist-credentials: false
- name: build SF
working-directory: Stockfish/src
run: make -j profile-build
- name: Checkout matetrack repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: vondele/matetrack
path: matetrack
ref: 9d77be284e8cba37e2f8a59e54cd2bcbf4e3ee4d
persist-credentials: false
- name: matetrack install deps
working-directory: matetrack
run: pip install -r requirements.txt
- name: cache syzygy
id: cache-syzygy
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
matetrack/3-4-5-wdl/
matetrack/3-4-5-dtz/
key: key-syzygy
- name: download syzygy 3-4-5 if needed
working-directory: matetrack
if: steps.cache-syzygy.outputs.cache-hit != 'true'
run: |
wget --no-verbose -r -nH --cut-dirs=2 --no-parent --reject="index.html*" -e robots=off https://tablebase.lichess.ovh/tables/standard/3-4-5-wdl/
wget --no-verbose -r -nH --cut-dirs=2 --no-parent --reject="index.html*" -e robots=off https://tablebase.lichess.ovh/tables/standard/3-4-5-dtz/
- name: Run matetrack th1
working-directory: matetrack
run: |
python matecheck.py --syzygyPath 3-4-5-wdl/:3-4-5-dtz/ --engine /home/runner/work/Stockfish/Stockfish/Stockfish/src/stockfish --epdFile mates2000.epd --nodes 100000 | tee matecheck1.out
! grep "issues were detected" matecheck1.out > /dev/null
- name: Run matetrack th4
working-directory: matetrack
run: |
python matecheck.py --syzygyPath 3-4-5-wdl/:3-4-5-dtz/ --engine /home/runner/work/Stockfish/Stockfish/Stockfish/src/stockfish --epdFile mates2000.epd --nodes 100000 --threads 4 | tee matecheck4.out
! grep "issues were detected" matecheck4.out > /dev/null
- name: Run matetrack th4 gameplay
working-directory: matetrack
run: |
python matecheck.py --engine /home/runner/work/Stockfish/Stockfish/Stockfish/src/stockfish --epdFile mates2000.epd --time 3 --timeinc 0.01 --threads 4 | tee matecheck4g.out
! grep "issues were detected" matecheck4g.out > /dev/null
- name: Run matetrack th4 go-mate
working-directory: matetrack
run: |
python matecheck.py --engine /home/runner/work/Stockfish/Stockfish/Stockfish/src/stockfish --epdFile matetrack.epd matedtrack.epd --bmMax 2 --mate 0 --threads 4 | tee matecheck4gm.out
! grep "issues were detected" matecheck4gm.out > /dev/null
total=$(grep "Total FENs:" matecheck4gm.out | awk '{print $3}')
bmates=$(grep "Best mates:" matecheck4gm.out | awk '{print $3}')
if [ $bmates -ne $total ]; then
echo "At least one go-mate search did not yield expected mate, see matecheck4gm.out" >&2
exit 1
fi
- name: Run matetrack th1 multiPV
working-directory: matetrack
run: |
python matecheck.py --engine /home/runner/work/Stockfish/Stockfish/Stockfish/src/stockfish --epdFile matetrack.epd matedtrack.epd --bmMax 10 --nodes 5000 --multiPV 20 --checkMultiPVs | tee matecheck1mpv.out
! grep "issues were detected" matecheck1mpv.out > /dev/null
- name: Run matetrack th4 multiPV
working-directory: matetrack
run: |
python matecheck.py --engine /home/runner/work/Stockfish/Stockfish/Stockfish/src/stockfish --epdFile matetrack.epd matedtrack.epd --bmMax 10 --nodes 5000 --threads 4 --multiPV 20 --checkMultiPVs | tee matecheck4mpv.out
! grep "issues were detected" matecheck4mpv.out > /dev/null
- name: Run matetrack th1 with --syzygy50MoveRule false
working-directory: matetrack
run: |
grep 5men cursed.epd > cursed5.epd
python matecheck.py --syzygyPath 3-4-5-wdl/:3-4-5-dtz/ --engine /home/runner/work/Stockfish/Stockfish/Stockfish/src/stockfish --epdFile cursed5.epd --nodes 100000 --syzygy50MoveRule false | tee matecheckcursed1.out
! grep "issues were detected" matecheckcursed1.out > /dev/null
- name: Run matetrack th4 with --syzygy50MoveRule false
working-directory: matetrack
run: |
grep 5men cursed.epd > cursed5.epd
python matecheck.py --syzygyPath 3-4-5-wdl/:3-4-5-dtz/ --engine /home/runner/work/Stockfish/Stockfish/Stockfish/src/stockfish --epdFile cursed5.epd --nodes 100000 --threads 4 --syzygy50MoveRule false | tee matecheckcursed4.out
! grep "issues were detected" matecheckcursed4.out > /dev/null
- name: Verify mate and TB win count for matecheckcursed[14].out
working-directory: matetrack
run: |
mates=$(grep "Found mates:" matecheckcursed1.out | awk '{print $3}')
tbwins=$(grep "Found TB wins:" matecheckcursed1.out | awk '{print $4}')
if [ $(($mates + $tbwins)) -ne 32 ]; then
echo "Sum of mates and TB wins is not 32 in matecheckcursed1.out" >&2
exit 1
fi
mates=$(grep "Found mates:" matecheckcursed4.out | awk '{print $3}')
tbwins=$(grep "Found TB wins:" matecheckcursed4.out | awk '{print $4}')
if [ $(($mates + $tbwins)) -ne 32 ]; then
echo "Sum of mates and TB wins is not 32 in matecheckcursed4.out" >&2
exit 1
fi