policy network

[BertTuyt]

With the policy network developed I asked Codex to optimize the LMR in relation to the policy output, in such a way that at least the number of nodes and/or time was reduced with 50%.

Codex did several experiments, the total time spent by Codex was around 1 hour.
I also asked Codex to write a report in the end.
All code changes were also implemented by the AI in an experimental version of the Damage engine (version 18.4).

I did not have time to study the extensive report in detail, see attached.

As a proof of the pudding a 158 games DXP match was played against KR, settings no book, 6p DB, 1 core, and 80 moves for 1 minute.
Match result 158 draws. See also attached file.

This result is not yet optimized, but as I believe we have reached a ceiling I don't expect there is much ELO to gain.

Bert

[Joost Buijs]

For running tests nowadays, I use 4" + 0.1' time control. This works out to about 10 seconds per game, which means 3 games per minute for both sides combined. With 32 cores, I can blast through a full ballot2 match within 2 minutes. Even then, there are at most 4 decisive games in a 158-game match. So, I think we really reached the ceiling.

Joost

[gwiesenekker]

Hi Bert,

Interesting. FYI, the latest version of GWD use 'sort moves randomly' as the 'policy network' combined with very aggressive LMR. The problem with very aggressive LMR was that GWD often got 'stuck in the wrong variation', but the 'sort moves randomly' compensates for that. GWD now also checks if it 'gets stuck in a PV', in which case it zaps the TT entries. There are some parameters that control how aggressive the LMR is, I can have GWD draw 100% against Kingsrow, but it then also draws 100% against Flits because it sees that 'all moves lead to a draw'. The default setting loses occasionally from Kingsrow but wins from Flits most of the time.

GW

[Joost Buijs]

There are some parameters that control how aggressive the LMR is, I can have GWD draw 100% against Kingsrow, but it then also draws 100% against Flits because it sees that 'all moves lead to a draw'. The default setting loses occasionally from Kingsrow but wins from Flits most of the time.

This is exactly what I already see here for a long time, when the engine is tuned in such a way that it won't lose from Kingsrow it plays worse against weaker engines. To win from weaker engines you have to play more speculative and aggressive, this fires back at you when you play strong engines like Kingsrow.

[BertTuyt]

Think that Sjende Blyn also uses that principle.
The settings for not losing (or winning) against Kingsrow are different from the settings to win (and not draw) against the weaker engines.
At least this is what I know.
I have no further details.
A pity that apparently there is no knob to turn which generates a +10/20 ELO against Kingsrow and a win towards others.....

Or to repeat some historical words, "Computerdraughts, Game , Set and Match...".

Bert

[BertTuyt]

Another match with a Damage experimental version (version 18.1, without policy !).
Again against Kingsrow.
Same settings as usual (1 core, 1 min/80 moves, 6p DB, no book).
Match result (Damage perspective), 1 win (game 1), 1 lost (game 102), and 156 draws.
I will issue a separate post about the loss in game 102.
Think the first game would have resulted in a draw, if both would have the access to 7p (or more) endgame databases.

Bert

[BertTuyt]

Here some notes on game 102.
See below the position after white (Kingsrow white, damage black) 11. 30-25

Black (Damage) now played 20-24?, and maybe already a decisive error.
This was followed by 12. 33-28 14-19 (think more or less forced, alternative 13-19).

Now Kingsrow initiated a small combination, without winning material.

13. 28-22 18x27
14. 32x21 16x27
15. 35-30 24x35
16. 45-40 35x33
17. 38x16 , see diagram...

Now white threatens directly the black outpost on 27 with 26-21, and the escape towards square 32 will not solve the problem.
Therefore black played 17. .... 6-11 18. 16x7 2x11

White now starts an attack on the outpost

19. 41-37 1-7
20. 43-38 13-18
21. 37-32 11-16
22. 32x21 16x27
23. 42-37 7-11
24. 37-32 11-16
25. 32x21 16x27
26. 47-42 19-23, see diagram

So far Damage has seen the whole move sequence, and the search did not go further.
Unfortunately this position was judged as "neutral", so there was no reason not to play the move 20-24 in the beginning.

If you redo the test with more search time and multi-core, then the move 20-24 is not played.
So with limited time control, and the neural networks not being perfect these problems can still occur.

Bert

[pontel]

Here some notes on game 102.
See below the position after white (Kingsrow white, damage black) 11. 30-25


diagram_1.png

Black (Damage) now played 20-24?, and maybe already a decisive error.
This was followed by 12. 33-28 14-19 (think more or less forced, alternative 13-19).

Now Kingsrow initiated a small combination, without winning material.

13. 28-22 18x27
14. 32x21 16x27
15. 35-30 24x35
16. 45-40 35x33
17. 38x16 , see diagram...


diagram_2.png


Now white threatens directly the black outpost on 27 with 26-21, and the escape towards square 32 will not solve the problem.
Therefore black played 17. .... 6-11 18. 16x7 2x11

White now starts an attack on the outpost

19. 41-37 1-7
20. 43-38 13-18
21. 37-32 11-16
22. 32x21 16x27
23. 42-37 7-11
24. 37-32 11-16
25. 32x21 16x27
26. 47-42 19-23, see diagram


diagram_3.png


So far Damage has seen the whole move sequence, and the search did not go further.
Unfortunately this position was judged as "neutral", so there was no reason not to play the move 20-24 in the beginning.

If you redo the test with more search time and multi-core, then the move 20-24 is not played.
So with limited time control, and the neural networks not being perfect these problems can still occur.

Bert

Hi Bert,

An inaccuracy in the opening leads to a similar position, one that has already been played by programs such as Truus and DAM 2.2. It seems that, despite being at a disadvantage, White is not necessarily lost, but it's worth checking with engines.

01. 34-30 20-25
02. 30-24 19x30
03. 35x24 18-23
04. 32-28? 23x32

[FEN "W:W46,47,48,49,50,41,42,43,44,45,36,37,38,39,40,31,33,24:B32,25,16,17,11,12,13,14,15,06,07,08,09,10,01,02,03,04,05."]

]João Pontel

[BertTuyt]

Joao, if I remember well this is also a move sometimes played by Flits.
And indeed it needs accurate play.
In the move sequence from Damage, I think it is already a loss after the initial exchange by Kingsrow.
But proving that is rather difficult.
Also not sure if this position or "simple" trap is known within the draughts world.

Bert