Chess Piece Value Optimization

Description

Parameters to be optimized:

'PawnValueEn', {'default': 92, 'min': 84, 'max': 120, 'step': 2} 
'BishopValueOp', {'default': 350, 'min': 300, 'max': 360, 'step': 3}
'BishopValueEn', {'default': 350, 'min': 300, 'max': 360, 'step': 3}
'RookValueEn', {'default': 525, 'min': 490, 'max': 550, 'step': 5}
'QueenValueOp', {'default': 985, 'min': 975, 'max': 1050, 'step': 5}

The default values are Deuterium's default and considered to be the best. The base engine will start from these values and the test engine will take the suggested values from optuna optimizer.

Command line

python tuner.py --engine ./engines/deuterium/deuterium.exe --opening-file ./start_opening/ogpt_chess_startpos.epd --concurrency 6 --games-per-trial 24 --base-time-sec 10 --inc-time-sec 0.10 --trials 1000 --study-name piece_value3

Tuning

python tuner.py --engine ./engines/deuterium/deuterium.exe --opening-file ./start_opening/ogpt_chess_startpos.epd --concurrency 6 --games-per-trial 24 --base-time-sec 10 --inc-time-sec 0.10 --trials 100 --study-name piece_value3
trials: 100, games_per_trial: 24
input param: OrderedDict([('PawnValueEn', {'default': 92, 'min': 84, 'max': 120, 'step': 2}), ('BishopValueOp', {'default': 350, 'min': 300, 'max': 360, 'step': 3}), ('BishopValueEn', {'default': 350, 'min': 300, 'max': 360, 'step': 3}), ('RookValueEn', {'default': 525, 'min': 490, 'max': 550, 'step': 5}), ('QueenValueOp', {'default': 985, 'min': 975, 'max': 1050, 'step': 5})])

[I 2020-09-13 10:26:15,917] A new study created in RDB with name: piece_value3
Warning, best param from previous trial is not found!, use an init param based from input param.
init best param: {'PawnValueEn': 92, 'BishopValueOp': 350, 'BishopValueEn': 350, 'RookValueEn': 525, 'QueenValueOp': 985}
Warning, init value is not found!, use init best value 0.5.
init best value: 0.5

starting trial: 0 ...
suggested param: {'PawnValueEn': 98, 'BishopValueOp': 300, 'BishopValueEn': 357, 'RookValueEn': 500, 'QueenValueOp': 1035}
init param: {'PawnValueEn': 92, 'BishopValueOp': 350, 'BishopValueEn': 350, 'RookValueEn': 525, 'QueenValueOp': 985}
init value: 0.5
[I 2020-09-13 10:28:09,176] Trial 0 finished with value: 0.5 and parameters: {'PawnValueEn': 98, 'BishopValueOp': 300, 'BishopValueEn': 357, 'RookValueEn': 500, 'QueenValueOp': 1035}. Best is trial 0 with value: 0.5.
starting trial: 1 ...

...

starting trial: 211 ...
suggested param: {'PawnValueEn': 106, 'BishopValueOp': 351, 'BishopValueEn': 345, 'RookValueEn': 550, 'QueenValueOp': 1030}
best param: {'BishopValueEn': 345, 'BishopValueOp': 351, 'PawnValueEn': 104, 'QueenValueOp': 1030, 'RookValueEn': 550}
best value: 0.5943281250000012
[I 2020-09-14 08:52:32,624] Trial 211 finished with value: 0.5962812500000012 and parameters: {'PawnValueEn': 106, 'BishopValueOp': 351, 'BishopValueEn': 345, 'RookValueEn': 550, 'QueenValueOp': 1030}. Best is trial 211 with value: 0.5962812500000012.
starting trial: 212 ...
suggested param: {'PawnValueEn': 106, 'BishopValueOp': 351, 'BishopValueEn': 345, 'RookValueEn': 550, 'QueenValueOp': 1030}
best param: {'BishopValueEn': 345, 'BishopValueOp': 351, 'PawnValueEn': 106, 'QueenValueOp': 1030, 'RookValueEn': 550}
best value: 0.5962812500000012
[I 2020-09-14 08:54:28,734] Trial 212 finished with value: 0.396 and parameters: {'PawnValueEn': 106, 'BishopValueOp': 351, 'BishopValueEn': 345, 'RookValueEn': 550, 'QueenValueOp': 1030}. Best is trial 211 with value: 0.5962812500000012.

best param: {'BishopValueEn': 345, 'BishopValueOp': 351, 'PawnValueEn': 106, 'QueenValueOp': 1030, 'RookValueEn': 550}
best value: 0.5962812500000012
best trial number: 211

Game test

The best parameters after 212 trials are used by deuterium_pv3_212_trials. It won against the default after 1000 games at tc=15s+100ms.

Score of deuterium_pv3_212_trials vs deuterium_default: 251 - 199 - 550  [0.526] 1000
...      deuterium_pv3_212_trials playing White: 138 - 94 - 268  [0.544] 500
...      deuterium_pv3_212_trials playing Black: 113 - 105 - 282  [0.508] 500
...      White vs Black: 243 - 207 - 550  [0.518] 1000
Elo difference: 18.1 +/- 14.4, LOS: 99.3 %, DrawRatio: 55.0 %