Search Parameter Optimization
Parameters to be optimized:
{'RazorMarginDepth1': {'default': 220, 'min': 170, 'max': 270, 'step': 2}}
{'RazorMarginDepth2': {'default': 235, 'min': 185, 'max': 285, 'step': 2}}
{'RazorMarginDepth3': {'default': 250, 'min': 200, 'max': 300, 'step': 2}}
{'GoodEvalPruningMargin': {'default': 60, 'min': 10, 'max': 110, 'step': 2}}
The default values are Deuterium's default and considered to be the best. The base engine will start from these values and the test engine will take the suggested values from optuna optimizer. In this tuning the base engine parameter values are not changed by using the flag --fix-base-param see command line. There are 100 games per trial done before sending the result to the optimizer. Time control is 10s+100ms. This is using Optuna Game Parameter Tuner v0.3.0.
python -u tuner.py --engine .\engines\deuterium\deuterium.exe --hash 128 --opening-file .\start_opening\ogpt_chess_startpos.epd --games-per-trial 100 --concurrency 6 --plot --study-name deuterium_search_study --fix-base-param --base-time-sec 10 --inc-time-sec 0.1 --pgn-output deuterium_search_study.pgn
input param: OrderedDict([('RazorMarginDepth1', {'default': 220, 'min': 170, 'max': 270, 'step': 2}), ('RazorMarginDepth2', {'default': 235, 'min': 185, 'max': 285, 'step': 2}), ('RazorMarginDepth3', {'default': 250, 'min': 200, 'max': 300, 'step': 2}), ('GoodEvalPruningMargin', {'default': 60, 'min': 10, 'max': 110, 'step': 2})])
[I 2020-09-21 10:04:41,458] A new study created in RDB with name: deuterium_search_study
Warning, best value from previous trial is not found!
study best value: 0.0
Warning, best param from previous trial is not found!.
study best param: {}
starting trial: 0 ...
suggested param for test engine: {'RazorMarginDepth1': 260, 'RazorMarginDepth2': 285, 'RazorMarginDepth3': 262, 'GoodEvalPruningMargin': 76}
param for base engine : {'RazorMarginDepth1': 220, 'RazorMarginDepth2': 235, 'RazorMarginDepth3': 250, 'GoodEvalPruningMargin': 60}
init param: {'RazorMarginDepth1': 220, 'RazorMarginDepth2': 235, 'RazorMarginDepth3': 250, 'GoodEvalPruningMargin': 60}
init value: 0.5
study best param: {}
study best value: 0.0
Actual match result: 0.495, point of view: optimizer suggested values
[I 2020-09-21 10:12:27,440] Trial 0 finished with value: 0.495 and parameters: {'RazorMarginDepth1': 260, 'RazorMarginDepth2': 285, 'RazorMarginDepth3': 262, 'GoodEvalPruningMargin': 76}. Best is trial 0 with value: 0.495.
...
starting trial: 32 ...
suggested param for test engine: {'RazorMarginDepth1': 180, 'RazorMarginDepth2': 221, 'RazorMarginDepth3': 264, 'GoodEvalPruningMargin': 32}
param for base engine : {'RazorMarginDepth1': 220, 'RazorMarginDepth2': 235, 'RazorMarginDepth3': 250, 'GoodEvalPruningMargin': 60}
init param: {'RazorMarginDepth1': 220, 'RazorMarginDepth2': 235, 'RazorMarginDepth3': 250, 'GoodEvalPruningMargin': 60}
init value: 0.5
study best param: {'GoodEvalPruningMargin': 52, 'RazorMarginDepth1': 238, 'RazorMarginDepth2': 199, 'RazorMarginDepth3': 252}
study best value: 0.57
Actual match result: 0.585, point of view: optimizer suggested values
[I 2020-09-21 14:47:27,943] Trial 32 finished with value: 0.585 and parameters: {'RazorMarginDepth1': 180, 'RazorMarginDepth2': 221, 'RazorMarginDepth3': 264, 'GoodEvalPruningMargin': 32}. Best is trial 32 with value: 0.585.The best parameter values are from trial 32 and are used by deuterium_search_study_trial_32. It won against the default after 1000 games at tc=10s+100ms.
Score of deuterium_search_study_trial_32 vs deuterium_default: 267 - 219 - 514 [0.524] 1000
... deuterium_search_study_trial_32 playing White: 154 - 99 - 247 [0.555] 500
... deuterium_search_study_trial_32 playing Black: 113 - 120 - 267 [0.493] 500
... White vs Black: 274 - 212 - 514 [0.531] 1000
Elo difference: 16.7 +/- 15.0, LOS: 98.5 %, DrawRatio: 51.4 %
Finished match
