Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calculate noise to adjust default initial value #3

Open
fsmosca opened this issue Sep 29, 2020 · 0 comments
Open

Calculate noise to adjust default initial value #3

fsmosca opened this issue Sep 29, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@fsmosca
Copy link
Owner

fsmosca commented Sep 29, 2020

Flag to implement: --calculate-noise

An example situation, when user uses the flag

--calculate-noise --games-per-trial 250 --base-time-sec 5 --inc-time-sec 0.10 ...

the tuner will run 4 matches of 250 games each match with the same engine using the same default parameter values. After the 4 matches, calculate the average result.

Since we are conducting matches with the same engine and param, the expected result is 0.5. If the average result is above 0.5 say 0.52 then the noise is 0.52 - 0.5 or 0.02. The default initial value is 0.5, this would be adjusted to 0.52 to take into account the estimated noise from the system that the tuner is run and from the tuning conditions that were defined such as time control and number of games.

In this case during trials, if result of a match is 0.51 the tuner will not accept that as the best value because 0.51 is lower than 0.52. However if match result is 0.53, then the best value is no longer 0.52 but 0.53 and the default param is no longer the best. The best param will be updated to the param that was suggested by the optimizer that resulted in a match result of 0.53.

@fsmosca fsmosca added the enhancement New feature or request label Sep 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant