From 541ee7e0383b491b86d1a23dcff669f2efad616d Mon Sep 17 00:00:00 2001 From: Sarah Segel <30651149+sarah-segel@users.noreply.github.com> Date: Tue, 1 Aug 2023 15:02:51 +0200 Subject: [PATCH] Version 2.0.2 (#1060) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * maint * fix wrongly imported func name in tests * Add reference for pSMAC #609 * Rename example file * Added roadmap to readme * Prettification * Add result merger * Reformat with black * Delete unnecessary import * Fix typing * Inject first trajectory entry from file from first rundir * Add test for ResultMerger * Fix docstring * Get runhistory and trajectory from pSMAC run Via the result merger. Searchers rundirs and merges found runhistories and calculates a new trajectory. * Get trajectory * Add regex to requirements * Fixed MO cache problem * Fix MO tests * Fixing MO in AbstractRacer * Accepting ints * Hacking in the sum cost for MO * Fix mypi * Bugfix * Make mypi happy * Fix tests * Make format * Remove num_obj in MO optimizer * A lot of restructuring * Solve importings * Solve importings * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Updated changelog * Still fixing :) * fix docstrings * Change directory of tests * Fix ambiguous variable * Del newlines * fix mypy * mypy and doc fix * Removed int type checking * fix tests * Always return List of incs * fix path issue in example * maint boing facade * Better docstring * Return values if no bounds passed * Removed import * Return only one incumbent * Rename n_optimizers -> n_workers * Update example * Update docstring * Remove legacy function * Update to new signature * Update changelog.md * Format black * Update test * Please pre-commit * Merge development * Fix tests * Fix typos * Format black * Updated text * Increased version * Improved readme * SMAC logo 50% * SMAC logo 50% * Removed dev-2.0 * Added doc workflow for latest tag * Fixing docs latest tag * Fixing docs latest tag * First steps new intensifier * Make it work again * MOVE random weight initialization to smbo ask method * ADD enable parego in unittests * Test issues * FIX formatting * Link to old documentation * MOVE random weight initialization to smbo ask method * ADD enable parego in unittests * FIX formatting * Fix unit tests * Add annotation * corrected log prob for tophat prior * Removed configurations from callback * Fixed tests * Fix mypy * Save docs as latest version too * Enabled all tests again * Make version an env * Added last version to docs * Global variable fix * Added v before version number * New Intensifier Implementation (#888) - Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now. - Expected behaviour for ask-and-tell interface ensured (also for Successive Halving). - Continuing a run is now fully supported. - Added more examples. - Updated documentation based on new implementation. * First benchmark steps * Added a warning * Set xi to 0.01 * Incorporated walltime in trajectory * Working version of benchmark * Changed requirements * Changed xi back to 0.0 again * No duplicate budgets * Fixed bug which updated tracker wrongly * Bugfix: Delete correct bracket+stage * Fixed tests * Added costs to the trajectory * Changed initial design max ratio to 0.25 * Make format * Added more details to intensifier * Finished benchmark * Added benchmark source files * Fixed tests because of initial design changes * Improved runhistory API * Fixed typo (#894) * Merge doc fixes * Doc fixes (#893) Co-authored-by: timruhkopf Co-authored-by: Carolin Benjamins Co-authored-by: dengdifan Co-authored-by: Alexander Tornede Co-authored-by: Sarah Krebs * Make format * Satisfied mypy * Enabled more tests * Increased version * Mypy * Mypy * Mypy * Fixed critical bug when using highest budget for inc selection * Added SH+HB comparison * Stuff * Updated report * Fixed docstring * Fixed image path * Removed vscode folder * Bugfixes * Doc fixes * Updated changelog * Make isort * Fix typos * Removed new line * Added alpha2 to docs selection * update description * Fix 531 correctly create integers in initial design This PR fixes a bug in which integer hyperparameters would not be created correctly in the initial design. This went unnoticed, except in the case where it was part of a condition and the function deactivate_inactive_hyperparameters was called. This PR fixes this bug by mapping the random float that was sampled by, for example and LHD or Sobol sequence, to the float representation of an integer by calling _transform and _inverse_transform of the integer hyperparameter. * Add contact paragraph to README. * Increase days until stale issues are closed by stalebot (#904) * Edit changelog * Add issue number * Closes #907 * Clarify origin of configurations (#908) * Random forest speed up (#903) Co-authored-by: René Sass * Add: workflow to update pre-commit versions (#874) Co-authored-by: René Sass * Add Git-Flow Workflow Specification * [DOC] acquisition function maximizers * Add stale reminder email action * Increase version number of ConfigSpace * Add guideline for updating copyright notice years to CONTRIBUTING.md * Update requirement for pyrfr (#920) * Update pre-commit: isort,flake8 version numbers * Format black * Fix mypy * Ac bench (#929) * Add Branin AC Task * Add ac optimization type * Nicer plotting * Add ac mode * Update ac task * Make new smac env name adjustable * Increase python version for smac 1.4 * Update benchmark * Update plotting * Goodby comments * Fix path * Fix 1.4 wrapper * Update benchmark results --------- Co-authored-by: Carolin Benjamins * [DOC] added documentation on the incumbent update * Feature/add version to make publish (#931) * Add version, correct package name and use bash in makefile * Increase version * Increase version in makefile, add places with versions to make publish * Fix typo * Transform integer feature names to strings (#930) * Transform int features to str * Fix pre-commit * Update version to 2.0.0 * Update version to 2.0.0 * Update CHANGELOG.md * Add information on deepcave to the readme * Correct doc version * fix(runhistory): set id of the sampled config (#951) * fix: Set config_id of config in runhistory * ci: Set dependabot to target development branch (#963) * feat: Option preventing SMAC setting up logging (#947) * Version 2.0.0 (#898) * Update runhistory2epm.py * pre_commit * Updated changelog * Fixed comment * Updated changelog * Some small improvements * Removed trailing space * Updated changelog * Restricted pynisher * Updated changelog * Link JMLR paper * Update bibtex entry Before: arxiv paper, now journal article. * Fixed citation in docs * Fixed callback typos * BOinG for mixed hyperparameter search space * fix format * maint * maint documentation * maint pre-commit * fix workflow * maint * Add dependabot for workflow files * Bump actions/setup-python from 2 to 4 (#849) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v2...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 2 to 3 (#850) Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * maint * Updated stale bot * Updated stale options again * Fixed typing * Increased version * rename check_point_in_ss to check_subspace_point * Make format * Uncomment test procedure * docstrings for boing facade * avoid typing namespace everywhere in epm_configuration_chooser * fix docstring in rh2epm_boing * Limit on dask * Trying to fix tests * Replaced typing in all files * docstring for boing subspace * Changed feature requests to feature * allow acq_optimizer_local_kwargs to be passed to boing subspace without acq_optimizer_local * fix pre-commit * Added any * Update facade * Fix typing * Match arguments from pSMAC with SMAC4AC signature * Add description of pSMAC to docstring * Add option to pass facade class to pSMAC * Add example for pSMAC * Removed docstring in readme's example * Corrected text * move construct_gp_kernels to smac.epm_gpytorch.boing_kernels * maint * fix wrongly imported func name in tests * Add reference for pSMAC #609 * Rename example file * Added roadmap to readme * Prettification * Add result merger * Reformat with black * Delete unnecessary import * Fix typing * Inject first trajectory entry from file from first rundir * Add test for ResultMerger * Fix docstring * Get runhistory and trajectory from pSMAC run Via the result merger. Searchers rundirs and merges found runhistories and calculates a new trajectory. * Get trajectory * Add regex to requirements * Fixed MO cache problem * Fix MO tests * Fixing MO in AbstractRacer * Accepting ints * Hacking in the sum cost for MO * Fix mypi * Bugfix * Make mypi happy * Fix tests * Make format * Remove num_obj in MO optimizer * A lot of restructuring * Solve importings * Solve importings * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Updated changelog * Still fixing :) * fix docstrings * Change directory of tests * Fix ambiguous variable * Del newlines * fix mypy * mypy and doc fix * Removed int type checking * fix tests * Always return List of incs * fix path issue in example * maint boing facade * Better docstring * Return values if no bounds passed * Removed import * Return only one incumbent * Rename n_optimizers -> n_workers * Update example * Update docstring * Remove legacy function * Update to new signature * Update changelog.md * Format black * Update test * Please pre-commit * Merge development * Fix tests * Fix typos * Format black * Updated text * Increased version * Improved readme * SMAC logo 50% * SMAC logo 50% * Removed dev-2.0 * Added doc workflow for latest tag * Fixing docs latest tag * Fixing docs latest tag * First steps new intensifier * Make it work again * MOVE random weight initialization to smbo ask method * ADD enable parego in unittests * Test issues * FIX formatting * Link to old documentation * MOVE random weight initialization to smbo ask method * ADD enable parego in unittests * FIX formatting * Fix unit tests * Add annotation * corrected log prob for tophat prior * Removed configurations from callback * Fixed tests * Fix mypy * Save docs as latest version too * Enabled all tests again * Make version an env * Added last version to docs * Global variable fix * Added v before version number * New Intensifier Implementation (#888) - Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now. - Expected behaviour for ask-and-tell interface ensured (also for Successive Halving). - Continuing a run is now fully supported. - Added more examples. - Updated documentation based on new implementation. * First benchmark steps * Added a warning * Set xi to 0.01 * Incorporated walltime in trajectory * Working version of benchmark * Changed requirements * Changed xi back to 0.0 again * No duplicate budgets * Fixed bug which updated tracker wrongly * Bugfix: Delete correct bracket+stage * Fixed tests * Added costs to the trajectory * Changed initial design max ratio to 0.25 * Make format * Added more details to intensifier * Finished benchmark * Added benchmark source files * Fixed tests because of initial design changes * Improved runhistory API * Fixed typo (#894) * Merge doc fixes * Doc fixes (#893) Co-authored-by: timruhkopf Co-authored-by: Carolin Benjamins Co-authored-by: dengdifan Co-authored-by: Alexander Tornede Co-authored-by: Sarah Krebs * Make format * Satisfied mypy * Enabled more tests * Increased version * Mypy * Mypy * Mypy * Fixed critical bug when using highest budget for inc selection * Added SH+HB comparison * Stuff * Updated report * Fixed docstring * Fixed image path * Removed vscode folder * Bugfixes * Doc fixes * Updated changelog * Make isort * Fix typos * Removed new line * Added alpha2 to docs selection * update description * Fix 531 correctly create integers in initial design This PR fixes a bug in which integer hyperparameters would not be created correctly in the initial design. This went unnoticed, except in the case where it was part of a condition and the function deactivate_inactive_hyperparameters was called. This PR fixes this bug by mapping the random float that was sampled by, for example and LHD or Sobol sequence, to the float representation of an integer by calling _transform and _inverse_transform of the integer hyperparameter. * Add contact paragraph to README. * Increase days until stale issues are closed by stalebot (#904) * Edit changelog * Add issue number * Closes #907 * Clarify origin of configurations (#908) * Random forest speed up (#903) Co-authored-by: René Sass * Add: workflow to update pre-commit versions (#874) Co-authored-by: René Sass * Add Git-Flow Workflow Specification * [DOC] acquisition function maximizers * Add stale reminder email action * Increase version number of ConfigSpace * Add guideline for updating copyright notice years to CONTRIBUTING.md * Update requirement for pyrfr (#920) * Update pre-commit: isort,flake8 version numbers * Format black * Fix mypy * Ac bench (#929) * Add Branin AC Task * Add ac optimization type * Nicer plotting * Add ac mode * Update ac task * Make new smac env name adjustable * Increase python version for smac 1.4 * Update benchmark * Update plotting * Goodby comments * Fix path * Fix 1.4 wrapper * Update benchmark results --------- Co-authored-by: Carolin Benjamins * [DOC] added documentation on the incumbent update * Feature/add version to make publish (#931) * Add version, correct package name and use bash in makefile * Increase version * Increase version in makefile, add places with versions to make publish * Fix typo * Transform integer feature names to strings (#930) * Transform int features to str * Fix pre-commit * Update version to 2.0.0 * Update version to 2.0.0 * Update CHANGELOG.md * Add information on deepcave to the readme * Correct doc version --------- Co-authored-by: dengdifan <33290713+dengdifan@users.noreply.github.com> Co-authored-by: dengdifan Co-authored-by: Matthias Feurer Co-authored-by: Carolin Benjamins Co-authored-by: eddiebergman Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Carolin Benjamins Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Katharina Eggensperger Co-authored-by: Eric Kalosa-Kenyon Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: timruhkopf Co-authored-by: Alexander Tornede Co-authored-by: Sarah Krebs Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com> Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: Jeroen Rook Co-authored-by: Tim Ruhkopf * feat: Option preventing SMAC setting up logging * fix: Run isort * Bump peter-evans/create-pull-request from 3 to 5 (#958) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3 to 5. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](https://github.com/peter-evans/create-pull-request/compare/v3...v5) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 2 to 4 (#945) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v2...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat: Option preventing SMAC setting up logging * fix: Run isort * fix: Add to changelog * feat: Option preventing SMAC setting up logging * fix: Run isort --------- Signed-off-by: dependabot[bot] Co-authored-by: René Sass Co-authored-by: dengdifan <33290713+dengdifan@users.noreply.github.com> Co-authored-by: dengdifan Co-authored-by: Matthias Feurer Co-authored-by: Carolin Benjamins Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Carolin Benjamins Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Katharina Eggensperger Co-authored-by: Eric Kalosa-Kenyon Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: timruhkopf Co-authored-by: Alexander Tornede Co-authored-by: Sarah Krebs Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com> Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: Jeroen Rook Co-authored-by: Tim Ruhkopf * Citation update (#961) * update CITATION.cff * update order in CITATION.cff * fix citation date * fix citation * maint * Adjust hyperband configuration distribution across brackets * Compute the actual differences between the isb keys. (#957) * Compute the actual differences between the isb keys. * Added change to CHANGELOG.md * Adjust schedule for stale reminder * fix(logging): Prevent automatic logging setup at init (#970) * fix(logging): Prevent automatic logging setup at init * fix(logging): Rely on `setup_logging` to handle arg * Fix validate method of smbo, update docstring. * Allow callbacks to be added to a specific index, make callback registering public * Update CHANGELOG.md * Fix broken references (#988) * Adapt developer install instructions to include pre-commit installation (#994) * [Feature] Initial Design now supports executing the default config provided by the user by means of the configspace he generated. (and is no longer ignored) * Pipe dask_client through facade for exec on hpc (#983) * Pipe dask_client through facade for exec on hpc * Handle pynisher's memory arg Before: Could only pass int via SMAC with the default unit B, but now tuple/list is possible * Update CHANGELOG.md * Make format * No printi --------- Co-authored-by: Carolin Benjamins * [Bug-fix] Pr was failing due to mutable additional configs default * [doc-fix] the count of initial design configs is computed slightly differently * Feature/metadata callback (#999) * Add example for using a callback to log run metadata to a file * Add issue number in changelog * Changelog formatting * Simplify metadata example * Simplify metadata example * Put callback in smac directory to allow importing it in custom repos * Move callback files to callback directory * Edit changelog * Correct metadata callback argument types * Additional metadata arguments in example * Set metadata default values in callback * Edit comment to fix PR * Documentation/dask client example (#1001) * Update requirements (dask_jobqueue) * Fix broken link * Ignore parallel example * Add parallelization example * Remove imports, add comment abt dask client * Add E's information * Update docs * Add comment about n_workers --------- Co-authored-by: Carolin Benjamins * Documentation/dask client example (#1001) * Update requirements (dask_jobqueue) * Fix broken link * Ignore parallel example * Add parallelization example * Remove imports, add comment abt dask client * Add E's information * Update docs * Add comment about n_workers --------- Co-authored-by: Carolin Benjamins * Feature/dask client (#1002) * Add info message that scenario.n_workers is ignored When passing a dask client, directly set n_workers in client. * Change info to warning --------- Co-authored-by: Carolin Benjamins * Update smac version (#1003) * Update smac version * Add hint on github release in make publish * Update copyright year * check if config in rh when storing state (#997) * check if config in rh when storing state * maint * remove unnecessary changes * Adapt developer install instructions to include pre-commit installation (#994) * Pipe dask_client through facade for exec on hpc (#983) * Pipe dask_client through facade for exec on hpc * Handle pynisher's memory arg Before: Could only pass int via SMAC with the default unit B, but now tuple/list is possible * Update CHANGELOG.md * Make format * No printi --------- Co-authored-by: Carolin Benjamins * update changelog * Increase tolerance for runhistory restoration test --------- Co-authored-by: Sarah Segel <30651149+sarah-segel@users.noreply.github.com> Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Carolin Benjamins Co-authored-by: Helena Graf * Add a workaround to be able to pass a dataset via dask.scatter (#993) * [feat] Support dask.scatter * [fix] Add shared_data arg in abstract_facade * Address the comments by Eddie * [fix] Fix errors raised by Eddie * [format] Apply black locally * [fix] Get retries back to 16 as it was a mistake * [doc] Update CHANGELOG --------- Co-authored-by: helegraf Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> * Adding Binder links for example (#976) * binder * maint * fix dependencies * maint docs * Revert "maint docs" This reverts commit 4fa1a0fae8dfa0b646e43273d50f72ddd76bf191. * add requirement --------- Co-authored-by: Difan Deng * Remove commit message from dependabot PRs (#1009) * Bugfix/fix broken link (#1012) * Remove commit message from dependabot PRs (#1010) * rename check_point_in_ss to check_subspace_point * Make format * Uncomment test procedure * docstrings for boing facade * avoid typing namespace everywhere in epm_configuration_chooser * fix docstring in rh2epm_boing * Limit on dask * Trying to fix tests * Replaced typing in all files * docstring for boing subspace * Changed feature requests to feature * allow acq_optimizer_local_kwargs to be passed to boing subspace without acq_optimizer_local * fix pre-commit * Added any * Update facade * Fix typing * Match arguments from pSMAC with SMAC4AC signature * Add description of pSMAC to docstring * Add option to pass facade class to pSMAC * Add example for pSMAC * Removed docstring in readme's example * Corrected text * move construct_gp_kernels to smac.epm_gpytorch.boing_kernels * maint * fix wrongly imported func name in tests * Add reference for pSMAC #609 * Rename example file * Added roadmap to readme * Prettification * Add result merger * Reformat with black * Delete unnecessary import * Fix typing * Inject first trajectory entry from file from first rundir * Add test for ResultMerger * Fix docstring * Get runhistory and trajectory from pSMAC run Via the result merger. Searchers rundirs and merges found runhistories and calculates a new trajectory. * Get trajectory * Add regex to requirements * Fixed MO cache problem * Fix MO tests * Fixing MO in AbstractRacer * Accepting ints * Hacking in the sum cost for MO * Fix mypi * Bugfix * Make mypi happy * Fix tests * Make format * Remove num_obj in MO optimizer * A lot of restructuring * Solve importings * Solve importings * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Updated changelog * Still fixing :) * fix docstrings * Change directory of tests * Fix ambiguous variable * Del newlines * fix mypy * mypy and doc fix * Removed int type checking * fix tests * Always return List of incs * fix path issue in example * maint boing facade * Better docstring * Return values if no bounds passed * Removed import * Return only one incumbent * Rename n_optimizers -> n_workers * Update example * Update docstring * Remove legacy function * Update to new signature * Update changelog.md * Format black * Update test * Please pre-commit * Merge development * Fix tests * Fix typos * Format black * Updated text * Increased version * Improved readme * SMAC logo 50% * SMAC logo 50% * Removed dev-2.0 * Added doc workflow for latest tag * Fixing docs latest tag * Fixing docs latest tag * First steps new intensifier * Make it work again * MOVE random weight initialization to smbo ask method * ADD enable parego in unittests * Test issues * FIX formatting * Link to old documentation * MOVE random weight initialization to smbo ask method * ADD enable parego in unittests * FIX formatting * Fix unit tests * Add annotation * corrected log prob for tophat prior * Removed configurations from callback * Fixed tests * Fix mypy * Save docs as latest version too * Enabled all tests again * Make version an env * Added last version to docs * Global variable fix * Added v before version number * New Intensifier Implementation (#888) - Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now. - Expected behaviour for ask-and-tell interface ensured (also for Successive Halving). - Continuing a run is now fully supported. - Added more examples. - Updated documentation based on new implementation. * First benchmark steps * Added a warning * Set xi to 0.01 * Incorporated walltime in trajectory * Working version of benchmark * Changed requirements * Changed xi back to 0.0 again * No duplicate budgets * Fixed bug which updated tracker wrongly * Bugfix: Delete correct bracket+stage * Fixed tests * Added costs to the trajectory * Changed initial design max ratio to 0.25 * Make format * Added more details to intensifier * Finished benchmark * Added benchmark source files * Fixed tests because of initial design changes * Improved runhistory API * Fixed typo (#894) * Merge doc fixes * Doc fixes (#893) Co-authored-by: timruhkopf Co-authored-by: Carolin Benjamins Co-authored-by: dengdifan Co-authored-by: Alexander Tornede Co-authored-by: Sarah Krebs * Make format * Satisfied mypy * Enabled more tests * Increased version * Mypy * Mypy * Mypy * Fixed critical bug when using highest budget for inc selection * Added SH+HB comparison * Stuff * Updated report * Fixed docstring * Fixed image path * Removed vscode folder * Bugfixes * Doc fixes * Updated changelog * Make isort * Fix typos * Removed new line * Added alpha2 to docs selection * update description * Fix 531 correctly create integers in initial design This PR fixes a bug in which integer hyperparameters would not be created correctly in the initial design. This went unnoticed, except in the case where it was part of a condition and the function deactivate_inactive_hyperparameters was called. This PR fixes this bug by mapping the random float that was sampled by, for example and LHD or Sobol sequence, to the float representation of an integer by calling _transform and _inverse_transform of the integer hyperparameter. * Add contact paragraph to README. * Increase days until stale issues are closed by stalebot (#904) * Edit changelog * Add issue number * Closes #907 * Clarify origin of configurations (#908) * Random forest speed up (#903) Co-authored-by: René Sass * Add: workflow to update pre-commit versions (#874) Co-authored-by: René Sass * Add Git-Flow Workflow Specification * [DOC] acquisition function maximizers * Add stale reminder email action * Increase version number of ConfigSpace * Add guideline for updating copyright notice years to CONTRIBUTING.md * Update requirement for pyrfr (#920) * Update pre-commit: isort,flake8 version numbers * Format black * Fix mypy * Ac bench (#929) * Add Branin AC Task * Add ac optimization type * Nicer plotting * Add ac mode * Update ac task * Make new smac env name adjustable * Increase python version for smac 1.4 * Update benchmark * Update plotting * Goodby comments * Fix path * Fix 1.4 wrapper * Update benchmark results --------- Co-authored-by: Carolin Benjamins * [DOC] added documentation on the incumbent update * Feature/add version to make publish (#931) * Add version, correct package name and use bash in makefile * Increase version * Increase version in makefile, add places with versions to make publish * Fix typo * Transform integer feature names to strings (#930) * Transform int features to str * Fix pre-commit * Update version to 2.0.0 * Update version to 2.0.0 * Update CHANGELOG.md * Add information on deepcave to the readme * Correct doc version * fix(runhistory): set id of the sampled config (#951) * fix: Set config_id of config in runhistory * ci: Set dependabot to target development branch (#963) * feat: Option preventing SMAC setting up logging (#947) * Version 2.0.0 (#898) * Update runhistory2epm.py * pre_commit * Updated changelog * Fixed comment * Updated changelog * Some small improvements * Removed trailing space * Updated changelog * Restricted pynisher * Updated changelog * Link JMLR paper * Update bibtex entry Before: arxiv paper, now journal article. * Fixed citation in docs * Fixed callback typos * BOinG for mixed hyperparameter search space * fix format * maint * maint documentation * maint pre-commit * fix workflow * maint * Add dependabot for workflow files * Bump actions/setup-python from 2 to 4 (#849) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v2...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 2 to 3 (#850) Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * maint * Updated stale bot * Updated stale options again * Fixed typing * Increased version * rename check_point_in_ss to check_subspace_point * Make format * Uncomment test procedure * docstrings for boing facade * avoid typing namespace everywhere in epm_configuration_chooser * fix docstring in rh2epm_boing * Limit on dask * Trying to fix tests * Replaced typing in all files * docstring for boing subspace * Changed feature requests to feature * allow acq_optimizer_local_kwargs to be passed to boing subspace without acq_optimizer_local * fix pre-commit * Added any * Update facade * Fix typing * Match arguments from pSMAC with SMAC4AC signature * Add description of pSMAC to docstring * Add option to pass facade class to pSMAC * Add example for pSMAC * Removed docstring in readme's example * Corrected text * move construct_gp_kernels to smac.epm_gpytorch.boing_kernels * maint * fix wrongly imported func name in tests * Add reference for pSMAC #609 * Rename example file * Added roadmap to readme * Prettification * Add result merger * Reformat with black * Delete unnecessary import * Fix typing * Inject first trajectory entry from file from first rundir * Add test for ResultMerger * Fix docstring * Get runhistory and trajectory from pSMAC run Via the result merger. Searchers rundirs and merges found runhistories and calculates a new trajectory. * Get trajectory * Add regex to requirements * Fixed MO cache problem * Fix MO tests * Fixing MO in AbstractRacer * Accepting ints * Hacking in the sum cost for MO * Fix mypi * Bugfix * Make mypi happy * Fix tests * Make format * Remove num_obj in MO optimizer * A lot of restructuring * Solve importings * Solve importings * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Fixing, fixing, fixing, ... * Updated changelog * Still fixing :) * fix docstrings * Change directory of tests * Fix ambiguous variable * Del newlines * fix mypy * mypy and doc fix * Removed int type checking * fix tests * Always return List of incs * fix path issue in example * maint boing facade * Better docstring * Return values if no bounds passed * Removed import * Return only one incumbent * Rename n_optimizers -> n_workers * Update example * Update docstring * Remove legacy function * Update to new signature * Update changelog.md * Format black * Update test * Please pre-commit * Merge development * Fix tests * Fix typos * Format black * Updated text * Increased version * Improved readme * SMAC logo 50% * SMAC logo 50% * Removed dev-2.0 * Added doc workflow for latest tag * Fixing docs latest tag * Fixing docs latest tag * First steps new intensifier * Make it work again * MOVE random weight initialization to smbo ask method * ADD enable parego in unittests * Test issues * FIX formatting * Link to old documentation * MOVE random weight initialization to smbo ask method * ADD enable parego in unittests * FIX formatting * Fix unit tests * Add annotation * corrected log prob for tophat prior * Removed configurations from callback * Fixed tests * Fix mypy * Save docs as latest version too * Enabled all tests again * Make version an env * Added last version to docs * Global variable fix * Added v before version number * New Intensifier Implementation (#888) - Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now. - Expected behaviour for ask-and-tell interface ensured (also for Successive Halving). - Continuing a run is now fully supported. - Added more examples. - Updated documentation based on new implementation. * First benchmark steps * Added a warning * Set xi to 0.01 * Incorporated walltime in trajectory * Working version of benchmark * Changed requirements * Changed xi back to 0.0 again * No duplicate budgets * Fixed bug which updated tracker wrongly * Bugfix: Delete correct bracket+stage * Fixed tests * Added costs to the trajectory * Changed initial design max ratio to 0.25 * Make format * Added more details to intensifier * Finished benchmark * Added benchmark source files * Fixed tests because of initial design changes * Improved runhistory API * Fixed typo (#894) * Merge doc fixes * Doc fixes (#893) Co-authored-by: timruhkopf Co-authored-by: Carolin Benjamins Co-authored-by: dengdifan Co-authored-by: Alexander Tornede Co-authored-by: Sarah Krebs * Make format * Satisfied mypy * Enabled more tests * Increased version * Mypy * Mypy * Mypy * Fixed critical bug when using highest budget for inc selection * Added SH+HB comparison * Stuff * Updated report * Fixed docstring * Fixed image path * Removed vscode folder * Bugfixes * Doc fixes * Updated changelog * Make isort * Fix typos * Removed new line * Added alpha2 to docs selection * update description * Fix 531 correctly create integers in initial design This PR fixes a bug in which integer hyperparameters would not be created correctly in the initial design. This went unnoticed, except in the case where it was part of a condition and the function deactivate_inactive_hyperparameters was called. This PR fixes this bug by mapping the random float that was sampled by, for example and LHD or Sobol sequence, to the float representation of an integer by calling _transform and _inverse_transform of the integer hyperparameter. * Add contact paragraph to README. * Increase days until stale issues are closed by stalebot (#904) * Edit changelog * Add issue number * Closes #907 * Clarify origin of configurations (#908) * Random forest speed up (#903) Co-authored-by: René Sass * Add: workflow to update pre-commit versions (#874) Co-authored-by: René Sass * Add Git-Flow Workflow Specification * [DOC] acquisition function maximizers * Add stale reminder email action * Increase version number of ConfigSpace * Add guideline for updating copyright notice years to CONTRIBUTING.md * Update requirement for pyrfr (#920) * Update pre-commit: isort,flake8 version numbers * Format black * Fix mypy * Ac bench (#929) * Add Branin AC Task * Add ac optimization type * Nicer plotting * Add ac mode * Update ac task * Make new smac env name adjustable * Increase python version for smac 1.4 * Update benchmark * Update plotting * Goodby comments * Fix path * Fix 1.4 wrapper * Update benchmark results --------- Co-authored-by: Carolin Benjamins * [DOC] added documentation on the incumbent update * Feature/add version to make publish (#931) * Add version, correct package name and use bash in makefile * Increase version * Increase version in makefile, add places with versions to make publish * Fix typo * Transform integer feature names to strings (#930) * Transform int features to str * Fix pre-commit * Update version to 2.0.0 * Update version to 2.0.0 * Update CHANGELOG.md * Add information on deepcave to the readme * Correct doc version --------- Co-authored-by: dengdifan <33290713+dengdifan@users.noreply.github.com> Co-authored-by: dengdifan Co-authored-by: Matthias Feurer Co-authored-by: Carolin Benjamins Co-authored-by: eddiebergman Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Carolin Benjamins Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Katharina Eggensperger Co-authored-by: Eric Kalosa-Kenyon Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: timruhkopf Co-authored-by: Alexander Tornede Co-authored-by: Sarah Krebs Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com> Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: Jeroen Rook Co-authored-by: Tim Ruhkopf * feat: Option preventing SMAC setting up logging * fix: Run isort * Bump peter-evans/create-pull-request from 3 to 5 (#958) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3 to 5. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](https://github.com/peter-evans/create-pull-request/compare/v3...v5) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 2 to 4 (#945) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v2...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat: Option preventing SMAC setting up logging * fix: Run isort * fix: Add to changelog * feat: Option preventing SMAC setting up logging * fix: Run isort --------- Signed-off-by: dependabot[bot] Co-authored-by: René Sass Co-authored-by: dengdifan <33290713+dengdifan@users.noreply.github.com> Co-authored-by: dengdifan Co-authored-by: Matthias Feurer Co-authored-by: Carolin Benjamins Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Carolin Benjamins Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Katharina Eggensperger Co-authored-by: Eric Kalosa-Kenyon Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: timruhkopf Co-authored-by: Alexander Tornede Co-authored-by: Sarah Krebs Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com> Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: Jeroen Rook Co-authored-by: Tim Ruhkopf * Citation update (#961) * update CITATION.cff * update order in CITATION.cff * fix citation date * fix citation * maint * Adjust hyperband configuration distribution across brackets * Compute the actual differences between the isb keys. (#957) * Compute the actual differences between the isb keys. * Added change to CHANGELOG.md * Adjust schedule for stale reminder * fix(logging): Prevent automatic logging setup at init (#970) * fix(logging): Prevent automatic logging setup at init * fix(logging): Rely on `setup_logging` to handle arg * Fix validate method of smbo, update docstring. * Allow callbacks to be added to a specific index, make callback registering public * Update CHANGELOG.md * Fix broken references (#988) * Adapt developer install instructions to include pre-commit installation (#994) * [Feature] Initial Design now supports executing the default config provided by the user by means of the configspace he generated. (and is no longer ignored) * Pipe dask_client through facade for exec on hpc (#983) * Pipe dask_client through facade for exec on hpc * Handle pynisher's memory arg Before: Could only pass int via SMAC with the default unit B, but now tuple/list is possible * Update CHANGELOG.md * Make format * No printi --------- Co-authored-by: Carolin Benjamins * [Bug-fix] Pr was failing due to mutable additional configs default * [doc-fix] the count of initial design configs is computed slightly differently * Feature/metadata callback (#999) * Add example for using a callback to log run metadata to a file * Add issue number in changelog * Changelog formatting * Simplify metadata example * Simplify metadata example * Put callback in smac directory to allow importing it in custom repos * Move callback files to callback directory * Edit changelog * Correct metadata callback argument types * Additional metadata arguments in example * Set metadata default values in callback * Edit comment to fix PR * Documentation/dask client example (#1001) * Update requirements (dask_jobqueue) * Fix broken link * Ignore parallel example * Add parallelization example * Remove imports, add comment abt dask client * Add E's information * Update docs * Add comment about n_workers --------- Co-authored-by: Carolin Benjamins * Documentation/dask client example (#1001) * Update requirements (dask_jobqueue) * Fix broken link * Ignore parallel example * Add parallelization example * Remove imports, add comment abt dask client * Add E's information * Update docs * Add comment about n_workers --------- Co-authored-by: Carolin Benjamins * Feature/dask client (#1002) * Add info message that scenario.n_workers is ignored When passing a dask client, directly set n_workers in client. * Change info to warning --------- Co-authored-by: Carolin Benjamins * Update smac version (#1003) * Update smac version * Add hint on github release in make publish * Update copyright year * check if config in rh when storing state (#997) * check if config in rh when storing state * maint * remove unnecessary changes * Adapt developer install instructions to include pre-commit installation (#994) * Pipe dask_client through facade for exec on hpc (#983) * Pipe dask_client through facade for exec on hpc * Handle pynisher's memory arg Before: Could only pass int via SMAC with the default unit B, but now tuple/list is possible * Update CHANGELOG.md * Make format * No printi --------- Co-authored-by: Carolin Benjamins * update changelog * Increase tolerance for runhistory restoration test --------- Co-authored-by: Sarah Segel <30651149+sarah-segel@users.noreply.github.com> Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Carolin Benjamins Co-authored-by: Helena Graf * Add a workaround to be able to pass a dataset via dask.scatter (#993) * [feat] Support dask.scatter * [fix] Add shared_data arg in abstract_facade * Address the comments by Eddie * [fix] Fix errors raised by Eddie * [format] Apply black locally * [fix] Get retries back to 16 as it was a mistake * [doc] Update CHANGELOG --------- Co-authored-by: helegraf Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> * Adding Binder links for example (#976) * binder * maint * fix dependencies * maint docs * Revert "maint docs" This reverts commit 4fa1a0fae8dfa0b646e43273d50f72ddd76bf191. * add requirement --------- Co-authored-by: Difan Deng * Remove commit message from dependabot PRs (#1009) --------- Signed-off-by: dependabot[bot] Co-authored-by: dengdifan Co-authored-by: René Sass Co-authored-by: Carolin Benjamins Co-authored-by: Carolin Benjamins Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Katharina Eggensperger Co-authored-by: Matthias Feurer Co-authored-by: Eric Kalosa-Kenyon Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: timruhkopf Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com> Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: Jeroen Rook Co-authored-by: Eddie Bergman Co-authored-by: Tim Ruhkopf Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com> Co-authored-by: Aron Bahram Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shuhei Watanabe <47781922+nabenabe0928@users.noreply.github.com> Co-authored-by: Difan Deng * Fix broken link --------- Signed-off-by: dependabot[bot] Co-authored-by: dengdifan Co-authored-by: René Sass Co-authored-by: Carolin Benjamins Co-authored-by: Carolin Benjamins Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Katharina Eggensperger Co-authored-by: Matthias Feurer Co-authored-by: Eric Kalosa-Kenyon Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: timruhkopf Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com> Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: Jeroen Rook Co-authored-by: Eddie Bergman Co-authored-by: Tim Ruhkopf Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com> Co-authored-by: Aron Bahram Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shuhei Watanabe <47781922+nabenabe0928@users.noreply.github.com> Co-authored-by: Difan Deng * Drop torch requirements (#1015) * Remove torch requirements * Fix format --------- Co-authored-by: Carolin Benjamins * Remove leftover CLI entry points (#1014) * Fix incumbent selection in case of SH+MO * Update CHANGELOG.md * Feature/GitHub actions pr draft (#1008) Update the workflows only to trigger extensive tests for PRs that are not in the draft stage --------- Co-authored-by: timruhkopf Co-authored-by: Helena Graf * Update stale report to include issues with recent interactions (#1025) * Add recently updated issues to list of reminders. * Add recently updated issues to list of reminders. * Add recently updated issues to list of reminders. * Raise error if resource limitation and parallelization is requested (#1023) * Add warnings for parallelism doc * Check for parallelization + pynisher Does not work together, throw error. Does not work bc Dask pickles jobs and pynisher runs the target function in a process. * Fix whitespaces --------- Co-authored-by: Carolin Benjamins * Raise an error for invalid scatter data (#1018) * [enhance] Add an error for scatter data * Update ChangeLog --------- Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> * fix(configspace): shows all keys in print (#1043) * Fix callback order (#1040) Co-authored-by: Carolin Benjamins * fix: dtypes (#1044) * Fix(config_selector): Budget checking (#1039) * Fix: target runner with partial func (#1045) * Feature/improve documentation (#1028) * Experimental instruction for installing SMAC in windows via WSL * More detailed documentation regarding continuing runs * Fixes * Fixes * Fixes * Update docs/10_experimental.rst Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com> * Fixes from PR * Fixes from PR * Fixes from PR * Fixes from PR * Fixes from PR --------- Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com> * Handle configspace as dictionary in mlp example (#1057) * Handle configspace as dictionary in mlp example * Adapt sgd loss to newest scikit-learn version * Add change to changelog * Handle configspace as dictionary in parego example * Correct get usage * Raise version number * Add missing period in docs. --------- Signed-off-by: dependabot[bot] Co-authored-by: dengdifan Co-authored-by: Carolin Benjamins Co-authored-by: René Sass Co-authored-by: Carolin Benjamins Co-authored-by: C. Benjamins <75323339+benjamc@users.noreply.github.com> Co-authored-by: Katharina Eggensperger Co-authored-by: Matthias Feurer Co-authored-by: Eric Kalosa-Kenyon Co-authored-by: Pascal <66594767+PascalIversen@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: timruhkopf Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede Co-authored-by: Alexander Tornede <44395781+alexandertornede@users.noreply.github.com> Co-authored-by: Sarah Krebs <30651149+sarah-krebs@users.noreply.github.com> Co-authored-by: helegraf Co-authored-by: Jeroen Rook Co-authored-by: Eddie Bergman Co-authored-by: Tim Ruhkopf Co-authored-by: Difan Deng <33290713+dengdifan@users.noreply.github.com> Co-authored-by: Aron Bahram Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shuhei Watanabe <47781922+nabenabe0928@users.noreply.github.com> Co-authored-by: Difan Deng Co-authored-by: Bastian Zimmermann <10774221+BastianZim@users.noreply.github.com> --- .github/workflows/dist.yml | 10 +++- .github/workflows/docs.yml | 14 ++++-- .github/workflows/pre-commit.yml | 8 +++- .github/workflows/pytest.yml | 27 +++++++---- CHANGELOG.md | 13 +++++ CITATION.cff | 2 +- Makefile | 2 +- benchmark/src/wrappers/v20.py | 2 +- docs/10_experimental.rst | 48 +++++++++++++++++++ docs/1_installation.rst | 8 ++++ docs/advanced_usage/10_continue.rst | 19 ++++++-- docs/advanced_usage/9_parallelism.rst | 33 +++++++++++++ docs/conf.py | 1 + examples/1_basics/5_continue.py | 12 ++++- examples/2_multi_fidelity/1_mlp_epochs.py | 6 +-- examples/2_multi_fidelity/2_sgd_datasets.py | 2 +- examples/3_multi_objective/2_parego.py | 6 +-- .../3_metadata_callback.py | 3 +- setup.py | 6 --- smac/__init__.py | 2 +- smac/facade/abstract_facade.py | 21 +++++++- smac/intensifier/successive_halving.py | 3 +- smac/main/config_selector.py | 16 +++---- smac/main/smbo.py | 12 +++-- .../gaussian_process/kernels/base_kernels.py | 8 ++-- smac/runhistory/runhistory.py | 6 +-- smac/runner/target_function_runner.py | 13 ++++- smac/utils/configspace.py | 17 ++++--- smac/utils/multi_objective.py | 2 +- tests/fixtures/models.py | 2 +- tests/fixtures/scenario.py | 2 +- .../test_initial_design.py | 2 +- 32 files changed, 254 insertions(+), 74 deletions(-) create mode 100644 docs/10_experimental.rst diff --git a/.github/workflows/dist.yml b/.github/workflows/dist.yml index ffac9f005a..0965ea2f1c 100644 --- a/.github/workflows/dist.yml +++ b/.github/workflows/dist.yml @@ -1,7 +1,7 @@ name: dist on: - # Manual trigger option in github + # Manual trigger option in GitHub workflow_dispatch: # Trigger on push to these branches @@ -10,14 +10,20 @@ on: - main - development - # Trigger on a open/push to a PR targeting one of these branches + # Trigger on open/push to a PR targeting one of these branches pull_request: + types: + - opened + - synchronize + - reopened + - ready_for_review branches: - main - development jobs: dist: + if: ${{ !github.event.pull_request.draft }} runs-on: ubuntu-latest steps: diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index e1b16334df..d89791712f 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -1,9 +1,9 @@ name: docs on: - # Manual trigger option in github - # This won't push to github pages where docs are hosted due - # to the gaurded if statement in those steps + # Manual trigger option in GitHub + # This won't push to GitHub pages where docs are hosted due + # to the guarded if statement in those steps workflow_dispatch: # Trigger on push to these branches @@ -12,8 +12,13 @@ on: - main - development - # Trigger on a open/push to a PR targeting one of these branches + # Trigger on open/push to a PR targeting one of these branches pull_request: + types: + - opened + - synchronize + - reopened + - ready_for_review branches: - main - development @@ -23,6 +28,7 @@ env: jobs: build-and-deploy: + if: ${{ !github.event.pull_request.draft }} runs-on: ubuntu-latest steps: - name: Checkout diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml index 5f5c400fc2..e9ba1a01bf 100644 --- a/.github/workflows/pre-commit.yml +++ b/.github/workflows/pre-commit.yml @@ -1,7 +1,7 @@ name: pre-commit on: - # Manually triggerable in github + # Manually triggerable in GitHub workflow_dispatch: # When a push occurs on either of these branches @@ -12,12 +12,18 @@ on: # When a push occurs on a PR that targets these branches pull_request: + types: + - opened + - synchronize + - reopened + - ready_for_review branches: - main - development jobs: run-all-files: + if : ${{ !github.event.pull_request.draft }} runs-on: ubuntu-latest steps: diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index f82e49c1d5..37e15a4729 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -1,7 +1,7 @@ name: tests on: - # Allow to manually trigger through github API + # Allow to manually trigger through GitHub API workflow_dispatch: # Triggers with push to main @@ -12,6 +12,11 @@ on: # Triggers with push to a pr aimed at main pull_request: + types: + - opened + - synchronize + - reopened + - ready_for_review branches: - main - development @@ -34,6 +39,7 @@ jobs: # General unit tests source-test: name: ${{ matrix.python-version }}-${{ matrix.os }} + if: ${{ !github.event.pull_request.draft }} runs-on: ${{ matrix.os }} defaults: @@ -44,7 +50,6 @@ jobs: fail-fast: false matrix: python-version: ["3.8", "3.9", "3.10"] - # python-version: ["3.8"] os: ["ubuntu-latest"] steps: @@ -70,10 +75,10 @@ jobs: - name: Tests run: | - pytest ${{ env.pytest-args }} ${{ env.test-dir }} - - # Post-hoc clean-up - make clean + echo "Running all tests..." + pytest ${{ env.pytest-args }} ${{ env.test-dir }} + # Post-hoc clean-up + make clean - name: Check for files left behind by test run: | @@ -89,6 +94,7 @@ jobs: # Testing with conda conda-tests: name: conda-${{ matrix.python-version }}-${{ matrix.os }} + if: ${{ !github.event.pull_request.draft }} runs-on: ${{ matrix.os }} defaults: run: @@ -98,7 +104,6 @@ jobs: fail-fast: false matrix: python-version: ["3.8", "3.9", "3.10"] - # python-version: ["3.8"] os: ["ubuntu-latest"] steps: @@ -120,11 +125,15 @@ jobs: - name: Tests run: | - pytest ${{ env.pytest-args }} ${{ env.test-dir }} + echo "Running all tests..." + pytest ${{ env.pytest-args }} ${{ env.test-dir }} + + # Testing a dist install dist-test: name: dist-${{ matrix.python-version }}-${{ matrix.os }} + if: ${{ !github.event.pull_request.draft }} runs-on: ${{ matrix.os }} defaults: @@ -135,7 +144,6 @@ jobs: fail-fast: false matrix: python-version: ["3.8", "3.9", "3.10"] - # python-version: ["3.8"] os: ["ubuntu-latest"] steps: @@ -161,4 +169,5 @@ jobs: - name: Tests run: | + echo "Running all tests..." pytest ${{ env.pytest-args }} ${{ env.test-dir }} diff --git a/CHANGELOG.md b/CHANGELOG.md index 807cbaf7ec..dbcac80e4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,16 @@ +# 2.0.2 + +## Improvements +- Add an error when we get an empty dict data_to_scatter so that we can avoid an internal error caused in Dask precautiously +- Add experimental instruction for installing SMAC in Windows via a WSL. +- More detailed documentation regarding continuing runs. + +## Bugfixes +- Fix bug in the incumbent selection in the case that multi-fidelity is combined with multi-objective (#1019). +- Fix callback order (#1040). +- Handle configspace as dictionary in mlp and parego example. +- Adapt sgd loss to newest scikit-learn version. + # 2.0.1 ## Improvements diff --git a/CITATION.cff b/CITATION.cff index 36435d6ecf..a984b24c58 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -9,7 +9,7 @@ date-released: "2016-08-17" url: "https://automl.github.io/SMAC3/master/index.html" repository-code: "https://github.com/automl/SMAC3" -version: "2.0.1" +version: "2.0.2" type: "software" keywords: diff --git a/Makefile b/Makefile index f7727cf7f6..ba7a4268b8 100644 --- a/Makefile +++ b/Makefile @@ -5,7 +5,7 @@ SHELL := /bin/bash NAME := SMAC3 PACKAGE_NAME := smac -VERSION := 2.0.1 +VERSION := 2.0.2 DIR := "${CURDIR}" SOURCE_DIR := ${PACKAGE_NAME} diff --git a/benchmark/src/wrappers/v20.py b/benchmark/src/wrappers/v20.py index 8ed0864ecb..9f2be56dcb 100644 --- a/benchmark/src/wrappers/v20.py +++ b/benchmark/src/wrappers/v20.py @@ -6,7 +6,7 @@ class Version20(Wrapper): - supported_versions: list[str] = ["2.0.1"] + supported_versions: list[str] = ["2.0.2"] def __init__(self, task: Task, seed: int) -> None: super().__init__(task, seed) diff --git a/docs/10_experimental.rst b/docs/10_experimental.rst new file mode 100644 index 0000000000..a075498c4a --- /dev/null +++ b/docs/10_experimental.rst @@ -0,0 +1,48 @@ +Experimental +============ + +.. warning:: + This part is experimental and might not work in each case. If you would like to suggest any changes, please let us know. + + +Installation in Windows via WSL +------------------------------ + +SMAC can be installed in a WSL (Windows Subsystem for Linux) under Windows. + +**1) Install WSL under Windows** + +Install WSL under Windows. This SMAC installation workflow was tested with Ubuntu 18.04. For Ubuntu 20.04, +it has been observed that the SMAC installation results in a segmentation fault (core dumped). + +**2) Get Anaconda** + +Download an Anaconda Linux version to drive D under Windows, e.g. D:\\Anaconda3-2023.03-1-Linux-x86_64 + +In the WSL, Windows resources are mounted under /mnt: + +.. code:: bash + + cd /mnt/d + bash Anaconda3-2023.03-1-Linux-x86_64 + +Enter this command to create the environment variable: + +.. code:: bash + + export PATH="$PATH:/home/${USER}/anaconda3/bin + +Input 'python' to check if the installation was successful. + +**3) Install SMAC** + +Change to your home folder and install the general software there: + +.. code:: bash + + cd /home/${USER} + sudo apt-get install software-properties-common + sudo apt-get update + sudo apt-get install build-essential swig + conda install gxx_linux-64 gcc_linux-64 swig + curl https://raw.githubusercontent.com/automl/smac3/master/requirements.txt | xargs -n 1 -L 1 pip install diff --git a/docs/1_installation.rst b/docs/1_installation.rst index 54f910a441..a835d9cf19 100644 --- a/docs/1_installation.rst +++ b/docs/1_installation.rst @@ -68,3 +68,11 @@ You must have `conda >= 4.9` installed. To update conda or check your current co Read `SMAC feedstock `_ for more details. + +Windows via WSL (Experimental) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +SMAC can be installed under Windows in a WSL (Windows Subsystem for Linux). +You can find an instruction on how to do this here: :ref:`Experimental`. +However, this is experimental and might not work in each case. +If you would like to suggest any changes, please let us know. diff --git a/docs/advanced_usage/10_continue.rst b/docs/advanced_usage/10_continue.rst index 8a254c4e59..e29c3f9c96 100644 --- a/docs/advanced_usage/10_continue.rst +++ b/docs/advanced_usage/10_continue.rst @@ -1,14 +1,23 @@ Continue ======== -SMAC automatically restores states where it left off if a run was interrupted or finished. To do so, it reads in old -files (derived from scenario's name, output_directory and seed) and sets the components. +SMAC can automatically restore states where it left off if a run was interrupted or prematurely finished. To do so, +it reads in old files (derived from scenario's name, output_directory and seed) and obtains the scenario information +of the previous run from those to continue the run. + +The behavior can be controlled by setting the parameter ``overwrite`` in the facade to True or False, respectively: + +* If set to True, SMAC overwrites the run results if a previous run is found that is consistent in the meta data with the current setup. +* If set to False and a previous run is found that + + * is consistent in the meta data, the run is continued. + * is not consistent in the meta data, the user is asked for the exact behaviour (overwrite completely or rename old run first). .. warning:: - If you changed any code and specified a name, SMAC will ask you whether you still want to resume or - delete the old run completely. If you did not specify a name, SMAC generates a new name and the old run is - not affected. + If you changed any code affecting the run's meta data and specified a name, SMAC will ask you whether you still + want to overwrite the old run or rename the old run first. If you did not specify a name, SMAC generates a new name + and the old run is not affected. Please have a look at our :ref:`continue example`. \ No newline at end of file diff --git a/docs/advanced_usage/9_parallelism.rst b/docs/advanced_usage/9_parallelism.rst index 9912c782f5..67e555215b 100644 --- a/docs/advanced_usage/9_parallelism.rst +++ b/docs/advanced_usage/9_parallelism.rst @@ -21,6 +21,39 @@ SMAC supports multiple workers natively via Dask. Just specify ``n_workers`` in When using multiple workers, SMAC is not reproducible anymore. +.. warning :: + + You cannot use resource limitation (pynisher, via the `scenario` arguments `trail_walltime_limit` and `trial_memory_limit`). + This is because pynisher works by running your function inside of a subprocess. + Once in the subprocess, the resources will be limited for that process before running your function. + This does not work together with pickling - which is required by dask to schedule jobs on the cluster, even on a local one. + + +.. warning :: + + Start/run SMAC inside ``if __name__ == "__main__"`` in your script otherwise Dask is not able to correctly + spawn jobs and probably this runtime error will be raised: + + .. code-block :: + + RuntimeError: + An attempt has been made to start a new process before the + current process has finished its bootstrapping phase. + + This probably means that you are not using fork to start your + child processes and you have forgotten to use the proper idiom + in the main module: + + if __name__ == '__main__': + freeze_support() + ... + + The "freeze_support()" line can be omitted if the program + is not going to be frozen to produce an executable. + + + + Running on a Cluster -------------------- You can also pass a custom dask client, e.g. to run on a slurm cluster. diff --git a/docs/conf.py b/docs/conf.py index d7b5598884..198361fb9e 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -12,6 +12,7 @@ "version": version, "versions": { f"v{version}": "#", + "v2.0.1": "https://automl.github.io/SMAC3/v2.0.1/", "v2.0.0": "https://automl.github.io/SMAC3/v2.0.0/", "v2.0.0b1": "https://automl.github.io/SMAC3/v2.0.0b1/", "v2.0.0a2": "https://automl.github.io/SMAC3/v2.0.0a2/", diff --git a/examples/1_basics/5_continue.py b/examples/1_basics/5_continue.py index 04cca4aed9..025856fee0 100644 --- a/examples/1_basics/5_continue.py +++ b/examples/1_basics/5_continue.py @@ -2,8 +2,16 @@ Continue an Optimization ^^^^^^^^^^^^^^^^^^^^^^^^ -SMAC can also be continued. In this example, an optimization of a simple quadratic -function is continued. We use a custom callback, to artificially stop the first optimization. +SMAC can also be continued from a previous run. To do so, it reads in old files (derived from scenario's name, +output_directory and seed) and sets the corresponding components. In this example, an optimization of a simple quadratic +function is continued. + +First, after creating a scenario with 50 trials, we run SMAC with overwrite=True. This will +overwrite any previous runs (in case the example was called before). We use a custom callback to artificially stop +this first optimization after 10 trials. + +Second, we again run the SMAC optimization using the same scenario, but this time with overwrite=False. As +there already is a previous run with the same meta data, this run will be continued until the 50 trials are reached. """ from __future__ import annotations diff --git a/examples/2_multi_fidelity/1_mlp_epochs.py b/examples/2_multi_fidelity/1_mlp_epochs.py index 9fd256c5d6..48c027d3bd 100644 --- a/examples/2_multi_fidelity/1_mlp_epochs.py +++ b/examples/2_multi_fidelity/1_mlp_epochs.py @@ -84,9 +84,9 @@ def train(self, config: Configuration, seed: int = 0, budget: int = 25) -> float # For deactivated parameters (by virtue of the conditions), # the configuration stores None-values. # This is not accepted by the MLP, so we replace them with placeholder values. - lr = config["learning_rate"] if config["learning_rate"] else "constant" - lr_init = config["learning_rate_init"] if config["learning_rate_init"] else 0.001 - batch_size = config["batch_size"] if config["batch_size"] else 200 + lr = config.get("learning_rate", "constant") + lr_init = config.get("learning_rate_init", 0.001) + batch_size = config.get("batch_size", 200) with warnings.catch_warnings(): warnings.filterwarnings("ignore") diff --git a/examples/2_multi_fidelity/2_sgd_datasets.py b/examples/2_multi_fidelity/2_sgd_datasets.py index 384d1c2246..09864a963a 100644 --- a/examples/2_multi_fidelity/2_sgd_datasets.py +++ b/examples/2_multi_fidelity/2_sgd_datasets.py @@ -89,7 +89,7 @@ def train(self, config: Configuration, instance: str, seed: int = 0) -> float: # SGD classifier using given configuration clf = SGDClassifier( - loss="log", + loss="log_loss", penalty="elasticnet", alpha=config["alpha"], l1_ratio=config["l1_ratio"], diff --git a/examples/3_multi_objective/2_parego.py b/examples/3_multi_objective/2_parego.py index d8fdc5ff8d..856c2e857f 100644 --- a/examples/3_multi_objective/2_parego.py +++ b/examples/3_multi_objective/2_parego.py @@ -66,9 +66,9 @@ def configspace(self) -> ConfigurationSpace: return cs def train(self, config: Configuration, seed: int = 0, budget: int = 10) -> dict[str, float]: - lr = config["learning_rate"] if config["learning_rate"] else "constant" - lr_init = config["learning_rate_init"] if config["learning_rate_init"] else 0.001 - batch_size = config["batch_size"] if config["batch_size"] else 200 + lr = config.get("learning_rate", "constant") + lr_init = config.get("learning_rate_init", 0.001) + batch_size = config.get("batch_size", 200) start_time = time.time() diff --git a/examples/4_advanced_optimizer/3_metadata_callback.py b/examples/4_advanced_optimizer/3_metadata_callback.py index 1f7078f941..ab35f28627 100644 --- a/examples/4_advanced_optimizer/3_metadata_callback.py +++ b/examples/4_advanced_optimizer/3_metadata_callback.py @@ -19,6 +19,7 @@ """ import sys + from ConfigSpace import Configuration, ConfigurationSpace, Float from smac import HyperparameterOptimizationFacade as HPOFacade @@ -65,7 +66,7 @@ def train(self, config: Configuration, seed: int = 0) -> float: branch="Name of Active Branch", commit="Commit Hash", command=" ".join(sys.argv), - additional_information="Some Additional Information" + additional_information="Some Additional Information", ) ], logging_level=999999, diff --git a/setup.py b/setup.py index 79249cd5f9..f194d36771 100644 --- a/setup.py +++ b/setup.py @@ -20,9 +20,7 @@ def read_file(filepath: str) -> str: return fh.read() -torch_requirements = ["torch>=1.9.0", "gpytorch>=1.5.0", "pyro-ppl>=1.7.0", "botorch>=0.5.0"] extras_require = { - "gpytorch": torch_requirements, "dev": [ "setuptools", "types-setuptools", @@ -40,7 +38,6 @@ def read_file(filepath: str) -> str: "pydocstyle", "flake8", "pre-commit", - *torch_requirements, ], } @@ -76,9 +73,6 @@ def read_file(filepath: str) -> str: extras_require=extras_require, test_suite="pytest", platforms=["Linux"], - entry_points={ - "console_scripts": ["smac = smac.smac_cli:cmd_line_call"], - }, classifiers=[ "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", diff --git a/smac/__init__.py b/smac/__init__.py index f3fe5805b9..ab27812974 100644 --- a/smac/__init__.py +++ b/smac/__init__.py @@ -19,7 +19,7 @@ Copyright {datetime.date.today().strftime('%Y')}, Marius Lindauer, Katharina Eggensperger, Matthias Feurer, André Biedenkapp, Difan Deng, Carolin Benjamins, Tim Ruhkopf, René Sass and Frank Hutter""" -version = "2.0.1" +version = "2.0.2" try: diff --git a/smac/facade/abstract_facade.py b/smac/facade/abstract_facade.py index adb1411123..a5db5ab521 100644 --- a/smac/facade/abstract_facade.py +++ b/smac/facade/abstract_facade.py @@ -92,8 +92,10 @@ class AbstractFacade: Callbacks, which are incorporated into the optimization loop. overwrite: bool, defaults to False When True, overwrites the run results if a previous run is found that is - inconsistent in the meta data with the current setup. If ``overwrite`` is set to False, the user is asked - for the exact behaviour (overwrite completely, save old run, or use old results). + consistent in the meta data with the current setup. When False and a previous run is found that is + consistent in the meta data, the run is continued. When False and a previous run is found that is + not consistent in the meta data, the the user is asked for the exact behaviour (overwrite completely + or rename old run first). dask_client: Client | None, defaults to None User-created dask client, which can be used to start a dask cluster and then attach SMAC to it. This will not be closed automatically and will have to be closed manually if provided explicitly. If none is provided @@ -310,6 +312,9 @@ def optimize(self, *, data_to_scatter: dict[str, Any] | None = None) -> Configur Best found configuration. """ incumbents = None + if isinstance(data_to_scatter, dict) and len(data_to_scatter) == 0: + raise ValueError("data_to_scatter must be None or dict with some elements, but got an empty dict.") + try: incumbents = self._optimizer.optimize(data_to_scatter=data_to_scatter) finally: @@ -455,6 +460,18 @@ def _validate(self) -> None: # Make sure the same acquisition function is used assert self._acquisition_function == self._acquisition_maximizer._acquisition_function + if isinstance(self._runner, DaskParallelRunner) and ( + self.scenario.trial_walltime_limit is not None or self.scenario.trial_memory_limit is not None + ): + # This is probably due to pickling dask jobs + raise ValueError( + "Parallelization via Dask cannot be used in combination with limiting " + "the resources " + "of the target function via `scenario.trial_walltime_limit` or " + "`scenario.trial_memory_limit`. Set those to `None` if you want " + "parallelization. " + ) + def _get_signature_arguments(self) -> list[str]: """Returns signature arguments, which are required by the intensifier.""" arguments = [] diff --git a/smac/intensifier/successive_halving.py b/smac/intensifier/successive_halving.py index e7061a190b..01b95c6d86 100644 --- a/smac/intensifier/successive_halving.py +++ b/smac/intensifier/successive_halving.py @@ -558,7 +558,8 @@ def _get_best_configs( # If we have more selected configs, we remove the ones with the smallest crowding distance if len(selected_configs) > n_configs: - selected_configs = sort_by_crowding_distance(rh, configs, all_keys)[:n_configs] + all_keys = [from_keys for _ in selected_configs] + selected_configs = sort_by_crowding_distance(rh, selected_configs, all_keys)[:n_configs] logger.debug("Found more configs than required. Removed configs with smallest crowding distance.") return selected_configs diff --git a/smac/main/config_selector.py b/smac/main/config_selector.py index 76ace17f84..7b2053eee2 100644 --- a/smac/main/config_selector.py +++ b/smac/main/config_selector.py @@ -266,14 +266,14 @@ def _collect_data(self) -> tuple[np.ndarray, np.ndarray, np.ndarray]: assert self._runhistory_encoder is not None # If we use a float value as a budget, we want to train the model only on the highest budget - available_budgets = [] - for run_key in self._runhistory: - budget = run_key.budget - if budget not in available_budgets: - available_budgets.append(run_key.budget) - - # Sort available budgets from highest to lowest budget - available_budgets = sorted(list(set(available_budgets)), reverse=True) # type: ignore + unique_budgets: set[float] = {run_key.budget for run_key in self._runhistory if run_key.budget is not None} + + available_budgets: list[float] | list[None] + if len(unique_budgets) > 0: + # Sort available budgets from highest to lowest budget + available_budgets = sorted(unique_budgets, reverse=True) + else: + available_budgets = [None] # Get #points per budget and if there are enough samples, then build a model for b in available_budgets: diff --git a/smac/main/smbo.py b/smac/main/smbo.py index 30e01c2e97..6138e4d64f 100644 --- a/smac/main/smbo.py +++ b/smac/main/smbo.py @@ -461,18 +461,22 @@ def _add_results(self) -> None: logger.info("Cost threshold was reached. Abort is requested.") self._stop = True - def register_callback(self, callback: Callback, index: int = -1) -> None: + def register_callback(self, callback: Callback, index: int | None = None) -> None: """ Registers a callback to be called before, in between, and after the Bayesian optimization loop. + Callback is appended to the list by default. Parameters ---------- callback : Callback The callback to be registered. - index : int - The index at which the callback should be registered. + index : int, optional + The index at which the callback should be registered. The default is None. + If it is None, append the callback to the list. """ + if index is None: + index = len(self._callbacks) self._callbacks.insert(index, callback) def _initialize_state(self) -> None: @@ -501,7 +505,7 @@ def _initialize_state(self) -> None: logger.info("Since the previous run was not successful, SMAC will start from scratch again.") self.reset() else: - # Here, we run into differen scenarios + # Here, we run into different scenarios diff = recursively_compare_dicts( Scenario.make_serializable(self._scenario), Scenario.make_serializable(old_scenario), diff --git a/smac/model/gaussian_process/kernels/base_kernels.py b/smac/model/gaussian_process/kernels/base_kernels.py index 044ad4fcaa..4dd6ff6bbf 100644 --- a/smac/model/gaussian_process/kernels/base_kernels.py +++ b/smac/model/gaussian_process/kernels/base_kernels.py @@ -250,12 +250,12 @@ def _signature(self, func: Callable) -> Signature: def _set_active_dims(self, operate_on: np.ndarray | None = None) -> None: """Sets dimensions this kernel should work on.""" - if operate_on is not None and type(operate_on) in (list, np.ndarray): + if operate_on is not None and isinstance(operate_on, (list, np.ndarray)): if not isinstance(operate_on, np.ndarray): - raise TypeError("The argument `operate_on` needs to be of type np.ndarray but is %s" % type(operate_on)) + raise TypeError(f"The argument `operate_on` needs to be of type np.ndarray but is {type(operate_on)}") - if operate_on.dtype != int: - raise ValueError("The dtype of argument `operate_on` needs to be int, but is %s" % operate_on.dtype) + if not np.issubdtype(operate_on.dtype, np.integer): + raise ValueError(f"The dtype of `operate_on` needs to be np.integer, but is {operate_on.dtype}") self.operate_on = operate_on self._len_active = len(operate_on) diff --git a/smac/runhistory/runhistory.py b/smac/runhistory/runhistory.py index a30e1356ee..aaf88889c8 100644 --- a/smac/runhistory/runhistory.py +++ b/smac/runhistory/runhistory.py @@ -435,7 +435,7 @@ def get_min_cost(self, config: Configuration) -> float: cost = self._min_cost_per_config.get(config_id, np.nan) # type: ignore if self._n_objectives > 1: - assert type(cost) == list + assert isinstance(cost, list) assert self.multi_objective_algorithm is not None costs = normalize_costs(cost, self._objective_bounds) @@ -443,7 +443,7 @@ def get_min_cost(self, config: Configuration) -> float: # Note: We have to mean here because we already got the min cost return self.multi_objective_algorithm(costs) - assert type(cost) == float + assert isinstance(cost, float) return float(cost) def average_cost( @@ -847,7 +847,7 @@ def load(self, filename: str | Path, configspace: ConfigurationSpace) -> None: for entry in data["data"]: # Set n_objectives first if self._n_objectives == -1: - if isinstance(entry[4], float) or isinstance(entry[4], int): + if isinstance(entry[4], (float, int)): self._n_objectives = 1 else: self._n_objectives = len(entry[4]) diff --git a/smac/runner/target_function_runner.py b/smac/runner/target_function_runner.py index 28837caa09..5431f5ed47 100644 --- a/smac/runner/target_function_runner.py +++ b/smac/runner/target_function_runner.py @@ -7,6 +7,7 @@ import math import time import traceback +from functools import partial import numpy as np from ConfigSpace import Configuration @@ -88,7 +89,17 @@ def __init__( @property def meta(self) -> dict[str, Any]: # noqa: D102 meta = super().meta - meta.update({"code": str(self._target_function.__code__.co_code)}) + + # Partial's don't have a __code__ attribute but are a convenient + # way a user might want to pass a function to SMAC, specifying + # keyword arguments. + f = self._target_function + if isinstance(f, partial): + f = f.func + meta.update({"code": str(f.__code__.co_code)}) + meta.update({"code-partial-args": repr(f)}) + else: + meta.update({"code": str(self._target_function.__code__.co_code)}) return meta diff --git a/smac/utils/configspace.py b/smac/utils/configspace.py index e6fdeddbdf..8f281ed3e9 100644 --- a/smac/utils/configspace.py +++ b/smac/utils/configspace.py @@ -169,12 +169,17 @@ def print_config_changes( if incumbent is None or challenger is None: return - params = sorted([(param, incumbent[param], challenger[param]) for param in challenger.keys()]) - for param in params: - if param[1] != param[2]: - logger.info("--- %s: %r -> %r" % param) - else: - logger.debug("--- %s Remains unchanged: %r", param[0], param[1]) + inc_keys = set(incumbent.keys()) + all_keys = inc_keys.union(challenger.keys()) + + lines = [] + for k in sorted(all_keys): + inc_k = incumbent.get(k, "-inactive-") + cha_k = challenger.get(k, "-inactive-") + lines.append(f"--- {k}: {inc_k} -> {cha_k}" + " (unchanged)" if inc_k == cha_k else "") + + msg = "\n".join(lines) + logger.debug(msg) # def check_subspace_points( diff --git a/smac/utils/multi_objective.py b/smac/utils/multi_objective.py index 2dec4da949..f959fe7836 100644 --- a/smac/utils/multi_objective.py +++ b/smac/utils/multi_objective.py @@ -29,7 +29,7 @@ def normalize_costs( costs = [] for v, b in zip(values, bounds): - assert type(v) != list + assert not isinstance(v, list) p = v - b[0] q = b[1] - b[0] diff --git a/tests/fixtures/models.py b/tests/fixtures/models.py index 1321176f27..29f9bf152e 100644 --- a/tests/fixtures/models.py +++ b/tests/fixtures/models.py @@ -71,7 +71,7 @@ def train(self, config: Configuration, instance: str = "0-1", budget: float = 1, # SGD classifier using given configuration clf = SGDClassifier( - loss="log", + loss="log_loss", penalty="elasticnet", alpha=config["alpha"], l1_ratio=config["l1_ratio"], diff --git a/tests/fixtures/scenario.py b/tests/fixtures/scenario.py index 6211939879..4830c68c68 100644 --- a/tests/fixtures/scenario.py +++ b/tests/fixtures/scenario.py @@ -51,7 +51,7 @@ def _make( instance_features=instance_features, min_budget=min_budget, max_budget=max_budget, - use_default_config=use_default_config + use_default_config=use_default_config, ) return _make diff --git a/tests/test_initial_design/test_initial_design.py b/tests/test_initial_design/test_initial_design.py index d12e503ff5..ed08db0d5b 100644 --- a/tests/test_initial_design/test_initial_design.py +++ b/tests/test_initial_design/test_initial_design.py @@ -122,6 +122,7 @@ def test_select_configurations(make_scenario, configspace_small): with pytest.raises(NotImplementedError): dc.select_configurations() + def test_include_default_config(make_scenario, configspace_small): scenario = make_scenario(configspace_small, use_default_config=True) @@ -133,4 +134,3 @@ def test_include_default_config(make_scenario, configspace_small): # if use_default_config is True, then the default config should be included in the additional_configs default_config = scenario.configspace.get_default_configuration() assert default_config in dc._additional_configs -