-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Initial paper version * Adjust SMT reference * Fix affiliation * Fix DOIs * Review statement of need * Add architecture figure * Fix typo * Fix examples doc section * Absolute links for sub-packages * Re-use README as crate doc * Absolute link for tutorial
- Loading branch information
Showing
7 changed files
with
201 additions
and
41 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -23,4 +23,8 @@ python/egobox/egobox.pyd | |
| *.npy | ||
| input.txt | ||
| output.txt | ||
| mopta08* | ||
| mopta08* | ||
|
|
||
| # JOSS | ||
| joss/paper.jats | ||
| joss/paper.pdf | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| @article{Jones1998, | ||
| title={Efficient global optimization of expensive black-box functions}, | ||
| author={Jones, Donald R and Schonlau, Matthias and Welch, William J}, | ||
| journal={Journal of Global optimization}, | ||
| volume={13}, | ||
| number={4}, | ||
| pages={455--492}, | ||
| year={1998}, | ||
| publisher={Springer} | ||
| } | ||
|
|
||
| @article{Bouhlel2016, | ||
| title={Improving kriging surrogates of high-dimensional design models by Partial Least Squares dimension reduction}, | ||
| author={Bouhlel, Mohamed Amine and Bartoli, Nathalie and Otsmane, Abdelkader and Morlier, Joseph}, | ||
| journal={Structural and Multidisciplinary Optimization}, | ||
| volume={53}, | ||
| number={5}, | ||
| pages={935--952}, | ||
| doi={10.1007/s00158-015-1395-9}, | ||
| year={2016}, | ||
| publisher={Springer}, | ||
| } | ||
|
|
||
| @inproceedings{Bartoli2016, | ||
| title={Improvement of efficient global optimization with application to aircraft wing design}, | ||
| author={Bartoli, Nathalie and Bouhlel, Mohamed-Amine and Kurek, Igor and Lafage, R{\'e}mi and Lefebvre, Thierry and Morlier, Joseph and Priem, R{\'e}my and Stilz, Vivien and Regis, Rommel}, | ||
| booktitle={17th AIAA/ISSMO Multidisciplinary analysis and optimization conference}, | ||
| pages={4001}, | ||
| doi={10.2514/6.2016-4001}, | ||
| year={2016}, | ||
| } | ||
|
|
||
| @article{Bartoli2019, | ||
| title = {Adaptive modeling strategy for constrained global optimization with application to aerodynamic wing design}, | ||
| journal = {Aerospace Science and Technology}, | ||
| volume = {90}, | ||
| pages = {85-102}, | ||
| doi = {10.1016/j.ast.2019.03.041}, | ||
| year = {2019}, | ||
| issn = {1270-9638}, | ||
| url = {https://www.sciencedirect.com/science/article/pii/S1270963818306011}, | ||
| author = {N. Bartoli and T. Lefebvre and S. Dubreuil and R. Olivanti and R. Priem and N. Bons and J.R.R.A. Martins and J. Morlier}, | ||
| keywords = {Surrogate modeling, Global optimization, Multimodal optimization, Mixture of experts, Aerodynamic shape optimization, Wing design}, | ||
| abstract = {Surrogate models are often used to reduce the cost of design optimization problems that involve computationally costly models, such as computational fluid dynamics simulations. However, the number of evaluations required by surrogate models usually scales poorly with the number of design variables, and there is a need for both better constraint formulations and multimodal function handling. To address this issue, we developed a surrogate-based gradient-free optimization algorithm that can handle cases where the function evaluations are expensive, the computational budget is limited, the functions are multimodal, and the optimization problem includes nonlinear equality or inequality constraints. The proposed algorithm—super efficient global optimization coupled with mixture of experts (SEGOMOE)—can tackle complex constrained design optimization problems through the use of an enrichment strategy based on a mixture of experts coupled with adaptive surrogate models. The performance of this approach was evaluated for analytic constrained and unconstrained problems, as well as for a multimodal aerodynamic shape optimization problem with 17 design variables and an equality constraint. Our results showed that the method is efficient and that the optimum is much less dependent on the starting point than the conventional gradient-based optimization.} | ||
| } | ||
|
|
||
| @article{Dubreuil2020, | ||
| title={Towards an efficient global multidisciplinary design optimization algorithm}, | ||
| author={Dubreuil, Sylvain and Bartoli, Nathalie and Gogu, Christian and Lefebvre, Thierry}, | ||
| journal={Structural and Multidisciplinary Optimization}, | ||
| volume={62}, | ||
| number={4}, | ||
| pages={1739--1765}, | ||
| doi={10.1007/s00158-020-02514-6}, | ||
| year={2020}, | ||
| publisher={Springer} | ||
| } | ||
|
|
||
| @article{SMT2019, | ||
| author = {Mohamed Amine Bouhlel and John T. Hwang and Nathalie Bartoli and Rémi Lafage and Joseph Morlier and Joaquim R. R. A. Martins}, | ||
| journal = {Advances in Engineering Software}, | ||
| title = {A Python surrogate modeling framework with derivatives}, | ||
| pages = {102662}, | ||
| year = {2019}, | ||
| issn = {0965-9978}, | ||
| doi = {10.1016/j.advengsoft.2019.03.005}, | ||
| year = {2019} | ||
| } | ||
|
|
||
| @misc{SMT2018, | ||
| author = SMTOrg, | ||
| title = {Surrogate Modeling Toolbox}, | ||
| year = {2018}, | ||
| publisher = {GitHub}, | ||
| journal = {GitHub repository}, | ||
| url = {https://github.com/SMTOrg/smt} | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| --- | ||
| title: 'egobox, a Rust toolbox for efficient global optimization' | ||
| tags: | ||
| - Rust | ||
| - design of experiments | ||
| - gaussian process | ||
| - mixture of experts | ||
| - surrogate-based optimization | ||
| authors: | ||
| - name: Rémi Lafage | ||
| orcid: 0000-0001-5479-2961 | ||
| affiliation: 1 | ||
| affiliations: | ||
| - name: ONERA, Université de Toulouse, France | ||
| index: 1 | ||
| date: "29 March 2022" | ||
| bibliography: paper.bib | ||
| --- | ||
|
|
||
| # Summary | ||
|
|
||
| Efficient global optimization (EGO) algorithms are used to tackle design optimization problems | ||
| that involve computationally costly models [@Jones1998]. Basically, the idea is to use gaussian | ||
| process regression to approximate the objective function and use probabilistic information | ||
| to select the promising next point where the optimum could be located. | ||
|
|
||
| Starting from that, one still has to tackle numerous challenges regarding the dimensionality, | ||
| multi-modality and computation time. Different algorithms may be develop to overcome | ||
| these hurdles [@Bartoli2019; @Dubreuil2020]. A key component of such surrogate-based algorithms | ||
| is the existence of an implementation of gaussian process regression also known as | ||
| kriging method [@Bouhlel2016]. | ||
|
|
||
| The Surrogate Modeling Toolbox [@SMT2018; @SMT2019] library addresses such concerns using | ||
| the Python programming language. The `egobox` library provides such key component to the Rust | ||
| machine-learning community while focusing on adaptative global optimization. | ||
|
|
||
| # Statement of need | ||
|
|
||
| Started by porting relevant parts from the SMT library in Rust, the `egobox` library aims | ||
| at providing building blocks useful to implement EGO-like algorithms. This open source library | ||
| will be used by research engineers needing to tackle design optimization problems using a | ||
| surrogate-based adaptative approach while taking advantage of the Rust programming | ||
| language. | ||
|
|
||
| Indeed research scientists reach for prototyping programming language such as Python to develop | ||
| new methods. Thanks to performant easy-to-use libraries like `numpy`, `scipy`, `scikit-learn` | ||
| the Python language has become popular in science computing. The ability of Python | ||
| to glue together different codes explains Python ecosystem is now a de-facto open source | ||
| standard for scientific software. Nevertheless, one might notice that the performances | ||
| of above Python libraries relies extensively on C/Fortran compiled code. | ||
|
|
||
| Thus, in the long run, a language like Python well-suited for prototyping may not be | ||
| suitable for maintainability or performance even with a strong software development discipline. | ||
| Performance issues may arise as new algorithms are built on top of the others | ||
| which at some point may not be tractable with such interpreted programming language. | ||
| As stated above, it is common to resort to compiled languages like C/C++ or Fortran | ||
| to implement computation-intensive methods or for embeding them in more constrained | ||
| environments where a Python runtime does not fit. | ||
|
|
||
| # Library features | ||
|
|
||
| With the above need in mind, the Rust programming language appears to be of interest | ||
| with its selling points being performance, reliability and productivity. | ||
| The language is meant to challenge C as a system language but also rely on strong typing, | ||
| and high level features such as functional programming, algebric data types, module | ||
| management. It has a strict approach regarding memory-safety management and | ||
| benefits from a state of the art tooling for software development. | ||
|
|
||
| The Rust community has developed scientific libraries like `ndarray`, `ndarray-linalg` | ||
| which can be seen as the `numpy` and `scipy` Rust counterparts. Last but not least, | ||
| the `linfa` project addresses the machine-learning domain with the purpose of being | ||
| the Rust equivalent of `scikit-learn`. | ||
|
|
||
| The `egobox` library relies on the above thriving Rust machine learning ecosystem and | ||
| focuses on providing some building blocks to implement efficient global optimization | ||
| algorithms. The library is organized in four sub-packages as follows: | ||
|
|
||
| * `doe`: sampling methods implementing Latin Hypercube sampling, popular sampling | ||
| method used to create design of experiments, | ||
|
|
||
| * `gp`: gaussian process regression also known as kriging algorithm used | ||
| as surrogate models for computationally costly black-box functions, | ||
|
|
||
| * `moe`: mixture of experts which aims at increasing the accuracy of a surrogate | ||
| approximation by clustering the design space, training and selecting the best | ||
| surrogate models on each cluster, | ||
|
|
||
| * `ego`: an efficient global optimization implementation with handling of inequality | ||
| constraints and mixed integer optimization through continuous relaxation. | ||
|
|
||
|  | ||
|
|
||
| Finally thanks to the `PyO3` project, the Rust language is well-suited to create | ||
| Python extensions which benefits from Rust strenghts while being integrated in the | ||
| Python ecosystem. | ||
|
|
||
| In order to increase the dissemination among the scientific community and demonstrate | ||
| actual optimization capabilities based on the library, we implemented a Python module `egobox` | ||
| as the binding of the implemented EGO-like Rust optimizer, namely `Egor`. | ||
|
|
||
| # Acknowledgements | ||
|
|
||
| I would like to thank my colleagues Nathalie Bartoli, Thierry Lefebvre and Sylvain Dubreuil | ||
| as their work on surrogate-based adaptative optimization has fueled this software development. | ||
|
|
||
| # References |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters