-
-
Notifications
You must be signed in to change notification settings - Fork 144
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update readme to reflect updates (python by default, rename dirs)
- Loading branch information
Showing
1 changed file
with
94 additions
and
55 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,86 +1,125 @@ | ||
# OpenML Python Container | ||
|
||
This docker container has the latest development version of openml-python downloaded and pre-installed. | ||
It can be used to run the unit tests or build the docs in a fresh and/or isolated unix environment. | ||
Instructions only tested on a Windows host machine. | ||
This docker container has the latest version of openml-python downloaded and pre-installed. | ||
It can also be used by developers to run unit tests or build the docs in | ||
a fresh and/or isolated unix environment. | ||
This document contains information about: | ||
|
||
First pull the docker image: | ||
|
||
docker pull openml/openml-python | ||
1. [Usage](#usage) take you over | ||
2. [Using local or remote code](#using-local-or-remote-code) is useful when testing your own latest changes | ||
3. [Versions](#versions) help you identify which image to use. | ||
4. [Developer](#for-developers) information about the Docker image. | ||
|
||
## Usage | ||
|
||
There are three main ways to use the image: running a pre-installed Python environment, | ||
running tests, and building documentation. | ||
|
||
docker run -it openml/openml-python [DOC,TEST] [BRANCH] | ||
### Running `Python` with pre-installed `OpenML-Python` (default): | ||
|
||
The image is designed to work with two specified directories which may be mounted ([`docker --mount documentation`](https://docs.docker.com/storage/bind-mounts/#start-a-container-with-a-bind-mount)). | ||
You can mount your openml-python folder to the `/code` directory to run tests or build docs on your local files. | ||
You can mount an `/output` directory to which the container will write output (currently only used for docs). | ||
Each can be mounted by adding a `--mount type=bind,source=SOURCE,destination=/DESTINATION` where `SOURCE` is the absolute path to your code or output directory, and `DESTINATION` is either `code` or `output`. | ||
|
||
E.g. mounting a code directory: | ||
To run `Python` with a pre-installed `OpenML-Python` environment run: | ||
|
||
docker run -i --mount type=bind,source="E:\\repositories/openml-python",destination="/code" -t openml/openml-python | ||
```text | ||
docker run -it openml/openml-python | ||
``` | ||
|
||
E.g. mounting an output directory: | ||
this accepts the normal `Python` arguments, e.g.: | ||
|
||
docker run -i --mount type=bind,source="E:\\files/output",destination="/output" -t openml/openml-python | ||
```text | ||
docker run openml/openml-python -c "import openml; print(openml.__version__)" | ||
``` | ||
|
||
You can mount both at the same time. | ||
if you want to run a local script, it needs to be mounted first. Mount it into the | ||
`openml` folder: | ||
|
||
### Bash (default) | ||
By default bash is invoked, you should also use the `-i` flag when starting the container so it processes input: | ||
``` | ||
docker run -v PATH/TO/FILE:/openml/MY_SCRIPT.py openml/openml-python MY_SCRIPT.py | ||
``` | ||
|
||
docker run -it openml/openml-python | ||
### Running unit tests | ||
|
||
### Building Documentation | ||
There are two ways to build documentation, either directly from the `HEAD` of a branch on Github or from your local directory. | ||
You can run the unit tests by passing `test` as the first argument: | ||
|
||
#### Building from a local repository | ||
Building from a local directory requires you to mount it to the ``/code`` directory: | ||
```text | ||
docker run openml/openml-python test | ||
``` | ||
|
||
docker run --mount type=bind,source=PATH_TO_REPOSITORY,destination=/code -t openml/openml-python doc | ||
see [the section below]((#using-local-or-remote-code) for running against local changes | ||
or a remote branch. | ||
|
||
The produced documentation will be in your repository's ``doc/build`` folder. | ||
If an `/output` folder is mounted, the documentation will *also* be copied there. | ||
### Building documentation | ||
|
||
#### Building from an online repository | ||
Building from a remote repository requires you to specify a branch. | ||
The branch may be specified by name directly if it exists on the original repository (https://github.com/openml/openml-python/): | ||
You can build the documentation by passing `doc` as the first argument, | ||
you should [mount]((https://docs.docker.com/storage/bind-mounts/#start-a-container-with-a-bind-mount)) | ||
an output directory in which the docs will be stored, on Windows: | ||
|
||
docker run --mount type=bind,source=PATH_TO_OUTPUT,destination=/output -t openml/openml-python doc BRANCH | ||
```text | ||
docker run --mount type=bind,source="E:\\files/output",destination="/output" openml/openml-python doc | ||
``` | ||
|
||
Where `BRANCH` is the name of the branch for which to generate the documentation. | ||
It is also possible to build the documentation from the branch on a fork, in this case the `BRANCH` should be specified as `GITHUB_NAME#BRANCH` (e.g. `PGijsbers#my_feature`) and the name of the forked repository should be `openml-python`. | ||
on Linux: | ||
```text | ||
docker run --mount type=bind,source="./output",destination="/output" openml/openml-python doc | ||
``` | ||
|
||
see [the section below]((#using-local-or-remote-code) for running against local changes | ||
or a remote branch. | ||
|
||
### Running tests | ||
There are two ways to run tests, either directly from the `HEAD` of a branch on Github or from your local directory. | ||
It works similar to building docs, but should specify `test` as mode. | ||
For example, to run tests on your local repository: | ||
*Note: you can forgo mounting an output directory to test if the docs build successfully, | ||
but the result will only be available within the docker container under `/openml/docs/build`.* | ||
|
||
docker run --mount type=bind,source=PATH_TO_REPOSITORY,destination=/code -t openml/openml-python test | ||
|
||
Running tests from the state of an online repository is supported similar to building documentation (i.e. specify `BRANCH` instead of mounting `/code`). | ||
|
||
## Troubleshooting | ||
## Using local or remote code | ||
|
||
You can build docs or run tests against your local repository or a Github repository. | ||
In the examples below, change the `source` to match the location of your local repository. | ||
|
||
### Using a local repository | ||
|
||
To use a local directory, mount it in the `/code` directory, on Windows: | ||
|
||
When you are mounting a directory you can check that it is mounted correctly by running the image in bash mode. | ||
Navigate to the `/code` and `/output` directories and see if the expected files are there. | ||
If e.g. there is no code in your mounted `/code`, you should double-check the provided path to your host directory. | ||
```text | ||
docker run --mount type=bind,source="E:\\repositories/openml-python",destination="/code" openml/openml-python test | ||
``` | ||
|
||
## Notes for developers | ||
This section contains some notes about the structure of the image, intended for those who want to work on it. | ||
on Linux: | ||
```text | ||
docker run --mount type=bind,source="~/repositories/openml-python",destination="/code" openml/openml-python test | ||
``` | ||
|
||
when building docs, you also need to mount an output directory as shown above, so add both: | ||
|
||
```text | ||
docker run --mount type=bind,source="./output",destination="/output" --mount type=bind,source="~/repositories/openml-python",destination="/code" openml/openml-python doc | ||
``` | ||
|
||
### Using a Github repository | ||
Building from a remote repository requires you to specify a branch. | ||
The branch may be specified by name directly if it exists on the original repository (https://github.com/openml/openml-python/): | ||
|
||
docker run --mount type=bind,source=PATH_TO_OUTPUT,destination=/output openml/openml-python [test,doc] BRANCH | ||
|
||
Where `BRANCH` is the name of the branch for which to generate the documentation. | ||
It is also possible to build the documentation from the branch on a fork, | ||
in this case the `BRANCH` should be specified as `GITHUB_NAME#BRANCH` (e.g. | ||
`PGijsbers#my_feature_branch`) and the name of the forked repository should be `openml-python`. | ||
|
||
## For developers | ||
This section contains some notes about the structure of the image, | ||
intended for those who want to work on it. | ||
|
||
### Added Directories | ||
The `openml/openml-python` image is built on a vanilla `python:3` image. | ||
Additionally it contains the following files are directories: | ||
|
||
- `/omlp`: contains the openml-python repository in the state with which the image was built by default. | ||
If working with a `BRANCH`, this repository will be set to the `HEAD` of `BRANCH`. | ||
- `/omlp/venv/`: contains the used virtual environment for `doc` and `test`. It has `openml-python` dependencies pre-installed. | ||
When invoked with `doc` or `test`, the dependencies will be updated based on the `setup.py` of the `BRANCH` or mounted `/code`. | ||
Additionally, it contains the following files are directories: | ||
|
||
- `/openml`: contains the openml-python repository in the state with which the image | ||
was built by default. If working with a `BRANCH`, this repository will be set to | ||
the `HEAD` of `BRANCH`. | ||
- `/openml/venv/`: contains the used virtual environment for `doc` and `test`. It has | ||
`openml-python` dependencies pre-installed. When invoked with `doc` or `test`, the | ||
dependencies will be updated based on the `setup.py` of the `BRANCH` or mounted `/code`. | ||
- `/scripts/startup.sh`: the entrypoint of the image. Takes care of the automated features (e.g. `doc` and `test`). | ||
|
||
## Building the image | ||
To build the image yourself, execute `docker build -f Dockerfile .` from this directory. | ||
It will use the `startup.sh` as is, so any local changes will be present in the image. | ||
To build the image yourself, execute `docker build -f Dockerfile .` from the `docker` | ||
directory of the `openml-python` repository. It will use the `startup.sh` as is, so any | ||
local changes will be present in the image. |