Skip to content

Commit

Permalink
Update readme to reflect updates (python by default, rename dirs)
Browse files Browse the repository at this point in the history
  • Loading branch information
PGijsbers committed Jul 27, 2023
1 parent 7709348 commit 7412f05
Showing 1 changed file with 94 additions and 55 deletions.
149 changes: 94 additions & 55 deletions docker/readme.md
Original file line number Diff line number Diff line change
@@ -1,86 +1,125 @@
# OpenML Python Container

This docker container has the latest development version of openml-python downloaded and pre-installed.
It can be used to run the unit tests or build the docs in a fresh and/or isolated unix environment.
Instructions only tested on a Windows host machine.
This docker container has the latest version of openml-python downloaded and pre-installed.
It can also be used by developers to run unit tests or build the docs in
a fresh and/or isolated unix environment.
This document contains information about:

First pull the docker image:

docker pull openml/openml-python
1. [Usage](#usage) take you over
2. [Using local or remote code](#using-local-or-remote-code) is useful when testing your own latest changes
3. [Versions](#versions) help you identify which image to use.
4. [Developer](#for-developers) information about the Docker image.

## Usage

There are three main ways to use the image: running a pre-installed Python environment,
running tests, and building documentation.

docker run -it openml/openml-python [DOC,TEST] [BRANCH]
### Running `Python` with pre-installed `OpenML-Python` (default):

The image is designed to work with two specified directories which may be mounted ([`docker --mount documentation`](https://docs.docker.com/storage/bind-mounts/#start-a-container-with-a-bind-mount)).
You can mount your openml-python folder to the `/code` directory to run tests or build docs on your local files.
You can mount an `/output` directory to which the container will write output (currently only used for docs).
Each can be mounted by adding a `--mount type=bind,source=SOURCE,destination=/DESTINATION` where `SOURCE` is the absolute path to your code or output directory, and `DESTINATION` is either `code` or `output`.

E.g. mounting a code directory:
To run `Python` with a pre-installed `OpenML-Python` environment run:

docker run -i --mount type=bind,source="E:\\repositories/openml-python",destination="/code" -t openml/openml-python
```text
docker run -it openml/openml-python
```

E.g. mounting an output directory:
this accepts the normal `Python` arguments, e.g.:

docker run -i --mount type=bind,source="E:\\files/output",destination="/output" -t openml/openml-python
```text
docker run openml/openml-python -c "import openml; print(openml.__version__)"
```

You can mount both at the same time.
if you want to run a local script, it needs to be mounted first. Mount it into the
`openml` folder:

### Bash (default)
By default bash is invoked, you should also use the `-i` flag when starting the container so it processes input:
```
docker run -v PATH/TO/FILE:/openml/MY_SCRIPT.py openml/openml-python MY_SCRIPT.py
```

docker run -it openml/openml-python
### Running unit tests

### Building Documentation
There are two ways to build documentation, either directly from the `HEAD` of a branch on Github or from your local directory.
You can run the unit tests by passing `test` as the first argument:

#### Building from a local repository
Building from a local directory requires you to mount it to the ``/code`` directory:
```text
docker run openml/openml-python test
```

docker run --mount type=bind,source=PATH_TO_REPOSITORY,destination=/code -t openml/openml-python doc
see [the section below]((#using-local-or-remote-code) for running against local changes
or a remote branch.

The produced documentation will be in your repository's ``doc/build`` folder.
If an `/output` folder is mounted, the documentation will *also* be copied there.
### Building documentation

#### Building from an online repository
Building from a remote repository requires you to specify a branch.
The branch may be specified by name directly if it exists on the original repository (https://github.com/openml/openml-python/):
You can build the documentation by passing `doc` as the first argument,
you should [mount]((https://docs.docker.com/storage/bind-mounts/#start-a-container-with-a-bind-mount))
an output directory in which the docs will be stored, on Windows:

docker run --mount type=bind,source=PATH_TO_OUTPUT,destination=/output -t openml/openml-python doc BRANCH
```text
docker run --mount type=bind,source="E:\\files/output",destination="/output" openml/openml-python doc
```

Where `BRANCH` is the name of the branch for which to generate the documentation.
It is also possible to build the documentation from the branch on a fork, in this case the `BRANCH` should be specified as `GITHUB_NAME#BRANCH` (e.g. `PGijsbers#my_feature`) and the name of the forked repository should be `openml-python`.
on Linux:
```text
docker run --mount type=bind,source="./output",destination="/output" openml/openml-python doc
```

see [the section below]((#using-local-or-remote-code) for running against local changes
or a remote branch.

### Running tests
There are two ways to run tests, either directly from the `HEAD` of a branch on Github or from your local directory.
It works similar to building docs, but should specify `test` as mode.
For example, to run tests on your local repository:
*Note: you can forgo mounting an output directory to test if the docs build successfully,
but the result will only be available within the docker container under `/openml/docs/build`.*

docker run --mount type=bind,source=PATH_TO_REPOSITORY,destination=/code -t openml/openml-python test

Running tests from the state of an online repository is supported similar to building documentation (i.e. specify `BRANCH` instead of mounting `/code`).

## Troubleshooting
## Using local or remote code

You can build docs or run tests against your local repository or a Github repository.
In the examples below, change the `source` to match the location of your local repository.

### Using a local repository

To use a local directory, mount it in the `/code` directory, on Windows:

When you are mounting a directory you can check that it is mounted correctly by running the image in bash mode.
Navigate to the `/code` and `/output` directories and see if the expected files are there.
If e.g. there is no code in your mounted `/code`, you should double-check the provided path to your host directory.
```text
docker run --mount type=bind,source="E:\\repositories/openml-python",destination="/code" openml/openml-python test
```

## Notes for developers
This section contains some notes about the structure of the image, intended for those who want to work on it.
on Linux:
```text
docker run --mount type=bind,source="~/repositories/openml-python",destination="/code" openml/openml-python test
```

when building docs, you also need to mount an output directory as shown above, so add both:

```text
docker run --mount type=bind,source="./output",destination="/output" --mount type=bind,source="~/repositories/openml-python",destination="/code" openml/openml-python doc
```

### Using a Github repository
Building from a remote repository requires you to specify a branch.
The branch may be specified by name directly if it exists on the original repository (https://github.com/openml/openml-python/):

docker run --mount type=bind,source=PATH_TO_OUTPUT,destination=/output openml/openml-python [test,doc] BRANCH

Where `BRANCH` is the name of the branch for which to generate the documentation.
It is also possible to build the documentation from the branch on a fork,
in this case the `BRANCH` should be specified as `GITHUB_NAME#BRANCH` (e.g.
`PGijsbers#my_feature_branch`) and the name of the forked repository should be `openml-python`.

## For developers
This section contains some notes about the structure of the image,
intended for those who want to work on it.

### Added Directories
The `openml/openml-python` image is built on a vanilla `python:3` image.
Additionally it contains the following files are directories:

- `/omlp`: contains the openml-python repository in the state with which the image was built by default.
If working with a `BRANCH`, this repository will be set to the `HEAD` of `BRANCH`.
- `/omlp/venv/`: contains the used virtual environment for `doc` and `test`. It has `openml-python` dependencies pre-installed.
When invoked with `doc` or `test`, the dependencies will be updated based on the `setup.py` of the `BRANCH` or mounted `/code`.
Additionally, it contains the following files are directories:

- `/openml`: contains the openml-python repository in the state with which the image
was built by default. If working with a `BRANCH`, this repository will be set to
the `HEAD` of `BRANCH`.
- `/openml/venv/`: contains the used virtual environment for `doc` and `test`. It has
`openml-python` dependencies pre-installed. When invoked with `doc` or `test`, the
dependencies will be updated based on the `setup.py` of the `BRANCH` or mounted `/code`.
- `/scripts/startup.sh`: the entrypoint of the image. Takes care of the automated features (e.g. `doc` and `test`).

## Building the image
To build the image yourself, execute `docker build -f Dockerfile .` from this directory.
It will use the `startup.sh` as is, so any local changes will be present in the image.
To build the image yourself, execute `docker build -f Dockerfile .` from the `docker`
directory of the `openml-python` repository. It will use the `startup.sh` as is, so any
local changes will be present in the image.

0 comments on commit 7412f05

Please sign in to comment.