diff --git a/.gitignore b/.gitignore index a21ac9e..fc5bb87 100644 --- a/.gitignore +++ b/.gitignore @@ -60,3 +60,7 @@ MANIFEST # Metaflow files .metaflow + +# Virtual Environment Directories +/venv +/py-orca-venv-* diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b1f8f3a..6a4685c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing -Welcome to `orca` contributor's guide. +Welcome to `py-orca` contributor's guide. This document focuses on getting any potential contributor familiarized with the development processes, but [other kinds of contributions] are also appreciated. @@ -16,7 +16,7 @@ behavior guidelines. ## Issue Reports -If you experience bugs or general issues with `orca`, please have a look +If you experience bugs or general issues with `py-orca`, please have a look on the [issue tracker]. If you don't see anything useful there, please feel free to fire an issue report. @@ -34,17 +34,17 @@ you help us to identify the root cause of the issue. ## Documentation Improvements -You can help improve `orca` docs by making them more readable and coherent, or +You can help improve py-`orca` docs by making them more readable and coherent, or by adding missing information and correcting mistakes. -`orca` documentation uses [Sphinx] as its main documentation compiler. +`py-orca` documentation uses [Sphinx] as its main documentation compiler. This means that the docs are kept in the same repository as the project code, and that any documentation update is done in the same way was a code contribution. The documentation is written using [CommonMark] with [MyST] extensions. :::{tip} Please notice that the [GitHub web interface] provides a quick way of - propose changes in `orca`'s files. While this mechanism can + propose changes in `py-orca`'s files. While this mechanism can be tricky for normal code contributions, it works perfectly fine for contributing to the docs, and can be quite handy. @@ -87,7 +87,7 @@ Before you work on any non-trivial code contribution it's best to first create a report in the [issue tracker] to start a discussion on the subject. This often provides additional considerations and avoids unnecessary work. -### Clone the repository +### Development Environment Setup 1. Create an user account on GitHub if you do not already have one. @@ -98,22 +98,25 @@ This often provides additional considerations and avoids unnecessary work. ```console git clone git@github.com:Sage-Bionetworks-Workflows/py-orca.git - cd orca + cd py-orca ``` -4. Install `pipx` to easily run Python CLI tools like `tox` and `pipenv`. - 4. Create an isolated virtual environment containing package dependencies, including those needed for development (*e.g.* testing, documentation) by running: ```console - pipx run tox -e pipenv + ./dev_setup.sh + source py-orca-venv-/bin/activate ``` + **Note:** + Ensure that you have a supported Python version installed on your local machine before running `dev_setup.sh`. `py-orca` currently supports Python version 3.10 and 3.11. You can [install](https://github.com/pyenv/pyenv?tab=readme-ov-file#installation) `pyenv`for easy Python version management. + + 5. Install [pre-commit] hooks: ``` - pipenv run pre-commit install + pre-commit install ``` `orca` comes with a lot of hooks configured to automatically help the diff --git a/README.md b/README.md index d4c6648..f7e4282 100644 --- a/README.md +++ b/README.md @@ -24,21 +24,6 @@ Refer to [`.env.example`](.env.example) for the format of their values as well a Once your environment is set, you can create a virtual environment, install the Python dependencies, and run the demonstration script (after downloading it) as follows. Note that you will need to update the `s3_prefix` parameter so that it points to an S3 bucket that is accessible to your Tower workspace. -### Using pipenv -```bash -# Create and activate a Python virtual environment (tested with Python 3.11) -pipenv shell - -# Install Python dependencies -pipenv install - -# If development dependencies are required to be installed -pipenv install -d - -# Run the script using an example dataset -python3 demo.py run --dataset_id 'syn51514585' --s3_prefix 's3://orca-service-test-project-tower-bucket/outputs' -``` - ### Manually creating a virtual environment ```bash # Create and activate a Python virtual environment (tested with Python 3.10) diff --git a/dev_setup.sh b/dev_setup.sh new file mode 100755 index 0000000..8660e97 --- /dev/null +++ b/dev_setup.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +# Usage: ./dev_setup.sh + +# Check if Python version argument is provided +if [ -z "$1" ]; then + echo "Please provide the Python version as an argument." + exit 1 +fi +# Check if Python version is supported +if [ "$1" != "3.10" ] && [ "$1" != "3.11" ]; then + echo "Unsupported Python version. Please use 3.10 or 3.11." + exit 1 +fi +# Set up and activate a Python 3.11 virtual environment +python$1 -m venv py-orca-venv-$1 +source py-orca-venv-$1/bin/activate +# Upgrade pip to latest version +pip install --upgrade pip +# Install airflow with constraints +pip install -r requirements-airflow.txt +# Install py-orca +pip install -e '.[all,testing,dev]' diff --git a/requirements-airflow.txt b/requirements-airflow.txt new file mode 100644 index 0000000..adefbcd --- /dev/null +++ b/requirements-airflow.txt @@ -0,0 +1,2 @@ +-c https://raw.githubusercontent.com/apache/airflow/constraints-2.7.2/constraints-3.10.txt +apache-airflow==2.7.2 diff --git a/tox.ini b/tox.ini index 0fa74ae..8102733 100644 --- a/tox.ini +++ b/tox.ini @@ -19,6 +19,8 @@ passenv = HOME *_CONNECTION_URI SETUPTOOLS_* +deps = + -r {toxinidir}/requirements-airflow.txt extras = testing all @@ -92,16 +94,3 @@ deps = twine commands = python -m twine check dist/* python -m twine upload {posargs:--repository {env:TWINE_REPOSITORY:testpypi}} dist/* - - -[testenv:pipenv] -description = - Refresh the Pipfile.lock file based on the latest dependencies in setup.cfg -skip_install = True -changedir = {toxinidir} -setenv = - PIPENV_IGNORE_VIRTUALENVS = 1 -deps = pipenv -commands = - pipenv lock --dev - pipenv install --dev