Skip to content

Latest commit

 

History

History
165 lines (106 loc) · 6.89 KB

README_DEV.rst

File metadata and controls

165 lines (106 loc) · 6.89 KB

A Modern Python Dev Setup Demo

Setup, Testing, and Closeout

To setup your local development environment, there are a couple steps. The intention here is to simultaneously enable high quality submissions while getting more 'time behind the wheel' with a full suite of modern python development tools.

Summary introductions to modern Python dev environments are available at this blog post and this blog post.

It's recommended to uninstall any existing system-wide Python3 interpreter before proceeding. Then, add tools to your system in the following order. Details about pyenv installation of ~/.profile and ~/.bashrc are posted here.

  1. pyenv.
    • Ensure your computer may support pyenv by following the directions here.
    • At a minimum, restart your terminal session: exec $SHELL.
    • It's possible/likely a full restart may be required.
    • pyenv --version to test install succeeded.
    • Install Python versions using the install command: pyenv install 3.8.5.
    • [optional] Set your preffered version as the default: pyenv global 3.8.5.
    • [optional] Global default may be checked with which python.
  2. Poetry
    • verify with poetry --version
    • poetry new <project_name> auto-creates a distributable boilerplate in folder <project_name>.
    • Inside <project_name>, pyenv local 3.8.5 sets a specific pyenv version for the project.
    • poetry env use python creates a new virtual environment for the package/project.
    • poetry install adds any dependencies from the config files and poetry itself.
    • poetry update updates any packages to their latest compatible version(s).
    • poetry shell moves the terminal to execute within the context of the virtual environment.
    • poetry run pytest automatically finds testsand executes tests, producing a report. [NOTE: lost of plugins
    • poetry show -v will printout the virtual environment path. Helpful when configuring IDEs.
    • poetry config virtualenvs.in-project true will host virtual environments inside the project root. This can make referencing the virtual environment easier when configuring IDEs or writing scripts. available to improve this output including HTML reports)
    • If poetry shell was used, simply calling pytest will also run tests.

the setup script listed below takes care of the rest of these installations

  1. pytest-cov
    • Several ways to install coverage but pytest-cov plugin is streamlined poetry add --dev pytest-cov.
    • pytest --cov=<project_name> tests/.
  2. pre-commit
    • poetry add pre-commit --dev.
    • Pre-commit allows automated tooling to automatically run when prepping a git commit.
  3. flake8
    • In IDE and CLI code quality assistant.
    • poetry add flake8 --dev.
  4. reorder-python-imports
    • More 'robust' alternative to isort.
    • poetry add reorder-python-imports --dev.
  5. add-trailing-comma
    • Ensure minimal diffs when making signature updates to functions and data structures.
    • poetry add add-trailing-comma --dev.
  6. setup-cfg-fmt
    • Ensure setup.cfg files are standardized.
    • poetry add setup-cfg-fmt --dev
  7. mypy
    • Maximize accurate use of python's (continuously expanded) typing system to minimize bugs.
    • poetry add mypy --dev
  8. black
    • Automatically apply linting best practices. Goodbye pedantic code reviews on style/format.
    • poetry add black --dev

Note

Poetry may have unmet dependencies listed with distutils and cleo. This may be avoided by not using pyenv to install a newer version of python than Ubuntu officially supports. Aside from using pyenv version to match the OS limitations, the following are additional fixes.

sudo apt-get install python3-distutils
sudo apt-get install python3-apt
source scripts/setup_environment.sh

The environment is correctly configured if the success message appears after running the scripts/1_after_pyenv_poetry_install-setup_env.sh script.

Testing your solution and applying the automated tooling can be done by running tox from the activated Pyenv + Poetry virtual environment.

tox

Tips

  1. exit instead of deactivate to have your shell exit the Poetry virtual environment.
    • If you can't use poetry shell to enter virtual environment because 'it already exists', try the following:
source "$( poetry env list --full-path | grep Activated | cut -d' ' -f1 )/bin/activate"
  1. To start a new project, try poetry new <project_name> and a decent default folder structure will be created.
  2. To add a reasonable pyproject.toml to an existing project: poetry init
  3. To manually activate a virtual environment:
    • pyenv versions to see which Python versions are installed.
    • poetry env use <python_version> to create a virtual environment with the preffered versions.
    • poetry shell to activate the new environment.
    • python -V in the activated virtual environment to verify the correct python version is being used.
    • poetry install and pre-commit install to ensure all dependencies and the pre-commit hook are added.

5. By default, Poetry creates virtual environments in the user profile cache. Likely, you'll want to have it created in the local project folder. To do this, set the poetry environment variable or add the poetry.toml file as shown in this repo.

  • poetry config virtualenvs.in-project true

6. If Sphinx or other tools are warning they can't find your module (dev_demo), ensure poetry show lists the module. If not, use poetry install to locally install in development mode. This is similar to pip install -e <module>.

TODO

  1. Better use of Sphinx autodoc to eliminate errors in Tox and improve auto-generated documentation.