Skip to content

Commit

Permalink
Merge branch 'develop' into feature_1463_pandas_future_warnings
Browse files Browse the repository at this point in the history
  • Loading branch information
@bug-or-feature
bug-or-feature committed Jan 13, 2025
2 parents f519e80 + d1cd6ca commit 6faaffd
Show file tree
Hide file tree
Showing 16 changed files with 227 additions and 200 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/os-test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,5 @@ jobs:

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
pip install -r requirements.txt
python -m pip install --upgrade pip setuptools
python -m pip install .
5 changes: 2 additions & 3 deletions .github/workflows/quick-test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,8 @@ jobs:

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
pip install pytest
pip install -r requirements.txt
python -m pip install --upgrade pip setuptools
python -m pip install '.[dev]'
- name: Test with pytest
run: |
Expand Down
5 changes: 2 additions & 3 deletions .github/workflows/slow-test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,8 @@ jobs:

- name: Install dependencies
run: |
python -m pip install pip setuptools wheel
pip install pytest
pip install -r requirements.txt
python -m pip install --upgrade pip setuptools
python -m pip install '.[dev]'
- name: Test with pytest
run: |
Expand Down
31 changes: 13 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,31 +57,26 @@ My live production system is always on the latest release. I make, and eat, my o

## Dependencies

See [requirements.txt](requirements.txt) for full details.
See the `project.dependencies` section in [pyproject.toml](pyproject.toml) for full details.

Make sure you get the python3 versions of the relevant packages, i.e. use: (in linux)

```
sudo pip3 install ....
```

(Your machine may not need pip3 if python3 is the default python on your machine)

## Installation

This package isn't hosted on pip. So to get the code the easiest way is to use git:
This package isn't hosted on pypi.org. So to get the code the easiest way is to use git:

```
git clone https://github.com/robcarver17/pysystemtrade.git
#
# then one of:
#
python3 setup.py install # normal
python3 setup.py develop # required if using ipython so sessions can see files inside subdirectories which would otherwise be inaccessible.
python3 setup.py install --user # avoids use of sudo
```
# clone the repo to your local filesystem
$ git clone https://github.com/robcarver17/pysystemtrade.git
# navigate to the project directory
$ cd pysystemtrade
Again, you may be okay with python rather than python3.
# either install pysystemtrade normally
$ python -m pip install .
# or install in editable mode, with development dependencies
$ python -m pip install --editable '.[dev]'
```

There is a more complete installation guide [here](docs/installation.md)

Expand Down
232 changes: 115 additions & 117 deletions docs/installation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,37 @@

## Introduction

This guide shows the quickest and easiest way to install the project in a virtual environment
This guide shows a couple of ways to install the project in a virtual environment.

## project files

Install the project files by cloning the GitHub repository. If you intend to contribute to the project, or run your own instance, you will likely want to clone your own fork

```
git clone https://github.com/<your_git_hub_id>/pysystemtrade.git
```

otherwise, you'll want the main repo

```
git clone https://github.com/robcarver17/pysystemtrade.git
```


## pyenv
## Option 1: pyenv + venv

pyenv allows easy installation of multiple versions of Python on the same machine. It allows the version of python used to be defined at the user and project level. It is a great tool, easy to use, and does its one job very well. It is worth reading the introduction to have an overview of how it works at a high level. It's not necessary to understand the technical internals
### pyenv

pyenv is a tool that makes it easy to manage multiple versions of Python on the same machine. It allows the version of python used to be defined at the user and project level. It is a great tool, easy to use, and does its one job very well. It is worth reading the introduction to have an overview of how it works at a high level. It's not necessary to understand the technical internals

https://github.com/pyenv/pyenv#how-it-works

Installation instructions for pyenv are here:

https://github.com/pyenv/pyenv#installation

## Python 3.10

pysystemtrade currently requires Python 3.10, so once pyenv is installed, the first step is to get that. Get the latest 3.10.x version, at the time of writing it is 3.10.13
First install Python itself. pysystemtrade currently requires Python 3.10 or newer

```
$ pyenv install 3.10
Expand All @@ -31,170 +46,153 @@ $ pyenv versions
3.7.14
3.8.5
3.8.6
3.8.10
3.8.13
3.8.15
3.8.16
3.9.6
3.9.13
3.10.4
* 3.10.13
* 3.10.15
```

Your output will be different, it's just an example

### venv

## project files
https://docs.python.org/3.10/library/venv.html

Once we have the correct version of Python, it's time to get the project files.
Now we want to create a virtual environment (venv) for the project. Doing this will keep all the dependencies for pysystemtrade separate from your other python projects

If you intend to contribute to the project, or run your own instance, you will likely want to clone your own fork
```
$ cd pysystemtrade
$ python -m venv .venv
```

This will create a brand new, isolated Python environment *inside the pysystemtrade project* at the directory
`<your_path>/pysystemtrade/.venv`. You can give your environment any name (the *.venv* bit).

Now activate the virtual environment

```
git clone https://github.com/<your_git_hub_id>/pysystemtrade.git
source .venv/bin/activate
```

otherwise, you'll want the main repo
Once your virtual env is activated, the prompt will change. It will look something like

```
git clone https://github.com/robcarver17/pysystemtrade.git
(.venv) $
```
This reminds you that your venv is active. You can exit the venv at any time by running `deactivate`

Now we will want to let pyenv know that we want to use Python 3.10 for this project

```
cd pysystemtrade
pyenv local 3.10.13
pyenv local 3.10.15
```

this creates a file at the top level of the project `.python-version` that lets the Python execution environment know to use version 3.10.13. We can check this by running python
this creates a file at the top level of the project `.python-version` that lets the Python execution environment know to use version 3.10.15. We can check this by running python

```
$ python
Python 3.10.13 (main, Nov 27 2023, 11:13:49) [Clang 14.0.0 (clang-1400.0.29.202)]
Python 3.10.15 (main, Nov 27 2023, 11:13:49) [Clang 14.0.0 (clang-1400.0.29.202)]
Type "help", "copyright", "credits" or "license" for more information.
>>>
< ctrl-D to exit >
```

## venv
### dependencies

https://docs.python.org/3.10/library/venv.html

Now we want to create a virtual env (venv) for the project. Doing this will keep all the dependencies for pysystemtrade separate from your other python projects
Now it's time to start setting up the venv. First update the basic tools

```
$ python -m venv venv/3.10.13
(.venv) $ pip install --upgrade pip setuptools
```

This will create a brand new, isolated Python environment *inside the pysystemtrade project* at the directory
`<your_path>/pysystemtrade/venv/3.10.13`. You can give your environment any name (the *venv/3.10.13* bit).

Now activate the virtual environment
And now install the project and its dependencies

```
source venv/3.10.13/bin/activate
(.venv) $ python -m pip install .
```

Once your virtual env is activated, the prompt will change. It will look something like
Or, if you intend to contribute to the project, you will need the optional development dependencies too, and will want to install in *editable* mode

```
(3.10.13) $
(.venv) $ python -m pip install --editable '.[dev]'
```
This reminds you that you're in a venv. (You can exit the venv at any time by running `deactivate`)


## dependencies

Now it's time to start setting up the venv. First check to see what is there

```
(3.10.13) $ pip list
```

You will probably be prompted to update pip at this time. Do whatever command it suggests.

And now install the dependencies

```
(3.10.13) $ pip install -r requirements.txt
```

### MacOS (ARM)

If you're running MacOS on one of the new ARM chips, the process is more complex. You'll need Homebrew and the Apple XCode Commandline Development Tools, configured for ARM. Doing that is beyond the scope of this document, type `homebrew apple xcode command line tools` into your favourite search engine. Once installed and configured, run installation script:

```
chmod u+x install_dependencies_apple_silicon.sh
./install_dependencies_apple_silicon.sh
```
Note: this may (unfortunately) become out of date and require some tweaking.

### Check dependencies

Check what is installed, should look something like

```
(3.10.13) % pip list
Package Version
--------------- ------------
blinker 1.7.0
click 8.1.7
contourpy 1.2.0
cycler 0.12.1
eventkit 1.0.3
exceptiongroup 1.2.0
Flask 3.0.1
fonttools 4.47.2
ib-insync 0.9.86
iniconfig 2.0.0
itsdangerous 2.1.2
Jinja2 3.1.3
joblib 1.3.2
kiwisolver 1.4.5
MarkupSafe 2.1.4
matplotlib 3.8.2
nest-asyncio 1.6.0
numpy 1.26.3
packaging 23.2
pandas 2.1.3
patsy 0.5.6
pillow 10.2.0
pip 23.3.2
pluggy 1.3.0
psutil 5.6.6
pyarrow 15.0.0
pymongo 3.11.3
pyparsing 3.1.1
PyPDF2 3.0.1
pytest 7.4.4
python-dateutil 2.8.2
pytz 2023.3.post1
PyYAML 5.3.1
scikit-learn 1.4.0
scipy 1.12.0
setuptools 65.5.0
six 1.16.0
statsmodels 0.14.0
threadpoolctl 3.2.0
tomli 2.0.1
tzdata 2023.4
Werkzeug 3.0.1
```

## pysystemtrade

And finally, install the project itself

```
(3.10.13) $ python setup.py develop
```
(.venv) % pip list
Package Version
----------------- -----------
black 23.11.0
blinker 1.8.2
click 8.1.7
contourpy 1.3.0
cycler 0.12.1
decorator 5.1.1
enum-compat 0.0.3
eventkit 1.0.3
exceptiongroup 1.2.2
Flask 3.0.3
fonttools 4.54.1
ib-insync 0.9.86
iniconfig 2.0.0
itsdangerous 2.2.0
Jinja2 3.1.4
joblib 1.4.2
kiwisolver 1.4.7
lz4 4.3.3
MarkupSafe 2.1.5
matplotlib 3.9.2
mock 5.1.0
mockextras 1.0.2
mypy-extensions 1.0.0
nest-asyncio 1.6.0
numpy 1.26.4
packaging 24.1
pandas 2.1.3
pathspec 0.12.1
patsy 0.5.6
pillow 10.4.0
pip 24.3.1
platformdirs 4.3.6
pluggy 1.5.0
psutil 5.6.7
pyarrow 17.0.0
pymongo 3.11.3
pyparsing 3.1.4
PyPDF2 3.0.1
pysystemtrade 1.8.2
pytest 8.3.3
python-dateutil 2.9.0.post0
pytz 2023.3
PyYAML 6.0.1
scikit-learn 1.5.2
scipy 1.14.1
setuptools 65.5.0
six 1.16.0
statsmodels 0.14.0
threadpoolctl 3.5.0
tomli 2.0.1
typing_extensions 4.12.2
tzdata 2024.2
tzlocal 5.2
Werkzeug 3.0.4
```

## Option 2: uv

[uv](https://github.com/astral-sh/uv) is packaging tool that combines the functionality of pip, pyenv, venv and more; read about it [here](https://docs.astral.sh/uv/). It is written in Rust and **extremely** fast

TBD


## test

Check stuff works

```
(3.10.13) $ python
(.venv) $ python
>>>
>>> from sysdata.sim.csv_futures_sim_data import csvFuturesSimData
Configuring sim logging
Expand Down
Loading

0 comments on commit 6faaffd

Please sign in to comment.