Skip to content

Commit

Permalink
Merge branch 'main' into 756-improve-docs
Browse files Browse the repository at this point in the history
Signed-off-by: Nicolas Höning <[email protected]>
  • Loading branch information
nhoening authored Jul 27, 2023
2 parents 0b47457 + c146ba6 commit 4f6e120
Show file tree
Hide file tree
Showing 28 changed files with 767 additions and 259 deletions.
33 changes: 16 additions & 17 deletions .github/workflows/lint-and-test.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
name: lint-and-test

on: ["push", "pull_request"]

on:
push:
pull_request:
types:
- opened
jobs:
check:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -35,7 +38,8 @@ jobs:
run: |
git fetch --prune --unshallow
git fetch --depth=1 origin +refs/tags/*:refs/tags/*
- uses: actions/cache@v2
- name: "Caching dependencies (txt)"
uses: actions/cache@v2
id: cache
with:
path: ${{ env.pythonLocation }}
Expand All @@ -46,21 +50,16 @@ jobs:
- run: |
ci/setup-postgres.sh
sudo apt-get -y install coinor-cbc
- name: Install FlexMeasures & dependencies for tests
if: steps.cache.outputs.cache-hit != 'true'
run: |
make install-pip-tools
make install-for-test
pip install coveralls
- name: Run all tests except those marked to be skipped by GitHub
run: pytest -m "not skip_github"
if: ${{ matrix.coverage != 'yes' }}
- name: Install FlexMeasures & exact dependencies for tests
run: make install-for-test
if: github.event_name == 'push' && steps.cache.outputs.cache-hit != 'true'
- name: Install FlexMeasures & latest dependencies for tests
run: make install-for-test pinned=no
if: github.event_name == 'pull_request'
- name: Run all tests except those marked to be skipped by GitHub AND record coverage
run: pytest -v -m "not skip_github" --cov=flexmeasures --cov-branch
if: ${{ matrix.coverage == 'yes' }}
- run: coveralls --service=github
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: pytest -v -m "not skip_github" --cov=flexmeasures --cov-branch --cov-report=lcov
- name: Coveralls
uses: coverallsapp/github-action@v2
if: ${{ matrix.coverage == 'yes' }}
env:
PGHOST: 127.0.0.1
Expand Down
7 changes: 6 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,10 @@
"python.linting.pylintEnabled": false,
"python.linting.flake8Enabled": true,
"workbench.editor.wrapTabs": true,
"python.formatting.provider": "black"
"python.formatting.provider": "black",
"python.testing.pytestArgs": [
"flexmeasures"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
}
18 changes: 16 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Note: use tabs
# actions which are virtual, i.e. not a script
.PHONY: install install-for-dev install-deps install-flexmeasures run-local test freeze-deps upgrade-deps update-docs update-docs-pdf show-file-space show-data-model clean-db
.PHONY: install install-for-dev install-for-test install-deps install-flexmeasures run-local test freeze-deps upgrade-deps update-docs update-docs-pdf show-file-space show-data-model clean-db


# ---- Development ---
Expand Down Expand Up @@ -39,13 +39,27 @@ install-for-dev:
make install-flexmeasures

install-for-test:
pip-sync requirements/app.txt requirements/dev.txt requirements/test.txt
make install-pip-tools
# Pass pinned=no if you want to test against latest stable packages, default is our pinned dependency set
ifneq ($(pinned), no)
pip-sync requirements/app.txt requirements/test.txt
else
# cutting off the -c inter-layer dependency (that's pip-tools specific)
tail -n +3 requirements/test.in >> temp-test.in
pip install --upgrade -r requirements/app.in -r temp-test.in
rm temp-test.in
endif
make install-flexmeasures

install-deps:
make install-pip-tools
make freeze-deps
# Pass pinned=no if you want to test against latest stable packages, default is our pinned dependency set
ifneq ($(pinned), no)
pip-sync requirements/app.txt
else
pip install --upgrade -r requirements/app.in
endif

install-flexmeasures:
pip install -e .
Expand Down
6 changes: 6 additions & 0 deletions documentation/api/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,16 @@ API change log

.. note:: The FlexMeasures API follows its own versioning scheme. This is also reflected in the URL, allowing developers to upgrade at their own pace.

v3.0-11 | 2023-07-20
""""""""""""""""""""

- Added REST endpoint for fetching one sensor: `/sensors/<id>` (GET)

v3.0-10 | 2023-06-12
""""""""""""""""""""

- Introduced the ``storage-efficiency`` field to the ``flex-model``field for `/sensors/<id>/schedules/trigger` (POST).
- Introduced the ``database_redis`` optional field to the response of the endpoint `/health/ready` (GET).

v3.0-9 | 2023-04-26
"""""""""""""""""""
Expand Down
17 changes: 17 additions & 0 deletions documentation/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,17 @@ FlexMeasures Changelog
v0.15.0 | July XX, 2023
============================

.. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).

New features
-------------

* Allow deleting multiple sensors with a single call to ``flexmeasures delete sensor`` by passing the ``--id`` option multiple times [see `PR #734 <https://www.github.com/FlexMeasures/flexmeasures/pull/734>`_]
* Make it a lot easier to read off the color legend on the asset page, especially when showing many sensors, as they will now be ordered from top to bottom in the same order as they appear in the chart (as defined in the ``sensors_to_show`` attribute), rather than alphabetically [see `PR #742 <https://www.github.com/FlexMeasures/flexmeasures/pull/742>`_]
* Having percentages within the [0, 100] domain is such a common use case that we now always include it in sensor charts with % units, making it easier to read off individual charts and also to compare across charts [see `PR #739 <https://www.github.com/FlexMeasures/flexmeasures/pull/739>`_]
* DataSource table now allows storing arbitrary attributes as a JSON (without content validation), similar to the Sensor and GenericAsset tables [see `PR #750 <https://www.github.com/FlexMeasures/flexmeasures/pull/750>`_]
* Added API endpoint `/sensor/<id>` for fetching a single sensor. [see `PR #759 <https://www.github.com/FlexMeasures/flexmeasures/pull/759>`_]
* The CLI now allows to set lists and dicts as asset & sensor attributes (formerly only single values) [see `PR #762 <https://www.github.com/FlexMeasures/flexmeasures/pull/762>`_]

Bugfixes
-----------
Expand All @@ -20,6 +25,18 @@ Infrastructure / Support
----------------------

* Add support for profiling Flask API calls using ``pyinstrument`` (if installed). Can be enabled by setting the environment variable ``FLEXMEASURES_PROFILE_REQUESTS`` to ``True`` [see `PR #722 <https://www.github.com/FlexMeasures/flexmeasures/pull/722>`_]
* The endpoint `[POST] /health/ready <api/v3_0.html#get--api-v3_0-health-ready>`_ returns the status of the Redis connection, if configured [see `PR #699 <https://www.github.com/FlexMeasures/flexmeasures/pull/699>`_]
* Document the `device_scheduler` linear program [see `PR #764 <https://www.github.com/FlexMeasures/flexmeasures/pull/764>`_].

/api/v3_0/health/ready

v0.14.2 | July 25, 2023
============================

Bugfixes
-----------

* The error handling for infeasible constraints in storage.py was given too many arguments. This caused the response from the API to be unhelpful when a schedule was requested with infeasible constraints. [see `PR #758 <https://github.com/FlexMeasures/flexmeasures/pull/758>`_]


v0.14.1 | June 26, 2023
Expand Down
177 changes: 177 additions & 0 deletions documentation/concepts/device_scheduler.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
.. _device_scheduler:

Device scheduler
===========

Introduction
--------------
This generic device scheduler is able to handle an EMS with multiple devices, with various types of constraints on the EMS level and on the device level,
and with multiple market commitments on the EMS level.

A typical example is a house with many devices. The commitments are assumed to be with regard to the flow of energy to the device (positive for consumption, negative for production). In practice, this generic scheduler is used in the **StorageScheduler** to schedule a storage device.

The solver minimises the costs of deviating from the commitments.



Notation
---------

Indexes
^^^^^^^^
================================ ================================================ ==============================================================================================================
Symbol Variable in the Code Description
================================ ================================================ ==============================================================================================================
:math:`c` c Commitments, for example, day-ahead or intra-day market commitments.
:math:`d` d Devices, for example, a battery or a load.
:math:`j` j 0-indexed time dimension.
================================ ================================================ ==============================================================================================================

.. note::
The time index :math:`j` has two interpretations: a time period or an instantaneous moment at the end of time period :math:`j`.
For example, :math:`j` in flow constraints correspond to time periods, whereas :math:`j` used in a stock constraint refers to the end of time period :math:`j`.

Parameters
^^^^^^^^^^
================================ ================================================ ==============================================================================================================
Symbol Variable in the Code Description
================================ ================================================ ==============================================================================================================
:math:`Price_{up}(c,j)` up_price Price of incurring an upwards deviations in commitment :math:`c` during time period :math:`j`.
:math:`Price_{down}(c,j)` down_price Price of incurring a downwards deviations in commitment :math:`c` during time period :math:`j`.
:math:`\eta_{up}(d,j)` device_derivative_up_efficiency Upwards conversion efficiency.
:math:`\eta_{down}(d,j)` device_derivative_down_efficiency Downwards conversion efficiency.
:math:`Stock_{min}(d,j)` device_min Minimum quantity for the stock of device :math:`d` at the end of time period :math:`j`.
:math:`Stock_{max}(d,j)` device_max Maximum quantity for the stock of device :math:`d` at the end of time period :math:`j`.
:math:`\epsilon(d,j)` efficiencies Stock energy losses.
:math:`P_{max}(d,j)` device_derivative_max Maximum flow of device :math:`d` during time period :math:`j`.
:math:`P_{min}(d,j)` device_derivative_min Minimum flow of device :math:`d` during time period :math:`j`.
:math:`P^{ems}_{min}(j)` ems_derivative_min Minimum flow of the EMS during time period :math:`j`.
:math:`P^{ems}_{max}(j)` ems_derivative_max Maximum flow of the EMS during time period :math:`j`.
:math:`Commitment(c,j)` commitment_quantity Commitment c (at EMS level) over time step :math:`j`.
================================ ================================================ ==============================================================================================================


Variables
^^^^^^^^^
================================ ================================================ ==============================================================================================================
Symbol Variable in the Code Description
================================ ================================================ ==============================================================================================================
:math:`\Delta_{up}(c,j)` commitment_upwards_deviation Upwards deviation from the power commitment :math:`c` of the EMS during time period :math:`j`.
:math:`\Delta_{down}(c,j)` commitment_downwards_deviation Downwards deviation from the power commitment :math:`c` of the EMS during time period :math:`j`.
:math:`\Delta Stock(d,j)` n/a Change of stock of device :math:`d` at the end of time period :math:`j`.
:math:`P_{up}(d,j)` device_power_up Upwards power of device :math:`d` during time period :math:`j`.
:math:`P_{down}(d,j)` device_power_down Downwards power of device :math:`d` during time period :math:`j`.
:math:`P^{ems}(j)` ems_power Aggregated power of all the devices during time period :math:`j`.
================================ ================================================ ==============================================================================================================

Cost function
--------------

The cost function quantifies the total cost of upwards and downwards deviations from the different commitments.

.. math::
:name: cost_function
\min [\sum_{c,j} \Delta _{up}(c,j) \cdot Price_{up}(c,j) + \Delta_{down}(c,j) \cdot Price_{down}(c,j)]
State dynamics
---------------

To simplify the description of the model, the auxiliary variable :math:`\Delta Stock(d,j)` is introduced in the documentation. It represents the
change of :math:`Stock(d,j)`, taking into account conversion efficiencies but not considering the storage losses.

.. math::
:name: stock
\Delta Stock(d,j) = \frac{P_{down}(d,j)}{\eta_{down}(d,j) } + P_{up}(d,j) \cdot \eta_{up}(d,j)
.. math::
:name: device_bounds
Stock_{min}(d,j) \leq Stock(d,j) - Stock(d,-1)\leq Stock_{max}(d,j)
Perfect efficiency
^^^^^^^^^^^^^^^^^^^

.. math::
:name: efficiency_e1
Stock(d, j) = Stock(d, j-1) + \Delta Stock(d,j)
Left efficiency
^^^^^^^^^^^^^^^^^
First apply the stock change, then apply the losses (i.e. the stock changes on the left side of the time interval in which the losses apply)


.. math::
:name: efficiency_left
Stock(d, j) = (Stock(d, j-1) + \Delta Stock(d,j)) \cdot \epsilon(d,j)
Right efficiency
^^^^^^^^^^^^^^^^^
First apply the losses, then apply the stock change (i.e. the stock changes on the right side of the time interval in which the losses apply)

.. math::
:name: efficiency_right
Stock(d, j) = Stock(d, j-1) \cdot \epsilon(d,j) + \Delta Stock(d,j)
Linear efficiency
^^^^^^^^^^^^^^^^^
Assume the change happens at a constant rate, leading to a linear stock change, and exponential decay, within the current interval

.. math::
:name: efficiency_linear
Stock(d, j) = Stock(d, j-1) \cdot \epsilon(d,j) + \Delta Stock(d,j) \cdot \frac{\epsilon(d,j) - 1}{log(\epsilon(d,j))}
Constraints
--------------

Device bounds
^^^^^^^^^^^^^

.. math::
:name: device_derivative_bounds
P_{min}(d,j) \leq P_{up}(d,j) + P_{down}(d,j)\leq P_{max}(d,j)
.. math::
:name: device_down_derivative_bounds
min(P_{min}(d,j),0) \leq P_{down}(d,j)\leq 0
.. math::
:name: device_up_derivative_bounds
0 \leq P_{up}(d,j)\leq max(P_{max}(d,j),0)
Grid constraints
^^^^^^^^^^^^^^^^^

.. math::
:name: device_derivative_equalities
P^{ems}(d,j) = P_{up}(d,j) + P_{down}(d,j)
.. math::
:name: ems_derivative_bounds
P^{ems}_{min}(j) \leq \sum_d P^{ems}(d,j) \leq P^{ems}_{max}(j)
Power coupling constraints
^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. math::
:name: ems_flow_commitment_equalities
\sum_d P^{ems}(d,j) = \sum_c Commitment(c,j) + \Delta {up}(c,j) + \Delta {down}(c,j)
2 changes: 1 addition & 1 deletion documentation/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@
# -- Options for intersphinx extension ---------------------------------------

# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {"https://docs.python.org/3/": None}
intersphinx_mapping = {"python": ("https://docs.python.org/3", None)}

# -- Options for copybytton extension ---------------------------------------
copybutton_prompt_is_regexp = True
Expand Down
2 changes: 2 additions & 0 deletions documentation/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ The platform operator of FlexMeasures can be an Aggregator.
concepts/inbuilt-smart-functionality
concepts/algorithms
concepts/security_auth
concepts/device_scheduler


.. toctree::
Expand All @@ -181,6 +182,7 @@ The platform operator of FlexMeasures can be an Aggregator.

tut/installation
tut/toy-example-from-scratch
tut/toy-example-expanded
tut/posting_data
tut/forecasting_scheduling
tut/building_uis
Expand Down
12 changes: 4 additions & 8 deletions documentation/tut/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
Installation & First steps
=================================

Getting FlexMeasures to run
-----------------------------
Preparing FlexMeasures for running
------------------------------------

This section walks you through installing FlexMeasures on your own PC and running it continuously.
We'll cover getting started by making a secret key, connecting a database and creating one user & one asset.
Expand Down Expand Up @@ -194,12 +194,8 @@ Finally, you can tell FlexMeasures to create forecasts for your meter data with
.. note:: You can also use the API to send forecast data.


Run FlexMeasures
------------------


Running the web service
^^^^^^^^^^^^^^^^^^^^^^^^^^
Running FlexMeasures as a web service
--------------------------------------

It's finally time to start running FlexMeasures:

Expand Down
Loading

0 comments on commit 4f6e120

Please sign in to comment.