Speed up search by avoiding the creation of intermediary objects

[Flix6x]

I'm trying to skip having the ORM create a potentially large number of TimedBelief objects.

[victorgarcia98]

Awesome!!

I created a small benchmark where I compare the speed of fetching the beliefs of 200 sensors that I have in my DB. The average improvement is of $4.53\pm 1.76$.

Two observations:

• It appears to cause a greater improvement when the base query takes more time (>=0.5)
• The new approach appears to be worse in 2 queries (not significant), but it happens for two queries that take very short on the baseline.

from flexmeasures.app import create
from flexmeasures.data.models.time_series import TimedBelief
from flexmeasures.data.models.time_series import Sensor

from flexmeasures.data.models.time_series import TimedBelief

app = create()

import time

FILENAME ="old.txt"

with app.app_context():
    sensors = Sensor.query.all()[:200]
    
    with open(FILENAME, "w+") as f:
        try:
            for sensor in sensors:
                t = time.process_time()
                TimedBelief.search_session(app.db.session, sensor)
                t1 = time.process_time() - t
                print(t1)
                f.write(str(t1) + "\n")
                f.flush()
        except KeyboardInterrupt:
            pass

benchmark.xlsx

[Flix6x]

@Ahmad-Wahid in our private meeting you mentioned a flexmeasures issue that was related to this PR, and that you posted relevant info on GitHub. Can you point me to it? Maybe it was this one, but it doesn't seem related to me.