Feature Request: Add support for history of day yield

[technics42]

It would be very nice, if you could add a history for day yield, which could be directly shown/exported in web-browser.
Storing day yield could be done on a defined hour of day, e.g. sunset or midnight. For storing data one could use littlefs - this should be ok, when only storing data once a day.

[kiu77]

Way too much for such a little Mikrocontroller. Not enough Power, not enough Storage, and the software would be the complete opposite of simpel, in my opinion.

[technics42]

Writing one/two values a day is very easy.
Storage is way enough for such little data.
You only need some bytes (<10) each day, so less than 4KB a year - I don't think it will run more than 1000 years ;)
Displaying history could be limited to some days/weeks/months or by selecting a range - exporting could be done for all data.

[kiu77]

If it is so simple for you: Do it. The source is open...

[rmayergfx]

Storing the data in flash is not a good idea. Thats why the data is avail with MQTT and Prometheus.
The next step after storing in flash is the option to backup the data to an external device and so on.

[technics42]

Actually storing values to flash is really simple (which is already done for configuration stuff) - getting into an existing project is some other kind of effort.
Of course I know the source is open and I could modify it, but that's no reason for not requesting a feature.
You shouldn't always fork a project and make your own stuff, if it would be of interest for other people, too - that's why you should ask for additional features in existing project.

The reason for not wanting this feature is surely not depending on power/storage.
Flash on ESP is designed to be written at least 10000 times - so writing once a day on it, isn't a bad idea.
(One could also add an external flash (SD-card), to make backups easier, but hey, why not using internal flash if it fits your needs)

Using MQTT is for other purpose ("realtime") and is way over-engineered for getting history of day yield, if you can get this value directly from inverter.

I guess for quite a view number of users it would be very nice to have such a function.
But of course, that's just my opinion (like yours) and it's only some suggestion.

[kiu77]

@technics42
Actually you're completely wrong.
Given that you can write values 10,000 times to a page. A page in ESP32 flash is 4K. So a flash memory contains 1,000 pages. This means:
Calculate 1 set of data is 20 values with 4 byte each = 80 byte.
To collect Data in RAM until they fill for example 1 page = 50 data sets is not an option, because you will loose the data if a reboot or a current failure occurs.
If you want to get a data point every minute the following calculation can be made:
The space of the flash chip can hold up to 50 x 1.000 data points = 50.000 minutes = 833.3 hours =34.7 days.
=> the flash can hold data for about a month. Not much.
=> if you choose 10 s Intervalls the flash can't contain even a weeks data.

The flash would last for 10,000 month of data written - but this number is irrelevant. The size is relevant.

Next aspect:
I too would find it nice if your wish would be realized - despite the fact that the space of the flash is too small for that. But the effort to program all this is high and I never would dare to ask somebody to do this job for me.
The argument with forks is not good, because if you wish to have that feature despite of the difficulties you should do it yourself. If your solution proofs to bee good there would be a good chance to be taken over to the major branch.

Long text short meaning: you better forget this ;-)

[lumapu]

what do you think about such a module? I have already three of them here 😀
Only problem I see with that: for ESP8266 only one SPI is available, we must share it in any way.

https://www.az-delivery.de/products/copy-of-spi-reader-micro-speicherkartenmodul-fur-arduino?variant=6107610710043

[kiu77]

Good idea. Even a cheap 4GB card would last for >10 years of data collection.

[technics42]

I am more than fine with that :-)
(actually a SD-card is the best solution for me - I also mentioned it above, but thought you guys wouldn't want to add extra hardware, if not really needed)

If you want another display to be supported, you could also think about something like ILI9341 including SD-card-adapter ;)

Just as an extra information: for storing data on internal flash, there would be also solutions that would fit, please keep in mind, that I only wanted to keep day yield from inverter, no single values/calculations! - but that's a different story
For SD-card you should also keep in mind, that this is also flash-memory and has limited write life span

[joedezibel]

Like I told Lukas already. It would be awesome to use this module to write the Data into a CSV file. A lot of people like me know nothing about raspberries and software stuff. The only Thing I would like to store are the "Yield Day" Data. So you can make it a curve in Excel for example ore Apple Numbers.

[stefan123t]

There was a PR for SD-Card support from Gerald R. quite some time ago.

The above question from @lumapu remains open: how can we add this as we may have only one SPI ?

Is there an option to work around this with some kind of Hardware Abstraction Layer (HAL) that will multiplex between NRF, CMT, whatever type of Display, ModBus Adapter, SD-Card, etc. ?
Examples given are the forks from LennartF22 of OpenDTU and RF24 which were necessary to do the switching.

Additional bonus: do we need some kind of port expansion logic circuitry in order to e.g. use only two GPIOs to address four SPI devices via CE ?

[technics42]

Usually you can share SPI by using different CS. So you need one additional pin for CS of SD-card-adapter. I guess this could be D0 (GPIO 16), if one is also using a display.

[stefan123t]

@technics42 did you check the project / fork OpenDTU-Database ?

@lumapu we investigated some of the issues with different SD-Card modules here in tbnobody/OpenDTU#2125 and in AhoyDTU here #1572
I guess this may be a duplicate of #1572, please decide which of the issues you want to keep #1063 or #1572 ?

[technics42]

No, I didn't check any OpenDTU-SW, as I am using ESP8266.
Btw, for using a SD-card you don't need an adapter/module, but you can directly connect SD-cards to ESPs...