Repository navigation
data collection throttling? #14
Description
Hey, great project! :)
I just set things up here, but sadly my poor little home assistant green using a sdcard as storage is totally unable to keep up with the dataflow here. ;(
I have 4 strings and 37 panels so the integration says 42 devices 464 entities
When I enable my waveshare and the sun is up, my server just goes completely unresponsive. Disabling it and it comes back up.
2026-04-13 07:42:01.668 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sensor.tigo_ts4_7_power_4 (<class 'custom_components.pytap.sensor.PyTapSensor'>) took 0.759 seconds. Please create a bug report at https://github.com/azebro/pytap/issues
2026-04-13 07:57:24.325 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration pytap which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
I don't think its cpu issues, but more i/o to the storage trying to save data from all those sensors? Not sure...
Would it be possible to add something to only store less of the dataflow? Say every minute or five minutes?
I guess I could also disable panel sensors I don't care too much about.
Failing that I guess it's time to upgrade the old ha green. :)
Activity
gabriel31337 commentedon May 6, 2026
Adding myself to the request above. The integration works like a charm in terms of how much information can it grab from the optimizers via rs485 converter, however, it just goes haywire with how much data it collects and stores. My HA installation went south with the performance. Would it be possible to make a configurable option of some kind of delay between data grab? 15 seconds would be sufficient for me... :-)
gabriel31337 commentedon May 21, 2026
@azebro there is a commit to implement this data throttling, will you merge it and release a new version maybe? :-)
azebro commentedon May 22, 2026
Have you tried to exclude some of the pytap sensors from recorder (and possibly set buffer time) in config.yaml? That will lower pressure on IO is yours is slow.
How many panels do you have?
I am running on 26 panels on RPi5 (on NVME and not SD though) with no impact in IO.
Let me evaluate the PR that has been opened, as wanted to avoid batching (but maybe a setting for whoever needs that would make sense)
gabriel31337 commentedon May 22, 2026
If I exclude the data from recorder, I will lose history. I want to keep the history of all the sensors' data types, just don't want to store every single change which happens every single millisecond. Buffer time won't help either, it will store all the data which are used for the graphs later on. I have 1.4 million records just for the overall performance sensor in just a week... 🥶
nirik commentedon May 23, 2026
Yeah, 37 panels here in 4 strings. ;(
gabriel31337 commentedon May 30, 2026
@azebro have you had a chance to look at this? Thank you.
azebro commentedon Jun 4, 2026
there is a feature branch you may want to start testing that introduces this functionality based on #15. I need to do a bit more testing myself before I create a release out of that. Unfortunately work takes priority.
gabriel31337 commentedon Jun 4, 2026
I know, I have mentioned it 2 weeks ago...
azebro commentedon Jun 4, 2026
Implemented in release 1.3.0
gabriel31337 commentedon Jun 5, 2026
@azebro thank you very much for publishing this, the performance of my HA skyrocketed! I use 24h graphs and now it's loading instantly.
nirik commentedon Jun 8, 2026
Sorry for being late here, but this works great here as well. ;) Thanks so much...