Spurious re-executions lead to delays on startup

[hubertp]

First discovered in #11882, it has been fixed in #11949 by avoiding cache invalidation when a visualization is triggered.
Sadly, self argument caching does not play well on startup and caused a serious enough regression that warranted a revert of the original PR.

This problem is particularly visible when initially reading (unnecessarily) really large files twice.