Add paragraph about how to fix ClientStorageError.

[jugmac00]

modified: doc/zeo-client-cache.txt

[jugmac00]

Hm, actually I have no clue why the build breaks, when I just add a paragraph to the documentation section.

• pypy build times out... ?
• Tests with failures: /home/travis/build/zopefoundation/ZEO/src/ZEO/tests/drop_cache_rather_than_verify.txt - I have not touched that file
• also, I have no clue why only the 3.5 build is failing

I guess I need a helping hand.

[jamadden]

Unfortunately there are a few tests that can be a bit flaky. Often a re-run will fix them. I've kicked off the broken builds again, let's see if that does it.

[jugmac00]

Thanks a ton @jamadden !!

[jamadden]

The actual underlying error is something like "AssertionError: Server behind client..."; "Client has seen newer transactions than server" is what is logged (which may or may not go somewhere useful).

Perhaps instead of (just?) putting this in the documentation, the actual error and/or log message should be modified to mention removing the cache? Something like "AssertionError: Client cache is out of sync with the server (server behind client: ...). Verify that this is expected and then remove the cache file at %s before restarting." % (self.cache,)" The ClientCache object would need to grow a useful __str__ and/or __repr__ to make this work (without poking into its internals).

Come to that, I'm not sure if those are always the right recovery steps..why would this have happened? Wouldn't it mean transactions were removed from the server? Perhaps the server was restored from an older backup? In that case, maybe the client cache is actually a way to recover data, and deleting it prematurely would be a bad thing?

[jugmac00]

@jamadden Thank you very much for taking your time.

I experienced this error today when I pulled a production db into my local dev environment. The only hint on how to fix this problem (without looking in the source code), was in the Plone 3 documentation:
https://docs.plone.org/3/en/manage/troubleshooting/transactions.html#zeostorage-client-has-seen-newer-transactions-than-server

I thought it would be a good idea to document this in the ZEO docs, but obviously there is a better way.

You are absolutely right - it would be far better to improve the error message over putting this in the documentation.

At the moment - besides my day job - I got some open issues/pull requests already for Zope (+ecosystem) and thus no time to "quickly" implement your solution.

Also, from a quick glance tox is not setup to run coverage, and from looking at the code and the tests it seems the branch ("Cache newer than server") is untested.

ZEO/src/ZEO/asyncio/client.py

Lines 488 to 493 in bd7cd06

	elif cache_tid > server_tid:
	self.verify_result = "Cache newer than server"
	logger.critical(
	'Client has seen newer transactions than server!')
	raise AssertionError("Server behind client, %r < %r, %s",
	server_tid, cache_tid, protocol)

At the end of the branch, there is an AssertionError - no resolution like invalidateCache or invalidateTransaction. Maybe that is on purpose, as you mentioned, it may be not always the best idea to delete the client cache, and maybe especially not automatically.

tl/dr
Obviously, this pull request should be closed.

As you proposed an imho very good solution, would you mind opening an issue or should I do it?

[dataflake]

Yes, this pull request should be closed. Just removing the cache file may paper over a deeper issue.

[jugmac00]

Gonna close this pr in favor of #142 - thanks for your input!