Review different caching layers in Alaveteli with an eye to simplifying

[crowbot]

We are caching in several places in the codebase. It would be good to do a review and see if there's an opportunity to simplify. Should also research what recommendations for caching in Rails 3/4 are.

[crowbot]

This is a precursor to #1475

[crowbot]

Specifically, it would be good to decide if we've going to turn on Varnish purging again for WDTK or take some other strategy for expiring cached pages.

[RichardTaylor]

A WhatDoTheyKnow user today was surprised by the site's caching practice. I think they may have seen changes a day before while logged in, then were surprised not to have seen them when viewing the site while not logged in the next day. They wrote:

I have just loaded up your website and, weirdly, all the corrections you made yesterday for me have reverted back to the incorrect and accurate versions.

[garethrees]

Adding a link led to confusion for me as it didn't immediately appear on the request page for a non-logged in user. Adding cache busting parameters to the URL (adding eg. ?odmflsd=dsnfnds) made the links appear. Slow caches / caches which don't update as the content update are surprising and confusing.

– #5483 (comment)

[garethrees]

Linking to #1168

[RichardTaylor]

I can't find the idea of a cache refreshing button for admins to use on pages/documents I recall that has been discussed. I think it might be a good idea, but better would be designing the system so it wasn't needed.

Cache delays do cause issues when running the site, admin actions aren't immediately reflected on the version of the site seen by non logged in users. We can:

• confuse correspondents by saying we've done something but it appears we haven't
• get reports of issues we've already dealt with
• have to add complexity to messages explaining action has been taken but might take a long time to fully take effect

I also just made a mistake when I wrote a tweet which was intended to point to a newly updated public body page, but I forgot about the slow caching issue, so readers were initially sent to the old version of the page, and deleted and re-issued the tweet with a cache busting parameter.

[RichardTaylor]

I had an issue I'd not experienced before this morning. I was creating censor rules and reloading a PDF to seek to see their effect. The PDF was being cached and I had to add URL parameters to break thought the cache.

Previously reloading / re-downloading a PDF has been sufficient to obtain the latest copy.

I don't yet know if this was an issue with one document or if it is a general issue.

[RichardTaylor]

I don't yet know if this was an issue with one document or if it is a general issue.

I am now doing other redactions and seeing the effects straight away as usual.

[garethrees]

While we've now reduced the attachment cache usage in #1549, it still grows pretty large, especially since mySociety hosts several Alavetelis.

We should probably use ActiveStorage with a S3 compatible backend so that we have more flexibility in how we distribute these files.

[sagepe]

Yes, and also get this enabled for raw_emails, files and other things, too. We'd need to consider the prioritisation for each area carefully.

[RichardTaylor]

This thread has previously been linked to #1168 "Requests not appearing immediately on request list page" but just to expand on that - sometimes what looks like it might be a caching issue will actually be a slow search index updating issue.

[garethrees]

https://blog.polyhaven.com/how-we-handle-80tb-and-5m-page-views-a-month-for-under-400/

[RichardTaylor]

At WhatDoTheyKnow we wanted to tweet a link to a page on which we had just added an annotation.

We had an inkling that we needed to be aware caching could delay the appearance of the annotation to non-logged in users.

Our experience is shown below, comments from three people in different places taken from Slack:

Just waiting for the annotation [...] to appear from the point of view of non logged in users

It shows up for me logged out and in incognito mode.

It didn't show up for me in incognito mode, I had to add ?no-cache . Not even with Ctrl + F5. (edited)

I didn't need to do that at all. Odd. Could be an ISP thing?

It was inconsistent for me too.
I could see the annotation via curl and safari new private window. Couldn't see it when logged out in a normal Safari window. All on the same computer.

I can't see it on my phone, either on WiFi or 4G. And I don't believe I've previously loaded the page on my phone.

visible on my phone on wifi, not visible on my phone on 4g. (A reload after switching to 4g made the annotation disappear)

It works for me wherever I try & via [ISP] or 4g

[RichardTaylor]

At the time of writing @confirmordeny and I are both currently experiencing

https://www.whatdotheyknow.com/body/list/estate_management_scheme

not being updated with changes (additions to the list) made around 12 hours ago

[RichardTaylor]

All slow cache issues are being closed in-favour of this issue; however this issue relates to a systematic change and often the issue is that cache-updates appear surprisingly slow at the moment / recently / on a specific page.

[garethrees]

All slow cache issues are being closed in-favour of this issue…

The underlying causes are likely to be the same (or if they're different, we won't really know until we've got a much clearer understanding of how all the layers interact).

[garethrees]

Should look at what our etags are doing and how they interact with varnish