Add method for cleaning up non-high scores

[peppy]

ie. scores with preserve = 0 set (and meeting various other conditions).

This should probably be implemented as a command which can be cron'd to run on an interval.

It should take into account things like:

• Whether a score is pinned
• Whether a score is part of a multiplayer game which should cause it to remain
• Whether the score is still the most relevant high score (and potentially change the preserve flag if not).
• Probably more things

Tasks

Preview Give feedback

1. Add command to clean up non-preserved scores #144
   size/M +1
2. Add command to revoke preserved flag from scores which no longer require it #145
   size/L +1
3. Delete replays from S3 when cleaning up non-preserved scores #243
   size/M +1
4. Check for multiplayer usage of score before marking it as non-preserved #244
   size/S +1
5. Always preserve multiplayer scores regardless of pass/fail osu-web#11118
   area:multiplayer +1
6. Check ranked state when determining user best score in mark non-preserved scores command #254
   size/L +1
7. Add queue to signal that a score may not need to be preserved anymore

[peppy]

As a note here, depending on ppy/osu-infrastructure#16 conclusion this command will need to be altered.

Put another way, we need to make sure that partition rotation doesn't happen to soon, to ensure that things like replay cleanup can be performed first.

[peppy]

Short term, this won't be running. We'll keep all scores around until it's ready to go.

[bdach]

@peppy do you possibly recall what the last task item here ("Add queue to signal that a score may not need to be preserved anymore") means? Who's this queue for? Who's pushing to it, who's consuming it?

[peppy]

I believe that was for osu-web to push to (ie. when a score was unpinned) so we could handle accordingly.

Right now, how we run clean-up is a bit up-in-the-air. My current plan is to run on all users which have recent scores, iterating all their scores. Which may work okay but also may take too much processing power, not sure yet. Could still use a queue for this though, and just push all recent user ids to the queue.

Not sure yet.

[bdach]

I see, thanks. I guess I'll leave that last item alone for now then?

There's one further web change I wanna make related to this issue that wasn't noted down before, just a little something I noticed in passing. Intent is to get it out today.

[bdach]

Aforementioned change is PR'd and linked in the OP.