Empty Space Hierarchy response for newly joined rooms

[morguldir]

Description

When joining a new space over federation, the space doesn't show any rooms in Element Web, I think this may be a somewhat recent regression (room works in conduwuit), there also might be some edge case here though, since seemingly not everyone experiences it

Steps to reproduce

• join a new space, e.g. #space:lix.systems
• open the space in element web
• see that it's empty
• look at the response to see that /hierarchy only returned itself

Homeserver

sulian.eu

Synapse Version

1.109.0r2

Installation Method

Docker (matrixdotorg/synapse)

Database

15.6

Workers

Multiple workers

Platform

Kubernetes+Nginx using the official docker images

Configuration

msc3266 (turning off didn't help)
msc2815
presence
multiple event persisters

Relevant log output

matrix-synapse-85bd95d945-6p9lz synapse 2024-06-13 03:56:46,009 - synapse.access.http.8008 - 473 - INFO - GET-577- <ip> - 8008 - {@morguldir:sulian.eu} Processed request: 0.008sec/0.001sec (0.001sec, 0.000sec) (0.000sec/0.006sec/3) 153B 200 "GET /_matrix/client/v1/rooms/!Hem4nC5nU7RWYJ74%3Alix.systems/hierarchy?suggested_only=false&limit=20 HTTP/1.1" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36.0 (KHTML, like Gecko) Chrome/124.0.6367.243 Safari/537.36.0" [0 dbevts]

{"rooms":[{"room_id":"!Hem4nC5nU7RWYJ74:lix.systems","world_readable":false,"guest_can_join":false,"im.nheko.summary.version":"10","children_state":[]}]}

Anything else that would be useful to know?

There wasn't a lot of information in the logs, so let me know if you have any spots where i could try to patch in some extra logging for this, also note that looking at the m.space.child state events contains the relevant rooms

[morguldir]

So I had set background tasks to the same worker as my user-events-persister (presence, to-device, receipts, etc), which due to presence probably never has a second free to run background tasks properly

I got a dedicated background worker and lots of things cleared up, like device login times, old purge jobs completed, i could see rooms of the spaces again and so on

I saw a few stuck jobs in scheduled_tasks, but i don't think i checked background_tasks before moving the worker, and it was probably very full as well 🤔