Messages do not send from my homeserver over federation

[gspbirel56]

Description

Hello! My homeserver setup has consisted of a main server running Synapse, an NFS machine for media_store, and a database server for PostgreSQL. Over the weekend, I attempted to upgrade my NFS machine and the worst case scenario happened: all of media_store is gone, and my most recent backup is from December.

I have since been able to set up the new drive, albeit without any data in media_store, but somehow this has resulted in issues with federation. On the matrix.org side, federation seems to be fine. However, I can break down the issue like so:

Things I have tried that do work:

• sending messages between users on my homeserver
• typing notifications between users on my homeserver and another (federated)
• receiving messages from other homeservers

Things I have tried that do not work:

• ping rooms from my homeserver (this may be unrelated)
• sending messages in 1:1 chats
• sending messages in group chats
• creating new 1:1 chats from my homeserver to others
• 1:1 calls with other homeserver users
• verifying others over federation

Steps to reproduce

I believe the description highlights the issue: this seems more holistic than a simple cause-effect bug

Homeserver

chrrreeeeesss.com

Synapse Version

1.104.0

Installation Method

pip (from PyPI)

Database

PostgreSQL 13 single server, always on PostgreSQL, no backup restore

Workers

Single process

Platform

• Hyper-V: 6 CPUs, 8 GB RAM for host machine
• FreeBSD 14.0-RELEASE-p6 for guest machine
• I have tried running as sysrc service and via synctl, but the issue is the same

Configuration

No response

Relevant log output

(...)/synapse/env/lib/python3.9/site-packages/synapse/handlers/federation_event.py", line 1189, in _compute_event_context_with_maybe_missing_prevs
    raise FederationError(
synapse.api.errors.FederationError: ERROR 403: We can't get valid state history.

Anything else that would be useful to know?

No response

[gspbirel56]

Recreating this as a support ticket as this may not be a bug.