MP+: utilize ReadWriteMany volumes for repository cache

[jpopelka]

Description

So far, we've been using only block storage (AWS EBS), which can't be accessed in ReadWriteMany mode.
Because of that, we need to have one repository cache for each worker - see #210

On MP+ we can also use AWS EFS file storage (basically NFS - network file system) which can be accessed in ReadWriteMany mode.

Hence, after we have both, stg & prod on MP+, we can consolidate all the RWO worker volumes (1+1 per long-running worker) we use for the repository cache into one(?) RWX.

And we can change workers from StatefulSet to Deployment at the same time, I guess.

Benefit

The long-running workers will use significantly less disk space (1 volume instead of 4 for 2 l-r workers)

Importance

gain/low, impact/low

Workaround

• There is an existing workaround that can be used until this feature is implemented.

Participation

• I am willing to submit a pull request for this issue. (Packit team is happy to help!)