Re-mount volumes on a restart or update

[olevski]

Currently we use the rclone daemon which needs to be running to keep the fuse mounts for storage working. When the daemon dies all mounts on the node that were mounted by the daemon are dead.

This is a huge problem where any interruption to the node servers results in many users of the driver to be suddenly left without access to the volumes they mounted.

We need to add some code that at startup will check all volumes that use csi-rclone and remount them if necessary. This way restarts or upgrades should result only in a short-lived interruption of access to mounted data.

[olevski]

So I did a bit of googling on this. If it is possible to just get the os to remount things on restart. I found these things that may be useful:

• https://github.com/rclone/rclone/wiki/rclone-mount-helper-script
• Automatic Mounting on reboot rclone/rclone#727
• Feature Request: mount.rclone helper rclone/rclone#992

The wiki on the rclone repo is the most relevant. The two issues have background on the request for remounting things at startup and all that.

[olevski]

Another good discussion: kubernetes/kubernetes#70013