Examples for copy triggers

[waza-ari]

Dear all,

wondering if there were any examples on how to use the PVC annotations for copy triggers? I'd love to use volsync for backup purposes, and for some workloads (mainly databases) I'd need to run a command within the pod to dump the database to a separate volume, which is then synced with volsync / restic.

I understand that volsync is not implementing hooks on purpose, and I understand the concept of labels. What I'm missing is the At this point the user can run commands to pause or quiesce their application. part, and how exactly this would be done.

[tesshuflower]

@waza-ari You're correct that VolSync is not implementing hooks - and will not run commands for the user to do things like a db dump.

Normally when the replicationsource schedule runs, a volumesnapshot would be taken of the PVC (to give us a consistent point-in-time copy), and then that snapshot would be converted into a temporary PVC that would be used to copy files during the sync cycle (a backup, replication to another side etc). This would happen every time a replicationsource sync runs.

If we look at this part of the docs: At this point the user can run commands to pause or quiesce their application.:

• Right here, the PVC (the sourcePVC in volsync replicationsource terms) has the following annotation on it from VolSync:
  volsync.backube/latest-copy-status: "WaitingForTrigger"

  This means that instead of taking that volumesnapshot of the sourcePVC, VolSync is now pausing and waiting for the copytrigger to be updated.

• So now you could run the db dump command on your db and then tell VolSync (via the next steps in copy triggers) that it can proceed to take the volumesnapshot only after you know your dump has completed and you're ready for a backup/replication.

Not sure this really answered your question - the copy triggers were added assuming someone is automating the user-side steps via their own controller or perhaps via some scripting mechanism.

[waza-ari]

Hi @tesshuflower , first of all, thank you very much for taking the time to write an extensive answer! This was pretty much what I understood as well, and the documentation is quite good in this regard. The interesting part starts here:

the copy triggers were added assuming someone is automating the user-side steps via their own controller or perhaps via some scripting mechanism.

This is the part where I currently don't understand how this would be done and was hoping for an example. Ultimately, I just need to find a way to run a command within the container. Writing a separate controller for such a simple use case doesn't seem to be the way forward. Any mechanism implemented here would need to monitor certain resources for the annotations, take appropriate actions and set the annotation as you mentioned. This part is where I'd like to understand if there are any real world use cases out there, or examples that could be built upon.

Ultimately, the goal for me is to run a single command and then sync / export the volume afterwards. Given that use case, VolSync simply might not be the right tool for the job (no fault of the tool of course). In our setup, everything is managed in Git via ArgoCD, thus we do not care about the K8s resources themselves. We're merely looking into data only backup and restore options. That combined seems to be a pretty niche use case though, as all tools I looked at so far have some issue with it. Velero requires the pod to be backed up as well, Stash doesn't work with ArgoCD, and VolSync doesn't have an easy to use option to run pre-backup commands. Again, not complaining, just saying that the search will continue. Anyway, thanks for taking the time again!

[tesshuflower]

@waza-ari copy triggers were essentially meant for users with controllers or some other mechanism that want to be able to schedule when to quiesce but potentially not have to even know about a replicationsource at all. So would need to be someone who can watch and update labels on the PVC.

Another possibility is you could look into the triggers for the replicationsource - you can possibly use a manual trigger and run something which does your quiesce and then triggers the replicationsource replication (or replicate on a known schedule). However a potential downside is you'd have to watch the replicationsource to know when it's complete so you could then unquiesce.