Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/enhance backup script #4833

Open
wants to merge 5 commits into
base: staging
Choose a base branch
from

Conversation

OpenSpaghettiSauce
Copy link

Sorry that I make another appearance regarding this script. I use it a lot and the latest THREADS was a great improvement.

This time, I added two things:

  • NODATE
    This skips the creation of the timestamp folder. Why? If you back up via s3fs, or backblaze/s3 client, you may opt for overwriting your backup files, and store them with the same name. This way, you can use bucket policy to practically never run out of disk space (well, never run out of money) and still have several snapshots available.

Usage: Specifying any value, such as NODATE=1 will work.

  • 'help'
    Adds a new option the user can call to see what the script can actually do.
    It can be called with ".sh help", or if the user simply calls the script without any argument, the script now tells them about this.

Feel free to change NODATE's name, or the text of help, of course. This worked for me, but I'm just a user of this great project. Thank you guys for your continued work on Mailcow!

@MAGICCC MAGICCC changed the base branch from master to staging November 2, 2022 17:19
This adds a THREADS like env var and it makes the script skip the creation of the ${DATE} folder.
This adds a "help" text, so the user knows their options. Calling just the script will now tell the user to either user 'backup', 'restore', or 'help'.
@DerLinkman
Copy link
Member

Nice nice. But one question i have left somewhere in my brain :D :

  • What happen if you restore this backup with the NO_DATE Variable? How does the Script behave then?

@OpenSpaghettiSauce
Copy link
Author

Okay that is an absolutely valid question! I'll test it out ASAP and report back/fix my branch.

@OpenSpaghettiSauce
Copy link
Author

Totally valid point.

I suppose the var support could be added here too but...
But, if I think about it more... why even check for a folder?

I mean, shouldn't the user just point the script to a folder, such as let's say /backup/mailcow-2022-11-03-01-49-04/?

I am saying this with utmost respect, this is your work, your project.
Just asking, like maybe it'd easier to drop the whole folder selection thing?

For sanity check, like "to make sure this is a proper backup folder", I would probably change the script again (yeah, I know, I should just close my account, sorry). Let's say put a "mailcow_backup.txt" within the folder, And the content would be like:
for var in *; do md5sum $var; done. And if the files in the folder check out, then it is a valid backup?

md5val backup_crypt.tar.gz
md5val backup_mariadb.tar.gz
md5val backup_postfix.tar.gz
md5val backup_redis.tar.gz
md5val backup_rspamd.tar.gz
md5val backup_vmail.tar.gz
md5val mailcow.conf

But then you also have the specific backup modes, like when you only backup some part, and not 'all'.

I am just thinking out loud, so to say. Let me know if you have any ideas...

@DerLinkman DerLinkman added this to the 2022-11 milestone Nov 6, 2022
@DerLinkman
Copy link
Member

I suppose the var support could be added here too but...

Yeah i think this is the way we want to go.

Not all users want to use a bucket for backup savings and therefore the policys.

So i guess to make everyone happy we´ll simply add the variable.

@OpenSpaghettiSauce
Copy link
Author

I'll add it then in a moment. Thank you.
Well I'm saying a moment but I swear I'll get 10 calls in the next 5 minutes. They sniff this out.

@DerLinkman
Copy link
Member

I'll add it then in a moment. Thank you. Well I'm saying a moment but I swear I'll get 10 calls in the next 5 minutes. They sniff this out.

Good man. I actually was about trying to do this as well. But please feel free.

@OpenSpaghettiSauce
Copy link
Author

No u.

Seriously, it's your project, you know how you like this to be implemented. And thank you for not forgetting about this, appreciate it!

@DerLinkman
Copy link
Member

No u.

Seriously, it's your project, you know how you like this to be implemented. And thank you for not forgetting about this, appreciate it!

Yes but please do that though I've deleted the things (which were not many to be honest) because I'm not sure how to implement it.

I tested it in Virtualbox with my own live server backups and it seemed to restore fine like this. I'm not sure if the help/explanation is enough, feel free to add more, change wording, etc.
Copy link
Member

@DerLinkman DerLinkman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment :)

@andryyy
Copy link
Contributor

andryyy commented Nov 30, 2022

Besides that it's "mailcow" and it needs a doc PR.

@OpenSpaghettiSauce
Copy link
Author

Sorry @andryyy , could you elaborate, please? Thank you!

Apparently I changed the problematic for loop on the VM, but I forgot to copy the script back to the git folder before committing. I put the for loop into the IF branches now.

TESTING IS UNDERWAY - I will comment again once it is 100% verified working (for me, again, sorry.)
@OpenSpaghettiSauce
Copy link
Author

Okay so I tested it, it worked for me (NODATE=1), please verify once time allows. Thank you.

@DerLinkman
Copy link
Member

Sorry @andryyy , could you elaborate, please? Thank you!

What @andryyy was saying is that firstly the "Mailcow" Names inside your PR have to be called "mailcow" with a small M and secondly that this new Parameter needs a entry in the mailcow docs repo: https://github.com/mailcow/mailcow-dockerized-docs

mailcow is mailcow.
@DerLinkman DerLinkman modified the milestones: 2022-12, 2023 Jan 17, 2023
@milkmaker
Copy link
Collaborator

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@milkmaker milkmaker added the stale Please update the issue with current status, unclear if it's still open/needed. label Mar 18, 2023
@milkmaker milkmaker closed this Mar 26, 2023
@DerLinkman DerLinkman reopened this Mar 26, 2023
@DerLinkman DerLinkman removed the stale Please update the issue with current status, unclear if it's still open/needed. label Mar 26, 2023
@milkmaker
Copy link
Collaborator

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@milkmaker milkmaker added the stale Please update the issue with current status, unclear if it's still open/needed. label May 25, 2023
@DerLinkman DerLinkman added neverstale Bot doesn't mark the issue or PR as stale and removed stale Please update the issue with current status, unclear if it's still open/needed. labels May 25, 2023
@DaGuich
Copy link

DaGuich commented Jun 10, 2023

Why is this PR blocked? Is the only thing missing the documentation?

@Programie
Copy link

Will this PR ever be merged? I'm really interested in the NODATE option so I can use the mailcow backup folder in my borg backup without having timestamped paths in my already timestamped borg backups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
add into docs needs testing neverstale Bot doesn't mark the issue or PR as stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants