/var/log/ not copied using rsync backup

[sbushnell]

Hi,

When I tested a restore that was backed up using rsync the /var/log/ directory was empty so several programs would not start. due to missing directories.
My backup directory '/mnt/backup/openhabian/openhabian-rsync-backup-20230218-073828/var/log/' is empty.
I see /var/log has the filesystem 'overlay2'... not sure what this means? 'overlay1' is also not copied. How can I add overlay1 & overlay2 to the backup?
Thanks. BTW BEST back ever!!!

Filesystem Size Used Avail Use% Mounted on
/dev/root 14G 9.3G 4.0G 70% /
/dev/mmcblk0p1 253M 50M 203M 20% /boot
/dev/zram1 868M 169M 638M 21% /opt/zram/zram1
/dev/zram2 468M 3.5M 430M 1% /opt/zram/zram2
overlay1 868M 169M 638M 21% /var/lib/openhab/persistence
overlay2 468M 3.5M 430M 1% /var/log

[framps]

I see /var/log has the filesystem 'overlay2'... not sure what this means? 'overlay1' is also not copied.

The overlay partitions are just in memory partitions and raspiBackup can only save partitions residing on any persistent storage like disks or SD cards.

Looks like OpenHab creates these partitions during startup and for some reasons this creation fails when the restored system is started.

/var/log is a directory used by the system. Looks like you use a customized OS and for some reasons the creation of this partition also fails during boot of the restored system.

I suggest to contact the OpenHab community why the creation of these overlay partition fails during backup and how this can be fixed.

Would be great either to add a link to the community you ask or at least let me know what's the root cause and how this can be fixed and I can document this on my webpage for OpenHab.

[sbushnell]

Hi, Thanks for your response! After further checking on the Openhab forums, I found that they use rsync to mirror the sd card to another sd card mounted on /dev/sda and the mirrored sd card has the overlay directories and /var/log files are all there. I use this ;openhabian' function already. To test, I did a rsync -a --exclude="/mnt/backup/*" --exclude="/media/pi/rootfs/*" --exclude="/media/pi/boot/*" / /mnt/backup/test/ and it did copy the overlay files to their respective directories. As you see I did the rsync from '/' root. Is the first time I have used rsync so I'm not exactly sure what I'm doing.... I got lots of errors of missing files etc but it did prove the overlay directories are copied to the back up. So don't understand why your rsync does not? Not sure where to go from here. I was thinking of adding a 'cp' to manually copy the overlay directories after each backup but probably not the best way. BTW, your software is also mentioned as a backup tool for Openhab :-) Thanks Cheers Scott https://community.openhab.org/t/automatic-backup-script/139796 [image: image.png]

…

On Sat, 18 Feb 2023 at 09:54, framp ***@***.***> wrote: I see /var/log has the filesystem 'overlay2'... not sure what this means? 'overlay1' is also not copied. The overlay partitions are just in memory partitions and raspiBackup can only save partitions residing on any persistent storage as disks or SD cards. Looks like OpenHab creates these partitions during startup and for some reasons this creation fails when the restored system is started. /var/log is a directory used by the system. Looks like you use a customized OS and for some reasons the creation of this partition also fails during boot of the restored system. I suggest to contact the OpenHab community why the creation of these overlay partition fails during backup and how this can be fixed. Would be great either to add a link to the community you ask or at least let me know what's the root cause and how this can be fixed and I can document this on my webpage. — Reply to this email directly, view it on GitHub <#618 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKSXXSURI3ZJMOESZCO65TWX76R3ANCNFSM6AAAAAAU74I2NQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[framps]

So don't understand why your rsync does not? Not sure where to go from here.

I think I know the reason: raspiBackup uses rsync option -x which causes rsync to stay on the root filesystem. Your in memory files reside on a different filesystem and thus are not copied.

You can remove this option in the following way: Add following line at the end of /usr/local/etc/raspiBackup.conf:

DEFAULT_RSYNC_BACKUP_OPTIONS="-aHA --delete"

Warning

If you have any other partitions mounted (e.g. some NAS partition) these partitions will also be saved by raspiBackup now ! Make sure you have no other partitions mounted or exclude them (use this option).

[sbushnell]

Hey Framps,
Worked perfectly! Thank you VERY much for your assistance with my issue! Excellent support!

I am still using the 'SD Mirror' function built into Openhabian image (soon to stop as the SD mirror is not 100% reliabable... some times the mirrored sd card doesn't boot) so the 'backup' sd card is always mounted. I used the -- exclude to not copy it. If you are going to add this info to your document for 'Openhab' users you may want to add to stop the openhab service at least for the initial backup as due to the time it takes, openhab changes some files and the rsync will fail. I have found that the subseqent backups work fine with openhab still running as they only take approximately 5mins doing daily backups.

[framps]

If you are going to add this info to your document for 'Openhab' users you may want to add to stop the openhab service at least for the initial backup as due to the time it takes, openhab changes some files and the rsync will fail. I have found that the subseqent backups work fine with openhab still running as they only take approximately 5mins doing daily backups.

There exists already a page for different applications which describes how to use raspiBackup (See here for the German page)

For all applications I asked the development team whether my statements are correct. Would be great to get a similar approval for your statement. Did you already talk to the dev team about your suggestion and have a link to a thread which covers the discussion? Otherwise we should open a thread and discuss your proposal 😉