sync.sh Automate gdrive Uploads and More

sync.sh - Automate gdrive uploads and More

Introduction

sync.sh uses gdrive program to push (sync) local files to a folder on the users google drive. For details on setting up gdrive security see https://github.com/pageauc/pi-timolo/wiki/How-to-Setup-pi-timolo-gdrive-sync

The default folder in config.py and sync.sh is

~/pi-timolo/media/motion

The folder location can be changed by editing pi-timolo.py and sync.sh variables (they must match each other). For more details see section below - How to Customize Operation of sync.sh

sync.sh Can also be re purposed to sync files other than pi-timolo images.

###How to Run sync.sh Run sync.sh script to test google drive syncing with specified local folder default is /home/pi/pi-timolo/motion. To run sync.sh executed the following

cd ~/pi-timolo
./sync.sh

The sync.sh script will perform the following

Local Source folder will be created by pi-timolo.py if it does not exist. The Remote Destination folder on google drive will be created by sync.sh if it does not exist. see sync.sh script for details sync.sh will perform the following operations

• Checks if FILES_TO_SYNC= script variable value exists eg *jpg Exits if no match found.
• Checks if gdrive is already running. Runs gdrive only if it is not already running to avoid spawning multiple instances
• Kills gdrive processes if it has been running too long. default is > 4000 seconds or 67 minutes. (edit the sync.sh script if you need a different time to kill time period).
• Optionally Looks for a pi-timolo.sync file created by pi-timolo.py indicating there are new files to sync otherwise it exits without attempting to resolve google drive files with specified local folder.
• Runs a gdrive push to sync local folder with the specified google drive subfolder. Only changed and new files are transferred.
• Reports if gdrive sync was successful or reports if errors were encountered
• Optionally checks if pi-timolo is running and restarts or reboots system if script force_reboot=true variable is set and pi-timolo.py is not running. I use this for headless remote operation.
• Optionally check for a google drive config.py file and remotely update and restart pi-timolo. This is handy for managing a camera settings remotely.

How to Customize Operation of sync.sh

If you need to sync another folder you will need to edit the sync.sh script using nano editor or similar

cd ~/pi-timolo
nano sync.sh

Edit variables in nano per comments. No spaces between = and variable and follow example case. Ctrl-x y to save changes.

# ---------------  Local to Remote File Sync Settings --------------
SYNC_ON=true                 # true - Do Remote Sync  false - Non
SYNC_DIR='media/motion'      # relative to this script folder - location of files to sync
FILES_TO_SYNC='*jpg'         # Set the type of files to sync * for all
CHECK_FOR_SYNC_FILE=true     # true if sync file below is required otherwise set to false always sync
SYNC_FILE_PATH=$PROG_DIR/pi-timolo.sync  # name of pi-timolo sync lock filename

# ------------------ Remote Sync Config Settings -------------------
# Remote configuration will copy a new configuration file from remote google drive 
# to local pi-timolo config.py file and restart pi-timolo.
REMOTE_CONFIG_ON=false                 # true - Check for remote config file and install, false - Ignore Checking
REMOTE_CONFIG_DIR='sync_config_cam1'   # Remote Config Folder on google drive (Will be Created if Does Not Exist)
REMOTE_CONFIG_FILE='config.py'         # Name of new pi-timolo config file on google drive
LOCAL_CONFIG_FILE='config.py'          # pi-timolo configuration variables file (default)

# ------------------ Remote Sync Wipe Settings -------------------
# Remote Wipe will erase all Files in the syc folder ater syncing 
# as long as a file wipe.me is located in the remote google drive
REMOTE_WIPE_ON=false                   # true - Check for wipe.me file and wipe sync folder, false - Ignore Checking
REMOTE_WIPE_DIR='sync_config_cam1'     # Remote Wipe Folder on google drive (Will be Created if Does Not Exist)
REMOTE_WIPE_FILE='wipe.me'             # Name of the wipe file on google drive
REMOTE_WIPE_SAFE=true                  # true - only synced files get deleted but this is slow, false- all jpg files get deleted 

# -------------------- Watch App Settings --------------------------
WATCH_APP_ON=false           # false - off  true - Check if app is running and restart or reboot
WATCH_APP='pi-timolo.py'     # App filename to monitor for Run Status
FORCE_REBOOT=false           # false - Restart pi-timolo.py if not running    true - Reboot if not running (Use with Caution)  

IMPORTANT - The latest version of sync.sh has better error checking. If a matching remote folder does not exist it will be created. Please review bash script code for details. Suggest you run sync.sh script from a crontab every 5 minutes or so.

How to setup crontab to schedule sync.sh

Edit the user or sudo crontab using command. (use of sudo is optional)

sudo crontab -e

Add example crontab entry per below then save and exit nano using ctrl-x y. First crontab line below will run sync.sh every 5 minutes

*/5 * * * * su pi -c "/home/pi/pi-timolo/sync.sh > /dev/null"
# Optional start of makevideo.sh and convid.sh 
# 0 22 * * * su pi -c "/home/pi/pi-timolo/makevideo.sh > /dev/null"
# */1 * * * * su pi -c "/home/pi/pi-timolo/convid.sh > /dev/null"

How to Manage sync for multiple Raspberry Pi's

If you have multiple cameras on different Raspberry Pi computers that sync to google drive, it is recommended that you change the motion folder name to something unique. You will also need to change the config.py motionDir= setting and sync.sh accordingly.

Important - Verify the config.py has the motion setting to allow triggering of the sync.sh process.

Enable pi-timolo.py syncLockFile

Use the config.py file using nano editor

cd ~/pi-timolo
nano config.py

then change motion setting variable per below. ctrl-x y to save change

createLockFile = True

The pi-timolo.sync file will then be created when motion images are created. Check if a pi-timolo.sync file exists in the pi-timolo folder otherwise run

./pi-timolo.py

Alternative DropBox solution

If you would like to copy/sync files to a DropBox folder then more information can be found here.
How to use DropBox with Raspberry Pi
Also see More info at GitHub dropbox-uploader.sh Project Repo

Note: You will need to add/modify pi-timolo project to implement the dropbox-uploader.sh alternative. Raise a github issue and if there is enough interest, If you want to develop this feature as a github push, I would be willing to test and document as part of the pi-timolo project. Currently I prefer the gdrive option so would not use dropbox feature.

How to Setup config.py for Remote Configuration