Replace Attic with Borg. #2

Author: slomkowski

.gitignore

@@ -2,8 +2,11 @@
 
 # config files
 conf.d/*
-!conf.d/*.example
+!conf.d/example-*.sh
+!conf.d/example-excludes.txt
 
 # IntelliJ
 *.iml
 .idea/
+
+_deploy*.sh

README.md

@@ -1,13 +1,13 @@
 # Simple backup scripts
 
-This is a collections of scripts, which automate the backup process of Linux server using [Attic](https://attic-backup.org/), a deduplicating backup tool. 
+This is a collections of scripts, which automate the backup process of Linux server using [Borg](https://borgbackup.readthedocs.io), a deduplicating backup tool.
 
 The starting point is `backup.sh`, which works in following way:
 * The scripts starts.
 * **premount** scripts are executed.
-* If Attic repository is remote, it's mounted.
+* If Borg repository is remote, it's mounted.
 * **predump** scripts are executed.
-* Attic creates backup point for each mountpoint defined in *general.sh*.
+* Borg creates backup point for each mountpoint defined in *general.sh*.
 * **postdump** scripts are executed.
 * If the repository is remote, it's unmounted.
 * **postmount** scripts are executed.
@@ -29,16 +29,16 @@ The plain file dump doesn't guarantee the database consistency. To address this
 
 Scripts dumps first 512 bytes for each drive defined in `PREMOUNT_MBR_PHYSICAL_DEVICES`. This area contains the MBR and partition table.
 
-## Copying Attic repository to secondary location
+## Copying Borg repository to secondary location
 
-`copy_attic_repository_rsync` side script can be used to copy the Attic repository to secondary remote location, which increases survivability in case of storage failure.
+`copy_borg_repository_rsync` side script can be used to copy the repository to secondary remote location, which increases survivability in case of storage failure.
 
 ## Installation
 
 * Install required packages:
 
 ```bash
-aptitude install sshfs attic rsync lbzip2
+aptitude install sshfs borgbackup rsync lbzip2
 ```
 
 * Checkout the repository, preferable to *root*'s home directory:
@@ -50,18 +50,18 @@ git clone https://github.com/1connect/backup-scripts
 * Go to *backup-scripts/conf.d* directory. There are several *.example* files. To enable the module, copy it's configuration file to it's proper name:
 
 ```
-cp general.sh.example general.sh
+cp example-general.sh general.sh
 ```
 
-You shouldn't change name of the *.example* files, because you'll run into conflicts when you decide to pull the new version from the repository. *general.sh* is main configuration file, you should have it enabled and configured.
+You shouldn't change name of the *example-* files, because you'll run into conflicts when you decide to pull the new version from the repository. *general.sh* is main configuration file, you should have it enabled and configured.
 
 * After you're done with configuration, run the *backup.sh* script in verbose mode:
 
 ```
 ~/backup-scripts/backup.sh -v
 ```
 
-First, it runs *premount* and *predump* modules. Then it initializes the Attic repository. You shouldn't set any password, because you won't be able to provide it during automated backup.
+First, it runs *premount* and *predump* modules. Then it initializes the Borg repository. You shouldn't set any password, because you won't be able to provide it during automated backup.
 
 The very first run deduplicates your whole filesystem, it'll take long. The subsequent runs are much faster.
 
@@ -78,18 +78,15 @@ The very first run deduplicates your whole filesystem, it'll take long. The subs
 List list existing backups and choose the one which suits your needs the best.
 
 ```
-attic list /my/backup/storage/myhostname/repository.attic/
+borg list /my/backup/storage/myhostname/repository.borg/
 ```
 
-Then access the files. The most convenient way is to use Attic *mount* feature:
+Then access the files. The most convenient way is to use Borg *mount* feature:
 
 ```
-attic mount /my/backup/storage/myhostname/repository.attic::{backup point} /tmp/restore
+borg mount /my/backup/storage/myhostname/repository.borg::{backup point} /tmp/restore
 ```
 
-**Warning!** Because of bug present in Attic, you always have to provide the full path to the repository.
-
-
 ### Restore MongoDB
 
 These commands will erase the content from the database and replace it with version from the backup.
@@ -101,7 +98,7 @@ mongorestore --db {database} --drop {database}
 
 ## Encryption keys
 
-If you enabled encryption, Attic stores the encryption keys in `~/.attic/keys`. Remember to backup them separately; you won't be able to use your backup if you loose them!
+If you enabled encryption, Borg stores the encryption keys in `~/.config/borg/keys`. All keys are encrypted with password `backup`. The key to the security is to hold them in secure location. Remember to backup them separately; you won't be able to use your backup if you loose them!
 
 
 

backup.sh

@@ -89,7 +89,7 @@ fi
 
 run_sidescripts premount
 
-ATTIC_OPTIONS=""
+BORG_OPTIONS=""
 
 if [[ ${SIDE_SCRIPTS_ONLY} -eq 0 ]]
 then
@@ -103,18 +103,20 @@ then
 
     out="${DST_LOCAL_LOCATION}/${HOSTNAME}"
     ${ECHO} mkdir -p ${out}
-    out+='/repository.attic'
-    export ATTIC_REPOSITORY_PATH="${out}"
+    out+='/repository.borg'
+    export BORG_REPOSITORY_PATH="${out}"
 
     if [[ "${VERBOSE}" -ne 0 ]]
     then
-        ATTIC_OPTIONS+=" --stats"
-        echo "** starting attic backup"
+        BORG_OPTIONS+=" --stats"
+        echo "** starting Borg backup"
     fi
 
-    if [ ! -d ${ATTIC_REPOSITORY_PATH} ]
+    export BORG_PASSPHRASE='backup'
+
+    if [ ! -d ${BORG_REPOSITORY_PATH} ]
     then
-        ${ECHO} attic init --encryption=${ENCRYPTION_MODE} ${ATTIC_REPOSITORY_PATH}
+        ${ECHO} borg init --encryption=${ENCRYPTION_MODE} ${BORG_REPOSITORY_PATH}
     fi
 
     for directory in ${SRC_DIRECTORIES}
@@ -144,10 +146,10 @@ then
             fi
         done
 
-        ${ECHO} ionice -c3 -t attic create ${ATTIC_OPTIONS} ${ATTIC_REPOSITORY_PATH}::$(full_date)-${prefix} ${directory} ${EXCLUDE_LIST}
+        ${ECHO} ionice -c3 -t borg create ${BORG_OPTIONS} ${BORG_REPOSITORY_PATH}::$(full_date)-${prefix} ${directory} ${EXCLUDE_LIST}
     done
 
-    ${ECHO} ionice -c3 -t attic prune ${ATTIC_OPTIONS} ${ATTIC_REPOSITORY_PATH} ${ATTIC_PRUNE_AGES}
+    ${ECHO} ionice -c3 -t borg prune ${BORG_OPTIONS} ${BORG_REPOSITORY_PATH} ${BORG_PRUNE_AGES}
 
     run_sidescripts postdump
 

conf.d/copy_attic_repository_rsync.sh.example

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+export POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_ENABLED=1
+
+export POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_USER=backup
+export POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_HOST=backup.example.org
+export POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_PORT=22
+
+export POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_PATH=/data/backup

conf.d/example-copy_borg_repository_rsync.sh

@@ -17,14 +17,15 @@ export DST_LOCAL_LOCATION='/tmp/backup'
 
 # You should set whether you want to encrypt your backup
 # valid values are: none, keyfile
+# don't use repokey mode since standard password 'backup' is provided to Borg calls
 #export ENCRYPTION_MODE=none
 export ENCRYPTION_MODE=keyfile
 
 # they should match with the filesystem mount points
 export SRC_DIRECTORIES="/ /boot"
 
-export ATTIC_EXCLUDE_FILE="excludes.txt"
+export BORG_EXCLUDE_FILE="excludes.txt"
 
-export ATTIC_PRUNE_AGES="--keep-within=14d --keep-weekly=8 --keep-monthly=6"
+export BORG_PRUNE_AGES="--keep-within=14d --keep-weekly=8 --keep-monthly=6"
 
 

conf.d/excludes.txt.example renamed to conf.d/example-excludes.txt

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+cd conf.d
+
+if [ -f copy_attic_repository_rsync.sh ]; then
+  mv copy_attic_repository_rsync.sh copy_borg_repository_rsync.sh
+fi
+
+sed -i -- 's/ATTIC/BORG/g' *.sh

conf.d/ftp.sh.example renamed to conf.d/example-ftp.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+
+RSYNC_OPTIONS="-rzx --safe-links --delete"
+
+[[ ${VERBOSE} -ne 0 ]] && RSYNC_OPTIONS+=" --progress --stats --verbose -h "
+
+userAndHost="${POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_USER}@${POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_HOST}"
+outputDirectory="${POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_PATH}/${HOSTNAME}"
+
+ssh -p ${POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_PORT} ${userAndHost} "mkdir -p ${outputDirectory}"
+
+rsync ${RSYNC_OPTIONS} -e "ssh -p ${POSTDUMP_COPY_BORG_REPOSITORY_RSYNC_SSH_PORT}" ${BORG_REPOSITORY_PATH} ${userAndHost}:${outputDirectory}
+