Add media backup script

This commit is contained in:
Markus Koch 2022-03-24 11:16:05 +01:00
commit 578953beff
2 changed files with 175 additions and 0 deletions

80
README.MD Normal file
View File

@ -0,0 +1,80 @@
# LTO Backup Scripts
Scripts to help me perform my backups to Ultrium LTO tapes.
## Movies Library
* `mkbak_media.sh`: Incremental subdirectory backup. Take each directory in my cwd, tar it, and append it to tape, one tape entry per directory. Create / extend a CSV index denoting when and to what tape ID the backup was made. Do not backup directories that were already backed up to other media (using the index). Note: Changes in directories will not be detected. The directory name is the only clue.
# LTO Notes
## General stuff
* dev/nst does not auto-rewind, dev/st does
* use mt-st seek / tell to position the tape at block indexes
* Remaining capacity: https://serverfault.com/questions/853302/determine-remaining-capacity-of-lto-tape
* `sg_logs -a $DEV | sed -n 's/\s*Main partition remaining capacity (in MiB):\s*\(.*\)/\1/p'`
* Disable compression: `mt-st -f /dev/nst0 compression 0`
* Make seek/tell work: `mt-st -f /dev/nst0 stsetoptions scsi2logical`
## Improve start/stop
If you have a slow source / destination, consider using mbuffer with a size of a couple of Gigs. But that might also break speed matching, causing start / stop rather than going slower... idk.
## Backups
### Encrypted tar using GPG
`tar cvO --exclude dir/excludeme dir/ | gpg --encrypt --sign --default-key KEY-ID -r KEY-ID -z 0 dd of=/dev/nst0 bs=512k`
To decrypt, simply use `gpg --decrypt /dev/nst0`
# LTFS on LTO-6
My drives are a little weird, to get it to work I need to edit LTFS sources.
`ltfs-2.4.3.1-10461/src/tape_drivers/linux/sg/sg_tape.c`: Type is incorrectly auto detected, we need to always `/* Use LogPage 0x31 */`
See this patch for inspiration
```
diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c
index f583079..1169e44 100644
--- a/src/tape_drivers/linux/sg/sg_tape.c
+++ b/src/tape_drivers/linux/sg/sg_tape.c
@@ -513,6 +513,12 @@ static int _raw_open(struct sg_data *priv)
return -EDEV_DEVICE_UNSUPPORTABLE; /* Unsupported device */
}
+ /* Special care about Quantum model B */
+ if (priv->vendor == VENDOR_QUANTUM) {
+ if (id_data.product_id[7] == ' ')
+ priv->vendor = VENDOR_QUANTUM_B;
+ }
+
if (priv->drive_serial[0]) {
/* if serial number is already set, compare it */
if (strcmp(priv->drive_serial, id_data.unit_serial)) {
@@ -2969,7 +2975,8 @@ int sg_remaining_capacity(void *device, struct tc_remaining_cap *cap)
memset(buffer, 0, LOGSENSEPAGE);
if ((IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) ||
- (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))) {
+ (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06)) ||
+ (priv->vendor == VENDOR_QUANTUM_B && IS_LTO(priv->drive_type))) {
/* Use LogPage 0x31 */
ret = sg_logsense(device, (uint8_t)LOG_TAPECAPACITY, (uint8_t)0, (void *)buffer, LOGSENSEPAGE);
diff --git a/src/tape_drivers/tape_drivers.h b/src/tape_drivers/tape_drivers.h
index a9d7bf5..6e11771 100644
--- a/src/tape_drivers/tape_drivers.h
+++ b/src/tape_drivers/tape_drivers.h
@@ -178,6 +178,7 @@ enum {
VENDOR_IBM,
VENDOR_HP,
VENDOR_QUANTUM,
+ VENDOR_QUANTUM_B,
};
enum {
```

95
mkbak_media.sh Normal file
View File

@ -0,0 +1,95 @@
#!/bin/bash
MEDIA_ID="$1"
MODE="append"
DEV="/dev/nst0"
LIB="./lib.csv"
if [ "$2" != "" ]; then
MODE="$2"
fi
function info()
{
echo -e "\e[34m[I] $1\e[0m"
}
function warn()
{
echo -e "\e[33m[W] $1\e[0m"
}
function err()
{
echo -e "\e[31m[E] $1\e[0m"
}
function succ()
{
echo -e "\e[32m[ ] $1\e[0m"
}
if [ "$MEDIA_ID" == "" ]; then
echo "usage: $0 <media ID> [rewind]"
exit
fi
info "Backing up new directories to $MEDIA_ID in $DEV."
if [ ! -f "$LIB" ]; then
echo "Directory;Media ID;Date" > $LIB
fi
info "Configuring tape drive"
# mt-st -f $DEV stsetoptions scsi2logical
mt-st -f $DEV compression 0
if [ "$MODE" == "append" ]; then
info "Moving tape to end..."
mt-st -f $DEV eod
else
warn "Rewinding tape. This will delete all existing data on tape. Are you sure? [ctrl+c to cancel]"
read a
mt-st -f $DEV rewind
fi
skipped=0
for dir in *; do
if [ ! -d "$dir" ]; then
continue
fi
exist=`cat "$LIB" | grep "^$dir;"`
if [ "$exist" != "" ]; then
info "Skipping $dir. Already backed up: $exist"
continue
fi
echo ""
info "Attempting to backup $dir"
lsize=`du -s "$dir" | sed "s/\s.*//"`
lsize=$(($lsize / 1024))
rsize=`sg_logs -a $DEV | sed -n 's/\s*Main partition remaining capacity (in MiB):\s*\(.*\)/\1/p'`
if [ $lsize -gt $rsize ]; then
skipped=$((skipped+1))
warn "Not enough space left on device (${rsize} MB remaining, ${lsize} MB required). Trying next directory."
continue;
fi
info "Appending $dir to tape ($lsize MB, $rsize MB available)"
tar cvO "$dir" | dd of=$DEV bs=512k
if [ $? -ne 0 ]; then
err "An error occured writing the backup to tape. Aborting."
exit
fi
date=`date`
echo "$dir;$MEDIA_ID;$date" >> $LIB
done
info "Rewinding and ejecting tape"
mt-st -f $DEV rewoff
echo ""
if [ $skipped -eq 0 ]; then
succ "Backup complete. All directories are on tape."
else
succ "Tape complete, but there is more data to be backed up."
warn "Insert a new tape and rerun the script to complete the backup."
fi