Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 howard
|
618c196bf0 | ||
 Codex
|
d73e00db1c | ||
|
Codex
|
a1ce3b19bc | ||
|
Codex
|
1bddfe2b3d | ||
|
Codex
|
e27387bd93 | ||
|
Codex
|
261e88f5dc |
No files matched your search
Executable → Regular
+35
-5
@@ -1,7 +1,37 @@
|
|||||||
# /etc/dockerbackup/.env
|
# Copy to .env and customise on each Docker server. Keep .env out of Git.
|
||||||
RCLONE_REMOTE="gdrive:backuplocation/sublocation"
|
SOURCE_BASE_DIR="/dockervolumes"
|
||||||
SOURCE_DIR="/mybackups"
|
BACKUP_ROOT="/dockerbackups"
|
||||||
PRUNE_ROOT="/mybackups"
|
CONTAINERS_FILE="containers.local.txt"
|
||||||
OUT_DIR="/tmp"
|
BACKUP_HOSTNAME="beaver"
|
||||||
|
|
||||||
|
# Main pipeline method order. Add future method names here.
|
||||||
|
BACKUP_METHODS="rclone smb"
|
||||||
KEEP_COUNT=5
|
KEEP_COUNT=5
|
||||||
|
PULL_IMAGES=1
|
||||||
|
START_TIMEOUT=90
|
||||||
|
WAIT_INTERVAL=3
|
||||||
|
START_RETRIES=1
|
||||||
|
LOCK_FILE="/run/lock/zorruno-docker-backup.lock"
|
||||||
ZSTD_EXTRA_OPTS="-T0 -12"
|
ZSTD_EXTRA_OPTS="-T0 -12"
|
||||||
|
|
||||||
|
# Optional retained MQTT status and command listener settings.
|
||||||
|
MQTT_REPORT_ENABLED=0
|
||||||
|
MQTT_HOST="beaver"
|
||||||
|
MQTT_PORT=1883
|
||||||
|
MQTT_USERNAME="dockerbackup"
|
||||||
|
MQTT_PASSWORD_FILE="/etc/zorruno-docker-backup/mqtt-password"
|
||||||
|
MQTT_STATUS_TOPIC="viewroad-status/dockerbackup/beaver"
|
||||||
|
MQTT_COMMAND_TOPIC="viewroad-commands/dockerbackup/beaver/run"
|
||||||
|
|
||||||
|
# Rclone destination becomes <remote>/beaver/.
|
||||||
|
RCLONE_ENABLED=0
|
||||||
|
RCLONE_REMOTE="gdrive:dockerbackups"
|
||||||
|
RCLONE_HOST_SUBDIR=1
|
||||||
|
RCLONE_EXTRA_ARGS=""
|
||||||
|
|
||||||
|
# The SMB share must already be mounted by the operating system.
|
||||||
|
# Destination becomes <mount>/beaver/ when SMB_HOST_SUBDIR=1.
|
||||||
|
SMB_ENABLED=0
|
||||||
|
SMB_MOUNT="/mnt/dockerbackups"
|
||||||
|
SMB_HOST_SUBDIR=1
|
||||||
|
SMB_RSYNC_ARGS="-a"
|
||||||
@@ -1 +1,2 @@
|
|||||||
.env
|
.env
|
||||||
|
containers.local.txt
|
||||||
@@ -1,83 +1,210 @@
|
|||||||
# Docker Backup Scripts
|
# Zorruno Docker Backup Scripts
|
||||||
|
|
||||||
Simple shell scripts for backing up Docker-related data and pruning old backups.
|
Modular backup scripts for Docker Compose servers such as Panda and Beaver.
|
||||||
|
Each server has its own untracked `.env` and `containers.txt`, while all script
|
||||||
|
logic remains common and can be updated through Git.
|
||||||
|
|
||||||
## Files
|
## Pipeline
|
||||||
|
|
||||||
- `docker-backup.sh` - create backups
|
`backup.sh` runs these stages in order:
|
||||||
- `dockerbackup_prune.sh` - prune old backups
|
|
||||||
- `containers.txt` - list of containers or backup targets
|
|
||||||
- `.env.example` - example environment file
|
|
||||||
|
|
||||||
## Reqirements
|
1. `stages/10-containers.sh` — stop, archive, pull, start and verify projects.
|
||||||
|
2. `methods/20-rclone.sh` — copy archives to an rclone destination when enabled.
|
||||||
|
3. `methods/30-smb.sh` — copy archives to a mounted SMB share when enabled.
|
||||||
|
4. `stages/90-prune.sh` — retain the newest configured number of `.zst`
|
||||||
|
archives locally, including legacy `.zst` and current `.tar.zst` names.
|
||||||
|
|
||||||
docker-compose
|
Every stage can also be run directly. The old `docker-backup.sh` name invokes
|
||||||
|
the container archive stage, while `dockerbackup_prune.sh` invokes the local
|
||||||
|
prune stage. Use the new `backup.sh` for the complete pipeline.
|
||||||
|
|
||||||
## BACKUPS: Example use docker-backup.sh
|
## Requirements
|
||||||
|
|
||||||
Configure the backup script, docker-backup.sh
|
- Bash 4 or later
|
||||||
Set up your list of containers in container.txt . The script assumes each container is in a subdirectory of one main docker directory
|
- Docker with `docker compose`, or legacy `docker-compose`
|
||||||
|
- GNU `tar` with zstd support
|
||||||
|
- `flock`
|
||||||
|
- `rclone` only when the rclone method is enabled
|
||||||
|
- `rsync` and `mountpoint` only when the SMB method is enabled
|
||||||
|
|
||||||
Edit the location of where you want docker backups stored in docker-backup.sh (default is "/dockerbackups") or use
|
The account running the scripts must be allowed to control Docker and read all
|
||||||
|
files under `/dockervolumes`.
|
||||||
|
|
||||||
./docker-backup.sh -s /home/me/mycontainers
|
## Installation on Beaver
|
||||||
|
|
||||||
**Actions:**
|
```bash
|
||||||
- It will recurse though the directories if they are listed in containers.txt
|
sudo git clone https://home.fox.co.nz/gitea/zorruno/zorruno-backupdockerscript.git /backupdockerscripts
|
||||||
- It will run docker-compose to take down the container, pull a new version if there is one, then bring the container back up.
|
cd /backupdockerscripts
|
||||||
- it will create a timestamped archive of each container in the backup directory, under a directory with that container name.
|
sudo cp .env.example .env
|
||||||
|
sudo cp containers.txt containers.local.txt
|
||||||
```
|
sudo chmod 600 .env
|
||||||
Usage: ./docker-backup.sh [-f containers_list_file] [-s source_base_dir] [-t] [-v] [-h]
|
sudo chmod +x backup.sh docker-backup.sh dockerbackup_prune.sh stages/*.sh methods/*.sh
|
||||||
|
|
||||||
Flags:
|
|
||||||
-t Test (dry run; no backups performed)
|
|
||||||
-v Verbose (debug logging to stderr)
|
|
||||||
-f FILE File containing list of container names
|
|
||||||
Default: '$containersfile'
|
|
||||||
-s DIR Base directory containing container/project folders
|
|
||||||
Default: '$SOURCE_BASE_DIR'
|
|
||||||
-h Show this help
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## PRUNE and REMOTE BACKUP dockerbackup_prune.sh
|
Edit `.env`. Beaver's Docker source is already represented by:
|
||||||
|
|
||||||
Set up the env file using the example.
|
```bash
|
||||||
```cp .env.example .env```
|
SOURCE_BASE_DIR="/dockervolumes"
|
||||||
|
BACKUP_HOSTNAME="beaver"
|
||||||
You don't need to worry about the rclone info (backups to google drive) it you won't be using that.
|
|
||||||
It will just backup up the file locally. Feel free to adjust for other remote backup systems.
|
|
||||||
|
|
||||||
If you run .dockerbackup_prune.sh it will prune the number of backup archives that are already there (based on your prune numbers), make a .zst archive of all container backups
|
|
||||||
then finally uploades to google drive and deletes the large archive (if that's what you set).
|
|
||||||
Then it will prune the number of archive files for each
|
|
||||||
|
|
||||||
**local archive only (no pruning)**
|
|
||||||
```./dockerbackup_prune.sh```
|
|
||||||
|
|
||||||
**local archive + real prune, keep new archive locally. Default prune is 5 archives)**
|
|
||||||
```./dockerbackup_prune.sh --apply```
|
|
||||||
|
|
||||||
**local archive + upload + prune dry-run (no actual prune), keep new archive locally (default location is /tmp)**
|
|
||||||
```./dockerbackup_prune.sh --rclone```
|
|
||||||
|
|
||||||
**full remote archive, google drive upload + real prune (default 5) + delete new local archive**
|
|
||||||
```./dockerbackup_prune.sh --rclone --apply```
|
|
||||||
|
|
||||||
**Full remote workflow with actual prune leaving 3 archives of each container and final full archive delete.**
|
|
||||||
**Specify location of docker archives, final archive dir and remote google drive folder.**
|
|
||||||
```./dockerbackup_prune.sh --root /mydockerarchives --/mybigbackupdolder --keep 3 --rclone /dockergooglebackup --apply
|
|
||||||
|
|
||||||
Usage: $0 [--apply] [--quiet] [--dry-run] [--keep N] [--root PATH] [--source PATH] [--outdir PATH] [--remote PATH]
|
|
||||||
|
|
||||||
Flags:
|
|
||||||
--rclone Enable rclone upload flow
|
|
||||||
--apply Actually delete pruned files; with --rclone also delete the new local archive
|
|
||||||
--quiet Minimal console output (no live progress)
|
|
||||||
--dry-run Simulate everything (no backup, no upload, no deletes)
|
|
||||||
--keep N Keep newest N .zst files per subfolder (default ${KEEP_COUNT})
|
|
||||||
--root PATH Root folder to prune (default ${PRUNE_ROOT})
|
|
||||||
--source PATH Folder to archive (default ${SOURCE_DIR})
|
|
||||||
--outdir PATH Where to write the .zst (default ${OUT_DIR})
|
|
||||||
--remote PATH rclone destination (default ${RCLONE_REMOTE})
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Choose and create a local staging path for `BACKUP_ROOT`. `/dockerbackups` is
|
||||||
|
the default, but it should be on a filesystem with enough free space to hold
|
||||||
|
at least `KEEP_COUNT` copies of every listed project.
|
||||||
|
|
||||||
|
Edit Beaver's untracked `containers.local.txt`. Each active line can be either a running
|
||||||
|
container name or a directory name beneath `/dockervolumes`. Blank lines and
|
||||||
|
text after `#` are ignored. If several entries resolve to the same Compose
|
||||||
|
project directory, that project is backed up only once. Keeping this host-local
|
||||||
|
file out of Git prevents a pull on Panda or Beaver from replacing its list.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Copy `.env.example` to `.env`; `.env` is ignored by Git. Important values:
|
||||||
|
|
||||||
|
| Variable | Purpose |
|
||||||
|
|---|---|
|
||||||
|
| `SOURCE_BASE_DIR` | Parent of Compose project directories |
|
||||||
|
| `BACKUP_ROOT` | Local archive staging and retention tree |
|
||||||
|
| `CONTAINERS_FILE` | Absolute path, or path relative to this repository |
|
||||||
|
| `BACKUP_HOSTNAME` | Host folder/name embedded in archive filenames |
|
||||||
|
| `BACKUP_METHODS` | Ordered method names, default `rclone smb` |
|
||||||
|
| `ZSTD_EXTRA_OPTS` | Options passed to `zstd`, default `-T0 -12` |
|
||||||
|
| `KEEP_COUNT` | Local archives retained per Compose project |
|
||||||
|
| `PULL_IMAGES` | Pull images before restarting (`1` or `0`) |
|
||||||
|
|
||||||
|
### Rclone
|
||||||
|
|
||||||
|
```bash
|
||||||
|
RCLONE_ENABLED=1
|
||||||
|
RCLONE_REMOTE="gdrive:dockerbackups"
|
||||||
|
```
|
||||||
|
|
||||||
|
Archives are copied beneath `<remote>/<BACKUP_HOSTNAME>/`. The method uses
|
||||||
|
`rclone copy`, so it does not delete older destination files. Set
|
||||||
|
`RCLONE_HOST_SUBDIR=0` when `RCLONE_REMOTE` already names the final host-specific
|
||||||
|
destination; otherwise the host name is appended automatically.
|
||||||
|
|
||||||
|
### SMB
|
||||||
|
|
||||||
|
Mount the SMB share through `/etc/fstab` or a systemd mount unit; do not put SMB
|
||||||
|
credentials in this repository. Then configure:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
SMB_ENABLED=1
|
||||||
|
SMB_MOUNT="/mnt/dockerbackups"
|
||||||
|
SMB_HOST_SUBDIR=1
|
||||||
|
```
|
||||||
|
|
||||||
|
The script refuses to copy unless `SMB_MOUNT` is an actual mount point. This
|
||||||
|
prevents a failed network mount from silently filling the server's root disk.
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
Run the complete pipeline:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo /backupdockerscripts/backup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Preview all stages without stopping containers, copying, or deleting:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo /backupdockerscripts/backup.sh --dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
Select a different container list for one complete pipeline run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo /backupdockerscripts/backup.sh --containers containers-special.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
Relative container-list paths are resolved from the repository directory;
|
||||||
|
absolute paths are also accepted. The selected list affects only the container
|
||||||
|
archive stage, after which configured transfer methods and pruning continue.
|
||||||
|
|
||||||
|
Run individual parts:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo stages/10-containers.sh --dry-run
|
||||||
|
sudo methods/20-rclone.sh --dry-run --force
|
||||||
|
sudo methods/30-smb.sh --dry-run --force
|
||||||
|
sudo stages/90-prune.sh --dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
`--force` runs a transfer method even when its `*_ENABLED` value is false. It
|
||||||
|
still requires the method's destination to be configured and available.
|
||||||
|
|
||||||
|
## MQTT reporting and Home Assistant trigger
|
||||||
|
|
||||||
|
Enable retained run results in the host's `.env`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
MQTT_REPORT_ENABLED=1
|
||||||
|
MQTT_HOST="beaver"
|
||||||
|
MQTT_PORT=1883
|
||||||
|
MQTT_USERNAME="dockerbackup"
|
||||||
|
MQTT_PASSWORD_FILE="/etc/zorruno-docker-backup/mqtt-password"
|
||||||
|
MQTT_STATUS_TOPIC="viewroad-status/dockerbackup/panda"
|
||||||
|
MQTT_COMMAND_TOPIC="viewroad-commands/dockerbackup/panda/run"
|
||||||
|
```
|
||||||
|
|
||||||
|
The retained status JSON includes overall status, timestamps, duration, exit
|
||||||
|
code, container totals, successful backups, dry-run counts, and error messages.
|
||||||
|
Final reporting runs from an exit trap, so failures in any earlier stage are
|
||||||
|
reported when MQTT remains available.
|
||||||
|
|
||||||
|
Install the optional root-run command listener with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp systemd/zorruno-docker-backup-mqtt.service /etc/systemd/system/
|
||||||
|
systemctl daemon-reload
|
||||||
|
systemctl enable --now zorruno-docker-backup-mqtt.service
|
||||||
|
```
|
||||||
|
|
||||||
|
It accepts only JSON containing the exact action `run` on the configured
|
||||||
|
command topic:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"action":"run"}
|
||||||
|
```
|
||||||
|
|
||||||
|
Publish commands without MQTT retain. The listener also ignores retained
|
||||||
|
messages, preventing an old command from starting another backup after a
|
||||||
|
listener or broker restart.
|
||||||
|
|
||||||
|
Restrict publish access to the command topic with broker ACLs. The listener
|
||||||
|
runs as root because the backup stage controls Docker containers. The existing
|
||||||
|
pipeline lock prevents concurrent runs.
|
||||||
|
|
||||||
|
## Adding another backup method
|
||||||
|
|
||||||
|
Add an executable script named `methods/NN-name.sh`, for example
|
||||||
|
`methods/40-restic.sh`. It should load `lib/common.sh`, accept `--dry-run`, and
|
||||||
|
return a non-zero status on failure. Add its short name to `.env`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BACKUP_METHODS="rclone smb restic"
|
||||||
|
```
|
||||||
|
|
||||||
|
The main pipeline discovers the numbered file from the short name and runs it
|
||||||
|
after the earlier methods. A failed method stops the pipeline, so pruning will
|
||||||
|
not run after a failed transfer.
|
||||||
|
|
||||||
|
## Archive and restore format
|
||||||
|
|
||||||
|
Archives are stored as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<BACKUP_ROOT>/<project>/<host>.<project>.<date_time>.tar.zst
|
||||||
|
```
|
||||||
|
|
||||||
|
Each archive contains the project's top-level directory rather than absolute
|
||||||
|
paths. Example restore into an empty temporary directory:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p /tmp/docker-restore
|
||||||
|
tar --zstd -xf backup.tar.zst -C /tmp/docker-restore
|
||||||
|
```
|
||||||
|
|
||||||
|
Inspect and verify restored content before replacing a live project.
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Main pipeline: container archives, configured methods, then local prune.
|
||||||
|
set -Eeuo pipefail
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
|
. "$SCRIPT_DIR/lib/common.sh"
|
||||||
|
DRY_RUN=0; SKIP_CONTAINERS=0; SKIP_PRUNE=0; CONTAINERS_OVERRIDE=""
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--dry-run|-t) DRY_RUN=1 ;; --skip-containers) SKIP_CONTAINERS=1 ;; --skip-prune) SKIP_PRUNE=1 ;;
|
||||||
|
--containers|-f) CONTAINERS_OVERRIDE="${2:?Missing containers file}"; shift ;;
|
||||||
|
-h|--help)
|
||||||
|
echo "Usage: ${0##*/} [--dry-run] [--containers FILE] [--skip-containers] [--skip-prune]"
|
||||||
|
echo "Methods run in BACKUP_METHODS order. Add methods/NN-name.sh and list name in BACKUP_METHODS."
|
||||||
|
exit 0 ;;
|
||||||
|
*) die "Unknown option: $1" 2 ;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
|
||||||
|
BACKUP_RUN_ID="$(run_id)"
|
||||||
|
BACKUP_STARTED_EPOCH="$(date +%s)"
|
||||||
|
BACKUP_RUN_LOG="$(mktemp "${TMPDIR:-/tmp}/zorruno-docker-backup.XXXXXX.log")"
|
||||||
|
export BACKUP_RUN_ID BACKUP_STARTED_EPOCH BACKUP_RUN_LOG
|
||||||
|
|
||||||
|
finish_run() {
|
||||||
|
local exit_code=$? finished status
|
||||||
|
trap - EXIT
|
||||||
|
set +e
|
||||||
|
finished="$(date +%s)"
|
||||||
|
if (( exit_code == 0 )); then
|
||||||
|
if (( DRY_RUN )); then status="dry_run_success"; else status="success"; fi
|
||||||
|
else
|
||||||
|
if (( DRY_RUN )); then status="dry_run_failed"; else status="failed"; fi
|
||||||
|
fi
|
||||||
|
if is_true "$MQTT_REPORT_ENABLED"; then
|
||||||
|
"$SCRIPT_DIR/stages/99-mqtt-report.sh" "$status" "$exit_code" \
|
||||||
|
"$BACKUP_STARTED_EPOCH" "$finished" "$BACKUP_RUN_ID" "$BACKUP_RUN_LOG" \
|
||||||
|
|| log ERROR "Unable to publish final MQTT result"
|
||||||
|
fi
|
||||||
|
rm -f -- "$BACKUP_RUN_LOG"
|
||||||
|
exit "$exit_code"
|
||||||
|
}
|
||||||
|
trap finish_run EXIT
|
||||||
|
|
||||||
|
require_command flock
|
||||||
|
mkdir -p "$(dirname "$LOCK_FILE")"
|
||||||
|
exec 9>"$LOCK_FILE"
|
||||||
|
flock -n 9 || die "Another backup run holds lock: $LOCK_FILE"
|
||||||
|
stage_args=()
|
||||||
|
(( DRY_RUN )) && stage_args+=(--dry-run)
|
||||||
|
[[ -n "$CONTAINERS_OVERRIDE" ]] && stage_args+=(--containers "$CONTAINERS_OVERRIDE")
|
||||||
|
log START "Backup pipeline for $BACKUP_HOSTNAME"
|
||||||
|
if is_true "$MQTT_REPORT_ENABLED"; then
|
||||||
|
"$SCRIPT_DIR/stages/99-mqtt-report.sh" running 0 "$BACKUP_STARTED_EPOCH" \
|
||||||
|
"$BACKUP_STARTED_EPOCH" "$BACKUP_RUN_ID" "$BACKUP_RUN_LOG" \
|
||||||
|
|| log WARN "Unable to publish MQTT running state; backup will continue"
|
||||||
|
fi
|
||||||
|
(( SKIP_CONTAINERS )) || "$SCRIPT_DIR/stages/10-containers.sh" "${stage_args[@]}"
|
||||||
|
method_args=(); (( DRY_RUN )) && method_args+=(--dry-run)
|
||||||
|
for method in $BACKUP_METHODS; do
|
||||||
|
method_script=""
|
||||||
|
while IFS= read -r candidate; do method_script="$candidate"; break; done < <(find "$SCRIPT_DIR/methods" -maxdepth 1 -type f -name "*-${method}.sh" -print | sort)
|
||||||
|
[[ -n "$method_script" ]] || die "Configured backup method not found: $method"
|
||||||
|
"$method_script" "${method_args[@]}"
|
||||||
|
done
|
||||||
|
(( SKIP_PRUNE )) || "$SCRIPT_DIR/stages/90-prune.sh" "${method_args[@]}"
|
||||||
|
log DONE "Backup pipeline completed successfully"
|
||||||
Executable
+24
@@ -0,0 +1,24 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Listen for an authenticated JSON command and run the backup pipeline.
|
||||||
|
set -Eeuo pipefail
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd -P)"
|
||||||
|
. "$SCRIPT_DIR/lib/common.sh"
|
||||||
|
. "$SCRIPT_DIR/lib/mqtt.sh"
|
||||||
|
|
||||||
|
require_command jq
|
||||||
|
require_command mosquitto_sub
|
||||||
|
[[ -n "$MQTT_HOST" ]] || die "MQTT_HOST is not configured"
|
||||||
|
mqtt_common_args
|
||||||
|
log MQTT "Listening on $MQTT_COMMAND_TOPIC"
|
||||||
|
|
||||||
|
# Never replay a retained command after this listener or the MQTT broker restarts.
|
||||||
|
# Home Assistant should also publish run commands without the retain flag.
|
||||||
|
mosquitto_sub "${MQTT_ARGS[@]}" -R -t "$MQTT_COMMAND_TOPIC" | while IFS= read -r payload; do
|
||||||
|
action="$(jq -er '.action // empty' <<<"$payload" 2>/dev/null || true)"
|
||||||
|
if [[ "$action" != "run" ]]; then
|
||||||
|
log WARN "Ignored invalid MQTT command"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
log MQTT "Accepted backup run command"
|
||||||
|
"$SCRIPT_DIR/backup.sh" || log ERROR "MQTT-triggered backup failed"
|
||||||
|
done
|
||||||
@@ -55,4 +55,3 @@ ttyd
|
|||||||
#weewx
|
#weewx
|
||||||
whoami
|
whoami
|
||||||
wireguard
|
wireguard
|
||||||
zigbee2mqtt
|
|
||||||
Regular → Executable
+3
-376
@@ -1,378 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# ORIGINAL AUTHOR: kmw (updates by zorruno)
|
# Compatibility entry point for stage A. Use backup.sh for the full pipeline.
|
||||||
# HISTORY:
|
|
||||||
# 2026-04-20 - add configurable SOURCE_BASE_DIR and -s option for source path fallback.
|
|
||||||
# 2025-08-22 - add start wait with one retry; fix find_working_dir typo; tidy.
|
|
||||||
# 2025-08-22 - tidy, stricter bash, compose detection, better checks, clearer logging.
|
|
||||||
# 2022-03-25 - add -t option for dry-run (no backups performed).
|
|
||||||
# 2022-01-15 - fix error reporting when a container is not found.
|
|
||||||
# 2022-01-09 - ignore trailing comments
|
|
||||||
# 2022-01-03 - display help; list from file; -f option
|
|
||||||
# 2021-10-16 - structure and logging
|
|
||||||
# 2020-05-ish - first version
|
|
||||||
|
|
||||||
set -Eeuo pipefail
|
set -Eeuo pipefail
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
timestamp_format='%Y-%m-%d_%H-%M-%S'
|
exec "$SCRIPT_DIR/stages/10-containers.sh" "$@"
|
||||||
|
|
||||||
# verbosity: set non-empty to enable debug
|
|
||||||
dbg=
|
|
||||||
# dbg=VERBOSE
|
|
||||||
|
|
||||||
# default containers list file
|
|
||||||
containersfile="containers.txt"
|
|
||||||
|
|
||||||
# dry run (or pass -t)
|
|
||||||
dryrun=
|
|
||||||
|
|
||||||
# base directory for backups
|
|
||||||
BACKUP_DIR="/dockerbackups"
|
|
||||||
|
|
||||||
# base directory for source docker project folders
|
|
||||||
SOURCE_BASE_DIR="${SOURCE_BASE_DIR:-/dockervolumes}"
|
|
||||||
|
|
||||||
# start/health wait tuning
|
|
||||||
START_TIMEOUT=90 # seconds to wait for service to be up
|
|
||||||
WAIT_INTERVAL=3 # seconds between polls
|
|
||||||
RETRIES=1 # number of restart retries on failure (per container)
|
|
||||||
|
|
||||||
# log file
|
|
||||||
logdir="$HOME/.local/share/log/${0##*/}"
|
|
||||||
mkdir -p "$logdir"
|
|
||||||
ymdhms="$(/bin/date +'%Y-%m-%d_%H:%M:%S')"
|
|
||||||
ymd="${ymdhms%%_*}"
|
|
||||||
logfile="$logdir/${0##*/}_${ymd}.log"
|
|
||||||
|
|
||||||
# compose command detection
|
|
||||||
COMPOSE_CMD=()
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_CMD=(docker compose)
|
|
||||||
elif command -v docker-compose >/dev/null 2>&1; then
|
|
||||||
COMPOSE_CMD=(docker-compose)
|
|
||||||
else
|
|
||||||
echo "ERROR: Neither 'docker compose' nor 'docker-compose' found." >&2
|
|
||||||
exit 3
|
|
||||||
fi
|
|
||||||
|
|
||||||
log() {
|
|
||||||
local timestamp
|
|
||||||
timestamp="$(/bin/date +"$timestamp_format")"
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
echo "$timestamp: ${0##*/}: $1" >>"$logfile"
|
|
||||||
if [ -n "$dbg" ]; then
|
|
||||||
echo "$timestamp LOG: ${0##*/}: $1" >&2
|
|
||||||
else
|
|
||||||
echo "$1" >&2
|
|
||||||
fi
|
|
||||||
shift
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
debug() {
|
|
||||||
local timestamp
|
|
||||||
timestamp="$(/bin/date +"$timestamp_format")"
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
echo "$timestamp DBG: ${0##*/}: $1" >>"$logfile"
|
|
||||||
if [ -n "$dbg" ]; then
|
|
||||||
echo "$timestamp DBG: ${0##*/}: $1" >&2
|
|
||||||
fi
|
|
||||||
shift
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
fail() {
|
|
||||||
local exitcode="$1"
|
|
||||||
shift
|
|
||||||
log "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
|
|
||||||
log "${0##*/}: FATAL ERROR"
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
log "$1"
|
|
||||||
shift
|
|
||||||
done
|
|
||||||
sleep 1
|
|
||||||
tidy_up
|
|
||||||
exit "$exitcode"
|
|
||||||
}
|
|
||||||
|
|
||||||
tidy_up() {
|
|
||||||
debug "tidy_up"
|
|
||||||
cd || log "Failed to cd in tidy_up"
|
|
||||||
trap - EXIT
|
|
||||||
trap - INT
|
|
||||||
}
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat 1>&2 <<EOF
|
|
||||||
Usage: ${0##*/} [-f containers_list_file] [-s source_base_dir] [-t] [-v] [-h]
|
|
||||||
|
|
||||||
-t Test (dry run; no backups performed)
|
|
||||||
-v Verbose (debug logging to stderr)
|
|
||||||
-f FILE File containing list of container names
|
|
||||||
Default: '$containersfile'
|
|
||||||
-s DIR Base directory containing container/project folders
|
|
||||||
Default: '$SOURCE_BASE_DIR'
|
|
||||||
-h Show this help
|
|
||||||
|
|
||||||
Example:
|
|
||||||
${0##*/} -f containers.servername.txt
|
|
||||||
${0##*/} -f containers.servername.txt -s /dockervolumes
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
parse_cmdline() {
|
|
||||||
while getopts ":f:hs:tv" opt; do
|
|
||||||
case "$opt" in
|
|
||||||
f) containersfile="$OPTARG" ;;
|
|
||||||
s) SOURCE_BASE_DIR="$OPTARG" ;;
|
|
||||||
h) usage; exit 0 ;;
|
|
||||||
t) dryrun=1 ;;
|
|
||||||
v) dbg=VERBOSE ;;
|
|
||||||
\?) echo "Invalid option: -$OPTARG" 1>&2; usage; exit 1 ;;
|
|
||||||
:) echo "Option -$OPTARG requires an argument" 1>&2; exit 1 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
shift $((OPTIND - 1))
|
|
||||||
}
|
|
||||||
|
|
||||||
toolcheck() {
|
|
||||||
local tool
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
tool="$1"
|
|
||||||
shift
|
|
||||||
debug "Checking for tool: '$tool'"
|
|
||||||
if ! command -v "$tool" >/dev/null 2>&1; then
|
|
||||||
fail 3 "Failed to find required tool: '$tool'"
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
compose() {
|
|
||||||
"${COMPOSE_CMD[@]}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Return "<project> <service>" for a container by inspecting labels
|
|
||||||
get_compose_identity() {
|
|
||||||
# arg: container_name
|
|
||||||
local name="$1"
|
|
||||||
local proj serv
|
|
||||||
proj="$(docker inspect "$name" 2>/dev/null | jq -r '.[0].Config.Labels["com.docker.compose.project"] // empty')"
|
|
||||||
serv="$(docker inspect "$name" 2>/dev/null | jq -r '.[0].Config.Labels["com.docker.compose.service"] // empty')"
|
|
||||||
if [ -z "$proj" ] || [ -z "$serv" ]; then
|
|
||||||
echo ""
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "$proj $serv"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Check current state/health for any container matching project+service
|
|
||||||
# Returns 0 if running and healthy (or no healthcheck), else 1
|
|
||||||
service_is_healthy() {
|
|
||||||
# args: project service
|
|
||||||
local proj="$1"
|
|
||||||
local serv="$2"
|
|
||||||
local ids
|
|
||||||
# find containers by labels (may be multiple if scaled; any healthy one counts)
|
|
||||||
ids="$(docker ps -a -q --filter "label=com.docker.compose.project=$proj" --filter "label=com.docker.compose.service=$serv")" || true
|
|
||||||
if [ -z "$ids" ]; then
|
|
||||||
debug "No containers found for project=$proj service=$serv"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
local id
|
|
||||||
for id in $ids; do
|
|
||||||
# status: created, restarting, running, removing, paused, exited, dead
|
|
||||||
local st hlth
|
|
||||||
st="$(docker inspect -f '{{.State.Status}}' "$id" 2>/dev/null || echo "unknown")"
|
|
||||||
if [ "$st" != "running" ]; then
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
# Health may not exist; treat missing as pass if running
|
|
||||||
hlth="$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$id" 2>/dev/null || echo "none")"
|
|
||||||
if [ "$hlth" = "none" ] || [ "$hlth" = "healthy" ]; then
|
|
||||||
debug "Container $id is running (health=$hlth)"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# Wait for service to become healthy/running within timeout
|
|
||||||
wait_for_service() {
|
|
||||||
# args: project service timeout_seconds
|
|
||||||
local proj="$1"
|
|
||||||
local serv="$2"
|
|
||||||
local timeout="$3"
|
|
||||||
local start now
|
|
||||||
start="$(date +%s)"
|
|
||||||
while true; do
|
|
||||||
if service_is_healthy "$proj" "$serv"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
now="$(date +%s)"
|
|
||||||
if [ $(( now - start )) -ge "$timeout" ]; then
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
sleep "$WAIT_INTERVAL"
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
down_backup_up() {
|
|
||||||
# args: WORKING_DIR PROJECT SERVICE
|
|
||||||
local WORKING_DIR="$1"
|
|
||||||
local PROJ="$2"
|
|
||||||
local SERV="$3"
|
|
||||||
|
|
||||||
local PROJECT_NAME
|
|
||||||
local BAK_YMDHMS
|
|
||||||
local BACKUP_CODE_DIR
|
|
||||||
local BACKUP_CODE_ZSTD
|
|
||||||
|
|
||||||
PROJECT_NAME="$(basename "$WORKING_DIR")"
|
|
||||||
BAK_YMDHMS="$(date +'%Y-%m-%d_%H-%M-%S')"
|
|
||||||
BACKUP_CODE_DIR="${BACKUP_DIR}/${PROJECT_NAME}"
|
|
||||||
BACKUP_CODE_ZSTD="${BACKUP_CODE_DIR}/$(hostname -s).${PROJECT_NAME}.${BAK_YMDHMS}.zst"
|
|
||||||
|
|
||||||
mkdir -p "${BACKUP_CODE_DIR}"
|
|
||||||
log "Backup target: ${BACKUP_CODE_ZSTD}"
|
|
||||||
|
|
||||||
if [ -n "$dryrun" ]; then
|
|
||||||
log "DRY-RUN: would run: compose down; tar --zstd; compose pull; compose up -d; wait for ${PROJ}/${SERV}"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
# stop, archive, start, then verify
|
|
||||||
compose down || log "WARNING: non-zero exit from 'compose down' (continuing)"
|
|
||||||
tar --one-file-system --zstd -cf "${BACKUP_CODE_ZSTD}" "${WORKING_DIR}"
|
|
||||||
ls -lh "${BACKUP_CODE_DIR}" || true
|
|
||||||
compose pull || log "WARNING: non-zero exit from 'compose pull' (continuing)"
|
|
||||||
compose up -d
|
|
||||||
|
|
||||||
if wait_for_service "$PROJ" "$SERV" "$START_TIMEOUT"; then
|
|
||||||
log "Service ${PROJ}/${SERV} is up."
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
log "Service ${PROJ}/${SERV} did not become healthy in ${START_TIMEOUT}s. Retrying ($RETRIES)."
|
|
||||||
local attempt=1
|
|
||||||
while [ "$attempt" -le "$RETRIES" ]; do
|
|
||||||
log "Retry $attempt: restarting ${PROJ}/${SERV}"
|
|
||||||
compose down || log "WARNING: retry: non-zero from 'compose down'"
|
|
||||||
compose up -d
|
|
||||||
if wait_for_service "$PROJ" "$SERV" "$START_TIMEOUT"; then
|
|
||||||
log "Service ${PROJ}/${SERV} is up after retry."
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
attempt=$(( attempt + 1 ))
|
|
||||||
done
|
|
||||||
|
|
||||||
fail 1 "Service ${PROJ}/${SERV} failed to start/healthy after ${RETRIES} retry(ies)."
|
|
||||||
}
|
|
||||||
|
|
||||||
find_working_dir() {
|
|
||||||
# args: CONTAINER_NAME
|
|
||||||
local name="$1"
|
|
||||||
local wd=""
|
|
||||||
if docker inspect "$name" >/dev/null 2>&1; then
|
|
||||||
wd="$(docker inspect "$name" 2>/dev/null | jq -r '.[0].Config.Labels["com.docker.compose.project.working_dir"] // empty')"
|
|
||||||
fi
|
|
||||||
if [ -z "$wd" ]; then
|
|
||||||
if [ -d "$SOURCE_BASE_DIR/$name" ]; then
|
|
||||||
wd="$SOURCE_BASE_DIR/$name"
|
|
||||||
elif [ -d "$HOME/prj/docker/$name" ]; then
|
|
||||||
wd="$HOME/prj/docker/$name"
|
|
||||||
else
|
|
||||||
wd=""
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
printf '%s' "$wd"
|
|
||||||
}
|
|
||||||
|
|
||||||
container_exists() {
|
|
||||||
docker ps -a --format '{{.Names}}' | grep -Fxq "$1"
|
|
||||||
}
|
|
||||||
|
|
||||||
backup_container() {
|
|
||||||
local container_name="$1"
|
|
||||||
log "Processing container '$container_name'"
|
|
||||||
|
|
||||||
local container_start_time container_finish_time container_duration container_duration_m
|
|
||||||
container_start_time="$(date +%s)"
|
|
||||||
|
|
||||||
if ! container_exists "$container_name"; then
|
|
||||||
log "Container not found: '$container_name' (skipping)"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
# capture compose identity before we stop anything
|
|
||||||
local identity proj serv
|
|
||||||
identity="$(get_compose_identity "$container_name" || true)"
|
|
||||||
if [ -z "$identity" ]; then
|
|
||||||
fail 1 "Could not determine compose identity (project/service) for '$container_name'."
|
|
||||||
fi
|
|
||||||
proj="$(echo "$identity" | awk '{print $1}')"
|
|
||||||
serv="$(echo "$identity" | awk '{print $2}')"
|
|
||||||
debug "Compose identity: project=$proj service=$serv"
|
|
||||||
|
|
||||||
local working_dir
|
|
||||||
working_dir="$(find_working_dir "$container_name")"
|
|
||||||
if [ -z "$working_dir" ]; then
|
|
||||||
fail 1 "Could not determine working_dir for container '$container_name'."
|
|
||||||
fi
|
|
||||||
log "Working dir: $working_dir"
|
|
||||||
if [ ! -d "$working_dir" ]; then
|
|
||||||
fail 1 "Working dir does not exist: '$working_dir'"
|
|
||||||
fi
|
|
||||||
|
|
||||||
local original_dir
|
|
||||||
original_dir="$(pwd)"
|
|
||||||
cd "$working_dir" || fail 1 "Could not cd to '$working_dir'"
|
|
||||||
|
|
||||||
down_backup_up "$working_dir" "$proj" "$serv" || log "WARNING: non-zero exit from backup function"
|
|
||||||
|
|
||||||
cd "$original_dir" || fail 1 "Could not cd back to '$original_dir'"
|
|
||||||
|
|
||||||
container_finish_time="$(date +%s)"
|
|
||||||
container_duration=$((container_finish_time - container_start_time))
|
|
||||||
container_duration_m=$((container_duration / 60))
|
|
||||||
log "Finished '$container_name' after ${container_duration}s (approx ${container_duration_m} min)"
|
|
||||||
}
|
|
||||||
|
|
||||||
trap tidy_up EXIT
|
|
||||||
trap 'exit 130' INT
|
|
||||||
|
|
||||||
parse_cmdline "$@"
|
|
||||||
|
|
||||||
debug "-----"
|
|
||||||
debug "logfile = '$logfile'"
|
|
||||||
debug "source_base_dir = '$SOURCE_BASE_DIR'"
|
|
||||||
|
|
||||||
toolcheck docker jq sed tar
|
|
||||||
|
|
||||||
if [ ! -f "$containersfile" ]; then
|
|
||||||
fail 2 "Containers file not found: '$containersfile'"
|
|
||||||
fi
|
|
||||||
if [ ! -r "$containersfile" ]; then
|
|
||||||
fail 2 "Containers file not readable: '$containersfile'"
|
|
||||||
fi
|
|
||||||
|
|
||||||
containers="$(sed -rn 's/^ +//; s/#.*//; s/ +$//; /^$/b; p' "$containersfile" || true)"
|
|
||||||
|
|
||||||
if [ -z "${containers:-}" ]; then
|
|
||||||
log "No containers listed in '$containersfile' (nothing to do)."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
script_start_time="$(date +%s)"
|
|
||||||
|
|
||||||
for container in $containers; do
|
|
||||||
backup_container "$container"
|
|
||||||
done
|
|
||||||
|
|
||||||
script_finish_time="$(date +%s)"
|
|
||||||
script_duration=$((script_finish_time - script_start_time))
|
|
||||||
script_duration_m=$((script_duration / 60))
|
|
||||||
|
|
||||||
debug "-----"
|
|
||||||
log "Script finished after ${script_duration}s (${script_duration_m} min)"
|
|
||||||
|
|
||||||
exit 0
|
|
||||||
# EOF
|
|
||||||
Regular → Executable
+2
-396
@@ -1,399 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
|
# Compatibility entry point. Use stages/90-prune.sh for new installations.
|
||||||
set -Eeuo pipefail
|
set -Eeuo pipefail
|
||||||
|
|
||||||
###############################################################################
|
|
||||||
# dockerbackup_prune.sh
|
|
||||||
#
|
|
||||||
# Default:
|
|
||||||
# A) Create a dated .zst backup locally and keep it
|
|
||||||
#
|
|
||||||
# With --apply:
|
|
||||||
# A) Create a dated .zst backup
|
|
||||||
# C) Prune *.zst under PRUNE_ROOT subfolders, keeping newest N per subfolder
|
|
||||||
# D) Keep the new local archive
|
|
||||||
#
|
|
||||||
# With --rclone:
|
|
||||||
# A) Create a dated .zst backup
|
|
||||||
# B) Upload via rclone
|
|
||||||
# C) Show what prune would do (unless --apply is also set)
|
|
||||||
# D) Keep the new local archive unless --apply is also set
|
|
||||||
#
|
|
||||||
# With --rclone --apply:
|
|
||||||
# A) Create a dated .zst backup
|
|
||||||
# B) Upload via rclone
|
|
||||||
# C) Prune for real
|
|
||||||
# D) Delete the new local archive
|
|
||||||
#
|
|
||||||
# Restore:
|
|
||||||
# zstd -d -c /path/backup.zst | tar -x -C /restore/target
|
|
||||||
###############################################################################
|
|
||||||
|
|
||||||
# --------------------------- Defaults (can be overridden) --------------------
|
|
||||||
SOURCE_DIR="/mybackups" # what to archive
|
|
||||||
PRUNE_ROOT="/mybackups" # where to prune .zst
|
|
||||||
PRUNE_PATTERN="*.zst" # files to prune
|
|
||||||
KEEP_COUNT=5 # keep newest N per subfolder
|
|
||||||
|
|
||||||
RCLONE_REMOTE="gdrive:location/sublocation" # rclone destination
|
|
||||||
OUT_DIR="/tmp" # where to write the .zst
|
|
||||||
BACKUP_PREFIX="panda-dockerbackup" # backup name prefix
|
|
||||||
|
|
||||||
# Optional zstd tuning, e.g. "-19 -T0" for max level, multithread
|
|
||||||
ZSTD_EXTRA_OPTS=""
|
|
||||||
|
|
||||||
# --------------------------- Load .env (same dir as script) -----------------
|
|
||||||
# Order of precedence:
|
|
||||||
# defaults < .env (same dir) < CLI flags
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
ENV_FILE="${DOCKERBACKUP_ENV:-$SCRIPT_DIR/.env}"
|
exec "$SCRIPT_DIR/stages/90-prune.sh" "$@"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
|
||||||
perms="$(stat -c '%a' "$ENV_FILE" 2>/dev/null || stat -f '%Lp' "$ENV_FILE" 2>/dev/null || echo '')"
|
|
||||||
if [[ -n "$perms" && ! "$perms" =~ ^(600|400)$ ]]; then
|
|
||||||
echo "$(date '+%F %T') [WARN] $ENV_FILE permissions are $perms; recommend 600." >&2
|
|
||||||
fi
|
|
||||||
set -a
|
|
||||||
# shellcheck disable=SC1090
|
|
||||||
. "$ENV_FILE"
|
|
||||||
set +a
|
|
||||||
fi
|
|
||||||
|
|
||||||
# --------------------------- CLI flags --------------------------------------
|
|
||||||
APPLY=0
|
|
||||||
QUIET=0
|
|
||||||
DRYRUN=0
|
|
||||||
USE_RCLONE=0
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<EOF
|
|
||||||
Usage: $0 [--rclone] [--apply] [--quiet] [--dry-run] [--keep N] [--root PATH] [--source PATH] [--outdir PATH] [--remote PATH]
|
|
||||||
|
|
||||||
Default:
|
|
||||||
Creates the local .zst archive and keeps it locally.
|
|
||||||
No rclone upload. No prune.
|
|
||||||
|
|
||||||
With --apply:
|
|
||||||
Creates the local .zst archive, prunes old .zst files for real, and keeps the new archive locally.
|
|
||||||
|
|
||||||
With --rclone:
|
|
||||||
Uploads the archive via rclone.
|
|
||||||
Prune also runs, but only as a dry-run unless --apply is also set.
|
|
||||||
The new local archive is kept unless --apply is also set.
|
|
||||||
|
|
||||||
With --rclone --apply:
|
|
||||||
Uploads the archive, prunes for real, and deletes the new local archive.
|
|
||||||
|
|
||||||
Flags:
|
|
||||||
--rclone Enable rclone upload flow
|
|
||||||
--apply Actually delete pruned files; with --rclone also delete the new local archive
|
|
||||||
--quiet Minimal console output (no live progress)
|
|
||||||
--dry-run Simulate everything (no backup, no upload, no deletes)
|
|
||||||
--keep N Keep newest N .zst files per subfolder (default ${KEEP_COUNT})
|
|
||||||
--root PATH Root folder to prune (default ${PRUNE_ROOT})
|
|
||||||
--source PATH Folder to archive (default ${SOURCE_DIR})
|
|
||||||
--outdir PATH Where to write the .zst (default ${OUT_DIR})
|
|
||||||
--remote PATH rclone destination (default ${RCLONE_REMOTE})
|
|
||||||
|
|
||||||
Environment file:
|
|
||||||
A .env file in the same directory as this script is loaded automatically.
|
|
||||||
You can override its path by setting DOCKERBACKUP_ENV=/path/to/.env
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--rclone) USE_RCLONE=1 ;;
|
|
||||||
--apply) APPLY=1 ;;
|
|
||||||
--quiet) QUIET=1 ;;
|
|
||||||
--dry-run) DRYRUN=1 ;;
|
|
||||||
--keep) KEEP_COUNT="${2:?}"; shift ;;
|
|
||||||
--root) PRUNE_ROOT="${2:?}"; shift ;;
|
|
||||||
--source) SOURCE_DIR="${2:?}"; shift ;;
|
|
||||||
--outdir) OUT_DIR="${2:?}"; shift ;;
|
|
||||||
--remote) RCLONE_REMOTE="${2:?}"; shift ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "Unknown option: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
shift
|
|
||||||
done
|
|
||||||
|
|
||||||
# --------------------------- Logging helpers --------------------------------
|
|
||||||
ts() { date '+%F %T'; }
|
|
||||||
log() { [[ $QUIET -eq 0 ]] && echo "$(ts) [$1] $2"; }
|
|
||||||
log_err() { echo "$(ts) [ERROR] $1" >&2; }
|
|
||||||
run() { if [[ $DRYRUN -eq 1 ]]; then log "DRY" "$*"; else "$@"; fi; }
|
|
||||||
|
|
||||||
truncate_and_print() {
|
|
||||||
local s="$1" cols
|
|
||||||
if [[ -t 2 ]]; then
|
|
||||||
cols=$(tput cols 2>/dev/null || echo 120)
|
|
||||||
else
|
|
||||||
cols=120
|
|
||||||
fi
|
|
||||||
(( cols < 20 )) && cols=120
|
|
||||||
if (( ${#s} > cols - 1 )); then
|
|
||||||
s="${s:0:cols-1}"
|
|
||||||
fi
|
|
||||||
printf '\r%s\033[K' "$s" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
cleanup_tmp=""
|
|
||||||
pipe_pid=""
|
|
||||||
pipe_pgid=""
|
|
||||||
pct_fd_open=0
|
|
||||||
path_fd_open=0
|
|
||||||
status_line_shown=0
|
|
||||||
|
|
||||||
cleanup() {
|
|
||||||
local why="${1:-EXIT}"
|
|
||||||
|
|
||||||
if [[ -n "$pipe_pid" ]]; then
|
|
||||||
if command -v ps >/dev/null 2>&1; then
|
|
||||||
pipe_pgid=$(ps -o pgid= "$pipe_pid" 2>/dev/null | tr -d ' ' || true)
|
|
||||||
fi
|
|
||||||
if [[ -n "$pipe_pgid" ]]; then
|
|
||||||
kill -TERM "-$pipe_pgid" 2>/dev/null || true
|
|
||||||
sleep 0.2
|
|
||||||
kill -KILL "-$pipe_pgid" 2>/dev/null || true
|
|
||||||
else
|
|
||||||
kill -TERM "$pipe_pid" 2>/dev/null || true
|
|
||||||
sleep 0.2
|
|
||||||
kill -KILL "$pipe_pid" 2>/dev/null || true
|
|
||||||
fi
|
|
||||||
wait "$pipe_pid" 2>/dev/null || true
|
|
||||||
fi
|
|
||||||
|
|
||||||
if (( pct_fd_open == 1 )); then exec 4>&- 4<&-; pct_fd_open=0; fi
|
|
||||||
if (( path_fd_open == 1 )); then exec 3>&- 3<&-; path_fd_open=0; fi
|
|
||||||
|
|
||||||
if [[ -n "$cleanup_tmp" ]]; then
|
|
||||||
rm -f "$cleanup_tmp/pct" "$cleanup_tmp/path" 2>/dev/null || true
|
|
||||||
rmdir "$cleanup_tmp" 2>/dev/null || true
|
|
||||||
fi
|
|
||||||
|
|
||||||
if (( status_line_shown == 1 )) && [[ $QUIET -eq 0 ]]; then
|
|
||||||
printf '\n' >&2
|
|
||||||
fi
|
|
||||||
|
|
||||||
case "$why" in
|
|
||||||
INT|TERM) exit 130 ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
trap 'log_err "Command failed: $BASH_COMMAND"; cleanup ERR; exit 1' ERR
|
|
||||||
trap 'cleanup INT' INT
|
|
||||||
trap 'cleanup TERM' TERM
|
|
||||||
trap 'cleanup EXIT' EXIT
|
|
||||||
|
|
||||||
# --------------------------- Preconditions ----------------------------------
|
|
||||||
[[ -d "$SOURCE_DIR" ]] || { log_err "Source dir not found: $SOURCE_DIR"; exit 1; }
|
|
||||||
[[ -d "$PRUNE_ROOT" ]] || { log_err "Prune root not found: $PRUNE_ROOT"; exit 1; }
|
|
||||||
command -v tar >/dev/null 2>&1 || { log_err "tar not found"; exit 1; }
|
|
||||||
command -v zstd >/dev/null 2>&1 || { log_err "zstd not found"; exit 1; }
|
|
||||||
command -v find >/dev/null 2>&1 || { log_err "find not found"; exit 1; }
|
|
||||||
|
|
||||||
if (( USE_RCLONE == 1 )) && (( DRYRUN == 0 )); then
|
|
||||||
command -v rclone >/dev/null 2>&1 || { log_err "rclone not found"; exit 1; }
|
|
||||||
fi
|
|
||||||
|
|
||||||
STAMP="${STAMP_OVERRIDE:-$(date +%F_%H-%M)}"
|
|
||||||
BACKUP_NAME="${BACKUP_PREFIX}-${STAMP}.zst"
|
|
||||||
BACKUP_PATH="${OUT_DIR%/}/${BACKUP_NAME}"
|
|
||||||
|
|
||||||
RCLONE_FLAGS=()
|
|
||||||
[[ $QUIET -eq 1 ]] && RCLONE_FLAGS+=("-q") || RCLONE_FLAGS+=("--progress")
|
|
||||||
|
|
||||||
ZSTD_FLAGS=("-q")
|
|
||||||
[[ -n "$ZSTD_EXTRA_OPTS" ]] && ZSTD_FLAGS+=($ZSTD_EXTRA_OPTS)
|
|
||||||
|
|
||||||
log "INFO" "Mode: $( [[ $DRYRUN -eq 1 ]] && echo 'DRY-RUN ALL' || ([[ $APPLY -eq 1 ]] && echo 'APPLY DELETES' || echo 'NO DELETES') )"
|
|
||||||
log "INFO" "Rclone mode: $( [[ $USE_RCLONE -eq 1 ]] && echo 'ENABLED' || echo 'DISABLED' )"
|
|
||||||
log "INFO" "Creating archive: $BACKUP_PATH (.zst)"
|
|
||||||
log "INFO" "Keeping newest $KEEP_COUNT file(s) per subfolder matching: $PRUNE_PATTERN"
|
|
||||||
log "INFO" "Source: $SOURCE_DIR"
|
|
||||||
if (( USE_RCLONE == 1 )); then
|
|
||||||
log "INFO" "Remote: $RCLONE_REMOTE"
|
|
||||||
fi
|
|
||||||
log "INFO" "Prune root (root itself will be skipped): $PRUNE_ROOT"
|
|
||||||
|
|
||||||
# --------------------------- Step A: create .zst with one-liner --------------
|
|
||||||
log "STEP" "A) Creating archive"
|
|
||||||
|
|
||||||
if [[ $DRYRUN -eq 1 ]]; then
|
|
||||||
log "DRY" "tar -C \"$SOURCE_DIR\" -cvf - . | pv -n -s \$(du -sb \"$SOURCE_DIR\"|cut -f1) 2>pct.fifo | zstd ${ZSTD_FLAGS[*]} -o \"$BACKUP_PATH\""
|
|
||||||
else
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
|
|
||||||
export LC_ALL=C
|
|
||||||
TOTAL_FILES=$(find "$SOURCE_DIR" -type f | wc -l | awk '{print $1}')
|
|
||||||
TOTAL_BYTES=$(du -sb "$SOURCE_DIR" | cut -f1)
|
|
||||||
|
|
||||||
tmpdir="$(mktemp -d)"; cleanup_tmp="$tmpdir"
|
|
||||||
path_fifo="$tmpdir/path"; pct_fifo="$tmpdir/pct"
|
|
||||||
mkfifo "$path_fifo"
|
|
||||||
have_pv=0
|
|
||||||
if command -v pv >/dev/null 2>&1 && [[ $QUIET -eq 0 ]] && [[ -t 2 ]]; then
|
|
||||||
mkfifo "$pct_fifo"; have_pv=1
|
|
||||||
fi
|
|
||||||
|
|
||||||
exec 3<> "$path_fifo"; path_fd_open=1
|
|
||||||
if (( have_pv == 1 )); then exec 4<> "$pct_fifo"; pct_fd_open=1; fi
|
|
||||||
|
|
||||||
if (( have_pv == 1 )); then
|
|
||||||
(
|
|
||||||
set -o pipefail
|
|
||||||
tar -C "$SOURCE_DIR" -cvf - . \
|
|
||||||
2> >(sed -u 's#^\./##' > "$path_fifo" || true) \
|
|
||||||
| pv -n -s "$TOTAL_BYTES" 2> "$pct_fifo" \
|
|
||||||
| zstd "${ZSTD_FLAGS[@]}" -o "$BACKUP_PATH"
|
|
||||||
) &
|
|
||||||
else
|
|
||||||
(
|
|
||||||
set -o pipefail
|
|
||||||
tar -C "$SOURCE_DIR" -cvf - . \
|
|
||||||
2> >(sed -u 's#^\./##' > "$path_fifo" || true) \
|
|
||||||
| zstd "${ZSTD_FLAGS[@]}" -o "$BACKUP_PATH"
|
|
||||||
) &
|
|
||||||
fi
|
|
||||||
pipe_pid=$!
|
|
||||||
|
|
||||||
files_seen=0
|
|
||||||
last_path=""
|
|
||||||
last_pct="0.0"
|
|
||||||
status_line_shown=0
|
|
||||||
last_print_time=0
|
|
||||||
|
|
||||||
while kill -0 "$pipe_pid" 2>/dev/null; do
|
|
||||||
if (( have_pv == 1 )); then
|
|
||||||
if IFS= read -r -t 0.03 pct_line <&4; then last_pct="$pct_line"; fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
if IFS= read -r -t 0.03 path_line <&3; then
|
|
||||||
last_path="$path_line"
|
|
||||||
[[ -n "$last_path" ]] && (( files_seen++ )) || true
|
|
||||||
fi
|
|
||||||
|
|
||||||
display_rel="$last_path"
|
|
||||||
dir_part="$(dirname -- "$display_rel" 2>/dev/null || echo .)"
|
|
||||||
base_part="$(basename -- "$display_rel" 2>/dev/null || echo .)"
|
|
||||||
[[ "$dir_part" == "." ]] && dir_part="."
|
|
||||||
|
|
||||||
if [[ $QUIET -eq 0 ]]; then
|
|
||||||
now_ns=$(date +%s%N)
|
|
||||||
if (( now_ns - last_print_time > 50000000 )); then
|
|
||||||
if (( have_pv == 1 )); then
|
|
||||||
line="[$(printf '%.1f' "${last_pct:-0}")%] archiving: ${dir_part}/ ${base_part}"
|
|
||||||
else
|
|
||||||
line="[${files_seen}/${TOTAL_FILES}] archiving: ${dir_part}/ ${base_part}"
|
|
||||||
fi
|
|
||||||
truncate_and_print "$line"
|
|
||||||
status_line_shown=1
|
|
||||||
last_print_time=$now_ns
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
while IFS= read -r -t 0.03 path_line <&3; do
|
|
||||||
last_path="$path_line"; (( files_seen++ )) || true
|
|
||||||
[[ $QUIET -eq 0 ]] && truncate_and_print "[${files_seen}/${TOTAL_FILES}] archiving: $(dirname -- "$last_path")/ $(basename -- "$last_path")"
|
|
||||||
status_line_shown=1
|
|
||||||
done || true
|
|
||||||
|
|
||||||
wait "$pipe_pid"
|
|
||||||
if [[ $QUIET -eq 0 ]]; then
|
|
||||||
printf '\n' >&2
|
|
||||||
status_line_shown=0
|
|
||||||
fi
|
|
||||||
|
|
||||||
exec 3>&- 3<&-; path_fd_open=0
|
|
||||||
if (( have_pv == 1 )); then exec 4>&- 4<&-; pct_fd_open=0; fi
|
|
||||||
rm -f "$path_fifo" "$pct_fifo" 2>/dev/null || true
|
|
||||||
rmdir "$tmpdir" 2>/dev/null || true
|
|
||||||
cleanup_tmp=""
|
|
||||||
fi
|
|
||||||
log "OK" "Archive step completed"
|
|
||||||
|
|
||||||
# --------------------------- Step B: optional rclone copy --------------------
|
|
||||||
if (( USE_RCLONE == 1 )); then
|
|
||||||
log "STEP" "B) rclone copy to remote"
|
|
||||||
if [[ $DRYRUN -eq 1 ]]; then
|
|
||||||
log "DRY" "rclone copy \"$BACKUP_PATH\" \"$RCLONE_REMOTE\" ${RCLONE_FLAGS[*]}"
|
|
||||||
else
|
|
||||||
run rclone copy "$BACKUP_PATH" "$RCLONE_REMOTE" "${RCLONE_FLAGS[@]}"
|
|
||||||
fi
|
|
||||||
log "OK" "Upload step completed"
|
|
||||||
else
|
|
||||||
log "SKIP" "--rclone not set; skipping upload"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# --------------------------- Step C: prune if apply or rclone ---------------
|
|
||||||
if (( APPLY == 1 || USE_RCLONE == 1 )); then
|
|
||||||
log "STEP" "C) Pruning $PRUNE_PATTERN under $PRUNE_ROOT (keep $KEEP_COUNT)"
|
|
||||||
export LC_ALL=C
|
|
||||||
prune_ok=1
|
|
||||||
|
|
||||||
while IFS= read -r -d '' dir; do
|
|
||||||
mapfile -d '' -t files < <(
|
|
||||||
find "$dir" -maxdepth 1 -type f -name "$PRUNE_PATTERN" -printf '%T@ %p\0' |
|
|
||||||
sort -z -n |
|
|
||||||
cut -z -d' ' -f2-
|
|
||||||
)
|
|
||||||
count=${#files[@]}
|
|
||||||
if (( count == 0 )); then
|
|
||||||
log "SKIP" "No matches in: $dir"
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
|
|
||||||
keep_from=$(( count - KEEP_COUNT ))
|
|
||||||
(( keep_from < 0 )) && keep_from=0
|
|
||||||
|
|
||||||
if (( count <= KEEP_COUNT )); then
|
|
||||||
log "KEEP" "At or below limit in: $dir (count=$count)"
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
|
|
||||||
keep_list=("${files[@]:keep_from:KEEP_COUNT}")
|
|
||||||
del_list=("${files[@]:0:keep_from}")
|
|
||||||
|
|
||||||
for k in "${keep_list[@]}"; do
|
|
||||||
log "KEEP" "Keep: $k"
|
|
||||||
done
|
|
||||||
for f in "${del_list[@]}"; do
|
|
||||||
if [[ $DRYRUN -eq 1 || $APPLY -eq 0 ]]; then
|
|
||||||
log "WOULD" "Remove: $f"
|
|
||||||
else
|
|
||||||
if rm -f -- "$f"; then
|
|
||||||
log "DELETE" "Removed: $f"
|
|
||||||
else
|
|
||||||
log_err "Failed to remove: $f"; prune_ok=0
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
done < <(find "$PRUNE_ROOT" -mindepth 1 -type d -print0)
|
|
||||||
|
|
||||||
if [[ $prune_ok -ne 1 ]]; then
|
|
||||||
log_err "Prune step encountered errors; aborting final delete"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
log "OK" "Prune step completed"
|
|
||||||
else
|
|
||||||
log "SKIP" "--apply and --rclone not set; skipping prune"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# --------------------------- Step D: remove local .zst -----------------------
|
|
||||||
if (( USE_RCLONE == 1 )); then
|
|
||||||
log "STEP" "D) Removing local backup"
|
|
||||||
if [[ $DRYRUN -eq 1 || $APPLY -eq 0 ]]; then
|
|
||||||
log "WOULD" "Remove local backup: $BACKUP_PATH"
|
|
||||||
else
|
|
||||||
if rm -f -- "$BACKUP_PATH"; then
|
|
||||||
log "DELETE" "Removed local backup: $BACKUP_PATH"
|
|
||||||
else
|
|
||||||
log_err "Failed to remove local backup: $BACKUP_PATH"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
else
|
|
||||||
log "KEEP" "Local archive retained: $BACKUP_PATH"
|
|
||||||
fi
|
|
||||||
|
|
||||||
log "DONE" "All steps completed successfully"
|
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -Eeuo pipefail
|
||||||
|
|
||||||
|
SCRIPT_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd -P)"
|
||||||
|
ENV_FILE="${DOCKERBACKUP_ENV:-$SCRIPT_ROOT/.env}"
|
||||||
|
|
||||||
|
SOURCE_BASE_DIR="/dockervolumes"
|
||||||
|
BACKUP_ROOT="/dockerbackups"
|
||||||
|
CONTAINERS_FILE="$SCRIPT_ROOT/containers.txt"
|
||||||
|
BACKUP_HOSTNAME="$(hostname -s)"
|
||||||
|
BACKUP_METHODS="rclone smb"
|
||||||
|
KEEP_COUNT=5
|
||||||
|
PULL_IMAGES=1
|
||||||
|
START_TIMEOUT=90
|
||||||
|
WAIT_INTERVAL=3
|
||||||
|
START_RETRIES=1
|
||||||
|
LOCK_FILE="/run/lock/zorruno-docker-backup.lock"
|
||||||
|
ZSTD_EXTRA_OPTS="-T0 -12"
|
||||||
|
MQTT_REPORT_ENABLED=0
|
||||||
|
MQTT_HOST=""
|
||||||
|
MQTT_PORT=1883
|
||||||
|
MQTT_USERNAME=""
|
||||||
|
MQTT_PASSWORD_FILE=""
|
||||||
|
MQTT_STATUS_TOPIC="viewroad-status/dockerbackup/$BACKUP_HOSTNAME"
|
||||||
|
MQTT_COMMAND_TOPIC="viewroad-commands/dockerbackup/$BACKUP_HOSTNAME/run"
|
||||||
|
|
||||||
|
RCLONE_ENABLED=0
|
||||||
|
RCLONE_REMOTE=""
|
||||||
|
RCLONE_HOST_SUBDIR=1
|
||||||
|
RCLONE_EXTRA_ARGS=""
|
||||||
|
SMB_ENABLED=0
|
||||||
|
SMB_MOUNT=""
|
||||||
|
SMB_HOST_SUBDIR=1
|
||||||
|
SMB_RSYNC_ARGS="-a"
|
||||||
|
|
||||||
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
|
set -a
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
. "$ENV_FILE"
|
||||||
|
set +a
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Ease migration from the original prune script's .env names.
|
||||||
|
if [[ -n "${SOURCE_DIR:-}" && "$BACKUP_ROOT" == "/dockerbackups" ]]; then
|
||||||
|
BACKUP_ROOT="$SOURCE_DIR"
|
||||||
|
fi
|
||||||
|
|
||||||
|
timestamp() { date '+%F %T'; }
|
||||||
|
log() {
|
||||||
|
local line
|
||||||
|
line="$(timestamp) [$1] $2"
|
||||||
|
printf '%s\n' "$line" >&2
|
||||||
|
if [[ -n "${BACKUP_RUN_LOG:-}" ]]; then
|
||||||
|
printf '%s\n' "$line" >>"$BACKUP_RUN_LOG"
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
die() { log ERROR "$1"; exit "${2:-1}"; }
|
||||||
|
|
||||||
|
is_true() {
|
||||||
|
case "${1,,}" in
|
||||||
|
1|true|yes|on|enabled) return 0 ;;
|
||||||
|
*) return 1 ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
require_command() {
|
||||||
|
command -v "$1" >/dev/null 2>&1 || die "Required command not found: $1" 3
|
||||||
|
}
|
||||||
|
|
||||||
|
require_positive_integer() {
|
||||||
|
[[ "$2" =~ ^[1-9][0-9]*$ ]] || die "$1 must be a positive integer (received: $2)" 2
|
||||||
|
}
|
||||||
|
|
||||||
|
resolve_path() {
|
||||||
|
if [[ "$1" = /* ]]; then printf '%s\n' "$1"; else printf '%s/%s\n' "$SCRIPT_ROOT" "$1"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
run_id() { date '+%Y-%m-%d_%H-%M-%S'; }
|
||||||
+19
@@ -0,0 +1,19 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
mqtt_common_args() {
|
||||||
|
MQTT_ARGS=(-h "$MQTT_HOST" -p "$MQTT_PORT")
|
||||||
|
[[ -n "$MQTT_USERNAME" ]] && MQTT_ARGS+=(-u "$MQTT_USERNAME")
|
||||||
|
if [[ -n "$MQTT_PASSWORD_FILE" ]]; then
|
||||||
|
[[ -r "$MQTT_PASSWORD_FILE" ]] || die "MQTT password file is not readable: $MQTT_PASSWORD_FILE"
|
||||||
|
MQTT_ARGS+=(-P "$(<"$MQTT_PASSWORD_FILE")")
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
mqtt_publish() {
|
||||||
|
local topic="$1" payload="$2" retain="${3:-0}"
|
||||||
|
[[ -n "$MQTT_HOST" ]] || die "MQTT_HOST is not configured"
|
||||||
|
require_command mosquitto_pub
|
||||||
|
mqtt_common_args
|
||||||
|
(( retain )) && MQTT_ARGS+=(-r)
|
||||||
|
mosquitto_pub "${MQTT_ARGS[@]}" -t "$topic" -m "$payload"
|
||||||
|
}
|
||||||
Executable
+26
@@ -0,0 +1,26 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Copy the local backup tree to an rclone destination.
|
||||||
|
set -Eeuo pipefail
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
|
. "$SCRIPT_DIR/../lib/common.sh"
|
||||||
|
DRY_RUN=0; FORCE=0
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--dry-run) DRY_RUN=1 ;; --force) FORCE=1 ;;
|
||||||
|
-h|--help) echo "Usage: ${0##*/} [--dry-run] [--force]"; exit 0 ;;
|
||||||
|
*) die "Unknown option: $1" 2 ;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
if ! is_true "$RCLONE_ENABLED" && (( FORCE == 0 )); then log SKIP "rclone method is disabled"; exit 0; fi
|
||||||
|
[[ -n "$RCLONE_REMOTE" ]] || die "RCLONE_REMOTE is not configured"
|
||||||
|
require_command rclone
|
||||||
|
[[ -d "$BACKUP_ROOT" ]] || die "Backup root not found: $BACKUP_ROOT"
|
||||||
|
read -r -a extra_args <<<"$RCLONE_EXTRA_ARGS"
|
||||||
|
destination="${RCLONE_REMOTE%/}"
|
||||||
|
is_true "$RCLONE_HOST_SUBDIR" && destination="$destination/$BACKUP_HOSTNAME"
|
||||||
|
args=(copy "$BACKUP_ROOT" "$destination" "${extra_args[@]}")
|
||||||
|
(( DRY_RUN )) && args+=(--dry-run)
|
||||||
|
log INFO "Copying backups to $destination"
|
||||||
|
rclone "${args[@]}"
|
||||||
|
log DONE "rclone backup method completed"
|
||||||
Executable
+29
@@ -0,0 +1,29 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Copy the local backup tree to an already-mounted SMB share.
|
||||||
|
set -Eeuo pipefail
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
|
. "$SCRIPT_DIR/../lib/common.sh"
|
||||||
|
DRY_RUN=0; FORCE=0
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--dry-run) DRY_RUN=1 ;; --force) FORCE=1 ;;
|
||||||
|
-h|--help) echo "Usage: ${0##*/} [--dry-run] [--force]"; exit 0 ;;
|
||||||
|
*) die "Unknown option: $1" 2 ;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
if ! is_true "$SMB_ENABLED" && (( FORCE == 0 )); then log SKIP "SMB method is disabled"; exit 0; fi
|
||||||
|
[[ -n "$SMB_MOUNT" ]] || die "SMB_MOUNT is not configured"
|
||||||
|
require_command rsync
|
||||||
|
require_command mountpoint
|
||||||
|
[[ -d "$BACKUP_ROOT" ]] || die "Backup root not found: $BACKUP_ROOT"
|
||||||
|
[[ -d "$SMB_MOUNT" ]] || die "SMB mount does not exist: $SMB_MOUNT"
|
||||||
|
mountpoint -q "$SMB_MOUNT" || die "SMB path is not a mounted filesystem: $SMB_MOUNT"
|
||||||
|
destination="${SMB_MOUNT%/}"
|
||||||
|
is_true "$SMB_HOST_SUBDIR" && destination="$destination/$BACKUP_HOSTNAME"
|
||||||
|
(( DRY_RUN )) || mkdir -p "$destination"
|
||||||
|
read -r -a rsync_args <<<"$SMB_RSYNC_ARGS"
|
||||||
|
(( DRY_RUN )) && rsync_args+=(--dry-run)
|
||||||
|
log INFO "Copying backups to $destination"
|
||||||
|
rsync "${rsync_args[@]}" "$BACKUP_ROOT/" "$destination/"
|
||||||
|
log DONE "SMB backup method completed"
|
||||||
Executable
+142
@@ -0,0 +1,142 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Stop, archive, update, start, and verify each listed Compose project.
|
||||||
|
set -Eeuo pipefail
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
|
# shellcheck source=../lib/common.sh
|
||||||
|
. "$SCRIPT_DIR/../lib/common.sh"
|
||||||
|
|
||||||
|
DRY_RUN=0
|
||||||
|
usage() {
|
||||||
|
cat <<EOF
|
||||||
|
Usage: ${0##*/} [--dry-run] [--containers FILE] [--source DIR] [--backup-root DIR]
|
||||||
|
|
||||||
|
Entries may be container names or Compose project directory names.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--dry-run|-t) DRY_RUN=1 ;;
|
||||||
|
--containers|-f) CONTAINERS_FILE="${2:?Missing containers file}"; shift ;;
|
||||||
|
--source|-s) SOURCE_BASE_DIR="${2:?Missing source directory}"; shift ;;
|
||||||
|
--backup-root) BACKUP_ROOT="${2:?Missing backup root}"; shift ;;
|
||||||
|
--verbose|-v) : ;; # Retained for compatibility; current logging is always concise.
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) die "Unknown option: $1" 2 ;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
|
||||||
|
CONTAINERS_FILE="$(resolve_path "$CONTAINERS_FILE")"
|
||||||
|
require_command docker
|
||||||
|
require_command tar
|
||||||
|
require_command zstd
|
||||||
|
[[ -d "$SOURCE_BASE_DIR" ]] || die "Docker source directory not found: $SOURCE_BASE_DIR"
|
||||||
|
[[ -r "$CONTAINERS_FILE" ]] || die "Containers file not readable: $CONTAINERS_FILE"
|
||||||
|
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE=(docker compose)
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1; then
|
||||||
|
COMPOSE=(docker-compose)
|
||||||
|
else
|
||||||
|
die "Neither 'docker compose' nor 'docker-compose' is available" 3
|
||||||
|
fi
|
||||||
|
compose() { "${COMPOSE[@]}" "$@"; }
|
||||||
|
|
||||||
|
resolve_project() {
|
||||||
|
local entry="$1" working_dir="" project=""
|
||||||
|
if docker inspect "$entry" >/dev/null 2>&1; then
|
||||||
|
working_dir="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project.working_dir"}}' "$entry" 2>/dev/null || true)"
|
||||||
|
project="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project"}}' "$entry" 2>/dev/null || true)"
|
||||||
|
fi
|
||||||
|
[[ -n "$working_dir" ]] || { [[ -d "$SOURCE_BASE_DIR/$entry" ]] && working_dir="$SOURCE_BASE_DIR/$entry"; }
|
||||||
|
[[ -n "$working_dir" ]] || return 1
|
||||||
|
[[ -n "$project" ]] || project="$(basename "$working_dir")"
|
||||||
|
printf '%s\t%s\n' "$project" "$working_dir"
|
||||||
|
}
|
||||||
|
|
||||||
|
project_is_ready() {
|
||||||
|
local project="$1" ids id status health
|
||||||
|
ids="$(docker ps -a -q --filter "label=com.docker.compose.project=$project" --filter "label=com.docker.compose.oneoff=False")"
|
||||||
|
[[ -n "$ids" ]] || return 1
|
||||||
|
for id in $ids; do
|
||||||
|
status="$(docker inspect -f '{{.State.Status}}' "$id")"
|
||||||
|
health="$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$id")"
|
||||||
|
[[ "$status" == running ]] || return 1
|
||||||
|
[[ "$health" == none || "$health" == healthy ]] || return 1
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
wait_for_project() {
|
||||||
|
local project="$1" started now
|
||||||
|
started="$(date +%s)"
|
||||||
|
while ! project_is_ready "$project"; do
|
||||||
|
now="$(date +%s)"
|
||||||
|
(( now - started < START_TIMEOUT )) || return 1
|
||||||
|
sleep "$WAIT_INTERVAL"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
backup_project() {
|
||||||
|
local project="$1" working_dir="$2" parent base output_dir output attempt
|
||||||
|
local -a zstd_args=()
|
||||||
|
parent="$(dirname "$working_dir")"
|
||||||
|
base="$(basename "$working_dir")"
|
||||||
|
output_dir="$BACKUP_ROOT/$project"
|
||||||
|
output="$output_dir/${BACKUP_HOSTNAME}.${project}.$(run_id).tar.zst"
|
||||||
|
log INFO "Project: $project ($working_dir)"
|
||||||
|
log INFO "Archive: $output"
|
||||||
|
if (( DRY_RUN )); then
|
||||||
|
log DRY "Would stop, archive, optionally pull, start, and verify $project"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$output_dir"
|
||||||
|
cd "$working_dir"
|
||||||
|
compose down
|
||||||
|
read -r -a zstd_args <<<"$ZSTD_EXTRA_OPTS"
|
||||||
|
if ! tar --one-file-system -cf - -C "$parent" "$base" | zstd "${zstd_args[@]}" -o "$output"; then
|
||||||
|
rm -f -- "$output"
|
||||||
|
log ERROR "Archive failed; attempting to restart $project"
|
||||||
|
compose up -d || true
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
log ARCHIVE "Created $output"
|
||||||
|
if is_true "$PULL_IMAGES"; then
|
||||||
|
compose pull || log WARN "Image pull failed for $project; starting with available images"
|
||||||
|
fi
|
||||||
|
compose up -d
|
||||||
|
|
||||||
|
if wait_for_project "$project"; then log OK "$project is running and healthy"; return 0; fi
|
||||||
|
for (( attempt=1; attempt<=START_RETRIES; attempt++ )); do
|
||||||
|
log WARN "$project did not become ready; restart attempt $attempt/$START_RETRIES"
|
||||||
|
compose down || true
|
||||||
|
compose up -d
|
||||||
|
if wait_for_project "$project"; then log OK "$project is running after retry"; return 0; fi
|
||||||
|
done
|
||||||
|
log ERROR "$project did not become ready"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
declare -A SEEN=()
|
||||||
|
failures=0
|
||||||
|
while IFS= read -r entry; do
|
||||||
|
entry="${entry%%#*}"
|
||||||
|
entry="${entry#"${entry%%[![:space:]]*}"}"
|
||||||
|
entry="${entry%"${entry##*[![:space:]]}"}"
|
||||||
|
[[ -n "$entry" ]] || continue
|
||||||
|
if ! resolved="$(resolve_project "$entry")"; then
|
||||||
|
log ERROR "Could not resolve container or project: $entry"
|
||||||
|
(( failures++ )) || true
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
IFS=$'\t' read -r project working_dir <<<"$resolved"
|
||||||
|
if [[ -n "${SEEN[$working_dir]:-}" ]]; then
|
||||||
|
log SKIP "Project already processed: $project ($working_dir)"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
SEEN[$working_dir]=1
|
||||||
|
backup_project "$project" "$working_dir" || { (( failures++ )) || true; }
|
||||||
|
done < "$CONTAINERS_FILE"
|
||||||
|
|
||||||
|
(( failures == 0 )) || die "$failures project backup(s) failed"
|
||||||
|
log DONE "Container backup stage completed"
|
||||||
Executable
+26
@@ -0,0 +1,26 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Prune local per-project archives, retaining the newest N in each.
|
||||||
|
set -Eeuo pipefail
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
|
. "$SCRIPT_DIR/../lib/common.sh"
|
||||||
|
DRY_RUN=0
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--dry-run) DRY_RUN=1 ;; --keep) KEEP_COUNT="${2:?Missing keep count}"; shift ;;
|
||||||
|
--root) BACKUP_ROOT="${2:?Missing backup root}"; shift ;;
|
||||||
|
-h|--help) echo "Usage: ${0##*/} [--dry-run] [--keep N] [--root DIR]"; exit 0 ;;
|
||||||
|
*) die "Unknown option: $1" 2 ;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
require_positive_integer KEEP_COUNT "$KEEP_COUNT"
|
||||||
|
[[ -d "$BACKUP_ROOT" ]] || { log SKIP "Backup root does not exist: $BACKUP_ROOT"; exit 0; }
|
||||||
|
while IFS= read -r -d '' dir; do
|
||||||
|
# Match both current *.tar.zst archives and legacy *.zst archives.
|
||||||
|
mapfile -d '' -t files < <(find "$dir" -maxdepth 1 -type f -name '*.zst' -printf '%T@ %p\0' | sort -z -nr | cut -z -d' ' -f2-)
|
||||||
|
(( ${#files[@]} > KEEP_COUNT )) || continue
|
||||||
|
for file in "${files[@]:KEEP_COUNT}"; do
|
||||||
|
if (( DRY_RUN )); then log WOULD "Remove $file"; else rm -f -- "$file"; log DELETE "Removed $file"; fi
|
||||||
|
done
|
||||||
|
done < <(find "$BACKUP_ROOT" -mindepth 1 -maxdepth 1 -type d -print0)
|
||||||
|
log DONE "Local prune stage completed; kept newest $KEEP_COUNT per project"
|
||||||
Executable
+43
@@ -0,0 +1,43 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Publish retained backup run state for Home Assistant.
|
||||||
|
set -Eeuo pipefail
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||||
|
. "$SCRIPT_DIR/../lib/common.sh"
|
||||||
|
. "$SCRIPT_DIR/../lib/mqtt.sh"
|
||||||
|
|
||||||
|
STATUS="${1:?Usage: ${0##*/} STATUS EXIT_CODE STARTED_EPOCH FINISHED_EPOCH RUN_ID LOG_FILE}"
|
||||||
|
EXIT_CODE="${2:-0}"
|
||||||
|
STARTED_EPOCH="${3:-$(date +%s)}"
|
||||||
|
FINISHED_EPOCH="${4:-$(date +%s)}"
|
||||||
|
RUN_ID="${5:-unknown}"
|
||||||
|
LOG_FILE="${6:-/dev/null}"
|
||||||
|
|
||||||
|
require_command jq
|
||||||
|
total=0; archived=0; dry_run=0; error_count=0
|
||||||
|
if [[ -r "$LOG_FILE" ]]; then
|
||||||
|
total="$(grep -c '\[INFO\] Project:' "$LOG_FILE" || true)"
|
||||||
|
archived="$(grep -c '\[ARCHIVE\] Created ' "$LOG_FILE" || true)"
|
||||||
|
dry_run="$(grep -c '\[DRY\] Would stop, archive' "$LOG_FILE" || true)"
|
||||||
|
error_count="$(grep -c '\[ERROR\]' "$LOG_FILE" || true)"
|
||||||
|
errors_json="$(sed -n 's/^.*\[ERROR\] //p' "$LOG_FILE" | jq -Rsc 'split("\n") | map(select(length > 0))')"
|
||||||
|
else
|
||||||
|
errors_json='[]'
|
||||||
|
fi
|
||||||
|
if (( EXIT_CODE != 0 && error_count == 0 )); then
|
||||||
|
errors_json="$(jq -cn --arg message "Pipeline exited with status $EXIT_CODE" '[ $message ]')"
|
||||||
|
error_count=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
payload="$(jq -cn \
|
||||||
|
--arg host "$BACKUP_HOSTNAME" --arg status "$STATUS" --arg run_id "$RUN_ID" \
|
||||||
|
--argjson exit_code "$EXIT_CODE" --argjson started "$STARTED_EPOCH" \
|
||||||
|
--argjson finished "$FINISHED_EPOCH" --argjson duration "$((FINISHED_EPOCH - STARTED_EPOCH))" \
|
||||||
|
--argjson total "$total" --argjson archived "$archived" --argjson dry_run "$dry_run" \
|
||||||
|
--argjson error_count "$error_count" --argjson errors "$errors_json" \
|
||||||
|
'{host:$host,status:$status,run_id:$run_id,exit_code:$exit_code,
|
||||||
|
started_at:($started|todateiso8601),finished_at:($finished|todateiso8601),
|
||||||
|
duration_seconds:$duration,containers_total:$total,containers_backed_up:$archived,
|
||||||
|
containers_dry_run:$dry_run,error_count:$error_count,errors:$errors}')"
|
||||||
|
|
||||||
|
mqtt_publish "$MQTT_STATUS_TOPIC" "$payload" 1
|
||||||
|
log MQTT "Published $STATUS result to $MQTT_STATUS_TOPIC"
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Zorruno Docker backup MQTT command listener
|
||||||
|
After=network-online.target docker.service
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=root
|
||||||
|
WorkingDirectory=/backupdockerscripts
|
||||||
|
ExecStart=/backupdockerscripts/bin/mqtt-trigger-listener.sh
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
Reference in new issue
Block a user