Remove personal values from environment example

.env.example

@@ -1,7 +1,37 @@
-# /etc/dockerbackup/.env
+# Copy to .env and customise on each Docker server. Keep .env out of Git.
-RCLONE_REMOTE="gdrive:backuplocation/sublocation"
+SOURCE_BASE_DIR="/dockervolumes"
-SOURCE_DIR="/mybackups"
+BACKUP_ROOT="/dockerbackups"
-PRUNE_ROOT="/mybackups"
+CONTAINERS_FILE="containers.local.txt"
-OUT_DIR="/tmp"
+BACKUP_HOSTNAME="examplehostname"
+
+# Main pipeline method order. Add future method names here.
+BACKUP_METHODS="rclone smb"
 KEEP_COUNT=5
+PULL_IMAGES=1
+START_TIMEOUT=90
+WAIT_INTERVAL=3
+START_RETRIES=1
+LOCK_FILE="/run/lock/zorruno-docker-backup.lock"
 ZSTD_EXTRA_OPTS="-T0 -12"
+
+# Optional retained MQTT status and command listener settings.
+MQTT_REPORT_ENABLED=0
+MQTT_HOST="examplehostname"
+MQTT_PORT=1883
+MQTT_USERNAME="dockerbackup"
+MQTT_PASSWORD_FILE="/etc/docker-backup/mqtt-password"
+MQTT_STATUS_TOPIC="backup-status/dockerbackup/examplehostname"
+MQTT_COMMAND_TOPIC="backup-commands/dockerbackup/examplehostname/run"
+
+# Rclone destination becomes <remote>/examplehostname/.
+RCLONE_ENABLED=0
+RCLONE_REMOTE="gdrive:dockerbackups"
+RCLONE_HOST_SUBDIR=1
+RCLONE_EXTRA_ARGS=""
+
+# The SMB share must already be mounted by the operating system.
+# Destination becomes <mount>/examplehostname/ when SMB_HOST_SUBDIR=1.
+SMB_ENABLED=0
+SMB_MOUNT="/mnt/dockerbackups"
+SMB_HOST_SUBDIR=1
+SMB_RSYNC_ARGS="-a"

.gitignore

@@ -1 +1,2 @@
 .env
+containers.local.txt

README.md

@@ -1,83 +1,213 @@
-# Docker Backup Scripts
+# Zorruno Docker Backup Scripts
 
-Simple shell scripts for backing up Docker-related data and pruning old backups.
+Modular backup scripts for Docker Compose servers.
+Each server has its own untracked `.env` and `containers.local.txt`, while all script
+logic remains common and can be updated through Git.
 
-## Files
+## Pipeline
 
-- `docker-backup.sh` - create backups
+`backup.sh` runs these stages in order:
-- `dockerbackup_prune.sh` - prune old backups
-- `containers.txt` - list of containers or backup targets
-- `.env.example` - example environment file
 
-## Reqirements
+1. `stages/10-containers.sh` — stop, archive, pull, start and verify projects.
+2. `stages/20-rclone.sh` — copy archives to an rclone destination when enabled.
+3. `stages/30-smb.sh` — copy archives to a mounted SMB share when enabled.
+4. `stages/90-prune.sh` — retain the newest configured number of `.zst`
+   archives locally, including legacy `.zst` and current `.tar.zst` names.
 
-docker-compose
+Every stage can also be run directly. The old `docker-backup.sh` name invokes
+the container archive stage, while `dockerbackup_prune.sh` invokes the local
+prune stage. Use the new `backup.sh` for the complete pipeline.
 
-## BACKUPS: Example use docker-backup.sh
+## Requirements
 
-Configure the backup script, docker-backup.sh
+- Bash 4 or later
-Set up your list of containers in container.txt . The script assumes each container is in a subdirectory of one main docker directory
+- Docker with `docker compose`, or legacy `docker-compose`
+- GNU `tar` with zstd support
+- `flock`
+- `rclone` only when the rclone method is enabled
+- `rsync` and `mountpoint` only when the SMB method is enabled
 
-Edit the location of where you want docker backups stored in docker-backup.sh (default is "/dockerbackups") or use
+The account running the scripts must be allowed to control Docker and read all
+files under `/dockervolumes`.
 
-./docker-backup.sh -s /home/me/mycontainers
+## Installation on a server
 
-**Actions:**
+```bash
-- It will recurse though the directories if they are listed in containers.txt 
+sudo git clone https://home.fox.co.nz/gitea/zorruno/zorruno-backupdockerscript.git /backupdockerscripts
-- It will run docker-compose to take down the container, pull a new version if there is one, then bring the container back up.
+cd /backupdockerscripts
-- it will create a timestamped archive of each container in the backup directory, under a directory with that container name.
+sudo cp .env.example .env
-
+sudo cp containers.txt containers.local.txt
-```
+sudo chmod 600 .env
-Usage:  ./docker-backup.sh [-f containers_list_file] [-s source_base_dir] [-t] [-v] [-h]
+sudo chmod +x backup.sh stages/*.sh
-
-Flags:
- -t          Test (dry run; no backups performed)
- -v          Verbose (debug logging to stderr)
- -f FILE     File containing list of container names
-             Default: '$containersfile'
- -s DIR      Base directory containing container/project folders
-             Default: '$SOURCE_BASE_DIR'
- -h          Show this help
 ```
 
-## PRUNE and REMOTE BACKUP dockerbackup_prune.sh
+Edit `.env`. A typical Docker source and example host name are represented by:
 
-Set up the env file using the example. 
+```bash
-```cp .env.example .env```
+SOURCE_BASE_DIR="/dockervolumes"
-
+BACKUP_HOSTNAME="examplehostname"
-You don't need to worry about the rclone info (backups to google drive) it you won't be using that.
-It will just backup up the file locally.  Feel free to adjust for other remote backup systems.
-
-If you run .dockerbackup_prune.sh it will prune the number of backup archives that are already there (based on your prune numbers), make a .zst archive of all container backups
-then finally uploades to google drive and deletes the large archive (if that's what you set).
-Then it will prune the number of archive files for each 
-
-**local archive only (no pruning)**
-```./dockerbackup_prune.sh```
-
-**local archive + real prune, keep new archive locally. Default prune is 5 archives)**
-```./dockerbackup_prune.sh --apply```
-
-**local archive + upload + prune dry-run (no actual prune), keep new archive locally (default location is /tmp)**
-```./dockerbackup_prune.sh --rclone```
-
-**full remote archive, google drive upload + real prune (default 5) + delete new local archive**
-```./dockerbackup_prune.sh --rclone --apply```
-
-**Full remote workflow with actual prune leaving 3 archives of each container and final full archive delete.**
-**Specify location of docker archives, final archive dir and remote google drive folder.**
-```./dockerbackup_prune.sh --root /mydockerarchives --/mybigbackupdolder --keep 3 --rclone /dockergooglebackup --apply
-
-Usage: $0 [--apply] [--quiet] [--dry-run] [--keep N] [--root PATH] [--source PATH] [--outdir PATH] [--remote PATH]
-
-Flags:
-  --rclone       Enable rclone upload flow
-  --apply        Actually delete pruned files; with --rclone also delete the new local archive
-  --quiet        Minimal console output (no live progress)
-  --dry-run      Simulate everything (no backup, no upload, no deletes)
-  --keep N       Keep newest N .zst files per subfolder (default ${KEEP_COUNT})
-  --root PATH    Root folder to prune (default ${PRUNE_ROOT})
-  --source PATH  Folder to archive (default ${SOURCE_DIR})
-  --outdir PATH  Where to write the .zst (default ${OUT_DIR})
-  --remote PATH  rclone destination (default ${RCLONE_REMOTE})
 ```
+
+Choose and create a local staging path for `BACKUP_ROOT`. `/dockerbackups` is
+the default, but it should be on a filesystem with enough free space to hold
+at least `KEEP_COUNT` copies of every listed project.
+
+Edit the server's untracked `containers.local.txt`. Each active line can be either a running
+container name or a directory name beneath `/dockervolumes`. Blank lines and
+text after `#` are ignored. If several entries resolve to the same Compose
+project directory, that project is backed up only once. Keeping this host-local
+file out of Git prevents a pull on any server from replacing its local list.
+When a listed directory contains other running Compose projects, the archive
+stage also stops those nested projects before archiving and starts them again
+afterward. Intentionally stopped nested projects are left stopped.
+
+## Configuration
+
+Copy `.env.example` to `.env`; `.env` is ignored by Git. Important values:
+
+| Variable | Purpose |
+|---|---|
+| `SOURCE_BASE_DIR` | Parent of Compose project directories |
+| `BACKUP_ROOT` | Local archive staging and retention tree |
+| `CONTAINERS_FILE` | Absolute path, or path relative to this repository |
+| `BACKUP_HOSTNAME` | Host folder/name embedded in archive filenames |
+| `BACKUP_METHODS` | Ordered method names, default `rclone smb` |
+| `ZSTD_EXTRA_OPTS` | Options passed to `zstd`, default `-T0 -12` |
+| `KEEP_COUNT` | Local archives retained per Compose project |
+| `PULL_IMAGES` | Pull images before restarting (`1` or `0`) |
+
+### Rclone
+
+```bash
+RCLONE_ENABLED=1
+RCLONE_REMOTE="gdrive:dockerbackups"
+```
+
+Archives are copied beneath `<remote>/<BACKUP_HOSTNAME>/`. The method uses
+`rclone copy`, so it does not delete older destination files. Set
+`RCLONE_HOST_SUBDIR=0` when `RCLONE_REMOTE` already names the final host-specific
+destination; otherwise the host name is appended automatically.
+
+### SMB
+
+Mount the SMB share through `/etc/fstab` or a systemd mount unit; do not put SMB
+credentials in this repository. Then configure:
+
+```bash
+SMB_ENABLED=1
+SMB_MOUNT="/mnt/dockerbackups"
+SMB_HOST_SUBDIR=1
+```
+
+The script refuses to copy unless `SMB_MOUNT` is an actual mount point. This
+prevents a failed network mount from silently filling the server's root disk.
+
+## Running
+
+Run the complete pipeline:
+
+```bash
+sudo /backupdockerscripts/backup.sh
+```
+
+Preview all stages without stopping containers, copying, or deleting:
+
+```bash
+sudo /backupdockerscripts/backup.sh --dry-run
+```
+
+Select a different container list for one complete pipeline run:
+
+```bash
+sudo /backupdockerscripts/backup.sh --containers containers-special.txt
+```
+
+Relative container-list paths are resolved from the repository directory;
+absolute paths are also accepted. The selected list affects only the container
+archive stage, after which configured transfer methods and pruning continue.
+
+Run individual parts:
+
+```bash
+sudo stages/10-containers.sh --dry-run
+sudo stages/20-rclone.sh --dry-run --force
+sudo stages/30-smb.sh --dry-run --force
+sudo stages/90-prune.sh --dry-run
+```
+
+`--force` runs a transfer method even when its `*_ENABLED` value is false. It
+still requires the method's destination to be configured and available.
+
+## MQTT reporting and Home Assistant trigger
+
+Enable retained run results in the host's `.env`:
+
+```bash
+MQTT_REPORT_ENABLED=1
+MQTT_HOST="examplehostname"
+MQTT_PORT=1883
+MQTT_USERNAME="dockerbackup"
+MQTT_PASSWORD_FILE="/etc/zorruno-docker-backup/mqtt-password"
+MQTT_STATUS_TOPIC="viewroad-status/dockerbackup/examplehostname"
+MQTT_COMMAND_TOPIC="viewroad-commands/dockerbackup/examplehostname/run"
+```
+
+The retained status JSON includes overall status, timestamps, duration, exit
+code, container totals, successful backups, dry-run counts, and error messages.
+Final reporting runs from an exit trap, so failures in any earlier stage are
+reported when MQTT remains available.
+
+Install the optional root-run command listener with:
+
+```bash
+cp systemd/zorruno-docker-backup-mqtt.service /etc/systemd/system/
+systemctl daemon-reload
+systemctl enable --now zorruno-docker-backup-mqtt.service
+```
+
+It accepts only JSON containing the exact action `run` on the configured
+command topic:
+
+```json
+{"action":"run"}
+```
+
+Publish commands without MQTT retain. The listener also ignores retained
+messages, preventing an old command from starting another backup after a
+listener or broker restart.
+
+Restrict publish access to the command topic with broker ACLs. The listener
+runs as root because the backup stage controls Docker containers. The existing
+pipeline lock prevents concurrent runs.
+
+## Adding another backup method
+
+Add an executable script named `stages/NN-name.sh`, for example
+`stages/40-restic.sh`. It should load `lib/common.sh`, accept `--dry-run`, and
+return a non-zero status on failure. Add its short name to `.env`:
+
+```bash
+BACKUP_METHODS="rclone smb restic"
+```
+
+The main pipeline discovers the numbered file from the short name and runs it
+after the earlier methods. A failed method stops the pipeline, so pruning will
+not run after a failed transfer.
+
+## Archive and restore format
+
+Archives are stored as:
+
+```text
+<BACKUP_ROOT>/<project>/<host>.<project>.<date_time>.tar.zst
+```
+
+Each archive contains the project's top-level directory rather than absolute
+paths. Example restore into an empty temporary directory:
+
+```bash
+mkdir -p /tmp/docker-restore
+tar --zstd -xf backup.tar.zst -C /tmp/docker-restore
+```
+
+Inspect and verify restored content before replacing a live project.

backup.sh

@@ -0,0 +1,164 @@
+#!/usr/bin/env bash
+# Main pipeline orchestrator.
+#
+# Execution order:
+#   1. Create cold archives for the selected Docker Compose projects.
+#   2. Run each enabled transfer stage listed in BACKUP_METHODS.
+#   3. Prune old local archives only after every earlier stage succeeds.
+#
+# An EXIT trap publishes the final MQTT result even when a stage fails. The
+# original exit code is preserved so cron, systemd and shell callers can detect
+# failure normally.
+set -Eeuo pipefail
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+. "$SCRIPT_DIR/lib/common.sh"
+DRY_RUN=0; SKIP_CONTAINERS=0; SKIP_PRUNE=0; CONTAINERS_OVERRIDE=""
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --dry-run|-t) DRY_RUN=1 ;; --skip-containers) SKIP_CONTAINERS=1 ;; --skip-prune) SKIP_PRUNE=1 ;;
+    --containers|-f) CONTAINERS_OVERRIDE="${2:?Missing containers file}"; shift ;;
+    -h|--help)
+      echo "Usage: ${0##*/} [--dry-run] [--containers FILE] [--skip-containers] [--skip-prune]"
+      echo "Transfer stages run in BACKUP_METHODS order. Add stages/NN-name.sh and list name in BACKUP_METHODS."
+      exit 0 ;;
+    *) die "Unknown option: $1" 2 ;;
+  esac
+  shift
+done
+
+if (( SKIP_CONTAINERS )); then
+  BACKUP_LOCAL_ARCHIVE_STATUS="skipped"
+else
+  BACKUP_LOCAL_ARCHIVE_STATUS="running"
+fi
+BACKUP_RCLONE_STATUS="not_configured"
+BACKUP_SMB_STATUS="not_configured"
+for configured_method in $BACKUP_METHODS; do
+  case "$configured_method" in
+    rclone) if is_true "$RCLONE_ENABLED"; then BACKUP_RCLONE_STATUS="pending"; else BACKUP_RCLONE_STATUS="skipped"; fi ;;
+    smb) if is_true "$SMB_ENABLED"; then BACKUP_SMB_STATUS="pending"; else BACKUP_SMB_STATUS="skipped"; fi ;;
+  esac
+done
+BACKUP_REMOTE_COPY_STATUS="pending"
+if [[ "$BACKUP_RCLONE_STATUS" == "not_configured" || "$BACKUP_RCLONE_STATUS" == "skipped" ]] && [[ "$BACKUP_SMB_STATUS" == "not_configured" || "$BACKUP_SMB_STATUS" == "skipped" ]]; then
+  BACKUP_REMOTE_COPY_STATUS="skipped"
+fi
+BACKUP_REMOTE_COPY_FINISHED_AT=""
+export BACKUP_LOCAL_ARCHIVE_STATUS BACKUP_REMOTE_COPY_STATUS BACKUP_RCLONE_STATUS BACKUP_SMB_STATUS BACKUP_REMOTE_COPY_FINISHED_AT
+
+BACKUP_RUN_ID="$(run_id)"
+BACKUP_STARTED_EPOCH="$(date +%s)"
+# Every script sourced through this process appends structured log lines here.
+# The MQTT reporting stage parses this temporary log to build its summary.
+BACKUP_RUN_LOG="$(mktemp "${TMPDIR:-/tmp}/zorruno-docker-backup.XXXXXX.log")"
+export BACKUP_RUN_ID BACKUP_STARTED_EPOCH BACKUP_RUN_LOG
+
+finish_run() {
+  local exit_code=$? finished status archived_count remote_success=0 remote_failure=0 remote_running=0 remote_dry_run=0 transfer_status
+  # Prevent recursion when this function exits with the pipeline's status.
+  trap - EXIT
+  # Reporting is best-effort and must never replace the real pipeline result.
+  set +e
+  finished="$(date +%s)"
+  if [[ "$BACKUP_LOCAL_ARCHIVE_STATUS" == "running" ]]; then
+    if (( DRY_RUN )); then
+      BACKUP_LOCAL_ARCHIVE_STATUS="dry_run"
+    elif (( exit_code == 0 )); then
+      BACKUP_LOCAL_ARCHIVE_STATUS="success"
+    else
+      archived_count="$(grep -c "\[ARCHIVE\] Created " "$BACKUP_RUN_LOG" || true)"
+      if (( archived_count > 0 )); then BACKUP_LOCAL_ARCHIVE_STATUS="partial"; else BACKUP_LOCAL_ARCHIVE_STATUS="failed"; fi
+    fi
+  fi
+  for transfer_status in BACKUP_RCLONE_STATUS BACKUP_SMB_STATUS; do
+    if [[ "${!transfer_status}" == "running" ]]; then
+      if (( DRY_RUN )); then printf -v "$transfer_status" "%s" "dry_run"; else printf -v "$transfer_status" "%s" "failed"; fi
+    elif [[ "${!transfer_status}" == "pending" ]]; then
+      printf -v "$transfer_status" "%s" "not_attempted"
+    fi
+  done
+  for transfer_status in "$BACKUP_RCLONE_STATUS" "$BACKUP_SMB_STATUS"; do
+    case "$transfer_status" in
+      success) (( remote_success++ )) ;;
+      partial) (( remote_success++ )); (( remote_failure++ )) ;;
+      failed) (( remote_failure++ )) ;;
+      running|pending) (( remote_running++ )) ;;
+      dry_run) (( remote_dry_run++ )) ;;
+    esac
+  done
+  if (( remote_running > 0 )); then
+    BACKUP_REMOTE_COPY_STATUS="running"
+  elif (( remote_failure > 0 && remote_success > 0 )); then
+    BACKUP_REMOTE_COPY_STATUS="partial"
+  elif (( remote_failure > 0 )); then
+    BACKUP_REMOTE_COPY_STATUS="failed"
+  elif (( remote_success > 0 )); then
+    BACKUP_REMOTE_COPY_STATUS="success"
+  elif (( remote_dry_run > 0 )); then
+    BACKUP_REMOTE_COPY_STATUS="dry_run"
+  else
+    BACKUP_REMOTE_COPY_STATUS="skipped"
+  fi
+  if (( exit_code == 0 )); then
+    if (( DRY_RUN )); then status="dry_run_success"; else status="success"; fi
+  else
+    if (( DRY_RUN )); then status="dry_run_failed"; else status="failed"; fi
+  fi
+  if is_true "$MQTT_REPORT_ENABLED"; then
+    "$SCRIPT_DIR/stages/99-mqtt-report.sh" "$status" "$exit_code" \
+      "$BACKUP_STARTED_EPOCH" "$finished" "$BACKUP_RUN_ID" "$BACKUP_RUN_LOG" \
+      || log ERROR "Unable to publish final MQTT result"
+  fi
+  rm -f -- "$BACKUP_RUN_LOG"
+  exit "$exit_code"
+}
+trap finish_run EXIT
+
+require_command flock
+mkdir -p "$(dirname "$LOCK_FILE")"
+# Keep descriptor 9 open for the whole run. Closing it automatically releases
+# the advisory lock, including on errors and signals that terminate the shell.
+exec 9>"$LOCK_FILE"
+flock -n 9 || die "Another backup run holds lock: $LOCK_FILE"
+stage_args=()
+(( DRY_RUN )) && stage_args+=(--dry-run)
+[[ -n "$CONTAINERS_OVERRIDE" ]] && stage_args+=(--containers "$CONTAINERS_OVERRIDE")
+log START "Backup pipeline for $BACKUP_HOSTNAME"
+if is_true "$MQTT_REPORT_ENABLED"; then
+  # The retained running state lets Home Assistant show that work is active.
+  "$SCRIPT_DIR/stages/99-mqtt-report.sh" running 0 "$BACKUP_STARTED_EPOCH" \
+    "$BACKUP_STARTED_EPOCH" "$BACKUP_RUN_ID" "$BACKUP_RUN_LOG" \
+    || log WARN "Unable to publish MQTT running state; backup will continue"
+fi
+if (( SKIP_CONTAINERS )); then
+  BACKUP_LOCAL_ARCHIVE_STATUS="skipped"
+else
+  "$SCRIPT_DIR/stages/10-containers.sh" "${stage_args[@]}"
+  if (( DRY_RUN )); then BACKUP_LOCAL_ARCHIVE_STATUS="dry_run"; else BACKUP_LOCAL_ARCHIVE_STATUS="success"; fi
+fi
+method_args=(); (( DRY_RUN )) && method_args+=(--dry-run)
+# BACKUP_METHODS contains short names (for example: "rclone smb"). Numeric
+# prefixes control directory ordering without becoming part of configuration.
+for method in $BACKUP_METHODS; do
+  method_script=""
+  while IFS= read -r candidate; do method_script="$candidate"; break; done < <(find "$SCRIPT_DIR/stages" -maxdepth 1 -type f -name "*-${method}.sh" -print | sort)
+  [[ -n "$method_script" ]] || die "Configured backup method not found: $method"
+  case "$method" in
+    rclone) if is_true "$RCLONE_ENABLED"; then BACKUP_RCLONE_STATUS="running"; fi ;;
+    smb) if is_true "$SMB_ENABLED"; then BACKUP_SMB_STATUS="running"; fi ;;
+  esac
+  "$method_script" "${method_args[@]}"
+  if (( DRY_RUN )); then
+    case "$method" in
+      rclone) [[ "$BACKUP_RCLONE_STATUS" == "running" ]] && BACKUP_RCLONE_STATUS="dry_run" ;;
+      smb) [[ "$BACKUP_SMB_STATUS" == "running" ]] && BACKUP_SMB_STATUS="dry_run" ;;
+    esac
+  else
+    case "$method" in
+      rclone) if [[ "$BACKUP_RCLONE_STATUS" == "running" ]]; then BACKUP_RCLONE_STATUS="success"; BACKUP_REMOTE_COPY_FINISHED_AT="$(date -u +%FT%TZ)"; fi ;;
+      smb) if [[ "$BACKUP_SMB_STATUS" == "running" ]]; then BACKUP_SMB_STATUS="success"; BACKUP_REMOTE_COPY_FINISHED_AT="$(date -u +%FT%TZ)"; fi ;;
+    esac
+  fi
+done
+(( SKIP_PRUNE )) || "$SCRIPT_DIR/stages/90-prune.sh" "${method_args[@]}"
+log DONE "Backup pipeline completed successfully"

bin/mqtt-trigger-listener.sh

@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+# Long-running MQTT command listener, normally managed by systemd as root.
+# Accepts only {"action":"run"}; broker authentication and ACLs should restrict
+# who can publish to MQTT_COMMAND_TOPIC.
+set -Eeuo pipefail
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd -P)"
+. "$SCRIPT_DIR/lib/common.sh"
+. "$SCRIPT_DIR/lib/mqtt.sh"
+
+require_command jq
+require_command mosquitto_sub
+[[ -n "$MQTT_HOST" ]] || die "MQTT_HOST is not configured"
+mqtt_common_args
+log MQTT "Listening on $MQTT_COMMAND_TOPIC"
+
+# Never replay a retained command after this listener or the MQTT broker restarts.
+# Home Assistant should also publish run commands without the retain flag.
+mosquitto_sub "${MQTT_ARGS[@]}" -R -t "$MQTT_COMMAND_TOPIC" | while IFS= read -r payload; do
+  action="$(jq -er '.action // empty' <<<"$payload" 2>/dev/null || true)"
+  if [[ "$action" != "run" ]]; then
+    log WARN "Ignored invalid MQTT command"
+    continue
+  fi
+  log MQTT "Accepted backup run command"
+  # backup.sh owns concurrency control. A command received during an active run
+  # fails safely on the same flock used by cron and manual invocations.
+  "$SCRIPT_DIR/backup.sh" || log ERROR "MQTT-triggered backup failed"
+done

containers.txt

@@ -1,58 +1,9 @@
 # Container names
 
-
-#bitnami
-#containers.panda.old
-#containers.panda.txt
-#dashbling
-#_disabled-chrome2mqtt
-#_disabled-chromecast-mqtt-connector
-#_disabled-huginn
-#_disabled-ipsec-vpn-server
-#_disabled-isd
-#_disabled-rtl-mqtt
-#_disabled-rtl-sdr
-#_disabled-speedy
-#_disabled-tasmoadmin
-#_disabled-tasmobackup
-zigbee2mqtt
-#_disabled-zigbee2mqttassistant
-#_disabled-zoneminder
 diun
-#docker-backup.sh
-#docker-compose.yml.sample
-docker-ntp
-#down-backup-up
-#evnex-mqtt
-gitea
 grafana
-#guacamole
+homeassistant
-#homeassistant
-#influxdb
 influxdb2
-mediawiki
 mosquitto
-mqcontrol-scripts
-#mqtt-chromecast
-#mqttview
-mqtt-watchdog
-nodered
-nsupdater
-ombi
-#openvpn-as
 pihole
 portainer
-rtl-433tomqtt
-rustdesk
-#smashing
-speedtest
-#tasmota-device-manager
-tautulli
-telegraf
-traefik
-ttyd
-#tvheadend
-#weewx
-whoami
-wireguard
-zigbee2mqtt

docker-backup.sh

@@ -1,378 +0,0 @@
-#!/usr/bin/env bash
-# ORIGINAL AUTHOR: kmw (updates by zorruno)
-# HISTORY:
-#   2026-04-20 - add configurable SOURCE_BASE_DIR and -s option for source path fallback.
-#   2025-08-22 - add start wait with one retry; fix find_working_dir typo; tidy.
-#   2025-08-22 - tidy, stricter bash, compose detection, better checks, clearer logging.
-#   2022-03-25 - add -t option for dry-run (no backups performed).
-#   2022-01-15 - fix error reporting when a container is not found.
-#   2022-01-09 - ignore trailing comments
-#   2022-01-03 - display help; list from file; -f option
-#   2021-10-16 - structure and logging
-#   2020-05-ish - first version
-
-set -Eeuo pipefail
-
-timestamp_format='%Y-%m-%d_%H-%M-%S'
-
-# verbosity: set non-empty to enable debug
-dbg=
-# dbg=VERBOSE
-
-# default containers list file
-containersfile="containers.txt"
-
-# dry run (or pass -t)
-dryrun=
-
-# base directory for backups
-BACKUP_DIR="/dockerbackups"
-
-# base directory for source docker project folders
-SOURCE_BASE_DIR="${SOURCE_BASE_DIR:-/dockervolumes}"
-
-# start/health wait tuning
-START_TIMEOUT=90     # seconds to wait for service to be up
-WAIT_INTERVAL=3      # seconds between polls
-RETRIES=1            # number of restart retries on failure (per container)
-
-# log file
-logdir="$HOME/.local/share/log/${0##*/}"
-mkdir -p "$logdir"
-ymdhms="$(/bin/date +'%Y-%m-%d_%H:%M:%S')"
-ymd="${ymdhms%%_*}"
-logfile="$logdir/${0##*/}_${ymd}.log"
-
-# compose command detection
-COMPOSE_CMD=()
-if docker compose version >/dev/null 2>&1; then
-  COMPOSE_CMD=(docker compose)
-elif command -v docker-compose >/dev/null 2>&1; then
-  COMPOSE_CMD=(docker-compose)
-else
-  echo "ERROR: Neither 'docker compose' nor 'docker-compose' found." >&2
-  exit 3
-fi
-
-log() {
-  local timestamp
-  timestamp="$(/bin/date +"$timestamp_format")"
-  while [ "$#" -gt 0 ]; do
-    echo "$timestamp: ${0##*/}: $1" >>"$logfile"
-    if [ -n "$dbg" ]; then
-      echo "$timestamp LOG: ${0##*/}: $1" >&2
-    else
-      echo "$1" >&2
-    fi
-    shift
-  done
-}
-
-debug() {
-  local timestamp
-  timestamp="$(/bin/date +"$timestamp_format")"
-  while [ "$#" -gt 0 ]; do
-    echo "$timestamp DBG: ${0##*/}: $1" >>"$logfile"
-    if [ -n "$dbg" ]; then
-      echo "$timestamp DBG: ${0##*/}: $1" >&2
-    fi
-    shift
-  done
-}
-
-fail() {
-  local exitcode="$1"
-  shift
-  log "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
-  log "${0##*/}: FATAL ERROR"
-  while [ "$#" -gt 0 ]; do
-    log "$1"
-    shift
-  done
-  sleep 1
-  tidy_up
-  exit "$exitcode"
-}
-
-tidy_up() {
-  debug "tidy_up"
-  cd || log "Failed to cd in tidy_up"
-  trap - EXIT
-  trap - INT
-}
-
-usage() {
-  cat 1>&2 <<EOF
-Usage:  ${0##*/} [-f containers_list_file] [-s source_base_dir] [-t] [-v] [-h]
-
- -t          Test (dry run; no backups performed)
- -v          Verbose (debug logging to stderr)
- -f FILE     File containing list of container names
-             Default: '$containersfile'
- -s DIR      Base directory containing container/project folders
-             Default: '$SOURCE_BASE_DIR'
- -h          Show this help
-
-Example:
-  ${0##*/} -f containers.servername.txt
-  ${0##*/} -f containers.servername.txt -s /dockervolumes
-EOF
-}
-
-parse_cmdline() {
-  while getopts ":f:hs:tv" opt; do
-    case "$opt" in
-      f) containersfile="$OPTARG" ;;
-      s) SOURCE_BASE_DIR="$OPTARG" ;;
-      h) usage; exit 0 ;;
-      t) dryrun=1 ;;
-      v) dbg=VERBOSE ;;
-      \?) echo "Invalid option: -$OPTARG" 1>&2; usage; exit 1 ;;
-      :)  echo "Option -$OPTARG requires an argument" 1>&2; exit 1 ;;
-    esac
-  done
-  shift $((OPTIND - 1))
-}
-
-toolcheck() {
-  local tool
-  while [ "$#" -gt 0 ]; do
-    tool="$1"
-    shift
-    debug "Checking for tool: '$tool'"
-    if ! command -v "$tool" >/dev/null 2>&1; then
-      fail 3 "Failed to find required tool: '$tool'"
-    fi
-  done
-}
-
-compose() {
-  "${COMPOSE_CMD[@]}" "$@"
-}
-
-# Return "<project> <service>" for a container by inspecting labels
-get_compose_identity() {
-  # arg: container_name
-  local name="$1"
-  local proj serv
-  proj="$(docker inspect "$name" 2>/dev/null | jq -r '.[0].Config.Labels["com.docker.compose.project"] // empty')"
-  serv="$(docker inspect "$name" 2>/dev/null | jq -r '.[0].Config.Labels["com.docker.compose.service"] // empty')"
-  if [ -z "$proj" ] || [ -z "$serv" ]; then
-    echo ""
-    return 1
-  fi
-  echo "$proj $serv"
-}
-
-# Check current state/health for any container matching project+service
-# Returns 0 if running and healthy (or no healthcheck), else 1
-service_is_healthy() {
-  # args: project service
-  local proj="$1"
-  local serv="$2"
-  local ids
-  # find containers by labels (may be multiple if scaled; any healthy one counts)
-  ids="$(docker ps -a -q --filter "label=com.docker.compose.project=$proj" --filter "label=com.docker.compose.service=$serv")" || true
-  if [ -z "$ids" ]; then
-    debug "No containers found for project=$proj service=$serv"
-    return 1
-  fi
-  local id
-  for id in $ids; do
-    # status: created, restarting, running, removing, paused, exited, dead
-    local st hlth
-    st="$(docker inspect -f '{{.State.Status}}' "$id" 2>/dev/null || echo "unknown")"
-    if [ "$st" != "running" ]; then
-      continue
-    fi
-    # Health may not exist; treat missing as pass if running
-    hlth="$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$id" 2>/dev/null || echo "none")"
-    if [ "$hlth" = "none" ] || [ "$hlth" = "healthy" ]; then
-      debug "Container $id is running (health=$hlth)"
-      return 0
-    fi
-  done
-  return 1
-}
-
-# Wait for service to become healthy/running within timeout
-wait_for_service() {
-  # args: project service timeout_seconds
-  local proj="$1"
-  local serv="$2"
-  local timeout="$3"
-  local start now
-  start="$(date +%s)"
-  while true; do
-    if service_is_healthy "$proj" "$serv"; then
-      return 0
-    fi
-    now="$(date +%s)"
-    if [ $(( now - start )) -ge "$timeout" ]; then
-      return 1
-    fi
-    sleep "$WAIT_INTERVAL"
-  done
-}
-
-down_backup_up() {
-  # args: WORKING_DIR PROJECT SERVICE
-  local WORKING_DIR="$1"
-  local PROJ="$2"
-  local SERV="$3"
-
-  local PROJECT_NAME
-  local BAK_YMDHMS
-  local BACKUP_CODE_DIR
-  local BACKUP_CODE_ZSTD
-
-  PROJECT_NAME="$(basename "$WORKING_DIR")"
-  BAK_YMDHMS="$(date +'%Y-%m-%d_%H-%M-%S')"
-  BACKUP_CODE_DIR="${BACKUP_DIR}/${PROJECT_NAME}"
-  BACKUP_CODE_ZSTD="${BACKUP_CODE_DIR}/$(hostname -s).${PROJECT_NAME}.${BAK_YMDHMS}.zst"
-
-  mkdir -p "${BACKUP_CODE_DIR}"
-  log "Backup target: ${BACKUP_CODE_ZSTD}"
-
-  if [ -n "$dryrun" ]; then
-    log "DRY-RUN: would run: compose down; tar --zstd; compose pull; compose up -d; wait for ${PROJ}/${SERV}"
-    return 0
-  fi
-
-  # stop, archive, start, then verify
-  compose down || log "WARNING: non-zero exit from 'compose down' (continuing)"
-  tar --one-file-system --zstd -cf "${BACKUP_CODE_ZSTD}" "${WORKING_DIR}"
-  ls -lh "${BACKUP_CODE_DIR}" || true
-  compose pull || log "WARNING: non-zero exit from 'compose pull' (continuing)"
-  compose up -d
-
-  if wait_for_service "$PROJ" "$SERV" "$START_TIMEOUT"; then
-    log "Service ${PROJ}/${SERV} is up."
-    return 0
-  fi
-
-  log "Service ${PROJ}/${SERV} did not become healthy in ${START_TIMEOUT}s. Retrying ($RETRIES)."
-  local attempt=1
-  while [ "$attempt" -le "$RETRIES" ]; do
-    log "Retry $attempt: restarting ${PROJ}/${SERV}"
-    compose down || log "WARNING: retry: non-zero from 'compose down'"
-    compose up -d
-    if wait_for_service "$PROJ" "$SERV" "$START_TIMEOUT"; then
-      log "Service ${PROJ}/${SERV} is up after retry."
-      return 0
-    fi
-    attempt=$(( attempt + 1 ))
-  done
-
-  fail 1 "Service ${PROJ}/${SERV} failed to start/healthy after ${RETRIES} retry(ies)."
-}
-
-find_working_dir() {
-  # args: CONTAINER_NAME
-  local name="$1"
-  local wd=""
-  if docker inspect "$name" >/dev/null 2>&1; then
-    wd="$(docker inspect "$name" 2>/dev/null | jq -r '.[0].Config.Labels["com.docker.compose.project.working_dir"] // empty')"
-  fi
-  if [ -z "$wd" ]; then
-    if [ -d "$SOURCE_BASE_DIR/$name" ]; then
-      wd="$SOURCE_BASE_DIR/$name"
-    elif [ -d "$HOME/prj/docker/$name" ]; then
-      wd="$HOME/prj/docker/$name"
-    else
-      wd=""
-    fi
-  fi
-  printf '%s' "$wd"
-}
-
-container_exists() {
-  docker ps -a --format '{{.Names}}' | grep -Fxq "$1"
-}
-
-backup_container() {
-  local container_name="$1"
-  log "Processing container '$container_name'"
-
-  local container_start_time container_finish_time container_duration container_duration_m
-  container_start_time="$(date +%s)"
-
-  if ! container_exists "$container_name"; then
-    log "Container not found: '$container_name' (skipping)"
-    return 0
-  fi
-
-  # capture compose identity before we stop anything
-  local identity proj serv
-  identity="$(get_compose_identity "$container_name" || true)"
-  if [ -z "$identity" ]; then
-    fail 1 "Could not determine compose identity (project/service) for '$container_name'."
-  fi
-  proj="$(echo "$identity" | awk '{print $1}')"
-  serv="$(echo "$identity" | awk '{print $2}')"
-  debug "Compose identity: project=$proj service=$serv"
-
-  local working_dir
-  working_dir="$(find_working_dir "$container_name")"
-  if [ -z "$working_dir" ]; then
-    fail 1 "Could not determine working_dir for container '$container_name'."
-  fi
-  log "Working dir: $working_dir"
-  if [ ! -d "$working_dir" ]; then
-    fail 1 "Working dir does not exist: '$working_dir'"
-  fi
-
-  local original_dir
-  original_dir="$(pwd)"
-  cd "$working_dir" || fail 1 "Could not cd to '$working_dir'"
-
-  down_backup_up "$working_dir" "$proj" "$serv" || log "WARNING: non-zero exit from backup function"
-
-  cd "$original_dir" || fail 1 "Could not cd back to '$original_dir'"
-
-  container_finish_time="$(date +%s)"
-  container_duration=$((container_finish_time - container_start_time))
-  container_duration_m=$((container_duration / 60))
-  log "Finished '$container_name' after ${container_duration}s (approx ${container_duration_m} min)"
-}
-
-trap tidy_up EXIT
-trap 'exit 130' INT
-
-parse_cmdline "$@"
-
-debug "-----"
-debug "logfile = '$logfile'"
-debug "source_base_dir = '$SOURCE_BASE_DIR'"
-
-toolcheck docker jq sed tar
-
-if [ ! -f "$containersfile" ]; then
-  fail 2 "Containers file not found: '$containersfile'"
-fi
-if [ ! -r "$containersfile" ]; then
-  fail 2 "Containers file not readable: '$containersfile'"
-fi
-
-containers="$(sed -rn 's/^ +//; s/#.*//; s/ +$//; /^$/b; p' "$containersfile" || true)"
-
-if [ -z "${containers:-}" ]; then
-  log "No containers listed in '$containersfile' (nothing to do)."
-  exit 0
-fi
-
-script_start_time="$(date +%s)"
-
-for container in $containers; do
-  backup_container "$container"
-done
-
-script_finish_time="$(date +%s)"
-script_duration=$((script_finish_time - script_start_time))
-script_duration_m=$((script_duration / 60))
-
-debug "-----"
-log "Script finished after ${script_duration}s (${script_duration_m} min)"
-
-exit 0
-# EOF

dockerbackup_prune.sh

@@ -1,399 +0,0 @@
-#!/usr/bin/env bash
-set -Eeuo pipefail
-
-###############################################################################
-# dockerbackup_prune.sh
-#
-# Default:
-#   A) Create a dated .zst backup locally and keep it
-#
-# With --apply:
-#   A) Create a dated .zst backup
-#   C) Prune *.zst under PRUNE_ROOT subfolders, keeping newest N per subfolder
-#   D) Keep the new local archive
-#
-# With --rclone:
-#   A) Create a dated .zst backup
-#   B) Upload via rclone
-#   C) Show what prune would do (unless --apply is also set)
-#   D) Keep the new local archive unless --apply is also set
-#
-# With --rclone --apply:
-#   A) Create a dated .zst backup
-#   B) Upload via rclone
-#   C) Prune for real
-#   D) Delete the new local archive
-#
-# Restore:
-#   zstd -d -c /path/backup.zst | tar -x -C /restore/target
-###############################################################################
-
-# --------------------------- Defaults (can be overridden) --------------------
-SOURCE_DIR="/mybackups"                             # what to archive
-PRUNE_ROOT="/mybackups"                             # where to prune .zst
-PRUNE_PATTERN="*.zst"                               # files to prune
-KEEP_COUNT=5                                        # keep newest N per subfolder
-
-RCLONE_REMOTE="gdrive:location/sublocation"         # rclone destination
-OUT_DIR="/tmp"                                      # where to write the .zst
-BACKUP_PREFIX="panda-dockerbackup"                  # backup name prefix
-
-# Optional zstd tuning, e.g. "-19 -T0" for max level, multithread
-ZSTD_EXTRA_OPTS=""
-
-# --------------------------- Load .env (same dir as script) -----------------
-# Order of precedence:
-#   defaults < .env (same dir) < CLI flags
-SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
-ENV_FILE="${DOCKERBACKUP_ENV:-$SCRIPT_DIR/.env}"
-if [[ -f "$ENV_FILE" ]]; then
-  perms="$(stat -c '%a' "$ENV_FILE" 2>/dev/null || stat -f '%Lp' "$ENV_FILE" 2>/dev/null || echo '')"
-  if [[ -n "$perms" && ! "$perms" =~ ^(600|400)$ ]]; then
-    echo "$(date '+%F %T') [WARN] $ENV_FILE permissions are $perms; recommend 600." >&2
-  fi
-  set -a
-  # shellcheck disable=SC1090
-  . "$ENV_FILE"
-  set +a
-fi
-
-# --------------------------- CLI flags --------------------------------------
-APPLY=0
-QUIET=0
-DRYRUN=0
-USE_RCLONE=0
-
-usage() {
-  cat <<EOF
-Usage: $0 [--rclone] [--apply] [--quiet] [--dry-run] [--keep N] [--root PATH] [--source PATH] [--outdir PATH] [--remote PATH]
-
-Default:
-  Creates the local .zst archive and keeps it locally.
-  No rclone upload. No prune.
-
-With --apply:
-  Creates the local .zst archive, prunes old .zst files for real, and keeps the new archive locally.
-
-With --rclone:
-  Uploads the archive via rclone.
-  Prune also runs, but only as a dry-run unless --apply is also set.
-  The new local archive is kept unless --apply is also set.
-
-With --rclone --apply:
-  Uploads the archive, prunes for real, and deletes the new local archive.
-
-Flags:
-  --rclone       Enable rclone upload flow
-  --apply        Actually delete pruned files; with --rclone also delete the new local archive
-  --quiet        Minimal console output (no live progress)
-  --dry-run      Simulate everything (no backup, no upload, no deletes)
-  --keep N       Keep newest N .zst files per subfolder (default ${KEEP_COUNT})
-  --root PATH    Root folder to prune (default ${PRUNE_ROOT})
-  --source PATH  Folder to archive (default ${SOURCE_DIR})
-  --outdir PATH  Where to write the .zst (default ${OUT_DIR})
-  --remote PATH  rclone destination (default ${RCLONE_REMOTE})
-
-Environment file:
-  A .env file in the same directory as this script is loaded automatically.
-  You can override its path by setting DOCKERBACKUP_ENV=/path/to/.env
-EOF
-}
-
-while [[ $# -gt 0 ]]; do
-  case "$1" in
-    --rclone) USE_RCLONE=1 ;;
-    --apply) APPLY=1 ;;
-    --quiet) QUIET=1 ;;
-    --dry-run) DRYRUN=1 ;;
-    --keep) KEEP_COUNT="${2:?}"; shift ;;
-    --root) PRUNE_ROOT="${2:?}"; shift ;;
-    --source) SOURCE_DIR="${2:?}"; shift ;;
-    --outdir) OUT_DIR="${2:?}"; shift ;;
-    --remote) RCLONE_REMOTE="${2:?}"; shift ;;
-    -h|--help) usage; exit 0 ;;
-    *) echo "Unknown option: $1" >&2; usage; exit 2 ;;
-  esac
-  shift
-done
-
-# --------------------------- Logging helpers --------------------------------
-ts() { date '+%F %T'; }
-log() { [[ $QUIET -eq 0 ]] && echo "$(ts) [$1] $2"; }
-log_err() { echo "$(ts) [ERROR] $1" >&2; }
-run() { if [[ $DRYRUN -eq 1 ]]; then log "DRY" "$*"; else "$@"; fi; }
-
-truncate_and_print() {
-  local s="$1" cols
-  if [[ -t 2 ]]; then
-    cols=$(tput cols 2>/dev/null || echo 120)
-  else
-    cols=120
-  fi
-  (( cols < 20 )) && cols=120
-  if (( ${#s} > cols - 1 )); then
-    s="${s:0:cols-1}"
-  fi
-  printf '\r%s\033[K' "$s" >&2
-}
-
-cleanup_tmp=""
-pipe_pid=""
-pipe_pgid=""
-pct_fd_open=0
-path_fd_open=0
-status_line_shown=0
-
-cleanup() {
-  local why="${1:-EXIT}"
-
-  if [[ -n "$pipe_pid" ]]; then
-    if command -v ps >/dev/null 2>&1; then
-      pipe_pgid=$(ps -o pgid= "$pipe_pid" 2>/dev/null | tr -d ' ' || true)
-    fi
-    if [[ -n "$pipe_pgid" ]]; then
-      kill -TERM "-$pipe_pgid" 2>/dev/null || true
-      sleep 0.2
-      kill -KILL "-$pipe_pgid" 2>/dev/null || true
-    else
-      kill -TERM "$pipe_pid" 2>/dev/null || true
-      sleep 0.2
-      kill -KILL "$pipe_pid" 2>/dev/null || true
-    fi
-    wait "$pipe_pid" 2>/dev/null || true
-  fi
-
-  if (( pct_fd_open == 1 )); then exec 4>&- 4<&-; pct_fd_open=0; fi
-  if (( path_fd_open == 1 )); then exec 3>&- 3<&-; path_fd_open=0; fi
-
-  if [[ -n "$cleanup_tmp" ]]; then
-    rm -f "$cleanup_tmp/pct" "$cleanup_tmp/path" 2>/dev/null || true
-    rmdir "$cleanup_tmp" 2>/dev/null || true
-  fi
-
-  if (( status_line_shown == 1 )) && [[ $QUIET -eq 0 ]]; then
-    printf '\n' >&2
-  fi
-
-  case "$why" in
-    INT|TERM) exit 130 ;;
-  esac
-}
-
-trap 'log_err "Command failed: $BASH_COMMAND"; cleanup ERR; exit 1' ERR
-trap 'cleanup INT' INT
-trap 'cleanup TERM' TERM
-trap 'cleanup EXIT' EXIT
-
-# --------------------------- Preconditions ----------------------------------
-[[ -d "$SOURCE_DIR" ]] || { log_err "Source dir not found: $SOURCE_DIR"; exit 1; }
-[[ -d "$PRUNE_ROOT" ]] || { log_err "Prune root not found: $PRUNE_ROOT"; exit 1; }
-command -v tar >/dev/null 2>&1 || { log_err "tar not found"; exit 1; }
-command -v zstd >/dev/null 2>&1 || { log_err "zstd not found"; exit 1; }
-command -v find >/dev/null 2>&1 || { log_err "find not found"; exit 1; }
-
-if (( USE_RCLONE == 1 )) && (( DRYRUN == 0 )); then
-  command -v rclone >/dev/null 2>&1 || { log_err "rclone not found"; exit 1; }
-fi
-
-STAMP="${STAMP_OVERRIDE:-$(date +%F_%H-%M)}"
-BACKUP_NAME="${BACKUP_PREFIX}-${STAMP}.zst"
-BACKUP_PATH="${OUT_DIR%/}/${BACKUP_NAME}"
-
-RCLONE_FLAGS=()
-[[ $QUIET -eq 1 ]] && RCLONE_FLAGS+=("-q") || RCLONE_FLAGS+=("--progress")
-
-ZSTD_FLAGS=("-q")
-[[ -n "$ZSTD_EXTRA_OPTS" ]] && ZSTD_FLAGS+=($ZSTD_EXTRA_OPTS)
-
-log "INFO" "Mode: $( [[ $DRYRUN -eq 1 ]] && echo 'DRY-RUN ALL' || ([[ $APPLY -eq 1 ]] && echo 'APPLY DELETES' || echo 'NO DELETES') )"
-log "INFO" "Rclone mode: $( [[ $USE_RCLONE -eq 1 ]] && echo 'ENABLED' || echo 'DISABLED' )"
-log "INFO" "Creating archive: $BACKUP_PATH (.zst)"
-log "INFO" "Keeping newest $KEEP_COUNT file(s) per subfolder matching: $PRUNE_PATTERN"
-log "INFO" "Source: $SOURCE_DIR"
-if (( USE_RCLONE == 1 )); then
-  log "INFO" "Remote: $RCLONE_REMOTE"
-fi
-log "INFO" "Prune root (root itself will be skipped): $PRUNE_ROOT"
-
-# --------------------------- Step A: create .zst with one-liner --------------
-log "STEP" "A) Creating archive"
-
-if [[ $DRYRUN -eq 1 ]]; then
-  log "DRY" "tar -C \"$SOURCE_DIR\" -cvf - . | pv -n -s \$(du -sb \"$SOURCE_DIR\"|cut -f1) 2>pct.fifo | zstd ${ZSTD_FLAGS[*]} -o \"$BACKUP_PATH\""
-else
-  mkdir -p "$OUT_DIR"
-
-  export LC_ALL=C
-  TOTAL_FILES=$(find "$SOURCE_DIR" -type f | wc -l | awk '{print $1}')
-  TOTAL_BYTES=$(du -sb "$SOURCE_DIR" | cut -f1)
-
-  tmpdir="$(mktemp -d)"; cleanup_tmp="$tmpdir"
-  path_fifo="$tmpdir/path"; pct_fifo="$tmpdir/pct"
-  mkfifo "$path_fifo"
-  have_pv=0
-  if command -v pv >/dev/null 2>&1 && [[ $QUIET -eq 0 ]] && [[ -t 2 ]]; then
-    mkfifo "$pct_fifo"; have_pv=1
-  fi
-
-  exec 3<> "$path_fifo"; path_fd_open=1
-  if (( have_pv == 1 )); then exec 4<> "$pct_fifo"; pct_fd_open=1; fi
-
-  if (( have_pv == 1 )); then
-    (
-      set -o pipefail
-      tar -C "$SOURCE_DIR" -cvf - . \
-        2> >(sed -u 's#^\./##' > "$path_fifo" || true) \
-      | pv -n -s "$TOTAL_BYTES" 2> "$pct_fifo" \
-      | zstd "${ZSTD_FLAGS[@]}" -o "$BACKUP_PATH"
-    ) &
-  else
-    (
-      set -o pipefail
-      tar -C "$SOURCE_DIR" -cvf - . \
-        2> >(sed -u 's#^\./##' > "$path_fifo" || true) \
-      | zstd "${ZSTD_FLAGS[@]}" -o "$BACKUP_PATH"
-    ) &
-  fi
-  pipe_pid=$!
-
-  files_seen=0
-  last_path=""
-  last_pct="0.0"
-  status_line_shown=0
-  last_print_time=0
-
-  while kill -0 "$pipe_pid" 2>/dev/null; do
-    if (( have_pv == 1 )); then
-      if IFS= read -r -t 0.03 pct_line <&4; then last_pct="$pct_line"; fi
-    fi
-
-    if IFS= read -r -t 0.03 path_line <&3; then
-      last_path="$path_line"
-      [[ -n "$last_path" ]] && (( files_seen++ )) || true
-    fi
-
-    display_rel="$last_path"
-    dir_part="$(dirname -- "$display_rel" 2>/dev/null || echo .)"
-    base_part="$(basename -- "$display_rel" 2>/dev/null || echo .)"
-    [[ "$dir_part" == "." ]] && dir_part="."
-
-    if [[ $QUIET -eq 0 ]]; then
-      now_ns=$(date +%s%N)
-      if (( now_ns - last_print_time > 50000000 )); then
-        if (( have_pv == 1 )); then
-          line="[$(printf '%.1f' "${last_pct:-0}")%] archiving: ${dir_part}/ ${base_part}"
-        else
-          line="[${files_seen}/${TOTAL_FILES}] archiving: ${dir_part}/ ${base_part}"
-        fi
-        truncate_and_print "$line"
-        status_line_shown=1
-        last_print_time=$now_ns
-      fi
-    fi
-  done
-
-  while IFS= read -r -t 0.03 path_line <&3; do
-    last_path="$path_line"; (( files_seen++ )) || true
-    [[ $QUIET -eq 0 ]] && truncate_and_print "[${files_seen}/${TOTAL_FILES}] archiving: $(dirname -- "$last_path")/ $(basename -- "$last_path")"
-    status_line_shown=1
-  done || true
-
-  wait "$pipe_pid"
-  if [[ $QUIET -eq 0 ]]; then
-    printf '\n' >&2
-    status_line_shown=0
-  fi
-
-  exec 3>&- 3<&-; path_fd_open=0
-  if (( have_pv == 1 )); then exec 4>&- 4<&-; pct_fd_open=0; fi
-  rm -f "$path_fifo" "$pct_fifo" 2>/dev/null || true
-  rmdir "$tmpdir" 2>/dev/null || true
-  cleanup_tmp=""
-fi
-log "OK" "Archive step completed"
-
-# --------------------------- Step B: optional rclone copy --------------------
-if (( USE_RCLONE == 1 )); then
-  log "STEP" "B) rclone copy to remote"
-  if [[ $DRYRUN -eq 1 ]]; then
-    log "DRY" "rclone copy \"$BACKUP_PATH\" \"$RCLONE_REMOTE\" ${RCLONE_FLAGS[*]}"
-  else
-    run rclone copy "$BACKUP_PATH" "$RCLONE_REMOTE" "${RCLONE_FLAGS[@]}"
-  fi
-  log "OK" "Upload step completed"
-else
-  log "SKIP" "--rclone not set; skipping upload"
-fi
-
-# --------------------------- Step C: prune if apply or rclone ---------------
-if (( APPLY == 1 || USE_RCLONE == 1 )); then
-  log "STEP" "C) Pruning $PRUNE_PATTERN under $PRUNE_ROOT (keep $KEEP_COUNT)"
-  export LC_ALL=C
-  prune_ok=1
-
-  while IFS= read -r -d '' dir; do
-    mapfile -d '' -t files < <(
-      find "$dir" -maxdepth 1 -type f -name "$PRUNE_PATTERN" -printf '%T@ %p\0' |
-        sort -z -n |
-        cut -z -d' ' -f2-
-    )
-    count=${#files[@]}
-    if (( count == 0 )); then
-      log "SKIP" "No matches in: $dir"
-      continue
-    fi
-
-    keep_from=$(( count - KEEP_COUNT ))
-    (( keep_from < 0 )) && keep_from=0
-
-    if (( count <= KEEP_COUNT )); then
-      log "KEEP" "At or below limit in: $dir (count=$count)"
-      continue
-    fi
-
-    keep_list=("${files[@]:keep_from:KEEP_COUNT}")
-    del_list=("${files[@]:0:keep_from}")
-
-    for k in "${keep_list[@]}"; do
-      log "KEEP" "Keep: $k"
-    done
-    for f in "${del_list[@]}"; do
-      if [[ $DRYRUN -eq 1 || $APPLY -eq 0 ]]; then
-        log "WOULD" "Remove: $f"
-      else
-        if rm -f -- "$f"; then
-          log "DELETE" "Removed: $f"
-        else
-          log_err "Failed to remove: $f"; prune_ok=0
-        fi
-      fi
-    done
-  done < <(find "$PRUNE_ROOT" -mindepth 1 -type d -print0)
-
-  if [[ $prune_ok -ne 1 ]]; then
-    log_err "Prune step encountered errors; aborting final delete"
-    exit 1
-  fi
-  log "OK" "Prune step completed"
-else
-  log "SKIP" "--apply and --rclone not set; skipping prune"
-fi
-
-# --------------------------- Step D: remove local .zst -----------------------
-if (( USE_RCLONE == 1 )); then
-  log "STEP" "D) Removing local backup"
-  if [[ $DRYRUN -eq 1 || $APPLY -eq 0 ]]; then
-    log "WOULD" "Remove local backup: $BACKUP_PATH"
-  else
-    if rm -f -- "$BACKUP_PATH"; then
-      log "DELETE" "Removed local backup: $BACKUP_PATH"
-    else
-      log_err "Failed to remove local backup: $BACKUP_PATH"
-      exit 1
-    fi
-  fi
-else
-  log "KEEP" "Local archive retained: $BACKUP_PATH"
-fi
-
-log "DONE" "All steps completed successfully"

lib/common.sh

@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+# Shared configuration, validation and logging helpers.
+#
+# Every executable script sources this file. Defaults are deliberately safe:
+# transfer methods and MQTT reporting are disabled until enabled in the
+# host-local, Git-ignored .env file.
+set -Eeuo pipefail
+
+SCRIPT_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd -P)"
+ENV_FILE="${DOCKERBACKUP_ENV:-$SCRIPT_ROOT/.env}"
+
+# Core archive pipeline defaults.
+SOURCE_BASE_DIR="/dockervolumes"
+BACKUP_ROOT="/dockerbackups"
+CONTAINERS_FILE="$SCRIPT_ROOT/containers.txt"
+BACKUP_HOSTNAME="$(hostname -s)"
+BACKUP_METHODS="rclone smb"
+KEEP_COUNT=5
+PULL_IMAGES=1
+START_TIMEOUT=90
+WAIT_INTERVAL=3
+START_RETRIES=1
+LOCK_FILE="/run/lock/zorruno-docker-backup.lock"
+ZSTD_EXTRA_OPTS="-T0 -12"
+# Optional MQTT reporting and command-listener defaults.
+MQTT_REPORT_ENABLED=0
+MQTT_HOST=""
+MQTT_PORT=1883
+MQTT_USERNAME=""
+MQTT_PASSWORD_FILE=""
+MQTT_STATUS_TOPIC="viewroad-status/dockerbackup/$BACKUP_HOSTNAME"
+MQTT_COMMAND_TOPIC="viewroad-commands/dockerbackup/$BACKUP_HOSTNAME/run"
+
+# Transfer-stage defaults. Host-specific destinations belong in .env.
+RCLONE_ENABLED=0
+RCLONE_REMOTE=""
+RCLONE_HOST_SUBDIR=1
+RCLONE_EXTRA_ARGS=""
+SMB_ENABLED=0
+SMB_MOUNT=""
+SMB_HOST_SUBDIR=1
+SMB_RSYNC_ARGS="-a"
+
+if [[ -f "$ENV_FILE" ]]; then
+  # Export loaded values so child stage scripts inherit the same configuration.
+  set -a
+  # shellcheck disable=SC1090
+  . "$ENV_FILE"
+  set +a
+fi
+
+# Ease migration from the original prune script's .env names.
+if [[ -n "${SOURCE_DIR:-}" && "$BACKUP_ROOT" == "/dockerbackups" ]]; then
+  BACKUP_ROOT="$SOURCE_DIR"
+fi
+
+timestamp() { date '+%F %T'; }
+log() {
+  local line
+  line="$(timestamp) [$1] $2"
+  printf '%s\n' "$line" >&2
+  # Directly executed stages have no BACKUP_RUN_LOG. Always return success so
+  # an informational log call cannot terminate a script running with set -e.
+  if [[ -n "${BACKUP_RUN_LOG:-}" ]]; then
+    printf '%s\n' "$line" >>"$BACKUP_RUN_LOG"
+  fi
+return 0
+}
+die() { log ERROR "$1"; exit "${2:-1}"; }
+
+is_true() {
+  case "${1,,}" in
+    1|true|yes|on|enabled) return 0 ;;
+    *) return 1 ;;
+  esac
+}
+
+require_command() {
+  command -v "$1" >/dev/null 2>&1 || die "Required command not found: $1" 3
+}
+
+require_positive_integer() {
+  [[ "$2" =~ ^[1-9][0-9]*$ ]] || die "$1 must be a positive integer (received: $2)" 2
+}
+
+resolve_path() {
+  # Host-local filenames are resolved relative to the repository, not the
+  # caller's current directory. Absolute paths pass through unchanged.
+  if [[ "$1" = /* ]]; then printf '%s\n' "$1"; else printf '%s/%s\n' "$SCRIPT_ROOT" "$1"; fi
+}
+
+run_id() { date '+%Y-%m-%d_%H-%M-%S'; }

lib/mqtt.sh

@@ -0,0 +1,23 @@
+#!/usr/bin/env bash
+# Shared mosquitto_pub/mosquitto_sub argument construction.
+# Passwords are read from a protected file rather than placed in .env or Git.
+
+mqtt_common_args() {
+  MQTT_ARGS=(-h "$MQTT_HOST" -p "$MQTT_PORT")
+  [[ -n "$MQTT_USERNAME" ]] && MQTT_ARGS+=(-u "$MQTT_USERNAME")
+  if [[ -n "$MQTT_PASSWORD_FILE" ]]; then
+    [[ -r "$MQTT_PASSWORD_FILE" ]] || die "MQTT password file is not readable: $MQTT_PASSWORD_FILE"
+    MQTT_ARGS+=(-P "$(<"$MQTT_PASSWORD_FILE")")
+  fi
+}
+
+mqtt_publish() {
+  local topic="$1" payload="$2" retain="${3:-0}"
+  [[ -n "$MQTT_HOST" ]] || die "MQTT_HOST is not configured"
+  require_command mosquitto_pub
+  mqtt_common_args
+  # Backup state is retained so Home Assistant receives the latest result after
+  # reconnecting. Command messages are handled separately and are never kept.
+  (( retain )) && MQTT_ARGS+=(-r)
+  mosquitto_pub "${MQTT_ARGS[@]}" -t "$topic" -m "$payload"
+}

stages/10-containers.sh

@@ -0,0 +1,213 @@
+#!/usr/bin/env bash
+# Stage 10: create a cold, compressed archive of each selected Compose project
+# directory, including any nested Compose projects.
+#
+# Entries may be Compose project-directory names or live container names. A
+# container name is resolved through its Compose labels. Duplicate entries that
+# resolve to the same working directory are processed only once.
+set -Eeuo pipefail
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+# shellcheck source=../lib/common.sh
+. "$SCRIPT_DIR/../lib/common.sh"
+
+DRY_RUN=0
+usage() {
+  cat <<EOF
+Usage: ${0##*/} [--dry-run] [--containers FILE] [--source DIR] [--backup-root DIR]
+
+Entries may be container names or Compose project directory names.
+EOF
+}
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --dry-run|-t) DRY_RUN=1 ;;
+    --containers|-f) CONTAINERS_FILE="${2:?Missing containers file}"; shift ;;
+    --source|-s) SOURCE_BASE_DIR="${2:?Missing source directory}"; shift ;;
+    --backup-root) BACKUP_ROOT="${2:?Missing backup root}"; shift ;;
+    --verbose|-v) : ;; # Retained for compatibility; current logging is always concise.
+    -h|--help) usage; exit 0 ;;
+    *) die "Unknown option: $1" 2 ;;
+  esac
+  shift
+done
+
+CONTAINERS_FILE="$(resolve_path "$CONTAINERS_FILE")"
+require_command docker
+require_command tar
+require_command zstd
+[[ -d "$SOURCE_BASE_DIR" ]] || die "Docker source directory not found: $SOURCE_BASE_DIR"
+[[ -r "$CONTAINERS_FILE" ]] || die "Containers file not readable: $CONTAINERS_FILE"
+
+if docker compose version >/dev/null 2>&1; then
+  COMPOSE=(docker compose)
+elif command -v docker-compose >/dev/null 2>&1; then
+  COMPOSE=(docker-compose)
+else
+  die "Neither 'docker compose' nor 'docker-compose' is available" 3
+fi
+compose() { "${COMPOSE[@]}" "$@"; }
+
+resolve_project() {
+  local entry="$1" working_dir="" project=""
+  # Prefer authoritative Compose labels when the entry names a container.
+  if docker inspect "$entry" >/dev/null 2>&1; then
+    working_dir="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project.working_dir"}}' "$entry" 2>/dev/null || true)"
+    project="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project"}}' "$entry" 2>/dev/null || true)"
+  fi
+  # Fall back to a project directory beneath SOURCE_BASE_DIR. This also allows
+  # useful dry-runs when the caller cannot access the Docker socket.
+  [[ -n "$working_dir" ]] || { [[ -d "$SOURCE_BASE_DIR/$entry" ]] && working_dir="$SOURCE_BASE_DIR/$entry"; }
+  [[ -n "$working_dir" ]] || return 1
+  [[ -n "$project" ]] || project="$(basename "$working_dir")"
+  printf '%s\t%s\n' "$project" "$working_dir"
+}
+
+project_is_ready() {
+  local project="$1" ids id status health
+  # Ignore one-off Compose jobs; all persistent project containers must be
+  # running and, where a healthcheck exists, healthy.
+  ids="$(docker ps -a -q --filter "label=com.docker.compose.project=$project" --filter "label=com.docker.compose.oneoff=False")"
+  [[ -n "$ids" ]] || return 1
+  for id in $ids; do
+    status="$(docker inspect -f '{{.State.Status}}' "$id")"
+    health="$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$id")"
+    [[ "$status" == running ]] || return 1
+    [[ "$health" == none || "$health" == healthy ]] || return 1
+  done
+}
+
+wait_for_project() {
+  local project="$1" started now
+  started="$(date +%s)"
+  while ! project_is_ready "$project"; do
+    now="$(date +%s)"
+    (( now - started < START_TIMEOUT )) || return 1
+    sleep "$WAIT_INTERVAL"
+  done
+}
+
+discover_child_projects() {
+  local root="$1" id row project working_dir
+  local -A seen=()
+  # Discover live Compose projects by their container labels. Only running
+  # projects are stopped/restarted, preserving intentionally stopped stacks.
+  while IFS= read -r id; do
+    [[ -n "$id" ]] || continue
+    row="$(docker inspect -f '{{printf "%s\t%s" (index .Config.Labels "com.docker.compose.project") (index .Config.Labels "com.docker.compose.project.working_dir")}}' "$id" 2>/dev/null || true)"
+    IFS=$'\t' read -r project working_dir <<<"$row"
+    [[ -n "$project" && -n "$working_dir" ]] || continue
+    [[ "$working_dir" == "$root"/* ]] || continue
+    [[ -n "${seen[$working_dir]:-}" ]] && continue
+    seen[$working_dir]=1
+    printf '%s\t%s\n' "$project" "$working_dir"
+  done < <(docker ps -q)
+}
+
+restart_children() {
+  local -a projects=("$@")
+  local record child_project child_dir failed=0
+  for record in "${projects[@]}"; do
+    IFS=$'\t' read -r child_project child_dir <<<"$record"
+    log INFO "Restarting nested project: $child_project ($child_dir)"
+    if ! (cd "$child_dir" && compose up -d); then
+      log ERROR "Unable to restart nested project $child_project"
+      failed=1
+    fi
+  done
+  return "$failed"
+}
+
+backup_project() {
+  local project="$1" working_dir="$2" parent base output_dir output attempt child_project child_dir record
+  local -a zstd_args=()
+  local -a child_records=() stopped_children=()
+  parent="$(dirname "$working_dir")"
+  base="$(basename "$working_dir")"
+  output_dir="$BACKUP_ROOT/$project"
+  output="$output_dir/${BACKUP_HOSTNAME}.${project}.$(run_id).tar.zst"
+  log INFO "Project: $project ($working_dir)"
+  log INFO "Archive: $output"
+  if (( DRY_RUN )); then
+    mapfile -t child_records < <(discover_child_projects "$working_dir")
+    log DRY "Would stop ${#child_records[@]} nested project(s), archive, then restart them with $project"
+    return 0
+  fi
+
+  mkdir -p "$output_dir"
+  mapfile -t child_records < <(discover_child_projects "$working_dir")
+  for record in "${child_records[@]}"; do
+    IFS=$'\t' read -r child_project child_dir <<<"$record"
+    log INFO "Stopping nested project: $child_project ($child_dir)"
+    # Track before down: Compose may partially stop a project and then fail.
+    stopped_children+=("$record")
+    if (cd "$child_dir" && compose down); then
+      :
+    else
+      log ERROR "Unable to stop nested project $child_project"
+      restart_children "${stopped_children[@]}" || true
+      return 1
+    fi
+  done
+  cd "$working_dir"
+  # Stopping the whole project creates a consistent filesystem-level backup.
+  if ! compose down; then
+    log ERROR "Unable to stop project $project"
+    compose up -d || log ERROR "Unable to restart project $project"
+    restart_children "${stopped_children[@]}" || true
+    return 1
+  fi
+  read -r -a zstd_args <<<"$ZSTD_EXTRA_OPTS"
+  if ! tar --one-file-system -cf - -C "$parent" "$base" | zstd "${zstd_args[@]}" -o "$output"; then
+    # Never leave a partial archive that a transfer stage could mistake for a
+    # valid backup. Make a best-effort restart before recording the failure.
+    rm -f -- "$output"
+    log ERROR "Archive failed; attempting to restart $project"
+    compose up -d || true
+    restart_children "${stopped_children[@]}" || true
+    return 1
+  fi
+  log ARCHIVE "Created $output"
+  if is_true "$PULL_IMAGES"; then
+    # Image updates are useful but not allowed to prevent restart with the
+    # already-cached image when a registry is temporarily unavailable.
+    compose pull || log WARN "Image pull failed for $project; starting with available images"
+  fi
+  compose up -d
+  restart_children "${stopped_children[@]}" || return 1
+
+  if wait_for_project "$project"; then log OK "$project is running and healthy"; return 0; fi
+  for (( attempt=1; attempt<=START_RETRIES; attempt++ )); do
+    log WARN "$project did not become ready; restart attempt $attempt/$START_RETRIES"
+    compose down || true
+    compose up -d
+    if wait_for_project "$project"; then log OK "$project is running after retry"; return 0; fi
+  done
+  log ERROR "$project did not become ready"
+  return 1
+}
+
+declare -A SEEN=()
+failures=0
+# Continue through the list so one failed project does not leave later projects
+# without a backup. The stage still exits non-zero after reporting all failures.
+while IFS= read -r entry; do
+  entry="${entry%%#*}"
+  entry="${entry#"${entry%%[![:space:]]*}"}"
+  entry="${entry%"${entry##*[![:space:]]}"}"
+  [[ -n "$entry" ]] || continue
+  if ! resolved="$(resolve_project "$entry")"; then
+    log ERROR "Could not resolve container or project: $entry"
+    (( failures++ )) || true
+    continue
+  fi
+  IFS=$'\t' read -r project working_dir <<<"$resolved"
+  if [[ -n "${SEEN[$working_dir]:-}" ]]; then
+    log SKIP "Project already processed: $project ($working_dir)"
+    continue
+  fi
+  SEEN[$working_dir]=1
+  backup_project "$project" "$working_dir" || { (( failures++ )) || true; }
+done < "$CONTAINERS_FILE"
+
+(( failures == 0 )) || die "$failures project backup(s) failed"
+log DONE "Container backup stage completed"

stages/20-rclone.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+# Stage 20: copy the local backup tree to an rclone destination.
+# Uses `rclone copy`, so files absent locally are not deleted remotely.
+set -Eeuo pipefail
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+. "$SCRIPT_DIR/../lib/common.sh"
+DRY_RUN=0; FORCE=0
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --dry-run) DRY_RUN=1 ;; --force) FORCE=1 ;;
+    -h|--help) echo "Usage: ${0##*/} [--dry-run] [--force]"; exit 0 ;;
+    *) die "Unknown option: $1" 2 ;;
+  esac
+  shift
+done
+if ! is_true "$RCLONE_ENABLED" && (( FORCE == 0 )); then log SKIP "rclone method is disabled"; exit 0; fi
+[[ -n "$RCLONE_REMOTE" ]] || die "RCLONE_REMOTE is not configured"
+require_command rclone
+[[ -d "$BACKUP_ROOT" ]] || die "Backup root not found: $BACKUP_ROOT"
+read -r -a extra_args <<<"$RCLONE_EXTRA_ARGS"
+destination="${RCLONE_REMOTE%/}"
+# Append the host name for shared destinations, or disable RCLONE_HOST_SUBDIR
+# when RCLONE_REMOTE already points at this host's final directory.
+is_true "$RCLONE_HOST_SUBDIR" && destination="$destination/$BACKUP_HOSTNAME"
+args=(copy "$BACKUP_ROOT" "$destination" "${extra_args[@]}")
+(( DRY_RUN )) && args+=(--dry-run)
+log INFO "Copying backups to $destination"
+rclone "${args[@]}"
+log DONE "rclone backup method completed"

stages/30-smb.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# Stage 30: copy the local backup tree to an already-mounted SMB share.
+# The script does not mount the share or store SMB credentials.
+set -Eeuo pipefail
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+. "$SCRIPT_DIR/../lib/common.sh"
+DRY_RUN=0; FORCE=0
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --dry-run) DRY_RUN=1 ;; --force) FORCE=1 ;;
+    -h|--help) echo "Usage: ${0##*/} [--dry-run] [--force]"; exit 0 ;;
+    *) die "Unknown option: $1" 2 ;;
+  esac
+  shift
+done
+if ! is_true "$SMB_ENABLED" && (( FORCE == 0 )); then log SKIP "SMB method is disabled"; exit 0; fi
+[[ -n "$SMB_MOUNT" ]] || die "SMB_MOUNT is not configured"
+require_command rsync
+require_command mountpoint
+[[ -d "$BACKUP_ROOT" ]] || die "Backup root not found: $BACKUP_ROOT"
+[[ -d "$SMB_MOUNT" ]] || die "SMB mount does not exist: $SMB_MOUNT"
+# Refuse an ordinary directory at the configured path. This prevents a failed
+# network mount from silently filling the server's local root filesystem.
+mountpoint -q "$SMB_MOUNT" || die "SMB path is not a mounted filesystem: $SMB_MOUNT"
+destination="${SMB_MOUNT%/}"
+is_true "$SMB_HOST_SUBDIR" && destination="$destination/$BACKUP_HOSTNAME"
+(( DRY_RUN )) || mkdir -p "$destination"
+read -r -a rsync_args <<<"$SMB_RSYNC_ARGS"
+(( DRY_RUN )) && rsync_args+=(--dry-run)
+log INFO "Copying backups to $destination"
+rsync "${rsync_args[@]}" "$BACKUP_ROOT/" "$destination/"
+log DONE "SMB backup method completed"

stages/90-prune.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+# Stage 90: prune local archives after successful archive and transfer stages.
+# Retention is applied independently to each immediate project directory.
+set -Eeuo pipefail
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+. "$SCRIPT_DIR/../lib/common.sh"
+DRY_RUN=0
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --dry-run) DRY_RUN=1 ;; --keep) KEEP_COUNT="${2:?Missing keep count}"; shift ;;
+    --root) BACKUP_ROOT="${2:?Missing backup root}"; shift ;;
+    -h|--help) echo "Usage: ${0##*/} [--dry-run] [--keep N] [--root DIR]"; exit 0 ;;
+    *) die "Unknown option: $1" 2 ;;
+  esac
+  shift
+done
+require_positive_integer KEEP_COUNT "$KEEP_COUNT"
+[[ -d "$BACKUP_ROOT" ]] || { log SKIP "Backup root does not exist: $BACKUP_ROOT"; exit 0; }
+while IFS= read -r -d '' dir; do
+  # Match both current *.tar.zst and legacy *.zst names. NUL delimiters keep
+  # paths containing whitespace safe. Files are sorted by modification time,
+  # newest first; the timestamp embedded in a filename is not trusted.
+  mapfile -d '' -t files < <(find "$dir" -maxdepth 1 -type f -name '*.zst' -printf '%T@ %p\0' | sort -z -nr | cut -z -d' ' -f2-)
+  (( ${#files[@]} > KEEP_COUNT )) || continue
+  for file in "${files[@]:KEEP_COUNT}"; do
+    if (( DRY_RUN )); then log WOULD "Remove $file"; else rm -f -- "$file"; log DELETE "Removed $file"; fi
+  done
+done < <(find "$BACKUP_ROOT" -mindepth 1 -maxdepth 1 -type d -print0)
+log DONE "Local prune stage completed; kept newest $KEEP_COUNT per project"

stages/99-mqtt-report.sh

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash
+# Stage 99: publish retained backup run state for Home Assistant.
+# Called once at pipeline start and again from backup.sh's EXIT trap.
+set -Eeuo pipefail
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+. "$SCRIPT_DIR/../lib/common.sh"
+. "$SCRIPT_DIR/../lib/mqtt.sh"
+
+STATUS="${1:?Usage: ${0##*/} STATUS EXIT_CODE STARTED_EPOCH FINISHED_EPOCH RUN_ID LOG_FILE}"
+EXIT_CODE="${2:-0}"
+STARTED_EPOCH="${3:-$(date +%s)}"
+FINISHED_EPOCH="${4:-$(date +%s)}"
+RUN_ID="${5:-unknown}"
+LOG_FILE="${6:-/dev/null}"
+LOCAL_ARCHIVE_STATUS="${BACKUP_LOCAL_ARCHIVE_STATUS:-unknown}"
+REMOTE_COPY_STATUS="${BACKUP_REMOTE_COPY_STATUS:-unknown}"
+RCLONE_STATUS="${BACKUP_RCLONE_STATUS:-unknown}"
+SMB_STATUS="${BACKUP_SMB_STATUS:-unknown}"
+REMOTE_COPY_FINISHED_AT="${BACKUP_REMOTE_COPY_FINISHED_AT:-}"
+
+require_command jq
+total=0; archived=0; dry_run=0; error_count=0
+# Derive counts and error messages from the shared run log. Keeping reporting
+# outside the work stages avoids coupling archive/transfer logic to MQTT.
+if [[ -r "$LOG_FILE" ]]; then
+  total="$(grep -c '\[INFO\] Project:' "$LOG_FILE" || true)"
+  archived="$(grep -c '\[ARCHIVE\] Created ' "$LOG_FILE" || true)"
+  dry_run="$(grep -c '\[DRY\] Would stop, archive' "$LOG_FILE" || true)"
+  error_count="$(grep -c '\[ERROR\]' "$LOG_FILE" || true)"
+  errors_json="$(sed -n 's/^.*\[ERROR\] //p' "$LOG_FILE" | jq -Rsc 'split("\n") | map(select(length > 0))')"
+else
+  errors_json='[]'
+fi
+if (( EXIT_CODE != 0 && error_count == 0 )); then
+  # Some commands can exit non-zero before emitting a structured [ERROR] line.
+  # Supply a fallback so a failed result never reports an empty error list.
+  errors_json="$(jq -cn --arg message "Pipeline exited with status $EXIT_CODE" '[ $message ]')"
+  error_count=1
+fi
+
+payload="$(jq -cn \
+  --arg host "$BACKUP_HOSTNAME" --arg status "$STATUS" --arg run_id "$RUN_ID" \
+  --argjson exit_code "$EXIT_CODE" --argjson started "$STARTED_EPOCH" \
+  --argjson finished "$FINISHED_EPOCH" --argjson duration "$((FINISHED_EPOCH - STARTED_EPOCH))" \
+  --argjson total "$total" --argjson archived "$archived" --argjson dry_run "$dry_run" \
+  --argjson error_count "$error_count" --argjson errors "$errors_json" \
+  --arg local_archive_status "$LOCAL_ARCHIVE_STATUS" --arg remote_copy_status "$REMOTE_COPY_STATUS" \
+  --arg rclone_status "$RCLONE_STATUS" --arg smb_status "$SMB_STATUS" \
+  --arg remote_copy_finished_at "$REMOTE_COPY_FINISHED_AT" \
+  '{host:$host,status:$status,local_archive_status:$local_archive_status,remote_copy_status:$remote_copy_status,rclone_status:$rclone_status,smb_status:$smb_status,remote_copy_finished_at:(if $remote_copy_finished_at == "" then null else $remote_copy_finished_at end),run_id:$run_id,exit_code:$exit_code,
+    started_at:($started|todateiso8601),finished_at:($finished|todateiso8601),
+    duration_seconds:$duration,containers_total:$total,containers_backed_up:$archived,
+    containers_dry_run:$dry_run,error_count:$error_count,errors:$errors}')"
+
+# Retention gives HA the latest known state after restarts or broker reconnects.
+mqtt_publish "$MQTT_STATUS_TOPIC" "$payload" 1
+log MQTT "Published $STATUS result to $MQTT_STATUS_TOPIC"

systemd/zorruno-docker-backup-mqtt.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=Zorruno Docker backup MQTT command listener
+After=network-online.target docker.service
+Wants=network-online.target
+
+[Service]
+Type=simple
+# Root is required because the listener launches a pipeline that stops and
+# starts Docker Compose projects and reads all project data.
+User=root
+WorkingDirectory=/backupdockerscripts
+ExecStart=/backupdockerscripts/bin/mqtt-trigger-listener.sh
+Restart=on-failure
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target