Skip to content [Tab]
~/posts/devops/automated-website-backups-using-a-pull-architecture-and-semaphore-ui.md
$ stat -x automated-website-backups-using-a-pull-architecture-and-semaphore-ui.md POSIX filesystem
Access Mode: -rw-r--r-- (0644)
Author: uid:1000(@f7234bv9s8)
File Size: 31.3 KB (3312 words)
Modified:

Automated Website Backups Using a Pull Architecture and Semaphore UI

attachment://featured-image.webp
Semaphore UI

1. What Tasks Should Be Automated and Why the “Pull” Approach Works Best

Manual backups fail when they are needed mostโ€”usually right before a rushed plugin update, framework migration, or server configuration change. A reliable backup pipeline for self-hosted web applications (such as Dockerized WordPress and Django sites) must automate three distinct operations:

  1. Consistent Database Dumps: Copying raw database files (/var/lib/mysql or /var/lib/postgresql) while a database engine is running produces corrupted, unrecoverable backups. Automated backups must invoke native dump utilities (mariadb-dump/mysqldump or pg_dump) inside the running container using transaction-safe flags (--single-transaction).
  2. Incremental Media and Asset Synchronization: Application uploads (wp-content/ in WordPress or media/ in Django) grow into gigabytes over time. Rather than compressing and downloading the entire directory every night, delta synchronization (rsync) transfers only newly created or modified files.
  3. On-Demand Pre-Change Checkpoints: Beyond scheduled nightly runs, operators need a one-click way to trigger a fast, database-only backup for a single website immediately before applying updates.

Why a Home Lab “Pull” Architecture Beats a VPS “Push” Architecture

In a traditional push setup, each public VPS holds credentials to upload backups to a central storage server. If a single VPS is compromised through an application vulnerability, an attacker can read those credentials and delete or encrypt your remote backups.

A pull architecture reverses the trust boundary:

  • Zero Inbound Ports on the Backup Server: Your Home Lab server sits behind NAT with no open firewall ports. Public VPS nodes cannot initiate connections to it.
  • Centralized Execution: The Home Lab initiates outbound SSH connections to each VPS, streams database dumps directly over stdout (writing zero temporary files to the VPS disk), and pulls changed files via rsync.
  • Containerized Credential Isolation: Database passwords never leave the VPS. The pull script reads Docker secrets or container environment variables directly inside the remote database container at runtime.

2. Directory and Folder Structure

Organizing orchestration files separately from backup staging storage keeps permissions clean and prepares the storage directory for downstream snapshot tools.

/opt/backups/                               # Home Lab script repository (mounted read-only into Semaphore)
โ”œโ”€โ”€ pull_wp_backups.sh                      # WordPress database & file pull script
โ””โ”€โ”€ pull_django_backups.sh                  # Django database & file pull script

/opt/semaphore/                             # Semaphore UI Docker stack directory
โ”œโ”€โ”€ docker-compose.yml                      # Container definition
โ”œโ”€โ”€ semaphore-data/                         # SQLite database (owned by UID 1001)
โ””โ”€โ”€ semaphore-config/                       # Semaphore config & isolated SSH config (owned by UID 1001)
    โ””โ”€โ”€ ssh_config                          # Custom SSH host aliases, ports, and key mappings

/mnt/homelab_storage/backups/               # Dedicated backup storage drive/pool
โ”œโ”€โ”€ site_one/                               # Isolated per-website directory (SITE_NAME)
โ”‚   โ”œโ”€โ”€ database/
โ”‚   โ”‚   โ””โ”€โ”€ wordpress_db.sql                # Latest atomic SQL dump (overwritten cleanly each run)
โ”‚   โ””โ”€โ”€ files_mirror/                       # Live delta mirror of wp-content/ or media/
โ””โ”€โ”€ site_two/
    โ”œโ”€โ”€ database/
    โ”‚   โ””โ”€โ”€ django_prod.sql
    โ””โ”€โ”€ files_mirror/

Design rationale:

  • Each website writes to its own SITE_NAME folder so multiple websites hosted on the same VPS never collide.
  • Database dumps overwrite a single canonical .sql file per site after passing an atomic integrity check, while files_mirror/ maintains an exact local mirror of remote uploads.

3. Backup Scripts and How They Work

Splitting WordPress and Django into dedicated scripts avoids bloated conditional branching while sharing identical safety mechanisms. Both scripts accept environment variables injected by Semaphore UI (with fallback support for CLI positional arguments).

WordPress Pull Script (/opt/backups/pull_wp_backups.sh)

#!/usr/bin/env bash

# WHAT: Enable strict error handling:
#   -e: Exit immediately if any command returns a non-zero exit code.
#   -u: Treat unset variables as an error and exit immediately.
#   -o pipefail: Fail a pipeline if any command in it fails (not just the last one).
# WHY: Prevents silent failures where a broken SSH tunnel or failed database dump
#      creates an empty/corrupt file and reports "Success" to Semaphore UI.
set -euo pipefail

# WHAT: Read configuration from Semaphore Environment Variables, with fallback to
#       positional CLI arguments ($1, $2, etc.) if you run the script manually.
# WHY: Semaphore UI passes Variable Group entries to Bash scripts as environment
#      variables, while positional fallbacks keep manual terminal testing easy.
VPS_HOST="${VPS_HOST:-${1:-}}"
SITE_NAME="${SITE_NAME:-${2:-}}"
DB_CONTAINER="${DB_CONTAINER:-${3:-}}"
DB_NAME="${DB_NAME:-${4:-}}"
INCLUDE_FILES="${INCLUDE_FILES:-${5:-false}}"
REMOTE_PATH="${REMOTE_PATH:-${6:-}}"

# WHAT: Optional override for custom Docker secret filenames inside /run/secrets/.
# WHY: Allows supporting non-standard secret names without editing the script.
DB_SECRET_NAME="${DB_SECRET_NAME:-}"

# WHAT: Loop through mandatory variables and abort if any are empty.
# WHY: Catches misconfigured Semaphore Variable Groups immediately with a clear
#      error message instead of failing midway through an SSH command.
for var in VPS_HOST SITE_NAME DB_CONTAINER DB_NAME; do
    if [ -z "${!var:-}" ]; then
        echo "ERROR: Required variable '$var' is empty. Set it under 'Environment variables' in Semaphore."
        exit 1
    fi
done

# WHAT: Define local staging paths on the Home Lab storage volume.
# WHY: Isolates each website into its own subfolder so multiple sites on the same
#      VPS never overwrite one another, and stages uncompressed files for Backrest.
SITE_BACKUP_DIR="/mnt/homelab_storage/backups/${SITE_NAME}"
DB_DIR="${SITE_BACKUP_DIR}/database"
LATEST_FILES="${SITE_BACKUP_DIR}/files_mirror"

# WHAT: Define both the final SQL dump path and a hidden temporary file path.
# WHY: Writing directly to the final file would destroy yesterday's good backup
#      if the network drops mid-transfer.
DB_DUMP_FINAL="${DB_DIR}/${DB_NAME}.sql"
DB_DUMP_TMP="${DB_DIR}/.${DB_NAME}.sql.tmp"

# WHAT: Create the database folder if missing and register an EXIT cleanup trap.
# WHY: The trap guarantees that if the script fails or is interrupted at any point,
#      the incomplete .tmp file is automatically deleted from disk.
mkdir -p "$DB_DIR"
trap 'rm -f "$DB_DUMP_TMP"' EXIT

echo "[1/2] Streaming WordPress database (${DB_NAME}) from ${VPS_HOST} (container:${DB_CONTAINER})..."

# WHAT: Connect to the VPS over SSH and execute an inline shell script inside the
#       running database container via stdin (-s), redirecting stdout to DB_DUMP_TMP.
# WHY: 1) Zero temporary dump files are written to the VPS disk.
#      2) Passing the logic via stdin (<< 'EOF') avoids nested quote-escaping issues
#         and keeps stdout strictly reserved for the raw SQL stream.
#      3) StrictHostKeyChecking=accept-new trusts new VPS host keys automatically on
#         first connect while still blocking changed keys (MITM protection).
ssh -o StrictHostKeyChecking=accept-new "$VPS_HOST" \
  "docker exec -i ${DB_CONTAINER} sh -s -- '${DB_NAME}' '${DB_SECRET_NAME}'" << 'EOF' > "$DB_DUMP_TMP"
set -eu
DB_NAME="$1"
SECRET_NAME="$2"

# WHAT: Auto-detect the MariaDB/MySQL root password across Docker secrets and env vars.
# WHY: Different Docker Compose stacks store credentials in different places
#      (/run/secrets/*, *_FILE pointers, or plain environment variables). Checking
#      all standard locations lets one script work across every WordPress VPS.
if [ -n "$SECRET_NAME" ] && [ -f "/run/secrets/${SECRET_NAME}" ]; then
    DB_PASS=$(cat "/run/secrets/${SECRET_NAME}")
elif [ -n "${MARIADB_ROOT_PASSWORD_FILE:-}" ] && [ -f "$MARIADB_ROOT_PASSWORD_FILE" ]; then
    DB_PASS=$(cat "$MARIADB_ROOT_PASSWORD_FILE")
elif [ -n "${MYSQL_ROOT_PASSWORD_FILE:-}" ] && [ -f "$MYSQL_ROOT_PASSWORD_FILE" ]; then
    DB_PASS=$(cat "$MYSQL_ROOT_PASSWORD_FILE")
elif [ -f "/run/secrets/db_root_password" ]; then
    DB_PASS=$(cat "/run/secrets/db_root_password")
elif [ -f "/run/secrets/mariadb_root_password" ]; then
    DB_PASS=$(cat "/run/secrets/mariadb_root_password")
elif [ -f "/run

Django Pull Script (/opt/backups/pull_django_backups.sh)

#!/usr/bin/env bash

# WHAT: Enable strict error handling (-e exit on error, -u error on unset vars,
#       -o pipefail catch failures inside pipes).
# WHY: Ensures Semaphore immediately marks the task failed if SSH, Docker, or
#      pg_dump encounters any error.
set -euo pipefail

# WHAT: Load target site variables from Semaphore Environment Variables, falling
#       back to CLI positional parameters ($1..$6) for manual runs.
# WHY: Decouples site-specific configuration from script logic so one script can
#      back up all Django websites.
VPS_HOST="${VPS_HOST:-${1:-}}"
SITE_NAME="${SITE_NAME:-${2:-}}"
DB_CONTAINER="${DB_CONTAINER:-${3:-}}"
DB_NAME="${DB_NAME:-${4:-}}"
INCLUDE_FILES="${INCLUDE_FILES:-${5:-false}}"
REMOTE_PATH="${REMOTE_PATH:-${6:-}}"

# WHAT: Optional override if your PostgreSQL password secret has a custom name.
# WHY: Supports non-standard /run/secrets/ filenames without modifying the script.
DB_SECRET_NAME="${DB_SECRET_NAME:-}"

# WHAT: Validate that all mandatory parameters are populated before connecting.
# WHY: Prevents undefined behavior or accidental writes to root directories.
for var in VPS_HOST SITE_NAME DB_CONTAINER DB_NAME; do
    if [ -z "${!var:-}" ]; then
        echo "ERROR: Required variable '$var' is empty. Set it under 'Environment variables' in Semaphore."
        exit 1
    fi
done

# WHAT: Define local staging directories and atomic temporary dump paths.
# WHY: Stages uncompressed SQL and delta-synced media files on the Home Lab disk
#      where Backrest can deduplicate them at the block level.
SITE_BACKUP_DIR="/mnt/homelab_storage/backups/${SITE_NAME}"
DB_DIR="${SITE_BACKUP_DIR}/database"
LATEST_FILES="${SITE_BACKUP_DIR}/files_mirror"

DB_DUMP_FINAL="${DB_DIR}/${DB_NAME}.sql"
DB_DUMP_TMP="${DB_DIR}/.${DB_NAME}.sql.tmp"

# WHAT: Ensure the database staging folder exists and clean up .tmp on any exit.
# WHY: Guarantees failed or aborted runs never leave behind partial SQL files or
#      corrupt the previous night's valid backup.
mkdir -p "$DB_DIR"
trap 'rm -f "$DB_DUMP_TMP"' EXIT

echo "[1/2] Streaming Django PostgreSQL database (${DB_NAME}) from ${VPS_HOST} (container: ${DB_CONTAINER})..."

# WHAT: SSH to the VPS and run the password-detection + pg_dump snippet inside the
#       PostgreSQL container via stdin (-s), streaming SQL output to DB_DUMP_TMP.
# WHY: Keeps database passwords inside the container, avoids using VPS disk space
#      for temporary dumps, and prevents shell-quoting errors over SSH.
ssh -o StrictHostKeyChecking=accept-new "$VPS_HOST" \
  "docker exec -i ${DB_CONTAINER} sh -s -- '${DB_NAME}' '${DB_SECRET_NAME}'" << 'EOF' > "$DB_DUMP_TMP"
set -eu
DB_NAME="$1"
SECRET_NAME="$2"

# WHAT: Use the container's POSTGRES_USER if defined, otherwise default to "postgres".
# WHY: Matches official PostgreSQL Docker image conventions automatically.
DB_USER="${POSTGRES_USER:-postgres}"

# WHAT: Locate the PostgreSQL password across Docker secrets and environment variables.
# WHY: Handles stacks using Docker secrets (POSTGRES_PASSWORD_FILE or /run/secrets/*)
#      as well as stacks passing POSTGRES_PASSWORD directly in the environment.
if [ -n "$SECRET_NAME" ] && [ -f "/run/secrets/${SECRET_NAME}" ]; then
    DB_PASS=$(cat "/run/secrets/${SECRET_NAME}")
elif [ -n "${POSTGRES_PASSWORD_FILE:-}" ] && [ -f "$POSTGRES_PASSWORD_FILE" ]; then
    DB_PASS=$(cat "$POSTGRES_PASSWORD_FILE")
elif [ -f "/run/secrets/postgres_password" ]; then
    DB_PASS=$(cat "/run/secrets/postgres_password")
elif [ -f "/run/secrets/db_password" ]; then
    DB_PASS=$(cat "/run/secrets/db_password")
elif [ -n "${POSTGRES_PASSWORD:-}" ]; then
    DB_PASS="$POSTGRES_PASSWORD"
else
    # Write diagnostic errors to stderr (>&2) so stdout remains pure SQL
    echo "ERROR: Could not locate PostgreSQL password in /run/secrets or environment variables." >&2
    exit 1
fi

# WHAT: Export PGPASSWORD for non-interactive authentication and run pg_dump.
# WHY:
#   PGPASSWORD: Required because pg_dump has no -p password flag like mysqldump.
#   --clean --if-exists: Includes DROP TABLE IF EXISTS statements before CREATE
#     statements so restoring into an existing database replaces objects cleanly.
#   Plain SQL format (default): Enables Backrest/Restic content-defined chunking
#     to deduplicate unchanged rows across daily snapshots.
export PGPASSWORD="$DB_PASS"
exec pg_dump -U "$DB_USER" --clean --if-exists "$DB_NAME"
EOF

# WHAT: Confirm the resulting dump file is larger than 0 bytes.
# WHY: Aborts before replacing the previous dump if pg_dump produced empty output.
if [ ! -s "$DB_DUMP_TMP" ]; then
    echo "ERROR: Database dump returned 0 bytes. Keeping previous backup intact."
    exit 1
fi

# WHAT: Atomically replace the previous SQL dump with the newly verified dump.
# WHY: Ensures zero downtime or partial-file visibility for Backrest snapshots.
mv "$DB_DUMP_TMP" "$DB_DUMP_FINAL"
echo "Database dump updated: ${DB_DUMP_FINAL}"

# WHAT: Conditionally sync Django user-uploaded media files when INCLUDE_FILES="true".
# WHY: Allows fast database-only backups before schema migrations while syncing
#      user uploads on schedule.
if [ "$INCLUDE_FILES" = "true" ]; then
    if [ -z "$REMOTE_PATH" ]; then
        echo "ERROR: INCLUDE_FILES is true, but REMOTE_PATH is empty."
        exit 1
    fi

    mkdir -p "$LATEST_FILES"
    echo "[2/2] Syncing Django media files from ${VPS_HOST}:${REMOTE_PATH}..."

    # WHAT: Incrementally sync remote media/project files to the local files_mirror.
    # WHY:
    #   -az --delete: Transfers only modified/new files and mirrors deletions.
    #   --chmod=...: Normalizes copied file permissions to 0755/0644 on the Home Lab
    #     so Backrest never fails with permission-denied warnings on restricted files.
    #   --exclude ...: Skips Python bytecode caches and log files that change
    #     constantly and do not need to be backed up.
    rsync -az --delete \
      --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r \
      --exclude '*.log' \
      --exclude '__pycache__/' \
      -e "ssh -o StrictHostKeyChecking=accept-new" \
      "${VPS_HOST}:${REMOTE_PATH}/" "${LATEST_FILES}/"

    echo "File mirror updated: ${LATEST_FILES}"
else
    echo "[2/2] Skipping file sync (INCLUDE_FILES=false)."
fi

Key Script Mechanisms Explained

  • Strict Error Handling (set -euo pipefail): Forces the script to abort immediately if any variable is unset, any command fails, or an SSH pipe breaks mid-streamโ€”reporting a clear failure status back to Semaphore UI.
  • Stdin Script Injection (docker exec -i ... sh -s << 'EOF'): Instead of passing complex password-lookup logic through nested SSH quotes, the script feeds the detection block over stdin (-s) while keeping stdout exclusively reserved for the raw SQL dump stream.
  • Automatic Secret and Environment Variable Detection: Checks custom secret names first, then Docker Compose *_FILE variables, standard /run/secrets/ paths, and finally plaintext environment variables (MARIADB_ROOT_PASSWORD, MYSQL_ROOT_PASSWORD, POSTGRES_PASSWORD).
  • Atomic Temporary Dumps (.sql.tmp + trap): Output is written to a hidden .tmp file first. If the network drops mid-transfer, the EXIT trap deletes the partial .tmp file and leaves the previous night’s valid .sql file intact. Only after confirming [ ! -s "$DB_DUMP_TMP" ] (non-zero file size) does mv atomically replace the target dump.
  • Permission Normalization (--chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r): Production VPS stacks often restrict secrets or configuration files to 0600 or 0400. Normalizing permissions to 0755 for directories and 0644 for files on the Home Lab staging disk prevents downstream snapshot engines from throwing permission denied warnings while leaving live VPS permissions untouched.

4. Installing and Configuring Semaphore UI

Semaphore UI runs inside Docker as internal user 1001 (semaphore). Proper host directory ownership and an isolated SSH configuration file prevent SQLite write locks and OpenSSH permission errors.

Step 1: Prepare Directories, SSH Config, and Permissions

  1. Create the required directories on your Home Lab:Bashsudo mkdir -p /opt/backups /opt/semaphore/semaphore-data /opt/semaphore/semaphore-config /mnt/homelab_storage/backups
  2. Save both scripts into /opt/backups/ and make them executable:Bashsudo chmod +x /opt/backups/pull_wp_backups.sh /opt/backups/pull_django_backups.sh
  3. Create a dedicated SSH config file for Semaphore at /opt/semaphore/semaphore-config/ssh_config (do not mount your host user’s ~/.ssh/config directly, as OpenSSH rejects config files whose owner does not match the running UID):Code snippetHost vps_alpha HostName 203.0.113.10 User backup_operator Port 22 IdentityFile ~/.ssh/homelab_backup_key StrictHostKeyChecking accept-new Host vps_beta HostName 203.0.113.20 User backup_operator Port 2222 IdentityFile ~/.ssh/homelab_backup_key StrictHostKeyChecking accept-new
  4. Apply strict ownership (1001:0) and permissions so Semaphore can write to its SQLite database, write to /mnt/homelab_storage/backups, and read its SSH key and config:Bashsudo chown -R 1001:0 /opt/semaphore/semaphore-data /opt/semaphore/semaphore-config /mnt/homelab_storage/backups sudo chmod -R 775 /opt/semaphore/semaphore-data /opt/semaphore/semaphore-config /mnt/homelab_storage/backups sudo chmod 600 /opt/semaphore/semaphore-config/ssh_config

(Ensure your backup_operator user on each VPS belongs to the docker group via sudo usermod -aG docker backup_operator so it can execute docker exec without sudo.)

Step 2: Deploy Semaphore UI with Docker Compose

Create /opt/semaphore/docker-compose.yml:

YAML

services:
  semaphore:
    image: semaphoreui/semaphore:latest
    container_name: semaphore_ui
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      SEMAPHORE_DB_DIALECT: sqlite
      SEMAPHORE_DB_HOST: /var/lib/semaphore/database.sqlite
      SEMAPHORE_DB_PATH: /var/lib/semaphore
      SEMAPHORE_TMP_PATH: /tmp/semaphore
      SEMAPHORE_ADMIN: admin
      SEMAPHORE_ADMIN_PASSWORD: ChangeThisStrongPassword!
      SEMAPHORE_ADMIN_NAME: Admin
      SEMAPHORE_ADMIN_EMAIL: [email protected]
      SEMAPHORE_ACCESS_KEY_ENCRYPTION: ReplaceWith32CharRandomBase64Key==
    volumes:
      - ./semaphore-data:/var/lib/semaphore
      - ./semaphore-config:/etc/semaphore
      - ./semaphore-config/ssh_config:/home/semaphore/.ssh/config:ro
      - ~/.ssh/homelab_backup_key:/home/semaphore/.ssh/homelab_backup_key:ro
      - /opt/backups:/opt/backups:ro
      - /mnt/homelab_storage/backups:/mnt/homelab_storage/backups:rslave

Start the container from /opt/semaphore:

Bash

docker compose up -d

Note on :rslave: Mounting /mnt/homelab_storage/backups with :rslave ensures that if an external drive or pooled storage mount initializes late during a Home Lab reboot, mount updates propagate into the container cleanly.

Step 3: Configure Semaphore UI Resources

Open http://HOMELAB_IP:3000, log in with your admin credentials, create a new Project (e.g., Website Backups), and configure the four required building blocks in the left sidebar:

  1. Key Store (Blank Key for Local Repository):
    • Open Key Store and click New Key.
    • Set Name to Local Folder and Type to None, then click Create.
  2. Repositories (Local Script Directory):
    • Open Repositories and click New Repository.
    • Set Name to Local Backup Scripts, URL or path to /opt/backups, Branch to main, and Access Key to Local Folder. Click Save.
  3. Variable Groups (One Group per Website):
    • Open Variable Groups and click New Variable Group.
    • Name the group after your website (e.g., WP - Site One).
    • Important: Leave Extra variables (the top section) completely emptyโ€”Semaphore only passes Extra variables to Ansible playbooks. Add all parameters in the Environment variables section at the bottom:
Variable NameExample ValueRequirement
VPS_HOSTvps_alphaRequired. SSH alias from ssh_config (or user@ip).
SITE_NAMEsite_oneRequired. Folder name created under /mnt/homelab_storage/backups/.
DB_CONTAINERsite_one_dbRequired. Exact MariaDB, MySQL, or PostgreSQL container name on the VPS.
DB_NAMEwordpress_dbRequired. Database name inside the container.
REMOTE_PATH/opt/sites/site_one/wp-contentRequired when INCLUDE_FILES=true. Remote path to sync into files_mirror.
DB_SECRET_NAMEcustom_db_secretOptional. Only needed if the secret file in /run/secrets/ uses a custom name.
  1. Task Templates (One Cloned Template per Website):
    • Open Task Templates and click New Template.
    • Select Bash as the template type.
    • Set Name to Backup - Site One, Playbook / Script Filename to pull_wp_backups.sh (or pull_django_backups.sh), Repository to Local Backup Scripts, and Variable Group to WP - Site One.
    • Expand Survey Variables and add a variable named INCLUDE_FILES (Title: Sync Files?, Default value: false, Type: String or Boolean), ensuring its target is set to Environment variable. This presents a prompt before every manual run so you can choose between a 5-second database-only dump (false) and a full database + file sync (true).
    • Save the template, then use the Clone button to duplicate it for each additional website, changing only the template name, script filename (if switching between WordPress and Django), and bound Variable Group.

5. Executing the Initial Backup and Setting Schedules

Before running tasks in the web UI, verify SSH connectivity directly from inside Semaphore’s container context:

Bash

docker exec -it semaphore_ui ssh vps_alpha "echo Connected && docker ps --format '{{.Names}}'"

Confirm it prints Connected along with your running container names without password prompts or host-key errors.

Running the Initial Full Backup

  1. In Semaphore UI, open Task Templates and click Run on Backup - Site One.
  2. In the Survey prompt, enter true for INCLUDE_FILES so the initial run populates both the SQL dump and the complete files_mirror/ baseline.
  3. Watch the live console output confirm [1/2] Streaming ... database followed by Database dump updated and File mirror updated.
  4. Verify the files on your Home Lab host:Bashls -lh /mnt/homelab_storage/backups/site_one/database/ ls -la /mnt/homelab_storage/backups/site_one/files_mirror/

Configuring Staggered Nightly Schedules

Open Schedules in the left sidebar and create one schedule per website Task Template, spacing start times 5 to 10 minutes apart (for example, 0 2 * * * for Site 1, 5 2 * * * for Site 2, 10 2 * * * for Site 3). Staggering execution prevents concurrent rsync and SQL streams from saturating your Home Lab network bandwidth or disk I/O, and ensures all pulls complete cleanly before your snapshot engine runs.

@f7234bv9s8 maintainer

Software engineer and writer at notes:own:vision:com.