A first attempt at a GhostBSD Automated Ports Tree Maintenance Script

vimanuelt

Here is a first attempt at automating the GhostBSD Ports Tree Maintenance support. Removing humans from the equation is the goal. It will take many iterations to get right. Feel free to review and improve it.

#!/bin/sh
# GhostBSD Automated Ports Tree Maintenance Script
# Author: Vester Thacker
# Description: Automates syncing, building, signing, publishing GhostBSD packages with monitoring and rollback

set -eu

PORTS_DIR="/usr/ghostbsd-src/ghostbsd-ports"
POUDRIERE_JAIL="ghostbsd"
PKGLIST="/usr/local/etc/poudriere.d/pkglist.txt"
REPO_DIR="/usr/local/poudriere/data/packages/${POUDRIERE_JAIL}-default"
WWW_DIR="/usr/local/www/packages"
LOG_FILE="/var/log/ghostbsd-ports-auto.log"
MAIL_RECIPIENT="root"
ZFS_DATASET="zroot/ghostbsd-ports"
SNAP_NAME="prebuild_$(date +%Y%m%d_%H%M%S)"
MONITOR_FILE="/var/log/ghostbsd-ports-monitor.log"

exec > "$LOG_FILE" 2>&1

log_monitor() {
  echo "[$(date)] $1" >> "$MONITOR_FILE"
}

log_monitor "===== Starting automated ports maintenance ====="
echo "[$(date)] Starting automated ports maintenance..."

# Step 0: ZFS Snapshot for rollback
if zfs list "$ZFS_DATASET" >/dev/null 2>&1; then
  echo "Creating ZFS snapshot: ${ZFS_DATASET}@${SNAP_NAME}"
  zfs snapshot "${ZFS_DATASET}@${SNAP_NAME}"
  log_monitor "Snapshot created: ${ZFS_DATASET}@${SNAP_NAME}"
else
  echo "ZFS dataset $ZFS_DATASET not found. Skipping snapshot."
  log_monitor "No snapshot created: ZFS dataset not found."
fi

# Step 1: Sync Ports Tree by force-resetting to upstream
cd "$PORTS_DIR"
echo "Force-resetting ports tree to upstream master..."
git fetch origin
git reset --hard origin/master || {
  log_monitor "Git reset failed. Attempting rollback."
  if zfs list "${ZFS_DATASET}@${SNAP_NAME}" >/dev/null 2>&1; then
    echo "Restoring snapshot..."
    zfs rollback -r "${ZFS_DATASET}@${SNAP_NAME}"
    log_monitor "Rollback completed."
  fi
  echo "Git operation failed. Aborting."
  exit 1
}
log_monitor "Ports tree synced successfully."

# Step 2: Update poudriere ports database
echo "Updating poudriere ports database..."
poudriere ports -u -p "$POUDRIERE_JAIL"
log_monitor "Poudriere ports database updated."

# Step 3: Build Updated Packages
echo "Starting poudriere bulk build..."
poudriere bulk -j "$POUDRIERE_JAIL" -f "$PKGLIST" || {
  log_monitor "Build failed. Rolling back to previous snapshot."
  if zfs list "${ZFS_DATASET}@${SNAP_NAME}" >/dev/null 2>&1; then
    echo "Restoring snapshot..."
    zfs rollback -r "${ZFS_DATASET}@${SNAP_NAME}"
    log_monitor "Rollback completed."
  fi
  echo "Build failed. Aborting."
  exit 1
}
log_monitor "Build completed successfully."

# Step 4: Generate and Sign Repo
echo "Generating and signing pkg repo..."
cd "$REPO_DIR"
pkg repo .  # Add signing options if needed
log_monitor "Repository signed."

# Step 5: Sync to Public Package Server
echo "Syncing built packages to web server directory..."
rsync -a --delete "$REPO_DIR/" "$WWW_DIR/"
log_monitor "Repository synced to web directory."

# Step 6: Cleanup or Notify
echo "[$(date)] Ports tree maintenance complete."
echo "Build and sync complete at $(date)" | mail -s "GhostBSD Ports Auto Build Complete" "$MAIL_RECIPIENT"
log_monitor "Maintenance cycle complete."

exit 0

GhostBSD Automated Ports Maintenance Pipeline
This script automates the entire lifecycle of updating, building, signing, and deploying the GhostBSD ports tree and binary package repository. It is designed to run unattended and includes ZFS snapshot rollback and monitoring support.

Features
- Force-resets ports tree from upstream to avoid merge conflicts
- Takes ZFS snapshot before build for safe rollback on failure
- Uses poudriere to rebuild packages from a predefined list
- Signs the repository using pkg repo
- Publishes packages to a web-accessible directory
- Logs activity and status to a separate monitoring file
- Sends completion or failure notifications via email

Prerequisites

FreeBSD or GhostBSD with ZFS
Git installed and configured
poudriere properly configured with a jail named ghostbsd
A populated ports tree at /usr/ghostbsd-src/ghostbsd-ports
ZFS dataset defined as zroot/ghostbsd-ports (customisable)
Mail service configured (for notifications)

Configuration
Adjust these variables in the script if your environment differs:

PORTS_DIR="/usr/ghostbsd-src/ghostbsd-ports"
POUDRIERE_JAIL="ghostbsd"
PKGLIST="/usr/local/etc/poudriere.d/pkglist.txt"
REPO_DIR="/usr/local/poudriere/data/packages/${POUDRIERE_JAIL}-default"
WWW_DIR="/usr/local/www/packages"
ZFS_DATASET="zroot/ghostbsd-ports"
MAIL_RECIPIENT="root"

Installation

Save the script as ghostbsd-ports-auto.sh.
Make it executable:chmod +x ghostbsd-ports-auto.sh
(Optional) Add to cron for scheduled execution: 0 3 * * * /path/to/ghostbsd-ports-auto.sh

Logs

Primary Log: /var/log/ghostbsd-ports-auto.log
Monitoring Log: /var/log/ghostbsd-ports-monitor.log

Rollback
If a build fails or the Git reset encounters issues, the script will automatically roll back to the latest ZFS snapshot taken before execution.

vimanuelt

Here is an updated version with port synchronization added.

#!/bin/sh
# GhostBSD Automated Ports Tree Maintenance Script
# Author: Vester Thacker
# Description: Automates syncing, tagging, building, signing, publishing GhostBSD packages
#              with monitoring, rollback and serialized standardisation via timestamped git tags

set -eu

# Configuration
PORTS_DIR="/usr/ghostbsd-src/ghostbsd-ports"
POUDRIERE_JAIL="ghostbsd"
PKGLIST="/usr/local/etc/poudriere.d/pkglist.txt"
REPO_DIR="/usr/local/poudriere/data/packages/${POUDRIERE_JAIL}-default"
WWW_DIR="/usr/local/www/packages"
LOG_FILE="/var/log/ghostbsd-ports-auto.log"
MONITOR_FILE="/var/log/ghostbsd-ports-monitor.log"
MAIL_RECIPIENT="root"
ZFS_DATASET="zroot/ghostbsd-ports"

# derive timestamp for tag and rollback snapshot
TIMESTAMP=$(date -u +"%Y%m%dT%H%M%SZ")
SNAP_NAME="prebuild_${TIMESTAMP}"
TAG_NAME="ports-${TIMESTAMP}"

# redirect stdout and stderr to log
exec > "${LOG_FILE}" 2>&1

log_monitor() {
  printf '[%s] %s\n' "$(date -u '+%Y-%m-%dT%H:%M:%SZ')" "$1" >> "${MONITOR_FILE}"
}

log_monitor "===== Starting automated ports maintenance ====="

echo "[\$(date)] Creating ZFS snapshot for rollback"
# Step 0: ZFS snapshot for rollback
if zfs list "${ZFS_DATASET}" >/dev/null 2>&1; then
  zfs snapshot "${ZFS_DATASET}@${SNAP_NAME}"
  log_monitor "Snapshot created: ${ZFS_DATASET}@${SNAP_NAME}"
else
  log_monitor "No snapshot created: ZFS dataset not found"
fi

# Step 1: Sync ports tree and tag snapshot
cd "${PORTS_DIR}"
echo "[\$(date)] Syncing ports tree and creating git tag ${TAG_NAME}"
if git fetch origin && git reset --hard origin/master; then
  git tag -a "${TAG_NAME}" -m "Ports snapshot ${TAG_NAME}" \
    && git push origin "refs/tags/${TAG_NAME}" \
    && log_monitor "Ports tree tagged: ${TAG_NAME}"
else
  log_monitor "Git sync failed. Rolling back to ${SNAP_NAME}"
  zfs rollback -r "${ZFS_DATASET}@${SNAP_NAME}" \
    && log_monitor "Rollback completed"
  echo "Git operation failed. Aborting." >&2
  exit 1
fi

# Step 2: Update poudriere ports database
echo "[\$(date)] Updating poudriere ports database"
poudriere ports -u -p "${POUDRIERE_JAIL}"
log_monitor "Poudriere ports database updated"

# Step 3: Build updated packages
echo "[\$(date)] Starting poudriere bulk build"
if ! poudriere bulk -j "${POUDRIERE_JAIL}" -f "${PKGLIST}"; then
  log_monitor "Build failed. Rolling back to snapshot ${SNAP_NAME}"
  zfs rollback -r "${ZFS_DATASET}@${SNAP_NAME}" \
    && log_monitor "Rollback completed"
  echo "Build failed. Aborting." >&2
  exit 1
fi
log_monitor "Build completed successfully"

# Step 4: Generate and sign repository
echo "[\$(date)] Generating and signing pkg repo"
cd "${REPO_DIR}"
pkg repo . # add -s keyid=<KEYID> if signing is required
log_monitor "Repository generated and signed"

# Step 5: Sync to public package server
echo "[\$(date)] Syncing packages to web directory"
rsync -a --delete "${REPO_DIR}/" "${WWW_DIR}/"
log_monitor "Repository synced to ${WWW_DIR}"

# Final: Notify and exit
echo "[\$(date)] Ports maintenance complete: snapshot ${SNAP_NAME}, tag ${TAG_NAME}"
echo "Ports auto build and sync completed. Snapshot ${SNAP_NAME}, tag ${TAG_NAME}" \
  | mail -s "GhostBSD Ports Auto Build Complete (${TAG_NAME})" "${MAIL_RECIPIENT}"
log_monitor "Maintenance cycle complete: ${TAG_NAME}"

exit 0

GhostBSD Ports Auto Maintenance

Automates the synchronization, tagging, building, signing and deployment of GhostBSD ports and packages with monitoring and safe rollback.

Features

Takes a ZFS snapshot before each build for safe rollback
Syncs the ports tree and creates a timestamped Git tag for a serialised snapshot
Updates the poudriere ports database and executes bulk builds
Generates and signs the package repository
Publishes the repository to a web‑accessible directory via rsync
Logs activities to separate build and monitoring files
Sends email notifications on completion or failure

Prerequisites

GhostBSD or FreeBSD with ZFS support
A configured Git clone of the ports tree at /usr/ghostbsd-src/ghostbsd-ports
poudriere installed and a jail named ghostbsd configured
A populated package list file (e.g. /usr/local/etc/poudriere.d/pkglist.txt)
Mail service configured to send notifications

Configuration

Edit the variables at the top of the script to suit your environment:

PORTS_DIR="/usr/ghostbsd-src/ghostbsd-ports"
POUDRIERE_JAIL="ghostbsd"
PKGLIST="/usr/local/etc/poudriere.d/pkglist.txt"
REPO_DIR="/usr/local/poudriere/data/packages/${POUDRIERE_JAIL}-default"
WWW_DIR="/usr/local/www/packages"
ZFS_DATASET="zroot/ghostbsd-ports"
MAIL_RECIPIENT="root"
LOG_FILE="/var/log/ghostbsd-ports-auto.log"
MONITOR_FILE="/var/log/ghostbsd-ports-monitor.log"

Installation and Usage

Save the script as ghostbsd-ports-auto.sh and make it executable:

   chmod +x ghostbsd-ports-auto.sh

(Optional) Schedule via cron for unattended runs. For example, to run nightly at 03:00:

   0 3 * * * /path/to/ghostbsd-ports-auto.sh

Verify that logs appear in /var/log/ghostbsd-ports-auto.log and monitoring entries in /var/log/ghostbsd-ports-monitor.log.

Logging and Monitoring

Primary log: Detailed script output in LOG_FILE
Monitoring log: High‑level events with timestamps in MONITOR_FILE
Email notifications: Sent to MAIL_RECIPIENT on success or failure

Rollback

If the script encounters an error during sync or build, it will automatically roll back the ZFS dataset to the snapshot taken at the start of the run.

vimanuelt

If you find the script useful, feel free to add it to a GitHub repository.

janeesmith

This script automates the maintenance of the GhostBSD ports tree, simplifying https://arzhost.com/) updates and management. It helps keep the system’s software ports organized and up-to-date efficiently.