Programmatic ZFS ARC Tuning

vimanuelt

Standard Operating Procedure (SOP)
Title: Programmatic ZFS ARC Tuning via Continuous Loop and devd Memory-Pressure Hook (Proof of Concept)
Version: 1.0
Date: 9 May 2025
Author: Vic Thacker

1 Purpose

This SOP describes the steps required to implement combined, programmatic tuning of the ZFS Adaptive Replacement Cache (ARC) on GhostBSD and FreeBSD. By deploying both a continuous-loop daemon and a devd memory-pressure hook, administrators ensure aggressive caching when memory is ample while preserving a defined minimum of free RAM under pressure.

2 Scope

Applies to all GhostBSD and FreeBSD hosts running ZFS where dynamic, responsive ARC management is desired; particularly systems with variable workloads or stringent memory constraints.

3 Prerequisites

FreeBSD 11 or later with ZFS enabled
Shell access with root privileges
Basic knowledge of FreeBSD rc.d, sysctl and devd
Text editor (e.g., vi, ee or nano)

4 Definitions

ARC: Adaptive Replacement Cache, ZFS’s in‑memory cache layer
arc.max: sysctl knob setting an absolute upper bound for ARC size
arc.sys_free: sysctl knob defining a byte‑precise free‑memory threshold under which ARC will evict
devd: FreeBSD’s userspace daemon for kernel event notifications

5 Responsibilities

System Administrator: Implements SOP, verifies configuration, monitors behavior
Operations Team: Reviews SOP annually and after major FreeBSD/ZFS upgrades

6 Procedure

6.1 Baseline Configuration

6.1.1 Open /boot/loader.conf (or /etc/sysctl.conf) in your editor
6.1.2 Add the following lines to establish default safety values:

vfs.zfs.arc.max="8G"            # Hard cap at 8 GiB  
vfs.zfs.arc.sys_free="536870912"  # Ensure 512 MiB free

6.1.3 Save and exit
6.1.4 Reboot or apply immediately via:

sysctl vfs.zfs.arc.max=8G  
sysctl vfs.zfs.arc.sys_free=536870912

6.2 Deploy the Tuner Script

6.2.1 Create /usr/local/sbin/adjust-arc.sh with the following content:

#!/bin/sh  
# Adjust ARC max and sys_free dynamically  
MIN_FREE_BYTES=$((512 * 1024 * 1024))  
ARC_FRAC=0.80  
  
total=$(sysctl -n hw.physmem)  
free_pages=$(sysctl -n vm.stats.vm.v_free_count)  
pagesize=$(sysctl -n hw.pagesize)  
free_bytes=$((free_pages * pagesize))  
  
target=$(( (total - MIN_FREE_BYTES) * ARC_FRAC ))  
arc_max=$(( (target / (1024*1024)) * (1024*1024) ))  
  
current_max=$(sysctl -n vfs.zfs.arc.max)  
if [ "$arc_max" -ne "$current_max" ]; then  
  sysctl vfs.zfs.arc.max="$arc_max"  
  echo "[adjust-arc] Set arc.max to $arc_max bytes"  
fi  
  
if [ "$free_bytes" -lt "$MIN_FREE_BYTES" ]; then  
  current_free=$(sysctl -n vfs.zfs.arc.sys_free)  
  if [ "$current_free" -ne "$MIN_FREE_BYTES" ]; then  
    sysctl vfs.zfs.arc.sys_free="$MIN_FREE_BYTES"  
    echo "[adjust-arc] Raised arc.sys_free to $MIN_FREE_BYTES bytes"  
  fi  
fi  
  
exit 0

6.2.2 Make the script executable:

chmod +x /usr/local/sbin/adjust-arc.sh

6.3 Configure Continuous-Loop Service

6.3.1 Create /usr/local/etc/rc.d/adjust_arc:

#!/bin/sh  
# PROVIDE: adjust_arc  
# REQUIRE: DAEMON  
# KEYWORD: shutdown  
  
. /etc/rc.subr  
  
name="adjust_arc"  
rcvar=adjust_arc_enable  
  
start_cmd="${name}_start"  
stop_cmd=":"  
  
adjust_arc_start() {  
  echo "Starting ARC tuner..."  
  /usr/local/sbin/adjust-arc.sh  
  while :; do  
    sleep 30  
    /usr/local/sbin/adjust-arc.sh  
  done &  
  echo $! > /var/run/${name}.pid  
}  
  
load_rc_config $name  
: ${adjust_arc_enable:="NO"}  
run_rc_command "$1"

6.3.2 Make the service script executable:

chmod +x /usr/local/etc/rc.d/adjust_arc

6.3.3 Enable in /etc/rc.conf:

adjust_arc_enable="YES"

6.3.4 Start the service:

service adjust_arc start

6.4 Configure `devd` Memory-Pressure Hook

6.4.1 Edit /etc/devd.conf and append:

notify 100 {  
  match "system" "memory";  
  action "/usr/local/sbin/adjust-arc.sh";  
};

6.4.2 Restart devd:

service devd restart

7 Verification

7.1 Confirm arc.max and arc.sys_free are in effect:

sysctl vfs.zfs.arc.max vfs.zfs.arc.sys_free

7.2 Simulate memory pressure (e.g. via a memory‑hog process) and check tuner output in /var/log/messages or console
7.3 Monitor ARC statistics:

sysctl kstat.zfs.misc.arcstats

8 Maintenance

8.1 Review thresholds (512 MiB free; 80% ARC fraction) quarterly
8.2 Update script logic if sysctl names or event semantics change in new FreeBSD releases
8.3 Test on staging before production rollout