UV Cann't load Pandas/numpy?

davidjayjackson

Hey,
When I try to load a package with ux add pandas, I get a screen full of errors. any fixes?

David

vimanuelt

Try:

 sudo pkg install -g 'GhostBSD*-dev'

davidjayjackson

Thanks, that sort of works.
The only thing that is a little screwy is if say I run:
uv add numpy (and it runs fine)
and then run
ux add pandas
It rebuilds numpy...

Thanks agin
David

vimanuelt

In that case, it’s best to build from source. That way, you know exactly what’s being built and can avoid problems that sometimes occur with precompiled packages. Don't use packages nor ports. :-)

Here's a script that fetches and builds from source. You may need to adjust it.

#!/bin/sh
# Build NumPy + pandas from source on GhostBSD/FreeBSD without ports/pkg.
# Installs into:   ~/local
# Creates venv at: ~/pandas-venv
# Logs to:         ~/bootstrap-pandas.log

set -eu

LOGFILE="$HOME/bootstrap-pandas.log"
exec > "$LOGFILE" 2>&1

MAKE_VER="4.4.1"
OPENBLAS_VER="0.3.27"
VENV_DIR="$HOME/pandas-venv"
PREFIX="$HOME/local"
SRC="$HOME/src"

# Choose Python
if command -v python3.11 >/dev/null 2>&1; then
    PYBIN="$(command -v python3.11)"
elif command -v python3 >/dev/null 2>&1; then
    PYBIN="$(command -v python3)"
else
    echo "No python3 found. Please install a system Python first." >&2
    exit 1
fi

USE_OPENMP="${USE_OPENMP:-0}"

log() {
    printf "\n[BOOTSTRAP] %s\n" "$*"
}

require_tool() {
    if ! command -v "$1" >/dev/null 2>&1; then
        echo "Missing required tool: $1" >&2
        exit 1
    fi
}

# --- Checks ---
for t in fetch tar sed awk uname cc; do require_tool "$t"; done
OSNAME=$(uname -s)
if [ "$OSNAME" != "FreeBSD" ]; then
    echo "This script is for FreeBSD/GhostBSD only." >&2
    exit 1
fi

mkdir -p "$SRC" "$PREFIX"/bin "$PREFIX"/lib "$PREFIX"/include "$HOME/.cache"

# --- Step 1: Build GNU make ---
if ! command -v gmake >/dev/null 2>&1; then
    log "Building GNU make $MAKE_VER"
    cd "$SRC"
    fetch "https://ftp.gnu.org/gnu/make/make-${MAKE_VER}.tar.gz"
    tar xzf "make-${MAKE_VER}.tar.gz"
    cd "make-${MAKE_VER}"
    ./configure --prefix="$PREFIX"
    make
    make install
    PATH="$PREFIX/bin:$PATH"
else
    PATH="$PREFIX/bin:$PATH"
    log "Using existing gmake"
fi

# --- Step 2: Build OpenBLAS ---
log "Building OpenBLAS $OPENBLAS_VER"
cd "$SRC"
fetch "https://github.com/OpenMathLib/OpenBLAS/archive/refs/tags/v${OPENBLAS_VER}.tar.gz"
tar xzf "v${OPENBLAS_VER}.tar.gz"
cd "OpenBLAS-${OPENBLAS_VER}"

if [ "$USE_OPENMP" = "1" ]; then
    gmake DYNAMIC_ARCH=1 NO_AFFINITY=1 USE_OPENMP=1
else
    gmake DYNAMIC_ARCH=1 NO_AFFINITY=1 USE_OPENMP=0
fi
gmake PREFIX="$PREFIX" install

CPPFLAGS="-I${PREFIX}/include ${CPPFLAGS:-}"
LDFLAGS="-L${PREFIX}/lib -Wl,-rpath,${PREFIX}/lib ${LDFLAGS:-}"
LD_LIBRARY_PATH="${PREFIX}/lib:${LD_LIBRARY_PATH:-}"
export CPPFLAGS LDFLAGS LD_LIBRARY_PATH PATH

# --- Step 3: Python venv ---
log "Creating Python venv at $VENV_DIR"
"$PYBIN" -m venv "$VENV_DIR"
. "$VENV_DIR/bin/activate"
python -V

log "Upgrading pip toolchain"
python -m pip install --upgrade pip setuptools wheel
python -m pip install ninja meson-python scikit-build-core Cython

# --- Step 4: NumPy ---
log "Building NumPy from source"
export NPY_BLAS_ORDER=openblas
export NPY_LAPACK_ORDER=openblas
export BLAS="${PREFIX}/lib/libopenblas.a"
export LAPACK="${PREFIX}/lib/libopenblas.a"
python -m pip install --no-binary :all: "numpy>=2.0,<3.0"

# --- Step 5: pandas ---
log "Building pandas from source"
python -m pip install --no-binary :all: pandas

# --- Step 6: Smoke test ---
log "Testing installation"
python <<'PY'
import numpy, pandas
print("NumPy:", numpy.__version__)
print("pandas:", pandas.__version__)
import numpy as np
print(np.eye(3) @ np.random.rand(3,3))
print(pandas.DataFrame({"x":[1,2,3]}).describe())
PY

log "Success! Activate with:"
echo "  . \"$VENV_DIR/bin/activate\""
log "NumPy & pandas built against OpenBLAS in $PREFIX/lib"

davidjayjackson

vimanuelt Thanks, I will give that a try.
David

davidjayjackson

Should I activate the UV environment and then perform the build?

David

vimanuelt

davidjayjackson
Stick with the standard venv created in the script unless you have a specific reason to use UV (e.g., reproducible builds across machines).
FreeBSD-based systems don’t yet have robust UV packaging integration, so plain venv + pip remains the most stable path.

vimanuelt

Here is an updated script to auto-detect uv. If uv is installed, it will use uv venv to create the environment; otherwise it falls back to python -m venv. Everything else (OpenBLAS, NumPy, pandas) stays the same.

#!/bin/sh
# Build NumPy + pandas from source on GhostBSD/FreeBSD without ports/pkg.
# Uses uv for the venv if available; otherwise falls back to python -m venv.
# Installs into:   ~/local
# Creates venv at: ~/pandas-venv
# Logs to:         ~/bootstrap-pandas.log

set -eu

LOGFILE="$HOME/bootstrap-pandas.log"
exec > "$LOGFILE" 2>&1

MAKE_VER="4.4.1"
OPENBLAS_VER="0.3.27"
VENV_DIR="${VENV_DIR:-$HOME/pandas-venv}"
PREFIX="${PREFIX:-$HOME/local}"
SRC="${SRC:-$HOME/src}"

# Pick a Python
if command -v python3.11 >/dev/null 2>&1; then
    PYBIN="$(command -v python3.11)"
elif command -v python3 >/dev/null 2>&1; then
    PYBIN="$(command -v python3)"
else
    echo "No python3 found. Please ensure a system Python is installed." >&2
    exit 1
fi

# Detect uv (Rust-based Python tool)
USE_UV=0
if command -v uv >/dev/null 2>&1; then
    USE_UV=1
fi

# Optional OpenMP for OpenBLAS (default off)
USE_OPENMP="${USE_OPENMP:-0}"

log() { printf "\n[BOOTSTRAP] %s\n" "$*"; }
need() {
    if ! command -v "$1" >/dev/null 2>&1; then
        echo "Missing required tool: $1" >&2
        exit 1
    fi
}

# --- Preflight ---
for t in fetch tar uname cc; do need "$t"; done
OS=$(uname -s)
if [ "$OS" != "FreeBSD" ]; then
    echo "This script targets FreeBSD/GhostBSD." >&2
    exit 1
fi

mkdir -p "$SRC" "$PREFIX"/bin "$PREFIX"/lib "$PREFIX"/include "$HOME/.cache"

# --- Step 1: GNU make (gmake) ---
if ! command -v gmake >/dev/null 2>&1; then
    log "Building GNU make ${MAKE_VER}"
    cd "$SRC"
    fetch "https://ftp.gnu.org/gnu/make/make-${MAKE_VER}.tar.gz"
    tar xzf "make-${MAKE_VER}.tar.gz"
    cd "make-${MAKE_VER}"
    ./configure --prefix="$PREFIX"
    make
    make install
    PATH="$PREFIX/bin:$PATH"
else
    PATH="$PREFIX/bin:$PATH"
    log "Using existing gmake"
fi
export PATH

# --- Step 2: OpenBLAS ---
log "Building OpenBLAS ${OPENBLAS_VER} (USE_OPENMP=${USE_OPENMP})"
cd "$SRC"
fetch "https://github.com/OpenMathLib/OpenBLAS/archive/refs/tags/v${OPENBLAS_VER}.tar.gz"
tar xzf "v${OPENBLAS_VER}.tar.gz"
cd "OpenBLAS-${OPENBLAS_VER}"

if [ "$USE_OPENMP" = "1" ]; then
    gmake DYNAMIC_ARCH=1 NO_AFFINITY=1 USE_OPENMP=1
else
    gmake DYNAMIC_ARCH=1 NO_AFFINITY=1 USE_OPENMP=0
fi
gmake PREFIX="$PREFIX" install

# Toolchain and runtime hints
CPPFLAGS="-I${PREFIX}/include ${CPPFLAGS:-}"
LDFLAGS="-L${PREFIX}/lib -Wl,-rpath,${PREFIX}/lib ${LDFLAGS:-}"
LD_LIBRARY_PATH="${PREFIX}/lib:${LD_LIBRARY_PATH:-}"
export CPPFLAGS LDFLAGS LD_LIBRARY_PATH

# --- Step 3: Create venv (prefer uv if present) ---
log "Creating Python virtual environment at ${VENV_DIR}"
if [ "$USE_UV" -eq 1 ]; then
    # Let uv manage the venv location, but still use pip inside it for builds
    uv venv "$VENV_DIR"
else
    "$PYBIN" -m venv "$VENV_DIR"
fi
# shellcheck disable=SC1090
. "$VENV_DIR/bin/activate"
python -V

# --- Step 4: Upgrade build toolchain ---
log "Upgrading pip/build backends"
python -m pip install --upgrade pip setuptools wheel
python -m pip install ninja meson-python scikit-build-core Cython

# --- Step 5: Build NumPy from source with OpenBLAS ---
log "Building NumPy from source (no wheels)"
export NPY_BLAS_ORDER=openblas
export NPY_LAPACK_ORDER=openblas
export BLAS="${PREFIX}/lib/libopenblas.a"
export LAPACK="${PREFIX}/lib/libopenblas.a"
python -m pip install --no-binary :all: "numpy>=2.0,<3.0"

# --- Step 6: Build pandas from source ---
log "Building pandas from source (no wheels)"
python -m pip install --no-binary :all: pandas

# --- Step 7: Smoke test ---
log "Smoke test"
python <<'PY'
import numpy, pandas, numpy as np
print("NumPy:", numpy.__version__)
print("pandas:", pandas.__version__)
print("A@A^T:\n", np.random.rand(3,3) @ np.random.rand(3,3).T)
print(pandas.DataFrame({"x":[1,2,3]}).describe())
PY

log "Done. Activate the environment with:"
echo "  . \"$VENV_DIR/bin/activate\""
log "NumPy & pandas linked against OpenBLAS in $PREFIX/lib"

davidjayjackson

Thanks, for the suggestions.
You can mark this resolved.

TX
David

vimanuelt

Thanks for the feedback.