Patch Management

---
name: patch-management
description: >
  Update and patch the managed hosts and their containers safely. Use whenever
  the user asks to update, upgrade, or patch a host, VM, node, or app; asks "are we
  patched", "any updates", "update proxmox/truenas/ubuntu", "update my
  containers", or wants to apply pending OS or image updates. Covers Debian/Ubuntu
  hosts, Proxmox, TrueNAS, and Docker containers. Diagnose what's pending first,
  snapshot before applying, confirm, then verify. This is about the MANAGED
  hosts, not the HomelabHero control plane.
---

# Patch management

Updates the machines HomelabHero manages and the containers on them. It does NOT
update the control-plane LXC itself - that is the operator-only `hh update`.

This is a state-changing workflow: diagnose what's pending, propose it, get a
go-ahead, and prefer a snapshot/backup first (use the backup-restore skill),
especially for OS, kernel, and hypervisor updates.

## Golden rules

- Snapshot/back up before an OS, kernel, or hypervisor update. Offer it every
  time for those; container image bumps are lower risk but still confirm.
- One host at a time. Apply, verify it came back (`hh test`, `hh overview`), then
  move on. Never update the whole estate in one shot.
- Watch for required reboots and say so; do not reboot without a go-ahead.
- Verify after: the service is back, the app works, nothing new is in
  `systemctl --failed` / the container is healthy.

## Debian / Ubuntu hosts

    hh run <alias> "apt-get update && apt list --upgradable 2>/dev/null"   # what's pending
    hh run <alias> "DEBIAN_FRONTEND=noninteractive apt-get -y upgrade"     # apply (confirm)
    hh run <alias> "ls /var/run/reboot-required 2>/dev/null && echo REBOOT-NEEDED" # kernel/libc?

## Proxmox

Update package lists and apply; kernel updates need a reboot. On a cluster do one
node at a time and migrate guests off first. Back up VMs (vzdump or PBS) before a
major upgrade.

    hh run pve1 "pveupdate && pveupgrade --shell no 2>/dev/null; apt list --upgradable 2>/dev/null"
    hh run pve1 "apt-get -y dist-upgrade"                      # apply (confirm; reboot if kernel)

## TrueNAS

TrueNAS is an appliance - do NOT `apt upgrade` it. Updates are train-based and are
safest applied from the web UI. Check availability over SSH, but hand the actual
apply to the UI unless the user asks otherwise (it reboots and manages the boot
environment for rollback).

    hh run truenas "midclt call update.check_available"   # is an update staged? (midclt needs no sudo)
    # Applying: TrueNAS UI -> System -> Update (creates a boot environment you can roll back to).

## Docker containers

Pull new images and recreate. Location depends on the layout in
`infra/docker-stacks.md`:

    # single-file layout
    hh run <dockerhost> "cd /mnt/<pool>/docker && docker compose pull && docker compose up -d"
    # per-stack (Dockge)
    hh run <dockerhost> "cd /opt/stacks/<stack> && docker compose pull && docker compose up -d"
    # reclaim space afterward (safe - images only, never volumes)
    hh run <dockerhost> "docker image prune -f"

If Watchtower or Tugtainer is running, image updates may be automated already -
check before doing it by hand. Note: Watchtower does not touch TrueNAS catalog
apps; those use `WATCHTOWER_DISABLE_CONTAINERS=ix*`, and catalog apps update from
the TrueNAS UI, not here.

## After

Confirm the host/containers are healthy (`hh overview`, `hh test <alias>`, or the
app's own check). If a reboot is pending, say so and let the user choose the window.
If an update broke something, that's a runbook entry (symptom, cause, fix).