Running a Patch

This page walks you through starting a patch run from the PatchMon web UI, from the initial click through dry-run validation, approval and live log streaming, to the final "patched" state. Everything here happens in the browser against a logged-in session.

Assumes you have the patching module enabled and the can_manage_patching permission. If the action buttons are missing, see Patching Overview for the permission matrix.

Starting Points

There are three entry points into a patch run:

All three funnel into the same Patch Wizard, a single modal used everywhere patching is initiated, so the mental model is identical regardless of where you started from.

The Patch Wizard

The wizard has a fixed six-step sequence, but it auto-skips steps that have no decision to make for your starting point. Unused steps are shown muted in the step indicator so you always see the full mental model.

Navigation is forward-only through the wizard; the Back button steps through enabled steps and skips the hidden ones.

Flow A: Patch All on a Single Host

This is the simplest case: update every out-of-date package on one host.

1. Open Hosts → select your host → Patching tab.
2. Click Patch all. The wizard opens at the Timing step (Hosts, Packages, Validate and Approval are all skipped for patch_all).
3. Review the effective patch policy shown on the Timing step. If the host has a delayed or fixed-time policy attached, the wizard tells you when the run will actually start (for example "Runs at 03:00 Europe/London").
4. If you need to bypass the delay, tick Run immediately. This sets schedule_override=immediate on the trigger call and fires the run as soon as the worker dequeues it.
5. Click Next to advance to Submit.
6. On Submit, read the per-host summary (host name, patch type, effective run time) and click Queue & patch.

What happens next on the server:

• A patch_runs row is inserted with patch_type=patch_all, dry_run=false, and the policy snapshot.
• A run_patch task is enqueued on the patching asynq queue.
• If the policy introduces a delay, the task is scheduled for the future and the run is shown as scheduled in Runs & History. Otherwise it goes straight to queued.
• The browser deep-links into the Run Detail page so you can watch the live terminal.

Note: patch_all cannot be dry-run. The agent's bulk-upgrade path (apt-get --with-new-pkgs upgrade, dnf upgrade, pkg upgrade, pacman -Syu) does not support a reliable simulation mode. If you want a dry-run, patch specific packages instead.

Flow B: Patch a Specific Package with Dry-Run

This is the richer flow and the one to use for anything security-sensitive. The dry-run runs first, you review the transaction, then you approve.

1. Open Patching → click into the package from a host, or start from Packages → package name → Patch this package.
2. The wizard opens at the Validate step (Hosts and Packages steps may be visible for fleet rollouts).
3. Click Run dry-run. The browser calls POST /patching/trigger with dry_run=true. For each target host, the server:
   • Creates a patch_runs row in pending_validation status.
   • Sends the run_patch command to the agent with DryRun=true.
   • The agent runs the package-manager simulation step (for example apt-get -s install <packages>).
4. The wizard polls each run until it terminates. While you wait you see:
   • The per-host status badge (pending validation → running → validated).
   • A live terminal excerpt for the currently running host.
5. When the dry-run completes, the run transitions to validated and the wizard shows:
   • The final Packages affected list (what would be installed, always a superset of the original request because of dependency resolution).
   • An Extra deps badge if dependency resolution pulled in packages you didn't originally ask for.
   • The captured stdout/stderr for the simulation.
6. Review the output. If the transaction looks correct, click Next to the Timing step, choose Run immediately or leave the policy delay, and then Approve & Patch on the Submit step.
7. The browser calls POST /patching/runs/{validationId}/approve. The server:
   • Marks the validation run approved (terminal state; the row and output are preserved for audit).
   • Creates a new patch_runs row with dry_run=false, linked to the validation via validation_run_id.
   • Enqueues the real run_patch task with the effective policy delay.
   • Returns the new run ID; the UI deep-links you into its Run Detail page if it's going to start immediately.

Retrying a stuck validation

If the agent was offline when you triggered the dry-run, the run will sit in pending_validation until it comes back. You have two options:

• Retry Validation: re-queues the same dry-run task. Call this once the agent is back online. Works via POST /patching/runs/{id}/retry-validation and is only available for patch_package runs.
• Skip & Patch: bypasses the dry-run entirely and queues the real patch run directly. Use this when you're confident about the change and the host won't come back soon. The UI labels the button amber to make it clear you're skipping a safety step.

Both options are available from the Runs & History table, from the Run Detail page, and inline on a per-row basis.

Submitting a patch_all run for approval

patch_all can't be dry-run, but you can still route it through an approval gate. In the wizard's Approval step tick Submit for approval. The run is created in pending_approval status with no execution task enqueued. A second reviewer with can_manage_patching can then open Runs & History, click Approve on that row, and the server builds the real execution run the same way as a validated approval. Until that happens the run sits in the DB and can also be deleted.

Flow C: Patch a Package Across the Fleet

Same as Flow B, but starting from the Packages page. The wizard discovers which hosts have the package out of date (only those show up in the Hosts step) and validates each in parallel.

1. Navigate to Packages → click the package name → Patch this package.
2. The wizard opens at Hosts. Tick the hosts you want to patch, or Select all.
3. Click Next to Validate. The wizard fans out dry-runs across the selected hosts with a bounded concurrency pool (5 at a time by default) to avoid hammering the queue.
4. When every host has reached a terminal validation state, you see a per-host results table. Hosts with failed dry-runs are flagged so you can exclude them from the approval step or retry them.
5. Timing lets you pick a per-host policy override, useful when your fleet mixes policies.
6. Submit fires POST /patching/runs/{id}/approve once per validated run. The UI tracks failures in a bulk-approve result banner when you come back to Runs & History.

Watching a Run: The Run Detail Page

Once a run is fired and not delayed, the UI redirects you into /patching/runs/{id}. The page has:

• A header with the host name, status badge, Awaiting inventory report pill (post-patch), and the primary action buttons for the current state (Approve & Patch, Retry Validation, Skip & Patch, Stop Run).
• A left-side Run summary card: host, type, initiated by, approved by, started / completed timestamps, link to the validation run if any, patch policy in effect, and packages affected.
• A right-side Shell output terminal for the live log stream.

Live log streaming

While the run is running, the Run Detail page opens a WebSocket connection to /api/v1/patching/runs/{id}/stream. The server's in-process patchstream hub fans out agent-published events to every connected browser:

• snapshot: sent once when you connect. Contains the current stage and whatever shell_output is already persisted, so the terminal pane is primed even if you arrived mid-run.
• chunk: a short piece of stdout/stderr from the agent, appended to the terminal as it arrives.
• done: a terminal stage (completed, failed, cancelled, validated, or dry_run_completed). The socket closes and the UI refetches run metadata to show the final state.

The header shows a pulsing green Live pill while the WebSocket is open. If you scroll up in the terminal to read earlier output, the UI stops auto-scrolling; scroll back to the bottom and it resumes.

If you arrive on a run that's already in a terminal state, the server sends a single snapshot message containing the persisted shell_output plus a synthetic done message, then closes the connection. The database is the source of truth, so you see exactly the same terminal contents as everyone else.

Copying output

When the run is not queued or running, a Copy output button appears above the terminal. It copies the full shell output to your clipboard. Use this for incident reports or to paste into a ticket.

The terminal normalises carriage returns. apt-get and dpkg use \r to overwrite progress bars on a single line, which would be invisible in a scrollback view. The UI converts \r to \n so every progress update becomes its own readable line.

Stopping a Running Patch

You can stop a run while it is in the running state. This is a hard stop with no graceful "let it finish the current package" behaviour.

1. On the Run Detail page, click Stop Run in the header.
2. Confirm in the dialog. The warning "Partially-installed packages may leave the host in an intermediate state" is there for a reason. Interrupting apt or dnf mid-transaction can leave dpkg or rpmdb needing manual repair.
3. The browser calls POST /patching/runs/{id}/stop. The server looks up the agent in the agentregistry, sends a patch_run_stop WebSocket message, and returns 202 Accepted.
4. The agent cancels the subprocess via SIGINT, collects whatever output it has, and reports a terminal cancelled stage back to the server.
5. The live stream closes and the run's final status is cancelled.

When Stop Run is not available

• The run is not in the running state (no subprocess to interrupt). For queued, scheduled, pending_validation, pending_approval, or validated, use Delete in Runs & History instead. That also removes any scheduled task from the asynq queue.
• The agent is not currently connected. The UI returns a 409 with "Agent is not currently connected". Wait for the agent to reconnect, or stop the agent's systemd service if you need the patch process killed at the OS level.

Post-Patch: The Awaiting Inventory Report Pill

When a patch_all or non-dry-run patch_package completes, the server sets awaiting_post_patch_report_run_id on the host. The Run Detail page shows an Awaiting inventory report pill next to the status badge, and the polling interval keeps ticking every 3 seconds.

The agent's next scheduled inventory report (usually within 60 minutes, sooner if triggered manually) updates the host's package list. The server clears the awaiting flag, and:

• If the host's last_update timestamp is newer than the run's completed_at, the pill changes to New report received.
• Otherwise the pill disappears. Absence is the soft success signal.

This is how you know the packages the patch run installed are now reflected in the Package Inventory view, not just that the apt command returned exit 0.

Troubleshooting Common Cases

The run sits in queued forever

The asynq worker picked up the task but cannot reach the agent, or the agent never received the WebSocket command.

• Check the agent's connection status on the Host Detail page (green Connected pill).
• Check the server logs for patching: entries around the task ID (patch-run-<run-id>).
• If the agent has been offline and just reconnected, wait up to 30 seconds for the WebSocket re-handshake.

"package manager ... not supported for patching"

The agent rejected the run because the host's detected package manager is not in the supported list (apt, dnf, yum, pkg, pacman). This is the error you see on Alpine (apk) hosts today. The run immediately transitions to failed with the message in error_message.

The run completes but the package inventory hasn't updated

The agent hasn't sent its post-patch inventory report yet. The Awaiting inventory report pill will flip automatically once it arrives. If it doesn't arrive within an hour, force a report from the agent CLI:

sudo patchmon-agent report

Approve returns 400 "Only validated... runs can be approved"

Another operator already approved or deleted the run while you were looking at it. Reload Runs & History; the row will now be in approved or gone.

Related Documentation

• Patching Overview: the three core concepts, module gates, and supported OS coverage.
• Patch Policies and Scheduling: control when approved runs actually fire.
• Patch History and Live Logs: the Runs & History table, filtering, and deeper dive into the live log stream.
• Managing the PatchMon Agent: agent CLI, service management, and troubleshooting connection issues.