Skip to main content
PathMon

Chapter 19 of 26

Notification Routes and Delivery Log

Updated Read the full guide

Overview

In PatchMon, a route (labelled Event Rule in the UI) connects one or more event types, and optionally a severity floor, a host scope, or a match rule, to a destination. When an event fires, the notifications engine evaluates every enabled route; each matching route produces a delivery to its destination.

Routes handle fan-out: one host_down event can notify your on-call ntfy topic, post to a Discord #alerts channel, and write an internal alert record, all from a single emit.

Both routes and the Delivery Log live under Reporting in the main navigation:

  • Reporting → Event Rules: create, edit, and disable routes.
  • Reporting → Delivery Log: every outbound delivery attempt, sent or failed.

Permissions

Action Permission
Create / edit / disable / delete routes can_manage_notifications
Read the delivery log can_view_notification_logs

Admins and superadmins bypass these checks.

Creating a route

  1. Go to Reporting → Event Rules.
  2. Click Add event rule. (Disabled until at least one destination exists. Create one first under Notification Destinations.)
  3. Fill in the modal:
Field Notes
Destination Required. Pick from the list of configured destinations. You can only route to enabled destinations; disabled destinations are skipped at delivery time.
Events Tick All events to match every event type, or tick individual events. Selecting every individual event collapses back to "All events".
Minimum severity Floor for the route. Events below this severity are ignored. Order is informational < warning < error < critical.
Host groups Optional. If any are selected, only events whose host is a member of at least one of the groups match. Leave empty for "any host".
Individual hosts Optional. If any are selected, only events for those specific hosts match. Leave empty for "any host".
Enabled On by default. Turn off to keep the rule for later without it firing.
  1. Click Add.

Event type reference

Pick from the same set documented in Alerts Overview. host_down, host_recovered, patch_run_completed, ssh_session_started, and so on. You can also select high-volume or low-volume events like user_login and account_locked to route sign-in telemetry.

Host group and host filters combined

If both host groups and individual hosts are set, the event must satisfy both filters. In practice you usually pick one or the other, not both.

Events without a host context (e.g. server_update, user_created) are filtered out by any host scope you add. Leave both scope fields empty to match those as well.

Severity, delay, and lifecycle

Per-type Alert delay in Alert Lifecycle applies before the route fan-out: if the event has a configured alert_delay_seconds, PatchMon enqueues the delivery with that delay. If a counterpart event fires within the window (for example, host_recovered while a delayed host_down is queued), the delayed notification is cancelled. Counterpart mapping:

Delayed event Cancelled by
host_down host_recovered
container_stopped container_started
host_security_updates_exceeded host_security_updates_resolved
host_pending_updates_exceeded host_pending_updates_resolved

Editing and deleting routes

Each row in Event Rules has Edit and Delete buttons.

  • Edit reopens the modal with the saved values. Save to update; the new criteria take effect for the next matching event.
  • Delete removes the rule entirely. Deliveries already enqueued finish, but no new deliveries are produced.

Disabled routes are displayed with a muted Disabled badge and do not receive deliveries. Disable is the safer option if you want to pause a rule temporarily.

How matching works

For each outgoing event, the server:

  1. Looks up all routes whose event_types include the event type (or the wildcard *).
  2. Drops routes whose destination is disabled.
  3. Drops routes whose min_severity is above the event's severity.
  4. For each remaining route, applies the host-group and host-ID filters.
  5. Deduplicates: events that repeat within a 2-minute window for the same destination are collapsed into one delivery. The fingerprint key is event_type + reference_id + destination_id + 2-minute bucket.
  6. Rate-limits: each destination is capped at 60 deliveries per minute. Deliveries over the cap are dropped with a warning in the server log.
  7. Enqueues an asynq task to the notifications queue with MaxRetry=5.

The queue worker then dispatches the delivery according to the destination's channel type (SMTP send, HTTP POST, ntfy publish, or internal alert write).

The Delivery Log

The Delivery Log tab shows every outbound notification attempt with its result. Use it when a destination is not receiving messages, a webhook recipient reports errors, or you want an audit trail of what went where.

Columns

Column Meaning
Time When the delivery was processed, shown as a relative time ("5m ago"). Hover for the exact timestamp.
Status sent for success (green), anything else for failure (red).
Event The event type that produced the delivery (e.g. host_down, patch_run_failed).
Destination The destination display name at the time of delivery. Shows the UUID if the destination has been deleted.
Reference reference_type:reference_id, clickable for host, patch_run, and alert references so you can jump to the source.
Error Error message returned by the delivery attempt. Empty for successful deliveries.

Pagination

The log is paginated at 50 rows per page. Use the left and right arrows at the bottom to move through history. The most recent deliveries are on page 1.

Use the Refresh log button in the page header to pull the latest entries without navigating away.

Retries

The notifications worker retries failed deliveries up to 5 times with exponential back-off (handled by asynq). Each attempt is recorded on the same delivery log row: the attempt_count field increments, and the row is upserted with the latest status and error_message. The provider message ID (e.g. SMTP queue ID, webhook Message-ID) is captured in provider_message_id when the remote end returns one.

If all five retries fail, the delivery row stays at the last failed state. There is no automatic escalation; diagnose the failure from the Error column.

Common failure reasons

Error (excerpt) Likely cause
connect: connection refused / i/o timeout Destination host is unreachable from the PatchMon server. Check firewall / network.
authentication failed / 535 5.7.8 Wrong SMTP credentials or token. Re-edit the destination and re-enter.
400 Bad Request from Discord/Slack webhook Webhook URL is wrong, revoked, or the rich payload is malformed for a customised Slack app.
403 Forbidden from ntfy Topic requires auth you have not provided, or token is expired.
destination disabled Someone disabled the destination between enqueue and delivery. Re-enable and re-trigger.

If an expected entry is missing, check that the route is enabled, the destination is enabled, the event passed the severity and scope filters, and the alert type itself is enabled in Alert Lifecycle.

Deduplication and rate-limiting in the log

Duplicates suppressed by the 2-minute dedup window do not appear in the delivery log; they are silently skipped before a delivery task is created. Rate-limited deliveries are also skipped silently (a warning goes to the server log, not the delivery log). If a destination suddenly stops receiving events, check:

  1. The destination is enabled.
  2. No route has been deleted.
  3. The per-minute rate cap is not being exceeded upstream. 60 messages/minute is per-destination.

Every notification includes an app_link in its metadata pointing back to the most relevant page in PatchMon:

Reference type Link
patch_run /patching/runs/<id>
host /hosts/<id>
alert /hosts/<host_id> if known, otherwise /
user /settings/users
test /reporting

Formatters for each channel render this as a clickable button (Discord/Slack rich embeds), an <a> tag (email), or a Click action (ntfy).