Agent Troubleshooting

Overview

This page is a quick-reference decision tree for the most common PatchMon agent symptoms. Each entry points to detailed steps in Managing the PatchMon Agent: Common Troubleshooting or another relevant section. Use this page to triage, then follow the linked section for the full fix.

If the problem is on the server side (container crashes, database migrations, CORS, reverse proxy), go to Server Troubleshooting instead.

The 30-Second Triage

Before anything else, run these three commands on the affected host:

sudo patchmon-agent diagnostics    # full picture in one shot
sudo systemctl status patchmon-agent    # (or: rc-service patchmon-agent status)
sudo journalctl -u patchmon-agent -n 50 --no-pager    # (or: tail -n 50 /etc/patchmon/logs/patchmon-agent.log)

On Windows (elevated PowerShell):

patchmon-agent diagnostics
Get-Service PatchMonAgent
Get-Content 'C:\ProgramData\PatchMon\patchmon-agent.log' -Tail 50

The diagnostics output includes system info, configuration status, network reachability, credential validity, and the last 10 log lines. In most cases it tells you immediately which part is broken. Full details: Managing the PatchMon Agent: diagnostics.

Decision Tree

Symptom → Likely Cause → Where to Look

Host Shows Pending

The host record was created but the agent has not sent its first report.

Quick checks:

sudo systemctl status patchmon-agent
sudo patchmon-agent ping
sudo patchmon-agent report      # force an immediate report

If ping fails, see Cannot reach server: DNS, TLS, or firewall.

If ping succeeds but the host stays Pending, the agent has not pushed a report yet. Run sudo patchmon-agent report and watch journalctl -u patchmon-agent -f for errors during report upload. A common cause is 413 Request Entity Too Large. See Server Troubleshooting: Upload/Body-Limit Errors.

Full details: Managing the PatchMon Agent: Agent Shows "Pending" in PatchMon.

Host WS Pill is Red

The host sent at least one report in the past, but its WebSocket has been disconnected for longer than the host_down threshold (default 30 seconds, configurable in Reporting → Alert Lifecycle). The host may still be alive — check the Reporting pill: if it's green, the agent is pushing HTTP reports normally and only the real-time control channel is unavailable.

Quick checks:

sudo systemctl is-active patchmon-agent    # should print "active"
sudo journalctl -u patchmon-agent -n 50 --no-pager

Common causes:

• Service stopped or crashed: sudo systemctl restart patchmon-agent and watch the logs for the underlying error.
• Reverse proxy not forwarding WebSocket upgrade headers: see Server Troubleshooting: Agent Can't Connect Over WebSocket.
• NAT / load balancer timing out idle connections: raise the proxy's idle timeout to at least 65 s. The agent sends WebSocket pings every 30 s.
• Temporary network blip: the agent auto-reconnects with exponential backoff. Wait 60 s and re-check. The WS pill goes amber for the grace window, then red.

Full details: Managing the PatchMon Agent: Agent's WS Pill is Red in PatchMon.

Host Reporting Pill is Stale

The agent hasn't pushed an HTTP report within its update interval and the WebSocket is also disconnected. This is the strongest indicator that the host is genuinely unreachable (as opposed to just losing the real-time channel).

Quick checks:

# From the affected host (if you can reach it):
sudo systemctl status patchmon-agent
sudo patchmon-agent ping
sudo patchmon-agent report       # force an immediate report

# From another host:
ping <host-ip>
ssh <host>                       # confirm host is alive

Common causes:

• Host is genuinely down (powered off, kernel panic, hardware fault). Check console / hypervisor.
• Network partition between the host and the PatchMon server. Verify outbound HTTPS to the server URL still works.
• Agent service stopped without WebSocket disconnect notice (e.g. host was suspended). sudo systemctl restart patchmon-agent once it's reachable.

If the host is online but only the Reporting pill is red while WS is also red, run sudo patchmon-agent report to push a fresh report and the pill should flip back to green.

Agent Won't Start

The service exits immediately or never stays running.

Diagnose by running the agent in the foreground with debug logging:

sudo systemctl stop patchmon-agent
sudo patchmon-agent serve --log-level debug
# (Ctrl+C to stop; then: sudo systemctl start patchmon-agent)

Common causes:

• Broken config.yml: YAML syntax error. Validate with yq e . /etc/patchmon/config.yml or reinstall to regenerate defaults.
• Missing credentials.yml: see Credentials file missing or corrupt.
• Invalid patchmon_server URL: must start with http:// or https://, no trailing slash.
• Port conflict (rare): the agent itself doesn't listen on any port, but if you've enabled the SSH-proxy integration with a static local port, check with ss -lntp | grep patchmon.
• Wrong architecture binary: file /usr/local/bin/patchmon-agent and compare against uname -m. If they disagree, reinstall from the UI so the correct binary is fetched.

Full details: Managing the PatchMon Agent: Testing and Diagnostics and Debugging a Problem.

Cannot Reach Server: DNS

Log or ping output contains no such host, dial tcp: lookup <hostname>, or server misbehaving.

Diagnose:

nslookup patchmon.example.com
# or
dig +short patchmon.example.com

Fix:

• If DNS resolution fails, fix /etc/resolv.conf or your internal DNS records.
• Inside containers (Docker/LXC/Kubernetes): make sure the container's DNS resolver can reach your internal DNS.
• If you use a /etc/hosts override, check the entry matches the URL in /etc/patchmon/config.yml.

Full details: Managing the PatchMon Agent: "Connectivity Test Failed".

Cannot Reach Server: TLS

Log contains x509: certificate signed by unknown authority, tls: failed to verify certificate, or (on Windows) Could not establish trust relationship for the SSL/TLS secure channel.

Preferred fix: install the CA into the system trust store:

• Debian/Ubuntu: copy CA to /usr/local/share/ca-certificates/ (must have .crt extension) and run sudo update-ca-certificates.
• RHEL/Rocky/Fedora: copy CA to /etc/pki/ca-trust/source/anchors/ and run sudo update-ca-trust.
• Alpine: sudo apk add ca-certificates, then same as Debian.
• Windows: import via certlm.msc into Local Computer → Trusted Root Certification Authorities.

Quick bypass (lab only): set skip_ssl_verify: true in /etc/patchmon/config.yml and restart the service, or set env var PATCHMON_SKIP_SSL_VERIFY=true.

Do not enable skip_ssl_verify in production. It disables TLS verification entirely. See the warning in Managing the PatchMon Agent: SSL Certificate Errors.

Full details: Managing the PatchMon Agent: SSL Certificate Errors.

Cannot Reach Server: Firewall

ping reports "connectivity test failed" but DNS and TLS are fine. Typically i/o timeout or connection refused.

Diagnose:

# TCP reachability on 443
timeout 5 bash -c "</dev/tcp/patchmon.example.com/443" && echo open || echo blocked
# or:
nc -vz patchmon.example.com 443

Fix:

• Open outbound TCP/443 (or whatever port your reverse proxy uses) to the PatchMon server on the host's / network's egress firewall.

• If the host is behind an outbound HTTP proxy, set HTTPS_PROXY and HTTP_PROXY in the agent's environment. On systemd, add them as Environment= lines to a drop-in at /etc/systemd/system/patchmon-agent.service.d/proxy.conf:

  [Service]
  Environment="HTTPS_PROXY=http://proxy.corp:3128"
  Environment="HTTP_PROXY=http://proxy.corp:3128"
  Environment="NO_PROXY=localhost,127.0.0.1"
  

  then sudo systemctl daemon-reload && sudo systemctl restart patchmon-agent.

Full details: Managing the PatchMon Agent: "Connectivity Test Failed".

Report Interval Not Updating

You changed the report interval in Settings → Agent Updates, but the affected host still reports on the old schedule.

Background: The server pushes update_interval changes to connected agents via the WebSocket. If the WebSocket is down, or the change was made just before a restart, the agent may be running on its cached value in /etc/patchmon/config.yml.

Fix:

# 1. Verify the WebSocket is up (host shows as Online in the UI)
# 2. Restart the agent so it fetches fresh values on startup
sudo systemctl restart patchmon-agent

# 3. Check the value the agent sees
sudo patchmon-agent config show | grep -i interval

The agent syncs update_interval, docker_enabled, and compliance_enabled from the server on startup and on every WebSocket-pushed setting change, then writes the result back to config.yml. No restart is normally required for runtime sync. A restart is only needed for the first startup sync.

Full details: Managing the PatchMon Agent: Configuration Management (see "When do changes require a restart?" table).

Auto-Update Failing

The agent log contains update failed, hash mismatch, binary verification failed, or update was performed X ago, skipping.

Diagnose:

sudo patchmon-agent --version
sudo patchmon-agent check-version
ls -la /etc/patchmon/.last_update_timestamp
ls -la /usr/local/bin/patchmon-agent.backup.*

Common causes:

• Update loop cooldown: the agent refuses to re-update within 5 minutes of the last attempt. Wait 5 minutes.
• Server has not shipped a new binary: verify on the server that the file in AGENT_BINARIES_DIR (or AGENTS_DIR) matches the expected architecture.
• Hash mismatch: the server must send a SHA-256 hash of the binary. If it doesn't, the agent refuses to install (this is mandatory for security, not a bug). Update your server to a release that provides hashes.
• SSL error during download: treat as Cannot reach server: TLS. skip_ssl_verify is explicitly blocked for binary downloads in production, so fix the certificate trust instead.
• Auto-update disabled: patchmon-agent check-version will say Auto-update disabled by server administrator. Enable it at Settings → Agent Updates → Master auto-update and the per-host toggle on the host detail page.

Manual force:

sudo patchmon-agent update-agent

Full details: Managing the PatchMon Agent: Agent Updates and Agent Not Auto-Updating.

Credentials File Missing or Corrupt

Log contains credentials file not found, failed to load credentials, or API credentials are missing.

Diagnose:

ls -la /etc/patchmon/credentials.yml
# expected: -rw------- 1 root root  ~120  <date>  /etc/patchmon/credentials.yml

sudo cat /etc/patchmon/credentials.yml
# expected:
#   api_id: "patchmon_abc123"
#   api_key: "<64 hex chars>"

If the file is missing:

# Reconfigure using the credentials shown at enrolment time.
# If you don't have them, regenerate from the UI:
#   Hosts -> <host> -> Show Credentials -> Regenerate
sudo patchmon-agent config set-api <API_ID> <API_KEY> <SERVER_URL>

If the file is present but permissions are wrong:

sudo chmod 600 /etc/patchmon/credentials.yml
sudo chown root:root /etc/patchmon/credentials.yml

If the YAML is malformed (e.g. manually edited and broken), either restore the latest backup (ls -la /etc/patchmon/credentials.yml.backup.*) or re-run config set-api.

Full details: Managing the PatchMon Agent: "Credentials File Not Found" and config set-api.

Logs Not Being Written

The agent is running but /etc/patchmon/logs/patchmon-agent.log is empty or missing.

Diagnose:

# Does the directory exist and is it writable by root?
ls -la /etc/patchmon/logs/
# expected: drwx------ 2 root root ...

# Check disk space
df -h /etc/patchmon

Common causes:

• Log directory missing: sudo mkdir -p /etc/patchmon/logs && sudo chmod 700 /etc/patchmon/logs.
• log_file points elsewhere: run sudo patchmon-agent config show and check the path.
• Disk full: df -h. Free space or change log_file to a different partition.
• Running as the wrong user (non-standard install): the agent must run as root. See Permission denied.
• systemd-only logging: on systemd, the agent logs to both the journal and the file. If the file is empty but journalctl -u patchmon-agent has entries, the file may be rotating correctly. Check for .log.gz backups.

The agent uses built-in log rotation (10 MB per file, 5 backups, 14-day retention). You do not need logrotate.

Full details: Managing the PatchMon Agent: Viewing Logs.

Service Keeps Restarting

systemctl status patchmon-agent shows activating (auto-restart) repeatedly, or the process count climbs.

Diagnose:

sudo systemctl status patchmon-agent
sudo journalctl -u patchmon-agent --since "15 minutes ago" --no-pager

Common causes:

• Invalid config.yml (YAML syntax error): the agent refuses to start; systemd restarts it every 10 s.
• Invalid credentials: the agent logs "invalid API credentials" and exits.
• Server unreachable for long: the agent will keep trying (it does not exit on network errors in normal operation), so constant restarts point at one of the two above.
• Binary crash: check dmesg for OOM kills or segfaults: sudo dmesg -T | grep -i patchmon.

The systemd unit uses Restart=always with RestartSec=10, which is correct for production but masks crash loops. Disable auto-restart temporarily to see the real error:

sudo systemctl edit patchmon-agent --force --full
# change: Restart=always  ->  Restart=no
sudo systemctl daemon-reload
sudo systemctl start patchmon-agent
# read the single run's output, then revert:
sudo systemctl edit patchmon-agent --force --full    # restore Restart=always
sudo systemctl daemon-reload

Full details: Managing the PatchMon Agent: Service Keeps Restarting.

Permission Denied

Any agent command exits immediately with permission denied or this script must be run as root.

Fix:

• Linux / FreeBSD: use sudo. The agent reads package databases, writes to /etc/patchmon/, and manages the system service; none of which work without root.
• Windows: open PowerShell with Run as Administrator. A non-elevated PowerShell cannot read installed-package inventory or manage the PatchMonAgent service.

If you are root and still see permission errors, check file ownership:

ls -la /etc/patchmon/config.yml /etc/patchmon/credentials.yml /usr/local/bin/patchmon-agent
# All three should be owned by root.

Full details: Managing the PatchMon Agent: "Permission Denied" Errors.

Escalation

If the decision tree above does not match your symptom, collect the following before asking for help:

# Everything in one command
sudo patchmon-agent diagnostics > /tmp/patchmon-diag.txt 2>&1
sudo journalctl -u patchmon-agent --since "1 hour ago" --no-pager \
  > /tmp/patchmon-journal.txt 2>&1
sudo tail -n 200 /etc/patchmon/logs/patchmon-agent.log \
  > /tmp/patchmon-log.txt 2>&1
sudo patchmon-agent config show > /tmp/patchmon-config.txt 2>&1
sudo patchmon-agent --version >> /tmp/patchmon-config.txt
uname -a >> /tmp/patchmon-config.txt

Redact the api_id before sharing the bundle publicly. The full api_id is sensitive even though the key hash is not shown by config show.

See Also

• Managing the PatchMon Agent: full CLI, service, log, and removal reference.
• Managing the PatchMon Agent: Common Troubleshooting: the detailed counterpart to this decision tree.
• Agent Configuration Reference (config.yml): every config parameter.
• Installing the PatchMon Agent: enrolment walkthrough.
• Uninstalling the PatchMon Agent: removal walkthrough.
• Server Troubleshooting: for server-side issues.