Skip to main content
PathMon

Chapter 2 of 15

Installing PatchMon on Kubernetes

Updated Read the full guide

Overview

There are two ways to run PatchMon on Kubernetes:

  • The community Helm chart, covered by most of this chapter. Best if you want values-driven configuration and upgrades handled for you.
  • Plain manifests, covered by Deploying with plain manifests. Best if you deploy with Argo CD or Flux, or you simply want to see and own every object. This is also the route to follow on k3s.

Either way, read the "Important: PatchMon 2.0 architecture" note below first, because it explains what changed from the 1.4.x Node.js stack.

The community Helm chart for PatchMon deploys the server on any Kubernetes 1.19+ cluster. It is maintained in a separate repository:

Verify against the latest chart. This page describes the chart values shape. The chart is community-maintained and may be a version or two behind the latest PatchMon release. Always check the chart's own README for the current value names and defaults before upgrading.

Important: PatchMon 2.0 architecture

The Helm chart was originally written against the 1.4.x Node.js stack, which shipped a separate patchmon-backend and patchmon-frontend image and used BullMQ for background jobs. PatchMon 2.0 consolidates everything into a single Go binary: the React frontend is embedded, chi serves both /api/* and the SPA, background jobs run on Asynq, and schema migrations are applied automatically at boot by golang-migrate.

Practical implications for the chart:

Concept 1.4.x (Node) 2.0+ (Go)
Containers patchmon-backend + patchmon-frontend Single patchmon-server
Queue BullMQ Asynq
Migrations Prisma golang-migrate (embedded, automatic)
Listen port backend 3001, frontend 3000 server 3000 (both /api/* and SPA)
RDP sidecar n/a guacamole/guacd (optional, required for in-browser RDP)

If your version of the chart still ships with separate backend and frontend deployments, set the frontend deployment to enabled: false and point your Ingress exclusively at the backend image tagged ghcr.io/patchmon/patchmon-server:2.0.0 or later, exposed on port 3000. Where the chart's values file uses backend.env.* keys, those now map to the single patchmon-server container's environment.


Prerequisites

  • Kubernetes 1.19+
  • Helm 3.0+
  • A PersistentVolume provisioner in the cluster (for PostgreSQL and Redis data)
  • An Ingress controller (e.g. NGINX Ingress) for external access (strongly recommended)
  • cert-manager for automatic TLS certificate management (optional)
  • Metrics Server for HPA (optional)

Container Images

Component Image Default Tag
Server ghcr.io/patchmon/patchmon-server 2.0.0
Database docker.io/postgres 17-alpine
Redis docker.io/redis 7-alpine
guacd (RDP sidecar, optional) docker.io/guacamole/guacd 1.6.0

Available tags (server image)

Tag Description
latest Latest stable release
x.y.z Exact version pin (e.g. 2.0.0)
x.y Latest patch in a minor series (e.g. 2.0)
x Latest minor and patch in a major series (e.g. 2)
edge Latest development build from the main branch. Unstable, for testing only.

Quick Start

The quickest way to try PatchMon on Kubernetes is the provided values-quick-start.yaml. It contains placeholder secrets and sensible defaults for a single-command install.

Warning: values-quick-start.yaml ships with placeholder secrets and is intended for evaluation only. Never use it in production without replacing all secret values.

1. Install the chart

wget https://raw.githubusercontent.com/RuTHlessBEat200/PatchMon-helm/refs/heads/main/values-quick-start.yaml

helm install patchmon oci://ghcr.io/ruthlessbeat200/charts/patchmon \
  --namespace patchmon \
  --create-namespace \
  --values values-quick-start.yaml

2. Wait for pods to become ready

kubectl get pods -n patchmon -w

The server pod runs embedded migrations on first boot. Follow the logs to watch them apply:

kubectl logs -n patchmon deploy/patchmon-server -f

3. Access PatchMon

If an Ingress is configured, open the host you set (e.g. https://patchmon.example.com).

Without Ingress, use port-forwarding:

kubectl port-forward -n patchmon svc/patchmon-server 3000:3000

Then open http://localhost:3000 and complete the first-time admin setup.


Production Deployment

For production, start from values-prod.yaml in the chart repository. The example below demonstrates how to:

  • Use an external Kubernetes Secret (managed by SOPS, Sealed Secrets, or External Secrets Operator) instead of inline passwords
  • Configure HTTPS with cert-manager
  • Set the CORS_ORIGIN to the external URL your users access (comma-separate multiple URLs with no spaces if PatchMon is reached from more than one origin)

1. Create your secrets

The chart does not auto-generate secrets. You must supply them yourself.

Required secrets for a PatchMon 2.0 deployment:

Key Description
postgres-password PostgreSQL password
redis-password Redis password
jwt-secret JWT signing secret used by the server
ai-encryption-key Encryption key used for AI provider credentials, bootstrap tokens, and other secrets at rest
oidc-client-secret OIDC client secret (only when OIDC is enabled)

Example: creating a Secret manually

kubectl create namespace patchmon

kubectl create secret generic patchmon-secrets \
  --namespace patchmon \
  --from-literal=postgres-password="$(openssl rand -hex 32)" \
  --from-literal=redis-password="$(openssl rand -hex 32)" \
  --from-literal=jwt-secret="$(openssl rand -hex 64)" \
  --from-literal=ai-encryption-key="$(openssl rand -hex 32)"

Recommended secret-management tools for production:

2. Create your values file

Start from values-prod.yaml and adjust:

global:
  storageClass: "your-storage-class"
  imageTag: "2.0.0"

fullnameOverride: "patchmon-prod"

server:
  env:
    # Comma-separate with no spaces to allow multiple origins, e.g.
    # "https://patchmon.example.com,https://patchmon.internal.lan"
    CORS_ORIGIN: "https://patchmon.example.com"
    ENABLE_HSTS: "true"
    TRUST_PROXY: "true"
  existingSecret: "patchmon-secrets"
  existingSecretJwtKey: "jwt-secret"
  existingSecretAiEncryptionKey: "ai-encryption-key"

database:
  auth:
    existingSecret: "patchmon-secrets"
    existingSecretPasswordKey: "postgres-password"

redis:
  auth:
    existingSecret: "patchmon-secrets"
    existingSecretPasswordKey: "redis-password"

secret:
  create: false

ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "86400"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
  hosts:
    - host: patchmon.example.com
      paths:
        - path: /
          pathType: Prefix
          service:
            name: server
            port: 3000
  tls:
    - secretName: patchmon-tls
      hosts:
        - patchmon.example.com

Verify against the latest chart. The exact value keys (server.* vs backend.*) depend on whether the chart has been updated for 2.0. If your chart still splits backend and frontend, set frontend.enabled: false and expose only the backend on port 3000.

3. Install

helm install patchmon oci://ghcr.io/ruthlessbeat200/charts/patchmon \
  --namespace patchmon \
  --create-namespace \
  --values values-prod.yaml

Configuration Reference

Verify against the latest chart. The table below reflects the original chart value names. 2.0 versions of the chart are expected to drop the separate frontend.* block and consolidate everything under a single server.* block. Always diff against the chart's values.yaml before changing things.

Global Settings

Parameter Description Default
global.imageRegistry Override the image registry for all components ""
global.imageTag Override the image tag for the PatchMon image (takes priority over individual tags) ""
global.imagePullSecrets Image pull secrets applied to all pods []
global.storageClass Default storage class for all PVCs ""
nameOverride Override the chart name used in resource names ""
fullnameOverride Override the full resource name prefix ""
commonLabels Labels added to all resources {}
commonAnnotations Annotations added to all resources {}

Database (PostgreSQL)

Parameter Description Default
database.enabled Deploy the PostgreSQL StatefulSet true
database.image.registry Image registry docker.io
database.image.repository Image repository postgres
database.image.tag Image tag 17-alpine
database.auth.database Database name patchmon_db
database.auth.username Database user patchmon_user
database.auth.password Database password (required unless existingSecret is set) ""
database.auth.existingSecret Existing Secret containing the password ""
database.auth.existingSecretPasswordKey Key inside the existing Secret postgres-password
database.persistence.enabled Enable persistent storage true
database.persistence.size PVC size 5Gi
database.resources.requests.cpu CPU request 100m
database.resources.requests.memory Memory request 128Mi
database.resources.limits.memory Memory limit 1Gi
database.service.port Service port 5432

Redis

Parameter Description Default
redis.enabled Deploy the Redis StatefulSet true
redis.image.tag Image tag 7-alpine
redis.auth.password Redis password (required unless existingSecret is set) ""
redis.auth.existingSecret Existing Secret containing the password ""
redis.auth.existingSecretPasswordKey Key inside the existing Secret redis-password
redis.persistence.enabled Enable persistent storage true
redis.persistence.size PVC size 5Gi
redis.resources.requests.memory Memory request 10Mi
redis.resources.limits.memory Memory limit 512Mi
redis.service.port Service port 6379

Server (PatchMon 2.0)

In 2.0 the server is a single Go binary that serves /api/* and the embedded React SPA on port 3000. In chart versions that have not yet been updated for 2.0, the equivalent values live under backend.* and frontend.enabled should be set to false.

Parameter Description Default
server.enabled Deploy the PatchMon server true
server.image.registry Image registry ghcr.io
server.image.repository Image repository patchmon/patchmon-server
server.image.tag Image tag (overridden by global.imageTag if set) 2.0.0
server.replicaCount Number of replicas 1
server.jwtSecret JWT signing secret (required unless existingSecret is set) ""
server.aiEncryptionKey Encryption key for secrets at rest ""
server.existingSecret Name of an existing Secret for JWT and encryption key ""
server.existingSecretJwtKey Key for JWT_SECRET inside the existing Secret jwt-secret
server.existingSecretAiEncryptionKey Key for AI_ENCRYPTION_KEY inside the existing Secret ai-encryption-key
server.resources.requests.cpu CPU request 100m
server.resources.requests.memory Memory request 256Mi
server.resources.limits.memory Memory limit 1Gi
server.service.port Service port 3000
server.autoscaling.enabled Enable HPA false

Migrations: The PatchMon server runs migrations automatically at boot via embedded golang-migrate. You do not need a dedicated migration Job; remove it from the chart if one is present.

Server environment variables

The server.env.* keys map directly to the environment variables the PatchMon binary reads. See the Environment Variables Reference for the full list.

Key Description Default
CORS_ORIGIN Allowed origin for CORS (must match the URL users type in their browser; comma-separate with no spaces to allow multiple, e.g. https://patchmon.example.com,https://patchmon.internal.lan) http://localhost:3000
ENABLE_HSTS Enable HSTS header for HTTPS false
TRUST_PROXY Trust proxy headers when behind an Ingress controller true
ENABLE_LOGGING Enable structured logging to stdout true
LOG_LEVEL Log level (debug, info, warn, error) info
JSON_BODY_LIMIT Max JSON body size 5mb
AGENT_UPDATE_BODY_LIMIT Max agent update body size 5mb
TZ IANA timezone for log timestamps UTC

Tip: Many of these can be changed later from the Settings UI without a restart of the whole cluster. See Settings in the web UI.

OIDC / SSO
Key Description Default
OIDC_ENABLED Enable OIDC authentication false
OIDC_ISSUER_URL OIDC issuer URL ""
OIDC_CLIENT_ID OIDC client ID ""
OIDC_CLIENT_SECRET OIDC client secret (put this in an existing Secret) ""
OIDC_REDIRECT_URI Callback URL (https://<host>/api/v1/auth/oidc/callback) ""
OIDC_SCOPES Space-separated scopes openid email profile groups
OIDC_AUTO_CREATE_USERS Auto-provision users on first login false
OIDC_DEFAULT_ROLE Default role for new OIDC users user
OIDC_SYNC_ROLES Sync roles from OIDC group claims on each login false
OIDC_DISABLE_LOCAL_AUTH Disable local username/password authentication false

Full OIDC configuration and group-to-role mapping variables are documented in the Environment Variables Reference.

guacd sidecar (optional, for RDP)

PatchMon 2.0 can proxy Windows RDP through the Apache Guacamole daemon. If you need in-browser RDP, add a guacd sidecar and point GUACD_ADDRESS at it:

server:
  env:
    GUACD_ADDRESS: "guacd:4822"

guacd:
  enabled: true
  image:
    repository: guacamole/guacd
    tag: "1.6.0"

Verify against the latest chart. guacd support was added after the original 1.4.x chart; if your chart version does not include a guacd.enabled option, you can deploy it as a separate Deployment and Service in the same namespace and set GUACD_ADDRESS to its ClusterIP hostname.

Ingress

Parameter Description Default
ingress.enabled Enable Ingress resource true
ingress.className Ingress class name ""
ingress.annotations Ingress annotations {}
ingress.hosts List of Ingress host rules see chart values.yaml
ingress.tls TLS configuration []

Required annotations for WebSocket support (agent WS and live patch streaming):

ingress:
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "86400"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"

For detailed reverse proxy configuration (Nginx, Caddy, Traefik), see Reverse proxy examples.


Persistent Volumes

PVC Component Purpose Default Size
postgres-data Database PostgreSQL data directory 5Gi
redis-data Redis Redis data directory 5Gi

No agent files volume in 2.0. In 1.4.x, agent binaries and SCAP compliance content were stored on a agent-files PVC. In 2.0 both are embedded in the server binary, so no application-specific volume is required. If your chart still declares backend.persistence, it can be removed.


Updating PatchMon

Using global.imageTag

helm upgrade patchmon oci://ghcr.io/ruthlessbeat200/charts/patchmon \
  -n patchmon \
  -f values-prod.yaml \
  --set global.imageTag=2.0.1

When the new pod starts, golang-migrate applies any pending schema migrations automatically. No manual Job is required.

Pinning individual tags

server:
  image:
    tag: "2.0.0"

Upgrading the chart version

helm upgrade patchmon oci://ghcr.io/ruthlessbeat200/charts/patchmon \
  --namespace patchmon \
  --values values-prod.yaml \
  --wait --timeout 10m

Check the chart releases page and the PatchMon releases page before upgrading.

Note: For about the first 24 hours after a release is published, PatchMon may still report an older version as the latest available. That is expected and is part of our phased rollout. See Why the newest version can take a day to appear in the Docker chapter.


Uninstalling

# Uninstall the release
helm uninstall patchmon -n patchmon

# Clean up PVCs (this deletes all data)
kubectl delete pvc -n patchmon -l app.kubernetes.io/instance=patchmon

Advanced Configuration

Custom image registry (air-gapped)

global:
  imageRegistry: "registry.example.com"

This changes every image pull to use the specified registry:

  • registry.example.com/postgres:17-alpine
  • registry.example.com/redis:7-alpine
  • registry.example.com/patchmon/patchmon-server:2.0.0
  • registry.example.com/guacamole/guacd:1.6.0 (when RDP is enabled)

Horizontal Pod Autoscaling

server:
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 10
    targetCPUUtilizationPercentage: 70

Note: Scaling the server beyond a single replica is safe as of 2.0. There are no writeable local file volumes, and background jobs are coordinated through Redis + Asynq. WebSocket connections (agent WS, SSH terminal WS, live patch streams) do need sticky sessions through the Ingress controller if you run multiple replicas; set nginx.ingress.kubernetes.io/affinity: cookie in your Ingress annotations.

Using an external database

Disable the built-in database and set DATABASE_URL to an external PostgreSQL instance:

database:
  enabled: false

server:
  env:
    DATABASE_URL: "postgresql://patchmon:password@external-db.example.com:5432/patchmon"

OIDC / SSO integration

server:
  env:
    OIDC_ENABLED: "true"
    OIDC_ISSUER_URL: "https://auth.example.com/realms/master"
    OIDC_CLIENT_ID: "patchmon"
    OIDC_REDIRECT_URI: "https://patchmon.example.com/api/v1/auth/oidc/callback"
    OIDC_SCOPES: "openid profile email groups"
    OIDC_BUTTON_TEXT: "Login with SSO"
    OIDC_AUTO_CREATE_USERS: "true"
    OIDC_SYNC_ROLES: "true"
    OIDC_ADMIN_GROUP: "patchmon-admins"

The client secret should live in a Kubernetes Secret and be mounted as OIDC_CLIENT_SECRET, not set inline.


Deploying with plain manifests

If you deploy through Argo CD or Flux, or you would rather own every object yourself, you can skip Helm entirely. The manifests below are a complete, working stack: PostgreSQL, Redis, the PatchMon server, a Service, and an Ingress.

They are written for k3s with Longhorn storage and the built-in Traefik Ingress controller, because that is a common self-hosted combination, but they work on any cluster once you change storageClassName and ingressClassName to suit.

Read the volume note before you deploy. The single most common mistake when writing PostgreSQL manifests by hand costs you your database on the first redeploy. It is explained in The PostgreSQL data directory below, and the manifests here already account for it.

1. Create the namespace and secrets

kubectl create namespace patchmon

kubectl create secret generic patchmon-secrets \
  --namespace patchmon \
  --from-literal=JWT_SECRET="$(openssl rand -hex 64)" \
  --from-literal=SESSION_SECRET="$(openssl rand -hex 64)" \
  --from-literal=AI_ENCRYPTION_KEY="$(openssl rand -hex 64)" \
  --from-literal=POSTGRES_PASSWORD="$(openssl rand -hex 32)" \
  --from-literal=REDIS_PASSWORD="$(openssl rand -hex 32)"

If you keep your manifests in Git, do not commit these in plain text. Use Sealed Secrets, SOPS, or the External Secrets Operator instead. The keys above are consumed as environment variables, so keep the names exactly as written.

2. Apply the manifests

Save this as patchmon.yaml, change patchmon.example.com to your own hostname in both the ConfigMap and the Ingress, and apply it.

apiVersion: v1
kind: ConfigMap
metadata:
  name: patchmon-config
  namespace: patchmon
data:
  # Must match exactly the URL your users open PatchMon on.
  CORS_ORIGIN: "https://patchmon.example.com"
  POSTGRES_HOST: "patchmon-db"
  POSTGRES_USER: "patchmon_user"
  POSTGRES_DB: "patchmon_db"
  REDIS_HOST: "patchmon-redis"
  REDIS_PORT: "6379"
  REDIS_DB: "0"
  PORT: "3000"
  # Required when running behind an Ingress controller.
  TRUST_PROXY: "true"
  TZ: "UTC"
---
# ------------------------------------------------------------------ PostgreSQL
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: patchmon-db-data
  namespace: patchmon
  annotations:
    # Keeps Argo CD from ever pruning or deleting the database volume.
    argocd.argoproj.io/sync-options: Prune=false,Delete=false
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: longhorn
  resources:
    requests:
      storage: 8Gi
---
apiVersion: v1
kind: Service
metadata:
  name: patchmon-db
  namespace: patchmon
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: patchmon-db
  ports:
    - name: postgres
      port: 5432
      targetPort: postgres
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: patchmon-db
  namespace: patchmon
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  replicas: 1
  # Never run two PostgreSQL pods against one ReadWriteOnce volume.
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app.kubernetes.io/name: patchmon-db
  template:
    metadata:
      labels:
        app.kubernetes.io/name: patchmon-db
    spec:
      containers:
        - name: postgres
          image: postgres:17-alpine
          ports:
            - name: postgres
              containerPort: 5432
          env:
            - name: POSTGRES_USER
              valueFrom:
                configMapKeyRef:
                  name: patchmon-config
                  key: POSTGRES_USER
            - name: POSTGRES_DB
              valueFrom:
                configMapKeyRef:
                  name: patchmon-config
                  key: POSTGRES_DB
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: patchmon-secrets
                  key: POSTGRES_PASSWORD
            # Do not change these two settings without reading
            # "The PostgreSQL data directory" below.
            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata
          volumeMounts:
            - name: db-data
              mountPath: /var/lib/postgresql/data
          readinessProbe:
            exec:
              command:
                - sh
                - -c
                - pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB" -h localhost
            initialDelaySeconds: 10
            periodSeconds: 10
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              memory: 512Mi
      volumes:
        - name: db-data
          persistentVolumeClaim:
            claimName: patchmon-db-data
---
# ----------------------------------------------------------------------- Redis
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: patchmon-redis-data
  namespace: patchmon
  annotations:
    argocd.argoproj.io/sync-options: Prune=false,Delete=false
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: longhorn
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: patchmon-redis
  namespace: patchmon
spec:
  selector:
    app.kubernetes.io/name: patchmon-redis
  ports:
    - name: redis
      port: 6379
      targetPort: 6379
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: patchmon-redis
  namespace: patchmon
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app.kubernetes.io/name: patchmon-redis
  template:
    metadata:
      labels:
        app.kubernetes.io/name: patchmon-redis
    spec:
      containers:
        - name: redis
          image: redis:7-alpine
          command: ["sh", "-c", "redis-server --requirepass \"$REDIS_PASSWORD\""]
          env:
            - name: REDIS_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: patchmon-secrets
                  key: REDIS_PASSWORD
          ports:
            - name: redis
              containerPort: 6379
          volumeMounts:
            - name: data
              mountPath: /data
          readinessProbe:
            exec:
              command:
                - sh
                - -c
                - redis-cli --no-auth-warning -a "$REDIS_PASSWORD" ping
            periodSeconds: 10
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              memory: 256Mi
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: patchmon-redis-data
---
# ---------------------------------------------------------------------- Server
apiVersion: apps/v1
kind: Deployment
metadata:
  name: patchmon-server
  namespace: patchmon
  annotations:
    argocd.argoproj.io/sync-wave: "1"
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app.kubernetes.io/name: patchmon-server
  template:
    metadata:
      labels:
        app.kubernetes.io/name: patchmon-server
    spec:
      # Stable identity for the agent presence registry. Keep this, and give
      # each server process a distinct value if several share one Redis.
      hostname: patchmon-server
      containers:
        - name: server
          image: ghcr.io/patchmon/patchmon-server:latest
          ports:
            - name: http
              containerPort: 3000
          envFrom:
            - configMapRef:
                name: patchmon-config
            - secretRef:
                name: patchmon-secrets
          env:
            # $(VAR) resolves against the envFrom entries above, so this
            # assembles the URL from the ConfigMap and Secret values.
            - name: DATABASE_URL
              value: "postgresql://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@$(POSTGRES_HOST):5432/$(POSTGRES_DB)"
          startupProbe:
            tcpSocket:
              port: http
            # The first start applies the schema migrations.
            failureThreshold: 40
            periodSeconds: 10
          livenessProbe:
            tcpSocket:
              port: http
            periodSeconds: 30
          readinessProbe:
            tcpSocket:
              port: http
            periodSeconds: 15
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              memory: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: patchmon-server
  namespace: patchmon
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: patchmon-server
  ports:
    - name: http
      port: 3000
      targetPort: http
---
# --------------------------------------------------------------------- Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: patchmon
  namespace: patchmon
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  ingressClassName: traefik
  rules:
    - host: patchmon.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: patchmon-server
                port:
                  name: http
  tls:
    - hosts:
        - patchmon.example.com
      secretName: patchmon-tls
kubectl apply -f patchmon.yaml
kubectl get pods -n patchmon -w

The server pod may restart once or twice on a brand new cluster while it waits for PostgreSQL and cluster DNS to come up. That is expected and it recovers on its own. Watch the migrations apply:

kubectl logs -n patchmon deploy/patchmon-server -f

Then open your hostname and complete the first-time admin setup.

The PostgreSQL data directory

This is the detail worth getting right, and it is the reason a hand-written PostgreSQL manifest can appear to work perfectly and still lose its data on the next deploy.

The postgres image stores its data in PGDATA, which defaults to /var/lib/postgresql/data. The obvious instinct is to mount your volume at the parent, /var/lib/postgresql, and let the data directory sit inside it. That does not work. The image declares VOLUME /var/lib/postgresql/data, and containerd honours image volumes by mounting a scratch directory over that exact path. Your persistent volume ends up underneath it, and PGDATA lands in the scratch directory instead:

/var/lib/postgresql       <- your PersistentVolumeClaim
/var/lib/postgresql/data  <- a per-container scratch directory, mounted on top

That scratch directory is keyed by container ID. It is created empty and destroyed with the container, so every redeploy, and every container restart from a failed probe or an out-of-memory kill, hands PostgreSQL an empty PGDATA. initdb runs again and you get a brand new empty database, while your persistent volume holds nothing but an empty data directory.

Mounting the volume at /var/lib/postgresql/data fixes that, but introduces a second problem on block storage such as Longhorn or Ceph. A freshly formatted ext4 volume already contains a lost+found directory, and initdb refuses to use a directory that is not empty:

initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
initdb: detail: It contains a lost+found directory, perhaps due to it being a mount point.
initdb: hint: Using a mount point directly as the data directory is not recommended.

So mount the volume at /var/lib/postgresql/data and point PGDATA at a subdirectory of it, exactly as the manifests above do:

env:
  - name: PGDATA
    value: /var/lib/postgresql/data/pgdata
volumeMounts:
  - name: db-data
    mountPath: /var/lib/postgresql/data

Redis needs no equivalent treatment. Its image declares VOLUME /data and the manifest mounts at /data exactly, so the explicit mount takes precedence.

To confirm your own deployment is correct, check that only one filesystem is mounted under the data directory:

kubectl exec -n patchmon deploy/patchmon-db -- grep postgresql /proc/mounts

One line means the volume is mounted correctly. Two lines means a scratch directory is sitting on top of your data, and the database will not survive a redeploy.

Notes for Argo CD

  • Sync waves. The database and Redis carry argocd.argoproj.io/sync-wave: "0" and the server carries "1", so the data services settle before the server starts. Without this the server crash-loops a few times on the first sync before recovering.
  • Protect the volumes. Both PVCs carry argocd.argoproj.io/sync-options: Prune=false,Delete=false, so removing the manifest from Git, or deleting the Argo CD Application, does not take the database with it.
  • Never sync a PVC with Replace. Replace=true, and the "Replace" option on a manual sync, delete and recreate the object. On a StorageClass with the default Delete reclaim policy, that destroys the volume and its data.

Optional: in-browser RDP

The server reaches Windows hosts over RDP through a guacd sidecar. It is only needed for that feature. To enable it, add the Deployment and Service below, then set GUACD_ADDRESS: "patchmon-guacd:4822" in the ConfigMap.

apiVersion: v1
kind: Service
metadata:
  name: patchmon-guacd
  namespace: patchmon
spec:
  selector:
    app.kubernetes.io/name: patchmon-guacd
  ports:
    - name: guacd
      port: 4822
      targetPort: 4822
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: patchmon-guacd
  namespace: patchmon
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: patchmon-guacd
  template:
    metadata:
      labels:
        app.kubernetes.io/name: patchmon-guacd
    spec:
      containers:
        - name: guacd
          image: guacamole/guacd:1.6.0
          ports:
            - name: guacd
              containerPort: 4822
          volumeMounts:
            - name: tmp
              mountPath: /tmp
          readinessProbe:
            tcpSocket:
              port: 4822
            initialDelaySeconds: 10
            periodSeconds: 10
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              memory: 512Mi
          securityContext:
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
      volumes:
        - name: tmp
          emptyDir:
            medium: Memory
            sizeLimit: 64Mi

Ingress controllers other than Traefik

Traefik proxies WebSockets without extra configuration, so the Ingress above needs no annotations. On NGINX Ingress you must raise the timeouts, or agent connections, SSH terminals, and live patch streams will drop about every 30 seconds:

metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "86400"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"

See Reverse proxy examples for the full list of WebSocket endpoints to verify.


Troubleshooting

Check pod status

kubectl get pods -n patchmon
kubectl describe pod <pod-name> -n patchmon
kubectl logs <pod-name> -n patchmon

Check init container logs (waiting for DB or Redis)

kubectl logs <pod-name> -n patchmon -c wait-for-database
kubectl logs <pod-name> -n patchmon -c wait-for-redis

Check migration logs

Migrations run inside the server pod on startup. Follow the startup logs:

kubectl logs -n patchmon deploy/patchmon-server --since=5m | grep migrate

You should see lines like [migrate] running migrations from embedded binary and either [migrate] applied successfully (version N) or [migrate] already up to date.

Health check

The server exposes a liveness probe at /health:

kubectl exec -n patchmon -it deploy/patchmon-server -- wget -qO- http://localhost:3000/health

Response is healthy (plain text) or a JSON structure when the Accept: application/json header is set.

Common issues

Symptom Likely cause Fix
Pods stuck in Init state Database or Redis not yet running kubectl describe sts -n patchmon
PVC stuck in Pending No matching StorageClass Run kubectl get sc and set global.storageClass
ImagePullBackOff Registry credentials missing Check imagePullSecrets and image path
Ingress returns 404 / 502 Ingress misconfigured or points at wrong port All traffic should go to server:3000
WebSocket connections drop every ~30s Ingress default read timeout too short Set proxy-read-timeout: "86400"
secret ... not found Required Secret not created before install Create the Secret or set secret.create: true
Database is empty again after every redeploy, and the setup wizard reappears Hand-written manifests mounting the volume at /var/lib/postgresql instead of at PGDATA See The PostgreSQL data directory
initdb: error: directory ... exists but is not empty Volume mounted straight at PGDATA on block storage, which contains lost+found Set PGDATA to a subdirectory of the mount. See The PostgreSQL data directory
CORS errors in browser CORS_ORIGIN doesn't match the URL users see Set it to the exact URL from the Ingress host. If the chart exposes more than one Ingress host, comma-separate them with no spaces, e.g. https://patchmon.example.com,https://patchmon.internal.lan

Support


See also