Configuration

Synthetic Monitoring works with the defaults from Deployment. This page lists the settings you can change: the Helm values of the KubeSense chart, extra synthetics-api settings, and the settings of agents you run in your own locations.

Settings that users change in the web app (monitor defaults, variables, tag rules) are described in Settings.

Helm Values

All values sit under the top-level synthetics: block of the kubesense (or kubesense-server) chart. Apply changes with helm upgrade and the same values files.

General

synthetics-api

Database passwords come from the release's existing secrets; you do not set them here.

In-cluster location and agents

The in-cluster agents have no access to the Kubernetes API. To run Kubernetes resource checks, install a separate agent with the synthetics-agent chart and k8sChecks=true (see Agents in your own locations).

Browser agents

Agent ingress

Agents outside the cluster reach synthetics-api through this Ingress. It routes only the agent endpoints (/api/v1/probes/* and /healthz); users always go through the KubeSense API.

synthetics:
  agentIngress:
    enabled: true
    host: synthetics-agents.example.com
    tls:
      enabled: true
      secretName: synthetics-agents-tls

synthetics-api Settings

Settings the chart does not expose go in synthetics.api.configFile (Kubernetes) or in synthetics-api.yaml (VM). Keys are snake_case; durations use s, m and h (720h is 30 days). An unknown key or a value of the wrong type stops synthetics-api at startup, and its log says which key.

synthetics:
  api:
    configFile:
      min_interval: 30s
      incident_retention: 4320h   # 180 days
      alert_event_retention: 720h  # 30 days

# common/config/synthetics/synthetics-api.yaml
listen_port: 8080
# ... mysql, clickhouse and remote_write as in Deployment
min_interval: 30s
incident_retention: 4320h   # 180 days
alert_event_retention: 720h  # 30 days

Run history in ClickHouse is kept for 30 days, uptime for 365 days, failure snapshots for 7 days and browser screenshots for 14 days. Change them under Settings → Data Retention (group Synthetics). The retention settings above (outages, alert events, CI batches, monitor versions) are synthetics-api's own clean-up of its MySQL database, once a day; they are set here, not on that page.

Agents in Your Own Locations

Agents in another cluster or on a VM are installed with the command a location shows (see Locations & Agents). The command sets the API address, location and token; add the settings below to it as needed.

Handover

When an agent goes offline, its monitors move to the other agents of its location (see Availability). An agent that stops is offline at once, and its monitors move within one check sync, about 30 seconds. An agent that dies is noticed only by its silence: its monitors move within probe_offline_after + nodata_sweep + the agents' check sync, about 30–90 seconds with the defaults. Lower settings shorten the gap at the cost of more requests to synthetics-api:

For example, probe_offline_after: 15s, nodata_sweep: 5s and agents at a 10s check sync bring the handover after a crash to about 15–30 seconds.

synthetics:
  handover:
    offlineAfter: 15s
    sweep: 5s
    sync: 10s   # in-cluster agents; set agent.syncInterval on agents of other locations

For no gap at all, turn on Run every monitor on every agent for the location.

Secrets Key

SYNTHETICS_SECRET_KEYS encrypts secure variables and monitor credentials at rest. Its format is kid:base64key[,kid:base64key] with 32-byte keys (openssl rand -base64 32). The first key encrypts; every key in the list decrypts.

To rotate the key:

1. Put a new key first and keep the old one after it, e.g. k2:<new>,k1:<old>, and restart synthetics-api. New secrets are encrypted with k2; old ones still decrypt with k1.

2. Re-encrypt the stored secrets once. Run the synthetics-api binary with -rotate-secrets and the same database settings and key list as the running service:

   KubernetesVM

   # The pod already has the passwords and keys in its environment;
   # copy the --mysql-* arguments from: kubectl -n kubesense get deploy kubesense-synthetics-api -o yaml
   kubectl -n kubesense exec deploy/kubesense-synthetics-api -- \
     /usr/local/bin/synthetics-api --mysql-host=<host> --mysql-username=<user> -rotate-secrets

   docker compose run --rm synthetics-api -config /etc/synthetics/synthetics-api.yaml -rotate-secrets

   It logs how many secrets it re-encrypted and exits with code 1 if one could not be decrypted.

3. Remove the old key from the list and restart synthetics-api.

Losing every key in use makes the stored secrets unreadable: re-enter them in the monitors and variables that use them.