Kubesense
Integrations/Uptime Monitoring

Uptime Monitoring

Overview

KubeSense can monitor the availability of internal and external endpoints from the Kubernetes cluster where the Sensor is installed. Uptime monitoring uses the Prometheus Blackbox Exporter through the OpenTelemetry (OTEL) Agent and forwards probe metrics to KubeSense.

Supported probe types are:

  • HTTP/HTTPS - Check web endpoints
  • TCP - Check TCP service reachability
  • ICMP - Check whether an IP address responds to ping
  • gRPC - Check gRPC endpoints

Each probe produces metrics such as probe_success and probe_duration_seconds. Targets can be grouped so one label set is applied to many URLs.

Prerequisites

Before configuring uptime monitoring, ensure that:

  1. KubeSense Sensor is installed in the target Kubernetes cluster.
  2. The OTEL Agent and uptime-monitor components are enabled.
  3. The Sensor namespace can resolve and reach every target.
  4. You can update the Sensor Helm values file.

Enable Uptime Monitoring

Add this configuration to the Sensor values file:

uptime-monitor:
  enabled: true

otel-agent:
  enabled: true
  uptime-monitor:
    enabled: true
    httpProbe:
      enabled: true
      targetGroups:
        - targets:
            - "https://api.example.com/health"
            - "https://web.example.com/health"
          labels:
            service: public-services
            env: production
            platform: external

The top-level uptime-monitor section configures the Blackbox Exporter. The otel-agent.uptime-monitor section configures the OTEL Agent to scrape it and forward probe metrics.

Apply the configuration:

helm upgrade -i kubesensor kubesense/kubesensor \
  --namespace kubesense \
  --create-namespace \
  -f values.yaml

Configure Targets and Labels

HTTP and HTTPS

Use targetGroups when several URLs share labels:

otel-agent:
  uptime-monitor:
    httpProbe:
      enabled: true
      targetGroups:
        - targets:
            - "https://service-01.example.com/health"
            - "https://service-02.example.com/health"
            - "https://service-03.example.com/health"
          labels:
            service: customer-services
            env: production
            platform: external
        - targets:
            - "https://service-04.example.com/health"
            - "https://service-05.example.com/health"
          labels:
            service: customer-services
            env: staging
            platform: external

The original string format remains supported:

otel-agent:
  uptime-monitor:
    httpProbe:
      enabled: true
      targets:
        - "https://example.com"
        - "https://api.example.com/health"

For different labels on individual targets, use:

targets:
  - target: "https://api.example.com/health"
    labels:
      service: api
      env: production
      platform: external

targets and targetGroups may be used together. targetGroups is recommended for large target lists.

TCP

otel-agent:
  uptime-monitor:
    tcpProbe:
      enabled: true
      targetGroups:
        - targets:
            - "database.example.com:5432"
            - "cache.example.com:6379"
          labels:
            service: data-services
            env: production
            platform: external

ICMP

otel-agent:
  uptime-monitor:
    icmpProbe:
      enabled: true
      targetGroups:
        - targets:
            - "8.8.8.8"
            - "1.1.1.1"
          labels:
            service: dns
            env: external
            platform: external

ICMP probing may require cluster and pod security settings that permit ICMP packets.

gRPC

otel-agent:
  uptime-monitor:
    grpcProbe:
      enabled: true
      targetGroups:
        - targets:
            - "grpc-01.example.com:443"
            - "grpc-02.example.com:443"
          labels:
            service: grpc-services
            env: production
            platform: external

Authenticated HTTP Probes

Authenticated targets are configured in two parts:

  1. Define a Blackbox Exporter module under uptime-monitor.config.modules.
  2. Reference that module from an otel-agent.uptime-monitor.httpProbeAuth group.

Targets in the same group use the same module and credentials. Create separate groups when credentials differ.

Bearer Token

uptime-monitor:
  enabled: true
  secretConfig: true
  config:
    modules:
      http_2xx_bearer_services:
        prober: http
        timeout: 5s
        http:
          valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
          follow_redirects: true
          preferred_ip_protocol: "ip4"
          headers:
            Authorization: "Bearer <bearer-token>"

otel-agent:
  uptime-monitor:
    enabled: true
    httpProbeAuth:
      - module: http_2xx_bearer_services
        targets:
          - "https://private-api-01.example.com/health"
          - "https://private-api-02.example.com/health"
        labels:
          auth_type: bearer
          service: private-api
          env: production
          platform: external

OAuth 2.0 Client Credentials

uptime-monitor:
  enabled: true
  secretConfig: true
  config:
    modules:
      http_2xx_oauth_services:
        prober: http
        timeout: 5s
        http:
          valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
          follow_redirects: true
          preferred_ip_protocol: "ip4"
          oauth2:
            client_id: "<client-id>"
            client_secret: "<client-secret>"
            token_url: "https://<tenant>.auth0.com/oauth/token"
            endpoint_params:
              audience: "<api-audience>"

otel-agent:
  uptime-monitor:
    enabled: true
    httpProbeAuth:
      - module: http_2xx_oauth_services
        targets:
          - "https://oauth-api-01.example.com/health"
          - "https://oauth-api-02.example.com/health"
        labels:
          auth_type: oauth2
          service: oauth-api
          env: production
          platform: external

Set secretConfig: true so the Blackbox Exporter configuration is stored in a Kubernetes Secret. Do not commit real tokens or client secrets to source control.

For ten bearer URLs and ten OAuth URLs, use one group for each authentication configuration when credentials are shared. If every URL has different credentials, define a module and group for each credential set.

Metrics and Labels

Labels in a target group are applied to every target in that group. The OTEL Agent also sets instance to the target URL:

labels:
  service: payments
  env: production
  platform: external

Use stable, low-cardinality labels such as service, environment, region, and platform. Do not use sensitive data or rapidly changing values as labels.

Scaling Guidance

The configuration supports 100 or more targets. Each target is probed once per scrape interval, so resource usage depends on the number of targets, probe timeout, and scrape interval.

  • Group targets that share labels and authentication settings.
  • Keep different bearer tokens and OAuth client credentials in separate groups.
  • Start with a 60-second scrape interval for approximately 100 targets and tune it for the required detection time.
  • Keep probe timeouts shorter than the scrape interval.
  • Provide sufficient CPU, memory, and network capacity to the OTEL Agent and Blackbox Exporter.
  • Avoid duplicate targets across authenticated and unauthenticated groups unless both probes are intentional.

Troubleshooting

Targets are not appearing

  1. Confirm uptime-monitor.enabled and otel-agent.uptime-monitor.enabled are true.

  2. Confirm the relevant probe type is enabled.

  3. Inspect the rendered configuration:

    helm template kubesensor kubesense/kubesense -f values.yaml

  4. Check the OTEL Agent and uptime-monitor pod logs:

    kubectl logs -n kubesense deploy/kubesensor-otel-agent
kubectl logs -n kubesense deploy/kubesensor-uptime-monitor

Probes are failing

  • Verify DNS resolution, network policies, target ports, and protocols.
  • For HTTP, check redirects, TLS certificates, and authentication.
  • For bearer authentication, check the token and Authorization header.
  • For OAuth2, check the client credentials, token URL, audience, and endpoint parameters.
  • For ICMP, verify that pod and cluster security settings permit ICMP probing.

Labels are missing

  • Put labels beside targets inside the same targetGroups entry.
  • For authenticated probes, put labels beside module and targets in the httpProbeAuth entry.
  • Check that label names and values are valid Prometheus labels.

Summary

Use uptime monitoring to collect endpoint availability metrics from the Sensor cluster. Use targetGroups to manage large target lists, define labels once for related URLs, and isolate authenticated targets by Blackbox Exporter module and credential set.

back

ArgoCD Metrics

next

External Service Logs & Metrics