MySQL

Integrating MySQL Metrics and Logs to KubeSense

KubeSense provides native monitoring for MySQL through the OpenTelemetry (OTEL) Collector, allowing you to capture key MySQL performance metrics and logs and visualize them in the KubeSense platform.

Prerequisites

Before you begin, ensure you have:

1. A MySQL server running version 5.7 or newer
   • You can check the server version with the SQL statement: SELECT VERSION();
2. A MySQL user with permissions required for metrics collection

   CREATE USER 'kubesense'@'%' IDENTIFIED BY '<PASSWORD>';
   GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'kubesense'@'%';
   FLUSH PRIVILEGES;

3. KubeSense deployed with an OTEL Collector that can reach the MySQL server
   • Install the OTEL Collector if not done already
   • Ensure the OTEL collector can connect to the MySQL endpoint using the monitoring user
   • For log collection, the OTEL collector must be able to read the MySQL log files
4. Access to modify the KubeSense Helm values file

Collecting MySQL Metrics

Step 1: Update KubeSense Helm Values File

Add the MySQL receiver configuration to your KubeSense Helm values file, within the kubesensor section:

otel-collector:
  config:
    receivers:
      mysql:
        # The hostname (and optional port) of the MySQL server.
        endpoint: <MYSQL_ENDPOINT>:3306
        # The username used to access the MySQL instance.
        username: kubesense
        # The password used to access the MySQL instance.
        password: <password>
        # Frequency for collecting metrics.
        collection_interval: 10s
        # Delay applied before the receiver starts scraping.
        initial_delay: 10s
        # tls:
        #   insecure: false
        #   ca_file: /etc/ssl/certs/ca-certificates.crt
        #   cert_file: /etc/ssl/certs/mysql.crt
        #   key_file: /etc/ssl/certs/mysql.key
    appendToPipelines:
      metrics:
        receivers:
        - mysql

Step 2: Apply the Configuration

Upgrade your KubeSense deployment with the updated values file:

helm upgrade kubesense kubesense/kubesense \
  -f values.yaml \
  --namespace kubesense

Collecting MySQL Logs

Step 1: Update KubeSense Helm Values File for Log Collection

Add the MySQL log collection configuration to your KubeSense Helm values file:

otel-collector:
  config:
    receivers:
      filelog/mysql:
        include: ["/var/log/mysql/*.log"]
        start_at: beginning
        operators:
          - type: regex_parser
            # Adjust the pattern to match your MySQL log format.
            regex: '^(?P<ts>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?Z?)\s+(?P<level>[A-Z]+)\s+(?P<message>.*)$'
            timestamp:
              parse_from: attributes.ts
              layout: '%Y-%m-%dT%H:%M:%S'
            severity:
              parse_from: attributes.level
            on_error: send
          - type: move
            if: attributes.message != nil
            from: attributes.message
            to: body
          - type: remove
            if: attributes.level != nil
            field: attributes.level
          - type: add
            field: attributes.source
            value: mysql
    processors:
      batch/mysql:
        send_batch_size: 10000
        send_batch_max_size: 11000
        timeout: 10s
    service:
      pipelines:
        logs/mysql:
          receivers:
          - filelog/mysql
          processors:
          - batch
          - resource
          exporters:
          - otlphttp/kubesense_logs

Step 2: Apply the Configuration

Upgrade your KubeSense deployment with the updated values file:

helm upgrade kubesense kubesense/kubesense \
  -f values.yaml \
  --namespace kubesense

MySQL Metrics Collected

KubeSense automatically collects and enriches the following key MySQL metrics:

MySQL Log Attributes

When collecting MySQL logs, the following attributes are available:

Visualization in KubeSense

Once configured, MySQL metrics and logs appear within the KubeSense dashboards, enabling you to:

• Monitor MySQL workload and query throughput in real time
• Track connection usage and thread activity
• Analyze buffer pool utilization and data transfer volume
• Review MySQL logs alongside other platform telemetry
• Configure alerts for critical MySQL performance indicators

Configuration Parameters

Here's a breakdown of the MySQL receiver configuration:

Troubleshooting

Issue: Cannot connect to MySQL server

Solution: Verify the endpoint is reachable from the OTEL collector pod.

Issue: Authentication failed

Solution: Confirm the monitoring user credentials and privileges:

SHOW GRANTS FOR 'kubesense'@'%';

Issue: Logs not being collected

Solution: Ensure the log path exists and is accessible:

kubectl exec -n kubesense <otel-collector-pod> -- ls -la /var/log/mysql/

Issue: Metrics not appearing in dashboard

Solution: Check the OTEL collector logs for errors:

kubectl logs -n kubesense <kubesense-otel-collector-pod> -f

Best Practices

• Security: Use a dedicated read-only monitoring user with scoped privileges
• Network: Ensure OTEL collector pods can reach the MySQL host over the required port
• Logging: Enable general and slow query logs for deeper visibility into query behavior
• Performance: Tune collection_interval and initial_delay to balance load and timeliness
• Lifecycle: Rotate MySQL logs regularly to avoid disk pressure on the collector nodes

Conclusion

With KubeSense and the OTEL collector, you can observe MySQL performance and logs alongside the rest of your infrastructure, enabling faster troubleshooting and a unified view of your database health.