Instrumentation
Send OpenTelemetry traces from your applications to KubeSense.
KubeSense captures traces automatically with eBPF. To get application-level spans, add OpenTelemetry instrumentation to your services. These include framework handlers, database client calls, and outgoing HTTP calls.
KubeSense accepts standard OpenTelemetry (OTLP) traces, so you can use the official OpenTelemetry agents and SDKs. Most languages also support zero-code instrumentation, which needs no changes to application code.
How traces reach KubeSense
Your application (OpenTelemetry agent / SDK)
│ OTLP/HTTP
▼
(optional) OpenTelemetry Collector ── adds Kubernetes metadata, batching, retries
│ OTLP/HTTP
▼
KubeSense trace ingest (kubesense-kubecol, port 33443)
│
▼
APM → Services, Traces, Service MapIngest endpoints
| Signal | Protocol | Inside the KubeSense cluster | From outside the cluster |
|---|---|---|---|
| Traces | OTLP/HTTP | http://kubesense-kubecol.kubesense.svc.cluster.local:33443 | http://<KUBESENSE_INGEST_HOST>:33443 |
| Metrics (optional) | Prometheus remote write | http://kubesense-metrics-scraper.kubesense.svc.cluster.local:30060/api/v1/write | http://<KUBESENSE_INGEST_HOST>:30060/api/v1/write |
<KUBESENSE_INGEST_HOST> is the internal load balancer or hostname that exposes KubeSense ingestion ports to your network. See Networking.
Logs: Application logs are collected by the KubeSense log sensor, so the instrumentation in this section exports traces only. Set OTEL_LOGS_EXPORTER=none so the OpenTelemetry SDKs don't also export logs.
Port 33443 accepts OTLP over HTTP only: Use http/protobuf as the exporter protocol. gRPC exporters sending to port 33443 fail. Set the endpoint as a base address without a path, for example http://kubesense-kubecol.kubesense.svc.cluster.local:33443. SDKs and the collector's otlphttp exporter add /v1/traces themselves.
Prerequisite: enable trace ingestion on KubeSense
The trace ingest listener on port 33443 is disabled by default. Enable it in your KubeSense server values and upgrade the release:
kubecol:
external_agent_traces:
enabled: truehelm upgrade kubesense kubesense/kubesense -n kubesense -f values.yamlIf applications outside the KubeSense cluster send traces (Linux or Windows VMs, or other Kubernetes clusters), expose port 33443 through an internal load balancer and allow inbound traffic from the application subnets.
Required resource attributes
Set these resource attributes on every service, either in the application (OTEL_RESOURCE_ATTRIBUTES) or in an OpenTelemetry Collector:
| Attribute | Value | Purpose |
|---|---|---|
service.name | Your service name (set with OTEL_SERVICE_NAME) | Service name shown in APM |
kubesense.cluster | Name of the cluster or environment, as shown in KubeSense | Groups the service under the right cluster in the cluster selector |
kubesense.env_type | k8s for workloads on Kubernetes; legacy for applications on VMs (Linux or Windows) | Tells KubeSense how to correlate the service with infrastructure data |
Example:
OTEL_SERVICE_NAME=checkout-api
OTEL_RESOURCE_ATTRIBUTES=kubesense.cluster=prod-eks,kubesense.env_type=k8s,deployment.environment.name=prodChoose an approach
| Where the application runs | Recommended approach |
|---|---|
| Kubernetes | OpenTelemetry Operator: zero-code injection with one pod annotation |
| VMs (Linux or Windows) | Language agent configured with environment variables (see the language pages), sending directly to KubeSense or through a local OpenTelemetry Collector |
| Full control in code | Manual instrumentation: install the OpenTelemetry SDK packages and add the trace setup code (see each language page) |
Languages
| Language | Kubernetes (auto) | VM (auto) | Manual SDK |
|---|---|---|---|
| Node.js and NestJS | ✅ Operator | ✅ | ✅ |
| Java | ✅ Operator | ✅ | ✅ |
| .NET | ✅ Operator | ✅ Linux VM, Windows / IIS | ✅ |
| Go | ✅ otel go build, or Operator (eBPF) | ✅ otel go build | ✅ |
| Python (incl. Celery) | ✅ Operator | ✅ | ✅ |
| Ruby and Ruby on Rails | — | — | ✅ (setup is manual) |
| Next.js (server and browser) | ✅ Operator | ✅ | ✅ instrumentation.ts |
Auto means no application code changes. Manual SDK means installing the OpenTelemetry SDK packages and adding the trace setup code to the application. Each language page has a Manual instrumentation (SDK) section.
Manual SDK: where to send traces
With manual SDK instrumentation, the application exports traces itself. Configure it with the same environment variables as auto-instrumentation. Use base URLs without a path; the SDKs add /v1/traces themselves.
Option A: through an OpenTelemetry Collector (recommended)
OTEL_SERVICE_NAME=orders-api
OTEL_EXPORTER_OTLP_ENDPOINT=http://<COLLECTOR_HOST>:4318
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_RESOURCE_ATTRIBUTES=kubesense.cluster=<CLUSTER_NAME>,kubesense.env_type=k8s
OTEL_TRACES_EXPORTER=otlp
OTEL_METRICS_EXPORTER=none
OTEL_LOGS_EXPORTER=noneThe collector forwards traces to KubeSense:
exporters:
otlphttp/kubesense:
endpoint: http://kubesense-kubecol.kubesense.svc.cluster.local:33443
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp/kubesense]On Kubernetes, use the collector from Kubernetes (OpenTelemetry Operator). Its address is http://otel-collector-opentelemetry-collector.kubesense.svc.cluster.local:4318.
Option B: directly to KubeSense (no collector)
OTEL_SERVICE_NAME=orders-api
OTEL_EXPORTER_OTLP_ENDPOINT=http://<KUBESENSE_INGEST_HOST>:33443
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_RESOURCE_ATTRIBUTES=kubesense.cluster=<CLUSTER_NAME>,kubesense.env_type=legacy
OTEL_TRACES_EXPORTER=otlp
OTEL_METRICS_EXPORTER=none
OTEL_LOGS_EXPORTER=noneInside the KubeSense cluster, use http://kubesense-kubecol.kubesense.svc.cluster.local:33443 and kubesense.env_type=k8s.
Troubleshooting
| Symptom | Likely cause |
|---|---|
Connection refused on port 33443 | Trace ingestion is not enabled (kubecol.external_agent_traces.enabled), or the port is not exposed to the application's network |
Exporter errors mentioning gRPC, or unexpected EOF | The exporter uses gRPC. Set OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf |
Traces sent to kubesense-otel-agent:4317 never arrive | kubesense-otel-agent is not a trace receiver. Send traces to kubesense-kubecol:33443, or to your own OpenTelemetry Collector |
| Service appears under the wrong cluster, or not at all | kubesense.cluster or kubesense.env_type is missing or misspelled |
| Exporter errors for metrics or logs | The SDK tries to export metrics and logs to the trace endpoint. Set OTEL_METRICS_EXPORTER=none and OTEL_LOGS_EXPORTER=none |