Installation

This guide connects a Python or JavaScript LLM application to KubeSense using the Langfuse SDK or the OpenLIT SDK.

Both SDKs are OpenTelemetry-based. They create spans around your application and export them as OTLP/HTTP to the KubeSense kubecol LLM receiver, which maps them into the kube_llm.llm_span table. KubeSense is the ingestion target; it does not replace the SDK in your application.

Before you start

Configure the LLM application in the KubeSense dashboard before installing an SDK or sending telemetry. The setup order is:

1. Confirm that kubecol is running and that the LLM collector is reachable from the application.
2. Create an LLM application in Settings > LLM > Applications.
3. Copy the application's ID and use it as the application.id resource attribute.
4. Optionally add an LLM connection for KubeSense-managed provider features such as the playground or evaluations. You can complete this step later.
5. Optionally review or add model pricing patterns when you need custom pricing for models that are not already in the registry. You can complete this step later.
6. Make sure the instrumented application can reach the collector port. The default LLM collector port is 31443.

The application is the container for your LLM telemetry. The application ID associates incoming spans with the correct KubeSense project. Spans without it are accepted, but are assigned to the default application and are harder to find in the UI.

Step 1: Create an LLM application

In the KubeSense dashboard, open Settings, select the LLM tab, and choose Applications. Click New Application.

Enter an application name, an optional description, and the domain associated with the workload. The domain helps organize applications when one KubeSense installation monitors multiple teams or environments. Click Create.

After the application is created, copy its ID from the application list. You will use this value in OTEL_RESOURCE_ATTRIBUTES for every instrumented service that belongs to this application.

Step 2 (Optional): Add an LLM connection

An LLM connection stores the provider adapter, provider name, model selection, and API key used by KubeSense-managed LLM features. It is separate from the SDK connection that sends application traces to kubecol.

From Settings > LLM, select LLM Connections. Choose the application from the Project selector and click Add LLM Connection.

Choose the LLM adapter, enter the provider name, select the model, and enter the provider API key. Enable Set as default connection when this provider should be used by default for the selected application. Use Advanced Settings to add extra provider headers. Enable Default Models makes the adapter's default model list available to KubeSense features.

Keep provider API keys in the KubeSense connection configuration or an approved secret-management workflow. Do not put provider keys in application source code, public documentation, or client-side bundles.

Creating the connection is required only for KubeSense features that call a model on your behalf, such as the playground, LLM-as-a-Judge evaluations, and dataset experiments. It is not required for trace ingestion when Langfuse or OpenLIT exports directly from your application to kubecol. You can skip this step during initial instrumentation and add the connection later from Settings > LLM > LLM Connections.

Step 3 (Optional): Configure LLM models and pricing

The model registry maps an incoming model name to prices. kubecol uses it to fill in cost when a span carries token or media usage but no cost fields of its own — costs the SDK reports always win.

KubeSense ships pricing for thousands of models, so most workloads need nothing here. Add an entry when a model shows zero cost in the dashboard, or when your negotiated rate differs from list price.

Open Settings > LLM > Models.

Each row shows the model name, its match pattern, and the unit that prices are quoted in — TOKENS for text, IMAGES for image generation. Use Search models to check whether a model is already covered before adding one.

Add a model

Click Add Model.

Model Name is the display label in this list. Match Pattern is what actually does the work: a regular expression tested against the model name on each incoming span.

For an exact, case-insensitive match to one model name, use:

(?i)^gpt-4o-mini$

Escape the regex metacharacters that appear in real model names — . and / especially. The seeded entries show the convention:

(?i)^ai21\.j2-mid-v1$
(?i)^aiml\/flux-pro\/v1\.1$

Matching is always case-insensitive: kubecol prepends (?i) when it compiles the pattern, whether or not you included it.

Set prices

Prices are entered per single unit — per token, or per image — not per thousand or per million.

Add one row per usage type. Price Overview converts each entry to per-unit, per-1K, and per-1M as you type; use it to check your figure against the provider's pricing page, which usually quotes per million tokens.

For a model listed at $4.50 per 1M input tokens and $5.60 per 1M output tokens:

The usage types the collector recognizes:

A usage type you do not price contributes zero. Price at least input and output for a text model, and add cache_read and reasoning when your provider bills for them separately, or that spend will be invisible.

total is a fallback for models billed at one flat rate. It is applied only when neither input nor output produced any cost, so setting it alongside them has no effect.

The usage type is a free-text field, and only the names above are recognized. A typo such as inputs is saved without complaint and prices nothing — check the row names before saving.

Tokenizer records which tokenizer the model uses. It does not affect the cost arithmetic, which is driven entirely by the token counts already on the span.

Click Create.

Update prices for an existing model

Use the row menu on the Models list to edit, clone, or delete an entry. Cloning is the quickest route to a negotiated rate: copy the built-in entry, change the prices, and keep the pattern.

The registry is loaded into the collector and refreshed periodically, so a new or edited model takes effect on the next refresh rather than instantly.

How a model is matched

Every entry is tested in order and the first pattern that matches wins. Entries are ordered by model name length, longest first, so a more specific name is tried before a shorter one that might also match.

That ordering is the only tiebreak. Two patterns that can both match the same model name are ambiguous — anchor your patterns with ^ and $ so they match one model and nothing else.

To check what an incoming model is actually called before writing a pattern, open a trace in LLM Monitoring > Tracing and read the request and response model on the generation.

Custom pricing tiers

+ Add Custom Tier stores an additional named tier with its own prices and conditions, alongside the default Standard Pricing tier.

How this relates to the other settings

The three LLM settings are independent. The application scopes telemetry, the connection provides credentials for calls KubeSense makes on your behalf, and the model registry maps usage to pricing. Trace ingestion works with none of them configured beyond the application.

When to complete the optional steps

You can begin sending traces after completing Step 1 and configuring application.id. Add Step 2 later when you want to use a KubeSense-managed provider connection. Add Step 3 later when a model shows zero cost in the dashboard or when your negotiated rate differs from list price.

Choose your integration

After the required application setup, choose the tab for your application stack. All paths send OTLP/HTTP telemetry to the same KubeSense kubecol receiver.

pip install langfuse

export LANGFUSE_BASE_URL="http://<kubecol-host>:31443"
export LANGFUSE_PUBLIC_KEY="ks-public"
export LANGFUSE_SECRET_KEY="ks-secret"
export LANGFUSE_TRACING_ENVIRONMENT="prod"

export OTEL_SERVICE_NAME="my-llm-app"
export OTEL_RESOURCE_ATTRIBUTES="application.id=<application-id>,service.version=1.0.0"

from langfuse import get_client, propagate_attributes

langfuse = get_client()

def answer_question(user_id: str, session_id: str, question: str):
	with langfuse.start_as_current_observation(
		as_type="span",
		name="support-chat",
		input={"question": question},
	) as root:
		with propagate_attributes(user_id=user_id, session_id=session_id):
			with langfuse.start_as_current_observation(
				as_type="generation",
				name="answer",
				model="gpt-4o-mini",
				input=[{"role": "user", "content": question}],
			) as generation:
				answer = call_your_model(question)
				generation.update(
					output=answer,
					usage_details={"input": 128, "output": 64},
				)
		root.update(output=answer)
		return answer

# Required for short-lived processes, jobs, and serverless handlers.
langfuse.flush()

from langfuse import observe

@observe(as_type="generation", name="answer")
def answer(question: str) -> str:
	...

npm install @langfuse/tracing @langfuse/otel @opentelemetry/sdk-node

export LANGFUSE_BASE_URL="http://<kubecol-host>:31443"
export LANGFUSE_PUBLIC_KEY="ks-public"
export LANGFUSE_SECRET_KEY="ks-secret"
export LANGFUSE_TRACING_ENVIRONMENT="prod"

export OTEL_SERVICE_NAME="my-llm-app"
export OTEL_RESOURCE_ATTRIBUTES="application.id=<application-id>,service.version=1.0.0"

import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";

export const sdk = new NodeSDK({
	spanProcessors: [new LangfuseSpanProcessor()],
});

sdk.start();

import { propagateAttributes, startActiveObservation } from "@langfuse/tracing";

await startActiveObservation("support-chat", async (span) => {
	span.update({ input: { question } });

	const answer = await propagateAttributes(
		{ userId, sessionId },
		async () =>
			startActiveObservation(
				"answer",
				async (generation) => {
					const result = await callYourModel(question);
					generation.update({ output: result });
					return result;
				},
				{ asType: "generation" },
			),
	);

	span.update({ output: answer });
});

await sdk.shutdown(); // flush before a short-lived process exits

pip install openlit

export OTEL_EXPORTER_OTLP_ENDPOINT="http://<kubecol-host>:31443"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_SERVICE_NAME="my-llm-app"
export OTEL_DEPLOYMENT_ENVIRONMENT="prod"
export OTEL_RESOURCE_ATTRIBUTES="application.id=<application-id>"

import openlit

openlit.init(
	otlp_endpoint="http://<kubecol-host>:31443",
	service_name="my-llm-app",
	environment="prod",
)

# Create the provider client after openlit.init() so instrumentation is applied.
response = client.chat.completions.create(
	model="gpt-4o-mini",
	messages=[{"role": "user", "content": "How do I restart a deployment?"}],
)

npm install openlit

export OTEL_EXPORTER_OTLP_ENDPOINT="http://<kubecol-host>:31443"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_SERVICE_NAME="my-llm-app"
export OTEL_RESOURCE_ATTRIBUTES="application.id=<application-id>"

import Openlit from "openlit";

Openlit.init({
	otlpEndpoint: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
	applicationName: "my-llm-app",
	environment: "prod",
});

// Call your instrumented provider client after OpenLIT is initialized.
await runModelAndTools("How do I restart a deployment?");

http://<kubecol-host>:31443/v1/traces

Collector endpoints

The LLM collector listens on port 31443 by default (collector.llm_collector_config.listen_port). All trace endpoints accept the same OTLP TracesData payload:

Additional JSON endpoints support the evaluation pipeline:

Set the base URL, not the full path, in SDK configuration. Langfuse appends /api/public/otel/v1/traces and raw OTLP exporters append /v1/traces.

Post a score from an external pipeline

If your quality checks run outside KubeSense, post the result directly to the collector:

curl -X POST "http://<kubecol-host>:31443/v1/llm/scores" \
	-H "Content-Type: application/json" \
	-d '{
		"application_id": "<application-id>",
		"trace_id": "<trace-id>",
		"observation_id": "<span-id>",
		"name": "answer_relevance",
		"value": 0.92,
		"data_type": "NUMERIC",
		"source": "API"
	}'

data_type is NUMERIC, CATEGORICAL, or BOOLEAN; use string_value instead of value for the last two. source is API for externally posted scores — KubeSense uses EVAL for LLM-as-a-Judge and experiment results and HUMAN_ANNOTATION for reviewer scores. The scores appear on the trace's Scores tab and in Evaluation.

Shared resource attributes

Set these once when the application starts:

export OTEL_SERVICE_NAME="my-llm-app"
export OTEL_RESOURCE_ATTRIBUTES="application.id=<application-id>,service.version=1.0.0"

Both SDKs build their OTel resource with Resource.create(), which merges OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES, so these variables apply to Langfuse and OpenLIT alike.

Environment

The Environment field and facet are populated from langfuse.environment, gen_ai.environment, env, or kubesense.env, falling back to the OpenTelemetry deployment.environment / deployment.environment.name. The explicit keys win when more than one is present.

Kubernetes context

When these attributes are present, KubeSense fills the cluster, namespace, pod, node, container, workload, and region fields:

The Kubernetes Downward API is the usual way to supply these from a Deployment manifest.

Attribute mapping reference

kubecol stores every incoming span, whether or not it carries GenAI attributes, and keeps unmapped attributes in the span's attributes map. The table below lists the keys it promotes to indexed columns. The first non-empty value wins.

Per-message attributes emitted by Traceloop (gen_ai.prompt.N.*, gen_ai.completion.N.*) and OpenInference (llm.input_messages.N.*, llm.output_messages.N.*) are reassembled into the input and output columns.

Langfuse's own OTel attributes map one to one, so a Langfuse-instrumented application populates the input, output, usage, cost, model parameters, prompt link, level, and status-message fields without any extra configuration.

Trace source

The Source label is derived from resource attributes only:

The Langfuse SDK does not override telemetry.sdk.name, so Langfuse traffic is labelled otel by default. Add telemetry.sdk.name=langfuse to OTEL_RESOURCE_ATTRIBUTES if you want the langfuse label. Either way, filter on the langfuse.* attributes or on service name to separate workloads; the source label is a convenience, not the identity of the application.

Authentication

Authentication is disabled by default (collector.llm_collector_config.enable_authentication: false). TLS is also off by default on the LLM listener; enable is_tls_enabled with a certificate and key when the collector is reachable outside the cluster.

When authentication is enabled, kubecol parses the Authorization header as HTTP Basic and validates the decoded value against KubeSense access credentials. The Langfuse SDKs already send Basic auth built from LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY, so set those to valid KubeSense credentials. For raw OTLP exporters, pass the header explicitly:

export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic $(printf '%s' 'public:secret' | base64)"

When authentication is enabled, a request with a missing, malformed, or unrecognized Authorization header is rejected with 401.

The gate covers the trace endpoints only. The score, experiment, and eval-metrics endpoints are unauthenticated because kubeapi posts experiment items and spans to the collector without a credential. Keep the LLM listener on an internal address and restrict it with network policy.

Verify ingestion

1. Generate one LLM request in the instrumented application.
2. Check the kubecol logs for LLM ingestion errors.
3. Open LLM Monitoring > Tracing in KubeSense and filter by the application or service name.

If the process exits immediately after a request, call the SDK flush method (langfuse.flush(), await sdk.shutdown()). A successful HTTP response only confirms that the collector accepted the payload; it does not confirm that a short-lived SDK process finished exporting its batch.

Configuration reference

Langfuse

OpenLIT

Environment variables take precedence over openlit.init() arguments.

Kubernetes deployment checklist

• Use the collector's Kubernetes service DNS name instead of localhost when the app runs in another pod.
• Allow egress from the application namespace to the collector service and port 31443.
• Store SDK keys and provider keys in Kubernetes Secrets, not in a container image or manifest ConfigMap.
• Set application.id and service.name through deployment environment variables so every replica uses the same identity.
• Supply the Kubernetes attributes through the Downward API when you want cluster, namespace, and pod filters in the trace views.
• Keep the SDK batch processor enabled for long-running services and flush during graceful shutdown.

Send a raw OTLP test request

Confirm network reachability before debugging SDK configuration. An empty OTLP payload is accepted and returns 200:

curl -i -X POST "http://<kubecol-host>:31443/v1/traces" \
	-H "Content-Type: application/json" \
	-d '{"resourceSpans":[]}'

A 200 proves the collector is reachable and parsing OTLP JSON. A 413 means the request body exceeded the collector's decompressed-body cap. Anything else points at networking, TLS, or the listener configuration.

Troubleshooting