Kubesense

Script Steps

A Script step runs a short piece of JavaScript over the output of earlier steps. It is for the reshaping that templates make awkward: grouping rows, computing percentiles, picking a route from an alert's labels, or building a nested request body.

A script reads data and returns data. It cannot send anything, so a script on its own never reaches outside KubeSense; Slack and HTTP steps still do the sending.

Writing a script

Add Script from the Logic group of the step palette. The editor holds the body of a function: write statements, and end with return.

const rows = ctx.Steps.trace_samples.rows;
return { spans: rows.length, services: [...new Set(rows.map((r) => r.app_service))] };
PartWhat it is
Inputctx, which holds the same data templates see: ctx.Trigger, ctx.Inputs, ctx.Steps and ctx.Run. See What data is available for the fields
OutputWhatever you return. It must be JSON-shaped: objects, arrays, strings, numbers, booleans or null
Loggingconsole.log, info, warn, error and debug. The output shows in the dry run and on the run detail page

Later steps read the output like any other step's:

  • in a template, {{ .Steps.<script id>.<field> }}
  • in another script, ctx.Steps.<script id>.<field>

As with templates, a script can only read steps that run before it. Add them under Runs after, or the builder flags the reference.

info: ctx is a copy. Changing it inside a script does not change what later steps see. Return the data you want them to have.

Help in the editor

  • Completion. Typing ctx. offers Trigger, Inputs, Steps and Run, and ctx.Steps. offers only the steps upstream of this one. After a dry run, completion knows the real fields each step returned, so ctx.Steps.trace_samples.rows[0]. suggests app_service, duration, resource and the rest, with their types.
  • Problems. A syntax error, an empty script, a script over 64 KB, or a read of a step that does not run first appears in the Problems list and blocks Publish until it is fixed. Only reads written out as ctx.Steps.<id> or ctx.Steps["<id>"] are checked. A step named in a comment or a string is not a read, and a name built while the script runs (ctx.Steps[name]) cannot be checked.

What a script can and cannot do

Scripts run standard modern JavaScript: const/let, arrow functions, destructuring, template strings, ?. and ??, Map and Set, regular expressions, JSON, Math, Date, and array methods up to Object.groupBy.

There is deliberately no network, file, timer or module access. fetch, require, setTimeout and process do not exist. Use an HTTP step to call out.

LimitValue
Run time10 seconds
Memory growth while running256 MB
Script size64 KB
Returned value, as JSON1 MB
Console output kept16 KB, then truncated
One string built with repeat/padStart/padEnd1 MB
One regular-expression match2 seconds
Call depth1,000 nested calls

Each script runs in its own isolated process, so a script that misbehaves fails its own step and nothing else.

A dry run executes scripts for real. Unlike a Slack or HTTP step, a script has nothing to send, so there is nothing to simulate.

Samples

Rename the step ids (trace_samples, log_samples, error_rate) to match your workflow.

Slow and failing services from trace samples

Step id slow_services, after a Trace samples step.

const { rows, _truncated } = ctx.Steps.trace_samples;
const byService = {};
for (const r of rows) {
  const s = (byService[r.app_service] ??= {
    service: r.app_service, spans: 0, errors: 0, slowestMs: 0,
  });
  s.spans++;
  if (r.status_code === "ERROR") s.errors++;
  s.slowestMs = Math.max(s.slowestMs, Math.round(r.duration / 1e6)); // duration is in nanoseconds
}
const services = Object.values(byService).sort((a, b) => b.slowestMs - a.slowestMs);
console.log(rows.length, "spans across", services.length, "services");
return {
  services,
  worst: services[0]?.service ?? "none",
  text: services
    .map((s) => `• ${s.service}: ${s.slowestMs} ms slowest, ${s.errors}/${s.spans} errors`)
    .join("\n"),
  truncated: _truncated,
};

Post it with {{ .Steps.slow_services.text }}, which renders lines like • checkout: 2400 ms slowest, 1/2 errors.

warning: Check the status_code values your spans actually carry: open a dry run and look at the Trace samples step's output. Adjust === "ERROR" if they differ.

Latency percentiles of the sampled spans

const ms = ctx.Steps.trace_samples.rows
  .map((r) => r.duration / 1e6)
  .sort((a, b) => a - b);
const pct = (p) =>
  ms.length ? ms[Math.min(ms.length - 1, Math.floor((p / 100) * ms.length))] : null;
return { n: ms.length, p50: pct(50), p95: pct(95), p99: pct(99) };

These are percentiles of the sampled spans: the step's row limit, fetched slowest first, so they read high. For percentiles of all traffic, use a Query step.

Group error logs by message

Step id error_digest, after a Log samples step.

const { rows, _truncated } = ctx.Steps.log_samples;
const errors = rows.filter((r) => /^(error|fatal|critical)$/i.test(r.level));
const groups = new Map();
for (const r of errors) {
  // The same message with different ids or numbers counts as one pattern.
  const pattern = r.body.replace(/0x[0-9a-f]+|\b\d+\b/gi, "#").slice(0, 120);
  groups.set(pattern, (groups.get(pattern) ?? 0) + 1);
}
const top = [...groups].sort((a, b) => b[1] - a[1]).slice(0, 5);
return {
  errorCount: errors.length,
  text: top.length
    ? top.map(([p, n]) => `• ${n}× ${p}`).join("\n")
    : "No error logs in the window.",
  note: _truncated ? "Only the first rows were fetched; counts are a lower bound." : "",
};

Renders lines like • 2× payment # failed: timeout after 3000ms.

Route by the alert's labels

Step id route, in an alert-triggered workflow.

const alert = ctx.Trigger.alert ?? {};
const labels = alert.labels ?? {};
const ns = labels.namespace ?? "unknown";
const owners = { "prod-payments": "payments", "prod-checkout": "checkout" };
const team = owners[ns] ?? "platform";
return {
  team,
  page: alert.severity === "critical" && ns.startsWith("prod"),
  title: `[${(alert.severity ?? "info").toUpperCase()}] ${alert.rule_name}` +
    ` — ${labels.workload ?? ns}`,
};

A Condition step with {{ .Steps.route.page }} can then gate a paging step, and {{ .Steps.route.title }} makes the message headline.

Series over a threshold

After a Query step error_rate grouped by service.

const limit = ctx.Inputs.threshold ?? 5;
const hot = ctx.Steps.error_rate.series
  .filter((s) => s.value > limit)
  .sort((a, b) => b.value - a.value)
  .map((s) => ({ service: s.labels.app_service, value: Number(s.value.toFixed(2)) }));
return { any: hot.length > 0, count: hot.length, hot };

The keys under s.labels are whatever the query groups by.

A ticket body for an HTTP step

Step id ticket, shaped like a Jira issue.

const a = ctx.Trigger.alert;
const failing = ctx.Steps.trace_samples.rows
  .filter((r) => r.status_code === "ERROR")
  .slice(0, 3)
  .map((r) => `${r.app_service} ${r.resource} (${Math.round(r.duration / 1e6)} ms)`);
return {
  fields: {
    summary: `${a.rule_name}: ${a.labels.workload ?? ""}`.trim(),
    priority: { name: a.severity === "critical" ? "Highest" : "Medium" },
    labels: ["kubesense", a.severity, a.labels.namespace].filter(Boolean),
    description: [
      `Fired at ${a.starts_at} with value ${a.value}.`,
      failing.length ? "Failing spans:\n" + failing.join("\n") : "No failing spans sampled.",
      `Run: ${ctx.Run.id}`,
    ].join("\n\n"),
  },
};

Set the HTTP step's body to {{ toJSON .Steps.ticket }}. Building the object in a script and serialising it once means quotes and newlines in log text cannot break the JSON, which is easy to get wrong when a body is assembled in a template.

When a script fails

The step fails with one of these messages, shown on the step's Error tab:

MessageWhat to do
the script returned nothing — end it with return <value>Every path through the script needs a return
syntax error: …The line and column count from the first line of the editor
syntax error: the script has an unmatched closing braceRemove the extra }
the script threw: …An exception escaped, often reading a field of undefined. Use ?. and ??, and check the upstream step's output in a dry run
the script returned a value with no JSON formA function or symbol was returned. Return plain data
the script's return value cannot be serialisedUsually an object that refers to itself
the script's return value is … bytes as JSON; the limit is …Return a summary rather than every row
the script ran longer than 10sIt hit the 10-second limit. Look for a loop that never ends
the script was stopped: the step ran out of timeThe step's own time budget ran out first
the script recursed too deeplyA function calls itself without stopping
the script was stopped: memory grew by … MB / the script ran out of memoryIt held too much at once. Work over fewer rows, or keep counts instead of copies
a regular expression ran longer than …msThe pattern backtracks badly on this input. Simplify it, or anchor it
the script returned a promise that never settlesScripts cannot wait on timers or I/O. Return a value directly

When a script throws, the console output it wrote before the failure is kept, so a console.log just before the failing line often shows the cause.