Kubesense

Investigations

Investigations is the deep-analysis experience inside Agent SRE. Where Conversations is an interactive back-and-forth chat you drive yourself, an Investigation is an autonomous, multi-step analysis: from a single alert, Agent SRE forms competing hypotheses, tests each against your telemetry, and works the problem to a root-cause conclusion with a confidence score — no manual querying required.

Investigations is one of the three tabs in Agent SRE, alongside Conversations and MCP Integrations.

Conversations vs. Investigations

  • Conversations — quick, interactive questions and follow-ups where you steer each step.
  • Investigations — you hand Agent SRE a symptom (a firing alert), and it runs an end-to-end root cause analysis on its own, then presents its reasoning and conclusion.

The Investigation list

Open Agent SRE and select the Investigation tab to see every investigation that has been run.

Agent SRE — Investigations list

  • Search — find investigations by name or root cause text.
  • Filters — narrow by Source and Initiated by.
  • Status tabs — switch between All, In progress, and Completed.

Each row summarizes one investigation:

ColumnDescription
InvestigationsThe investigation name (from the alert it was started from)
SourceWhat triggered it — e.g. an Alert Event
Root CauseA one-line summary of the conclusion
VerdictThe outcome, e.g. Root cause found
ConfidenceHow confident Agent SRE is in the conclusion, as a percentage
DurationHow long the investigation took (e.g. Completed in 3m)
InitiatedWho started it
Created AtWhen it was run

Click any row to open the full investigation.

Starting an Investigation

Investigations are launched from a firing alert — either a single alert event or an alert rule.

From an alert event

Open an alert from Alert Events to see its detail page, then click Investigate in the top-right actions.

Alert Detail — Investigate

Agent SRE starts an investigation scoped to that specific firing event, using the alert's rule, threshold, data source, and labels as the starting context.

From an alert rule

On an Alert Rule page, use the Investigate dropdown. Because one rule can be firing for many series at once, you can choose the scope:

Alert Rule — Investigate dropdown

  • Investigate the whole rule — one run across every firing series (the full blast radius). For example, Investigate all 5 series.
  • Investigate one series — pick a single firing instance from the Firing instances list (each shows its labels, current value, and how long it has been firing) and investigate just that series.

Reading an Investigation

An investigation opens on its detail page, with the alert name, a completion status (e.g. COMPLETE), the Source, and who initiated it. Two views show the same analysis from different angles: the Hypothesis Tree and the Investigation Steps.

Hypothesis Tree

The Hypothesis Tree shows how Agent SRE reasoned about the alert. Starting from the source alert, it branches into the competing hypotheses it considered, each tagged with a verdict, and converges on a final conclusion.

Investigation — Hypothesis Tree

Each hypothesis carries one of three verdicts:

  • Verified — supported by the evidence
  • Rejected — ruled out by the evidence
  • Inconclusive — not enough evidence to confirm or rule out

Verified hypotheses can lead to further, more specific hypotheses. The Investigation Conclusion node summarizes the most likely root cause and shows the overall confidence (e.g. 88%). Use the zoom controls to pan around large trees.

Click any hypothesis node to open a panel with the full detail behind it:

Investigation — Hypothesis detail

  • The full hypothesis statement and its verdict, with Agent SRE's reasoning for that verdict.
  • Evidence — the specific findings that support or refute the hypothesis (metrics, span behavior, values observed).
  • Relevant Tool Calls — the exact queries Agent SRE ran to test it (e.g. an analyze-traces call with its filters, fields, grouping, and time window), so you can reproduce or verify the finding yourself.

Investigation Steps

The Investigation Steps view is the chronological trail of what Agent SRE actually did — from the initial Trigger to the final Conclusion.

Investigation — Investigation Steps

  • The left panel lists every step (e.g. Loaded alert configuration, Tested alert query semantics, Checked for deploy or scaling changes, Checked downstream dependency latency) so you can jump straight to one.
  • Each step card explains what was checked and why, and footnotes the number of tool calls it made and how long it took (e.g. 1 tool call · 18.6s).
  • The final Investigation Conclusion restates the root cause and its confidence.

Click any step to open a panel with its full detail:

Investigation — step detail

  • The step's full reasoning — what Agent SRE set out to test and why.
  • Tool Calls — every tool the step invoked (e.g. get-trace-or-log-fields, analyze-traces), each expandable to show the exact Input and Output JSON.

This makes every conclusion auditable: you can see exactly which signals were queried, which hypotheses were tested, and how Agent SRE arrived at its answer.