Analysis Commands

Commands that find unusual values, forecast trends, group similar messages and compare periods. They work on the current results, so they usually follow stats or timechart.

outlier

Caps or removes values far outside the usual range: above the third quartile plus param × the interquartile range (IQR), and, with uselower=true, below the first quartile minus that.

Syntax:

outlier [action=transform|remove] [param=<number>] [uselower=<bool>] [mark=<bool>] [<field> ...]

Without fields, every numeric column of the results is checked.

Example:

timechart span=5m avg(duration) as latency
| outlier action=remove latency

anomalydetection

Finds rows with an unusual value.

Syntax:

anomalydetection [method=histogram|zscore|iqr] [action=filter|annotate] [pthresh=<p>] [<field> ...]

• action=filter (default) keeps only the unusual rows. action=annotate keeps every row and adds isOutlier (1 or 0) and, for histogram, log_event_prob.
• pthresh defaults to 0.01.
• Without fields, every column except _time is checked.

Examples:

stats count by workload, level
| anomalydetection level

timechart span=10m count
| anomalydetection method=zscore action=annotate count
| where isOutlier = 1

predict

Forecasts a timechart series: fits a straight-line trend and continues it into the future.

Syntax:

predict <field> [as <name>] ... [future_timespan=<N>] [holdback=<N>]

• Adds prediction(<field>) (or the as name) with a 95% band, upper95(prediction(<field>)) and lower95(prediction(<field>)), from how far the values stray from the trend.
• future_timespan (default 5) adds that many future buckets, with the prediction and no actual value.
• holdback leaves the last N buckets out of the fit, to see how well it predicts them.
• It follows a timechart without by.

Example:

timechart span=1h count
| predict count future_timespan=24

cluster

Groups similar messages: messages that differ only in numbers, IDs, UUIDs, IP addresses or quoted values belong together.

Syntax:

cluster [field=<field>] [showcount=<bool>] [countfield=<name>] [labelfield=<name>] [labelonly=<bool>]

Without labelonly, the result has one row per group (its newest message), largest group first. Groups are numbered from 1, largest first.

Examples:

level=error
| cluster showcount=true
| table cluster_count, body

level=error
| cluster labelonly=true
| stats count, values(pod_name) as pods by cluster_label

Splunk's similarity threshold, t=, is accepted and ignored. To count message templates over long time ranges, tstats count by pattern is faster.

timewrap

Lines up the periods of a timechart side by side, for comparing today with yesterday or this week with last week.

Syntax:

timewrap <period>

The period is seconds to weeks: 1d, 1w, 12h. Each bucket moves onto the latest period, and its values go to that period's column: count_latest_day, count_1day_before, count_2days_before, … Periods are counted back from the end of the time range. A time range may hold at most 100 periods.

Example:

With a time range of the last 7 days:

timechart span=1h count
| timewrap 1d

This returns 24 rows, one per hour of the day, with a column for each of the seven days.