Anomaly detection

Anomaly Detection

A threshold catches a value you can name in advance. Anomaly detection catches the ones you cannot: it learns what is normal for every KPI and flags a value that leaves that range.

Overview

Anomaly detection runs on the KPIs of your services. A KPI is an LPQL search that gives one value, such as the error rate of checkout or the logins per minute (see Service Intelligence). Every time a KPI runs, LogPulse compares its value with what that KPI normally does at that moment. You see the result under Intelligence → Anomalies and on the Anomalies tab of each service.

Baselines

The baseline of a KPI is its normal range, learned per weekday and per hour: Monday 09:00 has its own baseline, Sunday 03:00 another. It holds the median, the quartiles and the spread of the values in that slot. Until a KPI has enough history (5 days by default) it shows as learning and raises no anomalies.

The slots follow the clock of your organization. Set the baseline time zone in the security settings, under Anomaly scoring. With the default, UTC, a rhythm that follows local office hours shifts an hour when the clocks change.

When a value is an anomaly

A value inside the usual range of its slot is normal. Outside it, LogPulse computes how far it is from the median with a robust z-score, which uses the median and quartiles so a few outliers in the history do not spoil the baseline:

z = |value − median| / (IQR × 0.6745)

Severity

Severityz-score above
Low1.5
Medium2.5
High3.5
Critical5.0

The thresholds are divided by the KPI's sensitivity: at 2x, a z-score of 2.5 is already Critical.

Guards against false alarms

RuleWhy
The spread never drops below a floor (per unit, and 1% of the median)A flat history would otherwise turn a tiny change into a huge z-score
A count varies at least by the square root of its medianGoing from 3 to 5 is normal noise, not an alarm
On a KPI with thresholds, only the bad direction countsA disk that fills up is an anomaly; one that empties is not
A drop is at most Medium, unless lower is worse for that KPISmall sources go quiet for many harmless reasons
A flat history gives at most LowIt cannot tell how unusual a change is
A deviation inside the thresholds is at most MediumIt is context, not an outage
A KPI that is mostly zero is only judged when activeIdle is normal for "CPU while driving"

Settings per KPI

You find these in the KPI editor, under advanced settings:

SettingRangeDefault
Anomaly sensitivity0.25x to 4x1x
Baseline days1 to 145
Anomaly cooldown5 to 1440 minutes30 minutes
AI investigationOff, or on from Low, Medium, High or CriticalOff

The cooldown is the quiet time before the same anomaly is raised again. With AI investigation on, an anomaly of at least the chosen severity starts the AI Investigator on it.

Working with anomalies

An anomaly shows the KPI, the entity that deviated, when it started, the value against the expected range, and its severity. View logs opens the search the KPI runs, narrowed to that entity. You can acknowledge an anomaly, dismiss it, or start an investigation by hand.

Feedback that tunes

Not an issue lowers the sensitivity of that KPI by one step (to 85%, never below 0.25x). Real issue raises a KPI that feedback lowered back up, never past the value it had before. A sensitivity you set by hand in the KPI editor always wins.

A KPI that is called not an issue three times at the same weekday and hour (the nightly batch, the Monday-morning login wave) stops raising anomalies for that hour. The rule appears under learned patterns, where you can switch it off. Thresholds are never suppressed.

Notifications

Thresholds decide the health of a KPI. An anomaly on a KPI with thresholds is recorded but does not change its health, and so does not notify. On a KPI without thresholds, an anomaly does change its health, and the service's notification routes are told: email, Slack, webhook, push or a pipeline.

Tip: Want to be woken up? Give the KPI thresholds. Want to see drift early? Look at the Anomalies page, or let the AI investigate from High.

API

CallWhat it does
GET /api/v1/detect/anomaliesAnomalies, filtered by status, service, severity and time
GET /api/v1/detect/anomalies/:idOne anomaly
GET /api/v1/detect/anomalies/timelineAnomalies over time
PUT /api/v1/detect/anomalies/:id/acknowledgeAcknowledge
PUT /api/v1/detect/anomalies/:id/dismissDismiss
PUT /api/v1/detect/anomalies/:id/feedbackReal issue or not an issue
POST /api/v1/detect/anomalies/:id/investigateStart an AI investigation
GET /api/v1/detect/patternsLearned patterns
PATCH / DELETE /api/v1/detect/patterns/:idSwitch a pattern off or remove it

Acknowledging, dismissing, feedback and investigating need alerts:update.