Why We Put AI at the Heart of LogPulse

Most log platforms in 2026 still optimise for the same workflow that existed in 2010: a human types a query, scans a result list, refines the query, repeats. The interface gets faster, the storage gets cheaper, the syntax gets prettier, but the cognitive load on the engineer at 3am is unchanged. We do not believe that workflow is the destination. We believe it is the floor. AI is what raises the ceiling, and that is why we built LogPulse with AI as a first-class component of the platform, not an autocomplete bolted onto the search bar.

This post explains where AI is today inside LogPulse, why we chose root cause analysis as the first surface to invest in, and where this is going. It is also a deliberate look forward: the long-term thesis is that the same primitives that make a useful Investigator also make a SIEM that catches threats no signature was ever written for.

The Problem with "Search Faster"

When something breaks in production, the engineer on call does not have a search problem. They have an explanation problem. They need to know what changed, what failed, what cascaded, and which of the seventeen alerts firing simultaneously is the cause and which are the consequences. Faster queries help, but the bottleneck is rarely the database. It is the time spent forming hypotheses, running them, reading the results, and forming the next one.

Every observability vendor has spent years optimising the wrong half of that loop. The query runs in 200ms; the human still takes 45 minutes to chain enough queries together to land on the answer. Improving query speed by another 10x changes nothing if the bottleneck is human reasoning time.

AI is the first technology in two decades that meaningfully attacks the second half of the loop. Not because it knows the answer up front, but because it can run dozens of hypotheses in parallel, read the evidence, and narrate its reasoning in a way the engineer can audit and correct.

What the Investigator Actually Does

The LogPulse AI Investigator is a streaming agent that runs on Anthropic Claude. When you ask it a question -- "why is the checkout API returning 500s in production?" -- it does not generate a single LPQL query and call it a day. It runs an iterative loop: form a hypothesis, call a tool, read the result, refine, call another tool, until it has enough evidence to synthesise a structured answer.

Under the hood, the investigator has access to a deliberately constrained tool surface: LPQL search, time-bucketed aggregation, field-context lookups, system-health snapshots, and the ability to spawn sub-investigations for parallel branches of inquiry. It runs in a streaming loop with extended thinking turned on, so the UI can show the engineer the model's reasoning as it happens. When the model emits multiple tool calls in a single turn, they fan out via Promise.allSettled and return concurrently.

The output is not a paragraph of prose. It is a structured Finding[] array, where every finding carries a title, a severity, an array of supporting evidence (the actual log lines, with their timestamps), and a confidence score. The web UI renders each finding as a clickable card that drills into the underlying logs, so the engineer can verify the claim before acting on it. This is deliberate -- AI that cannot be audited is AI that cannot be trusted in production.

interface Finding {
  title: string;            // human-readable claim
  severity: 'low' | 'medium' | 'high' | 'critical';
  evidence: Evidence[];     // actual log lines with timestamps
  confidence: number;       // 0..1, surfaced in the UI
}

Why Root Cause First

When we sat down to decide where AI should land first, we ranked features by two axes: how much value AI adds beyond what a faster query produces, and how forgiving the surface is when the model is wrong. Root cause analysis scored highest on both.

On the first axis, the lift is enormous. The investigator is doing the work the on-call engineer would otherwise do manually -- scanning related services, correlating timestamps, forming hypotheses, ruling them out. A model that runs that loop in two minutes instead of forty-five is genuinely useful, even when it gets some of the intermediate steps wrong, because the engineer is still in the loop and the evidence is auditable.

On the second axis, the blast radius of a wrong answer is small. The engineer reads the finding, looks at the evidence, and either agrees or does not. The Investigator never autonomously takes action on infrastructure. This is in deliberate contrast to surfaces where AI has been pushed prematurely: auto-remediation, auto-scaling decisions, auto-triage that closes tickets without review. Those surfaces punish wrong answers severely. Root cause analysis does not.

Starting with root cause also gives us the right learning signal. Every investigation produces feedback: explicit thumbs-up/thumbs-down, but also implicit signals like which findings the engineer clicked into and which they ignored. We feed positive-feedback investigations into a pattern memory store, keyed by a Haiku-extracted pattern signature. Subsequent investigations in the same organisation get matching insights injected into the system prompt. The platform learns the shape of your incidents over time, without ever training a model on your data.

The Second AI Surface: Co-Building Pipelines

Root cause analysis is one shape AI takes inside LogPulse. Building data pipelines is another. The Pipeline AI is a chat agent embedded directly in the visual ETL editor: same Claude backend, same auditable approach, different problem surface. Where the Investigator answers "why is this broken", the Pipeline Agent answers "how do I get this data in, in the shape I actually need it".

Building an ETL pipeline is a hypothesis-and-evidence loop too. You sketch a graph, run it, look at what failed, adjust. The bottleneck for most users is not the editor itself. It is knowing what an integration's auth header looks like, which node parses what, which sequence of operations turns the raw API response into a row that lands in the right index. New users hit a documentation wall before they hit the actual work. The Pipeline Agent is built to walk through that wall with them.

The agent sits in a side panel next to the canvas. It can read the current pipeline graph, the last run's output per node, the structured run logs that the engine writes into the tenant's internal indexes, and historical versions of the same pipeline for diff. It can also list and read other pipelines in the workspace as references. And it can mutate the canvas: add a node, insert a node between two existing ones, update a node's configuration, connect or disconnect edges, remove things. Every change it makes shows up on the canvas with a coloured "added" or "modified" badge, so you can see at a glance what the agent touched and audit each change before promoting the draft.

For integrations the model does not reliably know (Datadog's logs-ingest endpoint, Stripe's pagination cursor shape, Auth0's audit-log payload format) it can look up the official documentation via Anthropic-hosted web search, capped at three lookups per turn. And critically: it asks you first. The web-search protocol is not a model quirk; it is an explicit UX rule. Web pages can contain prompt-injection attempts dressed up as instructions, and we treat any third-party content as untrusted. So the agent surfaces a one-line confirmation -- "I'd like to look up the Datadog logs-API docs to confirm the auth header. OK to search?" -- and only searches once you say yes. Tenant secrets, API keys, payload values, and any other data from your environment are never put into a search query.

The result is a workflow where you describe what you want in plain language and the agent walks you through making it real. A common shape:

you: every hour, pull our staging error logs from the last hour
     and POST a markdown digest to this Slack webhook: https://...

agent: I'd like to look up the Slack incoming-webhook docs to confirm
       the request shape. OK to search?

you: yes

agent: [web_search → reads Slack docs]
       Slack expects a POST with a JSON body { "text": "..." } and a
       2xx response counts as delivered. Building this as four nodes:

         start → lpqlSearch → transform → httpRequest

       - lpqlSearch runs `level=error earliest=-1h` against your tenant
         logs and lands the rows on the payload.
       - transform folds the rows into a markdown summary string.
       - httpRequest POSTs { text: "<summary>" } to your webhook.

       I'll set the trigger to schedule every hour. Sound right?

you: yes, do it

agent: [add_node × 3, add_edge × 3, configures each node]
       Done. Three nodes badged "added" on the canvas. Want me to run
       it once now to verify the Slack delivery before we promote?

Most pipelines that fit this shape come together in a few minutes. The bulk of that time is the conversation, not the typing, and the conversation is where the value is, because the agent is asking the questions that catch the mistakes a copy-pasted template would have shipped silently. "Are you sure you want this triggered every minute and not every hour?" "This webhook URL looks like it includes a secret. Should I store it in pipeline secrets instead of inline?" "You're reading from index=apps and ingesting back into index=apps. That loops. Did you mean a different destination?"

The constraints are deliberate. The agent only mutates the current pipeline's draft, never historical versions and never another pipeline (it can read those for reference, but editing them is a UI action you take, not the agent). It refuses bulk-destructive actions (wiping all nodes, removing every edge) without explicit confirmation. And the node library it can build with is exactly the eighteen node types the engine actually supports, because the system prompt is grounded in the same NODE_DOCS reference a human reads in our docs. The agent cannot invent a node that does not exist; it has to compose what is there.

The Investigator and the Pipeline Agent are the same architectural pattern applied to two different problems. A constrained tool surface, an auditable action trail (badges on changed nodes, evidence on findings), a model that prefers to ask one clarifying question rather than guess, and an explicit consent rule before any operation that touches third-party content. That pattern is what makes both surfaces production-grade rather than demo-grade, and it is the foundation we are building everything else on top of.

Why This Is Step One, Not the Destination

The Investigator and the Pipeline Agent are the first two surfaces where we are willing to put AI in front of a paying customer, because the value is high and the failure mode is benign. But the deeper bet is that the same primitives -- a constrained tool surface, a structured output schema, an auditable evidence trail, pattern memory, an explicit consent rule before third-party content enters the conversation -- generalise to a problem that the security industry has been failing at for thirty years: detecting threats that no one has written a signature for.

Modern SIEMs are still fundamentally signature engines. They detect what someone has already seen and codified. They miss novel attack chains, lateral movement that does not match a rule, insider misuse that looks superficially normal. The industry has spent two decades writing more rules and calling it progress. The result is alert fatigue, false-positive rates that train SOC analysts to ignore the tool, and detection coverage that stops at the boundary of what the rule writers anticipated.

An AI agent that can iterate over evidence, form hypotheses, and produce structured findings with audit trails is the right shape for security detection too. The Investigator we ship today asks "why is this service erroring?" The Investigator we are building toward asks "is anything unusual happening across this organisation's logs in the last hour?", and answers it without requiring someone to have written a rule for that specific anomaly first.

We are not there yet. We have shipped the foundation: pattern detection via Drain3-style log templating, statistical anomaly detection, and the agent infrastructure that can reason over both. The hard work ahead is the security-specific tooling, the threat-model coverage, and the operational maturity that turns "an LLM noticed something weird" into "a SOC analyst can act on this with the same confidence they would act on a Sigma rule." That is a multi-year arc. Root cause analysis is how we prove the technology and earn the right to take it there.

The Macro: Every Organisation Is Figuring AI Out

Outside our product, the same conversation is happening in every enterprise we talk to. Heads of platform are standing up internal AI provisions. Data teams are being formed or restructured to own model evaluation, prompt management, and the guardrails that keep AI usage compliant. CIOs are deciding which models to centralise on, which providers to allow, and how to expose those provisions to product teams without losing control.

This is not hype-cycle activity. It is real org-design work, with budget attached, and it is happening in roughly every Series-C-and-above company in Europe right now. The teams driving it have a problem we recognise: they have invested in the AI capability, and now they need products to plug into it that respect the boundaries they have built. Not products that ship a hardcoded provider, log everything to a vendor cloud, and wave away the data-residency conversation.

When those teams ask us for choice, the request usually breaks down into three concrete reasons. The first is data sovereignty. Dutch and broader European organisations are facing tighter requirements from GDPR, BIO, and the EU AI Act. For municipalities, hospitals, water authorities, and regulated financial institutions, "where does my log data live during AI processing" is no longer an academic question. It is a compliance line item. An American AI provider may be excellent technically, but excellence does not override jurisdiction. The second is vendor lock-in. Mature IT organisations refuse to take a critical dependency on a single provider whose pricing, terms, or product roadmap can change unilaterally. The third is leverage on existing investment. Customers who already operate an Azure OpenAI tenant, a GPU cluster running vLLM, or a self-hosted Mistral or Llama deployment want that investment reflected in the tools they buy, not duplicated alongside it.

LogPulse is built to plug in. The AI module lives behind a clean interface: prompt blocks composed in code, tool schemas defined in one file, a single Anthropic client today that reads ANTHROPIC_API_KEY. Everything that talks to a model goes through it. Swapping the provider, changing the model tier, routing through an organisation-internal AI gateway -- these are configuration shapes, not architectural rewrites. The structured-finding output schema, the security guardrails, the evidence-rendering UI work the same regardless of where the actual inference happens.

We are honest about today: production LogPulse runs on managed Claude through Anthropic's API, in Europe-resident infrastructure where the model provider supports it. We do not currently offer a customer-facing "bring your own model" toggle, but that is the next step on this trajectory, not a maybe.

Three Deployment Shapes, One Product

The architecture we are building toward is one product with three deployment shapes, picked per customer based on what they actually need. Each shape is the right answer for a different operational reality.

The first shape is managed cloud AI. This is what every customer uses today. The Investigator runs against managed Claude via Anthropic's API, in EU-resident infrastructure, and we operate the entire pipeline. Setup time is zero, model performance is the strongest currently available, and the customer trades a small amount of architectural control for a lot of operational simplicity. For most teams, this is exactly the right trade.

The second shape is EU-hosted open-source inference, operated by us. For customers where data residency is a hard compliance requirement and a US-headquartered model provider is not acceptable regardless of where the metal lives, we are building toward AI features that run end-to-end on open-weight models hosted on European GPU infrastructure under our operational umbrella. Same Investigator UX, same structured-finding output, different inference backend. This is the shape that makes the public-sector and regulated-finance conversation possible without forcing the customer to operate their own GPUs.

The third shape is bring-your-own-endpoint. Customers who already run an Azure OpenAI tenant, an internal AI gateway, or a self-hosted Mistral, Llama, or vLLM deployment can point LogPulse at that endpoint and use it for inference. The customer keeps full control over which model is called, where the inference runs, what gets logged, and what gets billed. We run the product; they run the model. This is also the shape that scales to airgapped on-premises, where the AI endpoint is the customer's own GPUs inside their own perimeter and no traffic ever leaves their network.

None of these three is shipping in the trial today beyond the first. We are saying out loud that all three are the direction. We would rather have customers make the cloud-AI choice today knowing the on-prem shape is on the path, than have them adopt us today and discover later that the architecture cannot follow them where they need to go.

How We Get There

The non-AI parts of LogPulse are already structured for self-hosted deployment. ClickHouse, PostgreSQL, Redis, Fastify, React Flow. Every core component is open-source and runs anywhere a Kubernetes cluster does. The same containers that run our managed cloud run on a customer's infrastructure. We have written about this elsewhere: the kill-switch post lays out which dependencies are core and which are peripheral, and how long it takes to migrate each one.

The AI side is the part that needs new work. The first milestone is provider abstraction at the AI client layer, a clean dispatcher so the same Investigator code can target managed Claude, an EU-hosted open-weight model on our infrastructure, or a customer-supplied endpoint, through one interface. The Investigator's prompts, tool schemas, and finding shapes are designed to be portable, but model-specific behaviour (cache breakpoints, structured-output reliability, tool-use parallelism) varies enough that this is real engineering, not a configuration flip.

After that comes the EU-hosted tier: selecting and operating an open-weight model that meets the structured-output and tool-use bar the Investigator depends on, on European GPU infrastructure we run. Then the on-prem package: a Helm chart, a documented set of model requirements (so customers know what hardware they need to host their own endpoint), and a deployment guide for customers who want to run the entire stack inside their own perimeter, including airgapped.

Enterprise customers in regulated sectors (defence, critical infrastructure, healthcare networks under NIS2, public-sector bodies operating under BIO) have been telling us they need this for as long as we have been talking to them. They are right. The principle is simple: a customer should never have to choose between good AI features and control over their data. Cloud-only is not a viable answer for every customer, and pretending otherwise would be dishonest.

What We Are Asking You to Trust

The honest version of this post is that we are asking you to trust two things. First, that AI applied carefully to the right surface -- root cause analysis, with auditable evidence and a human in the loop -- is genuinely useful today, not a demo-quality novelty. We have shipped it, customers use it, and the structured-finding format is what makes it production-grade rather than a chatbot.

Second, that the foundation we are building on -- the agent loop, the tool surface, the pattern memory, the multi-provider architecture -- is the right shape for the next set of problems. SIEM that detects beyond signatures. EU-hosted inference for residency-bound customers. Bring-your-own-endpoint for organisations that have already invested in their own AI provision. Airgapped on-prem for the most demanding deployments.

Both of those bets are still bets. The Investigator gets things wrong sometimes; we tell you when it does and we use the feedback to make it better. The SIEM trajectory is multi-year work. The EU-hosted and bring-your-own-endpoint tiers are on the roadmap, not in the trial today. We would rather be straight about what is shipped, what is in flight, and what is further out, than oversell. The customers who choose us based on a clear-eyed picture of where we are tend to be the customers we want.

If you are running an organisation that is figuring out how to put AI to work, and you want a logging platform built to plug into that effort rather than duplicate it, talk to us. The AI Investigator and the Pipeline Agent are in every plan today. The conversation about which deployment shape is right for you -- managed cloud, EU-hosted, or your own endpoint -- is the conversation we want to be having with enterprise customers over the next year.