Send Azure logs via Event Hub logs to LogPulse

Azure Activity Log, Entra ID sign-ins and audit, Defender for Cloud alerts, Azure Firewall, WAF, AKS audit and other resource logs from one Event Hub.

In short
To send Azure logs via Event Hub logs to LogPulse, add Azure logs via Event Hub under Configure → Sources → Cloud. LogPulse reads the Azure Event Hub (Kafka endpoint) itself, without an agent. Events land in the audit, iam, firewall, web, endpoint, kubernetes indexes with source azure and fill the Cloud Audit, Authentication, Alert, Network Activity, Web Access data models.

Overview

How Azure logs via Event Hub data reaches LogPulse
Vendor
Azure Event Hub (Kafka endpoint)
Collect
Integration pollerreads new events, maps the fields
Store
index=audit
index=iam
index=firewall
index=web
index=endpoint
index=kubernetes
  1. Azure Event Hub (Kafka endpoint) to Integration poller
  2. Integration poller to index=audit
  3. Integration poller to index=iam
  4. Integration poller to index=firewall
  5. Integration poller to index=web
  6. Integration poller to index=endpoint
  7. Integration poller to index=kubernetes

LogPulse signs in to Azure logs via Event Hub with the credential you give it and fetches new events every five minutes by default. You can set the interval per integration, from one minute to 24 hours.

What you need

  • An Event Hubs namespace on the Standard tier or higher (Kafka endpoint)
  • A diagnostic setting per subscription or resource that streams to the hub
  • A shared access policy with Listen rights
  • In LogPulse: a role that may manage API keys. Cloud integrations fall under that right.

1. Create a credential in Azure logs via Event Hub

Field in LogPulseWhere you find it
Connection stringFrom a shared access policy with Listen rights, on the namespace or on the hub. Stored encrypted and never shown again.
Event hub (optional)The name of the hub. Not needed when the connection string is from the hub itself (it then ends with EntityPath=…).
Consumer group (optional)A consumer group for LogPulse alone, so it does not take events from another reader. Empty uses "logpulse".

2. Add the integration in LogPulse

Go to Configure → Sources → Cloud and choose Azure logs via Event Hub. Fill in the fields, tick the data sources you want and save. LogPulse first tests the connection with your credential and tells you what is wrong when it fails.

Tip: Secrets are stored encrypted and never shown again. To change one, paste a new value; the rest stays.

What arrives

Data sourceIndexSourceSourcetypeWhat it holds
Activity Logauditazureazure:activityWho created, changed or deleted what in your subscriptions, with role assignments and policy.
Entra ID sign-insiamazureazure:signinInteractive, non-interactive, service principal and managed identity sign-ins.
Entra ID auditiamazureazure:auditChanges to users, groups, roles, apps and provisioning.
Defender for Cloud alertsauditazureazure:defender:alertAlerts from continuous export for VMs, storage, Key Vault and containers.
Azure Firewallfirewallazureazure:firewallNetwork and application rule hits, NAT, threat intelligence and IDPS.
Web application firewallwebazureazure:wafApplication Gateway and Front Door WAF.
Defender XDR streaming (off by default)endpointazurems_defenderAdvanced hunting tables streamed to the hub.
Application Gateway and Front Door access (off by default)webazureazure:web:accessEvery request through your gateways. High volume.
AKS audit (off by default)kubernetesazureazure:aks:auditKubernetes API audit of your AKS clusters.
Other resource logs (off by default)auditazureazure:resourceKey Vault, Storage, SQL, App Service and every other category.

The index is the kind of data and the source is the product, so detection rules for that kind of data find Azure logs via Event Hub without changes.

Note: The Event Hubs namespace must be on the Standard tier or higher: LogPulse reads the Kafka endpoint, which Basic does not have.
Note: One hub can carry the logs of every subscription. LogPulse works out per record which log it is.
Note: A new consumer group starts at the newest events, not at the start of the retention.

Fields and data models

Data modelWhat you get
Cloud AuditActivity Log, Entra ID audit, AKS audit and resource logs with user, action, object and src_ip.
AuthenticationEntra ID sign-ins with the same fields as the Entra ID connector.
AlertDefender for Cloud alerts in the shared alert shape.
Network ActivityAzure Firewall with the 5-tuple and allowed or blocked.
Web AccessWAF and gateway requests with src_ip, http_method, url, status and http_user_agent.

Events are mapped to the data models above, with the same field names as other sources of the same kind. See field names.

The original record of Azure logs via Event Hub is kept with each event under raw, so nothing the vendor sent is lost.

3. Check the data

The first events arrive within a few minutes. Check what came in:

earliest=-1h index IN (audit, iam, firewall, web, endpoint, kubernetes) source IN (azure)
| stats count by index, source, sourcetype

The integration page in LogPulse shows when it last read events and the last error, if any.

Troubleshooting

The connection test names what is wrong. These are the answers you are most likely to see.

What you seeWhat to do
Azure refused the connection string.Use the connection string of a shared access policy with Listen rights, on the namespace or on the hub.
Name the event hub, or use a connection string of the hub itself (with EntityPath)Fill in the hub name, or copy the connection string from the hub instead of the namespace.
The namespace answered, but has no hub with that name.Check the hub name. A diagnostic setting without a hub name writes to hubs called insights-logs-<category>.
The test passes but no events arrive.Run the search above over a longer time. No results: check on the integration page that the data source is switched on and when it last read events. Azure logs via Event Hub may also simply have had nothing new.

Frequently asked questions

How do I send Azure logs via Event Hub logs to LogPulse?
Add the Azure logs via Event Hub cloud integration under Configure, Sources, Cloud in LogPulse and fill in Connection string. LogPulse reads the Azure Event Hub (Kafka endpoint) itself, every five minutes by default; there is nothing to install.
Which permissions does LogPulse need in Azure logs via Event Hub?
An Event Hubs namespace on the Standard tier or higher (Kafka endpoint). A diagnostic setting per subscription or resource that streams to the hub. A shared access policy with Listen rights. Read access is enough for collecting logs.
Which index does Azure logs via Event Hub data go to?
The index is the kind of data: audit, iam, firewall, web, endpoint, kubernetes. The source is the product: azure. Each data source has its own sourcetype, listed on this page.
Which data models does Azure logs via Event Hub fill?
Cloud Audit, Authentication, Alert, Network Activity, Web Access. Detection rules and dashboards written for a data model work on Azure logs via Event Hub without changes.
Why is a data source off by default?
To keep your daily ingest down until you need it: Defender XDR streaming, Application Gateway and Front Door access, AKS audit, Other resource logs are high volume or of use to fewer teams. Tick the data source in the integration to switch it on.
Does LogPulse read the whole retention of the Event Hub at the start?
No. A new consumer group starts at the newest events. Give LogPulse a consumer group of its own, so it does not take events from another reader.