Send Cisco Umbrella and Secure Access logs to LogPulse

Cisco Umbrella and Cisco Secure Access logs from the S3 bucket they are written to: DNS, web proxy, cloud firewall, intrusion prevention and the admin audit log.

In short
To send Cisco Umbrella and Secure Access logs to LogPulse, add Cisco Umbrella and Secure Access under Configure → Sources → Cloud. LogPulse reads the S3 log bucket (Cisco-managed or your own) itself, without an agent. Events land in the dns, web, firewall, network, audit indexes with source cisco_umbrella and fill the Network Resolution (DNS), Web, Network Traffic, Intrusion Detection, Change data models.

Overview

How Cisco Umbrella and Secure Access data reaches LogPulse
Vendor
S3 log bucket (Cisco-managed or your own)
Collect
Integration pollerreads new events, maps the fields
Store
index=dns
index=web
index=firewall
index=network
index=audit
  1. S3 log bucket (Cisco-managed or your own) to Integration poller
  2. Integration poller to index=dns
  3. Integration poller to index=web
  4. Integration poller to index=firewall
  5. Integration poller to index=network
  6. Integration poller to index=audit

LogPulse signs in to Cisco Umbrella and Secure Access with the credential you give it and fetches new events every five minutes by default. You can set the interval per integration, from one minute to 24 hours.

What you need

  • Logging to S3 turned on in Log Management
  • An access key that can list and read the bucket
  • In LogPulse: a role that may manage API keys. Cloud integrations fall under that right.

1. Create a credential in Cisco Umbrella and Secure Access

Field in LogPulseWhere you find it
S3 bucketIn Umbrella or Secure Access: Admin, Log Management. A Cisco-managed bucket or your own.
RegionThe AWS region of the bucket, shown next to it in Log Management.
Data path (optional)The folder before dnslogs/ and proxylogs/. Leave empty when the logs are at the root of your own bucket.
Access key IDFor a Cisco-managed bucket Cisco shows the key once when you set it up. For your own bucket: an IAM key with s3:ListBucket and s3:GetObject.
Secret access keyStored encrypted and never shown again.

2. Add the integration in LogPulse

Go to Configure → Sources → Cloud and choose Cisco Umbrella and Secure Access. Fill in the fields, tick the data sources you want and save. LogPulse first tests the connection with your credential and tells you what is wrong when it fails.

Tip: Secrets are stored encrypted and never shown again. To change one, paste a new value; the rest stays.

What arrives

Data sourceIndexSourceSourcetypeWhat it holds
DNSdnscisco_umbrellacisco:cloud_security:dnsEvery DNS query: identity, internal address, domain, category and whether it was blocked.
Web proxywebcisco_umbrellacisco:cloud_security:proxyWeb requests through the secure web gateway: URL, category, verdict and file scans.
Cloud firewallfirewallcisco_umbrellacisco:cloud_security:firewallConnections through the cloud firewall: source, destination, ports and verdict.
Intrusion preventionnetworkcisco_umbrellacisco:cloud_security:intrusionSignatures the IPS matched, with severity, CVEs and the action taken.
Admin audit logauditcisco_umbrellacisco:cloud_security:auditChanges in the dashboard: who changed which policy or setting, before and after.

The index is the kind of data and the source is the product, so detection rules for that kind of data find Cisco Umbrella and Secure Access without changes.

Fields and data models

Data modelWhat you get
Network Resolution (DNS)Queries with identity, domain, category and verdict.
WebRequests with URL, category, verdict and file scan results.
Network TrafficFirewall connections with source, destination, ports and verdict.
Intrusion DetectionIPS signatures with severity and CVEs.
ChangeDashboard changes with before and after.

Field names follow the CIM field names. These are the fields LogPulse sets on Cisco Umbrella and Secure Access events; a field is left out when Cisco Umbrella and Secure Access did not send a value.

  • action
  • blocked_category
  • bytes
  • bytes_in
  • bytes_out
  • category
  • cve
  • dest
  • dest_host
  • dest_port
  • direction
  • file_hash
  • http_content_type
  • http_method
  • http_referrer
  • http_user_agent
  • id
  • identities
  • object_category
  • query
  • record_type
  • reply_code
  • result
  • rule_id
  • severity
  • signature
  • signature_id
  • src
  • src_port
  • src_translated_ip
  • status
  • transport
  • url
  • user
  • user_type
  • vendor_action

The original record of Cisco Umbrella and Secure Access is kept with each event under raw, so nothing the vendor sent is lost.

3. Check the data

The first events arrive within a few minutes. Check what came in:

earliest=-1h index IN (dns, web, firewall, network, audit) source IN (cisco_umbrella)
| stats count by index, source, sourcetype

The integration page in LogPulse shows when it last read events and the last error, if any.

Troubleshooting

The connection test names what is wrong. These are the answers you are most likely to see.

What you seeWhat to do
AWS refused the key. Check the access key and that it may list this bucket and prefix.For a Cisco-managed bucket, rotate the key in Log Management and paste the new one. For your own bucket, give the IAM key s3:ListBucket and s3:GetObject.
The bucket is not in eu-central-1.Fill in the region shown next to the bucket in Log Management.
No log folders (dnslogs, proxylogs, …) under the bucket root yet.Check the data path: the folder before dnslogs/ and proxylogs/. Cisco writes a file every ten minutes per kind of log, so right after turning logging on there is nothing yet.
The test passes but no events arrive.Run the search above over a longer time. No results: check on the integration page that the data source is switched on and when it last read events. Cisco Umbrella and Secure Access may also simply have had nothing new.

Frequently asked questions

How do I send Cisco Umbrella and Secure Access logs to LogPulse?
Add the Cisco Umbrella and Secure Access cloud integration under Configure, Sources, Cloud in LogPulse and fill in S3 bucket, Region, Access key ID, Secret access key. LogPulse reads the S3 log bucket (Cisco-managed or your own) itself, every five minutes by default; there is nothing to install.
Which permissions does LogPulse need in Cisco Umbrella and Secure Access?
Logging to S3 turned on in Log Management. An access key that can list and read the bucket. Read access is enough for collecting logs.
Which index does Cisco Umbrella and Secure Access data go to?
The index is the kind of data: dns, web, firewall, network, audit. The source is the product: cisco_umbrella. Each data source has its own sourcetype, listed on this page.
Which data models does Cisco Umbrella and Secure Access fill?
Network Resolution (DNS), Web, Network Traffic, Intrusion Detection, Change. Detection rules and dashboards written for a data model work on Cisco Umbrella and Secure Access without changes.
Does LogPulse delete the Umbrella log files?
No. LogPulse lists and reads the files and never deletes them. Cisco or your own bucket lifecycle decides how long they stay.