Send AWS CloudTrail logs to LogPulse

CloudTrail management events, every control-plane call in an AWS account, through the CloudTrail LookupEvents API.

In short
To send AWS CloudTrail logs to LogPulse, add AWS CloudTrail under Configure → Sources → Cloud. LogPulse reads the CloudTrail LookupEvents API itself, without an agent. Events land in the audit index with source aws and fill the Cloud Audit data model.

Overview

How AWS CloudTrail data reaches LogPulse
Vendor
CloudTrail LookupEvents API
Collect
Integration pollerreads new events, maps the fields
Store
index=audit
  1. CloudTrail LookupEvents API to Integration poller
  2. Integration poller to index=audit

LogPulse signs in to AWS CloudTrail with the credential you give it and fetches new events every five minutes by default. You can set the interval per integration, from one minute to 24 hours.

What you need

  • AWS IAM access key (Access Key ID + Secret Access Key)
  • IAM permission: cloudtrail:LookupEvents (and sts:GetCallerIdentity for the connection test)
  • An AWS region (CloudTrail management events are global but queried per region)
  • In LogPulse: a role that may manage API keys. Cloud integrations fall under that right.

1. Create a credential in AWS CloudTrail

Field in LogPulseWhere you find it
AWS RegionThe region to query, such as eu-west-1. Management events are global but queried per region.
Access Key IDAn IAM access key with the cloudtrail:LookupEvents permission (and sts:GetCallerIdentity for the connection test).
Secret Access KeyStored encrypted and never shown again.
Trail ARN (optional)The ARN of one trail, such as arn:aws:cloudtrail:eu-west-1:123456789012:trail/my-trail.

2. Add the integration in LogPulse

Go to Configure → Sources → Cloud and choose AWS CloudTrail. Fill in the fields, tick the data sources you want and save. LogPulse first tests the connection with your credential and tells you what is wrong when it fails.

Tip: Secrets are stored encrypted and never shown again. To change one, paste a new value; the rest stays.

What arrives

Data sourceIndexSourceSourcetypeWhat it holds
Management Eventsauditawsaws:cloudtrail:managementAWS API calls and account activity: IAM, S3, EC2, console logins, and other control-plane operations. Normalized to the Cloud Audit data model.

The index is the kind of data and the source is the product, so detection rules for that kind of data find AWS CloudTrail without changes.

Note: This integration reads management events of one account and region with only an access key. For data events, organization trails or other AWS logs, use the AWS logs in S3 integration: it reads the trail bucket through an SQS queue.

Fields and data models

Data modelWhat you get
Cloud AuditManagement events with action, outcome, provider, user, src_ip, app and region.

Events are mapped to the data models above, with the same field names as other sources of the same kind. See field names.

3. Check the data

The first events arrive within a few minutes. Check what came in:

earliest=-1h index IN (audit) source IN (aws)
| stats count by index, source, sourcetype

The integration page in LogPulse shows when it last read events and the last error, if any.

Troubleshooting

The connection test names what is wrong. These are the answers you are most likely to see.

What you seeWhat to do
Authentication failed: invalid Access Key ID or Secret Access Key.Copy both values again, or create a new access key for the IAM user.
Credentials are valid but lack permission for sts:GetCallerIdentity.Give the key cloudtrail:LookupEvents, and sts:GetCallerIdentity for the connection test.
Events of another region are missing.LookupEvents is queried per region. Add an integration per region, or use the AWS logs in S3 integration with an organization trail.
The test passes but no events arrive.Run the search above over a longer time. No results: check on the integration page that the data source is switched on and when it last read events. AWS CloudTrail may also simply have had nothing new.

Frequently asked questions

How do I send AWS CloudTrail logs to LogPulse?
Add the AWS CloudTrail cloud integration under Configure, Sources, Cloud in LogPulse and fill in AWS Region, Access Key ID, Secret Access Key. LogPulse reads the CloudTrail LookupEvents API itself, every five minutes by default; there is nothing to install.
Which permissions does LogPulse need in AWS CloudTrail?
AWS IAM access key (Access Key ID + Secret Access Key). IAM permission: cloudtrail:LookupEvents (and sts:GetCallerIdentity for the connection test). An AWS region (CloudTrail management events are global but queried per region). Read access is enough for collecting logs.
Which index does AWS CloudTrail data go to?
The index is the kind of data: audit. The source is the product: aws. Each data source has its own sourcetype, listed on this page.
Which data models does AWS CloudTrail fill?
Cloud Audit. Detection rules and dashboards written for a data model work on AWS CloudTrail without changes.