Platform

Data Retention & Storage

How long LogPulse keeps your logs, where they are stored, and what you can do to keep a copy longer.

Overview

LogPulse stores your logs in ClickHouse, a columnar database, and keeps every log searchable for the full retention period of your plan. There are no storage tiers: a log from yesterday and a log from two months ago are in the same place and are searched the same way. When a log is older than your retention, it is deleted.

Retention per plan

Retention is set per organization and follows your plan:

PlanDaily ingestionRetention
Free1 GB/day7 days
Starter1 GB/day14 days
Pro10 GB/day60 days
Business50 GB/day90 days
EnterpriseCustomCustom, set in your contract

Retention applies to all indexes of the organization at once. There is no separate retention per index.

How deletion works

A cleanup job runs once a day and deletes every log whose timestamp is older than the retention of your organization. Logs can therefore stay searchable for up to a day longer than the retention. Deletion is permanent: LogPulse cannot bring deleted logs back.

Audit logs

The audit trail of your organization (sign-ins, user and role changes, API key use, configuration changes) is stored in the index _audit and kept for 180 days on every plan, also when your plan's retention is shorter.

Changing plans

The retention of your current plan counts, also during a trial. When a trial ends or you move to a plan with shorter retention, the next cleanup deletes the logs that are older than the new retention. Export what you want to keep before you downgrade.

Where your logs are stored

Shared plansDedicated environment
LocationScaleway, Amsterdam (NL)Scaleway, Amsterdam (NL), or as agreed
DatabaseClickHouse, one table shared by all organizationsClickHouse of your own
SeparationEvery row carries your organization id; every query is limited to itA separate environment
LayoutPartitioned by organization and dayPartitioned by organization and day

Log text and fields are stored compressed (ZSTD). How well that works depends on your logs; repetitive logs such as access logs compress much better than unique text. The Compliance & Trust page lists our subprocessors.

Indexes

An index is a label on every log, used to group and protect logs. You do not create indexes beforehand: send a log with index set and the index exists. Without one, the log goes to main. You use indexes to:

ForHow
Searchingindex=security narrows a search before it runs
AccessRoles limit which indexes a user can search (Settings → Roles)
Ingest priorityCritical, Normal or Low per index
VolumeCost Attribution shows the volume per index

Indexes starting with an underscore, such as _audit and _internal, are filled by LogPulse itself.

Seeing your volume

Cost Attribution in the app shows your log volume and quota use over a period, broken down by index, namespace, source or sourcetype. Settings shows the license usage of the last days. Use them to find the sources that send the most.

Daily quota

Daily ingestion usedWhat happens
80%Email to the owners of the organization
100%Notification in LogPulse; logs are still accepted
125%Ingest returns 429 with Retry-After until the quota resets at midnight UTC, and the owners get an email

The quota counts what you send per day, not what is stored. Retention does not count against it.

Reducing volume

Use a stream pipeline to drop debug logs, sample noisy sources, remove fields you never search or deduplicate repeated events before they are stored. Test it first on the original logs with raw log replay.

Keeping a copy in your own storage

To keep logs longer than your retention, send a copy to your own storage as they come in. A stream pipeline with a streamSink node delivers events to:

DestinationFormat
S3-compatible object storage (AWS S3, Scaleway, MinIO)Gzipped NDJSON, one object per batch, under prefix/YYYY/MM/DD/HH/
HTTP endpointNDJSON, or Splunk HEC format, per batch

The sink can store the event in LogPulse as well, or only forward it. It only sends what comes in after you set it up: logs that are already stored are not exported.

Compliance

NeedIn LogPulse
Audit trailKept 180 days in the _audit index, on every plan
Retention longer than your planA higher plan, an Enterprise contract, or a copy to your own storage
Personal data in logsMask or drop it at ingest with a stream pipeline, so it is never stored
Deleting specific logs (GDPR erasure)There is no self-service delete. Contact support

See Compliance & Trust for certifications, the DPA and subprocessors.

Not available

To be clear about what LogPulse does not offer today:

FeatureStatus
Hot, warm and cold storage tiersNot used: all data is in one store
Retention per indexNo: retention is per organization
Export of logs that are already storedNo: use a sink for new logs
Legal holdNo
Rehydrating archived logsNo
Note: Need one of these? Tell us at support; it helps us decide what to build next.