Data Retention & Storage
How long LogPulse keeps your logs, where they are stored, and what you can do to keep a copy longer.
Overview
LogPulse stores your logs in ClickHouse, a columnar database, and keeps every log searchable for the full retention period of your plan. There are no storage tiers: a log from yesterday and a log from two months ago are in the same place and are searched the same way. When a log is older than your retention, it is deleted.
Retention per plan
Retention is set per organization and follows your plan:
| Plan | Daily ingestion | Retention |
|---|---|---|
| Free | 1 GB/day | 7 days |
| Starter | 1 GB/day | 14 days |
| Pro | 10 GB/day | 60 days |
| Business | 50 GB/day | 90 days |
| Enterprise | Custom | Custom, set in your contract |
Retention applies to all indexes of the organization at once. There is no separate retention per index.
How deletion works
A cleanup job runs once a day and deletes every log whose timestamp is older than the retention of your organization. Logs can therefore stay searchable for up to a day longer than the retention. Deletion is permanent: LogPulse cannot bring deleted logs back.
Audit logs
The audit trail of your organization (sign-ins, user and role changes, API key use, configuration changes) is stored in the index _audit and kept for 180 days on every plan, also when your plan's retention is shorter.
Changing plans
The retention of your current plan counts, also during a trial. When a trial ends or you move to a plan with shorter retention, the next cleanup deletes the logs that are older than the new retention. Export what you want to keep before you downgrade.
Where your logs are stored
| Shared plans | Dedicated environment | |
|---|---|---|
| Location | Scaleway, Amsterdam (NL) | Scaleway, Amsterdam (NL), or as agreed |
| Database | ClickHouse, one table shared by all organizations | ClickHouse of your own |
| Separation | Every row carries your organization id; every query is limited to it | A separate environment |
| Layout | Partitioned by organization and day | Partitioned by organization and day |
Log text and fields are stored compressed (ZSTD). How well that works depends on your logs; repetitive logs such as access logs compress much better than unique text. The Compliance & Trust page lists our subprocessors.
Indexes
An index is a label on every log, used to group and protect logs. You do not create indexes beforehand: send a log with index set and the index exists. Without one, the log goes to main. You use indexes to:
| For | How |
|---|---|
| Searching | index=security narrows a search before it runs |
| Access | Roles limit which indexes a user can search (Settings → Roles) |
| Ingest priority | Critical, Normal or Low per index |
| Volume | Cost Attribution shows the volume per index |
Indexes starting with an underscore, such as _audit and _internal, are filled by LogPulse itself.
Seeing your volume
Cost Attribution in the app shows your log volume and quota use over a period, broken down by index, namespace, source or sourcetype. Settings shows the license usage of the last days. Use them to find the sources that send the most.
Daily quota
| Daily ingestion used | What happens |
|---|---|
| 80% | Email to the owners of the organization |
| 100% | Notification in LogPulse; logs are still accepted |
| 125% | Ingest returns 429 with Retry-After until the quota resets at midnight UTC, and the owners get an email |
The quota counts what you send per day, not what is stored. Retention does not count against it.
Reducing volume
Use a stream pipeline to drop debug logs, sample noisy sources, remove fields you never search or deduplicate repeated events before they are stored. Test it first on the original logs with raw log replay.
Keeping a copy in your own storage
To keep logs longer than your retention, send a copy to your own storage as they come in. A stream pipeline with a streamSink node delivers events to:
| Destination | Format |
|---|---|
| S3-compatible object storage (AWS S3, Scaleway, MinIO) | Gzipped NDJSON, one object per batch, under prefix/YYYY/MM/DD/HH/ |
| HTTP endpoint | NDJSON, or Splunk HEC format, per batch |
The sink can store the event in LogPulse as well, or only forward it. It only sends what comes in after you set it up: logs that are already stored are not exported.
Compliance
| Need | In LogPulse |
|---|---|
| Audit trail | Kept 180 days in the _audit index, on every plan |
| Retention longer than your plan | A higher plan, an Enterprise contract, or a copy to your own storage |
| Personal data in logs | Mask or drop it at ingest with a stream pipeline, so it is never stored |
| Deleting specific logs (GDPR erasure) | There is no self-service delete. Contact support |
See Compliance & Trust for certifications, the DPA and subprocessors.
Not available
To be clear about what LogPulse does not offer today:
| Feature | Status |
|---|---|
| Hot, warm and cold storage tiers | Not used: all data is in one store |
| Retention per index | No: retention is per organization |
| Export of logs that are already stored | No: use a sink for new logs |
| Legal hold | No |
| Rehydrating archived logs | No |