4.8 TB of logs a day, measured for 24 hours
We aimed for 5 TB a day on a high-availability setup at Scaleway in Amsterdam. This page shows what we measured, how we measured it, what did not go well and the raw data to check it yourself.
In short
LogPulse processed 10.64 billion events in 24 hours, about 4.8 TB of logs a day, on a high-availability setup of 8 nodes at Scaleway in Amsterdam, while 25 analysts searched and 200 detection rules ran. There were no server errors and no lost events, and new data was searchable within 3.8 seconds (p95). We measured this ourselves in November 2026. The raw data is on this page.
What we measured
24 hours, 1,440 minutes, from 14 November 2026 21:36 UTC.
Volume
4.8 TB/day
10.64 billion events in 24 hours, on average 55.4 MB/s. Peak 73 MB/s and 163,000 events per second.
Errors
0
No HTTP 5xx, no pod restarts and no lost events in the 24 hours.
Searchable after (p95)
≤ 3.8 s
Time from sending an event to finding it in search, in the slowest minute. Usually 2.4 s.
Compression
9.4×
8.2 TB of raw logs took 911 GB in ClickHouse (12.45 billion rows), on our generated data mix.
Attack to notable (median)
6 min
58 measured attack scenarios with rules running every 5 minutes. 90% within 11 minutes, the slowest after 16.
Load next to ingest
25 + 200
25 analysts searched, 2 dashboards refreshed every minute and 200 detection rules ran every 5 minutes.
How we tested
A production setup with high availability, the real network path and a workload like that of a SOC.
Setup
- ClickHouse
- 2 × 32 vCPU / 128 GB (POP2-32C-128G), 1 shard with 2 replicas
- Application
- 3 × 16 vCPU / 32 GB (POP2-HC-16C-32G), autoscaling 3 to 8 nodes. API 11 to 16 pods, 12 workers
- Kafka
- 3 × 8 vCPU / 32 GB (POP2-8C-32G), 12 hours retention
- Disks
- ClickHouse 2 × 1 TB block storage at 15K IOPS, Kafka 3 × 400 GB
- Other
- Managed Postgres with HA, Kubernetes (Scaleway Kapsule), load balancer LB-GP-L
- Load generators
- 2 × 16 vCPU / 32 GB in a separate node pool, not part of the measured setup
Scaleway, Amsterdam (nl-ams-3), November 2026. The same Helm chart and containers as LogPulse production and on-premises.
Workload
Data mix
Seven log types by volume: firewall 35%, DNS 15%, Windows 15%, network 10%, Linux 10%, web 10% and email 5%. Generated events with realistic fields, no customer data.
Real network path
The generators send to the public API host name over TLS, through the load balancer and the ingress, like a customer does. 30% of the requests are gzip-compressed.
Day curve
The target follows a day curve from about 40 MB/s at night to 75 MB/s during the day.
25 analysts and 2 dashboards
Simulated analysts search during office hours, from the last 15 minutes up to the last 24 hours. Two dashboards of 8 panels refresh every minute.
200 detection rules
10 rules for the attack scenarios plus managed rules from the LogPulse hub on the same indexes, 200 in total, each running every 5 minutes.
Attack scenarios
Every 10 minutes a known attack goes into the data: brute force on Windows and SSH, a port scan, a DNS tunnel, encoded PowerShell, a known bad IP, SQL injection, an ISO attachment, a sudo shell and exfiltration. A watcher measures the time until the notable.
Measurement window: 24 hours without intervention, from 14 November 2026 21:36 UTC to 15 November 2026 21:36 UTC. Before the window we upgraded the platform under load and restarted a ClickHouse replica and a Kafka broker. None of that lost an event. Those tests are not part of the 24-hour figures.
How each figure is measured
A reporter in the stack wrote one line per minute with every measurement. These are the definitions.
- Events and MB sent
- Counted by the load generators, uncompressed size before gzip.
- Errors
- HTTP status codes per second at the generators and at ingress-nginx, pod restarts from Kubernetes. The retry counter of the generators stayed the same during the whole window.
- Ingest latency
- Time from sending a batch to the response, p95 per minute, measured at the client. On average 0.9 s, slowest minute 2.3 s. The API itself answers in 30 to 50 ms; the rest is receiving the body at the ingress.
- Searchable after
- A probe writes a marked event and searches until it finds it. p95 per minute.
- Rules late
- The number of enabled rules whose next run is overdue, per minute.
- Attack to notable
- Time from injecting a scenario to the notable in the database, measured by a watcher per scenario.
- Storage
- Rows and compressed and uncompressed bytes of the log table from the ClickHouse system tables, at the end of the test.
What did not go well
These are the weak points of this test and what we did about them.
4.8 TB a day on average, not 5
During the day ingest stayed around 68 of the 75 MB/s. The ingress wrote every request of about 450 KB to disk first. That is fixed (the body now stays in memory and the ingress runs on several nodes), but not yet measured again.
Detection briefly late in 34 minutes
Rules of one organisation ran one after another. With 200 rules every 5 minutes that is right on the edge, so in 34 of the 1,440 minutes part of the rules (at most 108) ran late, each time back to 0 within minutes. Rules now run in parallel. That too still needs a new measurement.
Generated data
The events come from templates. Real logs vary more and usually compress less than 9.4×. Plan storage on your own logs.
2 days of retention
The test disks held about 2 days. At this load ClickHouse grows about 510 GB per day per replica, so longer retention needs more disk or tiering to object storage.
Restore not tested
We tested failover under load, not restoring a backup. That follows in the next run.
Half of the attack injections failed in the test tool
Every second injection broke off with a connection error in our own simulation tool. The detection times only count scenarios whose injection succeeded.
Our own test
We ran and measured this test ourselves, on one organisation and one stack. It is not an independent audit. That is why the raw minute data is downloadable and why we repeat the measurement in a pilot.
What this setup costs
Scaleway list prices of 14 November 2026, excluding VAT, at 730 hours a month. Infrastructure only, without the LogPulse licence.
As tested, 2 days of retention
~€4,000/month
€3,950 to €4,150. Postgres, control plane and load balancer are estimated.
30 days on fast disk
~€7,800/month
About 15 TB per replica on block storage at 15K IOPS.
30 days with tiering
~€4,900/month
7 days on fast disk, the rest on Object Storage. Searching older data is slower.
Sources: Scaleway Instances, Scaleway Storage.
Check it yourself
The raw minute data is public. Every figure on this page can be recalculated from it.
Minute data (CSV, 320 KB)
1,745 minutes with 30 measurements each, including the build-up before the window. The column in_window marks the 24 hours.
Summary (JSON)
The window figures as calculated from the CSV.
The load generators, the SOC simulation and the reporter are part of the LogPulse Helm chart as a soak test mode. Every figure of the reporter comes from a Prometheus or ClickHouse query in that code. Enterprise customers and pilot participants get the same tooling.
Want to know what LogPulse does on your hardware and with your logs? In a pilot we run the same test together, with your log mix and your retention.
Questions
About the test and what it means for you.
How many logs can LogPulse handle per day?
In our 24-hour test a high-availability setup of 8 nodes (2 ClickHouse, 3 application, 3 Kafka) processed 10.64 billion events, about 4.8 TB a day, without server errors or lost events. More nodes give more capacity. We size every Enterprise stack on its own volume.
Is this test representative of real use?
It comes close: the traffic used the public network path, analysts searched, dashboards refreshed and 200 rules ran. The data was generated, though, and it was one organisation. Real logs compress differently, so we measure again on your own logs in a pilot.
How does this compare with Splunk?
We did not run Splunk in the same test, so we do not publish a performance comparison. Splunk Cloud is priced on ingest volume; a reseller price document in the UK G-Cloud 15 catalogue (January 2026) lists $764.75 per GB per day per year at 5,000 to 9,999 GB a day. LogPulse Enterprise is priced per agreement. Ask us for a quote at your volume.
What do I need for 5 TB a day?
Start from the tested setup: 2 database nodes of 32 vCPU and 128 GB, 3 to 8 application nodes of 16 vCPU and 32 GB and 3 Kafka nodes of 8 vCPU and 32 GB, plus disk for your retention (about 510 GB per day per replica with this mix). The on-premises page has the profiles.
Was the test independently verified?
No. We ran it ourselves and publish the raw data so anyone can recalculate the figures. In a pilot you measure along with us.
Measure it on your own logs
We run the same test in a pilot, on your own hardware or as a dedicated stack in our EU cloud.
Measured by LayerCares B.V. on 14 and 15 November 2026 under the conditions on this page. Results depend on hardware, log mix, retention and configuration and are no guarantee of performance. Splunk is a trademark of its owner; LogPulse is not affiliated with Splunk. LogPulse is a trade name of LayerCares B.V.