Platform Limits & Quotas
This page documents LogPulse's service level objectives, rate limits, platform quotas, and ingest pipeline architecture. Understanding these boundaries helps you design reliable integrations and plan for production workloads.
SLOs & Reliability Targets
LogPulse tracks the following service level objectives internally and aims to meet or exceed these targets for all customers. These targets apply to the shared multi-tenant platform. Enterprise customers with dedicated infrastructure may negotiate custom SLAs.
Current Targets
| Metric | Target | Measurement | Notes |
|---|---|---|---|
| Platform uptime | 99.9% | Rolling 30-day window | Measured by the external probes of the status page |
| Ingest to searchable | A few seconds | From the API response until the log can be found | Longer while storage is under pressure; accepted logs wait in the buffer |
| Alert evaluation | Per rule | Each alert rule runs on its own schedule | The schedule you set is the delay |
Status Page
The public status page is live at status.logpulse.io. It shows current platform health and 30 days of history per component, measured every minute by probes that run on Cloudflare rather than on our own platform, so the page keeps answering during an outage. Each check writes a log line and then searches for it, so what it reports is the round trip a customer would experience, not just whether the API is reachable.
Automatic incident emails are not wired up yet, so the page is the place to look. To be notified by hand in the meantime, email [email protected] and we will add you to the list for incidents and planned maintenance.
Ingest Pipeline Architecture
LogPulse puts durability first. The API answers only once your logs are safely in a durable ingest buffer, replicated over several brokers. From there, ingest workers parse and store them in ClickHouse. If storage is slow or briefly down, logs wait in the buffer instead of being lost.
Client (HTTP / OTLP / LogPulse agent)
|
v
API (Fastify)
- API key, rate limit and daily quota
- Validation; request max 10 MB, events over 256 KB truncated
- Ingest priority: Critical / Normal / Low
|
v
Durable ingest buffer
- Acknowledged by all replicas before the API answers
- Critical has its own lane and is stored first
- Keeps the original logs: 60 minutes of raw replay
|
v
Ingest workers
- Parser rules and stream pipelines
- Merge many small requests per organization into few inserts
- Retry with backoff; a batch that keeps failing is parked
|
v
ClickHouse
- Partitioned by tenant_id and day
- Ordered by (tenant_id, index, timestamp, log_id)
- Queryable right after insertSee Ingest priority and Raw log replay.
Delivery Semantics
LogPulse provides at-least-once delivery. A log the API accepted (HTTP 200 with accepted in the body) is in the durable buffer. A worker marks a batch as done only after it is stored, so a crash or restart offers the batch again. A retried batch carries the same deduplication token, which prevents most duplicates; in rare cases an event can still be stored twice.
attributes.request_id). You can then use LPQL to deduplicate at query time: ... | dedup attributes.request_id.Batching & Flushing
Workers read the buffer in batches and merge the logs of one organization into one insert per second of arrival time. Many small requests therefore become few, larger inserts. Logs are normally searchable within a few seconds of being accepted. Critical logs get an insert slot first.
Failure Handling
| Scenario | Behavior | Customer Impact |
|---|---|---|
| ClickHouse slow | Logs wait in the buffer; storage status goes to Under pressure | Low priority refused with 429; Normal and Critical accepted. Slightly higher ingest latency, no data loss |
| ClickHouse unavailable | Batches stay in the buffer and are offered again until storage is back | Accepted logs are delayed, not lost; searches show older data meanwhile |
| Long outage | The backlog nears the buffer retention (by default after 6 hours of backlog) | Storage status Protecting storage: only Critical accepted, others get 429 with Retry-After |
| A batch keeps failing for another reason | Retried up to 6 times with backoff from 2 s, then parked in a dead-letter queue | Those logs are delayed; we replay them once the cause is fixed |
| Buffer unavailable | API returns 503 with Retry-After | Nothing accepted; the client retries |
| Network partition | API returns 503 or times out | Client should retry; logs already accepted (HTTP 200) are safe |
Backpressure & Throttling
When storage is under pressure, LogPulse refuses Low priority first, and only Critical when storage must be protected. Refused requests get HTTP 429 with a Retry-After header; the LogPulse agent, Fluent Bit and the OpenTelemetry collector keep the logs and send them again. See Ingest priority.
Daily ingestion quotas are enforced separately. At 80% of the daily quota, organization owners get an email. At 100% you get a notification, but logs are still accepted up to 125%. Beyond that, ingest returns HTTP 429 with a Retry-After until the quota resets at midnight UTC.
Rate Limits & Quotas
Rate limits protect the platform and ensure fair resource sharing between tenants. All rate limits are applied per organization and enforced at the API gateway.
API Rate Limits
| Endpoint | Limit | Window | Scope |
|---|---|---|---|
| POST /api/v1/logs (ingest) | 10,000 requests | 1 minute | Per API key |
| POST /v1/logs (OTLP) | 10,000 requests | 1 minute | Per API key |
| POST /api/v1/search/jobs (search) | 60 requests | 1 minute | Per credential |
| All other API endpoints | 600 requests | 1 minute | Per credential (API key, token or session); per IP without one |
| Sign-in, sign-up, password reset, email verification, invites | 20 requests | 15 minutes | Per IP address |
When a rate limit is exceeded, the API returns HTTP 429 with the following headers:
HTTP/1.1 429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 10000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 12X-RateLimit-Reset is the number of seconds until the window resets.
Daily Ingestion Quotas
| Plan | Daily Ingestion Limit |
|---|---|
| Free | 1 GB/day |
| Starter | 1 GB/day |
| Pro | 10 GB/day |
| Business | 50 GB/day |
| Enterprise | Custom |
The quota counts the bytes you send per UTC day and works the same on every plan: an email to the owners at 80%, a notification at 100%, logs still accepted up to 125%, and above that HTTP 429 with a Retry-After until midnight UTC. Nothing is queued or thrown away on our side; a client that respects Retry-After keeps the logs and sends them later.
Burst Controls
Rate limits count requests in a fixed one-minute window. There is no separate burst allowance: send many events per request rather than one request per event. A batch of a thousand events costs one request.
Tenant Fairness
Every organization has its own quota, and rate limits apply per credential, so one customer's traffic cannot use up another's limits. Workers slow down their inserts when storage answers slowly, and Critical logs get an insert slot first. Every search runs with a time and memory limit (see below), so a single query cannot take over the database.
Platform Limits
The following hard limits apply to all LogPulse plans. These limits are designed to ensure platform stability and predictable performance.
Event Size Limits
| Limit | Value | Notes |
|---|---|---|
| Maximum request body | 10 MB | Per HTTP request; can contain many events |
| Maximum event size | 256 KB | Larger events are stored, cut to 256 KB; the response says how many in truncatedEvents |
| Attribute values | Strings | Other types are rejected for that event; the rest of the request is accepted |
| Index names | Not starting with _ | Indexes like _audit are reserved for LogPulse |
Query Limits
| Limit | Value | Notes |
|---|---|---|
| Search time | 60 seconds | A search that runs longer is stopped |
| Rows per search | 1,000,000 | The newest rows are kept |
| Memory per query | 5 GB | Lower on small dedicated environments |
| Running searches | 3 per user, 6 per organization | Wait for one to finish, or cancel it |
| Search results | Kept 15 minutes | Page through them within that time |
index=main earliest=-1h lets the search skip every partition outside that hour and index.Organization Limits
| Resource | Free | Pro | Business | Enterprise |
|---|---|---|---|---|
| Team members | 2 | 10 | Unlimited | Unlimited |
| API keys | 5 | 20 | 50 | Unlimited |
| Alert rules | 5 | 50 | 200 | Unlimited |
| Pipelines | 3 | 10 | 100 | Unlimited |
| Dashboards | 1 | 20 | 100 | Unlimited |
| Saved queries | 5 | 100 | 500 | Unlimited |
| Agents | 5 | 100 | Unlimited | Unlimited |
| Services | 3 | 25 | 200 | Unlimited |
| KPIs | 15 | 150 | 1500 | Unlimited |
| Organizations per user | 5 | 5 | 5 | 5 |
Cardinality Guidance
Cardinality refers to the number of distinct values a field can have. High-cardinality fields (like request IDs or user session tokens) can degrade query performance and increase storage costs if used improperly. This section provides guidance on managing cardinality in LogPulse.
Best Practices
| Field Type | Cardinality | Recommendation |
|---|---|---|
| source, level, index | Low (< 100) | Ideal for filtering and grouping. Use freely in attributes. |
| service, environment, region | Low-Medium (< 1,000) | Good for filtering. Store as attributes. |
| host, pod, container | Medium (< 10,000) | Acceptable as attributes. Avoid grouping by them over large time ranges. |
| user_id, account_id | Medium-High (< 100,000) | Store as attributes but avoid aggregating across all values. Use with equality filters. |
| request_id, trace_id, session_id | Very High (> 100,000) | Put them in the event text and search them as a term. |
| Full URLs, stack traces | Unbounded | Store in the event body. Never use as attribute values. |
As a rule of thumb, keep attributes you group by below 100,000 distinct values per field. Search unique values as terms in the event text rather than as attribute filters.
"req-abc123") is fast. Attributes have no such index: a filter on a high-cardinality attribute reads every log in the time range. Put IDs you want to search in the event text.// Good: low-cardinality attributes for filtering
{
"event": "User login failed: invalid credentials for [email protected]",
"level": "warn",
"source": "auth-service",
"attributes": {
"environment": "production",
"region": "eu-west-1",
"error_type": "invalid_credentials"
}
}
// Bad: high-cardinality values as attributes
{
"event": "Request processed",
"attributes": {
"request_id": "550e8400-e29b-41d4-a716-446655440000", // Too many unique values
"full_url": "https://api.example.com/users/12345?page=3", // Unbounded
"stack_trace": "Error: timeout at Object.<anonymous>..." // Unbounded
}
}