Platform Limits & Quotas

This page documents LogPulse's service level objectives, rate limits, platform quotas, and ingest pipeline architecture. Understanding these boundaries helps you design reliable integrations and plan for production workloads.

Note
The targets listed on this page are engineering targets, not contractual SLAs. Contractual SLAs with financial backing are available with the Enterprise plan.

SLOs & Reliability Targets

LogPulse tracks the following service level objectives internally and aims to meet or exceed these targets for all customers. These targets apply to the shared multi-tenant platform. Enterprise customers with dedicated infrastructure may negotiate custom SLAs.

Current Targets

MetricTargetMeasurementNotes
Platform uptime99.9%Rolling 30-day windowMeasured by the external probes of the status page
Ingest to searchableA few secondsFrom the API response until the log can be foundLonger while storage is under pressure; accepted logs wait in the buffer
Alert evaluationPer ruleEach alert rule runs on its own scheduleThe schedule you set is the delay
Note
These are engineering targets. Contractual SLAs are available with the Enterprise plan.

Status Page

The public status page is live at status.logpulse.io. It shows current platform health and 30 days of history per component, measured every minute by probes that run on Cloudflare rather than on our own platform, so the page keeps answering during an outage. Each check writes a log line and then searches for it, so what it reports is the round trip a customer would experience, not just whether the API is reachable.

Automatic incident emails are not wired up yet, so the page is the place to look. To be notified by hand in the meantime, email [email protected] and we will add you to the list for incidents and planned maintenance.

Ingest Pipeline Architecture

LogPulse puts durability first. The API answers only once your logs are safely in a durable ingest buffer, replicated over several brokers. From there, ingest workers parse and store them in ClickHouse. If storage is slow or briefly down, logs wait in the buffer instead of being lost.

Ingest pipeline flow
Client (HTTP / OTLP / LogPulse agent)
  |
  v
API (Fastify)
  - API key, rate limit and daily quota
  - Validation; request max 10 MB, events over 256 KB truncated
  - Ingest priority: Critical / Normal / Low
  |
  v
Durable ingest buffer
  - Acknowledged by all replicas before the API answers
  - Critical has its own lane and is stored first
  - Keeps the original logs: 60 minutes of raw replay
  |
  v
Ingest workers
  - Parser rules and stream pipelines
  - Merge many small requests per organization into few inserts
  - Retry with backoff; a batch that keeps failing is parked
  |
  v
ClickHouse
  - Partitioned by tenant_id and day
  - Ordered by (tenant_id, index, timestamp, log_id)
  - Queryable right after insert

See Ingest priority and Raw log replay.

Delivery Semantics

LogPulse provides at-least-once delivery. A log the API accepted (HTTP 200 with accepted in the body) is in the durable buffer. A worker marks a batch as done only after it is stored, so a crash or restart offers the batch again. A retried batch carries the same deduplication token, which prevents most duplicates; in rare cases an event can still be stored twice.

Tip
If your application requires deduplication, include a unique identifier in the event attributes (e.g., attributes.request_id). You can then use LPQL to deduplicate at query time: ... | dedup attributes.request_id.

Batching & Flushing

Workers read the buffer in batches and merge the logs of one organization into one insert per second of arrival time. Many small requests therefore become few, larger inserts. Logs are normally searchable within a few seconds of being accepted. Critical logs get an insert slot first.

Failure Handling

ScenarioBehaviorCustomer Impact
ClickHouse slowLogs wait in the buffer; storage status goes to Under pressureLow priority refused with 429; Normal and Critical accepted. Slightly higher ingest latency, no data loss
ClickHouse unavailableBatches stay in the buffer and are offered again until storage is backAccepted logs are delayed, not lost; searches show older data meanwhile
Long outageThe backlog nears the buffer retention (by default after 6 hours of backlog)Storage status Protecting storage: only Critical accepted, others get 429 with Retry-After
A batch keeps failing for another reasonRetried up to 6 times with backoff from 2 s, then parked in a dead-letter queueThose logs are delayed; we replay them once the cause is fixed
Buffer unavailableAPI returns 503 with Retry-AfterNothing accepted; the client retries
Network partitionAPI returns 503 or times outClient should retry; logs already accepted (HTTP 200) are safe
Note
Replaying the dead-letter queue is done by the LogPulse team. Contact support if you suspect logs are stuck there.

Backpressure & Throttling

When storage is under pressure, LogPulse refuses Low priority first, and only Critical when storage must be protected. Refused requests get HTTP 429 with a Retry-After header; the LogPulse agent, Fluent Bit and the OpenTelemetry collector keep the logs and send them again. See Ingest priority.

Daily ingestion quotas are enforced separately. At 80% of the daily quota, organization owners get an email. At 100% you get a notification, but logs are still accepted up to 125%. Beyond that, ingest returns HTTP 429 with a Retry-After until the quota resets at midnight UTC.

Rate Limits & Quotas

Rate limits protect the platform and ensure fair resource sharing between tenants. All rate limits are applied per organization and enforced at the API gateway.

API Rate Limits

EndpointLimitWindowScope
POST /api/v1/logs (ingest)10,000 requests1 minutePer API key
POST /v1/logs (OTLP)10,000 requests1 minutePer API key
POST /api/v1/search/jobs (search)60 requests1 minutePer credential
All other API endpoints600 requests1 minutePer credential (API key, token or session); per IP without one
Sign-in, sign-up, password reset, email verification, invites20 requests15 minutesPer IP address

When a rate limit is exceeded, the API returns HTTP 429 with the following headers:

Rate limit response headers
HTTP/1.1 429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 10000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 12

X-RateLimit-Reset is the number of seconds until the window resets.

Daily Ingestion Quotas

PlanDaily Ingestion Limit
Free1 GB/day
Starter1 GB/day
Pro10 GB/day
Business50 GB/day
EnterpriseCustom

The quota counts the bytes you send per UTC day and works the same on every plan: an email to the owners at 80%, a notification at 100%, logs still accepted up to 125%, and above that HTTP 429 with a Retry-After until midnight UTC. Nothing is queued or thrown away on our side; a client that respects Retry-After keeps the logs and sends them later.

Tip
If you consistently approach your quota, upgrade your plan or use a stream pipeline to drop low-value logs (debug, health checks) before they are stored.

Burst Controls

Rate limits count requests in a fixed one-minute window. There is no separate burst allowance: send many events per request rather than one request per event. A batch of a thousand events costs one request.

Tenant Fairness

Every organization has its own quota, and rate limits apply per credential, so one customer's traffic cannot use up another's limits. Workers slow down their inserts when storage answers slowly, and Critical logs get an insert slot first. Every search runs with a time and memory limit (see below), so a single query cannot take over the database.

Platform Limits

The following hard limits apply to all LogPulse plans. These limits are designed to ensure platform stability and predictable performance.

Event Size Limits

LimitValueNotes
Maximum request body10 MBPer HTTP request; can contain many events
Maximum event size256 KBLarger events are stored, cut to 256 KB; the response says how many in truncatedEvents
Attribute valuesStringsOther types are rejected for that event; the rest of the request is accepted
Index namesNot starting with _Indexes like _audit are reserved for LogPulse

Query Limits

LimitValueNotes
Search time60 secondsA search that runs longer is stopped
Rows per search1,000,000The newest rows are kept
Memory per query5 GBLower on small dedicated environments
Running searches3 per user, 6 per organizationWait for one to finish, or cancel it
Search resultsKept 15 minutesPage through them within that time
Tip
Narrow before the first pipe: index=main earliest=-1h lets the search skip every partition outside that hour and index.

Organization Limits

ResourceFreeProBusinessEnterprise
Team members210UnlimitedUnlimited
API keys52050Unlimited
Alert rules550200Unlimited
Pipelines310100Unlimited
Dashboards120100Unlimited
Saved queries5100500Unlimited
Agents5100UnlimitedUnlimited
Services325200Unlimited
KPIs151501500Unlimited
Organizations per user5555

Cardinality Guidance

Cardinality refers to the number of distinct values a field can have. High-cardinality fields (like request IDs or user session tokens) can degrade query performance and increase storage costs if used improperly. This section provides guidance on managing cardinality in LogPulse.

Best Practices

Field TypeCardinalityRecommendation
source, level, indexLow (< 100)Ideal for filtering and grouping. Use freely in attributes.
service, environment, regionLow-Medium (< 1,000)Good for filtering. Store as attributes.
host, pod, containerMedium (< 10,000)Acceptable as attributes. Avoid grouping by them over large time ranges.
user_id, account_idMedium-High (< 100,000)Store as attributes but avoid aggregating across all values. Use with equality filters.
request_id, trace_id, session_idVery High (> 100,000)Put them in the event text and search them as a term.
Full URLs, stack tracesUnboundedStore in the event body. Never use as attribute values.

As a rule of thumb, keep attributes you group by below 100,000 distinct values per field. Search unique values as terms in the event text rather than as attribute filters.

Warning
The event text has a token index, so searching a unique value as a plain term ("req-abc123") is fast. Attributes have no such index: a filter on a high-cardinality attribute reads every log in the time range. Put IDs you want to search in the event text.
Good vs. bad attribute usage
// Good: low-cardinality attributes for filtering
{
  "event": "User login failed: invalid credentials for [email protected]",
  "level": "warn",
  "source": "auth-service",
  "attributes": {
    "environment": "production",
    "region": "eu-west-1",
    "error_type": "invalid_credentials"
  }
}

// Bad: high-cardinality values as attributes
{
  "event": "Request processed",
  "attributes": {
    "request_id": "550e8400-e29b-41d4-a716-446655440000",  // Too many unique values
    "full_url": "https://api.example.com/users/12345?page=3", // Unbounded
    "stack_trace": "Error: timeout at Object.<anonymous>..."   // Unbounded
  }
}