Platform Limits & Quotas

This page documents LogPulse's service level objectives, rate limits, platform quotas, and ingest pipeline architecture. Understanding these boundaries helps you design reliable integrations and plan for production workloads.

Note
LogPulse is currently in Early Access. The targets listed on this page are engineering targets, not contractual SLAs. Contractual SLAs with financial backing will be available with the Enterprise plan at general availability (GA).

SLOs & Reliability Targets

LogPulse tracks the following service level objectives internally and aims to meet or exceed these targets for all customers. These targets apply to the shared multi-tenant platform. Enterprise customers with dedicated infrastructure may negotiate custom SLAs.

Current Targets

MetricTargetMeasurementNotes
Platform uptime99.9%Rolling 30-day windowExcludes scheduled maintenance (announced 48h in advance)
Ingest latency (P99)< 500msTime from API response to queryableMeasured at the API gateway; does not include network transit to LogPulse
Query latency (P95)< 200msHot tier, simple queriesComplex aggregations or cold tier queries may exceed this target
Backup RPO< 1 hourPoint-in-time recoveryAutomated backups to separate availability zone
Backup RTO< 4 hoursFull platform recoveryTarget for complete platform restore from backup
Alert evaluation< 2 minutesFrom log ingestion to alert firingDefault evaluation interval; configurable per alert rule
Note
These are engineering targets during Early Access. We publish internal performance against these targets monthly. Contractual SLAs with credits for missed targets will be available with the Enterprise plan at GA.

Status Page

A public status page at status.logpulse.io is planned for Q3 2026. The status page will display real-time platform health, historical uptime, ongoing incidents, and scheduled maintenance windows. Subscribe to receive notifications about incidents affecting your region.

Ingest Pipeline Architecture

LogPulse uses a durable, buffered ingest pipeline to ensure log data is safely received and stored, even during transient infrastructure issues. The pipeline is designed to prioritize data durability over raw throughput -- your logs are persisted in a Redis-backed queue before being written to ClickHouse.

Ingest pipeline flow
Client (HTTP/OTLP)
  |
  v
API Gateway (Fastify)
  - Authentication & rate limiting
  - Schema validation (Zod)
  - Payload size check (max 10 MB)
  |
  v
Ingest Queue (BullMQ / Redis)
  - Durable, Redis-backed persistence
  - At-least-once delivery guarantee
  - Automatic retries with exponential backoff
  |
  v
Batch Writer
  - Batches 100-1,000 events per insert
  - Flushes every 5 seconds or when batch is full
  - Parallel writers for throughput
  |
  v
ClickHouse
  - Partitioned by tenant_id + date
  - ZSTD compressed columnar storage
  - Queryable immediately after insert

Delivery Semantics

LogPulse provides at-least-once delivery. Every log event accepted by the API (HTTP 202 response) is persisted in the Redis-backed ingest queue before the response is sent. If ClickHouse is temporarily unavailable, events remain in the queue and are retried automatically. In rare failure scenarios, an event may be written to ClickHouse more than once.

Tip
If your application requires deduplication, include a unique identifier in the event attributes (e.g., attributes.request_id). You can then use LPQL to deduplicate at query time: ... | dedup attributes.request_id.

Batching & Flushing

The batch writer accumulates incoming events and writes them to ClickHouse in batches of 100 to 1,000 events. A flush is triggered either when the batch reaches the target size or after a 5-second idle timeout, whichever comes first. This balances write efficiency with ingestion latency -- most events become queryable within 5 seconds of being accepted.

ParameterValueDescription
Batch size100 - 1,000 eventsAdaptive based on event size and throughput
Flush interval5 secondsMaximum time before a partial batch is flushed
Parallel writers4 per nodeConcurrent batch writers for higher throughput
Max queue depth100,000 eventsPer-node queue limit before backpressure activates

Failure Handling

When a batch insert to ClickHouse fails, the batch writer retries with exponential backoff. Failed events are retried up to 5 times with increasing delays (1s, 2s, 4s, 8s, 16s). After all retries are exhausted, events are moved to a dead-letter queue (DLQ) for manual inspection and replay.

ScenarioBehaviorCustomer Impact
ClickHouse slow (> 5s insert)Queue depth grows, batch size increasesSlightly higher ingest latency; no data loss
ClickHouse unavailable (< 5 min)Queue buffers events, automatic retryEvents delayed but not lost; queries return stale results
ClickHouse unavailable (> 5 min)Queue fills; oldest events moved to DLQAPI returns 503; events in DLQ are replayed on recovery
Redis unavailableAPI returns 503 immediatelyClient should retry with backoff; no events accepted
Network partitionAPI returns 503 or timeoutClient should retry; accepted events (HTTP 202) are safe
Note
Dashboard-based DLQ replay is on the roadmap. Currently, DLQ events are automatically replayed when the underlying issue is resolved. Contact support for manual DLQ inspection during extended outages.

Backpressure & Throttling

When the ingest queue approaches capacity, LogPulse applies backpressure to protect data durability. At 80% queue depth, a warning is logged internally. At 100% queue depth, the API begins returning HTTP 429 (Too Many Requests) with a Retry-After header indicating when to retry.

Daily ingestion quotas are enforced separately from queue-based backpressure. When your organization reaches 80% of its daily quota, organization Owners receive an email notification. At 100%, ingestion is throttled and excess logs are queued for up to 1 hour. If the quota is not increased, queued logs are discarded.

Rate Limits & Quotas

Rate limits protect the platform and ensure fair resource sharing between tenants. All rate limits are applied per organization and enforced at the API gateway.

API Rate Limits

EndpointLimitWindowScope
POST /api/v1/logs (ingest)10,000 requests/minSliding windowPer organization
POST /v1/logs (OTLP)10,000 requests/minSliding windowPer organization
GET /api/v1/logs (query)100 requests/minSliding windowPer organization
Search queries (dashboard)10 requests/minSliding windowPer organization
Auth endpoints5 requests/minSliding windowPer IP address
Public endpoints5 requests/minSliding windowPer IP address

When a rate limit is exceeded, the API returns HTTP 429 with the following headers:

Rate limit response headers
HTTP/1.1 429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 10000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1711929600

Daily Ingestion Quotas

PlanDaily Ingestion LimitOverage Behavior
Free100 MB/dayIngestion paused until next day
Starter1 GB/dayIngestion paused until next day
Pro5 GB/dayWarning at 80%; throttled at 100%
Business25 GB/dayWarning at 80%; throttled at 100%; 1h queue buffer
EnterpriseCustomCustom overage handling; burst allowance negotiable
Tip
Daily quotas reset at midnight UTC. If you consistently approach your quota limit, consider upgrading your plan or using pipelines to filter out low-value logs (debug, health checks) before they consume quota.

Burst Controls

Rate limits use a sliding window algorithm that allows short bursts above the per-minute limit. A burst of up to 2x the per-minute limit is allowed for up to 10 seconds, as long as the rolling 1-minute average stays within the limit. This accommodates deployment spikes and application restarts without dropping logs.

Tenant Fairness

Each organization has independent rate limits and quotas. One organization's traffic cannot starve another organization's resources. Rate limits, query concurrency slots, and ingest queue capacity are allocated per tenant at the application layer. On the storage layer, ClickHouse queries are subject to per-query memory limits and timeouts that prevent any single query from monopolizing cluster resources.

Platform Limits

The following hard limits apply to all LogPulse plans. These limits are designed to ensure platform stability and predictable performance.

Event Size Limits

LimitValueNotes
Maximum request body10 MBPer HTTP request; can contain multiple events
Maximum single event1 MBEvents exceeding 1 MB are rejected with HTTP 413
Maximum attribute count128 attributesPer event; additional attributes are silently dropped
Maximum attribute key length256 charactersKeys longer than 256 chars are truncated
Maximum attribute value length32 KBValues longer than 32 KB are truncated
Maximum event field (body)256 KBThe event/message body field

Query Limits

LimitValueNotes
Query timeout30 secondsQueries exceeding 30s are cancelled and return HTTP 408
Result set limit1,000 - 10,000 rows1,000 default; up to 10,000 with ?limit=10000
Concurrent queries per org5 (Pro) / 10 (Business) / Custom (Enterprise)Queries exceeding concurrency limit are queued for up to 10s
Memory per query2 GBQueries exceeding memory limit are cancelled
Scan limit10 billion rowsSafety limit to prevent runaway full-table scans
Tip
Use time bounds and partition-aligned filters in your LPQL queries to keep scans efficient. For example, index=main earliest=-1h scans only the last hour of the hot tier, which is significantly faster than unbounded queries.

Organization Limits

ResourceFreeStarterProBusinessEnterprise
Indexes1525100Unlimited
Team members2310UnlimitedUnlimited
API keys252050Unlimited
Alert rules31050200Unlimited
Pipelines131050Unlimited
Dashboards1520100Unlimited
Saved queries520100500Unlimited
Scheduled exports01520Unlimited
Organizations per user55555

Cardinality Guidance

Cardinality refers to the number of distinct values a field can have. High-cardinality fields (like request IDs or user session tokens) can degrade query performance and increase storage costs if used improperly. This section provides guidance on managing cardinality in LogPulse.

Best Practices

Field TypeCardinalityRecommendation
source, level, indexLow (< 100)Ideal for filtering and grouping. Use freely in attributes.
service, environment, regionLow-Medium (< 1,000)Good for filtering. Store as attributes.
host, pod, containerMedium (< 10,000)Acceptable as attributes. Avoid using in GROUP BY with large time ranges.
user_id, account_idMedium-High (< 100,000)Store as attributes but avoid aggregating across all values. Use with equality filters.
request_id, trace_id, session_idVery High (> 100,000)Store in the event body, not as indexed attributes. Search with full-text match.
Full URLs, stack tracesUnboundedStore in the event body. Never use as attribute values.

As a rule of thumb, keep indexed attributes below 100,000 distinct values per field. Fields exceeding this threshold should be stored in the event body and searched with full-text queries rather than attribute filters.

Warning
Storing high-cardinality values (like UUIDs) as indexed attributes inflates ClickHouse's skip indexes and increases merge overhead. This can degrade query performance for all users on the same cluster. If you need to search by request ID, include it in the event body and use event CONTAINS "req-abc123".
Good vs. bad attribute usage
// Good: low-cardinality attributes for filtering
{
  "event": "User login failed: invalid credentials for [email protected]",
  "level": "warn",
  "source": "auth-service",
  "attributes": {
    "environment": "production",
    "region": "eu-west-1",
    "error_type": "invalid_credentials"
  }
}

// Bad: high-cardinality values as attributes
{
  "event": "Request processed",
  "attributes": {
    "request_id": "550e8400-e29b-41d4-a716-446655440000",  // Too many unique values
    "full_url": "https://api.example.com/users/12345?page=3", // Unbounded
    "stack_trace": "Error: timeout at Object.<anonymous>..."   // Unbounded
  }
}

We use cookies to analyze site traffic and improve your experience. No cookies are placed without your consent. Privacy Policy