Platform Limits & Quotas
This page documents LogPulse's service level objectives, rate limits, platform quotas, and ingest pipeline architecture. Understanding these boundaries helps you design reliable integrations and plan for production workloads.
SLOs & Reliability Targets
LogPulse tracks the following service level objectives internally and aims to meet or exceed these targets for all customers. These targets apply to the shared multi-tenant platform. Enterprise customers with dedicated infrastructure may negotiate custom SLAs.
Current Targets
| Metric | Target | Measurement | Notes |
|---|---|---|---|
| Platform uptime | 99.9% | Rolling 30-day window | Excludes scheduled maintenance (announced 48h in advance) |
| Ingest latency (P99) | < 500ms | Time from API response to queryable | Measured at the API gateway; does not include network transit to LogPulse |
| Query latency (P95) | < 200ms | Hot tier, simple queries | Complex aggregations or cold tier queries may exceed this target |
| Backup RPO | < 1 hour | Point-in-time recovery | Automated backups to separate availability zone |
| Backup RTO | < 4 hours | Full platform recovery | Target for complete platform restore from backup |
| Alert evaluation | < 2 minutes | From log ingestion to alert firing | Default evaluation interval; configurable per alert rule |
Status Page
A public status page at status.logpulse.io is planned for Q3 2026. The status page will display real-time platform health, historical uptime, ongoing incidents, and scheduled maintenance windows. Subscribe to receive notifications about incidents affecting your region.
Ingest Pipeline Architecture
LogPulse uses a durable, buffered ingest pipeline to ensure log data is safely received and stored, even during transient infrastructure issues. The pipeline is designed to prioritize data durability over raw throughput -- your logs are persisted in a Redis-backed queue before being written to ClickHouse.
Client (HTTP/OTLP)
|
v
API Gateway (Fastify)
- Authentication & rate limiting
- Schema validation (Zod)
- Payload size check (max 10 MB)
|
v
Ingest Queue (BullMQ / Redis)
- Durable, Redis-backed persistence
- At-least-once delivery guarantee
- Automatic retries with exponential backoff
|
v
Batch Writer
- Batches 100-1,000 events per insert
- Flushes every 5 seconds or when batch is full
- Parallel writers for throughput
|
v
ClickHouse
- Partitioned by tenant_id + date
- ZSTD compressed columnar storage
- Queryable immediately after insertDelivery Semantics
LogPulse provides at-least-once delivery. Every log event accepted by the API (HTTP 202 response) is persisted in the Redis-backed ingest queue before the response is sent. If ClickHouse is temporarily unavailable, events remain in the queue and are retried automatically. In rare failure scenarios, an event may be written to ClickHouse more than once.
attributes.request_id). You can then use LPQL to deduplicate at query time: ... | dedup attributes.request_id.Batching & Flushing
The batch writer accumulates incoming events and writes them to ClickHouse in batches of 100 to 1,000 events. A flush is triggered either when the batch reaches the target size or after a 5-second idle timeout, whichever comes first. This balances write efficiency with ingestion latency -- most events become queryable within 5 seconds of being accepted.
| Parameter | Value | Description |
|---|---|---|
| Batch size | 100 - 1,000 events | Adaptive based on event size and throughput |
| Flush interval | 5 seconds | Maximum time before a partial batch is flushed |
| Parallel writers | 4 per node | Concurrent batch writers for higher throughput |
| Max queue depth | 100,000 events | Per-node queue limit before backpressure activates |
Failure Handling
When a batch insert to ClickHouse fails, the batch writer retries with exponential backoff. Failed events are retried up to 5 times with increasing delays (1s, 2s, 4s, 8s, 16s). After all retries are exhausted, events are moved to a dead-letter queue (DLQ) for manual inspection and replay.
| Scenario | Behavior | Customer Impact |
|---|---|---|
| ClickHouse slow (> 5s insert) | Queue depth grows, batch size increases | Slightly higher ingest latency; no data loss |
| ClickHouse unavailable (< 5 min) | Queue buffers events, automatic retry | Events delayed but not lost; queries return stale results |
| ClickHouse unavailable (> 5 min) | Queue fills; oldest events moved to DLQ | API returns 503; events in DLQ are replayed on recovery |
| Redis unavailable | API returns 503 immediately | Client should retry with backoff; no events accepted |
| Network partition | API returns 503 or timeout | Client should retry; accepted events (HTTP 202) are safe |
Backpressure & Throttling
When the ingest queue approaches capacity, LogPulse applies backpressure to protect data durability. At 80% queue depth, a warning is logged internally. At 100% queue depth, the API begins returning HTTP 429 (Too Many Requests) with a Retry-After header indicating when to retry.
Daily ingestion quotas are enforced separately from queue-based backpressure. When your organization reaches 80% of its daily quota, organization Owners receive an email notification. At 100%, ingestion is throttled and excess logs are queued for up to 1 hour. If the quota is not increased, queued logs are discarded.
Rate Limits & Quotas
Rate limits protect the platform and ensure fair resource sharing between tenants. All rate limits are applied per organization and enforced at the API gateway.
API Rate Limits
| Endpoint | Limit | Window | Scope |
|---|---|---|---|
| POST /api/v1/logs (ingest) | 10,000 requests/min | Sliding window | Per organization |
| POST /v1/logs (OTLP) | 10,000 requests/min | Sliding window | Per organization |
| GET /api/v1/logs (query) | 100 requests/min | Sliding window | Per organization |
| Search queries (dashboard) | 10 requests/min | Sliding window | Per organization |
| Auth endpoints | 5 requests/min | Sliding window | Per IP address |
| Public endpoints | 5 requests/min | Sliding window | Per IP address |
When a rate limit is exceeded, the API returns HTTP 429 with the following headers:
HTTP/1.1 429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 10000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1711929600Daily Ingestion Quotas
| Plan | Daily Ingestion Limit | Overage Behavior |
|---|---|---|
| Free | 100 MB/day | Ingestion paused until next day |
| Starter | 1 GB/day | Ingestion paused until next day |
| Pro | 5 GB/day | Warning at 80%; throttled at 100% |
| Business | 25 GB/day | Warning at 80%; throttled at 100%; 1h queue buffer |
| Enterprise | Custom | Custom overage handling; burst allowance negotiable |
Burst Controls
Rate limits use a sliding window algorithm that allows short bursts above the per-minute limit. A burst of up to 2x the per-minute limit is allowed for up to 10 seconds, as long as the rolling 1-minute average stays within the limit. This accommodates deployment spikes and application restarts without dropping logs.
Tenant Fairness
Each organization has independent rate limits and quotas. One organization's traffic cannot starve another organization's resources. Rate limits, query concurrency slots, and ingest queue capacity are allocated per tenant at the application layer. On the storage layer, ClickHouse queries are subject to per-query memory limits and timeouts that prevent any single query from monopolizing cluster resources.
Platform Limits
The following hard limits apply to all LogPulse plans. These limits are designed to ensure platform stability and predictable performance.
Event Size Limits
| Limit | Value | Notes |
|---|---|---|
| Maximum request body | 10 MB | Per HTTP request; can contain multiple events |
| Maximum single event | 1 MB | Events exceeding 1 MB are rejected with HTTP 413 |
| Maximum attribute count | 128 attributes | Per event; additional attributes are silently dropped |
| Maximum attribute key length | 256 characters | Keys longer than 256 chars are truncated |
| Maximum attribute value length | 32 KB | Values longer than 32 KB are truncated |
| Maximum event field (body) | 256 KB | The event/message body field |
Query Limits
| Limit | Value | Notes |
|---|---|---|
| Query timeout | 30 seconds | Queries exceeding 30s are cancelled and return HTTP 408 |
| Result set limit | 1,000 - 10,000 rows | 1,000 default; up to 10,000 with ?limit=10000 |
| Concurrent queries per org | 5 (Pro) / 10 (Business) / Custom (Enterprise) | Queries exceeding concurrency limit are queued for up to 10s |
| Memory per query | 2 GB | Queries exceeding memory limit are cancelled |
| Scan limit | 10 billion rows | Safety limit to prevent runaway full-table scans |
index=main earliest=-1h scans only the last hour of the hot tier, which is significantly faster than unbounded queries.Organization Limits
| Resource | Free | Starter | Pro | Business | Enterprise |
|---|---|---|---|---|---|
| Indexes | 1 | 5 | 25 | 100 | Unlimited |
| Team members | 2 | 3 | 10 | Unlimited | Unlimited |
| API keys | 2 | 5 | 20 | 50 | Unlimited |
| Alert rules | 3 | 10 | 50 | 200 | Unlimited |
| Pipelines | 1 | 3 | 10 | 50 | Unlimited |
| Dashboards | 1 | 5 | 20 | 100 | Unlimited |
| Saved queries | 5 | 20 | 100 | 500 | Unlimited |
| Scheduled exports | 0 | 1 | 5 | 20 | Unlimited |
| Organizations per user | 5 | 5 | 5 | 5 | 5 |
Cardinality Guidance
Cardinality refers to the number of distinct values a field can have. High-cardinality fields (like request IDs or user session tokens) can degrade query performance and increase storage costs if used improperly. This section provides guidance on managing cardinality in LogPulse.
Best Practices
| Field Type | Cardinality | Recommendation |
|---|---|---|
| source, level, index | Low (< 100) | Ideal for filtering and grouping. Use freely in attributes. |
| service, environment, region | Low-Medium (< 1,000) | Good for filtering. Store as attributes. |
| host, pod, container | Medium (< 10,000) | Acceptable as attributes. Avoid using in GROUP BY with large time ranges. |
| user_id, account_id | Medium-High (< 100,000) | Store as attributes but avoid aggregating across all values. Use with equality filters. |
| request_id, trace_id, session_id | Very High (> 100,000) | Store in the event body, not as indexed attributes. Search with full-text match. |
| Full URLs, stack traces | Unbounded | Store in the event body. Never use as attribute values. |
As a rule of thumb, keep indexed attributes below 100,000 distinct values per field. Fields exceeding this threshold should be stored in the event body and searched with full-text queries rather than attribute filters.
event CONTAINS "req-abc123".// Good: low-cardinality attributes for filtering
{
"event": "User login failed: invalid credentials for [email protected]",
"level": "warn",
"source": "auth-service",
"attributes": {
"environment": "production",
"region": "eu-west-1",
"error_type": "invalid_credentials"
}
}
// Bad: high-cardinality values as attributes
{
"event": "Request processed",
"attributes": {
"request_id": "550e8400-e29b-41d4-a716-446655440000", // Too many unique values
"full_url": "https://api.example.com/users/12345?page=3", // Unbounded
"stack_trace": "Error: timeout at Object.<anonymous>..." // Unbounded
}
}