Building Pipelines

LogPulse Pipelines let you build data and response workflows in a visual drag-and-drop editor. 27 node types in 7 categories fetch data from external sources, transform and enrich it, control the flow, load results into LogPulse and run response actions. Most steps need no code; JSONata and JavaScript are there when you need them.

Overview

The editor is a visual canvas built on React Flow. Drag steps from the palette, connect them and configure each one in its panel. A pipeline is a data, action or stream pipeline; the kind decides which steps you can use and what can start it.

Visual Drag & Drop

Build pipelines visually by dragging nodes from the palette and connecting them with edges.

27 Node Types

Extract, transform, route, load and respond with purpose-built steps in 7 categories.

Real-Time Execution

Watch pipelines execute in real-time with per-node status, timing, and payload inspection.

Version History

You edit a draft. Promoting it creates the version that runs, and you can roll back to an earlier version at any time.

Getting Started

Follow these steps to create your first pipeline:

1

Create a Pipeline

Navigate to Pipelines in the sidebar and click "New Pipeline". Give it a name and optional description.

2

Add a Start Node

A start node is automatically added. This is the entry point of your pipeline.

3

Build Your Flow

Drag nodes from the palette on the left and connect them by drawing edges between handles.

4

Configure Nodes

Click any node to open its configuration panel. Set URLs, mappings, conditions, and more.

5

Test & Activate

Run a test with a test event and inspect each step. Then promote the draft and switch the pipeline to active.

Tip: Use the snap-to-grid feature (enabled by default) to keep your pipeline visually organized. The grid snaps at 20px intervals.

Pipeline Kinds

You pick the kind when you create a pipeline. The kind decides which nodes you can use and what can start the pipeline.

KindWhat it is forRunsOnly this kind has
dataFetch, transform and write data into an index (ETL)Schedule, webhook or API key, manually, from another pipeline—
actionSOAR responses: approvals and response actionsLike data, plus alerts, KPIs, SLOs, health checks and notableswaitForApproval, responseAction
streamShape events inside ingest, between the parser rules and storageOnce per event, with no runs or run historystreamSample, streamDedupe, streamDrop, streamSink

A stream pipeline only allows steps without side effects and without waiting: start, transform, condition, redactMask and the four stream nodes. Instead of a run history it shows counters and samples per step.

Node Categories

Nodes are organized into 7 color-coded categories in the palette. Which nodes you can use depends on the kind of pipeline.

CategoryColorNode TypesDescription
TriggerEmeraldstartEntry point of every pipeline
ExtractBluehttpRequest, splunkSearch, lpqlSearchPull data from external APIs, Splunk or LogPulse itself
TransformViolettransform, json, csv, redactMask, mapCommonSchema, lookup, aiAgentModify, parse, mask, map and enrich data
Flow ControlAmbercondition, split, join, wait, callPipeline, streamSample*, streamDedupe*Branch, fan out and recombine, pause, or hand off to another pipeline
LoadCyanlogpulseIngest, upsertEntities, respondWebhook, streamDrop*, streamSink*Write events or entities, answer a webhook, or end a stream path
UtilitySlatelog, errorDebug logging and stopping a run with an error
RespondRosewaitForApproval†, responseAction†Approval gates and SOAR response actions

* Stream pipelines only. † Action pipelines only.

Drag any node from the palette onto the canvas to add it. Connect nodes by clicking an output handle (right side) and dragging to an input handle (left side).

Trigger Nodes

Every pipeline starts with a trigger node that defines how and when the pipeline executes.

startStart Node

The start node is the entry point and holds the test event for test runs. Every pipeline has exactly one. How the pipeline starts is set in the pipeline settings, not on the node.

A pipeline can be started in these ways:

ModeDescriptionConfiguration
ScheduledRuns on a cron schedule; the input is empty, so the pipeline fetches its own dataCron expression (5 or 6 fields) + timezone, default UTC
HTTP ingestionRuns on every request to its webhook URL or on events sent with a linked API key; the posted body is the inputWebhook token, or link the pipeline to an API key
ManualRun now in the editor or through the APIA promoted version
Alerts and automationAction pipelines only: started by an alert, KPI, SLO or health check route, or by an automation trigger on notablesA notification route or automation trigger that points at the pipeline
Other pipelinesA Call Pipeline step or a run_pipeline response actionThe calling pipeline
Note: Only the production version runs. A schedule or webhook set in the draft does nothing until you promote it, and a paused pipeline is skipped by every trigger.

Extract Nodes

Extract nodes pull data from external sources into your pipeline. They support dynamic configuration, authentication, and retry logic.

HTTP Request

httpRequest

Make HTTP requests to any external API. Supports all standard methods, custom headers, query parameters, request bodies, and advanced retry/redirect handling. All string fields accept Dynamic Values for runtime interpolation.

SettingTypeDescription
URLDynamicValueTarget URL for the request
MethodDynamicValueGET, POST, PUT, PATCH, or DELETE
HeadersKey-Value pairsCustom request headers (DynamicValue per header)
Query ParamsKey-Value pairsURL query parameters (DynamicValue per param)
BodyDynamicValueRequest body (for POST/PUT/PATCH)
TimeoutNumberRequest timeout in ms (default: 30000)
RetriesNumberNumber of retry attempts on failure
Retry DelayNumberDelay between retries in ms
Follow RedirectsBooleanFollow HTTP redirects (default: true)
Max RedirectsNumberMaximum redirects to follow (default: 5)
Validate SSLBooleanValidate SSL certificates (default: true)
Output PathStringWhere to store the response in the payload
Warning: HTTP requests are protected against SSRF attacks. Requests to private IP ranges, localhost, and cloud metadata endpoints are automatically blocked.

Authentication

Both HTTP Request and Splunk Search nodes support multiple authentication methods. Credentials can use Dynamic Values to reference environment variables securely.

Auth TypeFieldsDescription
None—No authentication
BasicUsername, PasswordHTTP Basic authentication (Base64 encoded)
BearerTokenBearer token in Authorization header
API KeyHeader Name, ValueCustom header with API key (default: X-API-Key)
Multi-StepSteps[], Token RulesSequential auth flow for OAuth/complex scenarios

Multi-step authentication enables complex OAuth and token-based flows. Define sequential steps that extract values from responses and use them in subsequent requests:

Multi-Step Auth Example
Step 1: POST /auth/login
  Body: { "username": "{{envVar:API_USER}}", "password": "{{envVar:API_PASS}}" }
  Extract: body → $.access_token → accessToken

Step 2: GET /api/data
  Header: Authorization = Bearer {{accessToken}}
  Extract: body → $.results → results

Each auth step can extract values from three sources:

SourceDescriptionExample
bodyExtract from JSON response body$.access_token
headerExtract from response headerX-Session-Id
cookieExtract from response cookiesession_cookie

Transform Nodes

Transform nodes modify, parse, enrich, and reshape data as it flows through the pipeline. They range from simple field operations to advanced JavaScript transformations and schema mapping.

Transform

transform

The general-purpose transform node supports both a simple mapping mode and an advanced JavaScript mode for complex transformations.

Simple Mode

In simple mode the node runs a list of operations in order; later operations see what earlier ones changed. Values can come from these sources:

Value TypeDescriptionExample
stringText, with {{ }} templates"{{ payload.user.name }}"
numberNumeric value42
booleanTrue or falsetrue
jsonataA JSONata expression on the message$count(payload.items)
messageCopy the value at a path in the messagepayload.user.id
pipelineContext / tenantContextRead a key from the context storecursor.lastSeenId
variableA pipeline or organization variable (older form)${pipeline.variables.api_url}
expressionSandboxed JavaScript (older form)data.level.toUpperCase()

Advanced Mode (JavaScript)

In advanced mode, write JavaScript that receives the input payload and returns the result. It runs in a sandbox, and data, payload, input and $ all point to the input payload.

JavaScript Transform Example
// Variables: data, payload, $ (all reference input payload)
const enriched = {
  ...data,
  processed: true,
  timestamp: new Date().toISOString(),
  severity: data.level === 'error' ? 'high' : 'normal',
  tags: [...(data.tags || []), 'processed']
};
return enriched;

Transform Operations

In simple mode the Transform node runs these operations in order:

OperationFieldsDescription
settarget, value, scopeWrite a value to a path. Scope message (default), pipelineContext or tenantContext
deletetargets[], scopeRemove one or more paths
replacetarget, from, toReplace text inside a string field
movefrom, toMove (rename) a field
keeptargets[]Keep only these paths and drop everything else
coercetarget, targetTypeConvert to string, number, boolean or array
extractsource, target, modeParse a string field with a regex, a delimiter (split) or key=value pairs (kv)
Transform Operations Example
Operation 1: set     → target: "env", value: "production" (string)
Operation 2: move    → from: "msg", to: "message"
Operation 3: delete  → targets: ["debug_info"]
Operation 4: coerce  → target: "status_code", targetType: number

JSON & CSV Parsing

jsoncsv

Dedicated nodes for parsing and formatting structured data formats. Use these when you need to convert between strings and structured objects.

JSON Node

ModeDescription
parseConvert a JSON string to an object. Specify input path (source string) and output path.
stringifyConvert an object to a JSON string. Specify input path (source object) and output path.

CSV Node

SettingDescription
Modeparse (CSV → array of objects) or format (objects → CSV string)
DelimiterField separator character (default: comma)
Has HeadersFirst row contains field names (default: true)
Input PathPath to the CSV string or array in payload
Output PathWhere to store the parsed/formatted result

Redact / Mask

redactMask

Automatically detect and mask personally identifiable information (PII) and sensitive data before it reaches storage. Supports 8 built-in detection patterns and custom regex rules.

Built-in Patterns

PatternDefault ActionExample
EmailReplace → [EMAIL][email protected] → [EMAIL]
PhoneReplace → [PHONE]+1-555-0123 → [PHONE]
Credit CardMask (keep last 4)4111-1111-1111-1234 → ****-****-****-1234
SSNReplace → ***-**-****123-45-6789 → ***-**-****
IP AddressReplace → [IP]192.168.1.1 → [IP]
JWT TokenReplace → [JWT]eyJhbGciOi... → [JWT]
API KeyReplace → [API_KEY]sk-abc123... → [API_KEY]
IBANMask (keep first 4)NL91ABNA0417164300 → NL91**************

Redaction Actions

ActionDescriptionOptions
maskReplace characters with mask charactermaskChar (default: *), style: fixed / keepLastN / keepFirstN, keepN
hashSHA-256 hash of matched valueIrreversible, consistent hash
removeDelete the field entirely—
replaceReplace with custom textCustom replacement string

Custom Rules

Define your own detection rules using regex patterns or field path matching for data types not covered by the built-in presets.

Match TypeDescriptionExample
regexRegular expression pattern matching/\b[A-Z]{2}\d{6}\b/ (passport numbers)
fieldPathMatch specific field pathsuser.email, request.headers.authorization
Tip: Enable the audit option to include redaction statistics in the output. This logs which fields were redacted and how many matches were found, useful for compliance reporting.

Schema Mapping

mapCommonSchema

Map incoming data to standard or custom schemas. This ensures consistent field naming across different data sources and enables compatibility with downstream tools.

Schema Templates

TemplateDescription
ECSElastic Common Schema: standard field naming for Elasticsearch/OpenSearch
CIMCommon Information Model: normalized event field schema
OTELOpenTelemetry: vendor-neutral observability standard
LogPulseNative LogPulse data models: custom models defined in your organization
CustomUser-defined mappings without a base template

The mapping mode determines how missing fields are handled:

ModeDescription
PermissiveMissing source fields are skipped. Partial output is produced without errors.
StrictAll required fields must be present. Pipeline errors if required fields are missing.

Fields can be coerced to specific types during mapping:

TypeDescription
stringConvert to string
intConvert to integer
floatConvert to floating-point number
booleanConvert to boolean
Note: Enable "Keep Original" to merge mapped fields with the original payload instead of replacing it. This is useful when you want to add standard field names alongside existing ones.

Lookup

lookup

Enrich events by matching a field value against a lookup dataset. This is useful for adding context like hostname-to-team mappings, IP geolocation data, or CMDB asset information.

SettingDescription
DatasetSelect a lookup dataset from your organization
Input Key PathDot-notation path to the lookup key in your payload
Lookup Key FieldField name in the dataset to match against
Output ModeHow to output matched data (see below)
Field MappingsOptional: map specific lookup fields to event fields

Output Modes

ModeDescription
mergeIntoEventMerge all lookup fields into the event payload
writeToTargetPathWrite lookup data to a specific path in the payload
embedLookupEmbed the full lookup record as a nested object

AI Agent

aiAgent

A model reads the message and your instructions, may call the LogPulse tools you allow, and answers as text or as JSON that matches a schema. It runs as the person who published the running version and uses AI credits, also in test runs.

SettingDescription
InstructionsWhat the model should do; {{ payload.x }} reads the message
ToolsLogPulse tools the model may call. Tools that propose a change only in action pipelines
Outputtext, or json with an output schema
Output PathWhere the answer lands in the message (default: ai)
Max Tool CallsMost tool calls in one step (default 5, at most 10)

Flow Control

Flow control nodes branch, fan out and recombine, pause a run, or hand the message to another pipeline.

Condition

condition

Route data to different branches based on field values. Each condition creates its own output handle. A message goes to every condition that matches, or to the default branch when none match.

Operators

OperatorSymbolDescriptionValue Required
eq==EqualsYes
neq!=Not equalsYes
gt>Greater thanYes
lt<Less thanYes
gte>=Greater than or equalYes
lte<=Less than or equalYes
containscontainsContains substringYes
not_containsnot containsDoes not containYes
regexregexMatches regular expressionYes
existsexistsField exists in payloadNo
not_existsnot existsField does not existNo

You can define several conditions, each with its own output and label. The default branch catches the messages that match none of them.

Advanced Mode (JavaScript)

In advanced mode, write JavaScript that returns an array of branch IDs (strings) or a single string. The event will be routed to the specified branch(es).

JavaScript Condition Example
// Must return an array of branch IDs or a single string
if (data.level === 'error' && data.status_code >= 500) {
  return ['critical-errors'];
} else if (data.level === 'error') {
  return ['errors'];
}
return ['default'];

Split & Join

splitjoin

Fan an array out into one message per element with split, then optionally re-merge them with join. Each fork carries a parts.id correlation marker so a downstream join knows which forks belong together, the same pattern Node-RED uses.

Split SettingDescription
Array PathDot-notation path to the array to fan out (e.g., payload.items)
Output PathWhere each element lands inside the per-fork message (default: payload)
ConcurrencyMost forks running at once. Empty: all at once; 1 processes the items one after another (a loop)
Rate LimitOptional throttle: release N forks per second instead of all at once
Continue on ErrorLog a failing fork instead of failing the split
Join SettingDescription
Mode"auto" matches an upstream split via parts.id; "manual" waits for a fixed count
Output PathWhere the joined array is placed (default: payload)
CountRequired in manual mode: number of messages to collect
Timeout (ms)Maximum wait for missing parts (default: 60 seconds)
When Incompletepartial (default) continues with the parts that arrived; fail fails the run

Splits and joins nest. When a split runs while parts metadata is already present, the existing parts is pushed onto a stack, and each join pops the stack on completion, so split → ... → split → ... → join → join unwinds cleanly.

Tip: Use concurrency or a rate limit on split when the next steps call rate-limited APIs. Skip the join when the per-fork step (such as logpulseIngest) is the goal and you don't need the results combined.

Wait

wait

Pause for a number of seconds, or until a moment (an ISO timestamp, a Unix time or a {{ payload.x }} template). A wait of more than a minute parks the run with status waiting; it resumes by itself, up to 30 days later. Test runs skip the wait.

Call Pipeline

callPipeline

Run the published version of another pipeline with this message, inside the same run. Its result lands at the output path (default: result). The called pipeline must finish in one go, calls nest at most 3 deep, and a data pipeline cannot call an action pipeline.

Load Nodes

Load nodes end a path: they write events to an index, update entities or answer the webhook caller.

LogPulse Ingest

logpulseIngest

Write processed events into LogPulse. Set the source type, target index and field mappings to control how events appear in search.

SettingDefaultDescription
Source Typeetl_pipelineValue for the sourcetype field in LogPulse
IndexmainTarget index for ingested events
Hostlogpulse-etlHost name to tag events with
Sourceetl:<pipeline id>Source identifier
Message PathpayloadWhat is written: an array becomes one event per element, an object one event
Timestamp Path—Path to the timestamp field
Level Path—Path to the log level field
Field Mapping—Optional custom field name mappings
Note: A test run writes nothing: the node returns what a real run would write as __ingestPreview and __ingestCount.

Upsert Entities

upsertEntities

Create or update entities (such as users and hosts) from a sync, for example the members of a Cloudflare account. Each item is { key, display_name?, roles?, attributes? }; roles are stored per source, so a sync only owns and revokes its own roles. Detections read them with | lookup identities.

SettingDescription
Items PathPath to the array of items (default: payload)
Entity TypeEntity type to write (default: user)
Role SourceShort source name, such as cloudflare (required)
Prune MissingRemove this source's roles from entities that are no longer listed
Total Count PathPath to the source's total count; pruning is skipped when a run only saw part of it

A test run only shows a preview and writes no entities.

Respond to Webhook

respondWebhook

The answer for a webhook caller that waits for the result (?wait=N or the header Prefer: wait=N, at most 30 seconds). Set the status code (default 200), the body (the message or a {{ payload }} template) and headers. Without this step a waiting caller gets the run's output.

Utility Nodes

Utility nodes help with debugging and error handling during pipeline development and execution.

log

Write a debug line to the run log, from the whole message, a path or a template. Optionally also to the pipeline_logs index. A log step ends its path.

error

Stop pipeline execution immediately and record an error. Use this in condition branches to halt processing when invalid data is detected.

Retries & Errors

Every working step (not start, split, join, wait, approval gates, log or error) can retry and decide what happens when it fails.

On errorWhat happens
stop (default)The run fails at this step
continueThe next steps run with the input plus an error field
branchOnly the error output of the step runs, with the error on the message. Use it for "notify me when the API is down"

Retries take a number of attempts, a delay and a backoff. Under Settings → Runs you also set the most runs at once (extra runs queue or are skipped), an idempotency key so a repeated webhook delivery starts no second run, and an error pipeline that runs after the last retry has failed.

Respond Nodes

Respond nodes only exist in action pipelines.

NodeWhat it does
waitForApprovalPause the run (status waiting_approval) until someone approves it in the Respond tab. With an expiry in hours it ends by itself as rejected (default) or approved; a rejection follows the rejected output when there is one
responseActionRun a response action: notify, create_ticket, block_ip, disable_user, isolate_host or run_pipeline. In a test run it only does a dry run
Warning: A pipeline with block_ip, disable_user or isolate_host is destructive. Put a Wait for Approval step before the action. An automation trigger on such a pipeline starts switched off, and only an owner can turn it on.

Stream Nodes

Stream nodes only exist in stream pipelines, which run once per event inside ingest. A failing step lets the event through unchanged. Test a stream pipeline on the original logs of the last 60 minutes with raw log replay.

NodeWhat it does
streamSampleKeep one in N events. With a key field the same key is always kept or always dropped
streamDedupeDrop an event whose key fields were already seen within a time window
streamDropEnd the path without storing the event; the reason shows in the counters
streamSinkSend the event to an external destination, and optionally store it as well

Dynamic Values

Text fields in step settings take templates instead of fixed values. {{ payload.user.name }} reads a field and any JSONata expression works too. A value that is a single {{ }} keeps its type.

TypeSyntaxDescriptionExample
Field{{ path }}A value from the message{{ payload.user.id }}
JSONata{{ expression }}Any JSONata expression on the message{{ $count(payload.items) }}
Variables{{ $vars.name }}, {{ $tenantVars.name }}Pipeline and organization variables{{ $vars.api_url }}
Context{{ $context.key }}, {{ $tenantContext.key }}Values from the context store{{ $context.cursor }}
System{{ $system.run_id }}Run details: run_id, pipeline_id, run_started_at and more{{ $system.run_started_at }}
Secrets{{ $secrets.name }}, {{ $tenantSecrets.name }}Pipeline and organization secrets; only in HTTP fields and response action parameters{{ $secrets.api_key }}

The older placeholders, such as ${pipeline.variables.X} and ${pipeline.secrets.X}, still work.

Note: Keep keys and passwords in pipeline or organization secrets, never in the steps. Secrets are stored encrypted and only resolve in HTTP fields and response action parameters, so they never end up in the message.

Context Store

Pipelines are stateless by default: every run starts fresh. The Context Store is the escape hatch for state that must survive across runs: incremental cursors, dedup caches, last-seen IDs, accumulated counters, or anything where 'what did the previous run leave behind' matters. Two scopes are available: pipelineContext (private to one pipeline) and tenantContext (shared across every pipeline in your organization).

ScopeVisibilityTypical use
pipelineContextPrivate to one pipelinePolling cursors, dedup caches, last-seen IDs
tenantContextShared across every pipeline in the orgCross-pipeline coordination, shared lookup lists
Note: Reads happen at run start (snapshot of the store), writes are buffered during the run, and on a successful run the diff is persisted. A failed run discards its writes, with no half-state pollution. The editor runs against a draft slot; promoting draft to production happens in the sidebar Context tab.

Pipeline Context

pipelineContext is bound to one pipeline. Use it for pipeline-local cursors, dedup caches, and any state you don't want to share. Values are JSON: strings, numbers, booleans, arrays, and objects all work. Object-typed values can be addressed with dot-paths on both read and write (writing to cursor.lastSeenId creates the cursor object automatically if it didn't exist).

Reading a value

Use a transform set operation with value type pipelineContext. The value field is the key (or dot-path into an object value).

Read pipelineContext into the payload
Operation: set
  target: payload.lastSeenId
  value:
    type: pipelineContext
    value: cursor.lastSeenId

Writing a value

Use a transform set operation with scope pipelineContext. The target is the key (or dot-path); the value is whatever JSONata or expression you want to persist.

Persist a cursor for the next run
Operation: set
  scope: pipelineContext
  target: cursor.lastSeenId
  value:
    type: jsonata
    value: $max(payload.id)

Tenant Context

tenantContext is shared across every pipeline in your organization. Use it when several pipelines need to coordinate: one pipeline writes a list of active customers, another pipeline reads it. Both pipelines see the same store; consumers always read the freshest persisted value.

Producer pipeline writes a shared list
Operation: set
  scope: tenantContext
  target: active_customers
  value:
    type: jsonata
    value: payload.customers
Consumer pipeline reads it
Operation: set
  target: payload.customers
  value:
    type: tenantContext
    value: active_customers

Common Patterns

Two patterns cover most real use cases. JSONata expressions can also read the snapshot directly via the $context and $tenantContext bindings, which is convenient when you want to mix the existing state into a computation.

Incremental polling cursor

The most common use of the context store: remember where the previous run stopped so the next run only fetches new data. The first run sees no cursor, so use $exists() to provide a fallback timestamp. Advance the cursor only after a successful ingest; failed runs leave it unchanged so the next attempt re-pulls the same window.

Read cursor with first-run fallback
Operation: set
  target: payload.since
  value:
    type: jsonata
    value: $exists($context.cursor.lastPolledAt)
             ? $context.cursor.lastPolledAt
             : "1970-01-01T00:00:00Z"
Advance cursor after a successful ingest
Operation: set
  scope: pipelineContext
  target: cursor.lastPolledAt
  value:
    type: jsonata
    value: $max(payload.timestamp)

Dedup cache

Skip records you've already processed by keeping a list of seen IDs in the context store. Filter the new payload to drop ids already in $context.seenIds, ingest, then append the new ids to the cache.

Append seen IDs to a cache array
Operation: set
  scope: pipelineContext
  target: seenIds
  value:
    type: jsonata
    value: $append($context.seenIds, payload.id)
Warning: Don't use the Context Store for secrets (use pipeline or tenant secrets instead, as they're encrypted at rest) or for large data sets (it's row-level state in Postgres, not a data warehouse). Keep entries small: cursors, counters, short lists. For bulk data, query a lookup dataset or LPQL on every run.

Pipeline Settings

Pipeline-level settings control how and when your pipeline runs, as well as debugging and testing features.

Trigger Modes

Set how the pipeline starts. Schedules and webhooks read these settings from the production version.

Scheduled Execution

Run the pipeline on a cron schedule in the schedule's timezone (default UTC). It runs only when the pipeline is active and promoted and the schedule is enabled. A scheduled run starts with an empty input, so the pipeline fetches its own data.

SettingDescription
What starts itOn the Start step or the Settings tab: on a schedule, when data comes in, or only when asked. Picking "On a schedule" switches the schedule on
Cron Expression5- or 6-field cron (e.g., */5 * * * * for every 5 minutes). Shortcuts such as @hourly are not accepted
TimezoneTimezone for the cron schedule (default: UTC)
PublishWhat starts a pipeline is part of the published version. The Start step says what the live version does and what is set but not published yet
Cron Examples
*/5 * * * *      Every 5 minutes
0 * * * *        Every hour
0 9 * * 1-5      Weekdays at 9:00 AM
0 0 * * *        Daily at midnight
0 */6 * * *      Every 6 hours

HTTP Ingestion

Send events through the pipeline with its webhook URL (a long token in the URL that you can renew and limit to IP lists) or with an API key linked to the pipeline. The posted body is the input. A caller that waits (?wait=N or Prefer: wait=N, at most 30 seconds) gets the answer of a Respond to Webhook step, or the run's output.

Note: A pipeline has one trigger mode: scheduled or HTTP ingestion. Run now works in both, on the production version.

Active / Inactive

The Active toggle is a master switch. A paused pipeline is skipped everywhere: schedule, webhook, API key, alerts and automations. You can still edit it and run tests.

Test Events

Test events are sample payloads that you save on the pipeline. A test run executes the draft with one of them, without changing production.

FeatureDescription
Create Test EventDefine a name and JSON payload on the Start step or the Settings tab. An action pipeline also offers ready-made examples of what an alert, KPI, SLO, health check or notable sends
Select & RunPick a test event on the Start step; Test starts the draft with it
View ResultsInspect input/output payloads at each node after a test run
Tip: A test run is not a real run. Some steps behave differently, as the table below shows.
In a test runWhat happens
LogPulse IngestWrites nothing; shows __ingestPreview and __ingestCount
Response ActionDoes not fire; returns a dry run
Upsert EntitiesWrites nothing; shows a preview
HTTP RequestRuns for real: the call is made
LogWrites to the pipeline_logs index when "log to index" is on
WaitIs skipped

Execution & Tracking

Every run is recorded with its status, the version it ran and the input and output of each step. The editor shows progress live over a WebSocket.

Run Status

StatusDescription
pendingThe run is created and about to start
queuedWaiting for a free slot under the most runs at once
runningThe pipeline is running
waitingParked by a Wait step; resumes by itself
waiting_approvalWaiting for someone to approve in the Respond tab
successFinished without errors
failedStopped with an error; can be retried or resumed from the failed step
dead-letterFailed after its last retry
cancelledStopped, or skipped because another run was busy

Node Execution Details

Each node in a run is tracked individually, allowing you to pinpoint exactly where failures occurred and inspect the data at each step.

Tracked DataDescription
Statuspending, running, success, failed, or skipped
DurationExecution time in milliseconds
Input PayloadData received by the node
Output PayloadData produced by the node
Error DetailsError message (if failed)

Real-Time Updates

The pipeline editor connects via WebSocket to receive live updates during execution. Nodes light up as they execute, showing status, duration, and errors in real-time.

EventDescription
pipeline:run:startedPipeline execution has begun
pipeline:run:finishedPipeline completed successfully
pipeline:run:failedPipeline failed with an error
pipeline:node:startedA node has started executing
pipeline:node:finishedA node has completed
pipeline:node:failedA node has failed
pipeline:log:emitA Log step wrote a line
Note: The editor opens the WebSocket connection itself when you open a pipeline. Payloads larger than 64 KB are stored separately and shown as a marker in the run.

Versioning

You always edit a draft, and test runs use the draft. Schedules, webhooks and triggers run the production version. Promoting saves the draft as a new version and makes it production.

FeatureDescription
DraftWhat you edit and what a test run uses. Saving changes only the draft
PromoteSaves the draft as a new version with a message and makes it production
ProductionThe version that schedules, webhooks and triggers run
RollbackPoints production at an earlier version; the draft stays as it is
Load versionOpens an earlier version as the draft to continue from it
Pinned runsA run keeps the version it started with, also when you promote during the run
Tip: Write a short message when you promote, so the version history shows what changed. The editor can suggest one from the changes.

API Reference

All pipeline operations are available via the REST API. All endpoints require authentication and are scoped to the current organization.

Pipeline Endpoints

MethodEndpointDescription
GET/api/v1/pipelinesList the pipelines with their last run
POST/api/v1/pipelinesCreate a pipeline
GET/api/v1/pipelines/:idA pipeline with its graph, production version and draft state
PUT/api/v1/pipelines/:idUpdate the draft and settings (including test events)
DELETE/api/v1/pipelines/:idDelete a pipeline
PUT/api/v1/pipelines/:id/activePause or resume a pipeline
POST/api/v1/pipelines/:id/validateCheck the graph without saving

Execution Endpoints

MethodEndpointDescription
POST/api/v1/pipelines/:id/test-runTest the draft with a payload
POST/api/v1/pipelines/:id/runRun the production version now
GET/api/v1/pipelines/:id/runsRuns of a pipeline, newest first, with a per-node breakdown
GET/api/v1/pipelines/:id/runs/:runIdOne run with its node executions
POST/api/v1/pipelines/:id/runs/:runId/retryRun a failed run again
POST/api/v1/pipelines/:id/runs/:runId/resumeResume a failed run from its failed step
POST/api/v1/pipelines/webhook/:tokenStart a pipeline through its webhook URL

Version Endpoints

MethodEndpointDescription
GET/api/v1/pipelines/:id/versionsThe promoted versions, newest first
GET/api/v1/pipelines/:id/versions/:versionIdOne version with its graph
POST/api/v1/pipelines/:id/promotePromote the draft to a new production version
POST/api/v1/pipelines/:id/rollbackPoint production at an earlier version
POST/api/v1/pipelines/:id/load-version/:versionIdLoad an earlier version into the draft