Integrations

Agents & fleet

The LogPulse agent is a lightweight Go control plane wrapped around Vector. Agents enrol once, pull their configuration from LogPulse, send periodic heartbeats, and pick up remote actions — including self-upgrades — without ever exposing an inbound port.

Overview

Every host or container that ships logs into LogPulse runs the logpulse-agent binary. The agent doesn't generate Vector configuration locally — it asks the LogPulse API for a config that was assembled from the group's inputs in the UI, writes it to disk, and runs Vector against it. All operator changes happen in the dashboard; agents apply them on their next poll.

Fleet model

Agents are organised into groups. A group has a name, a slug, a set of inputs (syslog listener, journald reader, Kafka consumer, …) and a monotonically increasing config_version. Every host that enrols into a group receives the same Vector configuration — that's the unit of fleet-wide change.

ConceptWhat it isWhere it lives
GroupA logical bundle of hosts that share a Vector configagent_groups table, /sources/agents UI
InputOne Vector source (syslog, file, journald, kafka, …) attached to a groupagent_inputs table
AgentA single host registered with a bearer token (AGT_…)agent_registrations table
Enrolment tokenShort-lived secret that lets new hosts join a groupagent_enrollment_tokens table
ActionA queued instruction for an agent (e.g. upgrade_agent)agent_actions table
Note: Group membership is mutable. Move an agent to a different group and its applied_config_version resets to 0 — the next poll delivers the new group's config without a reinstall.

Quick start

Install the agent

The agent runs on Linux (systemd) and Kubernetes (DaemonSet). For a Linux host, the install one-liner downloads the binary, verifies the SHA-256, installs a systemd unit, and waits for an enrolment token to start the agent:

bash
curl -fsSL https://get.logpulse.io | sudo sh

Pin a specific release by setting LOGPULSE_VERSION before piping:

bash
curl -fsSL https://get.logpulse.io | sudo LOGPULSE_VERSION=v0.1.0 sh

The installer creates these paths:

PathPurpose
/usr/bin/logpulse-agentBinary symlink (versioned target under /opt)
/etc/logpulse-agent/Static config + bootstrap token
/var/lib/logpulse-agent/State: applied config, Vector data-dir
/var/log/logpulse-agent/Agent + Vector logs

Enrol the agent

In the dashboard, open Sources → Agents → Groups, pick a group, and click Mint enrolment token. The token (prefix EBT_) is shown once — copy it, then run on the host:

bash
sudo logpulse-agent enrol \
  --api-url https://api.logpulse.io \
  --token EBT_<bootstrap-token>

The agent posts its hostname, platform, and version to LogPulse and receives back a long-lived AGT_ token. The bootstrap token is consumed (one of n uses); the agent token is written to /etc/logpulse-agent/agent-token and used for every subsequent request. From there the agent runs autonomously — start it with systemctl enable --now logpulse-agent.

Tip: Bootstrap tokens have a configurable expiry (default 24h) and a use count (default 100). For large fleets, mint a token with maxUses=number of hosts and ship it via your provisioning system; the token expires by itself once it's exhausted or the TTL runs out.

Groups & inputs

Each input attached to a group becomes a Vector source in the generated configuration. The agent's platform (linux or kubernetes) filters which inputs apply — a journald input is skipped on a Kubernetes pod, a kubernetes_logs input is skipped on a Linux VM. Inputs can be enabled or disabled per-input without deleting them; a disabled input is omitted from the next config.

Per-input routing overrides control where events land in LogPulse:

OverrideDefaultWhen to set it
indexOverridemainSplit prod vs staging, tenants, or applications into separate indexes
sourcetypeOverrideagent_type (e.g. agent_journald)Tag a specific input with a domain-specific sourcetype (e.g. nginx_access)
sourceOverridelogpulse-agentDistinguish multiple inputs of the same type — `source=prod-fw1` vs `source=prod-fw2`

Config versions

Every edit to a group (add/edit/delete an input, change routing overrides, rename the group) increments the group's config_version. Agents only re-fetch the config when their own applied_config_version falls behind — see Config polling below. The version is exposed in the dashboard so operators can verify rollout: an agent that still reports an old applied_config_version hasn't yet picked up the latest changes.

Control-plane protocol

Once enrolled, an agent only talks to LogPulse over HTTPS. Three endpoints make up the entire agent-side surface area: enrolment (once), config polling (on a tight loop), and heartbeats (slower loop, also reports action results). LogPulse never opens a connection back to the agent.

Enrolment — POST /api/v1/agents/enrol

Called once during install. Request body authenticates with the short-lived enrolment (bootstrap) token; response includes the agent's long-lived bearer token.

POST /api/v1/agents/enrol
{
  "bootstrapToken": "EBT_<bootstrap-token>",
  "hostname": "fw1.prod.example.com",
  "platform": "linux",
  "os": "Ubuntu 24.04",
  "kernel": "6.8.0-35-generic",
  "agentVersion": "0.4.1",
  "vectorVersion": "0.40.0",
  "groupSlug": "wakanda-enterprise"   // optional: overrides the token's default group
}

The response carries an AGT_ bearer token — shown once, stored hashed (SHA-256) at rest:

201 Created
{
  "data": {
    "agentId": "f1c4…",
    "agentToken": "AGT_<32-byte base64url>",
    "groupId": "6d0e…",
    "configVersion": 7
  }
}
Warning: AGT_ tokens are credentials. They're written to /etc/logpulse-agent/agent-token with mode 0600 owned by the logpulse user. Revoke a stolen or stale agent from the Agents tab; the next request from that token returns 403.

Config polling — GET /api/v1/agents/:id/config

The agent polls this endpoint at a short interval (~10s by default). It passes its currently applied version via the since query parameter; LogPulse short-circuits with 204 No Content when nothing has changed.

GET /api/v1/agents/f1c4.../config?since=7
Authorization: Bearer AGT_<token>

→ 204 No Content      # since >= group.configVersion, nothing to apply
→ 200 OK              # new config available
  Content-Type: application/yaml
  x-config-version: 8
  x-config-etag: sha256:9f8…
  Body: <Vector YAML>

The body is the same YAML the dashboard renders under YAML preview on the group page. Agents write it to /var/lib/logpulse-agent/vector.yaml, validate it with vector validate, and reload Vector on success. A failed validation aborts the apply — the agent keeps running the previous config and surfaces the error in its next heartbeat.

When the group has no enabled inputs the response body is the literal string {}. The agent detects this and stops Vector entirely (no placeholder pipeline, no wasted RAM).

Heartbeats — POST /api/v1/agents/:id/heartbeat

Heartbeats are how an agent says "I'm alive", reports its current state, and asks for queued actions. The default cadence is 30 seconds. The body is small but carries every field the dashboard needs to render the agents table:

POST /api/v1/agents/f1c4.../heartbeat
{
  "agentVersion": "0.4.1",
  "vectorVersion": "0.40.0",
  "appliedConfigVersion": 8,
  "vectorRunning": true,
  "vectorUptimeSeconds": 1837,
  "actionResults": [          // optional — results for previously dispatched actions
    { "actionId": "...", "status": "done", "result": { "previousVersion": "0.4.0" } }
  ]
}

The server updates these columns on agent_registrations:

ColumnSourceUsed for
last_heartbeat_atnow()Online/offline badge (5-min threshold)
agent_versionbody.agentVersionPer-host upgrade decisions
vector_versionbody.vectorVersionVector-version sweep, troubleshooting
applied_config_versionbody.appliedConfigVersionRollout progress badge
vector_runningbody.vectorRunningSurface failed configs in the UI
vector_started_atderived from uptime"Running for 2h 14m" indicator
went_offline_atcleared on recoveryDistinguishes brief flaps from extended outages

An agent is considered offline when its last_heartbeat_at is older than the constant AGENT_OFFLINE_THRESHOLD_MS = 5 minutes (defined in packages/types). The dashboard renders an "Offline" badge accordingly.

The response carries any queued actions for this agent — see below.

200 OK
{
  "data": {
    "pendingActions": [
      {
        "id": "...",
        "type": "upgrade_agent",
        "payload": { "targetVersion": "0.4.2" }
      }
    ]
  }
}

Remote actions

Actions are a generic queue for "do something on the agent" instructions. The current type is upgrade_agent; future types (restart Vector, drop cache, rotate token) plug into the same machinery. Status flows in one direction only:

StatusSet byMeaning
pendingServer (created)Queued, not yet handed to an agent
runningAgent (via actionResults)Agent has picked it up and is executing
doneAgentAction completed successfully — result payload available
failedAgentAction failed — error message in result
cancelledServer (operator)Action was cancelled before dispatch
expiredServer (sweep)Action wasn’t picked up before its deadline

The server validates state transitions: an action that's already done can't go back to running; an expired action stays expired. Agents only receive non-expired, non-cancelled actions in the pendingActions array (up to 10 per heartbeat).

Note: Action results are reported via the next heartbeat after execution finishes. There is no dedicated webhook back to LogPulse. This keeps the agent-side network topology trivially simple: outbound HTTPS only, no inbound port, no callback URL.

Remote upgrades

Selecting an agent in the dashboard and clicking Upgrade creates an upgrade_agent action with the target version in its payload. The action waits in pending until the agent's next heartbeat, then flows through the state machine:

Lifecycle of an upgrade
1. Operator clicks Upgrade → POST /api/v1/agents/:id/actions
   { type: "upgrade_agent", payload: { targetVersion: "0.4.2" } }
   Status: pending

2. Next heartbeat from the agent (≤ 30s later):
   Server returns pendingActions = [the action]
   Server transitions: pending → running (when the agent acks)

3. Agent runs the install script with the pinned version:
   curl -fsSL https://get.logpulse.io | sudo LOGPULSE_VERSION=v0.4.2 sh
   The new binary replaces the old one, systemd restarts the unit, the
   new process loads its existing token from /etc/logpulse-agent.

4. The first heartbeat from the upgraded agent carries actionResults:
   { actionId, status: "done", result: { fromVersion: "0.4.1", toVersion: "0.4.2" } }
   Server transitions: running → done

Bulk upgrades use the same machinery — the dashboard issues one action per selected agent and shows aggregate progress as results trickle in. Vector itself isn't upgraded by this action; Vector lives under /opt/logpulse-agent/vector/ and rolls forward with the agent release that bundles it.

Warning: Upgrades that take longer than the action's configured deadline are marked expired by the server sweep. The upgrade may still have succeeded on the host — check the agent's next heartbeat for the actual agentVersion rather than relying on the action status alone.

Troubleshooting

Agent returns 403 on every request

The bearer token was revoked from the Agents tab, or it doesn't match the prefix on the server side. Mint a fresh bootstrap token, run logpulse-agent enrol again, and restart the unit.

Config changes don't reach an agent

Check the agent's row in the dashboard. If applied_config_version stays behind config_version after a couple of polls, the agent likely failed validation — open the agent logs (journalctl -u logpulse-agent) and look for vector validate errors. The agent keeps the previous config running until a new one passes validation.

Agent shows Offline but the host is up

The agent hasn't sent a heartbeat in over 5 minutes. Common causes: outbound HTTPS to api.logpulse.io is blocked by a firewall, the agent process crashed (systemctl status logpulse-agent), or system clock skew is rejecting TLS. The dashboard tooltip on the Offline badge shows the exact last seen timestamp.

Upgrade action stuck in running

The upgrade started but never reported a result. Check the agent's agent_version in the next heartbeat — if it matches the target, the upgrade succeeded but the result-reporting heartbeat got lost; the action will expire by itself. If the version is unchanged, look at the install script's output in /var/log/logpulse-agent/upgrade.log.

We use cookies to analyze site traffic and improve your experience. No cookies are placed without your consent. Privacy Policy