Migrating from Splunk
Bring your Splunk setup along instead of rebuilding it. LogPulse reads your Splunk configuration and apps, turns them into its own inputs and parser rules, and tells you exactly what needs a person.
.conf files and Splunkbase apps to Sources → Move from Splunk: LogPulse turns them into agent inputs, parser rules and cloud integrations, keeps the Splunk sourcetypes and uses Splunk CIM field names. Run both side by side until the counts match, then switch off Splunk.Overview
A migration has six steps. You can run Splunk and LogPulse next to each other for as long as you like, so nothing has to happen in one weekend.
- Windows and Linux to Converted inputs
- Firewalls to Converted inputs
- Apps to HEC input
- Windows and Linux to Splunk today (optional)
- Converted inputs to HTTP API
- HEC input to HTTP API
- HTTP API to Ingest buffer
- Cloud integrations to Ingest buffer
- Ingest buffer to Workers
- Workers to ClickHouse
| Step | What you do | Where |
|---|---|---|
| 1. Take stock | List apps, sourcetypes, indexes and senders | Your Splunk servers |
| 2. Convert | Turn apps and .conf files into inputs and parser rules | Sources → Move from Splunk |
| 3. Move the data flow | Install the agent, point HEC senders at it | Agents |
| 4. Run side by side | Compare counts and fields per sourcetype | Search |
| 5. Searches and detections | Bring saved searches, alerts and correlation searches across | Search, Security |
| 6. Switch off Splunk | Stop sending to Splunk, keep or export old data | Your Splunk servers |
Splunk to LogPulse
| Splunk | LogPulse |
|---|---|
| Universal Forwarder | LogPulse agent (Vector, managed from LogPulse) |
inputs.conf | Agent inputs in an agent group |
props.conf | Parser rules: fields, multi-line events, event time |
transforms.conf | Parser rules (field extractions) and pipeline steps (drop, route, mask) |
| Index | Index per kind of data: windows, linux, firewall, web, dns, … |
| Sourcetype | Sourcetype (kept as is); the source is the product, such as paloalto |
| SPL | LPQL: the same pipe model, see LPQL vs SPL |
| Saved search / alert | Saved search / alert rule |
| ES correlation search | Detection (hub rules, Sigma or your own) |
| Splunkbase add-on | Built-in parsers, hub parser rules and cloud integrations |
Indexes work differently. In Splunk every team picks its own index names. In LogPulse an index is a kind of data (firewall, windows, dns), so detection rules know where to look. The converter maps each Splunk sourcetype to its LogPulse index and keeps the Splunk index name in the report.
1. Take stock
Start with what Splunk really uses, not with what is installed. btool shows the merged configuration Splunk runs with, and which file each line comes from:
# On a forwarder or indexer
$SPLUNK_HOME/bin/splunk btool inputs list --debug > inputs.txt
$SPLUNK_HOME/bin/splunk btool props list --debug > props.txt
# On a search head: which sourcetypes and indexes carry data (last 30 days)
| tstats count where index=* earliest=-30d by index, sourcetype
| sort - countWrite down per sourcetype how much data it is and who uses it (dashboards, alerts, ES). Sourcetypes nobody searches are a good moment to stop collecting.
2. Convert apps and .conf files
In the LogPulse app go to Sources → Move from Splunk. Add any of these:
- an app or add-on archive from Splunkbase (
.tgzor.spl); - a folder such as
$SPLUNK_HOME/etc/apps/<app>with itsdefaultandlocalfolders; - loose
inputs.conf,props.confandtransforms.conf, for example frometc/system/local.
The files are converted in your browser. You see the agent inputs, parser rules and pipeline steps it found, and a list of what needs you. Tick what you want, pick an agent group and apply. Inputs are created switched off by default, so nothing changes on your hosts until you turn them on. You can also download a report and a plain Vector configuration.
convert_splunk_config does the same conversion. The assistant can then test each parser rule on your real events and propose it.What converts
| Splunk setting | Becomes |
|---|---|
[monitor://…] | File input; a folder becomes a glob, blacklist extensions become excludes |
[WinEventLog://…] | One Windows Event Log input with all channels and event ID filters |
[udp://…] [tcp://…] | Syslog input, or a socket input for other data |
[http://…] | HEC input on the agent; senders keep their token |
EXTRACT- / REPORT- | Regex parser rules with the same field names |
KV_MODE / INDEXED_EXTRACTIONS | JSON, key=value or delimited parser rules |
LINE_BREAKER / BREAK_ONLY_BEFORE | Event breaking on the parser rule (multi-line events) |
TIME_PREFIX / TIME_FORMAT | Event time field and format |
FIELDALIAS- | Field renames on the parser rule |
TRANSFORMS- (nullQueue, sourcetype, index) | Pipeline steps: drop, keep only, set sourcetype or index |
SEDCMD- | Pipeline step that masks text |
Splunk regexes are PCRE. Most work as they are; the converter rewrites the few PCRE-only forms ((?P<name>), \A, possessive quantifiers) and checks every regex against the same safety check as when you save a parser rule. A regex that fails it is listed with the reason.
One difference to keep in mind: Splunk applies every extraction of a sourcetype and merges the fields. LogPulse uses the first parser rule that matches an event. A sourcetype with several extractions gets one rule per extraction and a warning. Check them on real events in Parsers, or let the parser assistant merge them into one.
Splunkbase apps
Download the apps you use from Splunkbase with your own account and add them to the page. They stay in your browser. LogPulse does not ship or copy any Splunk app; the conversion only reads your copy to set up your own inputs and parser rules.
For many popular products you may not need the app at all: the agent already parses Windows, Linux, FortiGate, Palo Alto, Cisco ASA, nginx, IIS and more, and cloud sources such as Microsoft 365, Entra ID and AWS CloudTrail are cloud integrations without an agent. Apps that pull data from an API (modular inputs) become the matching cloud integration where LogPulse has one, such as Okta, Cisco Duo, Proofpoint TAP, Netskope, Cisco Meraki, Google Cloud, Google Workspace, Jira and Confluence. You enter the credentials again: LogPulse does not read them from the app. Inputs without a counterpart yet are listed by name.
3. Move the data flow
LogPulse agent
The simplest route is to install the LogPulse agent on the hosts next to the Universal Forwarder. Both read the same files and event logs, so Splunk keeps working while you compare. When you are done, remove the forwarder.
HTTP Event Collector
Applications that send to Splunk's HTTP Event Collector can keep doing so. Add an HEC input to an agent (the converter does this for every [http://…] stanza, with the same tokens) and change the URL in the sender to the agent on port 8088. The Splunk sourcetype, source and index arrive as splunk_sourcetype, splunk_source and splunk_index.
Universal Forwarders
Universal Forwarders talk to Splunk in Splunk's own protocol (S2S, usually on port 9997). Receiving that protocol directly is on the LogPulse roadmap. Until then you have two options:
- Agent next to the forwarder (recommended): see above.
- Forwarder sends a copy as plain TCP to a socket input on an agent. You keep your forwarders, but the host, source and sourcetype are not sent along and multi-line events arrive as single lines. Fine for simple line logs during a trial.
# outputs.conf on the Universal Forwarder: keep sending to Splunk,
# and send a plain copy to a LogPulse agent (socket input, TCP 5514)
[tcpout]
defaultGroup = splunk_indexers, logpulse
[tcpout:splunk_indexers]
server = idx1.example.internal:9997
[tcpout:logpulse]
server = logpulse-agent.example.internal:5514
sendCookedData = false4. Run side by side
Let both run for a week or two. Compare per sourcetype how many events arrive and whether the fields you search on are there:
# Splunk
index=* earliest=-24h | stats count by sourcetype
# LogPulse
earliest=-24h index=* | stats count by sourcetypeDifferences usually come from a filter that ran at ingest in Splunk (a nullQueue transform) or from an input that is still switched off. The To check list of the converter tells you which.
5. Searches, alerts and detections
LPQL uses the same pipe model as SPL (stats, eval, where, top, timechart), so most searches carry over with small changes. See LPQL syntax and the guide LPQL vs SPL.
Field names carry over too: LogPulse uses the Splunk Common Information Model (CIM) as its field standard, so src, dest, user, action and signature mean the same in both. Searches that use the URL Toolbox macros (ut_parse, ut_shannon, ut_levenshtein) are rewritten to the LPQL functions url_*, shannon and levenshtein.
For security, start from the Detection Hub: it already holds rules for the common log sources, mapped to MITRE ATT&CK. Bring across your own correlation searches one by one as detections. See Security monitoring.
6. Switch off Splunk
When the counts match and your searches and alerts run in LogPulse, stop sending to Splunk: remove the Splunk group from outputs.conf or uninstall the forwarders. Keep Splunk read-only for as long as you need the old data, or export what you must keep and send it to LogPulse through the HTTP API.
What stays manual
| Splunk | What to do |
|---|---|
EVAL- | Calculated fields: an eval in the search, or a pipeline step |
LOOKUP- | Upload the CSV as a lookup and use it in searches and detections |
[script://…] [powershell://…] | An exec input on the agent, or a cloud integration for the same data |
| Modular inputs without a cloud integration yet | Listed by name in the report; send the data through the HTTP API or an agent |
| Windows file paths | The agent reads files on Linux; use the Windows Event Log input on Windows |
(?i) | Inline regex flags: write the pattern case-explicitly |
| Extractions on a field (… in field) | Fold into the main pattern or a pipeline step |
| Data models and accelerations | Not converted; see LogPulse data models for normalised fields |
Frequently asked questions
- Can I move from Splunk to LogPulse without rebuilding my parsers?
- Yes, for most of them. The converter reads props.conf and transforms.conf, including those inside Splunkbase apps, and turns EXTRACT, REPORT, KV_MODE, line breaking and field aliases into LogPulse parser rules with the same field names. What it cannot convert is listed with the reason.
- Do Universal Forwarders work with LogPulse?
- Not directly yet: receiving the Splunk forwarder protocol (S2S) is on the roadmap. Install the LogPulse agent next to the forwarder, or let the forwarder send a plain TCP copy to an agent during a trial.
- Does LogPulse use Splunk CIM field names?
- Yes. The Splunk Common Information Model is the LogPulse field standard, so fields such as src, dest, user, action and signature have the same names. Older LogPulse names stay available as aliases.
- Do my Splunk sourcetypes stay the same?
- Yes. Converted inputs and cloud integrations use the sourcetypes of the Splunk add-ons, so searches that filter on sourcetype keep finding the data. The index is the kind of data, such as firewall or iam, and the source is the product.
- What happens to add-ons that pull data from a cloud API?
- Many modular inputs, such as those for Okta, Duo, Proofpoint, Netskope, Meraki, Google Cloud, Google Workspace, Jira and Confluence, convert to the matching LogPulse cloud integration. You enter the credentials again; LogPulse never reads them from the app.