Syslog receiver¶

The exporter can listen for syslog and have OPNsense push its logs to it, instead of polling the API for them. It parses what it receives and enriches it from the OPNsense API before shipping it on through the log pipeline.

This is off by default. It needs configuration on both sides: the receiver on the exporter, and a logging target on the firewall.

If a log line from your firewall is not parsed the way you expect, the parsers are open source: see internal/logship/ on GitHub and report the format with a sample line.

Why this exists¶

A generic syslog collector (Alloy, Vector, rsyslog) can already receive these logs. What it cannot do is understand them. A raw firewall log line looks like this:

16,115,,6cafbc76-9f4d-4150-949e-e3c37dd0a596,igb0,match,block,in,4,0x0,,58,0,0,none,6,tcp,40,203.0.113.9,198.51.100.4,54321,22,...

Nothing there tells you which rule that was, what igb0 is called, or who owns those addresses. The exporter already holds an authenticated OPNsense API client, so it can resolve all of it at ingest - which is the one thing a general-purpose log collector structurally cannot do.

Set up the exporter¶

opnsense-exporter \
  --logs.enabled \
  --logs.syslog.enabled

The receiver listens on port 5514 for both UDP and TCP by default. (Not 514: that is a privileged port, and the container runs as a non-root user.)

If you run the exporter in a container, publish the port for both protocols - missing one is the most common reason nothing arrives:

ports:
  - "5514:5514/udp"
  - "5514:5514/tcp"

Set up the firewall¶

In the OPNsense UI: System → Settings → Logging → Targets → +

Then Apply.

Three of those deserve explanation.

Leave Applications, Levels and Facilities empty. Empty means all - that is how OPNsense's target model works. Selecting nothing forwards everything the box logs, which is what you want: the receiver ships unknown programs as generic records rather than discarding them, so anything you don't explicitly model still reaches Loki.

Prefer TCP. UDP is OPNsense's default and it works, but datagram loss is silent and unrecoverable - you will never know what you didn't receive. Firewall logs are the highest-volume stream on the box and the one most worth not losing.

Tick rfc5424. OPNsense leaves this off by default, which sends the legacy BSD format. The receiver parses both, so it will work either way - but RFC5424 carries a proper timestamp with a UTC offset, where the legacy format has no year at all and must be inferred.

Filtering (optional, off by default)¶

The receiver ships everything unless you tell it otherwise - an unknown program is never dropped, because that is the point of a catch-all receiver and your box runs plugins we have never heard of.

If you do pay per GB of ingest, a firewall at debug level is loud. radvd logs a timer tick every two minutes and says nothing; HAProxy logs every request. So:

--logs.syslog.exclude-programs=radvd,cron     # drop the known-useless
--logs.syslog.min-severity=notice             # drop info and debug, keep notice and worse

Syslog severity is inverted (0 = emerg, 7 = debug), so --min-severity=notice keeps everything at or above notice. Anything dropped is counted in opnsense_exporter_logs_rejected_total{reason="filtered"}, so nothing is discarded silently.

You can also filter on the firewall itself (the target's Applications/Levels/Facilities selectors). Use that for coarse cuts you never want to see; use the exporter for tuning you might change your mind about, since it needs no firewall config edit.

Derived metrics and sampling¶

The receiver counts what it parses. Every firewall, HAProxy, sshd, DHCP, audit, IDS, gateway, VPN lifecycle and supported FreeRADIUS access line it recognises increments a Prometheus counter at /metrics, so you get rates and totals without querying Loki at all:

No IP, port, SID, hostname, username, MAC, NAS/client identity, certificate subject or serial, IKE identity, SPI, reply text, credential or signature text becomes a label. Configuration-scale values such as gateway, interface, rule, backend, server and VPN connection names are protected by the per-family --logs.max-metric-keys budget. This is on by default; turn it off with --exporter.disable-log-events.

Gateway monitor (dpinger) alarms¶

On OPNsense 27.1.a_40, the recognised RFC5424 form is:

<PRI>1 <timestamp> <host> dpinger <pid> - [meta sequenceId="<sequence>"] MONITOR: <gateway> (Addr: <address> Alarm: none -> down RTT: <milliseconds> ms RTTd: <milliseconds> ms Loss: <percent> %)

The clear form is identical except for Alarm: down -> none. These are the only two transition directions currently counted: none -> down becomes event="alarm_started"; down -> none becomes event="alarm_cleared". opnsense_log_events_gateway_total carries only event and gateway (along with the exporter's standard opnsense_instance attribution). The address, RTT, RTTd, loss, sequence ID and message text are deliberately not metric labels.

Matching lines are enriched with gateway.event, gateway.name, gateway.address, gateway.alarm.previous, gateway.alarm.current, gateway.rtt_ms, gateway.rttd_ms and gateway.loss_percent. A dpinger line that does not match this grammar still ships as a generic record; it is not counted as an inferred gateway transition.

CARP state changes and demotion (kernel)¶

CARP transitions come from the FreeBSD kernel, not from OPNsense, under the RFC5424 APP-NAME kernel. They require net.inet.carp.log=1, which is the FreeBSD default. On OPNsense 27.1.a_40 (FreeBSD 15) the two recognised forms are:

<PRI>1 <timestamp> <host> kernel - - [meta sequenceId="<sequence>"] <<kpri>>[<kuptime>] carp: <vhid>@<device>: INIT -> BACKUP (initialization complete)
<PRI>1 <timestamp> <host> kernel - - [meta sequenceId="<sequence>"] <<kpri>>[<kuptime>] carp: demoted by <delta> to <total> (pfsync bulk start)

The <6>[754] prefix ahead of carp: is the kernel's own priority and monotonic counter. It is part of the message, both numbers vary freely, and the receiver tolerates it being absent.

event is closed to three values. A <FROM> -> <TO> line is state_changed, with from and to closed to master, backup or init (lowercased) and the OS device and VHID in interface and vhid. A demoted by line is demoted when the delta is positive and promoted when it is negative — FreeBSD has no separate promotion line, so the sign of the delta is the whole distinction. A demotion is global to the node and names neither an interface nor a VHID, so from, to, interface and vhid are all empty on those series.

The kernel's cause — initialization complete, master timed out, hardware interface up, pfsync bulk start, pfsync bulk fail, service disruption, and whatever a future FreeBSD adds — is deliberately not a metric label, and is not bucketed into a reason_class either: it is open-ended free text across FreeBSD versions. It ships on the record as carp.reason, alongside carp.event, carp.state.previous, carp.state.current, carp.interface, carp.vhid, carp.demotion.delta and carp.demotion.total. Those last two are the numbers that explain why a node demoted, which the opnsense_carp_demotion current-state gauge cannot retain.

Because this parser claims kernel, it sees every kernel line on the box. Anything that is not one of the two forms above — link-state changes, watchdog resets, ZFS, USB, arp — is not claimed: it ships as a generic record exactly as it did before, and is never counted as an inferred CARP transition.

OPNsense's own CARP syshook also logs transitions under APP-NAME opnsense, carrying the admin's VIP description. That source is deliberately not parsed: opnsense is the catch-all app-name for every log_msg() call on the box, so claiming it would put an unbounded variety of unrelated lines through a CARP matcher.

UPnP / NAT-PMP mapping events (miniupnpd)¶

The os-upnp plugin's daemon logs under the RFC5424 APP-NAME miniupnpd. Five forms are recognised, and all five are failures or expiries - see the warning below about what is not here:

<PRI>1 <timestamp> <host> miniupnpd <pid> - [meta sequenceId="<sequence>"] remove port mapping <external_port> UDP because it has expired
<PRI>1 <timestamp> <host> miniupnpd <pid> - [meta sequenceId="<sequence>"] could not find nat rule to delete iport=<internal_port> addr=<token>
<PRI>1 <timestamp> <host> miniupnpd <pid> - [meta sequenceId="<sequence>"] could not find redirect rule to delete eport=<external_port>
<PRI>1 <timestamp> <host> miniupnpd <pid> - [meta sequenceId="<sequence>"] Unauthorized to remove PCP mapping internal port <internal_port>, protocol UDP
<PRI>1 <timestamp> <host> miniupnpd <pid> - [meta sequenceId="<sequence>"] could not open lease file: <path>

They map onto opnsense_log_events_upnp_total{event,result,protocol} (plus the standard opnsense_instance label) with fully closed values:

expired is a mapping reaching the end of its lease and being torn down - the lifecycle working. cleanup_failed means the daemon and pf disagree about which rules exist. unauthorized is a PCP client trying to remove a mapping it does not own, which the plugin's default secure_mode refuses.

Ports ship on the record as upnp.port.external and upnp.port.internal alongside upnp.event, upnp.result and upnp.protocol, and are deliberately not metric labels - an ephemeral client port is unbounded and would mint a series per mapping. The raw record is therefore the only place the specific mapping behind a failure can be identified. The daemon's opaque addr= token and the lease-file path are not even attributes: they stay in the message body.

Recognised on OPNsense 26.7.1_1 and 27.1.a_40 with miniupnpd 2.3.9_2,1 / os-upnp 1.9.

FreeRADIUS access outcomes and credential handling¶

OPNsense 27.1.a_40 with os-freeradius 1.10.2 emits authentication outcomes under the RFC5424 APP-NAME radiusd. The captured normal-service forms are Login OK and Login incorrect. They become:

The corresponding counter is opnsense_log_events_radius_total{event,result,client_scope} (plus the standard opnsense_instance label). Usernames and the configured RADIUS client name are not attributes or labels.

FreeRADIUS is a stricter confidentiality boundary than the other syslog parsers. For every radiusd frame, the exporter replaces the original hostname, PID, structured data and message before parser dispatch, generic enrichment, debug capture, shape-key calculation, metric derivation, sampling or queue admission. Recognised access outcomes become fixed code-defined messages. Any other radiusd message becomes a fixed sanitized generic record and is not counted. A malformed RFC5424 or RFC3164 frame with a recognizable radiusd program token also fails closed to that safe generic record. This prevents an enabled FreeRADIUS password-log option from reaching the raw generic or debug-capture paths.

Configure the plugin and target manually on OPNsense:

1. In the FreeRADIUS general settings, select syslog as the log destination and enable authentication-request logging.
2. Leave both accepted-password (auth_goodpass) and rejected-password (auth_badpass) logging disabled.
3. Add an OPNsense syslog Target for program radiusd pointing at the exporter's syslog listener.

The exporter does not create that Target or write firewall rules.

Normal Accounting Start, Interim-Update and Stop requests returned Accounting-Response in the capture but emitted no syslog records. Accounting therefore remains unsupported and is not inferred from request traffic.

IPsec and OpenVPN tunnel lifecycle events¶

The vpn family counts tunnel lifecycle transitions for both VPN backends, from charon (IPsec) and from the per-instance openvpn_* programs. Its vocabularies are closed and resolved in code:

connection is the fourth label and the only one that is not code-defined. It is the configured name of the IPsec connection or OpenVPN instance, resolved from the id in the log line against the inventory the exporter already fetches for its IPsec and OpenVPN collectors (the same enrichment that adds ipsec.connection / openvpn.instance to every record). It is empty when the id could not be resolved, and it is never the raw UUID.

connection is populated for IPsec and is ALWAYS EMPTY for OpenVPN. That is by design, not a fault. Read this before reporting an empty label as a bug:

• IPsec: populated. Every captured charon line carries the connection id, so the label resolves whenever the tunnel is still in the fetched inventory. It is empty only for a tunnel deleted since the last inventory refresh, or before the first refresh completes.
• OpenVPN: always empty. None of the four captured OpenVPN lines contains the instance id — OpenVPN prints it only on its MANAGEMENT: Client connected from /var/etc/openvpn/instance-<uuid>.sock line, which is not one of the four shapes this family parses. There is therefore nothing on those lines to resolve, and the label will never populate. Operator workaround: attribute those events using the program attribute on the raw log record (openvpn_server40, openvpn_client2), which names the instance. Nothing is inferred from the program-name suffix: the exporter's OpenVPN instance inventory has no numeric key to match 40 against, so guessing would be a fabrication rather than a resolution.

Recognised grammar¶

Only the grammar captured on an isolated testbed running OPNsense 27.1.a_40 with strongSwan 6.0.7 and the OpenVPN server package 2.7.5 is counted. The shapes, with every value replaced by a placeholder:

charon:  <thread>[<tag>] <<connection-id>|<n>> generating IKE_AUTH response <n> [ N(AUTH_FAILED) ]
charon:  <thread>[<tag>] <<connection-id>|<n>> IKE_SA <id>[<n>] established between <local>[<local-id>]...<remote>[<remote-id>]
charon:  <thread>[<tag>] <<connection-id>|<n>> giving up after <n> retransmits
charon:  <thread>[<tag>] <<connection-id>|<n>> IKE_SA deleted

openvpn: <peer-context> [<common-name>] Peer Connection Initiated with [AF_INET]<address>:<port>
openvpn: <peer-context> SENT CONTROL [<common-name>]: 'AUTH_FAILED' (status=<n>)
openvpn: <peer-context> VERIFY ERROR: depth=<n>, error=<error text>
openvpn: <peer-context> SIGUSR1[soft,ping-restart] received, client-instance restarting

The strongSwan <thread> number and <tag> subsystem vary line to line and are not anchored on. Mapping: N(AUTH_FAILED) in a generated IKE_AUTH response is authentication_failed; IKE_SA … established between … is established; giving up after N retransmits is liveness_failed; IKE_SA deleted is terminated. On the OpenVPN side Peer Connection Initiated with is established, 'AUTH_FAILED' is authentication_failed, a VERIFY ERROR: in OpenVPN's depth=<n>, error=<text> form is certificate_failed, and the ping-restart SIGUSR1 is terminated.

Three points of tolerance, so the recognised set is neither narrower nor wider than it looks:

• N(AUTH_FAILED) is counted wherever it appears in the response's payload list, not only when it is the sole notify. Only generated responses count — the box rejecting a peer. The initiator-side parsed IKE_AUTH response … N(AUTH_FAILED) (the far end rejecting our credentials) is a different event, was not captured, and is deliberately not counted.
• certificate_failed matches OpenVPN's format string, so expired, revoked and depth-N rejections all count as the same event class. A line that merely mentions the words does not.
• [AF_INET6] is accepted alongside the captured [AF_INET], derived from OpenVPN's own address-family tag rather than from a capture, so an IPv6 peer's establishment is not silently uncounted.

What is deliberately not counted¶

Anything else, including every stable-release form. A read-only search of a production OPNsense 26.7.1_1's retained logs found zero usable lifecycle or failure records, so no stable-release grammar exists to parse and none is inferred from the development formats. Unmatched and version-new lines still ship in full as generic records with the body verbatim, and still appear in the debug capture as unmodelled signal — they are simply not counted as a transition the exporter did not witness.

These lines were captured for real and are deliberately left generic:

Identity handling¶

The captured lines carry usernames, certificate subjects and serials, IKE identities, peer addresses and ports. The parsers extract none of them: the only attributes they add are vpn.backend, vpn.event and vpn.result. Those values stay in the record body, which ships verbatim exactly as it did before this family existed, so an investigation still has the detail while the metric stays bounded and non-identifying.

A matched line keeps everything it had while it was generic: the peer.* address resolution, the interface resolution, the tunnel-id resolution and the verbatim body are all still there, with the VPN attributes added on top. Structured parsers normally skip the address scan — a parser that reports its own source and destination would otherwise report them twice — but these two extract no address at all, so the scan still runs for them and no existing query on peer.* for charon or openvpn_* lines changes behaviour.

Because the counters already carry the totals, you can stop shipping the raw lines they count and keep only the ones worth reading. That is --logs.syslog.sample (off by default):

--logs.syslog.sample     # keep firewall block/reject and HAProxy errors, drop the rest

With sampling on, the receiver keeps firewall block/reject lines and drops the passes, keeps HAProxy state changes and errors and drops per-connection noise, and keeps every low-volume program (sshd, DHCP, audit, IDS, gateway, RADIUS and the VPN lifecycle events) in full. A line is only ever dropped after its metric has been counted, so the counters stay complete even though the log stream is not. Sampling requires the log_events collector to be on (the exporter refuses to start otherwise), because counting first is the whole point.

Every shipped line then carries a sampled="true" attribute so a consumer knows the stream is incomplete and must use the counters for totals. Turn that stamp off with --logs.syslog.sampled-attribute=false if you would rather not have it.

TLS transport (optional)¶

The receiver can take syslog over TLS in addition to (or instead of) plain UDP/TCP - OPNsense's tls4/tls6 transports. It matters when the firewall ships across an untrusted segment; on a LAN-local link it is unnecessary.

opnsense-exporter \
  --logs.enabled \
  --logs.syslog.enabled \
  --logs.syslog.listen-tls=:6514 \
  --logs.syslog.tls-cert-file=/etc/exporter/syslog.pem \
  --logs.syslog.tls-key-file=/etc/exporter/syslog.key \
  --logs.syslog.tls-client-ca-file=/etc/exporter/senders-ca.pem   # optional, see below

Set --logs.syslog.tls-client-ca-file too: it's the only real sender authentication syslog has. With it, a sender must present a client certificate signed by that CA - the peer allowlist only filters by IP, it doesn't prove who's on the other end. Left empty, the listener encrypts but accepts any TLS client. On the firewall, set the target's Transport to TLS(4) and point it at the TLS port.

What you get¶

Structured parsers run for these programs; everything else ships as a generic record with its message body verbatim and its envelope as metadata.

Every record, structured or generic, also gets:

• an opnsense.subsystem attribute (opnsense_subsystem in Loki) with a value like firewall, auth, dhcp, ipsec, vpn, upnp, proxy, routing or ups, so you can select a whole class of events without enumerating program names;
• any IP address mentioned anywhere in the message resolved to a hostname, MAC and scope (self/local/remote);
• any interface device resolved to its friendly name (vtnet0 → LAN);
• for IPsec and OpenVPN, the tunnel UUID resolved to its name - charon logs <5e891b0c-…|8> sending DPD request, which is unreadable; the exporter turns it into ipsec.connection: "site-to-site" because it already has the API.

Firewall lines specifically¶

Firewall (filterlog) lines are parsed into structured fields and enriched:

So the line above arrives looking like this:

{
  "body": "block in on WAN: 203.0.113.9:54321 -> 198.51.100.4:22 (tcp)",
  "attributes": {
    "action": "block", "direction": "in",
    "interface": "igb0", "interface.name": "WAN",
    "rule.description": "Default deny / state violation rule",
    "src.ip": "203.0.113.9", "src.scope": "remote",
    "dst.ip": "198.51.100.4", "dst.scope": "self", "dst.service": "ssh",
    "tcp.flags": "S", "tcp.window": "64240"
  }
}

Every other program OPNsense routes through syslog-ng - the auth/audit trail (SSH logins, "action allowed X for user root"), configd, unbound, dnsmasq, kea, haproxy, frr, ipsec, openvpn, package installs, and a catch-all for everything else - ships as a generic record with its raw body and its envelope attributes.

Querying it in Loki¶

Two index labels select the stream; everything else is structured metadata, filtered with | after it:

{service_name="opnsense-exporter", service_instance_id="opnsense"}
  | opnsense_subsystem="firewall" | action="block" | src_scope="remote"

A real record from a live box, as it lands:

{
  "action": "pass", "direction": "in",
  "interface": "pppoe0", "interface_name": "AAISP",
  "rule_description": "[AAISP] Allow inbound ICMP (monitors)",
  "rule_id": "7ed3ec06-ecf8-4ca8-9a2a-bb346967850f",
  "src_ip": "3.123.217.248", "src_scope": "remote",
  "dst_ip": "81.187.237.31", "dst_scope": "self",
  "protocol": "icmp", "ip_version": "4", "network_type": "ipv4",
  "opnsense_source": "syslog", "opnsense_subsystem": "firewall"
}

Useful starting points:

{service_name="opnsense-exporter"} | opnsense_subsystem="audit"                    # who changed the config
{service_name="opnsense-exporter"} | opnsense_subsystem="auth" | auth_result="failed"
{service_name="opnsense-exporter"} | opnsense_subsystem="firewall" | action="block" | dst_scope="self"
{service_name="opnsense-exporter"} | program="filterlog" | src_hostname="WINSRV"

opnsense.source and opnsense.subsystem (namespaced so they can never collide with a Loki-reserved key) are the two attributes worth indexing if you query them often. Both ride on the OTLP resource for exactly that reason, and both can be promoted to real Loki labels with a one-off tenant config change - see the Loki label model. Nothing else should be promoted: src_ip as a label is one stream per address.

Multi-line messages¶

syslog-ng frames with newlines, not octet counts, so a message that itself contains newlines - a configd Python traceback, a cron command spanning lines - arrives as several frames of which only the first carries a <PRI> header. The receiver rejoins them: a line that does not begin with < cannot start a new message, so it is appended to the previous one. The assembled message is capped at 64KB like any other (the overflowing tail is dropped and counted as oversized), and a message with no successor to complete it is flushed after 250ms rather than waiting for the next line. Octet-counted frames carry their own length and are passed through untouched, as are UDP datagrams - one datagram is always exactly one message.

Resolving rule ids¶

A filterlog rule id is either a rule UUID (for rules you wrote) or a content hash (for the auto-generated ones: anti-lockout, default-deny, bogon blocks, DHCP-allow, IPv6 RFC4890). The rule inventory the exporter already collects only contains the first kind - on a stock box that is a small minority of the rules that actually match traffic. list_rule_ids resolves both, which is why the receiver uses it.

Lines where the rule id is 0 (NAT and floating-rule matches) carry no rule id at all by design. They get rule.ref (rule #16.115) instead of a description.

What you do not get¶

The receiver is not a total replacement for the API-polling sources. Three things have no usable syslog path, and their poll lanes remain:

• Per-query DNS (--logs.unbound.enabled) - Unbound's per-query log with blocklist/policy/rcode comes from OPNsense's reporting database. What arrives over syslog under program("unbound") is the resolver daemon log (cache maintenance, errors), which is a different stream entirely.
• CrowdSec (--logs.crowdsec.enabled) - CrowdSec logs to file only. Nothing it produces reaches syslog, and it ships no syslog notification plugin.
• Suricata payloads (--logs.ids.enabled) - the syslog copy of an EVE alert never carries the packet payload. See below.

Suricata alerts: pick ONE path¶

The receiver does parse Suricata EVE alerts when the box forwards them (the IDS syslog_eve setting, off by default). So does the ids poll lane, from the richer file-based eve.json.

Running both would ship every alert into Loki twice, with no dedupe - and a duplicated security alert is worse than a missing one, because it silently inflates every count anyone builds on it. The exporter therefore refuses to start with --logs.syslog.enabled and --logs.ids.enabled both set, rather than guessing which you meant.

Take the receiver unless you need the payload. Then turn syslog_eve on in the IDS settings and leave --logs.ids.enabled off.

Records from both paths carry the same attribute names (alert_sid, signature, src_ip, …), so a dashboard or alert rule does not care which path you chose.

Security¶

Syslog is unauthenticated. Anything that can reach the port can inject arbitrary log records into your observability stack. This is inherent to syslog, not to this implementation.

• Bind the receiver to a trusted interface.
• On a shared network, set --logs.syslog.allowed-peers to the firewall's address (--logs.syslog.allowed-peers=10.0.0.254/32). Senders outside the list are dropped and counted in opnsense_exporter_logs_rejected_total{reason="peer"}.
• Messages are capped at 64KB, concurrent TCP connections at --logs.syslog.max-conns, and idle connections time out - a peer cannot exhaust memory or goroutines.

Migrating from the poll lanes¶

--logs.firewall.enabled, --logs.diaglog.enabled and --logs.scopes are removed. The exporter fails to start with an error naming the replacement if any of them (or their environment variables) are still set.

Troubleshooting¶

Nothing arrives. Check opnsense_exporter_logs_shipped_total{source="syslog"} is climbing. If it is flat: confirm the port is published for both UDP and TCP, confirm the target on the firewall is enabled and points at the right address, and check opnsense_exporter_logs_rejected_total{reason="peer"} in case an allowlist is dropping it.

Records arrive but aren't enriched. Check opnsense_exporter_logs_enrich_refresh_errors_total and opnsense_exporter_logs_enrich_last_refresh_timestamp_seconds - the API client may be failing while the receiver keeps working. Enrichment failure never drops a record, so this shows up as plainer logs rather than missing ones.

Rules aren't labelled. A steady opnsense_exporter_logs_enrich_misses_total{table="rules"} means the rule snapshot is behind the box. It refreshes every 60s and on a miss (rate-limited), so a persistent rate points at an API permission problem on diagnostics/firewall/list_rule_ids.