← Back to Blog

An XDP and Suricata Firewall on a Small Gateway: Every Failure Path Passes the Packet

·26 min read
eBPFXDPSuricataLinuxsystemdNetworkingSecurity

A cyberpunk neon infographic titled "NODEGUARD", subtitled "XDP / eBPF firewall, observability, security" over a rain-slicked neon city, showing a Suricata IDS panel feeding a blocklist of addresses with TTLs, a malicious traffic stream marked DROPPED (XDP) beside a legitimate one marked PASSED (XDP), a central server whose Linux kernel lane runs the NIC XDP hook into eBPF, and a live status panel of packet counters under a green FAIL-OPEN, ENFORCEMENT OFF badge.

Two small Fedora 44 boxes run a passive Suricata IDS and an XDP program that blocks what it detects. One is an internet gateway whose operator depends on it for connectivity. The other is a node reachable only over its management tunnel. Both run stock distribution packages, including the Suricata 8.0.6 RPM with a 52,667-rule ruleset.

Where it sits is the whole design constraint: in the path of the only way back into the box. A security system on that path has one property that matters more than its detection quality, and it is not a feature you add at the end. Every failure path has to pass the packet.

The selection criterion was therefore never detection quality. It was failure direction, and it disqualified every inline option before merit came into it. NFQUEUE stops matched traffic entirely when its userspace listener dies and no bypass rule exists. AF_PACKET copy-mode IPS needs a dedicated two-port layer 2 pair with Suricata as the forwarder, which is fail-closed by construction and physically impossible on a multi-VLAN router. Suricata's own eBPF machinery is compiled into the Fedora 8.0.6 RPM, but the package ships zero .bpf objects, and built by hand it is capture bypass rather than policy. xdp-filter from xdp-tools speaks the libxdp dispatcher, the meta-program that multiplexes several XDP programs on one interface by priority, but it supports neither CIDR entries nor per-entry TTLs, so it stays on the shelf as a break-glass tool. A userspace-fed nftables set is the obvious remaining alternative and I did not evaluate it; that is a gap in the record, not a verdict on nftables.

The XDP program is 415 lines of BPF C including its comments, 287 of them code, and it was the smallest part of the work. What took the rest of the time was everything that follows from that one constraint.

One disclosure first, because the rest of this post argues against claiming a state you have not verified. Enforcement went live on one of the two hosts on the day of writing, after a 44 hour dry-run; the second follows after a week of clean operation. The threat-intel feed loader enforces three feeds on both, and the anomaly detector runs alerting-only. What follows is a design argument backed by drills, rehearsals and one soak, not by months of production.

The code is nodeguard. Every fragment quoted below is in that repository, alongside the arc42 design document, the ADRs the arguments come from, and the failure-mode table; the per-host addresses and network layout are not, and never will be.

Every failure path passes the packet

The whole contract is three lines at the top of the program, and everything below is an attempt to be worthy of it:

// SAFETY: every failure path returns XDP_PASS (see design.md section 2).
// The ONLY drop is an unexpired blocklist hit on a source address that did
// not match the allowlist, the WireGuard port pass, or the kill switch.

Read the per-packet decision as a list of ways to leave. A truncated Ethernet header passes, and so does a non-IP ethertype, before any configuration is even read; a truncated IP header passes as soon as the handler bounds-checks it. A set kill switch passes. A UDP datagram to the management tunnel's live port passes, before either map is consulted. Then an allowlist hit passes, a blocklist miss passes, and a blocklist hit whose expiry has gone by passes. Fifteen return XDP_PASS statements against two return XDP_DROP, one per address family, and the drop is the last thing either handler reaches:

	bv = bpf_map_lookup_elem(&block4, &key);
	if (!bv) {
		count(ST_PASS);
		return XDP_PASS;
	}
	if (bv->expiry_ns && bpf_ktime_get_ns() >= bv->expiry_ns) {
		count(ST_PASS_EXPIRED);
		return XDP_PASS;
	}
	__sync_fetch_and_add(&bv->hits, 1);
	count(ST_DROP_V4);
	return XDP_DROP;

Six of the eight stats slots name a pass reason and two name a drop, one per address family. Parse failure, non-IP, allowlist, WireGuard port and expiry each get their own number, so "we are passing a lot" and "we are passing a lot because the parser is falling over" are different graphs. The kill switch and an ordinary blocklist miss share the generic pass counter, which is the one place the audit trail is coarser than the decision tree: a latch is told apart by reading the config slot, not by the counters. And the counters cannot influence a verdict: count() does a map lookup, tests the pointer, increments, and returns void, so a missing slot is silently skipped and there is no path from a broken counter back into the decision.

The expiry has to live in the kernel, not in the sweeper

The obvious place to expire a block is the process that created it. A sweeper walks the map every ten minutes and deletes anything past its deadline. It is simple and it is testable, which is exactly why it is the trap.

If expiry is enforced from userspace, a dead daemon leaves every existing block enforced forever: the failure of a convenience component becomes permanent denial of service against whatever happened to be blocked at the time, false positives included. On a box whose enforcement point is its own uplink, that turns a software defect into a physical-presence recovery.

So the block value carries an absolute deadline and the kernel checks it on every hit:

struct block_val {
	__u64 expiry_ns; // INVARIANT: CLOCK_MONOTONIC ns; 0 = permanent (manual entries only)
	__u64 hits;
};

Userspace computes that field with CLOCK_MONOTONIC so that bpf_ktime_get_ns() in the program compares against the same clock base. An entry past its deadline passes while still sitting in the map, counted as pass_expired. The sweeper is garbage collection: it reclaims map slots, and by the time it runs, enforcement expiry has already happened. ADR 0003 freezes that against erosion: the sweeper must never grow enforcement semantics, and any TTL behavior change lands in the kernel source, not in Python.

The system property that falls out of this is the one I care about. With the responder dead, Suricata dead, and the sweeper dead, the blocklist stops enforcing anything within one TTL and the steady state is a pass-through no-op. The entries themselves stay in the map with nothing left to reclaim them; only enforcement decays. Nothing has to be alive for the box to stop blocking.

Be honest about what that costs. Protection quietly decays when components die and the datapath never signals the loss, so liveness has to be alarmed from somewhere else, and every block is lost on reboot besides, because the pins live on bpffs and CLOCK_MONOTONIC restarts. Fail-open is a security trade in the plainest sense: an attacker who can crash the program or prevent the attach gets an unfiltered path.

Any auto-blocker can be aimed at its owner

This is the failure mode I think about most, because it is the one where the system becomes a weapon pointed at its owner. It is not an eBPF problem, and detection here is passive by decision and never inline, so it is not an in-path IPS problem either. It is the shape of every alert-to-block pipeline: every fail2ban, every WAF auto-ban, every SIEM playbook that calls a firewall API. The hazard is a userspace component turning an observation into a policy write.

A blocklist keyed on IDS alert source addresses is trivially poisonable. An attacker who cannot see any return traffic sends one UDP or ICMP packet with a forged source: your DNS resolver, or a relay your management tunnel depends on. If that packet trips a severity-1 signature, a naive responder inserts the forged address into the block map and the host severs its own lifeline, from a single packet, with no position on the path and no need to observe the result.

TCP is different in kind. Sequence numbers make completing or continuing a handshake blind infeasible. So the gate is not a heuristic about the attacker, it is a demand for evidence that the peer can see our traffic:

    # SAFETY: anti-spoofing gate - blocks require evidence of a real flow
    # (TCP bidirectional, or UDP/ICMP hand-promoted via sids.conf); a
    # single spoofed packet cannot trigger a block.
    if proto != "TCP" and sid not in st.udp_ok:
        log(f"WOULD BLOCK ({proto.lower() or 'unknown'}, "
            f"not eligible): {tag}")
        return
    if proto == "TCP":
        if (flow.get("pkts_toclient", 0) < 1
                or flow.get("pkts_toserver", 0) < 2):
            log(f"WOULD BLOCK (no bidirectional flow evidence): {tag}")
            return

Neither counter is evidence on its own. Two packets toward the server is two forged packets; one packet toward the client only proves our host answered a forgery. The conjunction, in that order, is the proof: we emitted something carrying a value only the return path can learn, and then the peer sent something after that.

UDP and ICMP are log-only. A signature can be promoted by hand with a udp-ok line naming its SID, Suricata's signature id, and the config file's own header demands a written justification for why blind spoofing cannot trigger that signature first. ADR 0002 calls per-SID promotion a standing invitation to erode the gate, which is exactly right; the file shipped in the repository contains zero promotions.

What the gate does not do is the half that gets left out of write-ups. Real attacks that fit in one spoofable packet, amplification probes, ICMP scans, one-shot exploit datagrams, are never blocked automatically; they are logged and blockable by hand. The argument is scoped to a blind attacker throughout: someone who can observe your traffic can complete a handshake from any address and poison the list anyway. And the gate depends entirely on the alert records carrying flow.pkts_toserver and flow.pkts_toclient. If those fields stop appearing, every TCP alert fails and the responder quietly becomes a logger.

The dry-run is where this stopped being theory. The procedure calls for ENFORCE=no for 48 to 72 hours minimum while somebody reads every WOULD BLOCK line; the first host was armed at 44, which is the sort of small deviation worth recording rather than rounding away. In that window the anti-spoofing gate rejected all 362 single-packet reputation alerts and blocked nothing. What that proves is bounded and worth stating precisely: all 362 were reputation alerts on single spoofable packets, none carried bidirectional TCP evidence, and every one was discarded at the gate rather than reaching the inbound, allowlist and rate-cap checks behind it. Whether those downstream checks would have discarded them anyway was not measured, so the number is not a count of self-inflicted blocks prevented.

A health check cannot verify a path it has exempted

The datapath checks the allowlist before the blocklist, deliberately, so that no userspace ordering bug can block a protected range:

	if (bpf_map_lookup_elem(&allow4, &key)) {
		count(ST_PASS_ALLOW);
		return XDP_PASS;
	}
 
	bv = bpf_map_lookup_elem(&block4, &key);

Now consider the watchdog. Once a minute it runs a small set of lifeline probes; the shipped configuration pings the default gateway and sends a hand-built query to a public resolver. Both of those targets are allowlisted, because you do not want your own firewall breaking your own health check, which means their return traffic hits the first lookup above and never reaches the blocklist lookup at all.

So if an encoder defect, an inverted comparison, or one over-broad CIDR causes the program to drop the entire internet, every lifeline probe still succeeds and the watchdog reports green: the health check is structurally blind to the single failure this system can cause. An earlier revision of the design had precisely that blind spot; the review found it before deployment.

The fix is a second probe with the opposite property, the canary: a TCP connect to a target deliberately never allowlisted, so its return packets traverse the blocklist lookup like real traffic. Neither probe is a health check on its own; the diagnosis lives in the disagreement between them. Below, $ks is the kill-switch value read back from the config map, so this is the branch that runs while enforcement is still on; counter reads a consecutive-failure count and lifeline_ok is set if any lifeline answered:

if [ "$ks" = "0" ]; then
    if [ "$(counter canary_fail)" -ge 3 ] && [ "$lifeline_ok" -eq 1 ]; then
        drops=$(python3 "$NG_MAP" stats --json 2>/dev/null || echo '{}')
        soft_off_or_detach "OVER-BLOCK suspected (canary $CANARY_IP:${CANARY_PORT:-443} failed 3 cycles, lifelines fine); stats: $drops"
    elif [ "$(counter lifeline_fail)" -ge 5 ]; then
        soft_off_or_detach "all lifelines failing 5 cycles (may be an upstream outage)"
        set_counter lifeline_fail 0
    fi
else
    # ... latched branch: detach if lifelines stay dead 10 more cycles,
    # then the bounded auto re-arm

Canary failing while lifelines pass is over-blocking. Canary failing while lifelines also fail is an outage, and gets the other branch. Both actions turn enforcement off, and the counter reset behind the second only clears a failure tally; nothing in the enforcing branch turns anything on. The one automatic path that ever does sits behind that else, fenced three ways, and is the subject of the next section.

The part I would not have thought to add is that the rule is enforced by code rather than by discipline. The maps service refuses to reconcile the allowlist at all if any allow source covers the canary address:

    if a.canary:
        canary = ipaddress.ip_address(a.canary)
        for net in static_nets + gen_nets:
            if net.version == canary.version and canary in net:
                die(f"allow sources contain the canary target {canary} "
                    f"(entry {net}); the watchdog's over-block probe would "
                    "be blind. Remove the entry.", code=2)

ADR 0005 says why in one sentence: allowlisting the canary, even with good intentions during an incident, silently re-creates the blind spot. That is not a hypothetical operator, that is me at 2am wondering why one probe keeps failing.

The cost is real and accepted. A routine outage of the third-party canary target latches enforcement off with nothing locally wrong; that trade is made on purpose. Which target it is, and on which port, is a per-host value: the repository ships an example, and the real ones live in a private overlay next to the allowlists.

The kill switch has to read itself back

Turning enforcement off is a single map write to a config slot. It is hitless: no detach, no reattach, no link blip, which is what makes an aggressive automatic trip affordable in the first place.

The write goes through bpftool on a pinned map, and it can fail for reasons that have nothing to do with the caller: a missing pin after a bpffs reset, an SELinux denial, a tool absent or the wrong version. Taking the exit status as proof produces the one lie this tool must never tell. Mid-incident, operator and monitoring both believe the datapath is open, everyone stops looking, and the program goes on dropping.

    v=""
    if ng_cfg_set 1 1; then
        v=$(ng_cfg_get 1 2>/dev/null || true)
    fi
    if [ -n "$v" ] && [ "$v" != "0" ]; then
        touch "$marker"
        ng_log "kill switch SET ($(basename "$marker")); nodeguard passes everything" crit
    else
        ng_log "kill switch write FAILED; nodeguard is STILL ENFORCING (config[1] unchanged)" crit
        exit 1
    fi

The read-back is half of it. The other half is that the automated caller has somewhere worse to go when the gentle path fails:

soft_off_or_detach() {
    local why="$1"
    if /usr/local/sbin/nodeguard-off --watchdog; then
        ng_log "watchdog: $why; enforcement soft-off" crit
    else
        ng_log "watchdog: $why; soft-off FAILED, detaching XDP instead" crit
        systemctl stop nodeguard-xdp.service
    fi
}

Detaching the program blips the link, which is precisely the thing the kill switch exists to avoid; it is worse in every respect except the one that matters. A break-glass control that can fail needs a cruder control behind it that fails in the same direction.

The system is also allowed to un-break itself, exactly once. If the latch was set by the watchdog rather than by a person, and fifteen consecutive cycles come back fully clean, and it has not already done so since boot, the watchdog calls nodeguard-on and records the re-arm in a third config slot. A short upstream outage therefore heals itself, while a second latch in the same boot, and any manual soft-off, is human-only recovery. That bounded re-arm is the only automatic action anywhere in the system that closes the datapath again, which is why it is fenced on three sides rather than one.

The asymmetry in that path is deliberate: a failed re-arm leaves the latch markers in place and exits nonzero, so the recorded state never claims a recovery that did not happen, and map creation initializes the switch to enforcing only when the config map is brand new, so restarting the maps service can never silently re-arm what a human switched off.

Changing the interface you are connected through

The same reasoning shaped the first attach on the remote node. You are about to modify the only interface you can reach the machine through, the attach blips the link, and there is nobody in the building. So it runs detached under a transient systemd unit that writes its verdict to disk before any rollback, gives the operator ten minutes to confirm, reverts itself if nobody does, and retries at most once. Proving the drop worked had to run the other way, since nothing reaches that host inbound: block a cooperating public host, curl it, and show the returning SYN-ACK never arrives.

Four findings from the adversarial reviews

The repository records 28 findings from a 36-agent adversarial review: one blocker, nine major, eighteen minor. A second evaluation has since landed four more changes of the same kind. The numbers are less interesting than the shape. The findings the changelogs name cluster on one theme, the system reporting a state it had not verified, alongside ordinary correctness bugs: log-rotation and accounting errors in the responder, fragment handling in the datapath.

A failed tool read as a clean datapath. The obvious way to ask whether the program is attached is xdp-loader status, treating empty output as not attached. Empty output has two causes that look identical: nothing is attached, or the tool itself failed. Collapsing them is worse than useless here, because the watchdog's detached branch resets all three cycle counters and exits the cycle. A broken xdp-loader read as "detached" would zero the canary counter every minute, so the over-block trip could never reach its threshold while the program went on dropping: a broken diagnostic would have permanently disarmed the over-block detector. The fix is a three-valued contract, stated as an invariant that names its dependents:

# INVARIANT: three-valued contract - prints ids and returns 0 when the
# dispatcher was read; prints NOTHING and returns nonzero when xdp-loader
# itself failed, which callers must treat as UNKNOWN, never as detached.
# Dependents: nodeguard-watchdog, nodeguard-status, nodeguard-attach,
# nodeguard-detach, nodeguard-reload.

The unknown branch assumes still-attached and leaves the counters live, on the reasoning that a spurious soft-off fails open and is loud.

A sweep that deletes a block you just made. bpftool map dump is a snapshot, and the sweep runs on a ten-minute timer while the responder inserts blocks continuously. The damaging race is not the entry that vanished, it is the entry that came back: a block expires, the sweep snapshots it as expired, the responder re-blocks the same source milliseconds later, and the sweep deletes the fresh block by key. The attacker is silently unblocked and nothing logs an error. The fix is that the sweep does not trust its own snapshot; each candidate is looked up again under the lock the write path takes, and deleted only if it is still expired. The comment names which direction of the race may lose: leaving a corpse is harmless, deleting a fresh block is not, and that only holds because expiry is in the kernel, so an unswept corpse enforces nothing.

A restart that takes the WAN down. You edited the allowlist file. The obvious way to apply it is systemctl restart nodeguard-maps. But the attach unit declares Requires=nodeguard-maps.service, and a restart job propagates through Requires=, so restarting the maps unit stops the attach unit, whose ExecStop detaches the program. Detach and reattach is a multi-second carrier blip, so a read-only-looking config edit drops the link of the box you are editing it from. A reload job does not propagate, so the fix is one line of unit file and a note at the point of use:

# NOTE: the sanctioned allowlist-reconcile verb - reload jobs do not
# propagate through nodeguard-xdp's Requires=, so no detach and no WAN
# link blip. systemctl RESTART of this unit propagates and blips the
# link; use reload.
ExecReload=/usr/local/sbin/nodeguard-maps

Nobody thinks of restart versus reload as a blast-radius decision; they think of it as a style choice.

A journal that outlived the thing it described. The second review found the same shape one layer up. The responder keeps a journal of block windows so a repeat offender is not re-blocked while a block is already live, and those windows are wall-clock, up to a day long. Kernel blocks are not: they expire against CLOCK_MONOTONIC and the maps are gone entirely after a reboot. So a repeat offender could clear every gate and then be skipped by the window check, because the responder believed a kernel block covered it that had ceased to exist at boot. The worst case falls on exactly the wrong addresses: escalation lengthens the window for persistent offenders, so the most-escalated attacker gets the longest post-reboot free pass. Windows are now scoped to the boot that wrote them, and the detail I like is what happens when the boot id cannot be read at all:

        # SAFETY: stamp only a boot id that was actually read. An empty
        # stamp would compare EQUAL to an equally unreadable boot id at
        # the next load and carry windows across a reboot, which is the
        # suppression this scoping exists to remove; an absent stamp
        # loads as prior-boot and expires them.
        if self.boot:
            payload[JOURNAL_META_KEY] = {"boot_id": self.boot}

Two unknowns comparing equal is the same bug as an empty scanner report comparing clean. Refusing to write the stamp is what makes the unknown behave like an unknown.

The build breaks its own tools to see whether the output is honest

The monitoring rule on the counter path is that a value which cannot be read is omitted, so its item goes unsupported, with an explicit fail flag beside it rather than a fabricated zero. That rule is easy to write in a comment and easy to violate three refactors later, so the build tests it by sabotage: export the counters normally and assert all eight slots plus a stats_read_fail=0 flag, then move bpftool out of the way, export again, and assert stats_read_fail=1 with no counter line emitted. The drill spot-checks three of the eight names, which the export's all-or-nothing structure makes representative, since the status tool prints every counter name or none:

if printf '%s\n' "$kv_out" | grep -qE "^ng\.(pass|drop_v4|drop_v6)="; then
    echo "KV DRILL FAIL: counter lines emitted despite read failure"
    exit 1
fi

That is the same class of failure as a security scanner reporting green on zero files scanned, and it is checked by deliberately breaking the tool.

The rest of the build is in the same spirit. No compiler lands on the production hosts, so the object is built in a container and shipped as a file with its hash. The map specification is generated from that object rather than hand-written, because libbpf reuses a pinned map only on an exact parameter match, and a drifted creation script fails one of two ways: every attach fails at boot, or, worse, the loader silently creates parallel unpinned maps and the program enforces against maps nobody manages. Then the firewall looks attached, the units are green, and the blocklist you write into is a different set of maps from the one the kernel reads.

Generating the spec makes that drift detectable at start. The attach wrapper makes it fatal at runtime: after every attach it checks the pinned map ids against the ids the loaded program actually holds, and on divergence it unloads its own program and exits nonzero, logging map identity check failed; nodeguard unloaded, host runs OPEN. Running open beats enforcing against maps nobody manages. The check is a generalized rule rather than an exact set match, so an additive map rolls out and back hitlessly, and it is weaker for it: an out-of-band loader invocation can still auto-pin a map nobody manages, which ADR 0007 documents rather than denies.

The build also rehearses the production sequence in a network namespace, attaching the freshly compiled object against pre-created pins and firing crafted anomalous packets at it: the run fails if the drop total moves, because the protocol-sanity counters are count-only by contract and a build that quietly turned telemetry into a verdict must not ship.

What it is not

It is a blocklist firewall for small Linux gateways on stock Fedora packages, with a threat model of about 1 Gbps and gated public services; it is not a datacenter DDoS appliance and does not pretend to be. It is stateless: no connection tracking, no SYN proxying, no per-flow state in the datapath at all. The verdict is a function of the source address plus two whole-host inputs, the kill switch and a hard pass for UDP to the live WireGuard port. That second one is worth stating plainly, because it is a destination-port-keyed bypass sitting ahead of the blocklist lookup, so a blocked source can use it too; what it reaches is a cryptographically authenticated tunnel listener, and on IPv4 the pass applies only at fragment offset zero.

Blocking makes the IDS blind to what it blocked. XDP_DROP happens before delivery to the capture, so no packet from a blocked source ever reaches Suricata. The block TTL is therefore also the re-detection interval, and "no more alerts" is never evidence that an attacker stopped. The protocol-sanity counters stay counters for the same reason the responder demands evidence: impossible flag combinations and TTL outliers are visible and never actionable in the kernel, because dropping on them would bypass the anti-spoofing discipline entirely.

What is missing, and what it costs

There is no rate limiting, and the gap is named rather than hidden: nothing bounds a distributed low-and-slow flood, because the responder's caps bound new blocks, not throughput. A datapath limiter would be a second, independent drop condition, which conflicts with the invariant that the only drop is a blocklist hit, so it waits on evidence instead of being built speculatively. That evidence is already being collected. The volumetric anomaly detector runs in alerting-only mode, diffing successive counter snapshots against a rolling EWMA baseline, and its best rule is squarely on this post's own theme: an anomalous cycle updates no metric's baseline, not even a correlated metric that stayed under threshold, until that metric's own bounded skip streak forces adaptation, so an attack cannot train the detector into silence. A hitless reload discards exactly one cycle while keeping the baseline, because during a member swap both dispatcher members briefly count the same traffic.

Deploying it is a six-phase manual procedure, 0 prep through 5 steady state, not an install script; the deploy tool pushes files, syntax-checks every script, verifies every unit, and enables nothing, and the monitoring items exist before the first attach, because a thing that cannot raise an alarm when it breaks has no business being in the forwarding path.

Block counts are up to ten minutes stale, also on purpose. The block maps are LPM tries, the BPF map type whose keys are CIDR prefixes: cheap to look up, expensive to enumerate. Cloudflare's measured experience with them, roughly 573 dump operations per second at ten thousand entries and multi-second lockups freeing large ones, makes a per-minute trie walk a structural risk as the maps grow toward their ceilings of 65,536 entries for IPv4 and 16,384 for IPv6. So the walk rides the ten-minute sweep and the one-minute collection path reads a cache, which the maps service truncates whenever it creates a map, so after a reboot the counts report as unknown until the first sweep rather than as confident pre-reboot totals.

The one number I got badly wrong

It was Suricata's memory. A ruleset reload builds the new detect engine beside the old one and peaks near twice steady state, so I set MemoryMax provisionally at 10 GiB and 8 GiB and flagged it in the design document as something phase 1 had to measure. Measured, steady RSS is about 790 MB and 745 MB, peaking at 872 MB and 813 MB with a daily reload included. The guesses were six to ten times the real peak; the caps are now MemoryHigh 1.5 GiB and MemoryMax 3 GiB.

What I would tell someone building one

Decide what the failure direction is before you write the datapath, not after. Fail-open is not a feature of this system, it is the axis every other decision hangs from: it is why expiry is in the kernel, why every automatic action the watchdog takes on a suspected fault opens the datapath, and why the one path that closes it again is fenced to a watchdog-set latch, fifteen clean cycles and once per boot.

A health check that is exempt from the mechanism it monitors is decorative. This generalizes far past eBPF. A monitoring host inside the firewall's trusted zone, a synthetic check that skips the WAF, a load balancer probe on a bypass path: all the same shape, all reporting green through the exact failure they were bought to catch.

Read the value back. Almost every emergency control in every toolkit I have written takes an exit status as proof of effect and never reads the value back. The one that matters is the one you reach for while something is already wrong, which is the worst possible moment to be told a comforting lie by your own tooling.

Today one host is enforcing and the other is still reading its own WOULD BLOCK lines. Both export the same key-value snapshot once a minute, and the packet counters in it arrive complete or not at all, with an explicit read-fail flag beside them; the block counts stay absent until a sweep has actually walked the maps.