Rethinking Firewall Architecture

in the eBPF Era

Nishant

Software Developer @CloudArmour

IndiaFOSS · Security Devroom

You wrote the rule. The packet still got through. Here's why.

You think this is happening:

packet arrives → iptables checks rules → DROP

This is what's actually happening under a flood:

packet arrives

▼

kernel allocates sk_buff

← one per packet, every packet

▼

conntrack tries to track it

← table has a size limit

▼

conntrack table is full

▼

new packets can't be tracked

← legitimate traffic dies here

▼

your DROP rule can fail under load

← it's too far down the stack

What if you could drop the packet before the kernel touches it?

That's XDP(in native mode).

NIC driver

packet arrives from ring buffer — raw bytes, no struct

your XDP program runs here

read ethhdr, iphdr

check a BPF map

return XDP_DROP or XDP_PASS

only if XDP_PASS ▼

sk_buff → conntrack → iptables → socket

(the whole mess from slide 2 — only if you allow it)

BPF maps — the kernel and userspace talk through shared state

Your XDP program runs in the kernel.

Your daemon runs in userspace.

They can't share memory. But they can share a map.

daemon · userspace · Rust

map.insert(

ip, metadata

)

XDP program · kernel · C

bpf_map_lookup(

src_ip

) → XDP_DROP

write ─────────────────── read

/sys/fs/bpf/rfw/blk_ip_v4

​

BPF_MAP_TYPE_HASH

key: src IP val: reason, timestamp

​

pinned to bpffs · survives daemon restart

Two processes writing the same map. What could go wrong?

Classic setup — CLI talks directly to the kernel map:

firewall-cli ──aya──► blk_ip_v4

firewall-api ──aya──► blk_ip_v4

Now an attacker sends traffic from 50,000 different source IPs.

Your map is LRU. It has a size limit. New entries push old ones out.

CLI thinks

1.2.3.4 is blocked

✓

API thinks

1.2.3.4 is blocked

✓

Map says

1.2.3.4 is gone

✗

← evicted by the flood

Traffic from 1.2.3.4 passes.

Nobody knows. Both processes think the rule exists.

Make the API the only writer. CLI becomes HTTP.

firewall-cli

──► POST /block {"ip": "..."}

──► firewall-api

HTTP

aya

blk_ip_v4

──► XDP

Now:

· Only one process calls map.insert() and map.remove()

· No race. No divergence. No phantom blocks.

Bonus you didn't plan for:

CLI → HTTP → API handler → SQLite → BPF map

Every rule change hits the database before it hits the kernel. You get a full audit trail for free.

The BPF map is a projection of the database, not the source of truth.

The eBPF program is C. The userspace library is your choice.

The XDP program always compiles to BPF bytecode. C is the right language for it —

CO-RE, the verifier toolchain, and every kernel example is built around C.

Don't fight that. The userspace library is a different question.

Option

Language

What it is

libbpf

C

The reference library, ships with the kernel.

Most production systems use this.

libbpf-rs

Rust

Thin Rust bindings over libbpf.

Still needs libbpf installed.

aya

Rust

Pure Rust, no libbpf dependency.

Loader + map types + program attach in one library.

Why aya for a Rust daemon:

rfw-ebpf/src/checklist/blocklist.rs

map.insert(IPv4Key::from_ip(ip), IPMeta { .. }, 0)?;

IPv4Key: same Rust type in handler and map. Drift won't compile.

Let's block something.

machine A

$ ping 192.168.1.10

icmp_seq=1 0.4ms

icmp_seq=2 0.3ms

icmp_seq=3 0.4ms

​

timeout icmp_seq=4

timeout icmp_seq=5

timeout icmp_seq=6

​

icmp_seq=7 0.4ms

icmp_seq=8 0.3ms

◄

machine B (operator)

$ fw block 192.168.1.1

blocked

​

$ fw list block

192.168.1.1 manual just now

​

$ systemctl restart fw

(daemon restarts)

​

$ fw list block

192.168.1.1 manual just now

Rule active on the next packet. Daemon restarted. XDP never stopped. Rules came back from SQLite.

Static rules are the beginning, not the end

Problem

Why it's hard

You don't know the IP yet

Static blocklist is useless against a zero-day source.

Need behavioral detection — rate, pattern, reputation.

LRU is still exhaustible

Attacker sends from 100k IPs. Map fills up. Legitimate entries evict.

Per-CPU maps help. Don't fully solve it.

XDP_DROP is silent

You're dropping packets. You don't know which, how many, from where.

Ringbuf → SSE stream is the next piece.

Stateful rules need TC

XDP is stateless. ESTABLISHED/RELATED tracking needs

TC or netfilter BPF. Different hook, different return codes.

Rule priority needs a compiler

Hash maps have no order. Priority-based policy needs

LPM trie or a rule compiler that writes the right map type.

Four things worth remembering

1

iptables fails structurally under flood

conntrack exhaustion means your rules never fire.

XDP drops before any of that runs.

2

BPF maps are a kernel/userspace contract

Pinned to bpffs. No syscall per packet.

Survives restarts on both sides.

3

Who writes the map is a security decision

Two writers + LRU + attacker key space = phantom blocks.

One writer closes the race. Audit trail comes for free.

4

aya makes the type system span the boundary

Same Rust type in the map and the REST handler.

Drift between them won't compile.

Nishant

Software Developer @CloudArmour

Questions?

aya-rs.github.io/book/

eBPF + Rust, start here

docs.cilium.io/en/stable/bpf/

deepest BPF reference

lwn.net�

XDP design paper

nishant@cloudarmour.io