Kernel-Bypass Networking with XDP and AF_XDP: Line-Rate Packet Processing on Linux

On a 100GbE or 400GbE network link, a continuous stream of minimum-sized 64-byte Ethernet frames delivers over 148 million packets per second (Mpps). That gives a server exactly 6.7 nanoseconds to inspect, route, filter, or forward each individual packet before the network interface card (NIC) drops frames due to buffer overrun.

Under these extreme throughput demands, the traditional Linux kernel networking stack hits an insurmountable bottleneck. Allocating a struct sk_buff, traversing the netfilter firewall chain, traversing routing tables, handling software interrupts (ksoftirqd), and performing user-space socket context switches consumes several hundred nanoseconds per packet. Even on modern multi-core server hardware, the standard Linux networking stack caps out at only a few million packets per second.

Historically, solving this required complete kernel bypass via DPDK (Data Plane Development Kit)—a brute-force approach that unbinds the physical NIC from the operating system, consumes 100% of dedicated CPU cores in busy-polling loops, and sacrifices all native Linux networking tools (iproute2, ethtool, tcpdump, standard routing).

The Linux kernel solved this dilemma with XDP (eXpress Data Path) and AF_XDP (XDP Sockets). By executing sandboxed, JIT-compiled eBPF bytecode at the lowest possible software layer in the NIC device driver—before memory allocation and before the kernel network stack is even aware a packet exists—XDP achieves line-rate packet processing speeds of over 25+ Mpps per CPU core while remaining 100% integrated with the Linux OS.

In this systems engineering guide, we dissect the internal architecture of XDP and AF_XDP, analyze XDP action return codes, explore UMEM zero-copy ring buffers, and build a line-rate layer-4 DDoS mitigation firewall in C.

The sk_buff Tax: Why the Standard Linux Stack Collapses

To understand the revolutionary nature of XDP, consider the mandatory computational tax paid by every single network packet entering the standard Linux kernel:

Standard Linux Kernel Network Ingress vs. XDP Fast Path:

Physical NIC (100GbE DMA)
       │
       ▼
[ NIC Driver Rx Ring Buffer (NAPI Poll) ]
       │
       ├───> [ XDP Hook (eBPF Bytecode) ] ───> XDP_DROP (Zero sk_buff, Instant Drop!)
       │            │                          XDP_TX (Bounce back out same NIC)
       │            │                          XDP_REDIRECT (AF_XDP Zero-Copy Socket)
       │            ▼
       │     XDP_PASS (Pass to standard OS stack)
       │
[ 1. Allocate Heavy 'sk_buff' Metadata Struct (~240 bytes overhead) ]
       │
[ 2. Netfilter / iptables / nftables Packet Filtering ]
       │
[ 3. IP Routing Table Lookups & Subnet Evaluation ]
       │
[ 4. Transport Layer (TCP Reassembly / UDP Socket Demux) ]
       │
[ 5. Socket Queue Locking & Wake Up Application Thread ]
       │
[ 6. Copy Data from Kernel Memory to User Space Buffer via read()/recv() ]

A standard struct sk_buff is an intricate, heavyweight control block containing over 200 bytes of metadata, pointers, atomic reference counts, and timestamps. Allocating and freeing millions of sk_buff allocations per second places immense pressure on CPU cache hierarchies, memory allocators (SLUB), and interrupt handlers.

XDP Architecture: Driver-Level eBPF Execution

XDP hooks directly into the network driver's NAPI poll routine. When the NIC finishes DMA-transferring raw packet bytes into memory, XDP executes a specialized eBPF program against the raw memory buffer (represented as struct xdp_md):

AF_XDP: Zero-Copy User-Space Networking

While XDP is ideal for in-kernel packet inspection and filtering, applications often need to process payload data inside high-performance user-space daemons (such as DNS servers, WebSockets gateways, or packet sniffers).

AF_XDP (Address Family XDP) provides a high-speed channel to transfer raw packets from the NIC driver directly into user-space memory with zero memory copies and zero system call context switches.

AF_XDP UMEM Ring Buffer Architecture:

┌─────────────────────────────────────────────────────────────┐
│                    USER-SPACE PROCESS                       │
│                                                             │
│   [ Fill Ring ] ──────────(Supplies empty buffers)────────> │
│   [ Rx Ring ]   <─────────(Receives completed packets)───── │
│                                                             │
│            UMEM: Shared Memory Arena (Pinned Pages)         │
└─────────────────────────────┬───────────────────────────────┘
                              │
               LOCKLESS CIRCULAR RING BUFFERS
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                     NIC DRIVER / KERNEL                     │
│                                                             │
│   NIC DMA -> Direct Write to UMEM Buffer                    │
│   XDP Program executes XDP_REDIRECT -> Pushes to Rx Ring    │
└─────────────────────────────────────────────────────────────┘

The 4 Lockless Ring Buffers of AF_XDP:

1. Fill Ring (User $\to$ Kernel): User space places memory addresses of available, empty UMEM frame buffers here for the kernel/NIC to fill with incoming packets.
2. Rx Ring (Kernel $\to$ User): When a packet arrives, the driver populates an empty UMEM frame and places the frame descriptor onto the Rx Ring for the application to process.
3. Tx Ring (User $\to$ Kernel): User space places frame descriptors of outgoing packets here for transmission.
4. Completion Ring (Kernel $\to$ User): The kernel notifies user space when outbound packet frames have finished transmitting over the wire, so the memory can be safely reused.

Architectural Comparison: Linux Sockets vs. DPDK vs. XDP / AF_XDP

Hands-On: Building a High-Speed Layer-4 DDoS Firewall in C with XDP

Below is a production-grade eBPF XDP program written in C that parses Ethernet, IPv4, and UDP/TCP headers, dynamically checks an eBPF Hash Map blacklist, and drops unauthorized traffic at line rate:

#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/udp.h>
#include <linux/tcp.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>

// eBPF Map: Blacklisted IPv4 addresses (Key: IP as uint32_t, Value: Drop Counter)
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 100000);
    __type(key, __u32);
    __type(value, __u64);
} blacklist_map SEC(".maps");

SEC("xdp")
int xdp_firewall_filter(struct xdp_md *ctx) {
    void *data_end = (void *)(long)ctx->data_end;
    void *data = (void *)(long)ctx->data;

    // 1. Parse Ethernet Header with bounds checking
    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
        return XDP_PASS;

    // Only inspect IPv4 traffic
    if (eth->h_proto != bpf_htons(ETH_P_IP))
        return XDP_PASS;

    // 2. Parse IPv4 Header
    struct iphdr *ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > data_end)
        return XDP_PASS;

    __u32 src_ip = ip->saddr;

    // 3. Lookup source IP in eBPF Blacklist Map
    __u64 *drop_count = bpf_map_lookup_elem(&blacklist_map, &src_ip);
    if (drop_count) {
        // Increment dropped packet metric atomically
        __sync_fetch_and_add(drop_count, 1);
        
        // Discard packet immediately in the NIC driver ring buffer (< 5ns!)
        return XDP_DROP;
    }

    // 4. Example: Drop DNS amplification attack attempts (UDP destination port 53 flood)
    if (ip->protocol == IPPROTO_UDP) {
        struct udphdr *udp = (void *)ip + (ip->ihl * 4);
        if ((void *)(udp + 1) > data_end)
            return XDP_PASS;

        if (udp->dest == bpf_htons(53) && bpf_ntohs(udp->len) > 512) {
            // Drop oversized suspicious DNS queries
            return XDP_DROP;
        }
    }

    // Allow legitimate traffic to proceed to standard Linux TCP/IP stack
    return XDP_PASS;
}

char _license[] SEC("license") = "GPL";

Compiling and Attaching with iproute2:

# 1. Compile eBPF C source to BPF bytecode with Clang
clang -O2 -target bpf -c xdp_firewall.c -o xdp_firewall.o

# 2. Attach XDP program to network interface (e.g. eth0) in native driver mode
ip link set dev eth0 xdpgeneric off
ip link set dev eth0 xdp obj xdp_firewall.o sec xdp

# 3. Verify attachment
ip link show dev eth0
# Output: ... xdp/id:42 ...

# 4. View real-time packet processing metrics
bpftool prog show name xdp_firewall_filter

Summary & Engineering Checklist

XDP and AF_XDP represent the modern golden standard for high-performance Linux networking, providing the speed of hardware kernel bypass with none of the operational downsides:

• Deploy XDP at the Ingress Boundary: Use native driver XDP for DDoS mitigation, volumetric IP filtering, and Layer 4 load balancing (DSR) to discard or bounce traffic before wasting kernel memory.
• Leverage AF_XDP for User-Space Acceleration: When building ultra-low latency proxies or specialized protocol handlers, use AF_XDP UMEM rings to eliminate memory copy overhead.
• Maintain Full Linux Observability: Unlike DPDK, XDP preserves compatibility with tcpdump, ethtool, and standard monitoring tools via XDP_PASS.
• Verify NIC Driver Support: Ensure your cloud or physical NIC driver (e.g., i40e, ixgbe, mlx5, virtio_net, ena) supports native XDP mode for maximum hardware offload efficiency.