If you've been following my blog for a while, you know I have a soft spot for making packets go fast in software. I'm a network guy at heart, raised on routing and switching, but these days most of that love is written in Go.
Every few years I circle back to the same question: how fast can we push and pull packets on a plain Linux box? We've explored this with DPDK, with XDP-based routers, and with the kernel itself. Today we're going a level deeper into my favorite of the bunch: AF_XDP, using a Go library I wrote to make it easy: go-afxd .
If you want to skip ahead and see it in action, the blast and drop examples in the repo are a great place to start.
Some background
A while ago I wrote this article: High-Speed Packet Transmission in Go: from net.Dial to AF_XDP. In that article we worked through every way Linux gives you to send a packet from Go: net.Dial, raw sockets, AF_PACKET, pcap. At the end of that journey sat AF_XDP, and it beat everything else by a mile.
That was my first hands-on experience with AF_XDP, through a third-party library asavie/xdp. It showed off the potential, but it was quite low-level and, honestly, a bit fragile; some of you who tried it on real hardware ran into panics I couldn't fix from my side. So, building on that code, I wrote my own library. The goal: keep the raw speed, make it higher level and more reliable, and have it do the right thing for whatever hardware or cloud you're on, without you needing to understand every knob under the covers.
So, what is AF_XDP?
AF_XDP is a socket type in Linux, like the ones you already know. The mental model is close to AF_PACKET (the thing tcpdump is built on): instead of asking the kernel to "send this payload to this IP and port," you hand it complete Ethernet frames, MAC addresses and all, and you receive complete Ethernet frames back. You're operating at layer 2.
The difference is how much of the kernel you skip on the way to the wire. AF_PACKET still takes a fairly scenic route through the network stack. AF_XDP takes the express lane: your program and the kernel share a chunk of memory (the UMEM), carved into packet buffers, and packets move through lock-free rings in that shared memory. On a modern NIC and driver it goes one step further with zero-copy mode: the NIC's DMA engine writes incoming packets directly into your program's memory and reads outgoing packets straight from it. No copies, no socket buffers, no per-packet syscalls.
How fast is that in practice? On a bare-metal machine with a 10G Intel NIC, I measured three methods, all sending small 64-byte packets. Plain kernel UDP sockets, one sendto() per packet, topped out around 3 million packets per second no matter how many workers I threw at it. Adding UDP GSO trickery (UDP_SEGMENT, where the kernel slices one big send into many packets) more than doubled that to almost 8 million. And go-afxdp? ~13.5 million packets per second, which is virtually 10G line rate, with the receiving side keeping up without dropping a packet. At that point the wire is the bottleneck, not the software. I still think that's kind of amazing for a Go program you can go build like any other.
These numbers come from go-pktgen, my packet generator that benchmarks all the send methods side by side; I've updated it so its af_xdp mode now uses this new library. Check that repo if you want the full comparison table. Btw, AF_XDP shines even more on the RX path than TX.
The best part: you don't lose your NIC
When you open an AF_XDP socket with go-afxdp, the library compiles a small eBPF program and attaches it to the interface as an XDP program. XDP programs run inside the driver, for every incoming packet, before the kernel stack ever sees it (the same technology behind my XDP-based BGP peering router from a few years back). The program's job here is simple: if a packet matches the filter you asked for, redirect it into your socket's ring. That's it.
The key word is if. Everything that doesn't match is handed to the kernel stack, completely unaffected. Say you only want UDP port 9999: those packets land straight in your Go program, while your SSH session, DNS lookups, and everything else on the interface keep flowing like nothing happened. You can be surgical about exactly which traffic takes the fast path. This is exactly where DPDK hurts: it takes over the entire NIC, and the interface disappears from the kernel, SSH session and all. With AF_XDP the NIC stays a normal Linux interface.
What it looks like in Go
This is what receiving packets looks like. Open the interface with a filter, and you get one socket per NIC queue:
fleet, err := afxdp.Open("eth0", afxdp.WithUDPPorts(9999))
if err != nil {
log.Fatal(err)
}
// Info() tells you what you actually got: queues, XDP mode, zero-copy, driver.
// e.g. "eth0: 12 queues, zero-copy, native XDP, 8192x2048B frames, driver ixgbe"
info, _ := fleet.Info()
log.Printf("started: %s", info)
// One receive loop per queue: Fill -> Poll -> Receive -> Recycle.
for _, xsk := range fleet.Sockets() {
go func(xsk *afxdp.Socket) {
for {
xsk.Fill(xsk.NumFreeFillSlots()) // hand the kernel empty buffers
n, err := xsk.Poll(-1) // block until packets arrive
if err != nil {
return
}
descs := xsk.Receive(n)
for _, d := range descs {
packet := xsk.GetFrame(d) // the raw Ethernet frame ([]byte)
// ... do something with it ...
}
xsk.Recycle(descs) // give the buffers back for reuse
}
}(xsk)
}
Pretty simple, right? No BPF to write, no UMEM math, no ring management. The Fill -> Poll -> Receive -> Recycle loop is the whole programming model, and transmit is the same idea in reverse. This snippet is essentially the drop example, a UDP sink that counts how fast it can receive; its counterpart blast is the transmit side, a UDP flooder that fills every TX queue. Those two together are how I got the numbers in this article.
Notice the Info() line. AF_XDP can run in very different modes: native (in the driver) or generic (a slower fallback that works anywhere, even on veth), with zero-copy or copy. The difference between them is huge. The library picks the fastest mode your driver supports and then tells you what you got.
It should just do the right thing
That mode business is where my old frustrations came from, and where I spent the most effort this time. Whether you get the fast path depends on your driver, your queue configuration, even your MTU. That's a lot of tribal knowledge to expect from someone who just wants to receive or send packets quickly.
A concrete example: I recently tested on AWS, on a pair of c7gn.xlarge instances. Out of the box, AF_XDP on the ENA driver quietly falls back to generic mode, and in generic mode my receiver silently lost about 25% of the packets, with no counter anywhere admitting to it. Getting the fast path on ENA requires some unobvious things, including page-sized (4096-byte) packet buffers instead of the usual 2048. Miss that one and zero-copy silently doesn't engage. The library now detects the ENA driver and handles it automatically, so the stock examples come up as "zero-copy, native XDP" on EC2 with no code changes. The same test then ran at a clean, steady 5 million packets per second, completely lossless, at which point you hit AWS's per-instance packets-per-second allowance. But hey, that means the software is no longer the bottleneck.
That's the philosophy of the library, you shouldn't need to know that one particular AWS driver wants page-aligned frame sizes. It should just do the right thing.
How does it compare to DPDK?
Honestly, the performance is in the same ballpark, and that surprised me. DPDK is still the gold standard for absolute numbers, but the gap has become small and the price difference is enormous. With DPDK you lose the whole NIC, you're tied to specific drivers and a heavy build-and-tuning universe, and there's effectively no good Go story. AF_XDP works on any modern Linux kernel, keeps the NIC as a normal interface, lets you pick which traffic to bypass, and is one go get away. For me it's become the sweet spot: 95%+ of the performance at 10% of the operational pain.
Where this fits (and where it doesn't)
The cases where AF_XDP shines is anything that's packet-in, packet-out at high packet rates:
- Packet generators and performance testing. the blast and drop examples together are a poor man's IXIA, the same itch I used to scratch with DPDK's pktgen, now in a couple hundred lines of Go.
- UDP-based servers and protocols. There's a DNS server example in the repo that answers queries straight from the AF_XDP path. Tunnels, VXLAN, QUIC-adjacent experiments: anything UDP-shaped fits naturally.
- Network scanners, IDS-style monitoring, DDoS scrubbing. Anywhere you want to see a lot of packets cheaply, for a specific slice of traffic, without taking the box off the network.
And where it doesn't fit: TCP. With AF_XDP you're getting raw Ethernet frames, so TCP means bringing your own TCP stack in userland, retransmissions and congestion control included. The kernel gives you a world-class TCP implementation for free, and my strong suspicion is that a userland stack bolted onto AF_XDP would give back most of the performance while adding a pile of complexity. If your workload is TCP-shaped, use the kernel. AF_XDP is for when your workload is packets.
Wrap-up
To me, a reliable easy to use AF_XDP library felt like a missing piece in Go for years: kernel sockets are easy but top out in the low millions of packets per second, and DPDK is fast but heavy, using C, and takes your NIC away. AF_XDP sits right in between: a Go program, built with the normal toolchain, sending and receiving at 10G line rate with a couple of CPU cores, while the rest of the box carries on like normal.
The code is at github.com/atoonk/go-afxdp. If you try it on hardware I haven't, I'd love to hear about it, especially the failure cases. That's how the ENA quirks got found and fixed.
That's it; you made it to the end! Thanks for reading! Let's push some packets :)
Cheers
-Andree
