Docker vs Bare Metal: The Real Latency Tax on Solana Bots

Every Solana bot that gets shipped in a container eventually hits the same question: how much latency did that convenience cost? The honest answer is "less than you fear, more than zero, and it depends entirely on how you configured the host." I've benchmarked the same arbitrage executor in three deployment shapes — bare metal, Docker with default networking, and Docker tuned for low latency — and the spread between them is real but closeable. Here's what the numbers actually look like and how to get containers within a rounding error of native.

What we're actually measuring

Latency on a Solana bot is not one number. The path from "I decided to send" to "the leader saw my packet" crosses several layers, and Docker only touches some of them:

• Syscall and scheduling latency — how long your thread waits to run after an event wakes it.
• Network stack traversal — packet from userspace to NIC, including any bridge, NAT, or veth hops.
• Serialization jitter — the p99/p50 gap, which matters far more than the average for a bot racing a block.

Averages lie here. A bot that submits in 400μs on median but spikes to 4ms at p99 will lose the trades that pay. So I benchmark p50, p99, and the standard deviation of inter-send time under load, not just the mean.

The test rig: a bare-metal box with a dedicated NIC, pinned to a validator's region, running a synthetic sender that fires 50k sendTransaction calls at a mock TPU endpoint on the same LAN, then the same binary inside Docker two ways. Kernel 6.x, BBR congestion control, identical CPU.

The raw numbers

Default Docker networking is where the tax shows up, and it's larger than people expect.

Deployment	p50 send	p99 send	inter-send stddev
Bare metal, pinned	61μs	190μs	22μs
Docker, default bridge	118μs	640μs	140μs
Docker, host net + tuned	68μs	210μs	31μs

The default-bridge column is the trap. That 640μs p99 isn't Docker "being slow" in some vague sense — it's the docker0 bridge plus the veth pair plus NAT via iptables/nftables conntrack. Every outbound packet gets a connection-tracking lookup, and under burst load the conntrack table thrashes. The jitter (140μs stddev) is worse news than the median, because that variance is exactly what blows your p99 on the packets you care about.

Switch to --network=host and the veth pair and NAT vanish. The packet path becomes identical to bare metal because there is no longer a separate network namespace to cross. That single flag closes roughly 80% of the gap on its own.

Where the container tax comes from

Three mechanisms, in order of how much they hurt:

Network namespace and NAT. The bridge network puts your bot in its own netns connected by a veth pair, then SNATs outbound traffic. conntrack is the killer — it's stateful, it has a hash table with a fixed bucket count, and QUIC's connection churn (which is how Solana's TPU talks now) fills it fast. If you've read the QUIC TPU forwarding guide on direct leader transaction submission, you already know the TPU path opens and cycles a lot of connections; conntrack hates exactly that pattern.

CPU scheduling and cgroups. The cpu cgroup controller adds accounting overhead and, if you set CPU limits, the CFS bandwidth controller can throttle your thread mid-burst. A container with --cpus=2 doesn't get "2 cores of latency" — it gets throttled to a quota window (default 100ms), and when it exhausts the quota it stalls until the next period. For a latency-sensitive sender that's catastrophic. Never set CPU quotas on a hot bot; use --cpuset-cpus to pin instead.

cgroup memory and page faults. Memory cgroup accounting adds a small per-fault cost, and if you're near the limit the reclaim path introduces multi-millisecond stalls. Give the container headroom and lock critical pages.

Closing the gap

Here's the container config I actually ship. It gets Docker to within ~10% of bare metal on p99:

docker run -d \
  --network=host \
  --cpuset-cpus="2,3" \
  --cap-add=SYS_NICE \
  --ulimit rtprio=99 \
  --sysctl net.core.busy_poll=50 \
  --sysctl net.core.busy_read=50 \
  --memory=4g --memory-swap=4g \
  solana-bot:latest

Host-side, before you even start the container, tune the kernel. These matter regardless of Docker but they're what turns "acceptable" into "native":

# Pin IRQs off your bot's cores, keep them on housekeeping cores
echo 2 > /proc/irq/$NIC_IRQ/smp_affinity_list

# Disable the two biggest jitter sources
tuned-adm profile network-latency
echo 0 > /sys/devices/system/cpu/cpu2/cpufreq/... # or cpupower to set performance governor

# Busy-poll the NIC so you don't pay the interrupt wakeup on every packet
sysctl -w net.core.busy_poll=50

Inside the app, pin the sender thread and bump its scheduling priority (SCHED_FIFO via SYS_NICE), preallocate your buffers, and warm the UDP/QUIC sockets before you go live so the first real packet doesn't eat a cold-path fault. If you're running a market-making loop where every quote update races a competitor, this thread-level tuning is the difference between filling and getting picked off. We build this kind of pinning into the executors we ship for Solana market-making systems, because the container config alone doesn't save you if the hot thread is floating across cores.

The gotchas that eat your afternoon

• --network=host breaks port isolation. If you run multiple bots on one box they now share the host's port space. Plan your port map or you'll get bind conflicts at the worst time.
• Time source. clock_gettime inside a container is fine (it's vDSO), but if you're measuring latency with a virtualized clock in a VM-under-Docker setup, your numbers are fiction. Measure on the metal.
• conntrack still bites host mode if iptables has rules. Host networking skips the bridge but not necessarily every netfilter hook. Check nft list ruleset and make sure your bot's traffic isn't traversing a conntrack'd chain.
• Transparent hugepages. THP compaction causes multi-ms stalls. Set transparent_hugepage=never on latency-critical hosts.

So, container or metal?

For most desks: container, with host networking and the tuning above. You keep reproducible deploys, clean rollbacks, and the CI story, and you give up almost nothing measurable once conntrack and the veth pair are out of the path. The ~7μs p50 difference in the tuned row is below the noise floor of your RPC and staked-connection variance anyway — if you're not already running staked connections, that's a far bigger latency lever, and it's worth reading how staked connections and swQoS change your effective send latency before you obsess over container overhead.

Go bare metal only when you need the last few microseconds and full control of IRQ affinity, hugepages, and the scheduler without cgroup interference — a colocated market maker fighting for top-of-block, for example. Even then, most teams get there faster by fixing their submission strategy (running dual submission across sendTransaction and Jito bundles) than by shaving container overhead. The right monitoring matters too; a real-time latency dashboard that plots p99 per hop tells you where the actual tax is, which is rarely where you assumed.

The uncomfortable truth is that Docker vs bare metal is a 10% question sitting on top of a 10x question. Get your RPC, your region, and your submission path right first. Then tune the host, flip on host networking, and the container tax mostly disappears.

If you want the kernel tuning, IRQ pinning, and pinned-thread executor set up correctly on your own hosts, that's exactly the kind of work our infrastructure and data engineering team does.