Inter-Thread Communication

The fastest practical inter-thread communication is a busy-spinning SPSC ring buffer with cached indices, pinned to cores sharing an L3 cache, achieving 50–135 ns round-trip latency. The absolute physical floor is the cache-line transfer cost between cores: 16–25 ns on the best modern hardware (AMD Zen 3 same-CCX or Intel ring bus). Every other inter-thread mechanism — atomics, futexes, condition variables, sockets, eventfd — is a layer of additional latency stacked on top of this floor. The engineering challenge is staying as close to the floor as possible while passing meaningful data.

The hierarchy

The full latency ladder spans roughly six orders of magnitude. Three insights matter more than any single number:

1. Topology trumps algorithm. The same queue design varies 5× in latency depending on whether the threads share an L3 (16 ns transfer) or cross an interconnect fabric (107 ns on AMD Infinity Fabric, 150 ns cross-socket on Intel). taskset and isolcpus are the single highest-leverage optimizations.
2. FAA and CAS cost the same per instruction (~6–7 ns uncontended). The performance gap is semantic, not hardware: FAA always succeeds, while CAS retry storms collapse throughput under contention.
3. std::mutex is not the bottleneck most developers think. Modern futex-based implementations complete >80% of operations entirely in userspace, adding only ~25 ns over raw atomics. See synchronization-primitives.

The physical floor

AMD's Zen 3 has the industry-best 16 ns within a CCX (8 cores sharing L3) but balloons to 84–107 ns crossing chiplets. Intel's 4-core consumer parts offer predictable ~21 ns across all cores via a ring bus; server parts with mesh interconnects sit at 48–62 ns. Hyperthreads on the same core can signal each other in 16.5 ns since they share L1/L2 — but this is rarely practical for data-passing workloads. See cache-coherency for the underlying MESI/MOESI protocol details.

The synchronization cost ladder

Surprising data point: eventfd (4,353 ns RTT) is slower than UNIX domain sockets (1,439 ns) for thread signaling. Both are orders of magnitude slower than atomic-based approaches because every notification is a syscall. See synchronization-primitives for the full breakdown.

Why SPSC ring buffers win

Lamport (1977) proved correctness of an SPSC ring buffer that needs zero atomic read-modify-write operations. Each index is written by exactly one thread, so plain loads and stores with acquire-release ordering suffice. On x86 with Total Store Order, these compile to ordinary MOV instructions — the cheapest possible memory operations.

Three modern optimizations on top of Lamport's design:

• Local index caching (Erik Rigtorp): the producer keeps a cached copy of the consumer's read index and only re-fetches when the cache says the queue appears full. Yields 20× throughput improvement on AMD Ryzen 9 3900X (5.5M → 112M ops/sec) by reducing coherence traffic from ~3 cache misses per operation to near zero amortized.
• Power-of-two sizing replaces modulo with bitwise AND.
• Huge-page backing eliminates TLB misses across the ring buffer.

Measured SPSC RTT latencies:

The Disruptor's 52 ns per hop on 2011-era hardware is remarkable — approaching the raw core-to-core latency of that hardware. Its secret is the single-writer principle: each sequence counter is owned by one thread, and consumers batch-process available events when they wake.

MPMC pays for coordination

Multi-producer multi-consumer queues fundamentally cannot avoid atomic RMW on shared head/tail indices. The progression:

• Michael & Scott (1996) — CAS on linked-list pointers; basis for ConcurrentLinkedQueue.
• Vyukov bounded MPMC (~2010) — per-slot sequence number with CAS on shared indices.
• Morrison & Afek LCRQ (PPoPP 2013) — replaces CAS with FAA on the fast path; current state of the art for x86.
• LPRQ (PPoPP 2023) — drops LCRQ's double-width CAS, making it portable.
• Wait-free CQ (SPAA 2022) — fast-path-slow-path methodology.
• Aggregating Funnels (PPoPP 2025) — software FAA combining; up to 2.5× over plain LCRQ.

ETH Zurich (Schweizer, Besta, Hoefler) proved that CAS, FAA, and swap have essentially identical instruction-level latency — the popular belief that "FAA is faster than CAS" comes from FAA's semantic advantage (no wasted retries), not hardware cost. See faa-vs-cas.

Advanced techniques that take a different approach: flat-combining (Hendler et al., 2010) serializes operations through a single combiner thread, outperforming lock-free queues under very high contention. Elimination (Hendler-Shavit-Yerushalmi) lets matching push/pop pairs exchange values directly in a side array, bypassing the central structure.

How HFT firms squeeze out every nanosecond

The high-frequency trading playbook combines several layers, documented at core-pinning:

• isolcpus + taskset reduces tail latency from 120 µs to 30 µs at the 99.9th percentile.
• 1 GB huge pages cut TLB misses by ~99% across shared ring buffers.
• SCHED_FIFO real-time priority prevents preemption.
• Disabled hyperthreading eliminates resource contention on execution units.

Intel's 2024 optimization study on Sapphire Rapids hit 450 million 64-byte messages per second using lazy flow control — updating the read pointer only every Nth message. The CLDEMOTE instruction (hints the hardware to push a cache line toward L3 proactively) boosted core-to-core bandwidth from 7.3 GB/sec to 22 GB/sec when 8+ threads are involved.

DPDK-style kernel bypass applies the same principles to network I/O: poll-mode drivers in tight userspace loops achieve ~5 µs average UDP latency versus ~9 µs through the kernel stack. The pattern translates directly to inter-thread work.

Cross-language convergence

The performance gap between languages for inter-thread communication is far smaller than developers expect. Martin Thompson's canonical benchmark showed Java volatile ping-pong at 50 ns versus C++ atomic at 45 ns on identical hardware — only ~10% difference. Both emit LOCK-prefixed x86 instructions that bottleneck on the same cache coherence protocol. Rust SPSC ring buffers produce essentially identical machine code to C++ equivalents. This is the mechanical sympathy thesis: the hardware sets the speed, and the language barely matters.

Where languages diverge is in tail latency and GC. Java's GC pauses can inject multi-millisecond stalls into otherwise sub-microsecond paths — the Disruptor mitigated this through pre-allocated ring entries, and JCTools' SPSC queues use Unsafe to match native performance. Go's goroutine channels are convenient but significantly slower than dedicated lock-free queues. For Rust, kanal leads channel throughput, while raw SPSC ring buffers remain faster for 1P/1C.

The recipe

The fastest known inter-thread communication converges on:

1. SPSC ring buffer (Lamport 1977 + cached indices + 128-byte padding).
2. Busy-spin waiting (no syscalls).
3. Threads pinned to cores sharing an L3.
4. Huge pages backing the buffer.

This achieves 50–135 ns RTT — within 2–5× of the bare cache coherence floor. The lmax-disruptor proved this approach extends to multi-consumer pipelines at 52 ns per hop in Java.

See also

• the-fastest-queue — broader queue performance hierarchy
• concurrent-queues — full MPMC landscape
• mechanical-sympathy — the design philosophy
• thread-per-core — server architecture built on these primitives