What Is Memory Interleaving?

Memory interleaving is a technique for laying out data across multiple independent memory banks so that consecutive addresses land in different banks rather than the same one, letting several memory accesses proceed in parallel instead of queuing up behind a single bank’s fixed access latency. It’s a structural trick for hiding memory latency, distinct from making any individual memory chip faster.

The problem: one bank, one access at a time

A memory bank — the basic unit inside DRAM that can service a request — has a fixed cycle time: after starting an access, it needs a certain number of cycles before it can start the next one. If every consecutive address a program touches lives in the same bank, each access has to wait for the previous one to finish, even though the memory controller and the bus connecting it to the CPU could, in principle, be moving more data during that wait. This is the same latency-vs-throughput tension covered in memory bandwidth vs latency: a single bank’s latency is fixed, but a system with several banks can hide that latency by keeping more than one busy at once.

How interleaving spreads the load

Interleaving addresses this by splitting memory into multiple banks and mapping consecutive addresses to different banks rather than filling one bank before moving to the next. A common scheme distributes addresses using the low-order bits of the address to select the bank:

Address 0  -> Bank 0
Address 1  -> Bank 1
Address 2  -> Bank 2
Address 3  -> Bank 3
Address 4  -> Bank 0
Address 5  -> Bank 1
...

When a program reads a sequential block of memory — exactly the access pattern most programs actually produce, whether streaming an array or fetching a cache line — each successive address lands in a different bank. The memory controller can issue the access to bank 1 while bank 0 is still finishing its previous access, rather than stalling. The banks operate concurrently; the controller just interleaves the requests between them, which is where the technique gets its name.

Why this matters more as CPUs got faster than memory

Interleaving became important because processor clock speeds improved much faster than DRAM access latency did over the decades — a gap sometimes called the memory wall. A single memory bank’s raw latency stayed roughly the same order of magnitude relative to a CPU cycle even as CPUs sped up dramatically, meaning a CPU waiting on one memory bank at a time would spend an increasing fraction of its cycles idle. Interleaving doesn’t shrink that per-access latency; it hides it by overlapping many accesses so the aggregate throughput available to the CPU looks much better than any single bank’s latency alone would suggest — the same idea behind pipelining and out-of-order execution hiding latency elsewhere in the pipeline, just applied to the memory subsystem instead of instruction execution.

Interleaving granularity

Systems can interleave at different address granularities, and the choice shapes which access patterns benefit:

• Fine-grained interleaving spreads consecutive individual words or small blocks across banks, maximizing parallelism for sequential access but adding coordination overhead per access.
• Coarse-grained interleaving assigns larger contiguous chunks to each bank before moving to the next, which suits access patterns where each processor or thread tends to work within its own region of memory (reducing contention between them) at some cost to raw sequential-access parallelism.

Modern DRAM controllers also interleave across multiple structural levels simultaneously — channels, ranks, and banks — each adding another dimension of parallelism, with the controller’s scheduling logic deciding how to spread a given access pattern across all of them.

Interleaving vs other memory-parallelism techniques

	What it parallelizes	Where it lives
Memory interleaving	Multiple banks servicing overlapping accesses	Memory controller / DRAM layout
Wider memory bus	More bits moved per single access	Physical bus width
Multi-channel memory	Fully independent memory controllers	Motherboard/CPU memory architecture
Cache hierarchy	Avoiding main memory access entirely	On-chip L1/L2/L3 cache

These techniques stack rather than compete — a system can interleave banks within each channel of a multi-channel setup, while a cache hierarchy sits in front of all of it trying to avoid touching main memory at all for frequently accessed data. High-bandwidth memory standards used in accelerators, like HBM, push this further still by combining very wide buses with deep internal banking and interleaving to sustain the bandwidth modern GPUs and AI accelerators demand — see also DDR vs GDDR for how consumer and graphics memory diverge on exactly this tradeoff between latency and bandwidth.

Why this stays mostly invisible to software

Interleaving is implemented in the memory controller and DRAM addressing scheme, not in application code — a program doesn’t choose which bank an address lands in, and on general-purpose systems it usually can’t. It matters directly to systems engineers designing memory controllers and to workloads sensitive to memory access patterns — NUMA-aware software, for instance, cares about which physical memory region a thread touches for related but distinct reasons around locality across sockets, not banking within a single region. For most software developers, interleaving is one of the reasons sequential memory access tends to be faster than scattered random access in practice, even when both touch the same total amount of memory — the sequential pattern is exactly what interleaving is built to accelerate.

The takeaway

Memory interleaving spreads consecutive addresses across independent memory banks so a controller can overlap multiple accesses instead of waiting on one bank’s fixed cycle time each time. It doesn’t make any individual memory access faster — it hides latency by keeping several banks busy at once, which is why sequential access patterns benefit far more than scattered ones. The technique operates below the level software normally sees, but it’s a load-bearing part of why memory throughput has kept pace with CPU demands even as per-access DRAM latency improved only slowly.