Breaking the Memory Wall

by
Zhen Fang

Advised by
John Carter

The performance of memory-intensive applications is often limited by how fast the memory system can provide needed data. For local memory, the speed gap between the CPU and DRAMs leads to significant stalls when there is not enough locality in the applications's memory references for caches to be effective. For remote memory, the increasing network latency, in terms of processor clock periods, makes inter-node communications inordinately expensive. In a cache-coherent, non-uniform memory access (cc-NUMA) system, the multiple non-overlappable network latencies dictated by a write-invalidate coherence protocol exacerbate the memory latency problem.

For applications with poor memory locality, caches do not improve performance. In general, moving data through the memory system and memory hierarchy into caches and subsequently the processor core is inefficient if the data is not reused sufficiently. A natural solution to this problem is to selectively move computation to data. I propose to design and evaluate Active Memory Operations (AMOs), which offload computation on data to processing elements on those data's home memory controllers. Performing AMOs at the home node memory controller avoids moving large amount of data across the network and through the cache hierarchy, eliminates serialized network latencies dictated by cache coherence protocols, and introduces opportunities for additional parallelism.