Software Write Detection for a Distributed Shared Memory
Matthew J. Zekauskas and Wayne A. Sawdon
School of Computer Science
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213
{mattz,wsawdon}@cs.cmu.edu
Brian N. Bershad
Department of Computer Science and Engineering FR-35
University of Washington
Seattle, WA 98185
bershad@cs.washington.edu
Abstract
Most software-based distributed shared memory (DSM) systems rely
on the operating system's virtual memory interface to detect writes
to shared data. Strategies based on virtual memory page protection
create two problems for a DSM system. First, writes can have high
overhead since they are detected with a page fault. As a result, a
page must be written many times to amortize the cost of that fault.
Second, the size of a virtual memory page is too big to serve as
a unit of coherency, inducing false sharing. Mechanisms to handle
false sharing can increase runtime overhead and may cause data to
be unnecessarily communicated between processors.
In this paper, we present a new method for write detection that
solves these problems. Our method relies on the compiler and
runtime system to detect writes to shared data without invoking
the operating system. We measure and compare implementations of
a distributed shared memory system using both strategies, virtual
memory and compiler/runtime, running a range of applications on a
small scale distributed memory multicomputer. We show that the
new method has low average write latency and supports fine-grained
sharing with low overhead. Further, we show that the dominant cost
of write detection with either strategy is due to the mechanism
used to handle fine-grain sharing.