We use a Sender-Based Protocol (SBP) to allow us to avoid the copying inherent in the typical layered protocol models. Work on the Sender-Based Protocols has proceeded on two levels. First, we have increased the efficiency (and decreased the latency) of the lowest level of the software. The obvious motivation for this is that the software overhead of the protocols should not dwarf the fabric/interface components of the cost of communications. An additional goal is to avoid inducing detrimental memory system effects due solely to the implementation of the protocols, e.g., cache misses resulting from poor locality in the protocol data structures,

Second, the API for the protocols has evolved as our experience in writing applications to it grows. The majority of message passing codes we have examined assume a communications model in which buffering and copying are performed, by the transport layer, at both sending and receiving ends. Sender-based protocols perform neither of these. Our API has been developed to export sufficient flexibility and control to the application to enable it to provide such transport layer functions that the algorithms require in an application specific manner.

We have identified an experimental model for our studies of message passing/memory hierarchy interactions. We are studying three cases tha we feel define the design space:

1. cache-only -- all message data are inserted directly into the cache, which is similar to processor mediated network IO;
2. memory only -- all message are placed into main memory (and invalidated in the cache if present), which is analgous to common organizations where network controllers sit on an IO bus far from the processor;
3. an optimal case, where the message data are placed in the cache if and only if the next reference to the cache lines involved is a reference to the message data; the determination is made on a cache line by cache line basis.

We have now implemented a number of complete applications on top of our SBPs to use in our simulation studies:

• a BLACS library, which makes available to us the large body of SCALAPACK code;
• a parallel ray tracer previously implemented on the CM5 at Los Alamos;
• a Scalable Parallel Short-range Molecular-dynamics code by Peter Lomdahl, Dave Beazley, Niels Gronbech-Jensen, and Tim Germann of Los Alamos National Laboratory

Furthermore, we are embarking on an implementation of Split-C, a language well suited to message passing models such as our SBP. This should provide us with an additional group of application codes already existing in that language.

---

This work was sponsored by the Space and Naval Warfare Systems Command (SPAWAR) and Advanced Research Projects Agency (ARPA), Communication and Memory Architectures for Scalable Parallel Computing, ARPA order #B990 under SPAWAR contract #N00039-95-C-0018

---

Back to the Avalanche Project Home Page, or Computer Science Department Home Page.
Feedback to <avalanche@jensen.cs.utah.edu>.