Experiences with Locking in a NUMA Multiprocessor Operating System Kernel
Ronald C. Unrau, Orran Krieger, Benjamin Gamsa, Michael Stumm
Department of Electrical and Computer Engineering
Department of Computer Science
University of Toronto
unrau@eecg.toronto.edu
Abstract
We describe the locking architecture of a new operating system, Hurricane,
designed for large scale shared-memory multiprocessors.Many papers already
describe kernel locking techniques, and some of the techniques we use
have been previously described by others. However, our work is novel in the
particular combination of techniques used, as well as several of the individual
techniques themselves. Moreover, it is the way the techniques work together
that is the source of our performance advantages and scalability. Briefly, we
use:
- a hybrid coarse-grain/fine-grain locking strategy that has the low latency
and space overhead of a coarse-grain locking strategy while having the
high concurrency of a fine-grain locking strategy;
- replication of data structures to increase access bandwidth
and improve concurrency;
- a clustered kernel that bounds the number of processors that can
compete for a lock so as to reduce second order effects such as memory
and interconnect contention;
- Distributed Locks to further reduce second order effects, with
modifications that reduce the uncontended latency of these locks to
close to that of spin locks.