Swarm files can be updated by overwriting previous contents (physical updates) or by applying a semantic update procedure that Swarm later applies to all replicas. Swarm supports three distinct useful paradigms of shared data access: (i) whole-file access with physical updates to support unstructured variable-length byte sequences such as files, (ii) page-grained access to support persistent objects and other heap-based data structures, and (iii) whole file access with semantic updates to support structured data such as databases and directories.
To evaluate the feasibility of Swarm's design, we have built a prototype and used it to build four representative network services with distinct data characteristics and consistency needs.
Our first service, called SwarmFS, is a wide area peer-to-peer file system with a decentralized but uniform file name space. It provides ubiquitous file storage to mobile users via autonomously managed file servers. SwarmFS provides on a per-file-access basis, the strong consistency semantics of Sprite [3], the close-to-open consistency of AFS [1]and Coda [2] file systems, the weak eventual consistency flavors of Coda, Pangaea [5]and Ficus [4] file systems.
Our second application models how an online shopping service such as Amazon.com [] could deploy proxies to improve its responsiveness to wide area clients. It implements a database of sale items as a simple object database in a Swarm-hosted persistent page heap, employing strong consistency for data integrity. The shopping service deploys wide area proxy servers that operate on the shared database to improve responsiveness to wide area clients. We have devised an adaptive caching algorithm for Swarm to decide when and where to replicate data; data with good locality tends to be aggressively migrated or replicated, while data with poor locality or heavy write sharing tends to be managed via a more efficient centralized mechanism. Adaptive caching improves responsiveness by an order of magnitude even under modest locality.
Our third application illustrates how Swarm enables replication to be transparently added to an existing database while providing significant control over consistency, and the performance benefits of doing so. We built a replicated version of the BerkeleyDB [6] embedded database by hosting it in a Swarm file and using semantic updates for synchronizing replicas. We evaluate its performance using five different consistency requirements ranging from strong (appropriate for a conventional database) to time-based (appropriate for many directory services). We find that relaxing consistency requirements even slightly improves throughput significantly.
Our final application is an online chat service that stresses the ability of Swarm to efficiently synchronize a large number of replicas in real-time. It models applications that involve producer-consumer style interaction among a number of users or service components, such as online real-time chatting, Internet gaming and data logging and content dissemination. The application requires that events/data produced at one or more sites (such as a message typed by a user to a chat room or a player's move in a game) be propagated to other affected sites in real-time. A centralized server can be quickly overwhelmed by the CPU and network load imposed due to a high rate of concurrent events. Swarm's hierarchical replica networking helps incrementally scale the overall service to a large number of participants and a high rate of concurrent events.
Each of these applications employs a different paradigm for accessing replicated data, and achieves different consistency semantics using Swarm. Swarm successfully offloaded replication logic from each of these applications, and effectively exploited inherent locality in each of them. This shows that composable consistency is both powerful, i.e., expresses a variety of useful consistency semantics, as well as practical and useful, i.e., effectively supports wide area replication in diverse distributed applications.
This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.71)
Copyright © 1993, 1994, 1995, 1996,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999,
Ross Moore,
Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -notransparent -white -noshow_section_numbers swarm-overview.tex
The translation was initiated by Sai Rama Krishna Susarla on 2004-08-09