A Read-only Distributed Hash Table,

A Read-only Distributed Hash Table,
J Grid Computing (2011) 9:501–529
DOI 10.1007/s10723-011-9189-9
A Read-Only Distributed Hash Table
Verdi March · Yong Meng Teo
Received: 22 June 2010 / Accepted: 15 March 2011 / Published online: 27 April 2011
© Springer Science+Business Media B.V. 2011
Abstract A distributed hash table (DHT) is an infrastructure to support resource discovery in large
distributed systems. In a DHT, data items such as
resources, indexes of resources or resource metadata, are distributed across an overlay network
based on a hash function. However, this may not
be desirable in commercial applications such as
Grid and cloud computing whereby the presence
of multiple administrative domains leads to the
issues of data ownership and self-economic interests. In this paper, we present R-DHT (Read-only
DHT), a DHT-based resource discovery scheme
without distributing data items. To map each data
item back onto its resource owner, a physical host,
we virtualize each host into virtual nodes. Nodes
are further organized as a segment-based overlay
network which increases node failure resiliency
without replicating data items. We demonstrate
the feasibility of our proposed scheme by presenting R-Chord, an implementation of R-DHT
using Chord as the underlying overlay graph, with
V. March (B) · Y. M. Teo
Department of Computer Science,
National University of Singapore,
Computing 1, 13 Computing Drive,
Singapore 117417, Singapore
e-mail: [email protected],
[email protected]
Y. M. Teo
e-mail: [email protected]
lookup and maintenance optimizations. Through
analytical and simulation analyses, we evaluate
the performance of R-DHT and compare it with
traditional DHTs in terms of lookup path length,
resiliency to node failures, and maintenance overhead. Overall, we found that R-DHT is effective
and efficient for resource indexing and discovery
in large distributed systems with a strong commercial requirement.
Keywords Grid · Resource indexing
and discovery · Administrative domain ·
Overlay network
1 Introduction
Distributed hash tables (DHT) [31, 35, 45, 59] are
a decentralized lookup scheme that aims to provide scalable lookups with high result guarantee
for large distributed systems consisting of multiple
peer nodes. A DHT, as with a hash-table data
structure, provides an interface to retrieve a keyvalue pair. A key is an identifier assigned to a
resource; traditionally this key is a hash value associated with the resource. A value is an object to
be stored into the DHT; this could be the shared
resource itself (e.g. a file), an index (pointer) to
a resource, or a resource metadata. An example of a key-value pair is SHA1(file_name),
http://peer-id/file, where the key is the
502
V. March, Y.M. Teo
SHA1 hash of the file name and the value is the
address (location) of the file.
A DHT achieves its design objectives including
lookup scalability and result guarantee through:
1. Key-to-Node Mapping—Assuming that keys
and nodes share the same identifier space, a
DHT maps key k to node n where n is the
node closest to k in the identifier space; we
refer to n as the responsible node of k. Compared to other decentralized lookup schemes
such as [5, 17, 18, 20, 21, 38, 48, 57, 67], the
key-to-node mapping is perceived to increase
result guarantee (i.e., a lower number of false
negative answers) because determining the
existence of a key-value pair equals to locating
the node responsible for the key [47].
2. Data-Item Distribution—All key-value pairs
(data items) whose key is equal to k are then
stored at node n regardless of who owns these
key-value pairs. This is implemented in existing DHTs by providing a store operation
[23, 60]. The concept of data-item distribution
has been further exploited for various optimizations, including load balancing [29, 30, 37]
and high availability [22, 28, 42–44].
3. Structured Overlay Network—Nodes are organized as a structured overlay network which
strikes a balance between the routing performance and the overhead of maintaining
routing states. There are two important characteristics of the structured overlay network:
a. Topology
A structured overlay network resembles
a graph with a certain topology such as a
ring [62, 65], a torus [58], or a tree [4, 52].
b. Ordering of nodes
The position of a node in a structured
overlay network is determined by the
node identifier.
By exploiting the structured overlay network,
a DHT can locate any responsible node in
a short and bounded number of hops. For
example, the lookup path length in the Chord
ring is O(log N) [65].
Though DHTs have been shown to exhibit
good lookup performance, the assumption on distributing data items hinders the widespread com-
mercial adoption of DHT. In large distributed
systems such as Grid computing [26, 41] and cloud
computing [6, 11], nodes belonging to different
companies come under different administrative
domains. In such applications, distributing data
items among different administrative domains
leads to two major issues:
1. Regulatory Compliance—Distributing data
items works against the current practice of
controlling data location in order to enforce
copyright and to meet regulatory requirements [40, 66]. The emergence of cloud computing further exacerbates the importance of
data-item ownership and location [19, 27, 39].
In view of this, it may not be commercially viable for an organization to store its data items,
including pointers or indexes to resources, on
nodes belonging to other organizations.
2. Conflicting Self-Interests among Administrative Domains—Data-item distribution requires
all nodes in a DHT overlay to be publicly
writable. However, this may not happen when
nodes do not permit the sharing of their
storage resources to external parties due to
different economic interests. Firstly, nodes
want to protect their investment in storage infrastructure by not storing data items belonging to other nodes. Secondly, an individual
node may limit the amount of storage it offers;
this reduces the result guarantee when the
total amount of storage in the DHT is smaller
than the total number of key-value pairs.
In addition to the problem in enforcing storage policies, nodes also face a challenge when
their infrastructure is used by customers of
other parties [56, 63]. For example, when a
node stores data items belonging to other parties, its computing and network performance
may degrade as a result of participating in the
lookup requests belonging to other parties.
These two issues can be addressed by not distributing data items. However, by design, a DHT
assumes that data items are distributed across
overlay networks.
In this paper, we propose a resource discovery scheme based on a read-only DHT (R-DHT)
and, as an example, apply it to support decentralized resource indexing and discovery in large
A Read-Only Distributed Hash Table
computational Grids [26, 41]. R-DHT is a scheme
that enables DHTs to map keys to nodes without distributing data items [51]. By organizing
the administrative domains in a Grid as an RDHT overlay, each of these domains can control
their own resource discovery information but still
allowing the other domains to read it. R-DHT
reuses the functionalities in existing structured
overlay networks such as Chord [65] to leverage
on existing extensive research in this field. The
read-only mapping scheme in R-DHT virtualizes
a host (i.e., a physical entity that shares resources)
into nodes: one node is associated with each
unique key belonging to the host. Nodes are then
organized as a segment-based structured overlay
network and the node identifier space is divided
into two sub-spaces: key space and host identifier
space. Our scheme inherits the good properties of
DHTs, namely support for decentralized lookups
to minimize single point of failure, high result
guarantee, and bounded lookup path length. In
addition, R-DHT is (i) by design resilient to node
failures, (ii) supports the “pay-for-your-own” usage model whereby each node stores only its own
key-value pairs, and without depending on thirdparty publicly-writable nodes [8], and (iii) avoids
inconsistency of data updates as changes to a keyvalue pair are reflected immediately by the node
that owns it.
Our main contributions are as follow:
1. Design of R-DHT—R-DHT (read-only DHT)
is a DHT-based resource discovery scheme
whereby data items are not distributed across
the overlay network. R-DHT reuses the existing functionalities from conventional DHTs
and as such, improvements in a particular
DHT are beneficial to R-DHT as well. To
demonstrate this, we present R-Chord, an implementation of the R-DHT scheme that uses
Chord as the underlying overlay graph. RChord uses Chord’s join algorithm to construct its overlay network. In addition, the
algorithms for lookup and maintenance of
the overlay (i.e., stabilization) are similar to
Chord.
2. Lookup Optimizations—We show that the
lookup path length in R-DHT is at worst equal
to the corresponding DHT. The segment-
503
based overlay in R-DHT allows messages to
be routed by segments and finger tables to be
shared among nodes. When the number of
unique keys (K) is larger than the number
of hosts (N), e.g. in file-sharing P2P systems,
the lookup path length in R-DHT is bounded
by O(log N) as in traditional DHTs such as
Chord. However, when K ≤ N, e.g. in computational Grids, the O(log K)-hops lookup path
length of R-DHT is shorter than traditional
DHTs.
3. Lookup Resiliency to Node Failures—In the
event of node failures, we show that R-DHT’s
segment-based overlay reduces the number
of failed lookups, i.e., lookups that return a
false negative or false positive answer. In RDHT, lookups for key-value pairs shared by
many nodes are more likely to succeed even
without replicating key-value pairs. When one
of the nodes fails, only its own key-value pairs
become unavailable. The remaining key-value
pairs in other nodes can still be discovered because R-DHT exploits a segment-based overlay by using backup fingers. Thus, there is a
higher probability of finding resources of a
certain type when many nodes share resources
of the same type.
The remainder of this paper is organized as follows. Section 2 presents an overview of Chord. In
Section 3, we discuss the design of R-DHT and its
Chord-based implementations, namely R-Chord.
Theoretical and simulation analyses of the performance of R-DHT are presented in Sections 4 and
5, respectively. Section 6 discusses related work.
Finally, Section 7 concludes this article.
2 Background
2.1 Distributed Grid Resource Indexing
and Discovery
Throughout this paper, we use distributed resource discovery in a computational Grid as the
example of R-DHT applications. A computational
Grid facilitates the sharing of compute resources
from different administrative domains [26, 41].
Typically, Grid users search for specific resources
504
V. March, Y.M. Teo
where their job will be executed [54]. In a centralized scheme, an MDS (Monitoring and Discovery
System) [3] indexes all resources from all administrative domains and processes all user queries
(Fig. 1a). As the adoption of Grid technologies
increases, the central MDS becomes a potential
bottleneck and a single point of failure.
Recently, there is a growing interest in studying
the use of DHT-based resource discovery for large
computational Grids [10, 12, 64, 69]. Instead of
depending on a third-party central MDS, DHTbased schemes distribute indexes and queries
across administrative domains organized as nodes
in an overlay network. As illustrated in Fig. 1b,
each domain still deploys an MDS (or any other
schemes) for the intra-domain discovery. Then,
R-DHT is applied for the inter-domain resource
indexing and discovery. This enables a computational Grid to support scalable distributed resource discovery with high result guarantee, while
allowing each administrative domain to index only
its own resources.
2.2 Overview of Chord
Chord is a DHT implementation that supports
O(log N)-hop lookup path length and O(log N)
routing states per node, where N denotes the total
number of nodes [65] . Chord organizes nodes as
a ring that represents an m-bit one-dimensional
circular identifier space, and as a consequence, all
arithmetic operations are modulo 2m . To form a
(a) Centralized
Fig. 1 Resource discovery in a computational Grid
ring overlay, each node n maintains two pointers
to its immediate neighbors (Fig. 2a). The successor pointer points to successor(n), i.e., the immediate neighbor of n clockwise. Similarly, the
predecessor pointer points to predecessor(n), the
immediate neighbor of n counter clockwise.
Chord maps key k to successor(k), the first
node whose identifier is equal to or greater
than k in the identifier space (Fig. 2b). Thus,
node n is responsible for keys in the range of
( predecessor(n), n], i.e., keys that are greater than
predecessor(n) but smaller than or equal to n. For
example, node 32 in the figure is responsible for
all keys in (21, 32]. All key-value pairs whose key
equals k are then stored on successor(k) regardless
of who owns the key-value pairs; this is the dataitem distribution.
Finding key k implies that we route a request
to successor(k). The simplest approach for this
operation, as illustrated in Fig. 2c, is to propagate
a request along the Chord ring in a clockwise
direction until the request arrives at successor(k).
However, this approach is not scalable as its complexity is O(N), where N denotes the number of
nodes in the ring [65].
To speed-up the process of finding successor(k),
each node n maintains a finger table of m entries
(Fig. 2d). Each entry in the finger table is also
called a finger. The ith finger of n is denoted
as n. f inger[i] and points to successor(n + 2i−1 ),
where 1 ≤ i ≤ m. Note that the 1st finger is also
the successor pointer while the largest finger divides the circular identifier space into two halves.
(b) Decentralized
A Read-Only Distributed Hash Table
(a) Chord Ring
505
(b) Map and Distribute Keys to
Nodes
(d) The Fingers of Node 8
(c) Traverse the Ring to
Find successor (54)
(e) f i n d _successor (54) Utilizing
Finger Tables
Fig. 2 Chord lookup
When N < 2m , the finger table consists of only
O(log N) unique entries [65].
By utilizing finger tables, Chord locates
successor(k) in O(log N) hops with high probability [65]. Intuitively, the process resembles
a binary search where each step halves the distance to successor(k). Each node n forwards a
(a) Insert Node 25
Fig. 3 Join operation in Chord
request to the nearest known preceding node
of k. This is repeated until the request arrives
at predecessor(k), the node whose identifier
precedes k, which will forward the request to
successor(k). Figure 2e shows an example of
finding successor(54) initiated by node 8. Node
8 forwards the request to its 6th finger which
(b) Migrate Key 22 from Node 32
(c) Correct Successor and
Predecessor Pointers
506
V. March, Y.M. Teo
(b) virtualize : hosts → nodes
(a) Host in the Context of Computational Grid
Fig. 4 Hosts and nodes
points to node 48. Node 48 is the predecessor of
key 54 because its 1st finger points to node 56
and 48 < 54 ≤ 56. Thus, node 48 will forward the
request to node 56.
Figure 3 illustrates the construction of a Chord
ring. A new node n joins a Chord ring by locating
its own successor. Then, n inserts itself between
successor(n) and the predecessor of successor(n),
illustrated in Fig. 3a. The key-value pairs stored
on successor(n), whose key is less than or equal to
n, are migrated to node n (Fig. 3b). Because the
join operation invalidates the ring overlay, every
node performs periodic stabilizations to correct its
successor and predecessor pointers (Fig. 3c), and
its fingers.
3 R-DHT Design and Implementation
We start by introducing the terms resource type,
host and node.
Definition 3.1 (Resource Type) A resource type is
a list of attribute names that describes a resource.
The resource type determines the key assigned
to a resource. There could be many resource instances with the same type; these resources are
assigned the same key.
Table 1 Variables
maintained by host
and node
Definition 3.2 (Host) A host refers to a physical
entity that shares resources. Let Th denotes the set
of unique keys (i.e., resource types) owned by a
host whose host identifier is h.
Figure 4a illustrates how the above terminologies are applied in a computational Grid. In this
example, a host refers to the MDS server. The two
keys, T3 = {2, 9}, denote that host 3, representing administrative domain 3, indexes two types
of resources, resource types 2 and 9. Details on
assigning a key to a resource is discussed in [49].
Definition 3.3 (Node) A node refers to a logical
entity in an overlay network.
In terms of set theory, a multiple-valued function, virtualize : hosts → nodes, describes the relationship between hosts and nodes (Fig. 4b). A host
joins an overlay network as one or more nodes,
i.e., by assuming one or more identities in the
overlay. Clearly, each node corresponds to only
one host. The notion of “nodes” is equivalent to
“virtual servers” in Cooperative File System [22]
or “virtual hosts” in Apache HTTP Server [1].
Table 1 shows some of the variables maintained
by each host and Chord node. In addition to its
node identifier (n), each node maintains its states
Entity
Variable
Description
Host h
Node n
Th
f inger[1 . . . F]
successor
predecessor
a set of unique keys owned by host h
a finger table of F entries
the next node in the ring overlay, i.e., f inger[1]
the previous node in the ring overlay
A Read-Only Distributed Hash Table
in an overlay topology including f inger, successor,
and predecessor.
In presenting pseudocode, we adopt the notations from [65]:
1. Let h denotes a host or its identifier, and n denotes a node or its identifier, as their meaning
will be clear from the context.
2. Remote procedure calls or variables are preceded by the remote node identifier, while
local procedure calls and variables omit the
local node identifier.
In the following, we describe read-only mapping, the main concept in R-DHT. Then, we
discuss the construction of R-DHT overlays and
the lookup algorithm using Chord [65] as the
underlying overlay graph. Subsequently, we use
“R-DHT” to refer to read-only DHT in general,
and “R-Chord” to refer to a Chord-based R-DHT.
3.1 Read-Only Mapping
R-DHT is a read-only DHT where a node supports “read-only” accesses to its keys, and does
not allow keys belonging to other nodes to be
written (mapped) on it. The read-only property
ensures that keys are mapped onto the node that
owns and shares the resource. This node is called
the originating node. With its read-only property,
R-DHT addresses issues such as node autonomy
in placing key-value pairs, prevents stale data
items, and increases lookup resiliency to node
failures without the need to replicate keys.
As shown in Fig. 5, R-DHT achieves the readonly mapping by virtualizing a host into a number
of nodes where each node represents a unique
key shared by the host. The node identifier space
is divided into two sub-spaces: a key space and
507
a host identifier space. This ensures the uniqueness of node identifiers without compromising
R-DHT’s support for a flat naming scheme [8].
Shared resources of the same type are identified
by the same key and form a segment on the overlay graph. We exploit segment-based routing to
reduce lookup path length and improves lookup
resiliency to node failures without a need to replicate data items.
The basic idea of read-only mapping is to exploit the property of the DHT mapping whereby
a key can be mapped onto a specific node if
the identifier of the node is equal to the key
(Fig. 6). Thus, each node in R-DHT is a bucket
with one unique key, as opposed to conventional
DHT where each node is a bucket with a number
of unique keys. R-DHT realizes the key-to-node
mapping through virtualization and splitting of
node identifiers. Virtualization ensures that each
host can share different keys, whereas the splitting
of node identifiers prevents node collisions when
several hosts share the same key.
R-DHT virtualizes each host into a number of
nodes by associating node n to each unique key k
belonging to host h. Figure 7a shows an example
of virtualizing two hosts into four nodes, where
each host with two unique keys is virtualized into
two nodes. By making the node identifier equals
to its associated key, R-DHT ensures that keys are
not distributed. However, when nodes and keys
share the same identifier space, virtualizing several hosts sharing the same key results in collisions
of node identifiers (Fig. 7b).
To avoid the collision of node identifiers, a
node associated to key k shared by host h is
assigned k|h as its identifier. Each node can be
uniquely identified by its node identifier which is
the concatenation of the key (k) and the host identifier1 (h). Figure 8a shows an example of node
identifier where the key and the host identifier are
of the same bit-length2 , i.e., (m/2) bits. The m-bit
node identifier space is divided into 2m/2 segments.
Each segment Sk consists of 2m/2 consecutive node
1 A host identifier can be derived by hashing the host’s
IP address or an identifier obtained from a certification
authority [15].
Fig. 5 Proposed R-DHT scheme
2 In general, the key space and the host identifier space
need not be of the same size.
508
V. March, Y.M. Teo
(a) Conventional DHT Maps Key to Node Identifier
Closest to the Key
(b) R-DHT Maps Key to Node Identifier Equal to
the Key
Fig. 6 Mapping keys to node identifiers
Fig. 7 Virtualization
in R-DHT
(a) Virtualize Two Hosts into
Four Nodes
(a) m-bit Node Identifier
(b) Collision of Node Identifiers
(b) Segmenting Node-Identifier Space
Fig. 8 R-DHT node identifiers
Table 2 Comparison of
API in R-DHT with
conventional DHT
Operation
Host h joins overlay through existing host e
Host h shares new key k
Users at host h search for key k
API
DHT
R-DHT
h. join(e)
h.store(k)
h.lookup(k)
h.virtualize(e)
h.newKey(k)
h.lookup(k)
A Read-Only Distributed Hash Table
identifiers prefixed with k (Fig. 8b). Therefore,
each segment represents resources of the same
type that are offered for sharing by different hosts.
Segment indexing reduces lookup path length and
improves fault-tolerance.
R-DHT is designed to maintain API compatibility with conventional DHTs and supports the
flat naming scheme [8]. As shown in Table 2, the
R-DHT API requires the same arguments as its
DHT counterparts. The lookup(k) API of R-DHT
supports a flat naming scheme by abstracting the
location of keys in its argument. With the flat
naming scheme, queries are formulated as “find
resource type k”. This allows R-DHT to fully provide a hashtable abstraction where only the key
is required to retrieve key-value pairs. However,
systems that support only a hierarchical naming
scheme do not fully provide a hashtable abstraction of locating key-value pairs, since the hierarchical naming scheme requires users to specify
the location of the key as an argument in the
lookup operation, in addition to the key itself.
Thus, queries are formulated as “find resource
type k from host h”, which are reminiscent of
HTTP requests such as “retrieve index.html
from www.comp.nus.edu.sg”.
R-DHT can be implemented as a one-level
(i.e., flat) overlay network or a two-level (i.e.,
hierarchical) overlay network. In this paper, we
discuss only flat R-DHT: construction of overlay,
lookup algorithm, and maintenance of overlay
(Section 3.2). Hierarchical R-DHT can be implemented based on a hierarchical scheme for conventional DHTs such as March et al. [50].
509
Fig. 9 Virtualizing host into nodes
segments where segment Sk consists of nodes
prefixed with k. Thus, each segment Sk represents
resources of the same type shared by different
hosts.
Figure 10 compares Chord and R-Chord in a
Grid consisting of three administrative domains,
assuming that keys and host identifiers are 4-bit
long. In Chord (Fig. 10a), each host becomes a
Chord node and its data items, keys and indexes of
its compute resources, are distributed to another
3.2 R-Chord
(a) Chord Distributes Data Items among Three Nodes
R-Chord is a flat R-DHT that organizes nodes as
a (one-level) Chord overlay. Figure 9 shows how a
new host joins R-Chord, and how an existing host
shares a new key. Each new node joins the ring
overlay using Chord’s join protocol (lines 5 and
13 in Fig. 9). Nodes are organized as a logical ring
in clock-wise ascending order where each node
identifier (k|h) is interpreted as an integer. Let N
N−1
Th |
denote the number of hosts, and K = | h=0
denote the total number of unique keys in the
system. R-Chord divides its ring overlay into K
(b) R-Chord Virtualizes Three Hosts into Six Nodes
Fig. 10 Chord and R-Chord
510
V. March, Y.M. Teo
node whose identifier immediately succeeds the
key. For example, all keys with identifier 2 will be
stored on node 3. In R-Chord (Fig. 10b), each key
is mapped to its originating node. In this example,
host 3 owns two unique keys, i.e., T3 = {2, 9}, and
thus, we virtualize host 3 into two nodes with
node identifiers 2|3 = 35 and 9|3 = 147. Similarly,
we virtualize host 6 and host 9 into one node
and three nodes, respectively. These six nodes are
organized as an R-Chord ring based on their integral node identifier. The R-Chord ring consists of
three segments, namely segment S2 with node 2|3
and node 2|9, segment S5 with node 5|9 and 5|6,
and segment S9 with node 9|3 and node 9|9. These
three segments represent three keys or resource
types, namely key 2, key 5, and key 9.
R-DHT prevents stale data items when updated by their originating node, while conventional DHT must route the update to the node
where the data item is distributed. The data item
becomes stale before the update reaches the node.
In Fig. 10, when host 3 updates its key 9, Chord
routes the update to node 9 which stores the key.
On the other hand, in R-Chord the update is
(a) R-Chord
reflected immediately because key 9 is mapped
onto node 9|3 which is associated with the resource owner, host 3.
By design, R-DHT is inherently fault tolerant
without a need to incorporate data-item replications as a resiliency mechanism. Firstly, a node
failure does not affect keys belonging to other
nodes. Secondly, lookup(k) can still be successful
since R-DHT can route the lookup request to
other alive nodes in segment Sk . Figure 11 shows
an example when administrative domain 3 with
two shared resource types failed. Since administrative domain 9 is still alive, R-Chord can still
locate resource type 2 by routing a lookup(2)
request to node 2|9 (Fig. 11a). On the contrary,
lookup fails in Chord because it is routed to node
3 (Fig. 11b). Typically, a conventional DHT replicates keys to improve its resiliency. However, this
further increases the risk of stale data items, i.e.,
data items pointing to unavailable resources in the
inaccessible administrative domain 3 (Fig. 11c).
R-DHT supports a flat naming scheme where
users need to specify only key k when searching.
R-DHT bases its lookup on the underlying over-
(b) Chord without Replication
(c) Replication Introduces Stale Data Items
Fig. 11 Node failures and stale data items
A Read-Only Distributed Hash Table
511
a node n where pref ix(n ) = k, then key k is
successfully found (line 11); otherwise, the key
does not exist (line 14).
The direct application of Chord’s lookup is
not efficient because it does not exploit the
advantages of read-only mapping. In a system
N−1
Fig. 12 The fingers of node 2|3
lay’s lookup protocol. With Chord as the underlying overlay, each node maintains at most m unique
fingers (Fig. 12). Lookup for key k implies locating
the successor of k|0, i.e., the first node in segment
Sk . Figure 13 shows a direct application of Chord
lookup algorithm on R-Chord. If a lookup returns
|T |
with N hosts where each host has T = h=0N h
unique keys on average, R-Chord consists of N · T
nodes and hence, its lookup path length is
O(log NT) hops (based on Theorem 2 in [65]).
However, we reduce R-Chord lookup path length
to O(min(log K, log N)) hops (Theorem 4.1) by
exploiting our read-only mapping scheme using
two optimizations, namely routing by segments
and shared routing tables. The complete algorithm
is shown in Fig. 14.
3.2.1 Routing by Segments
(a) Main Algorithm
(b) Helper Functions
Fig. 13 Unoptimized R-Chord lookup
R-DHT divides its node-identifier space into segments, with each segment representing a key.
Therefore, locating key k equals to locating any
node within segment Sk , instead of strictly locating successor(k|0) only. R-Chord exploits this
segmentation by forwarding a lookup(k) request
from one segment to another segment (lines 4,
7, and 16 in Fig. 14a, and line 6 in Fig. 14b).
Each routing step halves the distance, in term
of segments, to the destination segment Sk (see
Lemma 4.1 for the proof). As such, in a system
with K segments, routing by segments reduces
lookup path length to O(log K). Since segments
are identified by the prefix of node identifiers,
our routing-by-segments scheme is essentially a
prefix-based routing optimization applied for the
Chord lookup protocol.
Figure 15 illustrates the processing of a
lookup(k) request initiated by node n1 . Without routing by segments, i.e., the direct application of Chord’s lookup protocol, the lookup path
is n1 → n2 → n3 → n4 → n5 because we always
locate node n5 = successor(k|0). However, with
routing by segments, we realize that one of the
intermediate hops, node n2 , has a finger pointing
to n6 . Though node n6 is not successor(k|0), it also
shares key k and is in segment Sk . Since the lookup
can then be completed at node n6 , the optimized
lookup path becomes n1 → n2 → n6 .
512
V. March, Y.M. Teo
Fig. 14 R-Chord lookup
exploiting R-DHT
mapping
(a) Main Algorithm
(b) Helper Functions
3.2.2 Shared Finger Tables
To limit the lookup path length at O(log N) hops
even when K > N, our routing algorithm utilizes
all the |Th | finger tables maintained by each host
h (lines 3 and 11 in Fig. 14a, and lines 3 and
14 in Fig. 14b). In other words, a node’s finger
table is shared by all nodes from the same host.
As such, visiting one host is equivalent to visiting
all the |Td | nodes which correspond to the host
A Read-Only Distributed Hash Table
513
(a) No Routing by Segments
(b) Routing by Segments
Fig. 15 lookup(k) with and without routing by segments
(Fig. 16). The proof that this optimization leads
to O(log N)-hop lookup path length, similar to
Chord, is presented in Theorem 4.1. However,
an intuitive explanation is as follows: since the
distance between any two points in the overlay is
at most N hosts, it takes O(log N) hops to locate
any segment. Thus, even though the number of
nodes in the overlay is greater than N due to
host virtualization, the lookup path length is not
affected.
3.2.3 Maintenance of Overlay Graph
As with Chord, R-Chord maintains its overlay through periodic stabilizations. The periodic
stabilization is implemented as two functions:
stab ilize_successor() and correct_ f ingers(). The
first function corrects successor, (i.e., the first
(a) Visiting Host 3 is Equal to
Visiting Node 2|3 and Node 9|3
in the Overlay
Fig. 16 Effect of shared finger tables on routing
finger, and predecessor pointers, whereas the later
correct the remaining fingers in a finger table. The
rate with which these functions are invoked is an
implementation consideration.
During periodic stabilization, R-Chord exploits finger flexibility which is inherent in our
segment-based overlay. Finger flexibility denotes
the amount of freedom available when choosing
a finger. A higher finger flexibility increases the
robustness of lookup since finger tables deplete
slower in the event of node failures [31]. Finger flexibility also allows proximity-based routing
to reduce lookup latency. However, our current
R-Chord implementation has yet to exploit this
feature.
The segment-based overlay improves the finger
flexibility of R-Chord whereby n. f inger[i] is allowed to point to any nodes in the same segment
as successor(n + 2i−1 ) – this is an improvement
(b) Visiting Host 6
(c) Visiting Host 9 −All Nodes
are Visited
514
Fig. 17 Finger tables with backup fingers
over O(1) finger flexibility of Chord3 . To exploit
finger flexibility, we employ the backup fingers
scheme which is reminiscent of Kademlia’s κbucket routing tables [52]. With such a scheme,
every R-Chord node n maintains backups for each
of its fingers. Thus, when f dies (i.e., points to
a dead node), n still has a pointer to segment
S prefix( f ) . The new structure of the R-Chord finger
table is shown in Fig. 17. We use n. f inger[i] and
b ackup(n. f inger[i]) to denote the main finger and
the list of backup fingers, respectively.
Figure 18 shows the algorithm to correct successors in R-Chord. When a new successor is
detected, the old successor pointer is added into
the backup list instead of being discarded (line 7).
Similarly, when a new predecessor is to be set, the
old predecessor is also added into the backup list
(line 19). We then ensure that the backup list of
successors and predecessors contains only nodes
with the same prefix as the new successor (line 9)
and predecessor (line 21), respectively.
Figure 19 shows that the algorithm to correct
finger f in R-Chord exploits finger flexibility.
When a new finger is added, we also construct
its backup list based on the entries piggybacked
from the remote node and the older valid backup
entries (lines 7–9 in Fig. 19a). As with lookups, the
finger-correction algorithm incorporates shared
finger tables (line 4 in Fig. 19a and line 3 in
Fig. 19b).
4 Theoretical Analysis
In this section, we analyze the lookup performance of R-Chord, and compare the overhead
3 To
improve its robustness in spite of its lower finger
flexibility, each Chord node caches additional entries in its
finger tables. Such scheme can also be adopted in R-Chord.
V. March, Y.M. Teo
of the mapping scheme in R-Chord with conventional Chord-based DHT, hereafter referred to
as Chord. Let N denote the number of hosts,
Th denote the set of unique keys owned by host
h, T denote the average number of unique keys
N−1 |T |
owned by each host (i.e., h=0N h ), and K denote
the total
number of unique keys in the system
N−1
Th |). The Chord overlay consists of
(i.e. | h=0
N nodes, one node per host, and the R-Chord
overlay consists of V = NT nodes with an average
of T nodes per host.
4.1 Lookup
In the following, we present the proof on the
lookup path length of R-Chord.
Lemma 4.1 (Lookup Path Length of Routing by
Segments) Routing by segments leads to O(log K)hops lookup.
Proof To analyze the lookup path length due to
our routing-by-segments optimization, we compare the finger tables in R-Chord and Chord.
According to Theorem 2 in [65], in a Chord system consisting of N nodes, the lookup path length
is O(log N) if Chord is able to route a lookup
request from one node to another such that each
step halves the distance to the destination node.
To achieve this, each node n maintains O(log N)
unique fingers where:
1. The distance between n and n. f inger[i + 1] is
twice the distance between n and n. f inger[i].
2. The largest finger of n points to
successor(N/2). As highlighted in Section 2.2,
the division operation in N/2 implies a
modulo by 2m .
We now show the similarity of finger tables in
R-Chord and Chord. Let S = O(N P(k ∈ Th )) denotes the average number of nodes in a segment.
In an R-Chord system consisting of V nodes, each
node n maintains O(log V) unique fingers where:
1. The first O(log S) of the O(log V) fingers
point to the segment containing n.successor.
The remaining O(log V − log S) fingers
point to O(log V − log S) different segments
because the distance between n and
A Read-Only Distributed Hash Table
515
Fig. 18 Successorstabilization algorithm
n. f inger[log(S + j + 1)] is twice the distance
between n and n. f inger[log(S + j)], where
0 ≤ j ≤ log N − log S.
2. The largest finger of n will point to
successor(N/2), which is a node in the
segment that succeeds segment K/2.
Fig. 19 Finger-correction
algorithm
(a) Main Algorithm
(b) Helper Functions
516
V. March, Y.M. Teo
distributed across hosts, based on observation oi
real Grid testbeds such as the TeraGrid.4
Using the same argument as in Chord, R-Chord
routes a lookup request from one segment to
another and each hop halves the distance, in terms
of the number of segments, to the destination segment. Since R-Chord consists of K segments, a
lookup will cost O(log K) hops.
Lemma 4.2 (Probability of a Host to own a Key)
The probability that host h owns key k, i.e., P(k ∈
K
Th ), is bounded by ln K−T
.
Theorem 4.1 (Lookup Path Length in R-Chord)
With shared f inger tables, the lookup path length
in R-Chord is O(min(log K, log N)) hops.
Proof Let k ∈ Th denote key k owned by host h.
We define the probability that host h owns key
k as
Proof If K ≤ N then log K ≤ log N, assuming
that K, N > 0. Thus, according to Lemma 4.1, this
theorem is true.
Consider K > N. When host h processes
lookup(k), we choose two consecutive keys s, u ∈
Th where
P(k ∈ Th ) = P(e1 ) + P(e2 |e1 ) + P(e3 |e1 , e2 )
1. s < k < u
2. There is no v ∈ Th such that s < v < u
The two keys, s and u, are associated with two
nodes, namely node s|h and node u|h, respectively. The destination segment Sk will be located
between node s|h and node u|h, and the distance between s|h and u|h is O(K/T). Since K ≤
V, then K/T = O(V/T) = O(NT/T) = O(N).
Thus, according to Theorem 2 in [65], lookup(k)
can be routed from node s|h to segment Sk in
O(log N) hops.
As shown, the lookup performance in R-Chord
is at worst comparable to the lookup performance
in Chord. With shared finger tables, R-Chord’s
lookup path length is equivalent to Chord where
the number of hops to reach a certain node is
affected by the number of hosts in the physical
network (N) instead of the number of nodes in the
overlay network (V).
4.2 Overhead
The following theorems compare the maintenance
overhead in R-Chord and Chord in terms of the
cost of virtualization, number of fingers per host,
cost of updating data items, and overhead of
replication. We assume that keys are uniformly
+ . . . + P(eT |e1 , . . . , eT−1 )
=
T
P(ei |e1 . . . ei−1 )
i=1
where ei and ei denote that the ith key of host h is
equal and not equal to k, respectively.
Assuming that T K and k is uniformly drawn
from {1, ..., K}, we approximate P(k ∈ Th ) using
the first-order Markov process as follows. Firstly,
we consider K resource types as K balls, each
with a unique color. If we pick T balls sequentially, then the probability that the ith ball is col1
ored with k is K−i+1
. This leads to the following
equation:
⎧
0
if T = 0
⎪
⎪
⎪
⎨1
if T = 1
K
T
K
P(k ∈ Th ) = 1
1
⎪
⎪
=
if T > 1
⎪
⎩
K
−
i
+
1
i
i=1
i=K−T+1
Since Hx =
x
1
i=1
P(k ∈ Th ) =
⎧
⎨0
⎩
1
K
i
= ln x + O(1), then
if T = 0
if T = 1
if T > 1
H K − H K−T
⎧
if T = 0
⎨0
if T = 1
= K1
⎩
K
ln K−T
if T > 1
4 https://www.teragrid.org/web/user-support/compute_resources
A Read-Only Distributed Hash Table
Theorem 4.2 (Cost to Join Overlay) The cost for
a host to join R-Chord and Chord is O(|Th | log2 V)
and O ( log2 N + | Th | log N + K ln K K
),
−T
respectively.
Proof R-Chord virtualizes a host into |Th | nodes
in an overlay graph of size V. Since a node join
costs O(log2 V), the host join costs O(|Th | log2 V).
In Chord, a host join consists of a nodejoin operation, |Th | store operations to store
the key-value pairs belonging the new host, and
migrations of key-value pairs. The node-join
operation costs O(log2 N) and each store operations costs O(log N). The migration process
moves O(N/K) unique keys from an existing
node, which is the successor of the new node,
to the new node n. As there are O(N P(k ∈ Th ))
key-value pairs per unique key, the migration
K
costs O(K ln K−T
). Therefore, the host join costs
2
K
O(log N + |Th | log N + K ln K−T
) in total.
Theorem 4.2 shows that the cost to join
R-Chord is higher than Chord. This is because
R-Chord replaces a store operation with a join operation, and the cost of a join operation is higher
than a store operation.
The following theorem shows the overhead of
maintaining an overlay, i.e., stabilization cost, in
terms of the number of messages sent.
Theorem 4.3 (Cost of Stabilizations) In R-Chord,
the stabilization cost to correct all f ingers including successor pointers is O(V log N log V) messages. Since N ≤ V, the stabilization cost is also
(V log2 V) messages. On the other hand, the stabilization cost in Chord is O(N log2 N) messages.
Proof The R-Chord overlay consists of V nodes
and each node maintains O(log V) fingers; this
sums up to a total of O(V log V) fingers. Correcting the ith finger of node n is performed by locating the node that succeeds n + 2i−1 . With shared
finger tables as the only lookup optimization, it
requires O(log N) hops to correct the finger (see
Theorem 4.1). Thus, the cost of stabilization for
correcting all fingers is O(V log V log N) hops.
In Chord, the overlay consists of N nodes and
each node maintains O(log N) fingers. Correcting
517
each finger is performed by locating successor(n +
2i−1 ), which costs O(log N) hops. Thus, the cost of
stabilization is O(N log2 N) hops.
Theorem 4.3 states that a higher number of
fingers implies a higher overhead in maintaining
an overlay graph. This affects the scalability of RChord particularly when a host is virtualized into
many nodes. To reduce the overhead of periodic
stabilizations, which are employed by the current
implementation of R-Chord to correct fingers,
nodes need not correct all their fingers each time
the stabilization procedure is invoked. Instead,
each invocation corrects only a subset of a node’s
fingers, e.g. the successor pointer and another
randomly-chosen finger; this is similar to Chord’s
current implementation of periodic stabilizations.
The drawback of this approach is it increases the
number of incorrect entries in a finger table, which
increases the lookup path length. However, as
long as the successor pointer, i.e., the first finger,
is maintained, the lookup will still terminate at the
correct node.
Theorem 4.4 (Finger Flexibility) The f inger f lexK
ibility in R-Chord and Chord is O(N ln K−T
) and
O(1), respectively.
Proof Assume that successor(n + 2i−1 ) is in segment Sk . In R-Chord, the ith finger of node n can
point to any node in segment Sk . The number of
nodes in this segment is equal to to the number
of hosts that own key k, which is O(N P(k ∈ Th ))
K
hosts. Hence, the finger flexibility is O(N ln K−T
).
In Chord, the ith finger of node n must
point to successor(n + 2i−1 ), and hence, the finger
flexibility is O(1).
As mentioned in Section 3.2.3, a higher finger
flexibility increases the robustness of lookup in
the presence of node failures. Higher finger
flexibility also allows proximity-based routing to
reduce lookup latency.
Theorem 4.5 (Cost to Add Key) In Chord, adding
a key-value pair costs O(log N). In R-Chord,
adding a key-value pair whose key k already exists
in host h (i.e., k ∈ Th before the addition) costs
O(1), while adding a key-value pair whose key is
518
new for host h (i.e., k ∈
/ Th before the addition)
costs O(log2 V).
Proof In Chord, a key-value pair is stored on the
successor of the key. This costs O(log N).
In R-Chord, if k ∈ Th before the addition, then
no new node is created and hence, the cost is O(1).
However, if k ∈
/ Th before the addition, then a
new node is created and joins the R-Chord system.
This costs O(log2 V).
In applications such as P2P file sharing, sharing
a new file is equal to adding a new resource type.
However, in a computational Grid, a resource
type consists of many resource instances, and an
administrative domain can add new instances to
one of its existing resource types. Theorem 4.5
shows that using R-Chord, the administrative domain does not need to notify other nodes in the
R-Chord overlay.
Theorem 4.6 (Number of Replicas) In R-Chord,
the total number of key-value pairs with the same
K
key is O(N ln K−T
). In Chord, assuming that each
key-value pair is replicated O(log N) times, then
the total number of key-value pairs with the same
K
key is O(N ln K−T
log N).
Proof Given N hosts, the number of key-value
pairs with the same key is O(N P(k ∈ Th )). Since
R-Chord does not redistribute and replicate keyvalue pairs, the number of key-value pairs with the
K
same key is also O(N ln K−T
). In Chord, because
each key-value pair is replicated O(log N) times,
then the total number of key-value pairs with the
K
log N).
same key will be O(N ln K−T
R-Chord does not need to replicate data items
to improve the lookup resiliency in the event
of node failures. This eliminates the network
bandwidth required to replicate data items and
the complexity in maintaining consistency among
replicas.
Even when data items are not replicated, dataitem distribution can still lead to the problem of
inconsistent data items. In conventional DHTs,
updates must be propagated to the node responsible for storing the data item. This is shown in the
following corollary.
V. March, Y.M. Teo
Corollary 4.1 In Chord, the cost of a host to update
its key-value is O(log N). In R-Chord, the cost is
O(1).
Proof In Chord, the cost for a host (the originating node) to propagate an update on its key-value
pair to another node costs O(log N), according to
Theorem 2 in [65]
In R-Chord, the key-value pair is mapped to its
originating node. Hence, the cost of updating the
key-value pair is O(1).
Corollary 4.1 shows that R-Chord improves the
performance of a host in updating its data items.
In the case of computational Grids, updates occur when an administrative domain changes the
configuration of its shared resources, or changes
the number of resource instances of a resource
type.
When an administrative domain withdraws all
resources of the same type from the Grid, the
key-value corresponding to the resource type is
deleted from the overlay network. The cost of a
deletion in Chord and R-Chord is compared in
Corollary 4.2.
Corollary 4.2 In Chord, the cost of a host to delete
its key-value is O(log N). In R-Chord, the cost is
O(log V log N).
Proof In Chord, a key-value is deleted by notifying successor(k), and the cost to lookup for
successor(k) is O(log N). In R-Chord, deleting
a key-value requires a node to depart from the
overlay network. The departed node invalidates
O(log V) fingers in the system, since the node
is pointed to by O(log V) nodes and one finger
per predecessor is invalidated. Correcting each
of stale finger costs O(log N) (see Theorem 4.1.
Thus, the total cost becomes O(log V log N).
4.3 Costs Comparison
Table 3 summarizes the performance analysis of
R-Chord. Recall that N denotes the number of
hosts, T denotes the number of unique keys
owned by each host, K denotes the total number of unique keys in the system, and V = NT
A Read-Only Distributed Hash Table
519
Table 3 Comparison of Chord and R-Chord
Property
Chord
Lookup a key
Host join
O(log N)
O log2 N + |Th | log N + K ln
# unique fingers per host
O(log N)
Finger flexibility
O(1)
Stabilization
Add a key that exists
Add a new key
Update a key-value pair
O(N log2 N)
O(log N)
O(log N)
O(log
N)
K
log N
O N ln
K−T
O(log N)
# key-value pairs with the same key
Delete a key
denotes the number of nodes in an R-Chord overlay network. We show that the lookup path length
in R-Chord is shorter than Chord and at worst
equal to Chord. However, our mapping scheme
increases the cost for a host to join an overlay. In
addition, each host in R-Chord has more fingers
to correct due to the host being virtualized into
nodes. Thus, the scalability of R-Chord is determined by the number of nodes associated to
each host. When each host is virtualized into one
node only, the scalability of R-Chord is equal to
traditional Chord.
5 Simulation Analysis
In this section, we evaluate R-DHT by simulating
an R-Chord-based resource indexing and discovery scheme in a large computational Grid. As illustrated in Fig. 1b, a computational Grid consists
of many administrative domains, each of which
shares one or more compute resource types. Each
administrative domain, represented by its MDS
server, joins an R-Chord overlay and stores only
its own resource metadata. The key of each data
item is determined by the type (i.e., attributes) of
compute resource associated with the data item.
Thus, the number of unique keys owned by a
host denotes the number of unique resource types
shared by an administrative domain.
To facilitate our experiments, we implement RChord using the Chord simulator [2]. Let m = 18
bits unless stated otherwise. The network latency
R-Chord
K
K−T
O(min(log K, log N))
O(|Th | log2 V)
O(|Th | log V)
K
O N ln
K−T
O(V log V log N)
O(1)
O(log2 V)
O(1)
K
O N ln
K−T
O(log V log N)
between hosts is exponentially distributed with a
mean of 50 ms, and the time for a node to process
a request is uniformly distributed in the interval
of [5, 15] ms. Each finger is assumed to have at
most four backups. In the following subsections,
we compare the lookup path length, resiliency to
node failures, time to correct overlay, and lookup
performance on incorrect overlays.
5.1 Lookup Path Length
To verify Theorem 4.1, we measure the average lookup path length of 500,000 lookups. Each
lookup requests a randomly-selected key and is
initiated by a randomly-chosen host. Assuming
that T denotes the average number of unique
keys per host, each host has |Th | ∼ U[0.5T, 1.5T]
unique keys.
As shown in Fig. 20, the average lookup path
length in R-Chord is 20–30% lower than in Chord.
When K > N (for K = NT), R-Chord’s overlay
consists of K segments and each segment consists of one node. According to Theorem 4.1, the
lookup path length of R-Chord is affected only
by N, and hence, increasing K does not increase
the lookup path length. However, for K ≤ N, the
lookup path length increases with K.
Figure 20 also shows that in R-Chord, increasing T reduces the average path length, which can
be explained as follows. First, as each host maintains O(|Th | log NT) unique fingers an increase in
T also increases the number of fingers per hosts.
Several studies such as [33, 61] also reveal that
520
V. March, Y.M. Teo
(a) N = 10,000 Hosts
(b) N = 25,000 Hosts
Fig. 20 Average lookup path length
maintaining more fingers reduces the lookup path
length. Secondly, an increase in T also increases
the number of segments occupied by a host, and
hence, each host has a higher probability to be
visited.
Our simulation result confirms Theorem 4.1,
i.e., when K > N, the lookup path length in
R-Chord has the same upper bound as Chord.
When K ≤ N, the lookup path length in R-Chord
is shorter than Chord.
Scalable lookup is achieved by maintaining
finger tables in hosts. To understand the trade-off
between lookup performance and space requirement of finger tables, we compare the average
number of fingers in R-Chord and Chord. The
result in Table 4 confirms that in R-Chord, the
number of fingers per host is proportional to
Table 4 Average number
of fingers
K
T
5,000
1
2
4
8
1
2
4
8
NT
the number of unique keys, both per host (T) and
total (K) (Theorem 4.3). As T is increased, the
number of nodes in an R-Chord overlay increases.
Therefore, the number of fingers maintained by
the host also increases. On the other hand, in
Chord, the number of fingers per host depends on
the number of hosts (N) only, and is independent
of T and K.
Table 4 also shows that the number of backup
fingers is determined by finger flexibility. When
there are more than one nodes per segment, as
is the case with K = 5,000, backup fingers can be
exploited. When K = NT, each segment consists
of exactly one node. Therefore, no backup fingers
can be exploited. For K = 5,000, we further observe that the number of backups per finger increases as T is increased. A higher T leads to a
Chord
R-Chord
# Fingers/host
# Fingers/host
# Fingers/node
# Backups/finger
16.2
16.2
16.2
16.2
16.2
16.2
16.2
16.2
15.4
32.7
69.3
146.6
16.2
34.4
72.8
153.3
15.4
16.3
17.3
18.3
16.2
17.2
18.2
19.2
3.0
3.8
3.9
4.0
0
0
0
0
A Read-Only Distributed Hash Table
521
(a) K = NT Unique Keys
(b) K = 5,000 Unique Keys
Fig. 21 Average lookup path length with failures (N = 25,000 hosts)
higher number of nodes per segment. As such,
finger flexibility is increased, and the number of
backups per finger grows towards the predefined
maximum of four backups per finger.
5.2 Resiliency to Simultaneous Failures
To evaluate the resiliency when no churn occurs, we measure the average lookup path length
and failed lookups with the following procedures.
First, we setup a system of N = 25,000 hosts where
all hosts have T unique keys on average. Then, we
fail a fraction of hosts5 simultaneously, disable the
periodic finger correction after the simultaneous
failures, and simulate 500,000 lookup requests.
We define a lookup to have failed if it results in (i)
a false negative answer, where existing resources
cannot be located (i.e., at least one node with
the key is alive but cannot be found), or (ii) a
false positive answer where stale data items are
returned. We also assume that Chord stores a keyvalue pair only to successor(key) and does not
further replicate the key-value pair to other nodes.
5 In R-Chord, one host fail results in simultaneous node
fails.
Finally, we exploit the property of finger flexibility
in R-Chord by maintaining a maximum of four
backups per finger.
Figure 21 shows the average lookup path length
with 25% and 50% of simultaneous host failures.
When K > N (for K = NT), the average lookup
path length in R-Chord shows a trend similar
to that of in Chord, i.e., the lookup path length
increases as more hosts fail (Fig. 21a). Because
each segment consists of only one node, R-Chord
cannot exploit finger flexibility. Hence, as the
percentage of host failures increases, the number of valid fingers, which points to alive nodes,
reduces in each node’s finger table. For K ≤ N
(Fig. 21b), R-Chord has a shorter average path
length than Chord and the lookup path length is
not significantly affected by the number of failed
hosts. The reason is as follows. Firstly, R-Chord
provides O(log K)-hops lookup path length only if
each node has a correct finger table. In the case of
node failures, the finger table has a reduced number of valid fingers, which increases the lookup
path length. However, since K ≤ N implies each
segment consists of more than one node, R-Chord
effectively exploits finger flexibility to maintain
the number of valid fingers.
In terms of failed lookups, Fig. 22 shows that for
K > N and K ≤ N, R-Chord is 70 and 95% lower
522
V. March, Y.M. Teo
(a) K = NT Unique Keys
(b) K = 5,000 Unique Keys
Fig. 22 Percentage of failed lookups (N = 25,000 hosts)
than in Chord, respectively. For (K = NT) > N
(Fig. 22a), Chord has more failed lookups because
key-value pairs are stored on a different host. In
the event that the host which stores the key-value
pair fails, a lookup request to that host will be unsuccessful though the host that owns the key-value
pair is still alive. For K ≤ N (Fig. 22b), R-Chord
achieves even less failed lookups (95% lower than
Chord) because R-Chord exploits the property
that each key is available in a segment consisting
of several nodes. Hence, even if some of these
nodes fail, R-Chord can still reach the remaining
hosts in the segment through the backup fingers.
Thus, R-Chord offers better resiliency to simultaneous failures in comparison with conventional
DHTs.
5.3 Time to Correct Overlay
The correctness of an overlay network is crucial to
the lookup performance in a DHT. To ensure the
correctness of an overlay in the event of membership changes, each node periodically corrects its
fingers, i.e., periodic stabilization. However, the
larger size of the R-Chord overlay (Theorem 4.2
increases the stabilization cost (Theorem 4.3). To
amortize the maintenance overhead, periodic stabilization in R-Chord is performed less aggresively, similar to Chord, where each invocation of
the stabilization procedure corrects only a subset
of a node’s fingers, e.g. the successor pointer and
another randomly-chosen finger. However, this
may increase the time required to correct the
overlay of R-Chord.
In this experiment, we evaluate the time required to correct the overlay topology, which is
measured starting from the time when the last
host arrives. To facilitate this measurement, we
quantify the correctness of an overlay using stabilization degree (ξ ) which is derived by averaging
the correctness of all finger tables in the overlay
network (ξn ).
N−1
ξ=
n=0
N
ξn
0≤ξ ≤1
(1)
and
⎧
⎨ 0 if n. f inger[1] is incorrect
ξn = F ⎩
F
(2)
where 0 ≤ ξn ≤ 1, F is the number of correct
fingers in n, and F is total number of fingers
in n. Note that we do not consider backup fingers
in calculating ξn .
The experiment is performed as follows. We
simulate a system consisting of N hosts with
a mean arrival rate λ = 1 host/second (Poisson
A Read-Only Distributed Hash Table
523
distribution). The number of unique keys per host
is |Th | ∼ U[2, 5] unique keys and therefore, T =
3.5 unique keys. We assume that the total number
of unique keys in the system is K = 3N keys. This
K approximates NT keys where each segment
consists of one node on average. We then periodically measure ξ starting from the time when the
last host arrives.
A node joins a ring overlay through a randomly chosen existing node. We use the nodejoin process as described in [65]. First, a new
node n starts the join process by adding n =
f ind_successor(n) as its successor pointer. After
one or more rounds of finger correction, there
will be at least one other node pointing to n. At
this time, the join process completes. Note that RChord uses the f ind_successor() which incorporates shared finger tables (Fig. 19b).
Each node invokes the finger correction every
[0.5 p, 1.5 p] seconds (uniform distribution). Each
invocation of finger correction will correct the successor pointer (n. f inger[1]) and one other finger.
Correcting n. f inger[i] is done by updating it with
the node returned by f ind_successor(n + 2i−1 ).
Figure 23 reveals that at larger p (960 seconds
in this experiment), nodes in Chord correct their
successor pointer (i.e., n. f inger[1]) faster than
R-Chord. For examples, when N = 25, 000 hosts,
ξ in Chord increases from 0.36–0.59 in the first
three hours, which is faster than R-Chord (0.31–
0.38). The same behavior is also observed in N =
50,000 hosts. This is because ξn puts more priority
on the successor pointer (see (2)). Hence, by correcting successor pointers faster, Chord increases
its ξ faster than R-Chord.
In summary, although R-Chord has a larger
overlay and each of its hosts has to correct more
fingers, R-Chord does not require a longer time
than Chord to fully correct its overlay. This is due
to shared finger tables reducing the time to locate
the correct successor when correcting a finger.
(a) N = 25,000 Hosts
(b) N = 50,000 Hosts
5.4 Lookup Performance under Churn
Churn refers to membership changes in an overlay
network. Churn temporarily causes the overlay
network to be inconsistent and as such, lookup
performance is reduced until the overlay is fully
corrected through the stabilization. In R-DHT,
churn occurs in two ways. Firstly, host arrivals,
host fails, and host leaves result in simultaneous node joins, node fails, and node leaves, respectively. Secondly, adding a new unique key
to a host also causes a node join (Theorem 4.5).
When the frequency of membership changes (i.e.,
the churn rate) is high, lookup performance may
Fig. 23 Correctness of overlay ξ (measured every three hours starting from the last host arrival)
524
decrease because the larger overlay of R-DHT
magnifies the impact of churn on the correctness
of overlay topology.
To evaluate the ability of R-DHT to cope with
churn, we simulate lookups under various churn
rate. Due to the churn, the R-Chord ring overlay
keeps changing because of host arrivals, failures,
and leaves. When a host fails, its corresponding
nodes silently disappear from the overlay. On the
other hand, when a host leaves, its corresponding
nodes gracefully notify their successor and predecessor to update their fingers accordingly. In addition, a node leaving a Chord ring migrates all data
items belonging to other nodes to its successor.
In this experiment, we assume that each host
has |Th | ∼ U[4, 12] unique keys (which results
in T = 8 unique keys), and each node invokes
the finger correction procedure every [30, 90] seconds (uniform distribution). We first set up an
overlay network of 25,000 hosts, followed by a
number of churn events (i.e., arrivals, fails, and
leaves) produced by 25,000 hosts in a duration
of one hour. Thus, there will be about N =
25,000 alive hosts at any time within this duration.
During this one-hour period, we also simulate
a number of lookup events and keep the ratio
of arrive:fail:leave:lookup to 2:1:1:6. The mean
arrival rate of these events, λ, will model the
churn rate. Assuming that these events follow a
Poisson distribution, our simulation uses λ B = 10
events/second and λG = 40 events/second. These
are derived from the measurements on peer lifetime by Bhagwan et al. [9] and Gummadi et al.
[32], respectively.6 Table 5 presents the result for
6λ
B is obtained as follows. Bhagwan et al. [9] measures
that on average, each host performs 6.4 joins, and 6.4
fails per day [9]. We interpret the measured fail events
as consisting of 3.2 host fails and 3.2 host leaves. Thus,
including lookups, there are 32 events per day. With 25,000
hosts come and go repeteadly, there are 800,000 events per
day, which is approximately one event every 100 ms.
Similar steps as above are used to derive λG . Gummadi
et al. [32] measures 24 joins and 24 fails per host per day,
and we interpret the measured fails as consisting of 12 host
fails and 12 host leaves. Given 25,000 hosts and a ratio of
arrive:fail:leave:lookup = 2:1:1:6, there are approximately
one event every 25 ms.
V. March, Y.M. Teo
various K, from 5,000 (K < N) to 150,000 (K ∼
NT).
The average lookup path length (Table 5a)
again confirms Theorem 4.1 and the result from
Section 5.1. Though the number of nodes in RChord’s overlay is at least three times Chord,
when K < N the average lookup path length is
shorter than Chord since R-Chord routes lookups
by segments. When K ≥ N, the average lookup
path length is not worse than Chord due to the
shared finger tables.
Table 5b shows that lookup resiliency in RChord is comparable to Chord (from 8% lower
to 9% higher than Chord). Under a churn rate
of λ B and λG , R-Chord has a lower percentage of
failed lookups when K ≤ 100,000 and K ≤ 50,000,
respectively. The results indicate the importance
of exploiting finger flexibility through backup
fingers. In R-DHT, lookup resiliency is increased
due to the property that a key can be found
in several nodes of the same segment. Hence,
it is important that a segment can be reached
as long as it contains one or more alive nodes.
R-Chord addresses this issue by maintaining
backup fingers as redundant pointers to a segment. With a higher number of nodes per segment
(i.e., finger flexibility), backup fingers are more
effective in reducing the impact of a higher churn
rate to lookup resiliency. As K is increased, the
number of nodes per segment decreases. Because
finger flexibility is reduced, there are less redundancy to be exploited through backup fingers.
Considering that R-Chord’s overlay is eight times
larger than Chord’s overlay, we conclude that the
decrease is reasonable.
In summary, the result in this subsection suggests that R-Chord achieves better resiliency
when finger flexibility is exploited. When RChord cannot exploit finger flexibility, it can still
achieve comparable resiliency as Chord because
by not distributing data items, failure of a host
affects only its own data items.
6 Related Work
In this section, we first compare and contrast
R-DHT with structured P2P systems that support
A Read-Only Distributed Hash Table
Table 5 Lookup
performance under churn
(N ∼ 25,000 hosts)
K
525
λ B = 10 events/second
λG = 40 events/second
Chord
Chord
R-Chord
(a) Average lookup path length
5,000
8.4
4.1
7,500
8.5
4.4
10,000
8.5
4.6
15,000
8.5
4.9
25,000
8.5
5.3
50,000
8.5
5.9
75,000
8.5
6.2
100,000
8.6
6.4
125,000
8.6
6.5
150,000
8.6
6.6
% of failed lookups
5,000
2
<1
7,500
2
<1
10,000
3
1
15,000
4
1
25,000
4
1
50,000
7
5
75,000
10
7
100,000
7
5
125,000
6
7
150,000
8
16
the no-data-item-distribution scheme. Secondly,
we discuss the current status of distributed resource indexing and discovery in a computational
Grid.
6.1 Structured P2P with No-Store Scheme
We discuss three structured P2P that also support
the no-data-item-distribution scheme, namely
SkipGraph [7], Structella [13], and SkipNet [34].
SkipGraph [7] supports the no-store scheme
by associating a key to a node and organizing
nodes as a skip-list-like topology. It is assumed
that each key is shared only by one node, e.g.
resources of the same type are shared only by
one administrative domain. Our proposed scheme
generalizes SkipGraph by first, allowing a key to
be associated with several nodes. Secondly, our
scheme can be applied to structured overlay with
different underlying topologies.
Structella [13] organizes nodes as a structured
overlay (i.e., a Pastry ring [62]) but each node
manages only its own keys, similar to R-DHT.
However, unlike R-DHT, Structella does not map
keys onto nodes and does not fully exploit the
9.1
9.1
9.3
9.1
9.2
9.3
9.5
9.4
9.2
9.4
7
9
9
13
15
20
20
23
30
27
R-Chord
4.7
5.0
5.4
5.8
6.5
7.2
7.7
8.0
8.1
8.3
2
2
3
4
9
13
26
25
26
34
structured overlay to route lookup messages. Instead, it adopts the routing schemes used in unstructured P2P such as flooding and random walk.
The structured overlay network only serves to
reduce the overhead of those schemes. The authors reported that Structella offers similar results
guarantee as unstructured P2P. To improve results guarantee, the authors propose to distribute
and replicate data items to a number of nodes as
their ongoing work [14]. In contrast to Structella,
R-DHT inherits the property of DHTs which is
the strong result guarantee.
SkipNet [34] supports content locality to map a
key onto a specific node. This is achieved through
the hierarchical naming scheme: put(n|key) maps
a key to node n, and lookup(n|key) retrieves the
key. Compared to our proposed scheme, SkipNet
provides greater flexibility for a host to decide
where its data items are stored. However, the
hierarchical naming scheme does not directly supports queries such as “find resources of type k in
any hosts”. In contrast, though R-DHT addresses
node autonomy only by ensuring that data items
are stored on their originating host, it is compatible with flat naming scheme.
526
V. March, Y.M. Teo
Table 6 Comparison of R-DHT with related work
Characteristic
SkipGraph
Structella
SkipNet
R-DHT
Mapping scheme
High result guarantee
Key shared by many hosts
Controlled data placement
Flat naming scheme
Overlay network
Yes
Yes
No
No
Yes
Skip List
No
No
Yes
No
Yes
Pastry
Yes
Yes
Yes
Yes
No
Multi-level ring
Yes
Yes
Yes
No
Yes
Multiple choices
Table 6 summarizes the comparison of R-DHT
with the three no-data-item-distribution scheme.
6.2 Resource Discovery in Computational Grid
We classify distributed Grid information systems
based on their overlay topology into unstructured
overlay networks [25, 36, 46, 53] and structured
overlay networks (DHT) [10, 12, 16, 64, 69].
In unstructured overlay networks, resource information is replicated either to all nodes [46]
or a subset of nodes [25, 36, 53]. Replicating resource information to a smaller number of nodes
reduces the bandwidth overhead, but this potentially reduces the lookup accuracy. Iamnitchi et
al. [36] propose to replicate information based on
the small-world effect, Forestiero et al. [25] propose to replicate resource information based on
a probabilistic scheme, and Moreno-Vozmediano
[53] propose for each node to replicate its information to a set of neighbor nodes. To process
a lookup, a query message is forwarded to a
number of nodes based on heuristics such as
TTL (time-to-live). However, heuristics do not
guarantee that a lookup will successfully find resources. In contrast, DHT-based systems provides
stronger lookup guarantee and scalable lookup
performance.
MAAN [12], self-organizing Condor pools [10],
XenoSearch [64], RIC [69], and bitmap trees [16]
are examples of Grid information systems that
are based on conventional structured overlay networks. Compared to such schemes, our R-DHTbased Grid information system increases the
autonomy of administrative domains by not distributing data items. In addition, our scheme does
not introduce stale data items when the overlay
topology changes, and it is resilient to node failures without a need to replicate data items.
7 Conclusions
A distributed hash table maps each key onto a
node to achieve good lookup performance. A
typical DHT realizes this mapping through the
store operation and as a result, key-value pairs
are distributed across the overlay network. For
applications where distributing key-value pairs is
not desirable, we proposed R-DHT, a new DHT
mapping scheme without the store operation. RDHT enforces the read-only property by virtualizing a host into nodes subjected to the unique
keys belonging to the host, and dividing the node
identifier space into two sub-spaces, a key space
and a host identifier space. By mapping data items
back onto its owner, R-DHT is inherently fault
tolerant. In addition, it increases consistency of
data items because updates need not be propagated in overlay networks. R-DHT maintains API
compatibility with existing DHTs. In addition,
R-DHT lookup operations exploit not only existing DHT lookup schemes, but also routing by segments, shared finger tables, and finger flexibility
through backup fingers.
Through theoretical and simulation analyses,
we showed that in a Chord-based R-DHT consisting of N hosts and K total unique keys, the
lookup path length is O(min(log K, log N)) hops.
This result suggests that our proposed lookup
optimization schemes reduce lookup path length
compared with conventional DHTs. We further
demonstrate that R-DHT lookup is resilient to
node failures. In our simulations, when 50% of
nodes fail simultaneously, the number of failed
lookups in R-Chord is 5–30%. This is lower
compared to Chord where at least 60% of its
lookups fail. Our simulation also shows that under
churn, lookup performance of R-Chord is comparable to Chord even though R-Chord overlay is
A Read-Only Distributed Hash Table
eight times larger: (i) R-Chord lookup path length
is shorter, and (ii) number of failed lookups in RChord is at most 8% worst than Chord.
The host-to-nodes virtualization in R-DHT increases the size of overlay network in terms of
the number of nodes. This leads to higher overhead in maintaining an overlay network. In RChord system that virtualizes N hosts into V
nodes where V ≥ N, the maintenance overhead
is O(V log V log N). With the same number of
hosts, the maintenance overhead in Chord is
O(N log2 N). Our simulation analysis further revealed that when the stabilization period is large,
R-Chord requires more time to correct its overlay into a ring topology. In a separate study,
we show that overlay-network maintenance in RChord can be reduced using hierarchical R-DHT
topology where nodes are organized as a two-level
overlay network [50].
Though R-DHT addresses the data-ownership
issue while maintaining good lookup performance, real applications such as Grid and cloud
require multi-attribute range queries. While we
have investigated a simple multi-attribute range
query scheme based on a space-filling curve [49],
scalability of query processing remains a challenge
due to the large number of resource types and attributes. To further improve lookup performance,
we are also extending R-DHT with proximityaware routing and a selective data-item distribution scheme. Proximity-aware routing reduces
lookup latency by considering geographical distance among nodes during lookups [24, 55, 68]. Selective data-item distribution potentially increases
lookup performance via caching, increases the
availability of replicable resources, and reduces
load imbalance. This can be achieved by exploiting the segments in a R-DHT overlay network.
References
1. Apache HTTP server. http://httpd.apache.org
2. The Chord Project. http://www.pdos.lcs.mit.edu/chord
3. Globus toolkit—Information Service. http://www.
globus.org/toolkit/mds/
4. Aberer, K., Cudr-Mauroux, P., Datta, A., Despotovic,
Z., Hauswirth, M., Punceva, M., Schmidt, R.: P-Grid: a
self-organizing structured P2P system. SIGMOD Rec.
32(2), 29–33 (2003)
527
5. Adamic, L.A., Lukose, R.M., Puniyani, A.R.,
Huberman, B.A.: Local search in unstructured
networks. In: Handbook of Graphs and Networks:
from the Genome to the Internet, pp. 295–316 (2003)
6. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D.,
Katz, R. H., Konwinski, A., Lee, G., Patterson, D. A.,
Rabkin, A., Stoica, I., Zaharia, M.: Above the clouds:
a Berkeley view of cloud computing. Technical Report
UCB/EECS-2009-28, Electrical Engineering and Computer Sciences, University of California at Berkeley
(2009)
7. Aspnes, J., Shah, G.: Skip Graphs. In Proc. of the 14th
Annual ACM-SIAM Symp. on Discrete Algorithms,
pp. 384–393. ACM/SIAM Press, USA (2003)
8. Balakrishnan, H., Lakshminarayanan, K., Ratnasamy,
S., Shenker, S., Stoica, I., Walfish, M.: A layered naming architecture for the internet. In Proc. of ACM
SIGCOMM, pp. 343–352. ACM Press, Germany
(2004)
9. Bhagwan, R., Savage, S., Voelker, G. M.: Understanding availability. In Proc. of the 2nd Intl. Workshop on Peer-to-Peer Systems (IPTPS’03), pp. 256–267.
Springer-Verlag, USA (2003)
10. Butt, A. R., Zhang, R., Hu, Y. C.: A self-organizing
flock of Condors. In Proc. of the ACM/IEEE SC2003
Conf. on High Performance Networking and Computing, p. 42. ACM Press, USA (2003)
11. Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J.,
Brandic, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst.
25(6):599–616 (2009)
12. Cai, M., Frank, M., Chen, J., Szekely, P.: MAAN: a
Multi-Attribute Addressable Network for Grid information services. J. Grid Computing 2(1):3–14 (2004)
13. Castro, M., Costa, M., Rowstron, A.: Should we build
Gnutella on a structured overlay? ACM SIGCOMM
Comput. Commun. Rev. 34(2):131–136 (2004)
14. Castro, M., Costa, M., Rowstron, A.: Debunking some
myths about structured and unstructured overlays. In
Proc. of 2nd Symp. on Networked Systems Design and
Implementation (NSDI’05), pp. 85–98. USENIX Association, USA (2005)
15. Castro, M., Druschel, P., Ganesh, A., Rowstron, A.,
Wallach, D. S.: Secure routing for structured peer-topeer overlay networks. In Proc. of the 5th USENIX
Symp. on Operating Systems Design and Implementation (OSDI 2002), pp. 299–314. USENIX Association,
USA (2002)
16. Chang, R.-S., Hu, M.-S.: A resource discovery tree
using bitmap for Grids. Future Gener. Comput. Syst.
26(1):29–37 (2010)
17. Chawathe, Y., Ratnasamy, S., Breslau, L., Lanham, N.,
Shenker, S.: Making Gnutella-like P2P systems scalable. In Proc. of ACM SIGCOMM, pp. 407–418. ACM
Press, Germany (2003)
18. Cheng, A.-H., Joung, Y.-J.: Probabilistic file indexing
and searching in unstructured peer-to-peer networks.
In Proc. of the 4th IEEE Intl. Symp. on Cluster Computing and the Grid (CCGRID 2004), pp. 9–18. IEEE
Computer Society Press, USA (2004)
528
19. Cody, E., Sharman, R., Rao, R. H., Upadhyaya, S.: Security in Grid computing: a review and synthesis. Decis
Support Syst 44(4):749–764 (2008)
20. Cooper, B. F.: Quickly routing searches without having to move content. In Proc. of the 4th Intl. Workshop on Peer-to-Peer Systems (IPTPS’05), p. 163–172.
Springer-Verlag, USA (2005)
21. Crespo, A., Garcia-Molina, H.: Routing indices for
peer-to-peer systems. In Proc. of the 22nd IEEE Intl.
Conf. On Distributed Computing Systems (ICDCS
2002), pp. 23–33. IEEE Computer Society Press,
Austria (2002)
22. Dabek, F., Kaashoek, M. F., Karger, D., Morris, R.,
Stoica, I.: Wide-area cooperative storage with CFS.
In Proc. of the 11th ACM Symp. on Operating Systems Principles (SOSP’01), pp. 202–215. ACM Press,
Canada (2001)
23. Dabek, F., Zhao, B. Y., Druschel, P., Kubiatowicz, J.,
Stoica, I.: Towards a common API for structured peerto-peer overlays. In Proc. of the 2nd Intl. Workshop on
Peer-to-Peer Systems (IPTPS’03), pp. 33–44. SpringerVerlag, USA (2003)
24. Dabel, F., Cox, R., Kaashoek, F., Morris, R.: Vivaldi:
a decentralized network coordinate system. In Proc. of
ACM SIGCOMM, pp. 15–26 (2004)
25. Forestiero, A., Mastroianni, C., Spezzano, G.: QoSbased dissemination of content in Grids. Future Gener.
Comput. Syst. 24(3):235–244 (2008)
26. Foster, I., Kesselman, C. (eds.): The Grid: blueprint for
a New Computing Infrastructure. Morgan Kaufmann
Publishers (1999)
27. Foster, I., Zhao, Y., Raicu, I., Lu, S.: Cloud computing
and Grid computing 360-degree compared. In Proc. of
Grid Computing Environment Works. (GCE), pp. 1–
10. IEEE Computer Society Press, USA (2008)
28. Ghodsi, A., Alima, L. O., Haridi, S.: Symmetric replication for structured peer-to-peer systems. In Proc. of the
3rd Intl. Workshop on Databases, Information Systems
and Peer-to-Peer Computing (DBISP2P 2005), p. 12.
Spinger-Verlag, Norway (2005)
29. Godfrey, B., Lakshminarayanan, K., Surana, S., Karp,
R., Stoica, I.: Load balancing in dynamic structured
P2P systems. In Proc. of INFOCOM 2004, pp. 2253–
2262. IEEE Press, China (2004)
30. Godfrey, P. B., Stoica, I.: Heterogeneity and load balance in distributed hash tables. In Proc. of INFOCOM
2005, pp. 596–606. IEEE Press, USA (2005)
31. Gummadi, K., Gummadi, R., Gribble, S., Ratnasamy,
S., Shenker, S., Stoica, I.: The impact of DHT routing
geometry on resilience and proximity. In Proc. of ACM
SIGCOMM, pp. 381–394. ACM Press, Germany (2003)
32. Gummadi, P. K., Saroiu, S., Gribble, S.: Measurement study of Napster and Gnutella as examples of
peer-to-peer file sharing systems. Multim. Syst. Journal
9(2):170–184 (2003)
33. Gupta, A., Liskov, B., Rodrigues, R.: Efficient routing for peer-to-peer overlays. In Proc. of 1st Symp.
on Networked Systems Design and Implementation
(NSDI’04), pp. 113–126. USENIX Association, USA
(2004)
V. March, Y.M. Teo
34. Harvey, N. J. A., Jones, M. B., Saroiu, S., Theimer,
M., Wolman, A.: SkipNet: a scalable overlay network
with practical locality properties. In Proc. of the 4th
USENIX Symp. on Internet Technologies and Systems
(USITS’03), pp. 113–126. USENIX Association, USA
(2003)
35. Hsiao, H.-C., King, C.-T.: A tree model for structured
peer-to-peer protocols. In Proc. of the 3rd IEEE Intl.
Symp. on Cluster Computing and the Grid (CCGRID
2003), pp. 336–343. IEEE Computer Society Press,
Japan (2003)
36. Iamnitchi, A., Ripeanu, M., Foster, I.: Locating data
in (small-world?) peer-to-peer scientific collaborations.
In Proc. of the 1st Intl. Works. on Peer-to-Peer Systems
(IPTPS’02), pp. 232–241. Springer-Verlag, USA (2002)
37. Karger, D. R., Ruhl, M.: Simple, efficient load balancing algorithms for peer-to-peer systems. In Proc.
of the 3rd Intl. Workshop on Peer-to-Peer Systems
(IPTPS’04), pp. 131–140. Springer-Verlag, USA (2004)
38. Kashani, F. B., Shahabi, C.: Criticality-based analysis
and design of unstructured peer-to-peer networks as
“complex systems”. In Proc. of the 3rd IEEE Intl.
Symp. on Cluster Computing and the Grid (CCGRID
2003), pp. 351–358. IEEE Computer Society Press,
Japan (2003)
39. Kaufman, L. M.: Data security in the world of cloud
computing. IEEE Secur. Priv. 7(4):61–64 (2009)
40. Kienle, H. M., German, D., Tilley, S., Muüller, H. A.:
Intellectual property aspects of web publishing. In
Proc. of the 22nd Intl. Conf. on Design of Communication: the Engineering of Quality Documentation
(SIGDOC), pp. 136–194. ACM Press, USA (2004)
41. Krauter, K., Buyya, R., Maheswaran, M.: A taxonomy
and survey of Grid resource management systems for
distributed computing. Intl. Journal of Softw., Practic.
and Exper. 32(2):135–164 (2002)
42. Kubiatowicz, J., Bindel, D., Chen, Y., Eaton, P.,
Geels, D., Gummadi, R., Rhea, S., Weatherspoon,
H., Weimer, W., Wells, C., Zhao, B.: OceanStore:
an architecture for global-scale persistent storage. In
Proc. of the 9th Intl. Conf. on Architectural Support
for Programming Languages and Operating Systems
(ASPLOS-IX), pp. 190–201. ACM Press, USA (2000)
43. Landers, M., Zhang, H., Tan, K.-L.: Peerstore: better
performance by relaxing in peer-to-peer backup. In
Proc. of the 4th Intl. Conf. on Peer-to-Peer Computing
(P2P 2004), pp. 72–79. IEEE Computer Society Press,
Switzerland (2004)
44. Leslie, M., Davies, J., Huffman, T.: Replication strategies for reliable decentralised storage. In Proc. of the
1st Workshop on Dependable and Sustainable Peerto-Peer Systems (DAS-P2P 2006), pp. 740–747. IEEE
Computer Society Press, Japan (2006)
45. Li, J., Stribling, J., Gil, T. M., Morris, R., Kaashoek,
M. F.: Comparing the performance of distributed hash
tables under churn. In Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS’04), pp. 87–99.
Springer-Verlag, USA (2004)
46. Li, W., Xu, Z., Dong, F., Zhang, J.: Grid resource discovery based on a routing-transferring model. In Proc.
A Read-Only Distributed Hash Table
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
of the 3rd Intl. Workshop on Grid Computing (GRID
2002), pp. 145–156. Springer-Verlag, USA (2002)
Loo, B. T., Huebsch, R., Stoica, I., Hellerstein, J. M.:
The case for a hybrid P2P search infrastructure. In
Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS’04), pp. 141–150. Springer-Verlag, USA
(2004)
Lv, Q., Cao, P., Cohen, E., Li, K., Shenker, S.: Search
and replication in unstructured peer-to-peer networks.
In Proc. of the 2002 Intl. Conf. on Supercomputing
(ICS 2002), pp. 84–95. ACM Press, USA (2002)
March, V., Teo, Y. M.: Multi-attribute range queries
on read-only DHT. In Proc. of the 15th Intl. Conf.
on Computer Communications and Networks (ICCCN
2006), pp. 419–424. IEEE Communications Society
Press, USA (2006)
March, V., Teo, Y. M., Lim, H. B., Eriksson, P., Ayani,
R.: Collision detection and resolution in hierarchical
peer-to-peer systems. In Proc. of the 30th IEEE Conf.
on Local Computer Networks (LCN 2005), pp. 2–9.
IEEE Computer Society Press, Australia (2005)
March, V., Teo, Y. M., Wang, X.: DGRID: a DHTbased Grid resource indexing and discovery scheme for
computational Grids. In Proc. of the 5th Australasian
Symp. on Grid computing and e-Research (AusGrid
2007). Australian Computer Society Inc, Australia
(2007)
Maymounkov, P., Mazières, D.: Kademlia: a peer-topeer information system based on the XOR metric.
In Proc. of the 1st Intl. Workshop on Peer-to-Peer
Systems (IPTPS’02), pp. 53–65. Springer-Verlag, USA
(2002)
Moreno-Vozmediano, R.: A hybrid mechanism for
resource/service discovery in ad-hoc Grids. Future
Gener. Comput. Syst. 25(7):717–727 (2009)
Németh, Z., Sunderam, V.: Characterizing Grids: attributes, definitions, and formalisms. J. Grid Computing 1(1):9–23 (2003)
Ng, T. S. E., Zhang, H.: Predicting internet network
distance with coordinates-based approaches. In Proc.
of the 21st Annual Joint Conf. of the IEEE Computer and Communications Societies (IEEE Infocom),
pp. 170–179 (2002)
Nielson, S. J., Crosby, S. A., Wallach, D. S.: A taxonomy of rational attacks. In Proc. of the 4th Intl. Workshop on Peer-to-Peer Systems (IPTPS’05), pp. 36–46.
Springer-Verlag, USA (2005)
Ramaswamy, L., Gedik, B., Liu, L.: A distributed approach to node clustering in decentralized peer-topeer networks. IEEE Trans. Parallel Distrib. Syst.
16(9):814–829 (2005)
Ratnasamy, S., Francis, P., Handley, M., Karp, R.,
Shenker, S.: A scalable Content-Addressable Network.
529
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
In Proc. of ACM SIGCOMM, pp. 161–172. ACM
Press, USA (2001)
Ratnasamy, S., Stoica, I., Shenker, S.: Routing algorithms for DHTs: some open questions. In Proc.
of the 1st Intl. Workshop on Peer-to-Peer Systems (IPTPS’02), pp. 45–52. Springer-Verlag, USA
(2002)
Rhea, S., Godfrey, B., Karp, B., Kubiatowicz, J.,
Ratnasamy, S., Shenker, S., Stoica, I., Yu, H.:
OpenDHT: a public DHT service and its uses. In Proc.
of ACM SIGCOMM, pp. 73–84. ACM Press, USA
(2005)
Rodrigues, R., Blake, C.: When multi-hop peer-topeer lookup matters. In Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS’04), pp. 112–122.
Springer-Verlag, USA (2004)
Rowstron, A., Druschel, P.: Pastry: scalable, distributed object location and routing for large-scale peerto-peer systems. In Proc. of IFIP/ACM Intl. Conf.
on Distributed Systems Platforms (Middleware 2001),
pp. 329–350. Springer-Verlag, Germany (2001)
Shneidman, J., Parkes, D. C.: Rationality and selfinterest in peer to peer networks. In Proc. of the 2nd
Intl. Workshop on Peer-to-Peer Systems (IPTPS’03),
pp. 139–148. Springer-Verlag, USA (2003)
Spence, D., Harris, T.: XenoSearch: distributed
resource discovery in the XenoServer open platform.
In Proc. of the 12th IEEE International Symp. on High
Performance Distributed Computing (HPDC-12),
pp. 216–225. IEEE Computer Society Press, USA
(2003)
Stoica, I., Morris, R., Karger, D., Kaashoek, M. F.,
Balakrishnan, H.: Chord: a scalable peer-to-peer
lookup service for internet applications. In Proc. of
ACM SIGCOMM, pp. 149–160. ACM Press, USA
(2001)
Wohlmacher, P., Pharow, P.: Applications in health
care using public-key certificates and attribute certificates. In Proc. of the 16th Annual Computer Security Applications Conf. (ACSAC), pp. 128–137. IEEE
Computer Society Press, USA (2000)
Yang, B., Garcia-Molina, H.: Designing a super-peer
network. In Proc. of the 19th Intl. Conf. on Data Engineering (ICDE 2003), pp. 49–61. IEEE Computer
Society Press, India (2003)
Zeinalipour-Yazti, D., Kalogeraki, V.: Structuring
topologically aware overlay networks using domain
names. Computer Networks: Int. J. Computing and
Telecomm. 50(16):3064–3082 (2006)
Zhu, C., Liu, Z., Zhang, W., Xiao, W., Xu, Z., Yang,
D.: Decentralized Grid resource discovery based on
resource information community. J. Grid Computing
2(3):261–277 (2004)
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement