J Grid Computing (2011) 9:501–529 DOI 10.1007/s10723-011-9189-9 A Read-Only Distributed Hash Table Verdi March · Yong Meng Teo Received: 22 June 2010 / Accepted: 15 March 2011 / Published online: 27 April 2011 © Springer Science+Business Media B.V. 2011 Abstract A distributed hash table (DHT) is an infrastructure to support resource discovery in large distributed systems. In a DHT, data items such as resources, indexes of resources or resource metadata, are distributed across an overlay network based on a hash function. However, this may not be desirable in commercial applications such as Grid and cloud computing whereby the presence of multiple administrative domains leads to the issues of data ownership and self-economic interests. In this paper, we present R-DHT (Read-only DHT), a DHT-based resource discovery scheme without distributing data items. To map each data item back onto its resource owner, a physical host, we virtualize each host into virtual nodes. Nodes are further organized as a segment-based overlay network which increases node failure resiliency without replicating data items. We demonstrate the feasibility of our proposed scheme by presenting R-Chord, an implementation of R-DHT using Chord as the underlying overlay graph, with V. March (B) · Y. M. Teo Department of Computer Science, National University of Singapore, Computing 1, 13 Computing Drive, Singapore 117417, Singapore e-mail: [email protected], [email protected] Y. M. Teo e-mail: [email protected] lookup and maintenance optimizations. Through analytical and simulation analyses, we evaluate the performance of R-DHT and compare it with traditional DHTs in terms of lookup path length, resiliency to node failures, and maintenance overhead. Overall, we found that R-DHT is effective and efficient for resource indexing and discovery in large distributed systems with a strong commercial requirement. Keywords Grid · Resource indexing and discovery · Administrative domain · Overlay network 1 Introduction Distributed hash tables (DHT) [31, 35, 45, 59] are a decentralized lookup scheme that aims to provide scalable lookups with high result guarantee for large distributed systems consisting of multiple peer nodes. A DHT, as with a hash-table data structure, provides an interface to retrieve a keyvalue pair. A key is an identifier assigned to a resource; traditionally this key is a hash value associated with the resource. A value is an object to be stored into the DHT; this could be the shared resource itself (e.g. a file), an index (pointer) to a resource, or a resource metadata. An example of a key-value pair is SHA1(file_name), http://peer-id/file, where the key is the 502 V. March, Y.M. Teo SHA1 hash of the file name and the value is the address (location) of the file. A DHT achieves its design objectives including lookup scalability and result guarantee through: 1. Key-to-Node Mapping—Assuming that keys and nodes share the same identifier space, a DHT maps key k to node n where n is the node closest to k in the identifier space; we refer to n as the responsible node of k. Compared to other decentralized lookup schemes such as [5, 17, 18, 20, 21, 38, 48, 57, 67], the key-to-node mapping is perceived to increase result guarantee (i.e., a lower number of false negative answers) because determining the existence of a key-value pair equals to locating the node responsible for the key [47]. 2. Data-Item Distribution—All key-value pairs (data items) whose key is equal to k are then stored at node n regardless of who owns these key-value pairs. This is implemented in existing DHTs by providing a store operation [23, 60]. The concept of data-item distribution has been further exploited for various optimizations, including load balancing [29, 30, 37] and high availability [22, 28, 42–44]. 3. Structured Overlay Network—Nodes are organized as a structured overlay network which strikes a balance between the routing performance and the overhead of maintaining routing states. There are two important characteristics of the structured overlay network: a. Topology A structured overlay network resembles a graph with a certain topology such as a ring [62, 65], a torus [58], or a tree [4, 52]. b. Ordering of nodes The position of a node in a structured overlay network is determined by the node identifier. By exploiting the structured overlay network, a DHT can locate any responsible node in a short and bounded number of hops. For example, the lookup path length in the Chord ring is O(log N) [65]. Though DHTs have been shown to exhibit good lookup performance, the assumption on distributing data items hinders the widespread com- mercial adoption of DHT. In large distributed systems such as Grid computing [26, 41] and cloud computing [6, 11], nodes belonging to different companies come under different administrative domains. In such applications, distributing data items among different administrative domains leads to two major issues: 1. Regulatory Compliance—Distributing data items works against the current practice of controlling data location in order to enforce copyright and to meet regulatory requirements [40, 66]. The emergence of cloud computing further exacerbates the importance of data-item ownership and location [19, 27, 39]. In view of this, it may not be commercially viable for an organization to store its data items, including pointers or indexes to resources, on nodes belonging to other organizations. 2. Conflicting Self-Interests among Administrative Domains—Data-item distribution requires all nodes in a DHT overlay to be publicly writable. However, this may not happen when nodes do not permit the sharing of their storage resources to external parties due to different economic interests. Firstly, nodes want to protect their investment in storage infrastructure by not storing data items belonging to other nodes. Secondly, an individual node may limit the amount of storage it offers; this reduces the result guarantee when the total amount of storage in the DHT is smaller than the total number of key-value pairs. In addition to the problem in enforcing storage policies, nodes also face a challenge when their infrastructure is used by customers of other parties [56, 63]. For example, when a node stores data items belonging to other parties, its computing and network performance may degrade as a result of participating in the lookup requests belonging to other parties. These two issues can be addressed by not distributing data items. However, by design, a DHT assumes that data items are distributed across overlay networks. In this paper, we propose a resource discovery scheme based on a read-only DHT (R-DHT) and, as an example, apply it to support decentralized resource indexing and discovery in large A Read-Only Distributed Hash Table computational Grids [26, 41]. R-DHT is a scheme that enables DHTs to map keys to nodes without distributing data items [51]. By organizing the administrative domains in a Grid as an RDHT overlay, each of these domains can control their own resource discovery information but still allowing the other domains to read it. R-DHT reuses the functionalities in existing structured overlay networks such as Chord [65] to leverage on existing extensive research in this field. The read-only mapping scheme in R-DHT virtualizes a host (i.e., a physical entity that shares resources) into nodes: one node is associated with each unique key belonging to the host. Nodes are then organized as a segment-based structured overlay network and the node identifier space is divided into two sub-spaces: key space and host identifier space. Our scheme inherits the good properties of DHTs, namely support for decentralized lookups to minimize single point of failure, high result guarantee, and bounded lookup path length. In addition, R-DHT is (i) by design resilient to node failures, (ii) supports the “pay-for-your-own” usage model whereby each node stores only its own key-value pairs, and without depending on thirdparty publicly-writable nodes [8], and (iii) avoids inconsistency of data updates as changes to a keyvalue pair are reflected immediately by the node that owns it. Our main contributions are as follow: 1. Design of R-DHT—R-DHT (read-only DHT) is a DHT-based resource discovery scheme whereby data items are not distributed across the overlay network. R-DHT reuses the existing functionalities from conventional DHTs and as such, improvements in a particular DHT are beneficial to R-DHT as well. To demonstrate this, we present R-Chord, an implementation of the R-DHT scheme that uses Chord as the underlying overlay graph. RChord uses Chord’s join algorithm to construct its overlay network. In addition, the algorithms for lookup and maintenance of the overlay (i.e., stabilization) are similar to Chord. 2. Lookup Optimizations—We show that the lookup path length in R-DHT is at worst equal to the corresponding DHT. The segment- 503 based overlay in R-DHT allows messages to be routed by segments and finger tables to be shared among nodes. When the number of unique keys (K) is larger than the number of hosts (N), e.g. in file-sharing P2P systems, the lookup path length in R-DHT is bounded by O(log N) as in traditional DHTs such as Chord. However, when K ≤ N, e.g. in computational Grids, the O(log K)-hops lookup path length of R-DHT is shorter than traditional DHTs. 3. Lookup Resiliency to Node Failures—In the event of node failures, we show that R-DHT’s segment-based overlay reduces the number of failed lookups, i.e., lookups that return a false negative or false positive answer. In RDHT, lookups for key-value pairs shared by many nodes are more likely to succeed even without replicating key-value pairs. When one of the nodes fails, only its own key-value pairs become unavailable. The remaining key-value pairs in other nodes can still be discovered because R-DHT exploits a segment-based overlay by using backup fingers. Thus, there is a higher probability of finding resources of a certain type when many nodes share resources of the same type. The remainder of this paper is organized as follows. Section 2 presents an overview of Chord. In Section 3, we discuss the design of R-DHT and its Chord-based implementations, namely R-Chord. Theoretical and simulation analyses of the performance of R-DHT are presented in Sections 4 and 5, respectively. Section 6 discusses related work. Finally, Section 7 concludes this article. 2 Background 2.1 Distributed Grid Resource Indexing and Discovery Throughout this paper, we use distributed resource discovery in a computational Grid as the example of R-DHT applications. A computational Grid facilitates the sharing of compute resources from different administrative domains [26, 41]. Typically, Grid users search for specific resources 504 V. March, Y.M. Teo where their job will be executed [54]. In a centralized scheme, an MDS (Monitoring and Discovery System) [3] indexes all resources from all administrative domains and processes all user queries (Fig. 1a). As the adoption of Grid technologies increases, the central MDS becomes a potential bottleneck and a single point of failure. Recently, there is a growing interest in studying the use of DHT-based resource discovery for large computational Grids [10, 12, 64, 69]. Instead of depending on a third-party central MDS, DHTbased schemes distribute indexes and queries across administrative domains organized as nodes in an overlay network. As illustrated in Fig. 1b, each domain still deploys an MDS (or any other schemes) for the intra-domain discovery. Then, R-DHT is applied for the inter-domain resource indexing and discovery. This enables a computational Grid to support scalable distributed resource discovery with high result guarantee, while allowing each administrative domain to index only its own resources. 2.2 Overview of Chord Chord is a DHT implementation that supports O(log N)-hop lookup path length and O(log N) routing states per node, where N denotes the total number of nodes [65] . Chord organizes nodes as a ring that represents an m-bit one-dimensional circular identifier space, and as a consequence, all arithmetic operations are modulo 2m . To form a (a) Centralized Fig. 1 Resource discovery in a computational Grid ring overlay, each node n maintains two pointers to its immediate neighbors (Fig. 2a). The successor pointer points to successor(n), i.e., the immediate neighbor of n clockwise. Similarly, the predecessor pointer points to predecessor(n), the immediate neighbor of n counter clockwise. Chord maps key k to successor(k), the first node whose identifier is equal to or greater than k in the identifier space (Fig. 2b). Thus, node n is responsible for keys in the range of ( predecessor(n), n], i.e., keys that are greater than predecessor(n) but smaller than or equal to n. For example, node 32 in the figure is responsible for all keys in (21, 32]. All key-value pairs whose key equals k are then stored on successor(k) regardless of who owns the key-value pairs; this is the dataitem distribution. Finding key k implies that we route a request to successor(k). The simplest approach for this operation, as illustrated in Fig. 2c, is to propagate a request along the Chord ring in a clockwise direction until the request arrives at successor(k). However, this approach is not scalable as its complexity is O(N), where N denotes the number of nodes in the ring [65]. To speed-up the process of finding successor(k), each node n maintains a finger table of m entries (Fig. 2d). Each entry in the finger table is also called a finger. The ith finger of n is denoted as n. f inger[i] and points to successor(n + 2i−1 ), where 1 ≤ i ≤ m. Note that the 1st finger is also the successor pointer while the largest finger divides the circular identifier space into two halves. (b) Decentralized A Read-Only Distributed Hash Table (a) Chord Ring 505 (b) Map and Distribute Keys to Nodes (d) The Fingers of Node 8 (c) Traverse the Ring to Find successor (54) (e) f i n d _successor (54) Utilizing Finger Tables Fig. 2 Chord lookup When N < 2m , the finger table consists of only O(log N) unique entries [65]. By utilizing finger tables, Chord locates successor(k) in O(log N) hops with high probability [65]. Intuitively, the process resembles a binary search where each step halves the distance to successor(k). Each node n forwards a (a) Insert Node 25 Fig. 3 Join operation in Chord request to the nearest known preceding node of k. This is repeated until the request arrives at predecessor(k), the node whose identifier precedes k, which will forward the request to successor(k). Figure 2e shows an example of finding successor(54) initiated by node 8. Node 8 forwards the request to its 6th finger which (b) Migrate Key 22 from Node 32 (c) Correct Successor and Predecessor Pointers 506 V. March, Y.M. Teo (b) virtualize : hosts → nodes (a) Host in the Context of Computational Grid Fig. 4 Hosts and nodes points to node 48. Node 48 is the predecessor of key 54 because its 1st finger points to node 56 and 48 < 54 ≤ 56. Thus, node 48 will forward the request to node 56. Figure 3 illustrates the construction of a Chord ring. A new node n joins a Chord ring by locating its own successor. Then, n inserts itself between successor(n) and the predecessor of successor(n), illustrated in Fig. 3a. The key-value pairs stored on successor(n), whose key is less than or equal to n, are migrated to node n (Fig. 3b). Because the join operation invalidates the ring overlay, every node performs periodic stabilizations to correct its successor and predecessor pointers (Fig. 3c), and its fingers. 3 R-DHT Design and Implementation We start by introducing the terms resource type, host and node. Definition 3.1 (Resource Type) A resource type is a list of attribute names that describes a resource. The resource type determines the key assigned to a resource. There could be many resource instances with the same type; these resources are assigned the same key. Table 1 Variables maintained by host and node Definition 3.2 (Host) A host refers to a physical entity that shares resources. Let Th denotes the set of unique keys (i.e., resource types) owned by a host whose host identifier is h. Figure 4a illustrates how the above terminologies are applied in a computational Grid. In this example, a host refers to the MDS server. The two keys, T3 = {2, 9}, denote that host 3, representing administrative domain 3, indexes two types of resources, resource types 2 and 9. Details on assigning a key to a resource is discussed in [49]. Definition 3.3 (Node) A node refers to a logical entity in an overlay network. In terms of set theory, a multiple-valued function, virtualize : hosts → nodes, describes the relationship between hosts and nodes (Fig. 4b). A host joins an overlay network as one or more nodes, i.e., by assuming one or more identities in the overlay. Clearly, each node corresponds to only one host. The notion of “nodes” is equivalent to “virtual servers” in Cooperative File System [22] or “virtual hosts” in Apache HTTP Server [1]. Table 1 shows some of the variables maintained by each host and Chord node. In addition to its node identifier (n), each node maintains its states Entity Variable Description Host h Node n Th f inger[1 . . . F] successor predecessor a set of unique keys owned by host h a finger table of F entries the next node in the ring overlay, i.e., f inger[1] the previous node in the ring overlay A Read-Only Distributed Hash Table in an overlay topology including f inger, successor, and predecessor. In presenting pseudocode, we adopt the notations from [65]: 1. Let h denotes a host or its identifier, and n denotes a node or its identifier, as their meaning will be clear from the context. 2. Remote procedure calls or variables are preceded by the remote node identifier, while local procedure calls and variables omit the local node identifier. In the following, we describe read-only mapping, the main concept in R-DHT. Then, we discuss the construction of R-DHT overlays and the lookup algorithm using Chord [65] as the underlying overlay graph. Subsequently, we use “R-DHT” to refer to read-only DHT in general, and “R-Chord” to refer to a Chord-based R-DHT. 3.1 Read-Only Mapping R-DHT is a read-only DHT where a node supports “read-only” accesses to its keys, and does not allow keys belonging to other nodes to be written (mapped) on it. The read-only property ensures that keys are mapped onto the node that owns and shares the resource. This node is called the originating node. With its read-only property, R-DHT addresses issues such as node autonomy in placing key-value pairs, prevents stale data items, and increases lookup resiliency to node failures without the need to replicate keys. As shown in Fig. 5, R-DHT achieves the readonly mapping by virtualizing a host into a number of nodes where each node represents a unique key shared by the host. The node identifier space is divided into two sub-spaces: a key space and 507 a host identifier space. This ensures the uniqueness of node identifiers without compromising R-DHT’s support for a flat naming scheme [8]. Shared resources of the same type are identified by the same key and form a segment on the overlay graph. We exploit segment-based routing to reduce lookup path length and improves lookup resiliency to node failures without a need to replicate data items. The basic idea of read-only mapping is to exploit the property of the DHT mapping whereby a key can be mapped onto a specific node if the identifier of the node is equal to the key (Fig. 6). Thus, each node in R-DHT is a bucket with one unique key, as opposed to conventional DHT where each node is a bucket with a number of unique keys. R-DHT realizes the key-to-node mapping through virtualization and splitting of node identifiers. Virtualization ensures that each host can share different keys, whereas the splitting of node identifiers prevents node collisions when several hosts share the same key. R-DHT virtualizes each host into a number of nodes by associating node n to each unique key k belonging to host h. Figure 7a shows an example of virtualizing two hosts into four nodes, where each host with two unique keys is virtualized into two nodes. By making the node identifier equals to its associated key, R-DHT ensures that keys are not distributed. However, when nodes and keys share the same identifier space, virtualizing several hosts sharing the same key results in collisions of node identifiers (Fig. 7b). To avoid the collision of node identifiers, a node associated to key k shared by host h is assigned k|h as its identifier. Each node can be uniquely identified by its node identifier which is the concatenation of the key (k) and the host identifier1 (h). Figure 8a shows an example of node identifier where the key and the host identifier are of the same bit-length2 , i.e., (m/2) bits. The m-bit node identifier space is divided into 2m/2 segments. Each segment Sk consists of 2m/2 consecutive node 1 A host identifier can be derived by hashing the host’s IP address or an identifier obtained from a certification authority [15]. Fig. 5 Proposed R-DHT scheme 2 In general, the key space and the host identifier space need not be of the same size. 508 V. March, Y.M. Teo (a) Conventional DHT Maps Key to Node Identifier Closest to the Key (b) R-DHT Maps Key to Node Identifier Equal to the Key Fig. 6 Mapping keys to node identifiers Fig. 7 Virtualization in R-DHT (a) Virtualize Two Hosts into Four Nodes (a) m-bit Node Identifier (b) Collision of Node Identifiers (b) Segmenting Node-Identifier Space Fig. 8 R-DHT node identifiers Table 2 Comparison of API in R-DHT with conventional DHT Operation Host h joins overlay through existing host e Host h shares new key k Users at host h search for key k API DHT R-DHT h. join(e) h.store(k) h.lookup(k) h.virtualize(e) h.newKey(k) h.lookup(k) A Read-Only Distributed Hash Table identifiers prefixed with k (Fig. 8b). Therefore, each segment represents resources of the same type that are offered for sharing by different hosts. Segment indexing reduces lookup path length and improves fault-tolerance. R-DHT is designed to maintain API compatibility with conventional DHTs and supports the flat naming scheme [8]. As shown in Table 2, the R-DHT API requires the same arguments as its DHT counterparts. The lookup(k) API of R-DHT supports a flat naming scheme by abstracting the location of keys in its argument. With the flat naming scheme, queries are formulated as “find resource type k”. This allows R-DHT to fully provide a hashtable abstraction where only the key is required to retrieve key-value pairs. However, systems that support only a hierarchical naming scheme do not fully provide a hashtable abstraction of locating key-value pairs, since the hierarchical naming scheme requires users to specify the location of the key as an argument in the lookup operation, in addition to the key itself. Thus, queries are formulated as “find resource type k from host h”, which are reminiscent of HTTP requests such as “retrieve index.html from www.comp.nus.edu.sg”. R-DHT can be implemented as a one-level (i.e., flat) overlay network or a two-level (i.e., hierarchical) overlay network. In this paper, we discuss only flat R-DHT: construction of overlay, lookup algorithm, and maintenance of overlay (Section 3.2). Hierarchical R-DHT can be implemented based on a hierarchical scheme for conventional DHTs such as March et al. [50]. 509 Fig. 9 Virtualizing host into nodes segments where segment Sk consists of nodes prefixed with k. Thus, each segment Sk represents resources of the same type shared by different hosts. Figure 10 compares Chord and R-Chord in a Grid consisting of three administrative domains, assuming that keys and host identifiers are 4-bit long. In Chord (Fig. 10a), each host becomes a Chord node and its data items, keys and indexes of its compute resources, are distributed to another 3.2 R-Chord (a) Chord Distributes Data Items among Three Nodes R-Chord is a flat R-DHT that organizes nodes as a (one-level) Chord overlay. Figure 9 shows how a new host joins R-Chord, and how an existing host shares a new key. Each new node joins the ring overlay using Chord’s join protocol (lines 5 and 13 in Fig. 9). Nodes are organized as a logical ring in clock-wise ascending order where each node identifier (k|h) is interpreted as an integer. Let N N−1 Th | denote the number of hosts, and K = | h=0 denote the total number of unique keys in the system. R-Chord divides its ring overlay into K (b) R-Chord Virtualizes Three Hosts into Six Nodes Fig. 10 Chord and R-Chord 510 V. March, Y.M. Teo node whose identifier immediately succeeds the key. For example, all keys with identifier 2 will be stored on node 3. In R-Chord (Fig. 10b), each key is mapped to its originating node. In this example, host 3 owns two unique keys, i.e., T3 = {2, 9}, and thus, we virtualize host 3 into two nodes with node identifiers 2|3 = 35 and 9|3 = 147. Similarly, we virtualize host 6 and host 9 into one node and three nodes, respectively. These six nodes are organized as an R-Chord ring based on their integral node identifier. The R-Chord ring consists of three segments, namely segment S2 with node 2|3 and node 2|9, segment S5 with node 5|9 and 5|6, and segment S9 with node 9|3 and node 9|9. These three segments represent three keys or resource types, namely key 2, key 5, and key 9. R-DHT prevents stale data items when updated by their originating node, while conventional DHT must route the update to the node where the data item is distributed. The data item becomes stale before the update reaches the node. In Fig. 10, when host 3 updates its key 9, Chord routes the update to node 9 which stores the key. On the other hand, in R-Chord the update is (a) R-Chord reflected immediately because key 9 is mapped onto node 9|3 which is associated with the resource owner, host 3. By design, R-DHT is inherently fault tolerant without a need to incorporate data-item replications as a resiliency mechanism. Firstly, a node failure does not affect keys belonging to other nodes. Secondly, lookup(k) can still be successful since R-DHT can route the lookup request to other alive nodes in segment Sk . Figure 11 shows an example when administrative domain 3 with two shared resource types failed. Since administrative domain 9 is still alive, R-Chord can still locate resource type 2 by routing a lookup(2) request to node 2|9 (Fig. 11a). On the contrary, lookup fails in Chord because it is routed to node 3 (Fig. 11b). Typically, a conventional DHT replicates keys to improve its resiliency. However, this further increases the risk of stale data items, i.e., data items pointing to unavailable resources in the inaccessible administrative domain 3 (Fig. 11c). R-DHT supports a flat naming scheme where users need to specify only key k when searching. R-DHT bases its lookup on the underlying over- (b) Chord without Replication (c) Replication Introduces Stale Data Items Fig. 11 Node failures and stale data items A Read-Only Distributed Hash Table 511 a node n where pref ix(n ) = k, then key k is successfully found (line 11); otherwise, the key does not exist (line 14). The direct application of Chord’s lookup is not efficient because it does not exploit the advantages of read-only mapping. In a system N−1 Fig. 12 The fingers of node 2|3 lay’s lookup protocol. With Chord as the underlying overlay, each node maintains at most m unique fingers (Fig. 12). Lookup for key k implies locating the successor of k|0, i.e., the first node in segment Sk . Figure 13 shows a direct application of Chord lookup algorithm on R-Chord. If a lookup returns |T | with N hosts where each host has T = h=0N h unique keys on average, R-Chord consists of N · T nodes and hence, its lookup path length is O(log NT) hops (based on Theorem 2 in [65]). However, we reduce R-Chord lookup path length to O(min(log K, log N)) hops (Theorem 4.1) by exploiting our read-only mapping scheme using two optimizations, namely routing by segments and shared routing tables. The complete algorithm is shown in Fig. 14. 3.2.1 Routing by Segments (a) Main Algorithm (b) Helper Functions Fig. 13 Unoptimized R-Chord lookup R-DHT divides its node-identifier space into segments, with each segment representing a key. Therefore, locating key k equals to locating any node within segment Sk , instead of strictly locating successor(k|0) only. R-Chord exploits this segmentation by forwarding a lookup(k) request from one segment to another segment (lines 4, 7, and 16 in Fig. 14a, and line 6 in Fig. 14b). Each routing step halves the distance, in term of segments, to the destination segment Sk (see Lemma 4.1 for the proof). As such, in a system with K segments, routing by segments reduces lookup path length to O(log K). Since segments are identified by the prefix of node identifiers, our routing-by-segments scheme is essentially a prefix-based routing optimization applied for the Chord lookup protocol. Figure 15 illustrates the processing of a lookup(k) request initiated by node n1 . Without routing by segments, i.e., the direct application of Chord’s lookup protocol, the lookup path is n1 → n2 → n3 → n4 → n5 because we always locate node n5 = successor(k|0). However, with routing by segments, we realize that one of the intermediate hops, node n2 , has a finger pointing to n6 . Though node n6 is not successor(k|0), it also shares key k and is in segment Sk . Since the lookup can then be completed at node n6 , the optimized lookup path becomes n1 → n2 → n6 . 512 V. March, Y.M. Teo Fig. 14 R-Chord lookup exploiting R-DHT mapping (a) Main Algorithm (b) Helper Functions 3.2.2 Shared Finger Tables To limit the lookup path length at O(log N) hops even when K > N, our routing algorithm utilizes all the |Th | finger tables maintained by each host h (lines 3 and 11 in Fig. 14a, and lines 3 and 14 in Fig. 14b). In other words, a node’s finger table is shared by all nodes from the same host. As such, visiting one host is equivalent to visiting all the |Td | nodes which correspond to the host A Read-Only Distributed Hash Table 513 (a) No Routing by Segments (b) Routing by Segments Fig. 15 lookup(k) with and without routing by segments (Fig. 16). The proof that this optimization leads to O(log N)-hop lookup path length, similar to Chord, is presented in Theorem 4.1. However, an intuitive explanation is as follows: since the distance between any two points in the overlay is at most N hosts, it takes O(log N) hops to locate any segment. Thus, even though the number of nodes in the overlay is greater than N due to host virtualization, the lookup path length is not affected. 3.2.3 Maintenance of Overlay Graph As with Chord, R-Chord maintains its overlay through periodic stabilizations. The periodic stabilization is implemented as two functions: stab ilize_successor() and correct_ f ingers(). The first function corrects successor, (i.e., the first (a) Visiting Host 3 is Equal to Visiting Node 2|3 and Node 9|3 in the Overlay Fig. 16 Effect of shared finger tables on routing finger, and predecessor pointers, whereas the later correct the remaining fingers in a finger table. The rate with which these functions are invoked is an implementation consideration. During periodic stabilization, R-Chord exploits finger flexibility which is inherent in our segment-based overlay. Finger flexibility denotes the amount of freedom available when choosing a finger. A higher finger flexibility increases the robustness of lookup since finger tables deplete slower in the event of node failures [31]. Finger flexibility also allows proximity-based routing to reduce lookup latency. However, our current R-Chord implementation has yet to exploit this feature. The segment-based overlay improves the finger flexibility of R-Chord whereby n. f inger[i] is allowed to point to any nodes in the same segment as successor(n + 2i−1 ) – this is an improvement (b) Visiting Host 6 (c) Visiting Host 9 −All Nodes are Visited 514 Fig. 17 Finger tables with backup fingers over O(1) finger flexibility of Chord3 . To exploit finger flexibility, we employ the backup fingers scheme which is reminiscent of Kademlia’s κbucket routing tables [52]. With such a scheme, every R-Chord node n maintains backups for each of its fingers. Thus, when f dies (i.e., points to a dead node), n still has a pointer to segment S prefix( f ) . The new structure of the R-Chord finger table is shown in Fig. 17. We use n. f inger[i] and b ackup(n. f inger[i]) to denote the main finger and the list of backup fingers, respectively. Figure 18 shows the algorithm to correct successors in R-Chord. When a new successor is detected, the old successor pointer is added into the backup list instead of being discarded (line 7). Similarly, when a new predecessor is to be set, the old predecessor is also added into the backup list (line 19). We then ensure that the backup list of successors and predecessors contains only nodes with the same prefix as the new successor (line 9) and predecessor (line 21), respectively. Figure 19 shows that the algorithm to correct finger f in R-Chord exploits finger flexibility. When a new finger is added, we also construct its backup list based on the entries piggybacked from the remote node and the older valid backup entries (lines 7–9 in Fig. 19a). As with lookups, the finger-correction algorithm incorporates shared finger tables (line 4 in Fig. 19a and line 3 in Fig. 19b). 4 Theoretical Analysis In this section, we analyze the lookup performance of R-Chord, and compare the overhead 3 To improve its robustness in spite of its lower finger flexibility, each Chord node caches additional entries in its finger tables. Such scheme can also be adopted in R-Chord. V. March, Y.M. Teo of the mapping scheme in R-Chord with conventional Chord-based DHT, hereafter referred to as Chord. Let N denote the number of hosts, Th denote the set of unique keys owned by host h, T denote the average number of unique keys N−1 |T | owned by each host (i.e., h=0N h ), and K denote the total number of unique keys in the system N−1 Th |). The Chord overlay consists of (i.e. | h=0 N nodes, one node per host, and the R-Chord overlay consists of V = NT nodes with an average of T nodes per host. 4.1 Lookup In the following, we present the proof on the lookup path length of R-Chord. Lemma 4.1 (Lookup Path Length of Routing by Segments) Routing by segments leads to O(log K)hops lookup. Proof To analyze the lookup path length due to our routing-by-segments optimization, we compare the finger tables in R-Chord and Chord. According to Theorem 2 in [65], in a Chord system consisting of N nodes, the lookup path length is O(log N) if Chord is able to route a lookup request from one node to another such that each step halves the distance to the destination node. To achieve this, each node n maintains O(log N) unique fingers where: 1. The distance between n and n. f inger[i + 1] is twice the distance between n and n. f inger[i]. 2. The largest finger of n points to successor(N/2). As highlighted in Section 2.2, the division operation in N/2 implies a modulo by 2m . We now show the similarity of finger tables in R-Chord and Chord. Let S = O(N P(k ∈ Th )) denotes the average number of nodes in a segment. In an R-Chord system consisting of V nodes, each node n maintains O(log V) unique fingers where: 1. The first O(log S) of the O(log V) fingers point to the segment containing n.successor. The remaining O(log V − log S) fingers point to O(log V − log S) different segments because the distance between n and A Read-Only Distributed Hash Table 515 Fig. 18 Successorstabilization algorithm n. f inger[log(S + j + 1)] is twice the distance between n and n. f inger[log(S + j)], where 0 ≤ j ≤ log N − log S. 2. The largest finger of n will point to successor(N/2), which is a node in the segment that succeeds segment K/2. Fig. 19 Finger-correction algorithm (a) Main Algorithm (b) Helper Functions 516 V. March, Y.M. Teo distributed across hosts, based on observation oi real Grid testbeds such as the TeraGrid.4 Using the same argument as in Chord, R-Chord routes a lookup request from one segment to another and each hop halves the distance, in terms of the number of segments, to the destination segment. Since R-Chord consists of K segments, a lookup will cost O(log K) hops. Lemma 4.2 (Probability of a Host to own a Key) The probability that host h owns key k, i.e., P(k ∈ K Th ), is bounded by ln K−T . Theorem 4.1 (Lookup Path Length in R-Chord) With shared f inger tables, the lookup path length in R-Chord is O(min(log K, log N)) hops. Proof Let k ∈ Th denote key k owned by host h. We define the probability that host h owns key k as Proof If K ≤ N then log K ≤ log N, assuming that K, N > 0. Thus, according to Lemma 4.1, this theorem is true. Consider K > N. When host h processes lookup(k), we choose two consecutive keys s, u ∈ Th where P(k ∈ Th ) = P(e1 ) + P(e2 |e1 ) + P(e3 |e1 , e2 ) 1. s < k < u 2. There is no v ∈ Th such that s < v < u The two keys, s and u, are associated with two nodes, namely node s|h and node u|h, respectively. The destination segment Sk will be located between node s|h and node u|h, and the distance between s|h and u|h is O(K/T). Since K ≤ V, then K/T = O(V/T) = O(NT/T) = O(N). Thus, according to Theorem 2 in [65], lookup(k) can be routed from node s|h to segment Sk in O(log N) hops. As shown, the lookup performance in R-Chord is at worst comparable to the lookup performance in Chord. With shared finger tables, R-Chord’s lookup path length is equivalent to Chord where the number of hops to reach a certain node is affected by the number of hosts in the physical network (N) instead of the number of nodes in the overlay network (V). 4.2 Overhead The following theorems compare the maintenance overhead in R-Chord and Chord in terms of the cost of virtualization, number of fingers per host, cost of updating data items, and overhead of replication. We assume that keys are uniformly + . . . + P(eT |e1 , . . . , eT−1 ) = T P(ei |e1 . . . ei−1 ) i=1 where ei and ei denote that the ith key of host h is equal and not equal to k, respectively. Assuming that T K and k is uniformly drawn from {1, ..., K}, we approximate P(k ∈ Th ) using the first-order Markov process as follows. Firstly, we consider K resource types as K balls, each with a unique color. If we pick T balls sequentially, then the probability that the ith ball is col1 ored with k is K−i+1 . This leads to the following equation: ⎧ 0 if T = 0 ⎪ ⎪ ⎪ ⎨1 if T = 1 K T K P(k ∈ Th ) = 1 1 ⎪ ⎪ = if T > 1 ⎪ ⎩ K − i + 1 i i=1 i=K−T+1 Since Hx = x 1 i=1 P(k ∈ Th ) = ⎧ ⎨0 ⎩ 1 K i = ln x + O(1), then if T = 0 if T = 1 if T > 1 H K − H K−T ⎧ if T = 0 ⎨0 if T = 1 = K1 ⎩ K ln K−T if T > 1 4 https://www.teragrid.org/web/user-support/compute_resources A Read-Only Distributed Hash Table Theorem 4.2 (Cost to Join Overlay) The cost for a host to join R-Chord and Chord is O(|Th | log2 V) and O ( log2 N + | Th | log N + K ln K K ), −T respectively. Proof R-Chord virtualizes a host into |Th | nodes in an overlay graph of size V. Since a node join costs O(log2 V), the host join costs O(|Th | log2 V). In Chord, a host join consists of a nodejoin operation, |Th | store operations to store the key-value pairs belonging the new host, and migrations of key-value pairs. The node-join operation costs O(log2 N) and each store operations costs O(log N). The migration process moves O(N/K) unique keys from an existing node, which is the successor of the new node, to the new node n. As there are O(N P(k ∈ Th )) key-value pairs per unique key, the migration K costs O(K ln K−T ). Therefore, the host join costs 2 K O(log N + |Th | log N + K ln K−T ) in total. Theorem 4.2 shows that the cost to join R-Chord is higher than Chord. This is because R-Chord replaces a store operation with a join operation, and the cost of a join operation is higher than a store operation. The following theorem shows the overhead of maintaining an overlay, i.e., stabilization cost, in terms of the number of messages sent. Theorem 4.3 (Cost of Stabilizations) In R-Chord, the stabilization cost to correct all f ingers including successor pointers is O(V log N log V) messages. Since N ≤ V, the stabilization cost is also (V log2 V) messages. On the other hand, the stabilization cost in Chord is O(N log2 N) messages. Proof The R-Chord overlay consists of V nodes and each node maintains O(log V) fingers; this sums up to a total of O(V log V) fingers. Correcting the ith finger of node n is performed by locating the node that succeeds n + 2i−1 . With shared finger tables as the only lookup optimization, it requires O(log N) hops to correct the finger (see Theorem 4.1). Thus, the cost of stabilization for correcting all fingers is O(V log V log N) hops. In Chord, the overlay consists of N nodes and each node maintains O(log N) fingers. Correcting 517 each finger is performed by locating successor(n + 2i−1 ), which costs O(log N) hops. Thus, the cost of stabilization is O(N log2 N) hops. Theorem 4.3 states that a higher number of fingers implies a higher overhead in maintaining an overlay graph. This affects the scalability of RChord particularly when a host is virtualized into many nodes. To reduce the overhead of periodic stabilizations, which are employed by the current implementation of R-Chord to correct fingers, nodes need not correct all their fingers each time the stabilization procedure is invoked. Instead, each invocation corrects only a subset of a node’s fingers, e.g. the successor pointer and another randomly-chosen finger; this is similar to Chord’s current implementation of periodic stabilizations. The drawback of this approach is it increases the number of incorrect entries in a finger table, which increases the lookup path length. However, as long as the successor pointer, i.e., the first finger, is maintained, the lookup will still terminate at the correct node. Theorem 4.4 (Finger Flexibility) The f inger f lexK ibility in R-Chord and Chord is O(N ln K−T ) and O(1), respectively. Proof Assume that successor(n + 2i−1 ) is in segment Sk . In R-Chord, the ith finger of node n can point to any node in segment Sk . The number of nodes in this segment is equal to to the number of hosts that own key k, which is O(N P(k ∈ Th )) K hosts. Hence, the finger flexibility is O(N ln K−T ). In Chord, the ith finger of node n must point to successor(n + 2i−1 ), and hence, the finger flexibility is O(1). As mentioned in Section 3.2.3, a higher finger flexibility increases the robustness of lookup in the presence of node failures. Higher finger flexibility also allows proximity-based routing to reduce lookup latency. Theorem 4.5 (Cost to Add Key) In Chord, adding a key-value pair costs O(log N). In R-Chord, adding a key-value pair whose key k already exists in host h (i.e., k ∈ Th before the addition) costs O(1), while adding a key-value pair whose key is 518 new for host h (i.e., k ∈ / Th before the addition) costs O(log2 V). Proof In Chord, a key-value pair is stored on the successor of the key. This costs O(log N). In R-Chord, if k ∈ Th before the addition, then no new node is created and hence, the cost is O(1). However, if k ∈ / Th before the addition, then a new node is created and joins the R-Chord system. This costs O(log2 V). In applications such as P2P file sharing, sharing a new file is equal to adding a new resource type. However, in a computational Grid, a resource type consists of many resource instances, and an administrative domain can add new instances to one of its existing resource types. Theorem 4.5 shows that using R-Chord, the administrative domain does not need to notify other nodes in the R-Chord overlay. Theorem 4.6 (Number of Replicas) In R-Chord, the total number of key-value pairs with the same K key is O(N ln K−T ). In Chord, assuming that each key-value pair is replicated O(log N) times, then the total number of key-value pairs with the same K key is O(N ln K−T log N). Proof Given N hosts, the number of key-value pairs with the same key is O(N P(k ∈ Th )). Since R-Chord does not redistribute and replicate keyvalue pairs, the number of key-value pairs with the K same key is also O(N ln K−T ). In Chord, because each key-value pair is replicated O(log N) times, then the total number of key-value pairs with the K log N). same key will be O(N ln K−T R-Chord does not need to replicate data items to improve the lookup resiliency in the event of node failures. This eliminates the network bandwidth required to replicate data items and the complexity in maintaining consistency among replicas. Even when data items are not replicated, dataitem distribution can still lead to the problem of inconsistent data items. In conventional DHTs, updates must be propagated to the node responsible for storing the data item. This is shown in the following corollary. V. March, Y.M. Teo Corollary 4.1 In Chord, the cost of a host to update its key-value is O(log N). In R-Chord, the cost is O(1). Proof In Chord, the cost for a host (the originating node) to propagate an update on its key-value pair to another node costs O(log N), according to Theorem 2 in [65] In R-Chord, the key-value pair is mapped to its originating node. Hence, the cost of updating the key-value pair is O(1). Corollary 4.1 shows that R-Chord improves the performance of a host in updating its data items. In the case of computational Grids, updates occur when an administrative domain changes the configuration of its shared resources, or changes the number of resource instances of a resource type. When an administrative domain withdraws all resources of the same type from the Grid, the key-value corresponding to the resource type is deleted from the overlay network. The cost of a deletion in Chord and R-Chord is compared in Corollary 4.2. Corollary 4.2 In Chord, the cost of a host to delete its key-value is O(log N). In R-Chord, the cost is O(log V log N). Proof In Chord, a key-value is deleted by notifying successor(k), and the cost to lookup for successor(k) is O(log N). In R-Chord, deleting a key-value requires a node to depart from the overlay network. The departed node invalidates O(log V) fingers in the system, since the node is pointed to by O(log V) nodes and one finger per predecessor is invalidated. Correcting each of stale finger costs O(log N) (see Theorem 4.1. Thus, the total cost becomes O(log V log N). 4.3 Costs Comparison Table 3 summarizes the performance analysis of R-Chord. Recall that N denotes the number of hosts, T denotes the number of unique keys owned by each host, K denotes the total number of unique keys in the system, and V = NT A Read-Only Distributed Hash Table 519 Table 3 Comparison of Chord and R-Chord Property Chord Lookup a key Host join O(log N) O log2 N + |Th | log N + K ln # unique fingers per host O(log N) Finger flexibility O(1) Stabilization Add a key that exists Add a new key Update a key-value pair O(N log2 N) O(log N) O(log N) O(log N) K log N O N ln K−T O(log N) # key-value pairs with the same key Delete a key denotes the number of nodes in an R-Chord overlay network. We show that the lookup path length in R-Chord is shorter than Chord and at worst equal to Chord. However, our mapping scheme increases the cost for a host to join an overlay. In addition, each host in R-Chord has more fingers to correct due to the host being virtualized into nodes. Thus, the scalability of R-Chord is determined by the number of nodes associated to each host. When each host is virtualized into one node only, the scalability of R-Chord is equal to traditional Chord. 5 Simulation Analysis In this section, we evaluate R-DHT by simulating an R-Chord-based resource indexing and discovery scheme in a large computational Grid. As illustrated in Fig. 1b, a computational Grid consists of many administrative domains, each of which shares one or more compute resource types. Each administrative domain, represented by its MDS server, joins an R-Chord overlay and stores only its own resource metadata. The key of each data item is determined by the type (i.e., attributes) of compute resource associated with the data item. Thus, the number of unique keys owned by a host denotes the number of unique resource types shared by an administrative domain. To facilitate our experiments, we implement RChord using the Chord simulator [2]. Let m = 18 bits unless stated otherwise. The network latency R-Chord K K−T O(min(log K, log N)) O(|Th | log2 V) O(|Th | log V) K O N ln K−T O(V log V log N) O(1) O(log2 V) O(1) K O N ln K−T O(log V log N) between hosts is exponentially distributed with a mean of 50 ms, and the time for a node to process a request is uniformly distributed in the interval of [5, 15] ms. Each finger is assumed to have at most four backups. In the following subsections, we compare the lookup path length, resiliency to node failures, time to correct overlay, and lookup performance on incorrect overlays. 5.1 Lookup Path Length To verify Theorem 4.1, we measure the average lookup path length of 500,000 lookups. Each lookup requests a randomly-selected key and is initiated by a randomly-chosen host. Assuming that T denotes the average number of unique keys per host, each host has |Th | ∼ U[0.5T, 1.5T] unique keys. As shown in Fig. 20, the average lookup path length in R-Chord is 20–30% lower than in Chord. When K > N (for K = NT), R-Chord’s overlay consists of K segments and each segment consists of one node. According to Theorem 4.1, the lookup path length of R-Chord is affected only by N, and hence, increasing K does not increase the lookup path length. However, for K ≤ N, the lookup path length increases with K. Figure 20 also shows that in R-Chord, increasing T reduces the average path length, which can be explained as follows. First, as each host maintains O(|Th | log NT) unique fingers an increase in T also increases the number of fingers per hosts. Several studies such as [33, 61] also reveal that 520 V. March, Y.M. Teo (a) N = 10,000 Hosts (b) N = 25,000 Hosts Fig. 20 Average lookup path length maintaining more fingers reduces the lookup path length. Secondly, an increase in T also increases the number of segments occupied by a host, and hence, each host has a higher probability to be visited. Our simulation result confirms Theorem 4.1, i.e., when K > N, the lookup path length in R-Chord has the same upper bound as Chord. When K ≤ N, the lookup path length in R-Chord is shorter than Chord. Scalable lookup is achieved by maintaining finger tables in hosts. To understand the trade-off between lookup performance and space requirement of finger tables, we compare the average number of fingers in R-Chord and Chord. The result in Table 4 confirms that in R-Chord, the number of fingers per host is proportional to Table 4 Average number of fingers K T 5,000 1 2 4 8 1 2 4 8 NT the number of unique keys, both per host (T) and total (K) (Theorem 4.3). As T is increased, the number of nodes in an R-Chord overlay increases. Therefore, the number of fingers maintained by the host also increases. On the other hand, in Chord, the number of fingers per host depends on the number of hosts (N) only, and is independent of T and K. Table 4 also shows that the number of backup fingers is determined by finger flexibility. When there are more than one nodes per segment, as is the case with K = 5,000, backup fingers can be exploited. When K = NT, each segment consists of exactly one node. Therefore, no backup fingers can be exploited. For K = 5,000, we further observe that the number of backups per finger increases as T is increased. A higher T leads to a Chord R-Chord # Fingers/host # Fingers/host # Fingers/node # Backups/finger 16.2 16.2 16.2 16.2 16.2 16.2 16.2 16.2 15.4 32.7 69.3 146.6 16.2 34.4 72.8 153.3 15.4 16.3 17.3 18.3 16.2 17.2 18.2 19.2 3.0 3.8 3.9 4.0 0 0 0 0 A Read-Only Distributed Hash Table 521 (a) K = NT Unique Keys (b) K = 5,000 Unique Keys Fig. 21 Average lookup path length with failures (N = 25,000 hosts) higher number of nodes per segment. As such, finger flexibility is increased, and the number of backups per finger grows towards the predefined maximum of four backups per finger. 5.2 Resiliency to Simultaneous Failures To evaluate the resiliency when no churn occurs, we measure the average lookup path length and failed lookups with the following procedures. First, we setup a system of N = 25,000 hosts where all hosts have T unique keys on average. Then, we fail a fraction of hosts5 simultaneously, disable the periodic finger correction after the simultaneous failures, and simulate 500,000 lookup requests. We define a lookup to have failed if it results in (i) a false negative answer, where existing resources cannot be located (i.e., at least one node with the key is alive but cannot be found), or (ii) a false positive answer where stale data items are returned. We also assume that Chord stores a keyvalue pair only to successor(key) and does not further replicate the key-value pair to other nodes. 5 In R-Chord, one host fail results in simultaneous node fails. Finally, we exploit the property of finger flexibility in R-Chord by maintaining a maximum of four backups per finger. Figure 21 shows the average lookup path length with 25% and 50% of simultaneous host failures. When K > N (for K = NT), the average lookup path length in R-Chord shows a trend similar to that of in Chord, i.e., the lookup path length increases as more hosts fail (Fig. 21a). Because each segment consists of only one node, R-Chord cannot exploit finger flexibility. Hence, as the percentage of host failures increases, the number of valid fingers, which points to alive nodes, reduces in each node’s finger table. For K ≤ N (Fig. 21b), R-Chord has a shorter average path length than Chord and the lookup path length is not significantly affected by the number of failed hosts. The reason is as follows. Firstly, R-Chord provides O(log K)-hops lookup path length only if each node has a correct finger table. In the case of node failures, the finger table has a reduced number of valid fingers, which increases the lookup path length. However, since K ≤ N implies each segment consists of more than one node, R-Chord effectively exploits finger flexibility to maintain the number of valid fingers. In terms of failed lookups, Fig. 22 shows that for K > N and K ≤ N, R-Chord is 70 and 95% lower 522 V. March, Y.M. Teo (a) K = NT Unique Keys (b) K = 5,000 Unique Keys Fig. 22 Percentage of failed lookups (N = 25,000 hosts) than in Chord, respectively. For (K = NT) > N (Fig. 22a), Chord has more failed lookups because key-value pairs are stored on a different host. In the event that the host which stores the key-value pair fails, a lookup request to that host will be unsuccessful though the host that owns the key-value pair is still alive. For K ≤ N (Fig. 22b), R-Chord achieves even less failed lookups (95% lower than Chord) because R-Chord exploits the property that each key is available in a segment consisting of several nodes. Hence, even if some of these nodes fail, R-Chord can still reach the remaining hosts in the segment through the backup fingers. Thus, R-Chord offers better resiliency to simultaneous failures in comparison with conventional DHTs. 5.3 Time to Correct Overlay The correctness of an overlay network is crucial to the lookup performance in a DHT. To ensure the correctness of an overlay in the event of membership changes, each node periodically corrects its fingers, i.e., periodic stabilization. However, the larger size of the R-Chord overlay (Theorem 4.2 increases the stabilization cost (Theorem 4.3). To amortize the maintenance overhead, periodic stabilization in R-Chord is performed less aggresively, similar to Chord, where each invocation of the stabilization procedure corrects only a subset of a node’s fingers, e.g. the successor pointer and another randomly-chosen finger. However, this may increase the time required to correct the overlay of R-Chord. In this experiment, we evaluate the time required to correct the overlay topology, which is measured starting from the time when the last host arrives. To facilitate this measurement, we quantify the correctness of an overlay using stabilization degree (ξ ) which is derived by averaging the correctness of all finger tables in the overlay network (ξn ). N−1 ξ= n=0 N ξn 0≤ξ ≤1 (1) and ⎧ ⎨ 0 if n. f inger[1] is incorrect ξn = F ⎩ F (2) where 0 ≤ ξn ≤ 1, F is the number of correct fingers in n, and F is total number of fingers in n. Note that we do not consider backup fingers in calculating ξn . The experiment is performed as follows. We simulate a system consisting of N hosts with a mean arrival rate λ = 1 host/second (Poisson A Read-Only Distributed Hash Table 523 distribution). The number of unique keys per host is |Th | ∼ U[2, 5] unique keys and therefore, T = 3.5 unique keys. We assume that the total number of unique keys in the system is K = 3N keys. This K approximates NT keys where each segment consists of one node on average. We then periodically measure ξ starting from the time when the last host arrives. A node joins a ring overlay through a randomly chosen existing node. We use the nodejoin process as described in [65]. First, a new node n starts the join process by adding n = f ind_successor(n) as its successor pointer. After one or more rounds of finger correction, there will be at least one other node pointing to n. At this time, the join process completes. Note that RChord uses the f ind_successor() which incorporates shared finger tables (Fig. 19b). Each node invokes the finger correction every [0.5 p, 1.5 p] seconds (uniform distribution). Each invocation of finger correction will correct the successor pointer (n. f inger[1]) and one other finger. Correcting n. f inger[i] is done by updating it with the node returned by f ind_successor(n + 2i−1 ). Figure 23 reveals that at larger p (960 seconds in this experiment), nodes in Chord correct their successor pointer (i.e., n. f inger[1]) faster than R-Chord. For examples, when N = 25, 000 hosts, ξ in Chord increases from 0.36–0.59 in the first three hours, which is faster than R-Chord (0.31– 0.38). The same behavior is also observed in N = 50,000 hosts. This is because ξn puts more priority on the successor pointer (see (2)). Hence, by correcting successor pointers faster, Chord increases its ξ faster than R-Chord. In summary, although R-Chord has a larger overlay and each of its hosts has to correct more fingers, R-Chord does not require a longer time than Chord to fully correct its overlay. This is due to shared finger tables reducing the time to locate the correct successor when correcting a finger. (a) N = 25,000 Hosts (b) N = 50,000 Hosts 5.4 Lookup Performance under Churn Churn refers to membership changes in an overlay network. Churn temporarily causes the overlay network to be inconsistent and as such, lookup performance is reduced until the overlay is fully corrected through the stabilization. In R-DHT, churn occurs in two ways. Firstly, host arrivals, host fails, and host leaves result in simultaneous node joins, node fails, and node leaves, respectively. Secondly, adding a new unique key to a host also causes a node join (Theorem 4.5). When the frequency of membership changes (i.e., the churn rate) is high, lookup performance may Fig. 23 Correctness of overlay ξ (measured every three hours starting from the last host arrival) 524 decrease because the larger overlay of R-DHT magnifies the impact of churn on the correctness of overlay topology. To evaluate the ability of R-DHT to cope with churn, we simulate lookups under various churn rate. Due to the churn, the R-Chord ring overlay keeps changing because of host arrivals, failures, and leaves. When a host fails, its corresponding nodes silently disappear from the overlay. On the other hand, when a host leaves, its corresponding nodes gracefully notify their successor and predecessor to update their fingers accordingly. In addition, a node leaving a Chord ring migrates all data items belonging to other nodes to its successor. In this experiment, we assume that each host has |Th | ∼ U[4, 12] unique keys (which results in T = 8 unique keys), and each node invokes the finger correction procedure every [30, 90] seconds (uniform distribution). We first set up an overlay network of 25,000 hosts, followed by a number of churn events (i.e., arrivals, fails, and leaves) produced by 25,000 hosts in a duration of one hour. Thus, there will be about N = 25,000 alive hosts at any time within this duration. During this one-hour period, we also simulate a number of lookup events and keep the ratio of arrive:fail:leave:lookup to 2:1:1:6. The mean arrival rate of these events, λ, will model the churn rate. Assuming that these events follow a Poisson distribution, our simulation uses λ B = 10 events/second and λG = 40 events/second. These are derived from the measurements on peer lifetime by Bhagwan et al. [9] and Gummadi et al. [32], respectively.6 Table 5 presents the result for 6λ B is obtained as follows. Bhagwan et al. [9] measures that on average, each host performs 6.4 joins, and 6.4 fails per day [9]. We interpret the measured fail events as consisting of 3.2 host fails and 3.2 host leaves. Thus, including lookups, there are 32 events per day. With 25,000 hosts come and go repeteadly, there are 800,000 events per day, which is approximately one event every 100 ms. Similar steps as above are used to derive λG . Gummadi et al. [32] measures 24 joins and 24 fails per host per day, and we interpret the measured fails as consisting of 12 host fails and 12 host leaves. Given 25,000 hosts and a ratio of arrive:fail:leave:lookup = 2:1:1:6, there are approximately one event every 25 ms. V. March, Y.M. Teo various K, from 5,000 (K < N) to 150,000 (K ∼ NT). The average lookup path length (Table 5a) again confirms Theorem 4.1 and the result from Section 5.1. Though the number of nodes in RChord’s overlay is at least three times Chord, when K < N the average lookup path length is shorter than Chord since R-Chord routes lookups by segments. When K ≥ N, the average lookup path length is not worse than Chord due to the shared finger tables. Table 5b shows that lookup resiliency in RChord is comparable to Chord (from 8% lower to 9% higher than Chord). Under a churn rate of λ B and λG , R-Chord has a lower percentage of failed lookups when K ≤ 100,000 and K ≤ 50,000, respectively. The results indicate the importance of exploiting finger flexibility through backup fingers. In R-DHT, lookup resiliency is increased due to the property that a key can be found in several nodes of the same segment. Hence, it is important that a segment can be reached as long as it contains one or more alive nodes. R-Chord addresses this issue by maintaining backup fingers as redundant pointers to a segment. With a higher number of nodes per segment (i.e., finger flexibility), backup fingers are more effective in reducing the impact of a higher churn rate to lookup resiliency. As K is increased, the number of nodes per segment decreases. Because finger flexibility is reduced, there are less redundancy to be exploited through backup fingers. Considering that R-Chord’s overlay is eight times larger than Chord’s overlay, we conclude that the decrease is reasonable. In summary, the result in this subsection suggests that R-Chord achieves better resiliency when finger flexibility is exploited. When RChord cannot exploit finger flexibility, it can still achieve comparable resiliency as Chord because by not distributing data items, failure of a host affects only its own data items. 6 Related Work In this section, we first compare and contrast R-DHT with structured P2P systems that support A Read-Only Distributed Hash Table Table 5 Lookup performance under churn (N ∼ 25,000 hosts) K 525 λ B = 10 events/second λG = 40 events/second Chord Chord R-Chord (a) Average lookup path length 5,000 8.4 4.1 7,500 8.5 4.4 10,000 8.5 4.6 15,000 8.5 4.9 25,000 8.5 5.3 50,000 8.5 5.9 75,000 8.5 6.2 100,000 8.6 6.4 125,000 8.6 6.5 150,000 8.6 6.6 % of failed lookups 5,000 2 <1 7,500 2 <1 10,000 3 1 15,000 4 1 25,000 4 1 50,000 7 5 75,000 10 7 100,000 7 5 125,000 6 7 150,000 8 16 the no-data-item-distribution scheme. Secondly, we discuss the current status of distributed resource indexing and discovery in a computational Grid. 6.1 Structured P2P with No-Store Scheme We discuss three structured P2P that also support the no-data-item-distribution scheme, namely SkipGraph [7], Structella [13], and SkipNet [34]. SkipGraph [7] supports the no-store scheme by associating a key to a node and organizing nodes as a skip-list-like topology. It is assumed that each key is shared only by one node, e.g. resources of the same type are shared only by one administrative domain. Our proposed scheme generalizes SkipGraph by first, allowing a key to be associated with several nodes. Secondly, our scheme can be applied to structured overlay with different underlying topologies. Structella [13] organizes nodes as a structured overlay (i.e., a Pastry ring [62]) but each node manages only its own keys, similar to R-DHT. However, unlike R-DHT, Structella does not map keys onto nodes and does not fully exploit the 9.1 9.1 9.3 9.1 9.2 9.3 9.5 9.4 9.2 9.4 7 9 9 13 15 20 20 23 30 27 R-Chord 4.7 5.0 5.4 5.8 6.5 7.2 7.7 8.0 8.1 8.3 2 2 3 4 9 13 26 25 26 34 structured overlay to route lookup messages. Instead, it adopts the routing schemes used in unstructured P2P such as flooding and random walk. The structured overlay network only serves to reduce the overhead of those schemes. The authors reported that Structella offers similar results guarantee as unstructured P2P. To improve results guarantee, the authors propose to distribute and replicate data items to a number of nodes as their ongoing work [14]. In contrast to Structella, R-DHT inherits the property of DHTs which is the strong result guarantee. SkipNet [34] supports content locality to map a key onto a specific node. This is achieved through the hierarchical naming scheme: put(n|key) maps a key to node n, and lookup(n|key) retrieves the key. Compared to our proposed scheme, SkipNet provides greater flexibility for a host to decide where its data items are stored. However, the hierarchical naming scheme does not directly supports queries such as “find resources of type k in any hosts”. In contrast, though R-DHT addresses node autonomy only by ensuring that data items are stored on their originating host, it is compatible with flat naming scheme. 526 V. March, Y.M. Teo Table 6 Comparison of R-DHT with related work Characteristic SkipGraph Structella SkipNet R-DHT Mapping scheme High result guarantee Key shared by many hosts Controlled data placement Flat naming scheme Overlay network Yes Yes No No Yes Skip List No No Yes No Yes Pastry Yes Yes Yes Yes No Multi-level ring Yes Yes Yes No Yes Multiple choices Table 6 summarizes the comparison of R-DHT with the three no-data-item-distribution scheme. 6.2 Resource Discovery in Computational Grid We classify distributed Grid information systems based on their overlay topology into unstructured overlay networks [25, 36, 46, 53] and structured overlay networks (DHT) [10, 12, 16, 64, 69]. In unstructured overlay networks, resource information is replicated either to all nodes [46] or a subset of nodes [25, 36, 53]. Replicating resource information to a smaller number of nodes reduces the bandwidth overhead, but this potentially reduces the lookup accuracy. Iamnitchi et al. [36] propose to replicate information based on the small-world effect, Forestiero et al. [25] propose to replicate resource information based on a probabilistic scheme, and Moreno-Vozmediano [53] propose for each node to replicate its information to a set of neighbor nodes. To process a lookup, a query message is forwarded to a number of nodes based on heuristics such as TTL (time-to-live). However, heuristics do not guarantee that a lookup will successfully find resources. In contrast, DHT-based systems provides stronger lookup guarantee and scalable lookup performance. MAAN [12], self-organizing Condor pools [10], XenoSearch [64], RIC [69], and bitmap trees [16] are examples of Grid information systems that are based on conventional structured overlay networks. Compared to such schemes, our R-DHTbased Grid information system increases the autonomy of administrative domains by not distributing data items. In addition, our scheme does not introduce stale data items when the overlay topology changes, and it is resilient to node failures without a need to replicate data items. 7 Conclusions A distributed hash table maps each key onto a node to achieve good lookup performance. A typical DHT realizes this mapping through the store operation and as a result, key-value pairs are distributed across the overlay network. For applications where distributing key-value pairs is not desirable, we proposed R-DHT, a new DHT mapping scheme without the store operation. RDHT enforces the read-only property by virtualizing a host into nodes subjected to the unique keys belonging to the host, and dividing the node identifier space into two sub-spaces, a key space and a host identifier space. By mapping data items back onto its owner, R-DHT is inherently fault tolerant. In addition, it increases consistency of data items because updates need not be propagated in overlay networks. R-DHT maintains API compatibility with existing DHTs. In addition, R-DHT lookup operations exploit not only existing DHT lookup schemes, but also routing by segments, shared finger tables, and finger flexibility through backup fingers. Through theoretical and simulation analyses, we showed that in a Chord-based R-DHT consisting of N hosts and K total unique keys, the lookup path length is O(min(log K, log N)) hops. This result suggests that our proposed lookup optimization schemes reduce lookup path length compared with conventional DHTs. We further demonstrate that R-DHT lookup is resilient to node failures. In our simulations, when 50% of nodes fail simultaneously, the number of failed lookups in R-Chord is 5–30%. This is lower compared to Chord where at least 60% of its lookups fail. Our simulation also shows that under churn, lookup performance of R-Chord is comparable to Chord even though R-Chord overlay is A Read-Only Distributed Hash Table eight times larger: (i) R-Chord lookup path length is shorter, and (ii) number of failed lookups in RChord is at most 8% worst than Chord. The host-to-nodes virtualization in R-DHT increases the size of overlay network in terms of the number of nodes. This leads to higher overhead in maintaining an overlay network. In RChord system that virtualizes N hosts into V nodes where V ≥ N, the maintenance overhead is O(V log V log N). With the same number of hosts, the maintenance overhead in Chord is O(N log2 N). Our simulation analysis further revealed that when the stabilization period is large, R-Chord requires more time to correct its overlay into a ring topology. In a separate study, we show that overlay-network maintenance in RChord can be reduced using hierarchical R-DHT topology where nodes are organized as a two-level overlay network [50]. Though R-DHT addresses the data-ownership issue while maintaining good lookup performance, real applications such as Grid and cloud require multi-attribute range queries. While we have investigated a simple multi-attribute range query scheme based on a space-filling curve [49], scalability of query processing remains a challenge due to the large number of resource types and attributes. To further improve lookup performance, we are also extending R-DHT with proximityaware routing and a selective data-item distribution scheme. Proximity-aware routing reduces lookup latency by considering geographical distance among nodes during lookups [24, 55, 68]. Selective data-item distribution potentially increases lookup performance via caching, increases the availability of replicable resources, and reduces load imbalance. This can be achieved by exploiting the segments in a R-DHT overlay network. References 1. Apache HTTP server. http://httpd.apache.org 2. The Chord Project. http://www.pdos.lcs.mit.edu/chord 3. Globus toolkit—Information Service. http://www. globus.org/toolkit/mds/ 4. Aberer, K., Cudr-Mauroux, P., Datta, A., Despotovic, Z., Hauswirth, M., Punceva, M., Schmidt, R.: P-Grid: a self-organizing structured P2P system. SIGMOD Rec. 32(2), 29–33 (2003) 527 5. Adamic, L.A., Lukose, R.M., Puniyani, A.R., Huberman, B.A.: Local search in unstructured networks. In: Handbook of Graphs and Networks: from the Genome to the Internet, pp. 295–316 (2003) 6. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., Konwinski, A., Lee, G., Patterson, D. A., Rabkin, A., Stoica, I., Zaharia, M.: Above the clouds: a Berkeley view of cloud computing. Technical Report UCB/EECS-2009-28, Electrical Engineering and Computer Sciences, University of California at Berkeley (2009) 7. Aspnes, J., Shah, G.: Skip Graphs. In Proc. of the 14th Annual ACM-SIAM Symp. on Discrete Algorithms, pp. 384–393. ACM/SIAM Press, USA (2003) 8. Balakrishnan, H., Lakshminarayanan, K., Ratnasamy, S., Shenker, S., Stoica, I., Walfish, M.: A layered naming architecture for the internet. In Proc. of ACM SIGCOMM, pp. 343–352. ACM Press, Germany (2004) 9. Bhagwan, R., Savage, S., Voelker, G. M.: Understanding availability. In Proc. of the 2nd Intl. Workshop on Peer-to-Peer Systems (IPTPS’03), pp. 256–267. Springer-Verlag, USA (2003) 10. Butt, A. R., Zhang, R., Hu, Y. C.: A self-organizing flock of Condors. In Proc. of the ACM/IEEE SC2003 Conf. on High Performance Networking and Computing, p. 42. ACM Press, USA (2003) 11. Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., Brandic, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25(6):599–616 (2009) 12. Cai, M., Frank, M., Chen, J., Szekely, P.: MAAN: a Multi-Attribute Addressable Network for Grid information services. J. Grid Computing 2(1):3–14 (2004) 13. Castro, M., Costa, M., Rowstron, A.: Should we build Gnutella on a structured overlay? ACM SIGCOMM Comput. Commun. Rev. 34(2):131–136 (2004) 14. Castro, M., Costa, M., Rowstron, A.: Debunking some myths about structured and unstructured overlays. In Proc. of 2nd Symp. on Networked Systems Design and Implementation (NSDI’05), pp. 85–98. USENIX Association, USA (2005) 15. Castro, M., Druschel, P., Ganesh, A., Rowstron, A., Wallach, D. S.: Secure routing for structured peer-topeer overlay networks. In Proc. of the 5th USENIX Symp. on Operating Systems Design and Implementation (OSDI 2002), pp. 299–314. USENIX Association, USA (2002) 16. Chang, R.-S., Hu, M.-S.: A resource discovery tree using bitmap for Grids. Future Gener. Comput. Syst. 26(1):29–37 (2010) 17. Chawathe, Y., Ratnasamy, S., Breslau, L., Lanham, N., Shenker, S.: Making Gnutella-like P2P systems scalable. In Proc. of ACM SIGCOMM, pp. 407–418. ACM Press, Germany (2003) 18. Cheng, A.-H., Joung, Y.-J.: Probabilistic file indexing and searching in unstructured peer-to-peer networks. In Proc. of the 4th IEEE Intl. Symp. on Cluster Computing and the Grid (CCGRID 2004), pp. 9–18. IEEE Computer Society Press, USA (2004) 528 19. Cody, E., Sharman, R., Rao, R. H., Upadhyaya, S.: Security in Grid computing: a review and synthesis. Decis Support Syst 44(4):749–764 (2008) 20. Cooper, B. F.: Quickly routing searches without having to move content. In Proc. of the 4th Intl. Workshop on Peer-to-Peer Systems (IPTPS’05), p. 163–172. Springer-Verlag, USA (2005) 21. Crespo, A., Garcia-Molina, H.: Routing indices for peer-to-peer systems. In Proc. of the 22nd IEEE Intl. Conf. On Distributed Computing Systems (ICDCS 2002), pp. 23–33. IEEE Computer Society Press, Austria (2002) 22. Dabek, F., Kaashoek, M. F., Karger, D., Morris, R., Stoica, I.: Wide-area cooperative storage with CFS. In Proc. of the 11th ACM Symp. on Operating Systems Principles (SOSP’01), pp. 202–215. ACM Press, Canada (2001) 23. Dabek, F., Zhao, B. Y., Druschel, P., Kubiatowicz, J., Stoica, I.: Towards a common API for structured peerto-peer overlays. In Proc. of the 2nd Intl. Workshop on Peer-to-Peer Systems (IPTPS’03), pp. 33–44. SpringerVerlag, USA (2003) 24. Dabel, F., Cox, R., Kaashoek, F., Morris, R.: Vivaldi: a decentralized network coordinate system. In Proc. of ACM SIGCOMM, pp. 15–26 (2004) 25. Forestiero, A., Mastroianni, C., Spezzano, G.: QoSbased dissemination of content in Grids. Future Gener. Comput. Syst. 24(3):235–244 (2008) 26. Foster, I., Kesselman, C. (eds.): The Grid: blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers (1999) 27. Foster, I., Zhao, Y., Raicu, I., Lu, S.: Cloud computing and Grid computing 360-degree compared. In Proc. of Grid Computing Environment Works. (GCE), pp. 1– 10. IEEE Computer Society Press, USA (2008) 28. Ghodsi, A., Alima, L. O., Haridi, S.: Symmetric replication for structured peer-to-peer systems. In Proc. of the 3rd Intl. Workshop on Databases, Information Systems and Peer-to-Peer Computing (DBISP2P 2005), p. 12. Spinger-Verlag, Norway (2005) 29. Godfrey, B., Lakshminarayanan, K., Surana, S., Karp, R., Stoica, I.: Load balancing in dynamic structured P2P systems. In Proc. of INFOCOM 2004, pp. 2253– 2262. IEEE Press, China (2004) 30. Godfrey, P. B., Stoica, I.: Heterogeneity and load balance in distributed hash tables. In Proc. of INFOCOM 2005, pp. 596–606. IEEE Press, USA (2005) 31. Gummadi, K., Gummadi, R., Gribble, S., Ratnasamy, S., Shenker, S., Stoica, I.: The impact of DHT routing geometry on resilience and proximity. In Proc. of ACM SIGCOMM, pp. 381–394. ACM Press, Germany (2003) 32. Gummadi, P. K., Saroiu, S., Gribble, S.: Measurement study of Napster and Gnutella as examples of peer-to-peer file sharing systems. Multim. Syst. Journal 9(2):170–184 (2003) 33. Gupta, A., Liskov, B., Rodrigues, R.: Efficient routing for peer-to-peer overlays. In Proc. of 1st Symp. on Networked Systems Design and Implementation (NSDI’04), pp. 113–126. USENIX Association, USA (2004) V. March, Y.M. Teo 34. Harvey, N. J. A., Jones, M. B., Saroiu, S., Theimer, M., Wolman, A.: SkipNet: a scalable overlay network with practical locality properties. In Proc. of the 4th USENIX Symp. on Internet Technologies and Systems (USITS’03), pp. 113–126. USENIX Association, USA (2003) 35. Hsiao, H.-C., King, C.-T.: A tree model for structured peer-to-peer protocols. In Proc. of the 3rd IEEE Intl. Symp. on Cluster Computing and the Grid (CCGRID 2003), pp. 336–343. IEEE Computer Society Press, Japan (2003) 36. Iamnitchi, A., Ripeanu, M., Foster, I.: Locating data in (small-world?) peer-to-peer scientific collaborations. In Proc. of the 1st Intl. Works. on Peer-to-Peer Systems (IPTPS’02), pp. 232–241. Springer-Verlag, USA (2002) 37. Karger, D. R., Ruhl, M.: Simple, efficient load balancing algorithms for peer-to-peer systems. In Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS’04), pp. 131–140. Springer-Verlag, USA (2004) 38. Kashani, F. B., Shahabi, C.: Criticality-based analysis and design of unstructured peer-to-peer networks as “complex systems”. In Proc. of the 3rd IEEE Intl. Symp. on Cluster Computing and the Grid (CCGRID 2003), pp. 351–358. IEEE Computer Society Press, Japan (2003) 39. Kaufman, L. M.: Data security in the world of cloud computing. IEEE Secur. Priv. 7(4):61–64 (2009) 40. Kienle, H. M., German, D., Tilley, S., Muüller, H. A.: Intellectual property aspects of web publishing. In Proc. of the 22nd Intl. Conf. on Design of Communication: the Engineering of Quality Documentation (SIGDOC), pp. 136–194. ACM Press, USA (2004) 41. Krauter, K., Buyya, R., Maheswaran, M.: A taxonomy and survey of Grid resource management systems for distributed computing. Intl. Journal of Softw., Practic. and Exper. 32(2):135–164 (2002) 42. Kubiatowicz, J., Bindel, D., Chen, Y., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., Zhao, B.: OceanStore: an architecture for global-scale persistent storage. In Proc. of the 9th Intl. Conf. on Architectural Support for Programming Languages and Operating Systems (ASPLOS-IX), pp. 190–201. ACM Press, USA (2000) 43. Landers, M., Zhang, H., Tan, K.-L.: Peerstore: better performance by relaxing in peer-to-peer backup. In Proc. of the 4th Intl. Conf. on Peer-to-Peer Computing (P2P 2004), pp. 72–79. IEEE Computer Society Press, Switzerland (2004) 44. Leslie, M., Davies, J., Huffman, T.: Replication strategies for reliable decentralised storage. In Proc. of the 1st Workshop on Dependable and Sustainable Peerto-Peer Systems (DAS-P2P 2006), pp. 740–747. IEEE Computer Society Press, Japan (2006) 45. Li, J., Stribling, J., Gil, T. M., Morris, R., Kaashoek, M. F.: Comparing the performance of distributed hash tables under churn. In Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS’04), pp. 87–99. Springer-Verlag, USA (2004) 46. Li, W., Xu, Z., Dong, F., Zhang, J.: Grid resource discovery based on a routing-transferring model. In Proc. A Read-Only Distributed Hash Table 47. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 58. of the 3rd Intl. Workshop on Grid Computing (GRID 2002), pp. 145–156. Springer-Verlag, USA (2002) Loo, B. T., Huebsch, R., Stoica, I., Hellerstein, J. M.: The case for a hybrid P2P search infrastructure. In Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS’04), pp. 141–150. Springer-Verlag, USA (2004) Lv, Q., Cao, P., Cohen, E., Li, K., Shenker, S.: Search and replication in unstructured peer-to-peer networks. In Proc. of the 2002 Intl. Conf. on Supercomputing (ICS 2002), pp. 84–95. ACM Press, USA (2002) March, V., Teo, Y. M.: Multi-attribute range queries on read-only DHT. In Proc. of the 15th Intl. Conf. on Computer Communications and Networks (ICCCN 2006), pp. 419–424. IEEE Communications Society Press, USA (2006) March, V., Teo, Y. M., Lim, H. B., Eriksson, P., Ayani, R.: Collision detection and resolution in hierarchical peer-to-peer systems. In Proc. of the 30th IEEE Conf. on Local Computer Networks (LCN 2005), pp. 2–9. IEEE Computer Society Press, Australia (2005) March, V., Teo, Y. M., Wang, X.: DGRID: a DHTbased Grid resource indexing and discovery scheme for computational Grids. In Proc. of the 5th Australasian Symp. on Grid computing and e-Research (AusGrid 2007). Australian Computer Society Inc, Australia (2007) Maymounkov, P., Mazières, D.: Kademlia: a peer-topeer information system based on the XOR metric. In Proc. of the 1st Intl. Workshop on Peer-to-Peer Systems (IPTPS’02), pp. 53–65. Springer-Verlag, USA (2002) Moreno-Vozmediano, R.: A hybrid mechanism for resource/service discovery in ad-hoc Grids. Future Gener. Comput. Syst. 25(7):717–727 (2009) Németh, Z., Sunderam, V.: Characterizing Grids: attributes, definitions, and formalisms. J. Grid Computing 1(1):9–23 (2003) Ng, T. S. E., Zhang, H.: Predicting internet network distance with coordinates-based approaches. In Proc. of the 21st Annual Joint Conf. of the IEEE Computer and Communications Societies (IEEE Infocom), pp. 170–179 (2002) Nielson, S. J., Crosby, S. A., Wallach, D. S.: A taxonomy of rational attacks. In Proc. of the 4th Intl. Workshop on Peer-to-Peer Systems (IPTPS’05), pp. 36–46. Springer-Verlag, USA (2005) Ramaswamy, L., Gedik, B., Liu, L.: A distributed approach to node clustering in decentralized peer-topeer networks. IEEE Trans. Parallel Distrib. Syst. 16(9):814–829 (2005) Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable Content-Addressable Network. 529 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. In Proc. of ACM SIGCOMM, pp. 161–172. ACM Press, USA (2001) Ratnasamy, S., Stoica, I., Shenker, S.: Routing algorithms for DHTs: some open questions. In Proc. of the 1st Intl. Workshop on Peer-to-Peer Systems (IPTPS’02), pp. 45–52. Springer-Verlag, USA (2002) Rhea, S., Godfrey, B., Karp, B., Kubiatowicz, J., Ratnasamy, S., Shenker, S., Stoica, I., Yu, H.: OpenDHT: a public DHT service and its uses. In Proc. of ACM SIGCOMM, pp. 73–84. ACM Press, USA (2005) Rodrigues, R., Blake, C.: When multi-hop peer-topeer lookup matters. In Proc. of the 3rd Intl. Workshop on Peer-to-Peer Systems (IPTPS’04), pp. 112–122. Springer-Verlag, USA (2004) Rowstron, A., Druschel, P.: Pastry: scalable, distributed object location and routing for large-scale peerto-peer systems. In Proc. of IFIP/ACM Intl. Conf. on Distributed Systems Platforms (Middleware 2001), pp. 329–350. Springer-Verlag, Germany (2001) Shneidman, J., Parkes, D. C.: Rationality and selfinterest in peer to peer networks. In Proc. of the 2nd Intl. Workshop on Peer-to-Peer Systems (IPTPS’03), pp. 139–148. Springer-Verlag, USA (2003) Spence, D., Harris, T.: XenoSearch: distributed resource discovery in the XenoServer open platform. In Proc. of the 12th IEEE International Symp. on High Performance Distributed Computing (HPDC-12), pp. 216–225. IEEE Computer Society Press, USA (2003) Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. In Proc. of ACM SIGCOMM, pp. 149–160. ACM Press, USA (2001) Wohlmacher, P., Pharow, P.: Applications in health care using public-key certificates and attribute certificates. In Proc. of the 16th Annual Computer Security Applications Conf. (ACSAC), pp. 128–137. IEEE Computer Society Press, USA (2000) Yang, B., Garcia-Molina, H.: Designing a super-peer network. In Proc. of the 19th Intl. Conf. on Data Engineering (ICDE 2003), pp. 49–61. IEEE Computer Society Press, India (2003) Zeinalipour-Yazti, D., Kalogeraki, V.: Structuring topologically aware overlay networks using domain names. Computer Networks: Int. J. Computing and Telecomm. 50(16):3064–3082 (2006) Zhu, C., Liu, Z., Zhang, W., Xiao, W., Xu, Z., Yang, D.: Decentralized Grid resource discovery based on resource information community. J. Grid Computing 2(3):261–277 (2004)

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement