Digital TruCiuster, Version 2 eXcursion, OM, Atom, Integrated Directory Services, Common Directory Interface Technical Journal
Below you will find brief information for TruCiuster, eXcursion Version 2, OM, Atom, Integrated Directory Services, Common Directory Interface.
Advertisement
Advertisement
Digital
Technical
Journal
I
DIGITAL UNIX CLUSTERS
OBJECT MODIFICATION TOOLS
EXCURSION FOR WINDOWS
OPERATING SYSTEMS
NETWORK DIRECTORY SERVICES
Volume 8
Number
1
1996
Editorial jane C. Blake, Managing Editor
Helen
L.
Patterson, Ediror
Kathleen M. Srcrson, Editor
Circulation
Catherine M. Phillips, Adrninisrraror
Dororhea B. Cassady, Secretary
Production
Terri Autieri, Production Editor
AnneS. Karzcff, Typographer
Peter R. Woodbury, Illustrator
Advisory Board
Samuel H. Fuller, Chairman
Richard W. Beane
Donald
Z.
Harbert
William R . Hawe
Richard
J.
Hollingsworth
William A. Laing
Richard F. Lary
Alan
G.
Nemeth
Pauline A. Nist
Robert M. Supnik
Cover Design
The
"bot" colors on our cover retlecr rhe kind of performance delivered by 64-bir
Digital UNIX TruCiuster s y stems
.
A four node cluster made up of AlphaServer 8400
5/350 systems interconnected with the high-speed MEMORY C
HAN
N E
L and running the Oracle Universal Server with
Oracle Parallel Server recently achieved record
TPC-C performance of
30,390 tprnC. The design of the Digital UNIX
TruCiuster system is the opening topic in this issue.
The cover was designed by Lucinda O'Neill of Digital's Design Group.
The Digital
Technical(ournal is a rekreed journal published quarterly by Digital
Equipment Corporation, 30 Porter Road
L)02/ D 10, Lirrleron, !'vlassachusetts 01460.
Subscriptions ro thejounw/ are
(non-U.S. $60) for
$40.00 tour issues and $75.00
(non-U.S.
S
115) for eight issues and must be prepaid in U.S. funds. University and colkge professors and Ph.D. students in the electrical engineering and computer science tields receive complimentary sub scriptions upon request. Orders, inquiries, and address changes should be scm to the
Oigita/ Tecbnical(ournal at rhe published by address.
I n q ui
( ies can also be sent elec rronically ro [email protected]. Single copies and b<Kk issues are available tor $16.00 e a c h by calling DECdirect at
( 1-800-344-4825
1-800-DIGITAL
).
Recent issues of the journal arc also available on the Internet at http:/ j www
.digital.com/into/dtj.
Complete Digirallnternet listings can be obtained by sending an electronic mail message to [email protected].
Digital employees may order subscriptions through Readers Choice by entering VIX
PROFILE at the sysrem prompt.
Comments on rbe content of any paper are wdcomed and may be sent to the managing editor at the published-by or n e rwork address.
Copyright© 1996 Digital Equipment
Corporation. Copying without fee is per mitted provided that such copies are made for use in educational institut.ions by f.1ctlity members and are nor distributed for com mercial advantage. Abstracting with credit of Digital Equipment Corporation's author ship is perm i ned.
The information in the journal is subject to change without notice and should not be construed as a commitment by Digital
Equipment Corporation or by the compa nies herein represented. Digital Equipment
Corporation assumes no responsibility for any errors that may appear in the journal.
ISSN 0898-901X
Documenration Number EY-U025E-TJ
Book production was done by Quanric
Communications, Inc.
The following are trademarks ofDigit:d
Equipment Cmpmarion: AlphaSen·er,
DECnet, DECsafe, Digital, the
DIGITAL logo, eXcursion, l'vlanagcWORKS,
Open
J'v!SCP,
VMS, PATHWORKS,
Tn1Ciuster, and VAXcluster.
Adobe is a registered trademark of Adobe
Systems Incorporated.
DCE,
OSF, and Motif ate registered trademarks and Open Sofi.ware
Foundation is a trademark of Open
Software Foundation, Inc.
Hewlerr-Packacd is a n-adernark of
Hewlett-PKkard Company.
Himala)'a and Tandem c\rc rcgisteccd trademarks of Tandem Computers, Inc.
Inrd is a trademark of Intel Corporation.
MEMO
R
Y CHANNEL is a trademark of
Encore Compmer Corporation .
Microsoft, Visual C++, vVin32, and
Windows 95 are r e g ist e red trademarks and Windows, Windows tor Workgroups, and Windows
NT are tradem<lrks of
Microsoft Corporation.
NetWare and Novell are registered trademarks of Novell, Inc.
POSIX is a cegistered trademark of The
Institute of Electrical and Electronics
Engineers, Inc.
Oracle? is a trademark of Oracle
Corporation.
S3 is a registered trademark of S3
I ncorporared.
Sequent is a trademark of Sequent
Computer Systems, Inc.
SPEC is a registered trademark of the
Standard Performance Evaluation
Corporation.
StreetTalk is a tcadernark of Banyan
Systems, Inc.
Sun Microsystems is a registered trade mark of Sun Microsysrems, Inc.
TPC-C is a trademark of the Transaction
Processing Performance Council.
UNIX is a registered trademark in the
United Stares and other countries, licensed exclusively through X/Opcn Company Ltd.
X vVindow System is a trademark of the
Massachusetts Institute ofTechnology. http:/ j www
_digital.com/info/dtj
Contents
Foreword Don Harbert
3
DIG ITAL UNIX CLUSTERS
Design of the TruCiuster Multicomputer System for the Digital UNIX Environment
OBJECT MODIFICATION TOOLS
Delivering Binary Object Modification Tools for
Program Analysis and O ptimization
EXCURSION FOR WIN DOWS OPERATING SYSTEMS
Design of eXcu rsion Version 2 for Windows,
Windows NT, and Windows 95
N ETWORK DIRECTORY SERVICES
Integrating Multiple Directory Services
Design of the Common Directory Interface for DECnet/OSI
Wayne M. Cardoza, FrederickS. Glover, and
William E. Snaman., Jr.
Linda S. Wilson, Cr;1igA. Ncth, and
Michael]. Rickabaugh
John T. Freitas, James
G.
Peterson, Scot A. Aurenz,
Charles
P.
Guldenschuh, and r�uil
J .
Ranauro
Margaret Olson,
Laur;l
E. Holly, <lnd Colin Srrurr
Richard L. Rosenbaum <llld Stanley l. Goldf1rb
5
1 8
32
46
59
Digiral Technical )ou,·nJI
Vol.
8
'o. l 1996
Editor's
Introduction
2
Digital recently announced record breaking 30,390 tpmC performance on a Digital UNIX cluster of64-bit
RlSC AlphaServer systems. In rl1is issue, engineers from the UNIX team describe the key technologies that enable these near supercomputer performance levels as well as provide the cluster characteristics of high availability and scalability. Also pre sented in this issue are advanced
UNIX programming tools for maxi mizing performance, X server soft ware that supports the Microsoft f:1mily of operating systems, and new network directory services that sim pli�r management.
First defined by DigitaJ in the early
1980s, clusters are highly available, scalable multicomputer systems built with standard parts and oftering tJ1e advantages of single-computer systems.
Wayne Cardoza, Fred Glover, and
Sandy Snaman compare clusters with other types of multicomputer config urations and describe the major com ponents of Digital's newest cluster implementation, TruCiuster systems, for the 64-bit UNIX environment.
The cluster interconnect, called
MEMORY CHANNEL, is critical to the cluster's outstanding performance.
MEMORY CHANNEL implements clusterwide virtual shared memory and reduces overhead and latency by two to three orders of magnitude over conventional interconnects.
Also developed for the DigitaJ
UNIX environment (version 4.0) are two program analysis and optimiza tion tools-OM and Atom. The tool technology originated in Digital's
Western Research L1boratory, where researchers focused on providing performance diagnosis and improve ments tor large customer applications.
Software developers Linda Wilson,
Craig Neth, and Mike Rickabaugh from ilie UNIX Development Envi ronment Group describe the object modification tools and tl1e flexibility they provide over traditional tools that are implemented in the reaJm of compilers. In addition to demonstrat ing practical application of the tools, the authors examine the process of transferring technology from research to development.
For mixed operating system environments, Digital developed
Windows-based X server software, caJied eXcursion, to allow the win dows of a remote host running UNIX or Open VMS to display on a desk top running the Microsoft Windows operating system. The latest version of eXcursion, described here by John
Freitas, Jim Peterson, Scot Aurenz,
Chuck Guldenschuh, and Paul
Ranauro, is whoJJy rewritten to maxi mize graphics pertormance and to support the nil I range ofWindows plattonns: Windows, Windows 95, and Windows NT. Tbis new version is based on the X Window System version 11, release 6 protocol from the
X
Consortium.
Two network directory services that reduce complexity and increase choices tor network managers are the subjects of our next papers. The first is designed for multiple networked environments; Integrated Directory
Services (IDS) software integrates multiple services into one directory service-independent system. Margaret
Olson, Laura Holly, and Colin Strutt outline the problems that have lim ited the use of directory services and the different design approaches the team considered to simplit)r directory services use and make it more attrac tive. They then describe the IDS extensible, object-based framework, which comprises an application programming interface and a ser vice provider interface. Next, Rich
Rosenbaum and Stan Goldfarb present the Common Directory
Interface (CD!) for DECnet/OSI.
Implemented as shared libraries in the DigitaJ UNIX and Open VMS operating systems, COl is designed to give network managers a choice of directory services. The autl10rs describe the libraries and the registra tion tool set of management opera tions that is layered on a specialized
API.
Coming up in the}ourna/are papers about a new log-structured clusterwide file system caJJed SpiraJog, the 64-bit Open VMS operating system, speech recognition software, and the UNIX clusters message passing system and its use for pro gram parallelization.
Jane C. Blake
Managing Editor
Digital Technical Journal Vol. 8 No. l 1996
Foreword
Don Harbert
\lice
President.
UNIX
IJusiness
Digital not only invented dusters bur continues to set rhe standard by which all other cluster systems are measured. The VAXcluster success and that of Digital's latest UNIX clus ter systems derive ri·om superb engi neering that builds on the system definition put forth in the early 1980s by the VAX engineering team: an available, extensible, h.igh-pertormanu: multicomputer system built fi-om standard processors and a general purpose operating system, with char acteristics of borh loosely and tightly coupled systems.'
We in the UNIX community arc proud of our VAXcluster heritage and have engineered our products ro provide the same kinds of benefits to customers that V AXcluster systems provide.1 In the opening paper for this issue of the
journal,
members oftbe Digital UNIX engineering ream describe the multicomputer system for the Digital UNIX environ ment, called TruCiuster, wh.ich, like the VAXcluster system, is designed for high availability, scalability, and pertormance.
The technology, of course, is dif ferent, and the environment is open.
The fundamental concepts are never theless the same. The TruCiuster system is a loosely coupled, general purpose system connected by a high-performance interco!Ulect. It maintains a single security domain and is managed as a single system.
•
Nancv
P.
Kronenberg,
Hcmv lVI. Lew, and William
D.
Strecker, "VAXclusn:rs:
A Closely-Coupled Distributed
System,"
11011
Trausactions ou Oimputer.\)·ste/1/s, vol.4,.no.
2
(l'vby 1986): 130-146. t Digital has renamed VAXclustcr svstcms to
OpcnV,viS
Clustn svsrems.
Cluster services remain available even when other members are unavaiL!ble.
Like VAXcluster systems, TruCiuster systems implement a distributed lock manager, which provides synchro nization tor a highly parallelized distributed database system. The teclmology for the lock manager, however, is newly implemented tor the UNIX environment. Also com pletely new is the interconnect tech nology tor TruCiuster systems.
MEMORY
CHANNEL is a reliable, high-speed interconnect based on a design by Digital partner Encore
Computer Corporation. MEMOR.Y
CHAN EL addresses the unique needs of clusters by implementing clusterwide virrual shared memory; the interconnect reduces overhead and latency by two to three orders of magnitude.1 Because MEMORY
CHANNEL uses the industry standard PCI, designers can imple ment the network at very low cost.
We believe this interconnect tech nology puts Digital years ahead of the competition.
The TruCiuster system is rhe latest example of Digital's intent to remain a technology leader in the UNIX market.
We began by developing the first high-perf()rmance, 64-bit general-purpose operating system,
DEC
OSF
/l,
shipping in March
1993. The first Digital UNIX cluster release, DEC:sak Available Server
Environment, followed soon there after in April 1994. The announce ment in April 1996 ofTruCiustcr systems with MEMORY CHA
1 EL
:j: Richal'cl
B.
Gilkrr, "Memory
ChJtlllcl
Network f(>r PC: I," 1/:l:F :llicro
(february l':I<J6): 12-IR.
Digital Technical )ound
Vol. R
No. I 19':16
3
ag:tin places Digit:�! E1r ahead of the competition technologicallv. The pcrti:m11:tnce of these :waibble cluster systems now approaches that of very expensive supercomputers. System pcrti:m1unce has b..:en me:tsured at the record-breaking: rare of 30,390 tpmC 011 tour Alph:tScrver 8400 systems running Digital UNIX and th<.: Oracle Univns:tl Snver with
Or�Kie Parallel Server. The previous perti:>rmance record, 20,918 rpmC, was held by the proprietary Tandem
Him:tbya K10000-!12; Digital's open system cluster pertormance
1·ecord is 1.5 rimes the Tandem perfi:mnance record at one third the svsrem cost. ror Digital, clusters of high perti:mnance 64-bir systems are to
;1 great extent at the heart of irs commercial and technic:tl server smtet,'V. Digital UNIX has been de tined and engineered tor the ser ver business, specitically, tor the high perti:mll:tnce commerci:tl and large probkm/scientitic e11\'ironmcnr. To be successful in the open svstem m:tr ker, however, a company must reach outside itself to jointly engineer prod ucts with leading sortw;ll"C suppliers th:tt have rhe sofuv:tre customers need ro be comperiti\·c. Therefore, the tirst TruCiusrer implementation is designed with Digit:�l's partners m:ljor sottware comp<mics-to meet the requirements ti:>r high pcrtormance and tl111ctionalirv in the commerci:�l chrab:1se server marker.
The competitive challenge now is ro m:�inrain Digital's signitic:�nt lead in pro\·iding outstanding cluster per ti:mlwKe, a\·ail:�bilirv, and ;lft(xcbbil iry. hom :1 rechnologiul perspective, the imm<.:diate and achicv:�ble goal is to increase the number of cluster nodes J-i-om 4 to 10 or 20 nodes.
Within this r:�ng<.:, Digital maintains a simple cluster system model that otters the perf(mnance advJ!lt;lges of c.lustcring and :�voids the disadvan t:�ges, such :ts the ma.nagcmellt l�rob lems and qualiticarion healbches, of more compkx topologies. Further, the Digital UNIX organization will fOCllS Oil a new cJ LISter ti Je S\'Stem, configuration tloibilirv, man:1gement tools, and J cluster :�lias that :�I lows a single-system view tor clients and peers. The overall goal of this work is to e\·olvc row:�rd J more general computing ell\·ironment.
The kinds of tools rh:n both sim pli�' and enha1Ke pertormance arc exemplitied by the program ;l!Jalysis and optimization tools prcscmcd in this issue. Built on Digit:� I UNIX l'ersion 4.0 :tnd announced in April, these tools help software developers cxtr:tct maximum perti:.mnancc ti·om the system. The story of the tools dcvelopmcm is <lll excellent c:omple ofrhe direct application of research to products. The power ofrhc OM object moditicttion tool <lnd the analy sis tool with object modification
(Atom) \\\lS recognized bv d..:velopcrs e\·en :�s resc:trch progressed; in t:Kr, semiconductor designers developed
Atom tools to evaluate new Alpha chip implementations. The result of this close cooperation between research and dewlopmenr is ,llkmced programming tools tor customers.
These efti:>rts in the
UNIX org:�ni z:�tion arc manitestarions of Digital's commitment to open system�. Other areas of engineering where this com mirmem is app:trenr are also repre sented in this issue. For example, eXcursion sottware is kcv ro integra tion between Microsoft's Windows t:11nilv of products :�nd Digit:�l's
UNIX <llld OpcnViVIS products.
This whollv revised version both adds new fimction;llirv and conserves system resources. Another major area of strength tor Digital is irs networks products. Networks engineers describe two cxampks of network services that incre:�sc users' choices and extend system tl1ncrionaJirl', i.e., the Imegratcd Directory Scn·iccs
(IDS) and the Common Directory
Interbce.
DigitJI's srr:�regv is to continue to engineer products th:tr pro\·ide out stJnding performance and pric..:/ pcrti:mlla!Ke in open environments.
In all areas of engineering-systems, services, networking-our goal i� to set the standard lw which all others
:�rc measured.
4
Dig;iral Tcchnictl )ound
V"l. 8 No. I
1996
I
Wayne
M.
Cardoza
FrederickS. Glover
William E. Snaman, Jr.
Design of the TruCiuster
Multicomputer System for the Digital UNIX
Environment
The TruCiuster product from Dig ital provides an available and scalable multicomputer sys tem for the UNIX environment. Although it was designed for general-purpose computing, the first i mplementation is directed at the needs of large database applications. Services such as distri buted locking, fai lover ma nagement, and remote storage access a re layered on a high-speed cl uster i nterconnect. The initial implementation uses the M E MORY CHANNEL, an extremely relia ble, hig h-performance i nter connect specially designed by Digital for the cl uster system .
The primary goal for the first release of the TruClustcr system for the Digital UNIX ope r a ting svsn.:m was to develop J high-pertorm;�nce commen:ial database server environment running on a cluster of several nodes. Database applications oti:en require comput in g power and I/0 connectivity and bandwidth greater than that provided b�· most single systems. In addi tion, Jvaibbilitv is a kev require me nt tor enterp r ises that are dependent on datJbase services for no rm a l operations. These req uirements led us to implement a cluster of computers tllJt cooperate to provide services but fail independently. Thus, both performance and
Jvailability are addressed.
We chose an industry-st:mdard benchm;�rk to gauge our success in meeting pert(>rmance g o als
.
The
Transaction Processing Performance Council TPC-C benchmark is a widely accepted measurement of the capabi l it:v of large servers. Our goal was to achieve industry-leading numbers in excess of 30,000 t r a n sac tions per minute (tp m C) with a four-node TruCluster system.
The TruCiuster vers i on 1.0 product provides reliable, shared access to large amounts of storage, distributed synchr o niz
�u
i on k>r applications, efticient cluster communication, and application f:1ilover. The focus on database servers docs not mean that the
TruCJuster system is not suitable for other applica tions, but that the inC\·itJble design decisions and trade-offs tor tbe first product were made with this goal in mind .
Although other aspects of providing a singl e -syste m view of'' cluster are important, they are secondary objectives and will be phased into the product over time.
This paper begins wirh a brief comparison of com puter systems and presents the advantages of clustered computing. Next, it introduces the TruCiuster prod uct and describes the design of its key software compo nents and their relationship to database applications.
The paper then discusses the design of the MEMORY
CHANNEL interconnect for cluster systems, along with the design of the low-level software toundation for cluster synchronization and communication.
Finally, it addresses application tailover and co n f
i
g u r at i ons
. hardware
Digital
Tcdmic�l journal
Vol. 8 No.
I 1996
5
Brief Comparison of Computing Systems
Contemporary computing systems evolved ti·om centralized, single-node time-sharing systems imo sev eral distinct styles of multinode computer systems.
Single-node systems provided uniform accessibility to resources and services and a single-management donuin. They were limited with respect to scalability, however, and system tailures usuaUy resulted in a com plete loss of service to clients of the system.
Multinode computer systems include symmetric multiprocessing (SMP) systems and massively parallel processors (MPPs). They also include network-based computing systems such as the Open Software
Foundation Distributed Computing Environment
(OSF DCE), Sun Microsystems Inc.'s Open Network
Computing
(ONC), and workstation brrnsu Each of these systems addresses one or more of the bendits associated with clustered computing.
SMP configurcltions provide for tightly coupled, high-performance resource sharing. In their effective range, SMP S)'Stems provide the highest-performance single-system product for shared-resource applicJ tions. Outside that range, however, both hc1rdware and software costs increase rapidly as more processors are added to an SMP system. In addition, SMP avail ability characteristics are more closely associated with those ofsingle systems because an SMP svstem, by dd� inition, is composed of multiple processors but not multiple memories or
I/0 subsystems.
MPP systems such as the Intel Paragon series were developed to support complex, high-performance parallel applications using systems designed with hun dreds of processors. The indi,·idual processors of an
M
PP system were typically assigned to specific t:tsks, resulting in fairly special-purpose machines.
The DCE and ONC technologies provide support for common naming and :tccess capabilities, user account management, authentication, and the replica tion of certain services tor improved availability.
Workstation farms such as the Warson Research Cen tral Computer Cluster deliver support for the parallel execution of applications within multiple computer environments typicallv constructed using offthe-shelf software and hardware.' ONC, DCE, and f:mns pro vide their services and tools in support of heteroge neous, multivendor computing environments with hundreds of nodes. They are, however, much further away from realizing the benefits of a single-system view associated with clustered computing.
In the continuum of multi node computer systems, the advantage of the cluster system is its �1bility to provide the single-svstcm ,·iew and ease of manage ment associated with SMP systems and at the same time supply the E1ilure isolation and sca!Jbility of dis tributed systems.
Cluster systems have clear advantages over large scale parallel systems on one side and heterogeneous distributed systems on the other side. Cluster systems provide many cost and availability advantages over large parallel systems. They are built of standard build ing blocks with no unusual packaging or interconnect requirernenrs. Their I/0 bandwidth and storage con nectivity scJic well with standard components. They are inherently more tolerant of Eli lures due to looser coupling. Parallel or multiprocessor systems should be thought of as cluster components, nor as cluster replacements.
Cluster systems have a different set of advantages over distributed systems. First they are homogeneous in nature and more limited in size. Cluster systems can be more efficient when operating in more constrained environments. Data formats arc known; there is a single-security dom:1in; failure detection is certain; and topologies are constrained. Cluster systems also are likely to have interconnect pertorm:1ncc advantages.
Protocols are more specialized; interconnect charac teristics are more uniform; and high performance can be guaranteed. finally, the vendor-specific nature of cluster systems Jllows them to evolve Elster than het erogeneous distributed systems cllld will probably always allow them to have advJntJges.
There ln.: numerous examples of general-purpose clusters supplied by most computer vendors, including
AT&T, Digital, Hewlett-Packard, Imcrnational Busi ness 1\tbchines Corporation, Sequent Computer Sys tems, Sun Microsysrems, and Tandem Computers.
DigitJJ's Open VMS cluster system is generally accepted as the most complete cluster product oftering in the industrv, and it lchieves many of the single-svsrem m:magemcnt Jttributes' Much of the functionality of the OpenVMS cluster system is retained in Digital's
TruCiuster product offerings.
Structure of the TruCiuster System
Digital's TruCluster multicomputer system is :1 highly avJiL1ble and scal<lble structure of
UNIX servers that preserves mJny of the benefits of a centralized, single computer system. The TruCluster product is a collec tion of looselv coupled, general-purpose computer systems connected by a high-perfornlJnce intercon nect. It maintains J single security domain Jnd is ma.n
<lged as a single system. Each cluster node may be a uniprocessor or a multiprocessor system executing the Digital UNIX operating system. figure l shows a typical cluster configuration.
Each cluster member is isolated fi·om sofrware and hardware bults occurring on other cluster members.
Thus, the TruCluster system does not hcwe the tightly coupled, "bil together" characteristics of multiproces sor systems. Cluster services remain availab.le even
"vben individual cluster members are temporarily
6
Di�;iral Technical )ollrnal Vol. 8
No. I 1096
DISKS
Figure 1
Contiguration of a four node Cluster System unavailable . Other important availability objectives of the TruCluster server include quick detection of com ponent and member failures, on-line reconfigurarions to accommodate the loss of a fai led component, and cont inued service while sate operation is possible.
The TruCiusrer prod uct supports large, highly available database systems through several of its key components. First, the distri buted remote disk ( DRD ) facility provides reliable, transparent remote access to all cluster storage from any cluster node. Next, the d is tributed lock manager
( D LM )
enables the elements of a distributed database system to synchronize activity on i ndependent cluster nodes. Finally, elements of
Digi tal's D ECsafe Available Server Environment
( AS E )
provide application tailover.5 In support of all controls cluster membership and the transition of nodes in and out of the cl uster. Figure 2 is a block dia gram showing the relationships between components.
Each major component is described in the remain der of this paper. I n addition, we describe the high performance
M E M O RY
CHANNEL interconnect that was designed specifically tor the needs of cluster systems.
Distributed Remote Disk Subsystem
The distributed remote disk
( DRD) su bsystem was developed to support database appUcations by present ing a cl usterwide view of disks accessed through the character or raw device i nterface. The Oracle Parallel
Server
( OPS ) , which is a parallelized version of the
Oracle database technology, uses the DRD subsystem .
The DRD su bsystem provides a cl usterwide name space and access mechanism tor both physical and loa ical (logical storage manager or
LS M )
� olumes. Tl
� e
LSM logical device may be a concatenated, a striped , or a mirrored volume. D RD devices arc Jccessible ti-om any cluster member using the DRD device name.
This location independence allows databJse software to treat storage as a uniformly accessible duster resource and to easily load balance or tai l over activity between cluster nodes.
Cluster Storage Background through the U N I X tile system and an associ
� ted block device special til e . A disk device mav also be accessed through a character device special t ile or r:1w device that provides a direct, unstructured interface to the device and bypasses the block buffer cache.
Database mJnagement systems :1nd some other high-performance U N I X applications Jre often designed to tJke advantage of the character device spe cial file interbces to improve pertorm:1nce by avoiding additional code path length associated with the tile svs tem cache -'' ' The
I/0 profil e of these systems is ch acterized by large files, random acce
�
� r s to records, private datJ caches, and concurrent read -write sharing.
Overall Design of the DRD
The DRD su bsystem consists of four primary compo nents. The remote raw disk ( RRD ) pseudo-d river red i rects DIU) Jccess requests to the cluster member serving the storage device. The server is identified by information maintained in the DRD device data base
( RRDB
)
. Req uests to Jccess local
DRD devices are passed through to local device drivers. The block ship ping client ( BSC) sends requests for :1cccss to remote
DRD devices to the appropriate DIU) server and returns responses to the cal ler. The block shipping server (BSS) accepts requests ti·om
BSC clients, passes them to i ts locJI driver tor service, :1nd returns the results to the calling ESC client. Figure 3 shows the components of the DRD subsystem.
The D RD management component supports D RD device naming, device creation a n d deletion, device relocation, and device status requests. During the
DRD device creation process, the special device file designating the
D RD device is created on each cluster mem ber. In Jddi tion, the DRD device num ber, its cor responding physicJl device number, the network address of the serving duster member, and other con figuration parameters are passed to the DRD driver, which upd:1tes its local database Jnd communicates the information to other cluster members. The D RD driver may b e queried for device status and D RD data base information .
Clusterwide Disk Access Model
D uring the design of the
DRD ered both shared (multiported) and s
� rved
� iisk models.
A multiported d isk configuration provides good fai lure recovery and load balancing characteristics. On the
Digital Technical }ournal
Vol . 8 No. 1
1 996
7
8 r - - - - - - - - - - - - - - - - - -
I
NODE A
I
I r---
I
DATABASE INSTANCE t
I I
.....--:-t ----.
LOCK
I
MANAGER
INTERFACE r-----------t---+--+·
STANDBY
CONFIGURATION
MANAGER
�����
AVAILABI LITY
MANAGER
I
RAW DISK
INTERFACE
ER
I
I t'-------t----11--
1
I
COM
� l
I
I
I
I
I
II
I
I
I
I
I
, - - - - - - - - - - - - - - 1
I
NODE B
I
DATABASE INSTANCE
I
I
I
I
I
1 0 1
I
II
I
I
LOCK
MANAGER
I NTERFACE
DECSAFE ASE AVAILABILITY SERVICES
DISTRI BUTED LOCK MANAGER
I
DISTRIBUTED DIRECTORY SERVICE
I
I
RESOURCE MANAGER
I I
RESOURCE MANAGER
I
--
-+-1
!
DISTRIB UTED REMOTE DEVICE
I
'
.
RAW DISK
INTERFACE t
ACTIVE
I
CONFIGU RATION
MANAGER
I
..J
CONNECTION
MANAGER
AGENT l
AVAILABILITY
MANAGER
I
I
I + I
I
I
I
I
I
I
I
LOCAC
I
ORO
1/0
REDI RECTION
'"'cc cmvms t
I I
ORO BLOCK
SHIPPING
SERVER
ORO BLOCK
SHI PPING
CLIENT
MCMORV CHANNeL
SERVICES
1
I
I
I
I
:
DATA
MOVER
I I
:
I
I
ORO BLOCK
SHIPPING
CLI E NT
I
I
ORO BLOCK
SHI PPING
SERVER r
MCMORV CHANNEL
SERVICES
DAD
1/0
REDI RECTION
J
I
I I
LOCAL ""cE CR'"Rs
- - - l - - - - - - - ��-�" ;;;;���,"�� - - - - - -r - - -
J
I
I
II
MEMORY CHANNEL BUS ll
NODE C NODE 0
I
Figure 2
Software Componen ts
I
I
I
I
I
,
- - - - - - - - - - - -
ORO CLI ENT
- ,
I
I
I
I
I
I
DEVIC E DRIVER
I
L - - - - - - - - - - - - - - -
L - - - - - - - - - - -
Figure 3
D istri buted Remote D isk S u bsystem
Vol. 8 No. 1 1 996
other hand, !jO bus contention and hardware q ueuing delays from fu lly connected, sh:1red disk configurations can limit scalability. In addition, present standard I/0 bus technologies l imit configuration dist:mces.s As 3 consequence, we selected a served disk model for the
DIU) implementation. With this model, software queuing ;1lleviates the bus contention ;md bus queuing delays. This approJch provides improved scalabiljty Jnd fau l t isolation ;JS well as flexible storage configura tions 9·10 Full connectivity is not required, and extended machine room cluster configurations can be con structed usi ng standard networks ;1nd !jO buses.
The DlU) implementation supports cl usterwide access to DRD devices using a software- based emula tion of a fully con nected disk configuration . Each device is assigned to a single cluster mem ber at a time .
The member registers the device into the cluster wide namespacc and serves the device data to other cl uster members. �Jilure recoverv and load-balancing support are included with the DlU) device i mpJemcn tJtion. The bilure of a node or con trol ler is transpar ently masked when another node connected to the shared bus tJkcs over serving the d isk. As an option, au tomatic lo3d bJiancing can move service of the d isk ro the node generating the most requests.
In the Tru Ci ustcr version 1 .0 prod uct, data is transferred between req uesting and serving cl uster mem bers using the high - bandwidth, low-latency
MEMORY C H A N N E L i nterconnect, which a lso sup ports direct me mory access ( D MA) between the !jO ad3pter of the serving node and the m;lin memory of the req uesting node. The overall cluster design, how ever, is not dependent on the MEtVIO R.Y CHAN NEL intercon nect, :md alternative cl uster i nterconnects will be supported in future software rcle:�scs.
DRD Naming
The DigitJI U N I X operatjng system presently supports character device special file names f(>r both physical disk devices and LSM logical volumes and maintains a sepa rate device namespace for each. An important D lU) design objective was to develop a cl usterwide naming scheme integrating the physical and logical devices within the Dlill mmespace. We considered defining
J new, single mmespace to support all cluster disk devices. Our research, however, revealed plans to intro duce significant changes into the physic:1l device nam ing scheme i n a fu ture base system release and the compl ications of l icensing the logical disk technology ti-om a third parrv that maintains control over the logi cal volume namespace. These issues resulted i n deter ring a true cl usrerwide device namespace .
As an interim approach , we chose ro create a sepa rate, clusterwide DRD device namespace layered on the existing physical and logical device naming schemes. TrJnslations from D IU) device names i nto the underlying physical and logical devices are main tained by the DRD device mapping database on each cluster node. DRD device "services" are created by the cluster administrator using the service registL1tion facil ity. 1 1 E:1ch "add Service'' mamgement oper3tion generates a unique service number that is used in constructing the Dlill device special file name.
This operation also creates the new D RD device special ti le on each d uster member. A trad itional
UNIX-device-naming convention results i n the ere arion of D RD special device fi le n;Jmes in the form of
/dev/drd/d rd (service number f . ' 2
DRD Relocation and Failover
ASE fai lover (see the discussion in the section
Application Fai lover) is used to support DRD bilover and is fu l l y integrated within the cl uster prod uct. The device relocation policv defined d u ring the creation of a D RD device indicates whether the device may be reassigned to 3norher cluster member as a result of a node or controller fail ure or a load - balancing opera tio n . In the even t of a cl uster member fail u re, DRD devices exported by the failed member arc reassigned to an alternate server attached to the same shared ljO bus. During reassignment, the DRD device databases are updated on all cl uster mem bers and DRD ljO operations Jre resumed. Cl uster device services may also be reassigned d u ring a pl:mned relocation, such as for load babncing or mem ber removal. Any D RD operation in progress during a re location triggered by a fai l u re will be retried based upon rhc registered D RD retry pol icy. The retry mechanism must revalidate rhe database trJnslation map for the target D RD device because the server binding may have been modified .
Failover is thus transparent to database applications and allows them to ignore configu ration changes.
Several challenges result ti·om the support of m u ltiported disk configurations under various fai l u re scenarios. One of the more difficult problems is distin guishing a fjilcd member from a busy member or a communication bult. The ASE failover mechanism was designed to maintain data i nregriry d uring service tai lover, and to ensure that subsequent disk oper;Jtions are not honored ti-om a member that has been declared
"down " by rhe remai ning cluster members. This ASE mechanism, which makes usc of small computer sys tems interface (SCSI) target mode and device reserva tion, was integrated into the Tru Ci uster version 1 .0 product and supports the DRD service guarantees.
Other challenges relate to preserving seriali zation guarantees in the case of cluster member fai lure.
Consider a parallel application that uses locks to serial ize access to shared DRD devices. Suppose the applica tion is holding a write lock for 3 given data block and
Digir31 T<:chnical Journal Vol. 8 No. J 1996
9
issues an update for that block. Before the update operation is acknowledged, however, the local mem ber fai ls. The distri buted lock manager, which wi l l have been notified o f the member failure, then rakes action to release the lock. A second cooperating appli cation execu ting on another cluster member now acquires the write Joc k for that same data block and issues an update tor that block. If the fail ure had not occurred, the second appl ication would have had to wait to acquire a write lock for the data block unt i l the first application released the lock, presu mably after its write request h ad completed. This same serialization must be mai ntained during failure cond itions. Thus, it is imperative that the write issued by the first (now fai led) application partner not be applied after the write issued by the second application, even in the presence of a timing or net\vork retransmission Jnom aly that delays this first write.
To avoid the reordering scenario just described, we employed a solution cal led a sequence barrier in which the connection manager increments a sequence number each rime it completes a recovery transition rlut results in rcle:1sed locks. The seq uence number is comm u nicated to each D RD server, which uses the sequence n u mber as a barrier to prevent apply ing stale writes. This is similar to the im mediate com mand feature of the Mass Storage Control Protocol
( MSCP) used by Open VMS cluster systems to provide similar guarantees. Note that no application changes arc required.
As another example, client retransmissions of
DRD protocol requ ests that are nor id empotent em cause serious consistency problems. Request transac tion IDs and DRD server dupl icate transaction caches are employed to :woid undesirable efkcrs of client generated retransmissions . " '
Cluster member fa ilu res are mostly transparent to applications executing on client member systems.
Nondistributed applications may fail, but they can be
Jutomatiea l ly restarted by ASE facilities. DlU) devices exported by a serving member become unavailable for
�1 small amou nt of rime when the member tails. C luster f::�ilover activities that must occur before the DRD service is again available include detecting and veri�' i ng the member fai lure, purging the disk device SCSI hardware reservation, assigning an alrern::�re server, establishing the new reservation, and bringing the device back on - l i ne . A database application serving data from the DRD device at the time of the fai l ure may also have registered to have a restart script with a recovery phase executed prior to the restart of the database appl ication. A possible lack of transp:�rency may result if some client applications are not designed to accommodate this period of inaccessible DRD ser vice. The DRD retry request poli cy is configurable to accommodate applications interacting directly with a D RD device.
1 0
Digital Tcdm ical )ourn01 l
Vol . S No. 1 1 996
Distri buted Lock Manager
The distributed lock manager ( DLM ) provides syn chronization services appropriate for a highly parai
J.elized distri bu ted database system . Databases can use locks to control access to distributed copies of data buffers (caches) or to li mit concurrent access to shared disk devices such as those provided bv the DRD sub system. Locks can a lso be used tor controlling applica tion i nstance start-up and for detecting application i nstance fail u res. In addition, appl ications can use the locking services for their other synchronization needs.
Even though this is a completely new implementa tion , the lock manager borrows from the original design and concepts i ntroduced i n 1984 with the
VAXcl uster d istributed lock manager. '' These concepts were used in several recen t lock man:1.ger implementa tions for
UNIX by other vendors. In addition, the
Oracle P::trallel Server uses a locking :1pplication pro gramming i merface (API) that is conceptually similar to that oHcred here.
Usage of the DLM
The lock manager provides an API for request ing, releasing, and altering locks. "·'" These locks are requested on abstract names chosen by the applica tion . The names represent resources and may be orga nized in a hierarchy. When a process requests a lock on a resource, that request is either granted or denied based on examination of Jocks already granted on the resource. Cooperating components of an application use this service to achieve mutually exclusive resource usage . In addition, a mode associated with each lock request al lows tradi tional levels of sharing such as mul tiple readers excluding aU writers .
The API provides optional asynchronous request completion to allow queuing requests or overlapping mu ltiple operations for i ncreased performance.
Queuing prevents retry delays, elimin:�tes pol ling overhead, and provides a first in, first our ( FI FO) fair ness mechanism . In addition, asynchronous requ ests can be used as the basis of a signaling mechanism to detect component failures in a distri buted system . One component acquires an exclusive lock on a n amed resource. Other components queue incomparjble requests with asynchronous completion specified . If the lock holder fails or otherwise releases its Jock, the w:�iring requests are granted . This usage is sometimes referred to as a "dead man" lock.17
A process can request noti fication when a lock it holds is blocking another request. This allows elimina tion of many lock ca.lls by effectively cac hing Jocks.
When resource conten tion is low, a lock is acquired and held u mil :lllother process is blocked by that lock.
Upon receiving blocking noti fication, the lock can be released . When resou rce contention is high , the lock is acq uired and released immediately. I n addition, this
noti fication mechanism can be used as the basis of a ge neral signaling mec hanism . One component of the
3pplication acq uires an exclusive lock on a named resource with blocking noti fication specified. Other components then acquire incompatible locks on that resource, thus triggering the blocking notification.
This usage is known as a "doorbe ll" lock.'7
The D LM is often used to coordi nJte access to resources such as 3 distributed cache of database blocks. M ultiple copies of the data are held under compatible l ocks to permi t re:.1d but not write access.
When
:.1 writer wants an incompatible lock, readers are notified to downgrade their locks and the writer is granted the lock. The writer modifies the data before downgrading i ts lock. The reader's lock requests are again granted, and the reader tetches the latest copy of the data. A value block can also be associated with each resource. I ts value is obtained when a lock is granted and can be changed when cerrain locks are released .
The value block can be used to communicate any use ti.d information, i nduct ing the latest version n u mber of cached data protected by the resource .
Design Goals of the DLM
The overall design goa l of the lock manager was to provide services tor highly scalable database systems.
Thus correctness, robustness, scJ iing, a nd speed were the overrid i ng subgoals of the project.
Cardi.1l attention to design details, rigorous testing, internal consistency checking, and years of experience working with the VMS distri buted lock manager h ave
�11 1 contributed to ensuring the correctness of the implementation for the Digital U NI X system. Because the lock manager provides guarantees about the state of :1l l locks when either a lock holder or the node upon which it is running fails, it em ensure the i nternal lock state is consistent as far as surviving lock holders are concerned. This robustness permits the design of applications that can continue operation when a clus ter node f:1ils or is removed �or scheduled service . The choice of a kernel-based service :.1nd the use of a mes sage protocol also contri bute to robustness as dis cussed below.
I n terms of performance and scaling, the lock man
Jger is designed for minimal overhead to its users. The kernel-based service design provides h igh perfor mance by eliminating the context switch overhead
JssociJted with server daemons. The lock manager uses the kernel -locking features of the Digital
UNIX operating system for good scaling on SMP systems. A kernel- based service as opposed to a li brary also all ows the Jock manager to make strong guarantees about the internal consistency state of locks when a lock-holding process fails.
The message protocol contributes to cl uster scaling and performance through a scaling property that mai ntains a constant cost as nodes are added to the d uster. '< The message protocol also provides suffi ciently loose coupling to allow the lock manager to maintain internal lock state when a node fails. The use of messages controls the amou nt of i nternal state visi ble to other nodes and provides natural checkpoints, which limit the damage resulting ti·om the failure of
J cluster node.
DLM Communication Services
The D LM session service is a commun ication layer that takes advantage of M EMO RY CHANNEL fea tures such as guaranteed ordering, low error rate, and low latency. These features allow the protocol to be very simple with an associated reduction in CPU over head . The service provides connection establishment, delivery and order guarantees, and bufter manage ment. The connection manager uses the communi cation service to establish a c lunnel for the lock manager. The Jock manager uses the communi cation services to com municate between nodes. Because tl1c service h ides the details of the communication mecha n ism, Jlternative intercon nects can be used without changes to the lock manager's core routjnes.
The use of the MEMORY
CHANN EL i n terconnect provides a very low latency com munication path for small messages. This is ideal for the lock manager since lock messages tend to be very small and the users of the lock manager are sensi tive to latency since they wai t for the lock to be granted betore proceeding.
Small messages are sent by simply writing them into the receiving node's memory space. No other com munication setup needs to be performed . Many net work adapters and commu nication protocols are biased toward providing high throughput only when relatively l arge packets are used . This means that the performance drops off as the packet size decreases.
Thus, the MEMORY CHANNEL interconnect pro vides a better alternative for communicating smal l, latency-sensitive packets.
Connection Manager
The con nection manager deti nes an operating envi ronment for the Jock manager. The design allows gen eralization to other clients; b u t in the TruCluster version 1 .0 product, the lock manager is the only con sumer of the connection manager services. The envi ronment hides the details of dynamically changing con tlgurations. From the perspective of the lock man ager, the connection manager manages the addi tion
:tn d removal of nodes and maint ains a communication path between each node. These services a l l owed us to simpli�' the lock manager design.
The connection manager treats each node as a mem ber of a set of cooperating distributed components.
It maintains the consistency of the set by admitting and removing members under controlled conditions.
Dig;ir,ll Technical Journal VoU!
No. l
1996
1 1
The connecrion , .l:mager prm·ides contlgu r:.Hion rclatcd event notification and other support services to each mem ber of a set. It provides notification when members are added and removed. It
�1iso maim a in s
J list of current members. The connection manager J!so provides notification to clients when unsafe operation is possible as a result of partitioning. Partitioning exists when a member of a set is un JwJrc of the existence of a disjoint set ofsimibr clients.
The connection m�w:-�gcr can be extended in clicnt-spccitic wavs to tJcilitatc hJndling of mem bership change n·ents. Extensions are integral, well synchronized parts of the membership change mechanism. The lock mJnagcr uses :1n extension to distribute a globally consistent directory database :111d to coordinate Jock dat:tb:-�se rebuilds.
The connection manager nuintains a fully con nected web of communication clunnels between members of t he set. Jv1embership in the set is comin gcnt upon being
able
to comm u nicJte with J l l other me mbers of that set. The usc of the communication channels is ent irelv under the comrol of the lock m:Jn agcr or
:tny other client that may usc the connection manager in the future. 'When a client requests :-�dmis sion ro a set, the connection tn:-�Juger establishes a communication channel between the new client :1nd
:11! existing clients. It monitors these connections to ensure they remain fi.mcriona l . A connection hils when a comm unication ch:1nncl is unus:tble
between
a p:1ir of clients or when a client �1t either end of the channel tails. The connection lll<lllJgcr detects these cond itions and reconfigu res the set to contain onlv fu l ly connected members.
The combination of
::1 highly ;w�1ibblc communi cation channel, together wirh set membership :tnd synchronized membership chJnge responses, allows optimiz::Jtions in the lock manager's message prot o c ol.
The lock manager can send a mcss�1ge ro another node and know that either the mess:tgc will be delivered or th:Jt the configuration wi I I be altered so that it d oes not matter.
The use of the connection manager greath· sim pli tics the design and implementation of the J ock manager. The connection m<lll<lgcr allows most of the logic tor hand ling contigur:ttion changes and com munication errors to be moved away from main code p�Hhs. This increases mainline per�(mnance and simpli fies
the
logic, al lowing mon: e m phasis on correct and efticicn t operation.
Memory Channel Interconnect
Cluster performance is criticallv dependent on the cluster interconnect. This is due both to the high bandwidth requirements of bulk data transport tor
DRD and to the low latency required
tor
DLM opera tions. Although the cluster architecture allows for any high-speed interconnect, the initial implementation supports onlv the new
M EM O RY CHANNEL inter connect designed specitic:t l ly tiJr the needs of cluster svstems. This verv reliable, high-speed interconnect is based on a prn·ious interconnect designed
bv
Encore
Computer Corporation '' I t has enhanced been signitic:mtly
bv
Digital to impro1·c d�Ha integrity :tnd provide �iJr higher perh
> rman c e i n the future.
EJch cluster node h a s a
1VI EMORY C H ANNEL intcrbcc card that connects to a hub. The hub can be thought of as a switch that provides either broadcast or point-to-point connections berwcen nodes. It also pro vi des ord ering guarantees �md docs a portion of the error detection. The current implementation is an eight-node hub, but brger hubs arc planned.
The MEMORY CHANN EL interconnect pro
,·i des a 1 00-megabvte-per-sccond , memor y-mapped connection to other cluster mem be rs. As shown in figure
4,
cluster members mav map transkrs fi·om
t
he
MEM ORY CHANNEL interconnect di rectlv into their memory. The efkct is of
�1 write-only wi ndow into the memory of other cluster systems. Transkrs
�1 r e done with standard memory �Kccss instructions r�nhcr than special I/0 instructions or device access
MEMORY
CHANNEL
TRANSFER
NORMAL
MEMORY
WRITE
PAGE
In
PAGE
1--
NODE
0
ADDRESS SPACE
MEMORY CHANNEL
BUS ADDRESS SPACE
Figure 4
Tr:llls k rs Performed lw the M EM O RY CHAN NEL I nt e rconnect
MEMORY
CHANNEL
TRAN SFER
I
PAGE
NODE
1
MEM ORY
1 2
Digital
Tcchnid journal
Vol. 8 :-.io. I 1 996
protocols to �woid the overhe:�d usu�1 l l v present with rhc..:se techniq ues. The usc of mc..:morv store instruc tions resu l ts in extremely low I J t<..:�Ky ( two microsec onds) :�nd low overhead f(x J rranster of any length .
The MEMORY CHANNEL interconnect guaran tees essentially no undetectc..:d c..:rrors ( :1pproximately the same und etected error r;ne as Cl'Us or memory ),
Jl iowing the elimination of c..:hc..:cksums and other mc..:ciJJllisrns that detect software.: errors. The detected error r:Jte is also extremely low ( on rhe order of one nror per year per connec..:rion ) . Since recovery code oecutes verv infreq uently, we.: Jre assured that rela rivclv simple, brute-forc..:c recovery fi·om software errors is adequate. Using hardware error insertion, we have rested recovery code at error rates of many per second . Thus we are confident there :1re no problems at the :1crual rates.
Low-level MEMORY CHANNEL Software
Low-level sofuvare interfaces :1rc provided ro insulate the next bver of sofuvare (e .g., lock manager and dis tributed disks) trom the dctJils of r he MEMORY
CHANNEL i mplementation . We have taken the
:1ppro:1ch of providing a verv thin l:1yer to impact per f(xmance as l ittle as possible and :Jilow direct use of the
MEMO
RY C HANN EL inrerconncct. H igher-level sofrw�m: rhen isolates its use of MEMOR
Y
CHAN N E L in : 1 rcmsport layer that can I :Jter be modi fied for addi tional <.:luster interconnects.
The.: writc-onlv nature ofrhc..: MEMORY CHA.l'\JNEL inrnconnect le:1ds r.o some chal lenges in designing
:111d i m p lementing software. The onlv wa�' to see a copv of d ata written to the MEJ\,I ORY CHANNEL interconnect is to m�
1 p MEMORY CHANNEL trans fers to :1 norher region of memory on rhe same nod e .
This le:1ds t o two very visi ble programming con straints. hrsr, data is read and written rrom different addresses. This is not �1 natural programming style, and code must be written to treat a location as two vari
:1blcs, one t()l' read and one t('Jr write . Second , the ctkct of a write is delayed by the transkr latency. At t\vo microseconds, this is short but is enough ti me to execute hundreds of instructions. Hardware teatures
Jrc provided ro stall u nti l d:tta h:1s been looped back, bur \'CI')' c:�refi.d design is necessary to minimize t hese sr::dls and place them correctly. \tVe have had several subtle problems when an algorith m d id not include a stall and proceeded to read stale dar�1 that was soon overwritten by data in transi t. finding these problems is especially di fficu lt because much evidence is gone by the time the problem is observed . for example, con sider
�1 linked l ist that is implemented in a region of memory mapped to all duste r nodes through the
J\II EMORY CHANNEL interconnect. l f t\vo demems are inserted on the l ist without inserting proper waits tor rhe loop back delav, the efkct of the first insert will not be visible when the second insert is done. This resu lts in corrupting the list.
The difficulties just descri bed <1rc most obvious when de:1ling with distributed shared memory. Low level software i ntended to support appl ications is instead oriented toward a message- passing model .
This is especial ly apparent in the tcaturcs provided for error detection. The pri mary mechanisms allow either the receivi ng or the send ing node to check for anv errors over a bounded period of rime. This error c heck requires a special hardware transaction with each node and involves a loopback delay. If
�111 error occurs, the sender must retransmit a l l messages and the receiver must not use any data received in that time.
This mechanism works wel l with the expected error rates. H owever, a shared munory model makes it ex tremely difticult to bou nd the data affected by an error, unless each modification of a data element is separately checked for errors. Since this involves
J l oopbJck delav, man�' of the perceived efficiencies of sh ared memory may disappear. This is not to say that a shared memory model cannot be used . It is just that error detection and con trol of concurrent Jccess must be wel l - integrated, Jnd node tai l ures requ ire careful recovery. I n addition, the write-only nature of
MEMORY CHANNEL mappi ngs is more su ited to messJge passing than shared memory due to the extremely carefu l programming necessary to hand le de laved loopback at a separate ad dress .
AP!s :tre provided pri marily to mJnage resources, con trol memorv mappings, and provide synchroniza tion. MEMORY CHA.NNEL A Pis perti.)rn1 the follow ing tasks:
•
•
•
•
A l locJtion and mapping
- Al locate or deallocate the
CHANNEL address space.
M EMORY
- Map the MEMORY CHAN N E L int erconnect for receive or transmit.
Unmap the interconnect.
M E M O RY CHANNEL
Spin lock synchronization
- Create and delete spin lock regions.
- Acquire and release spinlocks.
Other synchronization
- Create :1nd delete write :1cknowledgment regions.
- Request write acknowledgment.
- Cre:Jte and delete sofuvare notification channels.
- Send noti fication.
- W:tit f(x notification.
Error detection and recovery
- Get cu rrent error coum.
- Check tor errors.
- Register for call back on error.
Vol .
8
No. 1
1 996 1 3
Higher layers o f software are responsi ble for transfer ring data, checking for errors, retrying transfers, and synchronizing their use of
M EM O RY CHA!'\J N EL address space after it is allocated .
Synchronization
Efficient synch ron ization mechanisms are essential for high-performance protocols over a cl uster inter connect.
M E M O RY CHA!'\JN E L hardware provides two i mportant synchronization mechanisms: first, an ordering guarantee that all writes arc seen in the same order on al l nodes, including the looped- back write on the originating node; second , an acknowledgment request that returns the current error state of al l other nodes. Once the acknowledgment operation is com plete, all previous writes are guaranteed either to have been received by other nodes or reported as a transmit or receive error on some node. We have implemented clusterwide software spinlocks based on these guaran tees. Spinlocks are used for many purposes, including i n ternode synchronization of other componen ts and concurrency control for the cl usterwide shared-mem ory data structures used by the low- level
1YI E M O RY
CHAN N E L software .
A spin lock is structured as a n array with one clement for each node. To acquire the spinlock, a node first bids for it by writi ng a value to the node's array ele ment. A node wins by seeing its bid looped back by the
M EM O RY CHAN N EL i nterconnect without seei ng a bid from any other node. The orderi ng guarantees of the
M E M ORY CHAN N E L ensure that no other node could have concu rrently bid and believed it had won .
Multiple nodes can realize they have lost, but more than one node cannot wi n. In case of a con flict, many different back-off techniques can be used . The win ning nod e then changes i ts bid va lue to an own value.
This last step is not necessary for correctness, but it does help with resolving contention and with various fai l u re recovery algorithms. AJI higher- l evel synchro nization is bui l t on com binations of spi n l ocks, order ing guarantees, and error acknowledgments.
Error Recovery and Node Failures
Most of the difficult problems in the low- level soft ware relate to error recovery and node tailures. I n spire of its reliabil ity, errors wil l occur in the
M E M O RY
C HAN N E L interconnect, and they must be hand led as transparently as possible . Transparency is key to sim plifYing the com munication model seen by higher level software. In addition, node fail ures from hardware or software faults are more freq uent than
M E MO RY C H A N N E L errors and must be dealt with even in the most inconvenient portions of the low level code. The
M EM O RY CHAN N E L interconnect is managed through a collection of distri buted data structures that must be kept consistent. Software locks arc used to synchronize access to these structures, but errors may l eave them in an inconsistent state .
Guaranteed error detection before the release of a lock allows operations to be redone in case of an e rror.
Thus, all sequences of
M EM ORY CHAN N E L writes must be idempotent to take advantage of rhis straight forward error-recovery technique.
If a node failure occurs, a su rviving node must make a l l data structu res consistent before it rekases locks held by the fai led node. To keep this a manageable task, we have written carefully structured algorithms to handJe each inconsistent state . I n general, struc tures arc changed such that a single atomic write com mits a change. ff a node fails before this lasr write, no recovery is necessary. As an example, consider a data structure that is completely initialized before being added to a l ist.
A single write is used to accomplish the list addition . I f a node tails, the last write was either done or not and, in either case, the list is consistent.
Complications arise when another node has
;1 receive error on the l ast write done by a failing node. In this case, the fai led node cJ.nnot retry after detecting the error, so the node with the receive error has a difrerenr view of the list than a l l other surviving nodes. To resolve this event, one node must propagate its view of the list to a l l other nodes before it releases the lock held b�, the tailed node. Any node can do this because each has a sel rconsistcnt view of r he list. If the node with the receive error propagates i ts view, the last ele ment added by the failed node is lost. This situation is no different, however, from having the node fai l a few instructions earl ier. The chal lenge is to design recov ery for all these cases and maintain our sanity by mini mizing the number of such cases.
Another interesting problem is maintaining a con sistent cou nt of errors across a l l nodes. This count is key to the error protocols of both the low-level
M EM ORY C HA N N E L software and higher layers since comparisons of a saved and a current value bou nd the period over which data is suspect. The cou nt may be read on one node, transferred with a message, and com pared to a cu rrent va lue on another node. Thus, a consistent value on all nodes is critical and must be mai ntained in the presence of arbi trary combi nations of receive and transmit errors .
( Although errors arc very infrequent, they may be cor related; so algorithms must work well tor error bursts . )
T h e write acknowledgment, described earl ier, guaran tees that other nodes have received a write without error. It is used both to imp.lement a Jock protecting the error cou nt and to guaran tee that all nodes have seen an updated cou nt. Updating the count is a slo\\' operation due ro multiple round-trip delays and long error ri me-outs, bur it is performed very infreq uently.
1 4
Digital Tt:chn ictl )ound
Vol . 8 No. I 19')6
Future Enhancements to MEMORY CHANNEL
Software
Fully supported MEMORY CHANNEL AP!s are cu rrently available only to other layers in the UNIX kernel tor two important reasons: First, M EMORY
CHANNEL is a new type of interconnect and we want to better understand i ts uses and advantages before commi tting to a ful l y fu nctional API for general use.
Second, many difficult issues of secu rity and resource l i mits will affect the final i nterface. To help Digital and i ts customers gain the necessary experience, a lim ited functionality version of a user-level M EMORY
CHANNEL API has been impl emen ted i n the version
1 .0 product. This interface su pports al location and mapping of MEMORY CHANNEL space along with spinlock synchronization . It is oriented toward sup port of paral lel computation in a cl uster, but we also expect it will serve the needs of many commercial appl ications. Once we have a better understanding of how high- level applications will use the M EMORY
CHA.t'\INEL intercon nect, we wil l extend the design and provide add itional AP!s oriented toward both commercial applications and technical computing.
Application Fa ilover
Digital's TruCi uster multicomputer system is a logical evolution of the DECsafe Avai lable Server Envi ronment (ASE). An ASE system is a m u l tinode con figuration with all nodes and all high l y available storage connected to shared SCSI storage buses.
Figure 5 shmvs an ASE configuration. Software on each node monitors the status of a l l nodes and of shared storage. In case of a fai l ur e , the storage and associated applications are tailed over to surviving sys tems. Planned application failover is accomplished by stopping the appl ication on one node and restarting the application on a surviving node with access to any storage associated with the appl ication. Application specific scripts control tailover and usually do not require appl ication changes.
Figure 5
Typical ASE
Con tiguration
In addition to supporting the appl ication fai l over mechanisms from ASE, the TruCluster system sup ports paral lel applications running on m u l tiple cluster nodes. In case of a fai l ure, the application is not stopped and restarted . I nstead , it may continue to exe cute and transparently retain access to storage through a distributed disk server. I n addition, more general hardware topologies are supported .
Hardware Configurations
The TruCi uster version 1 . 0 prod uct supports a maxi m u m of tour nodes connected by a high-speed
MEMORY CHAN N EL interconnect. The nodes may be any Digital UNIX system with a peripheral compo nent i nterconnect ( PCI ) that supports storage and the
MEMORY CHANNEL interconnect. Highly available storage is on shared SCSI buses connected to at least two nodes. Thus, a c luster looks l i ke multiple ASE systems joined by a cl uster interconnect.
Although tl1e l imitation to four nodes is temporary, we do not intend to support large numbers of nodes.
Ten to twen ty nodes on a high-speed interconnect is a reasonable target. A cluster is a component of a dis tribu ted system , not a replacement for one. If very large numbers of nodes are desired, a distributed system is built with cl uster nodes as servers and other nodes as cl ients. This allows maintaining a simple model of a cluster system without havi ng to allow for many comp lex topologies. Aside from simpl icity, there are performance advantages from targeting algoritllms for relatively sma ll and simple cluster systems.
Although the nu mber of nodes is intended to be small, the individual nodes can be high-end mu ltiprocessor systems. Th us, the overall computing power and the f/0 bandwidth of a cluster are extremely large.
Conclusions
With the comp letion of the first release of Digita l 's
TruCluster prod uct, we believe vve have met our goal of providing an envi ronment tor high- performance commercial database servers. Both the d istributed lock manager and the remote d isk services are meeting expectations and providing reliable, high-performance services for paral lel ized applications. The M EM O RY
CHAN N EL i nterconnect is proving to be a n excel lent cluster i n terconnect: Its synchronization and fai l ur e detection are especially compatible with many cluster aware components, which are enhanced by its low l atencies and simplified by i ts elimination of compJex error hand ling. The error rates have also proven to be as pred icted . With over 100 units i n use over the last year, we have observed only a very smal l nu mber of errors other than those attributable to debugging new versions of the hardware.
Digital Tedmical j ou rnal Vol . 8
No.
1
1996 I S
Detailed component performance measurements are stil l i n progress, but rough comparisons of D RD against local I/O have shown no signitlcant penalty in latency or throughput. There is of course additional
CPU cost, b u t it has n ot proven to be significant for real applications. DLM costs are comparable to VMS and thus meet our goals. Audited TPC-C results with t11e Oracle database also validated both our design approach and the implementation details by showing that database performance and scaling with addi tional cluster nodes meet our expectations.
The previous best reported TPC-C num bers were
20,9 1 8 tpmC on Tandem Computers' H imalaya
Kl0000- 1 12 system with the proprietary NonStop
SQL/MP database software. The best reported num bers with open database software were 1 1 ,456 tpmC on the Digital AlphaServer 8400 5/350 with Oracle7 version 7 . 3 . A four-node AlphaServer 8400 5/350 cluster with Oracle Paral lel Server was recentJy audited at 30,390 tpmC. This represents i ndustry-leadership performance with nonproprietary database software.
Future Developments
We will continue to evolve the TruCiuster prod uct toward a more scalable, more general computing envi ronment . In particu lar, we will emphasize distributed fi l e systems, configuration flexibility, management tools, and a single-system view tor both i n ternal and client applications. Work is u nder way tor a cluster fi le system wiili local node semantics across the cluster sys tem. The new cluster file system will not replace D RD but will complement it, giving applications the choice of raw access through D RD or ti.tiJ, local-file-system semantics. We are also lifting the four-node limitation and allowing more flexibility in cluster i n terconnect and storage configurations. A single network add ress for tl1e cluster system is a priority. Finally, fu rther steps in managing a mu lti node system as a single system will become even more i mportant as the scale of cluster systems i ncreases.
F urther in the future is a true single-system view of cluster systems that wil l transparently extend all process control, communication, and synchronization mechanisms across the entire cluster. An i mplicit trans parency requirement is pertormance.
Acknowledgments
In addition to the au thors, the fol lowing i nd ivid uals contribu ted d irectly to the cl uster components described in this paper: Tim B urke, Charlie B riggs,
Dave Cherkus, and Maria Vella for D RD ; Joe Amato and M itch Condylis for DLM; and Ali Rafieymehr for
MEMORY CHANN EL. Hai H uang, Jane Lawler, and
especially project leader Brian Stevens made many direct and indirect contributions to the project.
Thanks also to Dick B uttlar for his editing assistance.
References and Notes
1 . "Introduction ro DCE,"
OSF DCE Documentation Set
( Cambridge, Mass . : Open Soltware Foundation, 1991 ).
2. Internet RFCs 1 0 1 4, 1057, and 1094 describe ONC
X D R, RPC, and NFS protocols, respectively.
3. G . Pfister,
ln Searcb of Clusters
( U pper Saddle River,
N.J . : Prentice- HaJ J , Inc., 1 995): 1 9 -26.
4.
N.
Kronenberg, H. Levy, and W. Strecker, "VAXclusters:
A Closely-Coupled Distributed System,"
ACil1
Trans actions on Computer S)stems,
vol. 4, no. 2 ( May
1 986): 1 30-146.
5. L. Cohen and
].
Wi lliams, "Techn ical Description of the DECsatC Available Server Environment,"
Digital
Tecbn ical.fournal,
vol. 7, no. 4 ( 1 995 ) : 89-100.
6. TPC performance numbers tor U N IX systems are typi cally reported for databases using the character device interface.
7. The rile system inrerfaces on the Digital UNI X operat ing system arc bei ng extended to support di rect ljO, which results in bypassing the block bu ffer cache and reduci ng code path length for those applications that do nor benefit from use of the cache.
8. A fast wide differential (FVV D ) SCSI bus is limited to a maximum distance of about 25 meters tor example.
9. M. Devarakonda cr al., " Evaluation of Design Alterna tives for a Cluster File System,"
USENTX Conference
Proceedings, USEN IX Association, Berkeley, Calif.
(J anuary 1 995 ).
10.
J.
Gray and A. Reuter,
Tra nsaction Processin[;
Concepts and Techniques
(San Mateo, Calif. :
Morgan Ka utinan Publishers, 1 993 ) .
1 1 . This mechanism i s inherited fi·om the DECsafe Avail a ble Server management faci.lity, including rhe asemgr inrertace.
1 2. As an example, if the first DRD service created for a cluster is 1 , the D lW device special fr le name is
/dev/drd/drd
I and irs minor device number is also 1 .
1 3 . C. Juszczak, " Improving the Performance and Cor rectness of an NFS Server," USL�·.VJx Conference Pro ceedings,
USENJX Association, San Diego, Calif.
( Winter 1 989).
1 4. W. Snaman, Jr. and D . Thiel, "The VAX/VMS Distrib uted Lock Manager,"
Digital Techn ical journal,
vol. 1, no. 5 (September 1 987): 29-44.
1 5 . R . Golden berg, L. Kenah, and D. Dumas, VAX/VMS
Internals and Data Structures ( BecHard, Mass. :
Digital Press, 1 99 1 ) .
1 6
Digital Technical Journal
Vol.
8 No.
I
1 996
1 6.
TruC/uster AfJj>!icatinn Pmgrc.nnm ing inter/aces
Gu ide ( M�wncml , 1'vlass . : D i g it
� l
Eq uipment Corpora tion, Order No.
AA-QL8 PA-TE, 1 996).
1 7.
T.
Rengclr<l)'ln, P. Spi r o
, and W. Wri ght, " H igh Avai l clbilitY MedwJisms o f VAX DBMS Software," Digital
Techuica/ .fou rna!. vo l .
1 , no.
8 ( l:;ebruarv 1 989 ) :
88-98.
1 8 . f:'ncnre 91 Series
Tech nical St11111nary ( Fort La ud erd a l e , rh1.: Encore Compu ter Corporation,
1 99 1 )
.
Biographies
Wil l iam
E.
Snaman, Jr.
Sandy Snaman joined Digital in 1 980. H e is cu rrenrlv a consulting sotn1-are engineer
Group, where in Digital's U N I X Sott\\'Jre he c o nt r i bu t ed ro the TruCiusrc:r architec run: and design . He and members of his group designed and im plemented c luster eomponems such as the con nection mcmager, lock manager, cm d various aspects of cluster commun ications. Previously, in ing Group, he was the pr o j ec t the VMS Engi neer leader �(>r the port of the
VMSclustcr svstcm ro t he AJpha pl att(>rm and the technica l supervisor a n d project kad er tor area .
Sandy rhe VAXcluster cxcc uri,·e a l so reaches MS Windows programm in g and
C++ at
Uni
D<1niel W e b st e r Col lege. H e has a B.S. i n compu ter science a n d an M.S. in intormarion systems trom the v e rs i t y of Lowell.
Wayne M. Cardoza
Wavnc
C a rdo z c 1 is c 1 senior consulting engineer in the
U N I X E n gi n e er ing Group. He joined DigitJI i n 1 9 79
Jnd c o n t ri b ute d to v�rious areas ofrhc VMS kernel prior to joi ning rhe U N I X C roup ro work on rhe U N I X cl uster producr. W:1ync was also one: of the architects of PIUSM,
Jn e<Jrly Digital t()r this work. More recently, he participated ofthe Alplu AX
Al ph<L B e t o r
RISC: arc h i tecru n: ; he holds several patems
I' architecture and the
Open VMS port to e coming to DigitJI, Wc1vne wc1s in the design employed bv
Bdl Laboratories. He rec e iv ed a B.S. E. E . trom Southeastern
Massachuse tts Uni1·crsit\· and an
M.S . E . E. trom MIT.
Frederick S. Glover
Fred Gl o v e r is '' soft:wc1n: consulting engi neer <l nd the rec h
nicll di rector of the
Digi t a l U�J
X Bc1se Operating System
Croup. Since joining the Di g i ta l
U N I X
Group in 1 9 8 5 ,
!-'red has contributed to the de ve l o pm e nt of networking serv i c e s, local c1nd remote ti le systems, <1nd c l uster tech nol ogy.
He has served as the chair of the
I 1:-:TI-'
/lS IG Truste d
NFS Working
Croup,
:JS the chc1 i r ofrhe
OS!-'
Distribu ted
File Svsrem Working Group, to the I E F.F. POS I X
:md as Digita l 's representative
1 00 3 . 8 Transparem hie
Access ·work ing Group. P1·ior to Jo in i n g D ig i ta l , Fred was e m pl oyed bv l\T&T Bell Laboratories, where co-development of the R.I\1AS h i s contri butions i ncluded network communication s ub s ys t e m. He re n: i v e d B .S. and M.S. degrees in computer science trom Ohio State Uni v ersit y cmd con ducted h i s t h e s is resc:u·ch i n the areas oftault-tolcrant distributed computing and data flow arcbi tecru n: .
Digital TechnicJI /oum<ll
Vol .
8
No. 1 1 996
1 7
I
Linda S. Wilson
Craig A. Neth
NUchaei J. FUckabaugh
Delivering Binary Object
Modification Tools for
Program Analysis and
Optimization
Digital has developed two binary object modification tools for program analysis and optim ization on the Digital UNIX version 4.0 operating system for the Alpha platform. The tech nology origi nated from research performed at Digital's Western Research Laboratory. The
OM object modification tool is a transforma tion tool that focuses on postl ink optimizations.
OM can apply powerful intermodule and inter language optimizations, even to routines in sys tem li braries. Atom, an analysis tool with object modification, provides a flexible framework for customizing the transformation process to ana lyze a prog ram. With Atom, compilation system changes are not needed to create both si mple and sophisticated tools to directly diagnose or debug appl ication-specific performance prob lems. The linker and loader are enhanced to sup port Atom. The optim izations OM performs can be driven from performance data generated with the Atom-based pixie tool. Applying OM and Atom to commercial appl ications provided performance improvements of up to 1 5 percent.
H istorical ly on UNIX systems, optimization and pro gram analysis tools have been implemented primarily in the real m of compilers and r u n - time libraries. Such implementations have several d rawbacks, however.
For example, although the optimizations performed by compilers are effective, typical l�', they are limited to those that can be performed withi n the scope of a sin gJe source file. At best, the compiler can opti mize the set of files presented d u ri ng one compilation run.
Even the most sophisticated systems that save interme diate representations usually cannot perform opti mizations of cal l s to routines in system libraries or other libraries for which no sou rce or intermediate forms arc available . 1
The traditional UNIX performance analvsis tools, prof and gprof, req uire compiler support for inserting calls to pred efined ru n-time l i brary rou tines at the entry to each proced ure. The mon itor rou tines allow more user control over prof and gprof profi ling capa bil ities, but their usage requires modifications to the application source code. Because these capabilities are implemented as compilation options, users of the tools must recompile or, in the case of the mon itor rou tines, actually mod i fy their applications. For a l a rge applica tion, this can be an onerous requ i rement. Further, if the application uses libraries for w hich the source is unavai lable, many of the analysis capabilities are lost or severely impaired.
By comparison, object modification tools can per form arbitrary transformations on the executable program. The OM object modi fication tool is a trans formation tool that focuses on postli nk optimizations.
By pertcxming transformations after the link step, OM can apply powerful intermod u le and i nterlanguage optimizations, even to routines in system libraries.
Object transformations also have benefits in the area of program analysis. Atom, an analysis tool wirh object modification, provides a flexible framework tor cus tomizing the transformation process to analyze a pro gram. With Atom, compilation system changes are not needed to develop specialized types of debuggi ng or performance analysis tools. Appl ication developers can create both simple and sophisticated tools to directly diagnose or debug application -specific performance problems.
1 8 Digite1l TcchnicJI journal
Vol. 8 No. I 1 996
The OM and Atom technologies originated ti·om research performed at Digita l 's Western Research
Lab (WRL) in Palo Alto, California.' The software was developed into products by the Digi tal
Development Environment ( DUDE)
U N I X group a t
Digi ta l 's UNIX engi neeri ng site in Nashua, New
Hampshire. Both technologies are cu rrently shipping as supported products on D igital tor the Alpha platform :'
UNIX version 4.0
This paper first provides technical overviews tor both OM and Atom. An example Atom tool is presented to demonstrate how to use the Atom appli cation programming i nterface (API) to deve lop a cus tomized program analysis tool . Because OM and
Atom can be used together to enhance the etkctive ness of optjmi zations to :1pplication programs, the paper includes an overview regard ing the benefits of profi ling-directed optjmizations.
Subsequent sections discuss the product develop ment and tech nology transfer process for OM and
Atom and several design decisions that were made.
The paper describes the working relationship between
Wl�L and DUDE, the u til ization of the technology on
Independent Software Vendor (
ISV) applications, and the factors that drove the separate development strate gies for the two products. The paper concludes with a discussion about areas tor fu rther investigation and plans for future enhancements.
Technology Origins
Researchers at WRL investigating postlink opti miza tion techniques created OM in
1 992 '
Unlike compile time optirnizers, which operate on a single file, postlink optimizers can operate on the entjre executabk pro gram. For instance, OM can remove procedures that were linked into the executable but were never cal led, thereby red ucing the text spJce required by the pro gram and potential ly improving its paging behavior.;
Using the OM technology, the researchers fu rther discovered that the same bin ary code modi fication tech niques needed for optimizations could also be applied to the area of program instrumentation . I n tact, the processes o f instrumenti ng a n existi ng pro gram and generati ng a new execu table coul d be encapsu lated and a programmable interface provided to d rive the instr u mentation and analysis processes.
Atom evolved from this work -"·7
In 1 99 3 , WRL researchers Amitabh Srivastava and
Alan Eustace begJn planning with
DUDE engineers to provide OM and Atom JS supported products on the Digital
UNIX operating system . Different prod uct development and technology transfer strategies were used f(>r delivering the two tech nologies. The sec tion Product Development Considerations discusses the methods used and the f()rces that infl uenced the strategies.
Technical Overview of OM
OM performs transformations in three phases. It pro duces an i ntermediate representation , perf()rms opti mizations on thJt representation, and prod uces an executable i mage.
Intermediate Representation
In the fi rst phase, OM reads a special l y linked input til e that is produced by the linker, parses the object cod e, Jnd produces an i nrermed iJte representation of the instructions in the program . The flow informa tion and the program structure are maintained i n this representation.
Optimization ln tl1e optimization phase, OM performs various trans formations on the i n termediate representation created in the first phase. These transformations include
•
•
•
•
•
Tex t size reductions
Data size reductions
I nstruction and data reorganization to Improve cache behavior
I nstruction sched u ling and peephole optimization
User-directed procedure inlining
Text Size Reductions One type of text size reduction is the elimination of un used routi nes. Starting at the entry point of the image, OM exJmines the instruction stream for transfer-of-control instructions. OM fol lows each transter of control until it finds all reachJble routines i n the image . The remaining routines are potentiaUy unreachable and are candidates tor removal.
Before removing them, OM checks JJI candidates tor any add ress references. (Such references wi l l show up in the relocation entries for the symbol s . ) If no refer ences ex ist, OM em sJfdy remove the routine. A sec ond type of text size reduction is the elimination of most
GP
register reloading sequences.'·''
Data Size Reductions Because it operates on the entire program , OM performs optimizations that compilers are n ot able ro perform . One instance is with the addressJbiliry of globJl data. The general instruction sequence for accessing global data req uires the usc of a table of add ress constants ( the .litJ section ) and code necessary tor maintaining the cu rrent position in t.he table. Each entry in the address constant table is relo cated by the lin ker. Because OM knows the l ocation of
Jll global data, it c111 potentially remove the add ress entry while i nserting and removing code to more effi ciently reference the datJ directly. Removing Js many of the .lira entries Js possible l eaves more room in the data cache ( 0-cache) for the applicJtion 's global data.
Dig;ira.l
TcclllliGli
J o urml
Vo l .
R No.
I 1 ':196
1 9
20
This o pt im iz a t i on i s nor possible
:1t link time, b
c
c
1u s
c
the l inker c:tn neither i nsert nor re m ov e code .
Reorgan ization of the Image nizes :1 n i m; 1 g
B v deElll l t, O M re o r g a
c b1· reordering the routines in r h c i m ;1ge as determined by a depth -ti rst search of the routine order, starring at the main cnrrv poin t. I n the absence of profiling information , this orderi ng is usually better than the
In the linker's ordering. presence of p ro fi l in g kedback, OM pe rform
s
two more instruction-stream r e orck r i n gs: ( 1 ) r eorde r ing of r o uti n es based on b; 1 s i c block coums
;md
( 2 )
rourine o rd e r ing based on execution t i-cqu e n c l' .
OM first reorg;mizcs rou tines based on the basic block information co.l lected by a previous run of the i mage i n stru me nted with the Atom-b:tsed pixie tool. ON! L1ys rhe b asi c blocks to match the program's l i kely fl o II' of control . Branches are al ig ne d to match t he h;Hdll'are pred iction mechanism. As :1 resu l t,
OM packs together the most com mo1llv executed blocks. After b:tsic block reorganization, OM then reorders the routines in the image based on the cumul:ttivc basic block counts f()r each routi ne. The reorganized i m age i s ordered in a '''av similar to the wav the prof tool d isp!Jvs necu rion statistics. The reordering perr(mncd b1· OM is superior to that performed by cord, because cord docs 110t reorder basic blocks. cord is a U N IX profiling-directed opti mization utility that reorders pro ced ur e s in an exe cutable progrJm to cord(1) r e fer improve c1chc performance. The
e
n c e page on Digic1 l UNIX 1nsio n 4.0 describes the o p er a ti on of this urilir�· in more derai l .
Ela psed-time Performance
The o pt i m i z a ti o ns tllat
OM perf(>nm without p r o fi ling tCcd back can provid e elapsed -rime pedormancc i m p ro ve m ents o f u p ro
5 p e r c e n t. The teedback-dirccred optimizations can often pnwidc :11 1 <1dditioml i m p ro v ement of ti·om 5 ro lO p erce nt in elapsed time, t(lr a total i mprmTmcnr of up to 1 5 perce nt over an i m age not proc es sed by OM. S eve r a l commercial dJtabase progr;1 ms h:tve realized c l:t p s cdti me pe rfonnJn c e i mpro v e m e nt s ranging h·om 9 to l l percem with ked back.
Executable Image
Fi nally, i n the third phase, Olvl rc1ssembles
the tr;ms
f orm ed i ntermediate represcnc1rion into an exccu t;lhle i m a g e. ft pcrt(lrms relocation p r o ce ssi n g to reflect
Jny changes in data layo ut
or
program organization .
Technical Overview of Atom
The Atom tool ki t consists ofJ dril'er, an i nstrunKt1ta rion engine, Jnd an analysis run- time syste m . The
Atom engine performs transformations on an c :x c c u ta bl c program, converting it to an i n t
e
r med i a te torm. The e n gi ne then annotates rhc i n termcdi:ttc form and generates a nell', instru mcmed program .
Vol . 8 :-.i<>
I I Y96
The Atom engine is pr og r am m J b l e.
Atom a c c e p ts as i n p u t ;1 11 instrumentation rile and an :mal ysis ti le.
The i nstrumen tation fi le ddines the p oi nts at which the progrJm is to be instru mented and w lut an;1 l ysis routine is to be called
;1t uch ins t rume n t
J tion point.
The an;1 h·sis ti le (ddincci bttr in this section ) defines the anal ysis routines. Atom
<11iOII'S
i n s t ru m e nt ;Hion of a program at a very fi ne level of g ra nul arity. It supports instru mcnt:<tion before and after
•
•
•
•
•
Program execution
Shared l ibrarv loading
P ro ce du r e s
B:1sic blocks
I n d iv i d ual i nstructions
S u ppo r t i ng instru mcntJtion at these poi nts alloll's the development of a l l"i de
\";1riety of tools, a l l
11·irhin the Atom t
i
· am e
\\' ork.
Ex:�rnplcs
of' th e se tools are Gtche simulators, memory c hecking tools, and performance measu rcmenr tools . The h
· am
c
w o r k supports t h e cre ation of customi zed tools and can simpli
�
1ing d ec r ea se costs by rhe development of single-use tools.
The i nstrumentation tile is a
C language program that conrains calls to the
Atom
A P I .
The i ns tr u m c nt
; 1 tion ti le dcti nes anv arguments to be passed ro the analvsis routine. Arguments can instruction be register l'al ues, f i e lds, sym b ol names, addresses, ere . The instru mcntJtion the fil e is co m p ile d and then l i n ked with
Atom i n s tr u m e n t :t pertonn the i ns t rum
r
ion engine to c r e a te J tool to
e
nta t i on on ;1 target progr:1 m.
The
;malvsis file contains definitions ofrhc routines that arc called from the instru mentation p o i nts in rhe target program . The analysis rou tines record evenrs o r
process rbc
arguments that arc passed from
rhc
i n
s
t ru mentation points.
Bv co m
· enr ion , the instru me ntation and an:1 l vsis tiles arc n:<1ncd \Yi th the suffi xes inst.c and anal.c, r e s p e c t i v el v. Atom is i m·oked ;1s follows to crc:nc an i nstrumented e x ec u t abl e
:
% a t o m p r o g r a m t o o l . i n s t . c t o o l . a n a l . c
The atom co m m J n d is a d ri,·er that invokes the compiler :tnd linker to gencL1te the instrumented progra m . The rive steps of this process are
I . Compil e rhe instru mentJtion code .
2 . Link the instrumen tJtion code with t h e Atom i nsrrumcn t;ltion engi nc ro create an i ns rr u m e n ta tion rool .
3.
Compile the an a lysis code.
4. Lin k the :1 11:1lvsis cod e with the Atom Jnah·sis run ti me system, using the
U N IX ld tool with the
-r o pt i o n so the object may be used as inp u t to another link.
5 . Execute the instrumentation tool on the target program, providing rhe linked analysis cock as an argument.
T h e final step prod ucts an i nstrumented program l i n ked with th<:: analysis cod e . Figure l shows th<:: chang<::s i n memory l ayou t between the original pro gram and the instru ment<::d program.
An Example Atom Tool for Memory Debugging
The following d iscussion of an example Atom tool demonstrates how to usc the Atom
A P I to develop a customized program analysis too l .
A common development problem is locating the source of a memory overwrite. Figure 2 shows a con trived example program i n which the loop to ini ti�1lize an array exceeds the array boundarv and overwrites a
LOW
MEMORY
HIGH
MEMORY
STACK
READ-ONLY DATA
EXCEPTION DATA
- TEXT START
:
STACK
READ-ONLY DATA
EXCEPTION DATA
PROGRAM TEXT
ANALYSIS TEXT
INSTRUMENTED
PROGRAM TEXT
PROGRAM
TEXT
ADDRESSES
CHANGED
N E W DATA
START
-----........__
ANALYSIS gp -...__
ANALYSIS DATA
I
N I
T
IALIZED
/
OLD DATA
START
�
ANALYSIS DATA
UNINITIALIZED
(SET TO 0)
PROGRAM DATA
INITIALIZED
-- PROGRAM gp PROGRAM DATA
I N ITIALIZED
PROGRAM DATA
U N I N ITIALIZE D
HEAP
:
U N I NSTRUM ENTED
PROGRAM LAYOUT
PROGRAM DATA
U N I N ITIALIZED
HEAP
:
I NSTRUMENTED
PROGRAM LAYOUT
PROGRAM
DATA
ADDRESSES
UNCHANGED
Source · A. S rivastava and A. Eustace, '"ATOM: A System lor Building Customized Program Analysis Tools,'"
Proceedings of the SIGPLAN
'94
Conference on Programming Language Design and Implementation,
Orlando, Fla. (June 1 994). This paper is also available as Digital's Western Research Laboratory
(WRL) Research Report 94/2.
Figure 1 tvkmon· Lavout of I nstru mented Programs
1
2 l o n g n u m b e r s [ 8 J
=
{ Q } ; l o n g * p t r = n u m b e r s ;
3
4 m a i n ( )
5
{ i n t i ; 6
7
8
9
1 0 } f o r C i = O ; i < Z S ; i + + ) n u m b e r s [ i ) = i ;
Figure 2
ExJtnplc Progmm ll'ith Memory Overwrite
I * T h i s p o i n t e r i s o v e r w r i t t e n * I
I * b y t h i s a r r a y i n i t i a l i z a t i o n . * I
Di�irJ[ Ted111ical Journal
Vol . � No. l 1 996 2 1
poi nter v;1riabJe. I n this cJse, the initialization o f the rzumhers array detl.ned i n line I overwrites the con tents of the variable ptr ddined in l ine 2. This type of problem is often difticulr :md rime-consuming to locate with traditional debugging tools.
Atom can be used to develop a simple tool to loGlte the source of the overwrite . The tool would instr u ment each store i nstruction in the program a n d p:1ss the effective address of the store instruction and the value being stored to an Jnalysis routi ne. The ::m::dysis routine wou ld determine if the effective address is the address being traced and, if so, generJte a diJgnosri c .
The i nstru mentati on :md ;m ah'sis files f o r the mem_debug too l
Jre shown l ll
Figure 3 .
I nstrument! nit( ) registers the analysis routines with the Atom i nstrumentation engine a nd specifies rh::�t calls to the get_Jrgs( ) and open_ log( ) routines be inserted before the progrJm begins executing. A cal l ro the close _ log ( ) routi ne is dicr;ned when the pro tion engine calls I nstrument I nit( ) exactly once.
The Atom i nsu·umenrarion engine c1lls the
I nstrument( ) routjne once tar each executable objccr in the progr:lln. The routine instruments e::�ch store instruction that is not a stack opcrarjon wirl1 a c
:t l l to the analvsis routine mem_srore(
) . E;Kh call to the routine prO\;ides the address ofrhe store instruction, the target address of the store instruction, the value to be srorcd, and the file name, procedure n:�mc, and line nu mber.
The open_log( ) and close_Jog( ) analysis routines ::�rc selfexplanatory. The messages cou ld hJve been wrirren to the standard output, bccJusc, in this exJmple, they would not have interfered with the program output.
The ger_:�rgs( ) routine re;His the va l u e of the
MEM Df:JJI JG ;I NGS environ ment variable to obt:�in the datJ
-; ddress
� o be traced . The tool could luve been written to accept arguments from the command l i ne using the -too largs swi tch. The instrume n t
J tio n code would then JXISS the arguments to the analysis rou tine.
In the GlSe of this tool , using the environment v:�ri;1 bk to pass the arguments is bcneticial because the pro gram does not have to be reinsrrumented to tra c e
J new :tddress.
The mem_srore( ) routine is c1l led from each store instruction sire that was i nstrum e nted . If the tJrgct address of the store operation docs not march the trace add ress, the routine simply returns. If there is a matc h ,
J diagnostic is logged that gives i n fimnJtion about the location of the store.
To demonstrate how this tool would be used, sup pose one has determined b�, debugging that the vari able ptr is being overwritte n . The nm comma nd
IS used to determine the address ofptr as follows:
% n m - B p r o g r a m I g r e p p t r
O x 0 0 0 0 0 1 4 0 0 0 0 0 c 0 G p t r
I nstrument the program with the mem_debug tool .
22 Digit�!
Tcchnicll )oum�l
Vol . 8 No. l
1 996
% a t o m p r o g r a m m e m_d e b u g . i n s t . c m e m_d e b u g _ a n a l . c
Set the iv!Uv!_DtBUG_A NCSenviron ment v:�riable with the address to trace .
% s e t e n v M E M _D E B U G _A R G S 1 4 0 0 0 0 0 c 0
Run the instrumented program ,
% p r o g r a m - a t o m and view the log file .
% m o r e p r o g r a m . m e m_d e b u g . L o g
T r a c i n g a d d r e s s O x 1 4 0 0 0 0 0 c 0
A d d r e s s O x 1 4 0 0 0 0 0 c 0 m o d i f i e d w i t h \ v a l u e O x 1 6 : a t : O x 1 2 0 0 0 1 1 c 4 P r o c e d u r e : m a i n \
F i l e : p r o g r a m _ c L i n e :
9
Using this si mple Atom too l , the location of a mem orv overwrite can be detected q uickly. The instru m
� n ted program executes at nearly normal speed.
Tr:�d itional debugging methods to detect such errors are much more ti me-consuming.
Other Tools
An Jrca in which Atom capabilities have proven particu larly powcrfi.d is for hardware modeling and simulation .
Atom has been used as
:t te:�chi ng tool in university courses to trai n srudenrs on h:�rdware and operating sys tem design. Moreover, Digiol hardware designers have de\'eloped sophisticated Atom tools to evaluate designs tor ne\\' implementations oftbe Alpha cl�p.
The A tom tool ht contains 1 0 example tools that demonstrate the capabilities of this technology. The examples i nclude a branch pred iction tool, which is useful t(lr compiler designers,
:t procedure tr:�cing tool, which can be useful in following the How of un bm i li:�r code, and J simple cache simulation too
l
.
Atom Tool Environments
A.tulysis of certain types of programs can requi re use of speci:�lly designed Atom tools. for instance, to analyze a program thJt uses POSIX threads, an Atom
to
ol to hand le thre:�ds must also be designed , bec1use the analysis routines will be cal led from the threads in the appl.ic:�rion program. Therd(>re, the analysis rou ti nes need to be reentrant. They may also need to synchro nize access to data that is shared among the threads or manage cbta for i ndividual threJds. The thre
:� d m:tn agemenr in the analysis routines adds overhead to the execution time of the instrumented progr:� m; th1s overhe a d is not necessary for a non th readed progr:�m.
To effectively su pport both threaded and nonrhreaded programs, tWO rustinct versions of the same Atom tool. need to coexist. Designers developed tbe concept ot tool environ ments to add ress the issues of providing multiple versions of an Atom tool .
I *
* m e m_d e b u g _ i n s t . c
*
I n s t r u m e n t a t i o n f o r m e m o r y d e b u g g i n g t o o l
* T h i s t o o l i n s t r u m e n t s e v e r y s t o r e o p e r a t i o n i n a n a p p l i c a t i o n a n d
* r e p o r t s w h e n t h e a p p l i c a t i o n w r i t e s t o a u s e r - s p e c i f i e d a d d r e s s .
* T h e a d d r e s s s h o u l d b e a n a d d r e s s i n t h e d a t a s e g m e n t , n o t a
* s t a c k a d d r e s s .
*
* U s a g e : a t o m p r o g r a m m e m_d e b u g . i n s t . c m e m_d e b u g . a n a l . c
*
* I
# i n c l u d e < s t r i n g . h >
# i n c l u d e < c m p l r s l a t o m . i n s t . h >
I *
* I n i t i a l i z a t i o n s :
*
* r e g i s t e r a n a l y s i s r o u t i n e s d e f i n e t h e a n a l y s i s r o u t i n e s t o c a l l b e f o r e a n d a f t e r p r o g r a m e x e c u t i o n
*
* g e t _ a r g s ( ) r e a d s e n v i r o n m e n t v a r i a b l e M E M D E B U G A R G S f o r a d d r e s s t o t r a c e
* o p e n_ L o g ( ) - o p e n s t h e L o g f i L e t o r e c o r d o v e r w r i t e s t o t h e s p e c i f i e d a d d r e s s
* c l o s e_ L o g ( ) c l o s e s t h e L o g f i l e a t p r o g r a m t e r m i n a t i o n
* m e m s t o r e ( ) c h e c k s i f a s t o r e i n s t r u c t i o n w r i t e s t o t h e s p e c i f i e d a d d r e s s
* I v o i d I n s t r u m e n t i n i t C i n t a r g c , c h a r * * a r g v )
{
A d d C a L L P r o t o ( " g e t _a r g s ( ) " ) ;
A d d C a L L P r o t o ( " o p e n_ L o g ( c o n s t c h a r * ) " ) ;
A d d C a L L P r o t o C " c L o s e_ L o g ( ) " ) ;
A d d C a L L P r o t o C " m e m_ s t o r e C V A L U E , R E G V , L o n g , c o n s t c h a r * , c o n s t c h a r * , i n t ) " ) ;
A d d C a L L P r o g r a m C P r o g r a m B e f o r e , " g e t _a r g s " ) ;
A d d C a L L P r o g r a m ( P r o g r a m B e f o r e , " o p e n_ L o g " , b a s e n a m e C C c h a r * ) G e t O b j N a m e C G e t F i r s t O b j ( ) ) ) ) ;
A d d C a L L P r o g r a m C P r o g r a m A f t e r , " c l o s e_ l o g " ) ;
}
I *
* I n s t r u m e n t e a c h o b j e c t .
* I
I n s t r u m e n t ( i n t a r g c , c h a r * a r g v [ J , O b j * o b j )
{
P r o c
B l o c k
I n s t i n t
* p r o c ;
* b l o c k ;
* i n s t ; b a s e ;
I * b a s e r e g i s t e r o f m e m o r y r e f e r e n c e * I
I *
* S e a r c h f o r a l l o f t h e s t o r e i n s t r u c t i o n s i n t o t h e d a t a a r e a .
* I f o r ( p r o c
=
G e t F i r s t O b j P r o c C o b j ) ; p r o c ; f o r ( b l o c k
=
G e t F i r s t B L o c k ( p r o c ) ; p r o c b l o c k ;
=
G e t N e x t P r o c ( p r o c ) ) { b l o c k
=
G e t N e x t B l o c k C b l o c k ) ) { f o r ( i n s t
=
G e t F i r s t i n s t C b l o c k ) ; i n s t ; i n s t
=
G e t N e x t l n s t ( i n s t ) ) {
I *
* I n s t r u m e n t m e m o r y r e f e r e n c e s . S k i p $ s p b a s e d r e f e r e n c e s
* b e c a u s e t h e y r e f e r e n c e t h e s t a c k , n o t t h e d a t a a r e a .
* M e m o r y r e f e r e n c e s a r e i n s t r u m e n t e d w i t h a c a l l t o t h e
* m e m_ s t o r e a n a l y s i s r o u t i n e . T h e a r g u m e n t s p a s s e d a r e
* t h e t a r g e t a d d r e s s o f t h e s t o r e i n s t r u c t i o n ,
* t h e v a l u e t o b e s t o r e d a t t h e t a r g e t a d d r e s s ,
* t h e P C a d d r e s s o f t h e s t o r e i n s t r u c t i o n i n t h e p r o g r a m ,
* t h e p r o c e d u r e n a m e , f i l e n a m e , a n d s o u r c e L i n e f o r t h e
* P C a d d r e s s .
* I
Figure 3
Instrumentation and Analysis Code tor the mem_debug Tool
Digital Technical Journal
Vol. 8 No.
I 1 996 23
i f ( l s 1 n s t T y p e ( i n s t , I n s t T y p e S t o r e l l b a s e
=
G e t l n s t l n f o ( i n s t , I n s t R B l ;
{ i f ( b a s e
! =
R E G S P l
{
A d d C a l l l n s t ( i n s t , I n s t B e f o r e , " m e m_ s t o r e " ,
E f f A d d r V a l u e ,
G e t l n s t R e g E n u m ( i n s t , I n s t R A l ,
I n s t P C ( i n s t l ,
P r o c N a m e ( p r o c l ,
P r o c F i l e N a m e ( p r o c l ,
( i n t l l n s t l i n e N o ( i n s t l l ;
}
}
}
}
}
}
I *
* m e m_d e b u g . a n a l . c a n a l y s i s r o u t i n e s f o r m e m o r y d e b u g g i n g t o o l
*
* U s a g e : s e t e n v M E M D E B U G A R G S h e x a d d r e s s b e f o r e r u n n i n g
* t h e p r o g r a m .
* D i a g n o s t i c o u t p u t i s w r i t t e n t o p r o g r a m . m e m_ d e b u g . l o g
* I
# i n c l u d e < s t d i o . h >
# i n c l u d e < s t d l i b . h >
# i n c l u d e < s t r i n g . h >
# i n c l u d e < s y s l t y p e s . h > s t a t i c F I L E * l o g_ f i l e ; s t a t i c c a d d r t t r a c e _a d d r ;
I * O u t p u t f i l e f o r d i a g n o s t i c s * I
I * A d d r e s s t o m o n i t o r * I
I *
* C r e a t e L o g f i l e f o r d i a g n o s t i c s .
* I v o i d o p e n_ l o g ( c o n s t c h a r * p r o g n a m e l
{ c h a r n a m e [ 2 Q O J ; s p r i n t f ( n a m e , " % s . m e m_d e b u g . L o g " , p r o g n a m e l ; l o g_ f i l e
= f o p e n ( n a m e , " w " l ; i f <
! l o g _ f i l e l
{ f p r i n t f ( s t d e r r , " m e m_d e b u g : C a n ' t c r e a t e % s \ n " , n a m e ) ; f f l u s h ( s t d e r r l ; e x i t ( 1 l ;
} f p r i n t f ( l o g_ f i l e , " T r a c i n g a d d r e s s O x % p \ n \ n " , t r a c e _a d d r l ; f f l u s h ( l o g_ f i l e l ;
}
I *
* C l o s e t h e L o g f i l e .
* I v o i d c l o s e_ l o g ( v o i d l
{ f c l o s e < l o g_ f i l e l ;
}
I *
* G e t a d d r e s s t o t r a c e f r o m t h e e n v i r o n m e n t .
* I v o i d g e t _a r g s ( v o i d l
Figure 3
( cont inued )
24 Dig:inl Tcchniul journal
Vol. ll No. J 1 996
{ c h a r * a d d r ; i f ( l ( a d d r
= g e t e n v ( " M E M _D E B U G_ A R G S " ) ) )
{ f p r i n t f ( s t d e r r , " m e m_d e b u g : s e t M E M D E B U G A R G S t o h e x a d d r e s s \ n " ) ; f f l u s h ( s t d e r r ) ; e x i t ( 1 ) ;
} t r a c e a d d r ( c a d d r t ) s t r t o u l ( a d d r , 0 , 1 6 ) ;
}
I *
* T h e t a r g e t a d d r e s s i s a b o u t t o b e m o d i f i e d w i t h t h e g i v e n v a l u e .
* I f t h i s i s t h e a d d r e s s b e i n g t r a c e d , r e p o r t t h e m o d i f i c a t i o n .
* I v o i d m e m s t o r e ( c a d d r t u n s i g n e d L o n g c a d d r t c o n s t c h a r c o n s t c h a r u n s i g n e d
{ t a r g e t _ a d d r , v a l u e , p c ,
* p r o c ,
* f i l e ,
L i n e )
I * A d d r e s s b e i n g s t o r e d i n t o * I
I * V a l u e b e i n g s t o r e d a t t a r g e t _ a d d r * I
I * P C o f t h i s s t o r e i n s t r u c t i o n * I
I * P r o c e d u r e n a m e * I
I * F i L e n a m e * I
I * L i n e n u m b e r * I i f ( t a r g e t _a d d r t r a c e _a d d r )
{ f p r i n t f ( L o g _ f i l e , " A d d r e s s O x % p m o d i f i e d w i t h v a l u e O x % L x : \ n " , t a r g e t _a d d r , v a l u e ) ; f p r i n t f ( L o g _ f i l e , " \ t a t : O x % p " , p c ) ; i f ( p r o c
! =
N U L L )
{ f p r i n t f ( L o g _ f i l e , " P r o c e d u r e : % s p r o c > ; i f ( f i l e ! = N U L L ) f p r i n t f ( L o g_ f i l e , " F i l e : % s
} f p r i n t f ( L o g _ f i l e , " \ n " ) ; f f L u s h ( L o g_ f i L e ) ;
L i n e : % d " , f i l e , L i n e ) ;
}
}
Figure 3
(continued )
Tool environments accommodate seamless integra tion of specialized versions of tools into the Atom tool kit. They pro\'ide a means for extending the Atom ki t.
This f:tcility allows the addition of specialized Atom tools bv Digital's l ayered product groups or by cus tomers, while maintaining a consistent user i n terface.
The versions of the Atom tools hi prof, pixie,
Jnd
Third Degree that support POSIX threads are pro vided as a separate environ ment. hiprof is a perfor mance analvsis tool that collects data similar ro but with more precision than gprof. pixie is a basic block profi ling tool . Third Degree is a memory leak detec tion tool .
The fol lowing command invokes tbe Atom-based pixie tool for use on a nonthreaded program:
% a t o m p r o g r a m - t o o l p i x i e
The fOllowing comm:111d invokes the version of the pixie roo] that supports threaded programs:
% a t o m p r o g r a m - t o o l p i x i e - e n v t h r e a d s
Tools for orber specialized environments can be provided by defining a new environment name. For example, Atom tools written to work with a language specific run- time environment can be added to the
Atom tool kit by selecting an environment name for the category of tools. Similarly, tools designed to work on the kernel could be col lected into Jn environment.
The environ ment name is used in the names of rhe tool's instrumentation, analysis, and description files.
The description fi le for a tool provides the names of the instrumentation Jnd analysis fi les, as well as spe cial i nstructions for compiling and linki n g the tool .
For example, the pixie description tile tor threaded programs is named pixie.threads.dcsc. lt identifies the th readed versions of the pixie instrumentation and analysis ti les. The Atom driver builds rhe name of the description fi le from the argu ments to the
-tool and
-env switches on the command line. The contents of the description tile then d rive the subsequent steps of the build process.
Tool environments can be added without modifica tion ro the base Atom tech nology, thereby providing extensibil i ty to the tool kit while maintaining a consis tent i nterface.
Compact Relocations
Atom i nserts code into the text of the program , thus changing the location of routines. Atom req uires that relocation
intormatjon
be retained i n the
Digital Tedlllicll )oumJI
Vol. S
No. l 1996
2 5
executable image created bv the l i n ke r. This a llows
Atom to properly perform r-elocations on the insrru men ted executable.
Duri ng the normal process of linking, the relocation en tries stored in object files are climin;ltcd once thcv have been resolved. Because it effecrivdv rel i n ks th
� executable, Atom must have access to rl
; e relocation information .
Consider, tor example, an applic1tion that invokes a function through a statically initialized fi.mction pointer vJriablc, as shown in the followi ng code segment: v o i d f o o ( i n t a , i n t b )
{
} v o i d ( * p t r _ f o o ) ( i n t , i n t ) f o o ; v o i d b a r ( )
{
( * p t r_ f o o ) ( 1 , 2 ) ;
}
The Jdd ress of fu nction foo is stored i n rhc memory location referenced by the
ptr_foo
variable . When
Atom instruments this application, the add ress of
ji.;o
will change, and Atom needs to know to update the contents of the memorv location referenced bv
prr_jbo.
This is possible only i rhere is J relocation n:cor
� i pointing at this memory location. Addjng compact relo cations to the executable file solves this problem .
Compact relocations are smaller rh:m rcguL\l· re lo cations tor two reasons. First, the Atom system docs not req u i re all the information in the regu lar reloca tion records in order to instrument Jn executable.
Atom changes only the layout of the text segment, so relocation records that describe the data segments are not needed . Second, the remJining relocations can often be predicted by analyzing other p:u·ts of the cxecutJble file. This property is used to store a compact torm of the remai ning relocuion records.
Since compact relocation records arc represented in J difrerent form tban regular relocations, thev are stored in the .comment section of the object ti le .rather than in rhc normal relocation area.
Profiling-directed Optimization
OM and the Atom- based pixie tool can interoperate usi ng profiling-d irected optim i zation . The Atom based pixie too] is a basic block protilcr th;lt provides execu tion cou nts for each basic block when rhe pro gra m is r u n . The execution cou nts are then used as input to OM for performing optimizations on the exe cutJble that are d ri ven from actual run-time pert()r mancc data .
As an example, the fol l o\\'ing steps would be perkm11ed to utilize p rofi l ing-directed optimizations with
OM : l . % c c
- n o n _ s h a r e d - o p r o g r a m * . o
2 . % a t o m - t o o l p i x i e p r o g r a m
3. % p r o g r a m . p i x i e
4. % c c
- n o n _ s h a r e d - o m
- W L , - o m_i r e o r g_ f e e d b a c k , p r o g r a m * . o
I n step l , a nonsharcd version of the program is built. In step 2, the Atom- based pixie tool i nstru ments the progra m . Step
2 p roduces progra m . pixie and program .Addrs files. Step 3 resul ts in the exe cution of the instrumented program to generate a progra m . Counrs file. This ti le contains an execution cou nt for each basic bl ock in the progra m . The last step provides the basic block profi le as input to O M .
O M rearranges the text segment o f t h e program such that rhc most freq uently executed basic blocks and proced ures are placed in proximity to each other, rhus improving the instruction cache ( I -cache) hit rate.
Product Development Considerations
Bringing the OJ\11 and Atom tec hnologies from the lab oratory into use on current Digital UNIX production systems required lrcquem communication a nd coord i nation between WRL and DUDE engineers working on opposite coJsts of the U.S. The success of both proj ects depended upon establishi ng and mJintaining an atmosphere of cooperation among the engineers at the two locations. Common goals and criteria for bringing tJ1e technology to prod uct supported the teams d uring de,·elopment and planning work.
Among the product development consid erations tor OM and Atom were l . The products must Jdd ress a busi ness or customer requirement.
2 .
The products must meet customer expectations of features, usability, qu ality, and pertormance .
3.
Engineering, qu;�lity assurance, and documentJtion resou rces must be identified to ensure that the products could be enh;1tlced , updated to operate on ne\1' pl atform rclcJses, and supported throu gh out their l i fe cycles.
4.
The products must be released at the appropriate ti mes. Releasing J prod uct too early could resu lt in high support costs, perhaps at the expense of new development. Re leJsing a prod uct too late could compromise Digital's ability to leverage the new tech nology most effective ly.
Product Development and Technology Transfer
Process for OM
As part of their research and development et!orts,
WR.L engineers applied OM to large applications.
Researchers and Digital engi neers Jt ISV porti ng bbo ratories worked together to share intormation and to diagnose the performance problems of programs i n
26
Digit:ll
TedlJlical )ourn:ll
Vol.
8
No. I 1 9\!6
use on actual prod uction systems, such as relational database and
CAD applications. This cooperative eftort helped engineers determine the types of opti mizations that would benetit the broadest r:1nge of applications. In addition , the engineers were able to identity those opti mi zations that would be usefu l to specitic classes of applications and nlJke them switch-selectable through the OM i nterbce. The per tormann: improvements achieved on ISV :1ppl ications en:1bled OM to meet the criteria for :1ddressing cus tomer needs.
AJ though WRL researchers also :1pplied
OM to the
SPEC bench mark suite to me:1sure pedormance im provements, the primary tocus of the OM tech nology development was to provide perf(xmance im provements for applications currently in widespread use by the Digital U N I X customer bJSt:. With the focus of performance i mprove ments on l arge cus tomer appl ications,
OM satisfied a prominent Digital busi ness need tor inclusion in the D igital U N I X devel opmellt environment.
Engineers d iscussed the limitation that OM did not support shared libraries and the programs that used them . In this respect, the technology would not meet the expectations of all customers . Many ISV :1pplica tions and other performance-sensitive programs, how ever, are built nonshared to improve execution times.
Engi neers determined that the bendits tor this class of application outweighed this limitation of
OM, and, theref(xe, the limitation did not prevent moving forward to develop the prototype in to J prod uct.
Developers recognized the risks and support costs associJted with shipping the prototype, yet again decided that the proven benefits to existing applica tions outweighed these factors.
Because of the pressing business and customer needs tor this technol ogy, DUDE and WRL engineer ing concurred that
OM should be provided as a ful ly su pported component i n Digital U N I X version 3 . 0 .
Full prod uct development comm itmenrs from D U DE engineering, documentation, and quality assurance could not be made tor that release, however. Nte r discussion, WRL provided technical support :1 n d extensions to
OM t o address this need . D U D E engi neering agreed to provide consul ting support to WRL researchers on object tile and sym bol table f(xmats and on eval uations of text and data opti mizations.
The next issue the engineers faced was how to inte grate
OM i nto the existing development environment.
They evaluated three approaches.
The first approach was to make
OM a separate tool directly accessible to users as /usr/bi n/om . Thus, an application developer could utilize
OM as a separate step during the build process. This approach ottered two advantages. First, it was similar to the approach used to achieve the present internal use of OM and would require kw additional mod ificatjons to the
Digital U N I X development environment. The second advantage was that Atom and OM could be more easily merged i nto one tool since their usage would be similar. This merging would provide the potential efficiencies of a single stream of sources for the object modification technology.
A major disadvantage of this approach was that it put additional burden on the application developer.
OM req uires a specially linked input file p roduced by the linker. This intermediate input file is not a com plete executable nor is i t a p ure OMAGIC fi le.10 This approach would require customers to add and debug additional build steps to usc
OM on their applications.
The WRL and D U D E engineers agreed that the user complexity of this approach would be a significant bar rier to user acceptance of
OM.
The second approach was to change the compiler driver to i nvoke OM for linking an executable. With this approach, a switch would be added to the com piJer d river. If this switch was given , the driver would call /usr/libjcmpl rs/cc/om instead of the system linker to do the tina! link.
This approach had the advantage of reducing the complexity of the user interface for building an applica tion with
OM.
A developer cou ld specifY one switch to the compiler driver, and the driver would automaticaJiy invoke OM. This would al low a developer to introduce feedback-directed optimizations into the program by simply reli nking with the profiling information , thus making OM easier to use and less error-prone.
The disadvantage of this second approach was that the complex symbol resolution process in the linker wou l d need to be added to OM. The process of per forming symbo l resolution on Digital
UNIX
operating systems is nontrivial . There are special rules, boundary conditions, and constraints that the li nker must u nder stand. OM was designed to modifY an already resolved executable, and any problems introduced from adding linker semantics would discourage its use. Also, dupli cating l i n ker capabilities in
OM would req uire addi tional overhead in mai ntai ning both components.
The advantages and disadvantages of the second approach motivated the development of a third approach. The compiler drive r coul d be changed to invoke
OM during a postl ink optimization step. As in the second approach, a switch from the developer would trigger the invocation of
OM; however,
OM would be run atter the linker had performed symbol and li brary resolu tion.
The third approach is the one currently used . This method maintains separation between the l i n ki ng and optimization phases. When directed by the
-om switch, ld prod uces a specially linked object that will be used as input to OM. The compiler driver supplies this object as input to OM when the li nking is completed .
Digiral
Tc::d111ic�l )ou rn.1l
Vol .
8 No. l 1996
27
2K
The WRL : md D U D E engineers tound rh:n this fu nctional scpar�1tion also i mproved the cfricicnc1· of the development cfr(>rrs between WRL and DUi) E .
The sepa ration permitted concurrent Wl\L develop ment on OM and D U D E development on ld, with minimal inrerkrcncc. This approach Jllowcd more development ti me ro be dedicated to reclmic1l issues rather than de:- d i n g ll'ith sou rce manJgemcnt :111d i nte gration issues.
D U D E engineers :tddcd the OJ'vl sources into the
Digi tal U N I X development p updates
o
ol and intcgLncd
ti-om
WRL. WRL assu med responsibi lity t()r testing OM prior to providing source up(btes. As pre viously outlined, D U D E enginee rs i ntegrated su pport tc)r OM into the existi ng de1·e lopmenr Cill'iron menr tools tor the i nitial release.
Because of pro1·en pcrt()rmance impro1·uncnrs on
I SV applications, committed engineering cth>rts lw
WRL,
.__,
' b nccrs judged the technologv mature enough tc>r rele�1sc on prod uction systems. Efficient development str:ltc gies enabled Digital to rapidly turn this leadi ng-edge t echnology i nto a prod uct that bendits an import:mt segment of the Digital U N I X c ustomer b�1se .
W R.L com in ued engineeri ng support tc>r 0Jvl through the Digital U_:"\!I X 1-ersion
3.0
:md 3 . 2 releases.
Responsibilit1· t(>r the teclmolot,'\' gr:1du:�llv shifted ti·om WRL to D U D E m
·er the course of these releJscs.
Cu rrently, D DE fu llv supports �md en h�1 11ces OM while WlU. comi nues to provide consultation on the tech nology :tnd input for ti.1ture improv e m c ms.
Product Development and Technology Transfer
Process for Atom
W IU_ deploved c1rlv 1crsions of the Atom tool kit J t intern a l D igit:tl sites, I S V port i ng laboc1tories, and u n i,·ersiries, thus :�! l owing de1·cl opers to cxperimem with and evaluJte the Atom API. The cJrlv JV<l iLlbiJitl' of the tool kit promoted use of the Atom 'techno log).'·
User feedback :md requests for katu res helped the engi neers to more qu icklv and etketil'elv de1·elop J robust technologv fi'Oill the prorotl·pe .
Engineers thmughout Digital recogn i ze d Atom JS a unique and use fu l tec hnologv. Atom 's A P I , "·ith instrume nt:ttion <1 11d �lll<llysis eapJbilities doll'n to the instruction leve l , increased the power Jnd diversity of tools that could be nc::�red for soti:w;,rc :tnd h:lrdwJre development. Hardware development tc1ms used
Atom to sim u l ate the perto n n a nce of new Alpha impleme ntations. Software develope rs crc:�ted and sh:1red Atom tools t(>r debugging ;,nd mosuring pro gr�1111 ped(mn�lncc . The ,·:�lue of the Atom tec hnologv in soil-ing applicnion de,·elopment problems prm·ided the business justi fic a t ion t(>r developi ng the tcclmol ogy into a prod uct.
The prototl-pc version of Atom had se1-erJl
I i m i ta rions.
•
•
•
•
Like OM, the prototype version of Atom worked only on nonsh:1red applications. A prod u ction version of Atom would require support tor cJII sh�lred programs and si1Jred l i braries, since, bv debulr, programs arc built as cdl-sharcd program
�
.
A 1·iablc Atom prod uct ofkring needed to sup port these npes of progr::�ms, i n �1dd irion to non shared programs.
Progr::uns needed to be rel i n ked to rctJin relocatjon i n f(mn:ttion bd(>J'C Atom cou ld b e used . This add i rionJI build step impaired the usJbil i tl' ofAro m .
T h e Atom prototvpe p erform e d p oorly because it consu med a l�1rgc amoullt of memorv. 1VI uc h of the d ar:� collected about an excc u t:� ble t()r opti m i zation purposes 11·.1s not needed
[()r progr:t1n :tna ll·sis rr�l nst(mn<l rions .
The Atom API req u i red exte nsive d esign :md de,·e l opmenr ro support cJll-sh:-�rcd progr:� ms and sh:t red l i br�1rics.
The e ngineers decided to
aile)\\·
the OM and Atom te ch n
o
logie s to d il'erge so that the d i fkring requ i re mellts t()r optimi z ation and progra m Jn:�ll·sis cou l d be more etkctil'l·l l· :tddrcssed in eac h component.
Bec:�usc the cost of supporti ng :-� rcJuse ofrhe Atom proto t ype 1\'0uld have been high , WIU , a11d D U D E engi neer i n g developed a str:�tcgy tclr simultaneously releasi ng the Atom p rotot y pe while tC>C using engine e r
ing efforts on developing the prod uction version. An
A rom Ad1·� m c c d Dc1T iopmenr Kit ( A D K ) II' J
S released ll'irh Digit�1l U N I X 1·ersion 3 . 0
as
the initial step of the stra reg1·. The ADK prOI'ided customer <lCcess to the technologv 11 ith l i m i ted su ppon. Engineers 1 ic11ni the lack ofsuppmt for shared cxecur:.�blc objects as
Jn
::�ccept�1blc limir�nion tor rhe Atom A D K bur u n::�ccept ablc f(>r the tim I product.
In addition to <111owing WRL Jnd D U D E engineers ro t(xus 011 f1roducr development, this tirst str�ucgic step p ermim.
: d the engineeri ng te::Jms more time a nd flexi bilitl· to incremenullv add support f(>r Atom into otl1er prod uction components, such as the l inker ::�nd the lo:�der. for usabi l i tl' purposes, minor extensions ll'ere made to the loader to all o ll' i t ro auronutic:tllv load i nstrumented sh�1red libraries prod u ced by Atom tools.
The second step of the strategy W<lS to provide updJtcd Atom kits to users JS de1·cl opment of the soft ware progressed . These kits i ncluded the source code tor
nample
tools, m:.�nuals, and reference pages.
The
upd ate kits pnt<mncd t11·o fu nctions; ti1CI' su pported users and thel' pro1·ided feed back to the de1·elopment teams. D U D 1-'. and 'vVRL e n ginee r s posted information i n rern::� l lv ll'ithin DigitJ I II hen kits ll'erc <li'�Jilab le and de1-eloped ;1 m�1 i l ing l ist of Atom users. The Atom user
Vol . � �o. I 1 996
com nHmitY gre,,· to i nclude un iversities otem:t l customers.
<l!Ki se,·eral
Once the Atom ADK :tnd u pd < l te str:ttegv were est:tblished , D U D E en g i neering beg:� 11 to implement support t(>r Atom in the linker. As mentioned earl ier,
Atom i nserts text i nto J progr:un and req u ires reloca tion i n tonnation to cre:tte a correctly instru mented executable. The Atom prototype required a program to be l i n k ed to retain the reloc:ttion i n t(mnation , and this requirement presented
J
US<lbil itY problem tor users . ldeal lv, Atom wou ld be able to i nstrument the ot:cutables and shared libr,lries prod uced b v the linker. l1\' defa u l t
Mod ifying the l inker to retJin all trad itional reloca tion int(mnation by debult
W<lS not a cc eptable since the size increase in the executable wou l d have been prohibitive . [n some cases,
40
pncent of the object fi le consists of relootion rec o
r
ds. Engi neers did not view
<lll increase of that magn itude as <l viable solu tion. [n
:t d di tion, this solution con flicted with the goal of
Di gi tal U N I X versi on
3.0
to rt:d uce obj ect fi le size .
As a compromise,
D U D E engineeri ng i m p lemented comp<Kt re location support in the l in ker. Co m pact relocJtions prm·ided an acu.:prabk sol ution since they rt:qu irt:d t:1r less space th<m n:.:gul:tr re l oution records, typically less than
0 . 1
percent of the total tile size .
Another side effect of using comp:tct relocations as a solu tion wJs that i t introd uct:d a dependency between
Arom :tnd ld
.
Al l execut:t ble objects ro be processed by
Atom needed to have been gt:ne ratt:d with the l in ker that cont<lined compact reloc:nion support. There t(>re , to support Atom, l:tvert:d product libr:tries and third -parr\' li braries needt:d to be rt: l i nked with the comp:1ct relocation support.
In Digital U N I X version
3 . 0 ,
ld w:ts rnodi ned to genuart: compact reloc:nion int(mnarion in exe c u t:tblc objects. This a l lowed Atom to i nstru ment the debult output of ld.
Engi ncus viewed this capabi lity
;1s critical ro the usability and u l tim:tre su ccess of the
A rom tt:chnology.
The comp�lct relocation support i n l d
\\':tS rdint:d
;md
e xtended m·er the course of se,·eral
Digit:�! U N I X releases
Atom progressed .
;1S de,· c lopmenr work \\'ith
Concurrently, the WRL rese
::t r c h teJm expanded
JIH.l heg;m dn-elopment of tht: Atom Third Degree
;111d hi prof tools. WRL engint:ers also conr i n ued with
:tddi tions and i mprove ments
to
a suite of exampk
Atom tools.
After rhe rele::tse of Digital U N I X version
3 .0 ,
D U D E b egan design and deve l op m ent o f the prod uc tion ,·ersion of the core Atom rechnologv and the API .
D U D E engineers mod i tied :tnd extended the Atom
API as tool development progressed at WRL D uri ng peak development periods, en g inee rs discussed design issues daiil·
by
telephone Jnd electronic mail
.
Tht: original Atom AD K included the source code t(>r a nu mber of example Atom rools. Bec:tuse some of these tools contained h::trdware implementation dependencies, they would rt:quire ongoing and long tt:nn support to rem::tin operarion;11 on changing implementations of the Alpha <1rchitt: c tu re . For the sec; ond shipment of the Atom A DK in Digit:� I U N I X ver sion
3 . 2 ,
rhese high maintcnJIKe tools \\TIT remO\·ed
:tnd m:�de :t\·ai l:tble through unsupporrt:d c lunnels.
Bet\\'een releases of the AD I<.
011 the Digi t<J I U N I X operating
S\'Stem, the engi neering tt:<Jms continued to d e liver upd<Jte kits. Engineers schedu led delivery of tht: update kits to coincide with key milestones in the softw;HT development process. This srrategv ga,·e th e m more c ontro l over tht: development schedu l e
;1nd mi nimized risk. The update kits provided immedi
Jtc field test exposure tor the evolving Atom software.
The design , development, :tnd kit process was prac ticed i re r:ttively over a year to develop the origina l idc1s into J ful l product. T h e Atom update kits were provided tor
D i g ita l UNIX ve rsion
3 . 0
:tnd later
S\'S rt:ms, since most users did nor ha\'t: access to earlv \'er sions of Digita l U N IX 'usion kits
4 . 0 . Providing Atom t(>r use on pre-version
4 . 0 svste ms :tl l owed the sotrware ro be exercised in the ricld on acnwl applica tions prior to its i niti:tl release as a supported prod uct.
A l though support for earlier operating system versions added overhead and compkxiry to the process of pro viding tht: upd ate kits, tht: t:ngineni ng reams val ued the abu nd:�nce of user kedback that the process
\'iekkd . The benefits of usn input to the software dn·dopment process mmve ighed the on:rhc:td costs.
During Digital
UNIX
,·ersion
4.0
development ,
W R I . engi neers finalized the i mpkment:ttions of the hi prof and Third
Degree tools :tnd tr:tnskrred the tool
SOIII"C(S to D UDE tor fu r ther development. The vVRL developers l1:1d added su pport tor thrcJded applica tions on pre-version
4.0
Digital UN I X systems.
Because the i mplementation of threads c hanged in
\nsion
4.0,
D U D E e ngineers needed ro update the
Atom tools :teeordinglv.
D U D E engineers also developed an
Atom- based pixie roo! "·irh su pport tor threackd <l pplications. In
EKt, the Atom -based pixie tool rcplact:d the original version of pixie in Digiul U N I X \'ersion 4.0. The
A to m- b a s e d pixie allowed new katures such as sup port t(>r shared li braries :t nd rhre:tds to be more efticiently added into the prod ucr ofkri ng. The devel opment of:tn A tom -bJscd pixie tool solved the exten sibility problems that wert: being bced with the original \'ersion of pixie.
WRL. engineers a lso began
to
use Atom tor i nstru menting prt:-\ ersion
4.0
D i g it:tl U N IX kernels, de,·el op i ng speci
:t l tools for col l ecring kernel statistics.
Atom w:ts ex tended by D U D E engi neering as needed to support instrument:�tion :tnd analvsis of the kernel.
Di�iral Tcdmiol J ourn�l
Vol . il
No.
I
1996 29
The Atom roo! kit and example tools were shipped
With D 1gnal UNIX version 4.0. The pixie, h i p rof, and
Third Degree tools were shi pped with the Software
Development Environment subset of Digital
UNIX version 4.0. Research regarding usc of Atom for kernel instrumentation and analysis continues.
WRL
continues to share ideas and consults with D UDE on the future directions for the Atom technology.
Conclusions
Developing
OM
into a prod uct directly tl·om research proved to be chall enging. Problems and issues that needed to be addressed hrl.d to be hand led within the schedule constraints and pressures of a commi tted release plan .
In contrast, the ADK method used to deliver the
Atom product allowed the Atom developers to spend more time on prod uct development issues in an envi ronment relatively free from the pressures associated with daily schedules. The ADK mechanism, however, probably lim ited the exposu re of Atom technology at some customer sites.
The close cooperation of engineers from both research and development was necessary to accom pl ish the goals of the two projects. We believe that a colla borative development parad igm was key to suc cessfully bringi ng research to product.
Future Directions
This paper describes the evolution of the
OM and
Atom technologies through the release of the D igital
UNIX version 4.0 operating system. Digital plans ro investigate many new and improved capabilities, some intended for fi.Jture product releases. Plans are under way to
•
•
•
•
•
•
•
Provide support i n
OM t(x call-shared programs and shared libraries.
Support the use of Atom tools on programs opti mized with
O M .
I nvestigate provid ing an A P I to al low program mable, customized optimi zations to be delivered through O M .
Investigate reuse o f instrumented shared libraries by mul tiple call -shared programs that have been instrumented with the same Atom too l .
Research support for Atom tools that provide sys temwide and per-process analysis of shared li braries.
Extend Atom to improve ke rnel analysis.
Simplify the use of the profi ling-directed optimiza tion Facil ities of Atom and OM through an improved interface.
•
Extend the Atom tool kit to provide development support for thread-s:�fc program analysis tools.
In addition to enhancements to the Atom product, ongmal Atom- based tools are expected to become avail able through the development activities of students and educators at universities. I nternal Digital developers wiU con tinue to develop and share tools as wel l .
Acknowledgments
M:�ny people contributed to the development of the
O M and Atom products. The tollowing l ist gives recognition to those most actively involved. Amitabh
Srivastava led the research and development work at
WRL on
OM and Atom and mediated manv of the design d iscussions on the Atom design . Greg
L ueck of
D U DE designed and implemented the production version of Atom , compact relocations, and the Atom b:�sed p i x ie too l . Alan Eustace developed Atom exam ple tools, created the f-Irst Atom ADK, worked diligently with users, developed kernel tools, provided training and documentation on using Atom, and dis played eternal optimism. Russe l l Kao at
WRL con tributed the hi prof tool with thread support. Jeremy
D1on and Louis Monier at
WRL developed Third
Degree and an Atom-based code coverage tool called tracker. John WiJ liams and Chris Clark of DUDE com pleted the process of tu rning the hiprof, pixie and
Third Degree tools into prod ucts. Dick Butrlar pro
VH.lcd documentation on every component. Last but nor least, the authors wish to extend a fi nal thanks to
Jll the users who contri buted feedback to the
OM and
Atom development teJms.
References
I . F Ch ow, M. Himelst.:in, E. Killian, and L. Weber,
"Engi neeri ng a lUSC
Com piler System," Proceedini.!,S of' COJV/PCON San Francisco, Calif. ( M arch 1 9 86):
1 32 -1 37.
2.
VVestcrn
Research Laborarory, locned on the Web at http :/ jw-ww. research .digital .�om/wrl .
�.
R. Sites and R. Witek, Alpha AXPA rcbilec/ure
Rcfer
enu'
!Vfa nual, 2d ed . ( N ewron, Mass . : Digital Press,
1 99 S ) .
4. A. Srivastava and D . Wa l l , "A Practic<ll System ror
! n rcrmod ule Code Optimiz.ation at Link-rime," jour nal of' Progra mminl� languages. vol . 1 ( 1 99 3 ) :
1 - 1 8 . Also available <lS
( December 1992 ).
WRL
Research Report 9 2/6
S . A. Srj\'aStJ\':1\ '< Unrcach:�blc Proced ures in ()bjccr onenred Progra m m i ng,"
ACM f. OPf.AS. \'Ol.
I , no. 4
( Decem ber 1 992 ) : 355 -364. Also available
�s
WRL
Research Report 93/4 ( A u gust 1 993 ) .
30
Digital Tcchnictl Journal
Vol .
8 No. 1
1996
6. A. Eustace and A. Sri,·astc\\·a, "ATO M : A fl o i b k
I nterface
�or
B u i l d i n g High Performance Pmgr<\111
Analysis Tools," l'mceedin,f.4S of the Wlinter
1 995
CW:::YIX Conference. New
J
995
Orleans, La . (January
)
. AJso avail,lbk as WRL Technical Note T N - 44
(July
1 994 ) .
7.
A.
Srivastava and
A .
Eustace,
"ATO M : A
System for
B u ilding Custom ized Program Analysis Tools," Pro
ceedi118S of t be SIG'I'LAN 94 Conference un
Pro p,ramrn ing Lanp,uap,r:
Orlando, Fla.
Oes(r<n and !mplemr:ntation.
(J unc
1 994
). AJso ava ilabl e as
WRL
Research Report
94/2
( M arch
1 994 ).
8 . A. Srivastava and
D.
Wal l , "Link-Time Optimization of
Address Calculation on
;J
64-bit Architecture," Pro ceedings of the
S!G'PLAN 94
C
o n
f
e r e n c e on f'ro wwnm ing Languagr: /Jesip,n and
!mplemr:ntation.
Orla ndo, fla. ( J une
1 994).
AJso available as
WRL
Research Reporr 94/ 1 ( 1-'e bruarv 1 994).
9 . Digital
U;VIX
Ca//in/1, Standard for Alpha
5)•slems.
Order No.
AA PY8AC TE,
Digital U N I X version
4 . 0 or higher ( Maynard , Mass . : Digital Equ ipment Corpo ration,
1 996 ) .
1 0 .
Di{!,ital
UNIX Assemhlv Ictnguap,e Programmer s
Guide, Order No. AA- PS3 1 C-TE, Digital U N I X ver sion
4.0 or h igher ( !VL!ynard , l'vlass . : Digital Eq u i p ment Corporation, 1 996 ).
General Reference
} .
Larus and E. Sch narr, "EEL: Mach i ne- I nde pendent Exe cutable Editing,"
S!CI'LAN Conference on
Programm inp,
L a n g ua g e
Design and Implementation (June
1 995 ).
Biographies
Craig
A.
Neth
Craig eth is <\ princi pc1l software engi neer in the Digital
UNIX
Development Environment group, where he is the technical leader of link time tools. In prior positions at
Digital , Craig has worked on the OM object modi�ication tool and the VA X and
DEC
COBOL compilers, and led the de,·elopmenr of
D EC COBOL
''ersions l and 2 . Craig received a
B . S . in computer science from Purdue University in
1 984 and an
M .S . in (()mputer science �rom the
UniversitY of I l l i nois i n 1 9 86.
M ichael J. Rickabaugh
Michael Rickabaugh is a principal sofuvare engi neer in the Digital U N I X Devel opment Environmenr group.
He started his D igital career in 1 9 86 in the SEG/CAD
Engi neering group as a software engin ee r on the
DECSIM logic simul ation project. I n 1 99 1 , Michael transitioned to the DEC OSF I l AXP pro
j
ect ;�nd was a member of the original team responsi ble for del ive1·i ng the
U N I X development environment on the DEC
O S !-' I 1
AJ pha platform . He has si nce been a technica l contributor ro all aspects ofrhe Digital U
IX l ink rime tech nology as well as the creator of the A S
A)\
P assembler f()r the
Windows NT operating syste m . Michael received a B.S. i n elecrrical and computer engineering �l·om Carnegie
Mellon University in 1986.
Linda S. Wilson
As a principal software engineer in the Digi tal U N I X
Develop ment Environment group, Li nda Wilson leads the development of program analysis tools for the Digital
U N I X operating syste m. I n prior positions, she was respon sible f()r the del ivery oforhcr development environment components, including DEC:
FUSE, the dbx debugger, and run rime Jibraries on the U I .TRJ X and Digital
U N I X oper;Hing systems. Linda rccei,·ed <l B.S. i n computer sci wee from the Uni\'ersirv of N ebraska-Lincoln. Before joining Digital in 1 989, Linda held software e ngineering positions at Masscomp i n Westford , Massachusetts, and
Texas Instruments in Au sti n , Texas.
Digital T\:chni(:tl /ournal Vol . 8 No. I
1996
3 1
Design of eXcursion
Version
2
for Windows,
Windows
NT,
and
Windows 95
I
Jolm T. Freitas
James G. Peterson
Scot A. Aurenz
Charles P. Gu..ldenschuh
Paul J. Ranauro
Version 2 of the eXcursion product i s a complete rewrite of the successfu l Windows-based X server software package. Based on release 6 of the X Window System version 1 1 protocol, the new product runs on Microsoft's Wi ndows,
Windows NT, and Windows 95 operating sys tems. The X server is one of several components that com pose this package. The other com po nents are X Image Extension, the control panel
(which constitutes the user interface for product config uration), the error logger, the appl ication lau ncher, and the setup program. An interprocess communication facility enables the eXcursion components to communicate in a un iform fash ion under all three operating systems. A unique server design using object-oriented prog ram ming techn iques integrates the X graphics con text with the Windows device context i nto a combined state management faci l ity. The result ing implementation maxim ized graph ics perfor mance while conserving Windows resources, which are in l i m ited supply under the 1 6-bit version of the Windows operating system. The control panel was coded completely in the C++ programming language, thus making full use of the Microsoft Foundation Class li brary to m i n i m ize development time and to ensure consistency with the Windows user i nterface parad igm.
Digital developed the eXcursion family of d isplay server products to provide interopcra bi lity between desktop person:� I computers (res) running the
Microsoft Wi ndows operating svstcm and remote hosts running the X \iVindow System operating system u nder the UNIX or Open VMS oper:ning svstems. The fi rst version of the eXcursion X server was a 1 6-bit application written specifically tor Microsoft WindO\vs versions 3 . 0 :�nd 3 . 1 . As the pop u l arity of W i ndows increased :tnd desktop systems were connected to cor porate networks, the market tor X intcropcrability grew q u ickly. The 1 6 - bi t eXcursion code, much of which h:�d been ported from 3 2 - bi t
U N I X cod e , was
:�gain ported-this time ro Microsoft's Win32 appli cation progr:unrning interface (API ) to support the
Windows NT operating system . When release 6 of the X Window System version l l protocol ( X l l R6) appeared and
J new sample i mplemcnt:-ttion source kit bcome :�v;�ilable ti·om the X Consortium, the eXcursion t�.:am decided that it was ri me for a complete rewrite of the eXcu rsion sothva rc. M icrosoft had established the Win32
A P I as
J u niform cod ing inter face tor all its Windows- based oper:�ting systems. Since developrncllt tools such as 32-bit compilers and debuggers of sufficient quality and robustness had become :w;�ibble, it was now possible to i mplement a high qual ity , 32-bir product. Th is prod uct would sup port the entire range of Windows-based platforms, from note book PCs running the Wind ows operating system to high-end Alph;� systems running the
\rVind O\vs NT operating svstem.
Terminology
This paper incorporates certain convemions to clarify the distinction between the two window systems u n der consideration . X window rekrs to rbe collection of data structures, concepts, and oper:�tions that con stiture a \Vind ow, as defined in the X vVindow System envi ron ment. \\lin32 tcindut .
t ' refers to
:1 windO\\.
JS detined in tvlicrosoft's Win32 API .
\Vhe n referring to a window system ::�s opposed to a particubr wi ndow instance, X I'Vindou•
System is sometimes abbreviated to X. \l(linduws denotes the Microsoft Windows operating syste m.
32 Digit� I Tec hnic:1l j ournal
Vol . S No. I 1996
lore that the word
bitmap
has more th•m one meaning. In the X environment, a bitmap is a two dime nsional <1rray of bits, and a
j>ixnwj>
is
<1 two di mensional array of pixels, where each pixel may consist of one or more bits. Under the Win32
API,
the term bitmap is used exclusively; that is, no distinction is made between an array of depth I and •m array of
(kpth n . ln this paper, the term pix map is used i n its general sense to refer to X pixel arrays, and the term bitmap rdcrs to the Win32 concept.
Another common point of conf"l1sion when dis cussing the X Window System environment is the use of the terms seruer and
client.
To one familiar with til e and print servers, t h e meani ngs o f these rwo terms i n t h e X environment mav seem t o b e reversed . I n the X em·ironment, the server is a disp l av server, and the clients arc rhc applications reque ring display services.
The X server and the X client applications may reside on the S<1mc PC, but rl1e power of the eXcursion soft ware is in its abi lity to bridge the gap between the
Windows desktop and the trad itionJI X 1 1 UNIX and
Open VMS workstations. eXcursion Version
2
Product Goals
The design of eXcursion version 2 was driven pri marily by the fol lowing prod uct goals:
•
•
•
•
•
•
•
Support X Wi ndow System version l l , release 6.
Support the Microsoh Wi ndows, Windows NT,
Jnd Windows
95
operating systems.
Code the single source pool to Microsoh's Win32
API.
Exceed graphics performance of eXc ursion version
1 as mcasured with the standard benchmark tests
X l l pcrf:md X bench.
Preserve maintainability by using mod u lar coding
:111d limi ting changes of the sample implementation ti·om the
X
Consortiu m .
J\tLnimi7.e rel iabilirv by performing extended error checking and resource management.
Correct known protocol contonnance deticiencies in version 1. For example, in version I , plane mask support wJs implemented for only a few graphics operations. Version 2 would provide pbne mask support tor all graphics operJtions.
Components of eXcursion Version
2
In eXcursion version 1 , most of the hmctions provided by the product were combined in ;1 single executable.
To conserve resources and to partition the code for easier maintenance, version 2 is divided i nto several scpJratc components or mod u les. Some of these run
JS individual processes, and some arc built as dynamic link libraries ( D LLs) .
A
DLL is a shared memory l i brary modu l e that is linked to the call i ng program at run timc. eXcursion \'Crsion 2 is partitioned into the tollowing major components:
•
•
•
•
•
•
•
X server. The
X
server is the primary component of eXcursion version 2 . The
X
server process is respon si ble tor displaying wind ows and graphics on the
Windows desktop Jnd tor sending keyboard, mouse, and other evcnts to the client application .
X I mage Extension . X extensions are additions to the server that support t"lmcrionaliry not addressed by the core X l l protocol, such as displaving shaped
(nonrectangular) windows, handling large requests, testing/recording, and imagi ng. Al l extensions except the X lm:1ge Extension ( XJ E ) are i m p l e mented internal l y in t h e X server. Because o f i ts size,
X I E
is implemented as J pair of DLLs, one tor
XIE
version 3 and one tor X I E version
5 .
Control pan e l . As the primary user i ntertJ.ce, the control panel provides the user with access to the many con tiguration setti ngs. It is an i ndependent
Win32 application i mplemented using M icrosoti:
Visual C++ and the Microsoft Foundation Class
( MFC) library. lnt erprocess com munication library. The inter process comm unication
( ! PC ) l ibrary is an operat ing system-independcnt l i brary used by cooperating processes or tasks to communicate configuration and status information .
Error l ogger. The error logger is a simple Win32 application that records error and status intorma tion from other eXcursion components in a wi n dow, a file, or the Windows NT event log.
Application launcher. The appl ication launcher is a
Wi n32 appl ication that starts X client applications at the request of the X server or the control panel .
The application launcher is i nvisible to the user.
Registry interface. The registry i nterface is an operJting system-independent i nterface to the eXcursion configu ration proti k . The registry i nter face is implemented as a Win32 DLL.
X Server
The core of the eXcursion product is the X server, a
Win32 application that accepts X requests from client applications and transforms them i nto graphics on the
Windows desktop. The device independent portion of the server code is ported from the sample implementa tion provided by thc X Consorti u m . The device dependent portion treats the Wi n32 API as the de,·ice interface through which client requests are material ized on the screen . The eXcursion X server is il lus trated in Figure I .
Digiral
T.::chnical )ou rn.1l
Vol. 8
No. 1 1996
33
X REQUESTS
X EVENTS
NETWORK
TRANSPORT
INTERNAL WINDOW MANAG ER
� i
DEVICE-INDEPENDENT DEVICE-DEPE NDENT
CODE
CODE r---
WI N32 FU NCTION CALLS
WI N32 MESSAGES
Figure 1
The e Xcursion X Scrl'cr
The scrl'cr can operate in o n e o�- tii'O modes: s i nglc windoll' mode or multiwindoll' mod e . I n si n glc ll'i ndow mode, the sen-er creates
11·hich n:prcsenrs rbe one \rVi n 3 2 ll'ind oll',
X root 11·indm1·. All desce mbnr ll'indows :md their contents arc d L111·n i mo the ro o t l l 'i ndow us i ng Win32 fi. tnction calls. In multi ll'indoll' mod e
, the root windo11· is a l'irru�1 l ll'i ndoll'; that is, it is ne1·er drawn on the screen. E:1ch to p
- l evel child of the root wind m v has a corre s p o nd i ng \rVi n 3 2 window, which is created when the X wi ndow is dcsn:mbnts of a top
leve m apped . All l window ar c d r�1wn i nside the Wi n32 window with Win32 calls. M u l tiwindow mode r h c re bv creates a deskto p eill'ironmcm i n w hi ch
X app l i c a ti ons �1re peers of other
Win32 �1pplic:nions.
Single-11 in dow mode is usehi l h:Jr e
m
ul ati ng a c o m plete II'Orkstation e1wi ro
nm
ent i
n
cl ud i
n
g
the ll'ind oll' nnn:1gcr �md the
sess
i
on or deskto p
manJgcr. I n
mu lti windml· mode, drall'ing to :1nd getti ng in
p
u t t-i·om the root w c
i n
do
w
is r estricted b\' rbc X sen·cr to pl"CI'ellt
o
n
tl
icts
with the M icrosoft \rVi ndo11·s
S \'Stn
n
's use of the desktop 1\'i ndow. Despi te this n:srricrion , the multiwindo
w mode, when used
with the native ll'in
doll'
manager, provides the c leanest i megr:1tion of rh
c
X ;md \rV i ndows environmen
r
s.
Resource Management and Performance
Both the X and \Vi n32 S\'Ste m s h.11·c bui lt-in notion s of grap hics sr.lte and resource alloution. The se ma
n
tics and us:1ge of th e concept, ho11 e1·cr, �1 rc qu
i
re dif ferent in the
t11 0
ll'i ndoll' S\'Stem s .
I n X , g r ap hics St<lte is m
:� i n r ain cd i n �1 dar:� stru c ture knoll'n as a graph ics context ( GC ) . A GC h�1s an inde p e nd ent existence and mav be crcJted , dcs troved , u pd :ned , q u er i ed , and co p ied ar will lw the X J ppl ica
rion . During grap hics operations, �1 c_; c i s �1ssociarcd with th e
X ''d rawable" ( wi n d ow or pi x m � 1p) being drawn i nto, and infornution ddlnc the o in the GC is used to fu l l y p e r a tion . For exa m p l e , rhe GC may specit\r f(Jregrou nd or background in tcmn�1 tion. colors, line Sf)'ICS,
01"
�( mt
T h e Win 3 2 API has a co
n
c e pt
oiled
�1 dnicc con te\t ( DC ) , l
l
'hich a l so coma
i
ns
state i n tcmnarion bur
ll'hosc
p
u rp ose is more c loselv related ro
p
rm·i
d
i ng dc 1·icc i ndependence. Conscq u
cn
r lv, t\I'O d i fferent tvpes of DC:s are req uirt:d under the Wi n 3 2
API,
de pendi ng on ll'bcthcr rhc graph ics
o
perat
i
o
n
is d r�
1
1
1
ing to a 11· do
i
n
d
m
1· or to :1 bi tmap
.
Furthermore, a II i n
l
l ' DC mal' b e allocned either
p
er m a
n
e
n
t
!
�
or ti·om a cache, depe
n
d ing on its opecred l ifeti
m
e . Am· d r all' ing operation therdcll'c requ ires that both the GC used in the X gL
1
p
hi c
s
request Jnd the DC used
in
the ul ti mate Win 32 call be prop
c
rlv
set up and
synchro nize d . The m;ll1ncr in ll'hich this is done has a signi ti
cant etkcr
on the gr:1ph ics pcr rc m na
n
ce of the server.
.Before an
X gr�1phics o
p
ecnion cJn be started , rhe
GC must be va pre par
l
id �1 r c d . Va lidation is a process
o
f
i n
g the o utput devi
c
e to render the gr�1
p
h
ic
s properlv.
I n
the c1sc of the eXcursion server, the out put dc1 icc
is
:� Win32 DC. For e1u�· graphics com m:�nd , the CC must be checked for cha
n
ge s :llld the a ppropri ate Win32 objects .md stare 1·alues must be se lected into the DC This process can be
,
.
c
r
,
· time co
n
sumi
n
g . The kc\' ro max im izing pertormance
is
to recogn ize th:�r most
o
pc Lnions :trc repetitive . A tl-pical stream of X req
ue
sts tends ro contain 111:111\' co
m
m
�mds d i rected ar the same window with the sJme GC.
Therefore, the way to red is ro c:1ch
u
ce GC/DC val
i
da
t
ion rime
c
rhc most recent GC/DC
p
a ir so that subse quent comman
d
s th�1 t use rhe same combination need n ot tr
i
gger a \'�llidarion step. In some cases, graphics operat
i
ons
ll'il l
toggle between t\I'O or
more GCs. ( for
exa m ple, the C:o desti
p
v
Arca
o
perat
i
o
n
takes a s ource and a
n
a
ti
on . )
The pcdcmi1JI1cc in these c
as
e
s can be im pro
l
'e
d
Lw s im
p
h- cach
i
ng more than one rcccnr
GCjDC p:�i r. Tu ning c\pcriments on the sen·cr re1·ealed rhar '' c1che
size
bct\l'ccn
2 and
4
,,·as
suffi cient
to
max i m i ze
pcd(mn:JIJCe.
U
n
der the v
V
i ndOI\ S and \rVind ows 95 opcr;1ti ng S\'stems, are l i mi ted , �' c1ehe si ze
w
he re resources
of 2
is use d . Under the
W
i
n
do
w
s NT oper�ni ng systcl]], the cKhe size is 4 .
In t h e eXcursion server, the notion of
J c ac h e
d
GC/DC pai r i s cnc1psu
l
a ted in a C++ c
l
ass called a
WXDC. The WXDC: remembers the Win 3 2 obj ects th�1t have b een selected imo the
DC
and the last GC that w�1s used ll'ith it. As lo
n
g �1s these clements do nor c
han
ge from
one
gra
p
h
i
cs op
c
LH
i
on to the next, no I
'J
i id:�rion is
n
ecc
s
s
a
r\'. If
the client �1 pplication
ch a
n
ges the co
n
rents of
the (;c, Jll\' �1 ftecred obj ects
in
the DC �11-c tagged an d the not gr
a
ph
i
cs operation on that WX DC w
i
l l require ne11 o hju:rs to be selected into the DC.
Vnl.
K '-.'o. I 1 996
E,·cnts i n the
w i ndo
\\
'
S\'stcm em
also c1use WXDC
clements to become inv:1lid . t:or
example , if
the
win
dow
is m
ov
e
d on
the screen
bv
the
wi ndow man:wn irs clip
J ist may
hJve ch�1nged .
.
This causes
ro invJlidJtc the di
the WX�C
;
p
r
e
g ion in
i ts
DC. ( Clip list and region
arc ddined in the
t
(
J
I I
o w
i n
g
section . ) The
next graphics
operation
on th�lt wi ndow will requ i re the clip r
e
g i
o
n
to
be rec1lcuLued and r c lo :1
ded
.
Clipping in Single-window Mode
I n the X
Windo\\'
System environment, all
desccn dallts of the root
win
dow
have
.1
c
l i
p
l ist, whic h is
a
list of rectangles thJt ddi nes
the visi ble
are�1 of
t
h e win dow.
The
cl i p l ist is equal
to the area o
f the child
win
do w
minus Jny
�11-c�1s tb�1r arc occluded lw other
X
"
· i
n
d o \\
'S
.
Bdore d r
�
1
,
,
·
i n
g
into a descend:1
;
1t win
dow, the
server
m ust co mn
t
the dip
list
i
n
t o �1
Win 3 2
region . In t
h
e
Win32 API ,
a r eg i
o
n i s a polygonal �1rca,
110t
necessari ly
recrangu br,
rlut
can
be s
e
le
ct
e
d i n to a
DC tor clipping.
Bdcm.: initiJting
a
g
r
a
p
h i cs our put operJtion, cur renr reg
the tJrget WXDC
c
h
cck s to sce i f rhe
i
o
n
for t
h
e
window
is vJlid . If
it is
not, t
h
e
X clip l ist bined
is
converted to
:1
Win32
re
g i o n Jnd com
with the
diem-supplied
clip list
in
the
CC, if
�lll \'. The r
es
u lt
is
selected i n ro the output DC.
Clipping in Multiwindow Mode
I n
m u l tiwi ndow
m
od
e
,
the root wi ndow
is invisible.
E:1ch top - b·el
X
w i n d
ow
( tirsr- generation chil d of rhe root ) c orre
sp
on d s to :1 Win32
w
i
n
d o
w
on the desktop.
No cl
i ppin
g is necess�1ry tc>r rhese windows, because
W i
n
3
2 does this au t<Hll�1tic:1l ly. for
windows
below
rhe
rirst g e
n
er ati o n
, cl
i
p
pin
g
is
�
Kcom
p
li
sh
ed in
:1
man ner similar t
o
rh�u
uscd in singlc-window mode, o:ccpt th�lt thc offset of
the clip re
g
io
n
must
be
a
d ju
s
t cd
to
b
e
re l �ni,·e
to
the top-IC\'cl
windo\\' instead
of
reLni'
e to the r
o
o t
\\'indow.
Graphics Rendering
C r
�
1
ph
i c
s
r
e
nd
e
r i
n
g
is ar
the
hc1rr
of t he
X scrvcr. Two
of the core go�1ls for th e c
X
c ur
s
i o n
version
2 pmjcct
werc
to signi ticantlv
improve server
pertorm�1nce m·er rh�lt
of rhe e
X cu
r
s i o n vcrsion l
sen·er
and
to i lllf1l'O\'e scrvcr com pli:1ncc
to
the
X p
r
o
t
o
co
l
specific:1rio n .
hgure 2 comp::trcs t h c pcrt(mnance o f the e
X
c ursi on
\'CI'sion
2 sen·er
\\'ith
th�n of thc
\"Crsion
I
sen·er.
The
st�111Lbrd bench mark rests
X L 1 pu·f
and X
bench
were run o
v e
r
<1 local arc::t network to
eXcursion r
u nn i
n
g on
:1 66-meg<1herrz
vidco cJrd .
Pe ntium
processor with
;1 n S3
The
s:Imple X sen'Cr Uf10n \\'hich
the
eXcursion
X
sc
r\ '
L'
I
' is bJ
s
e d
pro,·idcs
�1
m�1c hinc-independcnr
byer th�lt
is
c:1pablc of
re ndering
:1l l X
graphics th
r oug h �1
sm:1ll set
of devicc-dcpendcnt fu nctions. I n tbc e X cur
s
io
n X
sen·er, rhc Win 3 2 limcrions provide rhe v i rru
�
1 l hardware i n tcrt:lce . For nuximum
pcd(>r m:Jncc, X
g r ::t p
hi
cs requ ests
�1re p
�
1
ss
e d to the Win32
1 00
90 z w
:2 w
>
0 a:
0..
� fz w u a:
0..
80
70
60
50
40
30
20
1 0
0
.--
I
(f) w z
0 f-
(f) w z
:::J r-
I r-
.-r-
� r-
(f) w z
0
_j
_j u:
(f) w z
0 f-
:::J en
(f) w z
0 u a:
<{ z
0 f-
X w fz
0
X w
_j
0..
:2
0 u
P E R FORMANCE BENCHMARK
(f) w z
0 f-
(f)
X
� a:
<{
:2
X
Performance
Benchmark eXcursion
Version 1 eXcursion
Version 2 Improvement
X Bench l i neStones fillS tones bl itStones arcS tones textS tones complexStones
XStones
1 3 5, 735
38,083
239, 740
74, 3 3 1
76.6%
9 5 . 2 %
59, 743
88,320 47.8 %
2, 1 72,720 3, 662,770
68. 6 %
1 56, 1 90
7 1 , 633
80, 057
2 1 4, 7 62
7 1 , 699
1 26,408
3 7 . 5 %
0. 1 %
5 7 . 9 %
X 1 1 perf
Xmark 1 . 6495 2. 5805 56.4%
Notes:
The test machine was a DECpc XL
566.
Si nce eXcursion version 1 did not support
16-bit fonts, the version 2 numbers were substituted to obta i n the X mark number.
Figure 2
Comparison of c
Xcursion
Version l
�u1d
Vc rsion 2
Pntorm�1nce
API
JS C;lrly as
p
o
s
s
i
ble
without c
o
mp
ro
m
i
s
i
n
g
the
requested rendering. Many
X
g r ap
h
ic s requcsts map nearly into Win32
c1
l l
s with
little
or no
datJ
m
a
n
i
p
u
lation.
Some
can not be complex gr:1phics requests, ho\\'e,·er,
pr<Ktica l l v m:1pped
into h
i
g
h
- l
e
vel vVi n 3 2 calls and ac hic,·c proper
pixelization. I n
such u
s
e
s
, the machine-i ndcpcmk n r f un
ct
i ons Jrc c;1 l led
;1S h e l p e
r
fu nctions to bre:1k the request do\\'n i
nto simpler
g
r
::t phic
s
requests.
GDI Context Switch ing
To reduce context
s
wir
c
h
i
n
g
,
W
i
n d o w
s
b a
t
c
h
es g r
aph
i c s device i n rerEKe
( G D I ) c:1Jis. The
debult G D J batch size i s 2 0 , b u t this l i m i t c:1n be
adj usted
per
thread. T
e s
tin
g
with �1
mix ofa l l
X requests siJo,,-cd rh
�
1t Jn m·erall
pe
do r
m
:1
n
c e
in crease
o
fa
b o u
G D I
r
9
percent
cou l d be
achieved
by bJtch limit to 30. At this
b·d,
t
h
increasi ng cr c is
110
m the
e
a
surable l at
e
nc y
, and, ti11·thennore, incre:1si ng r
h
c
batch size beyond this
po i
nt
had no measurable bcndit.
Vol . H No. I l \196
35
Some competing X scn·er products set the b:nc h size very high
( 10 0 )
at the beginning ofcvcry req uest and rlush the queue :lt the end . This approac h has no rnca surable benefit ove r our si mpler method, probably because the Windows operating system alrc:1dy pcr rorms timer-based f l ushi ng to prevent dr:1wing latency.
Similarly, whenever possi ble, Win32 graph ics cal ls are combi ned to red uce the overhead of context switch i ng. For example,
:111
X Polv
L
i ne request cou l d b e rendered with a series o f Win32 Li ncTo c:1lls, but i t is much more efficient to render the PolyLi ne request with a si ngl e Win32 PolyLine calL Similarly, a
Poly Rectangle X request is best rendered with a single
Poly PolyLine calL
Solid Fills
Many difkrent Wi n 3 2 resources such as pens, brushes, ronrs, and dip regions 111:1\' be req uired for anv given graph ics rtquest. The resources needed
::tre determintd by the gr::tphics operation itself:md the start of the X GC As norcd earl ier, these rtsou rces arc c reated as needed and managtd by the WX DC objects, removing si gnificant complexity and nearly redundam code from rhe actual graphics drawi ng rou t i nes.
Windows Pen stru ctures provide color and Lbsh pattern when drawi ng li ne objects. for dra\\'ing lines, segments, and arcs, the X ser \'er creates :md u�es
Pens that correspond to the
GC state. In some cases, holl' ever, exact pixcl izarion can nor be achie1·ed when usi ng
Windows Pens . Ex:�mples of this are dr:�1ving wide lines with r:�ste r oper:uions other than CXcopy or with l ong, dash patterns. In these cases, 111<1chi ne independ enr ti.ttKtions Jrc used to red uce the request to a set of spans (si ngle-width hori zonta l li ne s) to be ri l l ed . The use of Pens is :�lso ab:�ndontd i n speci al cases when the high ly optim i z ed GDI pattern block transfer ( PatBit) function cJn be used . P:uRlr ri l ls ITC tangular regions with speciricd colors or partnns. I r is r:1ster, for ex:�mpl c, to usc the ParRit function to d rJW verrjcal or horizont:tl lines than to use the \tVi ndows tradi tional line-drawing fu nctions.
Windows Brush structu res provide color :md pat tern \\'hen drawing r i l lcd recta ngles, ril led poil'gons,
:md fil led arcs. Aga i n , ror pertormance rtasons, the
PatB lt fu nction is often used even \\'hen there is a hi gher-level function th:�t seems to be a closer m•nch .
For examp le , PatRit can perform the X Po ly Poi nt req uest about 10 percent raster than S etPixe l V, the
Windows st:�ndard cal l f(>r setting single pixel vJ i ues.
Simi larly, PatB it can perrorm the X Polyfi l l
R
ect request about 14 percent faster than the Windows
Fi i i Rectangle d l .
Tile and Sti pple Fills
An X pix map c:�n be spcci tied as a pattern to be used when performi ng ti l l oper:nions .
·when the pixnup is created , it is rea l i zed :�s a Win 32 bitmap . When the pixm:1p has a depth greater than it is used as a co
1 ,
l
or rile that wi l l be used r(x the ril L If
36 l)igit.11 T.:chnic.11 )<>L11 11JI
Voi . S No. I 1 996 the pix map h:1s a depth of 1 , it c111 be used JS either a transp:�rcnt or an opaq ue stipple. An opaque stipple d raws both the GC's roregrou nd and b a c k g rou nd col ors, where the stipple is 1 and 0 respectively. A trans parent stipple is similar except that i t leaves the destination u ntouched wh ere the stipple is
0 .
When rht ri l e or opaque stipple i s 8 b�' 8 o r smaller, a Win32 color b rush is created and cached t()r the d r:n vi ng. On the Windows �T syste m, brushes brger than 8 b\'
8
c:m be created , but our experience has shown ir to be slower ro d raw wir!J them than i t is to perti:mn a series of bit block rranskr ( B itB l t ) opera tions h·om the tik/stipple bitmap ro the destination.
Transparent Sti pple Fills
There is a Win 3 2 functjon,
MaskB lt, rh:H seems ideal ly su ited r(Jr pedo rmin g trJnsp:�rcnt stipple ril ls. This fu nction , howe,·er, was not fu lly implement e d on all plarri:m11s :H the time 11·e d esigned tht eXcursion version 2 sortware product.
Without this fu ncrjon, there is no osy way in the vVin32 environ ment to perform the transpJren t stip ple operations. When rhe foreground color is either
0 or
Oxft:FF, the raster operation can be remapped to get the proper effect.
General rectan g ular fil ls th:�t d o nor meet the requ i rements ofrhc special case previ ous]\' ment ioned m ust be accomplished by fi rst con
,·erting the stipple bitmap to the depth of the d estination :1 nd then remappi n g the r:�srer operation.
In ge ner:1 l Clses that are nor rcctangubr ri l ls, mach i ne independent fu nctions :1re called to brtak down the request into spans
.
Image Requests
The Getlmage :md Putlmage requests are other examples of X gr a p hics requ ests that do not map ll'el l i nto the Win32 API . The only wav in the \tVin32 environment to pur image data o n rbe screen i s
to
rirst create a W i n 3 2 bitmap a n d initial ize i t wi th the innge d ata, and then ull the B i rBlr fu nc t ion to copy the bi tmap to the screen . X image data a lways Jisrs the top scan Jines first, whe reas the bottom scan li nes are listed fi rst in Wind ows b itmap data . Thcrd()tT , bdore t he bitmap is i nitialized, the
X i mage data must be scan- line Hipped. Si mil arly, the X Get I mage request req uires the usc of an inter mediate bitmap and also requi res the scm -line tl ip.
Plane Mask Support
Any graphics operation in X can be moditied by setting a plane mask in the GC The plane mask specifics which bits ofthe destination pixel arc a ll owed ro be changed . Without a pl:me mask, an
X gra phics operat ion may be ddi ned JS dst <- src 0 dst, where 0 is one of the 1 6 binary raster operat
i
ons
( e . g . , OR, AND, and XOR). When a plane mask is given, the r(JIIowing assignment ddines the destina tion pixel:
dst
<---
(( src @ dst)
& pm ) I
( dsr
& -pm )
Most video hardw�u-e de,·iccs support plane masking,
:md those rl1at do not su pport it
generally
provide E1st access to video
random -access
memory
(
l\AM
) . The
Win32
A P I ,
however, provides neither pl
a
n
e
lll<lsking nor direct video
1\AM
�
1 e c
css
.
To u nderstand w
h
y
,
vou must realize that Wi ndows h:-ts virtu:-tlized the color handling in
:m
a
t
te
m
pt
ro
mediate contlicts
bcrwcen
:-tpplications th:.lt would otherwise \\·ant to moddY th
e colormap
( the pix
c
l
-
to
-u> l or m
:1
p
p
i ng table). In tillS
,·irrual color
environmem,
the concept of pbn
c
111�1sks has no
me
a
n
i
n
g because Win32 applications need nor know the pLxel value th:-tt corresponds to :1
particular
color. See the section Color Resource Managemellt tor an explanation of how the
e Xcursion
software m:-tnages
ro :�ssign speciflc pixel val ues to colors.
_
I n the general plane mask c:1se, it is necess�lr\' tor the
X server � first sa,·e the contents of the destination i n
;1 bitmap . The gr:.�phics can then b e temporarily dr:J\\'11 ll'ithout r eg a rd to the pLme mask. Those bitS 111 the destination th a t �1re spcciticd by the plane mask
;ls being u naffected em rhcn be restored ti·om the s�wc�l bitm:-tp. This process wi l l work in ever�' case but IS Ind ficicnt since it involves SCI'er:-tl graphics opcr:ltlons bd(>rc achie,·ing th e tin:� I resu lt.
M:�n\' special cases un be reduced to one or t\\'O simple steps b,· modit':ing the sou rce color and raster opcc1tion. Ta ble l ShOll'S how rhc source color Jnd raster operation c1 n be set to
:�chin·e th e plane mask dkcr. The eXcursion X server uses these optim izations t()r many gr:�phics opcLltions when the source till is :1 solid color.
Internal Window Manager
I n the absence of
�1
windoll' m : 111 ager, the eXcursion servt:r creates :1 1 1 \\·indows
as
pop-up ll'i ndows. A l l \\'in doll'S, including
rop-Jc, c J
windoll's in m u lti\l'indow mode,
<11-e undecorated .
Thev
h�1,·e
no \"'i n32 borders, ririe
bars
or svsrem menus. To move, size,
rninimi7.c, maximiz
�
,
or
�
lose windows, the user must run a win dow m:�nager.
An eXcursion user �l lw;ws b:�s the option of u sing one of rhe
manv
X-b:1sed wi ndow managers al'aibblc, such JS the Mo-tif Windo"·
Man;1ger.
Holl'ever, lllJll\' users \\·ill W<lll t
a
ll'indo\1' manager pa
r:�
d
i
g
m
that is consistent with vVi ndows so that a l l ll'indO\I'S Oil the d
e
sktop have
t h
e
same
user intertace. To accom plish_ this,
a b u
i l
t
-
in window manager is provided as parr ot the eXcursion server. This internal window manager
is operative o n l v in mulriwi ndow
The internal window
m:-tnager, mode .
. .
the scnu, is function:� l l y
isol:�ted
tl-om
the rest
o f rhe code so that it can easilv
be
dis:-tbled . This al lo\\·s e xter nal \l'indow
ma nagers . to b
e used and a lso
hci lirarcs debugging by allowi ng problems
to be isolated. The ll'indow manager cre:1tes a " hook" into the
server's
window procedure, so
that
all vVin32 messages arc tirst
CX<lminecl bv
window m:1
;
t
he
"·indow managcL This gi,·es the
1agcr the
opportunity
to act on ll'indo\1'
managemem-�ebted messages
such as those that i nd i cate a� change i n the window's contigur:1tion o r state.
I f
the window manager decides to
handle a
message, it is removed ti·om the q ueue, and the server never sees ir. If rbe window manager is not
interested,
the mes sage is passed on to the normal ll'indo\1' procedu re . .
The purpose of the internal ll'indo\1' manager
Is t o ai\·e X wind oll's the same appear:-t ncc and beha\'JOr as
Y
Vin32 ll'indows th::Jt arc cr ea ted bv t\• pical desk top applic1tions, such as word processors a1
� d spreadsheets. When an X window is m:�pped . tor tbe first time, the internal wi ndow manager receives a W i n 3 2
WM_C: REATE message . Bct()re the window becomes visi ble on the screen, the window man ager alters the st\·lc of the Win32 windo"' to vVS OV E RLA P I ' E DW I N DOW. Win32 windows \\'Ith this- st\·lc ;liT auromaric1llv managed lw
Windows, ll'hicb ,h:-tnd lcs m ol
' i ng
, resizing, iconi f),ing, maximiz i n g
, and c l os i n g the wi ndows. Eac h of these :tcrions c auses a corresponding message to be sent to the server's window proced ure .
The i n tcrn:tl window ma nager interce pts the messages and dispatches them to the appropriate internal ti.1ncrion.
The r o l e of the internal \\'i ndo\\' m:1nagcr comple ments the ro le of the sen·er. The sen·er processes cl ient req uests on
X
\\'inclows �md translates them into opera tions on Win32 windows.
The inte rnal window man ager hand les Windows messages th at indicate changes to a vVin32 window :�nd translates them mro corre example
� the most important message rh:1t the window manager h<lnd lcs is
WM_Wl N DOWPOSC H A N G I N G .
Th.is message is scm just bdore anv change in the win do"''s position, size, stacking order, or l'isibiliry. If this message i ndicates that the window size c h a n g ed
, the window manager changes the size of the correspond ing X window :�nd sends a
Con6gureNorif)' event to the client.
Simi brly, the window man:tger translates other user-d irected. events such :-ts tocus change, win dow stacking, and iconification into changes to the u nderlying X data structures. I n most c:1ses, the win dow manager docs this bv calling into the de\'ICe independent l aver, thus simulating an X request that would occur trom an extern:�! window m an a g e r.
Mouse, Keyboard, and Input Focus
Mouse actions ;md keystrokes :�re received by the e Xcursion server as Win32 messages.
Each message comains i n f(mll;ltion about the window th:1r received the input and the rime of rhe in put.
For mouse moves
�md clicks, the scn·er u ses the \\'indo\\' intorm�uion to locate the corresponding X wi ndow and forwards an
X el'e nt ro th:lt winclo\1'. Keyboard input is t(xwarded to the window th:-tt currently has X foc us.
Digir.1l Technic.ll )oun1:11
Vol.
� �<>.
I
1 996 37
Table 1
Plane Mask Optim izations
Requested X Raster
Operation
GXclear
GXand
GXand Reverse
GXcopy
GXcopy
(src
& pm)
= pm
GXcopy
(src
& pm)
=
0
GXandl nverted
GXnoop
GXxor
GXor
GXnor
GXeq u i v
GXinvert
GXorReverse
GXcopyl nverted
GXorlnverted
GXnand
GXset src
0 dst 0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
1
0
1
0
0
0
0
0
1
1
0
0
0
0
0
1
0
1
0
0
1
0
1
0
0
1
0
Notes
2
1 0
2
2
7
4
1
6
8
8
8
5
7
9
6
3
Modified Source Color and
Raster Operations src
<---
-p m, rop <--- a nd src
<--src
I -pm src
<--src
I
-pm src
<---
-p m, rop
<--xor src
<---
- p m, rop <--- a nd src
<--src & pm, rop <--- or src
<--pm, rop
<--or src
<src
I
-pm, rop <--- a n d src
<--src & pm s rc
<--src
& pm s rc
<--s rc & pm src src
<--src
& pm
-p m, rop
<--xor src
<src
I
- p m src
<pm, rop
<--xor src
<--src & pm src
<---
-pm, rop <--- xor src
<---
- pm, rop <--- a nd s rc
<---
-src
& pm, rop <--- or src
<--src
I
- p m src
<--src
<--src
I
-pm
-pm, rop
<--xor src
<--pm, rop <--- or
Notes:
1 . dst is uncha nged when src equals 1 for these raster operations. Therefore, to preserve the value of dst when pm eq u a l s 0, set src equal to 1 .
2. dst is uncha nged when src equals
0 for th ese raster operat ions. The refore, to p reserve the va lue of dst when pm equals
0, set src equal to
0.
3. This operation sets all dst bits to pm i nto dst.
1 except where the plane mask equals 0. This can be done s i m ply by ORing
4. T h i s operation clears a l l dst bits except where the plane mask equa l s 0 . This ca n be done s i mply by ANDing pm into dst.
5. XORing with
1 has the effect of invert i n g . To invert only where pm eq uals 1, XOR pm with dst.
6. These operations a re performed in two steps. Note that dst is i nverted when src equals operation with src set to
1 where pm equals
0 . dst is now correct except that it is i n verted where pm equals 0.
The second operation of XORing with the i nvert of pm cor rects this.
1 . F i rst perform the
7. Th ese operations a re performed in two steps. Note that dst is i nverted when src equals
0.
Fi rst perform the operation with src set to
0 where pm equals
0 . dst is now correct except that it is inverted where pm equals
0.
The second operation of XORing with the i nvert of pm corrects t h i s .
8 .
T h i s operation is performed in two steps. Fi rst d s t is set to
0 whenever pm e q u a l s
1 .
Then dst is set t o
1 when ever both pm and src eq u a l
1 .
The two spec i a l cases can be redu ced to operations that use GXset and GXcl ear.
9.
1 0.
T h i s operation is performed in two steps. F i rst dst is set to 0 whenever pm eq u a ls 1 . Then dst is set to 1 when ever pm equals
1 and src equals
0. dst is unchanged; therefore, no operation is req u i red .
3:-l i)i�ir;11 Tn:hnical Journal
Vol . 8 No. I 1 996
The X server is a single Jppl icatio n in the vVin32 cn\·iron ment rh�H "owns" a l l rile X windO\\'S it crcltcs. from rhe user's perspective, thou gh
,
there 111�1y appear to be more than one
X
Jppl icar i on running, each with its own col
l
ection of windows. The user expects to be a b le to shift the kevbo:1rd t(>eus tl·om one window to another i n rhe s:1me tls hion that focus is shitrcd bct\\'ccn other :1ppl ications. \t\fben an e x te rn al \\'indo\\' m:1nager is i n usc, focus comrol is straightt(xward.
The \\'i ndow manJge r , usi ng \\'harever sem:mric it
\\·as designed tor, mon i tors mouse events Jnd shifts t(>eus accordingly. H owever, the semantic model for this may or may not be consistent with the Win32 mod e l . ln either case
,
the window decorations, e.g., borders, title bars, and menus, arc al most guaranteed to be different. A user who wants a consistcnr user in tcrhce model across Jl ! a pplications must emplov the interna l \\'indo\\' m:.n:1ger.
Ar Jll\' given rime, one wind o\\' on the screen Ius
Win32 focus and one X window has X focus. The two windows are not necessari l y the same. Since the X server cre:Jtcs and owns J l l the X windows in use, the server receives key bo
J rd input when any one of irs windmvs has \t\fi n 32 t(xus. The keystrok e s are nor necessari lv sent to r he undcrlving X window, however.
Thcv arc sent to rhe window that has X tocus. The internal \\'ind o\\' 111<1ll:tgcr Jssigns X tocus to the
X
\\'i n dow rhat receives Win32 toCLis. The cl ient receives noti tication of thi s event :tnd may decide to :tssign X t(xus to so m e other window, perhaps a child wi ndow.
The server m ust therctore
k
eep track of both the
X
window rhat cu rrcmly has focus and the stare of
Win32 foc us. When the server loses Win32 focus, the
X
ri xus is assi gned ro the root wi ndow. ·when the
Ser\·cr r c ee i\
· es Win32 focus
,
X tocus is assigned to the
X wi ndow th a t previous!\· hJd it. \Nhenever ch� m g c d bv :tn ap p l ic a tion
X
t i xus is
or Lw
the w i ndo w ma nager, the cu rrent X focus state is c:tchcd so that i t can be restored later, if necessary.
Font Management fonts and text fu nction�1lity make up a signi tion fi c ant por
of
anv graphics <1rchitccture. Both
the X
and the
Win32 S\'Stcms dctinc :1 rich set of te xt- rendering o p c rJtions and c a n process s c \· e r:tl tont tormats.
X and Win32 Fonts
The
X tont ma n a gement libr
:t ry is
:1 mod u lar arch itectu re th:tt detines an API t(x re a d ing and writing i n d ivid u•1 l rcmt tor m a t s. The m o d u l e that i mplements the API f or :1 given font format is cztl led a ren derer. This approach al lows X to support several tc>llt t o r mats
: tl1e l i brarv's renderer mod ul es com·ert ex t ern al formats to J single, internal b itmap format, which is u s ed tor all d r:twi ng oper
:t tions. The term
X./ollt refers to ton t
data in
this internal tormat.
The font management library supports both bitm a p and scalable outline tonrs. Bitmap t()llt gly p h s arc si m ply reto r matted :md used . Scal able tcm11:1ts, such as
Adobe Type 1 , JI-e rasterized on demand i n to the X font tormat.
For maximum performance, the server drJws text with native Win32 toms using the Win32 A P I . Win32 fonts are bitmap fonts in the FON f(mnat. Win32 functional irv co v ers the great ma j ority of te x t-drawi ng operations, but there a re a tew cases i n which it is either not possib le or not efficient to usc Win32 toms.
The server can also d r:tw d i rectly with the X tonts to provide fu ll
X t()llt support and complete text-d rawing functionality. This method uses Win32 Bit B i t
( ) opera tions to copv the ch�1racter glyp hs to the d isplay as bitmaps. Drawi ng speed with this method is :tccepr able but not maximu m .
Therefore, both X Jnd Wi n 3 2 fonts :tre used . The
Win32 fonts nuy be thought of as option a l acce lera tors: the server uses them whenever possible and falls b ac k to the X t(mts when necessJry. The decision to fall back can be made on a variety of cond i tions. This technique has also proved usefu l in worki ng around problems such as text-drawing bugs in i n d ividual video drivers .
Since scalable tont outli nes arc rJst c ri z ed into bitmaps at run time, the y are generally d rawn d i r ectly with the i n ternJ! X font format. The extrJ work of comp i li ng a comp:1nion vVin 3 2 tont at run rime gener ally outweighs its val ue as an accel e ra tor.
X bitmap tonts are most com monl y distributed in the B itmap Distribution Format ( BDl-"), an ASCII text sou rce fi l e . The eXcu rsio n team wrote a t(m t compil er tool that generates native Win32 ( FON t(xmat) fonts fl·om the
B
Df sources. The fonts created can be used by any Win32 appl ication .
The compi l er can generate either the com mon ly used version 2 t(mnat or t h e extended version 3 tor mat, which is n c ccs s J r y tor large tcm ts that re quire more than 64 k i l obytes ( KB) of glyph storage. figure 3 i l l ustrates the process of generati ng equivalent
X and
Win32 f(>nts ti·om �1 commo n source.
The X font tonn:1t contains extra i n t o rmJt i on (e.g., metrics and properties) thar cannot be derived from
BDF FONT
Figure 3
Font
Conversion
X FONT
WINDOWS (FON)
FONT
Digi r,ll T�(hnical journ.1l
Vol . 8 No.
I
1 996
39
rhe vVi n32 font. Therdon:, the X and Wi n32 to nts arc used rogether; the X i n t(mnation comes tl·om the
X font and the Wi n 32 r(mt is used Lw the Win32 API .
Real izing Win32 and X Fonts
When the X server fi rst opens a font, it invokes the function RealizeFont( ) .
This ti.mction gives the server a n opportun i ty to initial ize d:.1ta structu res
�1nd ped(mn
any format-specific operations necessarv
to m
ake the ti.lnt <l\'ailab l c .
To make a W
i
n 32 t(mt <1\":lii:Jble tor dr,m
· i ng, the server retrie,·es the
t
i lcn
: un
e
of the font from the scnu's l ook- up table �md registers it "·ith the Wi n 3 2
API using the function Add Font Resource( ). A h:mdlc ro th
e
font is obtained from CrcatcFontl ndircct( ), �111d the reafter the handle is selected i nto the desired
DC f(x d rawing operations.
H " the
Win32 real i zation hils
t
(
Jr any reason,
t
he code simply re1lizcs the X fon t instead . Failing to re:1lize a VVin32 font docs not neces sarily imply an error conditi on. Such t
a
i l ure happens in any case i n which the sen'Cr decides that it is best to use the X font d i rectJv.
The int ernal X tonr ti.m1ut is <1 set ofLhta structures.
The glyphs are stored in coll\'Cntioml arra\'S in user memory. To imp rove perti.mn<1ncc, the server reali zes
:.1n X font by writing a l l gln1hs ro a Win32 bitm a p in off-screen memory. CrcatcBitmap( ) returns a lund lc f(ll· later reference, and the glyphs in the bitmap arc indexed tor use in drawing operations.
Drawing with Win32 and X Fonts
The glyphs in X text strings are often kerned,
that
is, mnlapped tor best r v pogr
<
1p hic appearance. To draw with Wi n32 fon ts
, the SCITer emulates the way X draws text by using
Ex rTextOut( ) , w h ich uses an interch<1racter spaci n g vector to place the individual glyphs. The
f
o nt
' s
X met
rics are used directly to calculate th i s vector.
Glyphs tl·om X tonts arc dr:.1wn by performi ng
BitBits tl-om the Win32 bitmap to the t:trget window or bitmap . The server places the glyphs using the r(m t
' s
X metrics as d escribed in the previous paragrap h .
Color Resource Management
Alt hough some X Window Svstem concepts and struc tures 111:1p fairlv c lose ly to th
o
se in the \�Tin 3 2 S\'Ste m, color resou rce management is handled ,·crv differ en
t
ly
.
The difference is most
evidem
when dealing with pseudocolor video systems.
C
onseq uently
, this paper describes only this case .
The X Wi ndow Sys t e m environment shares 256 col
ornnp cells among :t i l appl i cations that use the default col ormap ( i . e . , those th<H do not have a pri,·are col onll<1 p ) . A ppl icati ons can allocate cel ls i n the defau lt c ol ormap to protect them tt·om modi fication Lw other applications. In contrast, the Win32 S\'Stem allows each appl ication comp lete access to the S\'Stem palette while the application h a s t(Kus and maps the palettes of the windows without t(Kus as best it can .
40
Digir,11
T<:(hni(a]
)oum.d
Vol .
8 No. I 1 9')6
In the X vVindow Svstem CIWironment, when an appliution n:serves a colormap cel l , it references the cel l wi th a pixel \·a lu e . Th i s value is an index into the colonnap and is used to look up the val ue that will
<
1ctu<1i ly be stored in scree n memory when that pixel value is used i n a d rawing opnation .
In the vVin32 system, color rnan<1gemcnt is handled by the palette manager through a p ale
t
te structure.
E
�
1cl 1 appl ication has a lo g ical palette
, and a singl e S\'S tcm palette contai ns the colors c ur renrlv mapped to the hardware colormap . App lications re
fe
re nce col ors rel a ti
,
·c
to
their l ogical p<1lette, :md the p
:.1
l ette man
.1gcr hand l es the mapping between the logical palette and the svstem palette . When J.n application is given t(>eus, the p <llette man
;
1 ger maps J l l the colors from
the
l ogical palette into the system pal ette. If the system palette d oes not have enough
empty
cel ls, the palette m�1 11<1gcr frees ce lls al locned to other app lications.
If
this occurs, the palette m:magcr wi ll attempt to remap the other applications' col ors into anv remain i n g tree ce l ls in the svstem colormap. If not enough cells are t i
ce
, a n\' re
m
ain i n g unmapped colors are mapped to the svstcm palette colors that m ost closclv matc h .
Beuusc of t h is \\"a\' o f hand l i ng color resource man agement, an appli cation docs nor know what value i s being stored i n screen memory r()f a n y particular color and the value stored f(Jr any color can change over the li ktimc of the app lication. This siruation presents sig nificant diffi c u l ties tor X operati o ns that req ui re exact kn o w ledge of the pixel val u es i n screen memory, such as the (;ctl mage operation and operations i n,
· o lYing pLmc 111<1sks. The sen-cr works arou nd the d i fficu lties b\ creatin g nm Win32 logical pal ettes.
The ti rst palette, i . e . , the working pal ette
, corre sponds exactl y to the X debult colonnap and does not al low sharing of the palette by Win32 applications.
\Nhcnever an X window has focus, the working palette is in usc. This causes the Win32 palette manager to set up the system pal e tte such that it d i rectly corresponds
to
the X co lorrnap, and operations thar arc pixel based
"urk properh-
.
The other palette, i . e . , the
idenrin·
palette, is set up ro cmrespond exacrlv to rhe system p�1lette . The iden tin· p:: tlctte is used w h enever no X window has too1s
.
Bcu use of the correspondence, no translation i s i nvolved benveen the ide
nti
ty
palette
and the system palette
,
which
allows the X server to know what pixel va lue is stored i n screen memory.
The X Window System envir o nment allows t(Jr p r i
V<ltc colormaps, which �11-c created and used by a single
<1ppli ution . The server creates :.1 working palette tor
C\U\' colormap created . When the col ormap is installed
( norma lly
Lw the windo\\· mamger \\'hen the X appl ica tion is gi, en tocus), the e Xcursion software installs the
" orking palette associated \\'ith the pr i
,· ate colornup .
The eXcursion X se rver c urre
m
ly supports the
PscudoColor visual class and the StaticGray depth 1
visuJI class, which i s main l y used tor bitmaps. eXcursion version I also supported
�l
StaticColor v isua l cbss for 1 6 -color video graphics :may (VG A ) displays. eXcursion version 2 treats VGA devices identic1lly to
PseudoC:olor de,·iccs and al lows the Windows palette m�
Huger to gencrJte d ithering pam.:rns tor the una,·ailable col ors.
Network Interface
With the rele:tse of X I I R6, the X Consortium com bined all transport-specific code into a single p!Jce in the sou rce tree, the X transport i n terface. The eXcursion te:nn extended the X transport intertace to include Network Computing D ev ice ' s ( N CD's)
Xremote s er i :� l line transport. Com bined with the transmission control protocol/internet protocol
(TCP/IP) and D ECnet transports, the eXcursion product
CliJ now execu te X sessions over Jnv of these tr:�nsports si m u l raneouslv. The c Xcu rsion pro d uc t supports anv TC:P /IP stack th�H com plies with the
Winsock \'ersion 1 . 1 implementation, PATHWO RKS
D ECnet protocol, and NCD's Xremote p rotocol tor serial line.
The X
transport interface prm·idcs fu nctions that
a
r
e common ro
all transports,
su
c
h as parsing an add ress into a docs
host and
port
number. T
h
e
i nt e r [Jce
not provide an abstraction
t(>r
the
se
le
ct
( )
caiJ, because ir assu mes that
this call
is
rr:msport in
de
p
e
n
dent.
Unt()rtu nately, the
X remote protocol
requ ires
�1n independent
select( ) mech:mism, and, thus, it
was necessarv
to
i
m
ple
ment
a select( )
� t b
s
trac
t
ion to
co
mbi
n
e the tra
nsport indepen
d
ent select( ) \\ith the
Xrcmote
select( ) .
Although somewhat comprom ised
Lw this add i tion, pe
r
to
r
m
a
n
c
e was a
problem only
w
h
e
n
the
Xrcmote protocol was used in
combi nation with
either the
TCP /IP or the
D ECnet p
ro
t
o
col.
X Image Extension e Xcursion \
' c rs
i
o
n 2 provides
,·ersions
3
and 5
of the
X I mage Extension
to support
a
w
i c k
range
of i maging
a
p
pl
ica
tio
ns
.
Because
it
is
a large
body of
code,
X I E i s
im ple mented
as a
pair
of Win32
D LLs
to consen·e
memorv on systems
t
h
a t wil l
nor be running applica
tions
that
usc X I E .
N
or
m
a
lly
,
�lCccss ro a
DLL
i s one-\\·:ty. App lic:trions
can load and
make function
calls i nto
a
D LL,
but because ir is
linked dynamical l y
�lt run time,
the D LL
code
cannot mJkc fu nction calls back
in
to
the calling appl ication . X I E , however, must ctll into the
device
dependent
laver
of the sen'Cr to
perrcmn
any
r
eq
u
i
re
d
d r
a\\
' in
g after processing
i ts i ma
g
i
n
g requests. To per mit this, Jn addition to
tbe
i n terbce
w<Js
designed .
W
h en the X I E
D LL
is i ni
t
ia
l
iz
ed , the caller supplies
a
Jist of
pointers ro
the
functions
needed by
the
X I E.
The DLL ri l ls an
<llT�lV with these poi nters and then calls back i n d i r e c t ly th rough the Jrray. On the
\1\Tindows operating system, this design cou ld c re at e a problem because u n d er Win32 AP!s, global data in a
D LL is not instanced; that is, the code is not reenrr�1nt.
The approach \\Orks i n this case b e c
<J us e there is onlv one copy of rhe D LL loaded . If another a p pJ i cn i on was shar i n g the DLL, the pointers wou l d be overwrit ten by the second initialization .
Control Panel
The
eXc u rsion
control
p
a ne
l
is
the prim ary interbcc through
which the user
con tigurcs �111d controls the product. Some other
compone nts
create
simple
w
i n
dows
or icons, but these fu nctions <Jre limited .
The control panel
constitutes 90 percent of the
user inter
bee t
o
r
the
eXcursion a
p
p
l
ic
at
i
o
n.
This ract makes rhe control p
an
el an ideal c<Jndidate tor the r
a pi
d
applica tion de\'elopmcnt features of
the
M icrosoft VisuJI
C++ e
m·iron
ment.
The control
p
Jn
cl is
a
vVin32
appli cation coded J lmost entirely i
n C++
and
linked
with
the Microsoft Foundation
Class library.
The main sent a
purpose of mJ nagc�Jble
the control
pa
n
e
l is to interf:tcc through which
the pre
user can view and mo
d
i f v
the eXcu rsion
con figuration pro ti le.
To do
this in a manner consistent \\'ith
the ne\\
Windows 95 she l l ,
the
Property Sheet
M�C
object
w<Js chosen. Property Sheets arc tabbed dialog
boxes
that
have the <Jdvant:�ge
of
organizing lar
g
e amoums
of
data
setti ngs
in a
compact
space . The�· arc
used
cxtensi,·el \' b\'
the
WindO\\'S 95 operating svstem and bv the most
recent
\'Crsions of
Microsoft
applicnions.
The Propcrt\' Sheet
object
is �l
su bclass of th
e
Windows object
and is essentially
a container tor the ta
bbe
d
p:tgcs. E:tc h
tab,
when
cl ic ked by the
user, di
s plays
a dia
l
o
g
box that
is subcbssed ti·om the
M�C
Propem·
Page
object. The individual
pages can
be
,·isually
configured
and revised
usi ng
rhe class
wizard teature of Microsoft VisuJI
C++ .
The designer simplv
selects
dialog box
controls such as buttons, d rop
l ists, or edi t tields :md
positions
them
on the
d ia
l
o
g
box.
The
code to
hand le user actions is rhen fi l led i n .
The eXcursion control pan el i s shown in hgure 4.
We constructed
an initial
prororvpe of the comrol
panel
J p
pli
cuion \\'ith about 60 pe rcent of the
tin�1l
function:tlitv in l ess than
one month.
lnterprocess Commun ication Library eXcursion ,·crsion 2
consists
of s
c
,·
er
al
cooperating processes
thJt
must com munic:�te :md
S\'IKhronizc with one started by
Jnother. When
a
remote
X
application
is
the server or the
control
pa
ne
l
, the applica tion l a
u
nch
er signals when the operation is complete.
Digit,11 Tc..:l1 11ic:d journal
Vul . R ;-\o. I I \J9(l
4 1
.,
><DM CP
..... .
.
.....
.
.......
.
,
!
Info
!
Accounts
· ... . ....... . . . . . ... ,; eXcursion Control Pa nel
E xtensions
Applications
D isplay Fonts
Communications
Keyboard Mouse
I
I
Logging
M odem
Access l
I eXcursion Control Panel
Copyright
€> 1 995
Digital Equipment Corporation
Computer N ame:
R egistered to:
Serial Number:
Installed Path:
Version:
S N O O T S
S N O O T S
21 309
C:\XCU R S I O N \x86\
V2. 1 . 309
X Server ------------------------------------------,
Server N umber:
Number of Links in U se:
N umber of Errors Logged:
0
8
0
O K
J I
�_= ce l
=='� I ..... J
Figure
4
The eXcursion Control l\111CI
I
R estart S erver
�
42
Error and status information is sent to the error logger by the other com ponents. When the user changes a contigurarion setting thro u gh the conrrol panel, the change must be com mu nicated tO the X sen·cr, if it is runn ing. In some uses, the change c1n t:� ke effect im mediately; i n other cases, the server unnot imple ment the change witl1out restarting. The control panel and the server must engage in a di:dog so th<lt the user can be int(>rmed as to wh:J.t action m ust be taken, ifanv.
The ! PC librar�' is Jn opcrJ.ti ng system-i ndependent
APJ th�H permits eXcursion compone nts to determine which other com ponents :tre present and to exchange comm: 111ds and configuration information .
The Windows NT operating system provides several bui lt-in !PC mecha nisms, but most arc nor on the Wi ndows
�wailablc or
\Nindo\\'S 95 syste ms. The onh' mcch:111ism that is u n i\'crsal to the three operating systems is the message -passing inrerhce in the Win32
API. This mechanism, while nor the most dlicicnt, is relatively straighttorward to implement. Since the per torm:lllec demands on the
I PC li brarl' were deter mined ro be verv light, this mech�111ism \\'JS chose n .
The d isadv:�mage of the Win32 message-passing i nterface is that it is \Vindow based , not process b�1sed.
Mess:�ges
�1 1-c received by a c:dlback procedu re that musr be :1ssociated \\'ith a \\'indo\\' before am· commu nication can t:�ke place. If <ln application has not vet cre:�red a \\'indow, or never creates a window, as is the c1sc with the :tpplication launcher, no com mu nication is possible. To remedy this, the J. PC li brary creates its own window when the calling process initializes. The
T PC \\'indm\· is ne\·er mapped to the screen, so it is nor
\·isible to the user. All i nterproccss com m u nicuion passes through the ! PC window.
The lPC li brary consists of a collection of un ique messages and an
A P I .
The messages arc registered with the Win32 fu nction RegisterWindowMcss:�gc.
This e nsures that the messages u sed l)\' the eXcursion
:�pplication do nor conflict with system messages or messages used by other applications. The eXcu rsion
! PC messages arc
• ipcComponcnrSrartedMsg, which th e
I PC posts to all compone nts \\'hen a component i nitial izes .
Vol . K No. I I ')<)(,
•
•
•
•
•
•
•
• ipcRestartServerMsg, which the J PC sends to the server to tell it to restart.
ipeRestartServerStatusMsg, which the
J PC
posts with the status of the restart request. ipclnq uireMsg, which the !PC sends to retrieve a data i tem from a component.
ipeProfi l eChanged Msg, which the control panel sends when the registry profile changes. ipc La unchOneCompleteMsg, which the applica tion launcher sends to noti�' the server of lau nch com pletion.
ipc LaunchAJ I CompleteMsg, which the application lau ncher sends to noti�' the server of launch com pletion . ipcH ideAI IWindowsMsg, which the server sends to a l l components to tell them to hide all their win dows. The e Xcu rsion application uses this message
to
execute the pause/resume Feature .
ipcShowAIJWindowsMsg, which the server sends to all components to tell them to show all their win do•vs. The eXcursion appl ication uses this message to execute the pause/resume featu re .
I n addition to sending and receiving messages, eXcursion processes can use the I PC l i brary to deter mine which other components arc running. The
I PC initialization proced ure creates a window with a unique name that identities the call ing component. To determ i ne whether a speci fic component is present in the system, the ! PC searches all windows on the system until it finds one with the correct name.
Error logger
The error logger is a Win32 application that receives nror and i n formational messages from other compo neJJts and either displays them in a window or logs them to a tile. On the vVindows
NT
operating system, information that may help system managers or users d iagnose problems may additionally be recorded in rhe Windows
NT
evellt log.
Application launcher
The application lau ncher is a Win32 app lication that hand les requests from the control panel or server to start
X
client applications. The client may reside on a remote host or on the same machine.
When the user req uests the server or control panel to start an X client application, it starts the eXcursion appl ication launcher in a separate process. The applica tion command, host name, account information, net
work transport, and command shel l are passed to the lau ncher in its command line arguments. The lau ncher makes the connection to the remote system, initiates
the command using the selected protocol ( rexec, rsh ,
DECnct object, or local com mand ), and sends an
IPC
message to the server indicating that a new appl ication is starting.
Registry Interface
The Windows NT operating system introd uced a new concept called the registry. This is a protected database mai ntai ned by the operating syste m, wherein Win32 appl ications may store configuration and state infor mation. The registry has a wel l -ddined
A P I
and a mainten:mce uti litv progr�u11 that is shipped with the Windows NT operating system . Under the
Windows operating system, configuration information is kept in simple text files, which are vul nerable to accidental or malicious tampering. At the time the design of eXcursion version 2 was under way, it was u n known which, if either, ofthese two metl1ods wou ld be ava i lable u nder the Windows
9 5
operating system.
Nevertheless, all th ree of these operating systems had to be su pported.
We designed an
API
for accessing the configuration intcm11ation in a manner i ndependent of the operating system . Knowledge of the oper:tting system and its reg istry access method is encapsulated in the li brary. Since several independent processes must access the informa tion, the l ibrary is built as a
DLL
to conserve memorv.
The in terface basically resem bles that of the Windm
;,
s
NT registry
A P I
but elimi n::ttes some of the complexity.
I f the eXcursion software has
nor
bee n configured when the registry interface ti rst accesses the profile, default values for al l settings arc selected to allow the software to function normal lv.
Summary
With computer systems based on the Microsoft
Windows operating system i ncreJsing i n power and decrc:tsing in price, Windows- based systems are appear ing on desktops that once held workstations running the U N I X or OpenVMS operati ng systems. Windows systems must be able ro access applications on remote tile and compute servers running in the X vVindow
System environment. Version 2 of rhe eXcursion prod uct provides desktop integration of X client applications with n�uive Win32 appl ications. Mod ul::tr cod ing tech niq ues, object-oriented progr:tmming, and selective use of the Microsoft Foundation Class li brary helped reduce development ti me, and improve performance, mai ntainability, and reliability.
General References
D .
Ci obs and A.
Leskowi rz ,
"eXcursion t()r
Windows: l nr cgr;Hing Two W i ndowi ng
Systems," Oip,ilal Technical
.Jou rnal. vol. 4, no.
I
( Winrcr 1 99 2 ) : 56-67.
Dig;ital Tcdmi�al Journal
Vol .
8
No. 1 1 <)96
43
44
X Window System
S. A n g c: br
;l l l n d t cr a l . , Dejlnition o/the l'urfillJ.!.
!.are1jin· the
X t · f I
Sa111ple Sener (
C:
11 nb r i d g e , M<lSs . :
X
Consor tium, I nc . , 1 994 ) . j . hdron ,
The X
Font Serricl! l'mtoco/.
I
·en-ion
2.0.
X
\ ('ISion
I ! .
Release
6
( C:unbridge, M �s s .
:
X
Consor tium, I nc . , 1 994 ) .
E .
I s r a e l Jnd
E . F ortu n e
, '/he
X
\l?indou•
,)·J,S/e/11
Serl 'el;
X
\ i.•J:..;iou
I I.
Release
5
(VVoburn, !Ybss.: Digit�! Press ,
1 99 3 ) .
0.
J o nes, lntmduction t o the
X
I.Fiudotl'
.):)•stem
( Engle wood Cli rlS, N . J . :
Prentice-H<lli, I n c . ,
1 91\ 9 ) .
K .
!';Jcbrd ;Jnd
D .
Lemke,
'lbe X
Font !.ihrwy ( Ca m bri d ge, ;V!Jss . : X Consortium, I n c , I 995 ) .
D.
Roscmh.1 l , lnfr?r-Ciient Conlllllllticotion Omt ·eutions
.1/muw/. \
C'l:,foll
2.0
( C a m bri dge, Mass . : X C:onsorri u m ,
I n c . , 1 994 ) .
R
..
Scilcitkr,
X
Windou· Svste111 Pmtocol.
X I
(•J:..;ion
I I .
Ne/e(fS(' 6
(
C
:
� m brid ge
, Mass . : X Co mm t ium
,
I n c . , 1 994 ) .
R.
Scileitler and J . Gerrys,
X
Windou•
S)'s/ellt ( Bedt(mi ,
M ass . : D igital Press,
1992 ) .
'fbe G{ 'I
Gu ide. lutemutionol Temliuolof{J'./(Jr
the
1\:iu dou s fnlel/ace (
R ed m o nd , Wash . : Microsoft Press, 1 993 ).
S. McConn..: II, Code O.>llijJ/ete:
Sojill'a re Co ns/
A
Practical flandh<HJIJ o( met io11
( R.:ci mond , Was h . : M ic r o so tr
Press,
1 993 ) .
C.
Petzol d, l'rr��nl llllllillg Window�.
2 d ed . ( RcJrnond,
Wash . : Microsoft Press, 1 990).
13 .
S t r ous t nr p
, 'the C++ l'mgr(fiJim ing Languap,e ( Kcad ing, Mass . : Add ison -Wesl ey P u b l ishing Company, 1 9 8 6 ) .
Biographies
Networks
M . H a l l e t al . , "Wind o11·s Sod�crs: An Open J n tertJce [()['
�l'tll' o rk l'rogr;unming u n der Microsolr Windows, V ersion l . J " ( I 99 3 ) .
K .
Pacbrd,
X
f)j_,jJia)' .\lanager
Contml l'w wl.
I
('1:-;io n
1 . 0. X I
(>n;fon
.1 1 .
Release
5
X
Consortium,
1 98 9 ) .
( Cm1lxidge, M�ss . :
,\·! I T
W. S t e
1·ens,
UNIX
Netn·ork Progrmnmin,t;
Cli Hs , N
)
.
:
Prentice-Hall, Inc.,
1 990 ) .
(
Engic ll'ood
X
'/i'anspou
Jntcrj(?C!! ( Dayton,
O h i o : N C R C:o rpor<ltiOll ,
1 99 3 ) .
John
T.
Freitas
Prcse nrk
<1 sofrw.1re enginee l:'rcit�s 11 orked ;lt Digi t;11 r at Atri<1 Software, john f or I 5 l'e<lr·s. !:'or the last fe11
\'e;Jrs, he ll'as <lss
<1S
Jn in di
1
· i d u
;
1 l o c i
.n
ed
11 irh D contributor, <1n i gi tal 's n i
, c
·ersi n
Xcursion pmd uct architect, <Hld
:1 d cs i g n
L:r.
Pre1·iouslv, he
1\·:1s i n the Worksution gro u p . John recein:d
J B.S.E.I:'.. ti'om N orrh c;Jstcrn l
'
· i n 1 96 7 . Wh ile in college, he ll lll'ked ;Js <1 co-op student on the Apoll o
Project at ,'v\ JT's Draper L1 boratorv. During the he worked l(>r
HJ n-;1rd
1 9 70s,
Unil'crsitv de1·clopi ng and rna i rl
t<lining rned iul com p ut i ng facilities <lt lvl assachuserts
Ccncral H o spi t<1 l .
Windows Operating Systems
R. Hhke, Of!limizii i,Q Windou�,·
.\T
Will!lrJit 'S Y!Nesoune
/\it. 1·ol . 3 ( Redmond, W,1sh . : Mic ro s o rr
Press, 1 99 :1 ) .
H .
Cu srn, Tusicle \'V-iudotn
,\T
( Red rn onJ, W;1sh . :
M i crosoft Press, 1 99 3 ) .
A . King, Tnside IFiudou �-;
95 ( Rcdrnond,
Wash : M i cr osoft
Press, 1 994 ) .
\Viu.-).2 l'm�ram mer \ Reference, l'ols.
W:1sh . : Microsoft- Press, 1 99 5 ) .
1 - S ( Redmond ,
Windows Programming
K.
Ch ristiJn, J Z1e ,\1ic msoji Gu ide to
C+ + l'mgrwnm ing
( Redmond , \V;Jsh . : Mi c roso f t
Press, 1 992 ) .
I'.
] ) i L1sci;1, Windotr s++: \.\// '/!il l,!.; Neusohle Wi urfoll
'S
Code in
C++
( Read in g, M <lSS . : Add ison- \Vesb· Publ ish i
n
g
C
:ornf1,1111·,
I 992 ) .
] ames G. Peterson
) ames Pert:rson is cmrc ntly a sotrware engineer at
DeLorme Ma
NT g ro u pp i n g. As
;1 member of Digit f
1
, James led t h e release� ofthc e ll'arc ti·om v e r si o n
1 . 1 th rough l'crsion
X a l c u
2 . 1 .
' s Wind oil's rsion sotr
1n
<ldd ition, he worked as
;J rc hitect <
1 n d in divid ual contributor on the e
Xcmsion project, concc ntrclting on graph ics <ll1d p e rf o r mance l-:;1rlicr, he 11orkcd in the
PATHWOR KS
<Hld R:1 i nb011 groups. hmes 11'<1S e mpl o v c d b1' C om p i o n
Corporation bd(>re jo m i ng Digital i n 1 9 84. He rccci 1'ed a B . A .
(
1 9 79 ) in J1l<lthcnnties ti-orn fnd i<lna Uni1·crsin· and an M . S .
(
1 98 1 ) in m�thernatics .1nd an
,\\.5. ( 1 984 ) in colllputer scie n c e
, both li'om the Uni1'crsm· of!llin ois.
V"l i-i No. I 1 996
Scot A . Aurenz
Scot Aurcnz is a �rincif'�l softwJr<.: engineer in the
Win do ws NT group where l1c ll'ork� on the develop111c11t of the e X cursion
PC
X ser v er . Scor h<1S contributed ro nunv projects
;H
Digital, including the L� ngu;�ge Sensitive Edito r
( D EC :ser LS E ) <1nd rile S U VA X 11·orkstJtion. Scot c;1111e to
Dig;i t;11 in
1 979 JS J Purd u e
U n i
,u si t\' co-op srudenr ;1nd b..:c·.1111e a fu l l - rinJC cmplmu·
.liter recci,·ing his
B.S. E . E. in
1 9X 2 .
H e recei\'(:d ;1n M . S . F . t-:. f rom rhe Uni, crsitl' of
I l linois in 1 98 6 .
Paul
J.
�1 nauro
Paul Ra nau ro joi ned Digi tJI in 1 987 and is a pri nc ip sofrware engi neer in the \Vindows NT group. He is a l respon sible tilr ;1pplication t;tilover t()r the Digir;
1 l Clusters t()l' vVi ndows
NT product. In t:<
1rl ier
II'Ork, he partici pa in the dc,·clopmenr ofrhc eXcursion softll
'
;
1r
<.:
Jnd the
A C M Sx� tr;ms;tcrion l
' rocc s s ing mo ni tor
, spcci t ic
1 l h
·
, ted in rhe implcmcnr;1tion of rhe RTI p rotocol
.
He ;tlso p;1r ri,ipared in the i m pkmcnt<ltion o f r he Manuhcruring
Messa ging Sen·icc
0 -, a ppl ic a tio n la\'l: r pro ro, o l t(Jr rlK
D l:::Comni prod uct ami ;\ ne n,·ork pertormJncc anail·zer.
Prior ro comi ng ro
Digi ta l
, he "·as a (OnsultJ nr ;1t In dex
Systems and
<l senior software engineer a t
Micom - [ nreriJn.
Paul h old s a
B.A. i n h istory ti·om the U n iversitv of
1Ybssachuserrs ;lt Boston .
Charles P. Guldensc h u h
C :h;1rlcs Gu ldenschuh is J principal softwJre engi neer i n
Dig;i t
;1 l
' s Windoll's NT group . H e i s responsible till· color su pport and sofr11·Jre in staiiJtion of rl1e eXc ursion prod uct.
Prc,·ioush', he II'Ot'kcd in the Real-Time Sofr11·;m:, l'roh:ssion;l l 300 Sofn, Jrc t-:ngincering, �nd RT- 1 1
Engit1l'l'ring gro ups. Ch;Hics joined Digital afrcr rccc i, ing his
B.S. in i nfonm1tion ;1nd c<>111flU t<.:r science ti·om rhe
Ccorg;i;1
1 nsriru re ofTcch nolog\' in
1 9 76.
Digital
Technical Journ;�l
Vol . � No. I l<J96
45
Integrating Multiple
Directory Services
I
Margaret
Olson
Laura
E. Holly
Colin
Strutt
The Integrated Directory Services (IDS) i nfra structure i m plements a directory-service independent i nterface. The IDS i nfrastructure is used by applications that store and retrieve information a bout resources in environments with either m ultiple d i rectory services or one of several di rectory services. The I DS interface isolates users and appl ication writers from the unique requirements of different directory services by providing a view of a single, logi cal d irectory service through a simple federa tion mechanism. To retrieve resources from the logical directory, IDS determ ines its phys ical location and converts the resource from a directory-specific to a canonical format.
Extensible schema tables represent the canon ical format for each resource and allow I DS to represent resources created using both the I DS interfaces and the directory-specific interfaces.
Digital has developed the Integrated Directory
Services
( I D S ) tec h nology ro provide a mechanism for integratjng m u l tiple d irectory services into a single sys tem . In this paper, we examine the development of the
I DS i n fi·astructure . We begin by ruscussing the prob lems faced by network directory appl ications. Next we describe our design goals, the
IDS i n frastructure, and our initial implemen tation on the
I'AT HWO lU(S prod uct. We concl ude with a brief discussion of plans br future development.
Di rectory Support in Multiple Environments
Al though rurectory services are a powerfu l mechan ism t(Jr distributing and accessing certain kjnds ofinfonna tion, relatively tew Jpplications choose to use them .
Digital's
PATHW O RKS application was in need of a directory for printers :md fi le shares.
�1 net\vork operating system
( N OS
PATH WORKS is
) integration product that gives users access to both M icrosoft's LA.!'\l
Manager and Novell's NetWare file and print shares. As we studied how to incorporate directory support into
PATHWOIU(S , we came to a better understanding of the problems faced by directory applications in general .
Networks are growing rapidly, as are the amount and kj ncf of information that can be accessed through the net\vork. We were certain that fu tu re network appl ication products wou ld have an even greater need tor a directory, and therefore a general sol ution was needed. We then set out to design a system that would remove the barriers to di rectory service <�pplication usage and deployment. We resolved the tension bet\veen the product dead line and the time req u i red ro implement rhe ge neral solution by design ing a complete solution and i m p lementi ng what was neces sary to prove the design and to meet the i m medi<�te needs of the
PAT HWO RKS product.
Existing Directory Services
There are a n u m ber of general -purpose directory ser vices. Some of the more frlmiliar include
Nove l l 's NetWare Directory Service
( N O S
X.500,
)
, the Cel l
Directory Service
( C DS
)
, and Banyan Systems'
46 Digital Ted1nid jound Vul. 8 No. 1 1 996
Strl:l:tT�1lk.
1 1 ln the past, dirl:ctorv sl:n·icl:s Wl:IT in rl:l ati\Tiv l i m i ted usc because most d irl:ctorv sn,·iccs wnl: til:d to l:ithcr an operating S\'Stcm or a transport or both.
In addition, d irector\' services \\'ere con tKctl:d to a m u l titude of application programming interbcl:s (APis) that were incomp<Hible and difticult to usc More recently, directory services h;we been tied to network operating systems or applications, rather than to host opeL1ting systems or transports. If any thing, the nu mber of"standard " A Pis Ius grown.
I n L1rge networks, this complexity has resu l ted in the proliferation of directories, often contJining over lapping i n �{>nllation. This ma kes the network man ager's job d i fticu lr, which in w rn creates tTsistancc to d i rector\' appl ications_ At the sanK rime, network and
N O S tcchnolog-v hJs developed to a poi nt whne :111 l:ver-incrctsing :1mounr of information is being shared
011 different machines. To give a simple example,
�1lmosr every server :1t Digital's Littleton site hJs a con nection to the h i gh -volume printer in thl: copy center, with a d i ffnent name on every server.
A directory wou ld simplify users' access to this single physicJ I resou rce by presenting a single name �()r the printer, i f onlv the application writer could �igurc out which dirccton· sen·ice to use Jnd how to use it.
Other Approaches
As discussed later i n the Design of the
�llld
Service Providers section,
I DS
I DS
hJmework defines both an APJ and �1 service provider inter��lCe. Support tc>r �1ny direc tory scn·ice
Clll be provided by writing a service prm·ider module. lYlicrosoft's
O L E
Direcrory Su·vices
(OLE
DS)
takes J similar approach to
I DS ,
with a more limited initiJI implementation .' A l though the current
I DS
implementation runs under Microsoft Windows, it w:Js designed to port to other systems.
OLE DS
depends on tCJtures of the vVi ndows operating systems.
The X/Open federated Naming ( XFN ) specificJtion was not complete at the time we were designing
I DS ,
�llld it did not include either :1 service provider interbce or
:1 rdcrence implementJtion 6 We did ex<1mine the
XFN d raft and designed the
I DS interfJce to be com p�niblc with
XFN , with a view toward su pporting the
XfN API in the future. Supporting the XFN interbces on top of
I DS wou ld be a rehtivelv srraightrc>rw:Jrd task, �md we llJve considered doing this_
The PATH WORKS Application
I n rhe
NOS environment, each
NOS h�1s irs own directon· or pseudo-directory_ NetW:Jre version 3 implcmnns the Binderv; NervVare 4 implements
N DS .7 TIK \'Jrious implementations of Microsott's
LAN Manager protocols provide a ''irtual directory b�1sed on i n tcm11ation maintained bv irs dom:1in con tro l lers.
In
:1 mu ltiple
NOS eil\'ironment, the user is presented with m u l tiple i n�(mn:Jtion sources ti·om the m ultiple directories. E\'Cil worse, the user mav be taced with multiple in�cm11ation sources even in a sin gle
N O S environment, since there ma\' be m ultiple
NetWare Binderies or LAN Manager domains.
Multiple
N OS
environments do not, in and of them selves, cause complexity and confusion . Problems arise when people within �1 single environment want to sh<ll'C resources across multiple environments. For example, consider a common local area network ( LA N ) configu ration where NetW:Jrl: is inst:Jikd on the c l ients :111d servers tor one departmellt and Microsoft's LAN
Manager ( contained within products such as
1'v1icrosott's Windows t(>r Workgroups, Windows 9 5 , a n d Windows
N'T'
operati ng systems, or the LAN
Server product ti·om l nternation:JI Business JV!Jchines
Corporation ) is insr:: d lcd on the clients and servers tor another department. I f each department's resources, users, and administration personnel Jre kept distinct, there is no problem . However, :111)' desire to al low users to share resources between d e partment s
, or to have common �ld min istrJtion over the departments i ntroduces administrative and user pro bl e ms.
I f a printer is to be shared by the two departments, it must be administered twice : once in the NetWare environ ment and once in the LAN Manager environ ment.
Users in the two dcp:1rrments use d i fferent nJmcs t(H· the same primer. LJter
NOS
i mplementations, such as
DigitJI 's
PATHWO RKS
,·ersion 5 . 0 or the networking sofuv:1re b u i l t into Microso�i:'s Windows 95 that pro vides support tor m u ltiple
N OS
protocols, do nothing to manage the mu l tiplicity of names for the same net work resource .
As we were contemplating the set of capabilities we needed to design tc>r the next generation of
PATHWOlU<.S
c l ient products, we reJiized that solv ing the connectivity problem i mplied i n a m u l tiple
NOS
environment w:1s not enough . User access and administrJtor control of
N O S
resources needed to be considerablv simpler.
As we looked Jt the problems i n larger networks, we saw the need t(>r the abilitv to pr o vid e more sophis ticated means to locJte
N OS
resources. TypicJIIv,
N OS
c l ient sott\\'are provides the means to bro\\ se the network to locate a resource . Hown·er, browsing requires the user ro know the location of the resource, specifically the name of the server, and to be able to choose the resource on rhc server by recognizing something about thl: resou rce n:1mc or J resource description provided by the :1d ministrator. What w:1s needed was a design thJt allows a user to search, Js we l l as bro\\'SC, t(>r
�l resource bJsed on \'Jrious attrib utes describing the resource .
Fin:11Jv, existing
NOS ell\'ironments have a fairly lim ited ,·ie\\' of the set of resources tllJt can be referenced .
Digir,,[ Tcdmid J uunnl
Vol . 8 No. 1 1 990
47
48
Both NetVVMe and 1·arious
LAN
J'vLm�1ger i m plemen t:nions pro1·ide support f<>r p ri mers and tile sh�1res .
We wamed to be able to extend the t\'pes of resou rces th�1t could be refere nced and lll<111Jged f[·om the new directory capabi lity that we were designing.
Thus we em barked on a design r()r the facility we initially cal.led
I DS , for lntegr.ned Directory Serl'iccs.
The
PA'rJ-fWO R.KS version 6.0 implementation was cventll�< l l v cal led Directory Assistant. We refer to this rcclmology <1S
I DS throughout this paper.
Design Goals
As we looked at the rcquircmcms of the PATHVVO RKS product, we found rh:n m:1nv of those req uiremenrs wJs integrated into the
PATH WO RKS applications and tool sets.
PATHWO RKS required the :1bility to
•
•
•
Gi1'C a single name to resources thJt em be accessed lw means of m u l tiple sen'Crs or protocols l ns u !Jte end users ti·om ch:mges in the w�1\' resou rces are a l locJted :1mong the servers
M:111:1ge resources in a n
NOS-i ndependent manner
\'Ve coul d not simply pick :1 di r ectory service and integrate it i nto
PATH VVO RKS, beclllse we cou ld not require that all c ustomers dcplov J parricubr directon· scn·ice Jt their site . The
PAT I:-I WO !\KS product is both
N O S and transport - i ndependent ; introducing such :1 dependence was u n:Jcccpt:< b l c . We q llickh· rc:�l ized thJt these were the req ui remenrs that kept ma111' other :1pplications from using di n:ctot'l' sen ices.
Our assu mption \\'JS that
111 JIW nct\\'Ork applica tions wou ld usc d irecton· scn·ices if thev could, bur that kw of them could assun1c or req u i re a p:1rtic u lJr d ir e cto r y service. Working fi·om that assu mption, we selected the follo w i n g d es ign requirements for
I DS :
•
•
•
•
•
•
•
Directory service independence
Abilitv to access existing d,m
Abi lirv to join disp:�r:Hc namesp:�ces i nto a single, logical namespace
Rcmm·:�l of barriers to successfu l dcplovmem of
:1 wide area network (WAN ) d i recton·
Abi lity to hide directory nJme syntax
Support of search
Support of application-specific directory entries
Directory Service Independence
Customers must be :�ble to choose the dirccton· sen icc in which the1· store resource i n formation . Some cus tomers ha1 c a prekrred director\' scn·ice, \\'hich the\'
11·:m t to cont i n u e to usc. Other customers, who arc not using a particu!Jr d i rectorv sen ice, prefer that Digital pn>�·ides the directorv sen·icc . I n :1 k11· cases, a cus tomer m ight wish or even need to store intonnation about d i tkrem resources in different directory sen·ices.
Ability to Access Existing Data
A great deal of i n f(mn�nion currently exists
1 11
�lppl ication-specific directorv services and in
NOS speci fic d i rectory scn·ices. A relarivelv large n u m ber of applications a lso usc the native i n terbces to store i n f(m11Jtion in the
N OS d irectories. Al lo\\'ing users to :Kccss this i n form:�tion d irect!\' through
I D S
\\'JS critical . We expressh· II'Jnted to :woid the need to d upliutc di recron· intorm�1tion in separate, incomp:�t i blc svsrems.
Ability to Join Disparate Namespaces into a Single,
Logical Namespace
Manv directory ser\'ices arc aimed Jt a specific applica tion or
J set of appl ic:nions. for example, c u rrent
X . SOO dcplovment s contain mostll' people i n form:J rion such as names, p hone n u m bers, and e lectronic mail :�dd resses. ( Note:
X . S OO is an exrrcmclv flexible di recrorv sen·ice that cJn be used to store almost an1· kind of' i n f(mnarion, bur f(>r historical reasons most deployments contain people i n f(m11ation . )
NOS d i rec tories contain i n t(xnl:ltion :�bou t
NOS resou rces such
:1s printers.
Conseq uently, many user environments hJI'e mu ltiple d irectory sen·ices, c1eh of which con rains critical business intonn:nio n . To access this exist ing d:na and present it to the user in
J meaningfu l 11·:11', these m u l tiple d i recron· n::unespaces must be joined i nto a single, logic:�l namespace .
Removal of Barriers to Successful Deployment of a WAN Directory
H ier:1rchicJI d i rectory services gener:�l l\' req u i re thJt the naming h ierarchy be designed berclrc the directory is deployed . Since t he hicr:1rchy consists of names, and names are sensitii'C and poliriul entities, this em be an e x tremely d i fficu l t task. Oq:�anizations also change m·er time, fu rther c o mpli c
Jti n g the problem of designing a name h ierarclw.'
Org:�nizations that successfu ll y deploy d i rectorv ser vices do so fi·om the bottom u p . The
NOS d irectories
:1rc deployed precisely because thev :woid the prob lems i n herent in a n:�me hier�1rclll'. An administrator c1n set up a Novell 3 .x Bindery t(lr a local organization without worrving about how the name o f one group rel:�tes to the names of a l l the other groups. The downside to the
NOS d irecto ries is that they have
,, l i m i ted :1bi l i t1· to scale be10nd a
LAN .
With
I DS ,
II'C
\\':Jilted to pro1·id e '' fr:�mework rh:1t wou l d gro\\· \\'ith the user's em·ironmenr. A user could stJrt ll'ith a loc1l d i rccrorv b u r incorporate th
:t t dircctorv imo
Jn enter prise or global di rectory "'hen the time \\'JS appropri
Jte, without affecting the end users or the applications.
Vol . H No. 1 1 9\1(>
Ability to Hide Directory Name Syntax
The syntax of the nJmcs in hicrJrchical directory ser vices varies nor only fi·om one directory service to another, bur in some c1ses fi·om one implementation of a single di recrorv service to another. The syntax for
Domain Name Svsrem names is ordered the same as a postal mail address, thJt is, fi·om the most-specific componcnt.
" · 111 For example, a machine at Digital might be bigAiplu.digit�1l .com. The X . SOO name order is usu:�lly ( d epend ing on the impleme ntation ) the reve rse. The corresponding X . SOO name might be: c = us;o= Digital ;cn = bigAipha. Particularh' in the
X . 500 case, difkrent svstems �md �1pplic:�tions also accept difrcrent sep:-trJtor cluracters.
Together, the lOS designers hJve much experience with a nu m ber of directorv services and their n:�me synt:�xes. Users and applications developers alike have been quick to poi1H out the problems with directory
Jumcs. These names :�rc cumbersome, conh1si ng, or just plain inconvenienr to type . The separator charac ters within
:-� directory n:�me may have special mean ings on some operating systems.
Because of these limitations, we decided that a name syntax specific to I DS would detract fi·om the value of the solution. An applicnion using I DS may choose to prescnr its own syntax, one that is suitable to i rs partic u lar environmenr Jnd prekrences. The AP I takes the object name and the context, as described in the
Contexts section. The service provider module uses these ro construct the n:1me in the native name synta x .
Support o f Search
Users need to locate resources in a nu mber of ways.
The most tamil i:�r met hod is to locate resources b\' knoll'ing the ir n:�mc; this is often reterred to as a white pages lookup, n�1med :�Iter the printed U .S. telephone direcrory of �1lph:1beticallv ordered names.
Searching for resources b:�scd upon information about the resources is referred to :�s �l vellow pages looku p, named after the printed U.S. telephone directory organ ized bv business category. To support vellow pages lookup, resources must be retrievable from the di rectory service based on their attri butes. For a printer, this might include the type of printer, the loca tion of the pri nter, whether it supports color or not, who is responsi ble f(.>r maintaining the printer, and other i n form:�tion. IDS needed to support both yellow pages and white p:1ges lookups.
Support of Application-specific Directory Entries
We saw a need to su pport two kinds of extensibi lity: the abi lity f(>r �111 application to crc:�te new kinds of d irectory entries, :�nd the :�bility for a customer ro add attri butes or other descri ptive i n f(xmation to the di rectorv entries created bv PATHWOR.KS or other applications. By providing appl iorions with the capa bil i ty to create new kinds of directory entries, the I DS designers al lowed IDS to be used by any application, regardless of i ts requ irements. By allowing the addi tion of attributes to existing directory entries, we allowed customers to easi ly :�dd i n formation that is speci fic to their organiz:�tion to application objects.
For example, a customer might add J speci fic cod e, such as an asset identification tag, to all printer direc tory entries.
Design of the IDS Fra mework and Service
Providers
I DS is an object- based system th:�t consists of a frame work and a set of service providers. For clarity, we fi.Jr ther divided the ti-amework into an A PI and a service provider interface ( S P T ) . The API consists of a su bset of the framework's objects and their public virtual methods. The SPI is a generalized, di rectory-service independent i ntertace (descri bed in detail later in this section ) . The SPI objects define the abstract interface to the directory service . We use the term
service prouiderto
reter to any directory service that provides
I DS storage. The service providers in teract with the framework through the SPI .
Framework
The fi·amework performs three major fu nctions:
•
•
•
It specities the IDS director y independent opera tions.
It
dispatches oper:�tions to directory-specific mod ules for execution.
It verifies that all I DS objects and operations do not violate the I DS schema.
F igure l ill ustrates the structu re of IDS. When an app�cation makes an API cal l, the fra mework examines tbe name information and calls the appropriate service provider. The service provider then ma kes the call to the appropriate native directory service client. When the d i rectory client returns the results, the service provider converts the resu lts i nto the I DS canonical form. The design supports ju nctions fl·om one d irec tory service to another, in that the result retu rned to the framework by the service provider may be only a reference to an object in another di rectory service.
The abstract interbce to the directory service ensures that I DS provides applica6ons with a consistent level offunctiona�ty without regard to which directory service a customer has in his or her environment.
Because the words "object" and "object class" are overloaded and overused in the industry, we define the words "resource" and "resou rce class" to denote objects represented in I OS. A
resource
is J di rectory
Digiral Teclmi(al Journal
Vol .
8 No. I 1996
49
IDS
APPL ICATION PROG R A M M I N G INTER FACE
SERVICE PROVIDER INTERFACE
'--
SERVICE
PROVIDER
SERVICE
PROV I D E R
SERVICE
PROV I D E R
I-
NATIVE
D I R ECTORY
CLI ENT
FRAMEWORK
NATIVE
D I R ECTORY
CLIENT
NATIVE
D I R ECTORY
CLIENT
I
�
�
�
NATIVE
DIR ECTORY
SERVER
NATIVE
DIRECTORY
S E R V E R
NATIVE
D I RECTORY
S E R V E R
Figure 1
Structure of the I ntegrated Directory Services e n try; it is a directory service object that represents some network object. A resource class is the defi nition of that type of d irectory entry. For example, the direc tory entr y that describes a specific printer is an I DS resource, and the I DS class that describes every printer entry is a resource class.
The framework provides extensibility by defining
C++ object classes that allow for the creation and manipulation of resources, attributes, and attribute val ues i n a type- independent manner. The type inde pendence al lows both appl ications and the ti·amework itself to manipulate I D S attributes and attribute values without knowing their types. As long as the new types are built on top of existing I DS system types, applica tion writers may ddine new IDS types without mod i fYi n g the service providers.
The fi·amework d ispatches d irectory operations to the appropriate service provider and maintains overall system state and integrity. Tt maintains a l ist of the service providers that are currently available a nd shows the errors encountered in any fai led loads.
This allows the svstem to continue to operate, albeit in a degraded st
� te, even though one of the service providers may be malfunctioning.
Before we d iscuss the design of the SPI, we describe tl1e framework's objects.
IDS Entry
The fundamental IDS object is the canon i c a l representation o f a d irectory entry, t h e IDS en try.
The I DS entry is a n abstract object. To create a resource cl ass, applications define a resource type and derive it from the IDS entry. IDS entry objects are cre ated and manipul ated through the API and translated into the appropriate native directory format by the ser vice providers. Derivatives of the IDS entry may define
J.d d i tional methods, but they may not override the
I DS entr y methods. The IDS entry methods are part of the framework.
The I DS entry methods tall i n to one of two otegories: those wh.ich manipulate the attributes and values contai ned in the I DS entry in a type-indepen dent manner, a nd those which perform operations on the directorv. Each I D S entry, each attribute, and each attri bute val
�
1 e contains a type. For convenience, deriv atives of the I DS entry may define additional methods that manipulate certain attributes or values di rectly.
For example, a derivation tl1at defines a printer might define a method to set rhe description attribute. The implementation of this method would cal l the general
I DS entry attri bute and value manipu lation method to set the value of the appropriate attribute.
As shown i n Figure 2, the IDS entry contains identi f),ing information and the attributes and attribute val ues that describe the resource. The context identi fies the service provider that performs directory opera tions on this entry and the location within that directory service in vvhich this entry is stored . The resource type detines the kind of resource that this entry represents. The resource name is the name by which applications and users refer to the entry.
The attributes of the entry are contained in a set.
Each attri bute in turn contains the vJiue or list of val ues associated witll the attribute.
Contexts
The context is an object that uniquely iden tif-ies a particular location in a particular namespace .
The IDS context i s very similar i n concept to the XFl'i context." All contexts contain the type i de ntifier for the directory service and an internal name. The type identi fier is used by the fDS framework to dispatch operations to the appropriate service provider. The internal name is the location within the directory ser vice described by this conrexr. The internal name is represented in the native syntax of the underlying di rectory service. The service provider is responsible tor setting and maintaining this internal name. ( See
Figure 2 . )
Attributes and Attribute Val ues
The type of an
::mribute defines the data type of its va lue or values.
The attribute value object is a canonical representation of an acmal attri bute value. The attribute val ue object de f-ines a set of methods tor accessi ng and manipu lat ing val ues. For each data type supported in IDS, there is a corresponding attribute value derivation in the
50
Digiral Tc-dmical
Jounul
Vol .
8
No.
I 1996
I
RESOURCE TYPE
IDS_PRINTER
SERVICE PROV I DER TYPE: LDAP
CONTEXT
LOCATION WITHIN SP: o=dec;ou=lkg
I
RESOURCE NAME
I
NIST GROUP PRINTER
ATTRIBUTE SET
ATTRI BUTE N
ATTRIBUTE
2
I
J
ATTRIBUTE
1
I
ATTRI BUTE TYPE
I
IDS-ATTR-MAINTAINER
ATTRIBUTE VALUE LIST
I
ATTRIBUTE VALUE DATA TYPE
I
ATTR IBUTE VALUE t
I
ATTR IBUTE VALUE DATA TYPE
I
ATTR I BUTE VALUE t
I
ATTR IBUTE VALUE DATA TYPE
I
ATTRIBUTE VALUE
I
I
I
I l i
I
DS-TYPE
I
JANE DO
-STRING
E
Figure 2
IDS
Entry
IDS framework . This allows applications, and the I DS framework itself, to manipulate attribute values with out knowing their types. The service providers, on the other hand , use the type information ro translate from the I DS data formats to their native data tonnats .
Types
To allow customers and third parties to identi�' their own IDS resources, the I DS type mechanism must uniquely identi�' objects. The two identifiers we considered using were universJI unique identifiers
(UU IDs) as defined by the Open Sofuvare Foundation
D istributed Computing Environment ( OSF DCE) and object identi tlers ( OIDs) JS defi ned by the open sys tems i nterconnection ( OS I ) standards.
" · ' 2
Some direc tory services identity attributes with OIOs, whi le others use UUIDs. For appl ications defin ing new resources, we wanted to avoid the necessity to obtain both an
OlD and a UUID . It is possible to encode a U U I D in an oro, but the reverse is not true.
We cou ld encode a UUID in an OlD by registering an OlD prefix. The prefix would indicate that the sequ ence after the prefix was a U U I D . U U I Ds are fixed-length structures generated from time sta mps and Ethernet addresses, and therefore arbitrary infor mation such as
:111
OlD cannot be encoded in them.
UUIDs are also e::tsier tor application writers to gener ate because nu merous systems ship with tools to generate them.
Certain directory services, for example
X . SOO, have external type definitions tor the directory entries. lt is possible to define a generic entry and then map arbitrary values inro that entry, but I DS entries would not be meaningfltl when viewed with the native di rec tory management tools. We fel t that this was unac ceptable, because it would make the management of
I DS entries in the namespace much more difficult.
Some systems use U U ! Ds to represent the type infor mation. We chose to use UU !Ds si nce they are both easy to generate and can be used in both U U I O and
OlD class definition systems. The use of OIDs wou ld require U U I Ds to be generated for U U I D -based systems and mappings to be maintained .
Digital
TcchniL·al Journal V(ll 8 Nu. 1 l996
5 1
Com m unities
An I DS com nH111irv i s both an adminis n·ativc grou pi ng mechanism and ,1 logic:�! loc
1 tion for
IDS resou rces . When people imcr�Kr wirh the I DS svs rem, rhev sec a com m u n i rv as the org�mizing principle.
The �1dminisrraror contro l s rl1e bou ndaries �111d mem bership of :111 IDS comi11UI1in·. T1·pic1llv, <1 com mun in· represents either a p�1rticuL1r locnion such as a b u i l d ing or a hmcrional grou ping such a s :1
\\ Ork group.
I n iri�1lly, \\'C co nsidered
:1 su p crco nrnt ro
j
o in multi ple directories i nro
:1 single logiol di rcctorv. This supercontnt wou ld ha1·c cont�1incd mul tipl e contexts, one
r(n
c�1eh n•pc of resource su pported bv I DS . We e1
· enrual lv subsu med the supcrconrot i mo a comrnu n i n• :111d oiled it a resource COI1tor l ist. An IDS com
mun in• is ston:d as �1 special object in rhe d i rectorv.
F.ach comm unity's resource comexr l ist describes rhc directories that 111�1kc up the com munirv. The resource context list is rhc kdcr:�rion mechanism by which I DS determi nes where resources of each type :�re stored .
Each entry in the resource context list is �1 pair of resource type :�nd context. As users and a pplic a t ions oper:�rc on entries in
:1 com munity, rhe I DS ti
· amework
(through
IDS enrry and communit\' methods) inspects the resource type and the com l nu nin· to
d
ercnni ne the context. Figure 3 i l l ustrates an IDS comnu1 11in•.
One of the probl ems we anticipated w�1s th�H large organi zations ll'ould nat u rallv re nd to han: 11
1 a nv IDS com muni ties: How wou ld r h c user idemi t-\· these; We considered Jn additionJI h icr:�rchv in
11 hich com m u ni ties ll'ou ld be mem bers o f other com m u n ities. Our usabi l i n· consultants cmpb
;
1siz e d that users should nor have to broll'se a hicrJrcll\' ro �Kcess resources . I n response, we developed the concepts of t h e loca l ami
. atcd \\'ith rhe machi ne J user is curremlv using
it represents a plwsical location . The home com m u 1
1 it)
' is the one with which the user is
J ssociat c d or belongs
. vVe e nv
i
sioned rh:�t the home com mu ni ty wou ld be the s:1me as the local communitv �1r the user's normal place of work, b u t thet-c is t10 req ui rement inherent in the design th:Jt things be orgJ n i zcd this W�1y. ror example, if a user is associ:ncd with the com m u n i rv at her work site and the nuc hinc she uses is
:1 l so Joc1tcd at that 1vork site, both her Joc1l communi ty and
COMMUN ITY
DEFAULT
CONTEXT --------
RESOURCE
CONTEXT
CONTEXT
RESOURCE CONTEXT LIST
OBJECT
TYPE
Svc Provider Type = F7801 DB7-F675-1 1 CD-ABC2-08002B1 87D1 A (ODBC)
External Name = IDS_Group Community lnlernal Name
=
E \\tuxedo\idsodbc\idsdbdir.mdb
Svc Provider Private = NULL
Svc Provider Type = EFF4B840-EC52-1 1 CD-9E5 E-08002BBA95CA (CDS)
External Name = ids_cell.lkg.dec.com
Internal Name = ids_cel l.lkg.dec.com
Svc Provider Private = NULL
Svc Provider Type = C723E850-A
1
A6-1 OAB-A699-08002B361 FC1 (LDAP)
External Name = c=us: o=dec;ou=IDS_Group Community
Internal Name = c=us:o=dec;ou =ID S_Group Community
Svc Provider Privale = YUMMY, 386. TCP/IP
52
KEY: c::::=J
COMMUNITY c::::=J
RESOURCE CONTEXT LIST
-
RESOURCE CONTEXT c=J
OBJECT TYPE
CONTEXT
Figure 3 l DS Com m u n i tv
\'ol. X :--:o I 1 996
her home comnHmiry rqm:senr this \\'Ork sire. If this user works at another work site and uses a different m:1.chine, her home community remains the s:1me, but her loc;1l comtnunity rdlecrs the community wlH.:re the new m:1chine resides. The concepts of local and home communities do nor red uce the nu mber of com munities, but they do provide a d irect method bv
\\'hich users can access the comm u nities that contain the resources the\' most hTquentlv use . The local :md home commu nities are a con,·en ience; users and :lppli cnions are in no wav restricted to those communities.
Search Support
Searching is handled by the search object. The search object contains a commu nity ( or list of communities ) , :1 resource tvpe, and Jn attribute tiltcr. The attri bute til tcr supports both equalirv and comp:1rison matching of attribute val ues and allows c:1llcrs to construct complex requests by conc:�ten;Hing comparisons together in a series of Boole:�n opera tions. For example, a cal ler cou ld construct a tilrcr tlut returned all pri mer objects that ( ( ( are locJtcd on F loor2 ) OR ( are located on Floor3 ) ) A N D (sup port color printing) ) . Combi ned with the lou I :md home commu nity supporr, ti l tcrs allow applications
:md users to express ideas such JS "print this at the closest printer that supports color, t\\'o-sided pri nting,
:�nd then transmit it to ;l n\' bcsi mile machine in 111\' home conHnu nirv. ''
The carch object's ddiult tiltcr returns all objects of the resource type in the locJI community. The search object resolves the community to a context :md passes it to the service provider. The service provider con structs a l ist of matching I I
)5 cntrv objects to return to the user. In I DS, the search object supports browsing.
The search object has mctl10ds that displ:�y a diJiog and construct tilrers bJsed on user input. vVhcn design ing the S\'Stem , we dcb:Jted whether it was bet ter f()r the se:1rc h object to cont<lin both the fil ter and rhe search d ialogs or whether the fi lter constru ction belonged in the IDS en try. We chose to keep the search dialogs separate h·om the I DS entry. Experience with i mplementing resources derived ti·orn the I DS enrn· h:�s shown this to be an error. Currently it is ncc css;ll'\' to derive ti·om two objects, I DS entrv and the sc1rch object, to implement a resource that has a resource-specific search dialog. We will be modirving the search and I DS entry objects so that the construc tion of the ti lters and the di:�log that constructs the ti ltcrs are I DS e ntry methods.
Schema
The ser\'icc prm·idcrs transbte between the n:�ti,·e directorv object and the I DS en tn·. I n genera l , directory sen·ice entries arc n o t self-describing. I n existing directory services, either J schema or the applic:�tion is expected to know the d irectory-spccitic f(mnat of the data . The btter is more common than the former, ,md in Jll\' case the schema methodologies
Jre unique to uch directOr\' service .
From the poinr of \'iew of the native dircctorv ser vice, IDS is the appl ic:�tion. To properly convert the data, the se rvice providers must knmv what it is. The service pro\'idcrs usc the schema to determine the cor rect attribute <1 1ld \'J iue types to use when constructing the
I DS entrv of;1 particular t\'pc .
The schema describes resource tvpcs, Jttri bu te tvpes, and attri bute \'alue data types. Logic:�l lv, the schema is a set of t:�blcs, one for each service provider, which maps the native nJme or type to the I DS name or type. These tables are read by the I DS schema com ponent when I DS is initialized. Bee:� use these tables are external to the system , thev can be modified by users or applicJtions.
There is one l i m i tation on the exte nsion of the schema: Ne\\' Jttribute and resou rce rvpcs can be ddined, but thev must be composed fi'om the prede fi ned IDS :lttri bute \'alue types that the scr\'ice providers can support. The service providers wou ld have to be mod i ticd to su pport Jddition�l attribu te va lue data types. This limitation is not as severe JS it
:lt first appears. A rich set of data types is ddined in the existi ng dirccrorv sen·ices, and a relarivclv sm:\11 set is in common us:1gc. Bv defining the I DS data types ro encompass the set of Lhta rvpes ddined lw existing di rectory services, we have reduced this l i mir:� rion to
J theoretical rather than a practical problem .
As a conseq uence of the use of schem:1, applications must specit)' rile resource type tor Jny I DS operation .
This is a limitation that in principle docs not exist in other directory svsrems. After some consideration, we concl uded that kw usefu l operations can be performed on an object whose type is un known . To perfcm11 an operation on objects of all rvpes, the schema c:�n be i nterrogated t()J' the l ist of all supporred I DS object types, and the opcr:�rion is then iterated over each type.
The System Object
The system object lo�1ds and initializes the service p roviders. On in iti:�lizarion, the svstem object constr ucts a list of the avai i:Jble service providers ti·om those ddined in a local configurJtion file.
The svstem object constructs and maint:1ins the l ist of known communities. The system object obtains this list using the following mechanisms:
•
•
•
Inspect a well- known location (if one exists) to see i f it cont:� i ns a cache of known comm unities.
For c:�ch sen·ice prm·ider, call the discmn method to ask the scn·icc provid er tor its list of known communi tics.
I f the system object is initializing for the tirsr rime, prompt the user to create a community.
Digir.1l T<:dmiul
)ourn;ll
Vol .
X No. I 1 996 5 3
Application Programming Interface
As mentioned previously, we divided the framework i nto an API and a service provider interface (SPI ) . The
APT consists of the search object methods, the IDS entry methods, the attribute object :�nd value object methods, and the system object methods necess:�ry to access communities.
Service Provider Interface
The SPI specifies the interface bet\veen the I DS frame work and the n ative d irectory services. I r defi nes the semantics for aU operations that may be performed on
I DS inf-ormation regardless of which directory service stores the i n f-ormation . The SPI effectively insu lates both the I DS framework and the I DS applications from the unique syntax and requirements of d i fferent di rectory services.
A directory-specific modu le, cal led
J seruice prouider lihrary, provides a directory-service-specific implemen tation of al l SPI operations and translates resource infor mation back and forth between the I DS entry and the service-provider-specific format. A service provider library must be implemented for each directory service to be supported by I DS. Any directory service or infor mation repository system that Gln provide the I DS SPI semantics may be an IDS service provider.
SPI Semantics
The IDS SPI defi nes the tc>l lowi ng main operations: create, read, search, modi�', discover, and delete. All SPI operations spec i t)' the name of the
IDS com munity upon which to operate. Each
I DS communi ty mai ntains a list of contexts rh:�t specify in which service provider IDS resources of a particular type arc stored and in what location within the service provid er. The SPI uses this comm unity name to retrieve rhe context i n formation that di rects the oper ation to the correct service provider library. \Vith the exception of the del ete operation, which requires an explicitly set context ( to be sure that an expl icitly located object is selected for de letio n ) , i f the cal ler does not set the commun i ty name, the local commu n ity is assumed .
The create, de lete, mod ify, and read functions all operate on a single I DS resource ar a rime. Each, therefore, provides an I DS entry object to idcnr it)' and/or describe the resource.
The create operarjon creates a new l
DS resource i n the d irectory. The create operation specifics the type of
IDS resource to be created, the resource's name, and the I DS attri butes and val ues associ:�ted with the resource. On a successfld create oper:�rion, rhc service provider constructs a unique directory-specific 1wme fi>r the new I DS resource and stores this name in the object's I DS entry. The service provider su bsequen tly may use this name to find the object more qu icklv rather than constructi ng it fi-om the name, resource type, and context information contained in rhc IDS entry.
54
Digital Tcchniul journal
Vol .
H
No.
I 1 996
Before constructi ng the resource in the directory, rhe operation valicbres the I DS entry against rhe schema to ensure that it does not violate the schema .
For ex:�mple, attem pti ng to create a resource without a required attribute va lue pair viol ates the schema and is flagged as an error. Conversely, the delete operation removes the I DS resource from the directory.
The mod i fY operation updates the attri bute :�nd values associated with the resource in the di rectory.
The modi�' operation su pports the following u pdate directives:
•
•
•
•
•
Add a new attribute and value.
Add a new value to :�n existing attri bute.
Replace a val ue of :�n existing attri bute.
Delete an :�ttributc and its associated values.
Delete a value frorn an existing attribute.
Each modi f)r directive is veri ried against the schema before being applied to the directory.
A read operation retrieves a uniquely specified
I DS resource fi·orn the di rectory, translates i t into
IDS entry format, and returns the I DS entry to the call er. The read fu nction is typicaUy used to com pare the di rectory format of an I DS resource to one maintai ned in memory by an application, or to process
IDS resources returned tl·om a search operation one at a rime.
The search fi.I JKtion identities and retu rns IDS resou rces that match the characteristics specified by rhe cal ler. To bound the scope of the search, the ca ller specifies the fol lowing search characteristics: resource type, commu nity n:�me or names to be searc hed, and a fil ter containing attri butes and associated values or value ranges.
The discover operation is called by the I DS system object ro fi nd all comm unities known to a given ser vice provider. Service providers for directory services that support a server solicitation and advertisement network protocol i mplement a discover function. In these directories, servers advertise their presence in response ro network sol icitation requests. The dis cover method uses the d irectory's native solicitation and advertisement protocol to discover local directory servers and then issues the appropriate operati ons to the server to determine if it has defined any I DS com munities. Service providers that do not have a solicita tion and advertisement protocol can i mplement an alternative discovery mechanism such as retrieving the community i n formation from a file or provide no dis covery mechanism.
Construction o f t h e System : Di rectory, Session, and
IDS Entry Objects
The SPI is constructed of three framework objects: the di rectory object, the session object, and the di rectory operation methods of the
I DS entrv object. The directory object is responsible
for service provider initi�1lization and termination,
mai nte nance
of session objects, and com munity dis covery. Each service provider exports one d irectory object to the I DS framework. The session object
implements
all the directorv operations on a service prm·ider. Session objects are obtained fl·om the service provider by means of the di rectory object. The fDS entry directorv operation methods determine the con text if i t
has
not been set, obtain a session object from the proper dirccrory, and d ispatch the operation to the
Jssociated service provider through the session object.
For eHicie ncy, session objects may be cached by the service providers.
Implementation Considerations
Once we bad establ ished our basic approach , we turned our attention to i m plementation decisions.
Client versus Server
Our first consider:�tion was whether to implement this technologv as sofTware executing on a server system or as soft\V�l i"C execu ting on a client system . The server solution had a n u m ber of attractive qualities: it woul d
110t be necess:Jn' to have a l l the native d i rectory cl ient s o n :�II the desktops, Jnd potentially complex pro cessi ng wou ld occur on
:� n :�ppropriate platform .
However, we identified two problems with the server solution . The tirst concerned security. To access rhc di rectory service on bei1Jlf of a particubr user, we would have to imperson:�te that client user on the server machine .
Although this c:1n be done without ex posing securi ty holes, doing so adds :�nother layer of complexity ro the problem. The second problem with rhc server solution w3s thJ.t it requ ired the customer to find a machine tc>r :1nd depl ov
:1 server prior to get r i n g started with the syste m . One of the design goals
WJS to remove lnrriers to directory deplovmcnt, and
II'C were concerned that a server solution would add a barrier. \V<.: s�l\\'
:1 need tor both client- and sen·er based solutions, and since the client solution was sim pler to implement, we chose to stJrt there.
Security
The I DS intcrbccs leav<.: security to the underlying directory services; we did not attempt to abstract
:1 general-pu rpose, �Kccss control or authcntic3tion interbce. The primary reason for this was a conviction that the
V
< l
S t m:�jority of current directory information is world 1-c:ld, and therefore a complex access control i n terbce wJs nor necess:�ry. An access control :�nd
Juthen ticarion l:!yer th:�t was di rectory-service independent would have :�d ded significantly to the complexity of the project, and we chose to postpone this prob l e m . ] D S docs pass requests d irectly to the native direnorv-sen·icc client; IDS does not alter or im personate the user's identity. In that sense , it
pertecrly p reserves the security inheren t i n the u nder lying directory services.
Filter Implementation
The imp lementation of the I DS attribute filter is based on the string fi l ter as defined i n RFC 1 777 -'' The
Lightweight Directory Access Protocol ( LDAP ) string til ter provided :1 convenient internal representation , and we would be J.ble to reuse the LDAP parsing a nd processing code that we had developed as part of an e:�rlier prod uct. We considered using SQL to construct
I DS attri bute filters, but chose not to do this for imple mentation convenience.
Service Provider Considerations
Initially, we though t that developi n g a directory service -independent interface wou ld not be d i fficult.
Most of the required oper:�tions such as read and write are straigh ttorward :�nd obvious. The implementation of such an i nterface, however, proved to be d i fficu lt because the underlying directory services have, in some cases, verv different native capabilities and sen1:1 ntics.
We chose to implement service provider libraries t(x the t(JIIowing th ree types of service providers :
•
•
•
Open D:�tab:�se Connect ( ODRC)-co m pl i:�nt dat:�b:�se
X . 500-based d irectory using the LDAP
DCE CDS
These service p roviders are representative of the types of directories th:Jt exist today. Table
1 high l igh ts some of the d i fferences :�mong the three di rectories. As this table ill ustrates, not all d irectories can nJtivcly su pport the semantics described by the I DS S P I .
I n these situations, we have fol lowed three J!terna tives: ( l ) the service provider l ibr:�ry implements the fu nctionali ty, ( 2 ) the IDS framework implements the fi.1 nctiona litv, or
( 3 ) in a small number of cases, the service provider c:�nnot implement the fu nctionality and remJins less functional .
Some operations c:�nnor be supported natively by only one or J sma l l handful of directory services. For these operations, we requ i re the service provider devel opers to i mplement (or emu late as best they can ) rhe fu nctionality in the specific service provider library for that directory.
For fu nctions that a number of service providers cannot su pport or that arc suHiciently difti cult to i mplement, we provide a common implementa tion or emulation in the IDS framework that service provider li braries c:�n cal l . For example, CDS does not natively su pport an attribute-based search mechanism .
Rather than :�ttempt to implement
:1
CDS search capa bili!J', we chose to provide an I DS ti·:�mework "prune" fu nction thJt :�pplies an I DS filter to :1 list of I DS entries
�md returns onlv those entries that s:�tish' all cond itions of the filte r. Service providers such as CDS can then
Vol . 8 1'\o. 1
1 996
55
56
Ta ble
1
Differences among the ODBC, X.SOO, a n d CDS Di rectories
Functional ity
Distributed di rectory service
H ierarchica l organ izati on of d i rectory in formation
Attrib ute-based sea rch
Attribute val ue-based search
Native schema su pport
User can extend IDS schema
Tra nsactional sema ntics
Tolerant of interm ittent conn ectivity
Provides security mecha nism on connections
ODBC
No
No
Yes
Yes
Yes
No
Yes
No
No x. soo
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes emul ate the I DS search function by enumerating all resources of a particu lar type and then cal l rhe prune function to pare down the l ist of resources.
The IDS schema implementation is another exJmple of a common capability we have provided tor all service providers to use. Not all service providers support object, schema and, of those that do, fe wer still can sup port user extension of the schema. We chose to allow user extensi bility and i mplemented a service- provider independent schema interface and mechanism.
In a tew instances, we determined that ir wou ld be too expensive i n terms of implementation time ro pro vide a service-provider-specific or a n 1 DS-ti-::llnework implementation of an SPI -mandated function . I n these cases, w e al lowed the service provider to remain noncompliant. For example, a cal l to i nitiate a session to a service provider specifies user name ;�nd p�1ssword argu ments. For those di rectories that su pport user name and password security mechanisms, we preserve that functionality. For directories such as the O D B C service provider that do n o t support these securitv mechan.isms, however, we provide no add i tional sec u rity measures. T h e cost to i mplement a n d deploy such a securi ty mechanism outweighs the gain of h::wing the add i tional features.
In addition, we fou n d that nor all d irectories pro vide the same semantics tor a particular opcr::1tion . for example, when updating a resource, service prm·idcrs handle existence checking of resource attributes d i fkr ently. If req uested to add an attribute value to an attri bute that does not yet exist, one service provider returns an error, while another implicitly creates the attri bute. We worked around problems of this type by carefu l ly specifying the semantics and error cond itions of al l SPI operatiom. Service providers that do not natively support these srr seman tics musr implement whatever add i tional fu nctional ity is req uired ro do so. For exam ple, the CDS service provider req uired additional fu nctions that determined and flagged whether or not a p::1rricular attribute existe d .
I n addition to all errors that are specific to sen·ice providers, '''e retu rn an error rhat is independent of any IDS ti·amework service pro1·id er. This adds another level of consistency across our sen·,ce provider impl ementations.
Current Applications
As with any foundation technology, the proof of its viabilitv l ies with the applications that employ it. In the
PATHWO RKS prod uct, we currentlv han� three appli cations th::tt usc I DS:
•
•
•
Network Connect
I DS Administration
Resource S)'nch ron izer
The Network Con nect applicJtion fi nds and con nects users' printers and tile shares . I t prm·i des a user i nte rface that a l l ows users to browse or search tor tile shares or pri ntns. Through Network Con nect, users can refer to resources by their logical name or their attributes. A single physical printer, with queues on several machi nes or several NOS systems, is presented to users as a single prin ter. Network Connect uses the
I DS API to access the IDS search capabi lities and to translate a pri mer or file sh;lrc's I DS name to its network-specific IUllle to con nect to the resource.
Network Connect mav be accessed through the
Windows version 3.1 Print M<ulJgcr Jnd rile Manager util ities and th rough the PATHWO RKS Network
Con nect utility.
The IDS Administration utility ( I DS Ad m in
) allows a network administrator to m�111;1ge I DS resources and com m u n ities. I DS Admin is int egrated into the
Digi tal ManagcWORKS Workgrou p Administrator for \.Yindows software producr.
1 · ' Ad min creates, mod ifies, �md deletes resources and communities. It a lso al lows users to browse IDS resources and commu nities in the ManageWORKS hierarchy and to search for I DS resources.
CDS
Yes
Yes
No
No
No
No
No
Yes
Yes
Diftiral Technic3l jou rn.\1
Vol. S
No. I 1 996
An ad mi ni strator c111 manage IDS resources manu a l l v th rough the ManageWORKS user interbce or can rely on information p rovided th rough the semi<lllto matic r esource collection utilities called the Resource
Gatherer an d Resource Synchronizer. The Resource
Gatherer periodically collects information about network LAN
M an a g er :md NctWare p ri nters and rile shares. The Resource Svnchronizer utilit\' processes the ga thered i n torn1 a t io n , updating the d ir ec ton
'
.
I t
�1 lso eliminates d u p lic.Hc entries a n d discards informa tion the administrator wishes to ignore. The ga therer and synchronizer allow the directory to be kept up-to d� ne , even if resources arc added or removed through the native NOS in terfaces.
Future Work
In the fu wre, we plan to improve the JDS extensibili t v mechanisms. Cu rrently, a lo c a l cop�· of the s c h ema ex ists on every client. Propaga ting the changes to each client wil l become a problem as users a nd applications extend the sc hema. \Vc arc c o nsidering storing either the schema or a pointer ro the schema in the di rectory.
The current I DS i m plementation run s on both the Windows v e r si on
3 . 1 a nd version
3 . 1 1 operating
S\'Stcms. We are currentlv porti ng i t to Wind o\\'s 95
�111<.l in vestigati ng ports to other operati ng svstcms, such as U N I X .
The implementation docs nor su pport the e n tire
I DS d esig n
:
Although resource c o n t e x t lists arc i mple mented , there is no rcason:1bk way tor a user or adminis trator to crc::nc them. The user intcrf.1cc work ti:lr these katu res in the I DS Ad min application has not vct been com pleted .
Summary
I DS pro vides a mcch:mism tor integrating m u l tiple di recto r y services into �1 s i ngle system . It is predicated
011 the abi l i t y ro define
:.1 common et of directory opc r
::�tions and on rhe type tion of info r m a tion . The implcmenr::� t h r ee verv difkrent servi c e providers - C DS,
X . 5 00, and ODBC - i nci ic::�tes that w e succeeded i n dctining t h e dirccton· o per:-�tions
.
The usc ofiDS i n the
PAT HWO RKS prod uct shows that it does add ress the practical aspects of the problem of integrating mu ltipk directories into a single, logical directory service.
Acknowledgments
We II'Ould like to rhan k rhe manv p a st and present lllcm bcrs of the I DS tc1m ll'ho contributed to the design and i mplementation of the product. Special thanks to K on st
� lnt i no s Baryiames, Anthony
Hin x man, David Magid, Tracy Teng, and Ta mar
\Vexler. vVe wou ld also like ro thank the mem bers of the Directo n
·
Task F orce, Dah Ming C h i u , Dennis
Giokas, and William Nichols.
References l . CC!Tr
Necr! li!lilel ldation
X 50 1
(
1 992 ) and
!nj'omwtion Technology - Ope11 S)'stenls lnter
WIII' Iection-Fhe DirectOl l': ,1,;Jodels. !SO/I EC
9594 · 2 : 1 992
( Gcnn-;1: Intern�rio n�1l Orga ni zation tor Sr;�ntbrdization/l n ternational Ekcrrotec hn ical
Commission, 1 992 )
.
2.
" Naming Concepts" i n Using
Net \!Vare Serl'ices for
NI.Ms ( Provo, Ut;l h : Nove ll, Inc., 1 99 3 ) .
3 . AE'S!Distrihutecl Comput inJ!,- Oirector)'
Serl'ices
( C1111 b1·idge, lV!ass . : Open Softw;ll·e fou ndation,
1 99 3 ) .
4. "S tn:erTalk N;lming Ser� icc" in E.\S Adlllillistrotor �\·
Plan11i11g Cu ide ( Westborough, lvbss. : Bam·an
Systems, fnc., 1 992 ) .
5 . " M icrosoft
Oi n.:crory Services Str�H<.:g)'," � \\' h i tc p•lper from the l� usi ness Systems Tcch nology Scries
( Redmond, W<lsh . : Microsoft Corpor� tion, 1 99 5 ) .
6 . )<J0j)(?l!
Crl l:'
.vJrxi/i'cation. Federated Atlllling· The xr-:v Specijlwtion ( Read i ng, U . K. : X/Open Com pan\' Ltd ., 1 995 ) .
7 .
"
B i n d e n
·
Scn·iccs" i n
Net Ware S)'stelll lnterjctce:
Techuical
Orerl 'ieu·
( Provo, Uta h : Nove l l , l nc.,
1 99 0 ) .
8 . S. R
J d ic a ti
,
" I m ple menting the DIT" i n
X. 500 /) ir ec
IOJy Sen 'ic<'S: Technulop,y and f)ep!o vment ( New
York : Van Nosn·;md Rei nhold, 1994 ) .
9.
P.
Mockapctris, " Dom<Jin NJmcs- Conu.:prs
�nd
Facilities," l nrcrnet Engi n eeri ng T1sk Fmcc, R.l'C:
1034 ( Ncl\'cmbcr 1 9 8 7 ) .
1 0 .
P.
M oc b p ctr i s, " Domain Names- 1 mplcmcntation and Spccitication," l mernet Engineering "L1sk force,
RfC 1 0
3 5 ( Novcmber 1 98 7 ) .
1 1 . Al-.'5/JJistrihuted
Computinp,- Nenwte l'mcedure
Call. AppenchY
A
( C:�mbridgc, M:�ss.: Of)t:n SoFtware
Fou n dation, 1 993 ) .
1 2 . CC!7T f?eco/11 lllc>I1 Ciat ion 208 ( 1992 ) ;l nd t ion Techuolo,L!..)'-Upen lnj(mna
S)'stems lnh·rconl lect ioii
Abstmct S)'ntox Notation One (/iS;V. ! J JSO/I EC:
8824- 2 : 1 992 ( Ccn e\·<1 : 1 n tern<ltional Org:�nizarion t(x
Stantilrdiz;Hi on/l nrcnutional Elcctrorcch nic1l Com m issi on,
1 992 ).
1 3 .
W. Yco ng, T. H m,·cs, ;lnd S. HardclStlc - Ki l l e,
" X . 5 0 0
Lightweight Di rccton' Access Protocol ," Internet
Engincc:ri ng T.1sk Fon.:e,
RFC 1 777
( M�11·ch 1 995 )
.
1 4 . D.
Giobs
� nd ) . Rok icki, "The Design of ,\r!Jn �lge
WORKS: A l ' scr I nred>1cc: hamcwork," f)<� ita/ 'f('ch-
11 ica/fou mal. l'ol. 6, no. 4 (
F:\11 1 994 ): 63-74.
Di!o\iral Tcd1 11ical jou rnal Vol. 1\ :\o l 1996 5 7
Biographies
M argaret Olson lv!ar-garet Olson is a consu l ting software e ngineer in the
Network Software Group. She was the project and tec h nical leJder ror th e IDS development project. for the l ast six years, she has had tec hnical leadership roles in Digital's
Di n:ctorv Services Group. Before joining Digi tal in
1 989, she worked in the networking and distributed. compu ting areas at Apollo Computer. She received a B.A. ( Sigma X i ) h·om Wel lesley College i n network licensing in
1 98 1 .
1 98 8 .
S h e published a paper on
Colin
Strutt
Colin Strutt is a consu l ring software engi neer and tech n ical director ror Teaming Software in rhe Network Software
Group, where he is helping to defi ne new PC- based soft ware prod ucts. Previously, he has held technical leadership rol es i n di rectories, ncrwork management, and term i nal server development, and before rhat l ed product d evelop menrs in F.tllerner servers and DECnet. He joined Digital in
1 9 80 fi-om British Ai rways in the U .K. He received a
B .A. ( honours) in
! 972 and a Ph.D. in
1978, both in com puter science fi·om the U niversity of Essex, U .K. He is a member of BCS and ACM. H e has two parents issued and several patents pending and has pu blished extensively, par ticularly on ma nagement technolog y
.
Laura
E.
Holly
I
.aura Holly is a pri ncipal engineer with the Network
SoFtw<lre Group. She was a key technical contri buro r ro the
I I)S development ef lort. Laura has previously con tributed to the areas of DCE, distri buted system, and knowledge- based system deve l opment. She joined Digital in
1 91\S
,,Iter receivi ng an A . B . ( high honors ) from Smith
College. Lama holds a patent and has publ ished se,·eral papers in the area of k n owledg e
based systems.
58 Dig;ir01l
Technical
Journal Vol .
8
, o.
I 1996
Design of the Common
Directory Interface for
DECnet/051
I
Richard
L. Rosenbatm1
Stanley
I.
Goldfarb
Digital has developed the Common Directory
Interface (CDI) as the means by which DECnet/
OSI can now access and manage node name and add ress information i n multiple directory ser vices. CDI com prises libraries for node name-to address translation and a tool set for managing and migrating node information among differ ent directory services. The Common Directory
Registration API is layered on top of a set of directory service wrapper routines to provide an extensible mechanism for adding new directory services. CDI gives customers greater flexibility in choosing a directory service and s u pports the new multiprotocol ca pabilities in DECnet/051, which support the open systems interconnec tion (OSI) standards.
The Common Directory Interface ( C D I ) provides the ability to store and retrieve DECnet node information from a variety of directory services. It consists of the
CD! Ebrary, which enables m u l tiple d irectory access, and the COl registration tool scr, which creates and maintains node/add ressing inf(xmation in m ul tiple directory services. CDT was developed for the DECnet/
OSI for OpenVMS operating system version
6.0
and for the DECnet/OSJ for Digital UNIX operating sys tem version
3.0.
This paper begins by presenting the product goals and the background of the
COl design. It then dis cusses the structure of the CD! components, the CD! li brary, and t h e
CDI registration tool set.
Design Goals
As the interface to DECnct node inf(Jrmation from multiple directory services,
CD! was designed to meet the following goals:
•
•
•
•
•
•
Give DECnct network administrators and users a choice of directory services.
Provide system adm inistrators with an easy- to usc node registration rool.
Enable easy and flexible configuration of directory choices.
Provide d evelopers of the DECnct protocol soft ware with a simple i nrernal interface that h ides the complexities and ditkrences between the various directory services.
Provid e a common design tor both DECnet/OSI platforms: the OpcnVMS and the Digital UNIX operating systems.
Interoperate with older, non-COT systems.
Background
In
1 99 1 ,
Digital updated i ts DECnet networking products to i nclude the use of
the
DECdns distributed directory scrvicc . 1 D ECd ns provided a highly scalable, distribu ted information source for translating node names to add resses and add resses to node names.
Digital Tcdmic1l )uurn;d
Vol. 8 No. I l9Y6
59
60
Initi:dlv, customer accepta nce of this name service ll'as low ror a n umber of reasons:
•
•
•
•
•
Adoption of this new tech nology req uired a signif-i cant learning curve.
Sign i f-icant planning was required before rhe
DECdns service could be deployed.
Users ofsm�l l l netll'orks d id not need the featu res of a d istributed naming sen·ice - rhe costs our weighed the be nefits. These customers req uested a naming service based on local files similar to the Phase JV DECnet prod uct.
Customers were deploying a nu mber of other directory se r vices in particul:�r the Donuin
Name S y stem for storing host inrorm:nion for transmission control protocol/internet protocol
(TCP/ I P ) networks.2
A ne\\' comprehensi\T sen·ice, X.SOO, h:�d the adqntage or' being an international stancbrd . '
These reasons, together with the need to directly sup port TCP / I P host n:1mes and ad dresses, prompted
Digi tal to incorporate new directory service choices in a new release of D ECnet/OSJ softw:� re .
CDI: Basic Design
Supporting mu ltiple name sen·ices requ i red decisions ro be made concerning naming synt<l \ , m u l tiple
�1ddress formats, �md local rile support. These decisions aft-Cered the design of both the CD! l ibrarv and the
CDI registration tool set.
Client-based versus Server-based Design
The earliest <lnd most h.l lldamental design decision
\\'�ls choosing between a clit:m- based or a sen·cr-based sol u tion. With a clienr-b�1scd design, suppmt for the \ ari ous directory services wou ld be accomplished through
;j variety or' cliellt b�lSt:d programming librarit:s. With a server-based desi gn, a single client li brary wou ld commtmicate with a new "mu lti headt:d '' server that wou l d fan out to tht: di rt:ctor\' servers.
Si nce c l ients outn umber servers, a client-based approach a ft-Ccrs more systems during the upgrade process. In spite of this d rawback, we chose a client based sol u tion for the following reasons:
•
•
•
•
Implementation of the client- bJscd design wou ld be less compln than the server dt:sign .
A c lient-based design did nor have the syntax and protocol transbrion issues of
<1 sen·er-based design.
'vVith a sen'Cr-based solution, client changes " ould stil l be required to support new native naming syn ta n�s .
For sm�1ll installations, no servn wou ld be needed if node i n rc>rmation was stored in a local t-ile: local ri le su pport was nor possi ble with a snver-only appro�u: h .
Naming Syntax
One of the most visible complications when su pport· ing mu ltiple naming services is the nt:ed to recognize d i tkrt:nt nanK synta\es. Table 1 givt:s the d i fferent syn uxes t< >r th ret: widely used di rectory services.
A further complication of supporting difkrent name
S\'IH:l\t:S was the usc of an internal D ECdns name format bv netm>rk management. On<.: of the goals of the CD! dt:sign II'JS to allow management requests to be exchanged \\'ith older, non-CDI systems.
For the initial implemenution, C D ! conti nues to su pport the internal DECdns format, r�nher than use a newer, non -D ECd ns specif-ic forma r alongside rhe existing one. As a resul t, C D ! is req u irt:d to map non- DECd ns names onto the DECdns format . For e\amplc ' the name bq.XI'Z
COl l i ri·om the Domain
Name Svstem maps omo the D ECdns name
DO. I fc1 1.\: bq .nz.wm ( actuJIIv onto the i n ternal
DECd ns fcm11 of this name).
Multiple Address Support
AJ ong with the in trod uction of C D ! , a major inno vation in this re kase of DECnerjOS I wa s direct sup port rc>r TCP/ I P transports in :�dd i tion to the existing
Ta ble 1
N a m i n g Syntax
Di rectory Service
Example Name
DECdns
Domain Name System
XYZ:.hq .sales.system 1 system 1 . sa les.hq.xyz.com
X. SOO
/c=
U
5/0=XYZ/ou =hq/o u=sa les/a p=syste m 1 /ae=D E Cn et
Notes:
The
X. 500 service is not supported by the first release of CD!.
The syntax shown for
X. 500 is commonly used but is not part of a standard.
\'o l . X ;\lo. J l lJ96
support r(lr D ECnet Phase IV <l lld
0 5 1 .
To simplit-\· the initi J I implementation,
IP add resses JIT rcrriC\Td on l \' ri·om the Domain N:� me System ( not from DECdns).
However, rhc design of CDI allows the rctrie,·:�l of both ki nds of add resses ti·om an\' su pported di rcctorv; f(>r example, OS! add resses can be obt
:� in cd from the
Domain Name System.'_;
Support of multiple protocols crc:�tcd :� norher nam i ng issue. Many customers already have a Domain
Name System in place in their networks. Otten DEC:net systems arc
;\
[
so ru nning
TCP /IP protocols •md ;u·c r eg
istered i n the Domain Name System, vet these systems arc nor running DECnet software m-er
TCP/ I P . for example, a SI'Stcm registered as hq.\\'Z.com ma\' be di rect!\' n.::�chJble with
TCP/TP bur nor 11 ith DECnct o1·er TCP/I
P
. In this case, it is possible rhar C D ! nuy rctrie1·c a vJlid lP add ress tor a remote SI'Stcm that is unreachable by the D ECnet protocol . for these rc:�sons, when
C D ! determines rh:�r both the
Donuin Name System and the D EC:dns naming service ( or a local fi l e ) are spe c iri e d in the seJrch pJth, ir docs not stop processing rhe se;m:h path u nr i l both the
I I' add ress and the OSI add ress l u vc been obr:�ined, or u mil rhe end of the list has been re;Khed . ln this way, if the desired remote system is not running
D ECnct oYer
TCP/ I P , an Jtte mpt to connect will be m;ldc through the DECnet protocol , using
3 connc c rion lcss network se r
\' ice
( CLNS ) O S !
<lddress. local ri le with the DECdns nammg servic e
. U n ror tunatelv, administrators of sma l l - and medium -sized
networks
fou n d th n
: the bendits of DECdns (sca l ing and centralized m;magemcnt) were outweighed
lw
i ts add i tion.:d complcxitv.
A subsequent version or· D ECnet/OSI i ntroduced the Local Naming Option. T h is al lo w e d approxi mately 1 5 0 nodes to be stored in a locJl tile, but many c ustomer s found this number to be too sma l l .
C D I supports a very brge local fil e : t h e s u pp o r t e d limit is 100,000 nodes, but there is no tixed i n ternal limit. In addition, through the use of the se.1rch p:1rh, customers em con figure the local fi le either as a ba c k u p to a distributed service, or as a way to provide greater performance. Note that both of these qualiti es are also provided in '' more au tomated wav by the
C D I cache (see rhe CDI Li br;m· Cache section for more intormarion ).
Security Considerations cor relies upon the security provided by the underlying directory services (or in the case of the locJI file, the file system ). Security of i ts remote management teaturcs depends on the network management security svstem.
CDI Libraries: Basic Design
Local File Support
Early versions of the DECner networking prod uct ofkn:d onlv '' local file tor node -to-Jddress informa tio n . The first release of
D ECnet/OS I replaced the
C D I is i m p l ememed as shared libraries on both the
Digital
U N I X and the OpcnVJv!S oper:ning svstems.
At the highest lei el, the design is identical on both sys te ms, as shown i n Figu re l . Narne-to-Jddress transl a tion req uests from the session control layer are passed through J single entry point in each
C D I li brary.
!
NETWORK
MANAGEMENT
I
DECN ET/OSI
APPLICATION
DECNET/OSI
APPLICATION f----------1:
L-t--
I
1
COl SHARED LIBRARY
----1
:
I
I
r - - - - - - - - -
�
I
I
LOCAL FI LE
I NT E R FACE
1
I
I
COMMON DIR ECTORY
I NTERFACE LIBRARY
�\�
T i
F
�
C E
I I
I
SESSION CONTROL LAYER
- - - - - - -
DECDNS
I
INTERFACE
I
I
FUTU RE
�--i-----1-----7--------
:
I
I
I
I
I r J
DOMAIN NAME
SYSTEM
S E RVER
DE CONS
S E RVER
Figure 1
Block Diagr.m1 of the
CDI
Libran·
Vol . 8 No. 1
1 996 6 1
Depending u pon the search path (descri bed bdow), the CD! libraries translate and forward the requ est to one or more di rectory services ( or they look up the information in a local file).
The CD! i mplementation was considerably more complex on the Open VMS operating system
( han on the Digital UNIX operating system due to the dif fering design of D ECnct/OS I on each system . O n the D i gital U N I X operating system, the DECnet/OSI session control layer consists of a shared libr:1rv that is linked with each network application . i-h mc resol u tion requ ests arc processed synchronously. On the Open VMS operating system, session comrol is a component of the N ET$ACP process. Since all name resolution requests are channeled through this single process, operations must be asynchronous ( requests must block concurrent operations) . I n addition, since multiple requ ests mav lx sim u l taneously outstandi ng, the l ib rary is multithreaded . Asynchronous, multi threaded oper:nions o n the OpenVMS operating system are impl emented using the asynchronous sys tem trap (AST) mechanis m . For these reasons, the
CD! implementation on OpenVMS was much brger and more complex.
CD/ Search Path
Another goal was to permit flexibility in determining a configu ration of directory services. The CDI design achieves this goal in rwo ways. First, i t allovvs admin istrators to select their service( s) of c hoice
:md to use them in any order. The search path is norm:�lly cre<lted during network con figuration and can be subse quently managed either loc:�lly or remotely. Second, it gives network users the ability to use short, :� blm::vi
:ned names i nstead o f potentially cumbersome ful l n:�mes. For example, they can use "system l " instead of
"system l .sales.hq . xyz.com ."
A single mechanism i n the CDI li brary-the CDI search path-provides these rwo capabilities. The search path consists of J series of director}' service/name temp late pairs, as shown in Figu re 2a.
When the CDI library is given a name to process, i t scans the search path , rep!Jcing t h e
" * " in the name template with the supplied name. For example, if the li brary was searching for the name frodo. it would use the directory services identified from the n:�mes gener ated shown in Figure 2b.
D uring network configuration, a def.1tl lt search path is automatically confi gu red based u pon the local node name and the admin istrator-specified dircctorv services. This search path behavior nu mber of existing
TC:P is similar to a lookup implementations.
CD/ Library Cache
Occasionally, name service lookups can take a long time to complete ( f(x example, if req uests �lre travers-
62
Disit,11 Technic�! journ,11 Vol . � �" I 1 996
DECdns
DECdns
DECdns
Domain
Domain
XYZ .hq.sales.'
XYZ: . D NA_Node_synonym.
·
·
. sales. hq . xyz.com
(a) Direnory
Service/Name
Tempbte Pairs frodo
XYZ:.hq.sales .frodo
XYZ: .DNA_Node_synonym. frodo fro do frodo.sales.hq.xyz.com
(DECdns)
(DECdns)
(DECdns)
(Domain)
(Domain)
(b) Address I .ookup tor
N<HlK
ji·odo
Figure 2
Using the
CD!
Search Path ing a slow network l ink, a lookup could take several seconds) . To i mprove performance, the CDI li brary incorporates J single cache thJt :�ccumu!Jtes node information ti·om all the directory services. Usually, the cache is consu lted before sending a req uest to a remote service. However, if session control deter mines th::Jt cached i n formation is stale-t<.>r example, if con nection to a node at a cached add ress reaches a node with J. different name-it wil l reissue the call, requesting that the cache be bypassed .
EJ.ch entry in the cache has a creation time stored with it. The c1che i tself has a " time -to-l ive" va lue that can be modified by the administrator. If a cache lookup fi nds :�n entry whose lifetime ( time since it was created ) is greater tl1an the time-to-l ive value, the c:1che entry is purged .
To prevent a period of low performance i mmed i ately after system start-up, the cache is preserved across system reboots by periodically checkpoi nting it to a disk ti le. The c hec kpoint interval is adjustable by the <ld ministrJtor.
CD! Registration Tool: Basic Design
The CD! registratjon tool provides fu nctions to create, modifv, rerume, display, and delete node name and
:�dd ress information in any of the supported directon' services. It runs on tl1e major D ECnet/OSI platforms, tl1e Open VMS and the Digital U N I X operating systems.
The basic req uirements for the CDI registration tool were the same as those for tbe CD! li brary. These three requirements were the need to:
•
•
Support different directory services tor storing node intormJtion
Access each directory service using the appropriate application progrJ.mming interfaces ( AP!s)
•
Store data i n each directory service using the
:1.ppropriate data types
I n addition, the tollowing requ irements were spe cific to the CDI registration tool:
•
•
•
Both a forms and console user interface bad to be provided . These had to work identically on all
DECnet/OSI operating S�'stem platforms.
Functions to transfer node intonnation between the various directory services had to be provided .
Other applications such as the DECnet/OSI network control language ( NCL) utility and other namespace management tools had to be Jble to access node name n1Jnagement functions.
The directory services su pported by the CDI regis trJ.tion tool are slightly different from those supported by the CDJ library. The CDI registration tool supports the DECdns, the local tile, and the D ECnet Phase IV database services.
The DECnet Ph:tse J V database is supported by the
CD! registration tool to al low administrators to use old Phase IV node intormation when popu lating the node names and addresses tor DECnet/OSI. The
Phase
IV database is not supported for node name-to add ress lookup by the CD! library.
Due to its lack of a remote update capability, the
Domain Name System is not supported by the CDI registration tool. Node n:tm e to-address information in the Domain Name System is managed using its native tools. Dynamic updating of the Domain Name
System servers is cu rrently under study by the I nternet
Engineering Task Force ( I ETF ) Domain Name
System Working Group.
Application Design
The design of the CDI registration tool uses a client based , multilayer approach. It is layered on top of a specia lized API , cal led the Common Directory
Registration (CDR) API . The CDR API di ffers from the API provided by the CD! library in that it presents a fu l l set of management operations, rather than just the lookup operations req ui red by DECnet/OS I .
I n this design, the CDI registration tool provides r()fmS and console user i nterfaces for node i n forma tion managemen t. It also provides functions beyond the basic ones provided by the CDR API, such as exporting from and importing to a di rectory service.
The function of the CDR API is to perform all under lying node name management operations i n a stan dardi zed manner. This layered approach was adopted to make node name management functions avai lable to applications other than the CDI registration tool.
The CDR API defi nes a node definition object. This contains all the intonnation that is exchanged between the CDR API and the application and is a canonical , directory-service-independent data representation of all information needed by the CDR API to manage node names and addresses.
To provide an extensible mechanisrn tor adding new directory services, the CDR AP I is layered on top of a set of d irectory service wrapper routines, one per supported directory service. Access to these wrapper rou ti nes is provided by a set of entry poi nt tables that can be extended to support new d irectory services.
The CDR API is responsible for accepting application requests and dispatching them to the correct directory service by means of the appropriate wr:tpper routine.
The CDR API wrapper routines are descr i bed later in this section .
Figure 3 shows the design of the CD! registration tool and the CDR API .
CD/ Registration Tool User Interface
The forms and the console user i nterbccs had to present exactly the same characteristics on both the
OpenVMS and the Digi tal UNIX operating systems.
Because no high-level software packages at the time could provide this level of user i nterface portability, we developed them for this application.
The console user i nterrace parses commands and dispatches them to the appropriate user req uest pro cessing routine, using a portable com mand p:�rser.
The forms user interface obtains input from task specific forms and dispatches the fu nction or ti.mctions associ ated with the form to the appropriate user request processing routine . The forms processor was written specifical ly for this appl ication because no existing libraries could provide the requi red level ofportability.
CD/ Registration Tool User Request Processing
Each user request maps into a specific request process i ng function as follows:
•
•
•
•
•
•
•
•
Register. Create a new node name entry in the directory service.
Add address . Add address intormation ro a node name en try.
Remove address. Remove add ress information tfom a node name entry.
Mod i f)' address. Replace the address information in a node name entry.
Update add ress. Replace the add ress information in one or more node name entries, using information obtained from the nodes themselves ( i f possi ble ) .
Modit)' synonym. Replace the node svnonym in a node name entry.
Rename. Change the name of a node n:�me emry.
Show. Display the information contained in one or more node name entries.
Digiral Tcchnictl journ.tl
Vol. 8
No. 1 1996
63
64 g
CDI REGI STRATION TOOL
I
FORMS USER I NTERFACE
FORMS DEFINITIONS
SHOW
PROCESSING
C D R API CALLS
I
REQUEST DISPATCHER
I
CO
�
SOLE
USER I NTERFACE
.
.
.
REGISTER
PROCESSING
CDR API CALLS
I
I
DELETE
PROCESSING
CDR API CALLS
I I
CDR API CALLS
CDR API
I r - - - - - - - - - -
1
I � - - - - - - - - - - - � - - - - - - - - - - J
I I
I I
I
ENTRY POINT ENTRY POINT
I
ENTRY POINT
SHOW
PROC ESSING
WRAPPER CALLS
REGISTER
PROCESSING
WRAPPER CALLS
D ELETE
PROCESSING
WRAPPER CALLS
ENTRY POINT
. . .
WRAPPER CALLS
DIR ECTORY SERVICE WRAPPER ROUTINE DISPATCHER
DECDNS
WRAPPER ROUTINES
I
I
LOCAL FILE
WRAPPER ROUTINES
I
I
I
PHASE IV DATABASE
WRAP P E R ROUTINES
I
I�
I
DECDNS
LOCAL FILE
D E C N ET PHASE
IV DATABASE
Figure 3
Blo..:k Diagram of the CDI Registr:�tion Tool ;md the C D R API
•
•
Dcregister. Delete one or more n ode name en tries by n
:� me, synonym, or address.
Re pai r
.
Fix any detected pr oblems or in..:onsistcn
..:ics i n the di re c t ory service tor one or more node name en tries.
•
•
Export.
Copy the information r(Jr one or more node
I U I Tle e ntries fl'Oill the d irCCtoJ'\' scn·i..:c into a teXt fl Je th;H can be copied between svsrcms, edited i f neccs san·, a nd i mported i nto am· other director\' scn·ice.
Import. Usc Jn export text ti.Je to register, mo d i
!'\
·
, or d c rcgistcr node name entries in a d irccton' service.
The request processi ng routines perr(mn ;11w required l
'
:tlid:tti on of the user req uest and tr
:t ns requests to cal ls
L ue those
into t
he CDR A P I .
Each request may map i nto one or more CDR API calls, depending on the complexity of the request. For example, register and dcre gistcr requests both map i nto single CDR APT c:tlls, and export and i mport requests m:�p i nr o several calls
.
i\1{ost req uests are straightrc)fll':trd i n their pro cessing rcqu i remcllts . For example,
:1 register req u est sim ph· calls the C D R A P I r egister enrrv poin t. The
C D R API t:t kes care oL:uw complic:ttions in processi ng the request.
Some requests can operate over m u l tjple node n�une e ntries
.
For cx::un pl e, the show request en u merates the node name entries, retrieves the i n rormation con tai ned in each node name e ntry, and displays the infor mation ro the user.
An export req ues t is s i mi la r to a show request, except that the resu lti ng i n ri:m11:1tion is written ro a tex t file i n a stand
: ml rom1<1t i nste a d of be ing disph
1
\
' e d to the ustr. The i mpor t request, however, is more complicated . This req uest must en umerate and show the c o nte nts of the d i recron
• service, and t hen com pare the re sults with the conrenrs of the tex t fiJe . Based on the sp eci tlc form of the im port request, it mav then re gister new node n:�nK entries, update the inr(>rma tion in exist i ng node name entries, or deregister listed node name emries.
The export and import requests make use of
a
text file to provide maxi m u m tlc.x ibiliry.
T
he use of a text r i le a l l ows the inform:�tion to be copied between dis similar pl
a
rtcmm such as the OpenVMS and the
D
i gital
UNIX
operating svstems, and allows the i n r(>r ma tion to be manipu lated usi ng standard tools such as batch riles, grep, awk, �111d text ed itors . This is parricu larlv usdi.d when applving a ch,mge
to all node
e
ntr
ies
.
Vul .
R No. 1 1996
For example, the contents of a directory service coul d be exported to a text file, the addresses i n the te xt file ch:111ged to rdlect a new routing area, and the results imported back into the di rectory to update the exist ing information .
The repair fu nction pertorms a show operation on all specified node names to determine if any consis tency errors arc found. This type of error can occur i n directory services that keep mu ltiple physical records tor each logical node name entry. D ECdns is one e x:1mple of this kind of directory service, because i t uses soft: links to map node synonyms and addresses b;Kk to their respective node name entries. If this type of error is found, the repair function re -registers the node S\'nonvm and address information to correct these inconsistencies.
The most complicated req uest is the update request.
This perfcmllS a show request tor the specitied node names <llld attempts to use the cu rrent add ressing int(xmation contained in the node name entry to make a network management connection to the node itsel f.
For each node name entry, it steps through the com pl ete set of registered addresses and tries each add ress in turn, using both a DECnet Phase IV connect and a
DECnetjOSI connect. I f a connect attempt is success fu l , it uses the appropriate network management requests to read the true addressing data. It then com pares this <Kidressing data to what it found in the direc tor\' service and makes an�r necessary corrections to the node n:�me entrv. The update operation does not oper ate on I P :�dd resses due to the lack of dynamic update c:tpabilities in the Domain Name System servers.
Bdore m:tking the CDR API caJ is, al l request process ing routines convert the user request dat:t into a node ckfinition object, which is discussed in the next section.
CDR API Node Definition Object
The node defi n ition object is the only input data pro vided to any of the CDR API entry poi nts. It stores the neccss<lrv data for any di rectory service operation, using a canonical representation . The node defin ition object contains the fol lowi ng: l .
Type of directory service to access
2.
N:tme of the node entry to access (depending on the oper:�rjon being pertormed, it nuy aJiow a fu lly qual ified mme, J synonym, an address, or wildcards)
3. Synonym name ( f(x DECnet Phase IV Jccess)
4. D ECnet l'h:tse IV network service access point
( NSAP) prefix ( tor use when a Phase IV add ress is specitied )
5 .
Add ress information
6. Di rectory names used tor reverse mapping of wnonym names and addresses back to the tli l ly qualified node name
The CDR API controls al l access to elements within the node definition object, which fu rther isolates the call i ng appl ication from the lower-level data structures.
CDR API Entry Points
Each CDR API entry point provides one logical fimc tion to the cal l i ng application . Each user request can translate i nto one or more CDR API functions. The functions are
•
•
•
•
•
•
•
•
•
Register. Create a new node name entry in the directory service.
Add address. Add address i n formation to a node name entry.
Remove address . Remove address information from a node name entry.
Mod i f)r address. Replace the address information i n a node name entry.
Modify synonym . Replace the node synonym in a node name entry.
Rename. Change the name of a node name entry.
Show. Return the information contained in one or more node name entries.
Deregister. Delete one or more node name entries by name, synonym, or address.
Enumerate. Return a series of node name entries, one at a time, based on a wildcard specification .
A l l node information passed ro a n d from the CDR
API is in the torm of a node definition object, as described previously. The CDR API functions validate the canonical information contained i n the node defi nition object and dispatch a d irectory-service-specitic function to handle the request.
CDR API Directory Service Wrapper Routines
Each directory service su pported by the CDR API has an associated set of directory service management wrapper routi nes. These routines provide entry points that are fu nctionally identical to those provided by the CDR API . The CDR API does the i nitial i nput argu ment validation, and the di rectory service wrap per routines perform the data manipulation in the u nderlying directory service.
The CDR API dispatches the appropriate directory service wrapper routine using a set of entry point tables.
This provides a means to easily extend the CDR API to include additional directory services in future versions.
CDR API Wrapper Routines for DECdns
In the DECdns name service, each node name entry contains all the information required to translate a node name to a synonvm or a set of node addresses.
However, no search mechanism exists to allow a
Digiral Tech nical )ou m;\l
Vol . 8 No. I 1 996 65
66 lookup of the node name entry based on the synonym or on an address . For this reason, all fu nctions that cre ate, modi!}', and delete node name entries ( register, modi!}' addresses, modifY synonym, re name, and deregister) must also create, modi f)', and delete reverse mapping entries.
Reverse mapping entries are based on a node's syn onym and addresses; they contain pointers to the true node name entry. These entries are used by the CDI l ibrary lookup functions and by the C D R API display fu nctions (show and enu merate) to Jccess the node name entry when given a synonym or address.
The use of reverse mapping entries req uires that m u l tiple directory service entries be created tor each registered node. These must be synchronized by prop erly ordering the creation and deletion of the various en tries when registering, modit),ing, or dcregisteri ng a node name. For example, when registering, the node name entry is created and its synonym and �1ddress val ues arc set bdore the reverse mapping e ntries are cre ated and set. Similarly, when deregistering, the reverse mapping entries are deleted bdore the node name entry is deleted . This prevents orphaned reverse map ping entries from being created , because they can always be found by starring from the intornurion con tained in the node name entry.
The repair ri.mction is provided i n case a register or deregister operation tai ls before completion. The repair rtiiKtion corrects the reverse mapping entries by re-registering all node name entries that show errors.
The CDI registration tool ( nor the
C D R AP I ) provides this h igher- level function .
CDR API Wrapper Routines for the Local Node File
Under the Open VMS operating system , the local node name tile is implemented using a record management system (1Uv1S ) -i ndexed fi l e . Under the Digital UNIX operating system, a DBM-indexed tile is used. On both systems, the file conten t is essentially the same.
The local node name tile contains J series of logical records, one for each node name e ntry in the directory service . Together, these records ddine each node's fu lly quali�ied name, i ts synonym, and irs add resses.
This logical record may be looked up using the ful l name, the synonym, or any o f the node's add resses.
Each logical record consists of
( 1 )
a node definition physical record , which contains all information related to the node, and
( 2)
zero or more reverse mapping physical records, which contain alternate keys for look ing up the node definition. Each reverse mapping record contains only the node name key in i ts record data. All the data used to describe the node is con rained i n the node definition record .
Because mu ltiple records compose a node name entry, operations that fail to complete can resu lt in inconsistencies in the local node file. Fortunately, these inconsistencies can be resolved usi ng the same synchronization algorithms as used for DECd ns.
CDR API Wrapper Routines for the DECnet Phase IV
Node Database
Access to the DECnet Phase IV node database is pro vided primarily to help users migrate their Phase IV node n:�me d ata to DECnet/OSI . No access is pro vided to this database by the CD! library for
D ECnet/OSI applications. Because this database con sists of a simple file, with one record per node name entry, none of the multi ple record synchronization problems exist.
Conclusion
The Common Directory I nterface, consisting of the CDI registration tool set and the CD! library, pro vides flexible and extensible d irectory service access for DECnet/OSI . Initial customer acceptance of these new capabilities has been high and futu re enhance ments are being studied .
Acknowledgments
The design and development of the Common
D i rectory In terface involved the contri butions of the entire directory services and DECnet engineering teams. vVe extend our thanks to a l l the team members, as well as to product and e ngineering management tor supporting this proj ect.
References
I . S. Marri n, J . J'v!cCann, and D. Oran, "Developmenr of the
VA.'<.
Distributed Name Service," Digital Techn ical
Journal. vol . l , no.
9
( J u ne
1 989): 9-1 5 .
2 . P. Mockaperris, "Doma in Names-Impl ementation and
Specir!cation,"
RFC 1 035,
Internet Docu ment ( N ovem ber
1 987).
3 . CC ITT Sixth Plenary Assembly, "The Directory
Ove rview of Concepts, Models and Services," Recom mendation
X.500 and
ISO 9'>94- 7.
Data
Com rn u n icarions dations
Networks
Direct01y. Recommen
X.500 to X. .52 7, CCITr
Blue Book, vol . xiii.R
( Geneva: l ntemationaJ Telecommu nications U n ion ,
1 989).
4.
R. Rosenbaum, " Using the Domain Na me System to
Store Arbitrary String Attributes," RFC 1 464, I nternet
Documenr ( M av 1 993).
5 . B. Manning and R. Colella, "The Dom.ain Name System
NSAP Resource Records,"
RFC 1 706,
Internet
Docu ment ( October
1994 ).
Vol . 8 No. J ! 996
Biographies
Richard L. Rosenbaum
Rich Rosm boum is a software engineering consultant in the l mnnet Software Busi ness Unit, where he is tocusing on the appl ic�tion of indexi ng and collaboration tt:chnolo gies to the World Wide Web. In his 1 7 ye<lrs with Digital, he has worked on networking products oper<�ting on
Digital's 1 6- , 32-, 36-, and 64-bit plartorrns. He is the
<:o-�uthor of several pare n rs on network software . Rich obtai ned ;.�
B.S. ti· o rn the State Un iversity of New York
<lt Stony Brook.
Stanley I. Goldfarb
Sr;m Gold brb is a princi pal software engineer with the
I nrn ncr Software Bu siness Unit. Since joining Digital in 1 976, he has contri buted to several network and ncr work ma1L1gement projects, including DECnet/RSX,
DECnct- PRO, DECner- DOS,
DECmcc,
DECnct/OSl , and PATH WORKS, and he has co-authored several parenrs on nt·rwork managemenr softwar e . He is cu rrently work ing on a Workgroup Web Foru m application to provide elccrmnic mail su bscription and disrri bmion services.
Sr�n holds B.S. �nd M.S. degrees in com purer science ti·om
Worcester Polytech n ic Institute and an M.S. i n managemenr ti-om Lesley CoUcge.
Digital Technical Journal
Vol. 8 No.
1 1996
67
I
Recent Digital
U.S. Patents
The td lowi ng patents were recently issued to Digital
Eq uipment Corporatjon. Titles :md names supplied to us by the U.S. Parent and Trademark
Oftice
;l re reprod uced exactly as thev appe:1r on the original published patent.
1)3 3 6 ,08 1
) ) 3 3 6 ,082
D336,290
1)336,636
1 ) 3 3 7,7 6 1
1)338 ,00 1
( ) 3 3 8 , 6 5 3
D :l 39 , .3 2 5
1)340 ,035
()342 , 5 2 3
D344,7 1 0
1!346 ,370
D347,624
1)341\,448
1!341\ ,672
1 ) 3 5 0 ,34 1
5 ,209,389
5 ,2 1 0,795
5 ,2 1 2 ,776
S.
S.
K.
1
Yi organ and M .
L.
Hcrticld
K.
Morg
<
ln ;l nd M .
L. Hc r ti cld
S . K.
M organ and i'vl.
L. Hertield
R . Ve no,
K.
Pal umbo , l'.
R.o;K h, P. B;HTon , and M . Freeman
S.
K.
Morgan and
M. l.. Herticld
M . Fal kner, M. Good, ;md M . \Viesenh;l h n
S.
K .
M organ :md M .
L.
He rticld
L.
Spencer and C:. Dcrsikas
R. 1-'aranda
S.
K.
Morgan and M.
L. H c r ti el d
S. K . Mo1·gan <lnd
M . L.
He t!ield
M .
L. Herticld and S .
K.
1'vl org;;m
W. McCarrby
M. Freeman
, R. Hcllm:g, R. i\t!Jsters,
,
C.
Will i:uns, C : . Brench,
K.
Palumbo, D . Snow, an d 1'.
Barron
C. E. Vai lbnr ,
} .
D. Read, ,md G . ) . Norqw1y
M. J. �a\ kner and M. W. Klcem;lll
C . Landry
K.
Sull ivan
Jnd P. C1ine
S. L i p ne r, M. Gasser, ;md B. W. Lampson
M. Kinden'ater ;md
�.
Zomdveld
5 ,2 1 4 ,963
5 ,2 1 5 ,608
5 ,2 1 6 ,655
5 ,2 1 6,672
5 , 2 1 8 , 5 1 3
5 , 2 2 0 , 2 7 1
5 , 2 2 3 ,7 1 0
5 , 2 2 3 ,806
5 ,2 2 3 ,996
5 ,2 2 4 , 2 3 5
D. Widder
R..
Strou d and
K. V on br an d t
P. Hearn ,
A.
P rent
; lki s
,
W. Le\\'is, and
F. Zayas
P. M .
Goodwin, D. W. Smelser, and
D. A. Tatosian
D. Brown
J. P;l\czynski
R. Pavlak
R . Curtis a nd D. Skcndzic
C. E.
Va i\L�nr and
).
D .
Rod
P.
Lison and W. !h ines
Electronic Device Moduk
Electroni c
Device Mod u l e
Enclosure f(>r Electronic Mod ule
Power Supply Door
Electronic Devi<:<: Module
Positioning Device
Power Supplv Module
Face Pi<lt<:
Ce n tral Processing Unit Enclosure
Cover f()r VVd ll -mounted Elccrmnic Equipment
Electronic Device Module
Network M u lti pkxm tc>r an
Ofticc Em·i ronment
CJrd Cage Enc losu re
Removable Rigid Disk Drive
Desktop Audio Enclosure
Display Monitor
Solder Pump Rushing
ScJI
Secure User Aurhenricarion ti-om Personal Comp uter
Computn System Compri s in g a Main Bus and an
A dd itional
ComnHI IJicuion
Means Directlv Connected between Proccssm ;lnd Main Memory
M erhod and App<lLltliS tc>r
Testing I n ner Lead Bonds
Co mposition �l lld M e thod tell'
B ond i ng Electrical
Compo ne nt s
Method and Appar.1 ru s t(>r Surbce Real locarion for
Improved M;lnu!:Kturing
Pmccss
Margin
Parallel Diagnostic Mode t<>r Test i ng Computer Mcmo r v
Plenum tor Air-impi ngement Cooling of Electroni c
Components
Cross Regu btor t(>r
J M u ltiple Output Power Supply
Optical Angular Position Sensing System for Use with
J
Galvanometer
Method ;md Appar;ltus t(>r Reducing Elecrrom<lgneric l nrertCrence and Emission Associated wit h
C omp uter
Ne twork lntert:lCcs
Com bi ned Shock Mount F1·a mc and Seal for a Ri gid
Disk Drive
E le c tr onic Component Cleani n g Apparatus
68 Dig;it;ll Technical Journal
Vol .
8
No. I 1 996
5 , 224,263
5 , 2 2 5 ,790
5,22 6,092
5 , 2 2 7,04 1
5 , 2 2 7,582
5 ,2 2 7,604
5 , 2 2 8 , 066
5 , 2 29,90 1
5 , 229,9 1 4
5,2 29,926
5 , 2 3 1 ,246
5 , 2 3 2 ,570
5 , 2 3 5 ,6 1 7
5 ,2 3 5 , 642
5 ,243,308
5 , 243,495
5 ,243,756
5 ,247,4 2 6
5 ,248,2 5 3
5 ,2 5 1 , 3 1 6
5 , 2 5 4,930
5 , 2 5 5 ,2 R7
5 , 2 5 5 , 375
5 ,2 56,060
5 ,2 5 6 ,9 75
5,260,864
5 ,260,928
5 ,260,945
5 ,2 60,999
5 , 26 1 ,002
5 , 2 39,260
5 , 2 39, 2 74
5,24 0,549
5,24 1 ,632
5 ,24 1 ,639
5 ,2 6 3 ,030
5 ,263,032
5 ,2 6 5 ,2 1 2
5,265,2 1 6
5 ,266,] 56
W. H;tmburgen
R . Noguchi , ) . Ri naldis, and P. Es1ing
K . Chen
B. Brogden,
L.
B rown
, and S. Husain
D. ). Vel:tsco,
).
P. Copebnd, D. C. Robinson, and R .
L. t:niLtnd..:z
C. l:'reedman
C:. Del'ane
1\1! . ,\'LJII:t r v
D . B3iln·
D. D. Donaldson ;md D. Wissell
) . Benson, D. Alcss3 ndri n i , Jnd W. Rerr
'vV. Haines, R . Ray mond, C. Bvu n,
E. J oh ns ,
D. R3"ip3ti,
Q .
Ng, 3nd G . Rauch
W. tvbll:trd
E. Wo bber, M. Abad i , A. Birrell, and
B. W. Lampson
D. Widder and D. Ringleb
K. Chi
W. Ha mb ur g cn 3nd ). Fitc h
T. Creed on, D. Smirl1, and A . O'Connell r:. h
:
J d bru gge
R . Curris 3nd B. Sh usterman
C.
E.
Va ilbnr,
W. Ha j .
A. D;liV
D. C:. Dal'ies, D. G . Von a t b , and R . A. Curris
N . Crook, P. B ruce, ;t nd R. Cal uszb j. m bu rge
Simone l l i 3nd Z. Arbanas
A. Jai n , N . Lee, and E. Keppclcr
T. I ..
R odehdh
: r
R . Wyman
B . E. William
) n
.
D. Read, 3nd G. Norquay and ) . Fitch
W Hamhu1·gcn and ) . Fitch
A. Philij)Ossian and E. Cutin·
P.
Anick
;\IH.i R..
Hynn
A. Philipossi;l ll and
R . Mellitz and
E. Cu l l ev
E.
Stearns
R
.)
. Perl man ;l!H.I C : . W. K:
.
1Lifi11 a n
P.
S. Rotkcr, ;md E.
W.
Ertd
B. Porter, C : . A. Meg3, 3nd R. L. M y er s
C. P. Murph y, T. Creedon, and C. D. Cremin
A. Nasr
Gentle Package E x traction Tool ;t nd M ethod
Tu nable
Wideband Active Fi lter
Method and Apparatus for Learning in
;1
Neural Nerwork
Drv Contact Electroplating Appar;Hus
Video Ampli fi er Asse mblv Mounr
Atmospheric Pressure G3scous-rl u x -assisted L
1 sc r
Rcflo"' Solderi ng
System and Method ror Measuring Computer Svste m
Time lnren·a!s
Sid e-bv-Side Readj\Vri tc Heads with RoLlrv Positioner
Cool ing Device that Cre;Hcs Longitudinal Vortices
Power Supplv Interlock for Distri buted Power Svstems
Appa ratus fo1· Sec u ring S h ielding or the Like
Nitrogen-containin g M a t e rials for We;Jr Protection and friction Redu ction
Transm ission Media Dri\·ing System
Access Control Su bsystem and Method f(>r Distri buted
Com puter Syste m
Usi ng Locally Cached Authen ticnion
Credentials
Semiconductor Probe and Alignmem System
Vol tage -controlled Ring Oscillator Using Complementary
Diffe rential Bu ffers tor Generati ng Multiple Phase Signals
Fixture and Method for Attaching Compo nents
P r og r a mm ab le Priori ty
A r bi ter
Method r or Updating Mod i fied Data ti·om a Cache
Add ress Location ro lvbin Memory and M3int3ining the Cache Add ress in Regi stration Me mory
Combined Di fferential- mode 3nd Com mon-mode
Noise Filter
Removable Enclosure Housing
;1
Rigid Disk Drive
Integrated Circuit Protection by Liq uid Encapsulation
Semiconductor H eat Remm·al A pp a r a t u s with
Non un irorm Condu ct
< mcc
Thermal Processing furnace with I mproved Plug How
Met hod and App a ratus r o r I nteg r atin g 3 Dvnamic Lexicon into a Full-text I nforma t i o n Rctriel'al Svstcm
Fault Detector tor a
Pl u rJ i i rv or. lhm:ries in Barren'
Backup Svstems
Transcei\
"
CI
"
Appara tus and Methods
H i gh Performance I nrerbcc between an Asvnchronous
Bus and One or More Processors or the Li ke
Red ucing Gas Recircu lation in Thcrn1<11 Processing Furmce
M<�nually Opnated Conti nuity/Shons Test Probe rc >r Bare
1 nrerconncction Packag e s
Conrigur<t ble In verter ror 1 2 0 VAC or 240 VAC Output
Apparatus and Method r(x FJbricating a Lens/Mirror Tower
I nterm ittent Component F
< li l ur e
MJnager and M ethod f(>r
M i n i mizing Disruption of Distributed Co m p u ter System
Filters in License Man;1gement System
Method of lssua ncc and Revoc3tion ofCe rr i ricues of Authenticity Used in P u b l ic Key Networks 3nd
Other Systems
Method and Apparatus for Encoding D3ra r(>r Storage on Magnetic Tape
Computer System Operation with Corrected Read
Data Fu nction
Sharing of Bus Access 3mong M u l tiple Stare Machines with
Mi nimal Wait Time <l nd Priori tization of Like Cycle Types
High-performance Asynchronous Bus 1 merrace
Methods oft:orming a Loc:tl [merconnccr ;md 3 High
Resistor Polvsi licon L.oad Iw Reac t ing Cob;Jit with P olysilicon
DigitJI
Tcdmic.1l journal
Vol . ll No. I 1 996
69
5 ,267, 1 1 2
5 ,267, 1 99
5 ,267,235
5 ,267,237
5 ,267,867
5 ,268,837
5 ,268,962
5 ,269 ,0 1 3
5 ,272 ,390
5 ,272 ,445
5 ,273,455
5 ,274,2 10
5 ,274,509
5 ,274,628
5 ,276,569
5 ,276,872
5 ,278,703
5 ,278,783
5 ,279, 865
5 ,280,437
5 ,280,608
5 ,28 1 ,869
5 ,283,560
5 ,285,007
5 ,286,9 1 9
5,287,263
5 ,287,359
5 ,2 87,500
5 ,287,5 0 1
5 ,287,5 1 7
5 ,289,046
5 ,289,328
5 ,289,347
5 ,29 1 ,529
5 ,293,486
5,293,487
5 ,294,842
S. Batra, S. Ramaswamy, and M . Mallary
R. J. Galuszka, A . ]. Walton, and C. Choi
C.
P.
Thacker
A .
T.
Townley
F. AghadeJ and C. W. Ho
M. Saylor
M. Abadi, M . Burrows, and 13. W. Lampson
K .
D. Abramson, H. B. B u tts, and
D . A . Orbits
H . A. Col l i ns, R . B. vVatson, and R . l kn <lian
S . G. Lloyd and H. Partovi
L. MacLellan
G . Freed m::w , P. Elmgren , and M. Brodeur
B . D. Buch
N . D Godiwala and K. M. Thaller
W. F. Even
D. B. Lomet and B. J. Salzberg
B . R u b
,
).
E. and R . Frame
Deroo, S. B.
]. Edmondson
R . P. Chebi and S. Mitral
D. A. Corliss
Skralv,
A . ) . Beverson, T. E. H u nr, and
G . P. Lid ington
]. R . Lundberg
J. F. Barrlett
A. SoJ ii,
A. E. Deluca,
T.
j.
M. Lewis, C. L. Leo,
)
.
Orr, D. T. Symmes, and R . A. Barker
]. W. Benson and D. T. Staffiere
1\tl. Shilo
W. Engelse
P.
Stoppani
D. B. Lomet
B . A. Maskas, J. A. Metzger, and G. ]. Harris
). A. Daly, ]. M . Gregorich, and G . ]. Br<lnd
G . Saliba
W. F. McCarthy, D. M. Snow, and
C. E. Brench
N.
A. Crook, P. L. Bruce, and R.
I.
Gal uszka
M . A. Jord an and D. ]. Don nelly
A. P. Russo, S. L. Rege, M.
F.
Kempf, <lnd
E.
T. S u l l ivan
R. 1 k.naian and R. B . Watson
Thin Film Read/Write Head for Minim izing Erase fri nging and Method of Making the Same
Apparatus for Sim u l taneous Write Access to a Single
Bit Memory
Method and Apparatus tor Resource Arbitration
Collision Detection and Signaling Circuit
Package for Mu ltiple Removable I ntegrated Circu its
Robotics Workstation
Computer Network with Modi tied Host-to- Host
Encryption Keys
Adaptive Memory Mano1gemenr Method for Coupled
!\'\emory M u l tiprocessor Systems
Method and Apparatus for Clock Skew Reduction through
Absolute Delay Regu lation
Resistance Tester Utilizing Regu lator Circuits
Torsion Bar Connecto r
L aser Bonding H ighly Rctb:tive Surtaces
On -thc-tly Splitting of Disk Data
131ocks
Usi ng Timed
Sampling of a Data Position I ndicator
Multisignal Synchroni zer with Shared Last Stage
Spind lc Control ler with Startup Correction of Disk Position
Concu rrency and Rcco,·crv for I ndex Trees with Nodal
Updates Using Mu ltiple Atomic Actions bv Which the Trees l n tegri tv is Preserved during Undesired
System
I ntcrruptions
Embedded Servo Banded Format t(>r Magnetic Disks f(ll·
Usc with a D::tta Processing Svstem
Fast Arc;J - Efficicnr M u l ti - bit Bi nary Adder with Loll' h111-our Signals
High Throughput
I nrerlcvel Dielectric Cap Fill ing Process
Structure ;md Method tc>r Direct Cali bration o r·
Regisn·ation Mcasmemem Systems to Actual
Semiconductor- Wafer Process Topographv
Program mable Sta l l Cycles
Reduced Voltage N M OS Output Driver
Computer System and Method ti:Jr Displaying I m ages with
Superimposed Parri:dlv Tra nsparent Menus
System tor Red ucing the Emission of High 1-"rcquency
Electromagnetic Waves from Computer Systems
Computer Cable Management System
I nrush C:urreilr Control Circuit
Syn chronous Decoder tor Self-clocking Signals
System for Allocating Storage Spaces R::tsed upon Required and Optional
Service
Attributes Having Assigned Priorities
Multilevel Transaction Recovery in a Database System
Which Loss Parent Transaction Undo Operation upon
Commit of Child Transaction
Self-compensating Voltage Level Shifting Circuit
Power Converter with Controller tor Switching between
Primary ;m d Battery Power Sources
Method and Apparanrs l·(>r V;lriable Density R..ead -after writing on Magnetic Tape
Enclosure t(>r Electronic Modules
Synch ronization Scheme
Dctcrrninistic Method for Allocation of;1 Shared Resource
Network Ad;lpter with H igh Tlmlllghput Data Tra nsfer
Circu it to
Optimize Network Data Tr;msrcrs, with Host
Receive Ring Resource Monitoring ;md Reporting
PVT Update Synchronizer
70
Digital
Technical Journal Vol . 8 No. l 1 996
5,30 I , 1 8 6
5,30 1 ,283
5,30 1 ,320
5,3 0 1 ,325
5 ,302,960
5 ,303,302
5 , 303,347
5,304,939
5,305, 1 85
5 ,305 ,354
5,305,389
5,306,994
5 ,307,2 1 7
5 ,307,256
5 ,307,336
5,3 07,345
5, 307,492
5, 308,429
5,309,035
5,309,294
5 ,309,45 1
5,309 ,569
5 ,3 1 1 ,08 1
5,294,994
5,297, 107
5,297,29 1
5 ,297,992
5 ,299,206
5,3 1 3,369
5 , 3 1 3,50 l
5,3 1 3,577
5,3 1 3 ,595
5,3 1 5 ,597
D. C. Robinson, ) P. Copeland, D .
}.
Velasco,
R. L. Fernandez, and S. D. Vendirri
J. A. Metzger and P. j. Gr�lffam
D. L. M u rphy
D. Bai ley, P. Martino, and B. Arsenau l t
A. Bcwcrson and C. J . Devane
R. Galuszk<l, A. vValron, and S. Bryant
C : . P. Thacker and D. Ha rtwell
P. J. Cerqua, S. M. Kennedy, J. D . McAtee, and P.J . Piccolomini
T R. Benson l nregrated Computer Asse mbly lnrerconnect Arrange ment tor Electronic Components
D isposed on a
Ci rcuit Board
System for Linking Program Unirs by Binding Symbol
Vecror I ndex in rhe Symbol Table inro Calling I mage to Obtain Cu rrent Va lue of the Target I mage
Method and Apparatus for Liquid Spill Contain menr
A General Process for Finding Patterns in Large Logic
Traces (or Other Large Bi nary Am1ys) Using Multipl e
Concurrent Finite Automata with Cross-communication
H igh Speed Transmission Line I n terlace
Dynamic Arbitration for System Bus C:onrrol i n
M u lr.iprocessor Data Processing System
Workflow Man agement and Conrrol System
P. Roers
M. Bu rrows
S. L. Rege and D. A. Gagne
D. C. Davies
V Samarov, ). DeCarolis, R.. Patel, G. Piche,
G. Skurr, and S. Norris
N. D. Godiwala and K. M. Thaller
M. L. Palmer
Use ofSrack Depth to Identify Arch itecture and Ca l ling
Standard Dependencies in Machine Code
Mu lti -element Suscepti bi lity Room
Network Packet Receiver with Bu ffer Logic fo r
Reassembling Interleaved Data Packers
Attri bute Based M ultiple Data Structu res in Hosr tor
Network Received Traftic
Tracking Peak Detector
Coplanar Heatsink and Electronics Assem bly
L. Supino
G. A. Sa l i b''
R . Si lverstein
Aborting Synchronizer
Predictive Cache fo r I m proved Pedornunce in Retri eving
Cached Data
Automatic P hase Margi n Compensation Control Circuit and Method tor Disk Drives
M agnetic Head for Very High Track Density Magnetic
Recording
Trickle Charge Circuit fo r an O ff l ine Switching
Power Supply
"'lulti-disk Optical Storage System N. K. Lee , A. Jain,
E.
Kcppeler, and
M. Bouchard
S.-T Ben-Michael
:mel
P. Lozowick
T. R . Benson
S. j . Brad lcy
H. Collins, R. . Warson , and R . I knaian
D. Cahalan
E.
S . Noya, M. N. R.osich, and
R. .
M. Arnott
N. A. W�1rchol
D. D . Donaldson , R. A. Dame, and
R . E. Nikel
Method and Apparatus tor Cur-th rough D<Ha Packer
Transkr in a B ridge Device
Mapping Assembly Language Argu ment List Rdcrcnces in Translating Code for Difterent Machine Archi tectures
System for Bonding a H eatsink to a Se miconductor
Chip Package
Method and Apparatus tor Clock Skew Reduction through
Absol ute Delay Regulation
Method and Circu itry ro Provide True Voltage Rias to a M agneroresistive H ead
Dara and Parity Prefetching for Red uncLmr Arrays of
Disk Drives
Sel f-con figuring Bus Termination Component
Data Bus Using Open Drain Drivers and DiHcrential
Receivers Together with Disrributed Termi n�uion
I mpedances
Red uced Tolerance Interconnect System M . Lewis,
L.
Trescel er, R. Marri nez, anel
R . 'T'usler
C. P. Thacker
K. J'vlei ncrth, C. Case, R. Gamache,
B . fanning, and C. Franklin
M. Lewis :mel R. ltwey
W. C. Mal lard and H . S. Yang
Method and Apparatus tor Deskewing D igital Da t�l
Tra nshltion of Vi rtual Addressing in a Computer
G raphics System
Automatic Signal Te rmi 1ution System tor '' Computer Bus
Method and Means for Auromarica lly Detecting and
Correcting a Polarity Error in Twisted-p<l i r Media
Digital Technical Journal
Vol . 8 No. I
1 996 7 1
5 ,32 1 ,703
5,32 1 ,806
5,32 1 ,8 10
5,325,495
5,325,528
5,327,4 1 6
5,327,424
5,327,435
5,329,426
5,330,920
5,33 1 ,496
5,332,487
5,3 33 ,097
5,333,098
5,333 ,260
5,333,262
5,333,3 1 5
5,333,744
5,334,043
5,335,226
5,335,235
5,3 1 5,602
5,3 1 5,696
5 ,3 1 5,698
5,3 1 5 ,707
5,3 1 6,642
5,31 6,965
5 ,3 1 7,527
5,3 1 7,693
5,3 1 9,385
5 , 3 1 9,678
5,3 1 9 ,743
5,3 1 9 ,760
5 , 3 1 9,766
5,3 1 9,785
5,32 1 ,373
5,32 1 ,693
R. M. Arnott, E. S. Noya, and M . N . Rosich
K.
Mei nerth, C. Case, B. Fanni ng, and
J.
I rwin
K. Meinerth, C. Case, ]. I rwin, and B . Fanning
S . Bryant and M. Seaman
D. Young, S. Randall, S. Shaw, and A. Wylde
A. Philipossian , H . Soleimani , and B. Doyle
S. B ritton, R. Allmon, and S. Samudrala
] . -C. E . Cuenod , and P. A. Sichel
F. Fernando
S . H o and N . Darcy
S . Duna, A . Roy, and N. Rao
P. Sroppani and C. Saether
Optimized Stripe Detection for Red undant Arrays of
Disk Drives
Graphics Command Processing Nlethod in a Computer
Graphics System
Method and Apparatus for Varying Com mand Length in a Computer Graphics System
Multiprocessor Buffer System
Oscillation Device for Plating System
Nlethod of Decreasing the Field Oxide Etch Rate in
I solation Technol ogy
Leading One/Zero Bit Detector for Floating Point
Operation
Computer Peripheral Device Network with Peripheral
Address Resetting Capabilities
Quad rant-based Binding of Poi nt.er Device Bu ttons
Clocking System for Asynchronous Operations lntelligenr and Compact Bucketing Method for Region
Queries in Two-di mensional Space
Translation Buffer tor Virtual Machi nes with Add ress
Space Matc h
Dupl icate Tag Store without Valid Indicator
A. H. Mason, P. T. Robinson, R. Witek, and ]. S. Hall
B. A. Maskas,
].
A. Metzger, N . D . Godiwala, and
K.
M. Thaller
K. M . Thaller
R . Curtis and B. S husterman
R . Perlman
Polling of ljO Device Status Comparison Performed in the Polled l/0 Device
Combined Difrerential -mode and Common-mode
Noise F ilter
Multicast Ad dress in a Local Area Network Where the Local Area Network has Inadequate Mu lti cast
Addressing Capability
Data Recovery after Error Correction Failure
Residue Bufter tor Graphics System
L.-J. Weng
K. l'vleinerth, C. Case, ]. I rwin, A. Masucci ,
S. Krishnaswami, and A. Moezzi
K. Meinerth,
C.
Case,
J.
Irwin, and
B. Fanning
E. McLellan
). Klein
L. Neville, A. Jain, and A. L. Gutierrez
R. Perlman
N. Warchol
A. Villani
A. Philipossia n, B. Doyle, and H . Soleimani
S . Batra and A . Wu
D. Young, S. Ra ndall, S. Shaw, and A. Wylde
G . Ch ristensen and J. Marceca
A. E. Deluca, S. W. Stefanick, C. L. Leo,
T. ]. Orr, D. T. Symmes, and H . Wright
R. U lichney
R. Ulichney
Address 1'vlethod tor Computer G raphics System
Red ucing Branch Delay in Pipelined Compu ter System
Distri buted Computation Recovery Management System and Method
Surface Selection .i'vlcchanism for Optical Storage System
Automatically Contiguring Paral lel Bridge Num bers
Method tor Testing a Processor Mod ule in a Com purer
System
Clip-on H eat Sink
Method of Controlling Gate Oxide Thickness in the
Fabrication of Semiconductor Devices
Thin Film Magnetic Transducer with a Multitude of
Magnetic Flux In teractions
Method and Plating Apparatus
Disk Drive Holder and I n terconnection System
Apparatus tor Storing Storage Devices
I m aging System with Mu ltilevel Di thering Using Bit Shifter imaging System with Mu ltilevel Dithering Using Two
Memories
Data Storage System and Method with Device
I ndependent File Directories
Modular Equipment Support System R . -A. Locicero, S. Morgan, M. Romm,
E. Mangan, and M . Banrly
G. Dvorak and L. Wolfe
R . A. Williams
R . M. Arnott
Test Fixture for Electronic Components
Communications System with Re liable Col lision Detection
Method and Apparatus
FIFO: Based Parity Generator (FBPG)
72
Digital Technical journal Vol . 8 No. 1 1 996
Call for Authors from Digital Press
Digital
Press is an imprint of Butterworth- Heinemann, a major international p
ub
l isher of professional books and
a
member of the Reed Elsevier group. Digital
Press is
the
authorized p ublisher for Digital Equipment Corporation : The two companies are working i n partnership to identifY and publ ish new books under the
Digital Press impri n t and create opportu nities for authors to publish their work.
Digital Press is com mitted to publishing high
-
quality books on a wide variety of su bjects. We would like to hear from you if you are writing or thinking about writing a book.
Contact: Mike Cash , Digital Press Manager, or
Liz McCarthy, Assistant Editor
DIGITAL PRESS
3 1 3 Washington Street
Newton,
MA
02 1 5 8- 1 626
U .S.A.
Tel :
( 6 1 7) 928-2649, Fax:
(
6 1 7) 928-2640
E-mai l : Mike.Cash@BHein .rel.co.uk or
LizMc@world .std .com
ISSN 0898-901X
Printed in
U .S.A. EY- U02SE·T)/96 6
14
20.7 Copvright � Digital
Equipment Corporation
Download
Advertisement
Key features
High availability and scalability
Provides a single-system view
Supports large, highly available database systems
Offers advantages over large-scale parallel systems and heterogeneous distributed systems
Integration of multiple directory services
Object modification tools
Clusterwide virtual shared memory
High-performance interconnect
Industry-standard benchmark
Frequently asked questions
The TruCiuster system is a highly available and scalable multicomputer system for the UNIX environment. It provides a single-system view and is managed as a single system. Each cluster node may be a uniprocessor or a multiprocessor system executing the Digital UNIX operating system.
The TruCiuster system offers several advantages over large-scale parallel systems and heterogeneous distributed systems. They provide high availability, scalability, and performance, are built of standard building blocks, and have a low-cost interconnect. They are also more tolerant of failures due to looser coupling and offer better performance and availability compared to single-node systems.
The TruCiuster system is a collection of loosely coupled, general-purpose UNIX servers connected by a high-performance interconnect. It maintains a single security domain and is managed as a single system. Each cluster node may be a uniprocessor or a multiprocessor system executing the Digital UNIX operating system. Key components include the Distributed Remote Disk (DRD), Distributed Lock Manager (DLM), and DECsafe Available Server Environment (ASE).
eXcursion Version 2 is Windows-based X server software that allows the windows of a remote host running UNIX or OpenVMS to display on a desktop running the Microsoft Windows operating system. It has been rewritten to maximize graphics performance and support the full range of Windows platforms: Windows, Windows 95, and Windows NT.
The object modification tools OM and Atom are developed for the Digital UNIX environment (version 4.0) to maximize performance. They help software developers extract maximum performance from the system by providing program analysis and optimization capabilities. They are implemented as shared libraries in the Digital UNIX and OpenVMS environments.
Integrated Directory Services (IDS) is a network directory service that simplifies management by integrating multiple services into one directory-service-independent system. The IDS extensible, object-based framework provides a common interface for network management and reduces the complexity of managing multiple directory services.
Related