Digital TruCluster Version 1.0, OM, Atom, excursion Version 2, Integrated Directory Services, Common Directory Interface, DECsafe Available Server Environment, MEMORY CHANNEL, UNIX Version 4.0, Oracle Universal Server software Technical journal

Editorial
Jane C. Blake, Managing Editor
Helen L. I'arrerson, Editor
Kathleen Ail. Stctson. Editor
Circulation
Catherine M. Phillips, Administrator
Dororhea B. Cassady, Sccrcrar!.
Production
Tcrri Autieri, Production Editor
Anne S. Katzeff, Typographer
Peter R. Woodbury, Illustrator
Advisory Board
Samuel H . Fuller, Chairlnan
lbcliarcl W. Beane
Donald %. Harbcrr
William R. Hawe
llchnrd J . Hollings~\lorth
Willianl A. L i n g
H~ch.lrdF. Lary
Ann G. Nerncrh
Paul~ncA. Ntsr
Robert IM. Supn~k
The Digilal Technicul,/oun~alis a refereeu
journal published quarterly by Digital
Equipment Corporation, 3 0 Porter Road
LJ02/D10, Littleton, ~Massachuserrs01460
Subscriptions to the foutnal are $40.00
(non-U.S. 560) for four issues and $75.00
and must
(non-U.S. % 115 ) for eight jss~~es
be prepaid in U.S. funds. University arid
college professors anci P1i.D. students in
the electrical engineering and cornpurcr
science tields receive complimentary subscriptions upon request. Orders, inquil-ics,
and address changes should bc sent to thc
Iligi~nlTecbr~ical,/o~iri~al
at the publislledby address. Inquiries can also be sent clcctronically to [email protected]. Singic copies
and back issucs arc available For $16.00 each
by calling DECdirect at 1-800-DIGITAL
(1 -800-344-4825).Recent issues of the
,/orrrr7nl are also available 011rlie Internet
at htrp://~n\~\~.digitaIIcom/info/drj.
Conlplete 1)igiral Internet listings can
be obtained by sending an electronic mail
message to [email protected]~n.
Digital enlployees may order subscriptions
through Readers Choice by entering VTX
PROFILE at the systcni prompt.
Comments on the content of any paper
are welcomed and may be sent to the
managing editor at the published-by or
~lerworkaddress.
Copyright Q 1996 Digital Equipment
Corporation. Copying without fee is permitted provided that such copies are made
for use in educational institutions by faculty
members and are not distributed for commercial advantage. Abstracting with credit
of Digital Ecluipment Corporation's authorship is permitted.
Cover Design
The "hot" colors on our cover retlect the
kind of performance delivered by 64-bit
Digital UNIX TruCluster systems. A fournode cluster made up of Alphaserver 8400
5/350 systems interconnected with the
higl-speed MEIMOKYCHANNEL and
running the Oracle Universal Server with
Oracle Parallel Server recently achieved
record TPC-C performance of 30,390
t p n C . The design of the Digital UNIX
TruClustcr system is the opening topic
in rhis issue.
The cover was designed by Lucinda O'Neill
of Digital's Design Group.
The information in the Jozirnal is subject
to change without notice and should not
be construed as a commitment by Digital
Equipment Corporation or by the conipanits herein represented, Digital Equipme~lt
Corporation assumes n o responsibility for
any errors that ma)! appear in the Journnl.
Documentation Number EY-U025E-TJ
Book production was done by Quantic
Con~munications,Inc.
The followit~gan-c tratlcmarks of Digital
Equipment Corporarion: Alph'iSer\.er,
DECnet, DECsafc, Digital, the DIGITAL
logo, excursion, iManagcWORKS, IMSCP,
OpenViMS, PATHWORKS, TruCluster,
and VAXcluster.
Adobe is a regisrercd rradcmark of Adobe
Systems Incorporarcd.
DCE, OSF, and i\.lorit'arr regisrcred
trademarks and Open Sofnvare
Foundation 1s a rradeniark of Open
Sofnvare Foundation, Inc.
He\\~lctt-PackatdIS a trademark of
Hc\vletr-P.~ckardConipnny.
M~malayaand Tallde~ll,\re ~.egistered
tr~dcrnarksofTandcm C o ~ n p ~ ~ t cInc.
rs,
Illrcl 1s a rradcmark of Intel Corpor~rioll.
IMEMO~U'CHANNEL is a trademark of
Encore Computer Corporarion.
Microsoft, Visual C++, Win32, and
Windows 9 5 are rcgistcred trademarks
and Windows, Windows for Workgroups,
and Windows N T are trademarks of
Microsoft Corporation.
Netware and Novell are registered
trademarks of Novell, Inc.
POSIX is a registered trademark of The
Institute of Electrical and Electronics
Engineers, Inc.
Oracle7 is a trademark of Oracle
Corporarion.
S3 is a registered trademark of S3
Incorporated.
Sequent is a trademark ofsequent
Computer Systems, Inc.
SPEC is a registered trademark of the
Standard Performance Evaluation
Corporation.
StreetTalk is a trademark of Banyan
Systems, Inc.
Sun Microsysre~nsis a registered trademark of Sun Microsysrems, Inc.
TPC-C is a trademark of thc Tralisaction
Processing I'crformance Council.
UNIS 1s a registered trade~narkin the
United Stares and other cou~itries,licensed
esclusivcly through X/Open Company Ltd.
X Window System is a trademark ofthe
Massachusetts Institute ofTechnology.
Contents
Foreword
Don Harbcrt
DIGITAL UNlX CLUSTERS
Design of the TruCluster Multicomputer System for
the Digital UNlX Environment
Wayne M. Gal-doza, I.'redc~-ickS. Glover, and
Mlilliarn E. Snarnan, Jr.
OBJECT MODIFICATION TOOLS
Delivering Binary Object Modification Tools for
Program Analysis and Optimization
Linda S. Wilson, Craig A. Ncth, and
Michacl J. Rickabaugh
EXCURSION FOR WINDOWS OPERATING SYSTEMS
Design of excursion Version 2 for Windows,
Windows NT, and Windows 95
John T. Frcitas, Jnmcs G. I'ctcrso~i,Scot A. Aurcnz,
Charles P. Guldcnsclu~h,and Paul J . h n ~ u r o
32
Integrating Multiple Directory Services
Margaret Olson, Laura E. Holly, and Colin Strutt
46
Design of the Common Directory Interface
for DECnetIOSI
Richard L. Roscnbaum and Stanlcy 1. GoldE~rb
59
NETWORK DIRECTORY SERVICES
Editor's
Introduction
Digital recently announced recordbreaking 30,390 t p m C performance
on a Digital UNIX cluster of 64-bit
RISC AlphaScrver systems. I n this
issue, engineers from the UNIX team
describe the key technologies that
enable thcse near supercomputer
performance levels as \vcll as provide
the cluster characteristics of high
availability and scalability. Also presented in this issue are advanced
UNIX programming tools for masimizing performance, X servcr software that supports tlie Microsoft
family of operating systems, and new
network directory serviccs that simplifj management.
First defined by Digital UI the early
1980s, clustcrs are highly available,
scalable multicomputer systems built
with standard parts and offering the
advantages ofsingle-computer systems.
Wayne Cardoza, Fred Glover, and
Sandy Snaman compare clusters with
other types ofmulticoniputer configurations and describe the major components o f Digital's newest cluster
in~plementation,TruClustcr systems,
for the 64-bit UNIX environment.
T h e cluster interconnect, called
MEMORY CHANNEL, is critical t o
the cluster's outstanding performance.
MEIVORY CHANNEL implements
clusterwide virtual shared memory
and reduces overhead and latency by
two to three orders of magnitude
over conventional interconnects.
AJso developed for the Digital
UNIX environment (version 4.0) are
two program analysis and optimization tools-OM and Atom. T h e tool
technology originated in Digital's
Westcrn Rescarch Laboratory, whcre
2
Digital Technical J o ~ ~ ~ . n a l
rescarchers focused on providing
performance diagnosis and improvements for large customer applications.
Sofnvarc developers Linda Wilson,
Craig Neth, and Mike Rickabaugh
from the UNIX Development Environment Group dcscribe the object
modification tools and the flexibility
they provide over traditional tools
that are implcmented in the realm o f
compilers. In addition to demonstrating practical application of the tools,
the authors cxaminc the process o f
transferring technology from research
to development.
For mixed operating system
environments, Digital developed
Windows-bascd X scrver software,
callcd excursion, to allow the windows of a remote host running UNIX
or OpenVMS to display o n a desktop running the Microsoft Windows
operating system. T h e latest version
of excursion, described here by John
Freitas, Jim Peterson, Scot Aurenz,
Chuck Guldenschuh, and Paul
Ranauro, is \vholly rewritten t o masimize graphics performance and t o
support thc k~llrange of Windows
platforms: Windows, Windows 95,
and Windows NT. This ncw version
is based 011the X Window System
version 11, release 6 protocol from
the X Consortium.
T w o network directory services
that reduce con~plexityand increase
choices for n c ~ v o r kmanagers are the
subjects of our next papers. T h e first
is designed for lnultiple networked
environments; Integrated Directory
Services (IDS) software integrates
multiple scrvices into one directoryservice-independent systcm. Margaret
\Jo1. 8 No. 1 1996
Olson, Laura Holly, and Colin Strutt
outline the problems that have limited thc use ofdirectory serviccs and
the different design approaches the
team considered t o simpliljr directory
services use and make it more attractive. They then describe the IDS
extensible, object-based framework,
which con~prisesan application
programming interface and a service provider interface. Nest, Rich
Rosenbaum and Stan Goldfarb
prcsent thc Common Dircctory
Interface (CDI) for DECnet/OSI.
Implemented as shared libraries in
thc Digital UNIX and OpenVMS
operating systems, CDI is designed
to give network managers a choice
of directory serviccs. T h e authors
describe the libraries and thc registration tool set ofmanagement operations that is layered o n a specialized
API.
Coming up in the./nunznlare
papers about a new log-structured
clusterwide file system called Spiralog,
the 64-bit OpenVMS operating
system, speech rccognition software,
and the UNIX clusters messagepassing system and its use for program parallelization.
Jane C . Blakc
Managing Editor
Foreword
Don Harbert
VicePresident, lhVIX D~isiness
Digital not only invented clusters but
continues to set the standard by
which all other cluster systems are
measured. T h e VAXcluster success
and that o f Digital's latest UNIX cluster systems derive from superb engineering t h ~ builds
t
o n the system
definition put forth in the early 1980s
by the VAS engineering team: an
available, extcnsiblc, high-performancc
multicomputer system built from
standard processors and a generalpurposc opcraring system, with chal-actcristics of both loosely and tightl!l
coupled systcn~s.*
We in the UNIX community arc
proud of our VAXcluster lleritage
and have engineered our products
t o provide the sarnc kinds o f bencfi ts
to customers that VAXcluster systems
provide! I n the opening paper for
this issue o f r h c ~ ~ o u r n nmembers
l,
of the Digital UNIX engineering
tcam describc the n~ulticomputer
system for thc Digital UNIX environment, called TruCluster, which, like
the VAXclustcr system, is designed
for liigh availability, scalability, and
performance.
T h e technology, ofcourse, is different, and the cnvironmcnt is open.
The fi~ndanicntalconcepts are nevertheless the same. T h e TruCluster
system is a loosely coupled, generalpurpose system connected by a
high-performance interconnect. It
maintains a single security domain
and is managed as a single system.
* Nancy 1'. I(ronclil)crg, Heliry M. LC\.?,
and Willianl 1). Srreckcr, "VAXclusrcrs:
Cluster services rcmaiil available even
wlien other members arc una\~ailablc.
Like VAXcluster systems, TruClustcr
systems implcrncnt a distributed lock
nianager, which provides synchronization for a highly parallelized
distributed database system. T h e
tecluiology for the lock manager,
however, is nc\illy iniplemented for
the UNIX cnvironmcnt. Also completel!! new is the interconnect technology tor TruCluster systems.
MEMORY CHANNEL is a rcliablc,
high-speed interconnect based o n
a design by Digital partner Encore
Computer Corporation. MEMORY
CHANNEL addresses the unique
needs ofclustcrs by implementing
clustcr\vidc virtual shared memor!l;
the interconnect reduces overhead
and latency by nvo to tliree orders
of magnitude.' Rccai~scMEMORY
CHANNEL uses the industrystandard P(:I, dcsigncrs can implcmcnt the ncnvorlc at \!cry lo\\; cost.
We believe this interconnect technology puts Digital pears ahead of
the competition.
T h e T r ~ ~ C l u s tsystem
cr
is the latest
example of Digital's intent t o re~nain
a technology lcadcr in thc UNIX
market. Wc began by developing
the first high-perfbrmance, 64-bit
general-purpose operating system,
L)EC OSF/l, shipping in March
1993. T h c frst Digital UNIX clustcr
rclcasc, DE(:safc Available Server
Environment, Follo\vcd soon thereafier in April 1994. The an.nounccmcnt in April 1996 o f T r ~ ~ C l u s t e r
systems with MEMOICY CHANNEL
A Closely-CouplcJ Disrriburcd Sysrt.ni,"
AC,M T r ~ ~ n . t c l i o or/
~?s
<:ompuler.S)sic,n~.s,
\lol. 4, 110.2 (1L1.ly 1986): 130-146.
t
Illgiral has rcn~tncd\'AXclustcr systems
to 0pel1ViMSCluster sysrcms.
$ Richard 13. Gillctt, "Mcmory C l l a ~ ~ ~ l c l
Nctwo~-kfor P( :I ," Il;/k:llio.o
(February 1996): 12-18.
Vol. 8 No. 1
I
3
again places Digital tir allcad of thc
competition technologically. Tllc
performance of thcsc a\lailable clustcr
systcn~snow approaches that of very
expmsive supcrcomp~ltcrs.System
pcrformance has bccn measured at
the record-breaking rate of 30,390
tpmC o n four AlpliaSer\,cr 8400
systems runlling Digital UNIX and
the Oracle Uni\rcrsal Scrver \vith
0 r ~ c I cParallel Scrvcr. The pre\~ious
pesfi)rniance record, 20,9 18 tpmC,
was held by the propricrary Tandcni
Himnlaya 1<10000-112; Digital's
opcn s!lstcm clustcr pcrformance
record is 1.5 timcs the Tandem
pcrfi)r~~lance
record at one-third
the systcm cost.
For Digital, clt~stcrsof highpcrk)nnance 64-bit systcms are
to ;I great cstent at the hcart of its
co1nmcrcial3nd technical scrvcr
strategy. Digital U N I S has been
def ncd and engineered for the ser\.cr b~~sincss,
specifically, for the highperformance co~nrncrcialand largcp~)blcm/scic~ltificcn\'ironmcnt. To
be succcssfi~lin the opcn system market, ho\\re\w-, a conipnny must reach
outsiclc itself to joi~itlyengineer prod~ ~ civith
t s leading s o f t w ~ r csuppliers
that lin\,e the sohvare customers
~ ~ c cto
c i be compctiti\.c. Tllerefol-e,
the first Tri~Clustcrimplementation
is designed with Digital's partnew11injors o h ~ ~ acompanies-to
re
meet
t l ~ crcq~~iremenrs
ti)r high pcrformancc
anci f~~nctionalinr
in the commcrci31
d ~ ~ t a b aserver
sc
market.
T h e conlpetiti\lc challcngc ~ i o \is\ ~
to mnint,lin Digital's significant lcad
in providing outsrandiug cluster pcrti)rmancc, a\.ailability, and nfhrdabilin. From a technological perspecti\,c,
the immediate and achievable goal
is t o increase the n ~ ~ n i bof
c rclustcr
nodes h-om 4 to 10 o r 2 0 nodes.
Within this mngc, Digital maintains
a simple cluster system n ~ o d c that
l
offers thc pcrforlnancc ad\~nntnges
of clusterins and avoids the disadvantages, such as the mviagcJncnt problc~iisand qu;ilification hcadachcs, of
more complex topologies. Further,
the Digital UNIX organization \\fill
focus on ,I ncur cluster tile systcln,
configuration flexibility, ~nanagcment
tools, and a clustcr alias that allo\\.s
a single-system vie\\! for clients and
pccrs. T h e overall goal of this work
is to c \ ~ o l \ toward
~c
a more general
computing cn\.iro~~ment.
T h e kinds o f tools that both simp l i and
~ cnliancc pcrfor~nanccarc
cxemplifcd by the program a~lalysis
and optimization tools prcscntcci
in this issuc. 11~1ilr
o n Digital U N l S
version 4.0 and announced in April,
these tools help sofn\rare dc\jclopcrs
extract ~ n a x i r n ~pcrfi)r~uancc
~m
tiom
the system. T h e story of the tools
ilc\~elopmcntis a n excellent csn~nple
of the direct application of research
t o products. Thc po\ver of the O M
objcct ~moditicationtool and tllc analysis tool \\it11 objcct rnodifc;ition
(Atom) \\,as rccog~lizedby de\,clopcrs
c\xn as research progressed; in hct,
sc~luconductordesigners dc\.clopcd
Atoni tools to c\duate nc.\\,Alph;~
clup i~nple~ncntations.
The result
of tlus close cooperation b ~ h \ ~ c c l l
research and dc\,elopmcnt is ,id\,;~nced
program~lli~lg
tools for c ~ ~ s t o ~ l l c r s .
Thcsc efforts in the UNlX organization arc manifestations of 1)igital's
commitmcot t o open systems. Other
3rcns ofcngi~lccring\\rlicl.cthis commitment is apparent are also represented in this issue. For exa~rlplc,
\'ol. S No. 1
19'16
cXc~~rsion
sohvarc is key t o intcgration bcnvcen microso off's Windows
Eirnily of prodt~ctsand Dig~tal's
UNIX and OpenVMS products.
This \\~lioll!lreviscd \icrsion both
adds ncnr fi~nctionnlityand conserves
systcm resources. Another major area
ofstrength for Digital is its networks
products. Networks cngincers
Jcscribc nvo csamplcs ofnenvork
scr\liccs that incrcasc users' choices
and cstcnd system f ~ ~ n c t i o n d i n ,
i.c., the Integrated I>ircctory Scr\.iccs
(IDS) ;lnd the Common 13irectory
Intcrhce.
1)jgital's strate&?)is to c o ~ i t i n i ~t oc
cnginecr products that provide outsmnding performance and price/
performance in opcn en\t ~ ~ - o ~ l ~ ~ ~ c ~ l t ~
I n all areas of engi~lccring-systems,
scr\~iccs,nen~rorlting-our goal is
to set the stanciard by \\.hich all othc~-s
arc ~ncasured.
'
I
Design of the TruCluster
Multicomputer System
for the Digital UNlX
Environment
The TruCluster product from Digital provides
a n available a n d scalable multicomputer syst e m f o r t h e UNlX environment. Although it was
designed f o r general-purpose computing, t h e
first implementation is directed a t t h e n e e d s
of large d a t a b a s e applications. Services such
as distributed locking, failover m a n a g e m e n t ,
a n d r e m o t e s t o r a g e access a r e layered o n a
high-speed cluster interconnect. T h e initial
implementation uses t h e MEMORY CHANNEL,
a n extremely reliable, high-performance interconnect specially designed by Digital f o r t h e
cluster system.
Wayne M. Cardoza
Frederick S. Glover
William E. Snaman, Jr.
The pri~na-!Igoal for the first release of the-.TruCLuster
system For thc Digital U N I S opcrati~igs!lstcn~ was to
develop a liigli-perforninnce commercial database
server environment running on a cluster of several
nodes. l>at,~bascapplications oftcn require computing
power and I/O connecti\lity and band\\lidtli greater
than that provided by niost single systems. In addition, availability is a key rcqi~irementfor cntcrprises
tliat are dependent on database services for nor~iial
operations. These requirenicnts led 11sto implement a
cluster ofconipi~tersthat cooperate to provide serviccs
but fail independently. Thus, both performance and
a\gailabilit\. are addressecl .
We chosc an industry-standard benchmark to gauge
our success in meeting performance goals. Tlie
Transaction Processing Performance Council TPC-C
bcnchniark is a \videly accepted measurcmcnt of the
capdbilip of large servers. Our goal \\#;isto achie\~e
industry-leading numbers in excess of 30,000 transactions per miuutc (tpmC) with a four-node TruCluster
system.
Tlie Tri~Cluster version 1.0 product pro\!jdes
reliable, shared access to large amounts of storage,
distributed synchroniziltion for applications, efficient
cluster co~n~niinication,
and application fi~ilover.The
focus 011 database servers docs not mean that the
TruClustcr system is not suitable for other applications, but that the inevitable design decisions and
trade-offs fix the first product \\!ere made \\;it11 this
goal in mind. Although other aspects of pro\liding
e r i~iiportant,they
a single-system view of a c l ~ ~ s t are
are secondary objectives and \\,ill be phased into the
product o\.er tinie.
This pdper begins \vith a brief coniparison of computer spstclns and presents the advantages of clustered
computing. Next, it introduces the TruCluster product and describes the design of its key softcvare cornponents and thcir relationsliip to database applications.
Tlie paper then disci~ssesthe design of the MEMORY
CHANNEL interconnect for cluster systems, along
with the design of the low-level sofnvarc foundation
for cluster s)lncliro~iizution and co~iimunication.
Finally, it addresses application failover and hardware
configurations.
Digital Technical Journal
Vol. 8 No. 1
1996
---
5
- --
Brief Comparison of Computing Systems
Cluster systems have clear advantages over largescale parallel systems o n one side and heterogeneous
Contemporary computing systems evolved tiom
distributed systems o n the other side. Cluster systems
centralized, single-node time-sharing systems into s a provide Inany cost and availability advantages over
era1 distinct styles of m ~ ~ l t i n o dcomputer
e
systelns.
large parallel systems. They are built ofstandard buildSingle-node systems provided uniform nccessibility
ing blocl<snrith no unusual packaging or interconnect
to resources and services and a s i ~ i g l e - ~ n a ~ i a g e ~ i i e ~requirements.
it
Their 1/0 band\vidth and storage condomain. They were li~nitedwith respect to scalability,
ncctivity scale well with sta~ldardcomponents. They
ho\\/ever, and system failures usuaU!~resulted in a comare inherently more tolerant of failures due to looser
plete loss of service to clients of the systeni.
coupling. 1'~irallelo r rni~ltiprocessorsystems should be
Multinode computer systems include symnlctrii:
thought of as cluster components, not as clustcr
replace~ne~~ts.
multiprocessing (SMP)systelns and massi\lcly parallel
processors (MPPs). They also include network-based
Cluster systems have a different set of advantages
computing systcms such as the Open Sohitare
over distributed systems. First they arc homogeneous
Foundation 1)istributcd Computing Environment
in nature and more limited in size. Cluster systems can
(OSF DCE), Sun ~\/IicrosystenlsInc.'s Open Network
be more efficient \\/hen operating in more constrained
and \vorkstation Lir~ns.',2
Each of
environments. Data formats arc known; tlicre is a
C o ~ n p u t i ~(ONC),
lg
single-security domain; failure detection is certain; and
these systenls addresses one or more of the benefits
associated with clustered computing.
topologies are constrained. Cluster systems also are
SM1' configurations provide for tightly coupled,
likely to liavc interconnect performance advantages.
Protocols are more specialized; interconnect charachigh-performance resource sharing. In their effective
range, SMP systems provide the highest-perh)rma~lcc
teristics are more lu~~iform;
and high perfol-mance can
single-system procluct for shared-resource applicabe gu.lrantecd. Finally, the vendor-spccific nature of
cluster systems allows them to evolve faster than hettions. Outside that range, however, both hard\vare
and sohvare costs increase rapidly as more processors
erogeneous distributed systems and \\,ill probably
always al lo\-\,thcm to have advantages.
are added to an SiM1' systcm. In addition, SMP availThere arc numerous examples of general-p~~rpost:
ability characteristics are more closely associated \\;it11
those ofsingle s!istclns because an SMl'system, by defy
clusters supplircl by most computer \!e~~dors,
incl~~ding
ATkT, lljgital, He\vlett-l'ackard, Intcmational Busiinition, is conlposed of multiple processors b ~ not
~ t
mu1tiple memories or I/O subsystems.
ness Machines Corporation, Sequent Computer Systems, Sun Microsystems, and Tandem Computers.
MPP systems such as the Intel Paragon serics were
13igitaJ's 0pc1iVMScluster system is gcncrally acccpted
developed t o support complex, high-performance
as the most complete cluster product offering in the
parallel applications using systems designed with hunindustry, a~lcl it achieves many of tile single-system
dreds of processors. Thc indi\ridual processors of a n
managen1cnt attributes.' Much of the fi~nctionalityof
M11
' ' system \\,ere typically assigned to spccitic tasks,
the OpenVMS cluster system is retained in Digital's
resulting in fairly special-purpose machines.
T r ~ ~ C l ~ sptreord ~ ~offerings.
ct
The DCE and O N C technologies provide support
for common naming and access capabilities, user
Structure of the TruCluster System
account managcmcnt, nuthcntication, and the rcplicntion of certain ser\!ices for inlpro\ui n\lailability
Digital's TruCluster multicomp~~ter
system is a highly
Worltstation farms SLICII as the Watson lksearch Cenavailable and scalable structure of UNIX servers that
tral Computer Cluster deliver support for thc pal-allel
preserves many of the benefits of a centralized, single
execution of applications within multiple computer
colnputcr system. The TruCluster product is a collecen\.ironments typically constructed using off-the-shelf
tion of loosel!l coupled, general-p~~rposc
computer
s o h a r e and hard~vare.'ONC, DCE, and Eirms prosysterns connected by a high-perforlnance interconvide their services and tools in support of heterogenect. It maintains a single security domain and is manneous, multivendor computing environ~nents with
aged as a single system. Each cluster node may be
hundreds of nodcs. They are, lio\ve\,er, much fi~rther
a uniprocessor or a multiprocessor system executing
away from realizing the benefits of a single-system vie\\/
the Digital UNIX operating system. Figure 1 sho\i~s
associated with clustered computing.
a typical cluster configuration.
In the continuum of n ~ ~ ~ l t i n coinputcr
odc
systcms,
Each cluster member is isolated ti-om sofnvare and
the advantage of the cluster s!lsteni is its ability to
hard\\,arc fai~ltsoccurring o n other cluster mcmbers.
provide the single-system lie\\. and ease of manageThus, the Tru(:luster system does not have the tightly
ment associated with SIMP systems and at the same
coupled, "fail together" characteristics of multiprocestime supply the hilure isolation and scalability of dissor systcms. Cluster services remain n\~ailableeven
tributed systems.
when individual cluster members are temporarily
6
Lligital '1kc.hnical Journal
\ o 8 No. I
1996
NODE 0
NODE 1
MEMORY CHANNEL
INTERCONNECT
4
Cluster Storage Background
Q
NODE 2
or a mirrored \,olume. DKD devices are accessible
from any cluster me~nberusing the DRD device name.
This location independence allows database sohvare
to treat storage as a u~iifor~nly
dccessible cluster
resource and to easily load balance o r h i 1 over activity
between cluster nodes.
NODE 3
%?5!&
DISKS
Figure 1
Co~lfigurationof a Fournodc Clt~stcrSystem
i~navailablc.Other important availability objectives of
the TruCluster server include quick detection ofcomponent and member failures, on-line reconfigurations
to accommodate thc loss of a failed component, and
continued service while safe operation is possible.
The TruCluster product supports large, highly
available database systems through several of its key
components. First, tlie distributed remote disk ( D m )
facility provides reliable, transparent reliiote access to
all cluster storage from any cluster nodc. Nest, the distributed lock manager (DLM) enables the elements of
a distributed database system to synchronize activity
011 independent cluster nodes. Finally, elcnients of
Digital's DECsafe Available Server Environment
(ASE) provide application failover.' In support of all
these components is the connection manager, \vhich
controls cluster menibcrship and the transition of
nodes in and out oftlie cluster. Figure 2 is a block diagram showing tlie relationships between components.
Each major component is described in the reniainder of this paper. In addition, we describe the highperformance MEMORY CHANNEL interconnect
that \itas designed specifically for the needs of cluster
systems.
Distributed Remote Disk Subsystem
The distributed remote disk (DRD) subsystem was
developed to support database applications by presenting a clusterwide \liew of disks accessed through the
character o r raw device interface. The Oracle l'arallel
Server (OPS), which is a parallelized version of the
Oracle database technology, uses the DRD subsystem.
The D W subs)lstern provides a clusterw~denamespace and access mechanism for both physical and logical (logical storage manager or LSM) volumes. The
LSM logical device may be a concatenated, a striped,
Disk devices on UNIX systems are commonly ~ccessed
through tlie UNIX file slatem and a n associated block
device special filc. A disk device may also be accessed
through a character device special file or ra\v device
that provides a direct, unstructured interfice to tlie
device and bypasses the block buffer caclic.
Database management systems and sonic other
high-performance UNIS applications are often
designed to take advantage of tlie character dcvicc spccia1 file interfaces to inipro\~eperformance by avoiding
additio~lalcode path length associ:lted with the f le system cache.".' Thc 1 / 0 profile of these systc~iisis characterized by large files, random access to records,
private data caches, and concurrent read-write sharing.
Overall Design of the DRD
T h e DRD subsystem consists of four primary coniponents. The remote rant disk ( M U ) pseudo-driver rcdirects DRD access requests to tlie cluster member
serving the storage device. The server is identified by
information maintained in the D W devicc database
(RRDB). Requests t o access local DIID devices are
passed through to local device drivers. The block shipping client (13SC) sends requests for acccss to renlote
1)KD devices to tlie appropriate DIU3 server and
returns responses to the ca.ller. 7Yhe block shipping
server (BSS) accepts requests from 13SC clients, passes
them to its local driver for service, and returns the
results to the calling BSC client. Figure 3 shows the
components of the DM3 s~~bsystem.
The DRI) management component supports DRD
device naming, device creation and deletion, device
relocation, and device status requests. During the
DRD device creation process, the special device file
designating the DRD device is created o n each cluster
member. In addition, the D 1 U device nunibcr, its corresponding physical device number, the nenvork
address of the serving cluster member, and other configuration parameters are passed t o the DRD driver,
which updates its local database and communicates
the information to other cluster nie~iibers.The DRD
driver may be queried for device status and DRD database information.
Clusterwide Disk Access Model
During the design of tlie DRD sitbsystem, \ire considered botli hared (~nultiported)and served disk ~iiodels.
A multiported disk configuration provides good failure
recovery and load balancing characteristics. 011 the
Digital '1-cchnical ] o ~ l ~ . n n l
\ / 0 1.
8 No. 1
1996
r------------------
I
1
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
DATABASE INSTANCE
RAW DISK
INTERFACE
MANAGER
INTERFACE
t
A
-----------
~NOD;-----
I
I
I
I
I
I
I
I
I
INTERFACE
t
A
+ DECSAFE ASE AVAILABILITY SERVICES
I 1 '
DISTRIBUTED DIRECTORY SERVICE
CONNECTION
MANAGER
AGENT
MANAGER
AGENT
AVAILABILITY
AVAILABILITY
MANAGER
MANAGER
DISTRIBUTED REMOTE DEVICE
DRD I10
REDIRECTION
DRD BLOCK
A
MOVER
DRD BLOCK
SHIPPING
4
--
SERVER
I
LOCAL DEVICE DRIVERS
1
MEMORY CHANNEL
SERVICES
OCAL DEVICE DRIVERS
SHARED
STORAGE
NODE C
NODE D
Figure 2
Sofi\\;arc Components
1
DRD CLIENT
I
I
I
DRD DEVICE
DATABASE
REMOTE DlSK DRIVER
II I
I
-
Figure 3
Distril)~~tcii
R c ~ ~ i oDisk
te S U ~ S ~ S ~ C I I I
Digital '1ic.hniznl Journal
IACCESS TO ldevldrdldrd5 I
I
Vol. 8 No. 1
1996
REMOTE DISK DRIVER
I
I
I
S
I
I
1
I
I
I
I
I
I
I
I
I
I
I
I
I
I
_1
SHARED SCSl BUS
--
I
I
I
I
CONFIGURATION
MANAGER
DISTRIBUTED LOCK MANAGER
CONFIGURATION
MANAGER
I
1
I
1
NODEA
I
DEVICE DRIVER
I
I
I
other hand, I/O b i ~ contention
s
and Iiard\vare queuing
delays from f ~ ~ lconnected,
ly
shared disk contigurations
can limit scalability. In addition, present standard 1/0
bus technologies limit configuration distance^.^ As a
consequence, wc selected a served disk model for thc
DIU) i~iiplementation. With this model, software
qi~euingalleviates the bus contention and bus queuing
delays. This approach provides i~nprovcdscalability and
fault isolation as well as tlexible storage configurat i o n ~ .Full
~ ' ~corlnectivity is not reqnired, and estencied
machine room cluster configurations can be constructed using standard networks and 1/0 buses.
The 1'>1U>implementation supports clusterwide
access to DM'> dcvices using a software-based emnlation of a fully connected disk configuration. Each
device is assigned to a single cluster member at a time.
The membcr rcgistcrs the device into the clustcr\vide narnespncc and serves the device data t o other
cluster members. Failure recovcrv and load-balancing
support are includcd with the 1)lW device implementation. The failure of a node or controller is transpurently masked ~vlienanother node connected t o the
shared bus takes over serving the disk. As an option,
automatic load balancing can move service of the disk
to the node generating the most rcqilests.
In the Tru(:lustcr version 1.0 product, data is
transferred between requesting and serving cluster
members using the high-bandwidth, low-latency
LMEIMORYCHANNEL interconnect, which also supports direct memory acccss (DIMA) benveen the I/O
adapter of the serving node and the 11i;iin memory of
the requesting node. The overall cluster design, lio\\lever, is not dependent on the MEhllOllY CHANNEL
i~lterconnect,and alternative cluster interconnects will
be supported in ti~ t u r sofhvare
e
releases.
DRD Naming
The Digital U N I X operating s!lstcm presently supports
character device special file names fix both physical disk
devices and LSM logical volumes and mainrains a separate de\licc namespace for each. An important DlU)
design objective was t o develop a clusterwide naming
scheme integrating the physical and logical devices
within thc D l U namespace. We considered defining
J. ne\\; single namespace to support all cluster disk
devices. Our research, however, revealed plans to inuoduce significant changes into the physical device naming scheme in a f i ~ t i ~ rbase
e system release and the
co~iiplicationsof licensing the logical disk technology
honi a third party that maintains control over the logical volume namespace. These issues resulted in deferring a true cli~sterwidedevice namespace.
As an interim approach, we chose to create a separate, clusterwide DRD device naniespace layered o n
the existing physical and logical device naming
schemes. Translations from 1)lU) device names into
the underlying physical and logical devices are maintained by the I)RD device mapping database on each
cluster node. 1)IW device "scr\licesn are created by
the cluster 'idministrator using the scr\fice registration facility." Each "add Service" management operation gencrates a ~lnique service number that is
used in constructing the DlUI dcvice special file name.
This operation also creates the new DRD device
special file on each cluster member. A traditional
UNIX-device-naming con\tention results in the creation of DRD special device file names in the form of
/de\!/drd/drd{ser\lice number)."
DRD Relocation and Failover
ASE hilover (see the discussion in the section
Application Failover) is used to support D1W failovcr
and is fi~llyintegrated within the cluster product. The
device relocation polic!~defined during the creation of
a DRD d e ~ i c cindicates whcthcr the device rnay be
reassigned to another cluster membcr as a result of
a node or controller failure or a load-balancing opcration. In the event o f a cluster mcmbcr failure, D m
devices exported by the failed member are reassigned
to an a1tcrn.m server attached to tlie same shared 1 / 0
bus. During reassignment, the 1)LW device databases
are updated on all cluster members and DRD I/O
operations are resumed. Cluster dcvjce services rnay
also be reassigned during a planned relocation, such
as for load balancing or me~iiberremoval. Any DRD
operation in progress during a relocation triggered by
a failure \\rill be retried based upon the registered DKD
retry policy. TIic retry mechanism must revalidate tlie
database translation map for the target DRD device
because the scrvcr binding may have been modified.
Failover is thus transparent to database applications
and allo\ia them to ignore configi~rationchanges.
Several challenges result fro111 the support of
multiported disk configurations under various failure
scenarios. One of the more difficult problems is distinguishing a failed member from a busy member or a
communicatio~~
fault. The ASE hilover mechanism was
designed to maintain data integrity during service
failover, and to ensure that subsequent disk operations
are not lionorcd fi-0111a member tliat has been declared
"do\vn" by the remaining cluster members. This ASE
mechanism, which makes usc of small computer systems interbce (SCSI) target mode and device reservation, \\!as integrated into the TruCluster version 1.0
product. and supports the DRD service guarantees.
Other challenges relate to preserving serialization
guarantees in the case of cluster member failure.
Consider a parallel application tliat uses loclts to serialize access to shared DRD devices. Suppose the application is holding a write lock for a given data block and
Iligirnl Technical Journal
Vol. 8
No. 1 1996
r
issues an update for tliat block. Before the update
operation is ackno\vledged, ho\vever, the local mcniber fails. The distributed lock manager, \vhich will
have been notified of the member failure, then takes
action t o release the lock. A second cooperating application executing on another cluster member now
acquires the write lock for that same data block and
issues an update for that block. If the failure had not
occurred, the second application would have had to
wait to acquire a \\!rite lock for the data block ~ ~ n tthe
il
first application rcleascd the lock, presumably aher its
write request had completed. This salne serialization
must be maintained during failure conditions. Thus, it
is imperative that the write issued by the first (now
failed) application partner not be applied aker the
write issued by tlie second application, even in the
prcscnce of a timing or nen\iorl<retransmissio~ianomaly tliat delays this first \vrite.
T o avoid the reorderilig scenario just described,
we employed a solution called a sequence barricr in
which the connection manager increments a secluence
number each time it completes a recovery transition
that results in rcleascd locks. The sequence number
is communicated to each DRD server, which uses
the sequence number as a barrier to prevent applying stale \\(rites. This is similar to the immediate command feature of thc Mass Storage Control Protocol
(MSCP) used by OpcnV1MS cluster systelns to provide
similar guarantees. Note that n o application changes
are required.
As another esan~ple, client retransmissions of
D1W protocol reclilests that are not ide~iipotentcan
cause serious consistency probler~ls.Ilecluest transaction IDS and DRD server duplicate transaction caches
are employed to avoid undesirable effccts of clicntgenerated retransmissions.';
Cluster member failures are mostly transparent to
applications executing on client mernbcr systen1s.
Nondisuibuted applications may fail, but they can be
autoniatically restarted by ASE facilities. DlU> devices
exported by a serving ~rieniberbecome unavailable for
a small amount of time when the member fails. Cluster
failover activities that ~ i i ~ l Os tC C L I ~before the I)RD
service is again a\~ail;~blc
include detecting and \fen@iug the member failure, purging the disk device SCSI
hardware reservation, assigning an alternate server,
establishing the new reservation, and bringing the
device back on-line. A database application serving
data from the DR1) device at the time of the failure
may also have registered to have a restart script \vitIi
a recovery phase esecuted prior to the restart of tlie
database application. A possible lack of transparency
may result if somc clicnt applications are not desigucd
to accommodate this period of illaccessible DRD service. The DRl) retry request policy is configurable
to accommodate applications interacting dircctly wit11
a DRD device.
Vol. 8 No. 1
1996
Distributed Lock Manager
The distributed lock manager (DLM) provides synchronization services appropriate for a highly paralJelized distributed database system. Databases can use
locks to control access to distributed copies of data
buffers (caches) or to limit concurrent access to shared
disk deviccs such as those provided by the DRD subsystem. Iacks can also be used for controlling application instance start-up and for detecting application
instalice failures. In addition, applications can use the
lockillg services for their other synchronization needs.
Eveti though this is a completely new ilnplementation, the lock manager borrows from the original
design and concepts introduced in 1984 with the
VAXcluster distributed lock manager.'" These concepts
were used in several recent lock manager implementations for UNIX by other vendors. In addition, the
Oracle Parallel Server uses a locking application programming interface (MI)that is co~~ceptually
similar
to that offered here.
Usage of the DLM
The Jock manager provides an API for requesting, releasing, and altering locks.'^'" These locks are
requested o n abstract names chosen by the application. The nanics represent resources and may be organized in a 1iierarcli)r.When a process requests a lock on
a resource, that request is either granted or denied
based o n csamination of locks already granted on the
resource. Cooperating components of an application
use this service to achieve mutually exclusive resource
usage. In addition, a mode associatcd with each lock
request allows traditional levels of sharing such as multiple readers cscluding all \vriters.
The API provides optional as!~nchronous request
completion to allow queuing requests o r overlapping
multiple operations for increased perforn~ance.
Queuing prevents retry delays, eliminates polling
overhead, and provides a first in, first out (FIFO) fairness mechanism. In addition, asynchronous requests
can be i~scdas the basis of a signaling mechanism to
detect component failures in a distributed system. One
component acquires an exclusi\,e lock o n a named
resource. Other components queue incompatible
requests \\lit11 as)rnchronous completion specified. If
the lock holder fails or otherwise releases its lock, the
waiting requests are granted. This usage is sometimes
referred to as a "dead man" lock."
A process can request notification when a lock it
holds is blocl<inganother request. This allows elimination of Inany lock calls by effecti\,ely caching locks.
When resource contention is lo\\.,, a lock is acquired
and held ~ ~ n tanother
il
process is blocked by that lock.
Up011 receiving blocking notification, tlie lock can be
released. When resource contention is high, the lock
is acquired and released immediately. In addition, this
notification mechanism call be used as the basis of a
general signaling meclianisni. Onc coniponent of thc
application acquires an exclusive lock on a named
resource tvith blocking notification specified. Other
components then acquire illcompatible locks o n that
resource, thus triggering the blocking notification.
This usage is hewn as a "doorbell" lock."
The DLlM is often used to coordinate access to
resources such as a distributed cache of database
bloclts. Multiple copies of the d'lta are held under
compatible locks t o permit read but not write access.
When a writer \\/ants an incompatible lock, readers are
notified to dourngrade their locks and the writer is
granted tlic lock. The \vriter ~iiodifiesthe data before
downgrading its lock. The reader's lock requests are
again granted, and the reader fetches the latest copy of
tlie data. A value block can also be associated with each
resource. Its value is obtained when a lock is granted
and can be changed when certain locks are released.
The value block can be used to colnniunicate any usefill information, including tlie latest version nuniber of
cached data protected by tlie resource.
Design Goals of the DLM
The overall clesign goal of the lock manager was to
provide services for highly scalablc database systcms.
T ~ ~ correctness,
LIS
robustness, scaling, and speed were
the overriding subgoals of the project.
C:arefi~ldttcntion to design details, rigorous testing,
internal consistency checking, and years of esperie~lce
\\,orking with tlie VMS distributed lock manager have
all contributed t o ensuring the correctness of the
ilnplcmentation for the Digital UNIX system. Because
the lock manager provides guarantees about the state
ofall locks cvhen either a lock holder or thc node upon
which it is running t'ails, it can ensure the internal lock
state is consistent as far as survi\rillg loclc holders are
concerned. This robustness pcr~lmitsthe design of
applications that can continue operation when a cluster node fails or is rel-noved for scheduled service. The
choice of a I<er~iel-bascd
scrvicc and thc use of a message protocol also contribute t o robustness as discussed belo\v.
In terms of performancc and scaling, the lock manJgcr is designed for minimal overhead to its users. The
kernel-based service design pro\fides high performance by eliminating the contest switch overhead
associated with server daemons. The lock manager
uses tlie kernel-locking features of the Digital UNIX
operating system for good scaling o n SMP systems. A
ker~lel-basedservice as opposed to a library also allows
the lock manager to make strong guarantees about the
internal consistency statc of locks \\hen a lock-holding
process fails.
The message protocol contributes to cluster scaling
and perforrnance through a scaling property that
maintains a constant cost as nodes arc added to the
cluster." The message protocol also provides sufficiently loose coupling to allow the lock manager to
maintain internal lock state cvhen a node fails. The use
of messages controls the amount of internal state visible t o other nodes and provides natural checkpoints,
which limit tlie damage resulting from tlie failure of
a cluster node.
DLM CommunicationServices
The LILM session service is a conimunicatio~ilayer
that takes advantage of MEMORY CHANNEL features such as guaranteed ordering, low error rate, and
low latency. These features allow the protocol to be
very si~nplcwith an associated reduction in CPU overhead. T h e service provides connection establishment,
delivery and order guarantees, and buffer managemcnt. The connection manager uses the communication service to establish a channel for the lock
manager. The lock manager uses the communication
services to communicate between nodes. Because the
service hides the details of the communication meclianis~n,alternative interconliects can be used \vithout
changes to the lock manager's corc routines.
The ilse of the MEMORY CHANNEL interconnect
provides a very low latency communication path for
small messages. This is ideal for the lock nianager since
lock rnessages tend to be very small and the users of
the lock manager are sensitive to latency since they
wait for the lock to be granted before proceeding.
Small rnessages are sent by simply \vriting them into
the receiving node's memory space. No other communication setup needs t o be performed. Many network adapters and coniniunication protocols are
biased to\vard providing high throughput only when
rclativcly large packets are used. This means that the
perforrnance drops off as the packet size decreases.
Thus, the iMEMOKY CHANNEL intercolinect provides a better alternative for communicating small,
latency-sensitive packets.
Connection Manager
The connection manager defines an operating environment for the lock manager. The design allows generalization to other cljentb; but in the TruCli~ster
version 1.0 product, the lock manager is the only consumer of the connection manager services. The environment hides the details of dynaniically changing
configurations. From the perspective of the lock manager, the connection manager manages the addition
and removal ofnodes and maintains a communication
path between each node. These services allowed us to
sirnpli@ the lock manager design.
The connection manager treats each node as a niember of a set of cooperating distributed components.
It maintains the consistency of the set by admitting
and removing members under controlled conditions.
The connection !.lanagel- pro\-ides configurationrclatcd event notification and othcr support serviccs
to each mcrnber of a set. It pro\.ides notification \\hen
members are added 2nd rcmovcci. It nlso maintains 3
list of current members. Tlic connection nlallager also
pro\jides notitication to clients when ~ ~ n s aoperation
fe
is ~.>ossible
as a result of partitioning. Partitioning exists
when a member o f a set is ~1113warc
oftlic csistence of
n disjoint set ofsirnilar clients.
The connection m;lnagcr can be extended in
client-specific \\pa!ls to t;?cilit,ltc handling of mcmbership change e\*ents. Estcnsio~uare integral, \veils!rnchronized parts of tlie me~nbership change
mcclia~~is~ii.
The lock Iiianagcr uses an estension to
distribute a globally consistent director!r database and
to coordinate lock database rebuilds.
The connection manager nlaintains a fully connected web of co~nmunic.itio~~
channels between
members of the set. Menibership in tlie sct is conringcnt upon bcing able to communicate with all other
~ n c n ~ b eof
r s that set. The use of the communication
channels is entirely i~ncicrthe control of tlic lock miinagcr or any other clicnt that may ~rsctlie connection
manager in the future. When a client recluests admission t o a set, the connection Inanager establishes a
communicatio~icha~lnclbct\\iccn the ne\v clicnt nnd
a11 existing clients. I t mo~litorsthese connections to
ensure they remain f~nctio~liil.
A connection hils
when a commirnication channcl is unusable bctwccn
3 pair of clients or \\rlicn 3 clicnt nt citlicr end o f the
clinnnel fails. The connection nianagcr detects these
conditions and reconfigurcs the set to contain only
fi~llyconnected members.
I lie combination of a IiighI!~a\railiiblc c o ~ i i ~ i i u ~ i i cation channel, togcthcr \\.it11 sct mcrnbersliip and
synchronized membership cliange responses, allo\vs
optimizations in the lock man,~gcr'smessdge protocol.
The lock manager can scnd a message to another nodc
and know that either the message will be deli\lered or
tliat the configuration \\,ill he ;iltercd so tliat it does
not matter.
--
I
MEMORY
CHANNEL
TRANSFER
The use of thc conncction rnnn'lger greatly s i n plitics the design and implcmcntation of tlie loci<
manager. The connection manager allo\\s most of
tlic logic for handling configurntion changes and com~nunicationerrors to be 111ovcdaway from main code
p~tlis.This illcreases mainline perfi)rmance and siniplitics the logic, allowing 11io1-ee~llphasis011 correct and
efficient operation.
Memory Channel Interconnect
<:luster performance is cr-iticnlly clepcnderit on tlic
cluster interco~inect.This is d i ~ cboth to tlie higliband\vidtli recllrirerncnts of bulk data transport for
1)111) and to tlie low Intenc!~rccji~ircdfor DI,M operntions. Although the cluster architecture allo\vs for any
high-spccd interco~incct,the initial implementation
supports only tlie new IMEIMOKY CHANNEL interconnect designed specifically for tlie needs of cluster
systcms. This very rcli;~blc,high-spccd interconnect is
based on a pre\.ious interconnect designed by Ellcore
C:ompter Corporation.'" It has been signiticantl!,
cnli.inccd by Digital to impro\'c d'ita integrity and
pro\'ide for higher perfi)rmance in the f i ~ n ~ r e .
Each cluster node has a MEIVORY CHANNEL
intcl-f~cccard tliat connects to n hub. The liub can bc
tl~oughtofas a switch that provides either broadcast or
point-to-point connections benvcen nodes. It also
provides ordering guarantees and docs a portion of
the crl.or detection. Tlie current implcmentatio~~
is an
eight-node hub, but larger l i ~ ~ arc
b s planned.
Tlic MElMORY CHANNEL interconnect pro\*idesa 100-1~1egaL~yte-pe1--scco1ic1,
memory-mapped
connection to other cluster mcmbcrs. As sho\\v in
Figure 4, clustcr members ~ n n yrnnp transfers fl-om t l ~ c
MEMORY CHANNEL interconnect directly into
tlicil- memory. The effect is of 3 \\?I-ire-only\\,indo\\!
into tlic mcmory of othcr cluster systclns. Transfcrs
;ire done with standard memory ;icccss instructions
rather than special I/<) instructions or device access
1
MEMORY
CHANNEL
TRANSFER
NORMAL
MEMORY
WRITE
NODE 0
ADDRESS SPACE
--
-
MEMORY CHANNEL
BUS ADDRESS SPACE
-
Figure 4
Transfcrs Pcrformcd by thc MEMORY CHANNEL Intcrconncst
Vol 8 No. I
1906
NODE 1 MEMORY
protocols to a\.oid the o\,erhcad us~~nll!,present \\lith
these techniques. The use of memory store instructions results in extremely lo\\, latency (nvo microseconds) and lo\v o\~erlie,~d
for a transfcr of any length.
Thc MEMORY CHANNEL interconnect guarantees csscntially no undetected errors (approsiniately
rile salne undetected error ratc as Cl'Us or n ~ e m o r ) ~ ) ,
allowing the elimination of chccksunls and other
rnccha~ljsmsthat detect software errors. The detected
error rate is also exrremel!~ lo\\, ( o n the order of one
crror per year per connection). Since rcco\,ery code
csecutes very infrequently, \\,c are assured t h ~ relat
tively simple, brute-force recovery from sofnvare
errors is adecluate. Using hard\\,are crror insertion, \\re
have tested recovery code nt el-ror rates of many per
second. T h i ~ we
s are confident there are n o problems
at tlie actual ratcs.
Low-level MEMORY CHANNEL Software
I,ow-level sofnvore interfaces are provided to insi~late
the nest laver of sofnv.1re (e.g., lock Inanager and distributed disks) from the details of the MElMORY
CHANNEL inipIenicnt.~tion. We have taken the
approach of providing a very thin layer t o i~ilpactperfor~nanceas little as possible and allow direct use ofthe
MEMORY CHANNEL interconnect. Higher-level
sofnva.re then isolates its use oflMEMOl<Y CHANNEL
in a tmnsport layer that can later be modified for additional clt~stcrinterconnects.
The \\,rite-only nature of the MEMORY CHANNEL
i ~ i t e r c o ~ ~leads
~ ~ c cto
t some challenges in designing
and imple~ncntingsohvnrc. The only \\lay to see a
copy of data \vritten to the IMEMORY CHANNEL
intcrconncct is t o niap MEMORY CHANNEL transfers to another region of mclnory o n the same node.
This lends to nvo very visible programming constraints. First, data is read and \\/ritten from different
addrcsscs. This is not a natural programming style, and
code must be Ivritten to treat a location as turo variables, one for read and one for write. Second, the
effect of a \\.rite is delayed by the tra~isfcrlatenc!!. At
microseconds, this is short but is enough time t o
csccute hi~ndredsof instructions. Hardivare features
nrc pro\.idcd to stall until data has been looped back,
h i ~ very
t
carefill design is neccss.lry to niinin~izethese
stalls and place them correctly. \iVe have had several
s ~ ~ b tproblems
lc
when an algorithm did not include a
stall ~ i n dproceeded to rend stale darn that was soon
o\~crwrittenby data in transit. Finding these problems
is especially difficult becai~scmuch cvidcnce is gone by
the time the problem is observed. For example, consider a linked list that is implemented in a region of
mcmor!l mapped t o all clustcr nodes through the
1MEMORY CHANNEL illterconncct. If nvo elelne~lts
are inserted o n the list \vithou t inserting proper waits
for the loopback delay, the effect ofthc first insert \\,ill
not be visible \vhen the sccond insert is done. This
results in corrupting the list.
The difficulties just described ,Ire most obvious
whcn dealing with distributed shared nicmory. Lowlevel software intended t o support applications is
instead oriented toward a ~iiessagc-passingmodel.
This is especially apparent in the features provided for
error detectio~l.The primary mechanisms allow either
thc receiving or the sending nodc to check for any
errors over a bounded period of tinlc. This error check
rcqi~ires3 special hard\vare transaction with each node
and involves a loopback delay. If an error occurs,
tI1c sc~ider must retransmit all messages and the
receiver must not use any data received in that time.
This mechanism works well with the expected error
ratcs. Howcver, a shared memory model makes it
extremely difficult to bound the data affected by an
error, i~nlesseach modification of a data element
is separately clwcked for errors. Sincc this involves
a loopback dcla!l, man!! of the perceived efficiencies
of shared memory may disappear. This is not t o say
that a shared memory model cannot be used. It is just
that error detection and control of concurrent access
lnust be well-integrated, and node hilures require
carcf~~l
recovery. In addition, the write-only nature of
MEMOltY CHANNEL mappings is more suited to
message passing than shared memory due t o the
cstrc~nclycarefill programming necessary to handle
delayed loopback at a separate address.
APTs are provided primarily t o Inanage resources,
control rnemor!, mappings, and provide synchronization. MEMORY C f i W N E L APIs perk)rni the following tasks:
Allocation and mapping
or deallocate the MElVORY
CHANNEL address space.
- Map the MEMORY CHANNEL, interconnect
For receive or transmit.
- Unmap
the
MEMORY
CHANNEL
interconnect.
Spinlock synchronization
- Create and delete spinlock regions.
- Accluire and release spinlocks.
Other synchronization
- Create and delete n~ritc acknowledgment
regions.
- l t e q ~ ~ ewrite
s t ack~~o\vlcdgmcnt.
- Create and delete sofwarc ~iotificationchannels.
- Send notification.
- Wait fbr notification.
Error detectioll and rcco\,el-!I
- Get cilrrellt error c o i ~ n t .
- Check for errors.
- Register for callback o n c u o r
- Allocate
Higher layers of software are responsible for transferring data, checking for errors, retrying transfers, and
synchronizing their use of MEMORY CHKilNEL
address space after it is allocated.
Synchronization
Efficient synchronization mechanisms are essential
for high-performance protocols over a cluster interconnect. MEMORY CHAi-UNEL Ilardware provides
two important syncl~ronizatio~i
niechanisms: first, an
ordering guarantee that alJ writes arc sccn in the same
order on all nodes, including the looped-back write on
the originating node; second, an acknowledgment
request tliat returns the current error state of all other
nodes. Once the acknowledgment operation is complete, all previous \vrites are guaranteed either to Iia\~e
been received by other nodes or rcportcd as a transniit
or receive error 011 some node. We have implemented
clusterwide software spinlocks based on these guarantees. Spinlocks are used for many purposes, including
internode synchronization of other components and
concurrency control for the clusterwide shared-memory data structures ~ ~ s ebyd the lo\v-lcvel MEMORY
CHANNEL sofnvarc.
A spilllock is structured as an array with one element
for each node. T o acquire thc spinlock, a node first
bids for it by writing a value t o the node's array element. A node wins by seeing its bid looped back by the
MEMORY CHANNEL interconnect \vithout seeing
a bid from any other node. The ordering guarantees of
the MEMORY CHANNEL ensure that n o other node
could have concurrently bid and belicved it had won.
Multiple nodes can realize they have lost, but more
than one node cannot win. In case of a conflict, many
different back-off techniques can bc used. The winning node then changes its bid v a l ~ ~toean own value.
This last step is not necessary for correctness, but it
does help with resolving contention and \vith various
failure recovery algorithms. All higher-level synchronization is built on combinations of spinloclts, ordering guarantees, and error aclu~owlcdgmcnts.
Error Recovery and Node Failures
Most of the difficult problems in the low-level software relate to error reco\/ery and node failures. In spite
of its reliability, errors \\rill occur in tlic MEMORY
CHANNEL interconnect, and they must be handled
as transparently as possible. Transparency is key to s i n plifiing the communication model seen by higherlevel sohvarc. I n addition, nodc failures from
hardulare o r sohvare faults are more frequent than
MEMORY CHANNEL errors and must be dealt \\/it11
even in the most inconvenient portions of the lowlevel code. T h e MEMORY CHANNEL intercon~lect
is managed through a collection of distributed data
14
Digital Tcclitiic.~lJo~trn.il
\rol 8 No. I
1996
structures that must be kept consistent. Sofnvare locks
to s)~nchronizcaccess to these structures, but
errors may leave them in an inconsistent state.
Guaranteed error detection before the relcase of a lock
allo\vs operations to bc rcdone in case of an error.
Thus, all sequences of MEMORY CHANNEL \\!rites
I T I L I Sbe
~ idernpotent to take advantage of tliis straightforward error-recovery technique.
Ifa node failure occurs, a sur\~i\lingnode must make
all data structures consistent before it releases locks
hcld by the failed nodc. T o Itecp this a manageable
task, we have written caref~~lly
structured algorithms
to handle each inconsistent state. In general, structures arc changed such that a single atomic write comniits a change. If a node fails before this last \\,rite, 110
recovery is necess'lry. As an example, consider n data
structure that is co~llplctelyinitialized bcforc being
added to a list. A single write is used to acco~nplishthe
list addition. If a node (ails, the last \vrite was either
done or not and, in either case, the list is consistent.
Complications arise \vl~enanother node has n receive
crror o n the last write done by a failing node. In this
case, the failed node ca111iotretry after detecting the
error, so the node cvith the receive error has a different
view of the list than all other surviving nodes. To
rcsolve tlus cvent, OIIC node must propagate its view of
the list to all other nodes before it releases the lock
held by the failed ~ i o d eAn!{
. node can d o tliis because
each has a self-consistent vie\\. of the list. If the node
\\it11 the receive error propagates its \vie\\!, the last element added by the failed node is lost. This sjt~~ation
is
no different, however, from having thc nodc fail a few
instructions earlier. Thc challenge is to design recoilcry for all these cases and maintain our sanity by minimizing the n ~ ~ n l bofsuch
er
cases.
Another interesting problem is maintaining a consistent count of errors across all nodes. This count
is Itcy to the error protocols of both the lo\\,-level
MEMORY CHANNEL software and higher layers
since cornparjsons of a saved and a current \lalue
bou~ldthe period ovcr which data is suspect. The
count may bc read o n one node, transferred with
a message, and cornpared to a current value on
another nodc. Thus, a consistent value on all nodes
is critical and must be maintained in tlic presence of
arbitrary combinations of rcceive and transmit errors.
(Although errors arc \!cry infrequent, they may be correlated; so algoritlims 111ustwork well for crror bursts.)
The write acknowledgment, described earlier, guarantees that other nodes havc received a svritc ~\itliout
error. It is used both to implement a lock protecting
tlic error C O L I I Tand
~
to guarantee tliat all nodes have
sccn an ~ ~ p d a t ecount.
d
Updati~igthe count is a slo\\.
operation due to multiple round-trip delays and long
crror time-outs, but it is performed very infrequently.
31-cused
Future Enhancements to MEMORY CHANNEL
Software
Fully supportcd MEMORY CHANNEL APIs arc
currently available only to other layers in the UNIX
kernel for two important reasons: First, MEMORY
CHANNEL is a new type of interconnect and we want
t o better understand its uses and advantages before
committing to a fully functional API for general use.
Second, many difficult issues of security and resollrce
limits will affect the final interface. T o help Digital
and its customers gain the necessary experience, a limited functionality version of a user-level MEMORY
CHANNELAPI has been implemented in the version
1.0 product. This interface supports allocation and
mapping of MEMORY CHANNEL space along with
spinlock synchronization. It is oriented toward support of parallel conlputation 111 a cluster, but we also
expect it will serve the needs of many commercial
applications. Once we have a better understanding of
how high-level applications will use the lMEMORY
CHANNEL interconnect, we will extend the design
and provide additional APIs oriented toward both
commercial applications and technical computing.
Application Failover
Digital's TruCluster multicomputer system is a logical
evolution of the DECsafe Available Server Environment (ASE). An ASE system is a rnultinode configuration with all nodes and all highly available
storage connected to shared SCSI storage buses.
Figure 5 shows an ASE configuration. Software on
each node monitors the status of all nodes and of
shared storage. In case of a failure, the storage and
associated applications are failed over to surviving systems. Planned application failover is accomplished by
stopping the application on one node and restarting
the application on a surviving node with access to any
storage associated with the application. Applicationspecific scripts colitrol failover and usually d o not
require applicatio~ichanges.
SCSI BUS 1
NODE 0
II
NODE 1
SCSI BUS 2
Figure 5
Typical ASE Contiguration
II
In addition to supporting the application failover
mechanisms from ASE, the TruCluster system supports parallel applications running o n multiple cluster
nodes. In case of a failure, the application is not
stopped and restarted. Instead, it may continue to cxecute and transparently retain access t o storage through
a distributed disk server. I11 rtdd~tion,more general
hardware topologies arc supported.
Hardware Configurations
The TruCluster version 1 . 0 product supports a maximum of four nodes connected by a high-speed
MEMORY CHANNEL interconnect. Thc nodes may
be any Digital UNlX system with a peripheral component interconnect (PCI) that supports storage and the
MEMOlZY CHANNEL interconnect. Highly available
storage is on shared SCSI buses connected t o at least
two nodes. Thus, a cluster looks like multiple ASE
systems joined by a cluster interconnect.
Although the limitation to four nodes is temporary,
we d o not intend to support large numbers of nodes.
Ten t o nventy nodes o n a high-speed interconnect is
a reasonable target. A cluster is a component of a distributed system, not a replacement for one. If very
large numbers of nodes are desired, a distributed
system is built with cluster nodes as servers and other
nodes as clients. This allows maintaining a simple
lnodel of a cluster system \\~ithouthaving to allou~for
many conlplex topologies. Aside from sirnpjicity, there
are performance advantages from targeting algorithms
for relatively small and simple cluster systems.
Although the nuniber ofnodes is intended to be snlall,
the individual nodes can be high-end multiprocessor
systems. Thus, the overall computing power and the
I/O bandwidth of a cluster are extremely large.
Conclusions
With the con~pletionof the first release of Digital's
TruCluster product, \ve believe we have met our goal
of providing an environment for high-performance
commercial database servers. Both the distributed loclc
manager and the remote disk services are meeting
expectations and providing reliable, high-performance
services for parallelized applications. The MEMORY
CHANNEL interconnect is proving to be an excellent
cluster interconnect: Its synchronization and failure
detection are especially compatible with many clusteraware components, which are enhanced by its low
latencies and simplified by its elimination of conlplex
error handling. The error rates have also proven to be
as predicted. With over 100 units in use over the last
year, we have observed only a very small number of
errors other than those attributable to debugging new
\iersions of thc 11a~ad ware.
1)igiril Teclillical Journal
Vol. 8 No. 1
1996
15
Detailed component performance measurements
are still in progress, but rough comparisons of DRD
against local 1/0 have shown n o significant penalty in
latency o r throughput. There is of course additional
CPU cost, but it has n o t proven t o be significant for
real applications. DLM costs are comparable to VMS
and thus meet o u r goals. Audited TPC-C results with
the Oracle database also validated both our design
approach and the implementation details by showing
that database performance and scaling with additional
cluster nodes meet our expectations.
The previous best reported TPC-C numbers were
20,915 tpmC o n Tandem Computers' Himalaya
K10000-112 system with the proprietary Nonstop
SQL/MP database software. TIie best reported numbers with open database software were 11,456 tpmC
o n the Digital Alphaserver 8400 5/350 with Oracle7
version 7.3. A four-node Alphaserver 8 4 0 0 5/350
cluster with Oracle Parallel Server was recently audited
at 30,390 tprnC. This represents industry-leadership
performance with nonproprietary database sofware.
Future Developments
We will continue t o evolve the Tri~Clusterproduct
toward a Inore scalable, more general computing environment. I n particular, we will emphasize distributed
file systems, configuration flexibility, management
tools, and a single-system view for both internal and
client applications. Work is under way for a cluster file
system with local node semantics across the cluster systern. The new cluster file systelii will not replace D 1 D
but will compleme~itit, giving applications the choice
o f raw access through DRD or fi~ll,local-file-syste~ii
semantics. We are also lifting tlie four-node limitation
and allowing more flexibility in cluster interconnect
and storage configurations. A single network address
for tlie cluster system is a priority. Finally, further steps
in managing a multinode system as a single system will
become even more important as the scale of cluster
systems increases.
Further in the future is a true single-syste~iiview o f
cluster systems that will transparently extend all
process control, communication, and synchronization
mechanisms across the entire cluster. An implicit transparency requirement is performance.
Acknowledgments
I n addition t o the authors, the following individuals
contributed directly to the cluster components
described in this paper: Tim Burke, Charlie Briggs,
Dave Cherkus, and Maria Vella for DRD; Joe Amato
and Mitch Condplis for DLM; and Ali Rafieymehr for
IMEMOKY CHANNEL. Hai Huang, Jane Lawler, and
16
Digital Technical J o u m i ~ ~
Vo1.8 No. 1
1996
especially project leader Brian Stevens made many
direct and indirect contributions to the project.
Thanlts also to Dick Buttlar for his editing assistance.
References and Notes
1. "Introduction to DCE," OSFDCE Docurnenlalion Set
(Carnbridgc,&lass.:Open Sohvare Foundation, 1991).
2. Internet RFCs 1014, 1057, and 1094 describe ONC
XDR, RPC, and NFS protocols, rcspectivcly.
3. G. Pfister, In Search of Clusten- (Upper Saddle River,
N.J.: Prentice-HJI, Inc., 1995): 19-26.
4. N. ICronenberg, H. Lcvy, and W. Strcckcr, "VAXclustcrs:
A Closely-Coupled Distributed System," AChJ Transactions on Computer Systems, vol. 4, no. 2 (May
1986): 130-146.
5. L. Cohcn and J. Williams, "Technical Dcscription of
the DECsafe Available Server Environment," Digital
7echnical.Jo~~rr~a1,
vol. 7, no. 4 (1995): 89-100.
6 . TPC performance numbers for UNIX systems are typically reportcd for databases ~rsi~lg
tlic character devicc
interface.
7. The file system interfaces on the Digital UNIX operating system arc being extended to support direct 1/0,
which results in bypassing the block buffer caclic and
reducing code path length for those applications that
do not benctit from use oftlic cache.
8. A fast \vide differential (FMID) SCSl bus is limited to
n maximum distance of about 25 meters for example.
9. M. Devarakonda ct al., "Evaluation of Design Altcrnatives for a Cluster File System," LISEVIX Conference
Proceed~ngs,USENIX Association, lierkeley, Calif.
(January 1995).
10. J . Gray and A. Reuter, Trans~lclionProcessingC,o?zc~q/.s and Tecbniqr.les (Sari iMateo, Calif.:
Morgan Kaufinan Publishers, 1993).
11. This mcchanism is inherited fi-om the DECsafc Available Server management facility, including the asemgr
interface.
12. As an esample, if tlic first DK1) servicc created for a
cluster is 1, the DlW device special file name is
/dev/drd/drd 1 and its minor device number is also 1.
13. C. Juszczak, "Improving the Perfori~ianccand Correctness of an NFS Server," USE.VIXXConference Proceedings, USENIX Association, San Dicgo, Calif.
(\Vinter 1989).
14. W. Snaman, Jr. and D. Thiel, "The VAX/VMS DistribTechrzrcal Jo~orlrnnl,
uted Lock Manager," D~~q~lwl
vol. 1,110. 5 (September 1987): 29-44.
15. R. Goldrnbcrg, L. Iknali, and D. Dumas, V A Y V M S
Interrzals and Dat~z Stmlclr~res(Bcdford, Mass.:
Digital Prcss, 1991).
16 . TniC11I.$/vI.A/)/di~~l/iol?
P~.o,gtzrtnmirigIntrr/ilcc~.s
G ~ t i ~(1\/13!111;11.d,
lc
mass.: Digital F.quipnient Corporation, Order No. AA-QL8PA-TE, 1996).
17. T. Rengarajan, P.Spiro, and W. Wright, "High Avail.lbilin Mccha~~isrns
of\lAS DBMS Sobvare," Digilul
Tech~lical,/orit~t~nl.
vol. 1 , no. 8 (February 1989):
88-98,
(Fort Laud18. Encor.c~91 .Sct-ics Techrrical Sl~r~rtrzary
erdalc, F'la.: t.,~icorc Computer Corporation, 1991).
Biographies
William E. Snanian, Jr.
Sandy Snam;in joincd Digital in 1980. Hc is c~ls~.cntly
a
consulting sohvare engineer in lligiral's UNIX Sofnvare
Group, wherc lie contributed t o tlie 'I'ruCluster architccture and design. H e and members of his group designed
and implemented cluster components such as the connection nianagcr, lock manager, and various aspccts of
cluster con~~nunications.
Previouslv, in the VMS Enginecring Group, lie \\{astlie project lender k)r the port of tlie
VMSclustcr systcnl to thc Alpha platform and tlic technical
supervisor and project leader for the VAXclustcr csccutivc
area. Sandy also teaches MS Windows programming and
C++ at Daniel kvcbster College. H e has a B.S. in computer
science and an 1M.S. in information systems from the
University of Lo\\fcll.
Wayne M. Cardoza
LVaync Cardoza is .I senior consulting cngi~iccrin rhc
UNIX Engineering Group. H e joincd 1)igital in 1979
and contributed t o various areas of the VMS kernel prior
to joining thc UNlS (Group t o work o n the UNJX clustcr
product. kVaync \\,as also o n e o f t h e architects of PRISM,
an early Digital RISC architecturc; lie holds several patents
for tllis ~vork.More ~,cccntly,he participated in the design
of tlie Alplia AXl' ;ircllitecture dnd the Opcn\lMS port to
Alpli'~.13eforc comi~igto Digital, LV.iync w ~ employed
s
hy
Bell Laboratories. H c received a R.S.t:.F.. from Soi~theastcrn
~ M ~ ~ s s a c h ~ ~Univcrsicy
sctts
and an M.S.E.E. from IMIT.
Frederick S. Glover
Fred Glo\;cr is 3 sofnvarc consulting cnginccr and the tcchnical director of the Digital UNJX Base Operating System
Group. Since joining the Digital UNIX Group in 1985,
Fred has co~itributedt o the de\relopnicnt of networking
services, local and remote tile systems, and cluster technology. H e has scrvcd JS tile chair o f the 1 EI'F/I'SI(; Trusted
NFS \Vorking Croup, ns the chair ofrlic OSF Llistributed
File S!rsteni Working Group, and as 13iginl's representarivc
to the IEEE I'OSIX 1003.8 Tr.1nspnrelic File Access Working Group. Prior t o joining Dig~tal,Frcd \\,as employed by
AT&T Bell Laboratories, where his contributions included
co-de\.elopment of the R I M S nenvork communication
subsystem. H e received B.S. and M.S. dcgrees in computer
science from Ohio State University and conducted his
thesis rcscarch in the areas of bult-tolerant distributed
computing and tiara flo\\~architecturc.
IXgital Technical Journal
Vo1. 8 N o . 1
1996
17
I
Delivering Binary Object
Modification Tools for
Program Analysis and
Optimization
Digital has developed two binary object
modification tools for program analysis and
optimization on the Digital UNIX version 4.0
operating system for the Alpha platform. The
technology originated from research performed
at Digital's Western Research Laboratory. The
OM object modification tool is a transformation tool that focuses on postlink optimizations.
OM can apply powerful intermodule and interlanguage optimizations, even to routines in system libraries. Atom, an analysis tool with object
modification, provides a flexible framework for
customizing the transformation process to analyze a program. With Atom, compilation system
changes are not needed to create both simple
and sophisticated tools to directly diagnose or
debug application-specific performance problems. The linker and loader are enhanced to support Atom. The optimizations OM performs can
be driven from performance data generated
with the Atom-based pixie tool. Applying OM
and Atom to commercial applications provided
performance improvements of up to 15 percent.
Vol. 8 No. 1
1996
Linda S. Wilson
Craig A. Neth
Michael J. Rickabaugh
Historically on UNIX systems, optimization and program analysis tools have been i~nplenientedprimarily
in the realm of compilers and run-time libraries. Such
implementations have several drac\!baclcs, Iio\vever.
For example, although thc optimizatio~isperformed
by compilers are effective, typicall!: they are limited to
those that can be performed \vithin the scope of a single source file. At best, the con~pilercall optimize the
set of files presented during one compilation run.
Even the most sophisticated systems that save iritermediatc representations usually cannot perform optimizations of calls to routines in system libraries or
other libraries for which no source o r intermediate
forms arc available.'
The traditional UNIX performance anal!rsis tools,
prof and gprof, require compiler support for inserting
calls to predefined run-time library routines at the
entry to each procedure. T h e monitor routines allow
more user control over prof and gprof profiling capabilities, but their usage requires modifications to the
application source code. Because these capabilities are
implemented as compilation options, users of the tools
must rccornpile or, in the case of the monitor routines,
actually rnodih their applications. For a large application, this can be an onerous requirement. Further, if
the application uses libraries for which the source is
~~ua\~ailable,
many of the analvsis capabilities are lost or
severely impaired.
Ry comparison, object modification tools can perform arbitrary transfor~nations on the exec~~table
program. The OM object modification tool is a transformation tool that focuses on postlink optimizations.
By perhrming transformations after the link step, 0 i M
can apply po\verfill intermodule and interlanguage
optimizations, even to routines in system librar~cs.
Object transformations also have benefits in the area
ofprogram analysis. Atom, an analysis tool with object
modification, provides a tlesible framework for customizing the transformation process t o analyze a program. With Atoni, compilatio~isystem changes are not
needed to develop specialized types of debugging o r
performance a~ialysistools. Application de\lelopcrs can
create both simple and sophisticated tools to directly
diagnose o r debug application-specific performance
problems.
The OM and Atom technologies originated from
rcscarch performed at Digital's Western Rcscarch
Lab (WRL) in Palo Alto, California.' The softwarc
was developed into products by the Digital UNIX
De\~cloprnent Environment (DUDE) group at
Digital's UNIX engineering site in Nashi~a, New
Hampshire. Both technologies are currently shipping
as supported products o n Digital UNIX vcrsion 4.0
for the Alpha platform."
This paper first provides tecl~nicaloverviews for
both OM and Atom. An cxample Atom tool is
presented to demonstrate ho\v to use the Atorn application programming interface (API) to develop a CLIStomized program analysis tool. Becausc OM and
Atom can be used together to enhance the effectiveness of optimizations t o application programs, the
paper includes an over\,ic\v regarding the bcncfits of
profiling-directed optimizations.
Subsequent sections discuss the product developmcnt and technology transfer process for O M and
Atom and several design decisions t l ~ a twere made.
The paper describes the working relationship between
WlU, and DUDE, tlic utilization of the technology on
Independent Sohvare Vendor (ISV) applications, and
the factors that drove tlie separate development strateg i c ~for the two products. The paper concludcs with
a discussion about areas for h r t l ~ e investigation
r
and
plans for fiiture enhancements.
Technology Origins
l<cscarcI~ersat WRL investigating postlink optimization techniques created 01M jn 1992.' Unlike compiletime optirnizers, which operate on a single file, postlink
optirnizers can operate on the entire exccutab.lc program. For instance, Oi\/I can remove procedures that
were linked into the executable but were never called,
thcrcby reducing the tcxt space required by thc program and potentially in~provingits paging behavior.'
Using the OM technology, tlie researchers f ~ ~ r t h e r
discovered that the samc binary code modif cation
techniques needed for optimizations could also be
applied t o the area o f program instrunientation. In
fact, the processes of instrumenting an existing program and generating a new esecutable could bc
encapsulated and a programmable interface provided
to drive the instrunlentation and analysis proccsscs.
Atom evolved from this work.".'
In 1993, WRI, researchers Amitabh Srivastava and
Alan Eustacc bcgan planning with DUDE engineers
to provide OM and Atom as supported products on
t l ~ cDigital UNIX operating systeni. Different product
dcvelopment and technology transfer stratcgics wcrc
used for delivering the two technologies. The scction Product Developlncnt Collsiderations discusses
thc methods used and tlic forces that influcnccd
the strategies.
Technical Overview of OM
OM performs tr.~nsformationsin three phases. It produces an intermediate representation, performs optimizations on that representation, and produces an
executable image.
Intermediate Representation
In the first phase, OM reads a specially linked input
file that is produced by the linker, parses the object
code, and produccs a11 intermcdiatc rcprcsentation
of the instructions in tlie program. The flow information and the program structure are maintained in
this reprcscntation.
Optimization
In the optirni~~lriion
phasc, OM pcrfornls \various transformations on the intermediate reprcscntation crcatcd
in the first phase. These transformations includc
Text size reductions
Data sizc reductions
Instruction and data reorganization to improve
cache behavior
Instri~ctionscheduling and peephole optimization
User-directed procedure inlining
Text Size Reductions O n e typc of text sizc rcduction
is the elimination of i~nusedroutines. Starting at the
entry point of the image, OM examines the instruction
stream for transfer-of-control instructions. OM follows each transfer ofcontrol until it finds all reachable
routines in the image. The remaining routines are
potentially ~lnrcac11'1lslcand are candidates for remo\~al.
Before removing them, OM checks all candidates for
any address references. (Such references will sho\-\~up
in the relocation entries for the symbols.) If no references exist, O M can safely remove the routine. A second type of text size reduction is the eliluination of
most GP register reloading sequences.")
Data Size Reductions 13ccause it operates on the entire
program, OM perforn~sopdmizations that con~pilers
are not able to perform. One instance is with the
addressability of global data. The general instruction
sequence for acccssing global data recluircs tlie usc of
a table of address constants (the .lita sectioll) and code
necessary for maintaining the current position in the
table. Each entry in the address constant tablc is relocated by the linker. Because O M knows the location of
all global data, it can potentially remove the address
~ i g to morc effientry \\!bile inserting and r e ~ i i o \ ~ icode
ciently refercncc the data directly. Rernovi~xgas Illany
of the .lita entries as possible leaves more room in the
data cache (D-cache) for the application's global data.
Vol. 8 No. 1
1996
14
An Example Atom Tool for Memory Debugging
5. i-,~ccute the instrumcntatioli tool 011 the target
program, providing t l ~ clinked alialysis c o d c as an
al-gulncnt.
T h e follo\\ring discussion o f 311 es'lmple Atom tool
demonstrates how t o use t h e Atoni API t o d c \ ~ e l o pa
customized program nnalysis tool.
A c o m m o n d c \ ~ c l o p m e n tproblern is locating t h e
source o f a mcmory o\ler\\~rite.Figure 2 shows a contr~\%dexample program in \\~hichtlie loop t o initialize
an army exceeds tlie array boundary and ovcr\vrites a
T h e final step produccs an instrumented program
linlteci with the analysis coclc. Figure 1 slio\\ls thc
chnngcs in memory layout b c n \ ~ c c ntlie original program and t h e i n s t r ~ ~ m e n t cprograni.
d
i
LOW
MEMORY
-
STACK
-
TEXT START ---
READ-ONLY DATA
EXCEPTION DATA
PROGRAM TEXT
INSTRUMENTED
PROGRAM TEXT
I
ANALYSIS DATA
ANALYSIS gP 1
INITIALIZED
I
I
PROGRAM
TEXT
ADDRESSES
CHANGED
NEW DATA
START
I
ANALYSIS DATA
UNlNlTlALlZED
(SET TO 0)
/
I
PROGRAM gp --, PROGRAM DATA
INITIALIZED
INITIALIZED
PROGRAM DATA
UNlNlTlALlZED
PROGRAM DATA
UNlNlTlALlZED
UNINSTRUMENTED
PROGRAM LAYOUT
INSTRUMENTED
PROGRAM LAYOUT
PROGRAM
DATA
ADDRESSES
UNCHANGED
HIGH
MEMORY
Source: A. Srivaslava and A. Eustace, "ATOM: A System for Buildlng Customized Program Analysis Tools."
Proceedings 01 the SlGPLAN '94 Conference on Programming Language Design and Implementation.
Orlando. Fla. (June 1994). This paper is also available as Digital's Western Research Laboratory
(WRL) Research Report 9412.
Figure 1
Mc~nor!. 1.ayout of I n s t r u m c ~ ~ t cProgra~iis
d
1
2
3
4
5
6
7
8
9
10
l o n g n u m b e r s C 8 1 = CO);
l o n g * p t r = numbers;
/* This pointer
i s
overwritten
*/
m a i n 0
i n t
i;
for(i=O;
i<25;
numbersCi1
i++)
/ * by t h i s a r r a y
i n i t i a l i z a t i o n .
*/
= i;
1
Figure 2
E s . ~ ~ n p lP~.ogram
c
\\.it11
1Vlc111ol-y
O\,c~.\\lrirc
LJigital Tech11ic;ll Journ.ll
\'()I.S No. 1
1996
21
pointer \~nriable.I n this case, the initialization OF the
numhcrs army defined in li~lc1 over\vritcs the contents oftlie variableptrdcfined in line 2. This type of
problenl is often diffic~~lt
and time-consuming to
locate with tr;lditional debugging tools.
Atom call be used to develop a simple tool to locate
the source of the overu,~-ite.'The tool \\/auld instruInelit each storc instruction in the program a11d pass
the effective address of tlie store instruction and the
value being stored to an analysis routine. The analysis
routine would determine if the cffcctive addrcss is tlic
address being traced and, if so, generate a diagnostic.
The instrumentation and ilnnlysjs files for tlie
mem-debug tool are sho\\/n in Figurc 3.
InstrumentInitO) registers the analysis routines with
the Atom instrumentation engine and specifies that
calls to the get-args() and open-log() routines be
inserted before the program begins esecuting. A call
to the close-log() routine is dictated when the program tcrniinatcs execution. The Atom instrumcntation cngilie calls InstrumentInit() exactly once.
l'he Atom instrumentation engine calls the
Instru~ncnt()routine once tbr each executable objcct
in the program. Tlie routine instruments each store
uistruction tliat is not a stack ~ p e r a t i owith
~ l a call to tlie
analysis routinc ~ i i e n i ~ t o r c (Ench
) . call to tlie routinc
provides the ,~ddressof the storc instruction, the target
address of tllc store instruction, the \!;llue to be stored,
and the file Iianie, p r o c e d ~ ~ rname,
e
and linc nu~nbcr.
The ope^^-log() and close-log() analysis routines arc
self-explanatory. The messages could have been \\lrittcn
to tlie standard output, because, in this example, they
would n o t hiive interfered \\*it11tlie progran~o~rtput.
Tlie get-args() r o ~ ~ t i nrc;lds
c
tlic value of thc
il/lE~\~l-DE/~l/~;-tlR~;Senvironnlcnt
variable to obtain the
data addrcss to be traced. Tlie tool could hnvc been
written to accept arguments from the comrnanci linc
i~singthe -toolargs s\vitcli. The instrumentation codc
\\rould then pass the arguments to the analysis routine.
In the case of this tool, using the environment \rnrinbIc
to pass tlic arguments is beneficial because thc program does not have to be reinstrumented to trncc a
new address.
The mem-store() routinc is called from each storc
instruction site that \\;as instru~~iented.
If the target
address of the store operation does not match the
trace addrcss, tlie routine simply returns. If there is a
match, a diagnostic is logged tllat gives information
a b o ~ l the
t location of the storc.
T o demonstrate how this tool would be used, suppose one has determined by debugging that the variable ptr is being overwritten. The nm comnlancl is
used to dcter~llinethe addrcss ofptr: as follo\\.s:
% nm -B p r o g r a m
0x000001400000c0
I
grep p t r
G ptr
Instrument tlie program \\lit11 the mem-debug tool.
22
Digital 'Tc.chlric.~l]oul.nal
% atom program mem-debug.inst.c
mem-debug.ana1.c
Set the rMIM-DEBlIG-ARC;.Ten\~ironment
variable with
the address to trace.
% s e t e n v MEM-DEBUG-ARGS
1400000~0
Run the instrumented program,
and view tlie log file.
% more program.mem-debug.log
T r a c i n g address 0x1400000c0
Address 0x1400000c0 m o d i f i e d w i t h \
v a l u e 0x16:
a t : 0x1200011c4
Procedure: main\
F i l e : program-c
Line: 9
Using this simple Atom tool, the location ofa melnor)' o\vr\\~ritecan be detected quickly The instrumented progralii executes at nearly normal speed.
Tradition~ldebugging methods to detect such errors
are mi~chmore time-consuming.
Other Tools
An arcn in \\~liichAtom capabilities have pro\?ellparticula-ly po\verli~Iis for liardu~arcmodeling and sin1~1l:ation.
Atom has been used as n teaching tool in university
courses to train students o n hardware and operating system design. moreo over, Digital hardware designers have
de\.eloped sopllisticated Atom tools to evaluate designs
for nc\v implementations of the Alpha clip.
Thc Atom tool kit contains 10 example tools that
dcmo~lst~.ate
the capabilities of this technology. Tlie
examples i~lcludea branch prediction tool, which is
L I S ~ ~ for
L I ~compiler designers, a procedure tracing tool,
\vIiich c;ul be usehl in following the tlo\v of ~~nhnliliar
code, and a simple cache sim~~lation
tool.
Atom Tool Environments
Analysis of certain types of progra~iiscan require use of
specially designed Atom tools. For instance, to analyze
a program that uses POSIS threads, an Atom tool to
handle threads must also be designed, because the
analysis routines \ \ r i l l be called from the threads in the
~ p p l i c a t i oprogram.
~~
Thercforc, the analysis r o ~ ~ t i n c s
need t o be reentrant. They niay also need to syncliro,,izc , cccss
. . . . . to data that is shared alnorlg the threads o r
manage data for individual threads. T h e thread management in the analysis routines adds overhead t o the
execution time of the instr~~mented
program; this
o\~crheadis not necessary for a tionthreaded program.
To effecti\cly support both threaded and nontlireadcd
prograins, nvo distinct versions of the same Atom tool
need to coexist. Designers developed the concept of
tool environments t o address the issues of providing
~lii~ltiple
v~rsionsof an Atom tool.
,,
I: mem-debug-i
*
*
*
*
*
*
*
nst
-
.c
Instrumentation for
memory d e b u g g i n g t o o l
T h i s t o o l i n s t r u m e n t s e v e r y s t o r e o p e r a t i o n i n an a p p l i c a t i o n and
r e p o r t s when t h e a p p l i c a t i o n w r i t e s t o a u s e r - s p e c i f i e d a d d r e s s .
The a d d r e s s s h o u l d be an a d d r e s s i n t h e d a t a segment,
not a
stack address.
Usage:
atom program mem-debug.inst.c
mem-debug.ana1.c
I*
*
x
*
*
Initializations:
register analysis routines
define the analysis routines
program execution
t o c a l l
b e f o r e and a f t e r
*
*
*
get-args0
- r e a d s e n v i r o n m e n t v a r i a b l e MEM-DEBUG-ARGS
f o r address t o trace
open-log0
- opens t h e l o g f i l e t o r e c o r d o v e r w r i t e s t o t h e s p e c i f i e d address
close-log0
- closes the l o g f i l e a t program termination
* mem-store0 - checks i f a s t o r e i n s t r u c t i o n w r i t e s t o the s p e c i f i e d address
*I
v o i d I n s t r u m e n t I n i t ( i n t argc,
char **argv)
C
AddCallProto("get-argsOV;
A d d C a l l P r o t o ( " o p e n ~ 1 o g ~ c o n s ct h a r * ) " ) ;
AddCallProto("close~10g~
AddCallProto("mem~store(VALUE,REGV,long,const
c h a r *,const
char *,int)");
) ' I ) ;
"get-args");
AddCallProgram(ProgramBefore,
"open-Log",
AddCallProgram(ProgramBefore,
basename((char *)GetObjName(GetFirstObjO)));
AddCallProgram(ProgramAfter,
"close-Log");
If/
1
Instrument
Instrument(int
Proc
Block
Inst
i n t
/
each o b j e c t .
argc,
char
*proc;
*block;
*inst;
base;
*argvCI,
Obj
*obj)
/* base r e g i s t e r o f
memory
reference */
*
* Search
*I
for
for
a l l of
the
store
instructions i n t o the
data area.
(proc = GetFirstObjProc(obj);
proc;
proc = GetNextProc(proc)) C
f o r (block = GetFirstBLock(proc);
block;
block = GetNextBLock(block))t
for (inst = GetFirstInst(b1ock);
inst;
inst = GetNextInst(inst)) t
I*
*
*
*
*
*
*
*
*
*
I n s t r u m e n t memory r e f e r e n c e s . S k i p S s p b a s e d r e f e r e n c e s
because they reference the stack,
not the data area.
Memory r e f e r e n c e s a r e i n s t r u m e n t e d w i t h a c a l l t o t h e
mem-store
a n a l y s i s r o u t i n e . The a r g u m e n t s p a s s e d a r e
the t a r g e t address o f the s t o r e instruction,
t h e v a l u e t o be s t o r e d a t t h e t a r g e t address,
t h e PC a d d r e s s o f t h e s t o r e i n s t r u c t i o n i n t h e p r o g r a m ,
t h e p r o c e d u r e name,
f i l e name, a n d s o u r c e l i n e f o r t h e
PC a d d r e s s .
Figure 3
Instrumcntation and h a l v s i s Codc for t h e rneni-debug Tool
L>lglt~l
Technical
Journal
VoI. 8 No. 1
1996
23
(IsInstType(inst,
InstTypeStore)) C
base = G e t I n s t I n f o ( i n s t ,
InstRB);
i f ( b a s e ! = REG-SP)
C
AddCallInst(inst,
I n s t B e f o r e , "mem-store",
Ef fAddrValue,
GetInstRegEnum(inst,
InstRA),
InstPC(inst),
ProcName(proc),
ProcFi leName(proc),
(int)InstLineNo(inst));
i f
>
>
/
*
mem-debug.ana1.c
- a n a l y s i s r o u t i n e s f o r memory d e b u g g i n g
*
* U s a g e : s e t e n v MEM-DEBUG-ARGS h e x - a d d r e s s b e f o r e r u n n i n g
* the program.
* D i a g n o s t i c o u t p u t i s w r i t t e n t o program.mem-debug.log
*1
*
#include
#include
#include
#include
tool
<stdio.h>
<stdlib.h>
<string.h>
<sys/types.h>
static
FILE *log-file;
/ * O u t p u t f i l e f o r d i a g n o s t i c s *I
static
caddr-t
I*A d d r e s s
/
*
* Create
*I
trace-addr;
Log f i l e
void
open-log(const
C
char
for
char
diagnostics.
"progname)
nameC2001;
sprintf(name,
"%s.mem-debug-log",
= fopen(name,
"w");
log-file
i f
t o m o n i t o r *I
( ! Log-fi l e ) C
fprintfcstderr,
fflush(stderr);
exit(1);
"mem-debug:
progname);
Can't
create
%s\n",
name);
1
fprintf(1og-file,
fflush(1og-file);
"Tracing
address Ox%p\n\nU,
1
/
*
* Close
*I
the
log f i l e .
void
close~log(void)
C
fclose(1og-file);
>
1*
* Get
*I
address
void
get-argscvoid)
Figure 3 ( c o n t i n u e d )
t o
trace
from the
environment.
trace-addr);
C
char *addr;
i f ( ! ( a d d r = getenv("MEM-DEBUG-ARGS"))
C
f p r i n t f c s t d e r r , "mem-debug:
s e t MEM-DEBUG-ARGS
fflushcstderr);
exit(1);
t o hex address\nU);
>
trace-addr
= (caddr-t)
strtoul(addr,
0,
16);
1
/
*
*
*
*
The t a r g e t a d d r e s s i s a b o u t t o b e m o d i f i e d w i t h t h e g i v e n v a l u e .
I f t h i s i s the address being traced,
report the modification.
/
void
mem-store(
/* Address b e i n g s t o r e d i n t o * I
target-addr,
caddr-t
/ * Value being s t o r e d a t target-addr
value,
unsigned long
/ * PC o f t h i s s t o r e i n s t r u c t i o n * /
caddr-t
PC,
*proc,
/ * P r o c e d u r e name * /
const char
/ * F i l e name * /
* f i le,
const char
I * L i n e number * /
unsigned
1 in e )
C
i f (target-addr
== t r a c e - a d d r )
C
fprintf(1og-file,
" A d d r e s s Ox%p m o d i f i e d w i t h v a l u e O x % L x : \ n " ,
target-addr,
value);
fprintf(1og-file,
"\tat
: O x % p ",p c ) ;
i f ( p r o c ! = NULL) C
fprintfclog-file,"Procedure:
% s ",p r o c ) ;
i f c f i l e ! = NULL)
fprintf(1og-file,"File:
% s L i n e : %d", f i l e ,
Line);
*/
1
fprintf(Log-file,
fflush(log-file);
"\nu);
1
Figure 3 (continued)
Tool environments accom~iiodatesea~iilessintcgration of specialized vcrsions of tools into tlie Atom tool
kit. They pro\ride a mc,lns for extending the Atom Itit.
This facility allo\\rs the addition of specialized Atom
tools by Digital's layered product groups o r by cust o m ~ ~\\.hile
. ~ , maintaining a consistent user interface.
Thc vcrsions of thc At0111 tools hiprof, pixie, and
Third Degree that support POSIX threads are provided as a separate en\ironnient. hiprof is a performancc analysis tool that collects data similar to but
with more precision tllan gprof. pixie is a basic block
profiling tool. Third l k g r e e is a meliior!r leak detection tool.
The follo\\ring command invokes the Atorn-based
pixie tool for use on a nonthreaded program:
% atom program - t o o l
pixie
The follo\ving command invokes the version of the
that supports threaded programs:
pixie tool
% atom program - t o o l
p i x i e -env
Atom tool kit by selecting an cnvironmcnt name for
tlie category of tools. Similarly, tools designed to cvork
on the kernel coitld be collected into an environment.
The environ~nentname is used in the namcs of the
tool's instrumentation, analysis, and description files.
The description filc for a tool provides the names
of the instrunicntation and analysis files, as wcll as special instructions for conipiljng and linking the tool.
For example, the pixie description file for threaded
programs is named pixie.threads.dcsc. I t identifies
the threaded versions of the pixie instrun~entationand
analysis files. The Atom driver builds the name of
the description filc from the arguments to the -tool and
-env s\vitchcs on the command line. The contents
of the description file then drive the subsequent steps
of the build proccss.
Tool environments can be added \vitliout modification to the base Atom technology, thereby providing
estensibility to the tool kit cvhile maintaining a consistent interface.
threads
Tools for other specialized environments call be
provided by defining a ncw e~~vironnient
name. For
example, Atom tools \vritten to work with a languagespecitic run-time environment can be added to the
Compact Relocations
Atom inserts code into the test of the program, thus
changing the location of routines. Atom requires
that relocation information be rctaincd in the
Digital Tcchnic.~lJournal
\Iol. 8 No. 1
1996
25
eseci~tableimage created by the linker. This allows
Atom to properl!. perform relocations o n the instrumented executable.
During the normal process of linking, tlie relocation
entries stored in object files are eliminntcd once the!'
ha\~ebeen resol\led. Beca~~se
it cfkcti\,cJy relinlts tlie
cxecutable, Atom must Iiave access to the relocatioli
inforniatio~i.
Consider, for example, an application that invokes a
fi~nctionthrough a statically initialized filnction pointer
\~ariablc,as slio\\~nin the follo\\ing code scgmcnt:
void
C
foocint
a,
i n t
void (*ptr-foo)(int,
b)
int)
= foo;
void b a r 0
C
1.
2.
3.
% c c -non-shared
-0
program *.o
% atom - t o o l p i x i e program
% program-pixie
4. %
c c -non-shared
-om
-WL,-om-ireorg-feedback,program
*.o
In step 1, a nolisharcd \~crsionof the program
is built. In step 2, tlie Atom-based pixie tool instruments the program. Step 2 produces program.pixie
and program.Addrs flcs. Step 3 results i11 the csccution of the instrumented program to generate a
program.Counts file. This file contains an eseci~tion
count for eacli basic block in the program. The last
step pro\.ides the basic block profile as input to OM.
OM rearranges the text segment o f t h e program such
that the most frequently cxecuted basic blocks and
procedures are placecl in prosi~iiityt o each other, thus
improving tlie instruction cache (I-cache) hit rate.
...
(*ptr_foo)(l,2);
Product Development Considerations
J
The address of h ~ ~ c t i o n j i ~stored
o i s in tlie memory
location referenced by the j~/~:,/bovariable. When
Atoni instruments tlus application, the address of
fbo will change, asid Atom needs to knotv to update
the contents of the memory location referenced by
ptt: jix). This is possible only ifthere is a relocation rccord
l~ointj~ig
at tlus rnemor!, location. Adding compact relocations to the executable file sol\res this probleni.
Compact relocations are snialler than r c g ~ ~ l reloar
cations for nvo reasons. First, the Atoll1 system docs
not require all the information in tlie regular relocation records in order to instrument an esecutablc.
Atom changes only the layout of tlic tcst segment,
so relocation records that describe the data segments
are not needed. Second, tlie remaining relocations
can often be predicted by analyzing othcr parts of
the esecutable file. This property is used to store a
co~npactfor111 of the rer-rlaining relocntion records.
Since compact rclocatio~lrecords drc rcprcsentcd in a
different forrn than regular relocations, tiicy are stored
in the .cornliient section of the object f lc rather than
in the normal relocation area.
Profiling-directed Optimization
OM and tlie Atom-based pixie tool can interoperate
~ u i n gprofiling-directed optimization. The Atombased pixie tool is a basic block prof lcr tliat pro\lidcs
execution counts for eacli basic block \\,lie11the progmrn is ~ L I The
I ~ . execution counts are then used as
input to OM for performing optimizations 011the esecutable that are driven From actual run-time perfornlwcc data.
As an example, the follo\\,ing steps \\roulcl bc
pwforrned to utilize profiling-directed optimizations
~ l t l OM:
i
26
Digital Tcch~~icnl
Joul-nit
Bringing the OM and At0111 technologies fro111the laboratory illto use 011currelit Digital UNIX production
systems required frequent co~iimunicationand coordination benvccn WRL and DUDE engineers working
on opposite coasts of the U.S. The success of both projects depended upon cstablishi~lgand maintaining a n
atmosphere of cooperation among the engineers at tlic
t\\.olocatio~~s.
Common goals and criteria for bringing
the technology to product supported the teams during
de\.elopment and planning \vork.
Among the product development considerations
for OM and Atom \\/ere
1. The products must address a business o r customer
requirement.
2. The products must meet customer expectations of
features, usability, quality, and performance.
3. Eng~neering,q u a l ~ qassurance,
~
and doci~mentation
resources must be ~dentified t o ensure tliat thc
products coilld be enh,lnced, updated to operate
on nenr platform I-elcases, and supported througliout thcir life cyclcs
4. The products nlust be released a t the appropriate
times. lleleasing a product too early could result in
high support costs, perhaps at tlie expense of n e ~ !
de\lelopmeut. Releasing a product too late could
compro~nise1)igital's ability to le\lerage the new
technology most effccti\lely.
Product Development and Technology Transfer
Process for OM
As part of thcir research and de\relopment efforts,
WRL engineers applied OM to large applications.
Researchers and Digits engineers at ISV porting laboratories \\,orl<ed together to share information and to
diagnose the performance problems of programs in
i~seon actual production systems, such as relational
database and CAD applications. This cooperative
effort helped engineers determine the types of optimiz.ations that would benefit the broadest range of
applications. In addition, the engineers were able to
identi$ thosc optimizations that w o ~ ~ lbe
d usefill
to specific classes of applications and make them
switch-selectable through the OM interfixe. Tlie perfor~nanceimprovements achieved on ISV applications
enabled OM to meet the criteria for addressing CLIStolncr nccds.
Although WRL researchers also applied O M to tlie
S1'l-X benchmark suite to measure performance
improvements, the primary focus of the O M technology development \\!as to provide performance
impro\~enientsfor applications currently in \videspread
use by the Digital UNIX customer base. With the
focus of performance improvements on large customer applications, OM satisfied a prominent Digital
business need for inclusion in the Digital UNIX development environment.
Engineers discussed the limitation that O M did not
si~pportsharcd libraries and thc programs that used
them. In this respect, the technology would not meet
the expectations of all customers. Many ISV applications and other performance-sensitive programs, however, are built nonshared to improve execution times.
Engineers determined that the benefits for this class
of application ounveighed this limitation of OM,
and, therefore, the limitation did not prevent nloving
forward to develop the prototype into a product.
llevelopcrs recognized the risks and support costs
associated with shipping the prototype, yet again
decided that tlie proven benefits to existing applications out~vcighedthese factors.
Because of the pressing business and customer
needs for this technology, DUDE and WRL engineering concurred that O M should be providcd as a fully
supported component in Digital U N I S version 3.0.
FLIIIproduct devclop~nentcommitnie~~ts
from DUDE
engineering, docurrientation, and quality assurance
could not be made for that release, ho\vever. After
discussion, WRL provided technical support and
extensions to O M to address this necd. DU1)E engineering agreed to provide consulting support to WRL
researchers 011object file and synibol table formats nlld
on evaluations of text and data optimizations.
The next issue the engineers faced was how to integrate OM into the existing development environment.
They evaluated three approaches.
Tlie first approach was to make OM a separate tool
directly accessible to users as /usr/bin/om. Thus, an
application developer could i~tilizcOM as a separate
step during the build process. This approach offered
two advantages. First, it was similar to the approach
used to achieve the present internal use of 01M and
\vould recl~~ire
f e ~ vadditional modifications to the
Digital UNIX de\~elopmenten\~ironnient.The second
advantage was that Atom and O M could be more
easily merged into one tool since their usage would be
similar. This merging would provide the potential
efficiencies of a single stream of sources for the object
modification technology
A major disadvantage of this approach was that it
put additional burden o n tlie application developer.
O M requires a specially linked input file produced by
the linker. This intermediate input file is not a cornplete executable nor is it a pure OMAGIC file.'" This
approach ~ i o u l drequire customers t o add and debug
additional build steps t o usc OM on their applications.
The WRL and DUDE engineers agreed that the user
complexity of this approach would be a significant barrier to user acceptance of OM.
Tlie second approach was to change the conlpiler
driver to invoke OM for linking an executable. With
this approach, a switch would be added t o the compiler driver. If this switch was given, the driver would
call /usr/lib/cmplrs/cc/om
instead of the system
linker to d o the final link.
This approach had the advantage of reducing the
complexity of the user jntesL~cefor building an apylication with OM. A developer could specify one switch to
tlie compiler driver, and the driver would automatically
invoke OM. This would allow a developer to introduce
feedback-directed opti~nizationsinto the program by
sinlyly relinking with the profiling information, thus
rnalung OM easier to use and less error-prone.
The disadvantage of this second approach was that
the complex symbol resolution process in the linker
would need to be added to OM. The process of performing symbol resolution on Digital UNIX operating
systems is nontrivial. There are special rules, boundary
conditions, and constraints that the linker must understand. OM was designed to modify an already resolved
executable, and any problems introduced from adding
linker semantics would discourage its use. Also, duplicating linker capabilities in OM \\~ouldrequire additional overhead in maintaining both components.
The advantages and disadvantages of the second
approach motivated the development of a third
approach. The compiler driver could be changed to
invoke OM during a postlink optiniization step. As
in the second approach, a switch from the developer
\vould trigger the invocation of OM; however, O M
would be run after the linker had perfonned symbol
and library resolution.
The third approach is the one currently used. This
method maintains separation between tlie linking and
optimization phases. When directed by the -om switch,
Id produces a specially linlted object that will be used as
input to OM. The compiler driver supplies this object
as input to O M when the linking is completed.
Digital Technical journ,il
Vo1.8 No. 1
1996
27
The WKL anci 1)Ul)E engineers found that this
fi~nctionalscpar.~tionalso improved the cfficicnc!, of
tlie development efforts ben\~eenWRL and 1)UI)E.
[.he separation pcrmittcd concurrent WRL, dc\~clopmeut o n O M and I)Ul>E dcvcloymcnt o n Id, \\lit11
minimal interference. This approach allo\\,cd more
development timc to be dedicated to technical i s s ~ ~ e s
rather than denling \\fit11source management n ~ i i iintcgration issues.
1)UL)E engineers added the OM. soul-ccs into the
1)igital UNIS dc\,clopmcnt pool nod intcgr.~tcd
updates fro~-nWRL. WRL assumed rcsponsi bility for
testing OM prior to projtiding source ~lpdntcs.As pre\,iously outlineci, 1)Ul)E cngincers intcgt.ltcd support
h r 01M into the existing dcvelopma~tenvironment
tools for tlie initial release.
Because of pl-o\cn performance improvcmcllrs o n
ISV applications, committed engineering cfforts b!,
WIG, and testing ncti\.itics nt both I>igir~lsitcs, enginccrs judged the tcchnolop InatLlrc cnougli k)r rclensc
on production systems. Efficient de\.elop~ncntstl-ntcgjes enabled Digital to rapidly turn this leading-cdgc
c t bcncfits .in important
tcch~iologyinto a p r o d ~ ~ that
segment of the l>jgital U N I S c~lstotncrb,lse.
WKL continued engineering s ~ ~ p p o l -f0r
t OM
through the l>igital UNIS vcrsion 3.0 and 3.2 rclcascs.
Ilesponsi bility k)r the technology grniiuall!, sl~iftcd
fi.0111W l U to 1)Ul)E o \ ~ the
r coilrse of these ~.eIeascs.
C:urrcntly, L)Ul>E ffirlly supports and cnhnnccs 01M
while W l U continues to provide consultintion o n tlie
technology 2nd input for f i ~ t i ~ir~cn p r o \ ~ e ~ n c ~ i t s .
Product Development and Technology Transfer
Process for A tom
WKL deploycd c;~rly\,crsions of tllc Atom tool kit at
internal Digital sitcs, ISV porting Inbor;ltorics, and
universities, thus allo\\ling dc\.elopers to cspcl.irnc11r
\\:it11and evaluate the Atom Al'I. TIic early a\~ailnbiljn!
o f the tool kit promoted use of tlie Atom tcchnology.
User feedback and rcqucsts for features helped the
engineers to ~ilol-ccluickly and effcctivcly develop a
robust technolog\, fro111tlie prototype.
Engineers tlirougliout 1)igitnl rccognizcil Atoll1 as ;I
uniq~le and ~ ~ s c f tccl111ology.
i~l
Atom's Al'l, \\.it11
instrumentatio~~
nnd an.ilysis capabilities do\\,n to the
instruction level, increased the po\\*erand di\rcrsity of
tools tliat could bc created for software 2nd Ilard\\jarc
dc\!clopment. Hard\\jarc develop~iient tcnms ~~scci
Atom to simulate the performance of
Alpha
implcmentarions. Sofn\.are dc\*clopcrs crcntcci .~nd
shared Atom tools for debugging and measuring progrL~lii
perf'ol-msi~icc.
"l'lic \.al~leof tlie Atom tcclinolog.
in sol\ring application clc\~clopmentproblems pr.o\.iclcd
tlie business justification for developing the tcchnolo ~ illto
y a product.
The prototype \,ersion of Atom had several
limitations.
Like O M , the prototype version of Atom \vorkcd
only on nonshared applications. A production
version of Atom would r c q ~ ~ i rsupport
c
for callshilrcd programs and sharcd libr.lrics, since, by
d e t i ~ ~ lprograms
t,
are built as cnll-sharcci prograliis.
A \,iablc Atom product offering nccded to s ~ l p port these t!,llcs of progralns, in addition to nonshared programs.
Progmms ncedccl to be rclinkccl to retain relocation
i n h r ~ n a t i o before
~i
At0111coi~ldbe L I S C ~This
.
additional build step impaired the usability ofAtom.
Tlie Atom prototype performed poorly because it
consunled a 1;lrgc alnount o f ~ i ~ c m o rbIucIi
y.
of the
data collcctcd a b o l ~an
t esecr~tnblch r optimization
p ~ ~ r p o s c\\.,is
s 11ot ~iecded for 17rog1'x'n anal!'sis
transk)rm;itions.
TIie A t o ~ nAI'I rcqirired estensi\,c design and de\.elo p n c n r to support call-shared progra111s and
sharcd librnrics.
Tlic cngi~iccrsdecided to allo\\ the OM and Atom
technologies to diverge so tliat the differing requircments ti)^- optimization and program anal!rsis could be
~ n o r ccffccti\~cl!rnddrcsscd in cnch component.
Because tlic cost ofsupporting n rclcasc of the Atom
prototylx u-oi~ldlia\re been high, WRI. and 1)Ul)E
cliginccring dc\,clopcd a stratcgy for sim~~lcaneously
releasing the Atom prototype while focusing engincering efforts o n dc\rcloping tlie pl-odi~ctio~i
\ u s i o n . An
Atom Acl\s,inccci l)c\rclopment Kit (r\l)l<) \\,asrcleascd
~\vithl>igitnl UNIS version 3.0 as the initial stcp of the
strategy. The A1)K provided custonicr ;lccess to the
tcchnology \\.it11 limited support. E~lginecrs\.ic\\-cd
the Inclt ofsupport for sharcd csec~~tablc
objects as an
acceprablc li~i~itntio~i
for the Atom Al)K lmt unacceptable fix the f nal product.
In iiddition to allowing WRL and 1)UDE engineers
S p r o d ~ ~development,
ct
this ti rsr strstegic step
to ~ ) C L I OII
pcrmittcd the engineering teams Inorc timc and f l c ~ i bilin. to incrcmcntall!~;~cldsupport fix Atom illto otlier
production com~x)~ients,
such as the linker and the
londcr. For usabilin purposes, minor cxtc~isions\\.ere
made to the loader to allo\\, it to auto~nnticnllyload
i n s t r ~ ~ ~ n c nsI131.cd
t c d libruies p r o d ~ ~ c cby
d Atom tools.
The scconci stcp of the strategy \\us to provide
updated Atom kits to users as dcvdopmcnt of tlie software 1"-ogrcsscd. These kits included the source code
for csnmplc tools, manui~ls,and rcfcrc~lccpages. The
update liits pcrti)rli~cdt\\.o fi~nctio~is;
the!' supported
users nn~trllc!' pro\,ideci feedbucl< to the clc\.elop~iient
teams. 1)Ul)E and WRL engineers posted information
intcr~lally\\.itliin l>igital \\.lien kits \\,ere a\railable and
developed n m;liling list ofAtom ~~scl-s.
Tlie Atom user
community gre\\. t o include uni\.crsiries and several
cstcrnal cLIstoliicrs.
Once the Atom ADK a11d update strategy \\.ere
cstablishcd, 1lUL3E enginccring bcgan to i~mplement
support for At0111 in tlic linlccr. As mentioned carlier,
Atom inserts rest into a program and requires relocation informution to create a correctly instrumented
csccutnblc. Tlie Atom prototype required a program
to bc linked to retain the relocation intbr~nation,and
this rcquirenient prese~iteda us;ibility problem for
i~scrs.Iclcnll!~, At0111 \\,auld be ~ b l cto instri~mcntthe
cxcc~ltablesand shared libraries produced b!, default
by the linker.
I\/lodi+ing thc li~ikerto retail1 1111 traditio~ialrelocation information by dehult \\,;is not acceptable since
the size increase in the csecutablc \vould have been
In some cases, 4 0 pcrccnt of the object f le
p~.ol~ibiti\!c.
consists of relocation records. Engineers
did not \lie\\/
nn increase of that magnitude as 3 vi:lble solution. In
ncldition, tliis soliltion contlictcd \\,it11 tlie goal of
lligiral UNIX version 5.0 to reduce object file size.
As 3 compromise, D U1)E e ~ l g i ~ ~ c e r implemented
ing
c o ~ i i p ~ crelocation
t
support in the linker. Compact
relocations provided an acceptable solution since they
rcqi~ircdKir less space than regular relocation records,
typically less than 0.1 pcrccnt of the total file size.
Anothcr side effect of using compact relocations as a
solution Mas that it introduced a dependency benveen
Aton] nnd Id. All csecutablc objects to bc processed by
Arom needed to ha\~ebeen generated \\,it11 the linker
that contained compact relocation support. Therefore, to support Atom, la!,crcd product libraries and
third-party libraries needed to be relinked with tlie
compact relocation support.
In Digital UNIX version 3.0, Id \\Ins modified to
gcllcratc compact relocation information in esecurable objects. This aIlo~\~cd
At0111 to instrument the
dchult o ~ ~ t p of
u t Id. Engineers \jic\\lcd tliis capability
;IS cl-itic;ll to the usability and ultimate success of the
Arom rcchnology. The compact relocatio~~
support in
Id \\.As ~.cfincd,111d estc~ldcdoircr the course of se\zeral
1)igital UNIX releases as dc\,clopment \\,ark \virh
Atom ~xogrcssed.
(:oncurrcntl!., the \YRL. rcscarcli team expanded
and b c g n development of thc Atom Third Degree
and hiprof tools. WRL engineers also continued \\,it11
additions nnd impro\!cmcnts to .I suitc of exa~nplc
Atom tools.
After the release of Digital UNIX version 3.0,
l)Ul>E began dcsign and dc\celopmcnt of the production \.crsion of the core Atom teclinologv and the API.
1)UI)E engineers moditiecl and extended the Atom
Al'l ns tool dc~elopmcntprogrcsscci at W l U . During
peak development periods, engineers disci~sseddesign
issi~csdail!, by telephone and electronic rnail.
The original Atom A D K included the source code
for n nu~iiberof esamplc At0111 tools, Because some
of these tools contained hm-d\\,nre implementation
clcpcndencics, they \\,auld rccli~ircongoing and longterm support to remain operntionnl on changing
implementations of the Alpha architecture. For the sec;
ond s h i p ~ n e ~of
i t the Atom A1)K in Iligital UNIX version 3.2, thcse l~igll-maintcnancctools \\<ereremo\cd
and made available through unsupported channels.
Kcr\veen releases of the AI)I< on the lligital UNIX
operating s!,stem, the enginccring teams continued to
dcli\rcr i~pdatekits. Engineers scheduled delivery of
the update kirs to coincide \\,it11 key ~nilestonesin the
soft\\,,lrc development process. This strategy ga1.e
them more control o\.er the dc\~elopmentschedule
and minimized risk. The update kirs pro\lided irnmediate field test exposure for the evolving Atom sohvare.
The design, development, and kit process was practiced iteratively over a year t o develop the original
idcns into 3 fill1 product. The Atom updntc ltits \\'ere
provided for Digital UNIX version 3.0 and later sysrcms, since most users did not have access to early versions of l3igitnl U N I S \rcrsion 4.0. 17ro\/iJing Atom
kits for use o n pre-version 4.0 systems nllo\\,ed the
sofivare to be exercised in the field on actual applications prior to its initial rclcasc as a supported product.
Althoi~ghsupport for earlier operating system versions
added o\.crhead and complexity to the process of providing the update kits, the engineering tennis \~alued
the ahunciance of user feeclbnck that the process
yielded. The benefits of 11sc.r input to the so%\~are
dc\,cloprncnt process o~~t\\,cighcd
the o\,c~.licadcosts.
lluring Digital U N I S \*ersion 4.0 de\~elopment,
\YRI. engineers finalized the implementations of the
hiprof and Third Degree tools and transferred the tool
sources to DUDE for further development. The PVRL
developers had added support for threaded applications o n prc-version 4.0 lligital UNIX systems.
Because the implementation o f threads changed in
1-ersion 4.0, DU1)E engineers needed to update tlie
Arom tools accordingl!:
1)UL)E engineers also de\feloped an Ato~ii-based
pixie tool \\.ith support for threaded ;~pplications.In
KICT,the Atom-based pixie tool rcplnccd the original
vc1.sio11of pixie in Digit.11 UNlS \,usion 4.0. The
Atom-based pixie allo\ved ne\v fcatu~-ess ~ ~ cashsupport for shared libraries and thrcnds t o be more
efficiently added into the prociuct offering. Tlie develop~ilcntof an Atorn-based pixie tool sol\red the extensibility proble~lls that \vc~-c being faccd with the
original version of pixie.
WRL. engineers also began to use Atom for instrumenting prc-\.ersion 4.0 Digital UNIS Iter~~cls,
de\,eloping special tools for collecting kernel st'ltistics.
Atom \\,as extended by DUDE engineering as needed
to silpport instrumentation and analysis of the Iternel.
\'()I. 8 No. 1
1996
2'9
The Atom tool kit and example tools were shipped
with Digital UNIX version 4.0. The pixie, hiprof, and
Third Degree tools \\rere shipped with the Sofnvare
Development Environment subset of Digital UNIX
vcrsion 4.0. Research rcgarding LISC ofAtom for ker~icl
instrumentation and analysis continues.
WRL continues to share ideas and consults
with DUDE on the f ~ ~ t u directions
re
for the Atom
tecl~nologj/.
Conclusions
Developing OM into a product directly from research
proved to be challenging. Problems and issues that
needed to be addressed had to be handled within the
scliedulc constraints and pressures of a co~nmittcd
relcase plan.
In contrast, the ADK mcthod uscd t o deliver thc
Atom product allowed the Atom de\lelopers t o spend
more time o n product development issues in an environment relatively frec from the prcssures associated
\vith daily schedules. The ADI< mechanism, however,
probably limited tlie exposure of Atom technology at
some customer sites.
The close cooperation o f cngi~ieers fro111 both
research and de\lcloprnent was necessary to accomplish the goals of the two projects. Wc believe that a
collaborative development paradigm was key to successfi~llybringing research to product.
Future Directions
This paper describcs the c\wlution of the O M and
Atom technologies tl~roughthe release of the Digital
UNIX version 4.0 operating system. Digital plans t o
investigate many new and improved capabilities, somc
intended for future product releases. Plans are under
way to
Provide support in OM h r call-shared progranls
and shared librarics.
Support the use of At0111 tools on programs optimized \.c,itb OM.
Investigate providing an API to allo\v programmable, customized optimizations to be deli~~ercd
through OM.
Investigate reuse of instrumented shared librarics
by multiple call-shared programs that have been
instrumented with the same Atom tool.
Research support for Atom tools that provide systernwide and per-process analysis of shared libraries.
Extend Atom t o iniprovc kernel analysis.
Simplifj, the use of the profiling-directed optilliization facilities of Atom 2nd 01M through an
improved interface.
Vo1.8 No. l
1996
Extend the Atom tool kit to provide de\lelopment
support for thrcad-safc program arialjsis tools.
In addition to enhancements to the Atom product,
original Atom- based tools are expected to become available through the development activities of students and
educators at universities. Internal Digital developers will
continue to develop and share tools as bvell.
Acknowledgments
Many pcople contributed to the development of the
OM and Atom products. The following list gives
recognition to those most actively involved. Arnitabh
Srivastava led the research and development work at
WRL on OM and Aton1 and mediated many of the
design discussions o n the Atom design. Greg Lueck of
DUDE designed and implemented the production
version ofAtom, compact relocations, and the Atombased pixie tool. Alan Eustace developed Atom esample tools, created the first Atom ADK, worked
diligently with users, developed kerncl tools, provided
training and documentation on using Atom, and displayed eternal optimism. Kussell I<ao at WRL contributcd the hiprof tool with thread support. Jercmy
1)ion and Louis Monier at WRL developed Third
Degree and an Atom-based code coverage tool called
traclter. John Williams and Chris Clark of DUDE completcd the process of turning the hiprof, pixie and
Third Degree tools into products. Dick Buttlar providcd documentation on cvcry component. Last but
not Icast, the authors \\)is11to extend a final thanks to
all tlie users who contributed feedback to the OM and
Atom developmcnt teanls.
References
I. F. C:ho\v, M. Hirnelstcin, F,. Killian, and L. Weber,
"Gigineering a 1USC Compiler Systcm," Proceedings
(!/'COMPCOiV San Francisco, Calif. (Mdrch 1986):
132-1 37.
2. Wcstcrn Research Laboratory, locarcd o n thc Wcb :*
http://\~~~w.rc~e"'~l~.digitnl.co~~~/~~~rl.
3. I<. Sitcs and R. Wirclc, All,hcr~,YPArchilect~,o~e
I\)CJbt.once il/latz~,ral.
2d ed. (Ncwto~i,Mass.: Digital Prcss,
1995).
4. A. Srivastavn and D. WiiII, "A Practical System for
Intcrlnodulc Codc Optilriiznrion a t Link-time,"Jo~,~r17al 01' Progmnr~nirzgI.at7g1,ta~qe.v.vol. 1 ( 1993):
1-18. Also available as WIIL l<cscarch Report 92/6
(December 1992).
5 . A. Ssi\zastavn, "Unrc.~clinblc L'roccdurcs in Objccto l - ~ c ~ ~Programmilig,"
ted
AC;W I.OPIAS vol. 1, no. 4
(1)cccmber 1992): 355-364. Also available as WR1.
Rcscarch Report 93/4 (Aug~~st
1993).
". A. Eustace and A. Srj\,asrava, "ATOM: A t'lcsible
Interface for Buildi~ig High Performnncc l'rogram
Analysis Tools," I'r.oc.c~cx/ingsoJ' the Winto- 1995
l.:SB,YIX ConJerc~zce.New Orleans, La. (January
1995). Also availal)lc as WKL Technical Note '1'N-44
(July 1994).
7. A. Srivasta\la and A. Eustace, "ATOM: A System for
Building C ~ ~ s t o m i z cProgram
d
Analysis Tools," I'ro-
ceedir7gs of tbc SIGI'LAN '94 Cbnfe~x???cc
0 1 1 Progrwlnrning L a n g r ~ ~ /lesi,q~?
~ ~ q e and Ir?~ple~ner?lation.
Orlando, Fla. ( J u n c 1994). Also available as W I U
Research Report 94/2 (March 1994).
Craig A. Neth
Craig Neth is a pri~icipalsofnvare engineer in the Digital
UNIX Devclopnient Environment group, where he is the
8 . A. Srivastava and I). Wall, "Ii~lk-TimeOptimization o f
technical lcadcr of link-time tools. In prior positions a t
Address Calculation o n a 64-bit Architecture," ProDigital, Crnig has \\iorked o n the OM objcct ~nodification
c c d i n g s of the srG'I"L41v '94 Conjerence or? Protool and the VAX and 1)EC COBOL conlpilers, and led
grc~rnn?irTgLa~?gria~qc
Iksig17 and /n?ple~r~el-llntion. the develop~nento f DEC COBOL versions 1 and 2. Craig
Orlando, Fla. ( J u n c 1994). Also available as W I U
received a B.S.in computer science from Purdue University
Research Report 94/1 (February 1994).
in 1984 and an M.S. in computer science fro111the
Universin of Illinois in 1986.
9 . Digital LBVIX Calling Slarl~lardfor Alpha S,atc~lns.
Order No. M-I'YSAC-TE, Digital UNIX vcrsion 4 . 0
o r higher (Maynard, ILIass.: 1)igital Equipment Corporation, 1996).
J.ongt.iage Pro~rarr?n?ers
10. Digital UNM As.scnrh1~~
Guide, Order No. AA-PS3IC-T€, Digital UNlX vcrsion 4.0 o r higher (Maynard, Mass.: Digital Equipment Corporation, 1996).
General Reference
J . Larus and E. Schnarr, "EEL: Mdchine-Independent Exccutablc Editing," SI(;I'LAN Corderence on Prograrnm~ng
Lun~lrageDesign a r ~ dI1nplc~117erztafion
(June 1995).
Biographies
Linda S . Wilson
As a principal soft\varc engineer in the Digital UNIX
1)cvelopmcnr E ~ l \ ~ i r o n ~ ngroup,
c n r Linda Wilson leads
the development o f program analysis tools for the Digital
UNIX operating system. I n prior positions, she \jJas responsible for the deli\,ery of other dc\,clopnient e n \~' ~ r o n m c n t
components, including I)EC FUSE, the dbx dcbuggcr,
and run-time libraries on the ULTRlX and Digital U N I S
operating systems. Linda rccei\.cd a B.S. in C O I , I ~ L ter
I
science from the University ofNebraska-l,incoln. Before
joining Digital in 1989, L ~ n d aheld sofnvare engineering
positions at Massco~npin Westford, Massachusetts, and
Texas Instrunlents in Austin, Texas.
Michael J. Rickabaugh
Michael Rickabaugh is a principal sohvare engineer i ~ x
the Digital UNIX De\reloprnent Enviro~lnlentgroup.
H e started his Digital carecr in 1 9 8 6 in the SEG/<:AD
Engineering group as a soFn\.are engineer on the DECSIM
logic simulation project. I n 1991, Michael transitioned
to the DEC OSF/1 AXP project and \\.as a ~nernbcrof
the original tcnm responsible for delivering thc UNIX
de\~elopmentenvironrncnt o n the DEC OSF/1 Alpha
platform. H e has since been a technical contributor to
all aspects of the Digital UNIX link-time technology
as \veil as the creator of the ASAXP assembler k)r the
Windows N T operating system. 1MicI1at.l rccci\ui a B.S.
in clecrric,il and computer engineering korn Carncgic
~MellonUniversity in 1986.
John T. Freitas
James G. Peterson
Scot A. Aurenz
Charles P. Guldenschuh
Paul J. R m a u r o
Design of excursion
Version 2 for Windows,
Windows NT, and
Windows 95
Version 2 of the excursion product is a complete
rewrite of the successful Windows-based X
server software package. Based on release 6
of the X Window System version 11 protocol,
the new product runs on Microsoft's Windows,
Windows NT, and Windows 95 operating systems. The X server is one of several components
that compose this package. The other components are X Image Extension, the control panel
(which constitutes the user interface for product
configuration), the error logger, the application
launcher, and the setup program. An interprocess
communication facility enables the excursion
components to communicate in a uniform fashion under all three operating systems. A unique
server design using object-oriented programming techniques integrates the X graphics context with the Windows device context into a
combined state management facility. The resulting implementation maximized graphics performance while conserving Windows resources,
which are in limited supply under the 16-bit
version of the Windows operating system. The
control panel was coded completely in the C++
programming language, thus making full use
of the Microsoft Foundation Class library to
minimize development time and to ensure
consistency with the Windows user interface
paradigm.
Digital developed the eXcursion family of display
server products t o provide interopcrability between
desktop personal computers (PCs) running the
Microsoft Windo\vs operating s!lstcm and remote
hosts running the X Window Systcm operating system
under the U N I S or OpenVMS operating systems. The
first version of tlie excursion X server was a 16-bit
application written specifically for Microsoft Windo\vs
versio~is3.0 and 3.1. As the pop~~larity
of Windows
increased and desktop systems were connected to corporatc nctworlts, the marltet for S interoperability
grew quickly. The 16-bit excursion code, milch of
urhich had been ported from 32- bit U N I S code, \vas
again portccl-this time to Microsoft's Win32 application prograinming interface (API) to support the
Windo\vs N.1' operating svstern. Whcn release 6 of
the S Windon) System version 11 protocol (Sll R 6 )
appeared and a new sample implemcntntion source
l i t bcci~nlcavailable ti-om the X Consortiuni, the
excursion team decided that it \\,as t i ~ n cfor a completc
rewrite o f the excursio~l software. Microsoft had
established the \Win32 API as n uniform coding interbcc for all its Windo\vs-based operating systems. Since
development tools such as 32-bit co~npilers and
debuggers of sufficient quality and robustness had
becorne available, it \\.as now possible to implcme~ita
high-qunlity, 32-bit product.
p r o d ~ ~\cvto ~ ~supld
port the entire range of Windows-based platforms,
from notebook PCs running the Windo\vs operating
system to high-end Alpha systems running the
Windo\\;s NT operating s!lstem.
Terminology
This paper incorporates certain conventions to clarify
the distinction b e m ~ e ethe
~ ~ two window systems
under consideration. Xiilin~loujrefcrs to the collection
of data structures, concepts, and operations that cons t i t ~ ~ar e\\lindo\v, as defined in the X Window System
elivironnient. LL'ir132 il:i~dolllrefcrs to 3 \\,indo\\. as
defined in Microsoft's Win32 API.
When referring to a \vindo\v systcni as opposed to
a particular \viildo\\l instance, X IVi/il?~lorr~
S~fetn
is somcti~ncsabbreviated t o X . W/irrclorts denotes
tlie Microsoti Windows operating system.
Vol. 8 No.1
1996
Note that the \\lord h ~ r m ~has
~ pmore than one
meaning. In the X en\.ironment, a bitmap is a nvodimensional array of bits, and a ~ ) I A ~ I ~ I is
L I /a> twodimcnsio~ldl array of pixels, \i~lierceacl~piscl may
consist of one or more bits. Under the Win32 AI'I, t l ~ c
terln bitmap is used exclusively; that is, no distinction
is ~ n a d cbcnveen an array of depth 1 and nn arra!l of
dcptll 1 1 I n this paper, the term pix~napis used in
its general scnsc to refel- to X pixel arrays, and tlie
term bitmap refers to the Win32 concept.
Another conllnon point of confi~sion\\~licndiscussing the X Window System en\,ironmcnt is the use
of the terms .scr.oc.r.and clic.rzr.To one familiar \\fit11filc
anci print scrircrs, the meanings of tl~csctwo terms in
mdy seem to be rcvcrscd. I n the X
tlic S c~l\~iron~iient
cn\~ironnicnt,the serI7er is a display scrvcr, and the
clients arc the applications requesting display scr\,iccs.
The S server and tl.le X client applications may reside
on the s ~ ~ PC,
n c b ~ the
~ po\\!er
t
of the eXcursion software is in its ability t o bridge tlic gap bchvccn the
Windo\vs desktop and the traditioiial S 1 1 UNIX and
OpcnVMS workstations.
excursion Version 2 Product Goals
The design of cXcursion version 2 \\!as driven primarily
by the Follo\ving product goals:
S t ~ p ~ x )Xr tWindo\\, System version 11, release 6.
S u p ~ m r tthe Microsoft \/Vindo\\~s,Windo\\!s NT,
and Windows 95 operating systems.
<;ode tlic single source pool t o Microsofi's Win32
API.
Excccd gr.\phics performance ofcXcursion vcrsion
1 ns mcusured with the standard benchmarl< tests
XI lperfand Xbench.
Preserve maintainabiliy by using modular coding
and limiting changes of the saniple implen~entation
fi-om the S <:onsortium.
Mnsimize reliability by performing cstended error
chccking and resource managclnent.
C:orrcct kno\vn protocol co~~formance
deticiencies
1. For example, in version 1 , pl'inc mask
in \icrsio~~
s ~ ~ p p o\\,as
r t iinpleniented For o n l y '1 feu) graphics
operations. Version 2 would provide plane mask
support for all graphics operations.
Components of excursion Version 2
In cXc~~rsion
v e r s i o ~1,~ most of the fi~nctionsprovjdeci
by the product \iferecombined in a single executable.
T o c o n s ~ r \ ~resources
c
and t o partition the codc for
easier maintenance, version 2 is di\iidcd into several
separate colnponents or modules. Some of tllese run
as individual proccsscs, and some arc built as dynamic
. DLL is a shared memory
link libraries ( 1 I l . l ~ )A
library module that is linked to the calling program at
run time.
eXcursion version 2 is piirtitioned into the follo\\,ing
major components:
X server. l'he X scr\>cris tlie primary component of
eXcursion version 2. The X scrvcr process is responsible for displaying windows and graphics o n the
Windows desktop and for sending keyboard,
mouse, and other c \ ~ c ~ ito
t s the client application.
X Image Extension. X extensions are additions to
the server that support fi~nctionalin~
not addressed
by the core X11 protocol, such as displaying shaped
(nonrectangular) cvindo\\(s, handling large requests,
testing/rccording, and imaging. All estensions
except the X Image Estension (XIE) are implcmented internally in the S ser\*er. Because of its
size, XIE is iniplcnlwltcd as a pair of DLLs, one for
XIE \,ersion 3 and one for SIE version 5.
C o ~ ~ t r opancl.
l
As tllc primary user interface, tlic
control panel provides the user \\lit11 access to tlie
many configuration settings. It is an independent
Win32 application implcmcnted using Microsoft
Vis~ialC++ and the Microsofi Foundation Class
(MFC) li b~.ary.
I~lterprcxcss aco~nrnunication library. The interprocess communication (IPC) library is an operaring system-indcpendcnt library used by cooperating
processes or tasks to communicate configuration
a i d status information.
Error logger. The error logger is a simple Win32
application that records error and status information from othcr cXcursion components in a window, a file, or the Windoivs NT event log.
Application launcl~cr.The application lau~ichcris a
Win32 application that starts X client applications
at the request of the X server o r the control panel.
The application launcher is i~l\isiblcto the user.
1Zegistry interface. The registry interface is an
operating s!!stcm-indcpc~icic~it interface to the
excursion co~lfigi~ration
profile. The registry interface is implemented as a Win32 DLL.
X Server
T h e core of the eXcursio~lproduct is the X server, a
Win32 application that accepts X reqilcsts from clicnt
applications and transforms them iiito graphics on the
Windo\vs desktop. The device-independent portion of
the server code is ported from the sample implemcntation provided by the X Consortiuni. The devicedependent portion treats the Win32 API as the device
interface t h r o ~ ~ g\ilhich
h
clicnt rcqilests are materialized on the screen. ?'he eScursio11 S server is illustrated in Figure 1.
Digital Tecl~~~ic~~l
Jo\11~11,1l
\lo1 8 No. 1
1990
3.5
I
INTERNAL WINDOW MANAGER
X REQUESTS
NETWORK
DEVICE-INDEPENDENT
X EVENTS
TRANSPORT
CODE
I
I
WIN32 FUNCTION CALLS
DEVICE-DEPENDENT
CODE
t
WIN32 MESSAGES
Figure 1
Tlic cSc~~rsion
X Scr\~cr
.. .
l'llc scr\,cr can operate in one oFt\\,o~nodcs:singlc\\<indo\\.~nocie or rnulti\vindo\\ moric. I n single\\.indo\v mode, the server creates one Win32 \\indo\\;
\\,hich reprcscnts the X root \vinclo\\.. All descendant
\\,incio\\rsand their contents re ci1..1\\'11ill to t l ~ croot
n 111m~~lti\\,indo\\,
\\,indo\\, i sing Win32 f ~ ~ n c t i ocnlls.
mode, tlie root \vindo\v is a \irtt~aI\\.indo\v; t11at is, it is
never ciraivn on the screen. Each top-lc\vcl child of the
root \\?indo\\,Iias a corl-esponding Win32 indo\ do\\;
\vIiich is created \\+en the X \\!indo\\$is mnppeci. All
dcsccndants of a top-lc\,cl \\'indo\\! arc drnwn inside
the Win32 \\?indow \\,it11 Win32 cnlls. iMulti\\,indo\v
~ n o d cthereby creates a desktop enviro~lmcntin \vhich
S ,ipplications are peers of other Win32 npplicntio~~s.
Singlc-\\-indo\v mode is usef~ltbr emulating J complctc \\rorkstation environment including the \\.indo\v
mnrlagcr ;ind the scssio~lor dcsl<topmanager. In multi\\inclo\\.mode, cirn\\ring to nnd getting input from tlie
root \vindo\\. is restricted b!~tllc S scr\,cl to p r - ~ \ ~ c n t
conflicts with the Microsoft Windo\\,s system's L I S ~
of the desktop \\,indo\\,. Despite this restriction, tlie
~nnlti\vindo\\,mode, \\,he11~ ~ s \\,ith
c d rllc llnti\,c \\,indo\\, manager, pro\lides the clca~~cst
i~itcgrationof the
S and Windo\\~senvironments.
Resource Management and Performance
Both tlic S ,ind bVin32 systelns Ii,~\,c\>uilt-in lotions
ofgr~pliicsstate and resource allocation. Tlic scmantics and usage of the concept, lio\\c\.cr, arc quite difftrc~i
t i l l the n\,o n d o \ \ systems.
In X, gr~phicsstate is mnintnincd in n ci.ira 5tr~1ctu1.c
kno\\*nas a graphics contest (G<:). A (;<:has an indcpe~idcntexistence and may be created, destroyed,
[~pdnrcd,q ~ ~ e r i cand
d , copied at \\,ill hy the X application. During graphics operations, .I G<: is nssociatcti
with the X "drawable" (\vi~tdo\\:or pismnp) being
drawn into, and information in the G(: is used to f ~ l l y
dcf nc the operation. For example, the C;(: may specie
foreground o r backgroc~ndcolors, line snlcs, or font
information.
The Win32 API has a concept cnllcd a dc\.icc co11text (I)(:), \vhich also contains state infor~ii.itionbut
\\rliosc PLII.POX
is niorecI(~seI!~related to pro\.idirig
device illdependence. Conscq~~cnrly,t\\.o diffcrcnt
t y l x s o f LXs are required under the Win32 API,
\\xi
depcnciin~o n \\.licrhcr the graphics operation is dl-a\\.ing to n \\rincio\\.or to n bitmap. Furthermore, .I \\.indo\v DC may be ;~llocatedeither permanentl!, o r kom
a cache, depending o n its cspccted lifetime. Any dm\\.ing operation t11crck)rc rccluires that both the (;(:
uscd in the S grnpliics ~ . c q ~ ~and
c s ttlie DC ~1sei1
in rhc
ulti~natcWin32 call be properly set ~ r pand synclll-onized. The m;lnncr in \\.liicli this is done has a signiticant effect o n the graphics performance of the ser\,er.
Before an X graphics opcrntion can be started, tlic
GC must be \,alidntcti. Validatio~l is a process of
preparing the o u t p ~ l tdevice to render the gr.lphics
properly. I n the c;lsc of the eScursion server, the outp u t dc\.ic.c is n Win32 I)<:. For every graphics command, the C;<: mllst be cl~cckcdfor changes and the
appropriate Win32 objects and state \-ali~esmilst bc
selected illto the 1)C:. This process can be very timcco~lsr~rni~ig.
The kc!, to maximizing performance is to
rccognizc tlint most oper.1tions arc rcpetiti\~e.A typic.ll
stream of S rcqucsts tends to contain man!' comm.lllds
directed nt the same \vindo\v \\lit11 the same (X:.
Therefore, the \\,3y to reduce GC/DC \.alidation timc
is to cncllc the most rcccllt G(:/L'>C pair so that si~bscquent commands t113t use tlie same combination nccti
not trigger 3 validation step. In some cases, graphics
opcrations \\.ill toggle kct\\,ccn two or rnorc GCs. (For
esample, the (:op!:A~.ca opcr~ltiontakes a source a n d :I
destination.) The pcrfi)r~nanccjn these cases c'ln be
inipro\,cd b!. simpl!. caching Inore than one recclit
GC/DC pair. Tuning cspcl-ime~ltson the ser\.c.r
re\,ealcci tl1;lt 3 c;~c[lcsize bcn\reen 2 and 4 \\-as sufficient to mnsimizc pc~.fornin~~ct.
Under tllc WillJo\\.s
and Windo\\.s 95 operating systems, \\,here resources
are limited, n cachc size of 2 is used. Under the
\Vindo\\zs NT opcr,iting system, the cachc size is 4.
In the cScul.sion scl.vcr, the notion of n c~clicd
GC/D(; pair is cncupsulatcd in a C++ class wlled ,I
WSDC. The WSlX: remembers the Win32 objects thnt
have been sclcctcd into the DC and the last GC that \\,as
~lscd\\,it11it. As long .IS these elements d o not c11,lngc
from one gr;lpliics o p c r ~ t i o tc)
~ i thc ~ ~ e s110
t , \*nlidntion
is necessary. If the clic~ltappliciltion changes the conccnts of the (;<:,any affected objects in the 1)C nrc
tagged and the ~ i c sglapliics
t
opelation on that WSIX:
\\.ill recluirc nc\\. o0jccrs to be selected into tlic DC.
E\.cnts in the \\,indo\\ system can also c;li~seW X l X
clcmcnts to become in\.alid. For csaniplc, if tlic \\.indo\\. is ~no\rcdo n the sc1-cc11by the \\!indo\\, nianager,
its clip list may lia\lc clinngcd. -1'his causes the WXlX;
to in\rnlidatc the clip rcgion in its 1)C. (Clip list and
rcgion arc defined in tlic fi)llo\ving section.) The nest
graphics operation o n tli;lt \\lindo\\r \\?ill rccluire the
clip rcgion to be rccnlcu1;ltcd and reloaded.
Clipping in Single- window Mode
In tlic X Windo\\, Systc~iicnvironnlent, a11 rlcsccntialits o f t h e root \vinJo\\, li;i\,c ,I clip list, which is a list
of rectangles that dcfines tlic \iisible area of thc \\,illdo\\/. Tlic clip list is equal to the area of tlie child
\vindo\v mini~sany arcas that are occluded by otIic1X \\,inclo\\'s. Before dr.l\\.ilig illto a descciidai~t\\.illclo\v, the server mList con\,crt the clip list into ;I Win32
rcgion. 111the Win32 ,\PI, a region is a polygonal nrca,
iiot ncccssaril!~ ~.ectnngular,thnt 5311 be selected into
;I I)(: for clipping. 13ct?)1.cinitiating a gr~pllicso ~ i t piit operation, the tilrgct WSD<; checks to see if the
current rcgion for tlic windo\\r is valid. Ifit is not, the
S clip list is con\lct.tcd to a Win32 rcgion and combined \\.it11 the client-supplied clip list in the C<:, if
any. The result is sclcctcd into the output 1)C.
Clipping in Multiwindow Mode
In ~nulti\\.indo\\,~ n o d c rhc
, root \\.indow is invisible.
Each top-lc\~elX \\,indo\\,(first-generation cliilcl of tlic
root) corresponcls to a Win32 \vinclo\\l on the clcsktop.
N o clipping is necessary for thcsc \vindo\\.s, because
Will32 does tliis ,iuromLiticall!~.For \vindo\vs belo\\.
tlie tirst generation, clipping is nccomplishcti in a manner similar to that L I S C i~l l si~iglc-\\.i~ldo\~
node, csccpt
t11,lt tlic offset of tlic clip region must be adjusted to be
~.cl,~ti\,c
to the top-lc\,el \\,incio\\ instead of relati\,c to
tlic root \vindo\\:
Graphics Rendering
(;I-:lpliics rendering is ;lt the heart of the S scr\Icr. 'S\\~o
of thc core go;lls for tlic eXcursion version 2 projcct
\\,ere t o significnntly impro\,c server performclnec o\,cr
tli.it of the eScursion vcrsion 1 senrer and to i~npro\,e
scr\.cr conipliancc to the S protocol specjfcation.
Figure 2 colnpnrcs the performance of the eScursion
\,crsion 2 ser\.er \\'itti tll;lr of the \ u s i o n 1 sc~-\.c~..
Tlic
s ~ i ~ l d a rbenchmark
d
tests S 1lpcrf and Xbcnch \\'ere
run over a local arcn ~iet\\,orkto eXcursion run~ling
o n n 66-meg,lIicrtz I'c~iti~~rn
processor \\gitli ; I I ~S3
video card.
Tlic sample X scr\,cr ~ ~ p o\\,liich
n
tlie excursion X
scr\.ci- is based pro\'idcs .I m~cliinc-indepe1ic1c1lt
Ia!zcr
that is capable of rcnciering all X graphics tliro~~gli
.I
small set of dcvicc-dcpcndcnt fi~nctions. In the
cXcursion S scr\,er, tlic Win32 fi~nctionspro\ridc the
\irtu.ll Iiard\\,arc intcrLicc. For masimuln pcrti)~-rnnncc, X graphics rcclucsts are passed to the Win32
(I)
W
z
(I)
W
Z
eB ef?
z
l
F
4
X
x
x
W
LL
PERFORMANCE BENCHMARK
Performance
Benchmark
excursion excursion
Version 1 Version 2 Improvement
XBench
linestones
135,735
239,740
38,083
74,331
blitStones
59,743
88,320
arcstones
2,172,720 3,662,770
textstones
156,190
214,762
complexStones
71,633
71,699
XStones
80,057
126,408
XI I perf
Xmark
1.6495
2.5805
f i l Istones
76.6%
95.2%
47.8%
68.6%
37.5%
0.1 %
57.9%
56.4%
Notes:
The test machine was a DECpc XL 566.
Since excursion version 1 did not support 16-bit fonts, the version 2
numbers were substituted t o obtain the Xmark number.
Figure 2
Co~iipwisono f sXc~~rsion
\rcrsion 1 :lnd \krsion 2
Pcrform;lncc
API as early 11s possible \\,ithout compromising the
requested rcnderilig. Many X graphics 1.cc1ucsts ]nap
neatly into Win32 calls \\,it11 little or n o data 11ianip~1lation. Some complex grapliics rcclucsts, Iio\\,c\.cr,
cannot bc pr,lcticnll!, mapped into liigli-lc\,cl CVin32
calls and acliicvc proper piselization. 111such cases, the
machinc-indcpc~icic~it
f ~ n c t i o n sarc callcd as helper
fi~nctionsto brcnk tlie recl~lestdo\\fn into simpler
graphics rcclucsts.
GDI Context Switching T o reduce contest s\\,itcliing,
Windoivs batclics g ~ ~ p l i i cdevice
s
il.itc~.Kicc(G1)I)
cnlls. The dcL2ult G1)1 batch size is 20, but this limit
can be ndji~stedper thread. Testins \vitli a mix of all S
requests sl~o\\.cdtliat an o\,erall pcrformancc increase
of about 9 percent c o ~ ~be
l d achieved by increasing the
GDI batch limit to 30. At tliis level, tlicrc is rlo measi~rablelatency, a ~ i dfi~rtliermore,
,
iricrcasing tlic batch
size beyond tliis point had n o mcnsurahlc benefit.
Sonie competing X server products set the batch size
very high (100) at the bcgi~iningofc\,cry rcqucst and
flush the queue at tlle end. This appro1c.Ii I 13s n o mcasurable benefit over 0111- simpler method, probably
because the Windows operating system nlrcndy pcrforms tinier-based flushing to prevent dra\ving Intc~lcy.
Similarly, ~\~henevcr
possible, Win32 graphics calls
are combined to rcclucc the overhead of contest
s\vitcl~ing.For example, an S PolyLine rcclucst could
be rendered \\.it11 a series of Win32 1,incl'o c,ills,
but it is r n ~ ~ cmore
li
cfficicnt to render the I'olyLine
recluest with 3 single Win32 PolyLine call. Siniilarl!r, a
I'olyRectangle X request is best rendered wit11 a singlc
Pol!!PolyLine c ~ l l .
Solid Fills Many different Win32 resources s ~ ~ cash
pens, brt~shcs,fonts, and clip regions ma!, be rcquircd
for any given graphics request. The resources nccdcd
are determined by tlie graphics operation itself nnd the
state of the X GC. As ~iotcdearlier, these resources arc
cre'~tedas needed and ~n'ln.~gcd
by the WX1)C objects,
rcmoving significant complexity a ~ l dnearly redundant
code from the ncti~algraphics drawing routines.
Mlindo\vs Pen structures provide color rind dnsh
pattern mihen dra\ving line objects. For Jra\\ring lines,
segments, and arcs, the S scrver creates and LIKS l'ens
that correspond to the GC state. I n some cases, ho\\.ever, exact pjsclization cannot be acliie\,ed \vlicn usjng
Windows Pelis. Esamples of this ase dra\\ling \vide
lines with raster operations other than (;>;copy or
\vith long, dash patterns. In these cascs, machineindependent functions arc ~ ~ s etodreduce the rcclucst
to a set of spans (single-lvidth horizontal lincs) to be
filled. The use of l'ens is also abandoned in special
cascs \\+en the highly optimized GDI pattcrn block
tmnsfcr (PatBlt) function can be used. PatRlt fills rcctangular regions \\,it11specified colors or patterns. It is
~
PatBlt f ~ n c t i o nt o dra\v
h e r , for example, to L I S the
vertjcal or horizontal lincs than to use tht: Windo\vs
traditional line-clr.i\\jing f~~nctions.
Windows Brush structures provide color and pattern when dracving fillcd rectangles, fillcd polygons,
and filled arcs. Again, for performance I.easons, the
l'atBlt function is often used even \\,hen tlicrc is a
higher-level fi~nctionthat seems to be a closer 1natc11.
For example, PatRlt can perform the S I'olyPoint
rcquest about 10 pcrccnt faster than Setl'ixclV, the
Windows standard call for setting single pixel values.
Similarly, PatBlt can perform the X PolyFillRect
request a b o ~ ~14t pcrccnt faster than tlie \iVinclo\\a
FillRectangIe call.
Tile and Stipple Fills An X pismap can be specified as a
piittern to be ~ ~ s e \\,hen
cl
performing fill operations.
When the pisniap is created, it is realized as n Win32
bitmap. When the pixmap has a depth greater than 1,
it is used as a color tile that will be used for thc till. If
the pismap h;is a depth of 1, it can be used as either a
transparent o r an opaquc stipple. An opaque stipple
dra\\a both tlie (X's foreground and background colors, where tlic stipple is 1 and 0 respectively. A transparent stipple is similar except that it leaves the
destination ~ ~ n t o ~ ~ \+here
c h e d thc stipple is 0.
When tlie tile or opaque stipple is 8 by 8 or smaller,
a Win32 color brush is created and cached for the
dr,i\\ring. 0 1 1the Windo\\rs NT systcm, krushcs larger
than 8 b!, S can be created, but our csperience IIJS
shown it to be slo\ver t o draw \\pith them than it is to
perform a scries of bit block transfer (BitBlt) operations from the ti.le/stipple bitmap to tlie destination.
Transparent Stipple Fills There is a Win32 fi~nction,
~MaskKlt, that seems ideally suited fix performing
tmnsparcnt stipplc fills. This function, Iio\\fe\,er,\!,as
not fi~llyimplemented o n all platforms at the time \ve
designed the cXcursion version 2 soft\\*a~-e
product.
Without this fi~nctjon,there is n o easy \vay in thc
Win32 environment to perform the transparent stipple operations. When the foreground color is either
0 or OsE'FFE', the raster operation can be remapped
to get the proper effect. General rcctiingular fills that
d o not mect the req~~irements
oftlic special case previousl!. mentioned must be accomplished by first converting the stipple bitmap to the depth of the
destination ancl then remapping the raster operation.
In general cascs that are not rectangular fills, machineindependent fi~nctionsare called t o break down the
request into s p 3 m
Image Requests The GetImage and PutImage
I-equests are other examples of X graphics requests
that d o not nlnp \\,ell into tlie Win32 API.
onl!,
\tray i l l thr Win32 en\,ironment to put image data o n
the screen is to first create a Win32 bitnlap and initialize it with the image data, and then call thc BitBlr
fi~nctio~l
to copy the bitmap to tlic scrcen. X inlage
data al\vays lists tlie top scan lines first, \\/hereas the
b o t t o ~ nscan lines are listed first in Windows bitmap
data. Tlicrcfi)rc, before the bitmap is initialized, the
S imagc c1,ita must be scan-line tlippcti. Similarl!;
the X (;ctI~nagc request requires the use of an intermediate bitmap and also recl~~irrs
the scan-line flip.
Plane Mask Support h y graphics opcl-ation ill X can
be modified by setting a plane niask in tlic GC. The
plane mask specjfics \vhich bits of the destination pixel
arc allo\\aed to bc changed. Without a plane mask, an
X graphics operation may be defined as
dst t src @ dst,
\\?here @ is one of the 16 binar), raster operations
(e.g., OK, A N D , and XOR). When a plane mask is
given, the following assignment dcfincs the destination pixel:
dst + ((src @ dst) S: pm ) I (dst & -pm)
examined by the \\,indo\v managcr. This gives the
\\.indo\\, manager the opportunity to act on \vindo\v
Most video hardwnl-e de\*iccssupport plane masking,
nianagenient-relilteci messages such as those tliat indiand those that d o not support it generally provide List
cate a change i l l tlic \\~indow'sconfiguration or state.
access to vidco random-.~cccss memory (1lAiM). 'l'hc
If the willdo\\) ~nanagcl-decides to liclndlc a mcssage, it
Win32 A H , ho\vcvcr, provides neither plane miisking
is removed from tlic queue, and the server never sees
nor direct vidco lL4M ;icccss. To understand why, you
it. If the \vindo\v manager is not interested, tlie mesmust realize that Windo\\,s lias virtualized tlic color
sage is passed on to the normal \vindo\v proccdure.
hnnclling in an attempt to mediate contlicts bct\\,ecn
The p~rrposcof the internal \\,indo\\- ninnager is
npplicntions tliat \\*auld otlic~~\\~isc
\\.ant to nioditj' the
t o gi\re S \\ri~~do\\,s
tlle s.me appearance and belia\.ior
colormap (the pixel-to<olor mapping [able). I n this
as Win32 \\,indows tliat arc created by typical desk\.irtual color environment, the concept of planc masks
top applicntions, such as word processors and
lias n o mealiuig because Win32 applications ~ ~ c cnot
d
spreadsheets. Wlicn an X \.vindo\v is mapped for
I<no\zr the pixel value thnt corresponds to a particular
tbe first time, tlic intcrnal \\lindow manager receives
co.lor. See tlie section Color ltcsoi~rceI\/la~lage~~icnt
for
a Win32 WM-CItEATE message. Before tlie \\/indo\\)
an explanation of lio\\r tlic excursion sohvarc 1nan;lgcs
becomes \risible on the screen, the \\,indo\\r Inanto assign specif c pixel \.alucs to colors.
agcr alters tlic style of tlie Win32 \\.indo\\, to
In tlic general plane masl<case, it is necessar!, for the
WS-0VERLAl'l'El)~VINDOW. Win32 \\.indo\vs with
S scr\.cr to first s<i\-cthe contents of the dcstiuation in
this style arc autoniatically managed by Windo\vs,
;I bitmap. The graphics c.in then be temporaril!~drm\,n
\\hich hnndlcs ~iio\~ing,
resizing, iconitjing, maxin~iz\\'itIio~~t
regard to the planc mask. Those bits i l l the
ing, and closing thr windo\\fs. Each of tliesc actions
destination that are specified by die planc mask
causes a corresponding message to be sent to tlie
as being unaffected cnn then be restored from the sa\cd
scr\ler's windo\v procedure. The internal windo\\:
bit~nap.This process \\.ill \\rorlc in ever!! case but is inefmanager intercepts the messages and dispntclies theni
ticicnt since it in\rol\.cs sc\acral graphics operations
to the appropriate intcrnal fi~nction.
behrc achieving the f nal result. Many speci;ll c;iscs can
The role of the intcrnal \vindo\\. managcr coniplebe reduced to one or n1.o siniple steps b!. rnodieing the
ments the I-oleof tlie server. The server processes client
source color and rastcr opcrntion. Table 1 sho\\*sho\v
requests on S \vindo\\.s and translates theni illto operathe source color and raster opelation call be set to
tions on Win32 windou.s. The intcrnal wi~idowmanacliit.\o the plane niask etEct. The excursion S sc~.\,c~.
ager handles Windo\vs messages tliat indicate changes
i~scsthese optimizations for many graphics operations
to a Win32 window and translates them into con-e~ ~ hthe
e ~source
i
till is a solid color.
sponding changes to the u~iderlyingX windo\\/. For
example, the no st important message that the win do\\^
Internal Window Manager
manager handles is WMVINDOWPOSCHANGING.
I n the absence of ;I \\,indo\\ manager, tlie cScursion
This message is sent just before any change in the \\,inserver creates a11 \vindo\\rs as pop-up \vindo\\,s. All \\,indo\\.'s position, size, staclung order, o r \,isibilip: If this
iio\\,s, including top-lc\.cl \vindo\vs in ~iiulti\vindo\\,
message indicates tliat tlie \vindo\v size changed, tlie
mode, are undecorated. The!! have no Win32 borders,
\\findo\\/managcr changes the size of the corrcspondtitle bars, o r system menus. To move, size, minimize,
ing X \vindo~\~
and sends a Configul-eNotit) event to
mnximize, or close windo\\/s, the user IniIst r ~ ;I~\\)inn
tlie client. Si~nilarly,tlie window managcr trnnslates
d o ~nanager.
~ ~
other uscr-directed events such as focus clinngc, \\)inAn cXcursion user nl\\*ays has the option of using
do\\*stacking, and iconification into changes to the
one of tlic many X-based \vindo\\. managers available,
i~nderlyingX data structures. In most cases, tlie \\.inst~clias tlie Motif Windo\\. ~Ma~xiger.
Ho\\.c\,er, many
do\v manager does this by calling into the deviceusers \\.ill \\.ant 3 \\'i~iiio\\,
mnnager parndigm that is
incleprndent Iiiycr, rlius sinlulating an S ~.eclucstthat
co~isistcnt\\zit11 Windows so tliat all \\.indo\\,s on tlic
\vot~ldoccur froni an external \\lindow manager.
dcsktop have the sanic user interface. T o accomplish
this, a built-in window manager is provided as part of
Mouse, Keyboard, and Input Focus
the excursion scrvcr. This internal window manager
Mouse actions iind keystrokes are received by the
is olxrati\ze only in ni~~lti\\~j~ldo\\l
mode.
excursion scr\Icr as Win32 niessages. Each mcssage
The internal \\,indo\\, mnnagcr, although linl<cd \\zit11
co~ltninsinformation about the \\)indo\\ that recei\~ed
the server, is functionally isolated from the rest of the
tlie inpi~t2nd the time of the input. For IIIOUSC moves
code so that it can easily be disabled. Thls allo\vs esterand clicks, thc scr\,cr uses the \vindo\v information t o
nnl \vindo\\. managers to be used and also tici1i~tc-s
locate the corresponding S \\.indo\\, and for\v.lrds an
ciebugging by nllo\\,ing problems to be isolated. The
S
event to rliat \vi~ldo\\.Iceyboard input is for\varded
\vindo\v Iiianager crcnrcs n "hool<" into the scr\lcr's
to
the u,indoc\l tliat ci~rrcntl!~
lias S focus.
wj~ldowprocedure, so rliat a11 Win32 messages arc first
1)igit.d 'l'ccllnical jo~crclnl
Vol. 8 No. 1
1996
3,
Table 1
Plane Mask Optimizations
Requested X Raster
Operation
src 0
dst 0
0
1
1
0
GXclear
GXand
GXandReverse
1
1
Notes
Modified Source Color and
Raster Operations
src
t
-pm, rop <- and
src + src I -pm
src t -pm, rop c xor
src t -pm, rop t and
src t src & pm, rop c or
src t pm, rop t o r
GXcopy
GXcopy
(src & pm) = pm
GXcopy
(src & pm) = 0
GXandlnverted
GXnoop
GXxor
GXor
GXnor
src
<-SIC
I -pm, rop t and
src t src & prn
src t src & pm
src t src & prn
src t -pm, rop t xor
src t src I -pm
src + pm, rop +xor
src t src & pm
src t -pm, rop + xor
src t -pm, rop +and
src t -src & pm, rop c or
src t src I -pm
src t src I -pm
src t -pm, rop t xor
src + prn, rop t or
GXequiv
GXinvert
GXorReverse
GXorlnverted
GXnand
GXset
Notes:
1. dst is unchanged when src equals 1 for these raster operations. Therefore, t o preserve the value of dst when
pm equals 0, set src equal t o 1.
2. dst is unchanged when src equals 0 for these raster operations. Therefore, t o preserve the value of dst when
pm equals 0, set src equal t o 0.
3. This operation sets all dst bits t o 1 except where the plane mask equals 0.This can be done simply by ORing
pm into dst.
4. This operation clears all dst bits except where the plane mask equals 0. This can be done simply by ANDing
pm into dst.
5. XORing with 1 has the effect of inverting. To invert only where pm equals 1, XOR pm with dst.
6. These operations are performed in t w o steps. Note that dst is inverted when src equals 1. First perform the
operation with src set t o 1 where pm equals 0. dst is now correct except that i t is inverted where pm equals 0.
The second operation of XORing with the invert of pm corrects this.
7. These operations are performed in t w o steps. Note that dst is inverted when src equals 0. First perform the
operation with src set t o 0 where pm equals 0. dst is now correct except that it is inverted where pm equals 0.
The second operation of XORing with the invert of pm corrects this.
8. This operation is performed in t w o steps. First dst is set t o 0 whenever pm equals 1. Then dst is set t o 1 whenever both pm and src equal 1. The t w o special cases can be reduced t o operations that use GXset and GXclear.
9. This operation is performed in t w o steps. First dst is set t o 0 whenever pm equals 1. Then dst is set t o 1 whenever prn equals 1 and src equals 0.
10. dst is unchanged; therefore, no operation is required.
\',>I. 8 No.
1
1996
Tlic S server is n singlc application in thc Win32
c ~ i \ ~ i r o n ~ i tI1;it
ie~it
all tlic X windo\vs it crcntcs.
From the user's pery>ccrivc, tlioi~gh,there may appear
to be Inore than one >( application running, each with
its o \ \ r ~collectio~i
l
of \vindows. The user cspccts to
bc nb1.e t o shifi the keyboard focus fro111one \vindow
to another in tlie same fashion that f o c ~ ~issshifted
L>ct\\~ccn
other applications. When an estcrnal \\*indo\\,
manager is in use, ~ C L I cS o ~ l t r ~is l straiglitfor\\~nldird.
..I lie \vincio\v maliagcr, using \\,hatever sc~nanticit
\\.:IS designed for, monitors mouse evcnts and shifts
focus accordingly. Ho\\fcver, tlie semantic modcl for
tliis may or may not br consistent with the Win32
modcl. In cithcr casc, the \\iindo\\~decorations, c.g.,
bordcrs, title bars, and menus, arc almost guaranteed
to be different. A L I S ~ I ivlio
.
\\?ants a consisteot uscr
interface moc.lc1 across a11 applications must c~iiplo!~
the inter~ial\\'illdo\\, manager.
At any given rime, one \\.indo\\, o n the screen has
Win32 focus and onc X \vindo\v has X focus. Tlic two
\vil~do\vs'Ire not necessnrily the same. Since tlie S
server creates and owns all the S \vindo\\~sin i~sc,the
scrvcr rcceives ltcyboard input when any onc of its
\\rindows has Win32 fi)cus. The keystrokes are not
necessnrily sent to [lie undcrl!,ing S \\~indo\\r,ho\\~ever.
Thcy arc sent to tlic \\!indo\\?that has X focus. 'The
internal \\.indo\\, manager assigns X focus to the X \\.indo\\, that receives Win32 focus. The client receiires
notification of this eLrcnr and may decide to assign S
focus to sollle other windo\\), perhaps a child \\rindow.
The server must therefore keep track of both the
X \\lindo\v that currently has focus and the state of
Win32 focus. When the scn8crloses Win32 focus, the
X focus is assigncd t o tlic root \\indo\\'. When tlie
scrvcr rccei\.es Win32 focus, X f o c ~ is
~ sassigncd to tlic
X \\!indo\\, that prc\.iously had it. Whenever X focus is
cllangcd by an applic,~tio~i
or by the \\/indo\\,nidnager,
tllc current X focus st,~tcis cached so that it can be
restored later, if necessary.
L
L
~
\
~
~
~
~
"
Font Management
Fonts and test f~~nction,ility
mnlte up a signif cant portion of any graphics arcliitccture. Both the X and the
Win32 systcms dcfinc a rich set of test-rendering
opcratio~~s
and can proccss sc\reral font formats.
The fonr nlnnagcmcnt library supports both bitniap
and scalablc outline fonts. Bitmap font glyphs arc simply reformatted and i~sed.Scalable hrmats, such as
Adobe Typel, are rasterized on demancl into the X
font for~iiat.
For masi~numperformance, the scsver draws test
\\ith native Win32 fonts using the Win32 Al'I. Win32
fonts are bitmnp fonts in the F O N format. Win32
fi~nctionalinco\,ers tlie great majority of test-drawing
operations, but there are a fe\v c,lses in \i,hich it is
either not possiblc or not efficient to use Win32 fonts.
The server can also draw directly \vitli tlie X fonts to
provide f i l l 1 X font support and complctc test-drawing
fi~nctionality.Tliis u~ietl~od
uses Win32 BitBlt() opcrations to copy the ch.iracter glyphs to the display as
biunaps. L>ra\\ling speed with this method is acceptable but not ~nasimum.
Therefore, both S and Win32 fonts are used. The
Win32 fonts may be thought of as optional accclerators: the ser\,cr uses them whenever possible and falls
back to the S h n t s when necessary. The decision to
fall back can be madc on a variety of co~iditions.Tliis
technique has also proved useful in working around
problerns such as test-drawing bugs in individual
video drivers.
Since scalable font outlines arc rastcrized into
bitrnaps at run time, they are generally drawn directly
\vith the internal X font format. The extra \vork of
compiling a companion Win32 font at run time generally outweighs its \/due as an accelerator.
X bitmap fonts are most commonly distributed in
tlie Bitniap 13istribution Format (BDF), a11ASCII tcxt
source file. The eXcursion team \\!rote a font compiler
tool that ge~icrarcsn,lti\.e Win32 (FON format) fonts
from the BL)F sources. The fonts created can be used
by any Win32 application.
7'11e co~npilercan Senerate either tlie commo~lly
used version 2 format or the extended \icrsion 3 format, which is ~~cccssarp
for large fbnts that require
more than 64 kilobytes (KR) ofglyph storage. F i g ~ ~3r c
illustrates the process of generating cqi~i\~alcnt
X and
Win32 fonts fiom ;I common source.
The X font format contains extra information (e.g.,
metrics and properties) tliat cannot be dcrived from
X and Win32 Fonts T11c X font management library is
a modular architecture that defines an API for reading
n~id\\~ritingindividual fonr formats. The nlodt~lctliat
implcmcnts the API for a givcn font format is called a
I-endercr. Tliis approncli nllo\\.s X to support sc\~el-al
h n t formats: the library's renderer modules con\,ert
cstcrnal formats to a singlc, internal bitmap format,
\\,liicli is used for all draiving operations. The term
X,/~)III
refers to font dam in tliis internal format.
X FONT
LIBRARY
BDF FONT
Y
~ ~ ~ R S I O NWINDOWS (FON)
COMPILER
FONT
Figure 3
Font Conversion
Digital Tcch~iicalJourn.11
Val. 8 No. I
1396
39
tlie Win32 font. Therefore, tlic S and Win32 fonts arc
uscci together; the S infor~nl~tion
comes from the
S font and the Win32 h n t is used b\. tlie Win32 API.
Realizing Win32 and X Fonts When the )i server first
).
opens a font, it involtes the h ~ ~ i c t i oRealizeFont(
n
Tliis fi~nctiongives the server a n o p p o r t ~ ~ ~to~initialit)~
izc data structures and perform any format-specific
operations necessary to make the font available.
T o make a Win32 h n t n\.ailnble for dru\\.inp,
tlic scr\'cr retrie\.es tlic filename of the font from the
scr\zer's look-up table and registers it \\.it11tlie Win.32
AI'I using the f ~ n c t i o nAddFoutIkso~~rce().
A handlc
to the font is obtained from (:rcatcFontIndircct(), and
tllereaftcr the handle is sclcctcci into the desired 1)C
for drajving operations. If tlic Win32 realization fiiils
k ~ any
r reason, the code simply realizes the X font
instead. Failing to realize a Win32 font docs not necess.iril!l imply an error condition. Such failure hnppens in
any case in nlhich tllc ~ c l - \ ~dcciiics
cr
that it is best to
use tlie X font directly.
Tlie internal X font format is 3 set ofdata structilres.
The glyphs are storcd in convcntionnl arra!'s in i ~ s c ~ .
niemory. T o improi~epcrformnncc, the ser\,er I-calizcs
an S font by writing all glyphs to a Win32 bitmap in
off-screen memory. CrcatcBitmap() returns a 1i;indlc
For later reference, and the glyphs in the bitmap arc
indesed for use in drawing opcmtions.
Drawing with Win32 and X Fonts The glyphs i l l X test
strings are ofien kerned, tliat is, o\,crlapped for best
typogr~phicappearance. To dra\\r \\.ith Win32 fonts,
tlie scr\.er emulates the \\!A!, X ctl-a\~,stest b!, using
ExtrestOut(), \\lhicIi uses an intcrchnracter spacing
vector to place the indi\.idual glyphs. The font's X metr i c ~are used directly to calculate this \:ector.
Glyphs fi-om S fonts arc dra\\~nby performing
I3itBlts fi-om tlie Win32 bitmap to the target \ v i n d o ~ ~
or bitmap. The server places the glyphs ~lsingthe fi~nt's
X nietrics as described in the p r c \ i o ~ paragraph.
~s
Color Resource Management
Altlioi~glisome X Windo\\. System concepts and structures map fairly closely to thosc in the Win32 s!.steni,
differcolor resource management is handled
\\!lien dealing
ently. 'l'he difference is most c\~iiic~it
with pse~~docolor
video systems. (:onseclucntly, this
paper describes only this case.
The X Windoc\, System environment shares 256 colormap cclls among all applications that use tlie dcfault
colormap (i.e., those th,lt ilo uot h.i\~ea private color~n.ip).Applications can allocntc cclls in the default
color~napto protect them fro111~nodificationby otlicr
applications. I n contrast, the Win32 system allo\\'s
eacli application complete access to tile s!!steni palcttc
\vliile tlic application has focus and maps the palettes
of the ~\:indo\ilsw i t l i o ~ focus
~ t as best it can.
In thc X Windo\\. System cn\,ironme~lr,\\,lien an
application reserves a colormap cell, it references the
cell \\.it11 a pisel \,iilue. This \,aluc is an indes into tlie
colormap and is used to look i ~ ptlie value that will
;~ctunllybe storcd ill scr-ccn mcriior)l \\,hen that pixel
\~alucis used in a dra\\~ingoper.ltion.
In tlic Win32 systeni, color manngcnicnt is handled
hy the palette manager through a palcttc structure.
Eiicli ;ipplication lius a logical palctte, and a single system pdctte contains the colors currently mapped to
the hard\\~arccolorn~np.Applicatio~isreference colors
rclati\.c to their logical pnlcttc, and tlie palette n u n q e r lia~idlesthe mapping ben\lccn tlie logical palette
2nd thc system palette. M'licn an application is given
focus, tlie palette manager maps a11 tlie colors from the
logical palctte into the systelri palctte. If the system
palette docs not have e ~ ~ o ~empty
~ g l icclls, tlie palette
man;igcr frees cells allocated to other applications. If
this occurs, the palette mnnagcr \ \ r i l l attempt to remap
the otlicl. applications' colors into any remaining free
cclls in tlie system coloniiap. If ~ i o enough
t
cells are
free, any remaining unmapped colors are mapped to
the system palette colol-s that most closely match.
l3cc;i~1sc
ofthis \\,a! of handling color resource manaKclnent, an application
docs not kno\\r \\hat value is
. .
being stored in screen meliiory For any particular color
2nd the value stored for any color can change over the
lifctinic ofthe ~pplication.'Tliis situation presents significant difficulties for X operations tliat require exact
kno\vlcdge o f t h e pixel valucs in screen memory, such
as the GctImage operation and operations in\rol\ring
pl.lnc ~nnslts.T h c scr\,cr \\mrks around the difficulties
b!. crcuting tn.0 Win32 logical palettes.
-The tirst palette, i.e., the \\$orking palette, correspo~ldsexactly to thc X dcf'i~~lt
colormap and does not
allo\\i sharing of tlie palettc by Win32 applications.
CVhc~ier~er
an S \\~i~ido\\/
has toci~s,tlie \\!orlting palette
is in use. This causes tlic Win32 palette manager to set
LIPtlic system palettc such that it directly corresponds
to tlie X colormap, and operations that arc pisel based
\\.o~-k
properl!,.
.The other palctte, i.e., the identity palette, is set up
to correspond esactly to tlie system palette. The identity palette is used \vlienc\,cr n o X \\~indo\vhas focus.
Ilccu~~se
of the correspondence, n o translation is
in\~ol\recibcween tlie identity palctte and the system
palcttc, which allows thc X scrvcr to know what piscl
\~alueis stored in screen nicniory.
Tlie X Windo\v System cn\iiron~iientalloria for pri\,,~tccolormaps, \\rliich arc created anci ~lscdby a single
npplication. The server creates a \\lorking palette for
cvcry colorniap created. Wlic~lthe colorniap is installed
(normally by the \vindo\v manager \\,hen the X applicar i o l i is gi\,en focus), the excursion soh\,are installs thc
or or kin^ pi~lettcassociated \\*it11the private colormap.
Tlic cXcursion X ser\,cr currently supports the
I'scuclo(:olor \.isual class and the StaticGray depth 1
visual class, \vliicIi is rn~inly used for bitmaps.
cXcursion \rcrsion 1 also supported a StaticColor V ~ S L I ~ I
class h r I (,-color \ d c o graphics array (VGA) displays.
cXcursion version 2 treats VGA devices identically to
P s c ~ ~ d o C o ldc\iccs
or
and allo\vs the Windo\4.s piilettc
malager to generate ditl~ering patterns for the
i~navailablecolors.
Network Interface
With the release of X1 1K6, the S <:onsortiu~ncoliibined all transport-specific codc into a single place
in the sourcc tree, the S transport interface. The
eXcursion team exte~idedthe X transport interface to
include Ncnvork C:omputing l)e\,ice's (NCD's)
Sremote serial line transport. <:o~nbinedwith the
transmission control protocol/internet protocol
(T<:P/IP) and 1IECnct transports, the eXcursion
product c.111 no\\, execute S sessions ojler an!! of tliesc
transports simultaneously. Tlie eXcursion product
supports any T<:P/IP stack tli;lt complies \\-it11 the
Winsock \rcrsio~~
1.1 irnplenicntatio~l,PATHWOllI<S
1)ECnet protocol, and NC1)'s Xrcmote protocol for
scrial line.
Tlie X transport interfi2ce pro\sidcs fi~nctionsthat
are common to all transports, such as pal-sing an
address inro a host and port n ~ ~ n i h eTlic
r . interhcc
does not provide a11 aI>st~-~ctio~i
for tlic select() call,
because it assumes that this call is transport independent. Unforti~nntcly,the Xrcmotc protocol requires
.In indcpcndcnt select() mechanism, and, thus, it
\\.as necessary to implement a select() abstraction to
conibi~lcrhc transport-indcpc1icIc11tselect() \\.it11 the
Srelnote select( ). Although somc\\.Jiat co~npromisecl
by this addition, pcrformancc \\..is a problem only
\\,hen the Srcmotc protocol \vas used in conibjnation
with either the .I'(:P/IP or the 1I1-Xnct protocol.
X Image Extension
cXcursion \,crsio~l2 provides \versions 3 and 5 of the
S Image Extension to support a \vide rangc ofiniaging
applications. 1Sccnuse it is a Iargc body of code, S1E
is implemented as a pair of Win32 I)L,Ls to conser\.c
mclnory on slrstcms tliat \\,ill not be running applic.1tio~isthat use SIF,.
Normally, access to a DLL is one-\\,a!. Applications
can load 2nd ~nnkcfunction calls inro a DLL, but
becai~seit is linked dynamically at run time, the DLL
code cannot make fi~nctioncalls back into the calling
'~pplicntion.SIE, l~o\\,e\~er,
must call into the cic\ziccdependent layer of tlic set-\-erto perform any required
dra\\-ing after processing its imaging requests. To permit this, a n addition to the intcrhcc \\/as designed.
When the XIE DLL is initialized, the caller supplies a
list of pointcrs to the fi~nctionsnccded by the XIE.
-
1he DLI, fills an arra!l witti thcsc pointers dnd then
calls back indirectly through the array. On the
\Vindo\vs operating system, this design could create a
probleni because under Win32 Al'ls, global data in 1'
DLL is not instanced; tliat is, the cocic is not rccntr;int.
The approach \\.arks in this case bccai~sethere is only
one copy of the DLL loaded. If another app.licntion
\\,as sharing the DLL, the pointers \\,auld be ovcr\\,ritten by the second initialization.
7
Control Panel
The eXcursion control panel is the primary interface
through \\4iicl1 tlie user confgurcs 2nd controls tlic
product. Some other components create simple \\!indows or icons, but these functions are limited. The
control pancl constitutes 9 0 pcrccnt of the user intcrface for the cXcursion application. This fact malces the
control pancl an ideal candidate for the rapid application devclopnlcnt feati~resof the microso oft Visual
C++en\ironn~cnt.The control pancl is a Wj1i32 application coded 3lmost entirely in C++ and linked \\lit11
the Microsoft Foundation Class library.
.I he niflin pilrposc of the colltrol panel is to prcsent a managcable interface t l ~ r o i ~ g\\'hicIi
h
the user
can \fie\\.and ~ n o d i %
tlie eXcursion configuration profile. T o d o tliis in n Inannrl- consistent \\,it11 the ne\\.
Windows 95 shell, the l'ropcrty Sheet MFC objcct
\\/as choscn. Property Shccts arc tabbed dialog boxes
that have the advantage of organizing large amounts
of data settings in a compact space. They arc used
extensi\,cly by the Windo\\.s 95 opcr3ti1ig systc~liand
by the most recent \.ersions of h4icrosoft applic~tions.
Tlie l'ropcl-h Sheet objcct is 3 subclass of the
Windo\\rs ohjcct rind is essentially a containel- for tlic
tabbed pagcs. Each tab, \\!hen cliclted by tlie nser, displays a dialog box that is subclassed fi-om the MF<:
Property Pngc object. The indi\vidual pagcs can be
visually configured a ~ l drevised i~sjngthe class \\tizard
feature of Microsoti Visual C++.The designer simply
selects dinlog box controls such as buttons, drop lists,
o r edit fields and positions them o n the dialog box.
The codc to handle user actions is then f lled in.
Tile c S c ~ ~ r s i ocontrol
n
pancl is slio\\.n in Figi11.c 4.
We constructed an initial P I - o t o t y c of tlie control
panel application \\,it11 about 6 0 percent of the fnnl
fi~nctionalipin less than one month.
7 7
lnterprocess Communication Library
eXcursion \.crsion 2 consists of sc\wal cooper.iting
processes t h ~ must
t
conin~unicateand synchronize
\vith one anotlicr. \/VIien a rc~iiotcS application is
started by tlic server o r the control panel, the ~pplication launcher signals when tlie operation is complctc.
Vol.
S So. 1
I
4.
.-
I
XDMCP
I
Extensions
i
--
Communications
I
Modem
Figure 4
The cXcursion Control Panel
Error and status inforn~ationis sent to the error loggelby the othcr components. When tlic user changes
a configuration setting through thc control panel, the
changc tilust be communicated to the S scr\.cr, if it is
running. In some cases, the changc can take effect
immediately; in othcr cnscs, the ser\'el- cannot i~nplement thc change \vithout I-cstarting. The control panel
and the server must e n p g c in a dialog so tliat the user
can he informed as to \vIiat actio11must be taken, ifany.
Ilie IP<: library is an operating system-independent
API tliat permits cXcursion components to determine
\vhich otlicr components are present and to cschangc
co~nmandsand config~~l-ution
infor~n;~tio~i.
The Windows N'I' operating system providcs scvcral
built-in IPC mechanisms, but most arc nor n\.ailablc
on the Wi~ldo\vso r Wil~do\\.s95 systems. The onl!.
mechanism that is uni\*crsal to the three operating
systems is the rncssngc-passing i~iterfaccin tlic Win32
API. This mechanism, while not the most cfticicnt, is
rcJati\lcl!l straightfor\\jard to iniplerncnr. Sincc the pcro n tlic IPC library were detel-f o r m a ~ ~ cdclnands
c
mincci to be very light, this mcclianism \\as chosen.
P
.
'I'he disadvnntngc of the Win32 niessage-p~ssing
interface is that it is \\111do\v based, not proccss bnscd.
~Mcssagesarc I-ccci\.cd by n callback procedure that
must be associatccl nith a \\indo\\, heforc an!* communication can take place. If an application has not yet
cl-eared a \\,indo\\; o r never creates ,I \\li~ldo\\;ns is tlic
cnsc with the application launclicr, n o comml~nicatio~i
is possible. T o rc~iiedythis, the l.I'C: library crc.ltcs its
o\\,n windo\\, \vllcn the calling process initializes. The
IPC \\indo\\. is nc\,cr mappcd to the screen, so it is not
\,isiblc to the user. All intcrp~.occssco~niiiunic;~tio~i
passes t h r o ~ ~ gtlic
l i IPC \\,indo\\.
The IPC l i b r ~ r yconsists of n collection o f u~iiquc
lncssages and an API. The Incssagcs arc registered
\\fitli the Win32 function Registcr\Vindo\~~Mc,u:agc.
'This ensures tliat the messngcs used by the excursion
applicatioll d o not conflict \\?it11sgstcrn messages o r
Iiicssagcs i~sedby other applications. The e S c ~ ~ r s i o n
I PC messages arc
i~xC,om~x)1ic1itStartedh4sg,
\\rliicl~the IPC:posts to
a component initializes.
;1II ~ O I I I ~ O I I C I I\vlien
~S
ipcl<estartServerMsg, which the IPC sends to the
server to tell it to restart.
ipcRestartSer\7erStatusMsg, \vhich the IPC posts
with the status of the rcstart rcquest.
ipclnquireMsg, which the IPC sends t o retrieve a
data item from a component.
ipcProfileChangedMsg, \\lhich the control panel
scnds when the registry profilc changes.
ipcI,auncl~OneCo~iipleteMsg,
which the application launcher sends to noti@ the server of launch
completion.
ipcLaunchAIIComplete~Msg,\vhich the application
launcher sends to noti@ tlie servcr of lau~icliconipletion.
ipcHideAIWiridowsMsg, which the server sends to
all components t o tell them to hide all their windows. The excursion application uses this messagc
to execute the pause/resume feature.
ipcShowAllWindowsMsg, which the server scnds
to all corilponents to tell them to show all their uindows. Thc excursion application uses this message
t o execute the pause/rcsun~c feature.
In addition to sending and rccei\,ing messages,
eXcu~.sionprocesses can use the IPC library to determine which other components are running. T h e IPC
initialization procedurc creates a wi~ldowwith a
i ~ n i c l ~name
~ e that identifies thc calling component. T o
determine whether a specific component is present
in the system, the IPC scarchcs all windows 011 the
system until it finds one with the correct name.
Error Logger
'The error logger is a Win32 application that receives
error and informational messages from other components and either displays them in a window o r logs
them to a file. O n the Windows NT operating system,
inhrmarion that may help system managers or users
diagnose problems ma)( add~tlonallybe recorded In
the Windows NT e\,ent log.
Application Launcher
Thc application launcher is a Win32 application thi~t
handles requests from the control panel o r server to
start X client applications. The clicnt may reside o n
a rcmotc host or on tlie samc machinc.
When the user reclilests the servcr o r control panel
to start an X client application, it starts the excursion
application launchel- in a separate process. The application command, host name, account information, network transport, and command shell are passed to tlie
launcher in its command line arguments. The launcher
makes thc c o n n e c t i o ~to~ the rcmotc system, initiates
the command using the selected protocol (resec, rsh,
l)E<:nct object, o r local command), and scnds an IPC
mcssage to the server indicating that a new application
is starting.
Registry Interface
The Windows N T opcrating systcni i ~ l t r o d ~ ~ ac cncnf
d
concept called tlie registry. This is a protected database
maintained by tlie operating system, wherein Win32
applications may store config~~ration
and state information. The registry has a well-defined API and
a maintenance utility program that is shipped with
the Windows N T operating system. Under the
Windows operating systeln, config~~ration
information
is kept in simple text files, which are vulnerable to
accidental or malicious tampering. At the time the
design of excursion versio~i2 was under way, ~t was
unkno\vn which, if ejther, of these two methods would
be available under the Windows 95 opcrating system.
Nevertheless, all three of these operating sjatems had
to be supported.
VVc dcsigned an API for acccssi~lgthe configuration
i~lh)r~nation
in a manner independent of the operating
system. I(no\vledge ofthe operating system and its registry access method is encaps~~latcd
in the library. Since
sc\lcral independent processes must access the information, the library is built as a l)LI, to conserve memory.
The interface basically resembles that of tlie Windo\vs
NT registry API but eliminates some ofthe complexity.
If the eXcursion software has not been configured
\vhcn thc registry interface first acccsscs the profilc,
default values for all settings arc selectccj to allo\~,the
sofnvare t o function normally.
'
Summary
With computer systems bascd o n the Microsol?
Windows operating system increasing in power and
decreasing in price, Windows-based systems are appearing on desl<tops that once held \+,orltstations running
the U N IX o r OpenVMS operating systems. Windo\vs
systems must be able to access applications on remote
filc and c o ~ i i p ~ servers
~te
running in the X Windo\v
System environlnent. Version 2 o f tllc cXcursion product provides desktop integration of X client applications
with native Win32 applications. Modular coding techniclues, object-oriented programming, and selective use
o f the Microsoft Foundation Class library helped
rcducc development time, and improve performance,
maintainability, and reliability.
General References
1). Giokas , ~ n dA. Lesko\\.itz, "cXcursion for Windows:
Windo\ving Systcriis," l)/gitrrl Tc~cl7jlical
, J o ~ r r - nvol.
~ ~4,
l . no. 1 (Winter 1992):56-67.
Intcgt.,~ting
?'\\lo
Vol. 8 No. 1
1996
45
X Window System
717G
~ i 7 GI/;(/(,,
/ i i / o i . / / ~ ~ / 7?~ri//i//(1/og)~,/i)i.
io~/~~/
tho 11.711S . A~igcbrnnrldtct al., DeJi'//i/ioi/c!/'/hcl)ol?i/~,g
L~i~,c,t.,/i)/' do/i:i/ i / / ~ ~ / ~( l/ l~c c~ ic~~~ cl o~~b\Ji~sli.:
i d , ~ ~ i c s o s oPress,
fr
199.3 ).
i\/l.lsb.: S \ ' < : ~ I I s o I . Il~c)S 1 . 1 1 S'(IIII/I/CSewei. (C,n1l1l>ri~lgc,
S. Mc<:onncll, Ci)t/vCbl~l/d~/c:
A /Jrwc/icalkl~rritll?ook
oj'
t i ~ ~ ~I I iI ~i ,. , 1 9 9 4 ) .
S?/itrzrrc~ (,i)/~.<t/./iclio/~
(Rcdmond, Wash.: Microsoft
J . Fulton, 7 % A~' Fo17l Scw*ice I'tu/oc'ol. li~t:sio~c
2.0.
PI-as, 1993).
A' li>~:~io/r
1 7 . RE/~>LISC
6 (Cnmhrjdgc, i\/lnss.: S ConsorC:. I'ctzol~i, l'i.c~y/z~/lr/l/i//~q
W'ii/do/c:s,2d cd. (Rcdrno~id,
r i ~ ~ ~I ~i i c, . 1994).
,
Wash.: Microsoft I'rcss, 1990).
E. Israel and E. Fortilrle, 7hc) A' \ V ~ I I L / O
S)~.S/OIII
? I ~ .So~.i~c~r;
X li.t:iiot/ 1 7 . Kc~le~fse
5 (\%burn, ~M;iss.:lligital Prcss,
1993).
Biographies
(CamI<. I'acknrd and I). Lemke, 71oc S / i ) r / l /.ihrz/t:)~
bridge, h,l.iss.: S (:onsorrium, Inc., 1 9 9 5 ) .
I). Ilosc~~rh.ll,
/ / i / e ~ 1 - ( ~ /Ci ~O/ 7I I/ I I I I I I I I I(C. ~L) II /~ ~I P~I// II ~ O ~ I S
.Ilrrr~~/o/.
li,r:iiorr 2.0 (<:ambridge, Mass.: S <:on~ortium,
Inc., 1994).
I<. Sclicitlcr, .Y\Vi/i,7~lo~i'
S I ~ / CP~.olocol.
II~
,I'I iJ/sioii 1 I .
K c l ( ~ / s c6l ((:n~nbridge, blass.: X (:onsorriu~n, Inc., 1994).
R. Schcitlcr and J. Gettys, X l.Vi~rc/oir~
S ~ : S I W(Ilcdford,
II
Mass.: 1)igital l'rcss, 1992).
Networks
JM. Hall ct nl., "Windo\\.s Sockets: An Opcn Jntcrfncc for
Ncrivork PI-oglummingunder Microsoti \Vi~ido\\,s,Ltrsiori
l.I"(l99.~).
W. S r c \ , c n , I I I W iV~/ri'o,%?
P/.og/'colri~~i/r~
(Englc\\,ood
(Iifti, N.J.: Prcnticc-Hall, I ~ l c . 1990).
,
J o h n T. Freitas
Prcsclirly a soft\\..irc c ~ l g i ~ i cat
c rAtria Sofnrarc, John
t'rcitns \\.orkcJ ; ~ 1)igirnl
t
for 15 years. For the last
~\5oci,ltcd\\.it11 I)iginI1s cScursio~iprodl~cr
!7rars, lie \\,a
.IS a11indi\,id~~.ll
contributor, .in iircllitcct, dnd a designer-.
Previously, lie \\.as in rhc \\'orkstntion g r o ~ ~John
p . rcccivcd
3 B.S.E.E. kom Norrhcnstc~-nI'nivcrsiy in 1967. While
in collcgc, he \\ol.kcd as a co-op scudcnt o n rhc Apollo
I'rojcct a t rCIIT1s 1)npcr I.;iborntor!,. During the 1970s,
he \\,o~~licil
ti)^- H.ir\':~riiU~li\,crsin
dcs.cloping and ~ n a i n raining 1ncclic31c o l n l ) ~ ~ t ifacilities
~ig
at Massachusctrs
C;cncrnl Hospital.
X 'Ii-~~~rs/~ot-l
/r//cr~/ace
(Dayton, Ohio: N<:R (:orpol-ation,
1993).
Windows Operating Systems
1<. Rlnkc, 0/1/i1~7izir
I,C '1Ti17~locis
.\'T IL'i~rrlori:\'.\9'lZc..so11/cr
h'il. \,ol. 3 (Redmond, LVash.: 1\4icrosoti I'rcss, 1993).
14. (:11src1., I~tsi~lc\C71i~/ou:s,\.'I
(I<ciin~o~iii,
\,V;lsli.:
hlicrosoft I'rcss, 1993).
A. King, 717sirle\.L~ii,rcloirs9-5(Rcdmond,\.V;lsIi.: Microsoft
I'rcss, 1994).
i
.
RS L J J ~ I . Cvols.
~ ~ C1-5
C . (Rcdmond,
James G . Peters011
Janlcs I'cterson is currcnrly ;i sofr\vnre rngincer a t
DcLomic Mapping. As 3 m c ~ n b e of
r 1)igiral's Windo\v>
NT group, Jamcs Icii the ~.clc'~scs
of the cScursion softWindows Programming
gl~
2.1. 111.~ciiiitio~l,
\\,'ire fro111\ , c ~ - s i oI ,~I ~t l i r o ~ ~ \,crsio~i
lie \vorkcd '1s ,1rchitcct anti indi\,idual contributor o n
I<, (;11risti.111,117~,I!ic/.oso/iC ; I I ~lo
L ~&'++
> l~ro~rr~r~~rr/iri,q
thc eXcursion project, conccntl-aring o n graphics and
( I L d ~ n o n J\,Vash.:
,
Microsoft Prcss, 1992).
performallcc. E;ulicr, he \vorkcd in the PI\-I'HWOIIKS
\Vi?/i~/gI<cJI/.s(I!)/c~
\V.it/(lo~r:i
1.' l~iI.;lsci:~,\Yitido~~:~++:
2nd Railibo\\ groups. Jnnics \\.,IS employeti b!! Cornpion
Corporation bcforc joi~ling1)igital in 1984. H c rccci\,ccl
(.i)t/o i/i (.++ (Reading, h/lass.: r\ddi.;on-l\lcslc\, I'~1blishi11g
a B.A. ( 1979) in mathcnlntics from Indiana Uoivcrsin
( : ~ I I I ~ ~ I I1992).
I!.,
and an M.S. ( 1 9 8 I ) in mathcmnrics ~ n an
d X I S . (1984)
in computer science, hoth fi-om the Univcrsin of Illi~iois.
I
I
I
I
I
~
I
\.V.lsh.: microso oft Press, 1995).
Scot A . Aurenz
Scot Aurcnz is a principal sol.i\\,nrc engineer ill tlic
Windo\vs NT group \vhcrc he \vorks o n the develop~ncnr
o f the excursion PC: S scr\,cr. Scot has co~irributcdt o many
projects at Digital, inclt~di~lg
tllc 1-anguage Sensiti\,c Editor
(1)EC:sct LSE) and the SUVAS u.orkstation. Scot ciimc t o
1)igirnl ill 1979 as a I'unluc Univcrsin co-op stildcnr nnd
l>cc.lmc n full-ti111ccrnplo!,cc .~licrrccci\'~nghis II.S.F,.E.
in 1982. H c rccei\,cd .In h~1.S.E.k.
from rlic Uni\.crsiry of
Illinois in 1986.
Paul J. R a n a u r o
I'aul Ra11;iurojoincii 1)igital in 1987 .inti is n p ~ . i l i ~ i p ~ I
softwarc cnginccr in the \Vindo\vs N T group. I-lc is
rcsponsiblc fix .ipplication failover for the Digital (:lusters
for Windo\\,s NT product. In earlier \\,ark, he parricip;itcd
in tlic dcvclopmcnt of thc excursion sofr\\,arc and the
ACMSsp rrans.~ctioliprocessing monitor, spccific.iIly,
in the implcmcnratio~iof the RTI protocol. H e ;ilso p;irticipated in rlic implcmcnt;trio~iof rlic ~Vnnuf,icturing
hlcssaging Scr\ ice OSI application layer protocol Fi)r tlxc
L)ECo~iiniproduct and ;i network perforniancc analyzer.
Prior to conii~igt o l)igital, lie was a consultant at I~irics
Systcms and a senior software enginccr at Miconi-I~itcrla~l
Paul holds a l3.A. in history fi.om the Uni\lcrsity of
~Massachusctts;it Bosto~l.
Charles P.Guldensclluli
<:h:irlcs Guldc~ischuliis a principal software enginccr in
1)i~iwl's\Yindo\vs N.1- g r o u ~ >H. e is responsible for color
support slid sofi\\arc installation o f the cXcul-sion product.
I'IU iously, lie \\.orkcd in thc ICcnl-Time Sofi\v.~rc,
1'1.ofcssional 300 Soft\\ arc E~iginccring,slid 1<1'-1 1
F,11g11iccring
groups. (:ll.~rlcs joi~icclDigital r~ticrrccci\.ing
hi5 1i.S. in inforniarion and computer scicnce from the
Georgia Instirutc. o f l ' c c l i o o in 1976.
Val. 8 No.1
9 6
45
I
Integrating Multiple
Directory Services
The Integrated Directory Services (IDS) infrastructure implements a directory-serviceindependent interface. The IDS infrastructure
is used by applications that store and retrieve
information about resources in environments
with either multiple directory services or one
of several directory services. The IDS interface
isolates users and application writers from
the unique requirements of different directory
services by providing a view of a single, logical directory service through a simple federation mechanism. To retrieve resources from
the logical directory, IDS determines its physical location and converts the resource from
a directory-specific to a canonical format.
Extensible schema tables represent the canonical format for each resource and allow IDS t o
represent resources created using both the IDS
interfaces and the directory-specific interfaces.
Margaret Olson
Laura E. Holly
Colin Strutt
1)igital has developed the Integratecl 1)irectory
Scrvices (113s)technology to provide a mcch,~nismfor
integr~tingmultiple directory services into a single system. In this paper, \ve examine the development of the
11)s infi.astructure. We begin by discussing thc problems hced by net\\lorlc directory applications. Nest \vc
describe our design goals, the IDS infrastructure, and
our initial implementation o n the PA-1-HWOlWS
product. Mre conclude \\lit17 a brief discussion of plans
k)r f i ~ t ~ lcle\~elopment.
re
Directory Support in Multiple Environments
Although &rectory services are a powerfill mechanism
ti)r distributing and accessing certain kinds of information, relative.ly few applications choose to use them.
Digital's PATHWOIUG application \\/as in need of a
clirectory for printers nnd file shares. PATHWORKS is
;I 11envorkoperating system (NOS) integration product
tliat gives users ncccss to both Microsoft's LAlU
Manager and Novell's NetWare file and print shares. As
we studied how to incorporate directory support into
PATHWORKS, \ve came t o a better undcrstnnding of
t l ~ cproblems faced by directory applications in general.
Nenvorks are growing rapidly, as are the amount
and kind ofinformation tliat can be accessed through
the netu/ork. We \\we certain that fi~turcnetwork
,~pylicationp r o d ~ ~ c\tvso ~ ~ have
l d an even greater need
for a directory, and therefore a general solution was
needed. We then set out to design a system that would
remove the barriers to directory service application
usage a i d deployment. We resolved the tension
between the product deadline and the time required
to implement the general solution by designing a
coniplete solution and implementing whnt \\[as neccssary to prove the design and t o meet the inimediate
needs of the PATHWORKS product.
Existing Directory Services
There are a n ~ ~ m bof
e rgeneral-purpose directory services. Some of the more familiar include X.500,
Novell's NetWare llircctol-)I Service (NDS), thc Cell
1)irectory Service (CDS), and Banyan Systems'
46
l>igir'~lTechnical Jc~ul-n;~
StrcciI'.llk.' ' 111tlle past, director!( scr\.iccs \\,ere in rclati\,cl\, li~nitcdllse b e c a ~ ~ smost
e
directory scr\riccs
\\,ere tied to either an operating system o r .I tru~isport
c ~con~
o r both. 111 nddition, director!? s e nr' ~ \\.ere
ncctcd to n multiti~dcof applicatio~~
~x-ogl-a~nlning
i n t e ~ - ~ ~(Al'Is)
c c s that \\)ere i~icolnpatiblcand difficult
to ~ ~ siMore
c . ~.ecently,directory services I i ; 7 \ ~been tied
to nct\vork opcrati~igsystems o r applications, ~ l t l i c r
th;ln to Ilost olxrating systems o r transports. If anything, tllc numI7er of L'sta~idard"AI'ls 11~1sgro\vIi.
In I.irgc networks, this complcsity has resulted in
the proliferation of directories, often containing overlapping intbrmation. This makes the network manager's job ciifficult, \vhich in turn creates resistance to
ciircctol-y npplications. At the same time, nenvork and
NOS technology has developed t o a point \vliere an
cvcr-illcreasing amount of information is being shared
o n different machines. To give a silnplc example,
,~lmostevery server at l)igitalls Littlcton site has a connection to the lligll-\~olumcprintcr in the cop!/ center,
\\lit11 a different name on every server. A dircctory
\vould si~npli$ users' access to tliis singlc pliysic~l
resourcc by presenting a single name for tlic printer,
if only the .~pplication\\triter could figure out \vIiich
dil-cctory scr\icc to use and ho\v to use it.
Other Approaches
As discussed Inter in the Ilesign of t l ~ e11)s Frame\\.ork
,111iiSc~.\'iccl'roviders scction, 11)s dcfi ncs both nn hP1
nil .I service provider interface. Support for a n y dircctory service can be provided by \\,riting a service
provider mociulc. ~Uicrosoft'sOLE Directory Services
(OLE 1)s) takes a similar approach to 11)s) \\;it11 a more
limited initial implernentation.Altliougli the current
11)s implcmcntation rutis under Microsoti Windows, it
\\!as designed to port to other systems. OLE 1)s depends
o n features of the Windo\\s operating systems.
The >;/Open Fcderatcd Naming (XFN) specification
\4r.is not complete at the tinie \Ire \\,ere designing IDS,
,111dit tiid not include either a service provider i n t e r f i e
o r n rcfcrcncc implementation." We did cs;lrninc the
XFN d r ~ t anci
i designed the 11)s interface to be colnp.itiblc \\,it11 SFN, with a vie\\, ro\\,arci supporti~igthe
XFN AI'I in the firti~re.Supporting the SFN interhces
o n top o f 11)s \\,auld be a relati\~cl!)straiglithr\vard
tnsk, ,lnd \\,c lia\re considered doing tliis.
The PATH WORKS Application
In rhc NOS cnvil.on~nent,e:lcli NOS hns its own
ciircctor!. o r pseudo-directory. NctWarc version 3
implcmcnrs the Binder!,; NetW;irc 4 implements
N1)S.' The various implementations of Microsoft's
LAN Manager protocols provide a virtunl clircctory
bnscci o n infol-mation niaintained by its cio~nnincontrollers. 111 n multiple NOS cn\.iron~ncnt,the user is
presented \\.ith multiple information sources fro111the
multiple dircctorics. F,\vcn \\lorsc, the user niay be
faced \\,it11multiple information sources even in a sin~ l NOS
e
environment, sjnce there may be ~nultiplc
NetVVnre l<j~lderics
or LAN Mnnager domains.
M ~ ~ l t i pNOS
l e en\~iro~lme~xts
d o not, in and of themselves, causc co~nplcxityand confi~sion.I'roblems arise
\\>henpeople within n singlc environment want to share
resources XI-ossm ~ ~ l t i pc1l\~iro111iients.
lc
For example,
consider 3 c o ~ n ~ i i Ioc:ll
o ~ l area nenvorlc (LAN) configw
ration \\lliere NctWnrc is i~istallcclon tlie clients and
servers for one depart~nentand Microsofi's LAN
Manager (contnincd \\tithin products sucli ns
Microsofi's Windo\vs for Workgroups, Windo\\.s 95,
and Windo\\fs N'T' operating systelns, or the LAN
Server product from International Busi~iess~Uachines
Corporation) is installed o n the clients and servers for
a1iothe1-departtilent. If c,lch department's resources,
users, and administration personnel are kept distinct,
there is n o problem. Howc\lcr, any desire to allow
users to share resources bct\vccn departments, o r to
have cornmon nd~ninistrationo\ler the departme~lts
introduces ~dmi~iistrnti\,c
n~lci uscr problems. If a
printer is to he slinrcd by tlie nvo departments, it must
be administered t\\/ice: once i l l thc NetWare environment rind once in the LAN Malinger en\~ironmcnt.
Users in the tn.0 departments use different names for
the same printer. Later NOS imple~iientations,sucli as
Digytal's PATHCVOl<I<S\.crsion 5.0 or tlic ncnvorlting
sofnvare built into Microsoft's \iVindo\\ls 95 that provides support for multiple NOS protocols, clo nothing
to manage the rnultiplicjty of names for the same nctwork resource.
As \ve were contcmplnting the set of capabilities
we needed to design for the next generation of
PATHWORKS client products, we realized that solving the connectivity proble~liimplied in a multiple
NOS e~ivironmcntwas not enough. User access and
administrator control o f NOS resources needed to be
considerably simpler.
i\s \\.e looked at the problcms in larger networks,
\\.e saw the need for the ability to provide morc sophisticated mc,lns to locate NOS resources. T!.pically,
NOS client soft\varc pro\,ides the means to bro\\.sc
the ncnvork to l o c ~ t cn resource. Ho\\re\rer, browsing
requircs the user to kno\\l tlic location of the resource,
specifically the name of the server, and to be able to
clloose the r c s o ~ ~ r cocn the server by recognizing
something about the resourcc name o r a resourcc
description providcd by the adn~inistrator.What \\/as
needed \\,as a design that .~llo\\zsa user to search, as
e
on vario~lsattrib\\,ell as bro\\rsc, for .I resoi~r ~ based
utes describing the rcsoLlrcc.
Finall!; existing NOS cn\,iron~nents
have a fairl!, Iimired vie\\ of the set of resources that can be referenced .
Vol. 8 No. 1
1996
47
l<otli NetWare n~ld\rarioi~sLAN iLI;ln;iger implementations pro\,ide s ~ ~ p p ofor
r t pri~itcrs,ind file shnrcs.
Wc \\,anted to be able to catcnd tile types ofresources
from tlie ~ic\\,
thiit c o ~ ~ be
l d referencccl and m;~~lagcd
clircctory capability tliat \vc \Irere designing.
Thus we embarked on a design for the facility
initiall!~called IDS, for Integrated 1)ircctory Set-\,ices.
The P A T H W O W \ a s i o n 6.0 iniple~nentatiollwas
evcntu~llycalled Directory Assistant. \Ve refer to this
technology as 11)s t I i ~ - o ~ ~ this
~ l ipaper.
o~~t
Design Goals
As \\'c looked at the rcquircmc~lts of the I'AAI''HWORI<S
p r o d ~ ~ c\\!e
t , ~ O L Itlinr
I ~ ~
m.lny of those req~~irenients
c o ~ ~ techllicaIl!/
ld
be ~iicr\\~itll,III!~dircctory service tliat
\inasintegrated illto the 1'ATHWORKS applicatio~isanci
tool sets. PATHWOIWS I-cquil-cdtlie ;ibility to
Give a single name to resources that cnn be acccsscd
by means of multiple ~ c r \ ~ cor
r sprotocols
I~isi~latcend L I S C ~ S horn changes in the \\.ay
resources are allocarcd among the senrers
1h4;inage resources in Jn NOS-independent manner
We coi~ldnot si~nplypick n dircctory ser\licc and
integrate it illto PATH\IVOI<l<S,because \ve could not
rccli~irctliat all customers deploy a particular dircctor!~
scr\rice at their site. The PATt1WOl<I<S product is
both NOS- and transport-indc~>c~icie~~t;
introducing
SLICII a d e p c ~ i d c ~\\,as
~ ~i~~iacccpt.~blc.
e
LVc qilicld!' realized that these \Yere tlie requil-cmc~itsthat kept many
other applications from s sing dircctory services.
C)ur assumption \vas tliat ~ i i a ~ incn\.ork
y
applicntions \ \ ' o ~ ~ luse
d directory scr\.iccs if they coi~ld,bur
tli;it fc\v of them could nssumc or require a particular
director!/ service. Working from that assumption, \\,e
sclcctcd the follo\\~ingdesign rcq~~irements
for 113s:
llirectory service incicpcndcncc
Ability to access csisting d ~ t a
Ability to join dispar.lrc nLlmcspnccsinto a single,
logical nnmcspacc
l<emo\.al of barriel-s to succcssf~ldeploy~iientof
n \\-idcarea 11et\\~orIi
(\VAN) clircctor!,
Ability to hide directory lialnc synms
Support of scarcli
S ~ ~ p p oofr tapplication-specific directory entries
Directory Service Independence
(:~lstolncrsmust be ~ b l to
e clioosc. tlic director!, scr\ ice
ill \\,liic!i the!, store resource infi)rnlation. Some customers Ii;~\,c3 preferred dircctol.!, scr\,ice, \\,Iiich the!.
\\rho arc not
\\,;i~it
to c o ~ i t i n to
~ ~IISC.
e OtIicr c~~sto~iicrs,
using a particular directory scr\.icc, prefer that Digital
vo1. 8 No. 1
1990
pt-()\,idesthe director!, scr\~icc.In 3 fex~,cnses, a custolncl- might \\,is11 or cvcn need to store information
; I ~ O L different
II
resources in diffcrcnt dircctory services.
Ability to Access Existing Data
A great deal of i~ifor~nationcurrently exists in
application-specitic director!1 services and in NOSspecific dircctory scr\ficcs. A rclati\,cly large nurnbcr
o f applications also use the niitive interf~cest o store
information in the NOS directories. Allo\\,ing users
to .lcccss tliis information dircctl!, through 113s \\.as
critical. \Ye e~pressl!~\\,anreti to a\,oid tl?e need to
ciuplic.lte director!, informatio~~
in separate, incolnpatiblc s\,stcrns.
Ability to Join Disparate Namespaces into a Single,
Logical Namespace
Many directory services arc ailncd nt a specific application o r J set of applications. For csample, current
X.500 deployments contain mostly people infonnntion such as narnes, phone numbcrs, and electronic
mail addresses. (Note: S.500 is an extremcl!, flesible
di~-ccto~-\r
scr\,ice tliat can be used to store almost an!,
kinti of i n f o r ~ n a t i o ~but
~ , for Iiistorical reasons most
dcplo!,ments contain peoplc information.) NOS directorics contain infor~iintiona b o i ~ rN O S rcsources such
ns printers. Consecluently, ninny uscl- environments
lin\,c niultiple directory services, each of \\.hich contains critical business i~iformilrion.To access this exist\yay,
ing c1at.1 and present it to the L I S C i~l l a ~iiea~ii~igfi~l
rlicsc multiple directory namcspnces must be joined
into ;i single, logical naniespncc.
Removal of Barriers to Successful Deployment of
a WAN Directory
Hicr.~rchicnldirectory scr\iiccs gcnernll!, require that
tlic n,lming hierarchy be d c s i g ~ ~ cbefore
d
the directory
is cicplo!!cti. Sincc the hicrarcliy consists of names,
11nd n.1mrs are sensiti\~cnnei political entities, this cnn
be rill cstrcrnely difficult task. Or~nnizationsalso
c1i.ingc over tirne, ti~rthcrcomplicating the problem of
cicsigning a n;lme hierarchy."
01-gnnizationsthat succcssf~~ll!~
cleploy directory services d o so from the bottom up. The NOS directories
arc clcploycd precisely because they a\.oid the probIcms i~lhcrcntin a nanie hieral-chy. An administrator
call set up a No\lcll 3.s Bindery k)r a local organization
\\~itliout\\lorr!ling about ho\v tlic name o f one group
rclntcs to the nalncs of .ill the other groups. The
do\\,nsidc to the NOS directories is tliat they have
,I limitccl nbilit\r to scale bc!,ond a 1.AN. With IDS, \\.c
\\.n~ltcdto p~.o\.idea fra~iie\\,orktl~.ir\\,oi~ldgron. \\'it11
tlic user's environment. A user could start with a local
dircctory hilt jncorpornte that ~iircctoryillto an enterprise o r global directory \\,lien the time \\,as approprintc, \\.ithout affecting the end usel-s or tlic applicatio~~s.
IDS
FRAMEWORK
SERVICE PROVIDER INTERFACE
SERVICE
PROVIDER
n
NATIVE
DIRECTORY
CLIENT
NATIVE
DIRECTORY
SERVER
SERVER
H
NATIVE
DIRECTORY
CLIENT
SERVER
Figure 1
Structure of the Integrated Directory Services
entry; it is a directory scrvice object that represents
some netc\~orl<
object. A tpesoL(rct'class is tlie definition
of that type ofdirectory entr)l. For example, the directory entry that describes a spccific printer is an IDS
resource, and the IDS class that describes every printer
entry is a resource class.
The framework provides extensibility by dcfinu~g
C++ object classes that allow for the creation and
manipulation of resources, attributes, and attribute
values in a type-indcpendent manner. Tlie type independence allows both applications and the frame\\~ork
itself to 111anipulate IDS attributes and attribute values
~vithoutknowing their types. As long as the new types
are built o n top of existing IDS system types, application writers may define new IDS types without nlodifylng the service providers.
The framework dispatches directory operations to
the appropriate service provider and maintains overall
systeni state and integrity. It maintains a list of the
service providers that are currently available and
shows the errors encountered in any failed loads.
This allows the system to continue to opcrate, albeit
in a degraded state, even though one of the service
providers may be rnalhnctioning.
Before we discuss the design of the SPI, we describc
the framework's objects.
IDS Entry The funda~nentalIDS object is the canonical representation of a directory entry, the IDS entry.
vbl. 8 No. 1
1996
The 11)s entrp is an abstract object. To create a
resource class, applications define a resource type and
derive it from the IDS entry. IDS entry objects are created and manipulated through the API and translated
into the appropriate native director!l format by the service providers. Derivatives of the IDS entry may define
additional methods, but they may not override the
IDS entry methods. The IDS entrp methods are part
of the framework.
The IDS entry methods h11 into one of nvo
categories: those wl~ichmanipulate the attributcs and
values contained in the IDS entry in a type-independent manner, and those which perform operations on
the directory. Each IDS entry, each attribute, and each
attribute value contains a type. For convenience, derivatives of the IDS entry may defile additional net hods
that manipulatc certain attributes or values directly.
For example, a derivation that defines a printer might
define a method to set the description attribute. The
implementation of this method would call the general
IDS entry attribute and value manipulation method
to set the value of the appropriate attribute.
As shown in Figure 2, the IDS entry contains identiGing i~lf'or~nationand the attributes and attribute
values that describe the resource. The colltext identifies the service provider that performs directory operations on this entry and the location \vithin that
directory service in which this entry is stored. The
resource type defines the kind of resource that this
entry represents. The resource name is the name by
which applicatioris and users refer t o the entry.
The a t t r i b ~ ~ t of
e s the entry are contained in a set.
Each attribute in turn contai~lsthe value or list of v.11ues associated with the attribute.
Contexts Tlie context is an object that uniquely iden-
tifies a partici~Jarlocation in a particular riamespace.
Thc IDS contest is very similar in concept t o the SF&
c o n t e ~ tAll
. ~ contexts contain the type identifier fc,r
the directory service and an internal name. The type
identifier is used by the IDS fran~exvorkt o dispatch
operations to the appropriate service provider. The
internal name is the location within tlie directory service described by this contest. The internal name is
represented in the native syntax of the underljring
directory service. The service provider is responsible
for setting and maintaining this internal name. (See
Figure 2 . )
Attributes and Attribute Values The type of an
attribute defines the data type of its value o r values.
The attribute value object is a canonical representation
of an actual attribute value. The attribute value objcct
defi ncs a set of methods for accessing and nianipulating \lalues. For each data type supported in IDS, there
is a corresponding attribute value derivation in the
SERVICE PROVIDER TYPE: LDAP
LOCATION WITHIN SP: o=dec,ou=lkg
CONTEXT
ATTRIBUTE SET
AlTRIBUTE N
ATTRIBUTE 2
ATTRIBUTE 1
IDS-ATTR-MAINTAINER
ATTRIBUTE VALUE LIST
JANE DOE
ATTRIBUTE VALUE
Figure 2
IDS Entry
IDS framework. This allo\vs applications, and the IDS
frame\vork itself, to manipulate 'ittribute \ d u e s withO L Ikno\\ling
~
their types. The service providers, on the
other hand, use the type information to translate from
the IDS data formats to their native data formats.
Types T o allow customers and third parties to identi@
their own IDS resources, the IDS type mechanism
must i~niquelyidenti* objects. Thc two identifiers we
considered using \\/ere ~~ni\lcrsaluniclue identifiers
(UUIDs) as defined by the Open Sohvare Foundation
Distributed Computing Environment (OSF DCE) and
object identifiers (011)s) as defined by the open systems interconnection (OSI) ~tandards.",'~
Some directory ser\tices identil+ attributes witl~OIDs, while others
use UUIl3s. For applications defining new resources,
we wanted to avoid the necessity to obtain both an
OID and a UUID. It is possible to encode a UUID in
an OID, but the reverse is not ~ I . L I C .
We could encode a UUID in an 0 1 D by registering
an OID prefix. The prefis \vould indicate that the
sequence aker the prefix was a UUID. UUIDs are
fixed-length structures generated fro111 time stamps
and Ethernet addresses, and therefore arbitrary information such as an OID cannot be encoded in them.
UUIDs are also easier for npplicatio~~
writers to generate because numerous systems ship with tools to
generate them.
Certain directory services, for example X.500, have
external type definitions for the directory entries. I t
is possible t o define a generic entry and then map
arbitrary \ialues into that entry, but IDS entries would
not be meaningfill when viewed with the native directory management tools. We felt that this was unacceptable, because it would make the management of
IDS entries in the namespace much Inore difficult.
Some systems use UUIL)s to represent the type information. We chose to use UUIDs since they are both
easy t o generate and can be used in both UUID and
OID class definition systems. The L I S of
~ OIDs n~ould
require UUIDs t o be generated for UUID-based
systems and mappings to be maintained.
Digital Technical Journal
Vol. 8 No. 1
1996
51
Communities An 11)s comniunin is both an aci~nillistr;~ti\*c
srouping mcchnnism and ,I logical location for
IDS resources. When pcoplc interact \\.it11the TDS system, they scc a community .is tllc orpclnizins principle.
;idministrator controls tllc boundaries and nlelnbership of an 11)s cornmunit\,. Typically, a cornmunit\.
represalts either a partic~113rlocation sl~cll3s a building or .I fi~~ictionnl
grouping s ~ ~ casl in \\,ark g r o ~ ~ p .
Initi,~lly,\\!c co~lsidcrcda supcrcol~tcstto join multiple directories into a single logical directory. This
S L I ~ C ~ C O I I ~\\'o~~lti
C S ~ hn\.c cont.lincd m~~lriplc
contests,
one for c ~ c l itype o f I-csourcc suppo~.tcdb!, IDS. We
c\.c~iti~,~lly
s ~ ~ b s u m ctlic
d supcrco~ltcstinto 3 cornmun i p and called it a rcsourcc context list. An 11)s community is stored ;IS :I special object i l l the dircctory.
Each commn~iity'srcsollrcc context list describcs the
dircctorics that makc up the community. The resource
contcst list is the federation ~ncchanismby which IDS
deter~nincswhere rcsoilrccs of cacli type are stored.
Each entry in thc rcsourcc contest list is n pair of
resource type and contest. As users and applic;itions
operate o n entries in a co~nrni~nit!~,
tlic 11)s fi;~me\\lork
(through IDS entry and c o ~ n m ~ ~ nmethods)
ity
inspects
the rcsourcc type and the c o n l ~ n ~ ~to
n idcter~ninc
p
the
contest. Figl~re3 illustrates all 11)s community.
One of the prol.>lcms \vc anticipated \\,.is that large
organizatio~ls\\roulci uaturally tend to Ila\.e many 11)s
cornmuniries: Ho\\s \vould tllc uscr identi6 these? We
considered an additional liicrarch!, in \\.llicl~commLInities \\.auld be nlembcrs of other conimunitics. Our
usability cons~~ltants
cmph;isizrd that users should not
have to bro\\.se a hierarchy to ncccss resources. In
response, \Ire dc\reloped the concepts oftlie local nnci
the home c o ~ l l ~ i ~ ~ uTlic
i i t y local
.
comm~lnit\,is associated lvith the machine 3 Llscr is c i ~ r r c ~ ~11si11g-it
tly
represents a physical location. The home community
is the one \\it11 \\~l~ich
the uscr is associateti 01- bclongs.
We en\isioned that the home comm~~nit!,
\\,oi~I~i
L
x
the same as the local community at the user's normal
place of \\:ark, but there is no rccluircmcnt inherent in
the design that things he orsanizcd this \\lii!~. For
esamplc, iEa user is ,~ssociatcd\\iitli t l ~ cc o r n ~ ~ ~ u nati r ! ~
her work site and the machi~icshe uses is also located
at that work site, both her local community and
COMMUNITY
DEFAULT
CONTEXT 1
/
Svc Provider Type = F7801DB7-F675-1lCD-A8C2-08002B187DlA (ODBC)
External Name = IDS-Group Community
Internal Name = E.\\tuxedo\idsodbc\idsdbdir.mdb
Svc Provider Private = NULL
RESOURCE COKlUCT L16T
RESOURCE.
CONTEXT
Svc Provider Type = EFF4B840-EC52-11CD-9E5E-08002BBA95CA (CDS)
External Name = ids-cell.lkg.dec.com
Internal Name = ids-cell.lkg.dec.com
Svc Provider Private = NULL
CONTEXT
OBJECT
TYPE
-
PRINTERS
/
FILE SYSTEMS
KEY
n
COMMUNITY
1-
RESOURCE CONTEXT LIST
RESOURCE CONTEXT
OBJECT TYPE
CONTEXT
Figure 3
111s C : o m ~ n ~ ~ n i n
Svc Provider Type = C723E850-A1A6-1OAB-A699-08002B361FC1 (LDAP)
External Name = c=us;o=dec;ou=IDS-Group Community
Internal Name = c=us:o=dec;ou=IDS~Group Community
Svc Provider Privale = YUMMY. 386. TCPIIP
her home community represent this \\rork site. If this
user \\,arks at another \vork site and uses a different
~iiaclli~le,
her liome c o ~ n ~ n i ~ nremains
it\i
tlie saine, but
her local comrnunity reflects the community where
the ncc\( rnachi~lcresides. The concepts of local and
home communities d o 11ot reduce the number of
communities, but thcy d o pro\!ide a direct method by
\\'llicl~users can access t l ~ ccommunities that contain
the resources they most fi.cquently use. The local and
11o1nccommunities are n con\,enience; users and applications are in no \\lay rcstrictcd to those communities.
Search Support Searching is liaudled by the search
objcct. The search objcct contaius a community (or
list of commi~nities),a resource type, and an attribute
filter. l ~ l l eattribute filter supports both eql~ality, ~ n d
comparison matching of attribute \ d u e s and allo\vs
callers to construct co~nplcsrequests by concatenating
comparisons together in a series of Boolean opcrations. For examplc, a caller could construct a filtcr
t1i;lt rcturned all printer objccts tliat (((are located
o n Floor2) OR (are located o n Floor3)) AN13 (support color printing)). C:ombi~led with tlie local and
liome comrni~nitysupport, fi ltcrs allo\v applicatio~ls
and users t o express ideas SLICII 3s L'pri~itthis at tllc
closcst printer tliat supports color, n\,o-sided printing,
and then transmit it to an!, f.~csimile~nachinein my
IIOIIIC community."
-.
1he scarch object's dchult tiltcr returns all objects of
the resource type in the local co~nniunity.The scarch
object resolves the cornmiunity to a contest and pnsscs
it to the servicc pro\,idcr. :The ser\~icepro\,idcr constructs a list of matching 11)scntr!l objects to return to
the user. In IDS, tllc searcll objcct supports bro\\aing.
,.
1hc search objcct has ~ncthodsthat display a dialog
and construct filters based o n user input. When
designing tlie system, \\'c debated \\,herher it \\!as bcrtcr fbr the search objcct to contain both the filtcr and
the smrch dialogs or whether the filter construction
belonged in the 113s c11tl.y. We chose to Iteep the
scarcli dialogs separate from the IDS entry. Espcricncc
\\tit11 i~nple~iicnting
resources derived from the I I X
cntr! has sho\\!n this t o be an error. Currently it is ncccssary to derive ti-om nvo objects, IDS entry and the
sc;ircli object, to implc~ncntn resource t l i ~ thas n
resource-specific senrcli dialog. We \\rill be modit\ling
the scarcli and 11)s entry objects so that the construction of tlie filters and the dialog tliat constructs the
filters are IDS entry mcthods.
Schema The service pro\.iders translate bet\vcen the
nati\,e directory objcct and the IDS entry. I n general,
directory scr\,ice entries arc not self-describing. I n
existing directory sel-\,ices, either a schema o r tlic
application is expected to know the director!,-specific
format of the data. The latter is more common tll.un
the former, and in any case the schema mcthoclologies
are unique to CLICII directory servicc.
From tlie point of\ric\\l of the nati\,e directory service, IDS is the application. To properly convert the
data, the scr\licc providers must know \\!liar it is. The
scr\~icepro\lidcrs use the schema to detcr~ninctlic correct attribute and value types to use \vlien constructing
thc IDS elltry of n p;irticular ~ p c .
The schenia describes resource types, attribute
types, and nttributc \,slue data types. Logically, the
schema is a set o f tables, one for each scl.vicc provider,
which maps the nnti\lc name or type to tllc 11)s name
or type. Tllesc tahlcs are read by tlie IDS schcma component \\/lien 11)s is initializcd. Bccausc tlicsc t;~blcs
are external to the system, they can be modified by
users or applications.
There is one limitation o n thc cstcnsjou of the
schenia: Ne\v attribute and resource typcs can be
defined, bur tlic!~must be composed horn tlic predefined IDS attributc value types
that the scr\.ice
..
providers can support. The servicc pro\,idcrs \\,ould
have to be moclificti t o support additional attribute
value data typcs. This limitation is not as scvcre as it
at first appea1.s. A rich set of data types is defined in
the existing directory services, and a relatively small set
is in comnlon LIS~FC.
By defining tl.lc 11)s data typcs to
encoliipass tlic set of data nipes dctincd by existing
directory scrviccs, \ire ha\te reduced tliis lilniration t o
n tlieoreticnl r;ithcr than a practical problem.
As a conscquencc o f tlie use of schema, ~ipplications
I I I L I S ~spec$ tllc resoi~rcetype for any 11)s operation.
This is a limitation that in principle docs not exist in
other directory systems. M e r some consideration, \ve
concludcd that fc\v i~scfi~l
operations can be performed
on an object \\,hose type is u~lkno\\rn.To pcrfcxm an
operation on ol7jects of all types, the schema can be
interrogated for the list of all supporrcti 113s object
types, and the opcmtion is then iterated over each type.
The System Object Tlie system objcct l o ~ d s2nd
initializes the servicc pro\riders. O n initialization, the
systenl objcct constructs a list of the a\,nilable service
pro\iders from those defined in a local configuration
tile.
Tlie system objcct constructs and maintnins tlie list
ofluiown commu~litics.The system object obtains tliis
list using tlic follo\ving meclianisnis:
Inspect a \vcll-k~lou~n
location (if one exists) to see
ifit contai~lsa cache of luio\\:n con~mu~iitics.
For each ser\,icc pro\,idcr, call tllc disco\,cr method
to ask the ser\.icc prolrider for its list of kno\\tn
conimunitics.
If the system objcct is initializing for the ti 1st time.
prompt the user to create a cornmi~nity.
Application Programming Interface
As mentioned previousljl, we divided the framework
into an API and a service provider interface (SPI). Tlie
APT consists of tlie search object methods, tlie IDS
entry methods, the attribute object and value object
methods, and the system object methods necessary to
access communities.
Service Provider Interface
The SPI specifies the interface between the 11)s framework and the native directory serviccs. It defines tlie
semantics for aU operations that lnay be performed on
IDS information regardless of which directory service
stores the information. The SpI effecti\rely insulates
both the IDS frame\vork and the IDS applications
from the ~ ~ n i q usyntax
e
and requirements of different
directory services.
A directory-specific modu lc, cal led a scn!iceprol?idcr
lihmq~,provides a directory-service-specific iniplementation ofall SPI operations and translates resource in formation back and forth between thc 11)s cntry and
the service-provider-specific format. A service provider
library must be implemented for each directory service
to be supported by IDS. Any directory scrvicc or jnformation repository system that can provide the IDS SPI
semantics may be nn IDS ser\jice pro\lider.
SPI Semantics The IDS SPI defines the following
main operations: create, read, search, modifi: ciisco\,er,
and delete. All SPI operations specifi the namc of the
IDS community upon which to o p c r ~ t c .Each 113.5
co~nmunitymaintains a list of contexts that spccifil
in which service provider IDS rcsoilrccs of a partici~lar
type are stored and in what location within the service
provider. The SPI uses this com~.l~unity
name t o
rctricvc the context information that directs the operation t o the correct service provider library. With the
exception of the delete operation, \\~liiclirequires an
cxplicitly set context (to be sure that an cxplicitly
located object is selected for deletion), if the caller
does not set tlie comniunity name, the local c o n ~ n i u nity is assumed.
The create, delete, niodit), and redd f ~ n c t i o n sall
operate on a single IDS resource at a time. Each,
therefore, provides an IDS cl~tryobject t o idcntilj,
and/or describe the resource.
The create operation creates a new 113s resourcc in
the dircctory. The create operation specifics the type of
IDS resource to be created, the resource's name, and
tlic IDS attributes and values associated with the
rcsourcc. O n a successhl create operation, the scrvicc
provider constructs a uniql~edirectory-specific nilme for
tlie new IDS resource and stores this namc in the
object's IDS cntry. The scrvicc pro\~idcrsubscclucntly
may use this name to find the object morc cluicklv rather
than constructing it from thc name, resourcc e p c , and
contest information contained in the IDS cntry.
54
I)ig~rdl
~'CCIIIIIC;II
Journal
Before constructing the resource in the directory,
the operation validates the IDS entry against thc
sche~nato ensure that it does not violate the schema.
For example, attempting t o create a resource ~ ~ i t h o i ~ t
a recluircd attribute value pair \liolates the schema and
is flagged as an error. Conversely, the delete operation
removes the 113s resource frorn the directory.
The m o d i 6 operation updates the attribute and
values associated with the resource in the directory.
The niodit) operation supports the follo\ving updatc
directives:
Adda~ie\vattrib~~tca~id\lalue.
' Add a new value to an esisting attribute.
Replace a value of an esisting attribute.
~an I
~
~its ~
,D
Delete a \,slue frorn an esisting attribute.
Each m o d i 5 directive is verified against tlie scliema
before being applied t o the directory.
A read operntion retrieves a i~niquely specjfied
IDS resource from the directory, translates it into
IDS entry format, and rcturns the IDS entry to
the caller. Tlie read f ~ n c t i o nis typically used to compare the directory format of an IDS resource t o one
maintained in niclnory by an application, or to process
IDS resources returned from a search operation one
at a time.
The search k~nction identifies and returns 11)s
resources that match the characteristics specified by
the caller. T o bound tlie scope of the search, the caller
specifies tlic follo\ving search characteristics: resourcc
type, com~nunityname or namcs to be searched, and
a filter containing attributes and associated values or
valuc ranges.
The discover operation is called by the IDS system
object t o find all conimunities kno\vn t o a given service provider. Scrvicc providers for directory serviccs
that support a server solicitation and advertisement
nenvol-k protocol implement a discover fi~nction.In
these dircctorics, scr\.ers advertise their presencc in
response to nenvork solicitation requests. Tlie discover method uses the directory's native solicitatio~l
and advertiscmcnt protocol to discover local dircctory
servers and then issues the appropriate operations to
the server to determine if it has defined any IDS communities. Scrvice providers that d o not have a solicit.1tion and advertisemcnt protocol can implement an
alternative discovery mechanism such as retrieving the
co~iirnunityinformation from a file or provide 110 discovery mechanisni.
Construction of the System: Directory, Session, and
IDS Entry Objects rTlic SPI is constructed of three
framc\vork objects: tlic director!! object, tlie session
object, and the directory operntion methods of the
IDS entrv object. The directory object w responsible
for scrvicc provider initialization and termination,
~nnintcnanccof session objects, and con~munitydiscovery. Each service provider exports one directory
ohjcct to the IDS frarnc\vork. The sessio~l object
implements all the directory operations o n a servicc
provider. Session objects are obtained from the service
provider by nlcans of the directory object. The 11)s
cntry directory opcmtion methods deterrninc the contest if it lias not been set, obtain a session object from
the proper dircctory, and dispatch the operation to the
nssoci;ltcd scl-vice provider tluough the session object.
For cfficicncy, session objects may be cached by the
servicc providers.
perfectly presenres the sec~lrityinherent in the underlying directory services.
lmplementation Considerations
Service Provider Considerations
Once \ire had established our basic approach, \vc
t11rncd our attention to implementation decisions.
Client versus Server
Our first consideration was whether to implement this
tcchnolog, as sohvare csecuring on a server system or
as sofn\~nrccxccuting o n a client system. The server
solution had a num bcr of attractive qi~alities:it \\rould
not be necessary to lia\le all the native directory clients
on all the desktops, and potentially complex processing \\T~LIICIoccur on an appropriate platforni.
Ho\vcvcr, \vc identitied nvo problems with the server
solution. Tlic first concerned security. T o access the
director)! ser\!icc o n hchalf of a partic~llaruser, we
wot~ldhave to impersonate that client user on the
scrvcr nlachine. Although this can bc done without
exposing security holes, doing so adds another layer of
complexity to the problem. TIlc second probleni \\lit11
tllc scrvcr solution \\.as that it required the custolner
to find ;i machine for and deploy a server prior t o getting started \\!it11tllc sjrsteln. One of the design goals
\\.as to rcmovc barriers t o directory deployment, and
\\,e \\*ereconcerned that a server solution \\,auld add
'1 barrier, We sn\v a need for both client- and ser\.erbased solutions, and since the client solution \\!as s i n pler to implement, w chose to start there.
Security
The 11)s interfaces Ienvc security to the underlying
directory scrviccs; \ve did not attempt to abstract a
general-purposc, access control or authcntication
intcrtkc. ?'lie primary reason for this was a conviction
that the vast majority of current directory information
is world read, and thcrcfore a complex access control
intcrhcc \\,as not necessary. An access control and
a ~ ~ t h e n t i c a t i olayer
~ ~ that \vas director!!-serviceindependent w o ~ ~ llia\!c
d added significantly t o thc
complesity of the project, and \ve chose t o postpone
tliis problem. I l>S docs pass requests directly to tlie
native directory-scrvicc client; IDS does not alter
or inipcrso~iatethe t~scr's identity. In that sense, it
Filter lmplementation
Tlic implementation of the IDS attribute filter is based
o n the string filter as defined in RFC 1777.'; The
Lighnveiglit Directory Access Protocol (LDAP) string
filter provided a convenient internal representation,
and we \\lould be able t o reuse the LDAP parsing and
processing code that \ile had developed as part of an
earlier product. We considered using SQL to construct
111s attribute filters, but chose not to d o this for implemelltation convenience.
Initi,illy, \ye thought that developing a directoryser\~icc-illdependentinterface would not be difficult.
Most of the required operations such as read and write
are straiglitfor\\~ardand ob\~ious.The il-nplementatio~i
of such a11 interface, hou~ever,proved to be difficult
bcc;iuse the i~nderlyingdirectory services have, in some
cases, very different native capabilities and semantics.
We cliose to iruplement ser\lice provider libraries for
the k)llo\\*ingthree types ofservice providers:
Open Database Connect (0DRC)-compliant
database
S.500-based directory using the LDAP
1)CEC:DS
These scrvicc providers are reprcsentati\!c of tlic types
of directories that exist today. Table 1 highlights soliie
of tlie differences among the three directories. As
tliis table illustrates, not all directories can nativcly
support the semantics described by the IDS SPI.
In these situations, we have follo\ved three alternatives: ( 1 ) thc service provider library iniplcmcnts the
filnctio~iality, (2) the IDS framc\vork iniplemcllts
the f~nctionality,or ( 3 ) in a small rlumber ofcases, the
scrvicc provider cannot implement the fi~nctionalinl
and remains less fi~nctional.
Some operations cannot be supported nati\.ely by
only one or a small handful of directory ser\,ices. For
these operations, we require the service provider developers to ilnplement (or enli~lateas best they can) the
fi~nctionalityin thc specific servicc provider library for
that directory. For functions that a number of service
providers c;lnnot support or that arc sufficiently difticult to implement, \ve provide a cornliion iniplemcntation o r emulation in the IDS framework that service
provider libraries can call. For example, CDS does not
natively support an attribute-based search mcchallisrn.
Ilather than attempt to implement a CDS search capabi lit?:\ve chose to provide an IDS frame\i~orlc"prune"
fi~nctionthat applies an IDS filter to a list of IDS entries
and returns only those entries that satis6 all conditions
of the fi lter. Service providers such as CI)S call then
Table 1
Differences among t h e ODBC, X.500, and CDS Directories
Functionality
ODBC
X.500
CDS
Distributed directory service
Hierarchical organization of directory information
Attribute-based search
Attribute value-based search
Native schema support
User can extend IDS schema
Transactional semantics
Tolerant of intermittent connectivity
Provides security mechanism on connections
No
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
emulate the IDS search tinction by enun~enitingall
resources of a particular type and then call the prune
function to pare down the list of resources.
The IDS schema irnplcniclitation is another csaniple
of a common capability we have provided for all service
providers t o use. Not all service providers support
object, scliema and, of those that do, fewer still can SLIPport user extension o f the schema. We chose to ;illow
user extensibility and implemented a service-providerindependent sclic~naintcrfiacc and ~iiechanism.
In a few instances, \\re determined that it ~ v o ~ be
~ld
too expensive in terms of implementatior~time to provide a service-pro\lidcr-specjfic or an IDS-franic~/ork
implementation of an SPI-mandated h~nctioo. In
these cases, we allowed the service pro\lider to remain
noncompliant. For example, a call to initiate a session
to a ser\~iceprovider specifies user name and password
arguments. For those directories that support user
namc and pass\\~ordsecurity mechanisms, we preserve
that functionality. For directories such as the ODRC
service provider that d o not support thesc security
mechanisms, howcvcr, wc provide n o additio~ialsccurity measures. The cost to implement and deploy such
a security mechanism o ~ ~ ~ v e ithe
g h gain
s ofhaving the
additional features.
In addition, we found that not all directories provide the same semantics for a particular operation. For
example, when ~ ~ p d n t i na gresource, service pro\,idcrs
handle existence checking of resource attrib~~tes
diffcrently. If requested to add an attribute vnluc to an
attribute that dvrs not yet esist, one servicc provider
returns an error, while another implicitly creates the
attribute. We worked around problems of this type by
carcfiilly speci@ing the semantics and error conditions
of all SPI operations. Service providers that d o not
nnti\rely support tliesc SPI semantics must implenlent
whatever additional firrlctionality is required to d o
so. For example, the CDS service provider required
additional functions that determined and flagged
whether or not a pal-ticular attribute cxistcd.
56
Digital Technical Journ..,
Yes
Yes
Yes
N0
Yes
No
No
N0
N0
No
Yes
Yes
111additio~it o ;ill errors that are specific to ser\.ice
pro\jiders, uJc return an error that is iodcpendent
of any 11)s fi-anie\vorlc service provider. This adds
another lc\lcl of consistency across 0111. sewiceprovider iniplcmentations.
Current Applications
As \\,it11 any fo~rndationtechnolog\,, the proof of its
viability l~cswith the applications t h , c~r ~ ~ p l oit.y I n the
PATHWORIG product,
currentl\. I i ~ \ ~
three
c npplications that L I S 11)s:
~
Network Connect
IDS Adnlinistration
Resource Synchronizer
The Nct\vork Connect application finds and connects L I S C ~ S 'pri~itersand file shares. It pro\,ides 1' user
interface that allo\\ls users to bro\\.sc o r search for file
shares or printers. Through Net\\,ork Connect, ~lscrs
call refer t o resources by their logical name or their
attributes. A single physical prilitcr, ~vitliqueues on
several ~iiachincsor several NOS spstc~iis,is presented
to users as a single printer. Network Connect uses the
IDS API to access the IDS search capabilities and
to translate a printer or file share's 113s name t o its
network-specific name to connect to the resource.
Nenvork Connect may be acccsscd through the
Windo\\,s version 3.1 Print Manager anti File iianager
utilities and through the PATHWORKS Nenuork
Connect utility.
Tlic IDS Administration utility (11)s Admin) allows
a nebvorlt adn2inistrator to man.igc 11)s resources
and communities. IDS Admin is integrated into the
Digital iManagcWO1US \Vorkgroup Administrator
for Windo\\a sohvare product." Admin creates, modifies, and deletes resources and communities. It
also allo\vs Llscrs t o browse IDS resources and comrnLr nities in the ~\/lanageWORI<Shicrarch!l and to search
for IDS resources.
An adnli~ustratorcan manage IDS resources manuJ I I ~through the lMc~nagck\iON<Suser intercicc o r can
L\lesler. We \vould also lilce t o thank the members o f
the Director!, Task Force, D a h iMing Cliiu, Dennis
Giokas, and Willia~nNichols.
rely o n information pro\!ided through t h e semiautomntic resource collection utilities called the R e s o ~ u c c
References
Gatherer and Resource Synchronizer. T h e l<esourcc
G a t l ~ r r e r periodic:illy collects informntion a b o ~ ~ t
1. CCITi /?oco111117c11clntio17
X. 501 ( 1992) and
nenvork LAN Manager and N e t w a r e printers 2nd file
//7/~~1~117L~/i017
7?C-/?lloIogl~-ope11 ,S)!~/C~/Il.S In/l?rsharcs. T h e Resource Synchronizer utility processes
c o ~ ~ ~ t ~ c / i o ~Dit~ecto~j~:
/ - T / ~ ~ ~ ,l/otlc/.s. ISO/IEC
the gathered information, updating the directory. It
9594-2: 1992 (Geneva: Incernarionnl Organization
also eliminates duplicate entries and discards informafor St.qnd.l~.tiization/Inrernation.ll Elccrrotcclinical
tion the administrator islie lies t o ignore. T h e gatherer
Commission, 1992).
:lnd synchronizer allouj tlie dircctory t o be kept up-to2. "Naming <:onccpts" in LJsir?g1\i,/ U'i11.c~
,%>/.l~icc.s
J~I.
date, even if resources arc added o r removed t h r o ~ ~ g l l
i\il.!bL%~(l'ro\,o,
Utah: N o \ ~ l lInc.,
,
1993).
the native N O S intcrfaccs.
Future Work
111the f i ~ t i ~ rwe
e , plan t o improve the IDS estensibjlity
mechanisms. Curl-eotly, a local copy o f t h e schema
exists o n every client. Propagating the changes t o cach
client will become a p r o b l e ~ nas users and applications
cstcnd t h e schema. W e are considering storing either
tlic schcma o r a pointer t o tlic schcma in the dircctory.
T h e current 113s implc~nentation runs o n b o t h
the Wilido\\,s versioli 3 . I and \,ersion 3.1 1 o p e r n t i ~ i g
S!~S~CI-J~S.
We are currently p o r t i ~ i git t o Mlindo\\,s 95
anti in\,estigating ports t o otlicr operating systems,
such as U N I S .
TIic implementation docs ~ i o support
t
t l ~ centire
113s design: N t l i o u g h resource contest lists arc i ~ n p l c mcnted, tliel-e is n o reasonable \\ray for a L I S ~ oI r
ndnlinistrator t o crcatc tlicm. T h e Llscr interface \\lorl<
ti)r these features in the 11)s Admin application has n o t
!let bee11completed.
Summary
11)s provides a mechanism for integrating mi~ltiple
di~.cctor!! services into 3 single system. I t is predicated
on tlie ability t o define a c o m m o n set ofdirectory opcrntions and o n the type j~lformation.T h e i m p l c ~ n c ~ i t n tion o f three very ciiffcrcnt service pro\liders-<:I)S,
S.500, dnd ODI3C-i1idicates that we succecdcd in
d e f ning the dircctory opcratiol~s.T h e L I So~f IDS in the
PATHWORKS product sl~o\vsthat it does address tlie
p r ~ t i c a aspects
l
of the problem o f integrating multiple
directories into a single, logical dircctory scrvicc.
Acknowledgments
\Ve \ V O L I ~ Ijke
~
t o th.inlc the Inany past and p r e s e i ~ t
d the
~ n c m b e r so f t h e IDS team \\.ho c o n t r i b ~ ~ t cto
design and i m p l c n i c ~ ~ t a t i o ~f ithe product. Spccial
thanks t o Konsnntinos Baryianies, Anthony
H i l ~ s m a n , David Magid, Tracy Teng, and Talnnr
5. "Microsot't Directory Services Str.ltcgy," a \vhitc
paper from the l3usiness Systems Tccllnolog)~Scries
(Redmond, kviish.: blicrosoft C:orporarion, 1995).
6 . ,VOpo7 Cil I:' .5~)cc~t/icc7/i01~,
t7~~/rrzr/ctl
I\(/ 11lii7g.
The
A%\; .Ij)cc!/i'critio17(Reading, U. I<.: X/Opcn Corn-
px1y Ltd., 1995).
".
S. Radicati, "I~i~plcrncllting
the 13I'l'" in A'. 500 1)irc.cto/;)'.So~~ric(,.s.
7i~c/111o/qy)~
6 1 ~ 1 ~/ I1 c ~ / ( ! ) ~ I I(Nc\\f
I[JII/
York: \J\ln Nosr~undReinhold, 1994).
9. P. h.lock.ipcrris, "l)ol~~ainNames-(:onccprs
.111d
Facilities," lnrcrncr Engineering -F.lsk Force, RF<:
1034 ( N o \ , c n i b c1987).
~
10. P. ~Mockapctris, "l)oniai~l Na~ncs-Implc~iicnration
and Spccif c,~tion,"I ~ ~ t e r n Enginccrillg
ct
T ~ s k1401-ce,
liFC 1035 (Novcnibcr 1987).
1 1. AES/lli.st~~/i?/~/c~~/
C , ~ ~ I ~ I ~ > I I / ~ I ~ < ~ - / ? ( >/ I' I? I. O
O /C
O O~/LIY~
A ((:am bridge, Mass.: Open Sofn\.are
C'nll. Al$)c,~rrli.~
Foundation, (99.7).
12. C C l T l ? ~ ~ c . o ~ r ~ ~ ~ ~ c ~
208
r ~ c( l1992)
r r / i o ,and
r 11?/i)t.i11u~ - o p ~ ~l // l7 / C ~ / ~ ~ ~ o l t / l ~ ~ c / ~ o / / li0117 ' ~ ~ c ~ / 7 l t o / o < ~ ~s]'st~/ll.s
A/x/~zict. S I V ~ / [ ~ . ;\:ot~l/io~z
Y
0 ) (,/J.S~
,\, 1 ) lSO/lE(:
8824-2:1992 (C;cnc\a: I~ircrnationnlOrganiz'ition for
Sra1ld.~1~diznrio1i/I11rcr1ii1rional
F.lcctrotcsli1~ic.31
Cornmission, 1992).
13. W. Ycong, 'I'. Ho\\cs, ;lrld S . Hariic'~stle-I<illc,"S.500
Lighn\.cighc I>irccrory Access Protocol," lntcrncr
Engineering T.lsk Force, RFC 1777 (h/l;lrch 1995).
14. D.Giokx and J . ILokicki, "The 1)csib.n of ManageM'ORIG: A I'\cr Intcl.f~lceF~-a~llc\i,ork,"
lIi~i,il~~l
7i)c.h\vl.6, no. 4 (Fall 1994): 63-74,
~~ical.Jo/r/.r~~il.
Biographies
Colin Strutt
Margaret O l s o r ~
iblargarcr Olson is A consulting sofnvarc criginccr in the
Ncnvork Sofnvare Group. She was the projcct and technical leader for the IDS de\relopnlcnt project. For the fast
S I X years, slic has had technical leadcrsliip roles in Digital's
Directory Scrvices Group. Beforc joining Digital in 1989,
slic \vorkcd in tlie nenvorking and distributed c o m p ~ ~ t i n g
arcns at Apollo Coniputer. She recci\,cd n U.A. (Sigma Xi)
kom Wcllcslcy College in 1981. She published a paper on
ncnvork licensing in 1988.
Laura E . H o l l y
I .aura Holly js a principal cnginccr wit11 thc Ncnvork
SoFt\\larc G r o ~ ~Shc
p . was a key technical conrril)u~-orro
tlic I I)S dcvclop~ncnteffort. Laura has previously contributed t o the areas of DCE, distributed system, and
kriowlcdgc-bascd system de\~elop~ncnt.
Sllc joilicd Digital
In 1985 akcr rccci\,ing an A.B. (high holiors) fro111Sniith
<:ollcgc. Laura holds a patent and has published sc\.cral
pa~)crsin the area of kno\\,ledgc-based systems.
58
1)igiral '1.cchnical Journal
Vol. 8 No. 1
1996
Colin Strutt is a consulting sofrware engineer and technical
director for Tcalriing Sofn\iarc in tlie Network Sofhvarc
Group, where lie is helping t o dcfine new PC-based software products. Prc\,iously, he has hcld technical leadership
roles in directories, nenvork management, and terminal
server de\.elopnient, and before that led product developmenrs in Ethernet servers and DECnet. He joined Digical
in 1980 from British Airivnys in the U.IZ. H e recciveci a
B.A. (horiours) in 1972 nnd a 1'Ii.D. ill 1978, both in cornputer science ftom the University o f Esses, U.K. H e is a
member of BCS and ACM. He has hvo patents issued and
scveral parents p c ~ ~ d iand
~ i ghas published cstensivcly, pnrtitularly o n management technology.
I
Richard L. Rosenbaiun
Stanley I. Goldfarb
Design of the Common
Directory lnterface for
DECnetIOSI
Digital has developed the Common Directory
lnterface (CDI) as the means by which DECnetl
OSI can now access and manage node name and
address information in multiple directory services. CDI comprises libraries for node name-toaddress translation and a tool set for managing
and migrating node information among different directory services. The Common Directory
Registration API is layered on top of a set of
directory service wrapper routines to provide an
extensible mechanism for adding new directory
services. CDI gives customers greater flexibility
in choosing a directory service and supports the
new multiprotocol capabilities in DECnetIOSI,
which support the open systems interconnection (OSI) standards.
The Common Directory Interface (CDI) provides tlie
ability t o store and rctricvc DECnet node information
from a variety of directory ser\lices. It consjsts of the
CDI library, nrhich enables multiple directory access,
and the CDI registration tool sct, \vhich creates and
maintains node/addressing information in multiple
directory services. CI>I was developed for tlie DECnet/
OSI for OpenVMS opcrating system version 6.0 and
for the DECnet/OSl For Digital UNIX operating system version 3.0.
This paper begins by presenting the product goals
and the background of the CDI design. It then discusses the structure oftlie CDI components, the CDI
library, and the CDI registration tool sct.
Design Goals
As the interface to I)E(:nct node information from
multiple directory serviccs, CDI was designed to meet
the following goals:
Give DECnct ncnilo~-kadministrators and users
a dioice of directory ser\lices.
Provide systcm administrators with an easy-to-usc
node registration tool.
Enablc easy and tlcsiblc configuration of directory
choices.
Pro\lide dcvelopcrs of thc 1)ECnct protocol soh\\,are with a simple internal interface that hides the
complesities and diffizrences between the various
directory ser\,iccs.
Provide a common design for both DECnct/OSI
platforms: the OpcnVMS and the Digital UNIX
operating systems.
Intcropcrate with older, non-CDI systems.
Background
In 1991, Digital upddtcd ~ t sDECnet nenvorl<~ng
products to include the use of the D E C ~ I Idistribi~ted
S
director!/ service.' 1)ECdns providcd a highly scalable,
distributed information source for translating node
names to addresses and addresses to node nanics.
Vol. 8 No. 1
1996
59
I~iiti:ill!; customci- acccptmce of this ndlnc scr\,icc \\.ha
lo\v for a number ofrcasons:
1
Since clients outnumber sel-vcrs, a client-based
approach .~fti-ctsmore systems during the ~lpgrade
process. In spite of this drawback, L\IC chose a clientbased solution for the following reasons:
Adoption of this 11c\r,technology reqilircd 3 significant learning cur\fe.
Sigl~ificant planning cvas required before the
DECdns service could be deployed.
Users of sm~11nct\\forlisdid not need thc fc'itures of
a distributed naming sel-\.ice--the costs o ~ ~ t \\rcighed the I7elicfits. Thcsc cLrstomcrs rccl~~cstcd
a narrling scr\,icc based on local files similar t o
the Phase IV DECnct prodirct.
Customers were deploying a number of other
directory services-in
particular the I>omain
Name System-for storing host ~nformation for
transmission control protocol/internct protocol
(TCl'/IP) neb\ orl<s.'
A ne\v comprchc~lsive ser\.icc, S.500, had the
ad\,antage of being an international stun cia^-d:'
These reasons, togethel- with the need to directly support TCP/IP host namcs ,und addresses, pronipted
Dlgital to incorporate new directory service cl~oiccs
in a ne\v release o f l>bCnet/OSI sofn\rnre.
CDI: Basic Design
Supporting multiple name serrrices requircri decisions
to he made concerning naming syntax, n~i~ltiplc
address formats, ;ilid Ioc.lI file support. Tlicse decisions
affected the design of both tlie C131 librnry and the
CDI registration tool set.
Client-based versus Server-based Design
TIie earliest and most h~nda~nental
clesign ciccision \\,as
choosing bcn\,eeli ,I client-based 01. a scr~cr-bnsecisolution. With a client-based design, support for the \ x i ~ L I directory
S
scrviccs \\lo~~ld
bc accornplisl~cdthrough
a val-iety of client-bascd programniing librarics. With
a set-\let--based design, a single client libmry \ \ , o ~ ~ l d
c o ~ n n l ~ ~ ~ l i\vitIi
c a t c a nc\v "multihcadcd" scr\,cr that
\ \ , o ~ ~fan
l d out to the directory servers.
Iolplernentation of the client-based design would
be lcss complex than tlie server dcsign.
A client-b.lsed design did not Jiave tlie syntax and
protocol traoslation i s s ~ ~ of
e s a scr\rcr-based design.
With a scr\,cr-based solution, client ch.inges \r.ould
still be rccluired t o support nc\v native naming
syllt'lses.
For s~nallinstallations, n o server would be needed
if node information \vas stored in n local file: local
file support \\Ins not possible \\litti a scrver-only
appro.1ch.
Naming Syntax
One of t l ~ cmost \,isible complications when supporting multiple naming services is the need t o recognize
diffcrcnt name syntaxes. Tablc 1 gives the different
syntases fix tlircc lvidely used directory services.
A fi~rtlicrconiplication ofsupporting different name
syntaxes \\,as rhc use of an internal 1)ECdns name
format by nct\\.ork management. One of the goals of
the CDI dcsigri \\,as to allo\\r management requests
to be exchanged \vitli older, non-C1)I systems.
For the initial implementation, (:1)I continues to
support tlie internal DECdns for~nat,rather than use
;I 1 1 c \ \ ~ r no11-1)ECdns
,
specific f i ) r ~ ~ ~alongside
'it
the existing one. As a result, CI)I is rccluircd to map
non-I)F,(:dns names onto the 1)EC:dns format. For
esan~plc,rllc nalnc hq.~:\:,z.c.o~~i
fioln the 1)omain
Name Sysrcm maps onto the 1)ECdns name
DO.11. I/.\:. .11q . \ : l ~ . c o / l i (actually o n to the internal
DECdns form of this name).
Multiple Address Support
Along with the introduction of <:Dl, a major inno\.ation in this rclcase of DEC:nct/<)SI \\/as direct support for I'<:l'/IP transports in addition to the existinp
Table 1
Naming Syntax
Directory Service
Example Name
DECdns
Domain Name System
XYZ:.hq.sales.systeml
system 1.sales.hq.xyz.com
Notes.
The X.500 service is not supported by the first release of CDI
The syntax shown for X.500 is commonly used but is not part of a standard
s ~ ~ p ~ for
x ) r1)ECnet
t
Phase 1V al.ld OSI. To simpli%,the
initial implcn1cntatiol1, IP addrcsscs arc rct~-ic\.ccio~ll!'
frotn the l>otnain Name System (nor from 1IECdns).
Ho\\.c\.cr, rhc design of Cl>I allo\\.s tlic rctrie\-al of
botli kinds of addresses fiom any supported directory;
for cxnmplc, OSI addresses can be obtained from the
l ) o t n ~ i ~Ni, m e S)atetii.i,i
Suptwrt ofnlultiple protocols crcarcd anotlicr naming issue. Many customers already have ;I 1)omain
Name System in place in their networks. O h n 1)EC:net
systems ;1rc i11so running TCP/IP protocols and arc registered in tlic l>omain Name System, yet thcse systems
arc 11ot running 1)ECnet sohvare o\.cr 'I'CI'/IP. For
csamplc, a syste111 registered as l1cl.syz.com may be
dircctly I-eachable\\.it11 TCP/IP but not \\.it11 l>ECnct
over 'I'Cl'/IP. In this case, it is possible that <:Dl may
rctric\sc a valid IP address for a remote system thnt is
unrcnch,~bleby tlic DECnet protocol.
For tliesc reasons, \vlien CDI dctcrmines tllar botli
tlic 1)omain Name System and the l>E<:Jns naming
service ( o r a local file) are specified in the scnrch path,
it docs not stop processitlg the search path until both
t l ~ c 11' addl-css and the OSI address liavc been
obtained, or until the e ~ l dof the list has bccn rcnched.
I n this \\lay, ifthe desired remote systcm is not running
l)E(:nct o\*crTCI'/IP, an attempt to connect \\,ill be
made through the DECnet protocol, ~tsinga conncctionlcss ~ien\.orkser\,ice (CLNS) OSI nddrcss.
Local File Support
Early \,ersions of the DECner nenvorking product
offcrcd only .I local file for node-to-address information. The first release of DECnet/OSI rcplaced tlic
I
DECNETIOSI
APPLICATION
local file \\,it11 the 1)EC:dns naming service. Unfortunately, administrators of small- and medium-sizcci
networks found that the bcncfi ts of 13ECdns (scaling
and centralized managet~lcnt)\\.ere ounveighed by its
additional complcsity.
A subsecluent version of l>ECnet/OSI introduced
tlie Local Naming Option. This allowed approsimately 150 nodes to bc stored in a local file, but many
customers found this number to be too small.
CDI supports a very large local file: the supported
limit is 100,000 nodes, but tl~cl-cis 110 tixed internal
limit. In addition, tl~rougli the use of the search
patli, customers c'in c o n f i g ~ ~ tlie
r c local file either as
a backup t o a distributed scr\jice, or as a \\fayto prc)\,idc
greater performance. Note that both of these q~~alities
are also pro\,idcd in ;I more nutomated \\lay by tlie CL>I
cache (see the CDI Library Cache section for more
infor~nation).
Security Considerations
CDI relies upon the security provided by the underlying
directory services (or in the case of the local file, the f le
spstcm). Security of its remote management featurcs
depends on the ncn\,orlc tnanngement securinl system.
CDI Libraries: Basic Design
CDI is implemented as sIi3red libraries on both the
Digital UNIS and the OpcnVlMS operating systems.
At tlie highest le\,cl, tlie design is identical o n both s!.stems, as sllo\\/n in Figure 1. Nanic-to-address translation requests from the scssio~lcontrol layer are passed
through a single entry point in each CDI library.
I1
DECNETIOSI
APPLICATION
1 ...
-7
I
I
I
I
CDI SHARED LIBRARY
COMMON DIRECTORY
INTERFACE LIBRARY
INTERFACE
INTERFACE ...
mr&l
SERVER
Figure 1
Block l>iagram of the CDI Library
Diairnl 'Tc.cl,~lical Jout.n.~l
Vo1. 8 N o . 1
6
6i
Depending upon the search path (described belo\\.),
the Cl3I libraries transl'ltc and forward the request to
one o r liiore directory services (or they look LIPthe
inforn~ationin a local file).
The CDI implcmcntation was considerably more
cornplex on the OpenVMS operating system than on
the Digital UNIX operating systern due t o the differing design of l>E<:nct/OSI 011 each system. On
the Digital UNIX operating system, the DECnet/OSI
session control layer consists of a shared library that
is linked with each ncnvork application. Namc
resolution requests arc processed synchronously. 0 1 7
tlie OpenVMS operating system, session control is a
component of the NET$ACP process. Since all name
resolution requests arc channeled through this single
process, o p e r a t i o ~ ~I IsI L I S ~be asynchrono~~s
(req~~csts
must block concurrcnt operations). I n addition, sincc
multiple requests may be sin~ultaneouslyoutstancling,
tlie library is ~ n ~ ~ l t i t h r c a d Asynchronous,
cd.
~nultithreuded operations on thc OpenVlMS operuting
system are implemented ~ ~ s i the
r ~ gasynchrono~~s
system trap (AST) mechanism. For these reasons, the
CDI implementatio~lon OpenVMS was much largcr
and more complex.
CDI Search Path
h i o t h e r goal was t o permit flexibility in detern~ining
a configuration of directory services. The C1)I design
achieves this goal in nvo ways. First, it allo~\tsiidministrators t o select their service(s) of choice and t o use
them in any order. T h e search path is normally cre;ired
during nen\~orl<config~~rntionand can be subsccluently managed either locally or remotely. Second, it
gives network users thc ability to use short, abbreviated names instead of potentially cunibe~.somcfill1
names. For example, they can use "system 1" instcad of
"systernl .sales.hq.xpz.com."
A single mechanism in tlie CDI library-the C1)I
search patli-provides
these nvo capabilities. The
search path consists of a series of directory
service/name template pairs, as sho\vn in F i g ~ ~ 22.
re
When tlie CDI library is given a name to process, it
scans the search path, replacing the "*" in the name
template with rhc supplied name. For example, if the
library was searcliing for the name,fi.oc/u, it \vould use
tlie directory services identified from the names generated shown in Figure 2b.
During network configuration, a defiiult search
patli is autoniatically configured based upon the local
node name and the administrator-specified director!l
services. This search patli behavior is similar to a
number of existing T(:P/IP host name/address
lookup imnplementations.
CDI Library Cache
Occasionally, name service lookups can take a long
time t o complete (fbr example, if requests .ire travers-
DECdns
DECdns
DECdns
Domain
Domain
'
XYZ:.hq.sales.'
XYZ:.DNA-Node-synonym.'
'.sales hq.xyz.com
(a) Oircctory Service/Namc Tcrnplntc I'airs
frodo
XYZ:.hq.sales.frodo
XYZ:.DNA-Node-synonym.frodo
lrodo
frodo.sales.hq.xyz.com
(DECdns)
(DECdns)
(DECdns)
(Domain)
(Domain)
( b) Address I nokup for Name ,/i-o~lo
Figure 2
Using the ClDI Search Path
ing a slow net\\~orklink, a lookup could take several
seconds). To improve performance, the CDT library
incorporates a single cache that accum~~lates
node
information from all the directory services. Usually,
the cachc is consulted before sending 3 request to
a remote service. However, if session control determines that cached information is stale-for example,
if connection t o a node at a cached address reaches
a node with a different name-it will reissue the call,
requesting that the cache be bypassed.
Each entry in the cache has a creation tiriie stored
with it. The cache itselfhas a "time-to-live" value that
can be modified by the administrator. If a cache
lookup fi~tdsan entry whose lifetime (rime since it was
created) is greater than the time-to-live value, the
cache entry is p ~ ~ r g e d .
T o prevent a period of low performance imlnediately atier system start-up, the cachc is preserved
across system reboots by periodically checkpointing
it to a disk file. The checkpoint interval is adjustable
by tlie administrator.
CDI Registration Tool: Basic Design
The CDI registration tool provides fi~nctionsto create,
modifi: rename, display, and delete node name and
address information in any of the supported directory
services. It runs on the major DECnet/OSI platforms,
the OpenVlMS and the Digital UNIX operating systems.
The bdsic requirements for tlie ClDI registration
tool were the same as those for the C1)I library. These
three requirements were the need to:
Support different director!r services for storing
node information
Access each directory service using tlie appropriate
application programming interfaces ( APIs)
Store data in each directory service using the
appropriate data types
In addition, the following requirements wcrc spccifc to the CDI registration tool:
Both a forms and console user interface had to
be provided. These had to \vork identically on all
l>ECnct/OSI operating system platfor~ns.
Functions to transfer node information benveen
the various directory services had to be provided.
Other applications such as the DECnet/OSI
nenvork control languagc (NCL) utility and othcr
namespace nianagernent tools had to bc able to
access node name man'igelnent functions.
The directory services supported by tlie CDI registration tool are slightly different from those supported
by the CDI library. The CDI registration tool supports
the l>ECdns, tlie local f le, and the DECnet Phase IV
database services.
Tlie DECnet Phase IV database is supported by the
CI>I registration tool to allow administrators to use
old Phase IV node information when populating
the node names and addresses for DECnet/OSI. The
l'hase IV database is not supported for node name-toaddress lookup by the CDI library.
Due t o its lack of a remote update capability, the
llomain Name System is not supported by the CDI
registration tool. Node name-to-address information
in the Domain Name System is managed i~singits
native tools. Dynamic updating of the Domain Name
System servers is currently under study by the Internet
Engineering Task Force (IETF) Domain Name
Svstem Working Group.
directory-service-independent data representation of
all information needed by the CDR API to manage
node names and addresses.
To provide an extensible mechanism for adding new
directory services, the CDR API is layered on top of
a set of directory service wrapper routines, one per
supported directory service. Access to these \\(rapper
routines is proviclcd by a set of cntry point t ~ b l c sthat
can be extended to support neiv directory services.
The CDR API is responsible for accepting application
requests and dispatching them to the correct directory
service by means of the appropriate wrapper routine.
The CDR APT wrapper routines are described later in
this section.
Figure 3 shows the design of the Cl>I registration
tool and tlie C1)RAPI.
CDI Registration Tool User Interface
The forms and the console user interhccs had to
present exactly the same characteristics o n both the
OpenVMS and the Digital UNIX operating systems.
Becaiise no high-level software pacltages at the time
could provide this level of user intertice portability, we
developed the111for this application.
T h e console user interface parses commands and
dispatches them to the appropriate user request processing routine, using a portable command parser.
The for~iisuser interface obtains input from taskspecific forms and dispatches the function o r f~nctions
associated with the form to the appropriate user
request processing routine. Tlie forms processor
was written specifically for this application because
n o existing libraries could provide tlie required level
of portability.
Application Design
CDI Registration Tool User Request Processing
The design of the CDI registration tool uses a clientbased, multilayer approach. It is layered 011 top of a
specialized API, called the Common Directory
Ixegistration (CDR) API. T h e CDR API differs from
the API provided by the CDI library in that it presents
a fill1 set of management operations, rather than just
the lookup operations required by DECnet/OSI.
I n this design, the CDI registration tool provides
fornis and console user i~iterfacesfor node information management. I t also provides functions beyond
the basic ones provided by tlie CDR API, such as
exporting from and importing t o a directory service.
The fi~nctionof the CDR API is to perforrn all underlying node name management operations in a standardized manner. This layered approach was adopted
to make node name management functions available
to applications other than the CDI registration tool.
The CDRAPI defi nes a node definition object. This
contains all the information that is exchanged between
the CDR API and the application and is a canonical,
Each user recluest lndps into a specific request processing function as follows:
Register. Create a new node name cntry in tlie
directory service.
Add address. Add address information to a node
name en try.
Remove address. Remove address information
kern a node name entry.
M o d i 6 address. Replace the addrcss information in
a node name entry.
Update address. Replace the address information in
one o r more node name entries, using illformation
obtained from the nodes themselves (if possible).
Modifjl synonym. Replace the node synonym in a
llode name entry.
Rename. Change the name of a node name entry.
Show. Display the information containcd in one or
more node nanie entries.
Digital Technical Journ.11
Vol. 8 No. 1
1996
63
;---------------------,
cDI REGlsTRATloNTOOL
I
I
I
I
FORMS USER INTERFACE
FORMS DEFINITIONS
CONSOLE USER INTERFACE
COMMAND TABLES
REQUEST DISPATCHER
SHOW
PROCESSING
REGISTER
PROCESSING
DELETE
PROCESSING
CDR APl CALLS
CDR API CALLS
CDR API CALLS
I
I
I
I
I
CDRApl
;
I
I
I
I
I
I
I
I
I
I
r--------I
I
-------------I
I
I
I
l
l
I
l
l
I
I
I
I
t
----------
I
J
...
CDR API CALLS
I
I
I
I
I
I
I
I
I
ENTRY POINT
ENTRY POINT
ENTRY POINT
ENTRY POINT
SHOW
PROCESSING
REGISTER
PROCESSING
DELETE
PROCESSING
...
WRAPPER CALLS
WRAPPER CALLS
WRAPPER CALLS
WRAPPER CALLS
DIRECTORY SERVICE WRAPPER ROUTINE DISPATCHER
DECDNS
WRAPPER ROUTINES
LOCAL FILE
WRAPPER ROUTINES
PHASE IV DATABASE
WRAPPER ROUTINES
I
I
I
I
I
I
I
I
,1
I
I
Figure 3
I3loc.k l>iagr.i~iiof thc CDI licgistmtioi~Tool ;uid the CDR M'I
1)crcgister. Dcletc one or niorc nodc nnmc cntries
by name, synonym, or address.
1Xcp"ir. Fix any detected problcn~sor inconsistcncics in tlic directory ser\lice for one or more nodc
nnmc entries.
Export. <:opy the information h r one or more node
nalnc entries from die dircctory scl-\,iceinto a text fjle
that can be copied benveen systems, cditccl if 11eccssnr!; and imported into an!, otlicr dircctor!~scl-\,ice.
Import. Use an esport test file to rcgistcr, moditi; or
dcrcgistcr node name entries in a directory scr\.icc.
Tlir requcst processing ro~rtiliesperform a n y rccl~~ired
\~nlicintion of the user request and tl.nr~sl,ltc those
recILlests to calls into the CDR API. Each request may
map into one or Inore CDK MI calls, dependins on tlie
complcsity of the rcquest. For csa~nplc,register and
dcregistcr reqilcsts both map illto single (:l')R APT calls,
and export and import requests map into sc\~eralcalls.
Most rcqucsts arc straiglitfor\\.nrd in their processing rccluiremcnts. For example, a rcgistcr rccl~~est
simpl!, c3lls the C D R A P I register entry point. ?'he
(:Dl< API tnkcs care of any complications in pl-occssing
the recluest.
64
1)igiral '1i.chnical Journal
\Jf.)l. 8 No. 1
1996
Some reclucsts can opel-ate o \ t r ~iiultiplenode n;inic
entries. For csamplc, the sIio\\l request enumcratcs
the nodc name entries, retrieves the information contained in each nodc name cntr!r, ancl displays tlie information to the user.
Ai esport request is similar to a show request,
except that tlic resulting information is ~ r i t t e nt o a
test file in a standard format jnste3d of being displ'~\~cd
to tlie user. The i~iiportrequest, ho\ve\.er, is more
coniplicated. This reclilcst Ini1st enumerate and she\\,
the contents of the directory scr\,ice, and the11 compare the results \\~itlitlic contents of the test file. Rased
o n the specific form of the import request, it may then
register new nodc namc cntries, update the information in existing nocie Ilnnle entl-ics, o r deregister listed
node name cntries.
The export and import rcqucsts make use of a text
file to pro\,ide masim~rmflexibility. The use of a test
file allo\\.s the information t o be copied between dissimilar platforms such as the OpenVlMS and the
Digital U N I S operating systems, and allo\\rs the inforination to be lna~iip~~lutcd
sing standard tools such as
batch filcs, grep, awk, ;lncl test editors. This is particularly usetill \\.hen applying a cIi;lnge to all node entries.
For example, the contents of a directory service could
be csportcd to a text file, the addresses in the test file
changed to reflect a new routing area, ,lnd the results
in2lx)rted back into tlie directory to update the esisting information.
Tlie repair fi~nctionperforms a s l i o ~ operation
\~
on
all specified node names to determine if any consistency crrors arc found. This type ofcrror can occur in
directory services that Iwep multiple physical records
for cach logical node name entry. 13ECdns is one
csaniplc of this Idnd of directory service, because it
mes soft linlts to map node s!lnonyrns and addresses
back to their respective node name entries. If this type
of error is found, thc repair fi~nctionrc-registers the
~ ~ o ds!,nonym
c
and address informatio~ito correct
these inconsistencies.
The most complicated request is the update request.
This performs a sho\v request for the specified node
n;lmes and attempts to use the current addressing
informntion contained in the node name entry to make
3 network management connection to tlie node itself.
For each node name entry, it steps through the complctc set of registered addresses and tries each address
i l l turn, ~1si11g
both a DECnet Phase IV connect and a
l)ECnet/OSI connect. If a connect attempt is successfill, it i~scs the appropriate network management
requests t o read the true addressing data. It then conipJres this ,~ddressingdata to what it found in the director!, scr\~icc, ~ n dmakes any necessary corrections to the
node name entry. Tlie update operation does not operate on I I' addresses due to the lack of dynamic update
cap3hilitics in tlie Domain Name Systcm servers.
Before making the C D R M I calls, all recluest processing routines convert the user request data into a node
clefinition objcct, which is discussed in thc ncst section.
The CDRAPI controls all access to elements within the
node definition object, which further isolates the calling application from the lower-level data structures.
CDR API Node Definition Object
CDR API Directory Service Wrapper Routines
The nodc definition object is the only input data provided to any of the CDRAPI entry points. It stores the
neccss'lry data for any directory service operation,
using a canonical representatioll. Thc nodc definition
object contains tlie following:
Each directory service supported by the CDRAPI has
an associated set of directory service management
wrapper routines. These routines provide entry points
that are functionally identical to those provided by
the CDR API. The CDR API does the initial input
argument validation, and the directory service wrapper routines perform the data manipulation in the
underlying directory service.
The CDR API dispatches the appropriate directory
service wrapper routine using a set of entry point tables.
This provides a means to easily extend the CDR API to
include additional directory services in h t u r e versions.
1. Type oftiircctory service to access
2 Name of the node entry to access (depending on the
opcratlon b e ~ n gperformed, ~tmay allow a full\; qud16ed namc, a synonym, an address, or wldcards)
3. Syno~iynlname (for DECnet Phase IV access)
4. l)E(:nct l'hase IV network servicc acccss point
(NSAP) prefix (for use when a Phase IV address is
specified)
5. Aclclrcss information
6. L)irectory narnes used for reverse mapping of
synonym Iianles and addresses back to the fi~lly
qildlified node name
CDR API Entry Points
Each CDR API entry point provides one logical fiinction to the calling application. Each user request can
translate into one o r more CDR API functions. The
functions are
Register. Create a new node name entry in the
directory service.
Add address. Add address information to a node
name entry.
Remove address. Remove address information
from a node name entry.
Modify address. Replace tlie address information in
a node name entry.
Modify synonym. Replace the node synonym in a
node name entry.
Rename. Change the name of a node name entry.
Show. Return the information contained in one or
more node name entries.
Deregister. Delete one o r more node name entries
by name, synonym, or address.
Enumerate. Return a series of node name entries,
one at a time, based 011a wildcard specification.
All node information passed to and from the CDR
API is in the form of a node definition object, as
described previously. The CDR API functions validate
the ca~~onical
information contained in the node definition object and dispatch a directory-service-specific
function to handle the request.
CDR API Wrapper Routines for DECdns
In the DECdns name service, each node name entry
contains all the information required to translate a
node name to a synon!lni or a set of node addresses.
However, no search mechanism exists to allow a
L>iginl Technic11 Journal
I
Vol. 8 No. 1
1996
65
lookup of tlie node name entry based on the synonym
or on an address. For this reason, all filnctions that create, m o d i ~ and
,
delete node name entries (register,
modifi, addresses, rnodifii synonyni, rename, and
deregister) must also create, modifj: and delete reverse
mapping entries.
lleverse ~iiappingentries are based on a node's synonym and addresses; they contain pointers to the true
node name entry. These entries are ilsed by the CDI
library lookup functions and by the C D K API display
functions (sho\v and enumerate) t o access thc node
name entry when given a synonym or address.
The use of reverse mapping entries recliiires that
multiple clirectory service entries bc created for each
registered node. These must be s~~~ichronized
by properly orclering the creation and deletion ot'the \rarious
entries \\then registering, moditjliug, or dcregistering
a node name. For esample, \\then registering, the node
name entry is created and its synol1yrn and address values arc set before tlie reverse mapping entries are created and set. Similarly, when dcregistering, the reverse
mapping entries are deleted before the node name
entry is deleted. This prevents orphaned rcvcrse mapping entries from being created, because they call
always be found by starting from the information contained in the node name entry.
The repair function is provided in case a register or
deregister operation fails before completion. The
rcpair fi~nctioncorrects the reverse mapping cntries by
re-registering all node name entries that show errors.
The CDI registration tool (not the C1)RAPI) pro\lidcs
this higher-level function.
inconsistencies in the local node filc. Fortunately,
diese inconsistencies can be resolved using the same
synchronization algorithms as used for DECdns.
CDR API Wrapper Routines for the DECnet Phase IV
Node Database
Access to the 1)ECnet Phase IV node database is pro-
vided primarily to help users migrate their Phase IV
node name data to DECnet/OSI. N o access is provided to this database by the CDI library for
DECnet/OSI applications. Because this database consists of a simple file, with one record per node name
entry, none of the ~nultiplerecord synchronization
problems exist.
Conclusion
T h e Common Directory Interface, consisting o f
the CDI registi-ation tool set and the CDI library, provides flexible and extensible directory service access
for DECnet/OSI. Initial customer acceptance of these
new capabilities Iias been high and future enhancements are being studied.
Acknowledgments
T h e design and development of the Common
Directory Interface involved the contributions of the
entire directory services and DECnet engineering
teams. We extend our thanks to all the team members,
as \\/ell as to product and engineering management for
supporting this project.
References
Under the OpenVMS operating system, the local node
1. S. Martin, J. klcCann, and D. Oran, "Development of
name file is implemented using a record management
thc VAX Distributed N ~ m eService," Digilal Tech~iical
system (RMS)-indexed file. Under the Digital UNIS
.Jo~lrwnl.vol. 1, no. 9 (June 1989): 9-15.
operating system, a DBM-indesed file is used. On
both systems, the file content is esscntiall!l the same.
2. 1.' Mockapetris, "1)ornain Names-lrnplementation anu
The local node name file contains a series of logical
Specification," 1W(: 1035, Intcrnec Document (No\rcniber 1987).
records, one for each node nanic entry in the directory
service. Together, these records define each node's
3. CCITT Sixth Plenary Asscnibly, "The Director!-fillly qualified name, its synonym, and its addrcsscs.
O\rer\~ie\vof Conccpts, Models and Services," Reco1?7This logical record may be loolted up using the f ~ ~ l l
r n e n ~ l ~ ~ t i o iXz . 5 0 0 and I S 0 9594- 1. Dutu
name, the synonym, or any of the node's addl-esses.
Cornnzr./nicatio~zsNetujorks Directory: Recomtnc~r?Each logical record consists of (1)a node definition
~l~rtio17.s
S.500 to X. 52 7 , CC1'T.f Blue Book>vol. xiii.8
(Genev;l: International "~eleconi~nunicationsUnion,
physical record, which contains all infor~nationrelated
1989).
to the node, and (2) zero o r more reverse mapping
ph!rsical records, which contain alternate keys for look"Using the Domain Nalnc Systcm to
4. R. Rosenba~~m,
ing up the node definition. Each rcvcrsc mapping
Store Arbitrary String Attributes," RFC 1464, Intcrnec
record contains only the node name key in its record
Docu~nent(Mav 1993).
data. All the data used to describe the node is con5. B. manning and R. Colclla, "Thc Dolllain Name Systcm
tained in the node definition record.
NSAP Resource Records," RFC 1706, Internet DocuBecause multiple records compose a node name
ment (Octobcr 1994).
entry, operations that fail t o cornplete can r e s ~ ~in
lt
CDR API Wrapper Routines for the Local Node File
Biographies
Richard L. R o s e n b a u n ~
Rich l<c)scnbaum is a sofi\\.are engineering consultant in
the Inrcrnct Sofr\\are Business Unit, \vlierc Iic is focusing
o n the application of indexing and collaboration rechnologies t o rhc World Wide Web. I n his 1 7 years with Digital,
hc Iias worked o n networking products operating o n
1)igital'b 16-, 32-, 36-, and 64-bit platforms. H c is tlic
co-nuthor of several patents o n network s o f ~ \ f a r eIGch
.
obtained a B.S.from the State University o f New York
at Stony Brook.
Stanley I. Goldfarb
Stan Goldhrb is n principal sofn\tare cnginccr with tlic
Intcr~lctSohvarc Business Unit. Since joining Digital
in 1976, he has contributed t o several ~ietworkand net\vorlc man.lgcnicnt projects, including DEC:net/RSX,
1)ECnct-PKO, DECnet-DOS, DECmcc, l)F,Cnct/OSl,
and PATHWORKS, and Ihe has co-authored several parents
on net\vork Iiianagement sofnvare. H e is currently working o n a Workgroup Web Forum application t o providc
clccrronic mail subscription and distribution services.
Stan holds R.S. and 1h4.S.degrces in computer scicncc
k o m Worccstcr Polytechnic Illstirutc arid an b1.S.in
Inanagcnlenr from Lesley College.
Recent Digital
U.S. Patents
?'he following patents \\!ere recently issucd t o Digital
E q ~ ~ i p m eC
n ot r p o r a t i o n . Titles a n d n a m e s s ~ ~ p p l i e d
t o us by t h e U.S. Patent and Trademark Office are
reproduced exactly as the! appear o n t h c original
published patent.
5,223,996
5,224,235
Electronic 13evicc Module
S. K. Morgan alld M. 1,. HctficIJ
Electronic Device Module
S. K. Morgan and M. L. Hcrticld
Enclosure h r F,lccrronic M o c i ~ ~ l c
S. I<. Morgan and iM. L. tlcrfclti
R. Veno, I<. Palumbo, 1'. Roncli, P. R.irron,
Power Supply Door
and IM. Freeman
Electronic 1)cvic.c Module
S. K. Morgan and M . 1 .. Hctfcld
IM. Falkner, LM.Good, rind M . Wicsc~lhahn
Positioning Dcvicc
S. K . Allorgan and M. L. Hcrficld
Pouser Supply iMociulc
L. Spcnccr and (:. llctsilc.15
Face Platc
R . Faranda
Central Processing Unit Enclosure
Cover for \Ydll-mounted Electronic Ecluipmcnt
S. I(, klorgan and JM. 1,. Hctficld
S. I<. &lorgall and iL1. L. Hctfcld
Electronic L)c\,icc M o d i ~ l c
M. I.. Herticld and S. K. Morgan
Network ~Multiplesorfor an Office En\.ironment
W. ~McCarth!; R . Hcll\\,cg, K.iV13srcrs,
C ~ r dCage Enclosure
iLI. Freeman, C. L\lilliams, <:. I3rcncli,
I(. Palumbo, D. Snow, and 1'. Rarron
lGgid Disk l>ri\~c
C. E. V ~ i l l ~ ~J.nD
t , . licati, ,lnd C;. J . N o r q ~ ~ ~ Rcmovnble
y
M. J. Falkncr and 1M. W. Klce~nn~l
Desktop Audio Enclosure
Display Monitor
C . Landry
Soldcr Pump Kushi~lgSeal
I<. Sullivan and P. Cninc
Secure User Aurhcnricntio~ifrom I'ersonal Computer
S. Lipncr, iM. Gasser, and 1%. W. Lampson
M. Kindervater nnd F. Zandvcld
Cornpi~terSystem Comprising a Main Bus and an
Additio~ial< ; o m ~ n u ~ ~ i c aiblcans
t i o ~ ~Directly Co~~nccrcci
bet\\,een Processor and Main Memory
Merhoti and Appnratus for Testing Inner Lead Bonds
D. Widder
R . Stroud and K. Vo~ihrnncit
Composition and iblcthod for Bonding Elecrl.ical
Componcnrs
1.' Hearn, A. Prentakis, W. Lewis, 2nd
Method and Appar.ltus ti)r Surhce Reallocation for
Irnpso\~cd~Mn~lu'lcruringProcess Margin
F. Zayas
Parallel Diagnostic ~ModcFor Testing Computer ~Mcniory
1.' IM. Goodwin, 1). W. Smclscr, and
D. A. Tatosian
l'lcnum for Air-i~npingc~ncnt
Cooling of Electronic
11. Brown
Components
Cross I<cgulator for a Multiple O i ~ t p u Pou,e~t
Supply
J. Palczynski
Optical Angular Position Sensing Systcn~for Use \vith
K.Pa\rlak
Galvanomctcr
Method ; u ~ dApparatus k)r Reducing Electromagnetic
R . Curtis and 1). Skc~liizic
Intcrfcrcnce n ~ i dF'rnission Associated with (:ompurer
Ncr\vork lnterfnccs
Combincd Shock Mount Frame and Seal for a Rigid
(:. E. Vaillant and J. 17. R c ~ d
Disk Drive
Electronic Componcnr Cleaning Apparatus
P. Lison and W. Haincs
W. H;l~nburgc~l
11. Noguchi, 1. Rinaldis, and P. Esling
K. <:lien
13. Rrogdcn, I,, Bro\vn, and S. Husain
1). J. Vclasco, J. P. '.<:opeland, D. C. Robinson,
and l i . 1,. Fcrnnndcz
G . Ir'rccclma~l
Gentle Package Extraction Tool
Mcrhoc~
Tunablc Wideband Active Filrcr
Method and Apparatus for Lxarning in a Ncural Ncnvork
Dr!l Contact Electroplaring Appa~..lrus
Video Amplifier Assembly [Mount
Atmospheric Pressure Gaseous-flus-nssistcd Lnscr
Rctlo\v Soldering
System and Method for Measuring <:ompurer Sysrc~n
Time Intervals
M . 1Mallal-y
Side-by-Side Rcad/Writc Heads \virh Rot;~r! Positio~icr
Vortices
1). 13;iiIc\,
Cooling Device tlinr Crc;lres Longir~1di1l~11
Po\\~crSupply Inrerlock for Disrributcd I'o\vcr Systcnis
1). 1). I>o~laldsonand D.Wissell
J. Benson, I). Alessandrini, and W. l<ett
Appararus for Sec~rri~ig
Shielding o r the Like
W. Haincs, R . K.lyniond, C. Ryun, E. Johns,
N~trogen-containingM~rerialsfor Wear Protection and
1). Kavipari, Q.Ng, and G. Ri~uch
Friction Reduction
W. Mallard
TI-ansmission Media Driving System
E. PVobhcr, M. Abadi, A. Rirrcll, and
Access Control Subsystc~nand Mctllod for l>isrriburcd
Computer System Using Locally Cached Authe~iticarion
R. W. 1.anipso11
Credentials
Semiconductor Probe and Alignment Sysrcni
I). Widdcr and 1). Icngleb
K. Chi
Voltage-conrrolled Ring Oscillator Using <:oniplcn~cnrary
Differential Buffers for Generating M ~ ~ l t i p Phase
lc
Signals
W. Hanlburgcn and 1. Fitch
Fixture and Method for Atr~cl?ing(;oniponents
T. (;reedon, I). Smith, and A. O'Connell
Progranimable Priority Arbiter
F. Fcl(lbruggc
~Merhodfor Updating Modified I>an fi.o~n1' <;ache
Address Location t o Mnin Memory and Maintaining
the Cache Address in Registration Memory
Combined Differe~~tinl-mode
2nd <:o~llrnon-motic
Noise Filter
<:. F.. V~illant,J. I). licad, and .;( Norquay
Re~no\fablcEnclosure Housing a Rigid Disk 1)rivc
\.\I. H ~ ~ i i h u r g
nnd
e ~J.~ Fitch
Integrated Circuit r r o t c c r i o ~b~y Liquici Enc.lpsulnrio~~
W. H a ~ n b u r g c and
~ i 1. Firch
Semiconductor Heat Rcrnoval Appararus \\,irh
Non-uniform Conductance
Thermal Processing F ~ ~ r ~ l \\,it11
a c e lrnprovcd Plug Flo\v
A.I'liilipossian and E. (:ullcy
P. Anick .ind R. FIynn
Method and Apparatus for Integrating a I>ynaniic Lexicon
into a Full-rest 1nform;ition Rctric\.;il Systcrn
Fault Detector for a P l u r ~ l i poERatrcrics in Bartcry
Backup Systems
I>. (:. D;l\'ics, I>. G.V O I I ' I ~ and
J , R . A. Curtis Transceiver Appararus nud Mctl~ods
N. Crook, I? Bruce, and 11. Gilluszka
High Performalice Interface bcnvecn an Asynchronous
Bus and O n e o r Morc Processors o r the Like
r\. I'hilipossinn n~ldE. Culle!)
Reducing Gas Rccirci~l;lrionin "l'licrnlal I'roccssi~lg F ~ ~ r n n c c
R . Mcllitz and E. Srcarns
Manually Operared <:ontinuin/Shorrs Test Probe For Rarc
lnterconnccrion Packages
J. Simo~~clli
and %. Al-ban.1~
Confgurablc In\lertcr for 120 VAC or 240 VAC Ourput
A. Jain, N.Lcc, anti E. Kcppclcr
Apparatus and Method for Fabricating a Lcns/Mirror TOMW
T. I .. Rodchcffcr
Intermittent <:o~nponentFailure Manngcr and Mctlioci For
Minimizing 13isruption of Disr~.iburcd<:o~iiputerSystcm
Filters in License Management Systeni
~Mctliodof Issuance and Revocation of <:errif cntcs
ofAuthenticit!~ Used in Public I<cy Nctulorks and
Other Systems
1'. S. Ihrkcr, a11d E. W. Ertcl
Method and Appararus for Encoding I h r a for Storage
o n [Magnetic Tape
R. I'ortcr, (:. A. Mcga, and R. L. Myers
Computcr System Operation \virIi <:orrccrcd Kcad
Data Funcrion
D. E. William
Sharing o f Bus Access among Mulriplc State Maclii~lcs\\lit11
Minimal Wait Tinic and Prioritization of Like Cycle Types
(:. !I Murphy, .I'. (:reedon, ~ n c (:.
i D. Crc~nin High-performance Asynchronous 13us 111rcrfncc
A. Nasr
methods of Forming a Local Interconnect and a High
Resisror Pol\lsilicon Lnad by Rcacting <:obalt \vith Pol!lsilicon
Vol. 8 No. 1
1996
6~
S. Batra, S. Ramaswamy, and M . Mallarp
R . J . Galuszka, A . J. Walton, and C . Clioi
C. P. Thacker
A. T. Townley
F. Aghadel and C . W. H o
M . Saylor
M. Abadi, M. Burru\vs, and 13. W. Lampson
K. D. Abramson, H. B. Butts, and
D. A. Orbits
H. A. Collins, R . B. Watson, and R. Iknaian
S. G. Lloyd and H. Partovi
L. MacLellan
G. Freedman, P. Elmgren, and M. Brodcur
B. D. Buch
N. D. Godiwala and K. M . Thallcr
W. F. Even
D. B. Lornet and B. J. Salzbcrg
5,278,783
B. Rub, J. E. Deroo, S. R. Skraly, A. Solli,
and R. Frame
J. Edrnondson
5,279,865
5,280,437
R . P. Chebi and S. Mitral
D. A. Corliss
5,278,703
A. J. Bcverson, T. E,. Hunt, and
G. P. Lidington
J. R . Lundbcrg
J. F. Bartlett
A. E. Deluca, J. M. Lewis, C. L. Leo,
T. ,I.Orr, D. T. Symrncs, and R. A. Barker
J. W. Benson and D. T. Statliere
M. Shilo
W. Engelse
P. Stoppani
D. B. Lomet
B. A. Maskas, J. A. Metzgcr, and G. J. Harris
J . A. Daly, J. M . Gregorich, and G. J. Rr'uld
G. Saliba
W. F. McCarthy, D. M. Snow, and
C. E. Brench
N.A. Crook, P. L. Brucc, and R . J . Gnluszlta
M. A. Jordan and D. J. Donnelly
A. P. Russo, S. L. Regc, M. F. Kcmpf, and
E. T. Sulli\ian
R . Iknaian and R. B. Watson
70
Digital Technical Journal
Vol. 8 No. 1
1996
Thin Film Kcad/\Yritc Head for Minimizing Erase
Fringing and Method o f Making the Sarnc
Apparatus for S i ~ n ~ ~ l r a ~Writc
~ e o uAccess
s
to a Single
Bit Memory
Mcthod and Apparatus for Resource Arbitration
Collision lktection and Signaling Circuit
Packagc for Multiplc Rcmo\~ablcIntcgratcd Circuits
Robotics Workstation
<:o~nputcrNct\vork with Motfificd I-Iost-to-Host
Encryption Keys
Adaptive Memory Maniigcmenr Method for Coupled
Mc~nor)!Multiprocessor Systems
Mcthod and Apparatus for Clock Skew Reduction t h r o ~ ~ g h
Absolute Delay Regulation
Resist~~icc
Tester Urilizi~igRegulator Circuits
Torsion Bar Connector
laser Bonding Highly Rctlecti\.e Surhces
On-the-fly Splitting of Disk Lhta Blocks Using Tirncd
Sampling of a Data I'osition Indicator
~\/lultisignalSynchronizer with Shared L ~ sStage
t
Spindlc (:ontroller \vith Startup Correction of Disk Position
Co~lcurrcncyand Kccovcry fbr Index Trees \\.it11 Nodal
Updates Using multiple Atomic Actions by Which
the Trccs Integrity is I'rcscrved d~rringUndesired
S!~srcni Intcrsuptions
Embedded Servo Bandcd F o r ~ n a for
t magnetic Disks for
Use uith a Data Processing System
Fast Arc.1-'fficicnt h4ulti-bit Rinar): Acides \\fith Lo\\!
Fan-out Signals
High Throughput Intcrle\,cl Dielectric Gap Filling I'rocess
S t r u c r ~ ~<ind
r e Method for 1)ircct Calibration o f
Registration Mcasurcmcnt Systems t o Actual
Semiconductor W ~ f e Process
r
Topogmpliy
Programmable Stall (:yclcs
Rcduccd Voltage NIMOS Output Driver
<:oniputer Systcni and Method for 13isplaying I m ~ g c \\.ith
s
Supcrimposcd Partially Transparent Menus
Systcm for Ileducing the Emission of High Frcquency
Elcctromagnctic bV:~csfrom Computer Systems
C o m p ~ ~ tCable
c r Management System
Inrush <:ur~-c~lt
(:ontrol Circuit
Synchronous I>ccodcr for Self-clocking Signals
Systcni For Allocating Storage Spaces Rascd upon Required
and Optional Service Attributes Having Assigned Priorities
Multilevel l'r.insactio11 Rccovcr)r in a Databnsc Systcm
s
Transaction Undo Operation upon
Which 1 ~ 5 I'arcnt
Commit of Child Transaction
Self-compensating Voltage I,cvcl Shifting <:irc~rit
I'owes <:on\icrtcr \\lit11 C o ~ ~ t r o l lfor
c s Switching bctwecn
Primary and Battery I'owcr Sourccs
Mcthod and Apparatus I-i)rVariable I)c~)sityKcad-afterwriting on Magnetic Tape
F.nclosurc h r Electronic Modules
Sy~ichronizatio~~
Scllc~nc
1)ctcrnlinistic h/Icrliod for Allocation o f a Shared llesource
N c t ~ t o r kAdaptcr with High Thro~lghl>urData Transfcr
<:ircuir t o Optimize Nct\vork Data Tra~~sfcrs,
\vith Host
Rcccivc King Resource Monitoring and Reporting
PVI' Updatc Synchronizer
1). C. Robinson, 1. 1.' Copeland, D. J. Velasco,
R. L. Fcrnandcz, and S. D. Vcnditti
J. A. Mctzger and P. J. Graffam
1). L. Murphy
D. Railcy, P. Martino, and R. Arsenault
A. Bcavcrson and <:. 1. Dcvanc
I<. Galuszka, A. bvalton, and S. Bryant
(:. P. Thacker and D. Harn\;cll
P. J . Cerqua, S. M. Kenncdy, J. D. McAtee,
and P.J. Piccolomini
T. R . Renson
P. Rocrs
M.Burrows
S. L. Rcgc and D. A. Gagnc
D. <:. Davics
V. Saniarov, 1, l>cCarolis, R. Patcl, G. l'ichc,
G. Skutt, and S. Norris
N. 13. Godiwala and K . M . Thaller
M. I,. Palmcr
L. Supino
G. A. Salib;~
R . Silvcl-stein
N.K. Lcc, A. lain, E. Keppclcr, and
M. Rouchard
S.-T. Bcn-Michael and P. Lozowick
T. K.Bcnson
S. 1. Rradlc!,
H . Collins, R . Watson, and R . Iknaian
13. <:alialan
E. S. Noya, M . N . Rosich, and R . M . Arnott
N. A. Warchol
13. D.Donaldson, R . A. f>anlc, and
R. K. Nikcl
M . Lcwis, L. Trcsedcr, K. Martinez, ana
R . Tuslcr
C:. P. Thackcr
K. Mcincrth, C. Case, R . Gamache,
B. Fanning, and <:. Franklin
1\11. Lcwis rind R . Ravep
W. (:. Mallard and H . S. Yang
Integrated Computer Assembly
lnterconnect Arrangement for Electronic C o n ~ p o n e n t s
Disposed on a Circuit Board
System for Linking Program Units by Binding Symbol
Vector Index in the Symbol Table into Calling Imngc
t o Obtain Current Value of the Target Image
Method and Apparatus tbr Liquid Spill Cont?'
,~ n m e n t
A General Process for Finding Patterns in L.argc L.ogic
Traces ( o r Other Large Binary Arrays) Using Multiplc
Concurrent Finite Automata with <:ross-corii~n~~~iication
High Speed Transmission Line Interface
Dynamic Arbitration for System Bus Control in
Multiprocessor Data Processing System
Workflow Management and Contl-01 System
Use ofstack Depth to Identi+ Architecture and Calling
Standard Dependencies in Machinc Code
multi-element Susceptibility Room
NetIvork Packet Recei\sr with Buffcr Logic for
Reassembling Interleaved Data Packets
Attribute Based Multiple Data Structures in Host for
Net\vork Received Traffic
Tracking Peak Detector
Coplanar Heatsink and Electronics Asscmbly
Aborting Synchronizer
Predictive Cache for Improved Perforrn;~nccin Retrieving
Cached Data
Automatic Phase Margin C o ~ n p e n s ~ t i oControl
n
Circ~~it
and Method for Disk Drives
I\Iilgnetic Head for Very High Track Density Magnetic
Recording
Trickle Charge Circuit for an Off-linc S\\sitchi~lg
Power Supply
multi-disk Optical Storage Systcm
[Method and Apparatus for Cut-through Data Packet
Transfer in a Bridge Dcvice
Mapping Assembly Language Argument List Rcfc~.cnccs
in Translating Code for Different Machine Architectl~res
Systenl for Bonding a Heatsink t o a Semiconductor
Chip Package
method and Apparatus for Clock Skew Reduction through
Absolute Delay Regulation
Method and Circuitry to Provide Truc Voltage Rias to
a Magnetoresistive Head
Data and Parity Prefetching for Redr~nclantArrays of
Disk Drives
Self-conf guring Bus Termination Component
Data Bus Using Opcn Drain Drii~crsand I>iffcrcntinl
Receivers Together with Distributed Termination
Impedances
Reduced Tolerance lntcrconncct System
Method and Apparatus for Dcskcwing I>igitnl l3at.1
Translation o f Virtual Addressing in a Computcr
Graphics System
Automatic Signal Termination S)lstcm for a <:omputer Bus
Method and Means for Autoniarically 13ctccting and
Correcting a Polarity Error in Twisted-pair Media
D~gitall'echnicnl Journal
R. 1M.Arnott, E. S. Noya, and M. N. Rosich
K. Mcinerth, C. Case, B. Fanning,
and J. Irwin
K. Meinerth, C. Case, J. Irwin,
and B. Fanning
S. Bryant and M. Seaman
D. Young, S. Randall, S. Shaw, and A. Wylde
A. Philipossian, H. Soleimani, and B. Doyle
S. Britton, R. Allmon, and S. Samudrala
J.-C. E. Cuenod, and P. A. Sichel
F. Fernando
S. H o and N. Darcy
S. Dutm, A. Roy, and N. Rao
A. H. Mason, P. T. Robinson, R . Witek,
and J. S. Hall
B.A. Maskas, J. A. Metzger, N. D. Godiwala,
and I<. M. Thaller
K. M. Thaller
R . Curtis and B. Shusterman
R. Perlrnan
L.-J. Weng
K. Meinerth, C. Case, J. Irwin, A. Masucci,
S. l(rishnaswami, and A. Moezzi
K. Meinerth, C. Case, J. Irwin, and
B. Fa~lning
E. McLellan
J. IUein
L. Neville, A. Jain, and A. L. Gutierrrz
R. Perlman
N. Warchol
A. Villani
A. Philipossian, B. Doyle, and H. Soleimani
S. Batra and A. Wu
D. Young, S. Randall, S. Shaw, and A. \Vylde
G. Christensen and J. Marceca
A. E. Deluca, S. W. Stcfanick, C. L. Leo,
T. J. Orr, D . T. Symmes, and H. Wright
R. Ulichney
R . Ulichney
P. Stoppani and C. Saether
R.-A. Locicero, S. Morgan, M. Romrn,
E. Mangan, and M. Bantly
G. Dvorak and L. Wolfe
R . A. Willian~s
72
Digirnl Tcchnlcal Journal
Optimized Stripe Detection for Redundant Arrays of
Disk Drives
Graphics Command Processing Method in a Con~puter
Graphics System
Method and Apparatus for Varying Command Lcngth
in a Co~nputcrGraphics Syste~n
Multiprocessor Buffer System
Oscillation Device for Plating System
Method of Decreasing the Field Oxidc Etch Rate IJI
Isolation Technology
Leading One/Zero Bit Detector for Floating Point
Operation
Computer Peripheral Device Nctwork with Peripheral
Address Resetting Capabilities
Quadrant-based Binding of Pointer Device Buttons
Clocking System for Asynchronous Operations
Intelligent and Compact Bucketing Method for Region
Queries in Two-dimensional Space
Translation Buffer for Virtual Machines with Address
Space Match
Duplicate 'rag Store without Valid Indicator
Polling of 1 / 0 Device Status Compar~sonPerformed
in the Polled I/O Device
Cornbined Differential-rnode and Common- node
Noise Filter
Multicast Address in a Local Area Nenvork Where
thc Local Area Nenvork has Inadequate Multicast
Addressing Capabil~ty
Data Recovery aker Error Correction Failure
Residuc Buffer for Graph~csSystcm
Address ~Mcthodfor Computer Graphics Systenl
Reducing Branch Delay in Pipeli~~ed
C o r n p ~ ~ tS\~stem
er
Distributed Computation Recovery Managemcnc System
and Method
Surface Selection Mechanism for Optical Storagc System
Automatically Configuring Parallel Bridge N ~ ~ m b e r s
Method for Testing a Processor Module in a Computer
Systcm
Clip-on Heat Sink
Method of Controlli~lgGate Oxide Thich~essin the
Fabrication of Semiconductor Devices
Thin Film Magnetic Transduccr \\lit11a Multitude of
Mag~neticFlux Interactions
Method and Plating Apparatus
Disk Drive Holder and Interconnection System
Apparatus for Storing S t o r ~ g eDcviccs
Imaging System with Multile\~elDithering Using Bit Shiftcr
lmaging System with Multilevel Dithcring Using Two
Memories
Data Storage System and Method with Dcvicc
Independent File Directories
Modular Equipment Support Systcm
Test Fisturc for Electronic Components
Communications System with Reliable Collision Lletecrion
Method and Apparatus
FIFO: Based Parity Generator (FBPG)
Call for Authors
from Digital Press
Digital Press is an imprint of Butterworth-Hcinemann, a major intcmational publisher of professional boolcs and a member of thc Reed Elsevier group. Digital
Press is the a~~tliorized
publisher for Digital Equipment Corporation: The nvo
companies are working in partncrship to idcntify and publish new books under the
Digital Press imprint and create opportunities for authors to publish their work.
Digital Press is cornrnitted to publishing high-quality books on a wide variety
of subjects. We would like to hear from you if you are writing or thinking about
writing a book.
Contact:
Mike Cash, Digital Press Manager, or
Liz McCarthy, Assistant Editor
DIGITAL PRESS
313 Washington Street
Newton, MA 02158-1626
U.S.A.
Tel: (617) 928-2649, Fax: (617) 928-2640
E-mail: [email protected]
[email protected]
">