Digital Equipment Corporation Spike Optimizer, Memory access pattern analysis, OpenVMS Alpha VLM, PowerStorm 4DT PowerStorm 4DT Graphics Adapter, DART Network Adapter Technical Journal
Publicité
Publicité
Digital
Technical
Journal
I
SPIKE OPTIMIZE R FOR ALPHA EXECUTAB LES
ANALYSIS O F MEMORY ACCESS PATIERNS
OPENVMS ALPHA VLM
POWERSTORM 4 DT G RAPHICS ADAPTE R
FAST APPLICATION-LEVEL NETWORKING
Volume 9 Number 4
1997
Editorial
Jane
C.
Bbkc, Managing Editor
Helen L.
P a n er s o n
,
Editor
Kathleen M. Srer�on,
Editor
Circulation
Catherine M. Phillips , Manager
Kristine
JVI.
Lowe, Administrator
Production
Christa W. Jcssico, Production Editor
Elizabeth iv!cGrail, Typographer
Peter R. Woodburv, Illustrator
Advisory Board
Thomas
F.
Gannon, Cbair01<1l1 (Acting)
Scott E Cu tier
Donald
William z.
Harbert
A.
Laing
Richard
Alan G.
F.
La r y
Ncmah
Robert M. Supnik
Cover Design
The power of graphics workstations is mca smed by the speed at which the machine can cre;ue ;md manipulate
3-D objects .
The l'owerStorrn 4D60T graphics adapter design, a topic featured in this issue, com b.incs
Alpha
64-bit microprocessor technol ogy and modi tied rendering technology to attain high levels ofpcrtormance both on leading real-world CAD/CAM applicatjons such as Pro/ENGINEER, and on widely accepted industry benchmarks such as
OpenG L Viewperf. Om cover design is made up ofrepr·esentativc images fi·orn the
Viewperfbenchrnark program and standard vicwscts.
The cover was designed by Lucinda O'Neill of the
DIGITAL Industrial and Graphic
Design Group. The editors thank author
Benjamin Lipchak tor supplying rhc im used on the c o ve r
. a g es
The nigital Technicaljourrwlis a refereed journal published qu a rt e r l y by D ig i t 3 l
E qui p m e nt
C or p or at i o n , 50 Nagog l'uk,
AK02-3/B3, Ac t o n ,
MA
01720-9843. s
H<1rd-copv subscriptions can be ordered ending a to
D igita l E q u ip m en t Corporation) ro the p u bl is h c d check in bv a dd re
U.S. funds s s
.
G coc r a l
(m s u a b de payable s c r i pt i bv o n rates arc a nd
540.00 (non-U.S. S60) for four issues
$75.00 (nonU.S. $115)
University and c ol l ege professors
:1od Ph.D. students in the e lec t r i c al e ngi n e tor e r i ei n gh g t issues. a11d com puter science fi el d s rec e iv e co m pl i m e n t:�r y sub s c r i p ti o n s upon re qu e s t . DIGITAL customers rnav qualli)' tor gilt subs c ri p ti o n s and a r e encour
�lged ro con t ac t th e i r s a le s r e p r c s c nra ti n
.: s.
El e c tr o n i c s ub s c ri p ti o n s are a vai lab le at no charge by ac ces si ng URL http://www.d.igital.com/subscription.
This service will send an clecrronic mail notification when a n e w issue i s o n r h e
I nt e rne t. a v a il a ble
Single copies and back issues can be ordered by s en d in g rhe requested issue's volume a n d number and a check for $16.00 (non-U.S.
$18) each ro the p u bl i s h e d b y address. Recent issues are also available on rhe
J nr ernc r at http://www.digital.com/drj.
DIGITAL employees may order s u rjons r hr o u g h Readers C h o i b s c r i p c e at U
Rl. http://web•·c.das.dec.com.
Inq ui rie s
, ad d r e s s ch a n g es
, and compli mentary s u b scr i p t i o n o r de r s can be sent. to rhe Di.�ilal Tecbnica!Journa/ H rhe published- by a ddres s or rbc e le c tro n i c maiJ a dd r e s s
I n q u iri e s can also be made by c a l lin g rhcJournal o Fti ce at 978-264-7556.
Comments on rhe co n rc n r of an)' paprr and requests to conracr aurhors are welcomed and m a y be scm to rhc m a tbc p u b l i s he d
b y or el e ct r n a on i g c in g ma ectiwr at il a d dre ss.
C o p v ri g ht© 1998 D ig i t
C or p o r atio n
.
Cop mitted y in g a l
E q u ip m e nt wirhour kc is per p r ov i d e d that such copies arc made for use in educ.ational institutions bv laculrv members and a r c nor distributed f o
;· c o m
-. mcrci.li c1dvanrc1ge. Abs tracti n g with credit of D i g ita l E q u ipment Corpor<�tion's author s h i p is pe r mitt ed.
The information in rllcJournal is subjccr to c h a n g e without no ti c e and should not be ies c h o n stru e d as a comm i tm e nt by D igi t al
Equi pm e rei e n nt Corporation or by represented. Digital the com
E qui p me p a n n t
C orp or a t io n assumes no responsibility tor
any erro r s rhar m av appear in tbeJuurna/.
ISSN 0898-90
I X
Docurnemat.ion Number EC-P9257-18
Book production was done by Q u a n r ic
Communications, Inc.
The t o ll o wi n g are trademarks of Digital
F.
qu i p m e nr
Corp or at i on : AlphaServcr,
DEC, DECner, DEC,rarion, DIGITAL, rh c DIGITAL l o g o
,
DIGITAL UNIX,
Open VMS, PowcrStorm, and VA)(.
ARC/INFO is a r eg i st er e d t r a d er m r k of Environmental
S y s t em s Research lnstirurc, Jnc.
AuroCf\D and t\urodesk are r e gister e d trademarks of Autodcsk, Inc.
DirecL\, !v!inosoh, Visual C++, and
W i nd ows
NT arc re g i st ere d trademarks of M icro s o ft
Co r p o ra t io n
.
EDS cHJd U n ig r aph i c s a r c r e gi st e r e d trademarks of Electronic Dara Systems
Corporation.
Hewlcrr-Packard is a reg i st e re d trademark ofl-lewlctt-Packard Companv.
IBM is a r e g i s t e
Bu r e d trademark of· I nrernational s in e ss
N h ch i nes C o r po ra t ion .
Intel is a r e g i st e r e d t rad e m a
Co rp o rat ion. r k of i nr e l
,'v!icroSration is a registered rradernark of Bc nr lc v Svstc:ms, l n c o r p o r ,
1 te d.
MIPS, R4400, and RIOOOO arc t r a dem a r ks of
MIPS Technologies, Inc., a wh o ll y o w n e d su b s i d i a r y of Silicon Graphics, Inc.
O p e n
G
L and
Silicon Graph i cs Me re g i s tered trademarks and lndigo2 and IMPACT are trademarks of
S i l i co n
Graphics
Inc.
Oracle7 is a trademark of Oracle Corpor a r jo n
.
OrCAD is a r egis t e red rrc1demark ofOrCAD,
I n c o r po r a t ed.
Pro/ENGINEER is a r e gi st ere d trademark of Parametric Te c hn o l o gy C o rp or a tio n
.
SolidWorks is a reg i st e r e d rrademe1rk of
SolidWorks
C orpo r ati on
.
SPEC is ,1
P r e g is te r e d rrad e r n a r k of S ra n d a r d c rt on na n c e
E va l u at i on C o
• p or ati o n
.
Sun is a re gi s t e r e d trademark of Sun
J\t!icrosystcms,
Inc.
Svbasc is a r e g i st e re d trademark and SQ
Se r v e r is a t r a d e
L m a r k ofSvbase, Incorporated.
UNIX is a r e gi st e red trademark in rhc United
S rar e s a n d o r h c r c o u nt ri e s , l i cen s ed exc l u s i ve l y t hro u g h X/Open C o m p an v Lrd.
X
W i ndo w Svstern is a trademark ofrhe
M a s s
<
K h u sc t ts In sr i t u rc o f
T c c hn o lo g y
.
Contents
Opti mizing Al pha Executables on Windows NT with Spike
Analyzing Memory Access Patterns of Programs on Al pha-based Architectures
Open VMS Al pha 64-bit Very Large Memory Design
PowerStorm 4DT: A H igh-performance Graphics
Software Architecture
DART: Fast Appl ication-level Networking via
Data-copy Avoidance
Patents
RobertS. Cohn, Da1·id W. Goodwin, and
P.
GeoH!·cv Lowncv
Susanne
M.
Balle and Simon
C.
Stech', Jr.
Karen L. Noel and Nirin
Y.
Karkbanis
Benjamin
N . Lipchak, Thomas Frisinger,
Karen L.
B ircsak
,
Keith L
C:omdord, and
M ich a el I . Rosenblum
Robert J. Walsh
3
21
33
49
61
76
Digital Technical J ounul Vol. 9 No.4 1997
Editor's
Introduction
In 1992, DIGITAL announced the rastest 64-bit RISC microprocessor, r.he
Alpha, with a clock rate of 200 MHz.
Today's Alpha processor rem:�ins the leader in performance; the newest gen eration operates at 600 MHz, and the next generation will operate at greater than 1,000 MHz- gigahertz speed.
With the industry's most powerful processor in hand, DIGITAL's engi neers are working to applv Alpha in different areas of computing a.nd eftect optimal solutions to computing prob lems. Samples of that work arc pre sented in this issue and include programming performance tools, the
Open VMS operating system ror very large memory (VLM) applications, graphics adapters tor workstations, and the DART network acbpter ror high-end systems.
Spike is a profile-directed perror mance tool for optimizing Alpha exe cutablcs running on the Windows NT operating system. Designed specifi cally to improve the pcrrormance of large, call-intensive programs, such as commercial databases, CAD pro grams, compilers, and productivity tools, Spike has been shown to speed program execution by as much as 33 percent. Robert Cohn, Dave Goodwin, and Geoff Lowney describe Spike's two components. The Optimizer modifies code layout to improve instruction cache behavior and pcr rorms hot-cold optimization to reduce the number of instructions executed on tl-equcnt parhs through the program. The Optimization
Environment collects, manages,
;1nd applies profile information transparently for the programmer.
An experimental Atom-based per formance tool presented by Susanne
Balle and Simon Steely provides pro grammers with an understanding of the access pattern behavior of their technical applications. The tool gen erates histograms tor each memory reference in a program, thus aUowing the programmer to spot bottlenecks.
The authors step through an instruc tive case studv in the usc of the tool with Fortran programs, showing how different compiler switches affect tl1e execution of a program algorithm.
The
Open
VMS
Alpha operating system version 7.1 e xtends its support for VLM applications. The design work discussed by Karen Noel and
Nitin Karkhanis focused on increasing tlcxibilitv ror VLM applications and on adding svstem man:�gcment capa bilities. Areas reviewed are the shared memory objects designed to improve application scaling on the system, shared page tables to reduce applica tion start-up/shut-down times, and the physical memory reservation sys tem to allow cft-icient application use of system components, namely the translation buffer.
DIGITAL's PowerStorm series of graphics adapters rc.>r mid-range workstations provides exceptional pe rfor m ance on tl1e DIGITAL UNIX and the Windows NT o p er a ti n g sys tems. Benj Lipchak, Tom Frisinger,
Karen Bircsak, Keith Comeford, and Mike Rosenblum have written an inrormative rutorial about the
PowerStorm adapter design that was shaped in large parr Lw the existing competitive environment. Their dis cussion covers selected benchmarks and real-world performance experi ences, the advantages and disadvan tages in choosing a direct-rendering or an indirect-rendering scheme, and the ways in which the engineering team exploited the Alpha micro processor's exceptional Aoating point speed.
DART is a
622-megabit-per-second network adapter that connects gigabit class networks to gigabit-class 1/0 buses. It is designed to increase net work throughput and decrease system overhead. Bob Walsh explains tlut tl1e DART project, started in the late 1980s, anticipated the need to address fi.mdamental memory band width bottleneck issues fi-om a system level perspective. The main approach taken in the
DART adapter is data copy avoidance, without requiring changes to svstcrn call semantics.
The upcoming.fournalwill be a special issue that katures papers on programming languages and tools.
Topics include C and Fortran paral lclizing compilers, the C++ template facility, alias analysis algorithms, debuggcrs, and pcrtormancc tools for software running on the 'vVindows
NT, ll:-\IX, and Open VMS operating systems.
Jane C. Bbkc
;\lfana/.;ing tditor
2
Dif!,ital Technical Journal
Vol . 9 No.4 1997
Optimizing Alpha
Executables on
Windows NT with Spike
I
Robert
S.
Cohn
David W. Goodwin
P.
Geoffrey Lowney
Many Windows NT -based applications are large, call-intensive programs, with loops that span multiple procedures and procedu res that have complex control flow and contai n numer ous basic blocks. Spike is a profile-directed opti mization system for Alpha executables that is designed to improve the performance of these applications. The Spike Optimizer performs code layout to improve instr uction cache behavior and hot-cold optimization to reduce the number of instructions executed on the freq uent paths through the program. The Spike Optimization
Environment provides a complete system for performing profile feedback by handling the tasks of collecting, managing, and applying profile information. Spike speeds up program execution by as much as
33 percent and is being used to optimize applications developed by
DIGITAL and other software vendors.
Spike is a performance tool developed by
DIGITAL to optimize AJpha executables on the Windows
NT oper ating system. This optimization system has two main components: the Spike Optimizer and the Spike
Optimization Environment. The Spike O ptimi z er ' -' reads in an executable, optimizes the code, and writes out the optimized version. The Optimizer uses profile feedback ti·om previous runs of an application to guide its optimizations. Profile teedback is not commonly used in practice because it is difficult to coUect, manage, and apply profile informatjon. The Spike Optimization
Environment' provides a user-transparent profile feed back system that solves most of these problems, allowing a user to easily optimize large applications composed of m<my executables and dynamic link libraries
(DLLs).
Optimizing an executable image after it has been compiled and linked has several advantages. The Spike
Optimizer can see rhe entire image and perform inter procedural optimizations, particularly with regard to code layout. The Optimizer can use profile feedback easily, because the executable that is profiled is the same executable that is optimized; no awkward map ping of profile data back to the source language takes place. Also, Spike can be used when the sources to an application are not available, which is beneficial when
DIGITAL is working with independent software ven dors (ISVs) to rune applications.
Applications can be loosely classified into two cate gories: loop-intensive programs and call-intensive programs. Conventional compiler technology is well suited to loop-intensive programs. The important loops in a program in this category are within a single procedure, which is typically the unit of compilation.
The control flow is predictable, and the compiler can use simple heuristics to determine the frequently exe cuted parts of the procedure.
Spike is designed for large, call-intensive programs; it uses interprocedural optimization and profile teed back. In call-intensive programs, the important loops span multiple procedures, and the loop bodies contain procedure calls. Consequently, optimizations on the loops must be interprocedural. The control flow is
Digital Technic:1l journal VoL
9 No. 4 1997
3
4 complex, and pro hle ked back is required to accuratelv predict the ti·equcntlv e x e cuted parts of a program.
Call overhead is large tor these programs. Optimiza tions to reduce call overhead are most e f kctive with interprocedural intormation or profile feedback.
The Spike Optimiz e r implements rwo major optimiza tjons to improve the pertormance of tl1e call-intensi\'e programs just described. The first is code layout
:
'"
S pik e rearranges the code to improve loc a lit v and reduce the number of' instruction cache misses. The sec ond is hot-cold optimi
;
.
a rion (HCO
) : ' Spike optimizes the rrequent paths through a procedure at the expense of the inrrequenrly executed paths. HCO is particularly effective in optimizing procedures with complex con trol flow and high procedure call overhead.
The Spike Optimization Em·ironment pr
O
\· idcs a system for managing profile feedback optimization.'
The user interface is simple-it requires only two user interactions:
( l) the request to start ked back collec tion on an application and (2) the request to end col lection and to usc the feedback data to optimize tl1e applicatjon. Spike maintains a database ofprori le int(x mation. VVhen a user selects an application, Spike makes an entry in its database for the application and tor each of its component images. For each im a ge,
Spike keeps an instrumented \'ersion, an optimized
\"ersion, and profile intr x m a tion. VVhen the original a pplication is run, a transparency agent substitutes the instrumented or optimized version of the a p plication, as appropriate.
Tbis paper discusses the Spike performance tool and its use in optimizing Windows
NTbased applications running on AJpha processors. In the Following section, we describe the characteristics ofVVi ndows l'\l ·based applications. :-Je\t, we discuss the optimizations tiS<.:d in the Spike
Optimizer ::md e\·aluate their dkctil'cness.
We then present the Spike Optimiz a tion Environment for managing protile teedback optimization. A sum mary of our results concludes the paper.
Characteristics of Windows NT -based
Applications
To evaluate Spike, we selected applications rhat are tvpically used on Alpha computers running the
Windows
NT op erating system. These applic
:t t ions include commercial databases, co mputer aided design
( .AD) programs, compilers, and personal productiv ity tools. For comparison, we also included the bench mark programs from rhe SPECint95 suite." T:tble I i dentifies the applications and benchmarks, and the workloads used to e\ercise them. All programs <1re optimized l'ersions ofDIGITALAlpha bin
, l ri c s and are compiled with the same highly optimi zing back end that is used on the
UNIX and Open VMS systems.'' The charts and graphs in this paper contain dara ti·om a core set of a pplications. Note that we do nor have a full set o fmc1surements tor some applications.
In obtaining most of the profile-direcred optimiza tion results presented in this paper, we used the same in put for both training and timing so that we could know the limits of our app,·o:�ch. Others in the field ha\'C shown that a reasonablv chosen training input will vield reliable speedups tor other input sets.'" Our e
\ perience contirms this result. for the code lavout results presented in Figure ll, we used the official
SPF . t iming harness to measure the Sl'ECinr bench marks. This harness uses a SPEC tr a in ing input for profile collection and a different rekrence i n put tor riming runs."
Fig ure l is a graph rhat shows, f(Jr each applicat i on and benchmark, rhe size of the single executable or
DLL responsible tor the majmity of the execution time. The f)gure contains data fcx most of the applica tions and all t he benchmarks listed in Table l. Some
Windows NT based applications are very large. For example,
PTC h as 30 t i m es more instructions than
GCC, the largest SPECint95 benchmark. Large
Windo\\'s N
b a s e d applications have thousands of pmcedures and millions of basic blocks. With such programs, Spike achieves signihcant speedups by rear r a n ging rhe code to reduce instruction cache misses.
Code rearrangement should also reduce the working set of the program and the number of vi rrual memory page faults, although we have not measured this reduction.
To characterize a call-intensive applic a tion
, we looked at SQL�ERVR. We estimated the loop behav ior of'SQLSERVR by classifYing each of i ts procedures bv the a\·crage trip count of its most trequenrlv exe cuted loop, assigning a weight to each procedure based on the number of instructions executed in the procedure, and graphing the cumul a ti\ e distribution of instructions executed. The graph is presented in
Figure 2. Note that 69 percent of the execution time in SQLSERVR is spent in procedures that have loops w ith an average trip count less th<lll 2. Nearly all the run tirne is spent in procedures with loops with an a\'cragc trip count less than 16. An insigniti.cant amount ot· time is spent in procedures containing loops wirh high trip counts.
Of course, SQLSERVR exe c u tes manv loops, but the loop bo d i e s cross multi ple procedures To improve SQLSERVR per formance,
Spike uses code layout techniques to optimize code paths that cross multiple procedures. Also note that 69 percent of the e\ecution time is spent in procedures where the entrv basic block is the most ti·equentlv n:c cuted basic block. The entry basic bJock dominates the other blocks in the procedure, and compilers often find it a conl'enient location for placing instructions, such as register saves. In SQLSERVR., this placement is a poor decision. Our HCO targets this opportunit y to
\'ol. 9 1'\o 4 !997
Ta ble 1
Windows NT -based Applications for Alpha Processors and SPECint95 Benchmarks
Program Full Name Type
SQLSERVR
SYBASE
EXCHANGE
EXCEL
Microsoft SQL Server 6.5
Sybase SQL Server 11.5.1
Microsoft Exchange 4.0
Microsoft Excel 5 .0
Database
Database
Mail system
Spreadsheet
Word processing WINWORD
TEXIM
Microsoft Word 6.0
Welcom Software Technology
Texim Project 2.0e
Orcad MaxEDA 6.0 MAX EDA
ACAD
Project management
Electronic CAD cv
PTC
SOLIDWORKS
USTATION
EDS
MPEG
C1, C2
OPT, EM486
ESRI
VORTEX
GO
M88KSIM
Ll
COMPRESS
IJPEG
GCC
PERL
Autodesk AutoCAD Release 13
Computervision Pmodeler v6
Parametric Technology
Corporation Pro/ENGINEER
Release 18.0
SolidWorks Corporation
SolidWorks 97
Bentley Systems MicroStation 95
Electronic Data Systems
Unigraphics 11.1
DIGITAL Light & Sound Pack
Microsoft Visual C++ 5.0
DIGITAL FX!32 Version 1.2
Environmental Systems
Research Institute
ARC/INFO 7.1.1
SPECint95
SPECint95
SPECint95
SPECint95
SPECint95
SPECint95
SPECint95
SPECint95
Mechanical CAD
Mechanical CAD
Mechanical CAD
Mechanical CAD
Mechanical CAD
Mechanical CAD
MPEG viewer
Compiler
C1: front end
C2: back end
Emulation software
OPT: x86-to-Aipha translator
EM486: x86 emulator
Geographical
Information Systems
Database
Game
Simulator
LISP interpreter
Compression
JPEG compression/ decompression
C compiler
Interpreter
Workload
Transaction processing
Transaction processing
Mail processing
BAPCo SYSmark for
Windows NT Version 1.0
BAPCo SYSmark for
Windows NT Version 1.0
BAPCo SYSmark for
Windows NT Version 1.0
BAPCo SYSmark for
Windows NT Version 1.0
San Diego Users Group benchmark
Mechanical model
Bench97
Intake runner model
Rendering
Brake shoe model
MPEG playback
5,000 lines of C code
BYTEmark benchmark
Regional model
SPEC reference
SPEC reference
SPEC reference
SPEC reference
SPEC reference
SPEC reference
SPEC reference
SPEC reference move instructions from the entry basic block to less ti·equently executed b locks .
Figure 3 presents the loop behavior cbta tor many of the Windows NT-based applications listed in Table l.
Note that the a pplic a tion s bll into three groups. The most c11l-intcnsive a ppli c atio n s arc SQLSERVR,
ACAD, and EXCEL, which spend approximate!\·
70 percent of their run time in procedures ll'ith an a\·er
�lge trip count less than
2. C2, WINWORD, and
USTATION
Jre moderately cJII intensive; they spend approximately 40 percent of their run time in loops vvith an average trip count less than 2.
MAXEDA
�md
TEXJM are loop intensive; they spend approximately
10 p e r cent of their run time in loops with an
:
:werage trip count less than
2. TEXIM is dominated by a single loop with an average tr i p count of465. vVe further characterized the nonlooping proce dures by control How. If a proced ure consists ofonlv J few basic blocks, techniques such as inlining are effec tive. To estimate the control tlow complexity of
Digiral TcdmicJI )ournc�l
Vol.') \lo 4 !')')7
5
1 0,000
(j) zz oUl o::::J
_jo
1 ,000
��
1 0
_____ ....
....
-----
(j)
(j) w rr:
Q_
2
0
(.)
_j ::;;;: w
:::? co co
2
0
CCJ
_j w
X f-
0
0
CCJ
2 f-
<{
N w
X
<{
2
> a:
(j)
_j
0
(j) z
0 w f=
::::J
_j w
0 rr:
0 z
�
0
<{
(.)
> (j) w
0
Q_
A PPLICATION OR BENCHMA R K
KEY:
I NSTRUCTIONS
BA SIC BLOCKS
Figure
1
Size of vVindows
NT
-b,lscd Applications
:md
Benchmarks
4 8 1 6
AVERAGE TRIP COU NT
Figure
2
Loop Behavior ofSQI5ERVR
SQLSERVR, we classitled each of its procedures by the number of basic blocks, assigned a weight to each pro cedure based on the number of instructions executed in the procedure, and graphed a cun1ulative distribu tion of the instructions executed. vVe restricted this analvsis to procedures that have loops with
'111 average trip count less than
4.
(These procedures account tor 69 percent of the execution time of
SQLSERVR.)
The line labeled
ALL in Figure 4 represents the results of our analysis. Note that 90 percent of the run time of the non looping procedures is spellt in procedures with more than
16 basic blocks. The line labeled
FILTERED in Figure 4 represents the results when we ignored basic blocks that are rarclv executed. Note that
65 percent of the run time of the non looping pro-
1 00
(j) z
0 f-
��
80 rr:w tii
�
zw u:J�
60
2:0 40
��
2
&:l
20
2 4 8 1 6 32 64 1 28 256 5 1 2 1 K 2K 4K 8K 1 6K 32K 64K 1 28K 256K 5 1 2K
AVERAGE TRIP COUNT
KEY:
-+-
SQLSERVR
..,._ ACAD
--EXCEL
C2
-I}-
WI NWORD
--iJ-
USTATION
�
MAXE D A
-oTEXIM
Figure 3
Loop Behavior ofWindmvs NT -based Applications
6
Di�ital Technical )ourml Vol. 9 �o. 4 1997
I
'
'
-I
, , -
0 16 32 48 64 80
SIZE IN BASIC BLOCKS
96 1 1 2 1 28
KEY:
ALL
F I LTERED
Figure 4
Complcxitv of Procedures in SQLSERVR for Procedures with an Average Trip Count Less Than 4, Which Account ri:>1·
69 Percent of rhe Execution
Time cedures is spent in procedures with more than 16 basic blocks. In
SQLSERVR, procedures are large; many basic blocks arc executed, and many :trc not. Spi ke uses code layout and HCO to optimize the frequently executed paths through large procedures.
Figure 5 presents the control flow data for many of the Windows NT -based applications listed in Table l.
Again we measured only nonJooping procedures and ignored basic blocks that are rarely executed. Note that
Jll the JpplicJtions have large procedures. More than half the run time of the non looping procedures is spent in procedures that execute at least 16 basic blocks.
To estimate procedure call overhead, we counted the n umber of instructions executed i n the prolog and epilog of each procedure. This estimate is conserva tive; it ignores the cost of the procedure linkage and argument setup and measures only the number of instructions used to create or remove a frame from the stack and to save or restore preserved registers. I n
SQLSERVR, 15 percent o f all instructions are i n pro logs and epilogs. HCO removes approximately one half of this overhead .
The chart in Figure 6 shows the procedure call over head for most of the Windows NT based applications listed in Table l.
The overhead ranges fi-om 23 percent to
2 percent. The applicJtions are ordered according to the amount of run time in procedures with an average trip count less than
8 in Figure
3.
The call overhead is roughly correlated with the amount of run time in low trip count procedures. Figure 6 includes data for some of the SPECint95 benchmarks, which are ordered by the amount of run time in procedures with an average trip count less than
2.
The ;1mount of call overhead tor th e se benchmarks ranges from
24 percent to 0 percent and is more strongly correlated with the amount of run time in low trip count proced ures.
Optimizations
The Spike Optimizer is organized li ke a compi ler. It parses an execu table into an intermediate representa tion, optimizes the representation, and writes out an optimized executable. The intermediate representa tion is a list of Alpha machine instructions, annotated
1 00
(f)
�
80
Of='
::::lz
�if
60 zw we:.
�0
40
5�
::::J::::J
::;; � ow
20
0 16 32 48 64 80
S I ZE IN BASIC BLOCKS (FILTERED)
96
1 1 2 1 28
KEY:
-+SQLSERVR (69%)
--ACAD (82%)
-¢-
WINWORD (49%)
-o-
USTATION (44%)
-EXCEL (71% ) -6MAX E D A (1 3%)
C2 (44%)
Nole that the number that appears after the application name indica1es I he percentage of the lolal execution time spent in procedures with an average trip counlless than 4.
Fig u re 5
Compkxity of Procedures in 'vVindows NT-bascd Applications tor Procedures with
<Hl
Averc1ge Trip Count Less Than 4
Digir<tl Technic1l journal
Vol.
9
No.4
1 997 7
0
�
25
::l
&l
�
W O
20
1 5
0 0 f--
1 0
I g: frl
5
� � 0 a:
> a: w
(f)
_j
0
.-
0
<{
(.)
<{
.-
_j w
(.)
X w
,--
N
(.) n
0 a:
5: z
� n n z <{
0
0 f=
<{ f-w
X
<{
2
�
X w f--
:::J
APPLICATION OR BENCHMARK
X w f-a:
I n n
�
([)
�
0
(?
(? w
0..
:::2 ro
2
([)
([) w a:
0..
0
(.)
Figure 6
Proced u re Call (),·crhccld ( Time Spent in Prolog and Epi log ) with a sma l l amount of add itional i n formation. O n top of the i ntermediate representation, the optimizer builds compiler-lik.c structures, including basic blocks, procedures, a How graph, a loop graph, and a call grap h . 1 1 Images a rc large, and the algorithms and rep resentations used in the optimizer m ust be time and space efficient.
The Spike Optimizer performs a n interproccd ur;11 dataHo11· analysis to sum marize register usage 11·ithin the i mage.
1 1 This enables optimizations to usc and real locate registers . The i nterprocedural dat�lf1ow is bst, requiring less than 2 0 seconds on the largest applications we teste d . Memory dataHow is much more d i fficult to an�1lvze because of the l i mited i n for mation available in an executable, so the opti mizer
Optimi zations rewrite the inte rmediate repn:senta tion. The import::�nt optimizations are code Lwout and
HCO. The
Spike Opti mizer also pcrforrns additional optimizations to reduce the 0\'erhcad of shared libraries .
Code Layout
We dcri1·ed our code layout algorithm fi·om prior work on p rori.lc-guid cd code positioning by Pettis and
Hansen." The goal of the algorithm is to reduce instruction cache miss. Our algorithm consists of' th ree steps. The first step reorgani zes basic blocks so rh�H the most frequent paths in a procedure :�re sequential, which permits more efficient usc of cache lines :mel the exploitation of instruction prefetch . The second step places procedures in memorv to avoid i nstruction e1che contlicts . The third step spli ts procedures i nto hot and cold sections to i m p rm· c t h e p c rto r n n nc c of procedure p lacement.
The following example i l lustrates bas1c block reor ganization . Consider the flow graph in Figure
7,
,,·here each node is a basic block that contains t()ur i nstruc tions. The arms of the cond itional branches arc 1<1bclcd with thei r relatii'C p robabilities. Assume thJt the target is an A lpl u
2 1 1 64 proces so r.
1 · ' E�K h i nstruction is
4 bytes, and the instruction cache is organized into
32-byte lines; each cache l i ne l10lds two of the tour i nstruction basic bl ocks. A si m p le breadth-first code lavou t orders the code AR
CD
EF G H , and the com mon path
A BD FGH req u ires tou r cache l ines. Two cache
J ine s
( CD and
EF) each cont:� i n a basic block that is infi-cq ucntl\' used bur ll'bich m ust be resident in the cache for the ti-cq uently used block to be C\ccuted .
If we order the code so that the common path is adja cent
(AB DF GH
CE), the infi-equenrlv us e d blocks arc in the same l ine ( CE ) , and thev do not need to be i n the cache to execute the ti
· eq u entlv used blocks.
Straight- l i ne code is :llso better able to exploit instruction pre retch . On an instruction c1che miss, the
Alpha 2 1 1 64 processor p refetchcs the next four cache lines i nto a rctill bu ffer. After an i nstruction cache miss, the processor fi-cqucntlv is able to e\ecute a straight line code path without stalling if the code is i n the sccon d - i ci'CI e1chc. A branch that is t aken typically requires a n <ld d i tional cache miss i f the target of the branch is nor al readv in the instruction Clchc.
'vVc reorganize the ba s i c blocks using J simple, greed1· a l gorithm, similar to the t
LK e
picking algo-
Fig ure 7
Basic Block Rcorg�HliZ<ltion
O F
� B
G H
C E
8 Digitcll Technical )uu rn;ll Vol . lJ
:--:o. 4 l lJlJ7
rithm used in trace scheduling. " Our goal is to find
:1 new ordering of the basic blocks so that the tall through path is usually taken . vVe son the list of flow graph edges by execution count and process them i n order, beginning with the highest v:�lues. For each edge we make the destination basic block immediately fol low the source block, unless the source has already been :�ssigncd a successor or the destination has already been assigned a predecessor.
We pl:�ce procedures to avoid confl icts in the instruction cache.
An
Alpha
2 1 1 64
has a primary instruction cache of 8 kilobytes
(
KB) that holds
2 5 6
lines of
32
bvtes each. Two instructions contlict in the cache if thev arc more than
32
bytes ap:�rt Jnd map to the s;lrnc c1cbc l i ne, specificallv, if
address 0/32
mod
256
=
addressl /32
mod
2 5 6 .
Our stratcgv is to place procedures so that frequentlv called proced ures arc ncar the cal ler. Consider the simple example in Figure
8. Assume procedure A calls procedure
C in a loop . A and C map to the same cache lines, so on c:�ch call to
C, C replaces A in the cache, and on each retu rn ti-om
C, A replaces
C.
I f we reorganize the code such that C follows A, both A and C can fit in the cache at once,
:111d there Jrc no conflict misses when A calls
C.
We usc another greedy algorithm to pbcc proce dures. The example presented in Figure 9 illustrates the steps . vVc build a call graph and assign a weight to each edge based on the number of calls. I f there is more than one edge with the same source and destina tion, we compute the sum of the execution counts and delete all but one edge. Figure 9a shows the cal l graph.
To place the procedu res in the graph , we select the most heavilv weighted edge
(B to C ) , record that the two nodes should be placed :1djacently, collapse the t\-vo nodes into one
( B .C),
:1nd merge their edges ( :1s shown in Figure 9b ). We ag:1in select the most heavily weighted edge and continue ( Figure 9c) u ntil the graph is reduced to a single node A.D. B . C ( Figure
9d ). The final node contains an ordering of all the pro ccd u res. Speci:1l care is taken to ensure that we rarelv require a branch to span more than the maximum branch displacement.
The effectiveness of procedure placement is limited by large procedures. In the PERL benchmark from
SPEC,
which is one of the smallest p rograms we stu d ied, one freq uently executed procedure is larger than
3 2 KB ,
tou r times the size of the instruction cache on the Alpha
2 1 1 64
processor. In SQLSERVR., more than hal f the run time is spent i n procedures with more than
16
basic blocks. To address this problem, we split procedures into hot and cold sections and treat each section as an independent proced ur e when placing procedures. To split a procedure, we examine each basic block and usc a threshold on the execution count
16 KB
8 KB
0 w
A
B
,LJ
-�
8-KB P R I MARY
INSTRUCTION
CA CHE
1
16 KB
Figure 8
Procedure Pbcc mcnt
A
7 �
D� c
( a )
Figure 9
Steps in rhc Procedure
Placement Algor-ithm
A
� �
( b)
A . D
�1
B.C
( c )
Digital
TcchnicJI fournJI
A . D.B.C
( d )
Vol. 9 :--Jo. 4 1 997
to decide if a basic block i s col d . We usc a single threshold f(x the entire progr.1111. The threshold is chosen so that the total execution time for all the bJsic blocks below the threshold constitutes no more than
1 percent of the execution time of the progr�1111 .
Proced ures with both hot and cold basic blocks Jrc split; otherwise, they are lett intact.
Figure
1 0 i l l ustrates the importance of procedure splitting. The figure charts the speedup on SQLS E RYR, running on an Alpha
2 1 064 workstation,1; for the components of our code layout algorithm. The bar graph i ndicates that chaining basic blocks or pl acing proced ures results in a speed up of less than 4 percent, but placing procedu res after splitti ng yields a
IS per cent speedup. Using al l our optimizations ( c h ;�ini ng, spl itting, and placing) together produces a 1 6 percent speed up.
Figure I I presents the speedups fTom code la�'out tor the Windows NT -based appl ications and the SPECim benchmarks running on an Alpha
2 1 1 64 workstation .
Speedups range ft·om 45 percent to
0 percent; most
Note that this data is for
the SQLSERVR
application
Alpha
2 1 064 microprocessor.
Figure 1 0
Speed up for Code Llyou r lw
Optimization running on an appl ications show a noticeable i mprovement. The leftmost seven Windows NT -based applications
(SQLS E RV R through
TEX I M ) are ordered by the amount of time spe nt in proced ures with an average trip count less than
8 i n Figure
3.
Note that all but the most loop-intensive appl ication s how a significant speedup from code layout. Three programs show min imal speed up: that fits
TEXIM is dominated by a single loop in the instruction cache, and
IJ PEG and
COM PRESS are dominJtcd by two or three sma l l loops. These programs d o not have a n appreciable amount of instruction cache miss; changi ng the code layout cannot imprO\'e their performance.
Hot-Cold Optimization
Hot-cold opti mization i s J generali zation of the proced ure-splitting technique used in our code l avout a.lgoritll m .c vVe opti mize the hot part of the proced ure
( ignori ng the cold parr) bv elim inating all instructions that :�re required on.ly by the cold part. To i m p l ement this optimization, we create
::1 hot procedure by copy ing the frequently executed basic blocks of a proce d ur e . All c:t!ls to the original procedure are redirected to the hot proced ure. Flow p�nhs in the hot procedure tllat target basic blocks th:�t were not copied :�rc red i rected to t h e appropriate basic block in t h e original
(cold ) proced ure; that is, the tlows jump i nto the mid dle of the original proced ure. We then optimi7.e the hot proced ure, possi bly J t the e xpense of the rl ows that pass through the cold path .
HCO is best u n derstood by working through an exte nded cx:�mple. Consider the procedure f o o
(shown i n Figure 1 2 ), which is J simplified version of a proced ure ti·om the \iVindows NT kerne.l.
50
45
�
Q.
::>
0 w w
Q.
(f) i=' z w
0
40
35
30
25
20
1 5
1 0
5
0 a: 0
> <( a: 0 w
<(
-'
0
(f)
-'
(\J w
0
0
X w
0 z a: 0
0
5 z
�
�
::>
�
X w
1w z
<(
I u w w
<(
>-
(f)
>
(.')
(f) u
0
(f) (f)
>:: 0 a: w
0
0
:::;
0
(f)
(.') w
Q.
2
0
1-
([)
Q.
0
<t
2 w
APPLICATION OR BENCHMARK
Figure 1 1
Speedup ti-om
Code LJ\'ou r
1 0 Digit;1l Tccilnic;ll journal
Vol. 9 No. 4 J <Jl)7
:::;
X w
1a:
0
>
�
(f)
2 u
0
(.')
-' a: w
Q.
0
(.')
(.') w
(f)
(f)
Q. w
:::? a:
Q.
2
0
0
8
9
1 0
4
5
6
7
1
2
3
1 1
1 2
1 3 f oe : add l aO , a 1 , a O b n e s0, L2
L 1 : b s r f 1 a d d l s O , a O , t 1 s t l t 1 , 4 0 ( g p )
L2 : l d a s p , 1 6 ( s p ) s t q s O , O < sp ) s t q r a , 8 ( s p ) add l a 0 , 1 , s0 l d q sO , O ( s p ) l d q r a , 8 ( s p ) l d a s p , - 1 6 ( sp ) r e t ( r a ) a d j u s t s t a c k s a v e s O s a v e r a sO a O
=
= aO
+ a O
+ a 1 b r a n c h i f sO c a l l f 1 t 1
= a O
+ sO s t o r e t 1 r e s t o re s O r e s t o r e r a a d j u s t s t a c k r e t u r n
! =
0
Figure 1 2
Simpliticd Version of a
Kernel
Procedure from rhc
Windows �T
Assume that the always ta branch in line 6 of f o o is al most k en and that lines 7 through 9 arc almost never executed. When we copy rlK hot part of the pro cedu re, we exclude li nes 7 through 9 of f o o. The resul ting proced ure f o o 2 is shown in Fi gu re 1 3 .
1
2
3
4
5
6
7
8
9
1 0 f oo 2 : l da s p , 1 6 ( sp ) s t q s0, 0 ( s p ) s t q r a , 8 ( s p ) add l a 0 , 1 , s0 add l a O , a 1 , a O b e q s O , L 1 l dq sO, O < s p )
L dq r a , 8 ( s p ) l d a s p , - 1 6 ( s p ) r e t ( ra )
Figure
1 3
Hor Procedure
Note the reversal of the sense of the br:meh hom b n e in f o o to b e q in f o o 2 and the change of the branch's target from L 2 to
L
1 . All cal ls to f o o are redirect e d to the hot procedure f o o 2. If the branch in line
6 of f o o 2 is taken , then comrol transt-Crs to l ine
7 of f o o , which is in the middle of the or i ginal proce dure . Once passed to the origin a l proced ure, control nel'er passes back to the hot procedure . This kature of HCO enables opti mization; when optimizing the i hot procedure, we can relax some of the constraints m po s e d by the cold procedure.
So hr, we have set up t he hot procedu re t(>r opti mi zJtion
, but we have not made the proc e dur e
�my faster. Now we show how to optimize the procedu re .
The hot procedure n o longer contains
:1 call , so we em l delete the save and restore of the return add ress i n i n e s
3 and 8 o f f o o
2 i n Fi gure
1 3 .
I fthe branch trans fers control to L 1 i n the cold procedure f o o , we m ust arrange 6:x r a to be saved on the stack. In gen e r a l
, whenever we e nter the original pro c e d u r e ti·om t he hot proc ed ure, we must fix up the state to match the expected state . 'vVe cal l the fix-up operations compen sation code. To insert compensation code,
\\'e cre�ue
J stu b and red irect the branch in line 6 of f o o 2 to
1
2
6
7
8
9
3
4
5 branch to the stub. The stub saves r a on the stack and branches to L
1 .
Next, note that the instruction in line 5 of f o o 2 writes a
0 , bur the value of a
0 is never read in the hot procedure. a
0 is n ot truly dead, however, because it is sti l l read if the branch in l ine 6 of· f o o 2 is taken .
Therefore, we delete line
5 ti·om the hot proced ure and place a copy of the i nstruction on the stu b. HCO tries to eliminate the uses of prese rved registers in a procedure. Preserved registers can be more expensive than scratch registers because they must be saved and restored if they are used . Preserved registers are typi cally used when the lifetime of a val ue crosses a call. In the hot proced ure, no litCtime crosses a call and the use of a preserved register is unnecessary. We rename all uses of s
0 in the hot procedure to use a free scratch register t
2 . We insert a copy on the stub from t 2 to s
0 . 'vVe can now eliminate the save and restore instruc t i o ns in Jines 2 a nd 7 of Figure
1 3 and pl ace the save on the stub.
We have el i mi nated all references to tl1e stack in the hot proced ure. The stack adj usts on l ines l and 9 in F i gure
1 3 can be deleted trom the hot procedu re, and the initial stack adj ust c:m be placed in the stub.
The final code, including the stub s t u b
1 , is listed i n
F i gu re 1 4 . T he number of instructions executed in the ti·eq uent path has been reduced from stu b is ta ken, then the fu ll 1
10 to 3 . If the
0 instructions and an extra copv and branch are executed. f oo 2 : s t u b 1 : a d d l a O , 1 , t 2 beq t 2 , s t u b 1 r e t ( r a )
L d a s p , 1 6 ( s p ) s t q s O , O < s p ) s t q ra , 8 ( s p ) add l a 0 , a 1 , a 0 mov t 2 , s 0 b r L
1
Figure 14
Optimized Hor Proo:d u rc
Final lv, we \
\
'
Ou l d like to inJine the hot procedure
Copies of instruc t ions
1 and
2
. can be placed inline.
For the inlined branch, we must create a new stub th:n materializes the return address into r a before transtcr ring control to s t u b
1 .
Except t(.>r p:1rrjal inlining
HCO optimizations in
, we have implemented all the
Spike. These optimizations are
•
•
•
Partial dead code elimination 1c'-the removal of dead cod e in the hot proced ure
Stack pointer adjust elimin:nion-the removal of the stack a dj usts in the hot procedu re
Presen·ed reg i ster elimination-the removal of the save and restore of preserved registers in the hot procedure
Digiral TcdlJliC<ll Journal
Vol.
9
:-Jo. 4
1 99 7
I I
•
Peephole optim ization-the removal i n the hot pr ocedure or' self-assignments and cond itional branches with an always-false condition
Figure 1 5 shows coverage statistics tor th e H C O_ op timizations . CoverJge represents the percentage or execution time spent in each categorv. To compute coverage, we assigned e a c h procedure to a cnegorv a n d then for each category calcu lated the total n u m ber of instructions execu ted l)\' i ts procedures. The cate gory OPTIMI ZED indic1tes the set of proced ures optim i z e d
Lw
HC:O. The portion of the execution time spent in these proced ures is typical ly but often higher. The category
60 percent
INFR.EQ UEr1T is the set of p rocedu res whose execution times arc so sma l l
( less than
0
. l percent of the total time ) th�1t \\'e did nor think it
\\'JS worth,,·hilc to optimize the proced ures.
Icrnorincr proced ures us
. to optimize less than 5 percent of the instru ctions
.
1 1 1
J program, a significant red uction in optimizer nmc.
The category NO
SPLIT represents the proced ures thJt we cou ld not spl i t into bot and cold parts because all basic blocks had simi lar execution counrs. The cate
. stack pointer is modi ricd after the initi,tl stack adjust
. l l1 the prolog. We decided not to opti mi ze these proce d u res, but it is possible to do so with extra analysts.
Note that the execution ti me spent i n this category of procedures is small except for i n C2, where the cate gory contains two p roced ures and the coverage IS 7 percent. h n�1l ly, the category NO ADVANTAGE rep resents the procedu res that were split but that the optimizer ,,.,,s not able to i mpro,·e .
Figure
1 6 shows the O\·era l l reduction in path leng;h �1s a resul t of HCO, broken dmm by optimiu tion. Most of the reduction in path l cngt h comes equally h-om the removal of un necessary sa_ve and restore instructions Jnd from the removal of partial dead code . Stack pointer adjust elimin ation and peep hole opti mi zation result in sm.1ller additional gai ns
.
A l a rg e peephol e category is usu:�lly the result of a save and restore of�1 prcsen·ed register that is made unnec essarl· lw H CO; the restore is co m-er ted to a self assig
; lm
� nt Lw copy prop
:1 g; a tion
, which is then removed by peep hole optimization.
HCO is most effective on call
in tensive programs such as SQLS E RVR, ACAD, and
C2, where we e limin::�te calls when creating the hot procedures. for
WINWO R D , the speedup is sma ll because coverag.e is k_>,,
.
.
\I'C could not ri nd
. t o split the proced ures.
70 w e::. w
::;;;
1z
Q
60
50
40 u w
30
20
1 0
1 00
90
80
0
<{
-' w a
(f)
KEY:
CJ c::::::J
D
-
Figure 1 5
HCO
Covcr<lg,c lw F\L·cut ion Time
1 2
DigitJI Technic:1l
)ound
APPLICATION OR BENCHMARK
VoL 9 'io + 1 997
1 2
!='
1 0 a: w
� 8
I f-
(.') z
I f-
<i o._ z
6 u
::J
0 w a: 2 w
(f)
_j
0
(f)
KEY
0
PEEPHOLE
D
SP ADJUST
-
DEAD CODE
-
SAVE/R ESTORE
X w f-
APPLICATION OR BENCHMARK
Figure
1 6
Red uction in P:trh
Lmgrh As a
Resu lr of HC:O
For EXCEL, HCO was able to spl it the procedures, but there is often a call in the hot path. Inlining may help in optimizing EXCEL, but frequentl y the call is to a shared library.
HCO is less effective on loop-intensive programs such as USTATION, fvlAXEDA, and T
E
X
I M . HCO provides a framework for opti mizing loops, and
Chang, Mahlke, and Hwu have shown that eliminat ing the i n frequent paths in loops enables additional optimizations, such as loop invariant removal.17
However, our current i mplementation of Spi ke includes :�lmost no information about the aliasing of memory operations; it can only optimize operations to local stack locations, such as spills of registers.
A leaf procedure is a proced u re that does not contain a procedure call . Figure 17 compares the amount of time spent i n leaf procedures before a nd after H CO is applied. B y eli mi nating i nfrequent code, HCO is able to eliminate all calls i n proced ures that represent 10 percent to 20 percent of the execu tion time in C2, ACAD, SQLSERVR, and MAXEDA.
For the other Windows NT-based applications, the increase in time spent in leaf procedures is very sma l l .
Most Windows N T based applications spend much less than half the time i n leaf procedures. To improve the performance of these applications, an optimizer n eeds to improve the performance of code with calls in the frequent pat h .
Code size a n d its efkct on cache behavior i s a major concern for us. In IJrge applications, locality for i nstructions is present but not high . If an optimization decreases path length but also decreases locality as a side eftect, the net result can be a loss i n performance.
Figure
1 8
shows the total i ncrease in code size as a result of opti mization . HOT + COLD is the part of the increase that comes from replacing a single procedur e with the original procedure plus a copy of the hot part.
STUB is the increase attri buted to stub procedu res.
Overall, the i ncrease in size is small. The maximum increase is 1 1 .6 percent for C 2 . SQLSERVR has the best speedup and is only
3 . 1 percent larger. Looking at the i ncrease in total code size is misleading, however.
HCO is not applied to procedures that are executed infrequently, which typically accou nt for more than
95
percent of the i nstructions in a program, so tripling the size of optimized proced ures would resu lt i n only a modest increase in code size. Note that tripli ng the size of the active part of an application usuall�r disas trously decreases perf(xmance.
Digiral Technic.1J ) ourn<li Vol . 9 No. 4
1 997 1 3
1 00
90 i=' z w
0 a: w e:_ w
80
70
60
50
4 0
30
20
1 0
0 a:
> a: w
<J)
_j
0
(f)
0
<1:
0
<1:
KEY
-ORIGI NAL
-o-
AFTER HCO
_j w
0
X w
(\)
0
0 a: z
0
0
�
� f=
<1: f-
<J)
:J
<1:
0 w
X
<1:
::;;
�
X w f-
::::;
APPLICATION OR BENCHMARK
X w fa:
0
>
::;;
U5
>:::
0)
0)
::;;
0
(.9
(.9 w
Q_
::::?
<J)
<J) w a:
Q_
2
0
0
Figure 1 7
Time Spent i n
Leaf
Procc d u t·es before and after HCO
For this reason, we also measu red the i ncrease i n code size based o n the procedures that were optimized .
Figure 1 9 compares the total sizes of the hot proce d u res with the total sizes of the original proced ures from which they were deri,·ed. For each procedure, by copying just the ti·equent l y executed part of the proce dure, we excluded about
50 percent of the original.
Next, we eliminated code that was ti-equently executed but only reachable through an inffequently nccuted path and therefore u n reachable in the hot procedure.
This code usually represents only
1 percent of the total size of a proced u re. finally, we optimized the hot pro cedure, reducing the remaining code size bv about lO percent, which is 5 percent of the size of the original procedure. The fi nal sizes of the hot procedures as pcrcentJ.ges of the sizes of the originJ.I procedures are shown in the .line labeled HOT. l'v!aking the most freq ucntlv executed part of a program
50
percent to
80
percent smaller yields a big improvement in instruction cache behavior; however, it would be mis leading to <lttri bute tbis improvement to HCO, since our code layout optimization achieves the same result.
When HCO is enabled, the cache layout optimizations arc run after HCO. The baseline we compare against also has cache optimizations enabled, so improve ments attributed to HCO are improvements beyond those th::Jt the other optimizations can make. HCO does make the ti·eq uently executed parts
1 0
percent
UJ
<J)
<( w a:
(.)
�
�
1 4 w
0 1 2 a: tf
1 0
�
8
<J) z 6
4
2
0 a:
> a: w
<J)
_j
0
<1:
0
<1:
_j w
0
X w
(\)
0
0 a:
0
� z
Q f-
<1: f-
<J)
:J
<1:
0 w
�
X w f-
::::;
APPLICATION OR BENCHMARK
X w
Ia:
0
>
KEY
-TOTAL
---G-
HOT
+
COLD
STUB
="
U5
'<:
0)
="
0
C)
(.9 w
Q_
::::?
<J)
(j) w a:
Q_
="
Figure 1 8
Overa l l Increase in Code Size after
HCO
1 4
Digital
Technic� I Jound
Vol. 9 :---:o. 4 1997
�
1 80
�
1 60 ffi
1 40 a..
;;:;
1 20
� 1 00
(f)
� 80 w
(f)
60
40 w a: u
�
20
0 a:
> a: w
(f)
_j
0
(f)
KEY:
-+-
--o-
--o-
__.._
TOTAL
COLO
HOT
STUB
0
<( u
<(
_j u
X
LlJ
0 a:
0 s z
� z
0 f=
<(
1-
(f)
:::J
<(
0 w
:::;:
1-
:::J
APPLICATION O R BENCHMARK
X w
1a:
0
>
�
(f)
�
:::;:
0
(.')
(.') w a..
:::2
(f)
(f) w a: a..
:::;:
0 u
Figure 1 9
Size
of
Optimized
Procedures after
H CO smal ler, but we did not see significantly better i nstruc tion cache behavior when we ran programs with a cache simulator.
If we were to perform partial inlining, only the hot procedure would be copied . Since the hot procedure is less than half the size of the original procedure, partial in lining wou ld greatly reduce the growth in code size due to i nlining.
The line labeled
COLD in Figure
19 shows how the size of the cold procedure is affected by
HCO.
When we redirect all calls to the hot procedure, some code in the original procedure becomes unreachable. The amount of unreachable code is usual ly Jess than percent, which is much smaller than the
10
50
percent of the code we copied to create the hot procedure . The infrequent paths in a p rocedure often rejoin the fre quent paths, which makes it necessary to have copies of both types of paths in the original procedure.
The line labeled
STUB shows the code size of the stubs, which is very smal l . A stub contains the com pensation code we introduce on a transition from a hot routine to a cold routine. We also implemented a variation of
HCO that avoided stubs by reexecuting a procedure from the beginning instead of using a stub to reenter a routine in the middle. It is usually not pos sible to reexecute the procedure from the beginning because arguments have been overwritten. Given the small cost of stubs, we did not pursue this method.
The line labeled
TOTAL shows that
HCO makes the total code (HOT + COLD + STU B ) 20 percent to
60 percent bigger.
A procedure is partitioned so that there is less than a 1 percent chance that the stllb and cold part are executed, so their size should not have a significant effect on cache behavior as long as the pro file is representative.
Figure
20
shows how splitting affects the d istri bu tion of time spent among different procedure sizes for two programs where
H C O is eftective
( C2
and
SQLSERVR) and two programs where it is not
(�'\EDA and vVIl\TWORD ) .
For the graphs shown in parts a through d of Figure
20,
we classified each procedure by its size in instructions before and after
HCO and plotted two cumulative clistributions of exe cution time. The farther apart the two lines, the better
HCO was at shifting the distribution from large proce dures to smaller procedures. Note that most of the programs spend a large percentage of the time in large procedures, which suggests that optimizers need to handle complex control flow wel l , even if profile infor mation is used to eliminate infrequent paths.
Managing Profile Feedback Optimization
Profile feedback is rarely used in practice because of the difficulty of collecting, managing, and applying profile information. The Spike Optimization Environ ment' provides a system for managing profile feedback that simplifies this process.
The first step in profile-directed optimization is to i nstrument each image in an application so that when the application is run, profile information is collected.
I nstrumentation is most commonly done by using a compiler to i nsert counters into a program during compilation's or by using a post-link tool to insert counters into an image. '9·20 Statistical or sampling based profiling is an alternative to counter- based tech niques.2u2 Some compiler-based and post-link systems require that the program be compiled special ly, so that the resulting images are only usefi.!l for generating profiles. Many large appl ications have lengthy and
DigitJI Technic�[ Journal
Vol.
9
No
4
1997 1 5
1 6
70 90
I I
0 1 30 1 50 1 70 1 90
50 70 90 1 1 0 1 30 1 50 1 70 1 90
KEY
-o-
--+--
KEY
-o-
--+--
( a ) SQLSERVR ( b )
C2 i='
1 0 0 z
90 f= u w w a: w
� w
2 i=
80
70
60
50 z 40
0
30
::::l
20
1 0
0
1 0 30
KEY:
-o-
--+--
70 90 1 1 0
1 30
1 50 1 70 1 90
2 f= z
0 f=
::::l u i=' w a: w
� w w
1 00
90
80
70
60
50
40
30
20
1 0
0
1 0
1 1
0 1 30 1 50 1 7 0 1 90
KEY
-o-
--+--
( d ) 1\llAXEDA (c)
WI NWORD
Figure 20
Cumulati1·c Dis1 riburion of Execution Time l)\' Procedure Size before and after HC:O complex build procedures. For these applicatio ns, requiring a special reb u i ld of the application to col l ect profi les is an obstacl e to the use of profilc-din.:ctcd o ptimization .
Spike d i rectly instruments the final production images so that a special compilation is not required.
Spike does require that the images be l i n ked to include relocation i n rcm11ation ; howe1'er, incl uding this extra i n formation does nor i ncrease the n u m ber ot- 'instruc tions in the image and does not prn·ent t he compiler fi·om performing fu l l optim iLations when generating the i mage.
Most applications consist of a main executable :llld many
DLLs.
Instru menting all tl1e i m ages in an app l i cation ca n be d i fficult, especiallv when the user doing the profi le-directed optim ization does not kno\\· all the DLLs i n the application. Spike relie1·es the user of this task bv finding all tile DLLs that the app l icltion uses, even i f rhev are loaded dynamically with a c:dl to
Load Library.
After instrumentation, the next step i n profi le di rected opti m i Lation is to execute the i nstrumented appl ication and to collect profile i n f(mnation. Most profile-di rected opti mization systems req u i re the user to first exp l i citly create instrumented copi es of each image in an app licati o n . Then the user m ust assemble tbe instrumented images into a new version of the application and run it to collect protile inr(>rmati o n . As the pro fi l e i n f-ormation is generated, the user is responsi ble for l ocating all the profile information generated for each image and m erging that i n forma tion i nto a single set of profiles.
Our experience with users has shown that requ i ring the user to manage the instru mented copies of the i mages and the profile information is a frequent source of problems. For example, the user mav fai l to instru ment each i mage or mav attempt to i nstru ment an i mage that has already been instrumented . The user mav be unable to locate all the generated profi l e i n formation or m:ty i n cor recrlv com bine the information. Spik�: frees the user
\'ol. 9 �"- 4 1997
from these tedious and error-prone tasks by managing both the instrumented copy of each image and the profile information generated for the image.
After profile information is col lected, the final step is to use the profile information to optimize each image.
As with instrumentation, the typical profile-directed optimization system requires the user to optimize each image explicitly and to assemble the optimized appli cation. Spike uses the profile information collected for each image to optimize all the images in an application and assembles the optimized application for the user.
Spike Optimization Environment
The Spike Optimization Environment (SOE) provides a simple means to instrument and optimize large applica tions that consist of many images. The SOE can be accessed through a graphical interface or through a command-line interface that provides identical func tionality. The command-line interface allows the SOE to be used as part of a batch build system such as make.23
In addition to providing a simple-to-use interface, the SOE keeps the instrumented and optimized ver sions of each image and the profile information associ ated witl1 each image in a database. When an application is instrumented or optimized, the original versions of the images in the application are not modi fied; instead, the SOE puts an instrumented or opti mized version of each image into the database. When the user invokes the original version of an application, the SOE uses a transparency agent to execute the instrumented or optimized version.
The SOE allows the user to instrument and optimize an entire application using the fol lowing procedure:
1 . Register: The user selects the application or applica tions that are to be instrumented and optimized. The user needs to specifY only the application's main image. Spike then finds aU the implicitly linked images
( DLLs loaded when the main image is loaded) and registers that tl1ey are part of the application.
2 .
I nstrument: The user requests that an application be instrumented . For each image in the application, the SOE invokes the Spike Optimizer to instrument that image. The SOE places the instrumented ver sion of each image in the database. The original images are not modified .
3 . Collect profile information: The user runs tl1e origi nal application in the normal way, e.g., from a com mand prompt, from Windows Explorer, or indirectly through another program. Our transparency agent
(explained later in this section) invokes tl1e instru mented version of the application in place of the original version. Any i mages dynamically loaded by the application are instrumented on the fly. Each time the application terminates, profile information for each image is written to the database and merged
\vith any existing profile information.
4. Optimize : The user requests that an application be opti mized. For each image in the application, the
SOE i nvokes the Spike Optimizer to optimize the image using the collected profile information and places the optimized version of each image in the database.
5. Run the optimized version: The user runs the orig inal application, and our transparency agent substi tutes the optimized version, al lowing the user to evaluate the effectiveness of the optimization.
6. Export: The SOE exports the optimized i mages from the database, placing them i n a directory spec ified by the user. The optimized images can tl1en be packaged \vith other application components.
The Spike Manager is the principal user interface for the SOE. The Spike Manager displays the contents of the database, showing the applications registered with
Spike, the images contained i n each application, and the profile information collected for each i mage. The
Spike Manager enables the user to control many aspects of the instrumentation and optimization process, including specifYing which images are to be instrumented and optimized and which version of the application is to be executed when the original applica tion is invoked.
Transparent Application Substitution
(TAS)
is the transparency agent developed for the Spike system to execute a modified version of an application transpar ently, without replacing the original images on disk.
TAS was modeled after the transparency agent in the
DIGITAL
FX! 3 2 system24 but uses different mecha nisms. When the user invokes the original application, the SOE uses
TAS
to load an instrumented or opti mized version. With
TAS,
the user does not need to do anything special to execute tl1e instrumented or opti mized version of an application. The user simply invokes the original application in the usual way (e.g., from a command prompt, from Windows Explorer, or i ndirectly through another application), and the instrumented or optimized application runs in its place.
TAS
performs application substitution in two parts. First,
TAS
makes the ·windows
NT
loader use a modified version of the main i mage and DLLs.
Second,
TAS
makes it appear to the application that the original images were invoked .
TAS
uses debugging capabilities provided by the
Windows
NT
operating system to specify that when ever the main i mage of an application is invoked, the modified version of that i mage should be executed instead . In each image, the table of imported DLLs is altered so that instead of loading the DLLs specified in the original image, each image loads its modified counterparts. Thus, when the user invokes an applica tion, the Windows
NT
operating system loads the modified versions of the images contained in tl1e appli cation. Some applications load DLLs with explicit calls
Digital
Technical Journal Vol . 9 No. 4 1 997 1 7
to LoadLibrary. TAS intercepts those calls and i nstead loads the modified versions.
The second part ofTAS makes the modified version of the application appear to be the origi nal version of the application . Applications often use the name of the main image to rind other tiles. For example, if an instrumented image requests its ful l path n::tme, TAS instead returns the tl1 ll path name of the correspond ing original image . To do this, TAS replaces certain calls to kernel32 .dli in tJ1e instrumented and optimized images witl1 calls to hook procedures. Each hook procedure determines the outcome the call wou ld have had for the original application and returns tlut result.
Instrumentation
Spike instruments an image by inserting counters into it. Using the results of these coun ters, the optimizer can determine the number of ti mes each basic block and con trol tlow edge in the image is executed. Spike uses a spanning-tree technique proposed by Knuth" to reduce the number of counters required to fully instrumen t an image . For example, i n an i fthen-clse clause, coun ti ng the number of ti mes the i f and then statements are executed is enough to determine the number of times the else statement is executed .
Register usage i n formation i s used to find free registers for the instru mentation code, thereby red ucing the number of saves and restores necessary to fi·ee up reg ist
e
rs.
12
Typically, instrumentation makes the code 30 percent larger. As part of the profile, Spike a lso cap tures the last target of a jump or procedure cal l that cannot be determined statically.
Spike's profile i n formation is persistent; small changes to an image do not i nvalidate the profile infor mation collected f(x that image. Profile persistence is essential for applications that requ ire a lengthv or cumbersome p rocess to generate a profile, even when using low-cost methods l ike statistical sampling. For example, generating a good profile of a transaction processing system requires extensive stagi ng of the sys te m . Even when i t is possible to automate the genera tion of profi les, some ISVs find tJ1e extr� build time unacceptable. vVith persistence, the user can col lect a profile once and continue to use it for successive builds of a program as sma l l changes are made to it. Our experience with an rsv h as shown t hat the speedup trom Spike declines as the profile gets older, but using a two- or three-week-old protile is acceptable. It is also possible to merge a profile generated by an older image with a profile generated by a newer image .
·when using an old profi le, Spike must match up procedures in the current program with procedures in the profiled program. Spike discards profiles tor proce dures that have changed . Relying on a proced ure name derived trom debug i n formation to do the matching is not practical in a prod uction environment.
I nstead , Spike generates a signature bJsed on the flow graph of each procedure. Since signatu res are not based on the code , small changes to a procedure will not i nvalidate the profi le. Signatures are not unique, however, so it can be difficult to match two lists of sig natures when there are d i fferences. A minimum edit distance J lgorithm2" is used to fi nd the best match between the list of signatures of the current program and the Jist of signatu res of the p rofiled program.
Summar y
Many Wind ows NT-based appl ications are large, cal l intensive programs, with loops that cross mul tiple pro ced ures and proced ures that h ave complicated control
How and many basic blocks. The Spike opti mi zation system uses code layout and hot-cold opti mization to optimize cal l-i ntensive programs. Code lavout places tbe ti·equen tly executed portions of the program together in memory, thereby reducing instruction cache miss and improving pertonnance up to 33 per cent. Our code layout algorithm rearranges basic blocks so that the fa ll-through path is the common path. The algorith m also splits each procedure into a frequentlv executed ( hot) part and an intreg uentlv executed (cold ) part. The split procedures are pl aced so that tt·equent (cal ler, cal lee) pairs are adjacent.
The hot part of a procedure is the collection of the common paths through the proced ure. These paths can be optimized at the expense of the cold paths by removing instructions tl1at are required only if the cold patJ1s are executed. Hot-cold optimization exploits this opportunity by performing opti mizations that remove partially dead code and replace uses of preserved regis ters with uses of scratch registers. H ot-cold optimiza tion reduces the instruction path length through the call -intensive programs by 3 percent to
8
percent.
Protile feed back is rarely used because of the diffi culty of col lecting, managing, and applying profile information. Spike provides a complete system for profile feed back optimization that eliminates these problems. It is a practical system that is being actively used to optimize applications for Alpha processors running the Windows NT operating system .
Acknowledgments
Tryggve Fossum supported the development of Spike
!Tom tl1e beginning; he also implemented two of our early optimizations. David WaJJ helped us ger started parsing
Windows NT images. lvlike McCallig implemented our first symbol table pJckage. Norman Rubin contributed to the design oft.he tnnsparency agent. Many people helped collect tJ1e data presented in this paper, including Michelle
1 8 Digital Technical Jou nl:ll Vol.
9 l"o 4
1997
Alexander, Brush Brad le y
,
Bob Corrigan, Jeff Donsbach,
Hans Graves, John Henning,
Phil
Hutchinson,
Herb
Lane,
John
Matt
Lapine, Wei Liu, Jeff Seltzer,
Arnaud
Sergent,
Shakshober, and
Robert Zhu .
References l . R . Coh n , D . Goodwi n , P. G . Lowney, and N .
Ru b i n , "Spike: A n Optimizer for Alp ha/NT E x e c ur a b les,"
The USEN!X Windows NT \
'(l orksbop Pro ceedin
.
� s
.
Seattle, Wash . ( August 1 9 9 7 ) : 1 7-24.
2. A. Srivastava and D. Wal l , " Li n k-time Opti m i zation of Add ress Calcu l ation on a 64-bit Archi te<.:rure," t
> ro ceedings of the A CM SJGPLA N'94 Conji:!n.mce on Pro
.
f!.
r o m m ing Lanp,uage Design and lmplemen tatiou
,
Orlando, Fla. ( J u ne 1 994): 49-60 .
3 . L . Wi lson, C . Neth, a n d !'vi . Rickab:wgh, " De l iver i ng B i nary O bject Mod i fication Tools tor Program
Ana lvsis :llld Opti m i zation,"
Dip,ital Techn ical jOit r
; wl
, vol . 8, no. 1 ( 1 996 ): 1 8-3 1 .
4. S . McFarl i ng, " Program Optim ization for Instruc tion Caches,"
ASP LOS
1!1
Proceedinp,s
,
Boston ,
Mass. ( April 1 9 8 9 ) : 1 83-1 9 3 .
5 .
VI/.
Hwu a n d P. Chang, "Ach ieving H igh I nstruc tion Cache Performance with an Optimizing Com p i ler,"
Proccedinp, s of the Sixteenth A n 11 ual
International Symposiu m on Computer A rch itec ture. Jerusalem,· Israel ( J une 1 98 9 ) .
6 . K .
Pettis a n d R . Hansen, " Profile Guided Code
Positioning,"
Pro ceedings of the A CIH SJGPLAJ\''90
C u nj i:!r
ence
011
Prog r am m ing Ltmp,uage /Jesign and !mplemental'ion,
White Plains, N . Y . ( J u ne
1 99 0 ) : 1 6-27 .
7 .
R .
Coh n a n d P. G . Lowney, " Hot Cold Optimiza tion of Large Windows/NT Applications,"
MJCN0-
29,
Paris, France ( Decem be r 1 99 6 ) : 80-89 .
8 . I n formation about the S PEC benchmarks is ava i l
:� b l c from the Standard Performance Eva l u ation
Corporation at h ttp:/ / www.
specbench.org/.
9 . D . Blickstei n et a l . , "The G EM Optimizing Com pilcr Svstcm,"
D
( t:i t al Technical.fuunwl, vol . 4, no. 4
( 1 99 2 ) : 1 2 1 - 1 36.
1 0 . 13 . Calder, D. Grunwal d , and A . Srivastava, "The
Prcdictabilitv of B ranches i n Libr:�rics," Procee d inMs of the
'
Twenty
eighth A n n ual International s:vrnposium on Nhcroarchitectu re , A n n A rbor,
Mich. ( November 1 99 5 ) : 24-34.
1 1 . A. Aho, R . Sethi, and J. Ullman,
Compilers
.
Princi ples. Techniques, and Tools (
R cading
, Mass.: Addison
Wesley, 1 98 5 ) .
1 2 . D . Goodwin, " l n tcrproccdural D:naflow Analysis in
:1 11
Execu ta b le Optimi zer," Proceedings of the
A C\1 SIGPLA t\' '9 7 Conference on Pro,t: ra m rn ing l an g uap,e Design and I mpl e men t ation
.
Las
Vegas,
Nev. ( J u ne 1 99 7 ) : 1 2 2-1 3 3 .
1 3 . Alpha 2 1 1 64 Microprocessor Hardware Reference
Manual, O rde r No. EC-QAEQB -TE ( Maynard ,
Mass . : D i gital Eq u i pment Corporation, April
1 995 ) .
1 4 .
J .
Fisher, "Trace Sched u l ing: A Techn ique for
Global tvl icrocode Com paction ," IEEE Tran s ac tions on Computers, C-30, 7 ( J u ly 1 9 8 1 ) : 478-490.
1 5 .
DECchip 2 1 064 and DI:'Ccb ip 21 064A A lpha A XP
JI;Jicroprocessors Hardware I<eferen ce Manual,
Order No. EC-Q9ZUA-TE ( Maynard , Mass . : Dig ital Equ ipment Corporation, J une 1 994 ) .
1 6 .
] .
Knoop,
0 .
Ri.ithing, and B . Steften, " Partial Dead
Code Elimi nation," Pmceedin
,
�s of the A CM SIG
PLA J\. 94 Conference on Prog ra m m ing La n guage
Desig n and implementation,
1 994 ) : 1 47-1 5 8 .
Orlando, Fla. ( J une
1 7 . P. Chang, S . M a h l k e , a n d W . H w u , " U sing Profile
I n formation to Assist Classic Code Opti m izations ,"
Sojiware-Praclice and Experience
. vol . 2 1 , no.
12 ( 1 9 9 1 ) : 1 3 0 1 - 1 32 1 .
1 8 . P. G . Lowney et ell . , "The Mul tiflow Trace Sched u l ing Com p i ler,"
The jou rnal of Supercomputing, vol. 7, no. 1 /2 ( 1 99 3 ) : 5 1- 1 4 2 .
1 9 . A . Srivastava a n d A . Eustace, "ATO M :
A
System for
B u i l d i n g Custo m i zed Program Analysis Tools,"
Pro c e edin u s uf t h e A CM SJGPLA N '94 Conference on Programm ing Lwlgi.W!!,e nes1���·1 and
I mpl e me n tation. Orlando, Fla. ( J unc 1 994 ): 1 96-20 5 .
2 0 .
UJVIJPS-
1/
Referellce Jlifa nual (pixie a n d pixs t ats)
( S u n nyva l e , Calif. : M I PS Computer Systems,
1 99 0 ) .
2 1 .
J .
Anderson er a l . , " Continuous Profi l ing: W here
Have Al l the Cvcles Gone>" Proceedings of the Six t een t h A 011 Sy;nposium on Operatinp, System Prin c iples , Saint- Malo, France ( October 1 99 7 ) : 1 - 1 4 .
2 2 . X . Zhang et a l . , "System Su pport for A u tomatic
Profi l i n g and Opti m i zation,"
P rocee d inp,s of the
Sixteenth A CtH -�vmposium on Operating Systern
Pr i l l c ip les
, Saint-Malo, F rance ( October 1 99 7 ) :
1 5-2 6 .
2 3 . S . Feldman, "Make-A Program tor Maintaining
Compu ter Programs,"
Sojiware
-
P ra ctic e and
Experience, vol . 9 , no. 4 ( 1 979 ) : 2 5 5-2 6 5 .
24. R . H ookway and M . Hcrdeg, " DIG ITAL F X 1 3 2 :
Com bining Emu lation and Binary Translation,"
Digital Tech n ical .fourna!,
3-1 2 . vol . 9 , no. l
( 1 997 ) :
2 5 .
D . Kn uth, Tbe A rt of' Computer Prog r a mm ing Vol.
7, Fu rtdamenta! A igorith ms ( Read ing, Mass . : Add i son-Wesley, 1 9 73 ) .
2 6 .
W . M i l l er and E . Mevcrs, "A F i l e Comparison Pro
"ram
'
" sort u • areP;-ctcti c e and J::'.xpen·ence, vol . 1 1
( 1 985 ) : 1 0 2 5 - 1 040.
Digital Technical joumJI Vol . 9
No. 4 1 997
19
Biographies
Robert
S.
Cohn
Robert Cohn is a consu l ting engineer in the VSSAD
Group, where he works on adl'anced compiler technology for Alpha microprocessors. Since joining D I GITAL in
1992, Robert has implemented profile-feedback and trJCe scheduling in the GEM compi ler. He aJso implemented the code layout optimizations in U NIX OM. Robert has been a key contributor to Spike, implementing both hot-cold optimization and the code layout optimizations . Robert received a B .A. from Cornell University and a Ph.D. in computer science from Carnegie Mellon University.
P. Geoffrey Lowney
P. Geoffrey Lowney is a senior consulting engineer in d1e
VSSAD Group, where he works on compilers and architec ture to i mprove the performance of Alpha microprocessors.
Geoff is the leader of the Spike project. For Spike, he im plemented the infrastructure for parsing cxecutables.
Prior to joi ning DIGITAL in 199 1 , Geoff worked at
Hewlett Packard/ Apollo, Multiflow Computer, and New
York University. Geoff received a B.A. in mathematics and a Ph.D. in computer science, both from Yale University.
David
W.
Goodwin
David W. Goodwin is a principal engineer in d1e VSSAD
Group, where he works on architecture and compiler advanced development. Since joining DIGITAL in 1996, he has contributed to the performance anaJysis of the 2 1 1 64,
2 1 1 64PC, and 2 1 264 microprocessors. For the Spike pro ject, David implemented the Spike Optimization Environ ment and the interprocedural dataflow anaJysis. David received a B .S.E.E. from Virginia Tech . and a Ph.D. in computer science from the Universitv of California, Da1·is.
20 Digital Technical Journal Vol.
9
N o . 4
! 997
I
Susanne M. Balle
Simon C. Steely, Jr.
Analyzing Memory
Access Patterns of
Programs on Alpha based Architectures
The development of efficient algorithms on today's high-performance computers is far from straightforward. Applications need to take full advantage of the computer system's deep mem ory hierarchy, and this implies that the user must know exactly how his or her implementa tion is executed. The ability to understand or predict the execution path without looking at the machine code can be very difficult with today's compilers. By using the outputs from an experimental memory access profiling tool. the programmer can compare memory access patterns of different algorithms and gain insight into the algorithm's behavior, e.g., potential bottlenecks resulting from memory accesses.
The use of this tool has helped improve the performance of an application based on sparse matrix-vector multiplications.
The development of efficient algorithms on today's high-performance computers can be a challenge. One major issue in implementing high-performing algo rithms is to take full advantage of the deep memory hierarchy. To better understand a program's perfor mance, two things need to be considered: computa tional intensiveness and the amount of memory traffic involved. In addition to the latter, the pattern of the memory references is important because the success of hierarchy is attributed to locality of reference and reuse of data in the user's program .
In this paper, we investigate the memory access pat tern of Fortran programs. We begin by presenting an experimental Atom' tool that analyzes how the pro gram is executed. We developed the tool to help us understand how different compiler s>vitches impact the algorithm implemented and to determine if the algorithm is doing what it is intended to do. In addi tion, our tool helps the process of translating an algo rithm into an efficient implementation on a specific machine. The work presented in this paper focuses primarily on a better understanding of the behavior of technical applications. Related work for Basic
Linear Algebra Subroutine implementations has been described.2 In most scientific programs, the data ele ments are matrix-elements that are usua!Jy stored in two dimensional (2-D ) arrays (column-major in Fortran) .
Knmving the order o f array referencing i s important in determining the amOLmt of memory traffic.
I n the fi nal section of this paper, we present an example of a memory access pattern study and i llus trate how the use of our program analysis tool improved the considered algorithm's performance.
Guidelines on how to use the tool are given as wel l as comments about conclusions to be derived from the histograms generated.
Memory Access Profiling Tool
Our experimental tool generates a set of histograms for each reference in the program or in the subroutine under i nvestigation. The first histogram measures
Digital Technical
)ournaJ
Vol. 9 No. 4 1 997
2 1
strides ti·om the previous reference, the second his togram gives the stride ti·om the second -to-last rder ence, and so on, f(x a total of MAXEL h istograms for each memory reference in the part of the program we investigate . B y stride, we mean the distance bct\vecn two memory reterences ( load or store ) . We chose
a
J\1AXEL of five for our case study, but MAXEL
c a n
be given any value.
Two variants of this tool were implemented .
I.
The first version takes all memorv references into account in al l histograms.
2. The second version takes into account i n the next h istogram those memory references whose stride is more than 1 28 bytes. It does not consider in the
(
i
+ l
) th histogram
( i
=
1 , . . . ,5 ) strides that are less than 1 28 bytes in the ith histogram .
The second version o f the tool bas proven to be more usefu l in understanding the access patterns. It highlights memory accesses that are stride one for a while and then have a stride greater than 1 28 bytes.
The
choice of 1 2 8 bytes was arbitrary; the value can be changed.
The fol lowing bins are used in the b istogrJms: 0through 1 27-byte strides Jre accounted tor separately.
Strides greJter than or equal to 1 28 bytes arc grouped into the fol lowing intervals: [ 1 28 through 2 5 5 ] , [ 2 56 through 5 1 1 ] , [ 5 1 2 through 1 ,0 2 3 ] , [ 1 ,024 through
2 ,047 ] , [ 2,048 through 4,095 ], [ 4,096 through
8 , 1 9 1 ] , [ 8 , 1 92 through 1 6,383 ] , [ 1 6,384 through
32,767 ] , and [ 3 2 ,768 through infinity ] .
I n the n e x t section, w e present the output his tograms obtained with the second version of this experimental tool ror a Fortran loop. In our
ClSC studv, we chose to perform the histograms on a single arrav instead of all rdercnccs in the program . This method provided a clearer picture of the memory access pat tern for each array in the piece of the program under consideration. We present separate h istograms tor the
.loads and the stores of each array in the memory traffic of the subroutine we investigated .
\Vhen looking at memory access patterns, it is important not to include load instructions that per form prdetching. Even though prefetching adds to the memory traffic, its load i nstructions pollute the memory access pattern picture.
Case Study
In this section, we study and compare d i fferent ver sions of the code presented in Figure
l
usi ng our
experimental
memory access profili ng tool . We show that the same code is not executed i n the same way for different compiler switches. Often a developer has to delve deeply into the assembler of the given loop to u nderstand how and when the different i nstructions
33
34
35
36
37
29
30
3l
32
20
21
22
23
24
25
16
1 7
1 8
19
26
27
28
1 1
12
13
14
15
5
6
7
Q ( i ) =O, i =1 , n
2 do k 1 = 1 , 4
3
4 i nd e x = ( k 1 -1 ) * numrows do j = 1 , n p 1 = C O L S T R ( j , k 1 ) p 2 = C O L S T R ( j +1 , k 1 ) - 1 p3= [ s n i p ]
8 s umO=O . dO
9 ill s um 1 =0 . d 0 sum2=0 . d0 s um3=0 . d0 x1 P ( i ndex+ROW I D X ( p 1 , k 1 ) ) x2 x3 x4
P ( i ndex+ROW I D X ( p 1 +1 , k 1 ) )
P ( i n d e x + R OW I D X ( p1 +2 , k 1 ) )
P ( i nd e x+ROWI D X ( p1 +3 , k 1 ) ) d o k = p1 , p 3 , 4 s umO s u m 1 sum2 sum3 x 1 x 2 x 3 x 4 enddo s umO + AA ( k , k1 ) * x 1 s u m 1 + AA ( k+1 , k1 )
* x 2 sum2 + AA ( k + 2 , k 1 ) * x 3 sum3 + A A ( k+3, k 1 ) * x4
P ( i n d e x +ROW I D X ( k+4 , k 1 ) )
P ( i n d e x+ROWI D X ( k+ S , k 1 ) )
P ( i nd e x+ROW I D X ( k+6 , k 1 ) )
P ( i ndex+ROWI D X ( k+7, k 1 ) ) do k = p3+ 1 , p2 x 1 =P ( i ndex+ROWI D X ( k , k 1 ) ) s umO = sumO + AA ( k , k 1 ) * x 1 enddo
38
Y T E M P ( j , k 1 ) =s umO+s um1 + s um2+sum3 enddo do i
= enddo enddo
1 , n , 4
Q ( i )
=
Q ( i +1 )
Q ( i +2 )
Q ( i +3 )
Q ( i ) + Y T E M P ( i , k 1 )
Q ( i +1 ) + Y T E M P ( i + 1 , k 1 )
Q ( i +2 ) + Y T E MP ( i +2 , k 1 )
Q ( i +3 ) + Y T E M P ( i +3 , k 1 ) w h e r e n = 1 4 0 0 0 , r e a l * 8 AA ( 5 1 1 3 5 0 , 4 ) , Y T E MP ( n , 4 ) r e a l *8 Q ( n ) , P ( n ) i n t e g e r * 4 ROW I D X ( 5 1 1 3 5 0 , 4 ) , C O LS T R ( n , 4 )
Figure
1
Origi nJI Loo�)
are executed . The output h istograms from our tool ease that process and give a clear picture of the refer ence patterns. The loop presented in figure 1 imple ments a sparse matrix-vector multiplication and is part of a larger application. Ninery-six percent of tbe appli cation's execution t i m e i s spent in that loop. 'vVe ana lyze the loop compiled with t\vo d i fferent sets of compiler switches. To i llustrate the dkcrive use of the tool, we present the enhanced performance results due to ch�mges made based on the output histograms.
From lines 5 and 6 i n the loop shown in Figure l , we wou l d expect the array
COLSTR to be read stride one 1 00 percent of the time. Line 30 of the figure ind icates that lTEMP is accessed stride one through the wholc j loop . From l ines 33 through 36, we expect
YTEMP's stride to be equal to one most of the time and equal to the n u m ber of colu m ns in the array every time k1 is i ncremented. Q should be referenced 1 0 0
2 2
Dig;iLll TcchnicJI Jound Vol . 9 )Jo. 4
1 997
percent stride one tor both the loads and the stores
( l ines
33
through
36) .
As i l lustrated in lines 1 2 through 1 5, 2 1 through 24, and 2 7, RO WIDX is expected to be accessed with a stride of one between the p1 and p2 bounds of the k loop. Even though i t looks like the k loop is violating the array bounds of
ROWIDX in lines 2 1 through 24 for the last iteration of the loop, this is not the case. We expect array Pto have nonadjacent memory references since we have deliber ately chosen an algorithm that sacrifices this array's access patterns to improve the memory references of
Q and AA.
Original Code
We investigate the memory access patterns achieved by the loop in Figure 1 when compiled with the fol lowing switches: f 7 7 - g 3 - f a s t - 0 5
The - g 3 switch i s needed to extract the addresses of the arrays from the symbol table. For more infor mation on DIG ITAL Fortran compiler switches, see
Reference
3 .
From Figures 2 and
3 ,
we see that array Q is accessed as we expected , 1 00 percent stride one for the loads and the stores. Since Q is accessed contiguously in 100 percent of its m emory references, we will not have any entries in the next four histograms. As described i n
100 the previous section, we only record i n the next his togram the strides that are greater than 128 bytes in the current histogram .
Figure 4 illustrates that COLSTR i s accessed 50 percent stride zero and 50 percent stride one . This is unexpected since lines 5 and
6
in Figure 1 suggest that this array would be accessed stride one 100 percent of the ti me. The fact that we have entries only for the strides between the current and the previous loads indicates that the elements of COL57R are accessed in a nearly contiguous way. A closer look at Figure l tells us that the compiler is loading COLSTR twice. We expected the compiler to do only one load into a regis ter and reuse the register. The work-around is to per form a scalar replacement as described by Blickstein et al.• We put p2
=
COLSTR( l ,kl ) -1 outside the j loop and substituted inside the j loop p1 with p1
= p2
+
1 . Inside the
=
COLSTR(j, k1) j loop, p2 remains the same. Eliminating the extra loads did not enhance per formance, and a possible assumption is that the analy sis done by the compiler concluded that no gain would resu lt from that optimization .
Figures 5 and
6
show the strides for the loads and the strides for the stores for the array Y"IFJvJP. One more time, the implementation is not being executed the way we thought it would. In Figure 1 , Jines
33
through
36
suggested that ITEAIP would be referenced stride one through the whole i loop as well as with a stride i="
()
a: w
8:.. 60
Ul
0
<>:: s
LL
0 ftl
50
40
30
20
10
0
0
.,.
00
KEY:
0
1 STEPAGO
0
2
D
STEPS AGO
3 STEPS AGO
� �
.,.
00
(\J
("') ("') .,.
.,.
..,
STRIDES IN BYTES
0
,...
.,.
Lf)
Lf)
(\J
I
00
�
;;;
I
Lf)
N
4 STEPS AGO
•
5
STEPS AGO
("')
0.
I
(\J
;;;
,...
"'
I
�
Lf)
"'
.,.
I
-.:: a; a5
I
-.::
("')
("')
I
-.::
,...
"'
I
-.::
�
z
u::
�
I
-.::
C')
Figure 2
Strides for Array Q between the Current Load and the Load One through Five Steps Ago
Digital Technical Journal
Vol . 9 No. 4 1 997 23
50
_j
LL
0 a:
0 w e:_
(f)
0
<{
0
40
30
20
1 0
0 w
70 e:_
60
Cf) w a:
0
50
LL
0
40 a: m
2
::::> z
30
20
1 0
1 00
90
80
KEY:
D
1
0
0
,. co �
�
STEP AGO
D
2 STEPS AGO
•
3 STEPS AGO
,. co
N
N
(') ..,.
,. co N
LO LO
0
CJ)
STRIDES IN BYTES
,.
CJ)
N
I ro
�
;:: (')
LO
I
CJ)
LO
N
�
N
� o_
N
I
�
LO o_
,.
I
�
N a; ro
(')
(')
I
,.
I
� co
�
� r--
CJ) r--
N.
(')
I
�
LL
:;:;
I
(")
•
4 STEPS AGO
•
5 STEPS AGO
Figure 3
Strides tor
ArrJy Q between the Cu rrcnr Store Jnd the Srore One through hve Steps Ago
60
0
0
,. co N CJ) 0
N
"'
N co
N
(") (") ,.
,.
,.
N
Lll r--
N
LO
Lll
N
I co
�
;::
LO
I ill
Lll
N
(")
N o_
N
�
0
N.
I
�
LO
0) o_
"'
I
�
N a; ro
I
..,.
(') co
(")_
�
I
�
1'-
CD r--_
(")
I
�
�
�
:;:;
I
� KEY:
D
1 STEP AGO
D
2 STEPS AGO
D
3 STEPS AGO
•
4 STEPS AGO
•
5 STEPS AGO
STR I D ES IN BYTES
Fig ure 4
Strides for
Array
COLSIR between the Cu rrent Load and the
Load
One through five
Steps
Ago
24 llif'irol Technical Journal Vol . 9 No. 4 1 997
1 00
90
80 f='
_J
LL
0
0:
UJ
CD rJl
0
�
70
(.)
0:
UJ e:.
60
50
40
30
20
1 0
0
0
" ro N (!)
KEY:
D
1 STEPAGO
D
2
STEPS AGO
3
STEPS AGO
•
4 STEPS AGO
•
5 STEPS AGO
(") (") "
" ro N
L[)
(!)
L[)
0
(!)
1'-
N
L[)
L[)
N
"
(!)
I ro
�
STRIDES IN BYTES u:;
I
(!)
L[)
N
(")
-
N u:;
0
N-
L[)
0
:.:: :.::
N a;
<:15
:.::
(")
(")
I
1'-
(!)
1'-
I
:.:: :.::
�
(") u::
�
I
� :.::
(")
Figure 5
Strides for Array Y7EMPberween the Current Load and the Load One through Five Steps Ago
1 00
90
80 f=' w
70
UJ e:.
60
0
(f)
0 a:
50
40
:::;
:::> z
30
20
1 0
0
0
" ro � � 0
N
"
N ro
N
(") "
" ro N
(!)
L[)
0 1'-
L[)
L[)
N
I ro
�
STRIDES
IN
BYTES u:;
I
(!)
L[)
N
(")
-
N u:; o_
N
I
L[) o_ a;
(")
(")
1'-
1'-
"
<:15
"' C\1
:.:: :.::
N
" :.:: ro
�
(") z u::
:.::
(!)
�
I
:.::
KEY:
0
1 STEP AGO
D
2 STEPS AGO
3 STEPS AGO
•
4 STEPS AGO
•
5 STEPS AGO
Figure
6
Strides for Array YTEMP between the Current Store and the Store One through Five Steps Ago
Digital Technical Journal
Vol . 9
No. 4
1997 25
equal to the n umber of columns in the array when k 1 is i ncremen ted . Bv considering Figure
5 along with lines
33 through 36 in Figure 1 , we conclude tlut YTEMP is u nrol led by four in the kl -direcrion in t h e i loop. The fact that all strides between the current load and the load two loads back or three loads back or tou r loads back have a stride between 32K and infinity is consis tent with traversing a matrix along rows. Figure s hows that the
6 j loop is not unrolled by four i n the kl dir e ction, because a.ll the loads of >77:/11/P are 100 percent stride one. The compiler must split the k<l loop into two separate loops, the first consists of the j loop and the second consists of the i loop. The latter has been unrolled bv four in the /;>} -direction thcrebv el im inating the extra O\'erhead ti-om d1e /�I loop.
Figure 7 shows that the matrix A./1 is accessed a s we expected. The strides arc not greater than 1 28 lwtes or, in other words, a maximum stride of 1 6 eJements.
The tact that there i s n o stride other than the one between the current load and the previous load in the histograms shows that AA is referenced i n a controlled way. In this case, AA is accessed 39 percen t of its total loads in stride one and
23 percent in stride two.
From l i nes through
1 2 through
1 5 , 1 7 through
20, and 2 1
24 i n Figure l , we know that the arrays AA
;md RO WIDX should have relativelv similar behaviors .
O n l y t h e tour extra prefctches of RO IX7DX in l i nes 2 1 through 2 4 for the last iteration i n the j l oop differen tiate the access patterns of the two arrays. Figure 8 confirms that assumption . RO WID.X:. is referenced with controlled strides. Because RO 'W7DX is accessed cJose to contiguously, we wil l not have any entries in the next tour h istograms. As described in t.l1e previous sec tion, we only record in the next histogram the strides that are greater than 1 28 bytes in the current his togram . NO \Vff)X. is referenced 24 percent of its total loads in stride one and 34 percent in stride two.
As illustrated in Figure 9, array P is accessed exactly the way we expected it. VVhen desigrung this algorithm, we had to make some compromises. We decided to have AA
<UKi
Q retCrenced as closeJy as possible to stride one, thus giving up the control of P's references.
Bv examining tJ1csc arrays' access patterns, we can sec how they arc accessed and ll'bethcr or not the imple mentation is doing what it is supposed to do. i n Figure l is used on a larger matrix
[
11
=
I f'thc loop
75,000 and rl.A( 204427, 1 2 ) has
1 5
million nonzero elements], tJ1c execution time for the total appl ication on a single
2 1 1 64 processor of an AlphaServer 8400 5/625 system is
(
=
l
,970 seconds. The application executes 26 x 75
1 ,95 0 ) times the considered loop. When profiling the program , we measured that the loop under i nvesti gation takes 96 percent of the total execution time. It is tl1erdore a tair assumption to sav tJ1at any improvement i n t.l1is building block will imprm·e the O\'eral l perfor mance oftbe total program.
50
45
40
Ul
0
<(
0
...J w
(.)
0: w
� lL
0
0:
LU co
::;;
=> z
35
30
25
20
1 5
1 0
5
0
0 '<t
KEY:
0
1 STEP AGO
D
2 STEPS AGO
3 STEPS AGO co C\J <D
"'
C\J co
C\J
(') (")
"' '<t
.�
STR I D E S IN BYTES
0
<D r--
C\J
<0
C\J
I co
�
"'
I
<D
C\J
(") r--
"'
�
'<t
0
N '<t
C\J
;[)
I
�
I
�
;;; ro
I
�
(") r-co
"'-
<D r--_
�
I
� co
�
(')
� lL
I
�
�
�
I
�
(')
•
4 STEPS AGO
•
5 STEPS AGO
Figure 7
Strides t<>r Arral' AA between rhe Current J
.md
�nct rhc I .oact One rhrougil Fil'c Steps Ago
26 DigitJI Tcch nic.1l Jou rnal
Vol. 9
No. 4 1 997
40
35
!='
30 u w
�
(f)
0
<{
25
20
_j lL
0 a:
1 5
:::;;
::) z
1 0
5
0
0 ""
.�
STRIDES IN BYTES
I
""
<D
N
I
00
S' l[)
I
�
N
�
,._
N
I
� a;
""
I
N
I
�
I
00
C')
I
�
�
�
u:::
�
I
�
KEY:
0
1 STEP AGO
0
2 STEPS AGO
0
3 STEPS AGO
•
4 STEPS AGO
•
5 STEPS AGO
Figure 8
Strides for Array ROW7DX between the Current Load and the Load One through Five Steps Ago
50
45
40
_j lL
0 a: co
:::;;
::) z
!=' w u w
35
� 30
(f)
0
<{
25
20
1 5
1 0
5
0
0 "" 00
S' �
""
N
<D 0 ""
""
STRIDES IN BYTES
JLrfk.f
�
I
""
<D
N
I
00
S'
�
I
<D
�
N
�
,._ o_
N
I
�
0
""'
I
N a;
00
I
�
IIlf
�
�
<!) N" �
C') lL
I
00
I
<D
�
I
�
KEY:
0
1 STEP AGO
0
2 STEPS AGO
0
3 STEPS AGO
4 STEPS AGO
•
5 STEPS AGO
Figure 9
Strides for Array ? between the Current Load and the Load One through Five Steps Ago
Digital Technical Journal Vol .
9
No.4
1 997 27
Modified Code
I n this section, we describe a new experiment in which we used different compiler switches and changed the original loop to the loop in Figure 10. The code changes were based on the analysis
in
the previous section as well as on a more extended seties of investigations.
In
this example, we used the tol lowing compiler switches:
f 7 7 - g 3 - f a s t - 0 5 - u n r o l l 1
Lines 3, 5, and 6 fi.·om Figure 1 0 show that we imple mented tl1e scalar replacement technique as described by
Blickstein et al .' to avoid COL57R being loaded twice. From
Figure 1 1 , we sec that array COLS7R is now behaving as we expect: 100 percent of tl1e sui des for tl1e loads are stride one.
In our first attempt to optimize the original loop, we split the kl loop into r-vo loops in the s:�me way the com piler did as described
in
tl1e previous section. We then hand unrolled the
'rTEMP
array in the
kl
direction. Further analysis showed that a considerable gain cou ld be made by removing the
Y"fEMP
array and writing the results directly into Q. By replacing tl1e zeroing out of the Q array
4
5
6
7
8
2
3
9
10
I I
1 2 do k 1 = 1 , 4 i nd e x = C k 1 -1 ) * n u m r o w s p 2 = C O L S T R ( 1 , k 1 ) - 1 do j = 1 , n p 1 = p 2 + 1 p 2 = C O L S T R ( j +1 , k 1 ) -1 p3= [ s n i p ] s umO=O . dO sum1 =0 . d0 sum2=0 . d0 s um3=0 . d0 x 1 = P ( i nd e x + R O W I D X ( p 1 , k 1 ) )
13
14
15
16
17
1 8 x 2
=
P ( i nd e x + R OW I DX ( p 1 +1 , k 1 ) ) x 3 = P ( i nd e x+ROW I D X ( p 1 +2 , k 1 ) ) x 4 = P ( i nd e x + R OW I DX ( p 1 +3 , k 1 ) ) d o k = p 1 , p 3 , 4 sumO
= s umO + A A ( k , k 1 ) * x 1 sum1
= sum1 + A A ( k + 1 , k 1 ) * x 2
1 9
20
2 1
22
23
24
L9
30
31
32
25
26
27
28 sum2 = sum2 + A A ( k + 2 , k 1 ) * x 3 s um3
= s um3 + A A ( k + 3 , k 1 ) * x 4 x 1
=
P ( i n d e x + R O W I D X ( k+4, k 1 ) ) x 2
=
P ( i n d e x + R O W I D X ( k+ 5 , k 1 ) ) x 3 = P C i n d e x + R OW I D X ( k+6, k 1 ) ) x 4
=
P ( i n d e x + R OW I D X ( k + 7 , k 1 ) ) enddo d o k
= p3+ 1 , p2 x 1 = P ( i n d e x+ROW I D X ( k , k1 ) ) s umO = - sumO +
AA( k , k 1 ) * x 1 enddo i f < k1 . eq . 1 ) t h e n
Q ( j ) = s umO + s u m 1 + s um2 + s um3 e l s e
33
34
Q ( j ) e nd i f
35 enddo
36 enddo
=
Q ( j ) + s umO + s u m 1 + s u m 2 + s um3 w h e r e n = 1 40 0 0 , r e a l * 8 AA ( 5 1 1 3 5 0 , 4 ) r e a l * 8 Q ( n ) , P ( n ) i n t e g e r * 4 ROW I D X ( 5 1 1 3 5 0 , 4 ) , C O L S T R < n , 4 )
Figure 1 0
Mod i ticd Loop
( Figure l , line
l )
with an
JF
statement ( Figure 10, line 30), we further improved the performance
of
tl1e loop. The last
t\\'O
changes were possible because we decided that, for pertonnancc enhancement issues, the serial version of the code was going to be different from its parallel version .
Figures 1 2 and 1 3 show that Q's load and store access pattern is 1 00 percent stride one :ts we expected it to be.
For both RO U:I7DXand
AA,
we sec a significant increase in stride one references. Figure 1 4 shows tllat
AA
is now accessed 69 percent stride one instead of 39 percent.
RO WIDX's stride one increased to 52 percen t from 24 percent as il lustrated in Figure 1 5 . These r-vo arrays are the reason tor using the
- u n r o l l 1
switch . vVithout it, stride one for both arrays would stay approximately the same as in the previous study. The pattern of accesses of array P in Figure 16 is similar ro the prior pattern of accesses i n Figure 9 as expected.
To better understand the effects of the umoLiing, we counted the number of second-level cache misses for 26 calls to the loop, using
:: m
Atom
tooP
tlut simulated a
4-megabyte direct-mapped cache. By considering only tl1cse
26 mau-Lx-vector multiplications, we do not get a full picture
of
what is going on and how the different arrays interact.
Nevertheless, it gives us hints about what caused the improvement in performance. Usc of the cache tool on the whole application would increase the run time dramatical ly.
Twentv-six calls to the original loop ( Figure 1 ) have a total of l ,476,0 1 7,322 memory references, of ,,·hich
77,638,624 are cache m isses. The modified loop ( Figure
10), on the other hand, has fewer references due to the fact that we eliminated
:111
expensive array initialization at each step and removed tJlC temporary array
YT&WP.
The number of cache misses dropped from 77,638,624 to 72,384,348 or a reduction in misses of 7 percent. If we compi le the modified loop witl1out the
-u n r o l l 1
switch, the number of cache misses incre:�scs slightly. On tl1e 2 1 1 64 Alpha microprocessor, all the misses are cflectively performed in serial . This means that for memory bound codes like tl1e loop we arc currcntlv investigating, execution time primar i.lv depends on tllC number of cache misses.
The histograms illustr::�ting tl1c access strides for the d i f� ferent arm·s helped us design a more suitable algoritl1m for our architecture. By increasing the su·ide one references in the loads for the arrays
AA
and RO WJDX, eliminating the extra references in C0!57Nand Q, and improving the strides for Q, we increased tl1e performance of this application d ra matically. Counting the number of cache m isses gave us a better understanding as to why the new access patterns achieve enhanced performance. It also helped us under stand that not allowing the compiler to unroll the already hand-unrolled loops in the modified loop decreased the number of cache misses. The execution time for this appli cation
I n =
75,000 and AA( 204427, 1 2 ) has 15 million nonzero clements] decreased !Tom
l
,970 seconds to
l
,8 3 1 seconds o n a single 625-megahertz ( MHz) 2 1 1 64 Alpha microprocessor of an AlphaScrver 8400 5/625 system .
This i s a n improvcmcm
o f
1 39 seconds or 8 percent.
28
Digiral 'll:chnicJI journal
Vol . lJ No. 4
1 ')97
1 00
90
80 u.J
Ill
::<
:::> z
I= z u.J
(.) a: u.J
..J
LL.
0
70 e:.
60 rJ)
0
<{
50
40
30
20
1 0
0
0 "' <Xl "' <£>
0
"'
KEY:
D
1 STEP AGO
D
2 STEPS AGO
3 STEPS AGO
•
4 STEPS AGO
•
5 STEPS AGO
"'
CD
"' "'
<Xl
"' l() l() CD C\J
I
"' l()
<Xl
� l()
<£>
STRIDES IN BYTES o_
�
;n
"'
N"
I
�
Ol
;:;;
�-
.,<Xl
I
�
I
�
"'
M
<D
I
�
1'-
N"
M
I
�
�
� z u::
�
I
�
M
Figure 1 1
Strides for Array
COLSTR between the Current Load and the Load One through Five Steps Ago
1 00
90
80
I= z
(.)
70 u.J e:.
60 rJ)
0
<{
50
LL.
0 a: u.J
Ill
::<
:::> z
40
30
20
1 0
KEY:
D
1
STEP AGO
D
2 STEPS AGO
3 STEPS AGO
•
4 STEPS AGO
•
5 STEPS AGO
STRIDES IN BYTES l()
(\J
<Xl
� l()
I u:;
"' cJ
I
�
Ol
"'
I
� c;; a5 <£>-
1'-
"'z
I
"'
I
�
M
I
�
� u::
�
I
�
M
Figure 1 2
Strides for
Array
Q between the Current
Load and the
Load
One through Five Steps Ago
Digital Technical Journal Vol . 9 No. 4
1997 29
100
90 f='
rr:
80
70
� 60
f- 50
(fJ
0
40
30
20
10
0
0 " ro
�
(!) 0
N
KEY:
D
1
D 2
D
3
STEP AGO
STEPS AGO
STEPS AGO
•
4
•
5 ro
N
N
(')
(!)
(')
0
" ro
"
N l()
(!) l()
0
(!)
,__
N
STRIDES IN BYTES
l()
N
I ro
�
:;:;
I l()
N
(')
N o_
N
:;:;
,__
" o_
N
I
;,:: l()
"'
".
I
(J) a5
I
(') ro
(")_
(!)
I
;,::
,__
,__
N.
(')
I
>f-
� lL
�
I
N
(')
Figure
1 3
Strides for Ar r;n · Q bet\\'een the Currcnr Srore and the Store One through fi1·c S tep s Ago
70
60
0
_j lL
rr:
w w
(fJ
50
40
30
20
10
0
0
" ro N (!) ro
N
N
(V)
KEY
D
0
2
3
STEPS AGO
STEPS AGO
•
•
4 STEPS AGO
5 STEPS AGO
0
" ro
"
N l()
(!) l()
0
(!)
,__
N
STRIDES IN BYTES
l() l()
N
I
(')
:;:;
I
(!) l()
N
N
:;:;
,.__
<:1'
0 cJ
I
;,:: l()
"'
0
<:1'
I o; a5
I ro
(')
,__
(!)
,__
<D N
I I
;,:: ;,::
� z u:::
�
I
;,::
30
Fig ure 1 4
Strides tor Array AA between the Cu rrcnr Load and rhe
Load One through fi1-e Steps .\go
Digital Technical journ�l
Vol. 9 No. 4 1 997
60
50
!=' llJ ll cr llJ e:._
(f)
0
<(
_j lL
0 cr llJ
CD
40
30
20
1 0
KEY
0
0
...
.n ro
(\J
'�
(!) 0
(\J
.n
... ro
"' "' ... ... ...
(\J lD
(!) lD
0
(!) (\J lD lD
(\J
I
...
I ro
�
(!) lD
STRIDES IN BYTES
D
1 STEP AGO
0
2 STEPS AGO
3 STEPS AGO
•
4 STEPS AGO
•
5 STEPS AGO
"'
0
�
(\J
�
"o_
(\J
I
� lD m
0
" a5 <ri C\i z
I
�
(\J o;
I
�
"' "-
"' "-
I
� ro
i;
"' u::
I
�
�
I
�
�
"'
Figure 1 5
Strides for Array
HO WIDX between tile Current Load and the Load One through Five Steps Ago
50
45
40
!=' llJ ll llJ
35 e:._ 30
(f)
0
<( 25 lL
0 cr
20
2 1 5 z
1 0
5
0
0
... n ro
KEY
D
1 STEPAGO
D
2 STEPS AGO
D
3 STEPS AGO n
� �
0
N n
V
N
.I
4 STEPS AGO
•
5
STEPS AGO
CO
N n n
("')
M ..:;t
... n ro (\J lD
Il
(!) lD
0
(!)
�
"-
... lD lD
(\J
I ro
I l..
�
�
(!)
"' o_
�
I
(\J
�
1
STRIDES IN BYTES
"o_
(\J
I
� lD m
0
... -
I
�
(\J o; a5
I
...
irfll
�
"' "-
<ri
I
� ro
C\i
I
�
i;
� lL
�
�
I
�
Figure 1 6
Strides tor Array
P between the Current Load and the Load One through Five Steps Ago
Digital Technical Journal
Vol . 9
No. 4
1 997 31
Conclusion
The case study shows that, given the right program analysis tools, a program developer can rake better advantage of his or her computer system. The experi mental tool we designed was very usefu l in providing insight into the algorithm's behavior. The approach considered yields an improvement in performance of
8 percent on a 625-MHz 2 1 1 64 Alpha microproces sor. This is definitely a worthwhile exercise since a su b stantial red uction i n execution rime was obtained using straightforward and easy guidelines.
The data collected from a memory access profiling tool helps the user understand a given program as well as its memory access patterns. It is an easier and faster way to gai n insight into a program than examining the listing and the assembler generated by the compiler.
Such a tool enables the programmer to compare mem ory access patterns of different algorithms; therefore, it is very usefu l when optim izing codes. Probably i ts most important value is that it shows the developer if his or her implementation is doing what he or she thinks the algorithm is doing and highlights potential bottlenecks resulting fro m memory accesses. Optimiz i ng an application is an iterative process, and being able to use relative!�' easy-to-use tools l i ke Atom is a very important part of the process. The major advantage of the tool presented in this paper is that no source code is needed, so it can be used to analyze the perf(mnaJlce of program executables.
Acknowledgments
The authors wish to thank David Lafrance-Linden,
Dwight Manley, Jean-Marie Verd u n , and Dick Foster for ti·uitful discussions. Thanks to John Eck, Dwight
Manley, Ned Anderson, and Chuck Schneider for reading and commenting on earlier versions of the paper. Special thanks go to the anonymous referees tor comments and suggestions that helped us consider ably improve the paper. Finallv, we thank John
Krearsoulas and Dave Fenwick tor encouraging our col �laboration.
References
1 . 1-'ro,t.vammer's
Guide, Digital f �VIX Version 4 . 0, chapter
9 ( Maynard , Mass. , Digital E quipm e n t Corporation, tVb rch
1 996).
2 . 0 .
R r e we r
,
J.
Do ng a rr a the Arwlvsis
, and
D .
Sorensen,
Tuu!s tu ,1id in ojNJemoly
Access
Pattcrnsfor
Fortmn f'rn p,rcllns,
Technical Report, Argonne
National L1b or ar o ry
( June 1 988 )
3. OFC Fortran
Longuage Reference
M a n u al
(l'vbvnJrd ,
Mass ., Digital Equipment Corporation, 1 99 7 ) .
4 . D. B l ic k s r e in , P. Craig, C. Da,·idson , R. Faiman, K. Glos sop, R. Grove, S. H obbs, and W. Noyce, The GEM
Opti m izing Compiler System, Dt;qital Techniwl jour nal. YOI .
4, no. 4 ( 1 992 ) : 1 2 1-136.
Biographies
Susanne
M.
Balle
Susanne Bal le is curren tly a member of the High Perfor mance Computi ng Expertise Center. Her work areas arc pcrtormancc prediction for 2 1 264 microprocessor-based architectures, memory access pattern analysis, p arallel
Lmczos algorithms for solving very large eigenvalue pro b
lems, distributed-memorv matrix computations, as well as improving performance of standard industry and customer bench marks. Bdore joining DIG ITAL, she was a postdoc toral telJow at the Mathenutical Sciences Department at the
IBM Thomas
J.
Warson Research Center where she worked on high-performance mathematical software . From 1992 to
1 995, she consulted for the
Connection Machine Scientific
Sofu,·are
Library
( CMSSL) group at Thinking Mac h i nes
Corporation. Susanne received a
Ph.D. i n mad1ematics and an M.S. in mechanical engineering a nd computational fl u i d dynam ics from the Tech n ical U niversity of Denmark. She is a member ofS.!Alvl.
Simon
C.
Steely, Jr.
Simon Steely is a consulting engineer i n DIGITAL's
AlphaServcr Platform Development Group . I n h is 2 1 years at DIGITAL, he has worked on many projects, i ncluding development ofrhe PDP- 1 1 , VAX, and Alpha systems. His work has tocused on writing microcode, design i ng proces sor and system architecture, and doing performance analysis to make design decisions. In his most recent project, he w a s a member of the arcl1itccrurc team for a fi.
ttu r e system. In addition, he led the team developi ng the cache-coherencv p ro tocol on rhar project. His primary i nterests are computer architecture, pcrtormancc analysis, prediction technologies, cache/memorv hierarchies, and optimization of code for best performance. Simon has
J
B.S. in engineering !Tom the
Universirv of Ne\\' Mexico and is a member of!EEE and
AC.M . He holds 1 5 patents and has several more pending.
32
Digital Tcc llnic1l JournaJ Vol . 9 No. 4 1 997
Open VMS Alpha 64-bit
Very large Memory
Design
I
Karen
L.
Noel
Nitin Y. Karkhanis
The OpenVMS Alpha version 7 . 1 operating system provides memory management features that extend the 64-bit VLM capabil ities intro duced in version 7.0. The new OpenVMS Alpha
APis and mechanisms allow 64-bit VLM applica tions to map and access very large shared mem ory objects (g loba l sections). Design areas include shared memory objects without disk file backing storage (memory-resident global sections), shared page tables, and a new physi cal memory and system fluid page reservation system.
Database products and other applications impose heavy demands on physical memory. The newest version of
DIGITAL's Open VMS Alpha operating system extends its very large memory (VLM) support and allows large caches to remain memory resident. OpenVMS Alpha version
7 . 1
enables applications to take advantage of both 64- bit virtual addressing and very large memories consistent with the Open VMS shared memory model.
In this paper, we describe the new 64-bit VLM capabili ties designed for the Open VMS Alpha version
7 . 1
oper ating system. We explain application flexibility and the system management issues addressed in the design and discuss the performance improvements realized by
64-bit VLM applications.
Overview
A VLM system is a computer with more than 4 giga bytes ( G B ) of main memory. A flat, 64-bit address space is commonly used by VLM applications to address more than 4 GB of data.
A VLM system allows large amounts of data to remain resident in main memory, thereby reducing the time required to access that data. For example, database cache designers implement large-scale caches on VLM systems in an effort to improve the access times for database records. Similarly, VLM database applications support more server processes than ever before. The combination of large, in-memory caches and an increased number of server processes signifi cantly reduces the overall time database clients wait to receive the data requested . 1
The OpenVMS AJpha version
7 . 0
operating system took the first steps in accommodating the virtual address space requirements of VLM applications by introducing 64-bit virtual addressing support. Prior to version
7.0,
large applications-as well as the Open VMS operating system itself-were becoming constrained by the limits imposed by a 32-bit address space.
Although version
7.0
eased address space restric tions, the existing OpenVMS physical memory man agement model did not scale well enough to accommodate VLM systems. Open VMS imposes spe cific limits on the amount of physical memory a
Digital Technical journal
Vol .
9
No. 4 1 997
33
process can occupy.
As a result, applications lacked the abi l i ty to keep a verv large object in physical memory.
I n systems on whi ch the physical memory is not plen tifu l , the mechanisms that limit per-process memory util ization serve to ensure tair-::111d -equal access to a potentially scarce resou rce . However, on systems rich with memory whose intent is to service applications creating VLM objects, the limitations placed on per process memory utilization inhibi t
the
overa l l perfor mance of those applications. As
a
result, the benefi ts of a VLM system may not be completely realized .
Applications that require very large amounts of physical memory need additional VUvl su pport. The goals of the Open VMS Alpha VLM project were the fol lowing:
•
•
•
•
• fvlaxi mize the ope r
a
tin
g
system's 64 -bit capabi lities
Take full
advantage
of the Alpha Architecture
Not require excessive application change
Simpl i f)' the system management of a VLM system
Allow tor the creation of VLM
objects
that e x hibit the same basic characteristics, from the program mer's perspective, as other virtual memory objects created with the OpenVMS system service pro gramming interface
These
goals
became the foundation for the fol low
ing
VLM technology implemented in the OpenVlviS
Alpha version 7 . 1 operating syste m :
•
•
•
M emory-resident
global
sections-shared memory objects that do not page to disk
Shared page tables-page tables mapped by multiple processes, which in turn map to memory-resident global
s
ectio n
s
The reserved memory registry-a memory reserva tion system that supports memory-resident global sections and shared page tables
The remainder of this paper describes design areas of VLM support tor
Open the major
VMS and dis cusses the problems addressed by the design team, tl1e alternatives considered, and the benefits of the extended
VLM support in
Open VMS
AI pha version 7 . 1 .
Memory-resident Global Sections
We designed memory- resident global sections the existing shared memory model,
to
resolve appl i c
the
scal ing problems experi enced by
VLM
a
ti o
n
s on Open VMS . We focused our design on
using
the
64- bit addressing support. Our project goals i ncluded simpli f),ing system management and harnessing the speed of tJ1e Alph
a
microprocessor. Before describing memorv resident global sections, we provide a brief explanation of shared memory, process w orki
ng
sets, and a
page fault hand ler.
Global Sections
An
OpenVMS
global section
is a shared memory object. The memory within the global section is
s hared
among diflere n t processes in the syste m . Once a process has created a global section, others may map to the section to share the data.
Several
types of global sections can be created and mapped by calling
Open
VMS system services.
G lobal Section Data Structu res
Internally, a global section consists of several basic data structures that
are
stored in system address space and are accessible to all processes from kernel mode. When a global section is created , OpenVMS allocates and initializes a set of these data structu res. The relationship between the structures is i l l ustrated in
Figu re
1 . The sample global section is named "SHROBJ" and is
2 ,048
Alpha
pages or 1 6 mega bytes ( M B ) in size . Two processes have mapped to the global section by referring to the global
GLOBAL PAGE TABLE GLOBAL SECTION DESCRIPTOR GLOBAL SECTION TABLE ENTRY
HEADER
GSTX
OTHER INFORMATION
"SHROBJ"
u
SIZE
=
HEADER
GPTX
2.048 PAGES
REFCNT
=
4,096
FLAGS
�
GPTE
GPTE
PAGE 0
PAGE 1
GPTE PAGE 2.047
KEY:
GSTX GLOBAL SECTION TABLE INDEX
Figure 1
G lobal Sccrion Dara Structures
34
Digital T\:chnical Journal Vol . 9 No. 4 1 997
section data structures in their process page table entries ( PTEs) .
Process PTEs Mapping t o Global Sections
When a process maps to a global section, its process PTEs reter to global section pages in a one-to-one fashion. A page of physical memory is allocated when a process accesses a global section page for the first time. This results in both the process PTE and the global section page becoming valid. The page frame number ( PFN) of the physical page allocated is stored in the process
PTE. Figure 2 illustrates two processes that have mapped to the global section where the first process has accessed the first page of the global section.
When the second process accesses the same page as the first process, the same global section page is read from the global section data structures and stored in the process PTE of the second process. Thus the two processes map to the same physical page of memory.
The operating system supports two types of global sections: a global section whose original contents are zero or a global section whose original contents are read from a file. The zeroed page option is referred to as demand zero.
Backing Storage for Global Sections
Global section pages require backing storage on disk so that more fre quently referenced code or data pages can occupy physical memory. The paging of least recently used pages is typical of a virtual memory system. The back ing storage for a global section can be the system page files, a file opened by OpenVMS, or a file opened by the application. A global section backed by system page files is referred to as a page-file- backed global sec tion. A global section backed by a specified file is referred to as a fi le- backed global section.
When a global section page is i nvalid in all process
PTEs, the page is eligible to be written to an on-disk backing storage file. The physical page may remain in memory on a list of modified or free pages. Open VMS algorithms and system dynamics, however, determine which page is written to disk.
Process Working Sets
On OpenVMS, a process' valid memory is tracked within its working set lists. The working set of a process reflects the amount of physical m emory a process is consuming at one particu lar point in time.
Each valid working set list entry represents one page of virtual memory whose corresponding process PTE is valid . A process' working set l ist includes global sec tion pages, process private section pages, process pri vate code pages, stack pages, and page table pages.
A process' working set quota is limited to 5 1 2 MB and sets the upper limit on the number of pages that can be swapped to disk. The limit on working set quota matches the size of a swap I/0 request.2 The effects on swapping would have to be examined to increase working set quotas above 5 1 2 M B .
Process working set lists are kept in 32-bit system address space. When fi·ee memory is plentiful in the sys tem, process working set lists can increase to an extended quota specified in the system's account file for the user.
The system parameter, WSMAX, specifies the maximum size to which a process working set can be extended .
Open VMS specifies an absolute maximum value of 4 GB for the WSMAX system parameter. An inverse relation ship exists between the size specified for WSMAX and the number of resident processes Open VMS can support, since both are maintained in the 32-bit addressable por tion of system space. For example, specifYing the m<L"Xi mum value for WSMAX sharply decreases the number of resident processes that can be specified .
Shou ld OpenVMS be required to support larger working sets in the future, the working set lists would have to be moved out of 32-bit system space.
FIRST PROCESS PTEs GLOBAL PAGE TABLE SECOND PROCESS PTEs
VALID · PFN
I NVALID · GPTX
f-J
VALID - PFN
GPTE
L
'-
I NVALID · GPTX
I NVALID · GPTX
INVALID · GPTX
�
KEY:
GPTX GLOBAL PAGE TABLE I NDEX
GPTE GLOBAL PAGE TABLE ENTRY
Figure 2
Process and Global PTEs
GPTE
L
INVALID · GPTX
Digital Technical Journal
Vol . 9 No. 4
1 997 35
Page Fault Handling for Global Section Pages
The data within a global section may be heavily accessed by the many processes that are sharing the data. Therefore, the access time to the global section pages may influence the overall performance of the application.
Many hardware and software factors can influence the speed at which a page within a global section is accessed by a process. The factors relevant to this dis cussion are the following:
•
•
•
Is the process PTE vali d or invalid?
I f the process PTE is invalid, is the global section page valid or i nvalid?
If the global section page is i nvalid, is the page on the modified l ist, free page list, or on disk vvithin the backing storage file?
If the process PTE is invalid at the time the page is accessed , a translation invalid fault, or page fault, is generated by the hardware. The Open VMS page fault handler determines the steps necessary to make the process PTE valid.
If the global section page is valid, the PFN of the data is read from the global section data structures.
This is caUed a global valid fault. This type of fault is corrected quickly because the data that handles this fault is readily accessible from the data structures i n memory.
If the global section page is invalid, the data may still be witl1in a physical page on the modified or free page list maintained by OpenVMS. To correct this type of fault, the PFN that holds the data m ust be removed from the modified or free page list, and the global sec tion page m ust be made valid. Then the fault can be handled as if it were a global valid fault.
If the page is on disk within the backing storage file, an I/0 operation must be performed to read the data from the disk into memory before the global section page and process PTE can be made valid. This is the slowest type of global page fau lt, because performing a read ljO operation is much slower than manipulating data structures in memory.
For an application to experience the most efficient access to its shared pages, i ts process PTEs should be kept valid . An application may use system services to lock pages i n the working set or in memory, but typi cally the approach taken b y applications to reduce page fault overhead is to increase the user account's working set quota. This approach does not work when the size of the global section data exceeds the size of the working set q uota limit of 5 1 2 MB.
Database Caches as File-backed Global Sections
Quick access to a database application's shared mem ory is critical for an application to handle transactions quickly.
Global sections implement shared memory on
Open VMS, so that many database processes can share the cached database records. Since global sections must have backing storage on disk, database caches are eitJ1er backed by the system's page files or by a file cre ated by the database application.
For best performance, the database application should keep al l its global section pages valid in the process PTEs to avoid page fault and I/0 overhead .
Database processes write modified buffers from the cache to the database files on an as-needed basis.
Therefore, the backing storage file required by
Open VMS is redundant storage.
Vety Large Global Sections
The OpenVMS VLM project focused on VLM data base cache design. An additional goal was to design the VLM features so that other types of VLM applica tions could benefit as well.
Consider a database cache that is 6 GB in size.
Global sections of this magnitude are supported on
OpenVMS Alpha with 64- bit addressing support. If tl1e system page files are not used, the application must create and open a 6-GB file to be used as backing stor age for the global section.
With the maxi mum quota of 5 1 2 MB for a process vvorking set and with the maximum of a 4-GB working set size, no process could keep the entire 6-GB data base cache valid in its working set at once. When an
OpenVMS global section is used to implement the database cache, page fau lts are i nevitable. Page fault activity severely impacts the performance of the VLM database cache by causing unnecessary I/0 to and from the disk while managing these pages.
Since all global sections are pageable, a 6-GB file needs to be created for backing storage purposes. In the ideal case, the backing storage file is never used.
The backing storage file is actually redundant with the database files tJ1emselves.
VLM Design Areas
The VLM design team targeted very large global sec tions ( 4 GB or larger) to share data among many processes. Furthermore, we assumed that the global section's contents would consist of zeroed memory instead of originating from a
file.
The team explored whetJ1er this focus was too narrow. We were con cerned that implementing j ust one type ofVLM global section would preclude support for certain types of
VLM applications.
We considered that VLM applications might use very large amounts of memory whose contents origi nate ti-om a data file. One type of read-onJy data from a file contains program i nstructions (or code). Code sections are currently not pushing the limits of 32-bit address space. Another type of read-only data from a file contains scientific data to be analyzed by the VLM
36 Digiral Technical journal
Vol.
9 No. 4 1 997
application. To accommodate very large read -only data of this type, a large zeroed global section can be created, the data from the file can be read into mem ory, and then the data can be processed in memory.
If writable pages are initially read trom a file instead of zeroed , the data w i l l most likely need to be written back to the original file. In this case, the tile can be used as the backing storage for the data. This type of
VLM global section is supported on Open VMS Alpha as a tile-backed global section . The operating system's algorithm for working set page replacement keeps the most recently accessed pages i n memory. Working set q uotas greater than 5 1 2 MB and working set si zes greater than 4 GB help this type of VLM application scale to h igher memory sizes.
We a lso considered very large demand-zero private pages, "mal loc" or " heap" memory. The system page flies are the backing storage for demand-zero private pages. Currently, processes can have a page file quota as large as 32 G B . A VLM application, however, may not want these private data pages to be written to a page file since the pages are used i n a simi lar fashion as in-memory caches. Larger working set quotas also help this type ofVLM application accommodate ever mcreasmg memory sizes.
Backing Storage Issues
For many years, data base cach e designers and database performance experts had requested that the
Open VMS operating system support memory with no backing storage tiles. The backi ng storage was not only redundant but also wasteful of disk space. The waste issue is made worse as the sizes of the database caches approach the 4-GB range . As a result, the
OpenVMS Al pha VLM design had to allow for non file-backed global sections.
The support of64-bit addressing and VLM has always been viewed as a two-phased approach, so that function alit)' could be delivered in a timely fashion.' Open VMS
Alpha version 7.0 provided the essentials of 64 bit addressing support. The VLM support was '�ewed as an extension to tJ1e memory management model and was deferred to Open VMS Alpha version 7. 1 .
Working Set List Issues. Entries i n rl1e process work ing set list are not required for pages that can never be written to a backing storage file. The fundamental con cept of the Open VMS working set algorithms is to sup port the paging of data from memory to disk and back into memory when it is needed again. Since the focus of the VLM design was on memory that wou ld not be backed by disk storage, the VLM design team realized that these pages, although valid in the process PTEs, did not need to be in the process' working set list.
VLM Programming Interface
The Open VMS Alpha VLM design provides a new pro gramming i nterface for VLM applications to create, map to, and delete demand-zero, memory-resident global sections. The existing programming interfaces did not easily accommodate the new VLM features.
To justif)' a new programming interface, we looked at me applications mat would be calling the new system service routines. To add ress more than 4 GB of mem ory in rl1e flat Open VMS 64-bit address space, a 32-bit application must be recompiled to use 64-bit pointers and often requires source code changes as wel l .
Database applications were already modifYing meir source code to use 64-bit pointers and to scale rl1eir algorithms to handle VLM systems.1 Therefore, calling a new set of system service routines was considered acceptable to the programmers of VLM applications.
Options for Memory-resident Global Sections
To initialize a very large memory-resident global sec tion mapped by several processes, the overhead of hardware faults, allocating zeroed pages, setting process PTEs valid, and setting global section pages valid is eli m in ated by preallocating the physical pages for the memory-resident global section . PreaJiocation is performed by the reserved m emory registry, and is discussed later in this paper. Here we tal k about options for how the reserved memory is used.
Two options,
ALLOC and
FLUID, are available for creating a demand -zero, memory-resident global section .
ALLOC Option
The ALLOC option uses preallocated, zeroed pages of memory for the global section. When me ALLOC option is used , pages are set aside during system start-up specifically for tJ1e memory-resident global section. Preallocation of contiguous groups of pages is discussed in the section Reserving Memory during System Start- up. Preallocated memory-resident global sections are taster to initialize man memory resident global sections mat usc rl1e
FLUID option .
Run-time performance is improved by using the
Alpha Architecture's granu larity h int, a mechanism we discuss later i n this paper. To use the ALLOC option , the svstem must be rebooted tor large ranges of physi cally contiguous memory to be al located .
FLU I D Option
The
FLUID option allows pages not yet accessed within the global section to remain fluid within the syste m . This is also referred to as the t:ud t option because the page fault algorirl1m is used to allo cate the pages. When the
F LUID
(or fault) option is used, processes or the system can use the physical pages until they are accessed wit h i n the memory resident global section. The pages remain within the system's fluid memory unti l they are needed. This l:)'pe of memory-resident global section is more flexible than one that uses the ALLOC option. If an applica tion that uses a memory-resident global section is run on a system that cannot be rebooted due to system
Digital Tec hnical
)oum1 l Vol . 9 No. 4 1 997
37
availability concerns, it can still use the FLUID option.
The system will not al low this application to run unless enough pages of memory are available in the system for the memory-resident global section .
The system service i nternals code checks the reserved memory registry to determine the range of pages preallocated for the memory- resident global section or to determine if the FLUID option wi l l be used. Therefore the decision to use the ALLOC or the
FLUID option is not made within the system services routine interface. The system manager can determine which option is used by specifYing preferences in the reserved memory regisu·y. An application can be switched from using the ALLOC option to using the
FLUID option without requiring a system reboot.
Design Internals
The internals of the design choices underscore the mod ularity of the shared m emory model using global sec tions. A new global section type was easily added to the
Open VMS system. Those aspects of memory-resident global sections d1at are identical to pageable global sec tions required no code modifications to support.
To support memory-resident global sections, the
M RES and ALLOC flags were added to the existing global section data structures. The MRES flag indi cates that the global section is memory resident, and the ALLOC flag indicates that contiguous pages were preallocated for the global section.
The file-backing storage information within global section data structures is set to zero for memory resident global sections to indicate that no backing storage file is used. Other than the new flags and the lack of backing storage tile information, a demand zero, memory-resident global section looks to
Open VMS Alpha memory management like a demand zero, file- backed global section. Figure 3 shows d1e updates to d1e global section data structures.
One i mportant difference with memory-resident global sections is that once a global section page becomes valid, it remains valid for the life of the global section . G lobal section pages by definition can never become invalid for a memory-resident global section .
When a process maps to a memory-resident global section, the process PTE can be either valid for the
ALLOC option or invalid for the FLUID option .
When the ALLOC option is used, no page fau l ting occurs for the global section pages.
\¥hen a process first accesses an invalid memory resident global section page, a page fau l t occurs just as with traditional file- backed global sections. Because the same data structures are present, the page f:··wlt code initially executes the code for a demand-zero, file-backed global section page. A zeroed page is allo cated and placed in the global section data structures, and the process PTE is set valid.
The working set list manipulation steps are skipped when d1e M RES fl ag is encountered in the global sec tion data structures. Because d1ese global section pages are not placed i n d1e p rocess working set list, t hey are not considered in i ts page-replacement algo rithm. As such, the OpenVMS Alpha working set manipulation code paths remained unchanged.
System Management and Memory-resident Global
Sections
When a memory-resident global section is used i nstead of a traditional, pageable global section for a database cache, there is no longer any wasted page file storage required by OpcnVMS to back up the global section .
The od1er system management issue aJleviated by the implementation of memory-resident global sec tions concerns working set sizes and quotas. When a file backed global section is used for the database cac he, the database processes require elevated worki ng
GLOBAL PAGE TABLE
GLOBAL SECTION DESCRIPTOR GLOBAL SECTION TABLE ENTRY
HEADER
GSTX
OTHER INFORMATION
"SHROBJ"'
J
HEADER
GPTX
SIZE =
n
REFCNT
FLAGS 1X
INFORMATION =
0
A
\
r--t--
MRES
\
INVALID - DZRO,GSTX
INVALID - DZRO,GSTX
INVALID - DZRO,GSTX
PAGE 0
PAGE
1
PAGE
n - 1
GPTX GLOBAL PAGE TABLE INDEX
DZRO DEMAND ZERO
Figure 3
Memory- resident Global Section Data Structures
38 Digital Technical Journal Vo l. 9 No. 4 1997
set quotas to accommodate the size of the database cache . This is no l onger a concern because memory resident global section pages are not placed i nto the process working set l ist.
With the use of memory-resident global sections, system managers may reduce the val ue for the
WSMAX system parameter such that more processes can remain resident within the system. Recall that a process working set list is in 32-bit system address space, which is l imited to 2 G B .
Shared Page Tables
VLM applications typically consume l arge amounts of physical memory in an attempt to minimize disk I/0 and enhance overall application performance. As the physical memory requirements of VLM applications increase, the fol lowing second -order effects are observed due to the overhead of mapping to very large global sections:
•
•
•
•
Noticeably long application start-up and shut down times
Add itional need tor physical memory as the num ber of concurrent sharers of a large global section mcreases
Unanticipated ex haustion of the working set q uota and page file quota
A reduction i n the nu mber of processes resident in memory, resulting in increased process swapping
The first two effects are related to page table map ping overhead and size . The second two effects, as they relate to page table quota accou nting, were also resolved by a shared page tables i mplementation . The fol lowing sections address the first two issues since they uniquely pertain to the page table overhead.
Application Start-up and Shut-down Times
Users of VLM appli cations can observe long applica tion start-up and shut-down times as a result of creat ing and deleting very large amounts of virtual memory. A single process mapping to a very large virtual memory object does not im pact overall system performance. However, a great number of processes that simultaneously map to a very l arge virtual mem ory object have a noticeable impact on the system's responsiveness. The primary cause of the pertormance degradation is the accelerated contention for internal operating system locks . This observation has been witnessed on OpenVMS systems and on DIGITAL
UNIX systems (prior to the addition ofVLM support. )
O n OpenVMS, the memory management spinlock
(a synchronization mechanism) serializes access to priv ileged, memory-management data structures. \Ve have observed increased spinlock contention as the result of hundreds of processes simultaneously mapping to large global sections. Similar lock contention and sys tem unresponsiveness occur when multiple processes attempt to delete their address space simultaneously.
Additional Need for Physical Memory
For pages of virtual memory to be valid and resident, the page table pages that map the data pages must also be valid and resident. If the page table pages are not in memory, successful address translation cannot occur.
Consider an 8-GB, memory-resident global section on an
Open VMS Alpha system (witl1 an 8-kilobyte page size and 8-byte PTE size ) . Each process that maps the entire 8-GB, memory-resident global section requires
8 MB for the associated page table structures. If 1 00 processes are mapping tl1e memory-resident global sec tion, an additional 800 MB of physical memory must be available to accommodate all processes' page table structures. This further requires that working set list sizes, process page file quotas, and system page files be large enough to accommodate tl1e page tables.
When 1 00 processes are mapping to the same memory-resident global section, the same PTE data is replicated into the page tables of the 1 00 processes.
If each process could share the page table data, only
8 MB of physical memory would be required to map an 8-GB, memory-resident global section; 792 MB of physical memory would be availa ble for other system pu rposes.
Figure 4 shows the amount of memory used for process page tables mapping global sections ranging i n size from 2 to 8 G B . Note that as the nu mber of processes that map an 8 - G B global section exceeds
8,000
�
(3
7,000
w
�
�
6,000
5,000
f-'
�
4,000
� w
(.)
� ll.. u..
0
3.000
2,000
1,000 w
N
(il
KEY:
-o- 1 ,000 PROCESSES
-- 400 PROCESSES
-+-
200 PROCESSES
Fig ure
4
Process Page Table Sizes
Digital Technical
Journal
Vol .
9
No. 4 1 997
39
l ,000, the amount of memory used by process page tables is larger than the global section itself.
Shared Memory Models
We sought a sol ution to sharing process page tables that >vould alleviate the performance problems and memory utilization overhead yet stay within the shared memory framework provided by the operating system and the architecture . Two shared memory models are implemented on Open VMS, shared system address space and global sections.
The Open VMS operating system supports a n address space layout that i ncludes a shared system address space, page table space, and private process address space. Shared system address space is created by plac ing the physical address of the shared system space page tables into every process' top-level page table.
Thus, every process has the same lower-level page tables in its virtual-to-physical address translation path . In turn, the same operating system code and data are found in all processes' address spaces at the same virtual address ranges. A similar means could be used to create a shared page table space that is used to map one or more memory-resident global sections.
An alternative for sharing the page tables is to create a global section that describes the page table structure.
The operati ng system could maintain the association between the memory-resident global section and the global section for its shared page table pages. The shared page table global section coul d be mapped at the upper levels of the table structure such that each process that maps to it has the same lower- level page tables in its virtual-to-physical address translation path . This in turn would cause the data to be mapped by ali the processes.
Figure 5 provides a conceptual representation of the shared memory model . Figure 6 extends the shared memory model by demonstrati ng that the page tables become a part of the shared memory object.
The benefits and drawbacks of both sharing models are highlighted in Ta ble 1 and Table 2 .
Model Chosen for Sharing Page Tables
After examining the existing memory-sharing models on Open VMS and taking carefu I note of the composi tion and characteristics of shared page tables, the design team chose to implement shared page tables as a global section. In addition to the benefits listed i n Table 2 , the
UPPER LEVEL ---------------- LOWER LEVEL
LEVEL 1
LEVEL 2
LEVEL3
PAGE TABLE (PT)
r--
L1PT r--
I--
�
-
L2PT
r--
-
L3PT
L2PT
L.,
L3PT
'--
L2PT
....
L3PT
-
....
L3PT
�
=-u
--u
-
-u u
-u
SHROBJ
I
Figure 5
Shared Memory Object
40
Digiral Tcchnic1l Journal Vol. Y No. 4 1997
I
UPPER LEVEL -----------------+ LOWER LEVEL
LEVEL 1
LEVEL 2 LEVEL 3
PAGE TABLE
,...
L1 PT r-----
-
L2PT t---r---
I--
L3PT
'+
L2PT
[.._
L3PT
.......___
�u r�u
....
.......___
L2PT
-
�
L..3PT
'
!..,
L..3PT
-u ru
SHROBJ
Figure
6
Shared Memory Objects Using Shared Page Tables
Ta ble 1
Shared Page Ta ble Space-Benefits a nd Drawbacks
Benefits
Shared page table space beg ins at the same virtual add ress for a l l processes.
Drawbacks
The virtual add ress space is reserved for every process.
Processes not using shared page tab les are pen a l ized by a loss in available address space.
Shared page ta ble space is at least 8 G B in size, regard less of whether the entire space is used.
A sign ifica nt amount of new code would need to be added to the kernel s ince shared system space is man aged sepa rately from process add ress space.
Table
2
Global Sections for Page Tab l es-Benefits and Drawbacks
Ben efits
The same virtual add resses can be used by a l l processes, but this is not req u i red.
The amount of virtual address space ma pped by shared page tables is determined by appl ication need.
Sha red page ta bles are availab le only to those processes that need them.
Shared page tables al low for sign ificant reuse of existing global section data structures and process add ress space management code.
Drawbacks
Shared page tables a re mapped at di fferent virtual add resses per process unless additional steps are taken.
Digital Technical Journal Vol. 9 No. 4 1997
4 1
design team noticed that shared page table pages bear great resemblance to the memory-resident pages they map. SpecificaUy, for a data or code page to be valid and resident, its page table page must also be valid and resi dent. The ability to reuse a signjficant amount of the global section management code reduced the debug ging and testing phases of the project.
In the initjal implementation, shared page table global sections map to memory-resident global sec tions only. This decision was made because the design focused on the demands ofVLM applications that use memory-resident global sections. Should significant demand exist, the implementation can be expanded to allow the mapping of pageable global sections.
Shared page tables can never map process p1ivate data.
The design team had to ensure that the shared page table implementation kept process private data from entering a virtual address range mapped by a shared page table page. If this were to happen, it would compromjse the security of data access between processes.
Shared Page Tables Design
The goals tor the design of shared page tables included the following:
•
•
•
•
•
Reduce the time required for mu ltiple users to map the same memory-resident global section
Reduce the physical memory cost of maintaining private page tables for multiple mappers of the same memory-resident global section
Do not req uire the use of a backjng storage file for shared page table pages
Eliminate the working set list accounting tor tl1ese page table pages
Implement a design that allows upper levels of the page table hierarchy to be shared at a later time
Figure 6 demonstrates the shared page table global section model. The dark gray portion of the figure highlights the level of sharing supplied in Open VMS
Alp ha version 7 . 1 . The light gray portion highlights possible levels of sharing allowed by creating a shared page table global section consisting of upper-level page table pages.
Modifications to Global Section Data Structure
Table 2 noted as a benefit the ability to reuse existi ng data structures and code. Minor modifications were exacted to the global section data structures so that they could be used to represent a shared page table global section. A new flag,
SHARED_PTS ,
\Vas added to the global section data structures. Coupled with this change was the requirement that a memory resident global section and its shared page table global section be u niquely l i n ked together. The correspon dence between the two sets of global sections is man aged by the operating system and is used to locate the data structures for one global section when the struc tures for the other glo bal section are in hand . Figure 7 highlights the c hanges made to the data structures.
Creating Shared Page Ta bles
To create a memory resident global section, an application calls a system service routine. No flags or extra arguments are required to enable the creation of an associated shared page table global section.
The design team also provided a means to disable the creation of the shared page tables in the event that a user might find shared page tables to be undesirable.
To disable the creation of shared page tables, the reserved memory registry entry associated with the memory-resident global section can specif)r that page tables are not to be used . Witlun the system service routine that creates a memory-resident global section ,
GLOBAL SECTION DESCRIPTOR
GLOBAL SECTION TABLE ENTRY
HEADER
GSTX
"SHROBJ"
___r
HEADER
GPTX
SIZE =
n
REFCNT
FLAGS
j
FILE BACKING S T ORA
' it-
\
GSTX GLOBAL SECTION TABLE INDEX
GPTE GLOBAL PAGE TABLE ENTRY
GLOBAL PAGE TABLE
VALID GPTE
VALID GPTE
VALID GPTE
PAGE 0
PAGE 1
PAGE
n - 1
Figure 7
Data Strucwrc Modifications
Vol 9 No. 4 1 997
42 Digital Technical Journal
the reserved memory registry is examined for an entry associated with the named global section. If an entry exists and it specifies shared page tables, shared page tables are created . If the entry does not specify shared page tables, shared page tables are not created.
If no entry exists for the global section at all, shared page tables are created. Thus, shared page tables are created by default if no action is taken to disable their creation. We believed that most applications would benefit from shared page tables and thus should be created transparently by default.
Once the decision is made to create shared page tables for the global section, the system service routine allocates a set of global section data structures for the shared page table global section. These structures are initialized in the same man ner as their memory resident cou nterparts, and in many cases the fields in both sets of structures contain identical data.
Note that on current Alpha platforms, there is one shared page table page for every l ,024 global section pages or 8 MB. (The number of shared page table pages is rounded up to accommodate global sections that are not even multiples of8 MB in size . )
Shared PTEs represent the data within a shared page table global section and are initialized by the operating system. Since page table pages are not accessible through page table space' u n til the process maps to the data, the initia lization of the shared page table pages p resented some design issues. To i n itialize the shared page table pages, they must be mapped, yet they are not mapped at the time that the global section is created.
A simple solution to the problem was chosen . Each shared page table page is temporarily mapped to a sys tem space virtual page solely for the purposes of initial izing the shared PTEs. Temporarily mappi ng each page allows the shared page table global section to be fully i nitialized at the time it is created.
An interesting alternative for initializing the pages would have been to set the upper-level PTEs i nvalid, referencing the shared page table global section. The page fault handler could i nitialize a shared page table page when a process accesses a global section page, thus referenci ng an i nvalid page table page. The shared page table page cou ld then be initial ized through its mapping in page table space . Once the page is initialized and made valid, other processes referencing the same data wou ld incur a global valid fault for the shared page table page . This design was rejected due to the additional overhead of faulting during execution of the application, especially when the ALLOC option is used for the memory-resident global section.
Mapping to a Shared Page Table Global Section
Map ping to a memory-resident global section that has shared page tables presented new challenges and constt·aints on the mapping criteria normally imposed by the virtual address space creation routi nes. The map ping service routines require more stri ngent mapping criteria when mapping to a memory-resident global section that has shared page tables. These require ments serve rwo purposes: l .
Prevent process private data from being mapped onto shared page tables. If part of a shared page table page is unused because the memory-resident global section is not an even multiple of 8 MB, the process would normally be allowed to create private data there.
2. Accommodate the virtual addressing alignments required when mapping page tables into a process' address space.
For applications that cannot be changed to conform to these mapping restrictions, a memory-resident global section with shared page tables can be mapped using the process' private page tables. This capability is also useful when the memory-resident global section is mapped read-only. This mapping cannot share page tables witl1 a writable mapping because the access pro tection is stored within the shared PTEs.
Shared Page Table Virtual Reg ions
The virtual region support added i n OpenVMS Alpha version 7.0 was extended to aid in prohibiting process private pages from being mapped by PTEs within shared page tables. Virtual regions are l ightweight objects a process can use to reserve portions of its process virtual address space. Reserving address space prevents other threads in the process ti·om creating address space in the reserved area, un less they specifY the handle of that reserved area to the address space cre ation rou tines.
To control which portion of the address space is mapped with shared page tables, the shared page table attribute was added to virtual regions. To map a memory- resident gl obal section with shared page tables, the user must supply the mapping routine with the name of the appropriate global section and tl1e region handle of a shared page table virtual region.
There are two constraints on the size and alignment of shared page table virtual regions.
1 .
The size of a shared page table virtual region must be an even multiple of bytes mapped by a page table page. For an 8KB page system , the size of any shared page table virtual region is an even multiple of8 MB.
2 . The caller can specif)' a particular starting virtual address for a virtual region. For shared page table virtual regions, tl1e starting virtual address must be aligned to an 8-MB boundary.
If the operating system chooses the virtual address for the region, it ensures the virtual address is properly aligned.
Digital Technical Journal
Vol. 9 No. 4 1997
43
If either the size or the align ment requirement for a shared page table virtual regi on is not met, the service fails to create the region .
The size and alignment constraints placed on shared page table virtual regions keep page t:J.ble pages from spanning two different virtual regions. This allows the operati ng system to restrict process private mappings in shared p:1ge table regions and shared page table mappings in other regions by checking the shared page table's attribute of the region before starting the mapping operation.
Mapping within Shared Page Table Regions
The address space mapped within a shared page table virtual region also must be page table page <lligned . This ensures that mappings ro m u l tiple memory-resident global sec tions that have u nique sets of shared page tables do not encroach upon each other.
The map length is the only argu ment to the map ping system service routines that need not be an even multiple of bytes mapped by a page table page. This is allowed because it is possible for the size of the memory-resident global section to not be an even m u l tiple of bytes mapped by a page table page. A memory-resident global section that ti ts this length description wi ll have a portion of its last shared page table page unused .
The Reserved Memory Registry
OpcnVMS Alpha VLM support provides a physical memory reservation system that c<1n be exploited by
VLM appl ications. The main pu rpose of this system is to provide portions of the system's plwsicaJ memory to m u l tiple consumers . \Vhen nccessarv, a consumer can reserve J q u antity of physical Jddrcsscs i n an attempt to make the most efficient usc of system com ponents, namely the translation buffer. More efficient use of the CPU and i rs periphera l components leads to i ncreased appl ication performanc<.:.
Alpha Granularity Hint Regions
A translation buffer (TB) is a CPU component that caches recent virtual-to-physical address translations of va lid pag<.:s. The TB is a sma l l amount of very fast memory and theref(xe is only capable of cach ing a lim ited nu mber of translations. Each entry in the TB rep resents a singk successfu l virtual -to-physical address translation. TB entries arc purged either when a req uest is made by software or when the TB is fldl and a more recent translation needs to be cached .
The Alpha Architecture coupled with software can help make more dkctive use of the TB by allowi ng several contiguous pages (groups of 8, 64, or
5 1 2 ) to act as a single huge page . This single huge page is cal led a granularity hint region and is composed of contiguous virtual and physical pages whose respective first pages are exactly a ligned accord ing to the nu mber of pages in the region . \Vhen the conditions for a gran ularity hint region prevai l , the single h uge page is allowed to consu me a single TB entry instead of sev eral. Minimizing the nu m ber of entries consumed for contiguous pages greatly reduces turnover within the
TB, leading to higher chances of a
TB hit. I ncreasing the l ikelihood of a TB hit in turn minimizes the n u m ber o f virtual-to- physical translations performed by the CPU.5
Since memorv- residenr global sections are nonpage able, mappings to memory-resident global sections greatly benefit by exploiting granularity hint regions.
Unfortunately, there is no guarantee that a contiguous set of physical pages ( let alone pages that meet the alignment criteria) can be located one<.: th<.: system is initialized and ready for steady-state operations.
Limiting Physical Memory
One technique to locate a contiguous set of PFNs on
OpenVMS (previously used o n Alpha and VAX plat forms) is to limit the actual n u m be r of physical pages used by the operating system . This is accomplished by setting the PHYSTCAL _ MEMORY system parameter to a val u e smaller than the actuJI amount of physical memory avail a ble in the syste m . The system is then rebooted, and the PFNs that represent higher physical addresses than that speci fied by the p<�ramcter are al lo cated bv the application.
This technique works wel l for a single application that wishes to allocate or usc a range of PFNs not used bv the operating system. Unfortu nately, it sufkrs from the follo\\'ing problems:
•
•
•
•
•
It requires the application to determine rhe first page not used by the operating system .
It req uires a process running this appl ication to be highly p rivileged since the opcr:tting system does not check which PFNs arc being mapped.
Since the operating system docs not arbitrate access to the isolated physical addresses, only one applica tion can safely use them .
The Alpha Architecture allows for implementa tions to support discontiguous physical me mory or phys ical memory holes. Tbis mcms that there is no guarantee that the isolated physical add resses are successively adjacent.
The PFNs above the limit set :�re not managed by the operating system ( physical memory data struc tures do not describe these PFNs ) . Therefore, rhe pages above the limit cannot be reclaimed by the operating system once the :tppl ication is fi nished using them u nless the system is rebooted.
44
Digiral Ted1niol
Journ�l Vol . 9 �o. 4
1 997
The Reserved Memory Solution
The Open VMS reserved memory registry was created to provide contiguous physical memory for the pur poses of further improving the performance of VLM applications. The reserved memory registry al lows the system manager to specify multiple memory reserva tions based on the needs of various VLM applications.
The reserved memory registry has the ability to reserve a preallocated set of PFNs. This allows a set of contiguous pages to be preallocated with the appro priate alignment to allow an Alpha granularity hint region to be created with the pages. I t can also reserve physical memory that is not preallocated. Effectively, the application creating such a reservation can allocate the pages as required . The reservation ensures that the system is tuned to exclude these pages.
The reserved memory registry can specifY a reserva tion consisting of prezeroed PFNs. It can also specify that a reservatjon account for any associated page tables. The reservation system allows the system man ager to free a reservation when the corresponding consumer no longer needs that physical memory.
The memory reserved by the reserved memory reg istry is communicated to OpenVMS system tuning facilities such that the deduction in fluid memory is noted when computing system parameters that rely on the amount of physical memory in the system .
SYSMAN User I nterface
The Open VMS Alpha
SYSJ\tlAN utility supports the RESERVED_MEMORY command for manipulating entries in the reserved memory registry. A unique character string is specified as the entry's handle when the entry is added , modi fied, or removed. A size in megabytes is specified for each entry added.
Each reserved memory registry entry can have the following options: preallocated PFNs (ALLOC), zeroed
PFNs, and an allotment for page tables. VLM applica tions enter their u nique requirements for reserved memory. For memory-resident global sections, zeroed
PFNs and page tables are usua!Jy specified.
Reserving Memory during System Start-up
To ensure that the contiguous pages can be allocated and that run-time physical memory al location routines can be used, reserved memory allocations occur soon after the operating system's physical memory data struc tures have been initialized .
The reserved memory registry data file is read to begin tJ1e reservation process. Information about each entry is stored in a data structure. M u l tiple entries result in multiple structures being linked together in a descending-order linked list. The list is intentionally ordered in this manner, so that tJ1e largest reservations are honored first and contiguous memory is not frag mented with smaller requests.
For entries with the ALLOC characteristic, an attempt is made to locate pages that will satisfy the largest granularity hint region that fits within the request. For example, reservation requests that are larger than 4 MB result in the first page al located to be aligned to meet the requirements of a 5 1 2-page gran ularity hint region.
The system's fluid page counter is reduced to account for the amount of reserved memory specified in each entry. This counter tracks the number of phys ical pages that can be reclaimed from processes or the system through paging and swapping. Another system defined value, minimum fluid page count, is calculated during system irutialization and represents the absolute minimum n umber of fl uid pages the system needs to function. Deductions from the flu id page count are always checked against the minimum fluid page count to prevent the system from becoming starved for pages.
Running AUTOGEN, the OpenVMS system tuning utility, after modifying the reserved memory registry allows for proper initialization of the fl uid page counter, the mirumum fluid page count, and other sys tem parameters, thereby accommodating the change in reserved memory. AUTOGEN considers entries in the reserved memory registry before selecting values for system parameters that are based on the system's memory size. Failing to retune the system can l ead to unbootable system configurations as well as poorly tuned systems.
Page Tables Characteristic
The page table reserved memory registry characteristic specifies that the reserved memory allotment for a particular entry should include enough pages for its page table requirements. The reserved memory registry reserves enough memory to account for lower-level page table pages, although the overall design can accommodate allotments for page tables at any level.
The page table characteristic can be omitted if shared page tables are not desired for a particular memory-resident global section or if the reserved memory will be used for another purpose. For exam ple, a privileged application such as a driver could call the kernel -mode reserved memory registry routines directJy to use its reservation from the registry. In tJ1is case, page tables are already provided by the operating system since the reserved pages vvill be mapped in shared system address space.
Using Reserved Memory
Entries are used and returned to the reserved memory registry using a set of kernel-mode routines. These routines can be called by applications running in kernel mode such as the system service routines tJ1at create memory-resident
Digital Technical Journal
Vol 9 No. 4 1997
45
global sections. For an application to create a memory resident global section and use reserved memory, the global section name must exactly match tl1e name of the reserved memory registry entry.
After the system service routine has obtained the reserved memory for me memory-resident global sec tion, it calls a reserved memory registry routine again for me associated shared page table global section . I f page tables were not specified for the entry, tl1e system service routine does not create a shared page table global section.
A side benefit of using me ALLOC option for me memory-resident global section is tlut tl1e shared page tables can be mapped into page table space using gran u larity hint regions as well.
Returning Reserved Memory
The memory used by a memory-resident global section and its associated shared page table global section is returned to the reserved memory registry ( by calling a kernel-mode routine) when the global section is deleted. Reserved memory is only returned when a memory-resident global section has no more outstanding references.
Preallocated pages are not returned to the system's free page list.
Freeing Reserved Memory
Preallocated reserved mem ory that is unused or partially used can be freed to the system's free page list and added to the system's fluid page count. Reserved fluid memory is returned to me system's fluid page count only.
Once an entry's reserved memory has been freed, subsequent attempts to use reserved memory witl1 me same name may be able to use only tl1e FLUID option, because a preallocated set of pages is no longer set aside for the memory-resident global section. ( If the system's fl uid page count is large enough to accom modate the request, it will be honored. )
The ability to free unused o r partially used reserved memory registry entries adds flexibility to tl1e manage ment of the system. I f applications need more mem ory, me registry can still be run wim the FLUID option u ntil the system can be rebooted with a larger amount of reserved memory. A pool of reserved mem ory can be freed at system start-up so that multiple applications can use memory-resident global sections to a limit specified by the system manager in the reserved memory registry.
Reserved M emory Registry and Other Appl ications
Otl1er Open VMS system components and appucations may also be able to take advantage of the reserved memory registry.
Applications that relied upon modifications to the
PHYSICAL_MEMORY system parameter as a means of gaining exclusive access to physical memory can enter kernel mode and call me reserved memory reg istry kernel -mode routines directly as an alternative.
Once a contiguous range of PFNs is obtained, the application can map the pages as before.
Using and returning reserved memory registry entries requires kernel -mode access. This is not viewed as a problem because applications using the former method (of modifying me PHYSICAL_MEMORY system parameter) were already privileged . Using the reserved memory registry solves the problems associ ared witl1 me previous approach and requires few code changes.
Performance Results
In a paper describing me 64-bit option for me Oracle7
Relational Database System, ' r.he aumor underscores the benefits realized on a VLM system running the
DIGITAL UNIX operating system. The test results described in that paper highlight the benefits of being able to cache large amounts of data instead of resort ing to disk l/0 .
Although the Open VMS design team was not able to execute similar kinds of prod uct tests, we expected to realize similar performance improve ments for tl1e following reasons:
•
•
•
More of a VLM application's hot data is kept resi dent instead of paging between memory and sec ondary storage.
Application start-up and shut-down times are sig nificantly reduced since the page table structures for the large shared memory object are also shared.
The result is tl13t many fewer page tables need to be managed and manipulated per process.
Reducing d1e amount of PTE manipulations results in reduced Jock contention when hundreds of processes map ilie large shared memory object.
As an alternative to product testing, tl1e design ream devised experiments that simulate the simultaneous start-up of many database server processes. The exper iments were specifically designed to measure the scaling effects of a VLM system during application starr-up, not during steady-state operation. vVe performed two basic tests. In the first, we used a
7 . 5 -GB, memory-resident global section to measure tl1e time required for an increasing number of server processes to start up. All server processes mapped to me same memory-resident global section using shared page tables. The results shown in Figure
8
indicate that the system easily accommodated 300 processes.
Higher numbers of processes run simu ltaneously caused increasingly large amounts of system stress due to the paging of otl1er process data.
46 Digital Technical Journal
Vol. 9 No. 4 1997
1 0,000
9,000
(jJ
8,000
6
7,000
&]
6,000
�
5,000
_J
�
4,000
�
3,000 i=
2,000
Figure 8
Server
Starr-up Time versus Process Count
9,99 1 .61
1 , 000 o L-�--�--��--�--�--��--��
50 1 00 1 50 200 250 300 350 400 450 500 550
PROCESS COUNT
KEY:
--+--
M EMORY-RESI DENT GLOBAL SECTION/SHARED PAGE
TABLE
In another test, we used 300 processes to measure the time required to map a memory-resident global section with and without shared page tables. In this test, the size of global section was varied. Note that the average time requ ired to start up the server processes rises at nearly a constant rate when not using shared page tables. When the global section sizes were
5 GB
and greater, the side effect of paging activity caused the start-up times to rise more sharply as shown i n
Figure
9 .
The same was not true when using shared page tables. The time required to map the increasing sec tion sizes remained constant at j ust under three sec onds. The same experiment on an AlphaServer 8400 system with 28 GB of memory showed identical con stant start-up times as the size of the memory-resident global section was increased to 27 GB.
Conclusion
The OpenVMS Alpha VLM support avail able i n ver sion 7. 1 is a natural extension to the 64- bit virtual addressing support included in version 7.0. The 64- bit virtual addressing support removed the 4-GB virtual address space l imit and al lowed applications to make the most of the address space provided by Alpha sys tems. The VLM support enables database products or other applications that make significant demands on physical memory to make the most of large memory systems by allowing large caches to remain memory resident. The programming support provided as part of the VLM enhancements enables applications to take advantage of both 64- bit virtual addressing and very large memories in a modular fashion consistent with the Open VMS shared memory model . This combina tion enables applications to realize the full power of
Alpha VLM systems.
The Oracle7 Relational Database Management
System for OpenVMS Alpha was modified by Oracle
Corporation to exploit the VLM support described in this paper. The combination of memory-resident global sections, shared page tables, and the reserved memory registry has not only improved application start-up and run - time performance, but it has also simplified the management of Open VMS Alpha VLM systems.
Acknowledgments
1 6.000
1 5, 5 1 5.35
1 4,000 gr
1 2,000
8
1 0,000
�
8,000
_J
�
6.000
�
4.000
2,931 03
\
�--�--+---�--� --+ -� --� ��
2,000
The authors wish to thank other members of the
OpenVMS Alpha Kernel Group who contributed to the design and implementation of VLM support on
Open VMS Alpha: Tom Benson, Richard Bishop, Clair
Grant, Andy Kuehnel, and Dave Wall. Jef Kennedy and Tony Lekas of Oracle Corporation provided great assistance as well. We would also like to thank Ron
H iggins for his management support and Mike
Harvey for his long-standing mentorship. Margie
S herlock and Tom Benson were especia l ly helpfu l in assisting us with the writing aspects of this paper. We are also thankfu l to Gaitan D'Antoni who found the time in current project schedules to accommodate the writing of tJus paper.
References
KEY:
Q L---�--�--�--�--�--�--�--�
2.048 2,560 3,072 3.584 4,096 4,608 5 , 1 20 5,632 6, 1 44
GLOBAL SECTION SIZE (MEGABYTES) l . V.
GokhaJe, " Design of cle? Relational Database the 64-bit Option for the
Management System,"
Ora
Digital
Technicaljournal, vol . 8, no.
4
( 1 996): 76-82.
- MEMORY-RESIDENT GLOBAL SECTION/PRIVATE OR
PROCESS PAGE TABLE
--+--
M EMORY-RESIDENT GLOBAL SECTION/SHARED PAGE
TABLE
2. R . Golden berg and
S.
Saravanan , Open VMS
AXP
Inter nals and Data Structures, Version
1 . 5
( Newton, Mass. :
Digital Press, 1 994 ) .
Figure 9
Server Starr-up Time on an 8 - G B System
3 .
T
Benson,
K
Noel, a n d R. Peterson, "The OpenVMS
Mixed Pointer Sized Environment," Digital Technical
.Journal, vol. 8, no. 2 ( 1 996): 72-82 .
Digital Technical Journal
Vol. 9 No. 4 1 997 47
4. M. Harvey and L. Szubowicz, " Extending OpenVMS for 64-bit Addressable Virtual Memory,"
Digital
Tecb mcal]ourna!, vol. 8, no. 2 ( 1996 ) : 5 7-71 .
5 .
R.
Sites and
R.
Witek,
Alpba
AXP
Architecture Reference
Jvlanual,
2d ed . (Newton , Mass. : Digital Press, 1 99 5 ) .
General References
Open
VMS
Alpha Guide to 64-bit
A dd r es si n g and
VL!vl fea tures (Maynard, Mass . : Digital Equipmenr Corporation ,
Order No. AA-QSBCB-TE).
Open VJI!JS System Seruices Reference Manual. A -GET!v!SG
(l'vlaynard, Mass . : Digital Equipment Corporation, Order
No. AA-QSBMG-TE) and Open VMS System Services
Reference Manual. GETQUJ-Z
( lvlaynard, Mass . : Digi tal
Equipment Corporation, Order No. AA- QSBNB-TE ) .
For a more complete description o f shared memory creation on DIGITAL U NIX, see the
DIGITAl. UNIX Programmer's
Guide.
Biographies
Nitin
Y.
Karkhanis
Nitin l(arkhanis joined D IGITAL in 1 987. As a member of the Open VMS Alpha Executive Group, be was one of the developers of Open VMS Alpha 64-bit addressing sup port. He was also the primary developer for the physical memory hole ( ctisconriguous physical memory) support for
Open VMS Alpha. He is a coapplicant for two patents on
Open VMS VLM . Currently, Nitin is a principal software engineer i n the Open VMS NT Infrastructure Engineering
Group, where he is a member of the OpenVNlS D COM porting team. N itin received a B .S . i n computer science ti·om the University ofVermont in 1987.
Karen L. Noel
A consulting engineer in the Open VMS Engineering
Group, Karen Noel was the technical leader for the VLM project. Currently, as a mem ber of tl1e Open VMS Galaxy design team, Karen is contributing to the overall GalaA)' architecture and focusing on tl1e design of memory parti tioning and shared memory support. Afi:er receiving a
B .S . in computer science from Cornell University in 1 98 5 ,
Karen joined DIGITAL's RSX Development Group. In
1990, she joined tl1e VMS Group and ported several parts of tl1e VMS kernel from the VAX platform to the Alpha platform . As one of the principal designers of Open VMS
Alpha 64-bit addressing and VLM support, she has applied for nine software patents.
48
Digital Technical Journal
Vol . 9 No. 4 1 997
PowerStorm 4DT: A High performance Graphics
Software Architecture
I
Ben
j
amin N. Lipchak
Thomas Frisinger
Karen L. Bircsak
Keith L. Comeford
Michael !. Rosenblum
The PowerStorm 4DT series of graphics devices established DIGITAL as the OpenGL performance leader for mid-range workstations, both on the
DIGITAL UNIX and the Windows NT operating systems. Achieving this level of success required combining the speed of the Alpha microprocessor with the development of an advanced graphics subsystem architecture focused on exceptional software performance. The PowerStorm 4DT series of graphics adapters uses a modified direct-rendering technology and the Alpha CPU to perform geometry and lighting calculations.
The PowerStorm 4D40T, 4DSOT, and 4D60T mid range graphics adapters from DIGITAL have exceeded the performance of all OpenGL graphics devices cost ing as much as $25,000. In addition, these products achieved twice the price/performance ratio of com peting systems at the time they were announced.
The PowerStorm 4DT series of mid-range graphics devices was developed in 1 996 to replace the com pany's ZLX series. In its search for a vendor to replace the graphics hardware, DIGITAL found Intergraph
Systems Corporation. This company had been design i ng three-dimensional ( 3- D ) graphics boards for a few years and was then on its second-generation chip design. The schedule, cost, and performance of
Intergraph's new design matched our project's target goals. Intet·graph was building software for the
Windows NT operating system on its Intel processor based workstations, but was not doing any work for the UNIX operating system or the AJpha platform.
The goals of the PowerStorm 4 DT project ·were to develop a mid-range graphics product powered by the
AJpha microprocessor that would lead the industry in performance and price/performance.
This paper describes the competitive environment in the graphics industry at the conception of the
PowerStorm 4DT project. It discusses our design deci sions concerning the graphics subsystem architecture and performance strategy. The paper concludes with a performance summary and comparison in the industry.
Competitive Analysis
OveraJl performance oftoday's mid-range workstations is markedly better than that of just two years ago. This improvement is largely due to the dramatic increases in
CPU speeds, both in the number of instructions exe cuted per clock cycle and the number of clock cycles per second. Without trivializing the efforts of the CPU architects, such year-over-year increases in CPU perfor mance have become the trend of the last decade, espe cially with the AJpha microprocessor.
Digital Technical Journal Vol . 9 No. 4 1 997 49
More astounding is the central role that the graphics component of the workstation is p laying in defining the overall performance of the workstation. \Ve are in the age of visual computing. Whether or not an appli cation req uires 3-D graphics, even the most primitive applications often rely on a graphical user interface
( G U I ) . As such, the graph ical components of today's system-level benchmarks now carry significant weight.
More importantly, a prospective buyer often looks at results from standard graphics benchmarks as a gen eral indication o f a machine's overal l performance. In the computer-aided design/computer-aided manu facturing ( CAD/CAM ) market, a customer typically buys a workstation to run a set of applications that has a large 3-D component. Performance is measured by how fast a workstation can create and manipulate 3 - D objects. For the most part, this performance i s deter mined wholly by the graphics subsystem. The hard ware components of the graphics subsystem, however, vary from vendor to vendor and may or may not include the CPU.
Performance Metrics
Simply stated, the primary goal of the PowerStorm
4DT graphics device series was to provide the fastest mid -range OpenGL graphics performance while offer ing the best price/performance ratio . OpenGL is the i nd ustry-standard 3-D graphics application p rogram m ing i nterface (API) and associated library that pro vides a platform-independent i nterface for rendering
3-D graphics ]
QuantifYing performance can be an elusive goal.
Product managers in our Workstation Graphics G roup chose two mctrics to compare the performance of the
PowcrStorm 4DT adapter to our competitors' prod ucts. The first metric was performance on the industry standard OpenGL Viewperf benchmark, Conceptual
Design and Rendering Software (CDRS).' This bench mark vvas chosen for its universal acceptance in the
CAD/CAM and process control markets. vVben buyers compare graphics performance of two systems running
OpenGL, the Viewpcrf scores are among the first measurements they seek. The secon d measurement was performance on the Pro/ENGINEER application t!·om Parametric Technology Corporation (PTC ) .
The CDRS benchmark, a s shown i n Figure l , was established by the OpenGL Performance Characteri zation ( OPC) organization as one of several Viewperf viewsets. It emulates the variety of operations a user typically executes when running a CAD /CAM applica tion . Specifically, this benchmark uses a series of tests that rotate a 3-D model on the screen in a variety of modes, including wireframe vectors, smooth-shaded facets, texturing, and transparency. Performance is measured by how many frames per second can be generated. Higher frame rates equate to faster and smoother rotations of the model . Each test carries a
Figure 1
CDRS ViewperfBenchmark ofOpenGL Performance weight determined to roughly correspond to h ow important that operation is in a real-world CAD /CAM package. The test results arc geometrically averaged to produce a composite score. This single number is a representation of the graphics performance of any given system .
Although standard benchmarks are good perfor mance indicators, they cannot su bstitute for actual performance on an application . To ensure that the
PowerStorm 4DT adapter realized exceptional real world performance, the second metric chosen was the
CAD/CAM industry's market share leader, the Pro/
ENGINEER application. PTC provides the industry with a set of playback files called traiJ files. As s hown in
Figure 2, each tile contains a recording of a session i n which a user has created a nd rotated a 3 - D part. The recordings typically have large wireframe and smooth shad i ng components and little or no texture mapping.
Performance is measured by how quickly a system can play back a trail file. The C D RS benchmark stresses only the graphics subsystem, but the Pro/ENGINEER trail rile stresses the CPU and the memory su bsystem as well.
Graphics Hardware Standards
In 1 996, Silicon Graphics Inc. ( SGI ) captured the mid-range graphics workstation market with i ts
I ndigo2 Maximum IMPACT graphics su bsystem pow ered by the MIPS
RlOOOO microprocessor. DIGITAL,
Sun Microsystems, and International Business Machines
(IBM ) Corporation had yet to produce a product with the performance SGI offered ; instead, they competed in the low to lower mid-range graphics arena.
50 Digital Tcchn.ical Journal
Vol . 9 No. 4 1 997
Figure
2
Screen Capture from the
Pro/ENGINEER Trail
File Used ro Stress the PowerStorm 4DT Series
Hewlett- Packard was notably absent from either bracket due to its lack of a mid-range workstation with
OpenGL graphics capability. Mid-range workstations can be loosely classified as costing from $ 1 5 ,000 to
$40,000. Graphics performance in this price range dif fers, sometimes dramatically, from vendor to vendor.
Considering only raw graphics hardware perfor mance, a vendor had to offer a certain level of perfor mance to be competitive with SGI. By 1 996 standards, a competitive device needed to be capable of achieving the fol lowing:
•
•
•
•
•
•
1 million Gouraud -shaded, 25 -pixel, Z-buffered triangles per second
2 m i llion flat-shaded, antialiased, 1 0-pixel vectors per second
Trilinear, mipmapped, texture fill rates of 30 megapixels per second
24-bit deep color buffer
4-bit overlay buffer
4-MB dedicated or unified texture memory
•
•
Dedicated hardware support for double buffering and Z-buffering
Screen resolution ofl ,280 by 1 ,024 pixels at 72 hertz
I n 1 996, the PowerStorm 4D60T, the most advanced graphics adapter in the new series, was capa ble of the following:
•
•
•
•
•
•
•
•
1 . 1 million Gouraud-shaded, 2 5 - to 50-pixel,
Z-buffered triangles per second
2 . 5 million flat-shaded , antialiased, 10-pixel vectors per second
Trilinear, mipmapped, texture fill rates of greater than 30 megapixels per second
32-bit deep color buffer
8-bit overlay buffer
0- to 64-MB dedicated texture memory
Dedicated hardware support for double buffering
(including overlay planes) and Z-buffering
Screen resolution up to 1 ,600 by 1 ,200 pixels at 76 hertz
Digital
Technical journal Vol. 9 No. 4 1997 5 1
It is important to understand that these are hard ware maximums. The interesting work is not in achieving these rates under the best of conditions, but in achieving these rates under most conditions. To reiterate, building hardware that can theoretically per form well and building a system that performs well in benchmark applications are two distinctly different goals. The latter requires the former, but the former in no way guarantees the latter.
Different viewpoints on the best way to provide the highest level of performance have divided the industry into several camps. Workstation vendors must decide which approach best exploits the competitive advan tages of their systems. In the mid-range workstation market, our graphics philosophy is decidedly different from that of our competitors. For the most part,
DIGITAL is alone in its choice of a CPU- based, direct rendering graphics architecture.
In the next section, we discuss the various graphics design architectures in the industry, focusing on the design of the PowerStorm series and comparing it with SGI's approach.
Graphics Subsystem Architectures
The two essential choices for graphics subsystem design are deciding between indirect and direct rendering and choosing whether the CPU or an application-specific integrated circuit (ASIC) performs the geometry and lighting calculations. In this section, we discuss the advantages and disadvantages of both rendering schemes and calculation devices and explore designers' decisions for graphics subsystem architectures.
By order of occurrence, 3-D graphics can be divided into three stages:
( l ) transferal ofOpenGL API caJls to the rendering library,
( 2 ) geometry and lighting, and
( 3 ) rasterization. In the next section, we compare direct and indirect image rendering.
Direct Versus Indirect Rendering
Before the popularization of the Windows NT operat ing system and the personal computer, almost all graphics workstations used the X Window System or a closely related derivative. The typical X Window
System i mplementation is a standard client-server model .3 An application that draws to the screen requests the X server to manage the graphics hardware on its behalf.
The graphics API , either Xli b for two-dimensional
(2D) applications or OpenGL for 3-D, was the rime tiona! breaking point. Tradi tionally, client applications would make graphics API calls to do drawing or another graphics-related operation. These calls would be encoded and buffered on the client side. At some point, either explicitJy by the client or implicitly by tl1e
API l i brary, the encoded and buffered requests would be flushed to the X server. These commands would
5 2 Digital Technical Journal
Vol.
9
No. 4
1 997 then be sent to tl1e X server over a transport such as me Transmission Control Protocol/Internet Protocol
(TCP /IP), a local UNIX domain socket, or local shared m emory.
\Vhen me requests arrived at the X server, it would decode and execute them in order. Many requests would then require the generation of commands to be sent to the hardware. This client-server model was named indirect rendering because of the indirect way in which clients interacted with the graphics hardware.
Direct rendering is a newer method often employed i n the design of high-end graphics systems!·5 In this scheme, tl1e client OpenGL library is responsible for al I or most 3-D rendering. Instead of sending commands to the X server, the client itself processes the com mands. The client also generates hardware command buffers and often communicates directJy with the graphics hardware . In this rendering scheme, the X server's role is greatly diminished for 3-D OpenGL requests but remains the same for 2 - D Xlib requests.
The designers chose to support direct rendering for the PowerStorm 4DT adapter. Direct rendering offers considerably better performance than indirect render ing. Note, however, direct rendering does not pre clude indirect rendering. All devices that support direct rendering under the X Window System also support indirect rendering.
In the following subsections, we discuss tl1e advan tages and disadvantages of direct and indirect render ing. We also explain the impetus for making the
PowerStorm 4DT adapter the first graphics device from DIGITAL capable of direct rendering.
Indirect Rendering
One advantage of indirect ren dering that should never be underestimated is its proven track record. This technology is widely accepted and understood. It offers network transparency, which means a client and server need not reside on the same machine. A client can redirect its graphics to any machine running an X server as long as the two machines are connected on a TCP /IP network. This model worked well until faster CPUs and graphics devices were developed. The protocol encode, copy, and decode overhead associated witJ1 sending requests to the server became a bottJeneck.
The increased use of display lists provided an inter mediate solution to this problem. Display lists are a group ofOpenGL commands tlut can be sent to the X server once and executed mu ltiple times by referenc ing the display list ID instead of sending all the data each time. D isplay l ists dramatical ly reduced commu nication overhead and returned graphics to the point at whicl1 communication to tl1e X server was no longer the bottleneck.
Unfortunately, display lists had significant disadvan tages. Once defined, they could not be modified. To achieve performance using indirect rendering, almost
all OpenGL commands had to be collected into dis play lists. This caused resource problems because display lists could be quite large and had to be stored in the X server until explicitly deleted by the client.
Probably the greatest disadvantage was that display lists were generally awkward for application programs to use. Application programmers prefer the more su·aightforward method of immediate-mode pro gramming by which commands are called individually.
For these reasons, indirect rendering proved to be insufficient, even with the advent of display lists.
Direct Rendering
The PowerStorm 4DT project team was committed to designing a product with lead ersrnp performance for both the display-list-mode and immediate-mode renderi ng. The designers realized early that they would have to adopt direct rendering to address the performance problems with immediate mode indirect rendering.
As mentioned earlier, the philosophy behind classi cal direct rendering is that each client handles all
OpenGL processing, creates a buffer of hardware commands for the device, and then sends the com mands to the device without any X server interaction.
This model has several drawbacks. First, access to the graphics hardware is difficult to synchronize between clients and the X server. Second, windows and their properties such as position and size have to be main tained by the clients, which also requires a complex synchronization design . SGI used this model for its
IMPACT series of graprncs devices.
The PowerStonn 4DT designers took a more con servative approach, based largely on the same model.
One fundamental difference is that each client gener ates hardware command buffers in shared memory.
The client then sends requests to the
X server tel ling it where to locate the hardware commands. The
X server sets up the hardware to deal with window position and size and then initiates a direct memory access ( DMA) of the hardware command buffer to the graphics device.
Essentially, the X server becomes an arbitrator of bard ware buffers. This approach worked quite well , because the X server was the logical place for synchronization to occur and it already maintained window properties. We were able to have all the performance benefits of classi cal direct rendering without the pitfalls.
One implication of direct rendering is that the client and the server have to be on the same physical macrnne.
When first evaluating direct rendering, designers were curious to determine how often our customers used this configuration; that is, did most users perform their work and display their graphics on the same computer?
Our surveys showed that more than 95 percent of users did display their graphics locally. The remaining
5 percent rarely cared about performance. Today, this may seem obvious; two years ago, it could not be assumed.
Direct rendering offered a huge performance improvement to nearly all our customers. The perfor mance gains were two to four times the performance of indirect rendering.
Direct-rendering 2-D Most graphics device imple mentations use direct rendering only for OpenGL, because indirect rendering of immediate-mode
OpenGL is protocol rich.
As mentioned previously, the transferal of this protocol to tl1e
X server can be quite expensive. One interesting aspect of our design is its support for direct rendering of2 - D Xlib calls.
Other graphics vendors consider 2-D performance important only for 2 - D benchmarks. These bench marks, which largely stress the graphics hardware's ability to draw 2-D primitives quickly, can generate a lot of work for the hardware with relatively few requests. Unlike 3-D, these requests do not need much geometry processing before iliey can be sent to the hardware. This means mat very little protocol is needed to saturate the hardware.
As long as the proto col generation does not produce a bottleneck, indirect rendering performs as well as direct rendering. In addition, given that OpenGL benchmarks like CDRS have almost no 2 - D component, it seems reasonable to conclude that indirect-rendered 2- D should suffice.
Benchmarks often are not sufficiently representative of real applications, especially when they isolate 2 -D and 3-D operations. CAD/CAM applications typically have a substantial 2 - D GUI, which interacts closely with the 3-D components of the application. A bench mark that exercises both 2-D and 3-D by emulating a user session on an application will provide results that more accurately reflect the performance witnessed by an end user. These benchmarks simply measure how long it takes to complete a session, so both 3-D and
2 - D performance impact the overall score.
Our research showed that with a highly optimized
OpenGL implementation, in many cases it was no longer the 3-D components that slowed down a benchmark, but the 2 - D components. Further exam ination revealed that it was the same protocol bottle neck evident with indirect- rendered OpenGL.
Applications were generating relatively small drawing operations with many drawing attri bute changes intermixed, such as draw line, change color, dra\v line, change color, and so forth . This type of request stream tends to generate tremendous amounts of protocol, unlike 2 - D benchmarks that rarely change drawing attributes.
Accordingly, 2-D direct rendering presented itselfas the logical solution. With the direct-rendering infra structure and design already in place, developers sim ply needed to extend it for 2 -D/Xlib. This required the development of two additional l ibraries: the
Vectored
X library and the Direct
X library (unrelated to Microsoft's Direct.'\ API) .
Digital Technical Journal
Vol.
9 No. 4 1997 53
The Vectored
X library replaced the preexisting Xlib.
It aUows de,�ces that support direct rendering to vector, or redirect, X.lib function calls to direct-rendering rou tines instead of generating the
X protocol and sending it to the
X server. I f a graphics device docs not support direct rendering, it defau lts to the generic protocol generating routines. I t i s important to u nderstand that tiLis is a device-independent library responsible only for vectoring X.lib caUs to tl1e approptiate Library.
The Direct X library, on the other hand, is a device dependent library. It contains all the vectored functions that the Vectored
X l ibrary cal ls when the device sup ports direct rendering. This library operates in much the same way as the direct-rendering OpenGL l ibrary.
It processes the requests and places graphics hardware commands in a shared memory buffer. The X server later sends the buffer to the grapiLics device by DMA.
The entire fu nctionality of tl
1 e X Library is nor imple mented through direct rendering for several reasons. I n many cases, a shared resource resides i n tl1c server (e.g., the X server performs al l pixmap rendering). I n other cases, the hardware is not directly addressable bv the client (e.g., the X server handles al l ITa me bu fkr reads).
Often the client does not have access to all window information tJ1at the server maintains (e.g., the
X server handles all window to window copi es ) . Fortu nately, these operations arc eimer not fi-equenrlv used, not expected to be fast, or easily saturate the hardware.
Further details of the Vectored
X li brary and Direct
X library are beyond the scope of this paper. The con cept of direct-rendered 2 - D , however, is sou nd. It has helped D I GITAL outperform other vendors on many application benchmarks that were largely focused on
OpenGL but had significant 2-D components. Our
2-D d i rect-rendering technology has also enh;mccd
2 - D performance and response time for the mJnv thousands of exclusively 2- D appl ications for the
X Window System.
Geometry and Lighting
The geometry and lighting phase can be performed by the host CPU or by a specialized, high-speed ASIC, which is typical ly located on the graphics device.
Regardless of where these calculations take p lace, the general idea is that the user's vertices arc transformed and lit, men ted to the rasterizer. Since the rastcrizcr is on the graphics device, choosing me host to do the geometry and ligh ting implies that the transformed and lit vertices are then sent across the bus to the ras terizcr.
The use of a specialized ASI C impl ies that the user's vertices are sent across the bus, transformed and lit by the custom ASI C , and then ted d i rectly to the rasterizer. The in formation transferred across the bus is obviously ditkrent, but in terms of amount of dat:l per vertex, it is approximately the same. Therefore, bus bandwidth does not become a deciding factor for either design .
54 l.)igiral Tcrhniol journal Vol. 9
:\fo. 4 1 997
Host CPU Geometry and Lighting
T r :1 d i t i o n a l l y ,
DIGITAL has chosen the host
CPU t o perform the geometry and l ighting calculations. The Po\\"erStorm project designers chose this approach because of the
Alpha microprocessor's exceptionJI floating-point speed, and because almost a l l 3-D calculations i nvolve
Hoating- point values. At the time this project \\"as con ceived , the only general-purpose, widely Jvaibblc processor capable of reeding more than 1 million transformed and l it vertices per second to the hard ware was the Alp ha CPU.
A n additional benefit of having the Alpha CPU do the work was an overall cost reduction of the graphics device. Custom AS!Cs arc expensive to develop and manufacture.
Another important and related advantage is that our softwJre becomes proportionally Elster as clock speeds rise on available Alpha microprocessors. This results in a neJr l inear performance i ncrease without any add itional engineering cost. For example, using the same software, a 5 00-mcgahcrtz ( MHz) Alpha micro processor is able to prod uce 25 percent more vertices per second than a 400-MHz Alpha microprocessor.
Because of this, de\ elopers can write optimized Alpha code once and reuse it for successive generations of
Al p ha CPUs, reaping performance i mprovements with virtually no fltrther i nvested effort.
It is obvious that rendering can proceed no bstcr than vertices cJn be generated . If the OpenGL library can transform and light only 750,000 vertices per sec ond, and the graphics device can rasterize l mil lion, the etfective rendering rate wil l be 750,000. I n this example, the OpenGL geometry and lighting software stages are the bottleneck. However, if the nu mbers were reversed, and the hardware cou ld only rastcrize
750,000 vertices while the OpcnGL software provided
l
million, the rasterization hardware wou ld become the bottleneck.
Thus far, we have discussed two potential bottle necks: the OpenG L implementation itself atKi the ras teri zation hardware . The third and potcnti a l l v most damaging bottleneck mav be the client's Jbiliry to reed vertices to the OpenG L library. I t should be clc1r thJt this is the top level of vertex processing. The OpcnGL l i brJry can render no faster than the rate at which the client application feeds it vertices. Conseq uently, the rJstcrizcr can render primitives no raster than the
Open(J L library can produce them . Thus, J bottleneck in generating vertices for the OpenG L l i brary wi ll slow the entire pipeline. Ideally, we wou ld like eJch level to be able to produce at least as many vertices as the lower levels can consume.
Clearly, the performance of the application , in terms of handing vertices to the Open(;L librJry, is J fu nc tion of CPU speed . This is onlv an issue f(x Jpplica tions that have large computation overhead before renderi ng. Currentlv, almost a l l graphics benchmarks have little or no computation overhead in getting vcr-
rices to the OpenGL library. Most attributes are pre comp uted, since they are trying to measure only the graphics performance and throughput. For the most part, this holds true for t he traditional CAD/CAL\1 packages. However, some emerging scientific visual ization applications as well as some high-end CAD applications require significant compute cycles to gen erate the vertices sent t o the OpenGL li brary. For these applications, only the DIGITAL Alpha CPU based workstations can produce the vertices fast enough tor interactive rates.
There are some potential disadvantages to this design. Namely, the CPU is responsible tor both the application's and the graphics library's computations. If the application and the OpenGL i mplementation must contend tor compute cycles, overall performance will sufter. Analysis of applications revealed that typical 3-D and
2-
D graphics applications do internal calculations followed by rendering. Only under rare circumstances do the two processes mix with a substantial ratio. If the applications should start mixing their own processing needs with those of the OpenGL library, the notion of host-based geometry would need to be revisited .
Another potential disadvantage is the rate at which
Alpha CPU performance i ncreases versus the rate at which the rasterizer chip's performance i ncreases. The emerging generation of graphics devices is capable of rasterizing more than 4 mil lion triangles per second . It is unknown whether future generations of the Alph a
C P U will b e able t o feed t h e faster graphics hardware.
ASIC-based Geometry and lighting
Performing geom eb)' and lighting calculations \\�th a custom ASIC on the graphics device is often referred to as OpenGL in hard ware because most of the OpenGL pipeline resides in the
ASIC. The OpenGL library is limited to handing the API calls to the hardware. SGI has adopted the ASIC- based approach for many generations of workstations and graphics devices. In this section, we discuss why this method works for them and its potential shortcomings.
SGI workstations use either the R4400 or the RlOOOO
CPU developed by MIPS Technologies. Although these
CPUs have good integer performance, their floating point performance cannot generate the number of ver tices that the Alpha CPU can. As a consequence, SGI has to use the custom-graphics ASIC approach. One advan tage to the custom ASIC is the decoupling of graphics from the CPU. Since each can operate asynchronously, the application has full use ofthe CPU.
Typically, custom geometry ASI Cs, also known as geometry engines, perform better than a general purpose CPU tor several reasons. First, the custom
ASIC must perform only a well-understood and lim ited set of calculations. This al lows the ASIC designers to optimize their chip tor these specific calculations, releasing them ti·om the burden and complexity of general-purpose CPU design .
Second, the graphics engine and the rasterizer can be tightly coupled ; i n fact, they can be located on the same chip. This allows tor better pipelining and reduced communication latencies between the two components. Even if the geometry engine and raster i zer are located on different chips, which is not at all uncommon, a much stronger coupl i ng exists betvveen the geometry engine and the rasterizer than does between the host CPU and rasterizer.
Third, geometry engines can yield high perfor mance when executing certain d isplay l ists. The use of a display list al lows an object to be q uickly re-rendered from a different view by changing the orientation of the viewer and reexecuting the stored geometry. If the display list can fit within the geometry engine's cache, it can be executed locally without having to resend the display l ist across the bus for each execution . This helps alleviate the transportation overhead in getting the display list data over the bus to the graphics device.
It is u nclear how often this really happens since rasteri zation is typically the bottleneck. If the display list is fil led with many small area primitives, however, its use can result in noticeable performance gains. Geometry engines often have a limited amount of cache. If an application 's display list exceeds the amount of cache memory, p erformance degrades significantly, often to below the performance attainable without a geometry accelerator. Our research shows that display list sizes used by applications increase every year; therefore, cache size must i ncrease at the same rate to maintain display l ist performance advantages.
The primary disadvantage of using custom ASI Cs to perform the geometry and l ighti n g calculations is the expense associated with their design and manufacture.
In addition, a certain risk is i nvolved with their devel opment: hardware bugs can seriously i mpact a p rod uct's viability. Fixing the bugs causes the schedule to slip and the cost to rise. Hardware bugs discovered by customers can be devastating. ·with host-based geom etry, a software fix in the OpenGL library can easily be i ncorporated and distributed to customers.
A sometimes unrecognized d isadvantage of dedi cated geometry engines is that they are bound to fixed clock rates, with l ittle room for scalability. Although this is true of most CPU designs, CPU vendors can j us tifY the engineering effort required to move to a taster technology, because of competitive pressures and the larger volume of host CPU chips.
Rasterization
During the rasterization phase, pri mitives are shaded, blended , textured, and Z-buffered . In the early years of raster-based comp u ter graphics, rasterization was done using software . As computer graphics became more prevalent, graphics performance became an issue . Because rasterization is highly computational and requires many accesses to frame buffer memory,
Digital Technical Journal
Vol .
9 No. 4 1 997
55
it quickly became the performance bottleneck.
Specialized hardware was needed to accelerate the rasterization part of graphics. Fortunately, bardware acceleration of rasterization is well understood and is now the de facto standard. Today, nearly every graph ics device has rasterization hardware. Even low-priced commoctity products have advanced raster capabilities such as texture mapping and antialiasing.
In the next section, we relate our strategy for obtaining optimal graphics software performance from an Alpha processor-based system.
Performance Strategy
The goals of the PowerStorm 4DT program were largely oriented toward performance. Our strategy consisted of having a generic code path and then tun ing performance where necessary using Alpha assem bly and integrated
C
code.
Performance Architecture
The designers optimized the sofu¥are performance of the PowerStorm 4DT series within the framework of a flexible performance architecture . This architec ture provided complete functionality tlu·oughout the performance-tu ni ng process, as wel l as the flexibi lity to enhance the performance of selected, performance sensitive code paths.
I n this context, code paths refer to the vertex handling routines that conduct each vertex tl1rough the geometry, lighting, and output stages. Whereas most OpenGL
API calls simply modify state condi tions, these vertex routines perform the majority of computation. This makes them the most likely choices for optimization .
The Generic Path
A solid, all-purpose code base written in
C
and named the generic path offers fu ll coverage of all OpenGL code paths. The generic path incurs a signi ficant performance penalty because its universal capabilities require that it test for and handle every possible combination of state conditions. In fact, under certain conditions, the generic patl1 is incapable of driving the hardware at greater than
33
percent of its maximum rendering rate. The generic path assumes responsibility for the rare circumstances that are not deemed performance-sensitive and thus not wormy of optimization. It also acts as a safety net when high performance paths realize m id-stride that mey are not equipped to handle new, unanticipated conditions.
M u lticompi led S peed of Lig ht (SOL) Paths
H i g h performance SOL paths provide greatly increased per formance where such performance is necessary. U nder prescribed conditions, SOL paths replace the genetic path, yielding equivalent functionality with perfor mance many times mat of me generic patl1. SOL paths
56
Digital Technical Journal
Vol. 9
No. 4 1997 were written for the combinations of state conditions exercised most frequently by the target applications and benchmarks.
The developers responsible for performance tuning designed two classes of SOL paths. First, they gener ated a large n umber of SOL patl1s by compiling a
C
code template multiple times. Whereas the generic patl1 is composed of several routines, each correspond ing to a single stage of the pipeline, a multicompiled
SOL patl1 integrates these stages into a monolithic routine. Each compilation turns on and off a ctifferent subset of state conditions, resulting in integrated pams for every combination of the available conditions. This multicompilation of integrated SOL paths yields me following benefits:
•
•
•
The C compil er is allowed a broader overview of tl1e code and can more wisely schedule instructions.
In contrast, the generic pam is composed of several individual stages. These relatively short routines do not provide the C compiler with enough space or enough scope to make informed and effective, insu·uction-ordering decisions. MulticompiJing tbe various stages into a series of monolithic, i ntegrated routines relieves each of these problems.
The multicompilation assumes a fixed set of concti tions for each generated path. This eliminates the need for run-ti me testing of these conditions dur ing each execution of the path. Instead, such test ing is necessary only when state conctitions change.
Validation, as this testing is called, determines which new path to employ, based on the new state conctitions. Wim tl1e great n umber and complexity of state conditions influencing mis decision, vaJjda tion can be an expensive process. Performing vali dation only in response to state changes, rather tl1an for every vertex, results in significant perfor mance gains.
The SOL path coverage at least doubles every time that support for a new state condition is added to the template. Each new condition increases the number of combinations of conctitions being multi compiled into SOL paths by a factor of two or more. An adverse side effect of this strategy is that me compile time and resulting l ibrary size wi ll increase at me same rate as the SOL path coverage.
Assembly Lang uage SOL Paths
Hand-coded Alpha assembly language paths constitute the other class of high-performance SOL paths. These paths, designed specifically for extremely performance-sensitive concti tions, require much more time and attention to pro d uce. Taking advantage of the many features of me
Alpha microprocessor transforms assem bly language coding from a science i nto an art form.6 The Alpha assembly coders kept the following issues foremost in tl1eir minds:
•
•
•
•
The
2 1 1 64 and subsequent Alpha microprocessors are capable of quad- issuing instructions, which means that as many as four instructions can be i niti ated during each cycle. The combination of instruc tions that may be issued, however, depends on the computational pipelines and other resources employed by each i nstruction. Coders must care fu l ly order instructions to gain the maximum bene fit from the multiple-issue capability.
As a consequence of the above restrictions, inte ger and floating-point operations must be sched uled i n parallel. With few exceptions, only two floating-point a nd two integer instructions can be issued per cycle. Efficiency i n this case requ ires not only local instruction-order tweaking but also global changes at the algorithmic l evel . Integer and floating-point operations must be balanced throughout each assembly routine. If a particular computation can be easily performed using either i nteger math or floating-point math, the choi ce is made according to which pipeline has more free cycles to use.
Register supply is another factor that affects the design of an assembly language routine. Although the Alpha m icroprocessor has a generous number of registers
(
32 i nteger and 32 floating-point), they are still considered a scarce resource. The coder must organize the routine such that some calcula tions complete early, freeing registers for reuse by subsequent calculations.
The crucial performance aspect of assembly coding is transporting the data where and when it is needed . The latency of loading data from main memory or even from cache into a register can eas i ly become any routine's botdeneck. To minimize such latencies, load instructions must be issued well in advance of a register's use; otherwise, the pipeline will stall u ntil the data is available. In an ideal architecture with an infinite quantity of regis ters, all loads could be performed well in advance.
Unfortunately, due to the scarce amount of free registers, the number of cycles available between loading a register and its use is frequently lim ited .
Each o f these assembly language programming con siderations requires intense attention but yields unmatched performance.
Performance Tuning
After reviewing benchmark compa.tisons and recom mendations rrom independent software vendors, we determined which areas required performance improve ment. We approached performance tuning rrom two directions: either by increasing SOL path coverage or improving the existing SOL code.
Increasing SOL path coverage was the more straight forward but the more time-consuming approach. If an
SOL path did not exist for a specific condition, a new one would have to be written. Adding a new option to the multicompilation template required a significant effort in some cases. I mplementing a new assembly language SOL path always required significant effort.
I mproving the performance of an existing SOL path required an iterative process of profiling and recoding. We employed the DIGITAL Continuous
Profiling Infrastructure ( DCPI ) tools to analyze and profile the performance of our code .7 DCPI indicated where bottlenecks occurred and whether they were due to data cache misses, instruction slotting, or branch misprediction. This information provided the basis for obtaining the maximum performance from every line of code.
Development of 3-D Graphics on Windows NT
At the start of the PowerStorm 4DT project, the
Windows NT operating system was an emerging tech nology. The DIGITAL UNIX platform held the larger workstation market share, whi le Windows NT accounted for only a small percentage of customers.
For that reason, designers targeted performance for applications running on DIGITAL UNIX and devel oped
3 -D
code entirely u nder that operating system.
Nevertheless, we recognized the potential gains of developing
3 - D graphics for d1e vVindows NT system .
One of the company's goals was to b e among m e first vendors to provide accelerated OpenGL hardware and software for Windows NT.
Wid1 a concerted effort and a few compromises, me team developed the PowerStorm 4DT into the fastest
OpenGL device for Windows NT, a title that was held for more than 18 months. To achieve this capability, the designers made the following key decisions:
•
•
•
•
To write code that was portable between the
DIGITAL UNIX and Windows NT systems.
To dedicate two people to the integration of the
DIGITAL U NIX-based code into tl1e Windows NT environment. Most OpenGL code was operating system independent, but supporting infrastructure needed to be developed for Windows NT.
To use Intergraph 's preexisting 2- D code and to avoid writing our own. I ntergraph provided us with a stable 2-D code base for Windows NT. This code base had room for optimization, but further opti mization of the 3-D code took precedence.
To ship the graphics drivers for DIGITAL UNIX first, and the graphics drivers for Windows NT three months later. I n this way, we allowed the
DIGITAL UNIX development phase to advance unimpeded by the efforts to port Windows NT.
Digiral Technical Journal
Vol . 9 No. 4 1997 57
Results and Conclusion
In August of 1 996, the PowerStorm 4 D60T graphics adapter was best in its price category with a CDRS per formance number of 49.01 using a 500-J\rlHz Alpha processor. It yielded a new price/performance record of $ 3 2 1 per frame per second . At the same time, SGI attained a CDRS number of only 48.63 on a system costing nearly three tjmes as much.
Figure 3 shows the relative performance of the
PowerStorm 4D60T for fou r of the major Viewperf benchmarks. The viewsets are based on the following applications: CDRS, a computer-ajded industrial design package tfom PTC; Data Explorer ( DX), a scientific visualization package from IBM; DesignReview ( D RV), a model review package from Into·graph ; Advanced
Visualizer, a 3-D animation system from Alias/
Wavefront (AWadvs) .
The PowerStorm 4D60T mid-range graphics adapter easily outperformed the Indigo2 High IMPACT system from SGI by a wide margin and even surpassed SGI's more expensive graphics card, the Ind igo2 Maxi mum IMPACT, by a factor of more than 2 : 1 in price/ performance on these benchmarks. Figure 4 shows that the PowerStorm 4D60T was the performance leader in three of the four benchmarks. SGI has yet to produce a graphics product in this price range that outpertonns the PowerStorm 4D60T.
Acknowledgments
The authors would like to acknowledge the many other engineers vvho made the PowerStorm 4DT proj ect a succcssfi.1 l one, incl uding Monty Brandenburg,
Shih-Tang Cheng, Bill Clifford, John Ford, Chris
Kniker, Jim Rees, S huhua Shen, Shree Sridbaran,
0
(f) a:
0..
9,000
8,000
7. 000 w
6,000
::::;
<{ 5,000 a: w
0..
(f) a:
::5
_j
0
4,000
3,000
2,000
1 ,000
0
CDRS
KEY
•
POWERSTOR M 4D60T
0
SGI MAX I M U M I M PACT
D
SGI H I G H I M PACT
DX
DRV
Figure
3
Price/Performance Comparison of Graphics Adapters on View perf Benchmarks
60
AWADVS
�
20
<{ a: u..
CDRS
KEY:
•
SGI MAXI M U M I M PACT
[]I
SGI H I G H I M PACT
D
POWERSTORM 4D60T
DX
Fig u re 4
Performance of Graphics Adapters on ViewpcrfBench marks
58
Digital Tccbnical journal Vol . 9 No. 4 1997
DRV AWADVS
Bruce Stochvell, and Mark Yeager. We would also like to than k the Graphics Quality Assurance Group and the Workstation Application Benchmarking Group for their unending patience and cooperation.
References
1 . M. Segal and
K.
Ak e l ey, Tbe OpenG"L Graphics System.· A
Specijlcation ( M o u nt a i n Vi ew, Cal if. : Si l icon Graphics,
Inc . , 1 995 ) .
2 . T h e
OpenGL Performance Characterization Pro je ct
, h ttp:/ jwww.specbench .org/gpc/opc.static.
3. R.
Scheifler and J.
G
D ig it al Press, 1 992 ) . e t tys
,
X V?indow System ( B oston :
4. H . Gajewska, M . Manasse, and
J .
McCormack, "Why X
Is Not Our Ideal Window System," Software Practice and t.:Xperience ( October 1 99 0 ) .
5 . !VI . Kilgard, " D 1 1 : A Hig h Performance, Protocol
O p t i o n a l
,
Transport- Optional Window System with
X 1 1 Compatibility and Semantics," 77Je Ninth A nnual
X Technical
C on f e r e nc e,
Boston, Mass. ( 1995 ).
6. R. Sites and R. Witek, Alpha AX.P
Architecture Refer ence Manuai ( B oston: Digital Press, 1 995).
7.
J . Anderson et al . , "Continuous Profiling: Where Have AJJ t he Cycles Gone?" The 16th
A CM
Symposium on Ope r at
ing
Systems Principles, Sr. Malo, France (
1997): 1-14.
General References
J . Foley, A. van Dam, S. Feiner, and
J.
H u gh e s , Computer
Graphics Principles and Pmclice (Reading, Mass.: Addison
Wes l e y
,
1993).
M. Woo,
J.
N e i d er, and T. Davis, OpenGL Prog r am ming
Guide (Reading, Mass . : Addison -Wesl ey, 1 99 7 ) .
Biographies
Thomas Frisinger
Tom Frisinger w a s a senior software engineer i n the
Workstation Graphics Group at DIG ITAL for three years.
During that time, he contributed to nearly all aspects of the
PowerStorm 4DT project. As a member of the core soft ware engineering team, he helped develop software for the
4D40T, 4D50T, and 4D60T graphics adapters as we l l as the 4 D3 0 T and 4D5 1 T m o d el s. He was also part of the core software d esign team for the 4 D31 T gr ap h i cs acceler ator. Tom is currently doing research and development i n
PC graphics tor AT! Research , I nc.
Karen L. Bircsak
As one of the developers of the PowerStorm
4DT graphics adapter, Karen Bircsak designed and implemented enhancements to the
X li brarv and contri buted to other sof t w a re development areas.
A principal software engineer in the Workstations Graphics Group, Karen is c u r re ntly working on supporting new graphics hardware. Prior to joining D IGITAL in 1995, she held software engineering positions at Concurrent Computer Corporation and Data
General Corporation. She earned a B .S. in computer sci ence and engineering from the University of Pennsylvania in 1 984 and an M.S. in computer science from Boston
U ni v er s it y in 1 990.
Benjamin
N.
Lipchak
Benjamin Lipchak joined DIGITAL in 1995 to develop softw,He for the PowerStorm 4DT graphics adapter and later developed 3-D software for the PowerStorm 4D30T pro j e c t
.
A senior software eng in e er in the Workstation
Graphics G r oup
, he is cu r ren tly le a d i ng the software effort of a n e w graphics p r o j ect
. Benjamin received
B.S. ( highest honors) and M .S . d e g r ee s in computer science from
Worcester Polytechnic Institute. He is the recipient of the
Sa l i sbu r y
Award in Computer
Sc ien c e
.
Digit3l Technical Journal
Vol. 9 No. 4 1 997 59
Keith
L.
Comeford
Keith Comeford is a principal software engineer in the
Workstation Graphics Development Group. He is currently working on the next generation of graphics cards and accel erators for DIGITAL. Keith was the project leader for the
Windows NT cLivers for tl1e PowerStorm 4D40T/50T /60T graphics cards. I n previous project work, Keith contributed significantly to the G KS and PH IGS implementations in a variety of capacities from developer to project leader tor more man lO years. Keith joined DIGITAL in 1983 after receiving a B.S. in computer science from \Vorcester
Polytechnic Institute .
Michael I. Rosenblum
Mike Rosenblum is a consulting software engineer at
DIGITAL and the technical director for the Workstations
Business Segment Graphics Group. He was the project leader and architect of the PowerStorm 4DT series and implemented some of its 2-D DDX code. Currently, he is managing two graphics projects and consulting to the company on graphics-related issues. Mike joined D I GITAL in 1 98 1 , to work on the terminal driver in the VMS
Engineering Grou p. Later he helped design the company's first workstations. He has a B.S. in computer science from
Worcester Polytechnic I nstitute and is a member oftlhe
ACM .
60 Digital Technical Journal
Vol.
9
No. 4 1 997
DART: Fast Application level Networking via
Data-copy Avoidance
I
Robert J. Walsh
The goal of DART is to effectively del iver high bandwidth performa nce to the appl ication, without a change to the operating system cal l semantics. The DART project was started soon after the first DART switch was completed, and also soon after line-rate commun ication over
DART was achieved. In looking forward to giga bit class networks as the next hurdle to conquer, we foresaw a need for an integrated ha rdware software project that addressed fundamenta l memory bandwidth bottleneck issues through a system-level perspective.
The Ethernet supported large 1 00-node networks in
1 976. ' By 1 9 8 5 , 10 Mb/s Ethernet had been available for a while, even for PCs. However, high-performance hardware and software lagged, due to system bottle necks above the physical layer. The premier i mplemen tations for
UNIX were achieving only 800 kb/s ( 8 % of
10 Mb/s) in benchmark scenarios on common system platforms of the day.2
The deployment of lOO Mb/s fiber distributed data interface (FDDI) provided an order of magnitude bandwidth increase in the link speed around 1987.
However, the end system could not saturate the link on generally available machines and operating systems u ntil 1 993! when Transmission Control Protocol
(TCP ) improvements and a CPU capable of 400 mil lion operations per second became available:' Once again , high-performance hardware and software lagged the potential provided by the physical layer.
The current technological approach is switching.
Gigabit-class links and adapters, such as 622 Mb/s asynchronous transfer mode (ATM ), are becoming available. Since ATM links are dedicated point-to point connections, the use of 622 Mb/s in switch-to switch links and at the periphery implies that one ought to be able to move data at gigabit rates.
Switched capacity promises a lot to servers; how ever, mainstream systems are not currently capable of effectively using the bandwidth. The DART project attempts to avoid tl1e Ethernet and FDDI scenarios where end-system performance lags physical-layer potential .
One of the early goals was to go beyond simple benchmark scenarios where line rate communication connects a phony bit source to a phony bit sink, with the CPU saturated. The context for the work was to connect two applications at high speed, leaving CPU
©
1 997
IEEE.
Reprinted, with permission, from
IEEE Network,
July/August 1997, pages 28-38.
'The
TCP improvements included a small architectural update, the window scaling extension, to abstractly support the advertise ment of more than 64 kbytes of receive bu!kring. The rest of the improvements derived from implementation efforts to increase the actual buffering allocated to advertised
TCP windows, and to improve the segmentation of the
TCP byte stream into packets.
Digital Technical Journal
Vol. 9 No. 4 1997
61
resources avail able to execute the applications. In the past, the CPU had been saturated in Ethernet and
FDDI q uests for line rate communication.
Layering
The motivation for DART arises from the speci fic lay ering and abstraction used in BSD-derived UNIX sys tems, but the context is sufficiently general that the problem and sol ution have wide appl icability. Since various layers with i n system software will be refer enced repeatedly, we introduce them using Figure l .
The application generates and consu mes data . It tells the operating system which data to commu nicate when, by using read and write system calls.
The socket layer moves data between the operating system and the appJication . It a lso synchronizes the application with the networking protocols based on data and buffer availability.
The
!rampart protocol layer provides a connection to the remote peer. In the case ofTCP, the connection is a reliable byte stream . TCP takes on the responsibil ity of retransmitting lost or corrupted data, and of ignoring reception of retransmitted data that was pre viousJy received .
The network protocol layer provides an abstract address and path to the remote host. It hides the vari ous hardware-specific addresses used by the various media in existence. In the case of I P, fragmentation allows messages to traverse media which have different frame sizes.
A conventional driver layer moves data betvveen the network and tl1e system. It uses buffers and data struc tures whose representation percolates throughout all the operating system n etworking layers.
The DART Concept
DART i ncreases network throughput and decreases system overheads, while preserving current system call semantics. T he core approach is data copy avoidance, to better uti l ize memory bandwidth.
Memory bandwidth is a scarce resource that must not be squandered. I n DIG ITAL's transition from
MIPS processor systems to AJpha processor systems,
CPU performance increased more rapidly than main memory bandwidth. It took approximately 340 f.LS to move 4500 bytes on the MIPS-based DECstation
5000/200, and approximately 200 f.LS on the AJpha based DEC 3000/500. In the same time, the fixed per-packet costs were reduced by a tactor of three or more. (General trends are also stated i n Reference 4 . )
O n e breakdown o f networking costs is reported in
Reference 5 . The variable per-byte costs reported there are all associated with memory bandwidth, which is improving slowly. The fixed per-packet costs in the driver, protocol, and operati ng system overhead are aJ I general ly associated with the CPU , which is improving rapidly. Thus, we focus on the per- byte memory bandwidth issues as those most needing architectural improvement.
A traditional system fol lows the networki ng subsys tem model implemented within the BSD releases of
U N I X, shown in Figure 2 . An application uses the
CPU to create data
(
1
), the socket portion of the sys tem ca ll interface copies the data into operating system buffers (2 and 3 ), the network transport protocol checksums the data for error detection pu rposes ( 4 ), and the device driver uses programmed input/output
(1/0) or direct memory access ( DMA) to move the data to the network ( 5 ). Graphs showing the domi nant costs of checksumming and kernel buffer copies are presented i n Reterence 6 .
These five memory operations are a profligate waste of memory bandwidth. A system with a 300 M byte/s memory system would achieve at most 300*8/5
=
480 Mb/s I/0 rates. The system wou ld be saturated .
The DART model is shown in Figure 3 . The DART model is that data is created
( l ) and sent ( 2 ). Two memory operations make efficient use of the memory bandwidth .
APPLICATION
SOCKET
SYSTEM
TRANSPORT PROTOCOL
DRIVER
L__
_ _
_
_JI .
:_
!"----_
:
-L �
-
M
E
M o
_
__J r--��e���'0"AL
Figu re 2
BSD Copy-based Arch itecture
Fig u re 1
Software Layering
62 Digital
Technical Journal
Figure 3
DART
Zero-copy Architecture
Vol .
9 No. 4 1997
Squandering of memory bandwidth is avoided. A system with a 300-Mbyte/s memory system would encounter the larger bound of300*8/2
=
1 200 Mb/s for I/0 rates. Resources are available for the applica tion even when running at line rate.b
To support the DART concept, we need a system perspective that integrates the hardware and software changes implied by the DART model. Hardware is responsible for checksumming instead of software.
Hardware is solely responsible for data movement, instead of redundant actions by both hardware and software. These hardware changes are bounded and generic.
Operating system software retains the application interface and general coding of the BSD UNIX imple mentatjon. Extensive changes are unnecessary, since the focus is the core Jines that represent data move ment consumption of memory bandwidth. Extensive changes are also undesirable, since there is a large base of software written to the current properties of the
BSD networking su bsystem.
The DART Hardware
The first implementation of the DART concept is a high -performance 622-Mb/s ATM network adapter for the
PCI
bus called DART. DART's design reflects an awareness of the i nteractions of the components of the system in which it is placed . The
PCI
bus, main memory, cache, and system software can all be used efficien tJ y.
Store-and-Forward Buffering and DMA
DART is an adapter that connects a gigabit-class net work to a gigabit-class I/0 bus, and is appropriate for systems with gigabit-class memory systems. DART is focused on tl1e server market where a slight increase i n adapter cost can b e acceptable if the system perfor mance is significantly improved , since main memory and other costs dominate the cost of the DART adapter.
DART alleviates main memory bottlenecks through a store-and-forward design, as shown in Figure 4.
Traditional networkjng software subsystems and appli cations perform at least five memory operations to cre ate, copy, checksum, and communicate data. DART's
exposed buffering
allows data to be created and com municated with j ust two main memory operations.
The adapter memory is a resource that can be better utilized by exposing it to the operating system, and better performance results as weJ I. Tills is similar to the exposure of the CPU-internal mechanism i n tl1e CISC
RISC ( complex to reduced instruction set) transition.
DART
contains a n umber of features to make the store-ctnd:forward design effective.
DART's bus mas ter and receiver summarize network transport proto col checksums for software. DART's bus master provides byte-level scatter-gather data movement to support communication out of application buffers, not just operating system buffers. DART provides packet headers for software to parse so that software can direct the bus master to place received data i n the application's buffers when the application desires, without operating system copy overhead.
Buffering Design
An
ATM segmentation and reassem bly ( SAR) chip accesses virtual circuit state for each cell, and operates on 48-byte cell payloads. The pay load naturally corresponds to a burst-mode operation, leading to the use of synchronous dynamic DRAM
(SDRAM) to buffer cells. The circuit state is generally smaller and randomly accessed, leading to the use of static RAM (SRAM ) for control i nformation. Dividing the data storage architecture into two parts allows the interface designs to be tailored to the characteristics of the data type in question.
The DART prototype uses 1 6 Mbytes of SDR.AM for the data memory. The prototype uses
1
M byte of
SRAM for tl1e control memory. The SDRAM supports hardware-generated transmissions, aggregation of data for efficient PCI and host memory interactions; and buffering for received data u n61 the application i ndicates the proper destination for it. The SRAM con tains the SAR intermediate state; with a large number of virtual circuits and ATM's interleaving of packet contents, there is too much state to be recorded on crup at this time.
Packet Summarization for Software
The receiver parses the cells for the various packets which are interleaved on tl1e network connection, and reassembles the cells into packets. Once all the cells composing a packet have been received, a packet descriptor is prepended to the packet. The descriptor contains length, circuit number, checksum, and all other information that the driver may need to parse and process the packet.
Upon packet reassembly, a hardware-initiated DMA operation moves software-configured amounts of descriptor and packet contents to host memory. When lThc 1 200-Mb/s tigure i ncludes the cost of having the application write tJ1e data to memory. Some memory bandwidth might be consumed to fill the CPU's cache in order to execute the applica tion and opet·ating system . ln this scenario, if non-network band width is greater rhan 300*8 - 2* 1 000
�
400 Mb/s, data production would be rhc bottleneck and the network would run at Jess than line rate.
This is beneficial; the bottleneck h as been moved to the app l ica t i on .
'Some adapters segment (or re as s emb le
) from host memory, leading to
48-byte pa)'load transactions with host memory.
Transaction size should be an integraJ multiple of the cache block size, and should be a l i gn e d, in order to avoid wasting system bandwidth.
Digital Technical
Journal
Vol.
9
No. 4
1 997 63
CONTROL
�==�--------�-----.-----L--------�K===:>
TRANSMIT
INTERFACE
RECEIVE
DATA
MEMORY
Figure 4
DART
Bl ock
Diagram properly configured, the hardware provides the net work and transport headers, aJiowing software to determine where to place the packet data. Software data copies are avoided by allowing software to initiate a DMA operation to move the data to its final application desired location, rather than to some expedient, but inefficient, operating system buffer.
Receive Buffering
DART's store-and-forward receive bu ffers are divided into t\vo classes. The per-circuit class guarantees each circuit forward progress. Each circuit is individually allocated some bu ffers in which to store cells. No other circuit can prevent data from passing through such buffers. The shared class is pref erentially used, and avoids resource fragmentation problems. Any circuit can consume a shared buffer for an incoming cell.
Since software specifies when and where to store packet data, adapter bu ffers are recycled when soft ware decides to do so, and not independently by hard ware . Part of a packet may be stored in application buffers at one time, and other parts of the same packet may be stored in application buffers at later times.
Hardware cannot assume a one-to-one correspon dence bet\Veen receive DMA and complete packet consumption.
Flow control occurs in the socket layer based on transmit buffer availability, in tl1e transport layer based on remote receive buffer availability, in the driver based on adapter resource availability, and in the ATM layer based on cell buffer availability within the net work. Credit-based flow-control protocols for ATM are based on the source of a cell stream on a link decreasing a counter (quota) when a cell is sent, and increasing a counter when a credit is received 7 The decrement represents buffer consumption at the next hop. The credit advertises buffer availability to the source; the next hop has forwarded a cell and thus freed a bu ffer."
JF o rwarding the ccU is required for (per-circuit) butTers of which d1e rransmitter on the
Link was made aware du1ing link initialization.
The receiver on d1e Link can g e ne r a t e crcdirs immediately for (slured ) buffers hidden from the rransmitter during
Link initialization.
64 Digital Technical journal
Vol.
9 1o. 4 1 997
With FLOWmaster, the credit is conveyed across the link to the source of the cell stream by overlaying the virtual path identifier (VPI) field with me circuit to cred it. This is a nonstandard optional use of the ATM cell header. Quantu m Flow Control is a credit- based flow-control protocol for ATM tlut batches the credits into cells instead of overlaying the VPI field.
Since credit- based flow-control is based on buffer availability, credits advertising free buffers can poten tially be held up by software actions. The shared class allows immediate credit advertisement, and best enables line rate communication. The per-circuit class involves software packet processing in the credit advertisement latency. To advertise a credit for a cir cuit whose per-circuit quota is exhausted, either the circuit must recycle an adapter-buffered packet, or any circuit must recycle a shared-class, adapter-buffered packet.
A minimal memory that constantly ran out of per circuit buffers and flow-controlled the source would exhibit poor performance. DART uses a large data memory. Advertising (shared) buffers via credits keeps the data flowing through the overall net\vork and sys tems with high performance.
Transmit Buffering
Software performs all transmit bu ffer management. Software creates a free buffer list of its own design, allocates buffers from the list to hold packet data, and recycles buffers after observing packet completion events. Software makes the trade-off between large efficient bu ffers which may be incom pletely filled, and small buffers which waste less stor age but incur increased allocation, free, DMA specification, and transmit description overheads.
Peer-to-Peer VO
DART avoids system resource consumption in server designs by supporting peer-to-peer I/0. A traditional server would consume PCI bus and main memory bandwidth t\Vice by using main memory as tl1e store and- forward resource bet\Veen t\vo I/0 devices, as shown in Figure 5. The PCI bus is consumed during steps 2 and 5 . The main memory is consu med during
Figure 5
Traditional Server Architecture steps 3 and 4. On some systems, 1/0 operations com pete for cache cycles during steps 3 and 4, whether the cache is external to or internal to the CPU. Such resource consumption can cause the CPU to staJ I even though the CPU will never examine such data.
DART allows a single PCI bus transaction to move the data, as shown in Figure 6. This also avoids any main memory bandwidth consumption when a bridge isolates the PCI I/0 bus from the main system bus.
The cache is not consumed with nuisance coherence loads for data the CPU will never examine, and the
CPU does not have to contend with I/0 for cache or main memory cycles.
For peer-to-peer I/0 over DART, the CPU is only involved in initiating packet transmission. This is a rel atively smaJl burden, since only a l ittle bit of control information needs to be computed and communi cated to tl1e adapter.
To enable efficient peer-to-peer I/0, DART includes a bus slave as well as a bus master.
The bus slave makes the internal resources of the adapter visi ble on the
PC!
bus
through DART's PCI configuration space base address registers. Therefore, on the PCI bus, the data memory looks like a linear contiguous region of memory, just as main memory does. The bus slave supports both read and write operations for these typically internaJ resources.
Figu re 6
DART Server Architecture
DART provides efficient hand ling of small packets.
Typically, describing a number of smaJI packets for transmission is onerous for software, limiting the peak packet rate. DART's transmitter can automatically subdivide a large amount of data into sma.l l packets, eliminating a lot of per-packet overhead. This feature is appropriate for a video server, whose software can not possibly fill the network pipe if it must operate on
8-ce l l packets.
PC/ Interface
DART supports both 64- and 32-bit variants of the
PC£ bus. The network interface and DART memories provide prodigious bandwidtl1 . To fully take advan tage of them, a 64- bit PCI bus is recommended, but
DART will also operate on a 32-bit PCI bus.
Bus Reads and Writes
The DART archi tecture sup ports memory write-and-invalidate hints to tl1e bridge between the system bus and the PCI I/0 bus. Such a h.int informs the bridge that the I/0 device is only writing complete cache blocks. There is no need for read-modify-write operations on main memory cache blocks in such circumstances.
Write operations within a system are general.ly buffered . A path from the origin of the write to the finaJ destination can be viewed as a sequence of seg ments. As data flows tl1rough each segment, each recipient accepts data witl1 the promise of completing me operation, allowing each source to free resources and proceed to new operations. Thus, write paths are generally not performance-limiting as long as there is sufficient buffering to accept burst operations. In the
DART context, me bridge between the system bus and the PCI I/0 bus accepts DART's writes and provides buffering for h igh throughput.
However, read operations are more problematic.
When memory locations are shared between CPUs, caches may or may not be kept coherent by hardware.
Here, me memory locations are shared between tl1e CPU and I/0 device, and mere is no coherence support. Each
DART read suffers a round-trip time tlu·ough me bridge to access tl1e main memory. DART addresses this latency tlu-ough large read transactions (up to 5 1 2 bytes).
As an example, consider a simplified 64-bit bus where 540 Mb/s of data are written in 64-byte bursts, reads suffer 1 5 stall cycles u nti l the data starts to stream , and writes require a staJl cycle for the target to recognize its address. Address and data are time multiplexed at 33 MHz. Then writes consume 540 *
( l
+
l
+
8 )/ 8
=
675 Mb/s of bus bandwidth. Reads have 33 * 8
*
8 - 675
=
1 437 M b/s of bus band width into which they must fit. Thus, the minimum burst length L required is 540 *
(
1
+
1 5
+
L)
=
L
$
1 437.
The burst must be at least 9 cycles, 72 bytes, i n the ideal case. DART's large read burst size compensates for overheads like large read latencies.
Digital Tedmical Journal
Vol . 9 No. 4 1 997
65
66
Importance of Bus Slave Interface
The bus master inter face is appropriate tor sofuvare-generated transmis sions. A packet created by an application in main memory can be moved via
DMA
to the network.
The bus slave interface is appropriate for hardware generated transmissions. Another IjO device which is designed to always be bus master, like a d isk i n terface, can move data directly to the
DART
without interme diate staging in a memory. Peer-to-peer I/0, however, was a by-product of other concerns.
Data transfer within TCP is based on a stream of large data packets flmving i n one direction, and a stream of sma l l acknowledgments flowing i n the opposite direction. Traffic analysis studies often find a mix of smal l er and larger packets. One of tbe early concerns for the
DART
project was to make transport protocol generation of acknowledgmen ts inexpensive by avoiding
DMA.
A small packet, constructed entirely by the CPU anyway, cou l d be moved to the I/0 device instead of to main memor y. This is fu ndamen tal l y a short sequence of write operations that could easily be butTered, allowing the
CPU to proceed i n par a l lel on other work.
DMA from an application buffer to a device inter face is genera l ly specified to hardware by stating the physical addresses of the application buffer in main memory.
DMA
requires a guarantee that the data is at the specified locations. If the virtual memory system were to migrate the data to disk and recycle the physi cal memory for some other use, the parallel DMA activity would move the wrong data. Therefore,
DMA
operations are surrounded by page lock and unlock calls to the virtual memory system, to i n form it tlut certain memory locations should not be migrated .
Additional concerns that led to i ncorporation of the bus slave interface were related to the cost of page locking, and the cost of acquiring and releasin g
DMA
resources (e.g., in the bridge ) . An acknowledgment might be constructed in nonpaged kernel memory, but a small application packet would l i kely be con structed in application memory subject to paging.
Even if page locks were cached for temporal locality, it might be cheaper to simply move the data via pro grammed I/0.
The break-even poin t betwee n grammed I/0
DMA
and pro is system-dependent, but can be mea sured at boot time in order to learn an appropriate threshold to use tor such a decision. Demands on the main memory system from i ts various clients wi l l change over time, a n d a single measurement is only optimal for the sample's conditions. The suggestion here is to enable a quick judgment in the software. The intent is to make large gai ns and avoid egregious per formance errors. 'vVe suspect that fine-tuni ng the deci sion is l ess important, and requires the coJiection of excessive information during the n ormal operation of the system.'
Vol . 9 No. 4
1 997
Interrupt Strategy
As noted above, on-chip access rates tor the CPU increase more quickly tlun off-chip access rates. Interrupt processing and context switching are fiwdamentally off-chip actions; new register values m ust be loaded into the CPU, and t11e cache must be ptimed with data. Thus, t11e general system trend is that i nterrupt processing and context switching improve more slowly than raw processing performance.
DART
provides a programmable interrupt holdoff mechan ism . B y delaying interrupts, events can be batched to red uce various system overheads. I f the batching mechanism were not present, an interrupt per packet wou ld swamp system software at gigabit rates.
Since the interrupt delay interval is programmable, sofuvare may use adaptive a lgorithms to decrease interrupt latency if the system is idle, or to i ncrease the amoun t of batchi ng if the system is busy. The delay timer starts decrementi ng as soon as it is writte n .
Typical ly, the timer will b e written at t h e e n d of the interrupt service rou tine.
Interrupts can be divided i nto two classes bv software. Each class has its own delay interval , i n case sofuvare assigns distinct importance o r latency requirements to the classes.
The Dart Software
DART
provides increased performance with the same system calls, and with the existing system call seman tics. The only change is to the underlying implementa tion of the existing system call semantics.
Unmodified
existing applications can consume giga bit network bandwidth. The appl ication can assist the system software by using large contiguous data buffers, but it is not required. System software can specify byte level scatterjgat11er operations to t11e
DART
adapter in order to access arbitrary application buffers.
Changes to the system sofuvare are confined to a few locations above the d river layer, a n d are generic.
Successive high-bandwidth adapters tor other media can be supported by just writing d rivers; no changes will be needed above the driver layer. The shared set of upper- layer software changes are only needed to take maximum advantage of a
DART
-style adapter; a tradi tiona! copy- based i mplementation i s supported b y the hardware .
··Given the parallel nature of the environment
(oth er
1/0, ca che operations, and multiprocessor CPUs), a software system could o nly estimate non- DART memory loads.
Queued DMA o p e r atio n s may stan bter than expected, or finish before t he ir completion has been noticed.
CPU cache activity is de p en d ent on the program cxccur.ing ,lt that moment; tine-tun.ing is problematic. The focus of
DART has been me large g ai ns
, like avoiding copies, or allowing either DMA or pro g ra m m e d ljO to be used .
The focus has been on rhc structure of me svstem.
We developed a prototype UNIX driver to test the upper-layer changes, and executed a modified kernel against a user-level behavioral model of a DART-style adapter. The code was subjected to constant back ground testing on a workstation relied on for daily use.
The prototype driver supports buffer descriptors refer encing either kernel buffers or adapter buffers. The implementation effort to support kernel-buffered packets was minimal , and enables mu ltiple protocol families to be layered above the driver.
The software changes modify the existing upper level software, rather than bypassing it via a collapsed socket, transport, network, and driver implementa tion. The current UNIX net\vorking subsystem pro vides a rich set of features that needs to be comp.letely supported for backward compatibility.
Transmit Overview
A comparison of traditional transmission with DART transmission is shown in Table l .
For a traditional adapter, the system call layer copies application data to operating system buffers. With a DART adapter, the data is copied to the adapter. Uiomove is the copy function typically used within UNIX. The DART mechanism is to use an indirect function call through a pointer, rather than a direct function call to an address specified by the compiler's linker. H igh-performance copy functions are associated with the device driver.
The driver's copy function is free to use DMA or pro grammed I/0, depending on the length of the copy.
For a trad itional adapter, software wastes machine resources computing checksums. With a DART adapter, the checksum is computed by hardware as the data flows into the adapter. The adapter can patch the checksum into the packet header. The adapter can also move checksum summaries back to host memory so that they are available for retransmission algorithms.
For a traditional adapter, the driver instigates addi tional memory references to copy the data to the adapter for transmission. With a DART adapter, the data is already on the adapter, ready to be sent! Much of the data copy avoidance work is throughput-related.
In this instance, we also create the potential for a latency advantage for the DART model, since the data copy overlapped work in the system call, transport, net work, and driver layers of the operating system .
Receive Overview
In many ways, d1e receive path for net\vorking is usu ally considered more complicated than the transmit path, since the various demultiplexing and lookup steps are based on fields that historically have been considered too large to use simple table indexing oper ations. Also, the receive path requires a rendezvous bet\�'een the transport protocol and the application ( to unblock the application process upon data arrival) . So it should come as a pleasant surprise that the DART style changes for packet reception can be as simple and localized as two conditionals in the socket layer and one in the network transport layer.
Table 2 is a comparison of traditional receive pro cessing with DART receive processing. It is almost identical to the packet transmission comparison. The distinction is which portion of the DART adapter computes the checksum on behalf of the software
( receiver i nstead of DMA engine).
Interrupts
Transmit completion interrupts do not need to be eagerly processed . Software can piggyback processing to reclaim transmit buffers upon depletion of transmit buffer resources, upon unrelated packet reception events (e.g., User Datagram Protocol, UDP), and upon related packet reception events (e.g., TCP acknowledgment). The transmit completion events can be masked, or the hardware interrupt holdoff mechanism can be used to give them a longer latency.
Receive interrupts are batched to reduce overheads.
Short packets are fully contained in the initial packet summary which would be deposited in a kernel buffer.
Adapter buffers for short packets can be recycled immediately by system software. Long packets are not fully contained in tl1e initial packet summary provided software for parsing and dispatch. The summary is noticed during one interrupt, and scatter/gather I/0 completion into application buffers is noticed during another interrupt if performed asynchronously.
The side-effect of the decision to create a store-and forward adapter is tlut a received packet is related to t\vo interrupts. The intent is not to burden a system and cause multiple interrupts per packet. The d istinc tion between relation and causality is important.
When the system is under load , there is a steady stream of packets, and tl1us a steady stream of batched
Table
1
Tra nsmit Overview
System call layer
Protocol layer
Driver layer
Traditional
U iomove user buffer to kernel buffer
For all buffers for a l l bytes, update checksu m
Progra mmed 1/0 o r DMA
DART
* U i omove user buffer to adapter buffer
For all buffers, update checksum
Data is a l ready on the adapter !
Digital Technical journal Vol.
9
No. 4
1 997
67
Table
2
Receive Overview
Driver layer
Protocol layer
System call layer
Traditional
Program med
1/0 or DMA
For all buffers for all bytes, update checksum
U iomove kernel buffer to user buffer
DART
Data stays on adapter!
Use checksum computed by receiver hardware as packet was reassembled
Uiomove adapter buffer to user buffer i nterrupts. If 3 M bytes were transferred using a burst of l - kbyte packets, there would be 3000 packets.
Batching 20 packets/interrupt, there would be 1 50 interrupts to report packet arrivals. The first interrupt is just for packet arrival events, to allow header parsing.
The intent is for the next 1 49 i nterrupts to report 20 new arrivals and the DMA completion for 20 previous arrivals. A final i nterrupt would take care of the fi nal
DMA requests. In this case, the additional i nterrupt load for a DART adapter is minor: one interrupt for
3000 packets. The interrupt load is not doubled (even if one chooses to move received data asynchronously).
Store-and-forward latency is incurred because of the memory write and read on tJ1e adapter (to store data from the network and to later move it to the application's buffers).
DART
adapter memory oper ates at a high rate, over 4 Gb/s, to minimize this. Due to me intervening software decision concerning where to place
DART
data for large packets, the data may be placed at its initial location in host memory later than for a tradi tional adapter which fills kernel buffers.
However, store-and-forward reduces main memory bandwidth consumption, and quickly places the data at its
final
l ocation within the application buffers in host memory. The correct metric is latency to data availability to the application, not data latency to first reaching tJ1e system bus.
CSR Operations
Control and status registers ( CSRs) are used within hardware implementations to al low software to con trol the action of hardware, and for hardware to pre sent information to software. For example, a CSR can inform a device of me device's address on a bus.
In mis case, the CSR's definition is generic in the context of the bus definition. Alternatively, a CSR can be used to initialize a state machine within the hardware imple mentation. In mat case, me CSR's definition is specifi c to that version o f the device.
CSR reads are very expensive. Generally, a single CSR read is required for DART interrupt processing, and mat CSR is placed in the PCI clock domain of
DART in order to avoid operation retries on me PCI bus.
Most packet processing information is written to host memory by the adapter for quick and easy CPU access. For example, packet summaries are placed in one or more arrays i n host memory, and software can use an ownership bit in each array element to termi nate processing of such an array.
CSR writes are buffered; nevertheless, they can be minimized . The packet summaries in host memory are managed with a single-producer, single-consumer model. When tJ1e consumer and producer indices into an array are equal, the array is empty. When hardware's producer index is greater, mere are entries to be processed by software. ( Redundant information in array element ownership bits means that soft\vare does not actually need to read tJ1e
DART
adapter to perform the producer-consumer comparison. ) When the hard ware's producer index reaches the sofnva.re's consumer index minus one, the array is fu lly utilized . When soft ware has processed a number of packet summa.ties, the hardware ca.t1 be informed that they can be recycled by a single write of the consumer index to the adapter.
The
DMA
engine processes a Jist of "copy this from h ere to there" commands. By supporting a l ist of operations i nstead of a single operation, software can quickly queue an operation and move along to its next action without a lot of overhead. The copy commands reside in a.tl array witJ1in host memory, witJ1 a software specified base and a software-specified length.
DMA comma.t1ds also follow the producer-consumer model . However, since i nstructions are only read by
DART, there are no ownership-bit optimizations. To compensate for this, software can allocate a large array and cache a pessimistic value for the hardware's con sumer index in order to avoid CSR reads. Alternatively, the
DMA engine could petiodically be given instruc tions to
DMA such information to host memory.
A typical
DART
interrupt involves one CSR read and tJ1ree CSR writes, yielding an efficient interface. One read determines interrupt cause. One write informs the
DMA
engine of new copy commands for newly received data. AnotJ1er write informs the
DMA
engine that the
CPU processed a number of me packet summaries
DART
placed in main memory. A tJlird write initializes tJ1e interrupt delay register to batch future events.
Occasionally, an interrupt also involves an extra CSR read. The read discovers a large number of commands processed by tJ1e
DMA
engine, a!Jowing software to recycle entries i n the command queue and thereby issue more commands.
68 Digital Technical Journal Vol. 9 No 4 1997
Driver
The driver classifies received packets, and decides whether to continue to use adapter buffers for them, or to copy the data into kernel buffers. For the proto type, adapter-buffered packets are:
•
•
•
Long enough to contain maxi mal-length IP and transport protocol headers.
Version 4 IP packets ( buffering assumptions perco late throughout the layers of the system, so a proto col family must be updated and tested to support adapter-buffered packets).
TCP or UDP protocol packets. Other protocols lay ered over IP do not use adapter buffers, to make the scope of the effort manageable by handling just the common case.
The operating system uses a single mbufto describe a single set of contiguous bytes in a buffer which may be within or external to the mbuf structure. Mbufs can be placed in l ists to form packets from a number of noncontiguous buffers.
Received adapter-buffered packets are two mbufs long. The first mbuf contains the initial contents of the packet DMAed into memory by the adapter, that is the protocol headers and summary information from the adapter.
The second mbuf refers to the packet in adapter memory. For ATM, the received packet is stored in a linked list of buffers on the adapter. Programmed I/0 access to the buffers requires software to traverse the links, but this would not be done in practice since the
CPU read path to the I/0 device is unbuffered and high-latency. The DART DMA hardware would be used, and it wou ld traverse the links as-needed . The
DMA hardware allows the software to pretend the packet is contiguous.
Fields of the second mbuf are used in specific ways.
The length of the second mbuf does not contain the initial portion of the packet copied into the first mbuf, even though the adapter memory buffers tbe entire packet. The initial portion is replicated, but only the copy local to the CPU is accessed . The pointers of the second mbuf point to bogus virtual addresses, even though the adapter looks l ike an extension of main memory. This speeds software debugging by trapping inefftcient accesses to the adapter. Adjusting the length and pointer fields is still allowed in order to drop data from the front or back of the mbuf. The m_ext fields record the location and amount of adapter buffering used to hold the packet. They also point to a driver-specific buffer reclamation routine.
For TCP, or for UDP packets with nonzero check sums, the driver makes incremental modifications to the DART receive hardware's checksum. The hard ware computes the
1
's complement checksum over all the cell payloads except for the final ATM trailer bytes.
As a result, the driver modifies the hardware checksum to account for:
•
•
•
Contributions made by IP options
Construction of the pseudo-header which is not transmitted on the network
The transport layer checksum, which was zero when the checksum was computed but may be nonzero on the network
To transmit a packet, the transport and nenvork lay ers operate on protocol headers in main memory. The driver moves the headers to the adapter as part of transmitting a packet whose encapsulated data is in adapter buffers.
The if n et structure is the interface between the pro tocol layers and the driver. It contains, for example, fields expressing the maximum packet size on the directly connected network, the nenvork-layer address of the i nterface, and function poi nters used to enter the driver.
We add an (*if_ uiomove)() field to be associated with buffers as described below. It represents a driver entry to copy data to or from the adapter. We also add an (*if_ xmtbufalloc)() field to be used within the mbuf allocation loop of the transmit portion of the socket layer. This allows the socket layer to give prece dence to allocating (large ) adapter buffers over main memory buffers.
The driver always retains some transmit adapter buffers for its own use. When the system is busy, there will be TCP packets consuming adapter buffers. The packets are associated with the socket send queue.
There wiJJ also be packets on the interface send queue, which may or may not use adapter buffers. If tbe fi rst item on the interface queue uses just kernel buffers, then the driver must have reserved adapter buffers in order to complete the transmission and avoid transmit deadlock. At least one packet of adapter buffering must be reserved for the driver output routine.
UDP
UDP motivates many of the changes without getting involved in the complexity of retransmission and relia bility. Many of these changes are generic to UDP and
TCP: augmenting the buffer and interface descrip tions, discovering the availability of efficient buffers for a connection, and allocating and filling the efficient buffers.
One portion ofthe mbufis the stntctpkthdr, which is used only in the first mbuf of a packet. It summarizes interesting information about the packet, like i ts total length.
We add a protocolSum field to the pkthdr of the mbufso that the driver can com municate the received transport-layer checksum to the upper layers. The transport-layer checksum is not ignored, as it would
Digital
Technical
Journal Vol . 9 No. 4 1 997 69
be if checksums were negotiated away or cavalierly disregarded . The checksum is verified by the trans port layer as usual, but without accessing all the bytes of the packet. The protocolSum field is valid if an
J\ll_PR07DCOL_SUM bit is set in the mbuf m_flags field .
Another portion of the mbuf is the
struct m_ext,
which is used to describe data buffers external to the m buf structure. We add an
(*u iomove_ j)()
field so that the driver can communicate a buffer- or driver specific copy routine to the socket layer. Socket layer usage of the standard pre-existing uiomove routine assumes that the received data is in the address space and should be moved by CPU byte-copying. The indi rection al lows the data to be moved by programmed
IjO or D MA . The uiomove_f field is valid if an
M_UIOMOVE bit is set in the mbuf m_flags field.
Parameters to the uiomove_f function are an mbuf� an offset into the packet at which to start copying bytes, a number of bytes to copy, and the standard uio struc ture that describes where the application wants the data.
The UDP input routine performs protocol process ing on received UDP packets. Before the pseudo header is constructed for checksum verification , the
M_PROTOCOL_SUM bit is tested in order to skip
CPU-based checksumming. i f ( m->m_f l a g s
&
M_PROTOCOL_SU M )
{
N E T I O_COUNT ( rch_hw_sum ) ; a s se rt ( m->m_f l a g s
&
M_PKTHD R ) ; i f ( u i ->u i_sum
1 = m->m_p k t hd r . p rotoco L Sum )
N E T I O_COUNT ( r ch_hw_sum_bad ) ; goto badsum;
}
} goto ok;
{
Error processing ca.n be based on packets reformat ted into kernel buffers. The UDP output routine per forms protocol processing on transmitted UDP packets.
Checksum overhead avoidance is similar to the receive path; but instead of testing the M_PROTOCOL_SUM bit, the mbuf checksum field is assumed to be valid for all transmit mbufs referencing adapter buffers ( they have the M_UIOMOVE bit set ) . We assume that no adapter which saves the operating system the effort of data copying would forget to save the operating sys tem the etTort of checksumming. It does not make sense to eliminate some, but not all, of the per-byte overhead operations.
For UDP transmission, software recycles ( adapter) buffering after the packet has been transmitted.
Changes like checksum avoidance are based on adding a conditional to the existing code paths. For a
DART adapter, the test and branch penalty are small relative to the gain . For large external buffers, there are one or two M_PROTOCOL_ SUM tests per packet, depending on packet length and buffer size.
This could be viewed as a constant-time overhead .
70 Digital Technical Journal
Vol. 9
No. 4
1 997
The gain is avoiding the linear-time access of each byte within each packet.
For a traditional adapter, the test and branch repre sent overhead tor each packet. The cost of the added conditionals occurs in the context of a large code base between the system call interface and the driver, and that networking code provides a rich feature set through the use of condjtionals. If the added condi tionals are viewed as sigruficant, consider the approach of generating two binary files from a single source module. To avoid penal izing systems populated solely with traditional adapters, operating system software configuration procedures can choose not to incorpo rate the DART-conditionalized version of the code. A
DART adapter installed at a later date would still oper ate under such a software configuration, but would not reach its peal< performance until the software is recon figured to use the DART-conrutionalized version.
TCP
The TCP input routine performs protocol processing on received TCP packets. Before the pseudo-header is constructed for checksum verification, the M_PRO
TOCOL_SUM bit is tested in order to skip CPU-based checksumming. The only differences with the UDP input processing change are the names of the TCP header structure and TCP header checksum field.
All the adapter resources represented by the second mbuf of a received packet are consumed until the final reference to the packet is freed. If large packets are exchanged and the application is doing small reads, not until the final read is any storage reclaimed. Trus space consumption is represented on the socket receive queue, and therefore affects the advertised
TCP window.
The TCP output routine performs protocol pro cessing on transmitted TCP packets. The checksum overhead avoidance is similar to that done for UDP.
Checksum computations for transport-layer retrans missions are simplified by the association of checksum contributions with mbufs, rather than an association of checksums witl1 packets. The association with buffers instead of packets also simplifies handling of packets using a mix of kernel and adapter buffers.
For TCP transmission, software recycles (adapter) buffering after the packet has been acknowledged by the remote end of the connection. Between transmis sion and acknowledgment, the data is held on the socket's send queue . Previously, the socket code copied data from one mbuf into another whenever both mbufs' contents fit into one, trading increased
CPU load for space efficiency. For DART adapters, the copy decision is cut short.
We add a hytesSummed field to the mbuf so that when a packet is transmitted or retransmitted by the transport layer, code can double-check that all the data the checksum is supposed to cover is still present in the
buffer. For example, a TCP acknowledgment of part of an original packet generally leads to the sender delet ing its copy of the acknowledged data retransmitting the rest. The software implementation handles the generality of acknowledgments which are not com plete transmit mbufs, the unit covered by the protocolSum field . A retransmission must not send a packet with an improper transport-layer checksum , even if it means using a n algorithm linear in the num ber of bytes remaining in the buffer to recompute the checksum.
The transmitter's socket layer buffers data in seg ments convenient for both the network-layer protocol and the driver. Checksum contributions remembered for retransmission are recorded at a similar level of granularity. The transmitter is l i beral in what the receiver can acknowledge; the receiver's i mplementa tion affects efficiency, but not correctness.
Socket Data Movement
The copy from the network buffers to the application data space occurs in the soreceive routine, which uses information left in the mbu f by the device driver. The call(s) to uiomove become conditionalized as follows: i f (m->m_f lags
&
M_U I OMOVE)
{ asse r t ( m->m_f lags
&
M_EXT ) ; error
} e l se
=
( *m->m_ext . ui omove_f ) (m, mof f , Len, u i o ) ; error = u i omov e ( mtod ( m, caddr_t )
+ mof f , Len, u i o ) ;
The reverse copy in sosend i s similar.
The standard uiomove fu nction makes the opti mistic assumption that the addresses of user buffers provided by the application are valid. I f addresses are not valid, a trap occurs and situation-specific code is called.
To support drivers that use programmed l/0 movements with the application's buffer, an additional code point is added to the error processing so that an
EFAULT error is returned to the application.
Note that the changes are generic, and can be used with existing devices. The uiomove_f function can per form both copies to kernel buffers and protocol check summing for transmission over traditional adapters.
In the transmit portion of the socket layer, the appli cation data is moved to kernel buffers or to adapter buffers by sosend. In order to take advantage of DART adapters, sosend needs to know:
•
•
That the protocol layers between the socket and driver support DART-style buffering
That the driver supports DART-style buffering
In general, formatting data efficiently for transmis sion can require knovving the amount of headers that will be prepended by the various layers below the socket layer, so device alignment restrictions can be met. Due to protocol options and to the variety of media in existence, the amount prepended may vary from socket to socket. Given a socket, we introduced a function that computes:
•
•
•
A function pointer for allocating adapter-based buffers
A function pointer for moving data from user buffers to adapter buffers
The number of bytes required to prepend all headers
To simplif)r the prototype implementation effort, the fu nction disallows the use of adapter bufters for IP multicast packets.
When allocating adapter buffers, sosend uses the if_xmtbufalloc entry to allocate adapter buffers. Each time it does so, it passes a maximum number of bytes of buffering that attempts to allocate a buffer for the entire ( remaining portion of the ) packet. The driver indicates the actual amount of buffering allocated; sosend loops until all the necessary buffering is allo cated. The driver may decline to allocate an adapter buffer if the requested amount of buffering is small. At that time the driver can best decide if CPU-based byte copying from user buffers to kernel buffers, and also copying kernel buffers to the adapter, is preferable to programmed IjO or DMA from user buffers.
Once an adapter buffer aJlocation fails, no further allocations are attempted within a segment that will be passed to the lower layers. This ensures that drivers will see, at worst, an (internal ) mbuf containing headers, one or many adapter buffers containing data, and potentially one or many kernel buffers containing the rest of the packet. This simplifies the driver, and ensures that alignment restrictions are met without shuffling data around on the adapter. It also simplifies transport-layer checksum computation algorithms.
There is an unusual boundary case in which a long segment of transmit data may not immediately be copied to adapter buffers, even though the driver would prefer to do so. If the driver has many free transmit adapter buffers when the socket code starts to prepare a segment, it may not have any free buffers when the segment nears completion. This is because the socket layer runs at a lower interrupt priority level than the device driver, and buffers are allocated indi vidually. A device interrupt can lead to servicing the device output queue, consuming adapter buffers in order to transmit traditional kernel-buffered packets.
Rather than block and wait for transmit adapter buffer availability, the prototype software uses kernel buffers.
Both the socket and network protocol (TCP) layers contain segmentation algorithms. In the socket layer, the segmentation process is confused with the (cluster mbuf) buffer choice decision procedure. As part of eliminating that confusion, we introduce an if_bujlen field to the ifnet structure.
Digital Technical Journal
Vol .
9
No. 4
1 997
7 1
If the socket layer creates segments longer than the device frame size, excess work occurs in the lower lay ers ( e.g., TCP segmentation or IP fragmentation). If the socket layer creates segments shorter than the device frame size, the system foregoes large packet efficiencies.
A large 8-kbyte write that leads to eight
1 kbyte cluster mbufs being i ndividually processed by the lower layers might benefit from overlapped IjO of the first segment with computation of the last, but the
CPU would be wasted for a benefit that is only rele vant when a large number of such poorly chosen seg ments arc constructed. Such a write could go out as a single packet over an ATM network.
Socket Buffering and Flow Control
A number of papers have commented on the require ment for a reasonable amount of socket buffering to enable applications to "fill the pipe" with a " band width times delay" amount of data . 1 Delay includes the link distance, device interrupt latency, software processing, and I/0 queuing delays. It also includes interrupt delays that aggregate events for efficient soft ware processing.
The requirement for sufficient socket buffering is a lesson learned over and over again. Traditional solu tions include marginal increases in systemwide defaults, and application modification to request more buffering than the default. Faci lities like rsh imply that anything can become a network application, unbe knownst to the application author; so changes to applications arc a poor solution. Also, applications are insulated from the network by the network protocol and socket abstractions; no application should need to know the buffering requirements for high throughput for the media
du jour.
We introduce an
(*if sockhuj)()
entry that allows the driver to increase socket buffering. When local network-layer addresses are bound to socket connec tions, an interface is associated with the connection, and the driver is allowed to adjust the socket buffer quota.
For TCP server connections, the server may not be restricting incoming connections to a particular inter face. Overriding the default buffering value must be done on the socket created when the incoming SYN arrives, not on the placeholder server socket. The buffer al location needs to be determined as soon as possible, because the initial SYN packet also triggers the determination of the proper window scaling value.
UDP docs not queue packets on the socket send queue. Although calls to
if_sockhuf
from the socket layer arc independent ofthc protocol, the buffer quota only affects the maximum UDP packet size sent, not the number ofUDP packets that can be in fl ight at the same time. The socket is not charged for UDP packets queued on the driver output q ueue or UDP packets i n the hardware transmit queues.
The adapter buffer resources are disti nct from main memory mbuf and cluster resources. The socket data structure and support routines support consumption and quota numbers for adapter buffering that are dis tinct from the current main memory consumption and quota numbers. For example, a connection re directed from a DART adapter to a traditional adapter is quickly flow-control led in the socket layer as a result.
The large adapter buffer allocation does not enable it to hog main memory buffers and adversely affect other connections.
IP
The prototype software contains conditionals to enable or disable the use of adapter buffers for mes sages undergoing IP fragmentation. This only affects
UDP, since the socket layer segments appropriately for the TCP and driver layers. Software computes the amount of header space for the first fragment, and also the amount of header space for the fol lowing frag ments (which wil l not contain transport protocol headers) . This information is used during the socket layer's movement of application data to kernel or adapter buffers. U D P and IP receive the segments as a single message; the I P fragmentation code uses the fragment boundaries precomputed in the socket l ayer.
IP reassembly of received adapter-buffered packets was i mplemented in the prototype code to keep up with a transmitter using adapter buffers for IP frag mentation. The driver adjusts the hardware-computed checksum to ignore the contribution to the hardware sum caused by the successive JP fragment headers, which arc not presented to the transport layer.
Resource Exhaustion
The h ardware provides a scalable data memory. The memory holds received data until the application accepts it, and transmits data until the acknowledg ment arrives. The prototype provides 16 M bytcs, which was considered a significant q uantity after examining network s ubsystem buffering at centralized servers for several large "campus" sites.
When adapter memory is scarce, it should be allo cated to connections whose current data flows arc high-bandwidth flows. Low-bandwidth connections, connections blocked by a closed remote window, and connections over extremely Joss-prone paths will not be significantly impacted by the copying overhead associated with the use of kernel butTers.
Data Relocation
Reformatting data from adapter buffers to kernel buffers allows existing code to be ignorant of adapter butTered data. Socket-based TCP communication can use adapter buffers for high throughput, and other
72
Digital Technical Journal Vol . 9 No. 4 1 997
protocol environments can simultaneously use the familiar kernel buffers. DART support can be phased in by protecting legacy code with a conditional reloca tion call before entering or queuing data to the legacy code. Cache fill operations should be targeted to main memory, not adapter memory, for best performance in legacy code.
Relocation is also appropriate for error handling and other rarely executed code paths. For example, a multi-homed host may lose TCP connectivity through the first-hop router associated with a DART link, and be forced to send packets over another link.
The new communication path could use any net\vork interface, DART or otherwise . The soft\vare needs to be able to handle the scenario where the new adapter, or some system resource, has a constraint preventing it from transmitting packets located in
DART memory.
We selected a lazy evaluation solution which assumes that data sent over an old route will be deliv ered and acknowledged. An eager solution would incur a large burst of data relocation when the new route takes precedence, with the disadvantages that the work would be wasted for data which is acknowl edged, and the burst of activity consumes resources and incurs increased latency for other activities.
For TCP connections marked as using adapter buffers, a driver entry through (*if_
pktok)()
al lows the driver to comment on each outgoing packet. This implies that the driver also comments on TCP retrans mission packets. The driver has a chance to dou ble check constraints and trigger data relocation, if necessary. Drivers not supporting
if_pktok
always trig ger data relocation, and also lead to unmarking the
TCP connection .
Comparison to Other Methods
Traditional adapters contain minimal onboard mem ory and hide their buffering from tl1e CPU. Unable to manage a traditional adapter's buffers, a copy of data must be kept in host memory until it is acknowledged in case it needs to be retransmitted.
We felt copy-on-write approaches to using a tradi tional adapter would be i nadequate due to book keeping overheads experienced by other projects.
Also, the application may commonly reuse the same application buffer before the transport protocol semantics allow. For an unmodified application, this wou ld lead to blocking the application, or i ncurring both copy-on-write and data copy overheads. All applications are network- based when one considers net\Vorked file systems and pipes to remote program invocations; architectures that require applications to be recoded to interact with page mapping schemes
( e . g . , 8) are inadequate. Another obj ection is that copy-on-write focuses on packet transmission, ignor ing packet reception.
When a write is performed by an application using
DART, the application blocks only long enough to buffer the data, as for a traditional adapter. The copy of the application's data on DART enables retransmis sion for reliable communication. The application is free to immediately dirty its write buffer, and no per formance impact is associated with that action.
Van Jacobson's W1TLES paper design uses the CPU to copy data to and from the adapter via progran1med
I/0.9 Reading the adapter is an expensive operation, and in practice would provide worse receive performance than even a traditional adapter. The Medusa design is a
W1TLES variant that uses programmed I/0 transmis sion and addresses the receive penalry vvith system block move resources for reception.10 The Afterburner design used the same approach, achieving 200 Mb/s.' The
W1TLES approach keeps tl1e packet in adapter memory until it is copied to the application buffer.
To minimize resource consumption, the checksum and copy loop are combined. This means that the TCP acknowledgment is deferred until the application con sumes the data, which might be much later than nec essary. Applications read data at a rate of their own choosing. Care must be taken that this deferral does not lead to TCP messages to the data source that cause unnecessary data retransmission.
Unlike W1TLES, DART supports DMA to and from the adapter. Software can use DMA where appropriate, intelligently balancing the costs of programmed I/0 and DMA.
S ince DART provides the IP checksum with the packet, the TCP acknowledgment can be sent as soon as the packet is reassembled and reported to the CPU.
The acknowledgment contents and transmission time are tradi tional BSD UNIX; it states that the data has been received, and the offered window reflects buffer consumption until the application receives the data at its leisure.
Adapters have been built that offload protocol pro cessing.1 However, the cost ofTCP processing is low, and such an architecture introduces message-passing overheads that counterbalance the offloaded protocol processing efficiencies. CPU execution rates are scal ing well. The issue to address is the main memory bandwidth bottl eneck. Also, it is expensive and diffi cult to create, maintain, and augment the firmware for such an adapter. The firmware is tied to a single adapter, and replicates work done within tl1e operating system that can be shared by a number of adapters.
DART provides assist via checksumming methods.
It does not attempt to offload network- or protocol layer processing.
Performance
The simulation environment used to debug and test the chip design was also used to extract performance
Digital Technical Journal
Vol .
9 No. 4 1 997
73
information. The chip model used to fabricate the part is connected to a PCI bus simulation, some generic bus master devices, and some generic bus slave devices. The simulation environment is connected to and controlled by a TCL-based environment.
Within the TCL environment, the hardware design ers wrote a device driver. With this driver, DART copied packets from host memory, looped packets on an external interface, reported packet summaries, and copied packets into host memory. Both 64- and 32-bit
PCI buses were exercised. Target read latency of host memory was incorporated into the simulation ( the data presented in Figure 7 is based on a 1 6- cycle latency) . Credit-based flow-control operations were enabled since they consume additional control mem ory bandwidth, and therefore represent worst-case scenario operation, Similarly, a large number of virtual circuits were used to loop data, to prevent the use of on-chip, cached circuit state.
Because the TCL driver was written by hardware designers, and they were focused on designing and testing the chip, performance numbers extracted from their work suffer from a lot of CSR accesses. A real driver wou ld reduce the CSR operations and have i ncreased batching of interrupts and other actions.
CSR reads are costly, since tl1ey involve a round-trip time within the chip which crosses clock boundaries, in addition to the round-trip time between the CPU and the pins on the device. Crossing clock boundaries means that there are internal first-in first-out (FIFO) delays involved to deal with synchronization and meta-stability issues. To meet PCI latency specifica tions, the bus master is told to retry such operations, freeing the PCI bus for other use during the internal round-trip time. CSR writes are efficient, since they are buffered throughout the levels of the system.
The dip i n Figure 7 is near the 5 12-byte burst size used to read from host memory. Packet transmissions no longer fit in a single DMA burst, and incur tl1e extra cost of an additional short fetch. This incurs additional overhead cycles to place the address on the bus and for tl1e target to start to respond with the first bytes.
For each simu lation we extract numerous detailed statistics. Table 3 contains a few for 32- cell packets
( 1 5 36 bytes) on a 32 bit PCI bus. These particular fig u res are for the TCL driver, and include time intervals to initialize the adapter, to transmit before the first packets are received, and to receive after the last packet was transmitted.
Figure
7
DART Performance
74 Digital Technical Journal
DART 4 O R MORE VC, B I D I RECTIONAL,
FLOW-CONTROLLED PERFORMANCE
40%
30%
20%
1 0%
90%
80%
70%
60%
50%
0 1 0
20 30 40
PACKET LENGTH IN CELLS
50
KEY:
-+--
PERCENT OF LINE RATE (64·BIT BUS)
-\}-
PERCENT OF L I N E RATE (32-BIT BUS)
Vol. 9 No. 4 1 997
60 70
Table 3
Examples of Additional Statistics
Control memory idle
Data memory idle
PCI busy (frame or i rdy asserted)
PCI transferri ng data (irdy and trdy asserted)
CSR operations share of bus operations
79%
48%
75%
60%
41 %
Future Work
Due to the large amount of on board buffering, we do not expect DART to encounter resource exhaustion issues. However, some work will be appropriate to determine the best solution should buffering require ments exceed the electrical capabilities of the high speed
SAR-SDRAM interface. Is it efficient to move unacknowledged data off the adapter so that new transmit data can be moved from user space to the adapter in the socket layer? Is it efficient to block in the socket layer, waiting for adapter buffers to be freed by a future, or arrived but unprocessed, acknowledg ment? Is it efficient to use conventional kernel buffers to transmit when the space allocated to DART-style transmissions is exhausted?
DART structures the system software so that the operating system does not examine the application's data, which should be private to the application any way. This separation of control operations (on head ers) from data operations (primarily movement) is a common theme i n embedded system design for bridges and routers. DART provides a generic struc ture that enables high-performance networking in a variety of systems.
With features like peer-to-peer
I/0, one can con ceive of a system with multiple gigabit linl<s, where the bottlenecks have shifted from the system sofuvare to the application or service. We think DART-style adapters will enable and accomplish the delivery of high-bandwidth service to the application.
Acknowledgments
Robert Walsh implemented the transmitter and PCI bus interface. Kent Springer implemented the receiver and packet reporting fimctions. Steve Glaser imple mented the DMA engine. Tom Hunt implemented the external control
RAM interface, the external data
RAM interface, and the board design. Robert Walsh developed the prototype
UNIX changes. Phil Pears,
Mark Mason, James Ma, and Ken-ichi Satoh provided significant assistance in placing and routing the
ASI C .
We also had assistance from Joe Todesca, Elias
Kazan, and Linda Strahle . Bob Thomas participated in the initial concept and design. KK. Ramakrishnan pro vided some information on networking performance.
References
1 . Metcalfe, "Co(llputer/Network Inrerface Design:
Lessons from Arpanet and Ethernet," JEEE}SAC, vol . l 1 , no. 2 (Feb. 1 993).
2 . Wa lsh and Gurwitz, "Converting the BBN TCP /IP to 4.2 BSD," USENIX
1984 Summer Conf Proc. (June
1 984).
3 . Chang et al . , "High-Performance TCP/IP and
UDP /IP Networking in DEC OSF / 1 for Alpha AXP,"
Digital Tecbnicaljournal, vol . 5 , no. 1 (Winter 1993).
4. Dalton et al. , "Afterburner," IEEE Network (July 1 993).
5 . Clark et a l . , "An Analysis of TCP Processing Over head," IEEE Commun. i\1/ag (June 1 989).
6. Kay and Pasquale, "Measurement, Analysis, and
I mprovement of U D P /IP Throughput for the
DECstation 5000," USENJX 1993 Winter Conf Proc.
( 1 993).
7 . Owicki, "AN2: Local Area Network and Distributed
System," Proc. 12tb Symp. Principles of Dist. Comp.
(Aug. 1 99 3 ) .
8 . Smith and Traw, "Giving Applications Access ro Gb/s
Networki ng," IEEE Network ( July 1 9 8 3 ) .
9 . Van Jacobson, "Efficient Protocol Implementation,"
ACM S IGCOMM 1 990 tutorial (Sept. 1 990).
10. Banks and Prudence, "A High- Performance Network
Architecture for a PA-RJSC Workstation ," JEEE.fSAC, vol. l l , no. 2 ( Feb. 1 993 ).
Additional Reading
1 . Kay and Pasquale, "The Importance ofNon-Data Touch ing Processing Overheads in TCP /IP," Proc. SIGCOlvf.Jv/
:93 Symp. Commun. A rchitectures and Protocols
( 199 3).
2 . Ramakrishnan, "Performance Considerations m
Designing Network Interfaces,"
IEEE }SAC, vol. 1 1 , no. 2 (Feb. 1 993).
Biography
Robert
J.
Walsh
Robert Walsh has been working on high-speed networking since the beginning of the 1 980s. He developed network i ng software for BSD UNIX, BEN's Bu tterfly multiproces sor, and DIG ITAL's GIGAswitch/FDDI.
Digital
Technical journal
VoJ.
9
No. 4
1 997 75
I
Recent DIGITAL
U.S. Patents
The f o llo wi n g patents were recently issued to Digital Equipment Corporation. Titles and names supplied by the
U.S. Patent and Trademark Office are reproduced as they appear on the original published patent.
D39 1 ,927
5 ,596,2 1 8
5, 596,283
5 ,596,575
5 ,596,7 1 5
5,596,754
5,608,653
5,608,883
5 ,6 1 4,444
5,6 1 5 , 1 67
5,61 5,283
5 ,6 1 5,363
5,61 5 ,382
5 ,6 1 7,283
5,61 7,409
5 ,6 1 9 ,657
5 ,6 1 9 ,662
5 ,6 1 9 ,7 1 0
Robert T. Faranda and
Bradford G. Chapin
Hamid R. Soleimani, Brian Doyle, and
Ara Philipossian
Richard
I.
Mellitz and Michael V. Dowd
Notebook personal computer
Hot carrier-hard gate oxides by nitrogen imp l a n t ati o n before gate oxidation
Continuous motion electrical circuit intercormect test method and apparatus
Automatic network speed adapter Henry S. Yang, Donald L.
Post, and
Wcn-Yi H u a ng
Philippe Klein, David W. Maruska, and
Kevin W. Ludlam
David B. Lomet
Ricky S. Palmer and Larry G. Palmer
Robert R. K'lndo and Paul L. Goctin
Janos Farkas, Rahul Ja.irath, Matt SteU, and Sing-Mo Tzeng
Ani! K
Jain,
John H. Edmondson, and
Peter J. Ba1mon
Dale R. Doncbin
Steven M. Jenness
Vincent G. Gavin, Michael J. Seaman,
Neal A. Crook, and Bipin Misu-y
David B . Krakauer, Ka.izad Mistry,
Steven Butler, and Hamid Partovi
Cuneyt M. Ozveren, Hallam G. Murray, Jr.,
Gregory M. Waters, and Robert J. Simcoe
Ram Sudama, David M. Griffin,
Brad Johnson, Dexter Sealy,
James Shelhamer, and Owen H. Tallman
Simon C. Steely, Jr. , David J. Sager, and
David B. Fire, Jr.
Robert L. Travis, Jr., Andrew P. Wilson,
Neal
F.
J ac o b s o n
, and
Michael
J.
Renzullo
Method and apparan1s for testing high speed busses using gray-code data
Method for performing private lock management
Video teleconferencing for networked workstations
Adapter for intercoimecting single-ended and differen tial
SCSI buses to prevent 'busy' or 'wired-or' glitches from being passed from one bus to the other
Method of using additives witl1 silica-based slurries to enhance selectivity in me t a l CMP
Method for increasing system bandwidtl1 through an on-chip add ress lock register
Pattern recognition device
Object oriented computer architecture using ctirectory objects
Data transfer system for buffering and selectively manipu lating the size of data blocks being transferred between a processor and a system bus of a computer system
Self-referencing modulation circuit for CMOS integrated circuit electrostatic discharge protection clamps
Flow control witl1 smootl1 limit setting for multiple virtual circuits
Method for providing a security facility for a network of management servers utilizing a database of trust relations to verifY mutual trust relations between management servers
Memory reference tagging
Method and apparatus for object-oriented invocation of a server application by a client application
76 Digital Technical Journal
Vol . 9 No. 4 1 997
5,621 ,678
5,62 1 ,734
5,62 1 ,874
5 ,623,690
5,625 ,802
5 ,625,805
5 ,625,822
5,627,773
5 ,627,842
5 ,627,98 1
5 ,629,840
5 ,629,950
5,630,049
5,630,055
5 ,630,097
5,630, 1 66
5 ,6 3 1 ,908
5,633,867
5 ,634,0 14
5,634,023
5 ,636,355
5 ,636,366
5,638,259
Michael J. Barnaby and James W. Brissette
Bruce E. Mann, Darrell J . Duffy,
Anthony G. Lauck, and
William D. Su·ecker
Peter Lucas and Jeffrey A. Senn
Larry G. Palmer and Ricky S. Palmer
Hoe T. Cho, Maw
Z.
Jau, and
W. H ugh Durdan
David M. Fenwick, Daniel Wissel!,
Richard Watson, and Denis Foley
Bevin R. Brett
Gilbert M. Wolrich, Timothy C. Fischer, and John A. Kowaleski, Jr.
Joseph H. Brown and Dilip K. Bhavsar
Michael C. Adler, Steven
0. Hobbs, and
Paul G. Lowney
William R. Hamburgen, John S. Fitch, and Norman P. Jouppi
Nitin D. Godiwala, Kurt M. Thaller,
Jeffrey A. Metzger, and Barry A. Maskas
Wayne M. Cardoza, Jeffrey M. Diewald,
Jeffrey E. Nelson, Steven D. DiPirro,
James R. Goddard, Wendell B. Fisher, Jr. ,
Anne E . McElearney, and Richard Sayde
Peter J. Bannon, Ruben vV. Castelino, and Chandrasekhara Somanathan
David A. Orbits, Kenneth D . Abramson, and H. Bruce Butts, Jr.
Rodney Gamache, Stuart Farnham,
Michael Harvey, William A. Laing,
Kathleen Morse, and Michael Uhler
James B. Saxe
Michael Ben-Nun, Simoni Ben-Michael,
Simcha Perl, and Kadangode K.
Ramakrishnan
William B . Gist and Joseph P. Coyle
Michael C. Adler, Steven 0. Hobbs, and
Paul G. Lowney
Kadangode K. Ramakrishnan and
Prabuddha Biswas
Scott G. Robinson, Richard L. Sites, and
Richard T. Witek
William F. McCarthy, Colin E. Brench, and Daniel M . Snow
Programmable memory controller for power and noise reduction
Local area network with server and virtual circuits
Three dimensional document representation using strands
Audio/video storage and retrieval for multimedia work stations by interleaving audio and video data in data file
Apparatus and method for adapting a computer system to different architectures
Clock architecture for synchronous system bus which regulates and adjusts clock skew
Using sorting to do match up in smart recompilation
Floating point unit data path alignment
Architecture for system-wide standardized intra-module and inter-module fault testing
Software mechanism for accurately handling exceptions generated by instructions scheduled speculatively due to branch elimination
High powered die with bus bars
Fault management scheme for a cache memory
Method and apparatus for testing software on a computer network
Autonomous pipeline reconfiguration for continuous error correction for fills from tertiary cache or memory
Enhanced cache operation with remapping of pages for optimizing data relocation from addresses causing cache misses
Controlling requests for access to resources made by multiple processors over a shared bus
Method and apparatus for generating and implementing smooth schedules for forwarding data flows across cell based switches
Local memory buffers management for an ATM adapter implementing credit based flow control
Semiconductor process, power supply voltage and tem perature compensated integrated system bus termination
Software mechanism for accurately handling exceptions generated by speculatively scheduled instructions
Disk cache management techniques using non-volatile storage
System and method for preserving instruction state atomicity for translated program
Enclosure for electronic modules
Digital Technical Journal
Vol . 9 No. 4 1997 77
5,638,532
5,638,538
5 ,644,571
5,646,58 1
5 ,648,909
5,648,959
5,649,1 09
5,649,203
5,650,997
5,65 1 , 1 1 1
5,652,61 5
5 ,652,837
5,652,86 1
5,652,869
5,652,889
5 ,654,653
5 ,657,239
5 ,657,426
5,657,456
5,657,47 1
5 ,657,480
Robert C. Frame and Mark J. Foster
Stephen R. Van Doren, Denis J. Foley, and Maurice B . Steinman
Michael
J.
Seaman
Apparatus and method for accessi.ng SMRAM in a com puter based upon a processor employing system manage ment mode
Turbotable: apparatus tor directing address and commands between multiple consumers on a node coupled to a pipelined system bus
Apparatus for message fi l tering in a network using domain class
Low inductance electrical resistor terminator package
Static timing verification in the presence oflogically false paths
James
0.
Pazatis and Richard P. Evans
Larry L. B iro, Joel
J.
Grodstein,
Jeng-Wei Pan, and Nicholas L.
Rethman
Nicholas Ilyadis and Richard Graham
Martin Edward Griesmer, Parayath
Gopal Krishnakumar, and David Benson
Richard Lee Sites
Henry Sho-Che Yang, Anthony G. Lauck,
Kadangode K Ramakrishnan, and
William R. Hawe
Wi.l liam M . McKeeman and
August G . Reinig
Stewart Frederick Bryant and
Shaheedur Reza Haque
Nicholas A!Jcn Warchol and
Chester Pawlowski
David T. Mayo, David W. Hartwell, and
Hansel A. Col lins
Mark A. Herdeg, James A. Wooldridge,
Scott G. Robinson, Ronald F. Brender, and Michael V. lies
Richard Lee Sites
Inter-module interconnect for simultaneous use with distributed LAN repeaters and stations
Apparatus and method for maintaining forwarding information in a bridge or router using multiple free queues having associated free space sizes
Translating, executing, and re-translating a computer program for finding and translating program code at unknown p rogram addresses
Method and apparatus for use in a network of the ethernet type, to i mprove fairness by controlling col lision backofftimes in the event of channel capture
Method and apparatus for producing a software test system using complementary code to resolve external dependencies
Precision broadcast of composite programs including secondary program content such as advertisements
Mechanism for screeni.ng commands issued over a communications bus for selective execution by a processor
System for interleaving memory modules and banks
Joseph P. Coyle and Willian1 B . Gist
Joel J. Grodstein, Nicholas L. Rethman, and Jeng-Wei Pan
Keith Waters and Thomas M. Levergood
William B . Gist and Joseph P. Coyle
Richard Lary, Robert Willard,
Cathari.ne van Ingen, David Thiel,
VVilliam Watson, Barry Rubinson, and
Verell Boaen
Neal F. Jacobson
System for executing and debugging multiple codes in a multi-architecture environment using jacketing means tor jacketing the cross-domain calls
Alternate execution and interpretation of computer pro gram having code at unknown locations due to transfer instructions having computed desti nation addresses
Reduced system bus receiver setup time by latching unamplified bus voltage
Timing verification using synchronizers and timing constraints
Method and apparatus for producing audio-visual synthetic speech
Semiconductor process power supply voltage and temperature compensated integrated system bus driver rise and fall time
Dual addressing arrangement for a communications interface architecture
Method of recording, playback, and re -execution of of concurrently running application progran1 operational commands using global time stamps
78
DigitJJ
Technical Journal Vol.
9 No. 4 1997
5,658 , 1 66
5,659,71 3
5,659,739
5,659,753
5,659,775
5,664, 106
5,664 , 1 77
5,664,22 1
5,666,4 1 5
5,666,5 1 9
5,666,55 1
5,668,95 1
5,671 ,225
5,671 ,406
5 ,675,735
5,675,742
5,675,763
5,675,800
5,678,045
5,680,544
5,680,584
5,680,644
5 ,682,489
5,682,5 5 1
5,684,946
Mike Freeman, Stuart Keith Morgan, and Mike Romm
Paul M. Goodwin, David A. Tatosian, and Donald Smelser
Clark E. Lubbers, Susan G. Elkington, and Richard F. Lary
Dennis Joseph Murphy and
Robert Neil Faiman, Jr.
Alexander Stein and William Grundmann
Frank Samuel Caccavale
Edward S. Lowry
Mark F. Amberg, William K. Miller,
Frank M. Nemeth, and
Dwayne H. Swanson
Charles William Kautlnan
Peter C. Hayden
David M . Fenwick, Denis ]. Foley,
Stephen R. Van Doren, David W. Hartwell,
Elbert Bloom, and Ricky C. Hetherington
Rajendra K. Jain, K. K. Rama.krishnan, and Dah -Ming Chiu
Donald F. Hooper, Dave M. Tonge!, and
Michael B. Evans
Clark E. Lubbers and Susan G. Elkington
Shawn Gallagher, James Scott Hiscock,
Dahai Ding, Scott D ' Edwine Lawrence
Rajendra K. Jain, K. K. Ramakrishnan, and Dah-Ming Chiu
Jeffrey Clifford Mogul
Wendell Burns Fisher, Jr. and
Richard Sayde
J i.irgen Bertels
John Edmondson and Scott Taylor
Mark A. Herdeg and Michael V. Iles
David J. Sager
Jeffrey R. Harrow and Fred P. Messinger
Chester Walenry Pawlowski, Nicholas
Alien Warchol, David Gerard Conroy, and R. Stephen Polzin
James P. Ellis, Mike Kantrowitz, and
Will Sherwood
Modular coupler arrangement for use in a building wiring clistTi bu tion system
Memory stream buffer with variable-size prefetch depending on memory interleaving configuration
Skip list data structure enhancements
I nterface for symbol table construction in a multi language optimizing compiler
Topology independent system for state element conversion
Phase-space surface representation of server computer performance in a computer network
Data processing system having a data structure with a single, simple primitive
System for reconfiguring addresses of SCSI devices via a device address bus independent of the SCSI bus
Method and apparatus for cryptographic authentication
Method and apparatus for detecting and executing cross-domain cal ls in a computer system
Distributed data bus sequencing for a system bus with separate address and data bus protocols
Avoiding congestion system for reducing traffic load on selected end systems which utilizing above their allocated f:'lir shares to optimize throughput at intermecliate node
Distributed interactive multimedia service system
Data structure enhancements for in -place sorting of a singly linked list
Method and apparatus tor interconnecting network devices in a networking h u b
System for setting congestion avoidance flag at intermediate node to reduce rates of transmission on selected end systems which uti lizing above their allocated fair shares
Cache memory system and method for selectively removing stale aJiased entries
Metl1od and apparatus for remotely booting a computer system
Method and apparatus tor multiscript access to entries in a directory
Method for testing an on-chip cache for repair
Simulator system for code execution and debugging within a multi-architecture environment
Low delay means of communicating between systems on clifferent clocks
Method and device tor monitoring, manipulating, and viewing system information
System for checking tl1e acceptance of i/0 request to an interface using software visible instruction which provides a status signal and performs operations in response tJ1ereto
Apparatus and method tor improving tllC efficiency and quality of fl.mctional verification
Digital Technical Journal Vol .
9
No. 4
1 997 79
5,687,3 1 0
5,687,330
5,689,679
5 ,692 , 1 59
5,694,3 1 2
5,694,350
5,694,5 36
5,694,579
5 ,695,068
5,696,945
5,696,956
5 ,698 ,8 1 8
5,701 ,480
5 ,70 1 ,484
5,701 ,667
5,708,8 1 3
5,71 2,858
5,71 3,009
5,7 1 7,575
5,7 1 7,729
5 ,7 1 7,883
5,7 1 7,92 1
5,720,009
5 ,724,033
5 ,724,5 1 3
Paul Stuart Rotker and
Randall Dean Hinrichs
William B. Gist and Joseph P. Coyle
Norman Pau l Jouppi
Mark A. Shand
System for generating error signal to indicate mismatch in commands :md preven ting processi ng data associated with the received com mands when mismatch command has been determined
Semicond uctor process, power supply and temperature compensated system bus integrated i nterface architec ture with precision receiver
Memory system and method for selective m u lti-level caching using a cache level code
Configurable d igital signal interface using field programmable gate array to reformat data
Uninterruptible power supply with fau lt tolerance in a high voltage environment
Rou nding adder for floating point processor
Gerald J. B rand and Don L. Drin kwater
GiJ bert iYI . Wol rich, Timothy C. Fischer, and John A. Kowaleski , Jr.
Michel Gangnct and Jean-Manuel
Van Thong
Rah u l Razdan and Gabriel B ischoff
Robert AUison Hart and Richard
Harry Plourde
Larry D . Seiler, Robert S . McNamara,
Christopher C. Gianos, and
Joel J . McCormack
Rah u l Razdan, Bill Grundmann, and
Michael D. Smith
Colin Edward B rench
Yoav Raz
Yeshayahu Artsy
Stephen Michael Birch, Gerard
Michel Gavrel , and Zaftar Iqbal Memon
Hoe To Cho and Ming Huann Yuan
Nitin Dhiroobhai God iwala, And re\v Myer
Ebert, and Chester Walenty Pawlowski
John Anthony DeRosa, Jr., Benn Lee
Schreiber, Peter Chapman Hayden, and
Scott Wade Apgar
Jeffrey P. Copeland and Dennis Ro binson
Russel l Iknaian and Richard B. Watson, Jr.
David J. Sager
David Lomet and Betty Salzberg
Steven A. Kirk, William Barabash, and
WiJliam S . Yerawnis
Michael Burrows
Michael Ben-Nun, Simoni Ben- Michael, and Moshe De- Leon
Method and apparatus for automatic gap closing in computer aided drawing
Using pre-analysis and a 2 -state optimistic model to reduce computation in transistor circuit simulation
Probe card shipping and handling system
Method for quickly painting and copying shal low pixels on a deep fi·anlC buffer
Dynamically programmable reduced i nstruction set computer with programmable processor load ing on pro gram number field and program number register contents
Two part closely coupled cross polarized
EMI shield
Distributed m u lti-version commitment ordering protocols for guaranteeing serializability d uring transaction processing
Rou ting objects on action paths i n a distributed computing system
Method of manutacture of an interconnect stress test coupon
Programmable interrupt signal router
Test methodology for exceeding tester pin cou nt for an asic device
Method and apparatus for configuring a computer system
Board mounting system with self guiding interengagement
Low skew remote absolute delay regulator cbip
Method and apparatus for parallel execution of com puter programs using intormation providi ng for recon struction of a logical sequential program
Concurrency a nd recovery for index trees with nodal updates using mu ltiple atomic actions
Method of rule execution i n an expert system usi ng equival ence cl asses to group data base objects
Method for encoding delta values
Traffic shaping system for asynchronous transfer mode networks
80
Digiral Technical Journal Vol . 9 No. 4 1 997
ISSN 0898 -90 1 X
Prinred in
U.S.A.
EC-P9257- 1 8 /98 05 1 9 2 3 . 0 Copyrighr © Digiral Equipmenr Corporarion .
' ·
...
Télécharger
Publicité