Performance Evaluation of Transaction Processing Systems. Digital VAX 6000, VAX TPC Benchmark A, VAX DECtp, VAX DECdta, VAX ACMS, VAX Server, VAX ft, VAX 9000, VAX cluster, VAX KODA
विज्ञापन
विज्ञापन
Walter H. Kohler
Yun-Ping Hsu
Thomas
K.
Rogers
Wael H. Bahau-El-Din
Pegormance Evaluation of Transaction Processing
Systems
Performance and price/performance are important attributes to consider when evaluating a transaction processing system. Two major approaches to performance evaluation are measurement and modeling.
TPC
Bencl?mark
A
is an industry stan- price/performance. Digital has implemented
TPC
Benchmark
A
in a distributed transaction processing environment. Benchmark measurements were performed on the
VAX
9000 Model
210
and the
VAX
4000
Model
300
systems. Further a compre- hensive analytical model was developed and customized to model the performance behavior of
TPC
Benchmark
A
on Digital's transaction processing platforms. This model was validated using measurement results and has proven to be an accurate pe?formanceprediction tool.
Transaction processing systems are complex in nature and are usually characterized by a large number of interactive terminals and users, a large volume of on-line data and storage devices, and a high volume of concurrent and shared database accesses. Transaction processing systems require layers of software components and hardware devices to work in concert. Performance and price/performance are two important attributes for customers to consider when selecting trans- action processing systems. Performance is impor- tant because transaction processing systems are frequently used to operate the customer's business or handle mission-critical tasks. Therefore, a certain level of throughput and response time guarantee are required from the systems during normal oper- ation. Price/performance is the total system and maintenance cost in dollars, normalized by the per- formance metric.
The performance of a transaction processing system is often measured by its throughput in trans- actions per second
(TPS) that satisfies a response time constraint. For example, 90 percent of the transactions must have a response time that is less than
2 seconds. This throughput, qualified by the associated response time constraint, is called the maximum qualified throughput
(MQTh).
In a trans- action processing environment, the most mean- ingful response time definition is the end-to-end response time, i.e., the response time observed by a user at a terminal. The end-to-end response time represents the time required by all components that compose the transaction processing system.
The two major approaches used for evaluating transaction processing system performance are measurement and modeling. The measurement approach is the most realistic way of evaluating the performance of a system. Performance measure- ment results from standard benchmarks have been the most accepted form of performance assess- ment of transaction processing systems. However, due to the complexity of transaction processing systems, such measurements are usually very expen- sive, very time-consuming, and difficult to perform.
Modeling uses simulation or analytical model- ing techniques. Compared to the measurement approach, modeling makes it easier to produce results and requires less computing resources.
Performance models are also flexible. Models can be used to answer "what-if" types of questions and to provide insights into the complex performance behavior of transaction processing systems, which is difficult
(if not impossible) to observe in the measurement environment. Performance models are widely used in research and engineering com- munities to provide valuable analysis of design alternatives, architecture evaluation, and capacity planning. Simplifying assumptions are usually
Digital
Technical
Journal Vo1.
.3
No. IVtnfer
I991
46
Transaction Processing, Databases, and Fault-tolerant Systems
made in the modeling approach. Therefore, perfor- mance models require validation, through detailed simulation o r measurement, before predictions from the models are accepted.
This paper presents Digital's benchmark measure- ment and modeling approaches to transaction processing system performance evaluation. The paper includes an overview of the current industry standard transaction processing benchmark, the
TPC Benchmark A, and a description of Digital's implementation of the benchmark, including the distinguishing features of the implementation and the benchmark methodology. The performance measurement results that were achieved by using the TPC Benchmark A are also presented. Finally, a multilevel analytical model of the performance behavior of transaction processing systems with response time constraints is presented and vali- dated against measurement results.
TPC Benchmark A-An Overview
The
TPC
Benchmark A simulates a simple banking environment and exercises key components of the system under test
(SUT) by using a simple, update-intensive transaction type. The benchmark is intended to simulate a class of transaction pro- cessing application environments, not the entire range of transaction processing environments.
Nevertheless, the single transaction type specified by the TPC Benchmark
A standard provides a simple and repeatable unit of work.
The benchmark can be run in either a local area network
(LAN) or a wide area network
(WAN)
configuration. The related throughput metrics are tpsA-Local and tpsA-Wide, respectively. The benchmark specification defines the general appli- cation requirements, database design and scaling rules, testing and pricing guidelines, full disclo- sure report requirements, and an audit checklist.'
The following sections provide an overview of the benchmark.
Application Environment
The TPC Benchmark A workload is patterned after a simplified banking application.
In this model, the bank contains one or more branches. Each branch has 10 tellers and 100,000 customer accounts.
A transaction occurs when a teller enters a deposit or a withdrawal for a customer against an account at a branch location. Each teller enters transactions at an average rate of one every 10 seconds. Figure 1 illustrates this simplified banking environment.
Transaction Logic
Thc transaction logic of the TPC Benchmark
A workload can be described in terms of the bank environment shown in Figure 1. A teller deposits in or withdraws money from an account, updates the current cash position of the teller and branch, and makes an entry of the transaction in a history file. The pseudocode shown in Figure 2 represents the transaction.
100,000
ACCOUNTS
I i
I
I
100,000
ACCOUNTS
I
I
I
100,000
ACCOUNTS
I
I
TELLERS
0
TELLERS
0
CUSTOMERS
I
Figure
I
TPCBencbmark A Banking Environment
Vol.
3
No. Winter
1991 Digital Technical Journal
Per$omnce Evaluation of Transaction Processing Systems
R e a d 1 0 0 b y t e s i n c l u d i n g B i d , T i d , A i d , D e l t a f r o m t e r m i n a l
B E G I N T R A N S A C T I O N
U p d a t e A c c o u n t w h e r e A c c o u n t - I D
=
A i d :
R e a d A c c o u n t - B a l a n c e f r o m A c c o u n t
S e t A c c o u n t - B a l a n c e
=
A c c o u n t - B a l a n c e
+
D e l t a
W r i t e A c c o u n t - B a l a n c e t o A c c o u n t
W r i t e t o H i s t o r y :
A i d , T i d , B i d , D e l t a , T i m e - S t a m p
U p d a t e T e l l e r w h e r e T e l l e r - I D
=
T i d :
S e t T e l l e r - B a l a n c e
=
T e l l e r - B a l a n c e
W r i t e T e l l e r - B a l a n c e t o T e l l e r
U p d a t e B r a n c h w h e r e B r a n c h - I D
=
B i d :
S e t B r a n c h - B a l a n c e
=
B r a n c h - B a l a n c e
W r i t e B r a n c h - B a l a n c e t o B r a n c h
+
+
D e l t a
D e l t a
COMMIT T R A N S A C T I O N
W r i t e
2 0 0 b y t e s i n c l u d i n g A i d , T i d , D e l t a , A c c o u n t - B a l a n c e t o t e r m i n a l
Figure
2
TPC Benchmark
A
Transaction Pseudocode
Terminal Communication
For each transaction, the originating terminal is required to transmit data to, and receive data from, the system under test. The data sent to the system under test must consist of at least 100 alphanumeric data bytes, organized as at least four distinct fields:
Account-ID, Teller-ID, Branch-ID, and Delta. The
Branch-ID identifies the branch where the teller is located. The Delta is the amount to be credited to, or debited from, the specified account. The data received from the system under test consists of at least 200 data bytes, organized as the above four input fields and the Account-Balance that results from the successful commit operation of the transaction.
Implementation Constraints
The
TPC
Benchmark
A
imposes several conditions on the test environment.
The transaction processing system must support atomicity, consistency, isolation, and durability
(ACID) properties during the test.
The tested system must preserve the effects of committed transactions and ensure database consistency after recovering from
-
The failure of a single durable medium that contains datatbase or recovery log data
-
The crash and reboot of the system
-
The loss of all or part of memory
Eighty-five percent of the accounts processed by a teller must belong to the home branch (the one to which the teller belongs). Fifteen percent of the accounts processed by a teller must be owned by a remote branch (one to which the teller does not belong). Accounts must be uni- formly distributed and randomly selected.
Database Design
The database consists of four individual files/tables:
Branch, Teller, Account, and History, as defined in
Table 1. The overall size of the database is deter- mined by the throughput capacity of the system.
Ten tellers, each entering transactions at an aver- age rate of one transaction every 10 seconds, gener- ate what is defined as a one-TPS load. Therefore, each teller contributes one-tenth (1/10) TPS. The history area must be large enough to store the his- tory records generated during 90 eight-hour days of operation at the published system TPS capacity.
For a system that has a processing capacity of x
TPS, the database is sized as shown in Table 2.
For example, to process 20 TPS, a system must use a database that includes 20 branch records, 200 teller records, and 2,000,000 account records.
Because each teller uses a terminal, the price of the system must include 200 terminals. A test that results in a higher TPS rate is invalid unless the size of the database and the number of terminals are increased proportionately.
Digital Tecbnfcal Journal
Vol.
.J
IVO. I
Winter
1991
Transaction Processing, Databases, a n d Fault-tolerant Systems
Table 1 Database Entities
Record
Branch
Teller
Account
History
Bytes Fields Required
100
100
Branch-ID
Branch-Balance
Description
Identifies t h e branch across the range of branches
Contains t h e branch's current cash balance
Teller-ID
Branch-ID
Teller-Balance ldentifies the teller across the range of tellers ldentifies the branch where the teller is located
Contains the teller's current cash balance
Account-ID Identifies the customer account uniquely for the entire database
Branch-ID Identifies t h e branch where the account is held
Account-Balance Contains t h e account's current cash balance
Account-ID
Teller-ID
Branch-ID
Amount
Time-Stamp
Identifies the account updated by the transaction
Identifies the teller involved in the transaction
Identifies the branch associated with the teller
Contains t h e amount of credit or debit (delta) specified by the transaction
Contains the date and time taken between the BEGIN
TRANSACTION and COMMIT TRANSACTION statements
Table 2 Database Sizing
Number of Records Record Type
1
x x
1Oxx
100,000 x x
2,592,000 x x
Branch records
Teller records
Account records
History records
Benchmark Metrics
TPC Benchmark
A
uses two basic metrics:
Transactions p e r second (TPS)
-
TPS, subject t o a response time constraint, i.e., t h e MQTh, is measured while the system is in a sustainable steady-state condition.
Price p e r TPS (K$/TPS)
- and five-year maintenance costs associated w i t h o n e TPS.
Transactions per Second
To guarantee that t h e tested system provides fast response t o on-line users, t h e TPC Benchmark
A
imposes a specific response time constraint on the benchmark.
Ninety percent of all transactions must have a response time of less than t w o seconds. The TPC
Benchmark
A
standard defines transaction response time as the time interval between the transmission from the terminal of the first byte of the input mes- sage t o t h e system under test to t h e arrival at t h e terminal of t h e last byte of t h e o u t p u t message from t h e system under test.
The reported TPS is the total number of commit- ted transactions that both started and completed during an interval of steady-state performance, divided by the elapsed time of t h e interval. The steady-state measurement interval must be at least
15 minutes, and
90
percent of t h e transactions must have a response time of less than 2 seconds.
Price per TPS
The KS/TPS price/performance metric measures t h e total system price in thou- sands of dollars, normalized by t h e TPS rating of t h e system. The priced system includes all t h e components that a customer requires to achieve t h e reported performance level and is defined by t h e TPC Benchmark
A
standard as t h e
Price of the system under test, including all hard- ware, software, ant1 maintenance for five years.
Price of t h e terminals and network compo- nents, and their maintenance for five years.
Price of on-line storage for 90 days of history records at the published TPS rate, which amounts t o 2,592,000 records p e r TPS.
A
storage medium is considered t o b e on-line if any record can be accessed randomly within o n e second.
Price of additional products required for t h e operation, administration, o r maintenance of t h e priced systems.
Price of p r o d u c t s required for application development.
All hardware and software used in t h e tested configuration must be announced and generally available to customers.
W>l.
.?
No. l Winter
1991 Digilal Technical Journal
TPC Benchmark A Implementation
Digital's implementation of the TPC Benchmark
A
goes beyond the minimum requirements of the
TPC
Benchmark A standard and uses Digital's dis- tributed approach to transaction processing? For example, Digital's TPC Benchmark A implementa- tion includes forms management and transaction processing monitor software that are required in most real transaction processing environments but are not required by the benchmark. The fol- lowing sections provide an overview of Digital's approach and implementation.
Transaction Processing Software
Environment
The three basic functions of a general-purpose transaction processing system are the user inter- face (forms processing), applications management, and database management. Digital has developed a distributed transaction architecture (DECdta) to define how the major functions are partitioned and supported by components that fit together to form a complete transaction processing system.
Table
3
shows the software components in a typical
Digital transaction processing environment.
Distributed Transaction Processing
Approach
Digital transaction processing systems can be dis- tributed by placing one or more of the basic system functions (i.e., user interface, application manager,
Table 3 Transaction Processing Software
Components
Component Example
Operating system
Communications
Database
TP monitor
Forms
Application
VMS
LAT, DECnet
VAX RdbNMS
VAX ACMS, DECintact
DECforms
COBOL database manager) on separate computers. In the simplest form of a distributed transaction process- ing system, the user interface component runs on a front-end processor, and the application and data- base components run on a back-end processor. The configuration allows terminal and forms manage- ment to be performed at a remote location, whereas the application is processed at a central location.
The Digital transaction processing software com- ponents are separable because their clearly defined interfaces can be layered transparently onto a net- work. How these components may be partitioned in the Digital distributed transaction processing environment is illustrated in Figure
3.
TPC Benchmark A Test Environment
The Digital TPC Benchmark A tests are imple- mented in a distributed transaction processing environment using the transaction processing
TERMINALS
COMMUNICATIONS
FORMS
TP MONITOR
OPERATING SYSTEM
COMMUNICATIONS
FRONT-END
PROCESSORS
BACK-END
PROCESSORS
I
DATABASE
1
OPERATING SYSTEM
I
COMMUNICATIONS
I
DATABASE
STORAGE
Figure
3
Distributed Trnnsaction Processing Environment
Digital Technical Jorc.rnal Vol.
-3
No. I Wiizfer 1991
Transaction Processing, Databases, and Fault-tolerant Systems software components shown in Figure
3.
The user interface component runs on one or more front- end processors, whereas the application and database components run on one or more back- end processors. Transactions are entered from teller terminals, which communicate with the front-end processors. The front-end processors then communicate with the back-end processors to invoke the application servers and perform database operations. The communications can take place over either a local area or a wide area network. However, to simplify testing, the
TP<:
Benchmark
A
standard allows sponsors to use remote terminal emulators (RTEs) rather than real terminals. Therefore, the TPC Benchmark A tests base performance and price/performance results on two distinctly configured systems, the target system and the test system.
The target system is the configuration of hard- ware and software components that customers can use to perform transaction processing. With the
Digital distributed transaction processing approach, user terminals initiate transactions and communi- cate with the front-end processors. Front-end pro- cessors communicate with a back-end processor using the DECnet protocol.
The test system is the configuration of com- ponents used in the lab to measure the perfor- mance of the target system. The test system uses
RTEs, rather than user terminals, to generate the workload and measure response time. (Note: In previously published reports, based on Digital's
Debitcredit benchmark, the RTE emulated front- end processors. In the TPC Benchmark A standard, the
RTE emulates only the user terminals.) The
KTE component
Emulates the behavior of terminal users accord- ing to the benchmark specification (e.g., think time, transaction parameters)
Emulates terminal devices (e.g., conversion and multiplexing into the local area transport
[LATI protocol used by the DECserver terminal servers)
Records transaction messages and response times (e.g., the starting and ending times of individual transactions from each emulated terminal device)
Figure 4 depicts the test system configuration in the
LAN
environment with one back-end proces- sor, multiple front-end processors, and multiple remote terminal emulators.
h
REMOTE
TERMINAL
EMULATORS
h b
FRONT-END
PROCESSORS
dm
ETHERNET
BACK-END
PROCESSOR
DATABASE
Figure
4
Test Systern
Configurcltion
11 c, u ~ r c ~ r ~ r r c c c r n c o u c c o
We now present the results of two TPC Benchmark
A
tests based on audited benchmark experiments performed on the
VAX
9000
Model 210 and the
VAX
4000
Model 300 systems.'.' These two systems are representative of Digital's Large and small trans- action processing platforms. The benchmark was implemented using the
VAX ACMS transaction pro- cessing monitor, the VAX RdbrYMS relational data- base management system, and the DECforms forms management system on the
VMS operating system.
'rables
4 and 5 show the back-end system configu- rations for the
VAX
9000
Model 210 and the
VAX
4000
Model 300 systems, respectively. Table 6 shows the system configuration of the front-end systems.
Measurement Results
The maximum qualified throughput and response time results for the
TPC
Benchmark A are summa- rized in Table 7 for the
VAX
9000
Model 210 and the
VAX
4000
Model 300 systems. Both configiirations have sufficient main memory and disk drives such
Table 4 VAX 9000 Model 210 Back-end
System Configuration
Component Product Quantity
Processor
Memory
Tape drive
Disk controller
VAX
9000 Model 21 0
1
256
MB
TA81
KDM70
1
2
Disks
Operating system
RA92
VMS
5.4
16
1
Communications DECnet-VMS Phase
IV
1
TP monitor
Dictionary
VAX
VAX
ACMS
V3.1
CDDIPIus
V4.1
1
1
Application
VAX
COBOL
V4.2
Database system
VAX
RdbNMS V4.0
Forms management DECforms V1.2
1
1
1
Vol.
3
No. 1
Winter 1991
Digital Technical Journal
Pe$omnce Evaluation of Transaction Processing Systevns
Table 5 VAX 4000 Model 300 Back-end
System Configuration
Componen t Product Quantity
Processor
Memory
Tape drive
VAX 4000 Model 300 1
64 MB
Disk controller
Disks
Operating system
Communications
TP monitor
TK70
DSSl
RF31
VMS 5.4
DECnet-VMS Phase IV 1
VAX ACMS V3.1 1
VAX CDDIPIus V4.1 1
Dictionary
Application
Database system
Forms management
VAX COBOL V4.2
VAX RdbNMS V4.0
DECforms V1.2
1
1
1
1
3
18
1 that the processors are effectively utilized with no other bottleneck. Both systems achieved well over
90
percent
CPU
utilization at the maximum quali- fied throughput under the response time constraint.
In addition to the throughput and response time, the
TPC
Benchmark A specification requires that several other data points and graphs be reported.
We demonstrate these data and graphs by using the
VAX
9000
Model 210
TPC
Benchmark
A
results.
Response Time in Relationship to
TPS.
Figure
5
shows the ninetieth percentile and average
Table 6 Front-end Run-time System Configuration
Component Product
Processor VAXserver 31 00 Model 10
Memory
Disks
Operating system
Communications
TP monitor
Forms management
RZ23 (1 04 MB)
VMS 5.3
VMS 5.4
DECnet-VMS Phase IV
VAX ACMS V3.1
DECforms V1.2 u
0.4
20
KEY:
30 40
50 60
TRANSACTIONS PER SECOND
70 80
90TH PERCENTILE
Figure
5
VAX
9000 Response Time in
Relationship to Transactions per Second
response times at 100 percent and approximately
80
percent and 50 percent of the maximum qualified throughput. The mean transaction response time still grows linearly with the transaction rate up to the 70
TPS
level, but the ninetieth percentile response time curve has started to rise quickly due t o the high
CPU
uti- lization and random arrival of transactions.
Response Time Frequency Distribution. Figure
6 is a graphical representation of the transaction
Quantity
10 for VAX 9000 back-end
3 for VAX 4000 back-end
16 MI3 for VAX 9000 back-end
12 MB for VAX 4000 back-end
16
1 for VAX 9000 back-end
1 for VAX 4000 back-end
1
1
1
Table 7 Maximum Qualified Throughput
System
VAX 9000 Model 21 0
VAX 4000 Model 300
Response Time (seconds)
TPS (tpsA-Local) Average 90 percent
69.4
21.6
1.20
1.39
1.74
1.99
Maximum
5.82
4.81
Digital Technical Journal V i l .
.?
No. 1 Winter 1991
5
1
Transaction Processing, Databases, and Fault-tolerant Systems
14000
[ I+
AVERAGE
=
1.20
SECONDS
I 1-
90TH PERCENTILE
=
1 74
SECONDS
,
1-
MAXIMUM
=
5.82 SECONDS
I
I
I
I
I
I . ,
, ,
. .
,
,
RESPONSE TlME (SECONDS)
5
10
,
I
I
1
I
DATA COLLECTION
I
I
I
INTERVAL
I .
15 20 25
I
I
I
, I
,
30 35
TlME (MINUTES)
6
VAX 9000 Respotzse Tiirze Freqliencj~
Distrib~ition
response time distribution. The average, nine- tieth percentile, and maximum transaction response times are also marked on the graph. shown in Figure 7 demonstrate the sustainable maximum qualified throughput. The one-minute running average transaction throughputs dur- ing the warm-up and data collection periods of the experiment are plotted on the graph. This gr'cph shows that the throughput was steady during the period of data collection.
Average Response Time over Time. The results shown in Figure 8 demonstrate the sustain- able average response time in the experiment.
The one-minute running average transaction response times during the warm-up and data collection periods of the experiment are plotted on the graph. This graph shows that the mean response time was steady during the period of data collection.
Comprehensive
A n a l y t i c a l M o d e l
Modeling techniques can be used as a supplement or an alternative to the measurement approach.
The performance behavior of complex transaction processing systems can be characterized by a set of parameters, a set of performance metrics, and the
. rameters can be used to describe the different resources avail- able in the system, the database operations of trans- actions, and the workload that the transaction proccssing system undergoes. To completely rep- resent such a system, the size of the parameter set would be too huge to manage.
An
analytical model simplifies, through abstraction, the complex behav- ior of a system into a manageable set of parameters
Figlire 7
VAX
9000 Truizsc~ctionsper ooer Tirne
and policies. Such a moclel, after proper validation, can be a powerful tool for many types of analysis, as well as a performance prediction tool. Results can be obtained cluickly for any combination of parameters.
A
comprehensive analytical model of the perfor- mance behavior of transaction processing systems with a response time constraint was developed and validated against measurement results. This model is hierarchical and flexible for extension.
The following sections describe the basic con- struction of the model ant1 the customization made to model the execution of
TPC
Benchmark
A
on
Digital's transaction processing systems. The model can also be used to study different trans- action processing workloads in addition to the
TPC
Benchmark A.
Response
Time
Components
The main metric used in the model is the maxi- mum qualified throughput under a response time constraint. The response time constraint is in the
I
0 5
10
I ,
15 20
TlME (MINUTES)
25 30
I ,
35
Figure
8 VRY
9000 Ai,erage Response Time oLler Titne
Vo1. No. l Winter
1991 Digital Technical
Joirrnal
Performance Evaluation of Transaction Processing Systems
form of
"x
percent of transaction response times are less than y seconds."
To evaluate throughput under such response time constraint, the distribution of transaction response times is determined by first decomposing the trans- action response time into nonoverlapping and independent components. The distribution of each component is then evaluated. Finally, the overall transaction response time distribution is derived from t h e mathematical convolution of t h e compo- nent response time distributions.
The logical flow of a transaction in a front-end and back-end distributed transaction processing system that is used to implement TPC Benchmark
A
is depicted in Figure
9.
T h e response time of a transaction consists of three basic components: front-end processing, back-end processing, and communication delays.
Front-end processing usually includes terminal
I/O processing, forms/presentation services, and communication with t h e back-end systems. In the benchmark experiments, n o disk I/O activity w a s involved during t h e front-end processing.
Back-end processing includes the execution of application, database access, concurrency control, and transaction commit processing. The back-end processing usually involves a high degree of con- currency and many disk I/O activities.
Communication delays primarily include t h e communications between t h e user terminal and the front-end node, and t h e front-end and back- end interactions.
(Note: These response time components d o not overlap with each other.)
Within the back-end system, the transaction response time is further decomposed into two additional components, CPU delays and non-CPLJ, nonoverlapping delays. CPU delays include both the c p u service and t h e CPU waiting times of trans- actions. Non-CPU, nonoverlapping delays include:
Logging delays, which include the time for trans- action log writes and commit protocol delays
Database I/O delays, which include both waiting and service times for accessing storage devices
Other delays, which include delays that result from concurrency control (e.g., waiting for locks) and waiting for messages
Two-level Approach
The model is configured in a two-level hierarchy, a high level and a detailed level. The use of a hierarchy allows a complex and detailed model that considers many components and involves many parameters to be constructed easily. Because of the hierarchical approach, the model also provides flexibility for modifications and extensions, and validation of separate submodels.
The high-level model assumes the decomposition of transaction response times, as described in the
Response Time Components section, and models t h e behavior of t h e transaction processing system by an open queuing system, as shown in Figure 10.
The queuing system consists of servers and delay centers, which are connected in a queuing net- work with the following assumptions:
The front-end processing does not involve any disk I/O operation, and t h e load o n the front- end systems is equally balanced.
;D EE:;o"i:
TIME
1
COMMUNICATION
I
FRONT-END
I
COMMUNICATION
1
BACK-END
)
Figure 9 Response Time Components
Digital Technical Journal
Vo/.
3
No.
1
Winter 1991
5
3
Transaction Processing, Databases, and Fault-tolerant Systems
FRONT-END
PROCESSORS
COMMUNICATION
BACK-END
PROCESSORS
Figure 10
High-level Queuing Model for a Transaction Processing Sj~stern
The back-end is a shared-memory multiprocessor system with symmetrical loads on all processors
(or it can be simply a uniprocessor).
No intratransaction parallelism exists within individual transaction execution.
No mutual dependency exists between trans- action response time components.
Transaction arrivals t o t h e processors have a
Poisson distribution.
These assumptions correspond to Digital's TPC Bench- mark
A
testing methodology and implementation.
The front-end CPU is modeled as an
M/M/l
queu- ing center, and t h e back-end CPU is modeled as an
M/M/m queuing center. The transactions' CPU times o n the front-end and back-end systems are assumed t o be exponentially distributed (coefficient of vari- ation equal t o
1)
due to t h e single type of trans- action i n t h e benchmark. (Note:
An
approximation of M/G/m can be used t o consider a coefficient of variation o t h e r than
1
for t h e back-end transaction
CPU service time, especially in t h e multiprocessor case when t h e bus is highly utilized.) Database I/O, logging I/O, and o t h e r delays are modeled as delay centers, with appropriate delay distributions. For t h e model of t h e TPC Benchmark A workload, t h e cation and synchronization delays are combined into o n e delay center, called t h e LOD delay center, which is represented by a 2-Erlang distribution.
The major input parameters for this high-level model are the
Number of front-end systems and t h e front-end
CPU service time p e r transaction
Number of CPUs in t h e back-end system and the back-end CPU service time p e r transaction
Sum of the back-end database I/O response time, journaling
I/O response time, and other delay times (i.e., t h e mean for the LOD delay center's
2-Erlang distribution)
Response time constraint (in the form of
x
per- centile less than y seconds)
The main result from the high-level model is the ivlQTh. This high-level model presents a global pic- ture of the performance behavior and manifests the relationship between the most important parameters of t h e transaction processing system and MQTh.
Some of t h e input parameters in t h e high-level model are dynamic. The CPU service time of a t r a n s action may vary w i t h t h e throughput or number of processors, and t h e database I/O o r o t h e r delays may also depend o n t h e throughput.
A
good exam- ple of a dynamic model is a tightly coupled multi- processor system, w i t h o n e bus interconnecting the processors and with a shared common memory
(e.g., a
VAX
6000 Model 440 system). Such a system would run a single copy of t h e symmetrical multi- processing operating system (e.g., the
V M S
T h e average CPU service time of transactions is affected by both hardware and software factors, such as
Hardware contention that results from conflict- ing accesses to the shared bus and main memory and that causes processor speed degradation and longer CPU service time.
Processor synchronization overhead that results from t h e serialization of accesses t o shared data
54
Vo1.
.?
No. 1 Winter I991 Digital Technical Journal
Perjomuznce Evaluution of Trnnsaction Processing
Systems
structures. Many operating systems use spin- locks as the mechanism for processor-level synchronization, and the processor spins (i.e., busy-waits) in the case of a conflict. In the model, the busy-wait overhead is considered to be part of the transaction code path, and such contention elongates the transaction CPU service time.
Four detailed-level submodels are used to account for the dynamic behavior of these param- and LOD.
The CPU-cache-bus-memory submodel consists of many low-level parameters associated with the workload, processor, cache, bus, and memory com- ponents of multiprocessor systems. It models these components by using a mixed queuing network model that consists of both open and closed chains, as shown in Figure 11. The most important output from this submodel is the average number of CPU clock cycles per instruction.
The busy-wait submodel models the spin-lock contention that is associated with the two major
VMS spin-locks, called SCHED and IOLOCKB. This sub- model divides the state of a processor into several nonoverlapping states and uses probability analy- sis t o derive busy-wait time. The I/O grouping sub- model models the group commit and group write mechanisms of the
VAX
Rdb/VMS relational database management system. This submodel affects the path length of transaction because of the amortization of disk I/O processing among grouped trans- actions. The LOD submodel considers the disk I/O times and the lock contention of certain critical resources in the
VAX
Rdb/VMS system.
Integrating the
Two
of
the
Model
The two levels of the model are integrated by using an iterative procedure outlined in Figure 12. It starts at the detailed-level submodels, with initial values for the MQTh, the transaction path length, the busy-wait overhead, and the CPU utilization.
By applying the initialized parameters to the submodels, the values of these parameters are refined and input to the high-level model. The out- put parameters from the high-level model are then fed back to the detailed-level submodels, and this iterative process continues until the MQTh con- verges. In most cases, convergence is reached within a few iterations.
Model Predictions
The back-end portion of the model was validated against measurement results from numerous
Debitcredit benchmarks (Digital's precursor of the
TPC Benchmark A) on many
VAX computers with the
VMS operating system, running
VAX
ACMS and
VAX
Rdb/VMS software? With sufficient detailed parameters available (such as transaction instruc- tion count, instruction cycle time, bus/memory access time, cache hit ratio), the model correctly estimated the MQTh and many intermediate results for several multiprocessor
VAX systems. The model was then extended to include the front-end sys- tems. In this section, we discuss applying this com- plete end-to-end model to the
TPC
Benchmark A on two
VAX
platforms, the
VAX
9000 Model 210 and the
VAX
4000 Model 300 systems, and then compare the results. The benchmark environment and imple- mentation are described in the
TPC
Benchmark A
Implementation section of this paper.
MEMORY
m
I
I
KEY:
----
CLOSED CHAIN
- -
OPEN CHAIN
SOURCE
Figure
I 1
Digital Technical Journal Vo1.3
No. 1
Winter I991
55
Transaction Processing, Databases, and Fault-tolerant Systems
I N I T I A L I Z E :
T x n P L , M Q T h , B u s y W a i t P L , C p u U t i l i z a t i o n ;
L O D - s u b m o d e l ( i n p u t : M Q T h ; o u t p u t : L O D )
REPEAT
I / O - G r o u p i n g - s u b m o d e l ( i n p u t : M Q T h ; o u t p u t : D i o P e r T x n , T x n P L ~ ;
REPEAT
REPEAT
B u s y W a i t - s u b m o d e l ( i n p u t : T x n P L , B ~ s y W a i t P L , C p u U t i l i z a t i o n ,
D i o P e r T x n ; o u t p u t : B u s y W a i t P L ) ;
U N T I L ( B u s y W a i t P L c o n v e r g e s ) ;
C P U - C a c h e - B u s - M e m o r y - s u b m o d e l ( i n p u t : T x n P L , B u s y W a i t P L : o u t p u t : C p u U t i l i z a t i o n , A v g C p u S v c T i m e ) ;
U N T I L ( C p u U t i 1 i z a t i o n c o n v e r g e s ) ;
REPEAT
M Q T h - m o d e l ( i n p u t : A v g C p u S v c T i m e , L O D ; o u t p u t : M Q T h , C p u U t i l i z a t i o n ~ ;
L O D - s u b m o d e l ( i n p u t : M Q T h ; o u t p u t : L O D ) ;
U N T I L ( M Q T h c o n v e r g e s ) ;
U N T I L ( M Q T h c o n v e r g e s ) ;
Figure 12 The Iterative Procedure to Integrating Submodels
Because both the
VAX
9000 Model 210 and the
VAX
4000 Model 300 systems are uniprocessor systems, there is no other processor contending for the processor-memory interconnect and mem- ory subsystems. Such contention effects can there- fore be ignored when modeling a uniprocessor system. The transaction processing performance prediction for t h e
VAx
9000 Model 210 system is a successful example of the application of our analyt- ical model.
We needed an accurate estimate of TPC Bench- mark
A
performance o n t h e
VAx
9000 Model 210 system before a
VAX
9000 system was actually avail- able for testing. The high-level (MQTh) model was used with estimated values for the input parame- ters, LOD and transaction CPU service time. The estimated LOD was based o n previous measure- ment observations from the
VAx
6000 systems. The other parameter, back-end transaction
CPll senlice time, was derived from the
Timing information of the
VAx
9000
CPU
Memory access time and cache miss penalty of the
VAX
9000 CPU
Prediction of cache hit ratio of the v a
9000 sys- tem under the TPC Benchmark
A
workload
Transaction path length of the TPC Benchmark A implementation
Instruction profile of the TPC Benchmark
A implementation
The high-level model predicted a range of MQTh, with a high end of 70 TPS and with a strong proba- bility that the high-end performance a7as achievable.
Additional predictions were made later, when an early prototype version of the
VAX
9000 Model 210 system was available for testing.
A
variant of the
Debitcredit benchmark, much smaller in scale and easier t o run, was performed on the prototype system, with the emphasis o n measuring the CPtj performance in a transaction processing environ- ment. The result was used t o extrapolate the CPU service time of the TPC Benchmark
A
transactions o n the
VAX
9000 Model 210 system and to refine the early estimate. The results of these modifica- tions supported the previous high-end estimate of performance of 70 TPS and refined t h e low-end performance to be 62 TPS. The final, audited TPC
Benchmark
A
measurement result of the
VAX
9000
Model 210 system showed 69.4 TPS, which closely matches the prediction. Table 8 compares t h e results from benchmark measurement and the analvtical model outnuts.
Table 8 Measurement Compared to Model
Predictions
System
Measured
MQTh
Modeled
MQTh
VAX
9000
4000
21
0 69.4
Model
300
21
.5
70.0
20.8
Vo1.
3
No.
1 Winter
I991 Digital Technical Journal
Pe@ormunce Evaluation of Transaction Processing Systems
The
VAX
4000 Model 300 TPC Benchmark A results were also used as a validation case.
VAX
4000
Model 300 systems use the same CMOS chip as the
VAX
6000 Model 400 series and the same
28-nanosecond (ns)
CPU cycle time. However, in the
VAX
4000 series, the CPU-memory interconnect is not the
xMr
bus but a direct primary memory interconnect. This direct memory interconnect results in fast main memory access. The processor, cache, and main memory subsystems are otherwise the same
as
in the
VAX
6000 Model 400 systems.
Therefore, the detailed-level model and associated parameters for the
VAX
6000 Model 410 system can be used by ignoring the bus access time. The
TPC Benchmark A measurement results are within
7
percent of the model prediction, which means that our assumption on the memory access time is acceptable.
Conclusion
Performance is one of the most important attrib- utes in evaluating a transaction processing system.
However, because of the complex nature of trans- action processing systems, a universal assessment of transaction processing system performance is impossible. The performance of a transaction pro- cessing system is workload dependent, configura- tion dependent, and implementation dependent.
A
standard benchmark, like TPC Benchmark A, is a step toward a fair comparison of transaction pro- cessing performance by different vendors. But it is only one transaction processing benchmark that represents a limited class of applications. When evaluating transaction processing systems perfor- mance, a good understanding of the targeted appli- cation environment and requirements is essential before using any available benchmark result.
Additional benchmarks that represent a broader range of commercial applications are expected to be standardized by the Transaction Processing
Performance Council (TPC) in the coming years.
Performance modeling is an attractive alterna- tive to benchmark measurement because it is less expensive to perform and results can be compiled more quickly. Modeling provides more insight into the behavior of system components that are treated as black boxes in most measurement exper- iments. Modeling helps system designers to better understand performance issues and to discover existing or potential performance problems. Model- ing also provides solutions for improving perfor- mance by modeling different tuning or design alternatives. The analytical model presented in this paper was validated and used extensively in many engineering performance studies. The model also helped the benchmark process to size the hard- ware during preparation (e.g., the number of
RTE and front-end systems needed, the size of the database) and t o provide an MQTh goal as a sanity check and a tuning aid. The model could be extended to represent additional distributed configurations, such as shared-disk and "shared- nothing" back-end transaction processing systems, and could be applied to additional transaction p r o cessing workloads.
Acknowledgments
The Digital TPC Benchmark A implementation and measurements are the result of work by many individuals within Digital. The authors would like especially to thank Jim McKenzie, Martha Ryan,
Hwan Shen, and Bob Tanski for their work in the
TPC Benchmark
A
measurement experiments; and
Per Gyllstrom and Rabah Mediouni for their con- tributions to the analytical model and validation.
References
1. Transaction Processing Performance Council,
TPC Benchmark A Standard Specification
(Menlo Park,
CA:
Waterside Associates,
November 1989).
2. Transaction Processing Systems Handbook
(Maynard: Digital Equipment Corporation,
3.
TPC
Benchmark: A Report for the
VRY
9000
Model 210 System (Maynard: Digital Equipment
Corporation, Order No. EC-N0302-57, 1990).
4.
TPC
Benchmark: A Report for the
VAX
4000
Model 300 System (Maynard: Digital Equipment
Corporation, Order No. EC-N0301-57, 1990).
5.
L.
Wright,
W
Kohler, and
W
Zahavi, "The Digital
Debitcredit Benchmark: Methodology and
Results:' Proceedings of the International
Conference on Management and Performance
Evaluation of Computer Systems (December
1989): 84-92.
Digital Technical Journal
Vol.
3
No. 1
Winter 1991
57
डाउनलोड
विज्ञापन