CHAPTER 16
Analyzing Requests and Request
Plans: Application DBAs
This chapter describes how to analyze request plans using the EXPLAIN request modifier and the Query Capture Facility.
Using EXPLAIN to Analyze Request Plans
About the EXPLAIN Request Modifier
The EXPLAIN request modifier reports an English language summary of the request plan generated by the Optimizer to process any valid SQL request. The summary describes the
AMP steps the system would perform to resolve the request, but the request is not processed unless the Optimizer uses a dynamic plan and then the request is partially processed.
Note: To EXPLAIN a request, you must have the privileges required to execute the request.
EXPLAIN is useful for evaluating and optimizing SQL requests. It provides a summary of the access and join plans generated by the Optimizer for the specified SQL request, including information such as:
•
The indexes to be used
•
Any intermediate spool files that will be generated
•
The join types to be performed
•
Whether the requests in a transaction will be dispatched in parallel
•
The relative time it will take to complete the request
•
Whether a dynamic or static request plan will be used
Using EXPLAIN allows you to compare the relative performance of the different versions of a request and choose the most efficient SQL.
Recommendation: Always use EXPLAIN to analyze any new requests under development.
Subtle differences in the way a request is structured can produce significant differences in its resource impact and performance, even though the requests may produce identical end results.
Generating an EXPLAIN with SQL Assistant
1
In the SQL Assistant Query window, enter the SQL request to be EXPLAINed.
2
Press F6.
Note: Shortcut F6 always produces a static request plan. For the dynamic plan, use DYNAMIC
EXPLAIN request.
Database Administration 393
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Capturing Request Plan Steps
For more information, see Teradata SQL Assistant for Microsoft Windows User Guide
.
Generating an EXPLAIN with BTEQ
To generate an EXPLAIN for an SQL request, add the word “EXPLAIN” at the beginning of the request.
For example, the following SQL performs an EXPLAIN on a SELECT statement:
EXPLAIN SELECT * FROM Employee;
If you want a dynamic request plan instead, issue DYNAMIC EXPLAIN request. For example,
DYNAMIC EXPLAIN SELECT * FROM Employee;
Reference Information
Information on...
syntax and options for the
EXPLAIN request modifier interpreting the output of the
EXPLAIN request modifier for various request types
Teradata Visual Explain utility
Is available in...
SQL Data Manipulation Language
SQL Request and Transaction Processing
Teradata Visual Explain User Guide
Capturing Request Plan Steps
About the Query Capture Facility
SQL provides a powerful facility for analyzing the request plans the Optimizer creates for various data manipulation statements. This feature, called the Query Capture Facility, permits a DBA to capture access plan information from the Optimizer using the SQL statements
COLLECT DEMOGRAPHICS, DUMP EXPLAIN, INSERT EXPLAIN, and BEGIN QUERY
CAPTURE (see SQL Data Definition Language for more information about these statements) and then analyze the captured data using the Teradata Index Wizard or standard SQL DML statements.
The captured request plans are stored in a relational database called a Query Capture Database
(QCD) that you can create for that purpose. QCD is a user database whose space is drawn from available permanent disk space: it is not a set of system tables found in a fixed location.
Applications of QCF include:
•
Store all request plans for customer requests. You can then compare and contrast requests as a function of software release, hardware platform, and hardware configuration.
•
Provide data so that you can generate your own detailed analyses of captured request steps using standard SQL DML statements and third-party request management tools.
394 Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Relationship Between BEGIN QUERY CAPTURE and BEGIN QUERY LOGGING
•
Provide the input for these utilities: Teradata Index Wizard (which recommends index definitions for improved efficiency), and Teradata VE (which presents a graphical view of the flow of data during request execution).
About Capturing Request Plans Using DBQL
An alternative to using the Query Capture Facility is to capture request plans using the request BEGIN QUERY LOGGING WITH XMLPLAN. This request causes the request plan for every non-cached INSERT, UPDATE, SELECT, DELETE, MERGE, and EXEC statement to be logged as an XML document in DBC.DBQLXMLTbl. For details, see
“WITH Logging
Options” on page 269
.
Relationship Between BEGIN QUERY CAPTURE and BEGIN QUERY LOGGING
BEGIN QUERY CAPTURE requests always take precedence over BEGIN QUERY LOGGING requests. If you submit a BEGIN QUERY LOGGING request when there is already an active
BEGIN QUERY CAPTURE request on the same user, the query logging takes place for the user in all the sessions except for the CAPTURE session and the query capture session continues capturing queries.
This is true whether the BEGIN QUERY LOGGING statement was issued in the capture session or from some other session. The opposite is true when you submit a BEGIN QUERY
CAPTURE request when there is already an active BEGIN QUERY LOGGING request on the same user. The query capture takes place for the capture session and the query logging for the user continues for all the other sessions.
DBS Control Parameters for Tuning the Workload Cache Limits for Index and Partition Analysis
Index Wizard processing in Teradata Database allocates workload cache for the purpose of caching information retrieved from the QCD and other temporary working structures and needed during Analysis and Validation phases of recommending indexes and partitioning expressions for various SQL workloads.
Two DBS Control performance group parameters control the respective workload cache sizes for the Analysis and Validation phases.
•
IAMaxWorkloadCache
•
IVMaxWorkloadCache
The following table provides some of the details for these parameters. See Utilities for further information.
Database Administration 395
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About the Query Capture Database
Parameter Purpose
IAMaxWorkloadCache Defines the maximum size of the Index
Wizard workload cache for Analysis operations.
This parameter is applicable to both the
INITIATE INDEX ANALYSIS and
INITIATE PARTITION ANALYSIS statements.
IVMaxWorkloadCache Defines the maximum size of the Index
Wizard workload cache for Validation operations.
This parameter is applicable to all SQL statements issued within a session when
DIAGNOSTIC “Validate Index” has been enabled.
Valid Range
(megabytes)
32 - 256
1 - 32
Default
(megabytes)
8
48
About the Query Capture Database
A query capture database, or QCD, stores the Optimizer query analysis workload data and the statistical and demographic data needed to support query analysis.
The SystemFE DIP creates a default QCD called TDQCD beneath the SystemFE user, but you can also create your own QCD databases, and you can define multiple QCD databases for your systems.
You must be granted the INSERT privilege to use a BEGIN QUERY CAPTURE request to capture query plans in XML format at the session level and write data into TDQCD or any other user-defined QCD.
Teradata Database determines the size of TDQCD based on the number of system AMPs using the following equation.
TDQCD_size = 50MB +
number_of_system_AMPs x 100 x 1,024 bytes
You can use the modify_tdqcd.spl SQL stored procedure to modify the size of the TDQCD database based on the number of AMPs in a test system. The definition of modify_tdqcd.spl is as follows.
REPLACE PROCEDURE TDQCD.MODIFY_TDQCD(IN Basesize int, OUT Success int)
BEGIN
DECLARE TotalAmps INTEGER;
DECLARE CurrentSize INTEGER;
DECLARE TargetSize INTEGER;
SELECT COUNT(DISTINCT Vproc) FROM Dbc.DiskSpaceV INTO TotalAmps ; select ZEROIFNULL(permspace) into :CurrentSize from Dbc.DatabasesV where databasename =
'tdqcd';
396 Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About the Query Capture Database
SET TargetSize = Basesize + (TotalAmps * 100 * 1024);
IF (CurrentSize < TargetSize) then
CALL dbc.sysexecsql('MODIFY DATABASE TDQCD AS PERM = ' ||:TargetSize || 'BYTES');
END IF;
SET Success = 1;
END;
Use the Control Center feature of the Visual EXPLAIN tool (see “Procedure Using the Visual
EXPLAIN Utility” on page 398 ) or BTEQ using the CreateQCF table (see
“Procedure Using
BTEQ” on page 399
) to create your query capture databases.QCD tables are populated using various methods, including the following:
•
BEGIN QUERY CAPTURE
•
COLLECT DEMOGRAPHICS
•
COLLECT STATISTICS (QCD Form)
•
INSERT EXPLAIN … WITH STATISTICS
•
BEGIN QUERY CAPTURE
•
Invoking workload macros from the client-based Teradata Index Wizard utility.
Notice:
Because the QCD is often a set of user tables, you can change both their individual structure and the structure of the QCD; however, you should never change any of the QCD structures in any way.
The reason for this constraint is that the SQL statements BEGIN QUERY CAPTURE, DUMP
EXPLAIN, INSERT EXPLAIN, INITIATE INDEX ANALYSIS, RESTART INDEX ANALYSIS,
COLLECT DEMOGRAPHICS, and COLLECT STATISTICS (QCD Form) all assume the default physical model when they capture the specified query, statistical, and demographic information for insertion into the QCD tables.
The results of changing the structure of the QCD tables are unpredictable.
Deleting Query Plans from a QCD
6
7
Use the Control Center option of Visual EXPLAIN to delete query plans from a QCD.
3
4
1
2
5
Start Visual EXPLAIN.
Pull down the Tools menu and select Control Center.
Select the Manage QCD tab.
Select the Clean up QCD button.
Select the database from which query plans are to be removed from the Database scroll bar.
Select the Delete Database Objects radio button.
Select the OK button.
Database Administration 397
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Creating the Query Capture Database Tables
Creating the Query Capture Database Tables
About Creating QCD Tables
Before you can capture request plan information from the Optimizer white tree and statistical and data demographic information from their respective data structures, you must either create and secure the QCD tables or create your own user-defined query capture database.
This topic explains two ways to set up your QCDs.
Procedure Using the Visual EXPLAIN Utility
Perform the following procedure to create the tables and other database objects for the query capture database using Visual EXPLAIN.
If you do not have the Visual EXPLAIN tool, then see
“Procedure Using BTEQ” on page 399 .
1
2
Start Visual EXPLAIN.
Pull down the Tools menu and select Control Center.
7
8
9
10
11
5
6
3
4
Select the Manage QCD tab.
Select the Setup QCD button.
Select the Create all QCF database objects (tables and macros) radio button.
Type the name of the database owner in the Owner selection box.
Specify the Perm Space parameter in the Perm Space selection box.
Specify the Spool Space parameter in the Spool Space selection box.
Check the Fallback check box depending on your requirements.
Select the Create button.
Secure the QCD tables by granting the appropriate privileges to the users who will be analyzing its data.
You can implement additional secured access by creating views on QCD.
12
Populate the QCD tables using the appropriate tools.
•
BEGIN QUERY CAPTURE requests to capture request plans. For the syntax of this statement, see SQL Data Definition Language Syntax and Examples.
•
INSERT EXPLAIN requests to capture query plans. For the syntax of this statement, see SQL Data Manipulation Language.
398 Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Creating the Query Capture Database Tables
•
Any of the following statements, as appropriate, to capture statistical and demographic data:
•
COLLECT DEMOGRAPHICS
•
COLLECT STATISTICS (QCD Form)
•
INSERT EXPLAIN … WITH STATISTICS
•
The appropriate workload macros to create workloads for index analysis.
Procedure Using BTEQ
Perform the following procedure to create the tables for the query capture database using
BTEQ. This procedure does not create the macros that the Visual EXPLAIN utility needs to work with a QCD. You must create these macros from the Visual EXPLAIN Control Center before the utility can visualize or load QCD query plans.
The procedure assumes that you have already created QCD_database_name.
1
2
Start BTEQ.
Change your current database or user to the database in which the QCD tables are to be created as follows.
3
DATABASE QCF_database_name; where QCF_database_name is the name of the database you created for the QCD tables.
Perform the following steps in the indicated order.
a .SET WIDTH 254
b .EXPORT FILE = file_name
c SELECT TabDefinition
FROM systemfe.CreateQCF
ORDER BY SeqNumber;
4
d .EXPORT FILE = file_name
e .RUN FILE = file_name where file_name is the name of the file you create to contain the output of the SELECT request executed in
step c
.
Secure QCD by granting the appropriate privileges to the users who will be analyzing its data.
To use the BEGIN QUERY CAPTURE request to insert data into the QCD tables in
TDQCD, you must have the ALL privilege.
See SQL Data Control Language for information about how to implement column-level and row-level security for your tables.
You can implement additional secured access by creating views on QCD.
Database Administration 399
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Dropping Query Capture Database Tables
5
Populate the QCD tables using the appropriate tools.
•
BEGIN QUERY CAPTURE requests to capture query plans. For the syntax of this statement, see SQL Data Definition Language Syntax and Examples.
•
INSERT EXPLAIN requests to capture request plans. For the syntax of this statement, see SQL Data Manipulation Language.
•
Any of the following statements, as appropriate, to capture statistical and demographic data.
•
COLLECT DEMOGRAPHICS
•
COLLECT STATISTICS (QCD Form)
•
INSERT EXPLAIN … WITH STATISTICS
•
The appropriate workload macros to create workloads for index analysis.
Client tools like Visual EXPLAIN and the Teradata Index Wizard access QCD tables using the views and macros created using the Control Center feature of Visual EXPLAIN.
Dropping Query Capture Database Tables
About Dropping QCD Tables
This topic describes how to drop all the tables and other database objects from a QCD database.
Procedure Using Visual EXPLAIN
Perform the following procedure to drop the tables and other database objects for the query capture database using Visual EXPLAIN.
If you do not have the Visual EXPLAIN tool, then see
“Procedure Using BTEQ,”
below.
5
6
7
3
4
1
2
Start Visual Explain.
Pull down the Tools menu and select Control Center.
Select the Manage QCD tab.
Select the Clean up QCD button.
Select the database to be cleaned up from the Database scroll bar menu.
Select the Delete Database Objects radio button.
Select the OK button.
Procedure Using BTEQ
Perform the following procedure to drop the tables for the query capture database using
BTEQ. The procedure assumes that you have already created QCF_database_name. This procedure does not drop the views and macros on the specified QCD.
400 Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About Querying QCD Tables
1
2
3
Start BTEQ.
Change your current database to the database from which the QCF tables are to be dropped as follows.
DATABASE QCD_database_name; where QCD_database_name is the name of the database you created for the QCD tables.
Perform the following steps in order.
a .EXPORT FILE = file_name
b SELECT DelTable
FROM systemfe.CleanupQCF
ORDER BY SeqNumber;
c EXPORT RESET;
d .RUN FILE = file_name where file_name is the name of the file you create to contain the output of the SELECT request in
step c .
About Querying QCD Tables
You can analyze the information in QCD using several different approaches.
The following methods are typical ways to use a QCD to analyze query data.
•
To create and compare graphical EXPLAIN reports, use the Visual EXPLAIN tool.
•
To perform ad hoc requests of QCD, use interactive SQL DML statements.
•
To perform standardized analyses of QCD, create macros, stored procedures, or embedded
SQL applications to perform your standard requests and report the results.
See
“About QCD Request Macros and Views” on page 401 for a set of macros you can use
for this purpose.
•
To run index validation diagnostics for the Teradata Index Wizard, use the DIAGNOSTIC
“Validate Index” statement. See SQL Data Manipulation Language for the syntax of the
DIAGNOSTIC “Validate Index” statement.
About QCD Request Macros and Views
Teradata Database provides a set of views on the QCD tables to restrict their access. Various levels of access to QCD tables are granted to different user categories.
Teradata Database also provides a set of macros for maintaining various aspects of the query capture database. The Control Center feature of the Visual EXPLAIN client utility provides the interface to create the views and macros and to grant privileges to QCD users.
Database Administration 401
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About Querying QCD Tables
QCD Macro and View Versions
Two versions of all the views and macros are created in a QCD.
X
NonX
Version Definition
Access restricted to information inserted by the current user in the QCD.
Access permitted to information inserted by any user of the QCD.
These macros and views are usually performed by means of the Visual EXPLAIN and Teradata
Index Wizard client utilities.
User Categories
The following categories of users are defined on a QCD to enhance its security.
User Category
Normal
Power
Administrator
Description
Loads, views, and deletes own plans or workloads only.
Loads and views plans or workloads inserted by any user.
Deletes own plans or workloads only.
Loads, view, and deletes any plan created by any user.
Drops and deletes QCD tables.
QCD creator has Administrator privileges granted by default.
Specific User Category Privileges
The following table indicates the specific privileges granted to each user category.
User Category
Normal
Database Object Type
QCD tables
X views
X macros
Non-X views
Non-X macros
Privileges Granted
•
INSERT
•
UPDATE
•
SELECT on the following tables.
•
IndexRecommendations
•
SeqNumber
•
TableStatistics
•
DELETE on the AnalysisLog table
SELECT
EXEC none none
402 Database Administration
Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About Querying QCD Tables
User Category
Power
Administrator
Database Object Type
QCD tables
X views
X macros
Non-X views
Non-X macros
QCD tables
X views
X views use the value for
UDB_Key in the Query table to restrict access to request plans to the user who captures them, while the Non-X views do not restrict that access.
X macros
Non-X views
Non-X macros
•
DROP
•
EXECUTE
•
DELETE
•
DROP
•
INSERT
•
SELECT
•
UPDATE
•
DROP
•
EXECUTE
Privileges Granted
•
INSERT
•
UPDATE
•
SELECT on the following tables:
•
IndexRecommendations
•
SeqNumber
•
TableStatistics
•
DELETE on the AnalysisLog table
SELECT
EXEC
SELECT
EXEC (excluding DELETE plan and workload macros)
•
DELETE
•
DROP
•
INSERT
•
SELECT
•
UPDATE
•
DELETE
•
DROP
•
INSERT
•
SELECT
•
UPDATE
403
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Building Baseline Transaction Profiles
Building Baseline Transaction Profiles
Baseline profiles can provide information on typical resource usage by period and by user, on a daily, weekly, or monthly basis. Building baseline transaction profiles can help you track the effects of software upgrades, introductions of new users or new applications, and help you determine what a “healthy and normal” system should look like.
You can build baseline profiles for:
•
Single operations (such as FastLoad, full table scans, primary index INSERT ... SELECTs, select joins, and so on.)
•
Multiple, concurrently run transactions
Once defined and stored, baseline profiles can help you:
•
Compare current to profiled operations on a real-time basis.
•
Collect data instantaneously for a set interval.
•
Detect and resolve throughput anomalies.
You can gather performance metrics for profiling from:
•
ResUsage reports. See Resource Usage Macros and Tables.
•
DBQL. See Chapter 15: “Tracking Query Behavior with Database Query Logging:
Operational DBAs.”
•
Viewpoint.
404 Database Administration
