Teradata Release 15.10 Database User Manual
Anúncio
Anúncio
Teradata Database
Database Administration
Release 15.10
B035-1093-151K
September 2017
The product or products described in this book are licensed products of Teradata Corporation or its affiliates.
Teradata, Aster, BYNET, Claraview, DecisionCast, IntelliBase, IntelliCloud, IntelliFlex, QueryGrid, SQL-MapReduce, Teradata Decision Experts,
"Teradata Labs" logo, Teradata ServiceConnect, and Teradata Source Experts are trademarks or registered trademarks of Teradata Corporation or its affiliates in the United States and other countries.
Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc.
Amazon Web Services, AWS, Amazon Elastic Compute Cloud, Amazon EC2, Amazon Simple Storage Service, Amazon S3, AWS
CloudFormation, and AWS Marketplace are trademarks of Amazon.com, Inc. or its affiliates in the United States and/or other countries.
AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc.
Apache, Apache Avro, Apache Hadoop, Apache Hive, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.
Apple, Mac, and OS X all are registered trademarks of Apple Inc.
Axeda is a registered trademark of Axeda Corporation. Axeda Agents, Axeda Applications, Axeda Policy Manager, Axeda Enterprise, Axeda
Access, Axeda Software Management, Axeda Service, Axeda ServiceLink, and Firewall-Friendly are trademarks and Maximum Results and
Maximum Support are servicemarks of Axeda Corporation.
CENTOS is a trademark of Red Hat, Inc., registered in the U.S. and other countries.
Cloudera and CDH are trademarks or registered trademarks of Cloudera Inc. in the United States, and in jurisdictions throughout the world.
Data Domain, EMC, PowerPath, SRDF, and Symmetrix are either registered trademarks or trademarks of EMC Corporation in the United
States and/or other countries.
GoldenGate is a trademark of Oracle.
Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company.
Hortonworks, the Hortonworks logo and other Hortonworks trademarks are trademarks of Hortonworks Inc. in the United States and other countries.
Intel, Pentium, and XEON are registered trademarks of Intel Corporation.
IBM, CICS, RACF, Tivoli, IBM Spectrum Protect, and z/OS are trademarks or registered trademarks of International Business Machines
Corporation.
Linux is a registered trademark of Linus Torvalds.
LSI is a registered trademark of LSI Corporation.
Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United
States and other countries.
NetVault is a trademark of Quest Software, Inc.
Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries.
Oracle, Java, and Solaris are registered trademarks of Oracle and/or its affiliates.
QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation.
Quantum and the Quantum logo are trademarks of Quantum Corporation, registered in the U.S.A. and other countries.
Red Hat is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Used under license.
SAP is the trademark or registered trademark of SAP AG in Germany and in several other countries.
SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc.
Sentinel® is a registered trademark of SafeNet, Inc.
SPARC is a registered trademark of SPARC International, Inc.
Unicode is a registered trademark of Unicode, Inc. in the United States and other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Veritas, the Veritas Logo and NetBackup are trademarks or registered trademarks of Veritas Technologies LLC or its affiliates in the U.S. and other countries.
Other product and company names mentioned herein may be the trademarks of their respective owners.
T
HE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN
"
AS
-
IS
"
BASIS
,
WITHOUT WARRANTY OF ANY KIND
,
EITHER EXPRESS OR
IMPLIED
,
INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY
,
FITNESS FOR A PARTICULAR PURPOSE
,
OR NON
-
INFRINGEMENT
. S
OME
JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES
,
SO THE ABOVE EXCLUSION MAY NOT APPLY TO YOU
. I
N NO EVENT WILL
T
ERADATA
C
ORPORATION BE LIABLE FOR ANY INDIRECT
,
DIRECT
,
SPECIAL
,
INCIDENTAL
,
OR CONSEQUENTIAL DAMAGES
,
INCLUDING LOST PROFITS
OR LOST SAVINGS
,
EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES
.
The information contained in this document may contain references or cross-references to features, functions, products, or services that are not announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions, products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or services available in your country.
Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time without notice.
To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document. Please email: [email protected].
Any comments or materials (collectively referred to as "Feedback") sent to Teradata Corporation will be deemed non-confidential. Teradata
Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform, create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata
Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including developing, manufacturing, or marketing products or services incorporating Feedback.
Copyright © 2000-2017 by Teradata. All Rights Reserved.
Preface
Purpose
Audience
This book covers the basics of Teradata database administration, giving you the essential tasks and the procedures for doing them. Chapters in this book are either designed for all DBAs or split tasks into two major groupings indicated in the chapter title: those for application DBAs, who are concerned with the design and performance of queries, and administrative DBAs, who are responsible for the day-to-day operation and maintenance of the data warehouse.
This book provides more details on topics of interest primarily to database administrators.
Where complete syntax is necessary or in-depth details are required, this book may point you to other books with more extensive information.
This book is intended for database administrators (DBAs). It is also useful for system operators.
Supported Software Releases and Operating
Systems
This book supports Teradata® Database 15.10.x
Teradata Database 15.10.x is supported on:
•
SUSE Linux Enterprise Server 10 SP3
•
SUSE Linux Enterprise Server 11 SP1
•
SUSE Linux Enterprise Server 11 SP3 (Release 15.10.01 and later)
Teradata Database client applications support other operating systems.
Prerequisites
You should have some knowledge of the following:
•
Relational database concepts
•
Teradata SQL
•
The Teradata Database configuration
Database Administration 3
4
Preface
Changes to This Book
You might find it helpful to review the following publications:
•
Introduction to Teradata
•
Database Design
•
SQL Fundamentals
The following publications are directly related to the information in this book and you may find it helpful to have them available for reference:
•
SQL Data Definition Language
•
Utilities
•
Security Administration
•
Data Dictionary
•
The suite of Teradata Tools and Utilities client documents
Changes to This Book
Release
Teradata Database Release 15.10
September 2017
Teradata Database Release 15.10
July 2015
Description
Added information about the following target tables used in shredding the lock plan information in DBQLXMLLockTbl:
•
Shredded_LockTbl: blocking transaction lock shredding target table
•
Shredded_TransTbl: blocked transaction lock shredding target table
•
Updated information on mainframe channel and network connections.
•
The WITH option of END QUERY LOGGING is deprecated.
•
Specified that you cannot log use count information for account strings.
•
Added the QryLogParamJSON view. Changed the
QryLogParam view output to CLOB type.
•
Added step names and their explanations for DBQLStepTbl.
•
The Recovery Manager commands LIST ROLLBACK
TABLES and LIST CANCEL ROLLBACK TABLES now include DBQL and ResUsage tables.
•
Updated the description of the RollbackPriority DBS control field.
Database Administration
Preface
Additional Information
Release
Teradata Database Release 15.10
June 2015
Description
•
Moved the data compression chapter from this book to
Database Design.
•
Reflected that PDE tool mboxchk replaces dbschk in detecting database hangs and slowdowns.
•
Incorporated Trusted Sessions, CheckTable Enhancements,
MLOADX Enhancements, Load Isolation, Secure Zones,
Columnar Enhancements, DSA: Allow Restore Jobs To
Complete Even When Out of Space, SQL Equivalents of
Ferret Output, Support Audit Trail of DBS Control GDO,
New AWT Resource Throttle, NO COLUMNS suboption to the OBJECT logging option of BEGIN QUERY LOGGING, and Combining Multiple DBQL XML Plan Rows in One
Document.
•
Added expanded deadlock information and revised shredding instructions to the DBQLXMLLockTbl section.
•
Noted that Teradata logs a warning if the number of crashdumps saved on a node is near the limit.
•
Removed documentation on troubleshooting a snapshot dump. The Onsite System and Dump Analysis tool is for
Teradata Customer Support only.
•
Removed references to 30-character limits for object names.
Teradata Database supports object names up to 128 characters long in most cases. For more information on object naming see SQL Fundamentals.
Additional Information
URL
www.info.teradata.com/
Description
Use the Teradata Information Products Publishing Library site to:
•
View or download a manual:
1
2
Under Online Publications, select General Search.
Enter your search criteria and click Search.
•
Download a documentation CD-ROM:
1
2
Under Online Publications, select General Search.
In the Title or Keyword field, enter CD-ROM, and click
Search.
Database Administration 5
6
Preface
Product Safety Information
URL
www.teradata.com
www.teradata.com/t/TEN/ https://tays.teradata.com/ developer.teradata.com/
Description
The Teradata home page provides links to numerous sources of information about Teradata. Links include:
•
Executive reports, white papers, case studies of customer experiences with Teradata, and thought leadership
•
Technical information, solutions, and expert advice
•
Press releases, mentions and media resources
Teradata Customer Education delivers training that builds skills and capabilities for our customers, enabling them to maximize their Teradata investment.
Use Teradata @ Your Service to access Orange Books, technical alerts, and knowledge repositories, view and join forums, and download software patches.
Teradata Developer Exchange provides articles on using
Teradata products, technical discussion forums, and code downloads.
To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document. Please email [email protected]
.
Product Safety Information
This document may contain information addressing product safety practices related to data or property damage, identified by the word Notice. A notice indicates a situation which, if not avoided, could result in damage to property, such as equipment or data, but not related to personal injury.
Example:
Notice:
Improper use of the Reconfiguration utility can result in data loss.
Teradata Database Optional Features
This book may include descriptions of the following optional Teradata Database features and products:
•
In-Memory Optimization
•
Teradata Columnar
•
Teradata Row Level Security
•
Teradata Secure Zones
•
Teradata Temporal
Database Administration
Preface
Teradata Database Optional Features
•
Teradata Virtual Storage (VS)
You may not use these features without the appropriate licenses. The fact that these features may be included in product media or downloads, or described in documentation that you receive, does not authorize you to use them without the appropriate licenses.
Contact your Teradata sales representative to purchase and enable optional features.
Database Administration 7
Preface
Teradata Database Optional Features
8 Database Administration
Table of Contents
Database Administration
Preface
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Chapter 1: Setting Up Your Administrative Environment:
All DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
9
Table of Contents
10
Chapter 2: About Databases and Users in Teradata:
All DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
Chapter 3: Working with Databases: All DBAs
. . . . . . . . . . . . . . . . . .55
Chapter 4: Working with Tables and Views:
Application DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65
Database Administration
Table of Contents
Database Administration
Chapter 5: Working with Stored Procedures and
User-defined Functions: Application DBAs
. . . . . . . . . . . . . . . . . . . . . . 91
Example 1: Compiling and Executing a Stored Procedure with a Cursor . . . . . . . . . . . . 91
Example 2: Creating a Stored Procedure to Update Accrued Interest . . . . . . . . . . . . . . . 92
Chapter 6: Working with Users, Roles, and Profiles:
Operational DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
11
Table of Contents
12
Chapter 7: Loading and Exporting Data:
Application DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135
Maximum Number of Concurrent FastLoad, MultiLoad, and FastExport Jobs. . . . . . . . . . .143
Database Administration
Table of Contents
Chapter 8: Working with Sessions and Accounts:
Operational DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Database Administration
Chapter 9: Managing Space: Operational DBAs
. . . . . . . . . . . . . . . 171
13
Table of Contents
14
Finding Loosely Packed Tables with the SHOWFSP Command. . . . . . . . . . . . . . . . . . . .189
Chapter 10: Maintaining the Database: Operational DBAs
. .195
Database Administration
Database Administration
Table of Contents
Managing Vproc Status and Initializing AMP Disks with Vproc Manager. . . . . . . . . . . . . . 198
Chapter 11: Archiving, Restoring, and Recovering Data:
Operational DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Archiving and Restoring Tables That Use Block-Level Compression. . . . . . . . . . . . . . . 210
Archiving and Restoring Tables Using Temperature-based Compression. . . . . . . . . . . 210
Archiving and Restoring Column-Partitioned Tables and Databases . . . . . . . . . . . . . . 210
15
Table of Contents
16
Chapter 12: Managing Database Resources:
Operational DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .227
Managing the Database Workload with Teradata Active System Management . . . . . . . . . . .228
About Defining Name-Value Pairs for Session and Transaction Query Bands . . . . . . . .232
Chapter 13: Managing Queries: Operational DBAs
. . . . . . . . . . .243
Database Administration
Table of Contents
Chapter 14: Improving Query Performance Using COLLECT
STATISTICS: Application DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Skipping Unneeded Statistics Recollection with the THRESHOLD Option. . . . . . . . . . . . . 256
Database Administration
Chapter 15: Tracking Query Behavior with Database
Query Logging: Operational DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
BEGIN/REPLACE/END QUERY LOGGING Statements . . . . . . . . . . . . . . . . . . . . . . . . 268
17
Table of Contents
18
Teradata Dynamic Workload Management View of the Main Log Table:
View of the Parameterized Query Log Table: QryLogParamJSON[V] . . . . . . . . . . . . . . . . . .313
Shredded_LockTbl: Blocking Transaction Lock Shredding Target Table . . . . . . . . . . . .354
Shredded_TransTbl: Blocked Transaction Lock Shredding Target Table . . . . . . . . . . . .359
Database Administration
Table of Contents
Relationship Between DBQL Temporary Tables and DBQL History Tables . . . . . . . . . . . . 382
Effects of Dynamically Enabling/Replacing/Disabling Logging on Current Rules . . . . 390
Database Administration
Chapter 16: Analyzing Requests and Request Plans:
Application DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
Relationship Between BEGIN QUERY CAPTURE and BEGIN QUERY LOGGING . . . . . 395
19
Table of Contents
20
DBS Control Parameters for Tuning the Workload Cache Limits for Index and
Chapter 17: Testing Query Performance Using
Target-Level Emulation: Application DBAs
. . . . . . . . . . . . . . . . . . . . . .405
Capturing Target System Optimizer Cost Values and Dynamic AMP Statistical Samples . .408
Using Target Level Emulation to Capture Target System Cost Profile Data . . . . . . . . . .411
Capturing and Installing Costing Information Including the Active Cost Profile . . . . . . . . .413
Procedure Using SQL DIAGNOSTIC Statement Requests . . . . . . . . . . . . . . . . . . . . . . . .413
Database Administration
Database Administration
Table of Contents
Chapter 18: Working with System Information and
Global Defaults: Operational DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
Changing Time Zones, Currencies, and Formats for Date, Time, and Numbers. . . . . . . . . 421
21
Table of Contents
22
Chapter 19: Troubleshooting: Operational DBAs
. . . . . . . . . . . . . .435
Appendix A: How to Read Syntax Diagrams: All DBAs
. . . . . . .459
Appendix B: Teradata System Limits: All DBAs
. . . . . . . . . . . . . . . .465
User-Defined Data Type, ARRAY Data Type, and VARRAY Data Type Limits . . . . . . .477
Database Administration
Table of Contents
Database Administration
Appendix C: Performance Management: All DBAs
. . . . . . . . . . . . 483
23
Table of Contents
24
Database Administration
Table of Contents
Performance Considerations when Expanding or Upgrading the Database System . . . . . . 550
Database Administration
Appendix D: Handling Teradata Crashdumps:
Operational DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
Ad-Hoc Creation of DBC.Crashdumps with DIPCRASH . . . . . . . . . . . . . . . . . . . . . . . . 561
25
Table of Contents
Appendix E: Considerations for Administering a System with Unity Director: All DBAs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .579
Index
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .583
26 Database Administration
CHAPTER 1
Setting Up Your Administrative
Environment: All DBAs
This chapter describes user DBC and provides a brief overview of space management. The chapter also describes these initial administrative tasks:
•
Logging in to the operating system
•
Setting up the database administrator user
•
Creating a spool reserve database
•
Setting up Teradata Administrator
•
Establishing Viewpoint alerts
•
Changing the PasswordExpire parameter
Logging in to the Operating System
Teradata Database administrators who need access to system nodes should log on to the operating system as a user who is a member of the tdtrusted group or as the root user.
About Root Logons
Root access is required to start Teradata Database and install new versions. Users can perform all other Teradata administrative functions, including switching to a new Teradata version once it is installed, as a member of the tdtrusted group.
About Tdtrusted Logons
The tdtrusted group is created automatically at installation of Teradata Database. Tdtrusted access allows nearly all command-line monitoring and administration of Teradata. The advantages of using tdtrusted include greater security because root access is limited and greater accountability because administrative user actions are recorded under each user account rather than as the root user.
Non-root accounts that administer Teradata must be members of the tdtrusted group on all
Teradata Database nodes. Each customer site should determine how to implement this within site-specific account management policies. Tdtrusted logons rely on standard UNIX account management facilities.
Database Administration 27
Chapter 1: Setting Up Your Administrative Environment: All DBAs
About User DBC
For More Information
For more information about the users and groups created automatically during Teradata installation, see “Working with OS-Level Security Options” in the chapter “Setting Up the
Administrative Infrastructure” in Security Administration.
About User DBC
When you first install Teradata Database, it has only one user. This user is called DBC, and from it all other future databases and users in the system are created.
User DBC also initially owns all the space in the entire system. As you create new databases and users, any permanent space you grant them is subtracted from available permanent space in user DBC. Because all space ultimately comes from user DBC, user DBC owns all database objects.
User DBC contains data like a database, but unlike a database it has a password. The usable
(PERM) disk space in DBC initially reflects the entire system hardware capacity, minus space for the following system users, databases, and objects:
•
All the space available for your databases, users, journals, data tables, indexes, stored procedures, functions, and temporary tables. DBC owns all unallocated space.
•
Crashdumps user
•
SysAdmin user
•
Sys_Calendar database
•
TD_SYSFNLIB database
•
SQLJ, SYSLIB, and SYSUDTLIB databases for external routines and user-defined types.
For details, see SQL External Routine Programming.
•
SYSSPATIAL database for geospatial data types. For details, see SQL Geospatial Types.
•
DBCExtension database for Global and Persistent (GLOP) sets. For details, see
“DBCExtension Tables” in Data Dictionary.
•
SystemFE user for field engineers
•
TDPUSER user
•
The system Transient Journal (TJ) which stores the before-change image of every data row involved in a transaction, the row ID of every inserted row, and other control records used by the system to automatically recover data from transaction aborts or to complete transactions.
•
The system catalog tables of the Data Dictionary and the suite of user-accessible views defined on those tables (for details, see Data Dictionary).
This also includes a suite of special system query logging tables used to store query logging rules and data when you enable the DBQL feature. See
Behavior with Database Query Logging: Operational DBAs.”
28 Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up Teradata Administrator
Note: The setting of DefaultCharacterSet in the DBS Control utility determines the default character set for user DBC. For information on changing the character set for user DBC, see
“Default Server Character Set” on page 420 .
Notice:
Never alter the privileges for user DBC. Changing DBC privileges may cause installation, upgrade, maintenance, or archive procedures to end abnormally and consequently require
Teradata Customer Support to correct the problem.
Notice:
If you use MODIFY USER to change the default character set for user DBC, user-defined functions may no longer work for this user without being recompiled, you must rerun the
DIPUDT and DIPDEM scripts, and site-defined functions that use SQL_TEXT must be recompiled.
Setting Up Teradata Administrator
Set Up the Teradata Administrator GUI
1
2
From the principal administrative client, access Teradata Administrator, click
Start>Programs>Teradata Administrator.
The Please Select a Data Source dialog box appears. Do the following:
3
4 a
In the left field, highlight the alias name for your system.
b a
In the Root Database field enter DBC.
c
In What to Load, select All Databases and Users and click OK.
In the Teradata Database Connect dialog box do the following:
In the Username field enter DBC.
b
In the Password field, enter the user DBC password provided by your Teradata
Customer Service representative.
c
Do not change other fields. Click OK.
Set Preview Mode as the default so that SQL statements generated from using the GUI fields and controls will appear in the Query Window before the statement is executed.
a
From the Teradata Administrator menu, click Tools>Options.
b
Click the Confirmation tab.
c
Check the Preview SQL before submitting check box and click OK.
Note: Setting Preview Mode as the default is optional, but it will help new Teradata
Database users to become familiar with Teradata Database SQL statements. You can send generated SQL without previewing it at any time by unchecking the Preview SQL before
submitting check box.
Database Administration 29
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
Set Up the Data Directory for Teradata Administrator
The Teradata Administrator data directory stores generated SQL for later reference and use.
The default location is:
Documents and Settings\Network_UserId\My Documents\Teradata\Teradata Manager\Data.
If you can use the default location, skip this procedure.
If you want to change the location, use the following procedure to relocate the data directory.
1
2
3
From the Teradata Administrator main window, click Tools>Options.
Click the Directory tab.
Note the following directory display fields:
•
Old Path - The current path to the data directory
•
New Path - The new path to the data directory
Navigate to the desired directory.
The directory path changes automatically in the New Path box.
4
Click OK to save your path changes and exit the dialog box.
Setting Up the Database Administrator User
Create the principal database administrator, user DBADMIN (or the name your site uses), and grant space and privileges to the user before proceeding with database implementation. User
DBADMIN can then create databases, tables, users, and other objects in the space it owns.
Note: For larger systems, you may find it useful to create additional administrative users to share database administration duties. See the sections beginning with “Administrative Users” in Chapter 3 of Security Administration.
Creating the DBADM Profile
1
2
Profiles determine certain system resources available to member users, and can also set password control parameters.
From an administrative client, log on to Teradata Administrator as user DBC.
Create the profile for the database administrator, user DBADMIN:
a
From the menu, click Tools>Create>Profile.
b
Specify values for the following fields and controls:
Note: Do not enter values at this time for fields not listed.
30 Database Administration
Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
Field Description
Profile Name Required. The name of the database administrator profile.
Administrative procedures in this publication use the name DBADM for the administrator profile.
Note: User DBADMIN will be the only member of this profile. Because of space and ownership concerns, you must create a separate profile for other administrative users.
On the General tab...
Account Optional. An account string defines the following characteristics for profile member sessions in the database:
•
Session priority
•
Account ID (to assign charges for system time and resources)
•
Date stamp
•
Time stamp
Recommendation: Define at least one account string per profile. If necessary, you can add an account string later using a MODIFY PROFILE statement.
For more about accounts, see Chapter 1: “Setting Up Your Administrative
Password
Spool Space
Required. The temporary password for DBADMIN.
Recommendation: Create a simple temporary password. Each user is prompted to change the temporary password to a permanent, private password upon first logon to the database.
Optional. The spool space specification is only a limit to the amount of space available for intermediate query results or formatted answer sets to queries and volatile tables. The system borrows spool space for a user from unused permanent space anywhere in the system.
Recommendation: Set this value equal to the amount of space in the
Spool_Reserve database.
Temporary
Space
Not applicable. Required only when using global temporary tables and other advanced features.
Recommendation: Temporary space specification is not normally needed.
Specification of temporary space requires that you develop a strategy for use of the advanced features that need temporary space. If necessary, you can add a temporary space specification later with a MODIFY USER statement.
Comment Optional. Use this space to describe the profile or note contingencies.
On the Password tab...
Expire Required. The number in days until a password expires. Set this value to a positive, non-zero number to cause passwords to expire.
Recommendation: 90 days is a common password expiration interval.
31
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
Field
Other password controls
Description
Optional.
Recommendation: Set other password controls globally by updating
DBC.SysSecDefaults if required by your site security policy.
Note: Be sure to review the default password control values to determine whether they conform to your site security requirements.
For information on the default password control values and how to change them, see Chapter 6: “Managing Database Passwords,” in Security
Administration.
c d
Do not use the Users in Profile dialog box. You can specify profile membership in
Click the Create button to create the profile. The Query Window displays a generated
CREATE PROFILE statement similar to the following:
CREATE PROFILE "dbadm" AS
SPOOL=spool_space
TEMPORARY=NULL
ACCOUNT=('$H-DBC-MANAGER',' account__str2',' account_str3')
DEFAULT DATABASE="All"
PASSWORD = password
(EXPIRE=90,MINCHAR=NULL,MAXCHAR=NULL,MAXLOGONATTEMPTS=NULL,LOCKEDU
SEREXPIRE=NULL,REUSE=NULL,DIGITS=NULL,RESTRICTWORDS=NULL,SPECCHAR=
NULL)
f e
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
Close the Query Window and the Create Profile dialog box.
Reference Information
For step...
Information on...
2 setting up user accounts the default password control values and how to change them
Is available in...
“Creating User Accounts” on page 108 .
Security Administration
Creating User DBADMIN
1
2
From an administrative client, log on to Teradata Administrator as user DBC.
Create the database administrator as follows:
a
From the menu, click Tools>Create>User.
b
Specify values for the following fields and controls:
Note: Do not enter values at this time for fields not listed.
32 Database Administration
Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
Field Description
User Name
Owner
Password
Permanent
Space
Required. The name of the database administrator user.
All administrative procedures in this publication use the name DBADMIN for the principal administrator.
The owner of user DBADMIN, which is DBC.
Required. The temporary password for DBADMIN.
Recommendation: Any simple password will work that follows the default system password controls. Each user is prompted to change the temporary password to a permanent, private password at first logon.
Note: Be sure to review the default password control values to determine whether they conform to your site security requirements.
For information on password control options, see Security Administration.
Required. The space in bytes that contains all objects that user DBADMIN creates or owns.
Perm space can also be specified with a constant expression resulting in a numeric value in bytes. This allows you to specify perm space per AMP, which is useful when porting applications from a smaller test system to a production
system. See the example in step c on page 33 .
Because user DBADMIN will create and own nearly all Teradata databases and tables, assign it the majority of space on the system.
Recommendation: 60-65% of available system (DBC) space.
Spool Space Optional. The spool space specification is only a limit to the amount of space available for intermediate query results or formatted answer sets to queries and volatile tables. The system borrows spool space for a user from unused permanent space anywhere in the system.
Spool space can also be specified with a constant expression resulting in a numeric value in bytes. This allows you to specify spool space per AMP, which is useful when porting applications from a smaller test system to a production
system. See the example in step c on page 33 .
Recommendation: Spool space specification is not normally needed. Spool is already allocated in the DBADM profile.
Default
Database
Profile
Optional. The user or database containing the space in which the Teradata
Database stores or searches for new or target objects unless a different database is specified in the transaction SQL.
Recommendation: Specify DBADMIN.
Required. The name of the profile in which the user has membership. A user can be a member of only one profile.
Recommendation: Enter DBADM, the profile created for user DBADMIN in
33
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
Field
FALLBACK
Description
Optional. Specification of FALLBACK automatically creates a duplicate of each table stored in the user space, in addition to the duplicates already created by disk mirroring, to which the system can revert in the event of a failure.
Recommendation: Uncheck Fallback. FALLBACK is only recommended for individual mission-critical tables or tables so large that their size prevents timely backup to tape or external disk.
c
Note: You can adjust the space assignments if required, using the Modify User GUI.
Click Create.
The Query Window pops up and displays the generated CREATE USER statement, for example:
CREATE USER "DBADMIN" FROM "DBC"
AS PERM = 2e5 * (hashamp() + 1);
PASSWORD = "temp"
SPOOL = spool_space
PROFILE = DBADM
STARTUP = ''
NO FALLBACK
NO BEFORE JOURNAL
NO AFTER JOURNAL
The hashamp() function is a database built-in function that returns the highest AMP number in the system configuration. Because AMP numbers start from zero,
(hashamp() + 1) returns the total number of AMPs in the system. The example expression: 2e5 * (hashamp() + 1) means allocate 200,000 bytes on each AMP for perm or spool space for the user or database.
Note: Some options appear in the SQL that are not specified in the procedure. These are system defaults and should remain as they are. For example Before Journal and
After Journal are checked No by default.
d
From the Query window, click the Run Query button .
e
Close the Query window and the Create User dialog box.
3
Keep the Teradata Administrator utility open for use in the
“Granting Database Privileges to User DBADMIN” on page 35
, or close the utility by clicking File>Exit.
Note: After you create the administrative user DBADMIN, do not log on as user DBC to perform subsequent administrative activities except during activities that only user DBC can perform.
34 Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
Reference Information
Reference topics are arranged according to the first step in which the topic appears.
For step...
Information on...
1 logon to Teradata Administrator
2 user types and characteristics syntax and options for the
CREATE USER statement guidelines for managing of space
Is available in...
Teradata Administrator User Guide
Chapter 5: “Creating Users and Granting
Privileges,” in Security Administration
SQL Data Definition Language Syntax and
Examples
Chapter 9: “Managing Space: Operational
Chapter 6: “Managing Database Passwords,” in
Security Administration
the default password control values and how to change them
Granting Database Privileges to User DBADMIN
1
2
Log on to Teradata Administrator as user DBC.
Grant object access privileges to system tables and other objects owned by user DBC.
a
From the main window, click Tools>Grant/Revoke>Object Rights.
b
Specify values for the following fields:
Field
Database Name
Object Type
To/From User
Normal
Grant
Description
The name of the user or database that owns the objects on which the privileges are being granted.
Select DBC from the Database Name drop down list.
The object on which the privileges are given.
Select Database from the Object Type drop down list.
The name of the user that receives the privileges.
Select DBADMIN from the To/From User drop down list.
Specifies the privileges granted to the user. Check the following boxes:
•
Execute
•
Select
Specifies that the user is granted the WITH GRANT OPTION privilege so the user can grant the privilege to other users.
Check the Grant box.
c
Click the Grant button.
The Query window appears and displays the generated GRANT statement:
GRANT EXECUTE, SELECT ON "DBC" TO "DBADMIN" WITH GRANT OPTION.
Database Administration 35
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
3 d
From the Query window, click the Run Query button
Specify values for the following fields:
.
e a
Close the Query window.
Grant object-level database privileges to the database administrator on all objects subsequently created in DBADMIN space.
From the main window, click Tools>Grant/Revoke>Object Rights.
b
Field
Database Name
Object Type
To/From User
All
Grant
Description
The name of the user or database that owns the objects on which the privileges are being granted.
Select DBADMIN from the Database Name drop down list.
The object on which the privileges are given. Includes all databases and all objects created in the database(s).
Select Database from the Object Type drop down list.
The name of the user that receives the privileges.
Select DBADMIN from the To/From User drop down list.
Specifies all rights for all objects created in DBADMIN space.
Check the All box.
Specifies that the user is granted the WITH GRANT OPTION privilege, which means that the user can grant all included privileges to other users.
Check the Grant box.
c
Click the Grant button.
The Query window appears and displays the generated GRANT statement.
GRANT EXECUTE, SELECT, INSERT, UPDATE, DELETE, STATISTICS, DUMP,
RESTORE, CHECKPOINT, SHOW, EXECUTE PROCEDURE, ALTER PROCEDURE,
EXECUTE FUNCTION, ALTER FUNCTION, ALTER EXTERNAL PROCEDURE, CREATE
OWNER PROCEDURE, CREATE TABLE, CREATE VIEW, CREATE MACRO, CREATE
DATABASE, CREATE USER, CREATE TRIGGER, CREATE PROCEDURE, CREATE
FUNCTION, CREATE EXTERNAL PROCEDURE, CREATE AUTHORIZATION, DROP
TABLE, DROP VIEW, DROP MACRO, DROP DATABASE, DROP USER, DROP
TRIGGER, DROP PROCEDURE, DROP FUNCTION, DROP AUTHORIZATION ON
"DBADMIN" TO "DBADMIN" WITH GRANT OPTION;
4 d a
From the Query window, click the Run Query button .
e
Close the Query window.
Grant object-level privileges on DBC tables and views to DBADMIN.
From the main window, click Tools>Grant/Revoke>Object Rights.
36 Database Administration
Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
b
Specify values for the following fields:
Field
Database Name
Object Type
To/From User
Access
Grant
Description
The name of the user or database that owns the objects on which the privileges are being granted.
Select DBC from the Database Name drop down list.
The object on which the privileges are given. Includes all databases and all objects created in the database(s).
Select Database from the Object Type drop down list.
The name of the user that receives the privileges.
Select DBADMIN from the To/From User drop down list.
Specifies privileges for all DBC tables and views.
Check the Access box.
Specifies that the user is granted the WITH GRANT OPTION privilege, which means that the user can grant all included privileges to other users.
Check the Grant box.
c
Click the Grant button.
The GRANT statement generates and automatically submits the following SQL statements without previewing them.
GRANT EXECUTE, SELECT, STATISTICS, SHOW ON "DBC" TO "DBADMIN" WITH
GRANT OPTION
5 d b
From the Query window, click the Run Query button .
e
Close the Query window.
Grant additional system-level privileges not included with the object-level privileges granted in
a
Deselect the Preview Mode button at the top of the main screen. The subprocedure that follows generates a compound SQL statement that cannot be submitted from
Preview Mode.
From the main window, click Tools>Grant/Revoke>System Rights.
c
Specify values for the following fields:
Field
To/From User
Description
The name of the user that receives the privileges.
Select DBADMIN from the To/From User drop down list.
37
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
38
Field
All
Grant
Description
Specifies all rights for all objects created in DBADMIN space.
Check the ALL box.
Specifies that the user is granted the WITH GRANT OPTION privilege, which means that the user can grant all included privileges to other users.
Check the Grant box.
d
Click the Grant button.
The GRANT statement generates and automatically submits the following SQL statements without previewing them.
GRANT MONRESOURCE, MONSESSION, ABORTSESSION, SETSESSRATE,
SETRESRATE, REPLCONTROL, CREATE PROFILE, CREATE ROLE, DROP PROFILE,
DROP ROLE TO "DBADMIN" WITH GRANT OPTION;
6
GRANT UDTTYPE, UDTMETHOD ON SYSUDTLIB TO DBADMIN WITH GRANT OPTION;
e
Close the Grant/Revoke System dialog box.
Grant additional system-level privileges not covered by GRANT shown in step 5
.
a
Click the Query button, the lowest button on the left edge of the main screen to open the Teradata SQL Assistant query window. This allows you to construct and submit an
SQL statement without using the GUI interface.
b
Enter the following SQL statement:
GRANT CTCONTROL ON "DBADMIN" TO "DBADMIN" WITH GRANT OPTION;
7 c
Click the Run Query button .
Grant privileges on Sys_Calendar, which contains tables and views for date-related system functions.
a
Click the Preview Mode button to enable previewing SQL statements before submitting them.
b
From the main window, click Tools>Grant/Revoke>Object Rights.
c
Specify values for the following fields:
Field
Database Name
Object Type
Description
The name of the user or database that owns the objects on which the privileges are being granted.
Select Sys_Calendar from the Database Name drop down list.
The object on which the privileges are given. Includes all databases and all objects created in the database(s).
Select Database from the Object Type drop down list.
Database Administration
Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up the Database Administrator User
Field
To/From User
Normal
Grant
Description
The name of the user that receives the privileges.
Select DBADMIN from the To/From User list.
Allows specification of rights for the named objects.
Check the Select, Insert, Update, and Delete boxes.
Specifies that the user is granted the WITH GRANT OPTION privilege, that is the user can grant all included privileges to other users.
Check the Grant box.
d
Click the Grant button. The Query window appears and displays the generated SQL statement.
GRANT SELECT, INSERT, UPDATE, DELETE ON "Sys_Calendar" TO "DBADMIN"
WITH GRANT OPTION;
8 e a
From the Query window, click the Run Query button .
f
Close the Query window and the Grant/Revoke Object dialog box.
Click File>Define Data Source. The ODBC Data Source Administrator box appears.
On the System DSN tab select the data source, that is, the system name.
b
Click on the Configure button. The ODBC Driver Setup for Teradata Database dialog box appears.
c
Change the value of the Username parameter to DBADMIN.
d
Change the value of the password to the temporary password for DBADMIN specified
Note: User DBADMIN will be prompted to change the temporary password to a permanent, private password at first logon.
e
Click Yes to store the password, then click OK.
f
Click OK to exit the ODBC Data Source Administrator dialog box.
9
10
Click File>Exit to close Teradata Administrator.
Immediately reopen Teradata Administrator, which logs on to Teradata Database as
DBADMIN using the temporary password specified in step d
.
Note: You do not need to execute a GRANT LOGON statement for user DBADMIN or any other user created in the database. All users automatically have the right to log on to the database from any connected client, unless a REVOKE LOGON statement is issued.
11
When prompted, create a private password for user DBADMIN. This client is now configured for use by user DBADMIN. Conduct all future administrative tasks as user
DBADMIN or the administrative username used at your site
12
Exit Teradata Administrator by entering File>Exit.
39
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Creating a Spool Reserve Database
Reference Information
Reference topics are arranged according to the first step in which the topic appears.
Creating a Spool Reserve Database
Teradata strongly recommends that you give sufficient space under user DBC to be used as spool. There are two ways to do this: set aside permanent space to remain unused or create an empty database to hold the space. The advantage of allocating space under DBC to be used for spool is that the space will also be available for other system uses. However, there is also a risk that you may forget to maintain this reserved space and inadvertently assign it to other users and databases for PERM space usage. This could then result in a physical out-of-spool condition if there is no spool space left when processing queries.
An alternative is to create a database under DBC and allocate permanent space to be unused to hold the reserved spool space. Assigning permanent space to DBC to be permanently unused is reserving this physical space for spool usage. The advantage of creating a database to hold spool space means that space will always be set aside and available for spool usage.
A specific example of why it is important to reserve space for DBC is the use of spool space by system tables. Spool limits are allocated for each user or profile and actual space must be available for spool usage when a query is being processed. Sometimes, however, system tables like Transient Journal are allowed to use more space than is available in DBC. When DBC tries to use more space than it has, the system prevents new logons and refuses any data definition language SQL statements. This happens because logons and DDL statements require the system to add additional rows to DBC tables.
To avoid this situation, reserve plenty of space in DBC. This is even more necessary if you are using access logging or database query logging because the tables produced can use up a lot of space. Monitor the space usage for the logging functions often and regularly.
Be sure to not only allocate enough space to DBC but also to regularly clean up old tables by deleting rows from them or archiving them instead if you need the data for later.
40 Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Creating a Spool Reserve Database
To create a spool reserve database, submit a CREATE DATABASE statement and specify the amount of space you want to keep in reserve as the PERM parameter.
Do not create objects or store data in this database. As long as the reserve database remains empty, its PERM allocation remains available for use as spool space.
1
2
For example, assume you created an administrative user named DBADMIN. Since the space of this user is under your control, you can use it to create a child database named Spool_Reserve that never contains tables, as follows:
Log on to Teradata Database as user DBC.
Submit the following statement:
3
CREATE DATABASE Spool_Reserve FROM DBADMIN AS PERM =
n ;
where n is the number of bytes that is a specific percentage you have decided is appropriate.
Quit the session and log off.
For more information on determining the allocation of spool space, see “Sizing Spool Space” in Database Design. For more information on determining perm space requirements, see
“Calculating Total PERM Space Requirements” in Database Design.
Guidelines for Reserving Minimum Spool Space
Teradata Database dynamically allocates available permanent space as spool space when necessary. To make sure that reducing permanent space does not impact transaction processing, Teradata recommends that you reserve permanent space for spool requirements.
At minimum, the average uncompressed and non-fallback protected database should reserve
40% of space relative to CURRENTPERM. (For example, if CURRENTPERM is 100 GB, spool space will be 40 GB for a total MAXPERM of 140 GB.)
You may increase the reserve as needed depending on your actual workload. Use the following table to help you estimate your spool space requirements. Note that the percent of spool space suggested is relative to CURRENTPERM.
IF your system has the following fallback rate…
100%
100%
50%
0%
0%
AND the following compression rate…
0%
40%
20%
0%
40%
THEN consider reserving the following spool space percent (that is, this percent is relative to
CURRENTPERM)…
20%
30%
32%
40%
60%
Database Administration 41
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Using Viewpoint Alerts
Note: Because every site is different and can greatly vary, use these numbers as guidelines only.
Using Viewpoint Alerts
Before setting up Viewpoint alerts, make sure you have:
•
Installed a Teradata Viewpoint server and connected it to your system.
•
Enabled the RSS data collection system on your Teradata system.
•
Created the Teradata Viewpoint server logon user (viewpoint) in the database and given it the privileges necessary to logon and execute database monitoring and management functions.
•
Configured at least one administrative user to access and administer Teradata Viewpoint and enabled the Viewpoint data collectors.
See the Teradata Viewpoint Configuration Guide for instructions.
You can configure Teradata Viewpoint to perform alert actions when the database achieves or exceeds certain defined thresholds. You can specify the following alert actions individually or in any combination:
•
Write a message to the alert log
•
Send an email message to a specified address
•
Communicate the alert to a specified third-party application
•
Run a specified BTEQ script or other program
Each alert action set has a unique name and is triggered according to the rules for any alert that specifies the action set name.
Setting Up Alert Actions
The alert setup defines alert notification parameters and alert actions.
See Teradata Viewpoint User Guide for procedure details.
3
4
1
2
From the main Teradata Viewpoint portal, select Admin>Setup.
Use Delivery Settings/Email to configure communication with the host SMTP email server.
Select the Teradata system for which you are configuring the alert.
Use Alerts Presets/Action Sets to set up common alert actions, for example, an email message to all administrative users. Then you can set up monitoring parameters that invoke the associated alert action whenever Teradata Viewpoint detects that a parameter was exceeded.
Note: Specifying one or more roles in the action sends an email to each role member and is the most efficient way of identifying email recipients. You can list email addresses for any recipients that are not Teradata Viewpoint users.
42 Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Using Viewpoint Alerts
Creating Alerts
A Teradata Viewpoint alert is based on:
•
A threshold specified for a monitoring parameter, for example, when a database exceeds
95% space usage.
•
The time period during which the parameter must be over threshold to invoke the alert.
•
The name of an Action Set, for example, the alert notification, which defines who should receive the alert, and how the message is delivered.
•
A message that the system delivers to the list of users defined in the Action Set, if the threshold and time criteria are met.
To configure a sample alert:
1
2
From the Teradata Viewpoint main portal, select Admin>Teradata Systems.
From the Setup menu, select Alerts, and then configure an alert, for example, a System
Health alert that sends a message if the system is down, similar to the following example.
Database Administration
Note: The Alert Action name “General Alert” is created using
.
3
Check the Enabled box and click Apply.
Note: See Teradata Viewpoint User Guide for details and options.
43
Chapter 1: Setting Up Your Administrative Environment: All DBAs
Setting Up Teradata Viewpoint Alerts for Space Usage
Setting Up Teradata Viewpoint Alerts for Space
Usage
You can use Teradata Viewpoint to regularly monitor space usage, and catch most space problems. However, it may be useful to setup a few alerts in areas where space problems can seriously affect database function.
The following table contains some examples of common space usage alerts.
Alert
Crashdumps Space
The crashdumps database needs enough space to contain possible large data dumps that occur during crashes, and which are used in system recovery.
DBC Space
User DBC contains all system tables and logs.
Critical system functions depend on DBC having enough space.
Note: You can set the same alert parameters for
DBCMNGR and
SystemFE, which perform other critical system functions.
Setup Parameters
Alert rules:
•
Only include databases = crashdumps
•
Current Perm Max% > 70
Note: The threshold is set low because a crashdump can be very large.
Alert Action:
•
•
Do not run twice in 5 minutes
Severity: Medium
Alert rules:
•
Only include databases = DBC
•
Current Perm Max% > 80
Note: Use 90% for SystemFE.
Alert Action:
•
•
Do not run twice in 5 minutes
Severity: Medium
Response to Alert Occurrence
•
Add perm space to the crashdumps database.
•
Purge the crashdumps database of unneeded data and back up needed data to external media.
Add perm space to the database.
44 Database Administration
Chapter 1: Setting Up Your Administrative Environment: All DBAs
About the Global Default Parameters
Alert
Set up three alerts to identify when a database exceeds important space thresholds.
•
All DB 90%
•
All DB 95%
•
All DB 97%
Setup Parameters
Alert rules:
•
All databases
•
Current Perm Max%
> percent
Alert Action:
•
Email (Pager for critical alerts)
•
Do not run twice in 5 minutes
Severity: (apply one of the following to the related alert)
•
Medium (90%)
•
High (95%)
•
Critical (97%)
Response to Alert Occurrence
You should allocate additional perm space to a database as soon as possible after perm space usage exceeds 90%.
Databases that trigger the High or Critical alert levels before you can respond to a Medium alert are growing very fast and should receive more space immediately.
You can investigate which tables are using database perm space.
About the Global Default Parameters
There are many global defaults that you can change for your system, including international character set settings, time zone, business calendars, and many others. For more information, see
Chapter 18: “Working with System Information and Global Defaults: Operational DBAs.” .
Database Administration 45
CHAPTER 2
About Databases and Users in
Teradata: All DBAs
This chapter describes concepts important to understanding a Teradata environment:
•
The differences between databases and users
•
The difference between being the creator, owner, or parent of objects in the database hierarchy
•
The system users and databases created at database initialization
•
A recommended hierarchy for databases and users
About Databases and Users
A database or user is a uniquely named permanent space that can store objects like tables, indexes, procedures, triggers, functions and other databases and users. (Views, macros, roles, and profiles do not consume space; their definitions are stored in the Data Dictionary, which takes up space in DBC.) Each database and each user may optionally contain one permanent journal.
A database is a logical repository for database objects, privilege definitions, and space limits.
In Teradata, a user is the same as a database except it is an active repository that can log on to
Teradata Database with a password and establish a session.
Both databases and users may own objects such as tables, views, macros, procedures, and functions. Both may hold privileges. However, only users may log on, establish a session with
Teradata Database, and submit requests.
A user performs actions; a database is passive. Users have passwords and startup strings; databases do not.
Creator privileges are associated only with a user because only a user can log on and submit a
CREATE statement. Implicit privileges are associated with either a database or a user because each can hold an object and an object is owned by the named space in which it resides.
Space Used by Databases and Users
On a newly installed system, user DBC initially owns all the space in the database. Teradata
recommends that you create special administrative users that belong to DBC (see “Creating
) and create your databases and users from those special administrative users.
Database Administration 47
Chapter 2: About Databases and Users in Teradata: All DBAs
About Databases and Users
Database Administrator
Accounting
HR
DBC
Operations
Customer
Security Administrator
Audit Logs
1093A008
As you create objects or insert data rows, space is allocated as needed from the permanent space of the administrative user who is the immediate owner. If the administrative user grants other users the privilege to create more databases and users, those databases and users can only use the amount of space available to them.
When creating additional databases or users, you must always allocate a permanent space limit. (You define space limits at the database or user level, not at the table level.) The
MODIFY DATABASE/USER statement also allows you to specify the maximum limit of temporary, spool, and permanent space allocated for a database and user.
Every database and user has an upper limit of temporary, spool, and permanent space. For more information on the different types of space, see
Creating Versus Owning Objects in the Hierarchy
There is a difference between creating and owning objects in the database hierarchy. Owner privileges and creator privileges are different and determine default settings for undefined parameters during creation.
•
The creator of an object is the user who submitted the CREATE statement. Every object has one and only one creator.
Note: If the CREATE statement is executed within a macro, then the user who executed the macro is the creator of the object. The macro is just a vehicle for that creation.
•
The owner of an object is any database or user above the object in the database hierarchy.
•
The immediate owner (or parent) of a new user or database is the owner of the space in which the object resides. (In other words, a database or user that has space subtracted from its own permanent space to create a new object becomes the immediate owner of that new object.) The creator is the user who submitted the CREATE statement, but that user can specify a different database with the FROM database option of the CREATE USER or
CREATE DATABASE statement. For details, see “About Creating a Database or User” on page 49
.
48 Database Administration
Chapter 2: About Databases and Users in Teradata: All DBAs
About Databases and Users
The default database is the database of the creating user, but can be a different database if you precede the object name with the database name and a period separator
(databasename.objectname) in the CREATE statement.
An object must always have an immediate owner; that is, an immediate owner of one or more objects cannot be dropped. However, the immediate owner of a database or user can be changed. (See
“Transferring Ownership with GIVE” on page 62
.)
The following table lists the basic rules defining a creator, an owner, and an immediate owner.
IF …
you execute a CREATE statement that creates an object (anywhere in the Teradata
Database) an object is directly below you in the hierarchy you create an object in your own database
THEN …
you are the creator of that object. You are not necessarily an owner, or the immediate owner, of that object.
you are the immediate owner of that object.
you create an object in the space of another user or database (assuming you have the privilege that allows you to do so)
•
UserA owns UserB
•
UserB creates an object in the database of
UserB you are using directory server integration and UserC is a directory-based user mapped to database-based UserA, and UserC creates a database in UserD you are both the creator and the immediate owner of that object.
•
you are the creator of the object.
•
the other user or database is the immediate owner of the object.
•
UserB is the creator of the object.
•
UserB is the immediate owner of the object.
UserA is also an owner, but not the immediate owner of the object.
UserA is recorded as the creator and UserD is recorded as the immediate owner of the new database.
About Creating a Database or User
Users other than DBC (or a site administrative user already granted privileges on ALL objects in the database by DBC) must explicitly be granted the CREATE DATABASE and CREATE
USER privileges before they can create another user or database, even in their own space. For details, see “Privileges That Must Be Explicitly Granted” in Security Administration.
As you create users and databases, a hierarchical relationship evolves.
Database Administration 49
Chapter 2: About Databases and Users in Teradata: All DBAs
The System Users
DBC
A B
C D E
F
FF07A002
•
DBC owns everything in the hierarchy, and is the immediate owner, or parent, of A and B.
•
A owns C, D, and F. A is the immediate owner, or parent, of C and D.
•
C is the immediate owner, or parent, of F.
•
B is the immediate owner, or parent, of E.
The user who submits the CREATE DATABASE/USER statement is the creator of the database or user. The database or user whose permanent space is used to create a new database or user becomes the immediate owner of that new database or user.
In addition, that owner owns all databases or users below it in the hierarchy, because they are created from its original permanent space. The exception to this is if ownership of a specific database or user is transferred to another database or user.
The creator is not necessarily the immediate owner; a creator is the immediate owner only if the new database or user resides within the database of the creator (and thus is directly below the creator database in the hierarchy). With the appropriate privileges, the creator can create a new database or user somewhere else in the hierarchy. For more information on ownership privileges which are implicit privileges, as well as explicit privileges automatically granted to a creator, see Security Administration.
The System Users
Permanent data disk space includes database and user fixed PERM allocation. The Teradata database initialization routine creates three system users as follows:
•
DBC (at the top of the hierarchy)
•
SYSTEMFE (owned by DBC)
•
SYSADMIN (owned by DBC)
All permanent space not consumed by SYSTEMFE and SYSADMIN is allocated to user DBC.
System User DBC
The PERM space allocation of system user DBC is used for the following:
50 Database Administration
Chapter 2: About Databases and Users in Teradata: All DBAs
The System Users
This DBC space… Performs this service …
PERM provides ownership
TEMP
SPOOL
For this function …
Teradata production database space, which must accommodate the maximum PERM space allocated to each user and database by its creator.
The allocated space is used to store the rows in the data tables created by each user, including the following items.
•
Primary data table rows
•
Join index table rows
•
Hash index table rows
•
LOB and
XML subtable rows
•
Secondary index subtable rows
•
Optional permanent journal image rows
•
Optional fallback rows, which replicate the rows of primary data tables, index subtables, hash and join index tables, and permanent journals storing a before-image copy of every row affected by data modifications in all current sessions and Redo and Undo WAL records.
setting up the data dictionary.
creates and maintains the
WAL log/transient journal stores system table, index, macro, trigger, and view definitions provides permanent space provides temporary space provides maximum defined SPOOL space per creator-defined allocation to each user and database
•
Table headers.
Note that each materialized global temporary table also requires a minimum of 512 bytes from the PERM space of its containing database or user for its table header.
•
CRASHDUMPS database.
data inserted into materialized global temporary tables.
•
Dynamic allocation and release of intermediate and final result sets during query processing.
•
Volatile temporary tables.
System User SYSTEMFE
System user SYSTEMFE is owned by user DBC.SystemFE contains special macros and objects.
The quantity of PERM space allocated to SYSTEMFE is minimal.
Only Teradata field support personnel normally log on under this user name.
System User SYSADMIN
System user SYSADMIN is owned by DBC.SysAdmin contains special views and macros. The quantity of PERM space allocated to SYSADMIN is minimal.
Only Teradata field support personnel normally log on under this user name.
Database Administration 51
Chapter 2: About Databases and Users in Teradata: All DBAs
Other System Databases and Users
Other System Databases and Users
The following table lists some of the additional users and databases created during the installation process at System Initialization (Sysinit) or database initialization time.
System Database/User
CRASHDUMPS
Sys_Calendar
TD_SYSFNLIB
Description
Note: CRASHDUMPS is defined in the Data Dictionary when the
DIPCRASH script is run during the DIP phase of installation, migration, or upgrade.
This user is created for use by the system for logging internal errors.
Adjust the space allocation accordingly and monitor its usage. For
more discussion, see “Configuring the Crashdumps Database” on page 560
.
CRASHDUMPS provides temporary storage of dumps generated by the
Parallel Database Extensions (
PDE) software. You may need to allocate more PERM space, based on the size of your configuration, to accommodate at least three dumps.
Note: Sys_Calendar is defined in the Data Dictionary when the
DIPCAL script is run during the DIP phase of installation, migration, or upgrade.
This database contains tables and views related to the system calendar, including the Calendar view, which is a user-accessible tool for date arithmetic.
For more information, see “CALENDAR System View” in Data
Dictionary.
Note: The TD_SYSFNLIB system database is created when the
DIPDB or DIPALL scripts are run during the DIP phase of installation, migration, or upgrade.
This database contains the non-type-specific embedded services functions offered as part of the database release. For details on embedded services functions, see SQL Functions, Operators,
Expressions, and Predicates.
Note: Do not create any database objects in the TD_SYSFNLIB database. The TD_SYSFNLIB database is for internal use only.
52
Note: ALL, DEFAULT, and PUBLIC may appear as databases or user names but actually have no content, no tables, no views, and users cannot log onto them. They are defined with no
PERM space and are used by the database system software. For more information, see “Special
Keywords” in Data Dictionary.
User DBC not only owns all these system databases/users but has access to everything in the system because it has privileges on everything as well. Therefore, rather than use DBC to manage the system and risk accidentally changing the system users, create an administrative user to manage the system.
Database Administration
Chapter 2: About Databases and Users in Teradata: All DBAs
Recommended Hierarchy
Recommended Hierarchy
Teradata recommends this type of hierarchy for databases and users:
DBC
Crashdumps SystemFE SecAdmin DBAdmin SysAdmin TDPUser
Accounting_Database Spool Reserve Production DB DB Users
Acctg_Role user1 user2 user3
Acctg_V_M
View_1
View_2
.
..
Macro_1
Macro_2
Macro_3
Acctg_T
Table_1
Table_2
Table_3
Table_4
.
..
1093B006
Secure Zones Overview
The secure zones feature allows you to create one or more exclusive database hierarchies, called zones, within a single Teradata database system. Access to the data in each zone and zone administration is handled separately from the Teradata database system and from other zones.
Secure zones are useful in situations where the access to data must be tightly controlled and restricted. You can also use secure zones to support some regulatory compliance requirements for the separation of data access from database administration duties.
For example, consider the following use of secure zones. Suppose you have a multinational company or conglomerate enterprise with many subsidiaries. You can create a separate zone for each of the subsidiaries. If your company has divisions in different countries, you can create separate zones for each country to restrict data access to the personnel that are citizens of that country. Your corporate personnel can manage and access data across multiple zones while the subsidiary personnel in each zone have no access to data or objects in the other
Database Administration 53
Chapter 2: About Databases and Users in Teradata: All DBAs
Secure Zones Overview zones. A system-level zone administrator can manage the subsidiary zones and object administration can be done by either corporate DBAs or zone DBAs, as required.
With secure zones, you can ensure the following:
•
Users in one subsidiary have no access or visibility to objects in other subsidiaries.
•
Corporate-level users may have access to objects across any or all subsidiaries.
Another typical scenario is the case of cloud companies that host multiple data customers as tenants. Companies that offer cloud-based database services can host multiple tenants within a single Teradata Database system, using zones to isolate the tenants from each other as if they were running on physically segregated systems. Zone DBAs can administer database objects in their own zone as required. The tenant zones can be managed by a system-level zone administrator, where Teradata acts as the system administrator.
With secure zones, you can ensure the following:
•
Users in one tenant's zone have no access or visibility to objects within other zones.
•
Users in one tenant's zone cannot grant rights on any objects in the zone to any other users, databases, or roles of other zones within the system.
54 Database Administration
CHAPTER 3
Working with Databases: All DBAs
This chapter describes the best practices for database creation and steps through the tasks involved in initial database setup:
•
Creating the Tables database
•
Creating the Views database
•
Granting table access privileges to the Views database
The chapter also details how to drop or transfer ownership of a database or user.
About Database Creation
In Teradata Database, you must create database objects to contain other objects such as data tables, indexes, views, and macros, as well as users and other lower level databases. The disk space that you assign to a database determines the size and number of objects it can contain.
Databases implicitly own and have privileges on all objects created within their assigned space.
Individual objects contained in a database do not require a space allocation, but you can specify space limits for some objects.
Best Practices for Database Creation
To ensure data integrity, Teradata recommends that only administrators and high-level users have the privilege to access data tables. All other database activity should be done in views.
Teradata recommends that you create top-level databases to separate tables and views. You can create additional lower-level databases within the top level databases as required by site security policy and user access needs.
Database
Tables_Database(s)
Description
•
Owns perm space.
•
Contains all tables and data.
•
Normally accessible only to privileged administrative and batch users for:
•
Creating and modifying database objects
•
Performing data loads and updates
Database Administration 55
Chapter 3: Working with Databases: All DBAs
Best Practices for Database Creation
Database
Views_Database(s)
Description
•
Requires minimal perm space because it does not contain data.
•
Contains views of the tables in the Tables_Database.
•
Exists primarily to provide SELECT access to the data for all users.
Note: Where security policy requires it, you can designate a layer of views for load and update operations so that even batch users do not need to directly access data tables.
Creating the Tables Database
Use the following procedure to create the Tables Database that will contain all data tables.
1
Start Teradata Administrator and log on as user DBADMIN.
2
3
4
From the main screen, click the Preview Mode button to display generated SQL.
Click Tools>Create>Database. The Create Database dialog box appears.
Specify values for the following database parameters.
Syntax Element
Database Name
Owner
Permanent
Spool
Explanation
Required. The name of the master database containing all tables and data.
Recommendation: Use a name that provides a hint about the function of the database, for example, Tables, Tables_Database, or TBL_DB.
Required. The name of the user or database that owns the space in which the object is being created.
Recommendation: Enter the name of the primary administrative user,
DBADMIN, previously set up in
“Setting Up the Database Administrator User” on page 30 .
Required. The permanent space in bytes allocated to contain tables and other objects created in the database.
Recommendation: For the Tables database, enter a value or an expression resulting in a numeric value equivalent to 80% of permanent space allocated to user DBADMIN.
If you create child databases within the Tables database, divide the Tables database perm space among them according to the size of the tables each database contains.
Note: Tables do not have perm space allocations. They automatically take the space required to contain their data from the owning database.
Recommendation: Not applicable. Queries running against the Tables database draw from the spool space available to the user.
56 Database Administration
Database Administration
Chapter 3: Working with Databases: All DBAs
Best Practices for Database Creation
Syntax Element
Temporary Space Use TEMPORARY = n BYTES to define the space allowed by default for users to create global temporary tables within this database. The default is the largest value that is less than the owner temporary space and that is a multiple of the number of AMPs in the system. n must not exceed the owner temporary space.
If no default temporary space is defined for a database, the space allocated for global temporary tables created in that database is the maximum temporary space allocated to the immediate owner of the database.
The database/user must have adequate perm space to accommodate the global temporary table header on each AMP. Table header size varies by table definition and the maximum size for a table header is 1 MB.
Account
Explanation
Default Journal
Comment
Before Journal
Not applicable.
Recommendation: Specify accounts at the profile or user level. For information
on using accounts, see “Working with User Accounts” on page 158 .
The default location for journal images of tables created in the new or modified database.
Recommendation: Specify only if you use journaling. See the syntax elements
Before Journal and After Journal in this table.
Optional. You can enter text to describe the database.
Specifies journaling for “before” change row images as follows:
•
Yes - specifies a single journal.
•
No - specifies no journal.
•
Dual - specifies dual journals.
These are actual journaling requirements when specified in the Create Database dialog box.
This option is not normally specified for the initial database implementation.
After Journal
FallBack
Specifies journaling for “after” change row images as follows:
•
Yes - specifies a single journal on a different AMP from changed row.
•
No - specifies no journal.
•
Dual - specifies dual journals.
•
Local - specifies a single journal on the same AMP as changed row. The
Local option is only available for non-FallBack tables.
These are actual journaling requirements when specified on the Create
Database dialog box.
This option is not normally specified for the initial database implementation.
Not applicable, but enabled by default. Specifies that the system will automatically create a duplicate of each table stored in the database space, to provide backup in the event of a failure.
Recommendation: Uncheck the FallBack box. Fallback is required only for individual mission-critical tables or tables so large that their size prevents timely backup to tape or external disk.
57
Chapter 3: Working with Databases: All DBAs
Best Practices for Database Creation
5
Note: This example is the minimum recommended specification for creating the
Tables_Database. You can specify additional options later using the MODIFY DATABASE statement.
Click the Create button to create the database. The Query Window displays a generated
CREATE DATABASE statement similar to the following:
CREATE DATABASE "Tables_Database" FROM "DBADMIN"
AS PERM = 2e7 * (hashamp()+1);
NO FALLBACK
NO BEFORE JOURNAL
NO AFTER JOURNAL
6
7
8
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
Close the Query Window and the Create Database dialog box.
You can optionally create additional child databases within the Tables database to group tables by function or department, but it is not required.
Reference Information
For step...
Information on...
All options for setup and operation of
Teradata Administrator
3 and 4 syntax and options for:
•
CREATE DATABASE
•
DELETE DATABASE
•
DROP DATABASE
•
MODIFY DATABASE
Is available in...
Teradata Administrator User Guide
•
SQL Data Definition Language Syntax and Examples
•
SQL Data Definition Language Detailed
Topics
Creating the Views Database
Use the following procedure to create one or more Views Databases to contain all the views and macros required by the users.
1
Start Teradata Administrator and log on as user DBADMIN.
2
3
From the main screen, click the Preview Mode button
SQL.
to specify display of generated
Click Tools>Create>Database. The Create Database dialog box appears.
58 Database Administration
Database Administration
Chapter 3: Working with Databases: All DBAs
Best Practices for Database Creation
4
Specify values for the following database parameters.
Syntax Element Explanation
Database Name The name of the master database containing user-accessible views. To make data more easily accessible you can create several Views databases according to the needs of various departments or user groups, for example:
•
Views by department
•
Finance_Views
•
Personnel_Views_and_Macros
•
Purchasing_Views
•
Views by user type
•
General_User_Views
•
Update_User_Views
•
Batch_User_Views
You can also create Views databases by functional layer, for example:
•
A layer of read-write views where each view mirrors a table, usable for batch loads and updates without direct access to data tables
•
A layer of read-only views that contain selected columns from one or more tables to combine related information from multiple tables or restrict access
For a description of user types, see “Types of Users” on page 106 .
Owner Required. The name of the user or database that owns the space in which the object is being created.
Recommendation: Enter the name of the primary administrative user,
DBADMIN, previously set up in
“Setting Up the Database Administrator User” on page 30
.
Permanent
Space
Spool Space
The permanent space in bytes or an expression resulting in a numeric value allocated to contain space-consuming objects created in the database.
Recommendation: The Views database is not meant to contain data, but it is the best place to locate stored procedures that access the views. You can allocate a small amount of permanent space for stored procedures, for example 100 MB, and then adjust the allocation later using the MODIFY DATABASE statement.
Not applicable. Queries running in the Views database take spool space from the
Spool_Reserve database.
Temp Space
Account
Default Journal
Comment
Before Journal
Not applicable.
Not applicable.
Recommendation: Specify account at the profile or user level.
For information on using accounts, see
“Working with User Accounts” on page 158
.
Not applicable.
Recommendation: Not applicable, because journaling is not specified for views.
Optional. You can enter text to describe the database.
Not applicable.
59
Chapter 3: Working with Databases: All DBAs
Best Practices for Database Creation
Syntax Element
After Journal
FALLBACK
Explanation
Not applicable.
Not applicable. Fallback is required only for individual mission-critical tables or tables so large that their size prevents timely backup to tape or external disk, and this database contains only views of the data tables.
Recommendation: Uncheck Fallback box.
5
Note: This example is the minimum recommended specification for creating a
Views_Database. You can specify additional database options in the initial CREATE
DATABASE statement, or add them later using the MODIFY DATABASE statement.
Click the Create button to create the database. The Query Window displays a generated
CREATE DATABASE statement similar to the following:
CREATE DATABASE "Views" FROM "DBADMIN"
AS PERM = 0
NO FALLBACK
NO BEFORE JOURNAL
NO AFTER JOURNAL
6
7
8
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
Close the Query Window and the Create Database dialog box.
Grant access privileges to the Views database, as shown in “Working with Table Access
Privileges for Views” on page 61 . After completing the procedure, return to this section
and continue with the remaining tasks.
Reference Information
For step...
Information on...
All
3 options for setup and operation of
Teradata Administrator
•
CREATE DATABASE
•
DELETE DATABASE
•
DROP DATABASE
•
MODIFY DATABASE an overview of SQL procedures and how they are stored in the database
Is available in...
Teradata Administrator User Guide
•
SQL Data Definition Language Syntax and Examples
•
SQL Data Definition Language Detailed
Topics
SQL Stored Procedures and Embedded SQL
60 Database Administration
Chapter 3: Working with Databases: All DBAs
Best Practices for Database Creation
Working with Table Access Privileges for Views
You must grant table access privileges to all views databases to allow the views within each database to access tables in the Tables database.
Note: When table access privileges are granted to a Views database, the individual views inherit the right to access tables in the Tables database from the parent Views database.
Granting Table Access to Each Views Database
1
Start Teradata Administrator and log on as user DBADMIN.
2
3
4 f
On the main screen, click the Preview Mode button
SQL.
Click Tools>Grant/Revoke>Object Rights.
In the Grant/Revoke Objects dialog:
a
to allow display of generated
From the Database Name drop-down menu, select the Tables database. This is the object the privileges are granted on.
b
From the Object Type drop-down menu, select Database.
c
Click the To/From User tab and select a Views database. The privileges are granted to this object.
d
In the Normal field, check the following to select the privileges being granted:
•
For read-only views: Check Select.
•
For updatable views: Check Select, Insert, Update, and Delete.
e
Check the Grant box in the lower right-hand corner of the screen to give the database the privilege of extending the defined privileges to other databases. This specifies the
WITH GRANT OPTION clause.
Click the Grant button. The Query Window displays the generated GRANT statement, for example:
•
For read-only Views databases:
GRANT SELECT ON "Tables_Database" TO "database_name" WITH GRANT
OPTION
•
For the updatable Views databases:
GRANT SELECT, INSERT, UPDATE, DELETE ON "Tables_Database" TO
"database_name" WITH GRANT OPTION
5 g
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
h
Close the Query Window and the Grant/Revoke Objects dialog box.
Log off Teradata Administrator.
Database Administration 61
Chapter 3: Working with Databases: All DBAs
Dropping a Database or User
Reference Information
For Step...
Information on...
4 syntax and options for the GRANT
(SQL form) statement database privileges, including those conferred by object ownership
Is available in...
SQL Data Control Language
Security Administration
Dropping a Database or User
Dropping a database or user is a two-step process:
1
Delete all database objects from the specified database or user. For example:
2
DELETE DATABASE used_cars ALL; or
DELETE USER marco ALL;
Drop the empty database or user. For example:
DROP DATABASE used_cars; or
DROP USER marco;
If a journal table exists, first be sure that no data table references it. Then remove it. For example:
MODIFY DATABASE used_cars AS DROP DEFAULT JOURNAL TABLE;
For the syntax and more examples, see SQL Data Definition Language.
The database or user permanent space that the drop makes available is added to the permanent space of the immediate owner database or user.
Note: All physical database maintenance is performed automatically. You do not need to restructure or reorganize a database to reuse space. Some physical database maintenance activities required by other database systems (such as eliminating pointer chains and reorgs) do not apply to Teradata Database.
Transferring Ownership with GIVE
Transfer databases and users from one immediate owner to another using the GIVE statement, for example:
GIVE Finance TO Alice;
The GIVE statement also transfers all child databases and users as well as the objects (tables, views, macros, indexes, stored procedures, and triggers) owned by the transferred object.
62 Database Administration
Chapter 3: Working with Databases: All DBAs
Transferring Ownership with GIVE
Rules affecting transfer of ownership include:
•
You cannot give an object to children of the object.
•
The permanent space owned by the database or user is also transferred.
•
To use GIVE, you must have the explicit DROP DATABASE/USER privilege on the object being transferred and explicit DROP/CREATE DATABASE/USER privilege on the receiving object.
•
Any explicit privileges granted to others on a transferred user are not automatically revoked. You need to explicitly REVOKE explicit privileges.
•
Changing ownership hierarchy impacts implicit privileges. When a database or user has a new owner, the former owner loses implicit privileges on the object (and all objects below it in the hierarchy). The new owner (and those above him in the hierarchy) gains implicit privileges on the object and all objects below it in the hierarchy.
Transferring ownership affects space allocation. Plan carefully and check space distribution and object ownership before using the GIVE statement.
Database Administration 63
Chapter 3: Working with Databases: All DBAs
Transferring Ownership with GIVE
64 Database Administration
CHAPTER 4
Working with Tables and Views:
Application DBAs
This chapter describes essential tasks and best practices for working with tables and views:
•
Choosing a primary index
•
Creating tables or converting pre-existing tables from other vendors
•
Copying, dropping, and recreating tables
•
Creating views
•
Using BTEQ scripts to create database objects
Choosing a Primary Index
Defining the primary index (PI) for a table is the most critical aspect of table design.
The system uses the PI to assign each data row to an Access Module Processor (AMP). A well chosen PI balances the distribution across AMPs and helps optimize the performance of queries that access the table. PIs also enable efficient joins and aggregations.
Note: A Teradata Database query does not use a PI for access unless all columns in the index are in an equality predicate or an IN LIST.
After you identify the PI requirements for a table, you specify the PI as part of the CREATE
TABLE statement.
Guidelines for Choosing Primary Index Columns
There are three essential factors to consider when choosing a primary index:
•
Uniform data distribution (the most important consideration)
•
Optimal access to the data
•
The volatility of indexed column values
Use the following guidelines for selecting columns to be used as primary indexes:
•
Select columns that consist mainly of unique, distinct values.
This is the most important consideration. Columns with distinct values distribute data rows evenly across all AMPs in the configuration. This maximizes parallel processing by allowing all the processors to participate in processing the data rows for a request.
Avoid using columns with a small number of distinct values that are repeated frequently or columns that have many nulls. This will cause uneven distribution of data rows resulting
Database Administration 65
Chapter 4: Working with Tables and Views: Application DBAs
Choosing a Primary Index in some AMPs having more rows to process than others, and will increase the potential for performance bottlenecks.
Recommendation: Select columns having significantly more distinct values than the number of AMPs in the configuration.
•
Select columns that are most frequently used in equality predicate conditions.
•
Select columns that are most frequently used to access rows.
The PI provides the most efficient method to access data in a table. Therefore, choosing the PI on the most frequently used access path provides for optimal data access. For example, if the table is frequently joined with a specific set of tables, consider defining the
PI on the column set that is typically used as the join condition.
Equality conditions permit direct access to the row. Inequality conditions require additional processing.
•
Select columns that do not have any of the following data types: BLOB, CLOB, Period,
UDT, ST_Geometry, MBR.
•
Choose as few columns as possible for the PI to optimize its generality.
•
Select primary indexes so that query plans use AMP-local processing and avoid row redistributions.
•
Select columns that are not volatile.
Volatile columns force frequent row redistribution because Teradata Database assigns table rows to AMPs based on the PI values.
Note: Several DBSControl values, including those for AMP buffer size, affect actual row distribution.
Recommendation: Select a PI column set that will never be (preferred) or rarely be updated.
You may encounter conflicts when selecting the optimum primary index for a table. For example, choosing a PI that gives good distribution of data across the AMPs must be balanced against choosing a PI that reflects the most common usage pattern of the table. Also, some applications might favor one type of primary index, while other applications might perform optimally using a different primary index.
Recommendation: Make even data distribution, not data access patterns, the main priority when selecting a primary index. Tables and queries change. Access patterns change. However, data distribution does not. You can always add additional non-primary indexes, such as secondary and join indexes, to facilitate particular applications.
66 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Choosing a Primary Index
Reference Information
Information on...
indexing:
•
primary indexes
•
partitioned primary indexes
•
secondary indexes
•
join indexes (materialized views) no primary index tables referential integrity
Is available in...
•
Database Design
•
Introduction to Teradata
•
SQL Fundamentals
Database Design
SQL Fundamentals
Unique and Nonunique Primary Indexes
You can define the primary index as unique (UPI) or nonunique (NUPI), depending on whether duplicate values are allowed in the indexed column set.
UPIs provide optimal data distribution and are typically assigned to the primary key for a table. However, the primary key is not necessarily always the best candidate for a primary index. For a comparison of primary keys and primary indexes, see
Indexes, Primary Keys, and Foreign Keys” on page 68
.
When a table is designed with a NUPI, consider assigning a uniqueness constraint such as
PRIMARY KEY or UNIQUE to the primary or other alternate key of the table. This both enforces uniqueness and enhances retrieval for applications where the primary or other alternate key is frequently used as a selection or join criterion.
Row Partitioning
Primary indexes can be row partitioned or nonpartitioned. A partitioned primary index (PPI) permits rows to be assigned to user-defined data partitions on the AMPs. A PPI is defined to be either single-level or multilevel. Multilevel partitioning allows each partition to be subpartitioned.
The greatest potential gain in row partitioning a primary index is the ability to access only the rows of the qualified partitions while skipping partitions that do not contain rows meeting the search conditions of a query.
A PPI increases query efficiency by avoiding full-table scans without the overhead and maintenance costs of secondary indexes. A multilevel PPI provides multiple access paths to the rows in the base table and can improve query performance through row partition elimination at each of the various levels or combination of levels.
PPIs are designed to optimize range queries while also providing efficient PI join strategies. A range query requests data that falls within specified boundaries. A common example of this is queries that involve date ranges.
Note: Column partitioning is not supported for tables with a primary index.
Database Administration 67
Chapter 4: Working with Tables and Views: Application DBAs
Choosing a Primary Index
Recommendation: Consider defining a table with a PPI to support either of the following workload characteristics:
•
A majority of the queries in the workload specify range predicates on some column, particularly a date column, of the PPI table.
•
A majority of the queries in the workload specify an equality predicate on some column of the PPI table, and that column is either:
•
Not the only column in the primary index column set or
•
Not a primary index column
No Primary Index (NoPI) Tables
You can create tables without a primary index (NoPI). The performance of FastLoad and
Teradata Parallel Data Pump (TPump) Array INSERT operations can be enhanced by using
NoPI staging tables.
You can use NoPI tables as intermediate work tables where PI access is not a consideration.
The system automatically balances the distribution of NoPI tables across AMPs.
NoPI tables are designed for these circumstances:
•
Temporary use
•
When you have not yet determined a suitable primary index
•
Column-partitioning (which cannot be done on tables with a primary index)
In all other circumstances, Teradata recommends that you specify a primary index when you create a table.
Relationship of Primary Indexes, Primary Keys, and Foreign Keys
A primary key defines a column, or columns, that uniquely identify a row in a table. The values defining a primary key for a table:
•
Must be unique
•
Should not change
•
Cannot be null
The following table summarizes the differences between logical primary keys and physical primary indexes:
Primary Key
Important element of logical data model.
Used to maintain referential integrity.
Must be unique to identify each row.
Primary Index
Not used in logical data model.
Used to distribute and retrieve data.
Can be unique or nonunique.
68 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
Primary Key
Values should not be changed if you want to maintain data integrity and preserve historical relations among tables.
Cannot be null.
Does not imply access path.
Required by Teradata Database only if referential integrity checks are to be performed.
Primary Index
Values can change.
Can be null.
Defines the most common access path.
Defined for most production tables. (Some staging tables may not have a primary index.)
Teradata Database uses a unique primary or secondary index to enforce a primary key; therefore, the primary key can affect how Teradata Database distributes and retrieves rows.
Although the primary key is often used as the primary index for a table, it may or may not be the best column set to choose as a primary index. You should base your primary index decision on physical database design considerations that may favor columns other than those of the primary key.
A foreign key identifies table relationships. They model the relationship between data values across tables. A foreign key defines a column, or combination of columns, in a table. The foreign key column(s) must exist in the referenced tables as a primary key or a UNIQUE alternate key.
Teradata Database uses primary and foreign keys to maintain referential integrity between tables. Foreign keys are also used in join conditions.
Creating Tables
Converting Existing Database Tables to Teradata Tables
This procedure uses Oracle as an example to show how you might convert tables created in an
ANSI/ISO-compliant database to Teradata.
1
Identify each Oracle schema/user and all database objects (for example, tables, indexes, views, and materialized views) contained in each schema.
Oracle uses schemas to group database objects into functional groupings that are created and owned by users.
In Teradata, database objects are created and organized within the owning database or user:
•
A database is a collection of related database objects. A database also contains an allotment of space from which users can create and maintain their own objects, or other users or databases.
•
A user is similar to a database, having a space allocation and the capacity to contain objects. However, a user also has a password and can log on to the system, whereas a database cannot.
Database Administration 69
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
2
3
4
5
6
7
Identify the schemas that have objects you want to migrate to Teradata.
Create Teradata databases that correspond to the Oracle schemas and map the schemas to the corresponding Teradata databases.
Identify the DDL for each schema object.
Map Oracle data types to Teradata data types.
Map Oracle table elements to Teradata table elements. See “Creating Tables in Teradata” on page 75
for a description of common Teradata table options.
Oracle and Teradata tables have some common elements:
•
All tables have names.
•
All tables contain columns with data types.
•
Tables can be partitioned.
In Oracle, you must create tablespaces to store table data. You can partition a table and distribute specific partitions for storage in specific tablespaces.
In Teradata, the system uses the Primary Index (PI) to distribute table rows across
units of parallel execution (AMPs). See “Guidelines for Choosing Primary Index
.
You can achieve further partitioning of the data rows within each AMP by defining a
Partitioned Primary Index (PPI) for the table, which is similar to the type of partitioning that Oracle tables perform. However, it is easier to define PPI schemes in
Teradata because the partitioning is a logical organization of data within the database file system, and it requires no knowledge of table space definitions or physical disks.
For more information, and
“Row Partitioning” on page 67 .
•
Constraints can be defined on table columns.
•
Referential integrity can be defined.
•
Global temporary tables are supported.
•
The data block size can be defined to optimize the storage of row data.
•
Freespace can be defined to retain room after loading for future data growth.
•
Triggers can be defined.
Teradata has a number of additional table options, including:
•
Primary index - used for data distribution, data access, table joins, and aggregations.
For details, see
“Choosing a Primary Index” on page 65 .
•
FALLBACK option - allows mirroring of table data for recovery scenarios.
•
Journaling options - stores an image of each data row that has been changed to provide data protection in case of a failure.
Also, in Oracle you must enable a table to be accessed with parallel query processes, whereas parallel query process are automatically accepted in Teradata.
Translate the DDL for each schema object to Teradata DDL.
The DDL for creating tables differs between Oracle and Teradata primarily in how space utilization is defined for the tables.
70 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
In Oracle, database objects are created and stored in tablespaces. Tables can have space utilization parameters describing how they will use space in tablespaces.
In Teradata, database objects are created and stored in databases or users. The database or user definition also defines a space allocation. The system automatically transfers space from a database or user to owned tables as needed, so Teradata table definitions do not include database storage management clauses.
Recommendation: To facilitate loading data from your Oracle table into the new Teradata table, define the column names of the Teradata table to be the same as the column names of your Oracle table. You may use the ALTER TABLE statement to rename any of the columns in the Teradata table after loading the data into the table.
For details on using the CREATE TABLE statement, see
“Creating Tables in Teradata” on page 75
.
Considerations for Defining Tables
A table acquires data attributes when you define its columns in a CREATE TABLE or ALTER
TABLE statement with at least a name and a data type phrase.
Data attributes control the internal representation of stored data and determine how that data is presented to a user. You can use the FORMAT phrase to change the external representation returned by a SELECT query.
The table definition directly affects the performance of applications that access that table.
Proper planning helps you define a table correctly at the time of creation. The following table lists several things to consider when planning your tables.
Issue
Compression
(Algorithmic)
Compression
(Block-level)
Compression
(Multivalue)
Considerations
Specify the COMPRESS USING udfname option of the CREATE TABLE or ALTER TABLE statement to compress character data by table column and minimize storage. This option allows you to choose your own compression or decompression algorithms. Algorithmic compression works best on seldom-used data. See Database Design for details.
Specify the BLOCKCOMPRESSION = block_compression_option of the CREATE TABLE or ALTER
TABLE statement to compress data blocks of primary and fallback tables to minimize storage.
Software-based block-level compression works best on large volumes of seldom-accessed tables. See
Database Design for details.
Specify the COMPRESS phrase to compress up to 255 specific values or NULLs to zero space.
(Nullable columns with nulls are automatically compressed, when COMPRESS is specified.)
Use ALTER TABLE to immediately add, change, or delete compression on a column, column partition, or a table. See Database Design for details.
Database Administration 71
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
Issue
Default value of a column
Default versus
NULL for aggregation results
Considerations
When you create or alter a table, you can specify a default value for a column by using the DEFAULT function. Specifying a default value allows the system to return a value associated with the column.
If you do not specify a value, the default is null.
Note: You cannot assign a user-defined type (UDT) as an explicit default value to the column.
You can assign the column to return USER, CURRENT_DATE, CURRENT_TIME,
CURRENT_TIMESTAMP built-in values as the default value. For more information, see SQL
Functions, Operators, Expressions, and Predicates. For more information on DEFAULT and NULLs returned to the host, see SQL Data Types and Literals.
During aggregation, the unspecified value represented by a:
•
NULL, which is ignored in the calculation.
•
Default value, which is included in the calculation.
For example, assume you want the average salary of employees in Department 300 as follows:
SELECT DeptNo, AVG(Salary)
FROM Employee
GROUP BY DeptNo
WHERE DeptNo = 300 ;
If a salary is not known, the result differs depending on how the column is defined. If an unspecified value in the Salary column is defined with:
•
a DEFAULT value, then the result includes the default value as if it were the actual value. This may be far from the average result if all values were known.
•
NULL, then the result is the average of the salaries that are known. NULL is ignored. This may be closer to the actual average.
You can do the following to further control how unknowns are handled during aggregation:
•
exclude substitutions by making NULL the default value for an unknown, then use the NULLIF function definition; for example:
NULLIF(SALARY,defaultvalue)
In this case, the defaultvalue is ignored in computing the average.
•
change an unknown represented by a NULL into the default value, then use the
COALESCE function definition. For example:
COALESCE(Salary,defaultvalue)
where defaultvalue is the value to be used in computing the average.
72 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
Issue
Defining consistent data types and column names
Considerations
Columns for the same data in different tables should have the same type and name for ease of identification. For example, if the column title is EmpNo in one table, it should be EmpNo in another table. Consistency of data across tables is critical. Some considerations include:
•
Changing data in one table.
If you change data in one table, data in other tables may be affected. For example, updating a department number in the Department table also affects the Employee table, which contains a
DeptNo column. To maintain data integrity, you can use:
•
A macro, a stored procedure, a trigger, or an application program to update all the tables affected by a change.
•
RI constraints. (For details, see Database Design and SQL Data Definition Language.)
•
Joining the columns of one table with the columns of another table. Ensure the following:
•
Join columns are of the same data type and size.
•
Data is consistent in both tables. For example, to join two tables on the Employee Name column, the notation must be identical in both tables (for instance, lastname firstinitial, such as “Smith H”).
•
All row-level security-protected tables referenced in a join contain the same constraints.
•
Using tables to store large data objects. Create a table with LOB type columns. This includes:
•
Character Large Object (CLOB)- A CLOB column can store character data, such as simple text, HTML, or XML documents.
•
Binary Large Object (BLOB)- A BLOB column can store binary objects, such as graphics, video clips, files, and documents.
IDENTITY column IDENTITY is an optional attribute used to generate a number for every row inserted into the table on which it is defined. An identity column does not have to be the first column in the table or defined as an index. The IDENTITY attribute may be specified such that the generated number will be unique.
Joins involving
NULLs
If a column used in a join contains either NULLs or generic values, the results might be misleading.
For example, assume you want to join the Employee and Department tables to obtain a listing of employees and their workplaces:
SELECT Name, Loc
FROM Employee, Department
WHERE Employee.DeptNo = Department.DeptNo;
Undesired results may occur. The nulls in each table prevent matches. Employees with an unknown department number (Employee.DeptNo is NULL) and departments without employees are not listed.
To list an employee with an unknown department number and a department with no employees, use a different default value to represent unknowns in the DeptNo column, as follows:
1
In the Employee table, add a mock “unknown” employee who has a DeptNo equal to the value of the default used for the Department table.
2
To the Department table, add a mock “unknown” department that has a DeptNo equal to the value of the default used for the Employee table.
Note: In the example query, a full outer join could be used instead of the inner join to obtain information for non-matching rows when there is a NULL in DeptNo. (Also, as noted previously, you may use the COALESCE function to change a NULL to a value.) However, using outer joins to obtain non-matching rows may be more efficient.
For further discussion on the implications of using NULL and for information on how to design efficient joins, see “Joins and Hash Indexes” in Database Design.
Database Administration 73
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
Issue
NULL and the
GROUP BY clause
Nullability of a column
Considerations
If you use a column defined as NULL for grouping aggregate results, the results may be confusing.
For example, assume you submit this query to find the average salary for each department in the organization:
SELECT DeptNo, AVG(Salary)
FROM Employee
GROUP BY DeptNo;
The results can differ. If the DeptNo column is:
•
allowed to be NULL, and two employees have not yet been assigned a department number, then the result lists the computed average for those two employees under a NULL department number. This might be confusing.
•
defined with DEFAULT and the specified constant is meaningful (such as Unknown), is more valid.
Note: You can convert a NULL into a default value (with NULLIF) and a default value into NULL
(with COALESCE), as explained previously for aggregated results.
It may not always be possible to specify a value when inserting a new data row. For example, a new employee may not immediately have a job title. You can explicitly define the nullability of a column with a NULL or NOT NULL phrase as follows:
If no value is given for a column when a row is inserted, and the nullability of a column is:
•
defined as NOT NULL and a DEFAULT value is not specified, then the INSERT statement returns an error.
•
not defined, then a NULL is supplied automatically.
Note: It is best practice is specify NOT NULL for a column with a default value (if a default value is appropriate) unless there is a valid reason for allowing the column to be nullable. This saves space and allows for more efficient query plans in some cases. If the column does need to be nullable, be sure to determine, understand, and document the meaning of NULL for this column since null has various interpretations.
If an application program requests data from a column (without NULL indicators) that allows nulls and a NULL is found, a substitute value that is compatible with the data type of the column is returned to the application instead of a null. The substitute value (a zero, blank, or zero-length element string) might be misleading, because NULL is an unknown.
Note: Never use the phrase NOT IN on nullable columns always replace it with the phrase NOT
EXISTS. If NOT IN is used, it will make the database work a lot harder.
Also, note that any comparison to NULL results is an UNKNOWN value. NOT EXISTS, however, treats UNKNOWN values as FALSE as shown in EXPLAIN.
For example:
1 in (1,2,NULL) -> 1=1 or 1=2 or 1=NULL -> TRUE OR FALSE OR UNKNOWN -> TRUE
1 not in (1,2,NULL) -> 1<>1 and 1<>2 and 1<>NULL -> TRUE and FALSE and UNKNOWN ->
UNKNOWN
For more information on unexpected answer sets, see “Behavior of Nulls for NOT IN” in SQL
Functions, Operators, Expressions, and Predicates.
74 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
Issue
Other considerations
NoPI table
Considerations
Consider:
•
Whether or not to have a primary index. (See the NoPI row in this table.)
•
The columns to use in the primary index and whether the PI is unique or not.
•
If the table has a primary index, whether to have one or more levels of row partitioning.
•
If there is no primary index, whether to have column partitioning and, optionally, one or more levels of row partitioning.
•
What secondary indexes to define, if any, and whether they are unique.
•
What constraints are needed on each of the columns.
•
Row header compression for column partitions that have COLUMN format can reduce storage costs and enhance system performance. However, in some cases, such as wide column partitions that do not benefit from autocompression, ROW format (no row header compression) may be better.
•
Whether to allow concurrent reads on committed rows in a table while loading. A table must be set up as load isolated to use this feature. A load-isolated table requires more space than a nonisolated table and requires regular maintenance to remove logically deleted rows. For more
information, see “About Reading Committed Data While Loading to the Same Table” on page 138
.
Use a NoPI table when either you want to column-partition the table, a primary index does not add any value, you do not know yet what primary index to specify, or you need a staging table and restrictions on NoPI tables are acceptable.
To create a staging table, use a NoPI table which is a table without a PI. Using a NoPI table for staging purposes improves performance when loading data from FastLoad and TPump jobs because the table is not hashed by a PI and the rows do not have to be sorted in any particular order or redistributed to any particular AMP.
NoPI tables are useful for applications that load data into a staging table that needs to go through some sort of transformation or standardization. The data must be converted before being stored in another table. You can then apply the data from the staging table to a target table through an
INSERT... SELECT, UPDATE-FROM or MERGE-INTO statement.
It is best practice is to explicitly specify a PI or NoPI instead of relying on the default for the PI. The default may not be appropriate and may cause skewing of the data on the AMPs.
For more information on how to create a NoPI table, see “CREATE TABLE” in SQL Data Definition
Language.
Creating Tables in Teradata
1
2
The following procedure shows how to create tables using BTEQ, but you can also use
Teradata Administrator, SQL Assistant, or Teradata Studio.
Logon on to Teradata Database as user DBADMIN using BTEQ.
Create one or more tables using the CREATE TABLE statement. For example:
CREATE SET TABLE database_name.table_name, FALLBACK
(column_name data_type,
column_name data_type,
column_name data_type)
UNIQUE PRIMARY INDEX (primary_index_column);
Database Administration 75
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables where:
Syntax Element
MULTISET
SET
database_name data_type
The name of the database where the table will be created if it is different from the current database.
Recommendation: The database should be the Tables_Database or a database located within the Tables_Database. See
“About Database Creation” on page 55
.
If database_name is not specified, the system creates the table in the default database for the current session.
The name of the table to be created.
table_name
[NO] FALLBACK Specifies that the system will automatically create a duplicate copy of the current image of every row in this table to provide backup in case of a failure.
NO FALLBACK is the default.
To specify FALLBACK for all tables in a database, use FALLBACK in the
CREATE DATABASE statement.
Recommendation: Specify FALLBACK for mission-critical tables or tables so large that their size prevents timely backup to tape or external disk.
column_name
Specifies the name of one or more columns, in the order in which they and their attributes are to be defined for the table.
Up to 2,048 columns can be defined for a table. Note that other limits or feature restrictions may apply before you reach the column limit. For example, you can have no more than 32 large object (LOB) columns per table.
Specifies a single data type for each column.
Some data types have different names in Teradata than in other databases. For more information, see SQL Data Types and Literals.
[UNIQUE|NO]
PRIMARY
INDEX
primary_index_ column
Explanation
A MULTISET table allows duplicate rows in compliance with the ANSI/ISO
SQL 2011 standard. A SET table does not allow duplicate rows.
If there are uniqueness constraints on any column or set of columns in the table definition, then the table cannot have duplicate rows even if it is declared as
MULTISET.
Specifies the primary index. A table can have no more than one primary index.
If you do not explicitly assign a primary index, Teradata Database will choose a default primary index (unless you specify NO PRIMARY INDEX).
Recommendation: Explicitly define either PRIMARY INDEX, UNIQUE
PRIMARY INDEX, or NO PRIMARY INDEX because the default selected by
Teradata Database may not be optimal for the table.
Specifies the column(s) that define a primary index. If you specify more than one column, the index is created on the combined values of each column. You can specify up to 64 columns.
Recommendation: Choose columns that primarily have unique values, are frequently accessed, and have values that do not change. See
Choosing Primary Index Columns” on page 65 .
76 Database Administration
Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
3
4
Note: The example is the minimum recommended specification for creating a table. You can specify additional options, or add them later using the ALTER TABLE statement, see
“Optional Table-level and Column-level Elements” on page 77 .
To automate the creation of data tables by using BTEQ scripts, see “Using BTEQ Scripts to
Create Database Objects” on page 88
.
Set privileges on the table so that only designated administrative users can access the table directly.
Collect statistics on the newly created, empty table. This defines the columns, indexes, and partitions for a PPI table, as well as the data structures for subsequent collection of statistics and demographics.
Optional Table-level and Column-level Elements
The following are common optional table-level elements you can specify in the CREATE/
ALTER TABLE statement. For details about these table options and for a complete description of all table options, see SQL Data Definition Language Syntax and Examples.
Syntax Element
GLOBAL
TEMPORARY
JOURNAL
FREESPACE
Explanation
Creates a global temporary table. Global temporary tables have a persistent definition but do not have persistent contents across sessions. Each user session can materialize up to 2,000 global temporary tables at a time.
To use global temporary tables, create or modify the user or profile to have a temporary space limit by using the TEMPORARY = n BYTES option.
If no temporary space limit is defined for a profile, Teradata Databases uses the temporary space limit defined for the individual user-creator. If no temporary space is defined for a user, then the space allocated for any materialized global temporary tables referenced by that user is set to the maximum temporary space allocated for the immediate owner.
Specifies a Permanent Journal (PJ) for the table, which stores an image of each data row that has been changed with an INSERT, UPDATE, or DELETE statement. PJ images can reflect row values as they appeared before the change, after the change, or both.
To specify a PJ for all tables in a database, use the JOURNAL clauses in the
CREATE DATABASE statement.
Recommendation: Specify journaling for tables that need an additional level of data protection after you have analyzed data protection needs.
Specifies the percent of free space that remains on a cylinder during loading operations. The reserved free space allows table data to expand on current table cylinders, preventing data migration operations which can impact system performance. You can control this space at the global and table level.
Recommendation: If little or no table expansion is expected (the table is read-only), set FREESPACE to 0. Otherwise, choose a percentage that reflects the growth rate of your table (INSERTs minus DELETEs). Common settings are 5 to 15%. The maximum allowed is 75%.
77
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
Syntax Element
DATABLOCKSIZE
[UNIQUE] INDEX
PARTITION BY
Explanation
Specifies the maximum data block size for blocks that contain multiple rows. Larger block sizes enhance full table scan operations by selecting more rows in a single I/O. Smaller block sizes are best for transaction-oriented tables to minimize overhead by retrieving only what is needed. You can control the data block size at the global and table level.
Recommendation: Use a larger value if the database is used primarily for strategic work (decision support/complex queries). Use a smaller value if the database is used primarily for tactical work (OLTP).
Specifies secondary indexes for the table. Secondary indexes (SI) allow quick access to table data using an alternate, less frequently used path than the primary index. SIs improve performance by avoiding full table scans.
Specifies that the primary index is partitioned based on the value of the specified partitioning expression. A partitioned primary index (PPI) permits rows to be assigned to user-defined data partitions, enabling enhanced performance for range queries while providing efficient PI join strategies. See
.
The following are common optional column-level elements you can specify in the CREATE/
ALTER TABLE statement. For details about these column-level options and for a complete description of all table options, see SQL Data Definition Language Syntax and Examples.
Syntax Element Explanation
data_type_attributes
If you specify attributes for a column, you must define its data type before you define its attributes. The following attributes are supported:
•
NOT NULL
•
UPPERCASE
•
[NOT] CASESPECIFIC
•
FORMAT
•
TITLE
•
NAMED
•
DEFAULT or WITH DEFAULT
•
CHARACTER SET
•
WITH TIME ZONE
Recommendation: Teradata highly recommends that you specify the NOT
NULL attribute for columns that will never be null.
COMPRESS Specifies a set of distinct values in a column that is to be compressed to zero space. Column compression can enhance storage capacity, improve response time for table scans, and reduce disk I/O traffic.
Recommendation: For best results, use multivalue compression where:
•
Enough rows contain a compressible value (null, zero, blank, or constant value) in the compressed field to exceed the break even point.
•
The shortened row size eliminates one or more data blocks.
78 Database Administration
Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Creating Tables
Syntax Element
CONSTRAINT
Explanation
Specifies column-level contraints. The following constraints are supported:
•
UNIQUE
•
PRIMARY KEY
•
FOREIGN KEY
•
CHECK
•
REFERENCES
Recommendation: Specify a name for the constraint specification.
Example CREATE TABLE Statement
This table definition creates the Employee table in the database named Tables_Database:
CREATE SET TABLE Tables_Database.Employee, FALLBACK
(Associate_Id INTEGER,
Associate_Name CHAR(25),
Salary DECIMAL(8,2),
DOB DATE,
Job_Title
Dept_No
VARCHAR(25),
SMALLINT,
Marital_Status CHAR,
No_Of_Dependents BYTEINT)
UNIQUE PRIMARY INDEX (Associate_Id);
Using BTEQ script, see “Using BTEQ Scripts to Create Database Objects” on page 88
.
Reference Information
Information on...
an overview of tables, including global temporary and volatile tables
•
syntax, options, and required privileges for:
•
CREATE TABLE
•
ALTER TABLE
•
DROP TABLE
•
RENAME TABLE the default database data types and data type attributes
•
primary indexes
•
partitioned primary indexes
•
secondary indexes
•
join indexes (materialized views)
Is available in...
•
SQL Fundamentals
SQL Data Definition Language Syntax and Examples
SQL Fundamentals
SQL Data Types and Literals
•
“Choosing a Primary Index” on page 65
79
Chapter 4: Working with Tables and Views: Application DBAs
Copying a Table
Information on...
using the FREESPACE option to manage performance, and guidelines for calculating a
FREESPACE value using the DATABLOCKS option to manage performance and specifying a DataBlockSize value compressing column values column-level and table-level constraints
Is available in...
“FreeSpacePercent” in Utilities
Utilities
Database Design
Database Design
Copying a Table
Use the AS option of the CREATE TABLE statement to copy some or all of an existing table.
The following table describes several options for copying tables.
Use this statement … To copy an existing table as …
CREATE GLOBAL
TEMPORARY
TABLE AS
[tablename/
query_expression]
…WITH NO DATA
a global temporary table. Use WITH NO DATA, because global temporary tables are not populated until they are materialized by being referenced in a query.
If you want the table to inherit:
•
All of the column definitions, then specify tablename WITH NO DATA.
•
A subset of the column definitions, then specify (query_expression) and
WITH NO DATA.
The subquery form of CREATE GLOBAL TEMPORARY TABLE
AS...WITH NO DATA does not copy indexes and defines a default primary index which might not be appropriate. Teradata highly recommends specifying explicitly an appropriate PI or specify NO PRIMARY INDEX.
80 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Copying a Table
Use this statement … To copy an existing table as …
CREATE TABLE
AS
[tablename/
query_expression] …
WITH [NO] DATA
[AND
STATISTICS]
a permanent table or a volatile table. You choose what columns you want to copy and whether the table should be populated automatically, as follows. If you want the table to inherit:
•
All of the column definitions plus the contents, then specify tablename
WITH DATA and (for a volatile table) ON COMMIT PRESERVE
ROWS.
•
All of the column definitions but none of the contents, then specify
tablename WITH NO DATA.
•
A subset of the column definitions plus the contents, then specify
(query_expression) WITH DATA and (for a volatile table) ON COMMIT
PRESERVE ROWS.
•
A subset of the column definitions but none of the contents, then specify
(query_expression) WITH NO DATA.
•
The statistical histograms of the original table, then specify AND
STATISTICS. If USECOUNT is enabled, this option also copies use
count information from the original table. See “BEGIN/REPLACE/END
QUERY LOGGING Statements” on page 268 .
The subquery form of CREATE TABLE AS...WITH [NO] DATA does not copy indexes or partitioning and defines a default primary index, which might not be appropriate. Teradata highly recommends specifying explicitly an appropriate PI or specify NO PRIMARY INDEX.
Copying Statistics From a Base to a Target Table
Use the optional AND STATISTICS clause with the CREATE TABLE AS…WITH DATA statement to copy statistics from a base table to a target table. Using the AND STATISTICS clause saves time from having to collect statistics on the new table.
Note: You can use the shorthand STATS or STAT in place of STATISTICS.
If you choose not to copy statistics, you can either omit the AND STATISTICS clause or you can specify the AND NO STATISTICS clause. If you wish to copy the table using zeroed statistics, submit the WITH NO DATA AND STATISTICS option instead.
For more information on usage rules and syntax, see “CREATE TABLE (AS Clause)” in SQL
Data Definition Language.
Database Administration 81
Chapter 4: Working with Tables and Views: Application DBAs
Dropping a Table
Dropping a Table
Use one of the following statements to drop a table.
TO remove a … USE this statement …
permanent data table materialized global temporary table, before session end
DROP TABLE.
Dropping a table does not delete the views, macros, functions or stored procedures that referenced the dropped table. You need to explicitly drop or alter the reference.
You cannot drop a table defined with a trigger, hash index, or join index. First drop the index or trigger and then drop the table.
DROP TEMPORARY TABLE.
Note: Teradata Database automatically drops all materialized temporary tables at session end.
volatile table before session end DROP TABLE.
global temporary table definition
DROP TABLE.
Recreating a Table
You may need to recreate a data table to:
•
Change a default PI to a defined PI for a nonempty table.
•
Add or remove a PI for a nonempty table.
•
Change the PI to a different PI for a nonempty table.
•
Change a NUPI to a UPI in a populated table when there is no USI (however, usually you would just create the USI on the table, and then alter it).
•
Redefine the partitioning of a populated table. In some cases, you can still use ALTER
TABLE; see “ALTER TABLE” in SQL Data Definition Language.
•
Change a data type attribute that affects existing data. For rules on changing data types, see “ALTER TABLE” in SQL Data Definition Language.
•
Define or delete COMPRESS storage attributes for an existing column.
•
Add columns that would otherwise exceed the maximum for the number of columns defined during the life of a table.
•
Move the table to another database or user.
•
Change the groupings of columns into column partitions.
Note: It is best practice to access a table via a view, especially for tables used by multiple users.
82 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Recreating a Table
Use the SHOW TABLE or SHOW IN XML TABLE request to display the current table definition, which you can then modify and submit.
Using INSERT…SELECT
Use the INSERT…SELECT statement to load the current data rows quickly into a new table.
The procedure for recreating a table is as follows:
1
Select the explicit privileges of the old table with the following query:
2
SELECT username, accessright, grantauthority, columnname, allnessflag
FROM dbc.allrightsV
WHERE tablename = 'Employee' AND databasename = 'Personnel';
Note: Save the output for later use; you will need to recreate the explicit privileges on the new table.
Create a new table with a temporary name, such as Temp.
3
4
5
CREATE TABLE Temp_Employee
(col1 datatype, col2 datatype...)
To display the DDL for the current table, submit a SHOW TABLE or SHOW IN XML
TABLE request.
If any data types, columns, column attributes, or the column order are not compatible, use an INSERT…SELECT statement that constructs compatible values. If this is not possible, you may need to load data from an external source.
If the data types are compatible, you can transfer all data rows from the old table to the new table with a single INSERT … SELECT statement:
INSERT INTO Temp_Employee
SELECT *
FROM Employee ;
If the tables are not compatible, a more complicated SELECT that computes the values for the target table from columns of the source table is needed.
Use SHOW JOIN/HASH INDEX or SHOW IN XML JOIN/HASH INDEX to see the index definitions. If needed, drop the hash index or join index using DROP JOIN INDEX or
DROP HASH INDEX and recreate it using CREATE JOIN INDEX or CREATE HASH
INDEX.
Drop the old Employee table:
6
7
8
DROP TABLE Employee ;
Note: When the table is dropped, explicit privileges are also dropped because the Data
Dictionary references objects by ID rather than by name.
Rename the temporary table:
RENAME TABLE Temp_Employee TO Employee ;
Use the index definitions from step 4 to recreate join and hash indexes that you want to maintain for the new table.
Submit GRANT statements to re-establish the explicit privileges you saved in step 1 on the new version of Employee table.
Database Administration 83
Chapter 4: Working with Tables and Views: Application DBAs
Specifying Fallback Tables
Use the LOGGING ERRORS option to log errors. Errors include duplicate row errors, duplicate primary key errors, CHECK constraint errors, and more. For more information, see
“INSERT/INSERT … SELECT” in SQL Data Manipulation Language.
Specifying Fallback Tables
Fallback provides data protection at the table level by automatically storing a copy of each permanent data row of a table on a different or “fallback” AMP.
If an AMP fails, Teradata Database can access the fallback copy and continue operation. If you cluster your AMPs, fallback also provides for automatic recovery of the down AMP once you bring it back online.
Define fallback using the CREATE/ALTER TABLE commands. Fallback-protected tables occupy twice the permanent space as non-fallback tables and require twice the I/O for inserts, updates and deletes. However, the advantages in continued operation and data integrity are well worth the space.
You can define fallback for the following:
•
Primary data table
•
Secondary index subtable
•
Join index of any type, including aggregate join index
•
Hash index
Fallback is beneficial because it:
•
Permits access to table data when an AMP is offline.
•
Protects the accessibility and integrity of your data if base data becomes unavailable due to a software error.
•
Adds a level of data protection beyond disk array RAID.
•
Automatically applies changes to the offline AMP when it is back online.
Working with Views
Database views are created from one or more base tables or from other views. They are virtual tables that you can use to retrieve data from the underlying views or tables. A view does not contain data so it does not use any space. Only the view definitions are stored in the Data
Dictionary.
A view may present only a subset of the columns and rows in the base table or tables. In addition, a column defined in a view can be derived and does not need to exist in the underlying base tables. For example, you can display summed or averaged data in a view column.
84 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Working with Views
Views are generally used to:
•
Allow users to access only the rows and columns they need in a table.
•
Enforce security by restricting table access and updates.
•
Provide well-defined, well-tested, high-performance access paths to data.
•
Provide logical data independence, which minimizes the need to modify your applications if you restructure base tables.
•
Format the output of the data for reports.
•
Specify locking, such as specifying a less restrictive ACCESS lock.
To ensure data integrity and system security, Teradata recommends the following:
•
Set privileges so that general users cannot access or modify data tables directly.
•
Create views which allow general users to access or modify only the table data they need.
Use views to restrict users to performing specific queries and update functions.
•
Create views in a Views Database.
You can use the following utilities to create views in Teradata:
•
BTEQ
•
SQL Assistant
BTEQ and SQL Assistant allow you to submit CREATE VIEW DDL statements to Teradata
Database.
Creating Views Procedure
The following procedure shows how to create views using BTEQ:
1
2
Log on to Teradata Database as user DBADMIN using BTEQ.
Create one or more views using the CREATE VIEW statement. For example:
CREATE VIEW database_name.view_name (column_name, column_name) AS
SELECT clause; where:
Syntax Element
database_name view_name column_name
Explanation
The name of the database where the view will be created if it is different from the current database.
Recommendation: The database should be a Views Database or a database located within a Views Database. See
“About Database Creation” on page 55 .
If database_name is not specified, the system creates the view in the default database for the current session.
The name of the view to be created.
The name of a view column. If more than one column is specified, list their names in the order in which each column is to be displayed for the view.
Database Administration 85
Chapter 4: Working with Tables and Views: Application DBAs
Working with Views
Syntax Element
LOCKING clause The specified lock is placed on the underlying base table set each time the view is referenced in an SQL statement.
Recommendation: Create views with a LOCKING FOR ACCESS modifier to allow concurrent access of the data to read-only users and users who will modify the data. This prevents deadlocks and increases performance.
ACCESS locking is a tool that can be used to reduce the isolation of a user request allowing that request to access valid, but uncommitted data that is subject to change.
This allows users that are interested in a broad, statistical snapshot of the data, but that do not require precise results, to attain access to data while modifications to that data are occurring. This can greatly improve performance, because those users do not have to wait unnecessarily for the write operation to complete.
SELECT clause
Explanation
The SELECT statement which obtains the data for the view. The following are some of the options and clauses you can use:
DISTINCT option
Returns only one row from any set of duplicate rows in the SELECT result.
SELECT clause
(continued)
ALL option
Returns all rows, including duplicate rows in the SELECT result. This is the default.
TOP n or TOP m PERCENT option [WITH TIES]
Restricts a view to only n rows or m percent of the rows in an underlying base table. If you do not specify an ORDER BY clause, then a TOP n clause only specifies that any n base table rows be returned, not the TOP n.
This option provides a fast method to obtain a restricted, non-statistically random sample of table rows.
FROM clause
Specifies the set of base tables or views from which data for the view are selected.
WHERE clause
Specifies a conditional expression by which rows for the view are selected.
GROUP BY clause
Groups result rows by the values in one or more columns or by various extended GROUP BY operations on specified column expressions.
HAVING clause
Specifies a conditional expression by which the groups defined by a GROUP
BY clause are selected.
QUALIFY clause
Specifies a conditional ordered analytical function by which rows for the view are selected.
86 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Working with Views
Syntax Element Explanation
WITH CHECK OPTION clause
Integrity constraint option that restricts the rows in the table that can be affected by an INSERT or UPDATE statement to those defined by the WHERE clause. This option only pertains to updatable views.
Recommendation: Specify a WITH CHECK OPTION clause in all your updatable view definitions to prevent unauthorized modification of data rows.
ORDER BY clause
Specifies the order in which result rows are to be sorted. You can only specify an ORDER BY clause for a view definition if you also specify either the TOP n or the TOP m PERCENT option.
3
Note: The example lists the major options for creating a view. You can modify your view definition by using the REPLACE VIEW statement.
To automate the creation of views by using BTEQ scripts, see “Using BTEQ Scripts to
Create Database Objects” on page 88
.
Set privileges on the view as follows:
•
Provide general users read-only access so they can use the view to query data.
•
Provide privileged users read and write access so they can use the view to update data in the underlying tables.
Example 1: Creating a View to Limit User Access
The following statement creates a view of the Employee table so that it provides access only to the names and job titles of the employees in department 300:
CREATE VIEW Views_Database.Dept300 (Associate_Name, Job_Title) AS
SELECT Associate_Name, Job_Title
FROM Tables_Database.Employee
WHERE Dept_No = 300
WITH CHECK OPTION;
The WITH CHECK OPTION prevents using this view to insert a row into the Employee table, or to update any row of the Employee table where Dept_No is not 300.
Example 2: Creating a View to Generate a Report
The following statement creates a view which calculates the minimum, maximum, and average salary of each department and displays only those rows with an average salary of
$35,000 or higher.
Because the view definition specifies an ACCESS lock for the table, the view returns valid, but uncommitted data that is subject to change, particularly if the view selects data at the same time another user attempts to modify the salary data in the Employee table.
This view is designed for users who need quick access to data, but do not need precise results.
CREATE VIEW Views_Database.Dept_Salary (Dept_No, MinSal, MaxSal, AvgSal)
AS LOCKING TABLE Tables_Database.Employee FOR ACCESS
Database Administration 87
Chapter 4: Working with Tables and Views: Application DBAs
Using BTEQ Scripts to Create Database Objects
SELECT Dept_No, MIN(Salary), MAX(Salary), AVG(Salary)
FROM Tables_Database.Employee
GROUP BY Dept_No
HAVING AVG(Salary) >= 35000;
Reference Information
2
2
2
2
For step...
Information on...
2
•
syntax, options, and required privileges for:
•
CREATE VIEW/REPLACE
VIEW
•
DROP VIEW
•
RENAME VIEW the default database the SELECT statement, including usage of the options and clauses
2
2
•
rules for creating, replacing, and using views
•
updatable views
•
WITH CHECK OPTION clause
•
deleting rows using views
(DELETE, Basic/Searched Form)
•
updating rows using views
(UPDATE) specifying locking in views
CREATE RECURSIVE VIEW and
REPLACE RECURSIVE VIEW
Is available in...
SQL Data Definition Language Syntax and
Examples
SQL Fundamentals
SQL Data Manipulation Language
SQL Data Definition Language Syntax and
Examples
SQL Data Manipulation Language
SQL Request and Transaction Processing
SQL Data Definition Language Syntax and
Examples
Using BTEQ Scripts to Create Database Objects
Basic Teradata Query Utility (BTEQ) is a general-purpose, command-based application that allows users to:
•
Perform administrative tasks, including creation of database objects such as databases, tables, and views, using SQL DCL, DDL, and DML requests.
•
Format reports for both print and screen output.
You can automate the creation of tables, views, and other database objects using BTEQ scripts.
The following example procedure uses a BTEQ script to create a sample table.
1
Create a text file named Create_Emp_Table.sql with the following contents:
.LOGON tdpid/UserName,Password
88 Database Administration
Chapter 4: Working with Tables and Views: Application DBAs
Using BTEQ Scripts to Create Database Objects
3
4
2
CREATE SET TABLE Tables_Database.Employee, FALLBACK
(Associate_Id INTEGER,
Associate_Name CHAR(25),
Salary DECIMAL(8,2),
DOB DATE,
Job_Title VARCHAR(25),
Dept_No SMALLINT,
Marital_Status CHAR,
No_Of_Dependents BYTEINT)
UNIQUE PRIMARY INDEX (Associate_Id);
.quit
where:
•
tdpid is the name by which the system is known to the network.
•
UserName is the name used to log on to Teradata, such as DBADMIN.
•
Password is the password associated with UserName.
Create a batch file named Create_Emp.bat with the following contents: bteq < Create_Emp_Table.sql > Create_Emp_Table.log 2>&1 where:
2>&1 redirects the error messages to Create_Emp_Table.log to be printed along with the output messages.
Place Create_Emp.bat and Create_Emp_Table.sql in the same folder.
Execute Create_Emp.bat to create the Employee table in the database, Tables_Database.
Create_Emp.bat invokes BTEQ using Create_Emp_Table.sql as the input file. BTEQ executes the commands and statements in the Create_Emp_Table.sql file and writes its output and any errors to the Create_Emp_Table.log file.
Reference Information
Information on...
starting and exiting BTEQ running batch jobs to submit BTEQ commands and Teradata SQL
Is available in...
Basic Teradata Query Reference
Basic Teradata Query Reference
Database Administration 89
Chapter 4: Working with Tables and Views: Application DBAs
Using BTEQ Scripts to Create Database Objects
90 Database Administration
CHAPTER 5
Working with Stored Procedures and User-defined Functions:
Application DBAs
This chapter provides real-world examples of creating stored procedures and user-defined functions.
Creating Stored Procedures
Example 1: Compiling and Executing a Stored Procedure with a Cursor
In most circumstances, it is not optimal to use a cursor for high-volume table updates. For details, see: http://developer.teradata.com/blog/georgecoleman .
However, cursor logic is required to generate a sequence of dynamic SQL statements. This example assumes that there is a file called Sales_Dept_Grant.sql. The file contains this procedure:
REPLACE PROCEDURE Sales_Dept_Grant ( )
BEGIN
Declare GrantStmt VARCHAR(77) Default '';
For UserList as UserList CURSOR FOR
Select User_Name, Admin_Ind From Sales_Dept_Users
Do
CASE UserList.Admin_Ind
WHEN 0 Then
ELSE
Set GrantStmt = 'GRANT SELECT on Sales_Dept to ' || UserList.User_Name;
Set GrantStmt = 'GRANT ALL on Sales_Dept to ' || UserList.User_Name || '
With Grant Option';
END CASE;
CALL DBC.SysExecSql( :GrantStmt );
END For;
END;
If you are in the directory that contains this file, invoke BTEQ and enter these commands:
.LOGON <IP-addr>/<User-ID>
[Enter the Password]
.COMPILE FILE=Sales_Dept_Grant.sql /* This compiles the procedure */
CALL Sales_Dept_Grant(); /* This invokes the procedure */
If you are not in the directory that contains the file, then include the path of the file in the compile command:
.COMPILE FILE=C:\Teradata\Procs\Sales_Dept_Grant.sql
Note: To compile the procedure from Teradata SQL Assistant and other .NET, JDBC, or
ODBC applications, copy and paste the contents of Sales_Dept_Grant.sql into the command window. Only BTEQ uses the .COMPILE command.
Database Administration 91
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
Testing Background
This example was tested using these test tables:
CT Sales_Dept (Acct varchar(44), Sale_Date DATE, Gross DEC(18,2), Net DEC(18,2);
insert into Sales_Dept values ('Acme Bearings','2013-02-13',12345.00,1234.50)
;insert into Sales_Dept values ('Horse Hockey','2013-03-23',57890.00,5789.00)
;insert into Sales_Dept values ('My Jaxx','2013-04-02',678930.00,67893.00)
CT Sales_Dept_Users (User_Name varchar(32), Admin_Ind BYTEINT);
insert into Sales_Dept_Users values ('GC151000',1)
;insert into Sales_Dept_Users values ('dulles',0)
Example 2: Creating a Stored Procedure to Update Accrued Interest
SQL To Create a Savings Table
The following code, Savings.sql, creates a savings table and inserts test data into the table:
DROP TABLE Savings;
CREATE TABLE Savings
,FALLBACK
,NO BEFORE JOURNAL
,NO AFTER JOURNAL
(
AccountType
,AccountNumber
CHAR(1) NOT NULL
CHAR(20) NOT NULL
,OpeningBalance DECIMAL(9,3)
,AccruedInterest DECIMAL(9,3)
,LastYearEnd
,LastAccrualDate
DATE
DATE
)
UNIQUE PRIMARY INDEX( AccountType,AccountNumber );
INSERT INTO Savings ('S','1200121',10000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('S','1200122',20000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('S','1200123',30000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('S','1200124',40000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('S','1200125',50000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('C','1200121', 1000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('C','1200122', 2000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('C','1200123', 3000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('C','1200124', 4000.00,0,'2012-12-31','2013-01-01');
INSERT INTO Savings ('C','1200125', 5000.00,0,'2012-12-31','2013-01-01');
Stored Procedure to To Update Accrued Interest
The following stored procedure, SP_Accrue_Interest.sql, uses a cursor to select qualified records and update accrued interest.
REPLACE PROCEDURE SP_Accrue_Interest(IN Arg_AccontType CHAR(20), OUT Proc_Msg VARCHAR(80))
BEGIN
DECLARE pv_AccountTypeCHAR(1);
DECLARE pv_AccountNumberCHAR(20);
DECLARE pv_CalcBaseDEC(9,2);
DECLARE pv_InterestDEC(9,2);
DECLARE pv_ProcCountINT;
DECLAREcalc_cursor CURSOR FOR
SELECT AccountType, AccountNumber, (OpeningBalance+AccruedInterest) AS BaseAmount FROM Savings
WHERE(AccountType= :Arg_AccontType
OR 'A '= :Arg_AccontType)
92 Database Administration
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
AND LastAccrualDate< DATE;
OPENcalc_cursor;
FETCHcalc_cursor INTO pv_AccountType, pv_AccountNumber,pv_CalcBase;
SET pv_ProcCount = 0;
SET Proc_Msg= 'Processed ' || pv_ProcCount || ' records';
Calc_Loop:
WHILE (SQLCODE =0) DO
SET pv_ProcCount= pv_ProcCount + 1;
SET Proc_Msg= 'Processed ' || pv_ProcCount || ' records';
SET pv_Interest= pv_CalcBase * 0.010;
UPDATESavings
SET AccruedInterest= AccruedInterest + :pv_Interest
,LastAccrualDate= DATE
WHEREAccountType= :pv_AccountType
AND AccountNumber= :pv_AccountNumber;
FETCHcalc_cursor INTO pv_AccountType, pv_AccountNumber,pv_CalcBase;
END;
END WHILE;
CLOSE calc_cursor;
BTEQ Script to Call the Stored Procedure
The following BTEQ script, Bld_iudf.btq, calls the stored procedure and performs SELECTs from the savings table to test the stored procedure.
.SET WIDTH 240
.logon TDSystem/MyUserId,MyPass
.REMARK 'Choosing database';
DATABASE Test_Database;
.REMARK 'Building & populating savings table';
.RUN FILE=./Savings.sql
.REMARK 'Building stored procedure';
.COMPILE FILE=./SP_Accrue_Interest.sql
.REMARK 'Testing stored procedure';
SELECT * FROM SavingsORDER BY 1,2;
CALL SP_Accrue_Interest('C',"Messg");
SELECT * FROM SavingsORDER BY 1,2;
CALL SP_Accrue_Interest('S',"Messg");
SELECT * FROM SavingsORDER BY 1,2;
CALL SP_Accrue_Interest('C',"Messg");
SELECT * FROM SavingsORDER BY 1,2;
CALL SP_Accrue_Interest('S',"Messg");
SELECT * FROM SavingsORDER BY 1,2;
.REMARK 'Rebuilding & populating savings table';
.RUN FILE=./Savings.sql
SELECT * FROM SavingsORDER BY 1,2;
CALL SP_Accrue_Interest('A',"Messg");
Database Administration 93
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
SELECT * FROM SavingsORDER BY 1,2;
CALL SP_Accrue_Interest('A',"Messg");
SELECT * FROM SavingsORDER BY 1,2;
.REMARK 'Finished testing stored procedure';
.LOGOFF
.EXIT 0
Linux Code To Call the BTEQ Script
This Linux code calls the BTEQ script: bteq bld_iudfs.btq 2>&1 | tee bld_iudfs.btq.i1.out
Output of the Stored Procedure
BTEQ 14.00.00.02 Wed Apr 24 09:07:08 2013
+---------+---------+---------+---------+---------+---------+---------+----
.SET WIDTH 240
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.logon TDSystem/MyUserId,
*** Logon successfully completed.
*** Teradata Database Release is 14.10i.00.434
*** Teradata Database Version is 14.10i.00.434
*** Transaction Semantics are BTET.
*** Session Character Set Name is 'ASCII'.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.REMARK 'Choosing database';
Choosing database
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
DATABASE Felix_Test;
*** New default database accepted.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.REMARK 'Populating savings table';
Populating savings table
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.RUN FILE=./SavingsData.sql
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
DELETE FROM Savings ALL;
*** Delete completed. 10 rows removed.
*** Total elapsed time was 2 seconds.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200121',10000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 3 seconds.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200122',20000.00,0,'2012-12-31','2013-01-01');
94 Database Administration
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200123',30000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200124',40000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200125',50000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200121', 1000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200122', 2000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200123', 3000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200124', 4000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200125', 5000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
*** Warning: EOF on INPUT stream.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.REMARK 'Testing internal UDF';
Testing internal UDF
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
Database Administration 95
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
SELECT * FROM Savings ORDER BY 1,2;
*** Query completed. 10 rows found. 6 columns returned.
*** Total elapsed time was 6 seconds.
AccountType AccountNumber OpeningBalance AccruedInterest LastYearEnd LastAccrualDate
----------- -------------------- -------------- --------------- ----------- ---------------
C 1200121 1000.000 .000 12/12/31 13/01/01
C 1200122 2000.000 .000 12/12/31 13/01/01
C 1200123 3000.000 .000 12/12/31 13/01/01
C 1200124 4000.000 .000 12/12/31 13/01/01
C 1200125 5000.000 .000 12/12/31 13/01/01
S 1200121 10000.000 .000 12/12/31 13/01/01
S 1200122 20000.000 .000 12/12/31 13/01/01
S 1200123 30000.000 .000 12/12/31 13/01/01
S 1200124 40000.000 .000 12/12/31 13/01/01
S 1200125 50000.000 .000 12/12/31 13/01/01
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
CALL SP_Accrue_Interest('C',"Messg");
*** Procedure has been executed.
*** Total elapsed time was 2 seconds.
Proc_Msg
--------------------------------------------------------------------------------
Processed 5 records
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
SELECT * FROM Savings ORDER BY 1,2;
*** Query completed. 10 rows found. 6 columns returned.
*** Total elapsed time was 1 second.
AccountType AccountNumber OpeningBalance AccruedInterest LastYearEnd LastAccrualDate
----------- -------------------- -------------- --------------- ----------- ---------------
C 1200121 1000.000 10.000 12/12/31 13/04/24
C 1200122 2000.000 20.000 12/12/31 13/04/24
C 1200123 3000.000 30.000 12/12/31 13/04/24
C 1200124 4000.000 40.000 12/12/31 13/04/24
C 1200125 5000.000 50.000 12/12/31 13/04/24
S 1200121 10000.000 .000 12/12/31 13/01/01
S 1200122 20000.000 .000 12/12/31 13/01/01
S 1200123 30000.000 .000 12/12/31 13/01/01
S 1200124 40000.000 .000 12/12/31 13/01/01
S 1200125 50000.000 .000 12/12/31 13/01/01
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
CALL SP_Accrue_Interest('S',"Messg");
*** Procedure has been executed.
*** Total elapsed time was 1 second.
Proc_Msg
--------------------------------------------------------------------------------
Processed 5 records
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
SELECT * FROM Savings ORDER BY 1,2;
*** Query completed. 10 rows found. 6 columns returned.
*** Total elapsed time was 1 second.
AccountType AccountNumber OpeningBalance AccruedInterest LastYearEnd LastAccrualDate
----------- -------------------- -------------- --------------- ----------- ---------------
C 1200121 1000.000 10.000 12/12/31 13/04/24
C 1200122 2000.000 20.000 12/12/31 13/04/24
C 1200123 3000.000 30.000 12/12/31 13/04/24
C 1200124 4000.000 40.000 12/12/31 13/04/24
C 1200125 5000.000 50.000 12/12/31 13/04/24
S 1200121 10000.000 100.000 12/12/31 13/04/24
S 1200122 20000.000 200.000 12/12/31 13/04/24
S 1200123 30000.000 300.000 12/12/31 13/04/24
S 1200124 40000.000 400.000 12/12/31 13/04/24
S 1200125 50000.000 500.000 12/12/31 13/04/24
96 Database Administration
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
CALL SP_Accrue_Interest('C',"Messg");
*** Procedure has been executed.
*** Total elapsed time was 1 second.
Proc_Msg
--------------------------------------------------------------------------------
Processed 0 records
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
SELECT * FROM Savings ORDER BY 1,2;
*** Query completed. 10 rows found. 6 columns returned.
*** Total elapsed time was 1 second.
AccountType AccountNumber OpeningBalance AccruedInterest LastYearEnd LastAccrualDate
----------- -------------------- -------------- --------------- ----------- ---------------
C 1200121 1000.000 10.000 12/12/31 13/04/24
C 1200122 2000.000 20.000 12/12/31 13/04/24
C 1200123 3000.000 30.000 12/12/31 13/04/24
C 1200124 4000.000 40.000 12/12/31 13/04/24
C 1200125 5000.000 50.000 12/12/31 13/04/24
S 1200121 10000.000 100.000 12/12/31 13/04/24
S 1200122 20000.000 200.000 12/12/31 13/04/24
S 1200123 30000.000 300.000 12/12/31 13/04/24
S 1200124 40000.000 400.000 12/12/31 13/04/24
S 1200125 50000.000 500.000 12/12/31 13/04/24
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
CALL SP_Accrue_Interest('S',"Messg");
*** Procedure has been executed.
*** Total elapsed time was 1 second.
Proc_Msg
--------------------------------------------------------------------------------
Processed 0 records
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
SELECT * FROM Savings ORDER BY 1,2;
*** Query completed. 10 rows found. 6 columns returned.
*** Total elapsed time was 4 seconds.
AccountType AccountNumber OpeningBalance AccruedInterest LastYearEnd LastAccrualDate
----------- -------------------- -------------- --------------- ----------- ---------------
C 1200121 1000.000 10.000 12/12/31 13/04/24
C 1200122 2000.000 20.000 12/12/31 13/04/24
C 1200123 3000.000 30.000 12/12/31 13/04/24
C 1200124 4000.000 40.000 12/12/31 13/04/24
C 1200125 5000.000 50.000 12/12/31 13/04/24
S 1200121 10000.000 100.000 12/12/31 13/04/24
S 1200122 20000.000 200.000 12/12/31 13/04/24
S 1200123 30000.000 300.000 12/12/31 13/04/24
S 1200124 40000.000 400.000 12/12/31 13/04/24
S 1200125 50000.000 500.000 12/12/31 13/04/24
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.REMARK 'Populating savings table';
Populating savings table
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.RUN FILE=./SavingsData.sql
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
DELETE FROM Savings ALL;
*** Delete completed. 10 rows removed.
*** Total elapsed time was 2 seconds.
Database Administration 97
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200121',10000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200122',20000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200123',30000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200124',40000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('S','1200125',50000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200121', 1000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200122', 2000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200123', 3000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200124', 4000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
INSERT INTO Savings ('C','1200125', 5000.00,0,'2012-12-31','2013-01-01');
*** Insert completed. One row added.
*** Total elapsed time was 1 second.
98 Database Administration
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
*** Warning: EOF on INPUT stream.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
SELECT * FROM Savings ORDER BY 1,2;
*** Query completed. 10 rows found. 6 columns returned.
*** Total elapsed time was 1 second.
AccountType AccountNumber OpeningBalance AccruedInterest LastYearEnd LastAccrualDate
----------- -------------------- -------------- --------------- ----------- ---------------
C 1200121 1000.000 .000 12/12/31 13/01/01
C 1200122 2000.000 .000 12/12/31 13/01/01
C 1200123 3000.000 .000 12/12/31 13/01/01
C 1200124 4000.000 .000 12/12/31 13/01/01
C 1200125 5000.000 .000 12/12/31 13/01/01
S 1200121 10000.000 .000 12/12/31 13/01/01
S 1200122 20000.000 .000 12/12/31 13/01/01
S 1200123 30000.000 .000 12/12/31 13/01/01
S 1200124 40000.000 .000 12/12/31 13/01/01
S 1200125 50000.000 .000 12/12/31 13/01/01
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
CALL SP_Accrue_Interest('A',"Messg");
*** Procedure has been executed.
*** Total elapsed time was 1 second.
Proc_Msg
--------------------------------------------------------------------------------
Processed 10 records
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
SELECT * FROM Savings ORDER BY 1,2;
*** Query completed. 10 rows found. 6 columns returned.
*** Total elapsed time was 1 second.
AccountType AccountNumber OpeningBalance AccruedInterest LastYearEnd LastAccrualDate
----------- -------------------- -------------- --------------- ----------- ---------------
C 1200121 1000.000 10.000 12/12/31 13/04/24
C 1200122 2000.000 20.000 12/12/31 13/04/24
C 1200123 3000.000 30.000 12/12/31 13/04/24
C 1200124 4000.000 40.000 12/12/31 13/04/24
C 1200125 5000.000 50.000 12/12/31 13/04/24
S 1200121 10000.000 100.000 12/12/31 13/04/24
S 1200122 20000.000 200.000 12/12/31 13/04/24
S 1200123 30000.000 300.000 12/12/31 13/04/24
S 1200124 40000.000 400.000 12/12/31 13/04/24
S 1200125 50000.000 500.000 12/12/31 13/04/24
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
CALL SP_Accrue_Interest('A',"Messg");
*** Procedure has been executed.
*** Total elapsed time was 1 second.
Proc_Msg
--------------------------------------------------------------------------------
Processed 0 records
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
SELECT * FROM Savings ORDER BY 1,2;
*** Query completed. 10 rows found. 6 columns returned.
*** Total elapsed time was 1 second.
AccountType AccountNumber OpeningBalance AccruedInterest LastYearEnd LastAccrualDate
----------- -------------------- -------------- --------------- ----------- ---------------
C 1200121 1000.000 10.000 12/12/31 13/04/24
C 1200122 2000.000 20.000 12/12/31 13/04/24
C 1200123 3000.000 30.000 12/12/31 13/04/24
Database Administration 99
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating User-defined Functions
C 1200124 4000.000 40.000 12/12/31 13/04/24
C 1200125 5000.000 50.000 12/12/31 13/04/24
S 1200121 10000.000 100.000 12/12/31 13/04/24
S 1200122 20000.000 200.000 12/12/31 13/04/24
S 1200123 30000.000 300.000 12/12/31 13/04/24
S 1200124 40000.000 400.000 12/12/31 13/04/24
S 1200125 50000.000 500.000 12/12/31 13/04/24
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.REMARK 'Finishied testing internal UDF';
Finishied testing internal UDF
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.LOGOFF
*** You are now logged off from the DBC.
+---------+---------+---------+---------+---------+---------+---------+---------+---------+---------+----
-----+---------+---------+-
.EXIT 0
*** Exiting BTEQ...
*** RC (return code) = 0
For More Information
For more information about creating stored procedures, see SQL Stored Procedures and
Embedded SQL.
Creating User-defined Functions
Example: Creating a User-defined Function To Verify a Date
The following is an example of creating an external C language UDF.
SQL Definition
---- UDF Build DDL/SQL--
REPLACE FUNCTION my_yyyymmdd_to_date2
(
InputDate VARCHAR(8) CHARACTER SET LATIN
)
RETURNS DATE
LANGUAGE C
SPECIFIC my_yyyymmdd_to_date2
NO SQL
DETERMINISTIC
PARAMETER STYLE SQL
CALLED ON NULL INPUT
EXTERNAL NAME 'CS!my_yyyymmdd_to_date2!./my_yyyymmdd_to_date2.c'
;
C Function Definition
The following user-defined function, my_yyyymmdd_to_date2.c, validates whether the argument character string in YYYYMMDD format is a valid date. It returns either a date or returns a NULL if the string is not a valid date.
100 Database Administration
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating User-defined Functions
/*
my_yyyymmdd_to_date2.c
Teradata User Defined Function (UDF)
Calling
-------
my_yyyymmdd_to_date2(date_str);
SELECT my_yyyymmdd_to_date2('20130423') AS ValidDate;
Parameters
----------
date_str
Character string containing date to be validated
UDF Compilation
---------------
REPLACE FUNCTION my_yyyymmdd_to_date2
(
InputDate VARCHAR(8)
)
RETURNS DATE
LANGUAGE C
NO SQL
DETERMINISTIC
PARAMETER STYLE SQL
EXTERNAL NAME 'CS!my_yyyymmdd_to_date2!./my_yyyymmdd_to_date2.c'
;
*/
/* Must define SQL_TEXT before including "sqltypes_td"*/
#define SQL_TEXT Latin_Text
#include "sqltypes_td.h"
#include "stdio.h"
#include "string.h"
#define IsNull -1
#define IsNotNull 0
#define NoSqlError "00000"
#define YYYYMMDD_LENGTH 8
#define ERR_RC 99 void my_yyyymmdd_to_date2
(
VARCHAR_LATIN*InputDateString
,DATE *result
,int
,int
*inputDateStringIsNull
*resultIsNull
,char sqlstate[6]
,SQL_TEXT extname[129]
,SQL_TEXT specificname[129]
,SQL_TEXT error_message[257]
)
{ char input_integer[30]; int year_yyyy; int month_mm; int day_dd; char day_char[3]; char month_char[3]; char year_char[5];
int in_len,i;
/* Return Nulls on Null Input */ if ((*inputDateStringIsNull == IsNull))
{ strcpy(sqlstate, "22018") ; strcpy((char *) error_message, "Null value not allowed.") ;
Database Administration 101
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating User-defined Functions
}
*resultIsNull = IsNull; return;
} in_len = strlen(InputDateString); if ( in_len != YYYYMMDD_LENGTH )
{
*result = ( 1 * 10000 ) + ( 12 * 100) + 1;
*resultIsNull = IsNull; strcpy((char *) sqlstate, "01H01"); strcpy((char *) error_message, "InputDateString is of wrong length, must be in YYYYMMDD format"); return;
} if ( in_len != YYYYMMDD_LENGTH )
{
*result = ( 1 * 10000 ) + ( 12 * 100) + 2; return;
} strcpy(input_integer , (char *) InputDateString); for (i = 0; i<in_len; i++)
{ if (input_integer[i] < '0' || input_integer[i] > '9')
{
*result = ( 1 * 10000 ) + ( 1 * 100) + 3; return;
} else
{ input_integer[i] = tolower(input_integer[i]);
}
} sprintf(year_char,"%c%c%c%c",input_integer[0],input_integer[1],input_integer[2],input_integer[3]); sprintf(month_char,"%c%c",input_integer[4],input_integer[5]); sprintf(day_char,"%c%c",input_integer[6],input_integer[7]); year_yyyy= atoi(year_char); month_mm= atoi(month_char); day_dd = atoi(day_char);
/* Format output_date in internal Teradata format ((YEAR - 1900) * 10000 ) + (MONTH * 100) + DAY*/
*result = (( year_yyyy - 1900 ) * 10000 ) + ( month_mm * 100) + day_dd;
BTEQ Script To Call the User-defined Function
This BTEQ script calls the user-defined function and tests it by selecting from the my_yyyymmdd_to_date2.sql table:
.SET WIDTH 240
.logon TDSystem/MyUserId,MyPass
.REMARK 'Building External UDF';
DATABASE Test_Database;
.RUN FILE=./my_yyyymmdd_to_date2.sql
.REMARK 'Testing External UDF';
SELECT Felix_Test.my_yyyymmdd_to_date2('20130422')AS ValidDate;
SELECT Felix_Test.my_yyyymmdd_to_date2('201304')AS ValidDate;
SELECT Felix_Test.my_yyyymmdd_to_date2(NULL)AS ValidDate;
.REMARK 'Finished testing External UDF';
.LOGOFF
.EXIT 0
102 Database Administration
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating User-defined Functions
Linux Command To Call the BTEQ Script
The following Linux commands call the BTEQ script: bteq bld_xudfs.btq 2>&1 | tee bld_xudfs.btq.i1.out
For More Information
For more information about creating user-defined functions, see “UDF Code Examples” in
SQL External Routine Programming.
You can also download UDFs from the Teradata Developer Exchange, located at developer.teradata.com
.
Database Administration 103
Chapter 5: Working with Stored Procedures and User-defined Functions: Application DBAs
Creating User-defined Functions
104 Database Administration
CHAPTER 6
Working with Users, Roles, and
Profiles: Operational DBAs
This chapter describes the strategies that an administrator can use to control user access.
Specific topics include:
•
Assessing user needs to develop a user management strategy
•
Creating users and user accounts
•
Creating user profiles
•
Using roles to manage privileges
•
Granting privileges directly to users
•
Creating Teradata Viewpoint users
Overview of Establishing Users
To establish users for the first time, perform all the tasks in this chapter completely and in the order presented.
1
Group potential database users by functional group and determine database access needs.
See
“Assessing Database User Needs” on page 106
.
2
Set up user accounts to track resource usage and assign workload priorities. See
3
Set up profiles to define resources, account assignments, and password control parameters for groups of users.
“Creating User Profiles” on page 110
.
4
Create database users and assign profiles, space and accounts. See
5
Create roles to define privileges for users groups and grant role membership to users.
“Using Roles to Manage User Privileges” on page 117
.
6
Grant additional privileges that are not suitable for role assignment directly to users. See
“Granting Privileges Directly to Users” on page 125 .
User Access Methods
This book describes permanent database users who are created with the CREATE USER request, have PERM space on disk, and are authenticated by Teradata Database. For more information on granting privileges to users not managed in the database, for example directory users, see Security Administration.
Database Administration 105
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Types of Users
Types of Users
The following list identifies five typical user types and job functions.
Functional Category Description
General users
Update users
Batch users
Database end-users who only need to read the data or to run preexisting queries or macros to generate reports.
Privileged users who perform some general user functions and who also may need to insert, update or delete data, and create new database objects.
High-level users who typically perform batch-level functions, for example:
•
Load, update, and export operations, including creation and deletion of staging tables.
•
Data backup, archive, and restore operations.
Database programmers
Assistant administrators
Users who design and create queries, macros, and stored procedures, as well as database objects, for use by the user community. Programmers may require administrator privileges within a development database, while needing only limited privileges in the main production database.
Administrative users that assist the principal administrator, user
DBADMIN, created in
“Setting Up Teradata Administrator” on page 29
.
Assistant administrative users may have most or all of the same privileges granted to user DBADMIN, but have a much smaller permanent space allocation because they have a limited ownership of objects
Note: You can create additional user types where necessary to identify functional differences, however, the examples provided in this book are based on these user types.
Assessing Database User Needs
Assess database user needs and develop a user management strategy before you begin executing user management setup tasks.
1
2
Create a list of all users that require access to Teradata Database and identify each one according to user type. Minimize the number of user types to simplify user management.
Define user resource requirements for use in creating profiles:
•
Examine user space requirements.
•
Users that create or own databases, tables and other space-consuming objects require permanent storage space (perm space).
•
Users that execute SQL queries, macros, or stored procedures require spool space to contain the required temporary database structures. The spool specification in a
106 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Best Practices for Creating Users
3
4
5
profile limits the amount of available space a profile member can use.
•
Define user accounting requirements for use in specifying one or more account strings.
Each profile can include one or more accounts. Each account can specify:
•
A priority level (low, medium, high, and rush)
•
A four-character account identifier based on department, group, or function.
•
A date and time stamp
•
Define the user default database, that is, the database where the user most often works.
•
Define password control parameters. Refer to the chapters on password controls and managing passwords in Security Administration. Consider your site security policy and decide whether or not all users can share global password parameters, or if you need to set them by user group, in profiles.
Review the database objects (such as views, tables, macros, functions, and procedures) that users or user groups must access to do their job. Always define database privileges at the highest level that is appropriate for a particular user. For example, if a user requires the same privileges on all views in a database, assign privileges at the database level.
Identify groups of users with common database privilege requirements and create roles to define the privileges for each group. Consolidate minor differences in access requirement where possible to minimize the number of roles.
You may need to assign specialized privileges directly to individual users.
Best Practices for Creating Users
While there is no single set of procedures that would best fit all the varieties of system configurations possible and meet all site requirements, consider the following suggestions for best practices in creating users.
•
Create separate users for the security administrator and database administrator.
Establish a security administrator user to perform security-related tasks. The biggest threat to security is usually the misuse of information or privileges by authorized users. No one single user should have all the privileges for everything. Neither should an administrative user have access to something for which he does not need access.
•
Ensure that all users are uniquely identified. This means not allowing several users to log in to Teradata Database using the same username.
Setting up users to be unique enables you to effectively monitor user activities and helps you identify the source of a security breach if there is one. By disallowing users to use a generic or shared user account, each user is held accountable for his specific actions. In addition, unique users can be allowed to view or not view certain information that is protected by row-level security constraints. For more information, see Security
Administration.
•
Consider the function of the user. Create administrative users under separate users/ databases so that privileges can be granted from the owning user/database. For example,
Database Administration 107
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Creating User Accounts the HR database and Marketing database can have separate administrative users to manage privileges for their respective users.
•
For non-administrative users, if possible, assign the user to a role with the required privileges rather than granting privileges to the user directly. It is easier to use roles to manage privileges. Profiles should also be created for non-administrative users (see
“Creating User Profiles” on page 110 ).
•
Limit the permanent and spool space of users and grant additional space if it becomes necessary. Limit spool space using a profile allows you to protect the system from runaway queries.
Creating User Accounts
Each user logon to the database should specify a user account string.
Accounts are used by the database to:
•
Assign priority to individual sessions to manage the workload
•
Monitor resource usage
•
Charge for space or resources used
Teradata recommends that you specify at least one user account for each user. Users with similar responsibilities and resource requirements should be assigned to the same account(s) to reduce the overall number of accounts you must maintain.
You can specify accounts in:
•
A CREATE PROFILE or MODIFY PROFILE statement for the profile specified in the user definition (preferred).
•
A CREATE USER or MODIFY USER statement.
Note: If a user is a member of a profile that specifies one or more accounts, the profile account assignments supersede and invalidate any accounts specified in the user definition.
Reference Information
Information on...
specifying the account string at logon
Is available in...
•
Security Administration
•
The logon section in the User Guide for each Teradata tool or utility.
Creating Account Strings
Teradata recommends that you devise an account system and then identify the accounts required for various groups in the user community before attempting to specify account information in user profiles. An account string is limited to 128 characters.
108 Database Administration
Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Creating User Accounts
User account strings should follow this format:
‘$G0+account_ID&S&D&H’
Note: If a CREATE or MODIFY statement specifies more than one account string, the individual strings must be separated by commas and the entire list of strings enclosed by parentheses.
where:
Element
Required Account String Variables
Description
$G0+
account_ID
Optional. The performance group identifier. Priority scheduler uses the identifier to determine the priority of the session.
Recommendation: Specify one of the following instead of $G0+:
•
$R0+ = Rush
•
$H0+ = High
•
$M0+ = Medium (default)
•
$L0+ = Low
Note: Consider the following when specifying the performance group.
•
The 0 specification designates resource partition 0, the default.
•
The + character is a place holder that forces the account_ID to start at the fifth character, which is important for compatibility with certain administrative tools.
The account name. Uniquely identifies the account.
Recommendation: You can use the four characters of the
account_ID to differentiate accounts and to provide an abbreviated description of the account function.
For example:
•
A department, such as finance (FIN) or marketing (MKT)
•
A job function, such as general use (G), update (U), or batch processing (B).
A complete account ID might resemble one of the following:
FINB or MKTU
Note: Although more characters are allowed, using a fourcharacter account_ID maintains application compatibility.
Account String Expansion (ASE) Variables
Teradata Database tags the session with information specified by the ASE variables for later use in management of database sessions. Each ASE variable expands to the number of characters needed to express the related information. The string limit is enforced against the expanded form.
Specification of the ASE variables shown below is optional, but recommended. Other variables are available. For information, see
“Logging Resource Usage Data with Account String Variables” on page 162 .
109
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Creating User Profiles
Element
&S
&D
&H
Description
The ampersand (&) designates an account string expansion
(ASE) variable. Teradata Database automatically replaces these variables with actual values upon opening the session. You can use any or all of the following:
•
&S = the session number
•
&D = the date
•
&H = the time
Determining the Account for a User Session
The system determines the account for a user when the user logs on as follows:
•
If the user is assigned only one account, the logon uses that account by default.
•
If the user is assigned multiple accounts and the default account (the first listed in the operant user or profile definition) is appropriate for the session, the logon need not contain the account string.
•
If the user has multiple accounts and does not want to use the default, the logon string can specify another assigned account.
•
During a session, the user can submit a SET SESSION ACCOUNT statement to change the session to another assigned account.
•
If the user has no account assignment, the session defaults to the first account listed for the owning user or database. Such a use of default owner accounts is not recommended.
Creating User Profiles
About Profiles
You can create profiles to define resource parameters for groups of users with similar needs instead of defining them for each individual user. Then list the profile in a CREATE or
MODIFY USER statement to establish membership in a profile for each user.
Profiles can be based on the primary user types such as those shown in “Types of Users” on page 106
, or other factors that group users according to database resource requirements.
Use profiles to specify parameters such as:
•
Permanent and spool space allocations
•
User account(s)
•
Password control parameters
•
Default database assignment
•
Query band
Note: The value for a profile attribute supersedes and replaces any corresponding values specified in global defaults, or in a CREATE USER request.
110 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Creating User Profiles
Creating Profiles
1
Open your favorite client software to construct your SQL request. For example:
2
CREATE PROFILE profile_name AS
ACCOUNT = (‘account_str1’,’account_str2’)
DEFAULT DATABASE = database_name,
SPOOL = spool_space,
PASSWORD =
(EXPIRE = n_days,
MAXLOGONATTEMPTS = attempts,
LOCKEDUSEREXPIRE = n_minutes),
QUERY_BAND = 'Pair_name=pair_value;' [NOT] DEFAULT';
Include the parameters:
Parameter
Name
Description
The unique name for the profile. Include such things as the user type or user department to identify its function. For example, GenUser_Profile or
Finance_Profile. Consolidate user requirements where possible to avoid the unnecessary proliferation of profiles.
Account The database uses accounts for user job tracking and to prioritize requests.
Recommendation: Enter one or more account strings in accordance with
the format shown in “Creating User Accounts” on page 108
.
If you assign more than one account string to a profile, enclose each string in apostrophes, separate the strings with commas and enclose the complete specification in parentheses, for example:
(‘$M0+accl&S&D&H’,‘$L0+accr&S&D&H’, ‘$R0+acc51’)
Note: The content of an account string is limited to 128 characters.
Default Database Optional. The database the user is most likely to access.
If no name is specified, the system uses the name specified in the user definition (CREATE USER statement).
Recommendation: Specify the default database at the user level, rather than in the profile, to allow for differences in user default database requirements.
Database Administration 111
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Creating User Profiles
Parameter
Spool Space
Description
This value limits the amount of space available for intermediate query results or formatted answer sets to queries and volatile tables. Spool space for a user is shared among the queries that user is executing concurrently.
The system borrows spool space for a user from unused permanent space in the system.
Recommendation: Spool space requirements in a profile depend on the activities of member users. Begin by specifying spool space in bytes according to the following percentages of the total perm space allocation for the Spool_Reserve database.
•
General users: 5%
•
Update users: 10%
•
Batch users: 10%
•
Assistant administrative users: 10%
•
Database programmers: 10%
Periodically monitor usage patterns and modify spool space assignments when required using the MODIFY PROFILE statement.
Spool space can also be specified as part of a CREATE USER statement or modified as part of a MODIFY USER statement.
Note: Profile members that do not require permanent space must still be allocated a small amount of spool space in the profile, for example, 1 MB.
Temporary Space This is required only when using global temporary tables.
Recommendation: Initially omit a temporary space specification unless you have a predetermined temporary space requirement for this group of users. If necessary, you can add a temporary space specification later with a
MODIFY PROFILE statement.
Password Optional. The password control parameters allow you to apply different settings in the profile than are in the global default settings created in
DBC.SysSecDefaults, as part of step 2 in “Setting Up the Database
Administrator User” on page 30
Recommendation: Do not reset any password control parameter values in a profile unless the profile members cannot use the system defaults.
Cost Profile
Constraint
Query Band
Optional. The Optimizer cost profile name to be associated with the profile.
Optional. Name of one or more row-level security constraints, each followed by a list of the hierarchical levels or non-hierarchical categories, valid for the constraint, which are being assigned to the profile_name.
Optional. Creates a query band that is automatically set for the session at
logon. For more details, see “About Defining Name-Value Pairs for Profile
3
Specify profile membership as follows:
•
Assign a profile to each user as part of “Working with Database Users.”
or
•
After creating users, use a MODIFY PROFILE statement to include member users.
112 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Working with Database Users
4
Close the client software or go on to
.
Reference Information
Information on...
syntax and options for the
CREATE PROFILE statement using accounts to enforce session priorities information on monitoring and reallocating space global temporary tables password control options
Is available in...
SQL Data Definition Language Syntax and
Examples
•
“Managing the Database Workload with
Teradata Active System Management” on page 228
“Chapter 9 Managing Space: Operational DBAs” on page 171
Database Design
Security Administration
Dropping Profiles
DROP statements are recorded in the Data Dictionary. However, the result of a DROP
PROFILE, DROP DATABASE, or DROP ROLE statement is not cascaded to the user rows in
DBC.Dbase, so the corresponding default setting for each affected user is not reset to NULL.
When an affected user next logs on, the system does not return an error or warning.
If you drop a default profile, the system uses the following by default:
•
ACCOUNT, SPOOL, TEMP, and DEFAULT DATABASE specifications in the CREATE
USER or latest MODIFY USER statement
•
Password attributes defined at the system level in DBC.SecurityDefaultsV (see Security
Administration)
Note: If you re-create a profile with the same name after it was dropped, users defined for that profilename are assigned the profile parameters at the next logon. The effect of a profile recreation is not immediate.
Working with Database Users
About Users
This task creates individual database users, including lower level administrators, database programmers and other privileged users, and database end-users.
Creating Users
1
Start Teradata Administrator and log on as user DBADMIN.
Note: This procedure creates users individually. To automate the creation of users by using
BTEQ scripts, see “Using BTEQ Scripts to Create Database Objects” on page 88 .
Database Administration 113
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Working with Database Users
2
3
From the main screen, click the Preview Mode button generated SQL.
Do the following to create a user:
a
Click Tools>Create>User.
b
to allow future display of
In the Create User dialog, specify the values for the required fields and controls.
Field or Control
User Name
Owner
Password
Permanent Space
Spool Space
Explanation
Required. The name of an individual Teradata Database user.
Each database user should be uniquely identified for accurate logging. Do not allow users to share a username for accessing the database.
Note: The username must conform to the rules for SQL object names.
Required. The name of the database or user that owns the space in which the user is created.
Recommendation: DBADMIN. All users should be created in space owned by the database administrator.
Required. A temporary password. The user is prompted to change the temporary password to a permanent, private password at first logon.
May be required. Space in bytes that can be used to contain objects the user creates or owns. Permanent space is required for those users that create and store space-consuming objects.
Recommendation: Base the permanent space specification on user needs.
•
General users: Not required.
•
Update users: Required if users create and store objects.
•
Batch users: Required if users create and store objects.
•
Assistant administrators: Required if users create and store objects.
•
Database programmers: Required to create and store objects.
Optional. Specifies the limit to the amount of space in bytes available for temporary process constructs such as intermediate query results or formatted answer sets to queries and volatile tables. Spool space for a user is shared among the queries that user is executing concurrently.
The system borrows user spool space from any unused permanent space found on the system.
The spool space specification in a profile takes precedence over spool space specified in a CREATE USER statement.
Recommendation: Specify spool space as part of a profile unless the user requires special spool considerations.
If you do enter a value for an individual user, specify approximately 20% of the perm space available to the largest database the user will query.
114 Database Administration
Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Working with Database Users
Field or Control
Temporary Space
Account
Default Database
Profile Name
Explanation
Not applicable. Required only when using global temporary tables.
Specifying temporary space requires that you develop a strategy for use of the advanced features that need such space. If necessary, you can add a temporary space specification later with a MODIFY USER or MODIFY
PROFILE statement.
Optional. You can set up user account information to form the basis of various user management options.
Recommendation: Do not specify accounts at the user level, unless the user requires unique system priorities. Instead name a profile in the
Profile parameter that contains the needed account(s). For information,
see “Creating User Profiles” on page 110
.
Note:
If a user is a member of a profile that specifies one or more accounts, the profile account assignment supersedes and invalidates all account assignments specified in the user definition.
Optional. Identifies the database that owns the space in which the
Teradata Database stores or searches for new or target objects in an SQL query, unless a different database is specified in the query.
Note: Users can use the DATABASE statement to establish a new default database for the current session.
Recommendation: Specify the database most often accessed by the user.
•
General users: Specify a read-only Views database. If restrictions are necessary, use role privileges to allow access to particular views.
•
Update users: Specify either a read-only Views or an updatable Views database, depending on where the user normally works.
•
Batch users: Specify the Tables_Database, where batch operations take place.
•
Assistant administrators: Specify DBADMIN.
•
Database programmers: Specify the name of the development database.
Optional. Gives the user membership in the named profile.
For syntax elements that appear in both the CREATE USER statement for a user, and in a profile in which the user is a member, the profile values take precedence.
A user can be a member of only one profile.
Recommendation: Specify profile membership for all users. Select from the drop-down list and highlight the needed profile.
115
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Working with Database Users
116
Field or Control
Default Role
Explanation
Identifies the default role used for determining user privileges in the database for the current session. This is usually the role associated with the activities performed most frequently by the user.
Note: The user can employ the SET ROLE statement at the beginning of a session or transaction to change from the default role to another role. The
SET ROLE ALL statement allows access to all roles of which the user is a member.
Recommendation: You may specify a default role later after you have created the user and the roles. See
member of only one role.
Optional.
Recommendation: Do not use for initial database implementation.
Default Journal
Startup String
Before Journal
After Journal
FALLBACK
Collation
Time Zone
Default Date Form
Default Character
Set
Optional. Specification of FALLBACK automatically creates a duplicate of each table stored in the user space, in addition to the duplicates already created by disk mirroring. The system reverts to the FALLBACK tables in the event of a failure.
Recommendation: Specify FALLBACK only for individual missioncritical tables or tables so large that their size prevents timely backup to tape or external disk.
Optional. Database default values that apply to all users are already set as part of system installation and configuration.
Recommendation: Do not change the default values for a user unless the default value is not applicable.
Note: All users must use the same collation value.
c
Click the Create button. The Query Window displays the generated CREATE USER statement, similar to the following:
CREATE USER "GenUser1" FROM "DBADMIN"
AS PERM = 0
PASSWORD = "temp123"
STARTUP = ''
DEFAULT DATABASE = "Personnel_Views"
NO FALLBACK
NO BEFORE JOURNAL
NO AFTER JOURNAL
PROFILE=”GenUser_Profile”
d e
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
Close the Query Window and the Create User dialog box.
Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Creating Temporary Passwords for First Login
4
Log off Teradata Administrator or go on to “Using Roles to Manage User Privileges” on page 117 .
Reference Information
For Step...
Information on...
3 syntax and options for the CREATE
USER statement the creation and use of accounts
Is available in...
SQL Data Definition Language Syntax and
Examples
“Creating User Accounts” on page 108
Creating Temporary Passwords for First Login
Teradata Database requires you to associate a username with a password when you first create a user. You must specify a password, even if temporary. Otherwise the Parser rejects the statement as incomplete.
The security administrator can prompt users for a new password upon first login if:
•
At the time of installing a new system, the administrator has created generic passwords for all users and wants them to choose their own passwords.
•
New password requirements must be enforced. For example, if the security administrator wants to require mixed case and wants all users to change their passwords.
If you are using Teradata Administrator, simply check the “Temporary” box next to the password. This adds the 'FOR USER' phrase to the generated command.
Note: Teradata password rules do not apply to external authentication. For more information, see Security Administration.
To use SQL, see “Passwords” under “CREATE USER” in SQL Data Definition Language.
Using Roles to Manage User Privileges
Do the following to setup roles to manage user privileges:
1
Create roles, as shown in
2
Grant privileges to each role, as shown in “Granting Privileges to a Role.”
3
Grant role membership to users, as shown in
“Granting User Membership in a Role.”
Note: Some privileges cannot be granted to a role. You must grant those privileges directly to
a user. For information, see “Granting Privileges Directly to Users” on page 125 .
Database Administration 117
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Using Roles to Manage User Privileges
User Types and Minimum Required Privileges
The following table describes the minimal privileges required by the three basic user types.
User Type
General
Update
Privilege Requirements
Database end-users who typically read data and execute macros in a read-only
Views database.
Assign the following privileges on a read-only Views database to all General users:
•
EXECUTE
•
SELECT
The read-only Views database must have privileges on the Tables_Database, as
shown in “Working with Table Access Privileges for Views” on page 129 .
Privileged database users who update data generally require the following privileges:
Privileges required on a read-only Views database:
•
EXECUTE
•
SELECT
•
CREATE VIEW and DROP VIEW
•
CREATE MACRO and DROP MACRO
Privileges required on an updatable Views database:
•
EXECUTE
•
SELECT
•
INSERT, UPDATE, and DELETE
•
EXECUTE PROCEDURE
•
EXECUTE FUNCTION
•
CREATE VIEW and DROP VIEW
•
CREATE MACRO and DROP MACRO
Additional recommended privileges on an updatable Views database for database programmers:
•
SHOW
•
ALTER, CREATE, and DROP PROCEDURE
•
ALTER, CREATE, and DROP FUNCTION
•
ALTER and CREATE EXTERNAL PROCEDURE
•
CREATE TRIGGER and DROP TRIGGER
To provide update capability on views, you must grant the updatable Views database SELECT, INSERT, UPDATE, DELETE WITH GRANT OPTION privileges on the referenced tables in the Tables_Database, as shown in
“Working with Table Access Privileges for Views” on page 129
.
118 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Using Roles to Manage User Privileges
User Type
Batch
Assistant
Administrators
Database
Programmers
Privilege Requirements
Data movers, who typically perform batch-level functions, such as:
•
Batch data load, update, and export operations.
•
Data backup, archive, and restore operations, including creating and deleting staging tables.
Batch users must have the following privileges on the Tables_Database:
•
SELECT
•
INSERT, UPDATE, DELETE
•
DUMP and RESTORE
•
CHECKPOINT
•
CREATE TABLE and DROP TABLE
If you need one or more assistant administrators to help share administrative duties, they may require the same privileges as those granted to DBADMIN, or a subset of those privileges. Use the procedure shown in
Administrator User” on page 30
to create assistant administrators and grant the privileges they need.
Database programmers may require administrator-level privileges within the development database to allow them to create and test database objects.
Administrators can then deploy the objects to the production database.
Types of Privileges
You can explicitly grant database privileges to users, roles, or databases. Users also gain other privileges without a formal grant. Before you decide which privileges to explicitly grant, make sure you understand the privileges users receive by other means.
Privilege
Implicit Privileges
Ownership
Description
Users that own perm space have certain implicit (ownership) privileges on any object contained in the space they own, even if they did not create the object.
Note: You can transfer ownership using the GIVE statement.
Explicit privileges
Automatic
GRANT
When a user creates a database object, Teradata Database automatically grants privileges to:
•
The creator of the object
•
A newly created user or database
You can grant privileges:
•
Directly to a user or database
•
To a role, then GRANT membership in the role to one or more users
•
To an external role, then map the external role to one or more groups of directory users
Database Administration 119
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Using Roles to Manage User Privileges
Privilege
Inherited
Description
Privileges that a user acquires indirectly.
•
All users inherit the privileges of the system-generated user, PUBLIC, a role-like collection of privileges available by default, whether or not they have any other privileges. You can grant additional privileges to PUBLIC.
•
A user inherits all the privileges granted to each role of which the user is a member.
•
Directory users inherit the privileges of the database users and external roles to which they are mapped.
Note: The system logs automatic and explicit privileges in exactly the same way, and are indistinguishable as to how they were acquired. All privileges except implicit privileges are stored in the data dictionary in the DBC.AccessRights table, by each user.
For a detailed discussion of privileges, see “Teradata Database User Privileges” in Chapter 5:
“Creating Users and Granting Privileges” in Security Administration.
Privileges That Must Be Explicitly Granted
Some privileges cannot be acquired by indirect means and must be explicitly granted to a user, database, or role. For information, see “Privileges that Must Be Explicitly Granted” in Security
Administration.
Creating User Roles
1
Start Teradata Administrator and log on as user DBADMIN.
2
3
4
5
On the main screen, click the Preview Mode button generated SQL.
Click Tools>Create>Role.
to allow future display of
In the Roles dialog, use the Role Name field to specify a unique role name. The role name should also provide some indication of the role function or the group of users to which it applies, for instance:
•
FinanceUser_Role
•
Update_Role
Click the Create button. The Query Window displays the generated CREATE ROLE statement:
CREATE ROLE role_name
6
7
8
In the Query Window, click the Run Query button. A Query Complete message appears in the bottom left corner of the main window when the query executes.
Close the Query Window and the Roles dialog box.
Repeat this procedure to create the needed roles.
120 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Using Roles to Manage User Privileges
9
Log off Teradata Administrator or go on to “Granting Privileges to a Role.” .
Granting Privileges to a Role
Use the GRANT statement to grant privileges to the roles necessary to serve the basic user types. A GRANT statement takes the following basic form:
GRANT privilege
ON database_object_name
TO role_name
1
Start Teradata Administrator and log on as user DBADMIN.
2
3
From the main screen, click the Preview Mode button generated SQL.
Grant privileges to each role as follows:
a
Click Tools>Grant/Revoke>Object Rights.
b
to allow future display of
In the Grant/Revoke Objects dialog, specify values for the following:
Fields and Controls Explanation
Database Name Required.
•
To grant privileges on an entire database, specify the name of the database.
•
To grant privileges on an object, specify the parent database that contains the object.
Recommendation: When specifying privileges for:
•
General users, select a read-only Views database.
•
Update users, select a read-only Views database or an updatable Views database.
•
Batch users, select the Tables_Database.
•
Administrative users, select DBADMIN.
•
Database programmers, select a Development database.
Object Type Required.
Specify the type of database object on which privileges are being granted.
For example:
•
To grant privileges on the Tables_Database, specify Database.
•
To grant privileges on the tables located in the Tables_Database, specify Table.
Recommendation: Always grant privileges at the highest level that is appropriate, while observing security concerns and user “need to know.”
Database Administration 121
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Using Roles to Manage User Privileges
122
Fields and Controls Explanation
Objects Required when objects are displayed.
If the Object Type is Database, this field is blank and not required.
Otherwise, the Objects field displays all of the database objects of the object type contained in the named database.
For example, if Database Name is Tables_Database and Object Type is
Table, this field displays all of the tables contained in the
Tables_Database.
Select the specific database object on which privileges are being granted.
To/From User
Role
Not applicable for granting privileges to roles.
Required. The role to which the specified privileges are granted.
Select the name of a previously created role from the displayed list.
Public
Normal
Create
Drop
All
Optional.
Checking the Public box grants the specified privileges to PUBLIC, a group of rights accessible by any user with logon privileges, even if no other privileges have been granted.
Recommendation: Do not grant additional privileges to PUBLIC at this time.
Required. Specify the privileges by checking the boxes, based on the guidelines in
“User Types and Minimum Required Privileges” on page 118 .
Recommendation: Consider the privileges that the user may get without
formal granting, as shown in “Types of Privileges” on page 119
, before deciding on what privileges to grant.
Optional. Do not use for roles.
Grants all privileges that apply on the database object. This box is usually used only for the master administrator, DBADMIN.
Dictionary
Access
All But
Optional. Grants administrative privileges on data dictionary tables.
Optional.
Provides basic SELECT and EXECUTE privileges on the specified object.
Optional.
When the All But box is checked, role members receive all privileges except those checked in the Normal, Create, and Drop sections.
c
Not all privileges need to be granted using roles. It may be more efficient to grant some specialized privileges directly to a user, rather than creating a role with a membership of one.
Note: Any user can grant privileges on any database object it owns to any other user or role, excluding the CREATE and DROP privileges, which are not ownership privileges.
Click the Grant button. The Query Window displays the generated GRANT statement, for example:
Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Using Roles to Manage User Privileges
GRANT EXECUTE, SELECT, INSERT, UPDATE, DELETE, DUMP, RESTORE,
CHECKPOINT, CREATE TABLE, DROP TABLE ON "Tables_Database" TO
"Role_BatchUser_2"
4
5 d
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
e
Close the Query Window and the Grant/Revoke Objects dialog box.
Repeat this procedure to grant privileges to all roles.
Close Teradata Administrator or go to
“Granting User Membership in a Role.”
Reference Information
For Step...
Information on...
3 syntax and options for the GRANT
(SQL Form) statement using roles to manage user access
Is available in...
SQL Data Control Language
Security Administration
Granting User Membership in a Role
After creating database users, grant them membership in one or more roles to provide the required database access privileges.
1
Start Teradata Administrator and log on as user DBADMIN.
2
5
6
3
4
7
On the main screen, click the Preview Mode button generated SQL.
to allow future display of
Click Tools>Create>Role.
In the Roles dialog, select a role from the Role Name drop-down menu.
Click the Users tab.
Select all users that will be members of the selected role, and click Add.
The Query Window will display a GRANT statement similar to:
Grant "Role_1" To "GenUser1";
8
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
9
10
11
Close the Query Window and the Roles dialog box.
Repeat this procedure for each role to which a user requires membership.
Log off Teradata Administrator or go to “Defining the Default Role for a User.”
Database Administration 123
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Using Roles to Manage User Privileges
Reference Information
For Step...
Information on...
1 syntax and options for the
CREATE ROLE statement
2 using roles to manage user access syntax and options for the GRANT
(SQL form) statement
3 database privileges, including those conferred by object ownership syntax and options for GRANT
(Role form)
Is available in...
SQL Data Definition Language Syntax and
Examples
Security Administration
SQL Data Control Language
Defining the Default Role for a User
To ensure that role privileges are available to users at log on, you must define a default role for each user that is a member of one or more roles. If you do not specify a default role, the user must employ the SET ROLE statement to select the operant role. For example:
SET ROLE role_name; or
SET ROLE ALL;
You can also use the SET ROLE statement to change from one role to another role within a session.
To ensure that the user is operating with the appropriate role, specify the default role for a user as follows:
1
Start Teradata Administrator and log on as user DBADMIN.
2
5
6
3
4
7
On the main screen, click the Preview Mode button generated SQL.
Click the + next to DBADMIN to display the list of users.
Select the user you want to modify.
to allow future display of
Click Tools>Modify User.
In the Modify User dialog, click the Default Role drop-down menu and select a default role.
You can optionally select All Roles, which will allow access to all roles of which the user is a member.
Click the Modify button. The Query Window displays the generated MODIFY USER statement, for example:
MODIFY USER "GenUser1" AS
STARTUP = ''
124 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Granting Privileges Directly to Users
DEFAULT DATABASE = "Personnel_Views"
NO FALLBACK
NO BEFORE JOURNAL
NO AFTER JOURNAL
COLLATION = HOST
DEFAULT CHARACTER SET LATIN
DATEFORM=INTEGERDATE
TIME ZONE=NULL
DEFAULT ROLE="Role_GenUser_2"
8
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
9
10
11
Close the Query Window and the Modify User dialog box.
Repeat this procedure to specify the default role for each user who is a member of a role.
Log off Teradata Administrator or go on to “Granting Privileges Directly to Users.”
Reference Information
For Step...
Information on...
1 syntax and options for the SET ROLE statement
7 syntax and options for the MODIFY
USER statement, including the default role using roles to manage user access
Is available in...
SQL Data Definition Language Syntax and
Examples
SQL Data Definition Language Syntax and
Examples
Security Administration
Granting Privileges Directly to Users
Some privileges cannot or should not be granted to roles, but should instead be granted directly to users.
•
System-level privileges that cannot be granted to a role, as shown in
Privileges to User DBADMIN” on page 35 .
Note: System-level privileges are normally used by administrators and should not be granted to other users.
•
Object-level privileges that apply to too few users to require creation of a role.
Note: A user automatically has the privilege to grant most privileges on any database object the user owns to any other user or role. This excludes CREATE and DROP privileges, which are not automatic ownership privileges. The CREATE and DROP privileges must be granted directly to a user, and must include the WITH GRANT OPTION clause, before that user can grant these privileges to others. For information on ownership privileges, see Security
Administration.
Database Administration 125
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Granting Privileges Directly to Users
Prerequisites
You can only grant privileges to users that already exist in the database. For information on creating users, see
“Working with Database Users” on page 113 .
The Sys_Calendar database and Calendar view must exist. They are normally created as part of system installation by running the DIPALL script.
Example: Granting Privileges on Sys_Calendar to All Users
Sys_Calendar is a system database used to contain the Sys_Calendar.CalDates table and
Sys_Calendar.Calendar view. The Calendar view is a user-accessible tool for date arithmetic. It permits easy specification of arithmetic expressions and aggregation, and is particularly useful when requesting values aggregated by weeks, months, year-to-date, years, and so on. You can use the Calendar view to get attributes for any date between the years 1900 and 2100, such as which day of the week or which week of the month a date occurs.
1
2
Use the following procedure to grant SELECT privilege on Sys_Calendar to all Teradata
Database users.
Start Teradata SQL Assistant and log on as DBADMIN.
Enter the following statement in the main query window:
GRANT SELECT ON Sys_Calendar TO PUBLIC
3
4
Click the Execute button
Close Teradata SQL Assistant.
to run the query.
Reference Information
For Step...
Information on...
All Sys_Calendar and the Calendar view
Is available in...
Data Dictionary
Granting Privileges to a User
1
Start Teradata Administrator and log on as DBADMIN.
2
3
4
From the main screen, click the Preview Mode button generated SQL.
Click Tools>Grant/Revoke>Object Rights.
In the Grant/Revoke Objects dialog, specify the following:
to allow future display of
•
Use the Database Name, Object Type, and Objects fields to identify the database object which the user will receive privileges on.
•
In the To/From User tab, select the user who will receive the specified privileges.
•
In the Normal, Create, or Drop sections, check the privileges the user will receive.
126 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Granting Privileges Directly to Users
5
For details, see “Granting Database Privileges to User DBADMIN” on page 35
.
Click the Grant button. The Query Window displays the generated GRANT statement similar to:
GRANT privilege
ON database_object_name
TO username; where
•
The ON clause is based on the values of the Database Name, Object Type, and Objects fields.
•
The TO clause is based on the user selected from the To/From User tab.
6
7
8
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
Close the Query Window and the Grant/Revoke Objects dialog box.
Log off Teradata Administrator.
Automatic Privileges for Creators
When you create a user or database, the system automatically grants you privileges on the created database or user:
•
ANY
•
CHECKPOINT
•
CREATE AUTHORIZATION
•
CREATE DATABASE
•
CREATE MACRO
•
CREATE TABLE
•
CREATE TRIGGER
•
CREATE USER
•
CREATE VIEW
•
DELETE
•
DROP AUTHORIZATION
•
DROP DATABASE
•
DROP FUNCTION
•
DROP MACRO
•
DROP PROCEDURE
•
DROP TABLE
•
DROP TRIGGER
•
DROP USER
•
DROP VIEW
•
DUMP
Database Administration 127
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Granting Privileges Directly to Users
•
EXECUTE
•
INSERT
•
SELECT
•
STATISTICS (also for creators of tables)
•
RESTORE
•
UPDATE
Automatic Privileges for Created Users and Databases
When you create a user or database, it automatically gets the following privileges on itself:
•
ANY
•
CHECKPOINT
•
CREATE AUTHORIZATION
•
CREATE MACRO
•
CREATE TABLE
•
CREATE TRIGGER
•
CREATE VIEW
•
DELETE
•
DROP AUTHORIZATION
•
DROP FUNCTION
•
DROP MACRO
•
DROP PROCEDURE
•
DROP TABLE
•
DROP TRIGGER
•
DROP VIEW
•
DUMP
•
EXECUTE
•
INSERT
•
SELECT
•
STATISTICS
•
RESTORE
•
UPDATE
Note: These same privileges are automatically given to creators, with the exception of
CREATE USER, CREATE DATABASE, DROP USER, and DROP DATABASE.
128 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Granting Privileges Directly to Users
Reference Information
For Step...
Information on...
5 syntax and options for the GRANT
(SQL form) statement database privileges, including those conferred by object ownership
Is available in...
SQL Data Control Language
Security Administration
Working with Table Access Privileges for Views
You must grant table access privileges to all views databases to allow the views within each database to access tables in the Tables database.
Note: When table access privileges are granted to a Views database, the individual views inherit the right to access tables in the Tables database from the parent Views database.
Granting Table Access to Each Views Database
1
Start Teradata Administrator and log on as user DBADMIN.
2
3
4 f
On the main screen, click the Preview Mode button
SQL.
Click Tools>Grant/Revoke>Object Rights.
In the Grant/Revoke Objects dialog:
a
to allow display of generated
From the Database Name drop-down menu, select the Tables database. This is the object the privileges are granted on.
b
From the Object Type drop-down menu, select Database.
c
Click the To/From User tab and select a Views database. The privileges are granted to this object.
d
In the Normal field, check the following to select the privileges being granted:
•
For read-only views: Check Select.
•
For updatable views: Check Select, Insert, Update, and Delete.
e
Check the Grant box in the lower right-hand corner of the screen to give the database the privilege of extending the defined privileges to other databases. This specifies the
WITH GRANT OPTION clause.
Click the Grant button. The Query Window displays the generated GRANT statement, for example:
•
For read-only Views databases:
GRANT SELECT ON "Tables_Database" TO "database_name" WITH GRANT
OPTION
•
For the updatable Views databases:
Database Administration 129
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
About Viewpoint Monitoring Privileges
GRANT SELECT, INSERT, UPDATE, DELETE ON "Tables_Database" TO
"database_name" WITH GRANT OPTION
5 g
In the Query Window, click the Run Query button . A Query Complete message appears in the bottom left corner of the main window when the query executes.
h
Close the Query Window and the Grant/Revoke Objects dialog box.
Log off Teradata Administrator.
Reference Information
For Step...
Information on...
4 syntax and options for the GRANT
(SQL form) statement database privileges, including those conferred by object ownership
Is available in...
SQL Data Control Language
Security Administration
About Viewpoint Monitoring Privileges
All database users can benefit from using Teradata Viewpoint to monitor the database, but not all users need all Teradata Viewpoint capabilities.
You can use the examples in the following table to help determine the database monitoring functions needed by various user types, and in creating and assigning privileges to Teradata
Viewpoint roles.
Role Privileges
Default Teradata Viewpoint roles
Administrator Assign this role only to the Teradata Viewpoint administrator(s).
Has all privileges (default). This role can access the functions in the Admin tool menu to configure Teradata Viewpoint alerts, users, roles, and systems.
User
TD_SE (Teradata
Solution Engineer)
TD_TSS (Teradata
Technical Support
Specialist)
Includes minimal privileges
This role is intended for the Teradata Solution Engineer when they need access to system health and metrics. You can enable or disable this role, but do not reconfigure it.
This role is intended for Teradata Technical Support Specialists when they need access to your system SQL. You enable or disable this role, but do not reconfigure it.
Example custom roles
Note: Roles are based on the user types/duties shown in
.
130 Database Administration
Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
About Viewpoint Monitoring Privileges
Role
Database
Administrator
Database
Programmer/
Power User
Batch User
Update User
Privileges
Database administrators may require all Teradata Viewpoint monitoring functions to perform administrative duties in the database.
Suggested role privileges: Create a Database Administrator role that has privileges to use all portlets and portlet functions, except System
Administration, which should only be accessible to the Teradata Viewpoint administrator(s).
Note: You may find it useful to create an additional role that limits access to some portlets and functions for lower level administrators.
Database programmers perform functions similar to administrators, creating database objects, running ad hoc queries, and monitoring performance.
Suggested role privileges: Similar to Database Administrator, but without analytical functions.
Note: Programmer privileges in the database are often restricted to a development database, but Teradata Viewpoint monitors all databases within the system.
Batch users run load and export utilities, or who perform backup, archive and restore operations.
Suggested role privileges include:
•
For job planning:
•
Capacity Heat Map
•
Space Usage
•
Calendar
•
During job execution and troubleshooting:
•
System Health
•
My Queries or Query Monitor
•
Workload Monitor
•
SQL Scratch Pad
•
Console Utilities
•
Privilege to set preferences and share portlets
Users who perform miscellaneous insert/update/delete operations.
Suggested role privileges:
•
Calendar (no create or edit privileges)
•
System Health (View Summary only)
•
My Queries
•
Workload Monitor
•
SQL Scratchpad
131
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Setting Up Teradata Viewpoint Users
Role
General User
Privileges
Low level users that run macros and stored procedures to access data, which select data and generate reports.
Suggested role privileges:
•
Calendar (no create or edit privileges)
•
System Health (View Summary only)
•
My Queries
Setting Up Teradata Viewpoint Users
For users that are not already set up to access Teradata Viewpoint, as part of initial system installation, you must perform Teradata Viewpoint set up operations.
1
2
3
4
Log on as the Teradata Viewpoint administrator and select Admin, which includes a drop down list of administrative functions.
Use the Roles Manager to create Teradata Viewpoint roles and define sets of user privileges for each role.
When creating roles:
•
Define the portlets available to the role
•
Define whether user members can set their own preferences and share portlets.
•
Give or remove user permissions for portlet tools and functions.
Teradata Viewpoint provides two default roles, but you may find it useful to configure custom roles, based on the suggestions in
“About Viewpoint Monitoring Privileges” on page 130 .
Use the Users Manager to create Teradata Viewpoint users and assign membership in one or more roles according to user job function. Include an email address for each user, so that Teradata Viewpoint can send them alert emails, if necessary.
Instruct users to setup their Teradata Viewpoint profile by specifying their username, password, and account, so the system can verify that each user exists in the specified database system and that the account is valid for the user. Profile information is used by some portlets, for example:
•
My Queries portlet, to track query activity for the user
•
SQL Scratchpad portlet, to determine database access privileges for the user
Reference Information
Information on...
the duties typically performed by the user types for which customer roles are recommended
Is available in...
132 Database Administration
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Setting Up Teradata Viewpoint Users
Information on...
Is available in...
creating Teradata Viewpoint users, roles, and profiles
•
Viewpoint application Help screens
•
Teradata Viewpoint User Guide
Configuring User Dashboards
You can create custom dashboard pages, and then use the Add Content function to populate each page with a set of related portlets from among those available to the user.
The primary dashboard should contain the most important portlets for live monitoring of database activity, for example:
•
System Health (all)
•
My Queries (users) or Query Monitor (administrators and programmers)
•
1 or 2 other optional portlets, depending on the type of user
You can create additional dashboards to contain other portlets.
Note: You may need to limit the number of portlets per dashboard page. Displaying too many
Teradata Viewpoint portlets on one dashboard may make it hard to find needed data, and for some browsers, may slow portlet loading and response times.
For details about configuring dashboards and portlets, see Teradata Viewpoint User Guide.
Creating a Dashboard
From the Teradata Viewpoint portal:
1
2
Use the Add Page option to name the dashboard, for example, MainDashboard.
Use the Add Content function to select portlets to include in the dashboard.
Database Administration 133
Chapter 6: Working with Users, Roles, and Profiles: Operational DBAs
Setting Up Teradata Viewpoint Users
134 Database Administration
CHAPTER 7
Loading and Exporting Data:
Application DBAs
This chapter provides an overview of the different methods used to load data into or export data from Teradata Database. The chapter also provides guidelines on choosing the best utility for your purpose and examples of utility job performance analysis and capacity planning.
About Teradata Parallel Transporter
Teradata Parallel Transporter (PT) is an object-oriented client application that provides scalable, high-speed, and parallel data:
•
Extraction
•
Loading
•
Updating
These capabilities can be extended with customizations or third-party products.
Teradata PT uses but expands on the functionality of the traditional Teradata extract and load utilities: FastLoad, MultiLoad, FastExport, and TPump, also known as standalone utilities.
Teradata PT can load data into and export data from any accessible database object in Teradata
Database or other data store using Teradata PT operators or access modules.
About the Load Operator
The Load operator uses the Teradata FastLoad protocol to load a large volume of data at high speed into an empty table in Teradata Database.
The Load operator is typically used for the initial loading of Teradata tables. It inserts data from data streams into individual rows of a target table.
About the Update Operator
The Update operator emulates the Teradata MultiLoad utility to perform highly scalable and parallel inserts, updates, deletes, and upserts into new or pre-existing Teradata Database tables in a single pass.
The Update operator can also be used to insert new rows into the database, but it cannot perform select operations. The Update operator is not supported for column-partitioned tables.
Database Administration 135
Chapter 7: Loading and Exporting Data: Application DBAs
About Basic Teradata Query for Loading Tables
About the Export Operator
The Export operator uses the Teradata FastExport protocol to extract large amounts of data at high speed from Teradata Database using block transfers over multiple sessions. The Export operator reads data from one or more tables in Teradata Database.
About Basic Teradata Query for Loading Tables
Differences Between BTEQ and Teradata PT
BTEQ and Teradata PT can both load data into Teradata and can update tables, among other functions. Other similarities include:
•
Capacity: Both can send up to 64 KB per request and up to 1 MB per request with array support.
•
Capabilities: Both can support multiple sessions and can load, export, process DDL, and act as a command line interface.
The following table explains the major differences between BTEQ and Teradata PT:
BTEQ
Consumes less client CPU.
Loads or updates either all records or stops after x records that you specify with the REPEAT command.
Limited error handling.
Teradata PT
Consumes more client CPU.
Can also load or update only selected rows.
No checkpoint restart.
No SERIALIZE option.
Does not process multiple DML statements in a single pass of the input data.
Performs error handling and creates error log tables.
Checkpoint restart.
SERIALIZE option ensures the correct sequencing of transactions.
Can perform conditional apply or multiple apply.
Recommendation: If your job has millions of rows, use Teradata PT. If not, especially for a load job that will be used again in the future, consider both. See which performs better and causes less system overhead. BTEQ can be a good choice for smaller amounts of data, exploratory work, and DDL.
Updating Tables
Frequently, when updating the Teradata Database, you make the same change to many rows. It is helpful to eliminate the repetitive work required to update them individually. The REPEAT command serves this purpose.
136 Database Administration
Chapter 7: Loading and Exporting Data: Application DBAs
About Basic Teradata Query for Loading Tables
Updating Rows Using a Single Request
If you can describe all of the rows by some formula, then you can use a single request to update them all. The following request, for example, updates the salaries of all employees in department 600 of the database Personnel: update personnel.employee set salary=salary*1.07
where deptno=600;
Updating Rows Using Multiple Requests
If you cannot describe all of the rows by a formula, you must describe them individually. The following requests, for example, update the salaries of two employees: update personnel.employee set salary=salary*1.07
where empno = 10006; update personnel.employee set salary=salary*1.07
where empno = 10013;
This approach is convenient for a few updates, but if there were hundreds, the script would become very long. It would also be very inflexible; you would have to edit each entry to use the script again for another list of employees.
Updating Rows by Importing a File
A better approach would be to create a separate file (for example, RAISEEMP) containing two records in data format representing the employee numbers 10006 and 10013. Then use the
BTEQ IMPORT command with a REPEAT command to update the rows in the database.
Using the REPEAT Command When Importing a File
The REPEAT command appears before the request and specifies the total number of requests to be submitted.
The following BTEQ script using the BTEQ REPEAT command opens the character-format file RAISEEMP and repeats the update sequence twice:
.IMPORT data file=raiseemp
.REPEAT 2 using enumb (char(5)) update personnel.employee set salary=salary*1.07
where empno = :enumb;
For each employee, BTEQ reads a value from the RAISEEMP file for the variable called enumb and carries out the UPDATE for the row whose EmpNo equals the value of enumb.
Note: When using a file originally exported with the BTEQ EXPORT command as the source for an IMPORT command across a different platform type, ensure that the endianness type of both platforms is the same. You can verify this from the “Client Platform Byte Order” tab in the output of the SHOW CONTROLS command.
Loading Data into Teradata Database
BTEQ Input Stream
Use the following BTEQ input stream to load (import) data into the Teradata Database:
Database Administration 137
Chapter 7: Loading and Exporting Data: Application DBAs
About Reading Committed Data While Loading to the Same Table
.IMPORT DATA FILE=DeptData.imp
.REPEAT *
USING DeptNo (SMALLINT),
DeptName (VARCHAR(14)),
Loc (CHAR(3)),
MgrNo (SMALLINT)
INSERT INTO Personnel.newtable (DeptNo, DeptName,
Loc,MgrNo)
VALUES (:DeptNo, :DeptName, :Loc, :MgrNo); where:
•
The REPEAT * command repeats the next request until the data in the import file is exhausted.
•
The term newtable refers to an empty table to receive the data being loaded.
Reference Information
Information on...
BTEQ commands, including
PACK running batch jobs to submit
BTEQ commands and
Teradata SQL
Is available in...
Basic Teradata Query Reference
Basic Teradata Query Reference
Restarts and Aborts on BTEQ Jobs with Identity Column
If a restart occurs during a BTEQ import, BTEQ will resubmit the last uncompleted row insert after the system recovers from the restart. Identity column numbering will continue from there. It is possible, however, for a restart to cause duplicates because BTEQ may do a reinsert even if the previous insert has completed. Duplicates will not be detected if the target table is
MULTISET and not defined with a UPI.
If a session abort occurs during a mainframe-attached BTEQ import, the last uncompleted row insert will not be resubmitted and associated data may be lost. Associated data may also be lost if a workstation-attached BTEQ import session is aborted and there is no other session through which to resubmit the insert.
In both cases, manually restarting the import can result in duplicate rows if rows newly inserted before the session abort are not deleted unless the target table is defined with a UPI.
About Reading Committed Data While Loading to the Same Table
Users can concurrently read committed data in a table while the table is being loaded when all of these conditions are met:
138 Database Administration
Chapter 7: Loading and Exporting Data: Application DBAs
About Reading Committed Data While Loading to the Same Table
•
The table must be a load-isolated table.
You identify a table as load isolated using the ISOLATED LOADING option with CREATE
TABLE or ALTER TABLE requests. For example:
CREATE TABLE ldi_table01, WITH CONCURRENT ISOLATED LOADING FOR ALL
(col_a INT, col_b VARCHAR(32));
•
The load is performed using isolated loading.
As described later in this section, isolated loading can occur even when you don’t explicitly specify ISOLATED LOADING. If you want to ensure isolated loading use the ISOLATED
LOADING option with your SQL request (INSERT, UPDATE, MERGE, and DELETE).
For example:
INSERT WITH CONCURRENT ISOLATED LOADING INTO ldi_table01 SELECT * FROM src_Tab;
•
The SELECT request must use the LOAD COMMITTED locking option to read committed data from the table. For example:
LOCKING TABLE ldi_table01 FOR LOAD COMMITTED SELECT * FROM
ldi_table01;
You can use Teradata PT with the SQL Inserter and SQL Selector operators for load isolation because these operators perform SQL requests. (The Load and Update operators perform SQL requests when the MLOADX protocol is used; however, Teradata Database determines the protocol that is used for each job. If the Load or Update operator uses the MULTILOAD protocol, then concurrent reads will not be allowed.)
You can determine whether the ISOLATED LOADING option is being used for a load by using the DBQL step level logging facility. See DMLLoadID in
“Query Step Information Log Table:
.
If a load is performed on a load-isolated table using a SQL request without the ISOLATED
LOADING option then the default is to:
•
Allow concurrent reads on committed rows while the table is being all-AMP updated
•
Disallow concurrent reads while the table is modified with less than an all-AMP operation
Note: All SQL requests in a transaction must use the same type of concurrency as the first request in the transaction or the transaction will fail.
For details on CREATE TABLE and ALTER TABLE syntax, see SQL Data Definition Language
Syntax and Examples. For details on the SELECT, INSERT, UPDATE, MERGE, DELETE, and
LOCKING request modifier syntax, see SQL Data Manipulation Language.
For information on locking and load isolation, see SQL Request and Transaction Processing.
About Cleaning up Load Isolated Tables
Load Isolation uses row versioning to ensure that only the most recently committed data is retrieved. Logically deleted rows are created as an artifact of row versioning and, as a result, a load-isolated table typically grows faster than a table without load isolation. A DBA must physically delete the logically deleted rows at regular intervals to reclaim disk space and improve query performance.
Database Administration 139
Chapter 7: Loading and Exporting Data: Application DBAs
About FastLoad
Larger tables with more concurrent reads during updates will require more frequent cleanup.
When you collect summary statistics on your load-isolated tables, monitor the logically deleted rows (DelRowCount) as a percentage of physical rows (PhyRowCount). Perform cleanup when the table contains a significant percentage of logically deleted rows or you notice degraded query performance on the table while modifications occur.
To clean up logically deleted rows, you can use either the RELEASE DELETED ROWS option of an ALTER TABLE request or issue a stored procedure to perform cleanup on cascaded, logically deleted rows in load-isolated tables and underlying join indexes and secondary indexes.
Alter Table Cleanup Method
Use the RELEASE DELETED ROWS option in an ALTER TABLE request to interactively clean up logically deleted rows from a load-isolated table. For example:
ALTER TABLE Employee RELEASE DELETED ROWS;
For details on the syntax, see SQL Data Definition Language Syntax and Examples.
Stored Procedure Cleanup Method
Use a stored procedure to clean up logically deleted rows from multiple load-isolated tables and their underlying join indexes and secondary indexes. Use LDI_Clean_ANSI in ANSI mode and LDI_Clean in Teradata mode. The input and output is the same for both ANSI and
Teradata mode. (These stored procedures are created typically during Teradata Database installation but can be added later using the DIPDEM or DIPPOST scripts.)
The following example uses LDI_Clean in Teradata mode: call ldi_clean('database_name', 'table_name','Reset_flag_value'); where Reset_flag_value is either ‘Y’ or ‘N’ to specify whether you want to reset the incremental value of the current load to 0. For example: call ldi_clean('finance', 'acct_payable','y');
Teradata responds with the following output:
acct_payable altered, finance altered
Keep in mind the following points:
•
An exclusive lock is required while performing cleanup on a load-isolated table.
•
The DROP TABLE privilege is required to perform the cleanup.
About FastLoad
FastLoad allows you to load data from the client to your Teradata Database system. It allows fast loading of a single empty table that has no SIs, JIs, HIs, RI (FKs), or column partitioning.
FastLoad loads large amounts of data into an empty table, with or without a primary index, on Teradata Database. However, row size and the number of columns, more than any other factors, do affect performance.
140 Database Administration
Chapter 7: Loading and Exporting Data: Application DBAs
About FastExport
When you load large tables, consider the following things.
IF you are loading a large table…
with fallback without fallback
THEN…
•
Observe the suggestions for loading the large table without fallback.
•
Create the table without fallback protection.
•
Load the table.
•
Use BTEQ to alter the table to have fallback protection.
•
Initially, set session to the number of AMPs in your system. Then, experiment by reducing the number of sessions.
•
Avoid checkpointing too frequently.
•
Avoid NULLIF clauses, VAR fields, or indicator bits whenever possible.
•
Run concurrent FastLoad jobs. The maximum number of FastLoad jobs you can run is 30. The default is 5. (See Utilities for more information.)
You can also change the maximum number of jobs allowed by defining throttle rules through the Teradata Viewpoint Workload Designer portlet. For more information on FastLoads, see
Concurrent FastLoad, MultiLoad, and FastExport Jobs” on page 143
.
FastLoad does not support the following:
•
HIs
•
JIs
•
FKs
•
SIs (Both USIs and NUSIs are not supported)
•
LOBs
•
Column partitioning
For more information on this utility, see Teradata FastLoad Reference.
About FastExport
FastExport quickly exports data from a Teradata Database system to the client platform. This utility can format and export very large amounts of data very quickly. The maximum number of FastExport jobs you can run is 60. The default is 5. You can also change the maximum number of jobs allowed by defining throttle rules through the Teradata Viewpoint Workload
Designer portlet. (See “Maximum Number of Concurrent FastLoad, MultiLoad, and
FastExport Jobs” on page 143 for more information.)
To improve performance:
•
Do not use too many sessions.
•
Avoid using any option that causes the evaluation of individual fields within a layout, including NULLIF clauses and APPLY WHERE conditions.
Database Administration 141
Chapter 7: Loading and Exporting Data: Application DBAs
About MultiLoad
The following restrictions and limitations apply:
•
Exponential operators
•
Concatenated fields
•
Hexadecimal forms
For more information, see Teradata FastExport Reference.
About MultiLoad
MultiLoad allows you to upload data from the client to your Teradata Database. It operates on multiple tables simultaneously and can also insert, update, and delete data.
The maximum number of MultiLoad jobs you can run is 30. The default is 5. You can also change the maximum number of jobs allowed by defining throttle rules through the Teradata
Viewpoint Workload Designer portlet. (See “Maximum Number of Concurrent FastLoad,
MultiLoad, and FastExport Jobs” on page 143
for more information.)
To improve performance:
•
Minimize concatenation and redefinition of input data.
•
Restrict the number of NUSIs.
•
Make the PI of each data table unique or nearly unique.
•
Minimize the use of error tables.
•
Do not checkpoint too often.
•
Avoid using too many sessions.
MultiLoad does not support the following:
•
Aggregate operators
•
Exponential operators
•
Arithmetic functions
•
Concatenation of data files
•
Hexadecimal forms
•
NoPI tables
•
Foreign keys
•
Loading LOBs
•
Column-partitioned tables
For more information, see Teradata MultiLoad Reference.
You can perform the following steps as an alternative to using MultiLoad:
1
2
Create a staging table that is otherwise identical with the target table, but has no constraints, triggers, or indexes defined on it, such as a NoPI.
Import into the staging table from the external source (for example, using Fastload or
Teradata Parallel Data Pump Array INSERT operations).
142 Database Administration
Chapter 7: Loading and Exporting Data: Application DBAs
Maximum Number of Concurrent FastLoad, MultiLoad, and FastExport Jobs
3
4
5
(Optional) create an error table for the target table if error logging is desired.
Execute a MERGE into the target table using the staging table as source.
Drop the staging table and error table.
For information on NoPI tables or Teradata Parallel Data Pump Array INSERT operations, see
Database Design.
Maximum Number of Concurrent FastLoad,
MultiLoad, and FastExport Jobs
If you do not use the Teradata Viewpoint Workload Designer portlet to throttle concurrent load and unload jobs, the MaxLoadTasks and the MaxLoadAWT fields of the DBSControl determine the combined number of FastLoad, MultiLoad, and FastExport jobs that the system allows to run concurrently. The default is 5 concurrent jobs.
If you have the System Throttle (Category 2) rule enabled, even if there is no Utility Throttle defined, the maximum number of jobs is controlled by the Teradata dynamic workload management software and the value in MaxLoadTasks field is ignored.
For more information, see “MaxLoadTasks” and “MaxLoadAWT” in the chapter on DBS
Control in Utilities.
Utility Management
To manage load utilities (such as, FastLoad, MultiLoad, and FastExport) and the Teradata
ARC utility, you can classify them into a workload definition based on a combination of the following criteria:
•
Utility Type
•
“Who” criteria (such as, account name, user name, query band, and so on)
•
Database and table or view name (except for archive and restore jobs)
You can use the following utility-related Teradata Active System Management (TASM) rules to define how Teradata Database manages its workloads:
•
Utility Throttle (also called Utility Limit in the Teradata Viewpoint Workload Designer portlet). This rule enforces system wide limits on each type of the Teradata utilities, such as FastLoad, MultiLoad, FastExport, Teradata ARC, Teradata Parallel Transporter Load operator, JDBC FastLoad, and so on.
Database Administration 143
Chapter 7: Loading and Exporting Data: Application DBAs
Teradata Parallel Data Pump
•
Utility Workload Throttle. This rule enforces the limits based on the following criteria:
•
Utility Type
•
“Who” criteria (such as account name, user name, query band, and so on)
•
Database and table or view name (not available for archive or restore jobs)
To define a Utility Workload Throttle, create a workload definition with a throttle limit and the above qualification criteria (or a combination).
•
AWT Resource Limit. This rule specifies an AMP Worker Task (AWT) limit for utilities.
The rule enforces the limit based on the utility type (except the Archive and Recovery utility), request source (for example, username and account), and query band. The maximum value for the AWT Resource Limit rule is 70% of the value of the DBS Control utility field MaxAMPWorkerTasks.
•
Utility Session. This rule specifies how many utility sessions can be used for the various kinds of utilities. The supported qualification criteria are:
•
Utility Type
•
Account name, user name, query band, and so on
When the Teradata dynamic workload management software is enabled and there is no applicable user-defined Utility Session rule, the Teradata dynamic workload management software automatically uses the default session rules to select the number of sessions based on the utility type, system configuration, and optional data size.
You cannot delete default session rules. However, you can modify the utility session information (that is, the default numbers assigned to various utility kinds).
You can manage your load utilities, DSA, and the Teradata ARC utility by using Teradata
Viewpoint.
Teradata Parallel Data Pump
The Teradata Parallel Data Pump (TPump) utility allows real time updates from transactional systems into the data warehouse. TPump executes INSERT, UPDATE and DELETE requests, or a combination, to more than 60 tables at a time from the same source feed.
TPump is an alternative to MultiLoad. The benefits of TPump include:
•
Real time INSERTs and UPDATEs to more than 60 tables simultaneously
•
Low volume batch maintenance
•
Can provide continuous feed to the warehouse
The data handling functionality of TPump is enhanced by the TPump Support Environment.
In addition to coordinating activities involved in TPump tasks, it provides facilities for managing file acquisition, conditional processing, and certain Data Manipulation Language
(DML) and Data Definition Language (DDL) activities, on Teradata Database.
TPump has the following restrictions:
•
It does not support aggregate operators or concatenation of data files
144 Database Administration
Chapter 7: Loading and Exporting Data: Application DBAs
Loading Geospatial Data
•
TPump performance is severely affected by access or DBQL logging.
The TPump Support Environment enables use of variable substitution, conditional execution based on the value of return codes and variables, expression evaluation, character set selection options and more. For more information, see Teradata Parallel Data Pump Reference.
Restarts on TPump Jobs with Identity Column
TPump works on multistatement SQL requests. Each request has a specific number of statements depending on the PACK specification in the BEGIN LOAD command.
In ROBUST mode, each request is written into a restart log table.
If a restart occurs, TPump will query the restart log table and re-execute requests that are not logged. This means it may be possible for a restart to generate duplicates if an insert request is repeated. Duplicates will not be detected if the target table is not defined with a UPI.
TPump will flag an error if it is run in simple mode and the target table has an identity column
PI. This is because no restart log is used for restart recovery and duplicate rows could result if some requests are reprocessed.
For more information on this utility, see Teradata Parallel Data Pump Reference.
Loading Geospatial Data
The TDGeoImport data load program can convert geospatial data into a format that can be used by the Teradata geospatial UDTs.
TDGeoImport converts layers from ESRI, MapInfo, and TIGER/Line data sources and loads them directly into the database. This program can be used with geometries that have web representations of any size up to a maximum of 16MB.
Interpreting LOAD Utility Status
When observing the progress of load, archive, export, and other jobs, remember the following:
•
The system uses LSN to calculate the impact of all sessions.
•
All work the system performs during a session is charged to that session.
Because not all sessions are used during an operation, some sessions with the same LSN and user name may not report any activity.
The Query Session utility (QRYSESSN) can provide detailed status of each session involved in the load task.
Database Administration 145
Chapter 7: Loading and Exporting Data: Application DBAs
Choosing the Best Utility for Your Purpose
Choosing the Best Utility for Your Purpose
Some of the client utilities have similar functions. However, to optimize performance on resource utilization, it is important to choose the best utility for your job. Note that the following section does not include Teradata Parallel Transporter.
Definitions
To better understand the guidelines in this section, refer to the definitions in the following table.
Term
Small table
Large table
Small number of rows
Moderate number of rows
Large number of rows
Meaning
<1 million rows
>100 million rows
<5% of total rows
20 - 30% of total rows
40 - 50% of total rows
Guidelines for Inserting Rows
The following table provides some guidelines you can use to make the best choice of utilities when inserting rows. Keep in mind that you may find a better way through experimenting.
Task
Insert a small number of rows into an empty table
Insert a large number of rows into an empty table
Insert a small number of rows into a small populated table
First Choice
FastLoad
FastLoad
BTEQ, MultiLoad, TPump or
MERGE when the transaction density is too low for satisfactory performance
TPump, BTEQ, or MERGE Insert a small number of rows into a large populated table
Insert a moderate number of rows into a large populated table
Insert a large number of rows into a large populated table
MultiLoad or MERGE
MultiLoad or MERGE
Second Choice
TPump/BTEQ
TPump
TPump when the transaction density is too low for satisfactory performance
1
2
FastLoad rows into an empty table.
INSERT...SELECT rows into an empty new table.
146 Database Administration
Chapter 7: Loading and Exporting Data: Application DBAs
Choosing the Best Utility for Your Purpose
Task
Insert a large number of rows into multiple populated tables
Insert a large number of rows when logged on session are near their full capacity
Insert a large number of rows when the specific order of updates is important
Insert a large number of rows into multiple empty tables
First Choice
MultiLoad
INSERT…SELECT or MERGE
Tpump
Multiple FastLoads
Second Choice
TPump
Tpump
INSERT…SELECT or MERGE
MultiLoad
Guidelines for Deleting Rows
The following table provides some guidelines you can use to make the best choice of utilities when deleting rows. Keep in mind that you may find a better way through experimenting.
Task
Delete a small number of rows from any table
Delete a large number of rows from a large table
Perform DELETE operations on a large number of rows on multiple tables
Delete all rows from a table
First Choice
TPump
MultiLoad
MultiLoad
BTEQ
In this case, Teradata marks the table header with an indicator to say that the table is empty. Assuming there are no locks on the table when this delete is requested, the delete will complete in a second or two.
Second Choice
BTEQ
BTEQ
TPump
Database Administration 147
Chapter 7: Loading and Exporting Data: Application DBAs
Choosing the Best Utility for Your Purpose
Task
Delete some rows from a table
First Choice
BTEQ
Space permitting, it can also be an option to create an empty table, insert into this table the rows that you wish to retain and drop the original table and then rename the new table to the name of the original table.
Second Choice
MultiLoad deletes are good for deleting large volumes of data, but not all data, from a table.
MultiLoad reads the data block by block. It will “touch” each block consumed by the table and remove rows as appropriate, and replace the block.
This can be very efficient and will beat an
SQL delete in many cases as there is no rollback logging. However, MultiLoad must complete or else the table will remain in an unstable state. For MultiLoad to work best, drop SIs, run the MultiLoad delete, and then replace the indexes. This is quicker and ensures that the work table is small.
Note: You cannot delete rows from a table using the SQL MERGE statement.
Guidelines for Other Batch Operations
When updating, exporting, or doing other batch operations, use the guidelines in the following table to consider the best choice for your job.
Task
Merging a large number of rows when logged on session are near their full capacity
Merging a large number of rows when the specific order of updates is important
Export a small number of rows from any table
Export a moderate or large number of rows from a large table
Update a small number of rows in any table
Update a large number of rows in a large table
Perform multiple DML
(INSERT, UPDATE, DELETE) operations on a small number of rows on multiple tables
First Choice Second Choice
INSERT…SELECT or MERGE Tpump
Tpump
BTEQ
FastExport
TPump
MultiLoad
MultiLoad or TPump when the transaction density is too low for satisfactory performance
INSERT…SELECT or MERGE
FastExport
BTEQ
BTEQ
BTEQ insert table into a new one.
BTEQ
148 Database Administration
Chapter 7: Loading and Exporting Data: Application DBAs
Utility Job Performance Analysis and Capacity Planning
Task
Perform multiple DML
(INSERT, UPDATE, DELETE) operations on a large number of rows on multiple tables
First Choice
MultiLoad
Copy one or several tables to a different Teradata Database
Copy all tables in the Teradata
Database system to a different
Teradata Database system
ARC
ARC
Load an identity column table TPump (in ROBUST mode)
Load the data into staging table
Teradata PT Load Operator
Second Choice
TPump
FastExport and FastLoad
BTEQ .IMPORT
Teradata PT Update Operator
Utility Job Performance Analysis and Capacity
Planning
You can query the DBQL utility job log table, DBC.DBQLUtilityTbl, to analyze the performance of load or export utility jobs and do capacity planning. This table is populated if you use the WITH UTILITYINFO option with a BEGIN/REPLACE QUERY LOGGING request. For details on the contents of DBQLUtilityTbl and the corresponding view
QryLogUtility[V], see “Utility Job Log Table: DBQLUtilityTbl” on page 337 and
Utility Job Log Table: QryLogUtility[V]” on page 343
.
Examples of Using the DBQL Utility Job Log Table
You can use DBQLUtilityLogTbl to diagnose utility performance issues and do capacity planning. The following are just some examples of how to use the table.
Investigating Why a Job Ran Slower Today
In this scenario, a FastLoad job submitted today took significantly longer than yesterday. You want to determine the root cause. Although the example is specific to Fast-
Load, you can use this general approach for other utilities.
1
Retrieve the rows for the jobs yesterday and today from DBQLUtilityTbl using the following criteria:
•
Job attributes that uniquely identify this FastLoad job. This criteria returns all instances of this FastLoad job. Possible job attributes include:
•
UtilityName = 'FASTLOAD'
•
Username
•
Query band
•
Utility request
Database Administration 149
Chapter 7: Loading and Exporting Data: Application DBAs
Utility Job Performance Analysis and Capacity Planning
2
3
•
Job end time: either today or yesterday.
Compare various column values between the two rows to determine the possible causes.
Suggestions include:
a
Compare RowsInserted values. If the job today loaded significantly more rows, this could be the cause. If not, proceed to the next step.
b
Compare DelayTime values. An increase indicates the job today was delayed longer by
TASM.
c
Compare the elapsed time of each phase (PhasexEndTime - PhasexStartTime) to identify the phase with a significant increase in elapsed time.
d
If the phase with a significant increase in elapsed time is the Acquisition phase, check for an increase in MaxDataWaitTime, which might be caused by more load on the client machine or slower network response time.
e
Compare the resource usages of the phase with a significant increase in elapsed time to find additional clues. For example:
•
An increase in PhasexMaxCPUTime may indicate data skew.
•
An increase in Phase1 messages (Phase1BlockCount) may indicate that a smaller message size was used.
Examine related query log data in DBQLogTbl. To select the desired data, use:
•
LSN to select all rows of a particular job.
•
PhasexStartTime and PhasexEndTime to select rows of a specific phase.
Performance Anomaly Detection
In this scenario, you want to detect job instances whose elapsed time is 50% longer than the last 30-day average for the same job. This approach can be used to detect anomalies for other metrics, such as CPU time or I/Os.
1
2
Select rows from DBQLUtilityTbl for jobs ending within the last 30 days and store them in a temporary table called Last30dayJobInstances.
CREATE VOLATILE TABLE Last30dayJobInstances AS
(SELECT U.*,
SUBSTRING(UtilityRequest FROM 1 FOR 100) AS UtilReqShort
FROM DBC.DBQLUtilityTbl U
WHERE CAST(JobEndTime AS date) >= DATE-30)
WITH DATA
PRIMARY INDEX (UtilityName, UserName)
ON COMMIT PRESERVE ROWS;
Use the Last30dayJobInstances table to calculate the last 30-day average elapsed time of each job, grouped by attributes such as utility name, user name, and utility request. Store the results in another temporary table called Last30dayAverage. Each row contains the average elapsed time and job attributes.
CREATE VOLATILE TABLE Last30dayAverage AS
(SELECT AVERAGE((JobEndTime - JobStartTime) HOUR TO SECOND)
AS AvgJobElapsedTime,
UtilityName, UserName,
SUBSTRING(UtilityRequest FROM 1 FOR 100) AS UtilReqShort
150 Database Administration
Chapter 7: Loading and Exporting Data: Application DBAs
Utility Job Performance Analysis and Capacity Planning
3
FROM Last30dayJobInstances J
GROUP BY UtilityName, UserName,
SUBSTRING(UtilityRequest FROM 1 FOR 100))
WITH DATA
PRIMARY INDEX (UtilityName, UserName)
ON COMMIT PRESERVE ROWS;
Select job instances whose elapsed time is more than 50% of the average using the
Last30dayJobInstances and Last30dayAverage tables.
SELECT I.*
FROM Last30dayJobInstances I,
Last30dayAverage A
WHERE I.UtilityName = A.UtilityName
AND I.UserName = A.UserName
AND I.UtilReqShort = A.UtilReqShort
AND (
(EXTRACT(HOUR FROM ((I.JobEndTime - I.JobStartTime) HOUR TO SECOND)) * 3600) +
(EXTRACT(MINUTE FROM ((I.JobEndTime - I.JobStartTime) HOUR TO SECOND)) * 60) +
(EXTRACT(SECOND FROM ((I.JobEndTime - I.JobStartTime) HOUR TO SECOND)))
) > (
((EXTRACT(HOUR FROM A.AvgJobElapsedTime) * 3600) +
(EXTRACT(MINUTE FROM A.AvgJobElapsedTime) * 60) +
(EXTRACT(SECOND FROM A.AvgJobElapsedTime))
) * 1.5);
Capacity Planning
In this scenario, you calculate these monthly metrics for each utility for the previous
12 months:
•
Number of job instances
•
Average job elapsed time
•
Total processing volume (byte counts and row counts)
•
Average processing volume
•
Average throughput (rows per second and bytes per second)
SELECT TRUNC(CAST(JobEndTime AS DATE), 'RM') JobMonth, UtilityName,
COUNT(*) AS NumJobs,
AVG(
(EXTRACT(HOUR FROM ((JobEndTime - JobStartTime) HOUR TO SECOND)) * 3600) +
(EXTRACT(MINUTE FROM ((JobEndTime - JobStartTime) HOUR TO SECOND)) * 60) +
(EXTRACT(SECOND FROM ((JobEndTime - JobStartTime) HOUR TO SECOND)))
) AS AvgJobTime,
SUM(RowsInserted), AVG(RowsInserted),
SUM(RowsUpdated ), AVG(RowsUpdated ),
SUM(RowsDeleted ), AVG(RowsDeleted ),
SUM(RowsExported), AVG(RowsExported),
SUM(Phase1ByteCount), AVG(Phase1ByteCount),
SUM(Phase2ByteCount), AVG(Phase2ByteCount),
AVG(RowsInserted) / AvgJobTime AS InsRowsPerSec,
AVG(RowsUpdated ) / AvgJobTime AS UpdRowsPerSec,
AVG(RowsDeleted ) / AvgJobTime AS DelRowsPerSec,
AVG(RowsExported) / AvgJobTime AS ExpRowsPerSec,
AVG(Phase1ByteCount) / AvgJobTime AS AvgPhase1BytesPerSec,
AVG(Phase2ByteCount) / AvgJobTime AS AvgPhase2BytesPerSec
FROM (SELECT *
FROM DBC.DBQLUtilityTbl U
Database Administration 151
Chapter 7: Loading and Exporting Data: Application DBAs
Utility Job Performance Analysis and Capacity Planning
WHERE CAST(JobEndTime AS DATE) >= ADD_MONTHS(TRUNC(CURRENT_DATE, 'RM'), -12)
AND CAST(JobEndTime AS DATE) < TRUNC(CURRENT_DATE, 'RM') ) AS OneYearLog
GROUP BY JobMonth, UtilityName
ORDER BY JobMonth, UtilityName;
152 Database Administration
CHAPTER 8
Working with Sessions and
Accounts: Operational DBAs
This chapter describes the differences between ANSI and Teradata session modes and how to set the system session mode. The chapter also describes Teradata sessions: how to obtain information about them, troubleshoot problems with them, and improve their performance.
In the section on user accounts, the chapter describes how to create accounts, work with system accounting views, and use account string expansion to measure query and AMP usage.
About Session Modes and Transaction
Processing
Teradata Database supports two session modes:
•
ANSI
•
Teradata
In ANSI session mode:
•
Transaction processing follows the rules defined by the ANSI/ISO SQL:2011 standard.
•
For Teradata SQL features that support both the ANSI/ISO SQL standard syntax and the
Teradata SQL dialect, the ANSI/ISO SQL standard syntax is used.
•
The default case specification for character data is CASESPECIFIC.
•
When a character string requires truncation, the excess pad characters are truncated without reporting an error. Truncation of characters other than trailing nonblank characters results in a truncation exception.
•
Cursors are always positioned. Updating or deleting the most current fetched cursor row using the UPDATE/DELETE … WHERE CURRENT OF statement is valid.
•
The default table type is MULTISET, which allows duplicate rows.
•
The TRIM function trims both leading and trailing pad characters by default.
In Teradata session mode:
•
Transaction processing follows the rules defined by Teradata before the emergence of the
ANSI/ISO SQL standard. These rules largely conform with ANSI/ISO semantics, but in some cases differ in important ways from the ANSI/ISO semantics.
•
For Teradata SQL features that support both the ANSI/ISO SQL standard syntax and the
Teradata SQL dialect, the Teradata SQL syntax is used.
•
By default, character data is NOT CASESPECIFIC, except when the character set is
GRAPHIC. GRAPHIC character data by default is CASESPECIFIC.
Database Administration 153
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Transaction Processing in ANSI Session Mode
•
The default case specification for character data is NOT CASESPECIFIC. The exception is data of type CHARACTER (n) CHARACTER SET GRAPHIC, which is always
CASESPECIFIC.
•
When a character string requires truncation, the string is truncated silently with no truncation notification given. This is true for truncation of pad or non-pad characters.
•
Cursors are never positioned. Updating or deleting the most current fetched cursor row using the UPDATE/DELETE … WHERE CURRENT OF statement is not valid.
•
The default table type depends on whether the table has a primary index. Tables with a PI are SET (do not allow duplicate rows). Tables without a primary index (including columnpartitioned tables) are MULTISET (allow duplicate rows).
•
The TRIM function trims only trailing pad characters by default.
Note: Case specificity does not apply to CLOBs.
A transaction is a logical unit of work. The statements nested within the transaction either execute successfully as a group or do not execute. A Teradata SQL transaction can be a single
Teradata SQL statement, or a sequence of Teradata SQL statements.
Transaction Processing in ANSI Session Mode
In ANSI mode, transactions are always implicitly started and explicitly closed.
A transaction initiates when one of the following happens:
•
The first SQL request in a session executes
•
The first request following the close of a previous transaction executes
The COMMIT or ROLLBACK/ABORT statements close a transaction.
If a transaction includes a DDL statement (including DATABASE and SET SESSION, which are considered DDL statements in this context), it must be the last statement in the transaction other than the transaction closing statement.
If an error is found in a request, then that request is aborted. Normally, the entire transaction is not aborted; however, some failures will abort the entire transaction. Locks placed by erroneous statements are not released.
Sessions in ANSI mode do not support BEGIN TRANSACTION/END TRANSACTION statements.
Transaction Processing in Teradata Session
Mode
In Teradata mode, transactions can be either implicit or explicit.
A request that is not enclosed by the BEGIN TRANSACTION and END TRANSACTION statements is treated as an implicit transaction.
154 Database Administration
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Setting the Session Mode
An implicit transaction can be one of the following:
•
A single DML statement
•
A macro or trigger
•
A multistatement request
DDL statements are not valid in a multistatement request and are therefore not valid in an implicit transaction composed of a multistatement request.
Explicit transactions contain one or more statements enclosed by BEGIN TRANSACTION and END TRANSACTION statements. The first BEGIN TRANSACTION initiates a transaction and the last END TRANSACTION terminates the transaction. Explicit transactions can contain multistatement requests.
When multiple statements are included in an explicit transaction, you can specify only a DDL statement if it is the last statement, and is immediately followed by END TRANSACTION.
If an error occurs, the request fails, the entire transaction is aborted and all locks are released.
Sessions in Teradata mode do not support the COMMIT statement. The END
TRANSACTION statement terminates a transaction with commit, while the ABORT and
ROLLBACK statements terminate a transaction with a rollback.
Setting the Session Mode
A system parameter specifies the default transaction mode for a site. You can override the default transaction processing mode for a session using one of the following:
•
SessionMode field of the DBS Control Record
•
BTEQ command .SET SESSION TRANSACTION
•
Preprocessor2 TRANSACT() option
•
Session Mode field of the Teradata ODBC Driver Options
•
JDBC TeraDataSource.setTransactMode() method
•
SQL Assistant when defining a .Net data source
Reference Information
Information on...
•
transaction, request, and statement processing
•
transaction processing in ANSI and
Teradata session modes
Is available in...
SQL Request and Transaction Processing
Database Administration 155
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Checking for Logged On Sessions
Information on...
SQL statements for processing transactions:
•
BEGIN TRANSACTION/END
TRANSACTION
•
COMMIT
•
ABORT
•
ROLLBACK setting the SessionMode field of the DBS
Control Record using the SESSION TRANSACTION BTEQ command setting the Session Mode field in the Teradata
ODBC Driver Options dialog box setting the Session Mode field when defining a .Net data source using SQL Assistant
Is available in...
SQL Data Manipulation Language
Utilities
Basic Teradata Query Reference
ODBC Driver for Teradata User Guide
Teradata SQL Assistant for Microsoft Windows User
Guide
Checking for Logged On Sessions
You may need to check if sessions are running on the system to warn users when you want to bring Teradata Database down or disable logons. There are several ways to check if there are established sessions on the system.
•
Run the Query Session utility (see “Query Session (qrysessn)” in Utilities). If there are sessions, it will report as follows:
Session state query results : 08/07/16 14:28:59
Host Session PE DBC User ID
----- ------- ----- -------------------------------
1 1015 16383 JohnSmith
State details : IDLE
•
Use Teradata Administrator or Teradata Studio
•
Run a select on the DBC.SessionInfoV view.
Obtaining Session Information
To find information about a session, for example CPU or I/O data for a specific session, try one of the following tools.
•
Teradata Viewpoint. Using Teradata Viewpoint, you can view information about queries running in a Teradata Database system so you can spot problem queries. You can analyze and decide whether a query is important, useful, and well-written.
156 Database Administration
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Troubleshooting Problems with Sessions
•
PM/APIs. If you write your own application, use the MONITOR SESSION request or the
MonitorSession function to collect current status and activity information for sessions that are logged on to Teradata Database. PM/APIs allow you to immediately access session information but you must have proper privileges to submit the requests or use the functions. For more information, see Application Programming Reference.
•
DBC.AmpUsage view and ASE. Use ASE to generate 1 row into DBC.AmpUsage per AMP per session. Note that the AmpUsage rows are not written until the query completes. For
more information on ASE, see “Logging Resource Usage Data with Account String
•
DBC.SessionInfo view. Use this view to see the client interface that established the session, the logon date and time, and current role among other information. There is one row for each time a user is logged on.
•
DBQL information. You must have first already enabled logging by submitting the BEGIN
QUERY LOGGING statement for the users/accounts/applications you wish to monitor.
For more information, see “SQL Logging Statements” on page 266 .
Troubleshooting Problems with Sessions
When a session appears idle, it is also referred to as a hung session. Hung sessions can be a result of problems with lock contention, a transaction rollback on a job with an error, the TDP losing communication with Teradata Database, or some other problem. The tools listed in the following table can help troubleshoot hung sessions.
Run the following utilities…
Query Session (qrysessn)
Recovery Manager
(rcvmanager)
Lock Display (lokdisp)
Show Locks (showlocks)
To determine…
the status of each session. For more information on what the different statuses mean, see “Query Session (qrysessn)” in Utilities.
if the system is rolling back any transaction. See “Recovery Manager
(lokdisp in Utilities.
Note: Some rollbacks may take a long time (several hours) to process because every TJ on every AMP must be accessed and updated.
transaction locks blocking the system. See “Lock Display (lokdisp)” in Utilities.
if there are host utility locks or HUT locks.
You may have determine if there is a lock on a certain table cleared by release lock override and system running normally. See “Show
Locks (showlocks)” in Utilities.
If there is an error during an archive or the archive job is killed prematurely, the outstanding locks from the job may block other sessions. To resolve this problem:
Database Administration 157
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Working with User Accounts
1
2
3
4
First run Query Session (qrysessn) to determine which session are blocked.
Then run Recovery Manager to see if anything is in recovery.
Next, check to see if there are any locks present and what kind of locks they are.
•
If you notice DBC has a WRITE lock on anything, run the CheckTable utility to determine the state of the tables. If any tables are corrupted, try dropping and restoring the table.
•
If there are ARC jobs that are hung, release the locks by using the RELEASE LOCK
(databasename) ALL, OVERRIDE statement.
Use QuerySession to check whether formerly blocked sessions are now active.
Sometimes, you may have to abort the session and restart the system (tpareset) to clear a hung session. However, first contact Teradata Customer Support and determine if you should take a crashdump before restarting the system so that you can try isolating the problem causing the block.
You may run across problems with FastLoad or MultiLoad sessions. This is usually indicated by either a 2633, 2574, or 2738 error. Use the DBC.SessionInfoV view to determine the host number, username, and partition associated with the session number.
If a session experiences an SQL error and the system attempts to roll back the transaction, and other sessions become blocked during this time, you may need to issue aborts using Teradata
Viewpoint. See Help in Teradata Viewpoint for information on how to issue an abort.
For information on how to handle a slow or hung job, see the sections beginning with
“Investigating Query Blocks and Delays” on page 501 .
Working with User Accounts
Purpose of System Accounting
System accounting serves three important administrative functions:
•
Charge-back billing (for equitable cost allocation)
•
Capacity planning (to anticipate your needs)
•
Resource management (to identify CPU and I/O usage)
For more information on how to identify and analyze session behavior and apply resource
control, see Chapter 12: “Managing Database Resources: Operational DBAs.”
Charge-back Billing
You may need to charge users for their use of resources. Charge-back billing permits objective cost allocation of system resources across all users.
The user account string enables you to summarize resource usage by account ID. The system table DBC.Acctg tracks CPU use in seconds and I/O resources expended by a session. The I/O resource tracks the number of AMP to disk read and write operations generated by a given user or account, and charges them to the current account associated with a session.
158 Database Administration
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Creating Accounts
Use the DBC.AMPUsage view to access aggregations of DBC.Acctg contents.
Resetting the Resource Usage Counters
A systems administrator can use the CPU usage information in DBC.Acctg to bill users every month. At the beginning of each month, the systems administrator runs ClearAccounting to reset the resource usage counters back to zero:
EXEC DBC.ClearAccounting;
Capacity Planning
To plan for the resources needed to accommodate growth, you must know how the current workload is affecting the system. To assess the effect of the current workload, you can collect and analyze information about resource utilization.
Collecting and analyzing current resource usage information is one component of data analysis but another valuable component is the collection of historical usage statistics. The accounting feature can be used to determine the activity on current workloads, which assists you in anticipating future needs. For information on capacity planning for a new system or database, see Database Design.
Workload Management
You may need to control who gets specific system resources using Priority Scheduler (for SLES
10 systems) or the Workload Designer portlet in Viewpoint for SLES 11 systems. For
additional information, see “Managing the Database Workload” on page 507
.
Creating Accounts
For information on how to create a user account, see
“Creating User Accounts” on page 108 .
Working with System Accounting Views
Teradata Database provides the following views to support administration of accounts and accounting functions:
•
DBC.AccountInfoV
•
DBC.AMPUsage
By auditing the information in these views, you can see which users heavily or rarely use the system. Or, you can make adjustments to the priority assigned to specific accounts.
About DBC.AccountInfoV
The DBC.AccountInfoV view accesses the DBC.Dbase, DBC.Profiles, and DBC.Accounts dictionary tables to return information about all valid accounts for all databases, users, and profiles.
Database Administration 159
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Working with System Accounting Views
Use DBC.AccountInfoV to find out:
•
What accounts are valid for which user
•
The assignment of:
•
Priority Scheduling codes
•
Account string expansion variables (see
“Logging Resource Usage Data with Account
).
This dictionary table …
DBC.Accounts
DBC.DBase
DBC.Profiles
Stores …
all accounts for all databases, users, and profiles.
If an account is not specified at creation time, the default is used. The default is determined as explained in
“Determining the Account for a
.
DBC.Accounts is used to verify any account entered by a user at logon time or with a SET SESSION ACCOUNT statement.
the default account for each:
•
Database
•
User
If a database or user is defined with multiple accounts, the first is used as the default.
If a user is assigned a profile that is defined with one or more accounts, the first profile account is used as the default. (Profile accounts take precedence over user accounts.) the default account for each profile. If multiple accounts are defined, the first is used.
Note: If an account is not specified for a profile, the value is NULL in the
DefaultAccounts field for that profile.
Example
The following query selects all account strings with a RUSH priority code, and whether the name associated with an account is an individual user or a profile:
SELECT AccountName,UserName,UserOrProfile
FROM DBC.AccountInfoV
WHERE AccountName LIKE '$R%'
ORDER BY AccountName ;
In this example, the view returns:
AccountName
--------------
$R1$ABCD&S&D&H
$R1$EFGH&S&D&H
$R2$IJKL&S&D&H
$R2$MNOP&S&D&H
$R2&QRST&S&D&H
UserName
---------------
AcctsRecv
DBADMIN
DBC
SysAdmin
SystemFe
UserOrProfile
-------------
Profile
User
User
User
User
160 Database Administration
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Working with System Accounting Views
About the DBC.AMPUsage View
The DBC.AMPUsage view provides information about the usage of each AMP for each user and account. It also tracks the activities of any console utilities. By user, account, or console utility session, DBC.AMPUsage stores information about:
•
CPU seconds consumed
•
Number of read/write (I/O) operations generated
Note: AMPUsage reports logical I/Os explicitly requested by the database software, even if the requested segment is in cache and no physical I/O is performed.
DBC.AMPUsage uses the DBC.Acctg table to provide aggregated information by username, account ID, and AMP. Updates to the table are made periodically during each AMP step on each processor affected by the step. (This means if there are long-running steps, AMPUsage numbers show large increases periodically, instead of continuous incremental additions.) The data is collected and continually added to what is already in the table until you reset the
counters to zero (see “Resetting the Resource Usage Counters” on page 159
).
You can use the information provided by DBC.AMPUsage to do the following:
•
Bill an account for system resource use.
•
Determine what resources were used, by user and account string, after hours as well as during the day.
•
Summarize and archive the information, then zero it out on a per shift, per day, or per week basis.
•
Determine which session caused reduced performance (in-depth analysis can be performed with DBQL data).
•
Derive capacity needs to plan for expansion.
DBC.AmpUsage does not record the activity of parsing the query, or of processing on a query basis.
You can use query logging to capture query text, step information, and elapsed processing time, and to differentiate queries submitted by SQL-generating products that do not provide a variety of user IDs and account ids in the logon string. For instructions and a description of the data capture options, see
Chapter 15: “Tracking Query Behavior with Database Query
Use Teradata Viewpoint for a look at up-to-the-moment activity in real-time.
Database Administration 161
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Logging Resource Usage Data with Account String Variables
Example: Totalling CPU Time and I/O by User
This SQL statement requests totals for CPU time and I/O for user DBA01.
The totals are aggregates of all resources.
SELECT UserName (FORMAT 'X (16)')
,AccountName (FORMAT 'X (12)')
,SUM (CpuTime)
,SUM (DiskIO)
FROM DBC.AMPUsage
WHERE UserName = 'DBA01'
GROUP BY 1, 2
ORDER BY 3 DESC ;
For this example, AMPUsage returns the following rows:
UserName
--------
DBA01
DBA01
DBA01
AccountName
-----------
$M2$ABCD&S&D&H
$M2$EFGH&S&D&H
$M2$IJKL&S&D&H
SUM (CpuTime)
-------------
6,336.76
4,387.14
1.28
SUM (DiskIO)
-----------
505,636
303,733
166
For detailed information on these and all the system views, see Data Dictionary. For more information on how to use DBC.AMPUsage and other views to find problems and improve
performance, see “ASE Impact on PE and AMP Performance” on page 512
.
Logging Resource Usage Data with Account
String Variables
You can log resource usage data in the DBC.Acctg table by creating substitution variables in the account ID part of the user logon string. These Account String Expansion (ASE) variables let you to measure query and AMP usage more precisely. You can gather statistics for capacity planning, gather information for use in charge back and accounting, and track resource consumption.
The variables allow you to include date and time information in the string. Using ASE is a best practice. If ASE is activated, Teradata Database collects statistics based on the variables used during the active session, and stores the information into the underlying table of the
DBC.AmpUsage view.
Enabling Account String Expansion
Before a user can activate any of the ASE functions, there must be a matching account string in the DBC.UsersV view for that user and ASE function. This means that the user cannot simply add the ASE variables to any logon account string unless first authorized by the DBA. This permits a level of control over users and programmers who may or may not use ASE properly.
162 Database Administration
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Setting the Default Account String to Use ASE
For example, to create a user that can use ASE, define the variables in his user account string or to the profiles which he will use.
The DBA or any user with CREATE USER privileges can define the following:
CREATE USER john FROM finance AS PERM = 100000, ACCOUNT =
'&H1&MSIR&S&D&H', '$M2$SMIB&S&D&H';
While the recommended ASE variable combination is &S&D&H, others include time (&T) and the time of log on (&L), or a combination thereof. For more information on the different variables, see
“About the ASE Substitution Variables” on page 164 .
Setting the Default Account String to Use ASE
To enable ASE, define one or more of the ASE variables in an account string either directly in the user definition (with the CREATE USER statement), or in the definition of the profile assigned to the user (with the CREATE PROFILE statement). You should set up the account to default to automatically use ASE.
Examples
You assign two account strings to user TAW. The first entry is the default.
MODIFY USER TAW AS ACCOUNT=('$M2$acct&S&D&H','$M2$finr&S&D&H');
When TAW logs on without specifying an account, TAW will default to using the first account.
At query execution time, Teradata Database replaces &S with the session number, &D with the current date, and replaces &H with the current hour.
In another case, you assign two account strings to user DTG. However, the first (default) account string does not specify session, date, or hour expansion:
MODIFY USER DTG AS ACCOUNT = ('$M1$acct', '$M2$acct&S&D&H');
When DTG logs in without specifying an account, then the ASE logging does not occur. To initiate them, DTG must type
&S&D&H
in the account string either when he logs in or during the session:
•
Log in using
.logon DTG, mypassword, '$M2$acct&S&D&H'
•
Change the account during a session with the following statement:
SET SESSION ACCOUNT='$M2$acct&S&D&H'
Because assigning the first account to DTG without ASE variables means ASE logging does not occur by default, Teradata does not recommend listing the account without &S&D&H in the account string first.
In another case, assume you omit the ASE variables in the two account strings for user AM1:
MODIFY USER AM1 AS ACCOUNT=('$M2$acc1', '$M2$acc2');
AM1 cannot invoke the ASE feature at any time. Teradata does not recommend assigning the first account string without the &S&D&H variables.
Database Administration 163
Chapter 8: Working with Sessions and Accounts: Operational DBAs
About the ASE Substitution Variables
About the ASE Substitution Variables
The system resolves these variables at logon or at SQL statement execution time. For example,
$M$MAR&D becomes $M$MAR090704 for a logon at July 4, 2009 under the MAR account.
(The presence of Performance Group priority codes $R$, $H$, $M$, or $L$ are ignored by
ASE.)
The ASE variables may be used in any combination and in any order, subject to the constraints on length and position. These are:
•
The maximum length of an account string is 128 characters.
•
If priority codes are being used, the first ASE variable must begin after the priority codes.
If no priority codes are used, the first account string expansion variable can begin in any character position of the account string. You should use a priority code. Otherwise, without a priority code, all usage with this account defaults to Medium Priority in the
Default Resource Partition.
•
The increasing levels of granularity that result when additional rows are written to the
DBC.Acctg table.
There is a minor increase in overhead associated with managing the frequency with which the DBC.Acctg table is cleared.
Teradata recommends that if you use DBQL, you should only use &S &D &H jointly
(&S&D&H).
Variable
&D
&H
&I
Format
YYMMDD
Description
Date. The system substitutes the date it received the SQL request into the account string.
HH (24 hour clock) Hour. The system substitutes the hour of day it received the
SQL request. This is useful for identifying the large resource users during peak periods. In charge back systems, use &H to give users preferential billing rates for running queries during off-peak time periods.
If you use the &H variable without the &D variable, the system sums statistics collected for a specified hour on one day with existing statistics for the same hour on other days.
LLLLSSSSSSSSSRRRRRRRRR The system substitutes the logon host ID, current session number, and sequential request number.
Note: All the queries within one stored procedure CALL command are reported under one request number. The request number is the client request number, which is the request of the CALL.
If you need more detail about queries within a stored procedure, the information will be available in
DBC.DBQLogTbl.RequestNum.
Length
6
2
22
164 Database Administration
Chapter 8: Working with Sessions and Accounts: Operational DBAs
About the ASE Substitution Variables
Variable
&L
&S
&T
Format
YYMMDDHHMMSS.hh
SSSSSSSSS
HHMMSS (24 hour clock)
Description
Logon date-time substitution. The system substitutes the logon timestamp. This value does not change until the user logs off and then logs on again.
Because there is only one logon string for each session pool, the &L option generates only one row per session, regardless of the number of users connected to the pool.
If a group of users share user IDs and passwords, the system accumulates all DBC.AMPUsage statistics under the same user
ID. In this case, use the &L option to generate separate statistics and to monitor the LogonSource field of
DBC.LogOnOffV.
The system substitutes the current session number.
Time substitution. The system substitutes the time of day it received the SQL request.
Note: Using &T can be very resource intensive. If you notice an impact on system performance, delete rows from
DBC.Acctg and discontinue using &T.
This variable allows for one second granularity, causing the system to write a row for virtually every individual SQL request. If the system receives two or more SQL requests for the same user/account ID pair in the same second, the system sums AMP usage statistics. This summation can be any combination of subsecond requests, or a combination of subsecond requests with a longer request.
If the system receives a multistatement request, each individual
SQL statement in the request has the same timestamp; therefore, the row written to DBC.AMPUsage contains the summation of the statistics of the individual statements.
Using the &T variable without the &D variable causes the system to sum statistics collected for a specified time on one day with existing statistics for the same time on other days.
9
6
Length
15
Restrictions and Usage Rules
Below are known requirements that could influence the standard formats of ASE:
•
Account strings cannot exceed 128 characters and must be able to accommodate any
Performance Group name.
•
Be aware that the account string character count includes:
•
Separation characters, such as colons (:) in time fields and slashes (/) in dates. If a string consists of all the ASE variables, the result is 32 characters long. If an account string exceeds the character limit, characters to the right of the limit are truncated.
•
Performance group names.
•
You can intersperse ASE variables with literals, subject to the constraints of length and of position relative to performance group controls.
Database Administration 165
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Considerations for Assigning ASE Variables to Different Workloads
•
To the greatest extent possible and appropriate, request level usage detail should be captured. If DBQL is enabled, however, ASE becomes less important as a mechanism for enabling request level usage detail since DBQL can capture usage for queries.
Recommendation for account string setup: $H1$WXYZ&S&D&H, where:
$H1$ = Performance Group PG
WXYZ = Work Group Identifier
&S&D&H = ASE Variables for session, date, and hour
•
Where possible, associate usage detail to higher-level aggregation for production batch processing, such as to the job or job step level.
•
For requests requiring short response time, the account string setup should not materially impact the performance of the request.
•
Provide the detailed information necessary to effectively manage the system. When consistently adhered to, the proper information will be captured to facilitate a wide variety of analyses and produce a set of standard metrics by which to measure the system.
•
When combining ASE variables, keep the following in mind:
•
You can use multiple ASE variables in any order in the account string following the first ASE variable.
•
The account string limit is 128-characters.
•
Although it is perfectly legal to combine the &H and &T options, the information collected is redundant.
•
If you specify &H or &T without &D, statistics collected on one day at one time are combined with statistics collected on other days at that same time.
Considerations for Assigning ASE Variables to
Different Workloads
Each ASE assignment depends on the type of usage being performed by the user ID. This has implications related to user ID assignment.
In general, workloads can be broadly grouped into three categories as follows:
•
Multisession/Multirequest
This workload can be identified by work typically done by MultiLoad, FastLoad, TPUMP or multisession BTEQ. These types of activities are normally used for database maintenance. Each session used handles multiple requests over time. The workload for this type of work tends to be more predictable and stable. It runs regularly and processes the same way each time it runs.
•
Single Session, nontactical
This workload is typically initiated through a single session BTEQ, SQL Assistant,
MicroStrategy, or other query-generating tool. Ad hoc users, or single session BTEQ jobs in the batch process can generate this type of activity. The single session may generate one or many requests. The requests may be back to back, or there may be hours of idle time
166 Database Administration
Database Administration
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Considerations for Assigning ASE Variables to Different Workloads between them. Typically, the requesting user has very broad and informal response time expectations.
•
Single Session, tactical
This workload is similar to the Single Session workload category except that there is typically a very clear definition of response time and the response time requirements normally range between less than a second to a few seconds.
The following ASE variables are to be used for each of the workload categories listed above along with the rationale for selecting the ASE variables.
•
Multisession, Multirequest
For this workload, usage information need not be captured at the request level. Workloads in this category can
•
Processes the same request over and over again across the multiple sessions it establishes (such as TPUMP and multisession BTEQ).
•
Generate multiple internal requests that are not easily correlated to specific user generated activity (as is the case with MultiLoad and FastLoad).
As a result, capturing usage detail at the request level typically does not provide especially meaningful information. Therefore, the recommended standard is to capture usage at the session level using the '&S' ASE variable. The account string for User Ids performing this workload category would have the following format:
Account String Format: $XX$_&S
Length: 12-15 characters (depending on PG length)
Capturing session level information for this workload category provides several benefits, including:
•
All usage for a given job can be more easily captured. Furthermore, the job level usage can then be grouped to associate all batch processing to an application.
•
All usage for a given job step can be obtained. This can facilitate performance analysis for batch processes.
•
Session usage within a multisession utility can be better analyzed to determine the optimal number of sessions to log on to the system.
•
Single Session, nontactical
For this workload, request level usage detail is desired. This type of activity is typically the most difficult to manage and control in a mixed workload, data warehouse environment.
They also typically represent the greatest opportunity for optimization. Although request level detail requires some minor additional overhead to capture, the benefits of gaining additional visibility into the impact of each request outweighs the increased overhead in data collection. The account string for user IDs performing this workload category would have the following format:
Format: $XX$_&I
Length: Up to 128 characters
Capturing request level information in this manner has numerous benefits, including:
167
Chapter 8: Working with Sessions and Accounts: Operational DBAs
Considerations for Assigning ASE Variables to Different Workloads
•
Usage associated with each SQL request can be identified. By applying specific metrics such as total CPU used, total IO used, CPU skew percent, Disk to CPU ratio, and so forth, problem requests can quickly and easily be identified and addressed.
•
Request level usage detail can be correlated to SQL statements in DBQL to greatly simplify performance-tuning efforts. DBQL captures the date and time of the request as well as the session and request number of the request.
•
Performance tuning can become much more quantitative and definitive by comparing usage statistics for alternative query approaches. Capturing the consumption at the individual request enables this benefit.
•
Usage can be accumulated to the session level to provide same level aggregations and analysis to multisession, multirequest processing. As such, the same benefits can also be achieved.
•
Single Session, tactical
For this workload, high-speed performance and minimal response time are the primary objectives. Even if the Teradata Active EDW is not currently servicing this type of request, it is important to account for this type of work within the standard. Typically, this workload tends to be very predictable in nature with queries typically designed to be single
AMP retrievals. For this workload, capturing information at the request level is unnecessary for two reasons. First, the transactions are well defined and repeated over and over again. Second, the additional overhead required to record usage for each request would represent a meaningful portion of the overall work performed on behalf of the transaction. In other words, the additional overhead could materially impact request response time.
As a result, the account string for this workload can, as one option, target usage detail at the session level. The assumption in this case is that applications requiring high-volume, low response time requests will take advantage of session pooling to avoid the overhead of continually logging on and logging off. The account string for User Ids performing this workload category would have the following format:
Format: $XX$_&S
Length: 12-15 characters (depending on PG length)
Since this is the same ASE strategy as employed for the multisession, multirequest workload, all the same benefits would accrue. In addition, as it pertains to this particular workload category, the following benefits could also be achieved:
•
Usage by session could assist in determining the optimal number of sessions to establish for the session pool.
•
CPU and/or IO skew by session could help identify possible problems in the data model for the primary index retrievals.
168 Database Administration
Chapter 8: Working with Sessions and Accounts: Operational DBAs
About Using ASE With Client Utilities
About Using ASE With Client Utilities
Except for the utilities and variables noted in the table that follows, you can use ASE with any utility that uses a standard Teradata Database interface to log on, including:
•
BTEQ
•
FastLoad
•
MultiLoad
•
Teradata Parallel Transporter
•
TPump (except for &T)
•
FastExport
•
Teradata SQL Assistant (formerly known as Queryman)
•
Teradata Studio
The exceptions are described in the following table.
Do not use …
TPump
PMPC statement
ARC
With …
&T any ASE code any ASE code
Because …
&T generates a row in AMPUsage for nearly every SQL statement.
requests generated are not parsed and substituting variables cannot be expanded.
ARC generates two kinds of requests:
•
SQL type requests that go through the Parser and expand the ASE variables.
•
Direct requests from ARCMAIN to the AMP that bypass the Parser. As a result, substitution variables are not expanded, so the rows in DBC.AMPUsage contain literal ASE codes rather than the expected date, hour, and timestamp values.
Database Administration 169
Chapter 8: Working with Sessions and Accounts: Operational DBAs
About Using ASE With Client Utilities
170 Database Administration
CHAPTER 9
Managing Space:
Operational DBAs
This chapter describes space management:
•
Considerations for setting up disk space limits
•
How to find and resolve system, database, and user space issues
•
How to find and fix skewed tables
•
Creation of a macro for space usage reporting
•
Methods of viewing space utilization
•
How to manage peak values, data blocks, and cylinders
About Types of Space
Space (whether permanent, spool, or temporary) is measured in bytes. The following table lists the three types of database space.
Type of Space
Permanent (PERM)
Spool
Description
Permanent space allocated to a user or database is a repository for database objects, for example, tables, indexes, and journals. Perm space can be defined as a fixed quantity or as a constant expression.
When an object is created or data rows are inserted, the system allocates space as needed from the PERM space of the immediate owner. The space is returned automatically when no longer needed.
Note: Perm space allocated to a user or database that is currently unused is available for use as spool or temporary space.
A database or user with no PERM space can still own views, macros, and triggers but cannot have objects that require space such as tables, UDFs, stored procedures, HIs, JIs, or journals.
Spool space holds intermediate query results or formatted answer sets to queries and volatile tables. The system can use unassigned
PERM space for spool space.
When a user creates new users or databases, the amount of spool space for those new objects must not exceed the spool space limit of their immediate owner. If you do not specify a spool space limit, a new user automatically inherits the spool limit of its parent.
To more easily manage spool space, define the value for spool in a profile. You can then assign a profile to users and all of the users will inherit the spool space definition.
Database Administration 171
Chapter 9: Managing Space: Operational DBAs
About Identifying and Correcting System-level Space Problems
Type of Space
Temporary (TEMP)
Description
Temp space defines the number of bytes the system will use to store data for global temporary tables. The value you specify must not exceed the value of the immediate parent at the time of creation. If you do not specify a value, the maximum value defaults to that of the parent.
Note: Global temporary tables require both PERM and TEMP space. The database/user must have adequate perm space to accommodate the global temporary table header on each AMP.
Table header size varies by table definition and the maximum size for a table header is 1 MB. Temp space is required for storing the actual rows.
To more easily manage TEMP space, define the value for it in a profile. You can then assign a group of users to this profile and all of the users will inherit the TEMP space definition.
You can manage space using Teradata SQL, or with Teradata Viewpoint administrative tools.
Capacity Planning
This chapter covers space issues for an already running system. For information on capacity planning for a new system or database, see Database Design.
About Identifying and Correcting System-level
Space Problems
Several Teradata Viewpoint portlets as well as some Teradata views provide space usage information, which you can use to identify space problems.
Using Viewpoint to Find System Space Issues
Use the System Health portlet to track current system-level space usage. This portlet displays segmented bar graphs that show normal, degraded and critical space usage ranges.
Parameter
Total disk space
Description
Percentage of total disk space in use across the system
Actions
If system disk space usage moves beyond the normal range (80%), add disk capacity.
172 Database Administration
Chapter 9: Managing Space: Operational DBAs
Identifying Space Problems Using Viewpoint
Parameter
DBC disk space
Description
Percent of disk space assigned to DBC currently in use
Actions
The DBC database contains logs, system tables and views, and other functions necessary for system operation.
If DBC space usage moves beyond the normal range, reallocate perm space from
DBADMIN to DBC.
Identifying Space Problems Using Viewpoint
1
2
3
Use the Space Usage portlet to track detailed space usage parameters, including available space, current usage, and peak usage for each space type, by database or user.
Select the system you want to investigate and click Next>.
In the selection menu, select Database: By most space and click Next>.
Use the Database Total view to display perm space usage for all databases that have assigned perm space. The following columns may indicate perm space problems.
Parameter
Current
Perm%
Peak Perm%
Current Perm
Skew%
Description
Percentage of available perm space currently being used by each user and database.
The highest percentage of total available perm used by each user and database since the last reset (default 24 hours).
A measure of how the perm space for a database distributes across all AMPs. Higher percentages indicate uneven distribution.
Actions
Consider allocating additional perm space to any user or database when it reaches 90% perm space usage. See
Resolve Space Issues” on page 179 .
A high perm skew percentage indicates that one or more tables in the database is skewed, that is, it has a poorly designed primary index.
To identify skewed tables, see “Finding and
Fixing Skewed Tables Using Viewpoint” on page 180 .
4
Note: If you think there is a space problem, you can click on a database name to display the Details view, which shows the current perm, peak perm, and skew% for each table in the database.
Use the Users Over 85% Spool view to display any users that have exceeded the safe range of spool usage since the last reset.
Note: In some cases excess spool usage may be the result of poorly constructed queries rather than an actual shortage of spool.
Database Administration 173
Chapter 9: Managing Space: Operational DBAs
Identifying Space Issues by Querying System Views
Parameter
Current Spool%
Peak Spool%
Description
Percentage of available spool space currently being used by each user and database (list)
Largest percentage of available spool space recently used by each listed user and database since the last reset (default 24 hours).
Actions
Monitor regularly to determine whether certain users regularly exceed 85% spool.
•
Use Query Monitor to track query activity for users exceeding the limit, to determine whether the excessive spool usage is caused by poorly constructed queries.
•
Increase the spool space limit for users who need spool by modifying the user or applicable profile definition. See
“Increasing the Spool Space Limit for a
.
Identifying Space Issues by Querying System
Views
Querying the DiskSpaceV View to Find System Space Issues
The DBC.DiskSpaceV view returns AMP information about disk space usage at the database or user level. It also can report spool space usage. DiskSpaceV figures are calculated only for the space owned by the user submitting the query. To find information about the full system, log on as the topmost user in the hierarchy (usually your site administrative user).
You can determine the amount of space allocated to a profile by querying the MaxProfileSpool and MaxProfileTemp columns of DBC.DiskSpaceV. By querying MaxSpool and MaxTemp of the restricted view (DBC.DiskSpaceVX), you get the space limits for your individual user space. The following queries show the kinds of information you can obtain from
DBC.DiskSpaceV.
TO find the… SUBMIT the following query…
total disk space in the entire system
SELECT SUM(MaxPerm) FROM DBC.DiskSpaceV; disk space currently in use
SELECT SUM(CurrentPerm),
SUM(MaxPerm),
((SUM(currentperm) / NULLIFZERO(SUM(maxperm)) * 100))
(TITLE '%MaxPerm', FORMAT 'zz9.99')
FROM DBC.DiskSpaceV; disk space for a given database percent of disk space that is available for spool
SELECT sum(maxperm
FROM DBC.DiskSpaceV WHERE databasename='xxxx';
SELECT (((SUM(MaxPerm) - SUM(CurrentPerm)) /
NULLIFZERO(SUM(MaxPerm))) * 100)
(TITLE'% Avail for Spool', format'zz9.99')
FROM DBC.DiskSpaceV;
174 Database Administration
Chapter 9: Managing Space: Operational DBAs
Identifying Space Issues by Querying System Views
TO find the…
percent of space used by each database in the system
SUBMIT the following query…
SELECT Databasename (format 'X(12)')
,SUM(maxperm)
,SUM(currentperm)
,((SUM(currentperm))/
NULLIFZERO (SUM(maxperm)) * 100)
(FORMAT 'zz9.99%', TITLE 'Percent // Used')
FROM DBC.DiskSpaceV
GROUP BY 1
ORDER BY 4 DESC
WITH SUM (currentperm), SUM(maxperm); users who are running out of PERM space users using a lot of spool
SELECT Databasename (format 'X(12)')
,SUM(maxperm)
,SUM(currentperm)
,((SUM(currentperm))/
NULLIFZERO (SUM(maxperm)) * 100)
(format 'zz9.99%', TITLE 'Percent // Used')
FROM DBC.DiskSpaceV
GROUP BY 1 HAVING (SUM(currentPerm) / NULLIFZERO(SUM(maxperm))) > 0.9
ORDER BY 4 DESC;
SELECT databasename
,SUM(peakspool)
FROM DBC.DiskSpaceV
GROUP BY 1 HAVING SUM(peakspool) > 5000000000
ORDER BY 2 DESC;
Note: You can change the value 500000000 to whatever value is appropriate for your site.
Some sites with more space may have higher tolerance for higher spool usage and spool limits.
Querying the TableSizeV View
The DBC.TableSizeV view provides AMP information about disk space usage at the table level.
Optionally use viewnameVX for information on only those tables that the requesting user owns or has SELECT privileges on.
To find…
the table distribution disk space for a given table
Use the following query…
SELECT tablename (TITLE 'Table')
,currentperm (TITLE 'CurPerm')
,vproc (TITLE 'Amp')
FROM DBC.tablesizeV
WHERE databasename='xxx'
AND tablename = 'xxxx'
ORDER BY 2 DESC;
SELECT SUM(currentperm)
FROM DBC.tablesizeV
WHERE databasename='xxx'
AND tablename = 'xxxx';
If you submit a SELECT statement against DBC.TableSizeV and notice it takes a long time to process, replace the view definition by submitting the following query:
REPLACE VIEW DBC.TableSizeV
Database Administration 175
Chapter 9: Managing Space: Operational DBAs
Identifying Space Issues by Querying System Views
AS SELECT DataBaseSpace.Vproc,
Dbase.DatabaseName (NAMED DatabaseName),
Dbase.AccountName,
TVM.TVMName (NAMED TableName),
DataBaseSpace.CurrentPermSpace(NAMED CurrentPerm,
FORMAT '---,---,---,---,--9'),
DataBaseSpace.PeakPermSpace(NAMED PeakPerm,
FORMAT '---,---,---,---,--9')
FROM DBC.Dbase, DBC.DataBaseSpace, DBC.TVM
WHERE DataBaseSpace.TableID <> '000000000000'XB
AND DataBaseSpace.TableID = TVM.tvmid
AND TVM.DatabaseId = Dbase.DatabaseId
WITH CHECK OPTION;
This improved definition helps the Optimizer choose a better plan to process SELECT requests. (For more information, see Knowledge Article SD1000B999E.)
You may also want to look at the individual tables within the database to see if any are skewed.
Warning About AllSpaceV View
The DBC.AllSpaceV view provides space usage information at the object level (table, join index, permanent journal, or stored procedures) and the database/user level. However,
Teradata recommends using the DBC.DiskSpaceV and DBC.TableSizeV views instead because
DBC.AllSpaceV can return misleading results.
For example, the following query returns the current permanent space from the detail rows
and summary rows for each user or database. Note that the view sums data from both types of rows:
SELECT DatabaseName
,Sum(CurrentPerm)
FROM DBC.AllSpaceV
GROUP BY 1 having sum(currentperm) > 0
ORDER BY 2 desc;
DatabaseName Sum(CurrentPerm)
------------------------------ -------------------
DBC 47,183,872
Sys_Calendar 2,648,064
SysAdmin 1,380,352
SYSLIB 319,488
SystemFe 148,480
However, a similar query using DBC.DiskSpaceV will return the desired result and, on most systems, will run faster:
SELECT DatabaseName
,Sum(CurrentPerm)
FROM DBC.DiskSpaceV
GROUP BY 1 having sum(currentperm) > 0
ORDER BY 2 desc;
DatabaseName Sum(CurrentPerm)
------------------------------ -------------------
DBC 23,591,936
Sys_Calendar 1,324,032
176 Database Administration
Chapter 9: Managing Space: Operational DBAs
Finding Space Issues
SysAdmin 690,176
SYSLIB 159,744
SystemFe 74,240
The view DBC.DiskSpaceV includes only summary rows and the view DBC.TableSizeV includes only detail rows. The view DBC.AllSpaceV, however, includes both summary and detail rows and when you sum currentperm from AllSpaceV, you get the sum from both the summary and detail rows added together.
If you want space usage information at the database/user level, use the DBC.DiskSpaceV view which selects only summary (user/database) rows. If you want space usage information at the object level, use the DBC.TableSizeV view.
Finding Space Issues
1
2
3
Use the Space Usage portlet to track detailed space usage parameters, including available space, current usage, and peak usage for each space type, by database or user.
Select the system you want to investigate and click Next>.
In the selection menu, select Database: By most space and click Next>.
Use the Database Total view to display perm space usage for all databases that have assigned perm space. The following columns may indicate perm space problems.
Parameter
Current
Perm%
Peak Perm%
Current Perm
Skew%
Description
Percentage of available perm space currently being used by each user and database.
The highest percentage of total available perm used by each user and database since the last reset (default 24 hours).
A measure of how the perm space for a database distributes across all AMPs. Higher percentages indicate uneven distribution.
Actions
Consider allocating additional perm space to any user or database when it reaches 90% perm space usage. See
Resolve Space Issues” on page 179 .
A high perm skew percentage indicates that one or more tables in the database is skewed, that is, it has a poorly designed primary index.
To identify skewed tables, see “Finding and
Fixing Skewed Tables Using Viewpoint” on page 180 .
Database Administration 177
Chapter 9: Managing Space: Operational DBAs
Transferring Ownership and Permanent Space
4
Note: If you think there is a space problem, you can click on a database name to display the Details view, which shows the current perm, peak perm, and skew% for each table in the database.
Use the Users Over 85% Spool view to display any users that have exceeded the safe range of spool usage since the last reset.
Note: In some cases excess spool usage may be the result of poorly constructed queries rather than an actual shortage of spool.
Parameter
Current Spool%
Peak Spool%
Description
Percentage of available spool space currently being used by each user and database (list)
Largest percentage of available spool space recently used by each listed user and database since the last reset (default 24 hours).
Actions
Monitor regularly to determine whether certain users regularly exceed 85% spool.
•
Use Query Monitor to track query activity for users exceeding the limit, to determine whether the excessive spool usage is caused by poorly constructed queries.
•
Increase the spool space limit for users who need spool by modifying the user or applicable profile definition. See
“Increasing the Spool Space Limit for a
.
Transferring Ownership and Permanent Space
Transferring Permanent Space
With CREATE, GIVE, and DROP, you can transfer the permanent space of one database or user to another. This is particularly useful if you wish to transfer permanent space from a child of a child back to user DBC when user DBC is not the immediate owner. For information on dropping databases or users, see
“Dropping a Database or User” on page 62
. For information on transferring ownership of a database or user with GIVE, see
178 Database Administration
Chapter 9: Managing Space: Operational DBAs
Reallocating Perm Space to Resolve Space Issues
For example, assume the following hierarchy:
DBC
A B
D E
C
F
FF07A002
Also, assume that:
•
F has a MAXPERM of 10, a MAXSPOOL of 50, and a MAXTEMP of 25.
•
E has a MAXPERM of 10, a MAXSPOOL of 20, and a MAXTEMP of 15.
To increase permanent space for E:
1
From space owned by F, create temporary database X with a MAXPERM of 5:
2
CREATE DATABASE X FROM F AS PERM = 5 ;
The default is to allocate to a new database the same spool and temporary space as its owning database, so MAXSPOOL for X defaults to 50 and MAXTEMP for X defaults to 25.
The PERM allocation for X is taken from the space of its owning database; thus, the
MAXPERM of F is reduced to 5.
Give X to E by using the GIVE statement to transfer ownership:
3
GIVE X TO E;
Drop X with the following statement:
DROP DATABASE X;
This increases the MAXPERM of E to 15.
The MAXSPOOL and MAXTEMP of E are unchanged at 20 and 15, respectively.
Reallocating Perm Space to Resolve Space
Issues
1
2
In the Space Usage portlet, highlight the database or user that requires more space.
Note: You can also define the value for spool in a profile, and then assign profile
membership to a user. See “Creating Profiles” on page 111 .
Click the Add Space button and:
•
Enter your username and password (you must have DROP privileges on the database or user that requires more perm space).
Database Administration 179
Chapter 9: Managing Space: Operational DBAs
Increasing the Spool Space Limit for a User
•
Enter the space in megabytes and the name of the user or database that directly owns the perm space being transferred.
The system generates a MODIFY DATABASE or MODIFY USER statement that reallocates the space.
Increasing the Spool Space Limit for a User
Specify a new value for SPOOL in a MODIFY PROFILE or MODIFY USER statement, depending on where the user spool is defined, for example:
MODIFY [PROFILE profile_name | USER user_name ] AS
SPOOL = bytes ;
Finding and Fixing Skewed Tables Using
Viewpoint
The DBC.TableSizeV contains information that can help identify skewed tables. Table skew results from an inappropriate primary index. A query that references a skewed table may try to process more rows on some AMPs than others, and may run out of space:
1
•
SELECT statements may run out of spool space if the accessed object has a defined spool space limit.
•
INSERT and UPDATE statements may run out of perm space.
Use the following SQL statement to find skewed tables in the DBC.TableSizeV:
SELECT vproc AS
"AMP", TableName (FORMAT 'X(20)'), CurrentPerm
FROM DBC.TableSizeV
WHERE DatabaseName = 'database'
ORDER BY TableName, "AMP" ;
1
2
3
1
2
3
0
AMP
---
0
TableName
--------------------employee_upi_onempid employee_upi_onempid employee_upi_onempid employee_upi_onempid employee_nupi_ondept employee_nupi_ondept employee_nupi_ondept employee_nupi_ondept
CurrentPerm
------------
18,944
18,944
18,944
19,968
4,096
30,208
15,360
12,288
180 Database Administration
Chapter 9: Managing Space: Operational DBAs
Finding Skewed Tables by Querying the TableSizeV View
2
In this example, the answer set displays space allocations by AMP for two tables. The results show that:
•
CurrentPerm is similar across all vprocs for employee_upi_onempid. Permanent space distribution is relatively even across all AMPs in the system.
•
The table Employee_nupi_ondept is poorly distributed. The CurrentPerm figures range from 4,096 bytes to 30,208 bytes on different AMPs.
Redefine the primary index for any skewed tables that you find. See “Choosing a Primary
Finding Skewed Tables by Querying the
TableSizeV View
In this example, the SELECT request looks for poorly distributed tables by displaying the average, minimum, and maximum of the CurrentPerm figures allocated on each AMP to every table in the USER database. Each table is reported separately, ordered by name.
SELECT
TableName (FORMAT 'X(20)'),
MIN(CurrentPerm) AS "AMP Minimum",
AVG(CurrentPerm) AS "AMP Average",
MAX(CurrentPerm) AS "AMP Maximum"
FROM DBC.TableSizeV
WHERE DatabaseName = 'USER'
GROUP BY TableName
ORDER BY TableName;
TableName
-------------------employee_nupi_ondept employee_upi_onempid
AMP Minimum
-----------
4,096
18,944
AMP Average
------------
15,488
19,200
AMP Maximum
------------
30,208
19,968
The result displays two tables. Notice the results show that:
•
The table Employee_upi_onempid is evenly distributed. CurrentPerm is similar across all vprocs (the minimum and maximum are close to the average). Permanent space is relatively evenly distributed across all AMPs in the system.
•
The table Employee_nupi_ondept is poorly distributed. The CurrentPerm figures range from a minimum of 4,096 bytes to a maximum of 30,208 bytes on different AMPs, indicating a wide variance from the average.
Defining Temporary Space Limits
While some amount of permanent space is used to store the definition of a global temporary table, temporary space is used to hold rows of materialized global temporary tables.
Temporary space is allocated at the database or user level, but not the table level.
Database Administration 181
Chapter 9: Managing Space: Operational DBAs
Defining Temporary Space Limits
Define a temporary space limit with the TEMPORARY parameter of a CREATE/ MODIFY
PROFILE or CREATE/MODIFY USER/DATABASE statement.
Note: A profile definition overrides any user definition, it does not append settings to the definition.
The maximum and default limits for temporary allocation are determined as described in the following table.
IF you …
specify TEMPORARY in a CREATE/
MODIFY USER/
DATABASE statement do not specify a
TEMPORARY limit in a CREATE/
MODIFY USER/
DATABASE statement
AND a profile…
does not apply applies does not apply applies applies but the
SPOOL parameter is
NULL or NONE
THEN the limit is inherited from the...
immediate owner and may not exceed the limit of the immediate owner.
user who submitted the CREATE/MODIFY
PROFILE statement. The limit may not exceed the limit of that user. The limit is determined as follows:
•
If that user has a profile, the system uses the limit in the profile.
•
If a profile does not apply to that user, the system uses the limit in the CREATE/MODIFY
USER statement for that user.
•
If no TEMPORARY is defined for that user, the system uses the limit of the immediate owner.
immediate owner of the user.
profile specification.
specification for the user who submitted the
CREATE/MODIFY PROFILE statement, determined as follows:
•
If that user has a profile, the profile specification.
•
If a profile does not apply to that user, the specification in the CREATE/MODIFY USER statement for that user.
•
If no TEMPORARY is defined for that user, the limit of the immediate owning database or user.
When using the CREATE USER, CREATE DATABASE, or CREATE PROFILE statements to assign temporary space limits, keep your space limits in mind. Query the DBC.DiskSpaceV view to find the system levels for temporary space.
182 Database Administration
Chapter 9: Managing Space: Operational DBAs
Protecting Transactions by Reserving Cylinders for Perm Space
The following table describes the different types of temporary space.
Temporary Space
Level
CURRENTTEMP
PEAKTEMP
MAXTEMP
Description
This is the amount of space currently in use by Global Temporary Tables.
Note: This does not include the amount of permanent space required for the table header stored on each AMP for the base global temporary table definition.
This is the maximum temporary space used since the last session.
Temporary space is released when the session terminates.
MaxTemp specifies the limit of space available for global temporary table rows.
Note: This does not include the amount of permanent space required for the table header stored on each AMP for the base global temporary table definition.
The value may not exceed the limit of the:
•
Creator or modifier of the profile, when setting TEMPORARY in a profile
•
Immediate owner of the user being created or modified, if a profile does not apply
If you do not specify a value and the user is associated with a profile,
MaxTemp defaults to the value of the profile, if defined. If the profile
TEMPORARY is set to NULL or NONE, or the user is not associated with a profile, MaxTemp defaults to the value of the parent of the user.
Protecting Transactions by Reserving Cylinders for Perm Space
A transaction is aborted when it requires more space than is currently available for the requesting user. When you create a user or database, the system requests space on cylinders from the pool of free cylinders. To protect transactions, you can specify that a number of cylinders be reserved for transactions needing permanent space.
The number of free cylinders to keep in reserve for requests needing permanent space is determined by the File System field named “Cylinders Saved for PERM” in DBS Control (see
“Cylinders Saved for PERM” in Utilities). When a statement requires space, this field determines allocation as described in the following table.
Database Administration 183
Chapter 9: Managing Space: Operational DBAs
Creating a Macro for Space Usage Reporting
IF the statement needs …
permanent space spool space
AND there are…
one or more free cylinders less than one free cylinder more free cylinders than the amount specified in Cylinders Saved for PERM fewer free cylinders than the amount specified in Cylinders Saved for PERM
THEN the statement …
succeeds.
fails with a disk full error.
succeeds.
fails with a disk full error.
This means that requests needing SPOOL space might fail more often than requests needing
PERM space. This can be advantageous because failure of a request for spool space rarely involves a rollback.
To monitor and reallocate permanent disk space, you can use Teradata Viewpoint.
Creating a Macro for Space Usage Reporting
You can create a like the one shown below.
Example
The following macro shows space used.
CREATE MACRO superspace as (
SELECT databasename, SUM(maxperm), SUM(currentperm)
FROM DBC.DiskSpaceV
GROUP BY databasename
ORDER BY databasename
WITH SUM(maxperm), sum(currentperm);
);
When executed, the macro returns the following results.
execute superspace;
*** Query completed. 28 rows found. 3 columns returned.
*** Total elapsed time was 1 second.
DatabaseName Sum(MaxPerm) Sum(CurrentPerm)
-----------------------------------------------------------------macro
All 0 0 cliuser
CONSOLE
Crashdumps
CustomerTest
10,000,000
100,000
1,024,000,000
5,120,000,000
7,168
0
0
100,082,688
CUSTOMER_SERVICE dash
DBC
10,000,000
20,000,000
24,316,695,812
5,632
9,216
89,080,832
184 Database Administration
Chapter 9: Managing Space: Operational DBAs
Viewing Space Usage Trends for Databases and Users
Default
Test_User explainsample
EXTUSER mkt_bsktTest
PDTBASE
PUBLIC qttest
SQLJ
SysAdmin
SYSLIB
SYSSPATIAL
SystemFe
SYSUDTLIB
Sys_Calendar
TDPUSER testdb testuser udtuser
Sum(MaxPerm)
0
20,000,000
1,000,000
0
1,280,000,000
10,000,000
0
500,000,000
600,000,000
40,000,000
10,000,000
110,000,000
60,000,000
100,000,000
15,000,000
0
4,096
12,288
0
1,108,992
33,792
0
10,240
0
1,693,696
12,288
2,931,712
207,872
0
2,654,208
0
800,000
20,000,000
10,000,000
0
133,120
9,216
0
-------------------------------------
33,277,595,812 197,997,056
For the latest example queries for obtaining space usage information, see Knowledge Article
CA9001181E6 located in the Teradata Database Knowledge Repositories. This article is available through Teradata @ Your Service. Sign up for an account at http://tays.teradata.com
.
Viewing Space Usage Trends for Databases and
Users
The Capacity Heatmap portlet displays many of the same parameters as the Space Usage portlet, but allows you to see space usage trends over several months. A review of space usage trends in Capacity Heatmap can be useful when planning and scheduling batch jobs or other large space-using tasks, or when estimating growth in system disk requirements.
Other Ways to View Space Utilization
Teradata Administrator
You can use the Database menu in Teradata Administrator to view space usage. Select Child
Space to see the space usage for all child databases of the selected database.Select Table Space to see the space usage for all tables of the selected database.
You can also use the Object menu in Teradata Administrator to view space usage. Use Space
Summary to see current and perm peak usage of the specified table. Use Space by AMP to see current and perm peak usage of the specified table by AMP.
Teradata Administrator functions are also available through Teradata Studio.
Database Administration 185
Chapter 9: Managing Space: Operational DBAs
Managing Data Blocks
The Ferret Utility
You can also use the system utility, Ferret, which is started via the Database Console, to see space usage. Use the ShowSpace command to view space usage (perm, spool, and temporary) at the system level. Use the ShowBlocks command to view the allocation of permanent space at the table and subtable level.
The ShowBlocks command can provide a level of detail that is not available with
DBC.TableSize and Teradata Administrator. For example, ShowBlocks can answer “How much perm space is used for a secondary index?” ShowBlocks also provides subtable space information. You can multiply the typical block size times the number of blocks to determine subtable space usage.
SQL Macros
A set of File System macros can retrieve Ferret ShowBlocks data into a table for access via SQL requests. The PopulateFsysInfoTable macro retrieves data to a table created by the
CreateFsysInfoTable macro. ANSI versions of these macros are also available. For more information, see the chapter on file system information macros in SQL Functions, Operators,
Expressions, and Predicates.
Managing Data Blocks
A data block is the unit of physical I/O when Teradata Database writes data to disk. A data block contains one or more data rows from the same table. The system stores data blocks in segments, which are grouped in cylinders. You can set data block sizes for tables using SQL or set them globally using the DBS Control utility. If there is a conflict between the data block size value set for the system and for the table, the table value overrides the global value.
Setting Data Block Sizes Using SQL
Set data block sizes for tables using the CREATE TABLE or ALTER TABLE statement. The
DATABLOCKSIZE = n [BYTES/KBYTES/ KILOBYTES] specification allows you to define n as the maximum multi-row data block size used for storing rows of this table.
Teradata recommends that:
IF...
most queries to a table access a single row of data most queries to the table access data across the entire table
THEN...
define smaller data blocks for the table to enhance row retrieval efficiency.
define larger data blocks for the table to reduce the number of I/Os required to read the table.
186 Database Administration
Chapter 9: Managing Space: Operational DBAs
Managing Data Blocks
Setting Data Block Sizes Using the DBS Control Utility
You can set data block sizes globally using these DBSControl fields: PermDBAllocUnit,
PermDBSize, and JournalDBSize. These fields determine the maximum size of permanent data blocks that hold multiple rows. (A row that exceeds the size of a multirow data block is put into a block of its own.)
Example
To change the PermDBSize, you need to use the modify and write commands of the DBS Control utility. The following example changes the data block size to 254 sectors:
1
From the command-line prompt (or the Supervisor window of the Database Window):
2
3
4
dbscontrol
From the DBS Control window: display FILESYS
Use the modify command to change flag 3 (PermDBSize): modify FILESYS 3 = 254
Write the changes to the GDO: write
Note: The default and minimum data block size depends on the cylinder size your site uses.
Cylinder size is a global value, shown as:
•
Number of Disk Blocks Per Cylinder, using the PUT utility
•
SectsPerCyl, using the Filer utility FIB or SEGZERO command
Note: By default, Filer returns output from every vproc.
Setting Data Block Merging Limits
As part of full-table modify operations, where the system sequentially scans the table as part of the operation, Teradata Database can merge small DBs into larger ones as the modifications are performed. While this can make some of the initial modifications to the table more expensive, it makes future full table modifications and queries cheaper by reducing the number of DBs in the table, which reduces disk I/Os.
You can control the frequency of merges and the size of resulting data blocks by:
•
Setting the system-level MergeBlockRatio field in DBS Control.
•
Specifying the table-level MergeBlockRatio attribute in the CREATE TABLE or ALTER
TABLE statement.
Note: The table-level attribute overrides the system-level setting for any table that has the attribute. However, if the Disable Merge Blocks field in DBS Control is enabled, both the table-level attribute and the system-level setting will be ignored.
MergeBlockRatio settings allow you to make adjustments so that merged data blocks end up at a reasonable size (that is, the block does not split again quickly after it undergoes a merge).
To consider the right MergeBlockRatio setting, you must take into account the following:
•
How often you expect to access or modify certain tables
Database Administration 187
Chapter 9: Managing Space: Operational DBAs
Managing Cylinders
•
How much of the data is being modified
•
The desired general size of blocks across certain tables
Managing Cylinders
Setting Free Space Percent Limits
You can control the Free Space Percent (FSP) at the global and table level as described below.
For information on FSP, see FREESPACEPERCENT option in PACKDISK or Appendix C:
“Performance Management: All DBAs.”
Parameter
FreeSpacePercent field of DBS Control
FREESPACE = n of a
CREATE/ALTER TABLE statement
DEFAULT FREESPACE of an ALTER TABLE statement
Description
Global parameter that the system uses to determine the percentage of space to leave unused on each cylinder during bulk loading operations such as MultiLoad and FastLoad. This default can be overridden at the table level with the FREESPACE clause.
Choose a percentage that reflects the net growth rate of your data tables
(INSERTs minus DELETEs). For example, most sites choose 5 to 15%.
n percent of cylinder space to remain free on each cylinder when bulk loading this table (where n is an integer constant).
Note: Do not use the Ferret utility PACKDISK command to change this value once you have created a table or have modified its FREESPACE
PERCENT with an ALTER TABLE request.
Instead, submit an ALTER TABLE request to change the free space percent value for the table and then immediately afterward submit a
PACKDISK command which will pick up the new free space percent value (see the documentation for the Ferret utility in Utilities for more information and instructions for running PACKDISK).
Reset the current free space percent for a table at the global or table level.
DEFAULT FREESPACE resets Free Space to the value defined in the
FreeSpacePercent field in DBS Control.
188 Database Administration
Chapter 9: Managing Space: Operational DBAs
Managing Cylinders
Parameter
FREESPACEPERCENT option in PACKDISK
Description
Optionally specifies the percentage of cylinder space to leave free. This allows for future table growth without requiring the allocation of new free cylinders.
FSP can be specified for individual tables by using the FREESPACE option to CREATE TABLE and ALTER TABLE.
The FREESPACE value defined for a table with CREATE TABLE takes precedence over the FREESPACEPERCENT value with PACKDISK, unless PACKDISK includes the FORCE option. In that case, PACKDISK will pack to the specified FREESPACEPERCENT, but after that, the
AutoCylPack job running in the background will eventually repack the table cylinders to the FREESPACE value that was specified in CREATE
TABLE. For information on the FORCE and PACKDISK commands, see the Ferret Utility in Utilities.
Use the FREESPACE option of ALTER TABLE to add or modify an FSP value for an existing table. For more information on the FREESPACE option, see SQL Data Definition Language.
Note: Because the AutoCylPack (automatic background cylinder packing) feature monitors and maintains proper levels of cylinder free space using AutoCylPackFSP, you only need to use the
FREESPACEPERCENT option in PACKDISK in exceptional situations.
For a list of these exceptions, see “Ferret Utility (ferret)” in Utilities.
For more information on the DBS Control utility, see Utilities.
For information on the CREATE TABLE or ALTER TABLE statements, see SQL Data
Definition Language.
Finding Loosely Packed Tables with the SHOWFSP Command
Use the SHOWFSP command of the Ferret utility before running PACKDISK to estimate the effectiveness of PACKDISK commands. SHOWFSP helps you find loosely packed tables.
The SHOWFSP command gives an approximation of the number of cylinders that can be freed by PACKDISK. It may or may not match with the exact number of cylinders that will be freed. If you feel that PACKDISK reports a very different number of cylinders freed than
SHOWFSP, then perform a defrag with the DEFRAG command.
==========================================================
Ferret ==> showfsp
Fri Jun 06, 2003 12:06:02 : showfsp will be started on All AMP vprocs
Fri Jun 06, 2003 12:06:02 : to find all the tables specified greater than 0 cylinders
Fri Jun 06, 2003 12:06:02 : which could free up more than 0 cylinders
Fri Jun 06, 2003 12:06:02 : only tables with a current FSP greater than 0%
Fri Jun 06, 2003 12:06:02 : will be considered
Do you wish to continue based upon this scope?? (Y/N) y
Fri Jun 06, 2003 12:06:04 : ShowFsp has been started
On All AMP vprocs vproc 0 (0000) response
There are 6 tables larger than 0 cylinders on amp 0
Database Table fsp Recoverable Current
Name Name % Cylinders Cylinders
----------------- --------------------------- ----- --------- ---------
CUSTOMER ppitable3 25 1 5
Database Administration 189
Chapter 9: Managing Space: Operational DBAs
Managing Cylinders vproc 1 (0001) response
There are 6 tables larger than 0 cylinders on amp 1
Database Table fsp Recoverable Current
Name Name % Cylinders Cylinders
------------------ -------------------------- ----- ----------- ---------
CUSTOMER ppitable4 34 1 5
2 of 2 vprocs responded with the above tables fitting the criteria
ShowFsp has completed
For information on the settings and options for SHOWFSP, see “Ferret Utility (ferret)” in
Utilities.
Freeing Cylinders Using PACKDISK
The PACKDISK command of the Ferret utility reconfigures the contents of an entire disk or an individual table, leaving a percentage of free space for cylinders within a scope defined by the SCOPE command. PACKDISK uses the default Free Space Percent or a new percentage defined as part of the command to pack the entire disk or a table.
The allowable scope for PACKDISK is vprocs or tables but not both.
The system automatically moves data blocks from one cylinder to another, stopping when the required number of free cylinders is available. This operation runs when the number of free cylinders is critically low. You can use the PACKDISK command to force the packing to occur earlier.
Specifying Free Space Percent
PACKDISK packs either the entire disk or a single table, leaving a specified percentage of the cylinders occupied by the object empty, to account for subsequent insert and update
operations. This is the FSP you specify using the tools described in “Setting Free Space Percent
.
Setting the PACKDISK Scope
The PACKDISK command of the Ferret utility reconfigures the contents of an entire disk or an individual table, leaving a percentage of free space for cylinders within a scope defined by the SCOPE command.
Running PACKDISK
To run PACKDISK, enter in the command window of the Ferret partition, for example:
PACKDISK FREESPACEPERCENT = 25
(where 25 equals the percentage of cylinder free space). Key the command in uppercase, lowercase, or a combination of both.
Stopping PACKDISK
To stop PACKDISK, enter ABORT.
Packing applies only to entire logical cylinders, not to the space inside individual data blocks within those cylinders. Data block sizes are the same before and after the PACKDISK
operation. Also see “Managing Free Space” on page 524
.
190 Database Administration
Database Administration
Chapter 9: Managing Space: Operational DBAs
Managing Cylinders
Defragmenting Cylinders Using Defragment
Cylinders can become defragmented over time. Use the Ferret utility DEFRAGMENT command to defragment the cylinders on either an AMP or the system, depending on the
SCOPE options.
About AutoCylPack and MiniCylPack
AutoCylPack
AutoCylPack (automatic background cylinder packing) manages space on cylinders, finding cylinders with low or high utilization and returning them to their user-defined FSP (Free
Space Percent), that is, the percentage of storage space within a cylinder that AutoCylPack leaves free of data to allow for future table growth.
There is a system wide default for AutoCylPack. It can be overridden on a table-by-table basis by specifying the FSP clause in a CREATE TABLE or ALTER TABLE statement.
Although AutoCylPack runs as a background task issuing I/Os, you can adjust its level of impact. For the DBS Control fields that support AutoCylPack, see Utilities.
Performance
AutoCylPack helps reduce:
•
The chances that MiniCylPacks run. MiniCylPacks are strictly concerned with reclaiming whole cylinders.
•
The need for you to perform regular Ferret PACKDISK operations.
PACKDISK can be performed on a table, a range of tables or an entire system, but it affects performance of the foreground tasks and thus system performance.
If cylinders have higher utilization:
•
System performance improves because there is a higher data block to cylinder index ratio and more effective use of Cylinder Read. There will be less unoccupied sectors read in.
•
A table occupies less cylinders. This leaves more cylinders available for other uses.
MiniCylPack
A MiniCylPack moves data blocks in logical sequence from cylinder to cylinder, stopping when the required number of free cylinders is available.
The process continues until one cylinder is completely emptied. The master index begins the next required MiniCylPack at the location that the last MiniCylPack completed.
Teradata Database file system will start to MiniCylPack when the number of free cylinders drops to the value set by MiniCylPackLowCylProd. The default is 10.
The File Information Block (FIB) keeps a history of the last five cylinders allocated to avoid
MiniCylPacks on them.
Note: Spool files are never cylinder packed.
Use the DBS Control (see “MiniCylPackLowCylProd” in Utilities) to specify the free cylinder threshold that causes a MiniCylPack. If the system needs a free cylinder and none are available, a MiniCylPack occurs spontaneously.
191
Chapter 9: Managing Space: Operational DBAs
Managing Cylinders
2
3
Migrating Data Blocks
1
If space can be made available either by migrating blocks forward to the next cylinder or backwards to the previous cylinder, choose the direction that would require moving the fewest blocks.
If the number of blocks is the same, choose the direction of the cylinder with the most number of free sectors.
If Step 1 fails to free the desired sectors, try migrating blocks in the other direction.
If space can be made available only by allocating a new cylinder, allocate a new cylinder.
The preference is to add a new cylinder:
•
Before the current cylinder for permanent tables.
•
After the current cylinder for spool tables and while performing FastLoads.
When migrating either forward or backward, the number of blocks may vary because the system considers different blocks for migration.
Because of the restriction on key ranges within a cylinder, the system, when migrating backward, must move tables and rows with the lowest keys. When migrating forward, the system must move tables and rows with the largest keys.
The system follows special rules for migrating blocks between cylinders to cover special uses, such as sort and restore. There are minor variations of these special rules, such as migrating more data blocks than required in anticipation of additional needs, and looking for subtable breaks on a cylinder to decide how many data blocks to attempt to migrate.
Performance
Although cylinder packing itself has a small impact on performance, it often coincides with other performance impacting conditions or events. When Teradata Database file system performs a MiniCylPack, the operation frees exactly one cylinder.
The cylinder packing operation itself runs at the priority of the user whose job needed the free cylinder. The cylinder packing operation is the last step the system can take to recover space in order to perform a write operation, and it is a signal that the system is out of space.
Needing to pack cylinders may be a temporary condition in that a query, or group of queries, with very high spool usage consumes all available free space. This is not a desirable condition.
If space is a problem,
•
Enable AutoCylPack if you have not done so and specify an FSP of 0% for read-only tables using the CREATE TABLE or ALTER TABLE statement
•
Run the Ferret PACKDISK command.
Error Codes
MiniCylPacks are a natural occurrence and serve as a warning that the system may be running short on space. Tightly packed data can encourage future cylinder allocation, which in turn triggers more MiniCylPacks.
The system logs MiniCylPacks in the Software_Event_LogV with the following error codes.
192 Database Administration
Chapter 9: Managing Space: Operational DBAs
Giving One User to Another
Code
340514100
340514200
340514300
Description
Summary of MiniCylPacks done at threshold set via the DBS Control.
A MiniCylPack occurred during processing and a task was waiting for it to complete.
The system could not free cylinders using MiniCylPack. The MiniCylPack failed.
This means that the system is either getting too full or that the free cylinder threshold is set unreasonably high. Investigate this error code immediately.
Frequent 340514200 or 340514300 messages indicate that the configuration is under stress, often from large spool file requirements on all AMPs. MiniCylPacks tend to occur across all
AMPs until spool requirements subside. This impacts all running requests.
If table data is skewed, you might see MiniCylPacks even if Teradata Database has not used up most of the disk space.
Giving One User to Another
When you give one object to another, all space that belongs to it goes with it. If you drop the object, the space goes to the immediate owner. When you give databases or users, all descendents of the object remain descendents of the object. When you give the object to new parents, the ownership of the space is transferred, but the limits remain the same.
Adjusting Perm Space Limits Using SQL
Perm space limits are easy to adjust, for example:
CREATE DATABASE Temp FROM HR AS PERM = 2000000;
GIVE Temp TO Finance;
DROP DATABASE Temp;
Adjusting Perm Space Limits Using Viewpoint
1
2
In the Space Usage portlet, highlight the database or user that requires more space.
Note: You can also define the value for spool in a profile, and then assign profile membership to a user.
Click the Add Space button and:
•
Enter the username and password of a database user that has DROP privileges
(required for use of the MODIFY statement) on the database or user that requires more perm space.
•
Enter the space in megabytes and the name of the user or database that directly owns the perm space being transferred.
The system generates a MODIFY DATABASE or MODIFY USER statement that reallocates the space.
Database Administration 193
Chapter 9: Managing Space: Operational DBAs
Giving One User to Another
194 Database Administration
CHAPTER 10
Maintaining the Database:
Operational DBAs
This chapter describes system maintenance tasks that should be done on a regular basis, including purging old log data, using Vproc Manager to manage vproc status and initialize
AMP disks, and maintaining Data Dictionary tables and logs. The chapter also describes more occasional maintenance tasks, including applying software patches and resetting peak values in the DBC.DataBaseSpace table.
About Database Maintenance Tasks
The maintenance topics that follow are of two types:
•
Periodic maintenance tasks that should be done at regular intervals
•
Occasional maintenance tasks
Managing Accumulated Log Data
About Database Logs
Teradata Database contains a number of log tables that accumulate data, either automatically or after certain features are enabled. Because log data grows over time, you must periodically purge older information to avoid using up permanent space.
Logs Description
Logs that can be purged manually or using Teradata Viewpoint. See
Teradata Viewpoint” on page 197 .
Access Log
(DBC.AccLogTbl)
Data collected as defined in a BEGIN LOGGING statement, for each privilege check of an attempted user access of a database object.
DBQ Logs:
•
DBC.DBQLSummaryTbl
•
DBC.DBQLStepTbl
•
DBC.DBQLogTbl
•
DBC.DBQLObjTbl
•
DBC.DBQLExplainTbl
•
DBC.DBQLSqlTbl
Data collected during database query logging (DBQL), if DBQL is enabled.
Database Administration 195
Chapter 10: Maintaining the Database: Operational DBAs
Managing Accumulated Log Data
Logs
Event Log
(DBC.EventLog)
ResUsage Logs:
•
DBC.ResUsageSpma
•
DBC.ResUsageSvpr
•
DBC.ResUsageShst
•
DBC.ResUsageIpma
•
DBC.ResUsageIvpr
•
DBC.ResUsageScpu
•
DBC.ResUsageSldv
•
DBC.ResUsageSawt
•
DBC.ResUsageSps
•
DBC.ResUsageSpdsk
•
DBC.ResUsageSvdsk
Description
Data collected automatically by the system for each user logon event.
ResUsage tables collect data on system resource usage.
You can enable ResUsage data collection and set the collection rate using either:
•
The ctl utility SCREEN RSS command
•
Teradata Viewpoint data collectors
The collection rate determines how quickly data accumulates and when the logs should be purged.
SWEvent Log
(DBC.SW_Event_Log)
TDWM Logs:
•
DBC.TDWMSummaryLog
•
DBC.TDWMEventLog
•
DBC.TDWMExceptionLog
The system automatically inserts rows in response to software errors and system events, for use by Teradata Customer Service.
Logs for Teradata Viewpoint workload management functions.
Logs that must be purged manually purged
In Doubt transaction Log
(DBC.InDoubtResLog)
Contains a row for each transaction where completion was in doubt.
Note: Depending on site requirements, you may need to retain older data long after it should be removed from an active log. If so, you can create backup log tables to store log history and periodically copy data from the active logs to the backup tables. You can also archive copies of the backup tables on tape for long-term reference.
Reference Information
Information on...
configuration and use of DBQL using the ctl utility SCREEN RSS command to set up ResUsage logging and log intervals setting up Teradata Viewpoint data collectors
Is available in...
Chapter 15: “Tracking Query Behavior with Database
Query Logging: Operational DBAs”
the SCREENRSS command in Chapter 8: “Control
GDO Editor (ctl), in Utilities.
“Enable the Data Collectors” in “Configure
Viewpoint to Monitor a Teradata Database,” in
Teradata Viewpoint Configuration Guide.
196 Database Administration
Chapter 10: Maintaining the Database: Operational DBAs
Managing Accumulated Log Data
Deleting Old Log Data with Teradata Viewpoint
Configure log data cleanup as follows:
1
2
From the Teradata Viewpoint main screen, select Admin>Teradata System.
To configure specific logs for clean up, and specify the cleanup interval:
a b
In the SETUP column, select Log Table Clean Up.
In the TABLES column, select a log table category for cleanup, for example, DBQ Log.
c
Use default cleanup interval shown unless you have reason to change it.
Note: Coordinate the interval with your site requirements for storing log history.
3 d
Check the Enabled box.
e
Click the Apply button to activate the cleanup
To specify the time of day that all configured cleanup takes place:
a
From the SETUP column, select Clean Up Schedule.
b
Specify a time when the log tables are not active.
Note: The clean up operation locks the log table and delays logging in the table until log data deletion is complete.
c
Check the Enabled box.
d
Click the Apply button to set the cleanup schedule.
Deleting Old Log Data Manually from System Tables
Before deleting old log data, determine whether you need to retain records of the data set for deletion. For example, while Teradata recommends that you purge log data at least every 90 to
180 days, security policy for some sites may require retention of access log data for a much longer time period.
Note: You cannot delete data that is less than 30 days old.
Use the following procedure when deleting old log data from system tables:
1
If your site requires long term retention of log data, do the following:
a b c
Create a duplicate log table in another database using the Copy Table syntax for the
CREATE TABLE statement.
CT DBC.tablename AS databasename.tablename
For further information on using the Copy Table syntax, see “Create Table” in
SQL
Data Definition Language Syntax and Examples.
Back up the table to tape storage in accordance with your site back up policy.
Recommendation: Match the log backup/deletion schedule to the amount of log data you want to maintain locally in system tables. For example, to maintain 90 days of log data in a system table do the first back up at 120 days, then delete the last 30 days of data in the log. Back up the log data every 30 days after that.
Drop the duplicate table using a DROP TABLE statement.
Database Administration 197
Chapter 10: Maintaining the Database: Operational DBAs
Managing Vproc Status and Initializing AMP Disks with Vproc Manager
2
3
Log on to SQL Administrator as DBADMIN or another administrative user with DELETE privileges on database DBC.
In the Query window, enter an SQL statement to purge old log entries. For example:
DELETE FROM DBC.object_name WHERE (Date - LogDate) > number_of_days ; where:
Syntax Element
object_name
Date
LogDate number_of_days
Description
The name of the DBC object that allows log deletions, as shown in
“About Database Logs” on page 195 . For example, DeleteAccessLogV is
the delete object for the Access Log.
SQL syntax element for the current date.
The date contained in the LogDate column of the log view based on the specified number_of_days from the current date.
The number of days of data that you want to retain in the system table; the span of days from the current date to the oldest LOGDATE to be retained.
Managing Vproc Status and Initializing AMP
Disks with Vproc Manager
The Vproc Manager utility allows you to:
•
View or change vproc states
•
Initialize and boot a specific vproc
•
Initialize the storage associated with a specific vproc
•
Force a manual Teradata Database restart
You can start Vproc Manager from the DBW Supervisor window (start vprocmanager) or from the Linux command line (vprocmanager). Valid commands are STATUS, INITVDISK,
RESTART, BOOT, SET, HELP, and QUIT.
Vproc Manager Command
STATUS
INITVDISK <vproc ID>
Description
Provides STATUS information about vprocs.
Initializes the DBS File System on a vdisk. It applies only to vprocs with a status of
NEWPROC or FATAL AMP. Valid vproc IDs are decimals ranging from 0 to 16383. Hex numbers can be used with a trailing “x.”
198 Database Administration
Chapter 10: Maintaining the Database: Operational DBAs
Maintaining Data Dictionary Tables and Logs
Vproc Manager Command
RESTART [TPA] [NODUMP | DUMP = {YES,
NO}][<RestartKind] [<comment>]
BOOT <vproc ID>
SET
Description
Forces different types of DBS restarts:
•
NODUMP (the default) indicates no system dump.
•
DUMP = YES indicates a system dump.
•
DUMP = NO is equivalent to NODUMP.
•
ValidRestartKind is either COLD or
COLDWAIT.
•
Comment specifies the reason for the restart.
Reinitialize the AMP disk before an all-table rebuild and starts the DBS partitions on the specified AMP. It applies only to vprocs with a
VprocState of FATAL and a ConfigStatus of
Down.
Sets the state of a vproc to either ONLINE,
OFFLINE, or FATAL.
Maintaining Data Dictionary Tables and Logs
Managing the size of Data Dictionary tables (sometimes called logs) can help improve performance. Teradata Database does not delete system data on its own so you must manage the logs and tables yourself. Determine what you should archive and what you should delete as needed for your site. (With respect to keeping Data Dictionary data for historical purposes, archive tables using a Backup Application Software solution.)
Purging the System Logs
The system does not automatically purge logs or the tables underlying views such as the
AccessLogV, LogOnOffV, and Software_Event_LogV views. Teradata recommends that you copy data off the DBC tables into a separate database, archive the data (as desired), and then delete information from the DBC tables. Some DBC objects to consider purging include:
•
Resource usage (ResUsage) tables that have logging enabled.
•
DBQL logs, which include all DBQL tables that have logging enabled.
(The tables DBC.DBQLRuleTbl and DBC.DBQLRuleCountTbl are not part of the log maintenance list. These tables are automatically maintained by the Teradata SQL BEGIN/
END QUERY LOGGING statements; an error is returned if you attempt to delete their contents.)
•
DBC.SW_Event_Log
•
QCD.DataDemographics. (If you use Query Capture Facility [QCF] with the SQL
COLLECT DEMOGRAPHICS statement, you need to explicitly delete rows from this table, DataDemographics, in your user-defined QCD databases.)
Database Administration 199
Chapter 10: Maintaining the Database: Operational DBAs
Maintaining Data Dictionary Tables and Logs
Note: Entries in DataDemographics are deleted automatically when you use the INSERT
EXPLAIN WITH STATISTICS AND DEMOGRAPHICS statement. For more information, see “Query Capture Facility” in SQL Request and Transaction Processing and
“COLLECT DEMOGRAPHICS” in SQL Data Manipulation Language.
Also, the security administrator should purge the logs associated with access logging as well as any other security-related views as needed.In addition to maintaining the size of system tables and logs, you can reduce log file sizes as follows:
•
Use only DBC.ResUsageSpma instead of activating logging on multiple ResUsage tables.
ResUsageSpma may be able to provide all the information you need.
•
Use roles to manage privileges which help reduce the size of DBC.AccessRights. For other
tips on reducing the size of the DBC.AccessRights table, see “Housekeeping on an Ad-Hoc
•
Track only the information you need for DBQL. The DBQLogTbl may be able to provide all the information you need.
•
Track only the information you need for accounting.
•
Use active row filter mode where possible for ResUsage tables.
For example, if you want to keep the DBC.Acctg table from growing too large, use only the
ASE variables that report the level of information you need. Using &T or &I for ASE accounting will potentially make the DBC.Acctg table very large; but using &D or &L usually has less impact on disk space and may still provide enough level of detail. For more
information, see “Enabling Account String Expansion” on page 162
.
Clearing Out Values in the DBC.Acctg Table
The DBC.Acctg table logs CPU utilization and I/O statistics for each account a user owns on each AMP.
Updates to the table are made periodically during each AMP step on each processor affected by the step. (If there are long-running steps, AMPUsage numbers show large increases periodically, instead of continuous incremental additions.) Data is collected and added to what is already in the table until you reset the counters to zero.
If you use account string expansion, ASE codes can cause the table to grow even more quickly.
You may need to update DBC.Acctg on a regular basis to clear out values and reset the accumulators (For details, see
“Enabling Account String Expansion” on page 162
). However, use caution when purging rows containing accounts with ASE codes.
Notice:
Purge obsolete rows from DBC.Acctg carefully when there is an ASE string embedded in the account string by constructing the appropriate WHERE clause. Since ASE is embedded in the
AccountName text string instead of a separate field, careful SQL coding is required.
Use the following process to first save the data, then clear out DBC.Acctg and reset the counters:
200 Database Administration
Chapter 10: Maintaining the Database: Operational DBAs
Maintaining Data Dictionary Tables and Logs
1
2
To control growth, archive the entire predefined set of system tables, using ARC with an all-AMPs dump of database DBC. (For a list of the tables that are archived, see Teradata
Archive/Recovery Utility Reference.)
Note: If you archived data using a tool other than ARC, such as FastExport or BTEQ, then
ARC cannot reload it. If you cannot use the tool that created the archive to reload it, try the BTEQ IMPORT command.
After the accumulated AMP data is successfully summarized, and archived if necessary, run the ClearAccounting macro to reset the values of the CPU and I/O columns:
EXEC DBC.ClearAccounting;
If you use ASE but are not sure which rows to purge from the DBC.Acctg:
a
Create a table similar to DBC.Acctg with a non-unique primary index, date, and time columns.
b
Populate the table with rows that are created for each respective account by capturing the sum total of the relevant accounting fields at regular intervals.
c
Run queries against the table. Entries that show no change are candidate rows to purge.
Archiving and Resetting Accumulators and Peak Values
Tracking peak and count values can help you monitor the health of your system. When you no longer need old data, you should periodically archive and reset accumulated values.
The following table lists some recommended peak values you should consider purging when no longer needed or when archived.
IF you want to reset …
peak disk space values in
DBC.DatabaseSpace
accumulators in DBC.Acctg
THEN use …
DBC.ClearPeakDisk macro the ClearAccounting macro
For instructions and examples, see …
.
.
“Resetting Use Counts” on page 237
.
count values, AccessCount and
LastAccessTimeStamp fields, in:
•
DBC.Dbase
•
DBC.TVM
•
DBC.TVFields
•
DBC.Indexes
one of these macros:
•
ClearAllDatabaseUseCount
•
ClearDatabaseUseCount
•
ClearTVMUseCount
Correcting DBC.DataBaseSpace and DBC.DBase Values
As a result of very rare types of system failures, you might need to correct inconsistencies in the system tables DBC.DataBaseSpace and DBC.DBase. Use the Update DBC and Update
Space utilities only when you need to perform these tasks.
Database Administration 201
Chapter 10: Maintaining the Database: Operational DBAs
Maintaining Data Dictionary Tables and Logs
Utility
Update DBC
Update Space
Description
Recalculates the maximum allowed values for permanent, temporary, and spool space and uses those values to update the DBC.DBase system table and the DBC.DataBaseSpace system table.
Examines the storage descriptors and recalculates the current usage of space used by:
•
A single database and its individual tables
•
All databases in a system and their individual tables
Note: Values in DBC.DBase are global values. Values in
DBC.DataBaseSpace are local AMP values, that is, the global value divided by the number of AMPs in the system.
For instructions on each utility, see Utilities. For descriptions of the system tables and the views that access them, see Data Dictionary.
Collecting Statistics on Data Dictionary Tables
If you are using custom applications or business intelligence tools, you may find it useful to collect statistics on Data Dictionary tables because these applications and tools often query the
Data Dictionary for validation or information checks in the background.
Components within the Teradata Database software use internal express requests that are optimized single-AMP queries. Statistics on Data Dictionary tables will not benefit these internal activities but will instead benefit SQL submitted directly by a user or an application.
Any potential benefit from collecting statistics will depend on the amount and complexity of the dictionary access activity. Using DBQL can help determine the benefit although it requires the DBC logon to collect against Data Dictionary tables. Before you decide whether to collect statistics on Data Dictionary tables, examine the DBQL output to determine if your applications use a lot of resources to access the Data Dictionary.
Note: Not all companies find value in collecting statistics on dictionary tables. The value depends on the frequency and complexity of the queries that access the dictionary tables, the amount of resource they use, and how long they run on your platform.
The following tables are good candidates for statistics collection:
•
DBC.AccessRights
•
DBC.DBase
•
DBC.Indexes
•
DBC.Owners
•
DBC.Profiles
•
DBC.Roles
•
DBC.RoleGrants
•
DBC.TVFields
•
DBC.TVM
202 Database Administration
Database Administration
Chapter 10: Maintaining the Database: Operational DBAs
Maintaining Data Dictionary Tables and Logs
•
DBC.UDFInfo
Only hashed dictionary tables allow statistics collections. Do not collect statistics on these non-hashed Data Dictionary tables:
•
DBC.Acctg
•
DBC.ChangedRowJournal
•
DBC.DatabaseSpace
•
DBC.LocalSessionStatusTable
•
DBC.LocalTransactionStatusTable
•
DBC.OrdSysChngTable
•
DBC.RecoveryLockTable
•
DBC.RecoveryPJTable
•
DBC.SavedTransactionStatus
•
DBC.SysRcvStatJournal
•
DBC.TransientJournal
•
DBC.UtilityLockJournalTable
For more information, see https://developer.teradata.com/blog/carrie/2010/05/if-you-re-notcollecting-statistics-on-your-dictionary-tables-do-it-now .
203
Chapter 10: Maintaining the Database: Operational DBAs
Housekeeping on an Ad-Hoc Basis
Data Dictionary Tables That Require Maintenance
Dictionary Tables To Maintain
Reset accumulators and peak values using the DBC.AMPUsage view and the ClearPeakDisk macro provided with the software.
DBC.Acctg
Resource usage by account/user
DBC.DataBaseSpace
Database and table space accounting
Teradata automatically maintains these tables, but good practices can reduce their size.
DBC.AccessRights
User rights on objects
DBC.RoleGrants
Role rights on objects
DBC.Roles
Defined roles
DBC.Accounts
Account codes by user
Archive these logging tables (if desired) and purge information 60-90 days old. Retention depends on customer requirements.
DBC.SW_Event_Log
Database console log
DBC.ResUsage
Resource monitoring tables
DBC.EventLog
Session logon/logoff history
DBC.AccLogTbl
Logged user/object events
DBC.DBQL tables
Logged user/SQL activity
NetSecPolicyLogTbl
Logs dynamic security policy audit trails
NetSecPolicyLogRuleTbl
Controls when and how dynamic security policy is logged
Purge these tables when the associated removable media is expired and overwritten.
DBC.RCEvent
Archive/recovery events
DBC.RCConfiguration
Archive/recovery config
DBC.RCMedia
VolSerial for Archive/recovery
1093-001A
Housekeeping on an Ad-Hoc Basis
You should occasionally clean out rows or reset the fields in the following tables as needed:
• DBC.AccessRights table
When a user or database is created, there is a default number of privileges added to
DBC.AccessRights. These entries should be reviewed and obsolete or redundant entries deleted as necessary. Otherwise, this table can grow very large and can greatly impact space limits negatively. This table may also potentially become skewed.
In addition, when an object is dropped, the system may do a full table scan against
DBC.AccessRights. If DBC.AccessRights is large, this can adversely affect the performance of the drop operation. The smaller this table can be made, the better.
To reduce the size of the DBC.AccessRights table:
•
Determine which users have the most entries. The following query checks for the userid/database ID that has the most rows that hash to the same value:
204 Database Administration
Database Administration
Chapter 10: Maintaining the Database: Operational DBAs
Housekeeping on an Ad-Hoc Basis
.set retlimit 10
.set ret cancel
LOCKING ROW FOR ACCESS
SELECT COUNT(*) (TITLE '# of//Entries', FORMAT 'ZZZZZZ9'),
a.userid|| a.databaseid (TITLE 'UserID|DatabaseID'),
dbase.databasenamei (TITLE 'UserName')
FROM accessrights a , dbase
WHERE a.userid = dbase.databaseid
GROUP BY 2,3 ORDER BY 1 DESC,2 ;
•
REVOKE rights at the table level and GRANT rights at the database level to reduce the number of entries.
•
Convert individual privileges to role privileges where possible.
• DBC.TSETQueryText table
If you receive a 3610 error, the query text may be saved to the DBC.TSETQueryText table for use by the Teradata Customer Support Center. After the problem is resolved, you may need to manually delete the rows where the “Handled” column is set to 1 or has a timestamp that indicates the information is no longer needed.
•
Archive and recovery tables
When associated removable media is expired and over-written, clean out the following:
Table
DBC.RCConfiguration
DBC.RCMedia
DBC.RCEvent
Purpose
archive or restore configuration
VolSerial for archive or restore archive or restore events
•
Accumulated values in Data Dictionary tables
Resetting values can help you do better resource usage analysis, and keep cleaner historical records. The following fields should be reset when necessary:
Fields
PeakPermSpace,
PeakSpoolSpace, and
PeakTempSpace in the
DBC.DataBaseSpace table.
CPUTime and DiskIO of the
DBC.AMPUsage view (which corresponds to the DBC.Acctg table).
Purpose
Contains database and table space accounting information.
To reset the values, execute the macro:
EXEC DBC.ClearPeakDisk;
Due to the importance of tracking space usage in your system, you should try to run this macro regularly. Or, you can submit an UPDATE statement yourself.
Holds resource usage data by Acct/User. To reset the values, submit the following:
UPDATE DBC.AMPUsageV
SET CPUTime = 0,
DiskIO = 0
ALL;
205
Chapter 10: Maintaining the Database: Operational DBAs
Housekeeping on an Ad-Hoc Basis
Fields
AccessCount and
LastAccessTimeStamp of the following views:
•
DBC.ColumnsV
•
DBC.DatabasesV
•
DBC.IndicesV
•
DBC.TablesV
•
DBC.UsersV
Purpose
Tracks the number of times and the last time a particular object was accessed.
To reset these fields, use one of these macros:
•
ClearAllDatabaseUseCount
•
ClearDatabaseUseCount
•
ClearTVMUseCount
Updating Software (Applying Proper Patches)
Updating your software to the most current patch levels can sometimes help eliminate potential problems. This includes software patches for both the database and the client. While it is not possible or practical to continually update software every time there is a new version, it is a good practice to keep Teradata Database, as well as other software on your system, as current as reasonably possible. It is a good idea to monitor Tech Alerts and known fixes through Teradata @ Your Service ( http://tays.teradata.com
).
If you determine from a Tech Alert that a fix should be applied to your system, either you should contact your Teradata support personnel or your customer support representative will help initiate the process for the patch or upgrade. When preparing for patching or upgrading software, keep the following in mind:
•
Determine when system down time would least impact your operations.
•
Factor in time and work required to archive data.
•
Check if the gateway version, the PDE version, and DBS version are compatible.
For more information, contact your Teradata Customer Support personnel. Or, if you have a service account, check the service website.
206 Database Administration
CHAPTER 11
Archiving, Restoring, and
Recovering Data: Operational DBAs
This chapter provides an overview of methods for archiving and restoring data in Teradata
Database and then provides best practices and troubleshooting techniques for using the
Teradata Archive and Recovery utility (ARC).
Methods of Archiving, Restoring, and
Recovering Data
To archive, restore, and recover data in Teradata Database, you can use either:
•
Teradata Data Stream Architecture, which is accessible via the Viewpoint BAR Operations portlet
•
Teradata Archive and Recovery utility (ARC)
These programs can co-exist at a customer site; however, only the program that created the archive can read it and restore it. Teradata Data Stream Architecture cannot restore an archive created by ARC and vice versa.
This chapter describes ARC. For more information, see the documentation for Teradata Data
Stream Architecture.
About the Archive and Recovery Utility
The Archive and Recovery (ARC) utility backs up and optionally restores databases and database objects (for example, tables, views, macros, and stored procedures). The ARC utility performs four major tasks:
•
Archive
•
Restore
•
Copy
•
Recovery
Archive dumps information from the Teradata system onto portable storage media. Restore reverses the archive and moves the data from the storage media back to the database. Copy copies data from one system to another. Recovery uses information stored in permanent journals to roll back or roll forward row information.
Database Administration 207
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Best Practices
Common Uses for ARC
Teradata Database includes many automatic data protection features. However, these features do not cover all types of data loss. The ARC utility provides additional data protection for these situations:
•
Loss of an AMP vdisk for no fallback tables
•
Loss of more than one AMP in a cluster
•
Failed batch processes
•
Accidentally dropped tables, views, or macros
•
User errors
•
Disaster recovery
You can use ARC to:
•
Archive a database, individual database object, or selected row partitions from Teradata
Database to an archive file.
•
Restore a database, individual database object, or selected row partitions to Teradata
Database from an archived file.
•
Restore a copied database, table, or selected row or column partitions to a different
Teradata Database.
Note: Selected row partition archive, restores, or copies are only for PPI and MLPPI.
•
Recover a database to an arbitrary checkpoint by rolling it backward or forward using before- or after-change images in a permanent journal table.
•
Copy a database, individual database object, or selected row partitions to the same or a different Teradata Database from an archived file.
•
Delete a changed-image row from a permanent journal table.
Best Practices
Recommended Archive Tasks
Requirements for how often and when to archive data tables vary from site to site. For tables that change on a regular basis, the best practice is to archive data once a week. The following section provides suggestions for archiving on a daily and weekly basis.
2
3
Daily Archive Tasks
If you have permanent journals, perform the following procedure on a daily basis:
1
Submit a CHECKPOINT WITH SAVE statement for each journal table. This appends any changes stored in the active journal subtable to the saved journal table, and initiates a new active journal subtable.
Archive each saved journal.
Delete the saved journal subtable from the saved journal.
208 Database Administration
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Best Practices
Weekly Archive Tasks
Teradata recommends that you submit an all-AMPs ARCHIVE of all data tables (or selected partitions of a PPI table) that change often, at least on a weekly basis. However, you may need to adjust how often you perform routine archiving based on the amount of data and based on how often your data changes.
Archiving the Data Dictionary
The Data Dictionary tables are maintained in the default database associated with system user
DBC. This means that as they grow, they consume DBC permanent space, which is space also required by the TJ and other internal recovery journals. To protect your system software, you should archive the DBC database when:
•
A high volume of DDL statements has significantly increased the quantity of definitions for data tables, views, macros, indexes, triggers, stored procedures, roles, or profiles since your last DBC archive.
•
You plan to purge some dictionary logs that have grown very large due to a high volume of recorded activities, queries, or statistics.
•
You plan to upgrade your Teradata Database or migrate to a different platform.
•
The user may exclude tables being modified from the archive.
Note: When archiving partitions of the Data Dictionary you cannot specify the ALL
PARTITIONS option.
Note: Embedded services function rows are not stored or copied when you execute BAR on system database TD_SYSFNLIB or on DBC dictionary tables. To reactivate embedded services functions, execute the DIPALL option.
For more information about archiving the Data Dictionary tables and a full list of the tables archived by the system, see Teradata Archive/Recovery Utility Reference.
If you are performing an archive or restore of a table or database using row-level security constraints, you must have the OVERRIDE DUMP CONSTRAINT and OVERRIDE
RESTORE CONSTRAINT privileges in addition to the normal DUMP and RESTORE privileges.
If a RESTORE of a PPI row-level security table specifies the LOG WHERE option, it includes constraint columns in the resulting error table. Security constraints are not enforced for error row insertion, but are enforced for any subsequent access to the error table.
About Using COPY or RESTORE After a Constraint Change
Notice:
Do not change or delete values or value names in a constraint name:value pair if an archived table references the constraint. Row-level security of the table will be compromised if it is restored or copied from archive. Adding new valuename:value pairs to a constraint will not compromise the security of archived tables. For more information on row-level security and altering or dropping CONSTRAINT objects, see Security Administration.
Database Administration 209
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Best Practices
Archiving and Restoring Tables That Use Block-Level Compression
Small or highly skewed tables that use block-level compression may be incorrectly archived or restored as not compressed due to random-AMP sampling for block-level compression. If the random-AMP sample checks an AMP without rows for that table, ARC assumes that the table is empty and does not compress it during the archive or restore. If a system has both compressed and uncompressed tables, a DBA needs to be aware of the criteria determining whether a table is compressed and review the uncompressed tables to see whether they are archived and restored correctly.
Archiving and Restoring Tables Using Temperature-based Compression
Tables compressed with automatic temperature-based block-level compression usually restore to the same temperature and compression status they had originally. However, this may not happen if the tables are restored onto a different system.
When you specify a query band TVSTEMPERATURE option, it overrides the original temperature (and, therefore, compression status) of the data that was archived.
To be sure that you have enough space for the restore, you can:
•
Try the restore using the same amount of space that was required on the source system.
There is a good chance that the authorized space is sufficient. Do this only if the restore is not expected to take a long time.
•
Authorize the destination database for the amount of storage required if all the data is uncompressed (or as much as possible). To determine this amount, use the Ferret command UNCOMPRESS ESTIMATE.
•
Specify a query band option of COLD if there is not enough space on the destination system to authorize the potential uncompressed size of the data. This causes all the data in the table to be restored compressed if automatic temperature-based compression is enabled for the table on both the archive and restore system. While this may compress more data than required, at least the data can be successfully restored. Over time, the frequently accessed data in the table will warm up and be decompressed.
Archiving and Restoring Tables Inside Secure Zones
Zone users can only archive or restore users or databases in the same zone. The system tables archived for user DBC include DBC.Zones and DBC.ZoneGuests. Copying zone object is not supported.
Archiving and Restoring Column-Partitioned Tables and Databases
Users can archive, restore, and copy column-partitioned tables and join indexes and databases containing those tables and join indexes.
210 Database Administration
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Best Practices
Restoring or Copying a Column-Partitioned Primary AMP Index
Table
A nonempty column-partitioned primary AMP index table is invalid when restored or copied if the target system has the following differences from the source system:
•
A different number of AMPs
•
A different hash bucket size
•
A different hash function
To make the table valid when the number of AMPS differ, modify the table to have no primary index or drop the column-partitioned primary AMP index table.
If the target system has a different configuration or if the hash bucket size changes, ARC restores or copies a nonempty table with a primary AMP index but marks the table as invalid.
In these circumstances, perform an INSERT-SELECT into a new column-partitioned table with a primary AMP index; then drop the invalid table. Alternatively, alter the table to have no primary index.
Restoring or Copying a Column-Partitioned Primary Index Table
If a column-partitioned primary index table is restored/copied to system with a different number of AMPs than the source system or if the hash bucket size changes, ARC redistributes the physical rows based on the hash maps of the target system.
A nonempty column-partitioned primary index table is invalid if it is restored or copied and the target system hash function changes. In these circumstances, perform an INSERT-SELECT into a new column-partitioned primary index table; then drop the invalid table.
ARC does not permit the archive, restore, or copy of selected partitions of a columnpartitioned table or join index.
Selective Backup and Restore Rules
When performing a selective backup and restore on a table or database, the following rules must be met in order for the operation to be successful.
Database Administration 211
Type
Aggregate Function
JAR
External Stored Procedures
Standard Function
Trigger
Instance or Constructor
Method
Join Index
Macro
Hash Index
Stored Procedure
Queue Table
Table Function
Table
PPI Table
NoPI and Columnpartitioned Table
UDT
View
Authorization
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
N/A
Yes
Yes
Restore Same
Database
Yes
Yes
Yes
N/A
DB
Level
Yes
Yes
Yes
Yes
Yes
N/A
Object
Level
Yes
Yes
N/A
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
N/A
Yes
Yes
Copy Same
Database
Yes
Yes
No
N/A
DB
Level
Yes
Yes
Yes
Yes
No
N/A
Object
Level
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
N/A
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
No
No
N/A
Yes
Yes
Copy and Rename
Same Database
Yes
Yes
No
N/A
DB
Level
Yes
Yes
No
No
No
N/A
Object
Level
No
No
Yes
No
Yes
Yes
Yes
No
No
No
No
N/A
No
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
N/A
Yes
Yes
Copy Different
Database
Yes
Yes
No
N/A
DB
Level
Yes
Yes
Yes
Yes
No
N/A
Object
Level
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
N/A
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
No
No
N/A
Yes
Yes
Copy and Rename
Different Database
No
No
No
N/A
DB
Level
No
No
No
No
No
N/A
Object
Level
No
No
Yes
Yes
Yes
Yes
Yes
No
No
No
No
N/A
No
No
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
•
Although an operation may be allowed, there is no guarantee the object will function as desired. For example, the copied object may have a dependency on another database object that is not present.
•
For some copy operations, certain dependencies must be met before the operation can be successful. For example, if you are copying a database with a journal table, then the destination database must be created with a journal table.
SYSUDTLIB System Database
Methods and UDTs are stored in the system database, SYSUDTLIB. Database SYSUDTLIB is backed up or restored as part of a DBC backup.
Teradata Database handles SYSUDTLIB separately from DBC in the following instances:
•
When you copy the contents of SYSUDTLIB to another database using COPY FROM.
DBC is not included in this copy. You cannot specify SYSUDTLIB as the target database.
•
When you delete all objects in the system (except for permanent journal tables) so that
DBC can be restored:
DELETE DATABASE (DBC) ALL, EXCLUDE (DBC);
Troubleshooting
Checking for Lock Contentions Blocking ARC Operations
To determine whether an ARC operation is being blocked due to lock contention, check the
ARC log. (This log is usually located in the same location as your backup application.
However, some application configuration options may have relocated the logs.)
If it is not clear from the ARC logs whether there is a lock contention on an object, do the following:
•
Use the Show Locks (showlocks) utility to check for HUT lock contentions.
If the ARC log or the Show Locks output shows that HUT lock contentions exist, use the
RELEASE LOCK statement to release the locks. Log on to the ARCMAIN utility, get a session started and submit the following:
RELEASE LOCK(databasename)ALL, OVERRIDE;
All databases will display with either no lock found or lock release. Then log off and try to run your job again.
•
Check for other lock contentions (from other queries or applications) using the Lock
Display (lokdisp) utility.
If Lock Display shows lock contentions, then use the Viewpoint Lock Viewer Portlet to determine the source of the lock.
For more information on Show Locks (showlocks) or Lock Display (lokdisp), see Utilities.
Database Administration 213
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
Managing Space Availability Issues
If restoring from an archive exceeds available perm space, two DBS Control utility fields can be set temporarily to allow operations to complete and avoid out of space errors. The settings apply to all operations and all instances of space accounting, such as table inserts, so Teradata recommends setting these fields to a non-zero value only temporarily. For more information, see the DBS Control utility fields OverflowMaxPermSpacePercentage and
OverflowMaxPermSpaceKByteCount in Utilities.
Archiving After Recovering a Down AMP
When an AMP is offline during an all-AMP archive, non-fallback tables may only be partially archived. You need to:
•
Issue the ARCHIVE statement after the AMP comes back online.
This AMP was down during the all-AMP archive.
•
Perform a single-AMP backup.
•
Choose NO FALLBACK TABLES, to obtain a complete backup. (Fallback tables are always completely archived even if an AMP is down, because there is either a primary or fallback copy of the data on another AMP.)
Archiving or Restoring a Down Table
During an archive operation, the system checks the table header to verify if it contains any down status or down region information. If there is a down subtable or down region, it reports an error to ARCMAIN.
When archiving a table that is marked down or contains down regions, ARCMAIN either aborts the archive operation if it is a table-level archive, or skips the table and proceeds with the next object if it is a database-level archive operation. (In other words, in a multitable archive operation such as a database, cluster, or AMP archive, when the system encounters an error of down subtable or down region, ARCMAIN skips the table and proceeds with the next object.)
If you encounter an error message from ARCMAIN about a down region, do one of the following:
•
Run CheckTable utility at level 3 to identify possible table corruption.
•
Repair the down regions or reset the down status of the table or index.
•
Restore the down table or index status before running the archive operation again.
Then use the ALTER TABLE tablename RESET DOWN statement, and the system will delete the down regions of that table from the table header on each AMP.
Recovering Data
Some important concepts regarding data recovery operations are:
•
As with archive or restore operations, you use the ARC utility with permanent journals
(PJs) for data recovery operations.
214 Database Administration
Database Administration
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
•
The CHECKPOINT statement indicates a recovery point in a journal.
•
The CHECKPOINT WITH SAVE statement saves stored images before a row marker in an active subtable and appends them to the saved subtable.
•
ROLLBACK or ROLLFORWARD operations, which involve PJs, can use either current journals (active and saved subtables) or restored journals (restored subtable). ROLLBACK commands help you recover from one or more transaction errors and reverses changes made to a database or table.
•
ROLLFORWARD commands help you recover from hardware errors. These commands replace existing row data with after-change images.
•
DELETE JOURNAL command erases the contents of either the restored subtable or the saved subtable in the PJ.
•
There are several recovery control system views that contain information about ARC utility events.
•
You can cancel rollback if you decide it is quicker to restore the tables instead.
Data Recovery Using Roll Operations
The restore statement allows you to move information from archive files back to Teradata
Database. The restore operation can restore data or journal tables. When you restore a journal table, the system restores the information to a permanent journal subtable. Before you can use the tables, you must perform a rollback or rollforward operation to move the journal tables back to the data tables.
Roll operations can use either the current journal or the restored journal. If you specify the current journal, then the ARC utility uses information stored in both the active and saved subtables.
A permanent journal is checkpoint-oriented rather than transaction-oriented. The goal of the journals is to return existing data tables to some previous or subsequent checkpoint. For example, if a batch program corrupted existing data, the rollback operation would return the data to a checkpoint prior to the running of the batch job.A rollforward operation might occur after an all-AMP restore. After you move the data and journal archive files back to the database, the data tables would only include changes committed since the last full backup. Any intermediate changes would reside in the journal tables. The rollforward operation would replace the existing data with changes from the journal table.
In summary:
•
The RESTORE function copies journal archive files to the restored subtable of the PJ.
•
ROLLBACK and ROLLFORWARD statements apply journal table contents to data tables.
•
Roll operations can use either Current journal (active and saved subtables) or Restored journal (restored subtable).
CHECKPOINT Statement
Use the CHECKPOINT statement to indicate a recovery point in the Journal. The
CHECKPOINT statement places a marker row after the most recent change image row in the active subtable of a permanent journal.
215
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
Teradata Database assigns an event number to the marker row and returns the number in response. You may assign a name to the CHECKPOINT statement rather than use the event number in subsequent ARC activities.
The following table describes the options to the CHECKPOINT statement.
CHECKPOINT
Option Description
NAMED chkptname Checkpoint names may be up to 120 characters and 512 bytes long and are not case-specific. Teradata Database always supplies an event number for each checkpoint. Use the number to reference a checkpoint if a name is not supplied.
If there are duplicate checkpoint names in the journal and an event number is not specified:
•
Rollforward uses the first (oldest) occurrence.
•
Rollback uses the last (latest) occurrence.
USE LOCK
WITH SAVE
By default, the system acquires a read lock on all tables assigned to the journal being checkpointed. A checkpoint with save may optionally use an access lock.
The read lock suspends update activity for all data tables that might write changes to the journal table during checkpoint. This lock provides a clean point on the journal.
The access lock accepts all transactions that insert change images to the journal, but it treats them as though they were submitted after the checkpoint was written.
The access lock option requires that you also use the WITH SAVE option.A checkpoint with save under an access lock is only useful for coordinating rollforward activities from the restored journal, and then from the current journal.
The WITH SAVE option logically moves the contents of the active subtables of the identified journals to the saved subtable.
After you archive the saved area of the journal, you can delete this section of the current journal to make space for subsequent saved journal images.
The database automatically initiates a new active subtable. You can archive the contents of the saved subtable to an archive file.
Starting Rollbacks
Use the ROLLBACK statement to recover from one or more transaction errors. To use this statement, you must define the table with a before-image journal table. The ROLLBACK is performed to a checkpoint or to the beginning of the current or restored journal.
The system uses the before images to replace any changes made to the table or database since a particular checkpoint was taken.
The ROLLBACK command helps you recover from one or more transaction errors. It reverses changes made to a database or table. To accomplish this reversal, it replaces existing data table rows with before-change images stored in a permanent journal.
216 Database Administration
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
The before-change images must reside in either the restored or current subtables of a permanent journal. If you choose the current subtable for rollback procedures, the database uses the contents of both the active and saved subtables.
When you use the restored subtable for rollback procedures, you need to verify that it contains the desired journal table. If it does not, submit the RESTORE JOURNAL TABLE command with the appropriate removable storage media. This process ensures that you restore the correct subtable contents. Teradata Database does not have any simple tools for looking at journal subtables to determine that they contain the desired data.
Checkpoint names need to match existing names used with a previous CHECKPOINT statement. An eventno is the software-supplied event number of a previous checkpoint. You can supply either checkpoint names or event numbers or both. To find the checkpoint names or event numbers, select information about the checkpoint from the DBC.EventsV view.
If there are duplicate checkpoint names in the journal and an event number is not supplied, rollback stops at the first one encountered with a matching name.
The following illustrates a rollback procedure:
1
2
3
First, activate the ROLLBACK CURRENT JOURNAL statement to rollback any changes made since the journal table was archived. This statement rolls back the saved subtable first followed by the active subtable.
Next, run the RESTORE JOURNAL TABLE command to load the appropriate archive file into the restored subtable of the permanent journal.
Finally, submit the ROLLBACK RESTORED JOURNAL command to reverse the changes by replacing any changed rows with their before-image rows stored in the restored journal.
Repeat Steps 2 and 3 as necessary.
By default, the rollback procedure automatically deletes the contents of the restored subtable after successfully completing the command. The NO DELETE option overrides the default and has the following benefits:
•
Overrides automatic deletion of restored subtables, allowing you to:
•
Recover selected tables first
•
Later recover other tables that may have changes in the journal
•
Is used only for restored subtables
•
Is never used with current subtables
For information on the performance implications for rollbacks, see
Transaction Rollbacks” on page 503 .
Canceling Rollbacks
With the help of Recovery Manager utility, you can cancel rollbacks that are taking a long time to complete. Canceling rollbacks allows you to make tables available more quickly if the time it takes to restore specific tables is less than waiting for the rollback to complete on those tables.
Rollbacks usually take up twice as long as it would take for the runtime of a job. Rollback on tables with USIs take longer. Rollbacks on tables with NUSIs take even longer. If multiple
Database Administration 217
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting tables are involved with a complex join, rollbacks could take more than just a few hours.
However, be aware that canceling a rollback leaves the table in an unknown and unstable state if inserts or modifications were performed.
If you want to stop the rollback of a table because of an error or due to a system restart, use the
Recovery Manager utility to abort the rollback, delete the table, and then reload it from backup if needed.
Note: Before you cancel a rollback, you must first list the rollback tables. Then cancel the rollback on tables from the list. These two steps must be done in the same Recovery Manager session.
The Recovery Manager utility allows you to:
•
Cancel the rollback of tables during a system restart or an aborted, online transaction.
•
List the status of the tables being rolled back in the system.
•
Change the priority levels for rollbacks in user-aborted transactions.
For example, a typical process might be:
1
2
3
4
You notice that a rollback is taking too long.
You identify if a large table, or perhaps several tables, can be restored more quickly than the rollback will take.
If canceling rollback is quicker than waiting for rollback to complete, decide which tables to cancel rollback. Submit the LIST ROLLBACK TABLES statement first and then cancel rollback on the tables in that list.
Immediately do a DELETE ALL and restore the table if you need that table restored. If you do not need the table, be sure to still do the DELETE ALL operation.
218 Database Administration
Database Administration
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
The following table lists some of the Recovery Manager utility commands involved with canceling rollbacks.
Use this RcvManager command…
CANCEL
ROLLBACK ON
TABLE
LIST CANCEL
ROLLBACK TABLES
LIST ROLLBACK
TABLES
To…
cancel rollback on one or more tables. Use the command after running the
LIST ROLLBACK TABLES command and use this command only when:
•
The rollback of a table is likely to take longer than its restoration.
•
The table, such as a temporary table, is non-critical. Some things to note:
•
You must have the DBC password to execute this command.
•
The priority of the cancel is the same as the priority of the rollback.
•
CANCEL ROLLBACK command is not instantaneous. The system must still read all of the Transient Journal rows for the transaction even if it does not apply most of them, or at least the rows on the table on which rollback has been canceled.
•
You cannot cancel rollback on the referenced or referencing tables of
RI unless they are self-references.
You can perform the following operations on a table for which rollback has been canceled:
•
ARC DUMP - Only the table header is archived, not the data in the table. Then, you retain the DDL without having to recreate the table.
•
ARC RESTORE - Restore the table from archive using ARC. The table is made usable for update operations after it is restored from backup.
•
CheckTable - Skip the tables specified with the CANCEL ROLLBACK
ON TABLE command.
•
DELETE ALL - Delete all the rows in a table and make the table valid again.
•
DROP TABLE - Drop the table so you can create it again.
•
LOCKING…with READ OVERRIDE option - Retrieve a single table for
SELECT operations.
list all the tables for which rollback is pending cancellation as a part of an online user requested abort or during system recovery.
When rollback on a table has been canceled, the table is marked not valid.
The not valid tables do not appear on the list generated by the LIST
CANCEL ROLLBACK TABLES command. Therefore, it is highly recommended that you perform a DELETE ALL on the table after canceling rollback on it.
display all the tables that are currently undergoing rollback in the system.
The list is sorted in descending order of the TJ Row Count column.
This command displays records for the tables in a transaction having more than 10,000 rows to rollback on at least one AMP. The transaction must be in abort status.
An asterisk (*) follows the tables which cannot be specified in the CANCEL
ROLLBACK command because they are RI tables.
Note: DBC tables and Permanent Journals are not listed.
219
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
Use this RcvManager command…
LOCKING TABLE …
FOR READ
OVERRIDE
ROLLBACK
SESSION …
PERFORMANCE
GROUP
To…
perform a single table retrieve operation on a table for which rollback has been canceled using the CANCEL ROLLBACK ON TABLE command.
The tables for which rollback has been canceled can be viewed through this locking modifier option.
display or set the performance group setting of the rollback for the specified session. This command allows you to change the priority. By reducing the priority of less important rollbacks, you can increase the system resource availability for other processes in the system.
Rollback processing runs at the priority defined for the session, that is, the priority associated with the Performance Group for the session. If no priority is specified, rollback runs at RUSH priority by default.
For more commands, usage notes, and syntax diagrams for Recovery Manager, see Utilities.
Recovering Data with Rollforward
The ROLLFORWARD command helps you recover from a hardware error. It changes existing rows in data tables by replacing them with after-change images stored in a permanent journal.
The after-change images must reside in either the restored or current subtables of a permanent journal.
When you use the restored subtable for rollforward procedures, you need to verify that it contains the desired journal table.
If it does not, submit the RESTORE JOURNAL TABLE command with the appropriate portable storage media. This ensures that you restore the correct subtable.
Also, before you can rollforward, you must have a backup copy of the table rows and AFTER
Image journal rows since the last backup.
220 Database Administration
Database Administration
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
The following table lists descriptions of some of the options.
ROLLFORWARD Option Description
PRIMARY DATA During a rollforward operation, this option instructs the software to ignore secondary index and fallback row updates.
A BUILD operation rebuilds the invalidated fallback copy and indexes.
TO checkpointname, eventno
Checkpoint names need to match existing names used with a previous
CHECKPOINT statement.
An event number is the software-supplied event number of a previous checkpoint. You can supply either one or both of these.
To find the checkpoint names or event numbers, select information about the checkpoint from the DBC.EventsV view.
If there are duplicate checkpoint names in the journal and an event number is not supplied, rollback stops when it encounters with a matching name.
The following illustrates a rollforward procedure:
1
2
3
4
Submit the RESTORE DATA TABLE command.
Submit the RESTORE JOURNAL TABLE command to load the appropriate archive files into the restored permanent journal subtable.
Submit the ROLLFORWARD RESTORED JOURNAL command to replace existing data table rows with their after-image rows stored in the restored journal.
Submit the ROLLFORWARD CURRENT JOURNAL statement to rollforward any changes made since the journal table was archived. This statement rolled forward the saved subtable first followed by the active subtable.
PRIMARY DATA Option
This option replaces only primary row images during the rollforward process. It ignores secondary index.
If you use this option with a rollforward operation, you can reduce the amount of I/O. It also improves the rollforward performance when recovering a specific AMP from disk failure.
Unique indexes are not valid when recovering a specific AMP. Always submit a BUILD statement when the rollforward command includes the PRIMARY DATA option.
Therefore, the PRIMARY DATA Option:
•
Ignores secondary index rows
•
Reduces amount of I/O
•
Improves performance when recovering single-AMP
ROLLFORWARD Restrictions
The following table illustrates important restrictions on using the ROLLFORWARD statement.
221
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
ROLLFORWARD Restriction
AMP-specific Restore
All-AMP Restore
Description
If you perform a restore operation on a specific AMP rather than on all AMPs, the ROLLFORWARD command does not permit you to use the TO CHECKPOINT NAME option.
Following an AMP-specific restore, the system permits a rollforward only to the end of the journal.
You must follow up the restore process with a rollforward of the entire journal table.
When you perform an all-AMP restore, you choose whether to submit the ROLLFORWARD command with the TO
CHECKPOINT NAME option, or to the end of the journal.
The PRIMARY DATA option of the ROLLFORWARD statement indicates that the operation should ignore secondary index and fallback rows that will reduce the amount of I/O during rollforward.
If you use this option, follow up with the BUILD statement.
Use the DBC.EventsV view to determine event numbers and checkpoint names.
For example:
SELECT EventNum FROM DBC.EventsV WHERE CreateDate = 940819;
SELECT CheckPointName FROM DBC.EventsV
WHERE CreateDate = 940819;
Using Recovery Control Views
Several system views contain information about ARC utility events. You can use these views for recovery control.
This view… Provides information about … On table…
DBC.AssociationV
DBC.EventsVX
objects (databases, users, tables, views, macros, indexes, stored procedures) that you imported from another Teradata
Database system or created via the ARC utility COPY statement archive or restore activities, with a row (audit trail) for each archive and recovery event. Each time another ARC activity begins, ARC inserts a new row in the underlying system table of the EventsV view.
DBC.DBCAssociation
DBC.RCEvent
DBC.Events_ConfigurationVX
archive and recovery activities that did NOT affect all AMPs DBC.RCConfiguration
DBC.Events_MediaVX
archive and recovery activities that involved removable media
DBC.RCMedia
222 Database Administration
Database Administration
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
Finding Information about Imported Objects with the
AssociationV View
The AssociationV view allows you to retrieve information about an object imported from another Teradata Database. An existing object created with the ARC COPY command also appears in the AssociationV view. If you later drop a copied object from its new destination, the information is deleted from the AssociationV view and is no longer available.
The following example uses the AssociationV view to list all tables, views, or macros that were copied into the payroll database. The result of the query displays imported and current table names. The object column displays the current name of each table. The Source column provides the name of the original table. The event column shows the number assigned to the restore operation.
SELECT TRIM (DatabaseName)||'.'||TableName
(NAMED Object, FORMAT 'X (25)')
TRIM (Original_DatabaseName)||'.'||
Original_TableName
(NAMED Source, FORMAT 'X(25)')
EventNum (NAMED Event, FORMAT '9(5)')
FROM DBC.AssociationV
WHERE DatabaseName LIKE %Payroll%'
ORDER BY Object ;
This query returns event rows similar to the following:
Object Source Event
---------------------------------------------------------------
Payroll_Prod.DEPARTMENT PAYROLL_TEST.department 00014
Payroll_Prod.DEPT PAYROLL_TEST.dept 00014
Payroll_Prod.EMP PAYROLL_TEST.emp 00014
Payroll_Prod.EMPLOYEE PAYROLL_TEST.employee 00014
Payroll_Prod.NEWHIRE PAYROLL_TEST.newhire 00014
Tracking ARC Activity with the EventsV View
The EventsV view tracks ARC activity. ARC inserts a new row in the EventsV system table each time another ARC activity begins. The EventsV view returns a row for each activity tracked.
The following table describes events the system creates depending on the type of object on which the activity was performed.
A row for this event type …
Checkpoint Event Row
Delete Event Row
Dump Event Row
Restore Event Row
Rollback Event Row
Rollforward Event Row
Is created for each …
journal checkpointed.
journal deleted.
database or table archived.
database or table restored.
database or table rolled back.
database or table rolled forward.
223
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
The SQL statement in the following example requests a list of all ARC activity that took place
March 28th.
SELECT EventNum
,UserName (CHAR (12))
,EventType (CHAR (12))
,DatabaseName (CHAR (15))
FROM DBC.EventsV
WHERE CreateDate=990328
ORDER BY EventNum ;
EventNum
---------
180
181
UserName
------------
BRM
RPK
EventType
---------
Dump
Restore
DatabaseName
---------------
Payroll_Test
Personnel_Test
Tracking ARC Activity on Specific AMPs with the
Events_ConfigurationV View
The Events_ConfigurationV view contains rows for each archive activity that does not affect all AMPs in the database configuration. If the ARC command specifies all AMPs and there are one or more AMPs offline, a row is inserted in the system table for each offline AMPs. If the statement is for specific AMPs, a row is inserted for each specified and online AMPs.
The following example submits an SQL statement to find out which user did not release the utility locks on processor 2. Query results show three different users: AMT, ALK, and JLR.To determine who left the utility locks on processor 2, submit the following query.
SELECT CreateTime
,EventNum
,EventType (CHAR (12))
,UserName (CHAR (12))
,vproc
FROM DBC.Events_ConfigurationV
WHERE vproc= '2' ORDER BY 2 ;
CreateTime
-----------
14:06:22
16:06:39
18:12:09
EventNum
----------
1,153
1,159
1,164
EventType
----------
Dump
Dump
Restore
UserName
--------
AMT
ALK
JLR
Vproc
-----
2
2
2
Finding Information about Removable Storage Media with the
Events_MediaV View
The Events_MediaV view provides information about ARC activities that used removable storage media such as volume serial numbers assigned to portable devices.
The following example requests the volume serial number of a restore tape. The query results show two restore operations, each with their own serial number and data set name.
224 Database Administration
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
SELECT EventNum
,EventType (CHAR (12))
,UserName (CHAR (12))
,VolSerialID
,DataSetName (CHAR (12))
FROM DBC.Events_MediaV
ORDER BY EventNum ;
EventNum
----------
79
180
EventType
-----------
Restore
Restore
UserName
----------
PJ
PJ
VolSerialID
-------------
MPC001
MPC002
DataSetName
-------------
LDR.DMP1.JNL
RAN.DMP2.JNL
Restoring a Missing DBC.Database View
If you receive the error message “object DBC.Database does not exist” when opening Teradata
Administrator or Teradata Studio, your view DBC.Database is missing. You can restore this view by running the DIPALL or DIPVIEWSV script from the DIP utility.
Notice:
Before restoring DBC.Database, make sure you have backed up everything, including DBC.
Database Administration 225
Chapter 11: Archiving, Restoring, and Recovering Data: Operational DBAs
Troubleshooting
226 Database Administration
CHAPTER 12
Managing Database Resources:
Operational DBAs
This chapter describes strategies for managing system I/O, database workload, and tracking system object use and table changes. The chapter also describes query banding and provides an overview of resource usage trend analysis.
About Managing I/O with Cylinder Read
A data block is a disk-resident structure that contains one or more rows from the same table and is the smallest I/O unit for the Teradata Database file system. Data blocks are stored in physical disk sectors or segments, which are grouped in cylinders.
Teradata Database employs Cylinder Read to enable full-file scan operations to run more efficiently by reading a list of cylinder-resident data blocks with a single I/O operation. This reduces I/O overhead from once per data block to once per cylinder. This can significantly reduce the time it takes to do a full-table scan of large tables. During installation, Cylinder
Read is enabled by default.
Note: If you notice a performance hit from Cylinder Read, try decreasing the value of the
Cylinder Read Aging Threshold control field in the Ctl DBS screen. For information on this field, see Utilities.
Cylinder Read may improve performance during operations such as:
•
Full-table scan operations under conditions such as:
•
Large select
•
Merge INSERT ... SELECT
•
Merge delete
•
Sum, average, minimum, maximum, and count aggregates
•
Joins operations that involve many data blocks, such as merge or product joins.
Cylinder reads are not invoked when there is row partitioning elimination or column partitioning elimination.
For more information, see “Managing I/O with Cylinder Read” on page 523 .
About Tracking Cylinder Read Resource Usage
To track Cylinder Read behavior if you enable resource usage logging, see the Cylinder Read
Columns in the “ResUsageSvpr Table” in Resource Usage Macros and Tables.
Database Administration 227
Chapter 12: Managing Database Resources: Operational DBAs
Managing the Database Workload with Teradata Active System Management
Managing the Database Workload with
Teradata Active System Management
The system provides a number of optional Teradata Active System Management (TASM) features that provide workload management capabilities beyond simple performance groupbased session priorities. You can use the Teradata Viewpoint Workload Designer portlet to create TASM rule sets that regulate the database workload according to wide variety of criteria.
Do the following to implement TASM:
3
4
1
2
5
6
Determine workload management needs.
Create a ruleset and define the general parameters.
Define session rules
Define filter rules.
Define throttle rules.
Define workload classifications.
Determining Workload Management Needs
Before defining TASM workload management parameters, you should evaluate how your database is used. The objective is to identify job handling and query limitation needs that cannot be met by performance group assignments and scheduling.
•
Monitor database activity using the Teradata Viewpoint Capacity Heatmap portlet to understand overall workload concurrency and possible resource contention. For information on the Capacity Heatmap portlet, see Teradata Viewpoint User Guide.
•
Use the Teradata Viewpoint Query Monitor and Lock Viewer portlets to study resource contention at various times of day, and to understand blocking and wait time patterns.
•
Optionally, use the database query logging (DBQL) logging feature to determine query performance details. For information on tracking processing behavior with DBQL, see
“BEGIN/REPLACE/END QUERY LOGGING Statements” on page 268 .
•
Optionally, monitor the ResUsageSPS table to determine resource usage by performance group. For information on the ResUsageSPS table, see Data Dictionary.
Defining a Ruleset
You must create a TASM ruleset to contain any workload management rules that you create.
You can create multiple rule sets, but only one ruleset can active at a time. For instructions on using the Workload Designer portlet, see Teradata Viewpoint User Guide.
In the Ruleset General View:
1
2
3
Name the ruleset.
Use the default Interval settings.
Use the default Blocker settings.
228 Database Administration
Database Administration
Chapter 12: Managing Database Resources: Operational DBAs
Managing the Database Workload with Teradata Active System Management
4
5
Defining Bypass rules is optional, but not recommended for initial implementation.
Note: Database user DBC bypasses all rules by default.
Create the needed classification session, filter, and throttle rules within the ruleset to manage your database workload, as shown in the topics that follow.
Classification Settings
You can set the following classification parameters for the session, filter or throttle rules you create, to narrow the effects of the rules, unless you want a rule to apply globally.
•
Source: account name, account string, username, or client IP address
•
Target: The name of a database object that the query accesses, for example, a database, table, macro, view, function name, or method name.
•
Query characteristics, for example, statement type, join type, or full table scan
•
Query band
•
Utility (source of query)
Session Rules
You can use the sessions tab in a ruleset to set limits on the:
•
Number of concurrent query sessions.
•
Number of concurrent utility sessions
•
Number of concurrent sessions for a specific utility
•
The order of precedence of utility session rules
Filter Rules
A filter rejects a query before the query starts running. You can apply filters globally or according to classification rules.
For example, you can create a filter that prohibits a specific user (source classification) from running a query with an estimated processing time of longer than 15 minutes (query characteristic classification).
For information on Ruleset Filters, see Teradata Viewpoint User Guide.
Throttle Rules
A throttle limits the number of concurrent queries in a filter classification. When the throttle limit is reached, additional affected queries go into the delay queue.
You can create throttles to limit concurrency according to throttle classification, for example:
•
Limit a specific user (source classification) to running no more than 10 queries at a time
(query session limit)
•
Limit the number of utilities (utility classification) simultaneously executing against a table (target classification)
•
Limit the number of memory-intensive functions, such as XMLQUERY, and memoryintensive SQL queries executing concurrently
229
Chapter 12: Managing Database Resources: Operational DBAs
Managing Sessions and Transactions with Query Banding
Suggested Custom Throttles
The following example throttles can be useful for most implementations.
Throttle all at 52:
•
Limits the database to execution of 52 concurrent queries
•
Places additional queries into the delay queue
•
Prevents unusually large query demand from putting the system into flow control
Throttle High, Medium, and Low PG queries with estimated processing time > 1 second:
•
Sets a limit of 30 concurrently executing H, M, or L queries that have an estimated processing time > 1 second. There is no built-in limit on shorter queries.
•
Places additional long queries into the delay queue.
•
This throttle does not apply to short queries (<1 sec), or to an R query of any size.
About Ensuring a Consistent Response Time for a Workload
On SLES 11 systems, database administrators can specify a minimum response time for a workload, causing Teradata to hold request responses until that time has passed. A minimum response time provides consistent results regardless of changing conditions and helps to set reasonable expectations for users. For example, responses may naturally be very fast in a new system, but once more users and workloads have moved to the new system, responses may take longer. Specifying a minimum response time allows a database administrator to set reasonable expectations from the beginning.
The response wait time can be set from 1 to 3,600 seconds in the Hold Query Responses tab of the Workload Details screen in the Teradata Viewpoint Workload Designer portlet. This option does not apply to tactical workloads. For more information, see Teradata Viewpoint
User Guide.
Managing Sessions and Transactions with
Query Banding
How can you identify a unit of work such as a report that consists of multiple SQL requests that span multiple tiers in the enterprise, such as a web service? Use query banding.
A query band is a set of name-value pairs assigned to a session, transaction, or profile that you can use to identify the originating source of a query. This is particularly useful for identifying specific users submitting queries from a middle-tier tool or application. Query banding information helps you answer the following types of questions:
•
Which external users are using the most resources?
Access the information stored in the QueryBand field in the DBC.DBQLogTbl table when you enable query logging. See
Chapter 15: “Tracking Query Behavior with Database Query
For example:
230 Database Administration
Database Administration
Chapter 12: Managing Database Resources: Operational DBAs
Managing Sessions and Transactions with Query Banding
Sel Username (Format 'x(10)'), queryband(Format 'x(40)'), AMPCPUTime from qrylogV where ampcputime > .154;
*** Query completed. 9 rows found. 3 columns returned.
*** Total elapsed time was 1 second.
UserName QueryBand AMPCPUTime
---------- ---------------------------------------- --------------
TWMUSER1 =S> CLUser=KShort;Dept=Sales;Job=z995; 0.188
TWMUSER18 =S> CLUser=TJuli;Dept=DMgr;Job=x1235; 0.170
TWMUSER =S> CLUser=TJuli;Dept=DMgr;Job=x1234; 0.171
TWMUSER13 =S> CLUser=BPut;Dept=Mgr;Job=q2120; 0.173
TWMUSER1 =S> CLUser=KShort;Dept=Sales;Job=z995; 0.157
TWMUSER27 =S> CLUser=KShort;Dept=Sales;Job=z996; 0.186
TWMUSER2 =S> CLUser=DThom;Dept=Manuf;Job=a100; 0.156
TWMUSER28 ? 0.158
TWMUSER =S> CLUser=DGale;Dept=Mktg;Job=m125; 0.158
BTEQ -- Enter your DBC/SQL request or BTEQ command:
You can even use query bands to determine what functional area of an application is using the most resources.
• How can I control how a workload or type of query from a specific session uses system
resources?
Set up rules using Teradata Viewpoint Workload Designer portlet and associate query bands with filter rules or define them as workload definition attributes.
• Is there a correlation of CPU time by region?
Using Teradata Workload Analyzer, you can get a report of how CPU time is affected by queries coming from different regions. For example you can track CPU time in the
Atlantic, Central, Northeast, Northwest, Southeast, and so on.
• Are there some months when response time is slower than others? That is, are there correlations of response times by current month?
Using Teradata Workload Analyzer, generate reports to measure resource usage by periods of time such as by month or quarter. You could generate a report like the following:
231
Chapter 12: Managing Database Resources: Operational DBAs
Managing Sessions and Transactions with Query Banding
About Defining Name-Value Pairs for Session and Transaction Query Bands
To assign values to a query band for a session or transaction, submit a SET QUERY_BAND statement. You can submit a SET QUERY_BAND = NONE FOR SESSION or NONE FOR
TRANSACTION at any time to remove the query band. For the syntax and more extensive examples, see SQL Data Definition Language.
To set a query band for a profile, use a CREATE/MODIFY PROFILE request. For more
information, see “About Defining Name-Value Pairs for Profile Query Bands” on page 233
.
Example 1: Setting a Query Band For a Session
The following example sets a query band with two name:value pairs for the session.
SET QUERY_BAND = 'org=Finance;report=Fin123;' FOR SESSION;
The pairs in the query band string are org=Finance and report=Fin123.
Example 2: Removing a Query Band From a Session
The following example removes the query band for the session.
SET QUERY_BAND = NONE FOR SESSION;
The following request is equivalent.
232 Database Administration
Chapter 12: Managing Database Resources: Operational DBAs
Managing Sessions and Transactions with Query Banding
SET QUERY_BAND = '' FOR SESSION;
Example 3: Setting a Query Band for the Current Transaction
The following example sets a query band with two name:value pairs for the current transaction.
SET QUERY_BAND = 'Document=XY1234;Universe=East;' FOR TRANSACTION;
The pairs in the query band string are Document=XY1234 and Universe=East.
About Defining Name-Value Pairs for Profile Query Bands
To assign values to a query band in a profile, submit a CREATE/MODIFY PROFILE request.
The profile query band is automatically set for the session at logon. You can submit a
MODIFY PROFILE QUERY_BAND = NULL request at any time to remove the query band from the profile. For the syntax and more extensive examples, see SQL Data Definition
Language.
The profile has a [NOT] DEFAULT option to specify if the session and transaction query band can contain the same names as the profile query band. The DEFAULT option specifies that users can overwrite the values specified in the profile query band with the SET
QUERY_BAND statement for the session or transaction. If the DEFAULT clause is omitted, the profile query band is set to NOT DEFAULT, and users cannot change the profile query band with the SET QUERY_BAND statement for the session or transaction.
If the transaction, session, and profile query bands have the same name but with different values, the name/value pair used for workload management classification is the first one found in a query band searched in the following order:
•
Transaction query band
•
Session query band
•
Profile query band
Example: Creating a Query Band in a Profile
CREATE PROFILE salesprofile AS
SPOOL = 2e6*(HASHAMP()+1),
QUERY_BAND = 'GROUP=sales;AREA=retail;';
About Query Bands and Proxy Users
When using a middle-tier application to perform requests to Teradata Database on behalf of an application end user, use the SET QUERY_BAND statement. The SET QUERY_BAND statement adjusts the session active user and allows privilege checking and auditing based upon the PROXYUSER and PROXYROLE name-value pairs. The system validates the privileges of the current user or role to connect as a specified proxy user.
The proxy connection lasts for the life of the query band. The session query band remains set for the session until the session ends or the query band is set to NONE.
Database Administration 233
Chapter 12: Managing Database Resources: Operational DBAs
Managing Sessions and Transactions with Query Banding
The session query band is stored in the session table and recovered after a system reset. The transaction query band is discarded when the transactions ends (commit, rollback, or abort) or the transaction query band is set to NONE. The transaction query band is not restored after a system reset.
Finding the Origin of a Query Using Query Bands
A query band is a set of name-value pairs assigned to a session or transaction that identifies the originating source of a query. You define these identifiers and they get stored along with other session data in DBC.SessionTbl.
This is particularly useful for identifying specific users submitting queries from a middle-tier tool or application, since these often use pooling mechanisms that hide the identity of the users when each connection in the pool logs in to the database using the same user account.
For more information on query bands and how to assign them to a session or transaction, see
SET QUERY_BAND in SQL Data Definition Language.
After you create a query band set, retrieve the query band and the name-value pairs by using:
•
HELP SESSION or query the QueryBand field of the DBC.SessionInfo view.
•
System UDFs and external stored procedures. For example, to find the query bands of active sessions, use the MonitorQueryBand function:
BTEQ -- Enter your DBC/SQL request or BTEQ command:
SELECT t1.sessionno, MonitorQueryBand(Hostid, sessionNo, runvprocno)
FROM TABLE (MonitorSession(1,'*',0)) AS t1;
*** Query completed. 2 rows found. 2 columns returned.
*** Total elapsed time was 1 second.
SessionNo MonitorQueryBand(HostId,SessionNo,RunVprocNo)
----------- ---------------------------------------------------------------
1299
1298 =S> REPJOBID=495;REPORT=Sales;CLIENTMACHINE=XYZ12;REPTYPE=10;
For more information on the MonitorQueryBand function and a full list of available functions and procedures, see Application Programming Reference.
•
Query logging (DBQL). When you enable query logging, query band information is stored in the DBC.DBQLogTbl table. Use the DBC.QryLogV view to access this table. You can use the information to analyze queries after they execute.
BTEQ -- Enter your DBC/SQL request or BTEQ command: sel queryband(FORMAT 'X(50)'), querytext(FORMAT 'X(30)') from qrylogv order by
StartTime, queryid;
*** Query completed. 10 rows found. 2 columns returned.
*** Total elapsed time was 1 second.
QueryBand QueryText
-------------------------------------------------- ------------------------
? Begin query logging on a
=S> REPJOBID=495;REPORT=Sales;CLIENTMACHINE=XYZ12; sel count(*) from dbc.da
=S> REPJOBID=495;REPORT=Sales;CLIENTMACHINE=XYZ12; sel count(*) from dbc.ta
=S> REPORT=Costs;CLIENTMACHINE=XYZ12; SET QUERYChapter 11, Managing
C_BAND='REPORT=C
=S> REPORT=Costs;CLIENTMACHINE=XYZ12; sel * from tdwm.rules;
=S> REPORT=Costs;CLIENTMACHINE=XYZ12; sel * from tdwm.ruledefs
=S> REPORT=Costs;CLIENTMACHINE=XYZ12; sel * from configuration
=S> REPJOBID=99;REPORT=Acct;CLIENTMACHINE=BBB1; SET QUERY_BAND='REPJOBID
=S> REPJOBID=99;REPORT=Acct;CLIENTMACHINE=BBB1; sel database from dbc.da
? End query logging on all
234 Database Administration
Chapter 12: Managing Database Resources: Operational DBAs
Tracking Object Use and Table Changes
For more information on the database query log, see Chapter 15: “Tracking Query
Behavior with Database Query Logging: Operational DBAs.”
•
Teradata Viewpoint to track requests that meet a certain CPU or duration threshold.
Other Uses for Query Bands
Other uses for query bands include:
•
Preventing development work from running on the production platform at critical times.
For example, queries with a query band of
‘Application=Test;’
can be prevented from running, based on a TASM filter rule.
•
Recording the parameterized values that the query uses for each execution of tactical Java applications. These values are available in DBQL after the queries complete.
•
Choosing between dynamic plan execution or static query plan execution.
•
Setting the temperature of data being loaded into the system.
For More Information
For the syntax for setting a query band, see SQL Data Definition Language Syntax and
Examples. For the available query band values, see SQL Data Definition Language Detailed
Topics.
Tracking Object Use and Table Changes
You can track both object use and table changes to understand database use and provide an accurate foundation for automated statistics management.
Object use counts and last access timestamps provide a simple way to identify how frequently user queries access or use specific database objects. You can analyze system performance by noting objects that are heavily used and adjust accordingly. Or you can improve resource availability by deleting objects the system rarely accesses to reclaim disk space or tune frequently used objects.
Objects That Can Be Tracked
You can determine the number of times user queries access or use the following objects:
•
Columns
•
Databases
•
Indexes
•
Macros
•
Stored Procedures
•
Tables
•
Triggers
•
UDTs
•
UDMs
•
UDFs
•
Users
•
Views
•
Table statistics
Database Administration 235
Chapter 12: Managing Database Resources: Operational DBAs
Tracking Object Use and Table Changes
Note: Teradata Database does not collect object use counts and last access timestamps for
Data Dictionary tables or views or for the EXPLAIN, INSERT EXPLAIN, or DUMP EXPLAIN request modifiers.
Table Changes That Can Be Tracked
In addition to collecting access counts (which are done once per request), object use counts collect the number of updates, deletes, and inserts performed against database tables involved in the request. This information is useful in calculating table row count estimations, statistics extrapolations, and thresholds for statistics collection.
Teradata Database supports these requests and logs them to the Data Dictionary table
DBC.ObjectUsage:
•
Selects
•
Updates
•
Deletes
•
Inserts
•
Merges (including merge, merge-delete, and merge-update)
•
Upserts
•
Statistics access
Enabling Object Use Counts
To enable object use counts, use the WITH USECOUNT option of the BEGIN/REPLACE
QUERY LOGGING request. This request starts collection of object use counts in the Data
Dictionary table DBC.ObjectUsage. Object use counts are disabled by default, but it is a best practice to enable them for production databases to ensure the accuracy of automated statistics.
Teradata Database tracks two versions of these counts, which collect identical information:
•
User counts (a user can reset).
•
System counts (only the system can reset).
Controlling How Often Counts Are Recorded
The ObjectUseCountCollectRate field in DBS Control controls when the system writes the counts to the Data Dictionary. The field sets the number of minutes you want the system to wait before it writes the cached counts to the Data Dictionary table DBC.ObjectUsage. The recommended minimum value is 10 minutes. The default value is also 10 minutes. Any rate below 10 minutes may impact performance of systems with heavy workloads. The counts also write to disk when the cache is full.
Views That Report Count Data
These Data Dictionary views report the count data.
236 Database Administration
Chapter 12: Managing Database Resources: Operational DBAs
Tracking Object Use and Table Changes
This view...
ColumnsV[X]
DatabasesV[X]
IndicesV[X]
TablesV[X]
Users
DatabaseUseCountV[X]
ObjectUseCountV[X]
ColumnUseCountV[X]
IndexUseCountV[X]
StatUseCountV[X]
QueryStatUseCountV[X]
UpdateUseCountV[X]
DeleteUseCountV[X]
InsertUseCountV[X]
Provides this information...
AccessCount and LastAccessTimeStamp for selected columns.
AccessCount and LastAccessTimeStamp for selected databases.
AccessCount and LastAccessTimeStamp for hash indexes or join indexes.
AccessCount and LastAccessTimeStamp for selected tables, views, macros, stored procedures, triggers, and functions.
AccessCount and LastAccessTimeStamp for users that the requesting user owns or has MODIFY or DROP privileges on.
database use counts, by usage type.
object use counts, by object usage type.
column use counts, by usage type.
index use counts, by usage type.
statistics use counts, by table and statistics.
query statistics use counts, by table and statistics.
update count per table and column, by usage type.
delete count per table, by usage type.
insert count per table, by usage type.
Resetting Use Counts
When you enable query logging for a database, system and user use counts and timestamps are reset. However, when you disable query logging, only system counts and timestamps are reset. User use counts and timestamps are not reset. To reset them, use these macros:
This macro...
ClearAllDatabaseUseCount
ClearDatabaseUseCount
ClearTVMUseCount
ClearUserUseCount
ClearUserStatCount
ClearUserUDICount
Resets...
AccessCount and LastTimeStamp for all objects in the system.
AccessCount and LastTimeStamp of objects in a specified database.
the AccessCount and LastAccessTimeStamp for all of the objects in a specified table, view, macro, trigger, stored procedure, join index, user-defined type, user-defined method, or user-defined function.
all access counts (excluding statistics) for a particular user.
all statistics counts for a particular user.
all update, delete, and insert counts for a particular user.
Database Administration 237
Chapter 12: Managing Database Resources: Operational DBAs
Tracking Object Use and Table Changes
Note: You must have the EXECUTE privilege on the macros to use them.
To create these macros, check if the DIP utility script DIPVIEWSV has run. If not, run
DIPVIEWSV.
Note: All SQL statements used by the ClearDatabaseUseCount macros place write locks on the database objects involved.
Examples of Resetting Use Counts
TO reset the AccessCount and
LastAccessTimeStamp fields for…
the table tab1 the database db2
USE the following syntax…
Exec DBC.ClearTVMUseCount('db1', 'tab1');
Specify the databasename and the tablename in apostrophes, separated by a comma.
Exec DBC.ClearDatabaseUseCount('db2');
Specify databasename in apostrophes.
Exec DBC.ClearAllDatabaseUseCount; all the objects in the system
Usage Recommendations
When collecting object use counts, remember the following:
•
Do not enable collection when a Data Dictionary ARCHIVE or RESTORE is in progress.
Note: If the Data Dictionary is being archived and the archive gets blocked by the
ObjectUseCount collection, the system will turn off the ObjectUseCount collection. You must enable it again after the archive is complete.
•
The recommended value of ObjectUseCountCollectRate is 10 minutes or more. Setting
ObjectUseCountCollectRate to less than 10 minutes impacts system performance.
Copying the Use Counts of Table
CREATE TABLE AS...WITH DATA AND STATISTICS copies the statistics and data of an original table to a target table. If the USECOUNT option is enabled on the target database, the
AND STATISTICS option also copies the use count information. For more information, see
“Copying Statistics From a Base to a Target Table” on page 81 or SQL Data Definition
Language Detailed Topics, CREATE TABLE (AS Clause).
Renaming Objects Has No Effect on Use Counts
Renaming an object does not reset its use count. That is, neither the access count nor the last access timestamp changes when you rename an object. For example, the following statement:
RENAME TABLE Employee TO Emp; does not reset the use count for the table “Emp” to 0. The count remains what it was for
“Employee” with the timestamp it was last used. When “Emp” is next used, the system continues the tally and updates the timestamp. Similarly, for columns of a table, if you rename a column using the following request:
238 Database Administration
Chapter 12: Managing Database Resources: Operational DBAs
About Analyzing Trends with Account String Expansion Variables
ALTER TABLE t1 RENAME oldcolumnname TO newcolumnname; this does not reset the use count information for the column.
Because using the RENAME statement does not affect use count information, if you need to reset the count, manually reset it using the ClearTVMUseCount macro.
Example 1: Object Use Counts for Tables
Assume two databases, db1 and db2, each contain some tables and other objects. Assume also that count collection is enabled.
Now specify some SQL requests to access the tables in the two databases.
sel *from db1.tab1; sel *from db1.tab1 where col1=7; sel *from db1.tab1 where col2=7; sel *from db1.tab2; sel *from db2.tab3; sel *from db2.tab3 where col1=7; sel *from db2.tab4;
Then use the Data Dictionary view TablesVX to find the use counts for the tables.
sel DatabaseName, TableName, AccessCount, LastAccessTimeStamp from DBC.TablesVX where TableName IN ('tab1', 'tab2', 'tab3', 'tab4');
The output is:
*** Query completed. 4 rows found. 4 columns returned.
*** Total elapsed time was 1 second.
DatabaseName db1
TableName tab2
AccessCount 1
LastAccessTimeStamp 2006-12-17 14:51:53
DatabaseName db2
TableName tab4
AccessCount 1
DatabaseName db1
LastAccessTimeStamp 2006-12-17 14:51:53
TableName tab1
AccessCount 3
LastAccessTimeStamp 2006-12-17 14:51:53
DatabaseName db2
TableName tab3
AccessCount 2
LastAccessTimeStamp 2006-12-17 14:51:53
=============================================================
Note: If any of the named objects have not been accessed since the last use count reset, the value in the AccessCount column for that object appears as zero, and the
LastAccessTimeStamp will appear as a question mark.
About Analyzing Trends with Account String
Expansion Variables
You can use account string expansion variables to analyze resource usage trends. See “Logging
Resource Usage Data with Account String Variables” on page 162
.
Database Administration 239
Chapter 12: Managing Database Resources: Operational DBAs
About Resource Usage Trend Analysis
About Resource Usage Trend Analysis
About ResUsage Macros and Tables
You can use resource usage macros to report resource usage data. You can enable resource usage tables to:
•
Observe the balance of disk and CPU usage
•
Obtain pdisk usage information
•
Check for bottlenecks
•
Provide an overall history of the system operation
•
Monitor the utilization of AMP Worker Tasks (AWTs)
Resource usage data is useful for the following purposes:
•
Measuring system performance
•
Measuring component performance
•
Assisting with on-site job scheduling
•
Identifying potential performance impacts
•
Planning installation, upgrade, and migration
•
Analyzing performance degradation and improvement
•
Identifying problems such as bottlenecks, parallel inefficiencies, down components, and congestion
Enabling RSS Logging
You can enable the resource usage tables using the Control GDO Editor (ctl) utility or the database commands in Database Window (DBW).
Before you enable the resource usage tables, determine which tables apply to the resource usage macros you want to run. For more information, see Resource Usage Macros and Tables
ctl
You can set various Teradata Database configuration settings using ctl. The RSS screen in ctl allows you to specify the rates of resource usage data logging. For detailed information on starting ctl and modifying the settings, see ctl in Utilities.
DBW
Note: For instructions on starting the Database Window, see “Database Window (xdbw)” in
Utilities.
To enable RSS logging from DBW
1
2
Open the Supvr window.
Set the Node Logging Rate using the database command below.
240 Database Administration
Database Administration
Chapter 12: Managing Database Resources: Operational DBAs
About Resource Usage Trend Analysis
SET RESOURCE LOGGING
LOG
number
1099D002
3
where number is the number of seconds.
Note: A rate of zero disables the logging function.
Specify the table you want to enable logging to using the database command below.
SET LOGTABLE
tablename
ALL
ON
OFF
FE0CA030
4
where tablename is the suffix part of ResUsageXxxx. For example, for the
DBC.ResUsageSpma table, the tablename would be “Spma.”
After the table is enabled for logging, you can log rows in Summary Mode.
Note: To log rows in Summary Mode, you must enable the table specified in both the RSS
Table Logging Enable group and in the RSS Summary Mode Enable group.
(Optional) Enable Summary Mode on the table specified using the command below.
SET SUMLOGTABLE
tablename
ON
OFF
1095A010
Example
The following example shows you how to enable table logging and set the Logging rate using the database commands in DBW. Suppose you want to enable the ResUsageShst table and set the logging rate for 10 minutes (600 seconds). You would enter the following: set logtable shst on set resource log 600
241
Chapter 12: Managing Database Resources: Operational DBAs
About Resource Usage Trend Analysis
242 Database Administration
CHAPTER 13
Managing Queries:
Operational DBAs
This chapter provides:
•
Instructions for enabling and disabling Redrive protection for queries
•
Techniques for finding and resolving common problems with queries
Redrive Protection for Queries
Redrive protection makes database restarts transparent to applications and users. After a
Teradata Database restart, incomplete SQL requests are automatically resubmitted for sessions that have Redrive enabled.
Enabling Redrive Protection
To enable Redrive protection...
Do...
at the system level for all sessions by default at the session level at the user, account, or profile level
Set the RedriveProtection DBS Control field to 1.
1
2
Ensure that the RedriveProtection DBS Control field is set to 1.
Set the RedriveDefaultParticipation DBS Control field to 1.
Set the Redrive field in the Session Options parcel to 'Y'.
Insert or modify a row in the TDWM.Redrive table and set the value of the RedriveMethod column to 2 for the user, account, or
profile. For details, see “Setting Redrive Protection for Users,
Accounts, and Profiles” on page 244
.
Note: If any of the following are true, the session will not participate in Redrive:
•
The RedriveProtection DBS Control field is set to 0
•
The Redrive field in the Session Options parcel is set to 'N'
•
There is a row in the TDWM.Redrive table indicating that Redrive is disabled for this session
In addition, Recoverable Network Protocol must be enabled in order for you to use Redrive.
For details about Recoverable Network Protocol, see the DisableRecoverableNetProtocol DBS
Control field in Utilities.
Database Administration 243
Chapter 13: Managing Queries: Operational DBAs
Redrive Protection for Queries
The Redrive capability is assigned to a session at logon time. Once a session is enabled with
Redrive, you can subsequently use the Redrive query band to toggle Redrive protection off and on during the life of the session using this syntax:
SET QUERY_BAND = 'redrive=on;' FOR SESSION;
SET QUERY_BAND = 'redrive=off;' FOR SESSION;
For details on using the Redrive query band, see SQL Data Definition Language Detailed
Topics.
Disabling Redrive Protection
To disable Redrive protection...
Do...
at the system level for all sessions by default
1
2
3
Disable logons from the Supervisor window of Database
Window (DBW) or similar console subsystem interface, such as cnsterm.
Ensure that no sessions are logged on.
Set the RedriveProtection DBS Control field to 0.
1
2
Ensure that the RedriveProtection DBS Control field is set to 1.
Set the RedriveDefaultParticipation DBS Control field to 0.
at the session level at the user, account, or profile level
Set the Redrive field in the Session Options parcel to 'N'.
Insert or modify a row in the TDWM.Redrive table and set the value of the RedriveMethod column to 1 for the user, account, or
profile. For details, see “Setting Redrive Protection for Users,
Accounts, and Profiles” on page 244
.
Setting Redrive Protection for Users, Accounts, and Profiles
1
2
You can enable or disable Redrive protection for individual users, accounts, and profiles using the TDWM.Redrive table:
3
Ensure that TASM is running.
Insert, modify, or delete rows in TDWM.Redrive to specify whether particular users, accounts, or profiles will participate in Redrive.
From the Teradata Viewpoint Workload Designer portlet, issue the TDWM Activate command after changing the TDWM.Redrive table in order for the new changes to take effect.
The table definition for TDWM.Redrive is:
CREATE SET TABLE TDWM.Redrive ,FALLBACK ,
NO BEFORE JOURNAL,
NO AFTER JOURNAL,
CHECKSUM = DEFAULT,
DEFAULT MERGEBLOCKRATIO
(
ObjectType VARCHAR(10) CHARACTER SET UNICODE
NOT CASESPECIFIC NOT NULL,
ObjectName VARCHAR(128) CHARACTER SET UNICODE
244 Database Administration
Chapter 13: Managing Queries: Operational DBAs
Redrive Protection for Queries
NOT CASESPECIFIC NOT NULL,
RedriveMethod INTEGER NOT NULL)
UNIQUE PRIMARY INDEX ( ObjectType ,ObjectName );
Insert a row into the table or modify an existing row in the table to specify whether Redrive protection is enabled for a particular user, account, or profile:
Column Name
ObjectType
ObjectName
RedriveMethod
Valid Values
In order of precedence, where
USER is most specific and
PROFIL is least specific:
•
USER
•
ACCT (account)
•
ACCTSTR (account string)
•
PROFIL (profile)
The name of the object
•
•
1 (Disable Redrive)
2 (Enable Redrive)
Description
Specifies whether you are setting Redrive protection for a user, an account, or a profile.
The name of the user (or application), account, or profile for which you are setting Redrive protection.
For example, if ObjectType is USER, then ObjectName is the user name.
Determines whether Redrive is enabled or disabled for the user, account, or profile specified by ObjectName.
When there is a conflict, the most specific ObjectType is used. For example, suppose these rows are in TDWM.Redrive:
ObjectType
USER
PROFIL
ObjectName
Mark
Manager
RedriveMethod
1
2
If a session logs on as USER Mark and PROFIL Manager, since both rows in the table qualify, the most specific ObjectType is chosen. USER is more specific than PROFIL; therefore,
RedriveMethod 1 (Do not use Redrive) is chosen and the session will not participate in
Redrive.
If there is no row in TDWM.Redrive for the session, the RedriveDefaultParticipation DBS
Control field determines whether the session participates in Redrive.
Redrive attributes are assigned to users, accounts, and profiles at logon time, and these attributes remain for the entire session. The attributes are not changed for that session even if new entries are made to TDWM.Redrive that would apply to the session. However, once
Redrive is enabled, you can use the Redrive query band to turn Redrive off and on during the life of the session.
Database Administration 245
Chapter 13: Managing Queries: Operational DBAs
Redrive Protection for Queries
Restrictions
•
Redrive only hides database restarts from requests running in the SQL partition.
•
If a UDF or external stored procedure modifies data that is external to Teradata Database, then the request involving that function or procedure cannot be redriven.
•
If a database restart occurs while SELECT AND CONSUME requests are waiting for rows to consume, the restart is hidden from the application, but the queued up requests will not be in the same order as they were at the time of the restart.
•
If a PE node fails, then the restart cannot be hidden from applications that were logged onto the failing node.
•
Database restarts cannot be hidden if the restart occurs:
•
During execution of stored procedures.
•
After the first request of an explicit Teradata transaction or an ANSI/ISO transaction and the transaction placed a write lock or exclusive lock on a table in database DBC.
•
After the request is committed on the AMPs but before the PE is notified of the commit.
Example 1
When the session logs on and you have these settings...
Then the session...
RedriveProtection DBS Control field is 0 will not participate in Redrive.
•
•
RedriveProtection DBS Control field is 1
Redrive field in Session Options parcel is 'Y'
•
TDWM.Redrive contains a row matching your user name, account, account string, or profile with
RedriveMethod = 1 (Do not use Redrive)
•
RedriveProtection DBS Control field is 1
•
Redrive field in Session Options parcel is 'N'
•
TDWM.Redrive contains a row matching your user name, account, account string, or profile with
RedriveMethod = 2 (Use Redrive) will not participate in Redrive because the row in TDWM.Redrive specifies not to use Redrive.
will not participate in Redrive because the
Redrive field in the Session Options parcel is 'N'.
•
RedriveProtection DBS Control field is 1
•
Redrive field in Session Options parcel is 'Y'
•
TDWM.Redrive does not contain any matching row or it contains a row matching your user name, account, account string, or profile with
RedriveMethod = 2 (Use Redrive) will participate in Redrive.
Example 2
If you want to use Redrive, but do not want to change any application settings:
•
If you only want a single account to use Redrive:
a
Set the RedriveProtection DBS Control field to 1.
246 Database Administration
Chapter 13: Managing Queries: Operational DBAs
Recommendations for Common Query Problems
Related Topics
b
Set the RedriveDefaultParticipation DBS Control field to 0.
c
Add a row in TDWM.Redrive for the account and enter a value of 2 in the
RedriveMethod column for the account.
•
If you want all sessions to use Redrive except those from a particular account string:
a
Set the RedriveProtection DBS Control field to 1.
b
Set the RedriveDefaultParticipation DBS Control field to 1.
c
Add a row in TDWM.Redrive for the account string and enter a value of 1 in the
RedriveMethod column for the account string.
For information on...
the RedriveProtection and
RedriveDefaultParticipation DBS Control fields using the REDRIVE query band to enable or disable Redrive protection for a session
See...
Utilities
SQL Data Definition Language Detailed Topics
Recommendations for Common Query Problems
You can monitor queries using the Teradata Viewpoint Query Monitor portlet and other tools and utilities to identify poorly performing queries. Poor query performance is usually due to:
Query Problem
Stale statistics
Badly skewed data
Poor query construction
Result
The optimizer uses statistics to plan query execution. If the statistics are out of date, the optimizer may create a poor plan.
Recommendation: Review tables to see whether statistics are out of
date, and recollect statistics if required, as shown in Chapter 14:
“Improving Query Performance Using COLLECT STATISTICS:
Causes table rows to be distributed unevenly among AMPs and queries to execute unevenly, which slows processing time.
Recommendation: Monitor table skew. Redefine the primary index
if required. See “Finding and Fixing Skewed Tables” on page 249
.
Poor SQL syntax or missed punctuation can result in bad queries.
Recommendations:
•
Identify underperforming queries and the users responsible for
them. See “Identifying Poorly Performing Queries” on page 248 .
•
Abort underperforming queries when required. See
Poorly Performing Queries” on page 249
.
Database Administration 247
Chapter 13: Managing Queries: Operational DBAs
Identifying Poorly Performing Queries
Query Problem
Concurrency issues
Result
Some queries are delayed or prevented from executing because of lock contentions, or the relative priorities of concurrent queries.
Recommendations:
•
Identify the cause of a delayed query. See
Blocks and Delays” on page 501 .
•
Release a query from the throttle delay cue. See “Controlling
Poorly Performing Queries” on page 249
.
•
Order the release of requests from the throttle delay queue based on workload priority, instead of first in, first out. See the General tab in the Teradata Viewpoint Workload Designer portlet.
•
Change the priority for a database operation or user.
•
Establish more detailed concurrency controls using TASM.
Identifying Poorly Performing Queries
1
2
When notified of delays by a user or an alert, you can use the Teradata Viewpoint Query
Monitor portlet to find out if the delay is the result of a poorly performing query.
Note: The Query Monitor portlet has many functions beyond what is shown in this example procedure. See Teradata Viewpoint User Guide for detailed setup and options.
Monitor query activity using the By Session tab.
Look for poorly performing queries using the following Query Monitor parameters:
•
High CPU Use% combined with low REQ I/O accesses. If the CPU divided by the I/0 is
> 100, the query is probably bad.
•
High CPU Skew% with significant Delta CPU.
•
High spool usage.
Note: You can click on and sort the display by a parameter instead of by session number.
3
4
5
6
Select a session row that shows CPU characteristics indicating a bad query to display the
Details View, which provides information about the query, including the user and account that submitted the query.
Select the SQL tab to display the SQL for the query. Examine the SQL for errors and poor construction.
Select the Explain tab to display an abbreviated version of the step statistics and explain text (generated from a full EXPLAIN request), along with a listing of the active steps, completed steps, and steps yet to run.
Check for the following in the Explain:
•
The steps required to execute the query
•
The type of operation being performed, and whether it includes a product join
Often the query is stuck on the step where the product join occurs.
248 Database Administration
Chapter 13: Managing Queries: Operational DBAs
Controlling Poorly Performing Queries
7
You can copy the query from the SQL tab, run it from Teradata SQL Assistant or Teradata
Studio, and request a full EXPLAIN to get more details and validate the source of the problem.
Controlling Poorly Performing Queries
To control a query:
1
In the Query Monitor portlet, click on the session with the identified bad query. The
Details view appears.
2
You can use the Tools menu to:
•
Change the query priority
•
Abort the query
•
Abort and log off the session
Note: Use the following guidelines when considering whether to abort a session:
•
Do not abort the session if it is still running other queries.
•
If the session is not running other queries:
•
If the query is a SELECT statement, you can safely abort the query.
•
If the query includes an INSERT, UPDATE, DELETE, or other statement that changes data, aborting a partially completed query may cause data problems.
Finding and Fixing Skewed Tables
The DBC.TableSizeV contains information that can help identify skewed tables. Table skew results from an inappropriate primary index. A query that references a skewed table may try to process more rows on some AMPs than others, and may run out of space:
1
•
SELECT statements may run out of spool space if the accessed object has a defined spool space limit.
•
INSERT and UPDATE statements may run out of perm space.
Use the following SQL statement to find skewed tables in the DBC.TableSizeV:
SELECT vproc AS
"AMP", TableName (FORMAT 'X(20)'), CurrentPerm
FROM DBC.TableSizeV
WHERE DatabaseName = 'database'
ORDER BY TableName, "AMP" ;
Database Administration 249
Chapter 13: Managing Queries: Operational DBAs
Finding and Resolving Lock Contentions
0
1
2
3
1
2
3
AMP
---
0
TableName
--------------------employee_upi_onempid employee_upi_onempid employee_upi_onempid employee_upi_onempid employee_nupi_ondept employee_nupi_ondept employee_nupi_ondept employee_nupi_ondept
CurrentPerm
------------
18,944
18,944
18,944
19,968
4,096
30,208
15,360
12,288
2
In this example, the answer set displays space allocations by AMP for two tables. The results show that:
•
CurrentPerm is similar across all vprocs for employee_upi_onempid. Permanent space distribution is relatively even across all AMPs in the system.
•
The table Employee_nupi_ondept is poorly distributed. The CurrentPerm figures range from 4,096 bytes to 30,208 bytes on different AMPs.
Redefine the primary index for any skewed tables that you find. See “Choosing a Primary
Finding and Resolving Lock Contentions
Note: The command-line Lock Logger utility (dumplocklog) and related DBS Control settings are deprecated, but remain available for compatibility with prior releases. Database locks should be logged using the Database Query Log (DBQL). This lock information can be accessed by means of the Teradata Viewpoint Lock Viewer portlet, or by querying the
DBC.QrylockLogXMLV view.
•
For more information on DBQL, see Database Administration and the BEGIN QUERY
LOGGING statement in SQL Data Definition Language.
•
For more information on the Lock Viewer portlet, see Teradata Viewpoint User Guide.
About Lock Contentions
Teradata Database automatically locks database objects accessed by each transaction, to prevent or limit simultaneous access by other transactions. Teradata Database requests a lock on the data before a transaction begins and releases the lock when the transaction is complete.
Lock Level Description
Read Prevents write and exclusive locks. Generally caused by a SELECT.
Ensures consistency during read operations. Several users may hold concurrent read locks on the same data, during which no modification of the data is permitted.
250 Database Administration
Chapter 13: Managing Queries: Operational DBAs
Investigating Query Blocks and Delays
Lock Level Description
Write Prevents other read, write, and exclusive locks. Generally caused by an INSERT,
UPDATE, or DELETE statement.
Enables users to modify data while locking out all other users except those using access locks. Until a write lock releases, no new read or write locks are allowed.
Exclusive
Access
Prevents any other type of concurrent access. Generally caused by statements that make structural changes, for example, using an ALTER TABLE statement to add, drop, or change a column, when the statement uses the LOCKING modifier.
Exclusive locks are the most restrictive type of lock. When you hold an exclusive lock, other users cannot read or write to the data.
Prevents exclusive locks only. Caused by the LOCKING…FOR ACCESS or
LOCKING…FOR LOAD COMMITTED locking modifier.
When you use the LOCKING…FOR ACCESS locking modifier you can read data while changes are in process. However, if you use this modifier, concurrent requests may access data that has not been committed, causing inconsistent results. Use this modifier for decision support on large tables that are updated only by small, singlerow changes.
When you use the LOCKING…FOR LOAD COMMITTED locking modifier on loadisolated tables, you can read committed rows while concurrent changes are in process.
Unlike LOCKING…FOR ACCESS, this modifier returns only committed data. For
more information on Load Isolation, see “About Reading Committed Data While
Loading to the Same Table” on page 138
.
Investigating Query Blocks and Delays
Most blocks are momentary and do not require attention. However, if a block persists, you can investigate it to identify the cause and determine whether further action is required.
1
Use the Teradata Viewpoint Query Monitor portlet to monitor the parameters related to query blocks and delays.
State
Blocked
Description
Indicates that a session query is held up by a lock on an object that the query is attempting to access.
Blocked Time How long the query has been blocked.
Momentary locks are normal and not harmful. Persistent locks may indicate:
•
A runaway query or hung query that cannot complete and release its locks, and which should be aborted
•
A long-running query, such as large-scale update or backup operation, which should be rescheduled to avoid resource contention
Delayed Indicates that the query is in a delay queue caused by a workload rule.
Database Administration 251
Chapter 13: Managing Queries: Operational DBAs
Investigating Query Blocks and Delays
2
3
Click the session ID for the query to access the Details View.
•
If a query is blocked, see the Blocked By tab for details
•
If the query is delayed, the Delay tab provides details about the cause of the delay
You can also use the Lock Viewer portlet to find out more about the block, including:
•
Block Time
•
Database
•
Table
•
Delay
•
Blocked User
•
Blocking Level
•
Blocking User
252 Database Administration
CHAPTER 14
Improving Query Performance
Using COLLECT STATISTICS:
Application DBAs
This chapter describes how to generate the most efficient query plans by using automated or manual statistics to ensure accurate data demographics.
About Statistics Collection
Collecting statistics provides the Optimizer with the data demographics it needs to generate good query plans. The more accurate and up-to-date the statistics, the better the Optimizer can decide on plans and choose the fastest way to answer a query. The computed results are stored in the Data Dictionary DBC.StatsTbl for use during the optimizing phase of statement parsing. The Optimizer does not accept pragmas or hints, and bases query plans only on statistics.
Required Privileges
You must have STATISTICS privileges on a table (except a volatile table) to run a COLLECT
STATISTICS request.
Resource Considerations
Collecting statistics normally involves a full-table scan, so you may need to collect statistics for large tables during off-hours.
For very large tables, you can also collect statistics on a sampled subset of table rows instead of
all the rows for a table, using the SAMPLE clause. For more information, see “Sampling
Statistics with the USING SAMPLE Option” on page 256
About Automated Statistics Management
You have the choice of managing statistics collection manually or allowing Teradata Database to do it for you. To configure the Teradata system to automatically manage the statistics for one or more databases, the following steps are recommended:
Database Administration 253
Chapter 14: Improving Query Performance Using COLLECT STATISTICS: Application DBAs
When To Collect Statistics
1
2
3
4
Use the Teradata Viewpoint Stats Manager portlet to identify the databases and tables whose statistics should be automated. Alternatively, directly call the Automate-related
APIs available in the TDStats database.
Use the Teradata Viewpoint Stats Manager portlet to define jobs that collect the statistics identified in step 1. Consider defining Analyze jobs to identify additional statistics beneficial to query optimization. Alternatively, directly call Collect and Analyze-related
APIs in the TDSTSTATS database.
If you defined Analyze jobs in step 2, enable the STATSUSAGE logging option via the
BEGIN QUERY LOGGING statement for users or applications that access the objects identified in step 1.
Schedule the Collect and Analyze jobs defined in step 2 using either the Teradata
Viewpoint Stats Manager portlet or a different scheduling tool (for example, Cron).
For more information, see Application Programming Reference, Viewpoint documentation, and the Teradata Orange Book for Automated Statistics Management, 541-0009628.
When To Collect Statistics
Collect statistics at the following phases of table development:
•
Newly created, unpopulated tables.
Collect statistics on unpopulated tables to set up the interval histogram used in internal processing. This initial collection makes subsequent statistics collections faster. Make sure to recollect statistics after data is added.
•
Prototype phase, newly populated tables.
•
Production phase, after a significant percentage of change to the table or partition (~10% rows). For high volumes of very nonunique values, such as dates or timestamps, it may be advantageous to recollect at 7%.
Recommendation: Collect production phase statistics after you have created users and applied real world query loads to the database (up to about 3 months of querying).
•
Collect statistics in the first few weeks after an upgrade or migration during periods of low
CPU utilization.
Collecting Statistics
Use the COLLECT STATISTICS statement to define the table columns on which you want to collect statistics, for example:
COLLECT STATISTICS ON database_name.table_name COLUMN (column_1,
column_2 ...);
At minimum, you should collect statistics on the primary index (PI) of each table. For small tables (<100 rows per AMP) PI statistics may be sufficient.
254 Database Administration
Chapter 14: Improving Query Performance Using COLLECT STATISTICS: Application DBAs
About Stale Statistics
Recommendations for Large Tables
For best results on large tables, collect SUMMARY statistics only on non-partitioned tables.
Other recommendations for large tables include collecting statistics on:
•
All indexed columns
•
All frequently joined columns
•
Columns frequently referenced in WHERE clauses, particularly if the column contains very skewed, nonunique data
•
The system-derived column PARTITION (for all row and column-partitioned tables)
Reference Information
For more information on syntax and options for the COLLECT STATISTICS statement, see
SQL Data Definition Language.
About Stale Statistics
Although the system has some ability to extrapolate reasonable estimates for rows added since statistics were last collected, statistics that are far out of date (stale) may be worse for
Optimizer query planning than having no statistics at all.
Recommendation: Use the THRESHOLD option for COLLECT STATISTICS so the system automatically determines if statistics are stale and need to be recollected or if recollection can
be skipped. See “Skipping Unneeded Statistics Recollection with the THRESHOLD Option” on page 256
.
Note: If the current statistics are stale, but you do not want to recollect them because of resource contention, you can temporarily drop the statistics using the DROP STATISTICS command, and use the default statistics automatically collected by the system. However, you should recollect new statistics at the earliest opportunity.
About SUMMARY Statistics
Teradata recommends collecting SUMMARY statistics for both partitioned and nonpartitioned tables. This is a fast way to collect the current total number of rows in a table.
Collecting SUMMARY statistics is useful if collecting full statistics is too expensive. Collecting
SUMMARY statistics also allows more accurate extrapolation from previously collected full statistics, allowing resource-intensive full statistics collection to be delayed longer.
Database Administration 255
Chapter 14: Improving Query Performance Using COLLECT STATISTICS: Application DBAs
Skipping Unneeded Statistics Recollection with the THRESHOLD Option
Skipping Unneeded Statistics Recollection with the THRESHOLD Option
To skip unneeded statistics recollections, specify the THRESHOLD option of the COLLECT
STATISTICS request. With this option, if the amount of data that changed since the last statistics collection is below a specified threshold or the statistics are newer than a specified age, Teradata does not recollect statistics. Either Teradata can determine the appropriate threshold or you can specify a change percentage following the THRESHOLD keyword and/or the number of days. You can issue COLLECT STATISTICS requests at regular intervals without being concerned that you might be wasting I/O and CPU by collecting statistics too soon. Teradata skips the recollections for the statistics whose thresholds are not met and recollects the statistics for those columns that are over the threshold. For example, if the threshold is 10% data change and you request a recollection, Teradata skips recollection if less than 10% of the data changed since the last collection.If you specify the THRESHOLD option the first time you collect statistics, Teradata collects the statistics and remembers the
THRESHOLD option. When you recollect statistics, Teradata uses the saved threshold to determine whether statistics recollection is needed. To tell Teradata to ignore the
THRESHOLD option (without resetting it) the next time it collects statistics, use the FOR
CURRENT option after the percent specification.The following are the two kinds of thresholds that can be used to control the statistics recollections: change-based and timebased.
To use the THESHOLD option, the AutoThresholdPercentForStats field in the DBS Control utility Cost Profile configuration settings must be set to either:
•
0 (the default: apply the SYSTEM THRESHOLD PERCENT for all statistics that do not have the threshold percent defined)
•
1 (apply SYSTEM THRESHOLD PERCENT for only new statistics requests that do not have the threshold percent specified)
•
2 (allows user-specified thresholds but disables the SYSTEM THRESHOLD PERCENT).
You can query the LastCollectTimeStamp column of the Data Dictionary table DBC.StatsTbl to see whether data is collected or not.
The user-specified change threshold percent is not supported for statistics on views and queries.
For the syntax of the COLLECT STATISTICS statement, see SQL Data Definition Language.
Sampling Statistics with the USING SAMPLE
Option
If you find that recollection on columns with evenly distributed data (that is, data with little or no data skew) is too costly in terms of performance, try using sampled statistics instead.
256 Database Administration
Chapter 14: Improving Query Performance Using COLLECT STATISTICS: Application DBAs
Sampling Statistics with the USING SAMPLE Option
To use sample statistics, specify the USING SAMPLE option of the COLLECT STATISTICS request. You can collect sample statistics with a system-determined sample or you can specify a sample percentage. If you specify the sample percentage, Teradata reads the specified percent of rows to collect statistics.
If Teradata determines the sample percent, the system determines when to switch to sampling and the appropriate degree of sampling that can be used without sacrificing quality. The downgrade approach works like this:
1
2
3
4
5
The first time you request a sample statistics collection, Teradata collects full statistics.
This gives Teradata information it needs to determine when to downgrade to a smaller percentage.
On subsequent requests to recollect statistics, Teradata collects full statistics until it has collected enough statistics history.
Using the statistics history, Teradata evaluates the nature of the column (for example, skewed, rolling, or static). Teradata considers the column usage (detailed or summary data), which it maintains in DBC.StatsTbl.UsageType field, and the user-specified number of intervals to decide when to downgrade to sample statistics. The system is more aggressive in downgrading to sample statistics for the histograms whose summary data is used.
Teradata determines a sample percentage (2% to 100%) for recollecting statistics and decides which formula to use for scaling based on the history and nature of the column.
To ensure quality, Teradata compares the recollected statistics to the history. The system lowers the sampling percentage only if a smaller sample provides quality results.
For example, Teradata never switches to sampled statistics for small tables, skewed columns, a column that is member of the partitioning expression, or columns that require details.
After few sample statistics collections, Teradata may collect full statistics to verify and adjust the sample percentage and the extrapolation method.
Sample statistics are not supported on views and queries.
A DBS Control utility Cost Profile field AutoSampleStats defaults all the COLLECT
STATISTICS requests to the SYSTEM SAMPLE option. You can override the Cost Profile default for individual COLLECT STATISTICS requests by explicitly specifying the required sample percentage.
To determine the sampling percentage in use, you can query the Data Dictionary table column
DBC.StatsTbl.SampleSizePct.
Example 1
The following requests sample statistics on the o_orderkey with user-specified sample percentage. Statistics are collected by reading 10% of the table rows:
COLLECT STATISTICS
USING SAMPLE 10 PERCENT
COLUMN o_orderkey
ON orders;
Database Administration 257
Chapter 14: Improving Query Performance Using COLLECT STATISTICS: Application DBAs
Working with Special Cases
Example 2
The following requests system-determined sample statistics on the o_orderdatetime column.
The default sample percentage in DBS Control determines the number of rows sampled.
Teradata collects full statistics and remembers the sample option. Teradata considers downgrading the sample percentage after the system captures enough history and the column is identified as eligible for sample statistics:
COLLECT STATISTICS
USING SYSTEM SAMPLE
COLUMN o_orderdatetime
ON orders;
For more information, also see “Sampled Statistics” in SQL Request and Transaction
Processing.
For More Information
For guidelines and suggested tools for collecting statistics, see the following:
•
“COLLECT STATISTICS (Optimizer Form)” in SQL Data Definition Language
•
SQL Request and Transaction Processing:
•
“Query Capture Facility”
•
Submit the SHOW STATISTICS statement with either the SUMMARY option to see summary statistics or the VALUES option to see detailed statistics. For more information, see “SHOW STATISTICS (Optimizer Form)” in SQL Data Definition Language.
•
Teradata Viewpoint Stats Manager portlet described in Viewpoint documentation.
Working with Special Cases
Collecting Statistics on Tables Protected with Row-Level Security
If a table is protected by row-level security (at least one column is defined with a constraintcolumn-definition clause), you must have additional privileges, as follows:
To use COLLECT STATISTICS, HELP STATISTICS, and SHOW STATISTICS with the VALUES clause on...
a base table a join index a hash index a view a query
You must have the OVERRIDE SELECT
CONSTRAINT privilege on...
the base table or database.
the underlying tables or database.
the underlying tables or database.
the view or database.
the underlying tables or database.
258 Database Administration
Chapter 14: Improving Query Performance Using COLLECT STATISTICS: Application DBAs
Working with Special Cases
For copying statistics, in addition to the SELECT privilege, you must have the OVERRIDE
SELECT CONSTRAINT privilege if the source table is protected by row-level security.
Collecting Statistics on Null
When you collect statistics, Teradata Database records three different null counts: NumNulls,
PNullUniqueValueCount, and AllNulls.
•
NumNulls is the number of rows that have a null in at least one field. This includes rows with all fields null.
•
PNullUniqueValueCount is the number of unique values in rows with partial nulls.
•
All-Nulls is the number of rows that have all fields null. For example, in the following table, NumNulls count is 4, All-Nulls count is 2, and PNullUniqueValueCount is 2: x y
------ -------
1 a
? b
? ?
4 ?
? ?
Use the SHOW STATISTICS statement to report the number of nulls after collecting statistics.
By considering NumNulls, All-Nulls, and PNullUniqueValueCount counts, Teradata can better estimate the number of unique values in the presence of nulls.
Collecting Statistics on the Data Dictionary
For information on collecting statistics on the Data Dictionary, see
Data Dictionary Tables” on page 202
.
Database Administration 259
Chapter 14: Improving Query Performance Using COLLECT STATISTICS: Application DBAs
Working with Special Cases
260 Database Administration
CHAPTER 15
Tracking Query Behavior with
Database Query Logging:
Operational DBAs
This chapter describes how to track query behavior by using Database Query Logging
(DBQL). The chapter also describes how to maintain DBQL logs and archive DBQL data and the details of the DBQL tables and views.
DBQL Overview
Like other system tables, the predefined DBQL logs are created as relational tables in DBC during normal Teradata Database installation. However, while most system tables are populated automatically, you control whether or not to populate the DBQL tables.
If you choose not to use the feature, the tables remain empty.
When to Use DBQL
Use DBQL to:
•
Capture query/statement counts and response times. You can look in the DBQL logs to see if this is the first time this query has been broken and what the historical pattern of its execution has been.
•
Validate that nonunique secondary indexes (NUSIs) are actually used. You can free up space and improve table update performance by deleting NUSIs.
•
Determine which aggregate join indexes (AJIs) are used the most. This allows you to make a cost-benefit analysis on their AJIs, comparing usage against size and time to build.
•
Analyze SQL text and process steps. View the SQL text that inadvertently wiped out the system using DBQL historical data and then attempt to rebuild the lost data.
•
Log optimizer queries as XML documents.
•
Make further refinements to workload groups and scheduling.
•
Discover potential application improvements.
•
Understand basic workload trends, such as growth patterns and workload mix, and identify outliers.
•
Determine whether the query has redrive protection and if the request was redriven after a database failure.
DBQL provides a series of predefined tables and views that can store historical records of queries and their duration, performance, and target activity based on rules you specify.
Database Administration 261
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
DBQL Overview
DBQL is flexible enough to log information on the variety of SQL requests, from short transactions to longer-running analysis and mining queries that run on Teradata Database.
You can log information for an application name, a user or group of users, or a list of accounts.
How to Use DBQL
To use DBQL:
1
2
3
4
5
6
Grant the proper privileges to your administrative user for query logging. See
DBQL Administrative Privileges to Other Users” on page 391
.
Determine what type of information you want to collect. This chapter lists the descriptions of each field in each DBQL log. Use the descriptions to get an idea of what type of information you want to collect and then look at the options you can employ to control the type of details and the amount of data logged. The options are discussed in
“WITH Logging Options” on page 269
and
“LIMIT Logging Options” on page 276
.
Determine what you want to collect data for. Create DBQL rules to log queries of a user, group of users, an account, a list of accounts, or an application.
Teradata recommends creating rules for accounts so that queries will be logged for any user who logs in under that account.
Create temporary and history tables to hold log data in another database. For example, name these tables DBQLogTbl_Temp or DBQLStepTbl_Hist to associate the actual DBQL log with the temporary and history tables. Using these tables to load the DBQL data off
DBC keeps DBC from running out of space.
Use the BEGIN QUERY LOGGING statement to create logging rules or use the REPLACE
QUERY LOGGING statement to replace existing logging rules with new logging rules.
Teradata recommends using scripts to enable and disable logging to avoid typographical errors. You can also use Query Logging under the Tools menu of the Teradata
Administrator utility to create, enable, replace, and disable DBQL rules.
Note: You must have the EXECUTE privilege on the DBC.DBQLAccessMacro macro to enable or disable DBQL logging. That is, you must have this privilege to submit the
BEGIN QUERY LOGGING, REPLACE QUERY LOGGING, and END QUERY LOGGING statements.
As users establish sessions and submit queries, DBQL will automatically enable rules
according to the Best Fit Hierarchy. (See “Best Fit Rule Hierarchy” on page 264
.)
DBQL collects query information into a data cache based on rules you specify, flushes the cache periodically (see
“Options for Flushing the DBQL Cache” on page 373
), and writes the information to the DBQL dictionary tables. This information includes historical records of queries and their duration, performance data, and target activity.
During off-peak hours, load DBQL data off the DBC into your temporary tables in another database. Teradata recommends doing this nightly so that the DBQL logs do not take up space in DBC and impact the system due to space it needs for other system processing.
262 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
DBQL Overview
7
You can then load the data off these temporary tables into history tables for analysis and reporting. Examine data, create reports, and archive or delete old log information as necessary. Database Query Log Setup and Maintenance Scripts are available online at http://downloads.teradata.com/download .
Upon examining the query log data, you may decide you need to adjust rules. You can use the REPLACE QUERY LOGGING statement to replace or delete existing rules using the
END QUERY LOGGING statement.
Overview of Collection Options
DBQL is flexible enough to log information on the variety of SQL requests that run on
Teradata Database, from short transactions to longer-running analysis and mining queries.
Options enable you to control the volume and detail of the logged data. You can define rules, for example, that log the first 5,000 characters of any query that runs during a session invoked by a specific user under a specific account. Or you could define a rule to log queries that take more time to complete than the specified time threshold. You could even log queries that exceed a specific CPU time specified in hundredths of a second. For example, specify
“THRESHOLD=500 CPUTime” for queries that take longer than five seconds of AMP CPU time to complete.
The
controls what details the system logs. The
controls how much of information the system logs. Collection options include:
•
Default logging reports for each query with at least the leading SQL characters, the time of receipt, the number of processing steps completed, the time the first step was dispatched, and the times the packets were returned to the host.
•
Summary logging reports the count of all queries that completed processing time within specified time intervals, I/O criteria, or CPU usage criteria.
•
Threshold logging can log a combination of default and summary data:
•
Default data for each query that ran beyond the threshold limit
•
Summary counts of all queries that ran within the threshold time
Use elapsed time, I/O counts, or CPU time as a criterion for threshold logging of details or summary
•
Detail logging, which includes:
•
Default data
•
Any or all of the following:
•
Step level activity, including parallel steps
•
Object usage per query
•
Full SQL text
•
Explain text
•
XML query plan
•
Preventing logging for a specific user, user/account pair, user/account list, or application name.
Database Administration 263
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
DBQL Overview
Logging Rules for Applications
Enable or disable logging for a utility by specifying a rule for the application name. The system logs information for any user that logs on under the application name. For example, to log all queries that result from a FastLoad job, submit:
BEGIN QUERY LOGGING ON APPLNAME= 'FASTLOAD';
The application names you specify with the APPLNAME option are the names the system passes in the LogonSource string. (For a list of the commonly seen LogonSource string application names, see “Application Names” in SQL Data Definition Language Detailed
Topics.)
DBQL first searches for a rule for an application name before searching for rules for the user and account.
Creating logging rules by application name allows control over certain utilities, such as load and unload utilities, regardless of what user/account that the application runs under. For example, if you run utilities under the same users and accounts that are used for interactive queries, creating logging rules for utilities allows you to specify minimal logging for utilities and more extensive logging for interactive users.
Best Fit Rule Hierarchy
3
4
1
2
5
You may enable many different rules for various users, user accounts, or applications. DBQL checks for rules to apply in the following order:
All users under a specific application name
Specific user, specific account
Specific user, all accounts
All users, specific accounts
All users, all accounts
To create exceptions to the rules, use the WITH NONE option. You can create a rule to exclude certain sessions from a broader rule by using the WITH NONE option. For example, if you want to create a rule that enables logging for all users logging on under a specific application except for user1, you could create a rule to log all users for that application and create another rule using the WITH NONE option for user1 to disable logging just for user1.
(See the example under
“Scenarios of Logging Accounts” on page 376 .)
As each user logs on, DBQL first searches the rule cache for a rule, and if no rule is found, it searches the DBQLRuleTbl table. The searches are based on the order of this best fit rule hierarchy.
264 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Rules Validation
Rules Validation
The following table lists the rules DBQL validates when DBQL logging begins. DBQL checks against the application name, user name, and account string to validate rules.
IF a…
user logs on to start a session match is found in the rules cache match is not found in the rules cache match is found in the Data Dictionary match is not found in the Data Dictionary
“WITH NONE” rule is found
THEN DBQL …
checks the username, account string, application name, and zone ID (in secure-zoned systems) in the rules cache and also looks for this in the dictionary table.
logs according to any options in the rules table.
searches for a matching user or account in the
Data Dictionary.
creates an entry in the rules cache and logs according to any options in the rules table.
creates a “do not log” rule in the rules cache but does not perform logging.
creates a “do not log” rule in the rules cache but does not perform logging.
Note: If the account string of a session is changed via the SET SESSION ACCOUNT statement, the system ensures that any rules associated with the user and account string applies when the next query is issued from that session.
SQL Statements That Should Be Captured
Listed below are the requirements that should determine which SQL statements ought to be captured:
•
To the greatest extent possible, SQL text and the associated tables and views referenced should be captured and logged to facilitate performance tuning, access modeling, physical modeling modification considerations, and so on.
When enabled, DBQL captures all SQL statement types and stores them into an existing
20-character field named StatementType in DBQLogTbl.
In addition, usage of databases, tables, columns, indices, macros, views, triggers, stored procedure, and User-Defined Functions (UDFs) are logged in DBQLObjTbl. By querying
DBQLObjTbl information, DBAs are able to see which views and macros users access. This enables DBAs to delete unused objects.
•
It is assumed that user IDs and their associated account strings will adhere to the
conventions defined in “Logging Resource Usage Data with Account String Variables” on page 162 .
Database Administration 265
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Logging Statements
•
Step, explain, and XML plan logging can also be logged when needed for more detailed analysis, like performance tuning or troubleshooting.
SQL Logging Statements
When creating the necessary rules to enable SQL logging using DBQL, a rule correlates exactly to a specific SQL statement. For example, the following SQL statements show a BEGIN
QUERY LOGGING statement along with its corresponding END QUERY LOGGING statement.
BEGIN QUERY LOGGING WITH SQL ON ALL ACCOUNT='$H1$ABCD&S&D&H';
END QUERY LOGGING WITH SQL ON ALL ACCOUNT='$H1$ABCD&S&D&H';
The implication is that the SQL used to create the various rules should be retained so as to easily facilitate removing a rule.
The preceding account string format is considered a best practice. For more information, see
“Logging Resource Usage Data with Account String Variables” on page 162 .
All rules can be easily removed by the following command:
END QUERY LOGGING ON ALL RULES;
Note: Support will be removed for the WITH option of the END QUERY LOGGING statement. Use of the option results in the following message: “Deprecated option WITH is used and will soon be removed. Do not continue to use WITH option, refer to END QUERY
LOGGING statement syntax.”
SQL Logging Considerations
The BEGIN QUERY LOGGING and REPLACE QUERY LOGGING statements give the administrator flexibility in determining what SQL requests are to be logged based on the user
ID and/or account string. It also determines what level of detail is captured for a selected request. DBQL does not, however, allow for rules to be selective based on request type or object accessed. In other words, DBQL logs all request types and target database objects.
Therefore, the user ID and account string serve as the primary selection criteria.
Note: If a Teradata dynamic workload management software exception occurs, the Teradata dynamic workload management software causes rows to be written to the DBQLSqlTbl.
For information on this type of an exception, see Application Programming Reference or use
Teradata Viewpoint.
XML Query Plan Logging
The WITH XMLPLAN option of the BEGIN QUERY LOGGING statement provides more detailed Optimizer query plan information than the INSERT EXPLAIN statement modifier.
That plan information, structured and stored as an XML document, can be easily parsed and analyzed by those reading the log.
266 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Logging by Workload Type
The XMLPLAN option does not log query plans for EXPLAIN request modifiers, DDL requests, or for the INSERT EXPLAIN and DUMP EXPLAIN statements.
Query plan logging has benefits:
•
It does not require users to create a separate database to capture query plans.
•
It allows users to capture the plans for a set of queries easily via a single BEGIN QUERY
LOGGING or REPLACE QUERY LOGGING statement rather than having to issue a separate INSERT EXPLAIN or DUMP EXPLAIN statement for each one.
•
It provides more plan details than the INSERT EXPLAIN modifier.
Capturing DBQL Through the Account String
In the Teradata Active Enterprise Data Warehouse (EDW), the recommendation is to specify which requests will be captured through DBQL via the account string. Whether you are working in a non-TASM environment or a TASM environment, you should have 10 to 40 different account strings based on various workloads on the system. For more information on
TASM environments, see
“Managing the Database Workload with Teradata Active System
Management” on page 228 . The recommendation is to characterize the difference between
work groups using 4 characters (for example, ABCD). The 4 characters should identify each work group based on workload characteristics, such as business priority and by reporting.
Differentiating work groups in the account string allows you to report usage, such as CPU consumption by account string. The recommended account string format for all usage is
'$PG$ABCD&S&D&H'.
Note: TASM does not use the performance group constructs in the account string (that is,
$PG$).
The advantage to logging DBQL by account string is that it keeps the logging rules to a minimum and alleviates the need to log a DBQL rule every time a user ID is created.
Also the SQL logging needs the Teradata Active EDW to align with the workload
classifications outlined in “Managing the Database Workload” on page 507
. In other words, each SQL transaction within a given workload type, such as single session tactical, would be logged in a similar manner.
Note: You cannot log use count information for account strings.
SQL Logging by Workload Type
SQL logging is based on by the type of work being performed. In general, all Teradata Active
EDW workloads can be broadly grouped into three categories as follows:
•
Short subsecond known work (Tactical Work)
This workload can be identified by work typically done by TPUMP and single-AMP, subsecond tactical requests only.
•
Mostly long running work
Database Administration 267
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
This workload can be identified by work typically done by SQL Assistant, Teradata Studio,
Report Applications, MultiLoads, FastLoads, and BTEQ. It is typical mixed workload usage.
•
Mostly short subsecond requests (with occasional long running or unknown work)
This workload can be identified by usage that is typically tactical (millions of short, subsecond queries), but there is an unknown factor, like unauthorized usage of the user
ID, or a longer running component, that should be logged in a more detailed manner as to allow tuning or troubleshooting.
SQL Statements to Control Logging
Start and stop logging with the Teradata SQL statements BEGIN QUERY LOGGING,
REPLACE QUERY LOGGING, and END QUERY LOGGING. Only a user with EXECUTE privilege on DBC.DBQLAccessMacro can execute the statements. (You can also use Teradata
Administrator or Teradata Studio to control query logging.)
Note: You cannot issue a BEGIN/REPLACE/END QUERY LOGGING statement while running in ANSI session mode. In Teradata session mode, you cannot issue a BEGIN/
REPLACE/END QUERY LOGGING statement within a BT/ET transaction. Instead, use these statements outside a BT/ET transaction or, for ANSI session mode, log off and set the transaction mode to Teradata session mode.
BEGIN/REPLACE/END QUERY LOGGING Statements
The purpose of each of the Teradata SQL DDL statements used to enable and disable DBQL logging is described in the following table.
Statement
BEGIN QUERY
LOGGING
Purpose
When submitted by a user with EXECUTE privilege on DBQLAccessMacro, this statement enables logging for the named users, accounts, and applications triplet.
Log only a maximum of:
•
100 users
•
One user with 100 accounts
•
100 applications per statement. If logging is to be enabled for more than 100 users, use blocks of BEGIN QUERY LOGGING statements with 100 users each. For active sessions, logging begins when Teradata Database receives the next query.
You can also enable a rule to make sure no logging takes place for a user, account, or application name.
For more information, see
“Logging Scenarios” on page 374 .
268 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
Statement
REPLACE QUERY
LOGGING
END QUERY
LOGGING
Purpose
When submitted by a user with EXECUTE privilege on DBQLAccessMacro, this statement enables logging and deletes the existing rule in the
DBC.DBQLRuleTbl table if one matches the users, accounts, applications triplet, and then creates a new rule. If no rule exists, the new rule is added to the DBC.DBQLRuleTbl table.
When submitted by a user with EXECUTE privilege on DBQLAccessMacro, this statement stops logging for the named users, accounts, or applications.
Each END QUERY LOGGING statement should list fewer than 100 users/
100 accounts for one user/ 100 applications. DBQL calls a routine that commits the data and flushes all the DBQL caches.
Each time you issue an END QUERY LOGGING statement, the system will flush all the rules cache and will flush the DBQL data tables.
You can also flush the DBQL cache using other methods. See
Flushing the DBQL Cache” on page 373 .
For more information on END QUERY LOGGING, see
Dynamically Enabling/Replacing/Disabling Logging on Current Rules” on page 390
.
Every BEGIN QUERY LOGGING statement you enable must be disabled using the proper
END QUERY LOGGING statement syntax. For example, if you submit:
BEGIN QUERY LOGGING WITH SQL ON user1 ACCOUNT='Marketing';
you must use the following syntax to end logging for that user:
END QUERY LOGGING ON user1 ACCOUNT='Marketing';
Note: If you list more than one account, use parentheses. For example, ACCOUNT=
('Marketing', 'Finance').
For statement syntax, see SQL Data Definition Language. To enable other users to submit these
statements, see “Granting DBQL Administrative Privileges to Other Users” on page 391 .
WITH Logging Options
The WITH logging option that can be used with the BEGIN/REPLACE QUERY LOGGING statements enables logging into the appropriate tables as listed below.
The DBQL option…
WITH ALL
Logs to…
the following tables:
•
DBC.DBQLogTbl
•
DBC.DBQLExplainTbl
•
DBC.DBQLObjTbl
•
DBC.DBQLStepTbl
•
DBC.DBQLSQLTbl
Database Administration 269
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
The DBQL option…
WITH EXPLAIN
WITH LOCK= n
WITH [NO COLUMNS] OBJECTS
WITH NONE
WITH PARAMINFO
WITH SQL
WITH [DETAILED] STATSUSAGE
WITH STEPINFO
WITH USECOUNT
WITH UTILITYINFO
WITH [VERBOSE] XMLPLAN
Logs to…
•
DBC.DBQLogTbl
•
DBC.DBQLExplainTbl
DBC.DBQLXMLLockTbl
•
DBC.DBQLogTbl
•
DBC.DBQLObjTbl
nothing. This option disables logging on the specified user or account, list of users or accounts, or application.
DBC.DBQLParamTbl
•
DBC.DBQLogTbl
•
DBC.DBQLSQLTbl
DBC.DBQLXMLTbl
•
DBC.DBQLogTbl
•
DBC.DBQLStepTbl
DBC.ObjectUsage
DBC.DBQLUtilityTbl
•
DBC.DBQLogTbl
•
DBC.DBQLXMLTbl
Note: The WITH ALL option enables logging into all of the DBQL tables with the exception of DBQLXMLTbl and DBQLXMLLOCKTbl. The WITH NONE disables logging on the specified user/account, list of users, account list, or application.
270 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
The following table describes the different WITH options available in more detail.
Parameter
WITH ALL
WITH
EXPLAIN
WITH
LOCK=n
WITH NONE
Logging Behavior
Note: Use the WITH ALL option sparingly and only for selected users because it can consume excessive CPU resources and grow the logs (which consume DBC
PERM space) very quickly.
WITH ALL logs the information generated by these WITH rules: EXPLAIN,
OBJECTS, SQL, and STEPINFO. It does not log the information generated by
XMLPLAN, LOCK, and STATSUSAGE. The WITH ALL option generates:
•
One default row per query in DBQLogTbl that includes the first 200 characters of the SQL statement, unless you define LIMIT SQLTEXT=0.
•
One row per target object per query in DBQLObjTbl.
•
One row per step per query in DBQLStepTbl.
•
One or more rows per SQL statement in the DBQLSQLTbl.
•
One or more rows per query in DBQLExplainTbl.
This option generates the EXPLAIN for the query and logs it into the
DBQLExplainTbl. If the additional EXPLAIN text is greater than 31,000 characters, multiple rows are generated and the ExpRowNo field indicates the row number.
This option inserts a default row into DBQLogTbl.
WITH EXPLAIN generates and logs the unformatted EXPLAIN text for each request. It does not generate EXPLAIN text for requests preceded by the
EXPLAIN request modifier, nor does it log rows for cached requests.
Note: You cannot specify the WITH EXPLAIN option with the SUMMARY or
THRESHOLD options.
Logs in XML format in DBQLXMLLOCKTbl any lock contention longer than n centiseconds. The minimum acceptable value for n is 5.
You can access the DBQL lock log through the Viewpoint lock viewer portlet or query the system table DBC.DBQLXMLLOCKTbl or the view
DBC.QrylockLogXML[V].
This option causes no logging to take place for the specified user, user/account pair, user/account list, application name, or list of application names.
For example, if there is a rule for default logging on ALL users, you can exclude logging for a specific user user1, by submitting the following statement:
BEGIN QUERY LOGGING WITH NONE ON user1;
271
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
Parameter
WITH [NO
COLUMNS]
OBJECTS
WITH
PARAMINFO
Logging Behavior
This option logs database, table, column, and index information in
DBQLObjTbl.
This option can be used with SUMMARY or THRESHOLD.
This option inserts:
•
A default row in DBQLogTbl
•
One row per target object per query in DBQLObjTbl
Use this option selectively. Object data is useful for analyzing queries that make heavy use of join indexes and indexed access, but it can generate many rows.
By default, the WITH OBJECTS option logs a separate row for each column referenced in a query. The NO COLUMNS suboption turns off column logging.
This suboption is especially useful for tables with many columns. Use of NO
COLUMNS reduces overhead and controls the size of the DBQLObjTbl table.
Note: Any DBC database tables and columns used by the system while processing a query are not reflected in the DBQL object rows for that query. If a statement accesses a DBC table, the DBC table will not appear. This means, for example, that statements like CREATE TABLE or SELECT FROM DBC.xxx do not have objects logged through DBQL because all objects referenced by the plan generated for the statement are DBC tables and columns. However, other objects that are accessed by the statement will be logged in the DBC.DBQLObjTbl.
Also note that certain optimized requests may bypass the Dispatcher so that
DBQL will not log them.
The names of macros, views and triggers will be displayed. The tables and columns mentioned in the macro or view will be those of the base table, not the field names in the macro or view. Objects in DBQLObjTbl are those that the
Optimizer actually accesses, not necessarily the objects that appear in an SQL statement.
This option logs parameterized request variable names, types, positions, and values into DBQLParamTbl.
272 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
Parameter
WITH SQL
Logging Behavior
This option logs the full text of all SQL statements in DBQLSqlTbl. It inserts:
•
A default row in DBQLogTbl.
•
The entire SQL statement for each request for each user being logged. Large statements can cause multiple rows to be written to log the full query text.
Note: This option can be used with THRESHOLD.
DBQL is limited to logging information about base tables and logging direct SQL statements. Macros, views and triggers do not result in complete logging information. For example, SQL statements within a trigger are not logged by
DBQL. However, the SQL statement that triggers the action will be logged.
Note: If you set LIMIT SQLTEXT=0 when you specify the WITH SQL option, you avoid duplicate SQL in the DBQLogTbl.
Database Administration 273
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
Parameter
WITH
[DETAILED]
STATSUSAGE
WITH
STEPINFO
Logging Behavior
This option logs query optimizer statistics and usage recommendations for DML requests that undergo query optimization. Logged data is stored as an XML document in DBC.DBQLXMLTbl and is accessible from DBC.QryLogXML[V] view. WITH [DETAILED] STATSUSAGE does not log these requests: EXPLAIN,
INSERT EXPLAIN, and DUMP EXPLAIN.
Enable this option to provide data to automated statistics features, in particular the AnalyzeStatsUsage open API. Teradata recommends that you access the open
APIs via the Teradata Viewpoint Stats Manager portlet. For more information, see
“About Automated Statistics Management” on page 253 .
If logging option XMLPLAN is also enabled, the data from both options is logged in one document. If only STATSUSAGE is enabled and the optimized statement has no relevant statistics usage data, no document is logged to reduce overhead.
Note: This option requires the GRANT EXECUTE privilege on
DBC.dbqlaccessmacro. This option is not enabled when you specify the ALL option.
The DETAILED suboption logs in XML format summary statistics details from the existing statistics for all objects referenced in the query plan.
The DETAILED suboption logs the following statistics details:
•
StatTimeStamp
•
Version
•
OrigVersion
•
NumColumns
•
NumBValues
•
NumEHIntervals
•
NumHistoryRecords
•
NumNulls
•
NumAllNulls
•
NumAMPs
•
NumPNullDistinctVals
•
AvgAmpRPV
•
PNullHighModeFreq
•
AvgAmpRPV
•
HighModeFreq
•
NumDistinctVals
•
NumRows
•
CPUUsage
•
IOUsage
DETAILED does not support geospatial statistics.
This option logs AMP step-level information in DBQLStepTbl. When the query completes, it logs one row for each query step, including parallel steps.
The WITH STEPINFO option inserts a default row into DBQLogTbl.
Note: This option can be used with THRESHOLD.
274 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
Parameter
WITH
UTILITYINFO
WITH
USECOUNT
WITH
[VERBOSE]
XMLPLAN
Logging Behavior
This option logs utility information to DBC.DBQLUtilityTbl for the following utilities:
•
FastLoadProtocol: FastLoad, Teradata Parallel Transport (TPT) Load operator, and JDBC FastLoad.
•
MLOAD Protocol: MultiLoad and TPT Update operator.
•
MLOADX Protocol: TPT Update operator.
•
FastExport Protocol: FastExport, TPT Export operator, and JDBC FastExport.
•
Data Stream Architecture.
This option logs use count information for a database or user.
If you specify USECOUNT for a user, you can specify any of the other logging options. If you specify USECOUNT for a database, you cannot specify any other logging options.
Note: You cannot enable the WITH USECOUNT option on account strings.
This option logs the query plan in XML format in DBQLXMLTbl and inserts a default row into DBQLogTbl.
For example, the following statement logs the XML plan for all DML statements performed by User1.
BEGIN QUERY LOGGING WITH XMLPLAN LIMIT SQLTEXT=0 on User1;
This option does not log query plans for EXPLAIN request modifiers, DDL requests, or for the INSERT EXPLAIN and DUMP EXPLAIN statements.
Because the XML plan includes the query and EXPLAIN text, Teradata recommends that when you use the XMLPLAN option, do not also use the
WITH EXPLAIN and WITH SQL options. Set SQLTEXT to 0 (SQLTEXT=0) to avoid redundant logging if query and EXPLAIN text is not needed for DDL statements.
The XML schema for the log this option produces is maintained at http:// schemas.teradata.com/queryplan/queryplan.xsd
.
Note: WITH XMLPLAN cannot be used with the THRESHOLD and
SUMMARY limit options.
The VERBOSE suboption logs the VERBOSE EXPLAIN text of a query in XML format and contains details on SpoolAsgnList and HashFields that are not available with just EXPLAIN.
For more information on the SQL syntax, see “BEGIN QUERY LOGGING” or “REPLACE
QUERY LOGGING” in SQL Data Definition Language.
275
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
LIMIT Logging Options
The following options control how much and when the data gets logged into DBQLogTbl and
DBQLSummaryTbl.
Parameter Logging Behavior
LIMIT SQLTEXT=n This option determines how much SQL text to capture for queries logged to
DBQLogTbl.
Use this option to capture less than or more than the automatic first 200 characters in the default row. To turn off text capture in DBQLogTbl completely, specify 0. Specify LIMIT SQLTEXT=n where n is some value greater than 0 to control the amount of SQL text to be recorded in
DBC.DBQLogTbl. For example, to log 500 characters of the SQL, submit the following:
BEGIN QUERY LOGGING LIMIT SQLTEXT=500 ON USER2;
The maximum is 10,000 characters. If you specify LIMIT SQLTEXT but do not specify a value, up to 10,000 characters are logged in DBQLogTbl.
To store the complete statement regardless of length, specify the WITH SQL option; the system logs as many rows as needed to contain the full text in
DBQLSQLTbl.
If you specify the WITH SQL option and log to the DBQLSQLTbl, you may define LIMIT SQLTEXT=0 to avoid redundant logging in both the default row and DBQLSQLTbl.
LIMIT
SUMMARY=n1, n2,
n3 [units]
Use this option to group queries by duration intervals or number of I/Os, count each group, and store the count results in DBQLSummaryTbl. You can specify up to 3 intervals. A fourth interval is created by default as anything longer than n3.
If you specify all three intervals, the intervals are:
•
0-n1
•
n1-n2
•
n2-n3
•
> n3
SUMMARY is useful for tracking many short queries, such as for OLTP applications, because it does not log to DBQLogTbl. For an output example, see
“Summary Log Table: DBQLSummaryTbl” on page 331
.
The LIMIT SUMMARY option:
•
Flushes summary information at system-controlled intervals of 10 minutes.
•
Does not write rows if no data has been collected for a summary logging user/account in a 10-minute interval.
•
Cannot be used with LIMIT THRESHOLD.
•
Cannot be used with logging options.
You can define the intervals for LIMIT SUMMARY with one of the following:
276 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
Parameter
LIMIT SUMMARY
=n1, n2, n3 [units]
(continued)
Logging Behavior
Use the unit of… To specify summary data as based on…
CPUTIME AMP CPU time, where n1, n2, n3 are specified in hundredths of a second.
Note: Parser CPU Time is reported, but it is not used to place query data into summary buckets.
CPUTIMENORM normalized AMP CPU time, where n1, n2, n3 are specified in hundredths of a second.
Note: Normalized columns, such as
CPUTIMENORM, are for co-existence systems only.
ELAPSEDSEC elapsed time in seconds. If you do not specify units,
ELAPSEDSEC is the default.
ELAPSEDTIME
IOCOUNT elapsed time, where n1, n2, n3 are specified in hundredths of a second.
count of total I/Os, where n1, n2, n3 are specified in number of I/Os.
If you do not specify units, the summary is based on ELAPSEDSEC.
For example, to group queries based on elapsed time such as 0-1 seconds, 1-
2 seconds, 2-3 seconds, and log queries running greater than 3 seconds, submit:
BEGIN QUERY LOGGING LIMIT SUMMARY=1,2,3 ON ALL;
To summarize queries for 0.1, 0.5 and 1.0 seconds of CPU, (that is, four groups: 0-0.1, 0.1-0.5, 0.5-1.0, and greater than 1.0 CPU seconds) submit the following:
BEGIN QUERY LOGGING LIMIT SUMMARY=10,50,100 CPUTIME ON
ALL;
To count queries using normalized CPU time rather than “raw” CPU time, use the CPUTIMENORM modifier.
Note: Normalized columns, such as CPUTIMENORM, detect skew more accurately and are for co-existence systems only.
To group queries based on I/O, submit:
BEGIN QUERY LOGGING LIMIT SUMMARY=1000,5000,10000
IOCOUNT ON ALL;
For more information on the DBQL Summary table, see “Summary Log
Table: DBQLSummaryTbl” on page 331
.
277
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
Parameter
LIMIT
THRESHOLD =n
[units]
Logging Behavior
Queries that run in n “units” or less are counted in the DBQLSummaryTbl.
Queries that run more than n units are logged in the DBQLogTbl. If you do not specify n, the default is 5 units. Units can be one of the following:
Use the unit of…
CPUTIME
IOCOUNT
To specify threshold data as based on…
AMP CPU time, where n is specified in hundredths of a second.
Note: Parser CPU Time is reported, but it is not used to place query data into summary buckets.
CPUTIMENORM normalized AMP CPU time, where n is specified in hundredths of a second.
Note: Normalized columns, such as
CPUTIMENORM, are for co-existence systems only.
ELAPSEDSEC elapsed time in seconds. If you do not specify units,
ELAPSEDSEC is the default.
ELAPSEDTIME elapsed time, where n is specified in hundredths of a second. count of total I/Os, where n is specified in number of I/Os.
278 Database Administration
Database Administration
Parameter
LIMIT
THRESHOLD
(continued)
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
SQL Statements to Control Logging
Logging Behavior
LIMIT THRESHOLD can be used with these logging options:
•
OBJECTS
•
STEPINFO
•
SQL
The option ELAPSEDTIME is useful for specifying partial seconds.
For example, to log queries that use more than 8 CPU seconds in detail and tally queries that require less than 8 CPU seconds submit the following:
BEGIN QUERY LOGGING LIMIT THRESHOLD=800 CPUTIME ON
ALL;
THRESHOLD is specified in hundredths of a second. If you set
THRESHOLD=8, this would set the threshold at 0.08 CPU seconds.
The default is 5 hundredths CPU seconds. For example, if you submitted the following statement:
BEGIN QUERY LOGGING LIMIT THRESHOLD CPUTIME ON ALL; without assigning a numeric value for THRESHOLD, the system defaults to summarizing queries that use less than 0.05 CPU seconds and gives details for all queries using more than 0.05 CPU seconds.
To count queries using normalized CPU time rather than “raw” CPU time, use the CPUTIMENORM modifier.
Note: Normalized columns, like CPUTIMENORM, detect skew more accurately and are for co-existence systems only.
To base the threshold on the number of I/Os and log all queries with more than 5,000 I/Os, use the following:
BEGIN QUERY LOGGING LIMIT THRESHOLD=5000 IOCOUNT ON
ALL;
You can even combine THRESHOLD with SQLTEXT to capture more than just the first 200 characters of a query that runs longer than THRESHOLD seconds (because the SQL text of short queries is not logged in
DBQLogTbl).
Defining a threshold will determine whether to log a query or just count it, as follows:
IF a query …
completes at or under the threshold elapsed time, CPU seconds, or I/O count
THEN DBQL...
•
Increments the query count and adds the elapsed time of the query, CPU time, and I/O counts to the summary row for this session in the current collection period.
•
Stores the final count for the session as a row in
DBQLSummaryTbl.
threshold value DBQLStepTbl, and DBQLogTbl so you can examine query execution time through time stamps and the number and level of processing steps.
279
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
DBQL Tables, Views, and Macros
Mode Logging Option
The following option controls which CPU/IO collection algorithm Teradata uses for this query logging request.
Parameter
MODE = m
Logging Behavior
The CPU/IO collection algorithm to use for this query logging request, as specified by m. Possible values are:
•
0 = Use the default value for this release
(algorithm 1).
•
1 = Use the classic algorithm 1 with step adjustments.
•
2 = Use AMP algorithm 2 (diagnostic only).
•
3 = Use AMP algorithm 3 (which includes data on aborted and parallel steps).
If you do not specify MODE, the algorithm is controlled by the DBS Control utility field
DBQL CPU/IO Collection. For more information, see Utilities.
You must have the EXECUTE privilege on DBC.DBQLModeMacro to use this option.
DBQL Tables, Views, and Macros
Along with the DBQL tables, additional DBQL components includes a security macro and a view for each table. These are all created in database DBC by the DIP utility during installation. For details, see “Database Initialization Program (DIP)” in Utilities.
The purpose of each object is listed in the following table.
THIS dictionary object …
DBQLAccessMacro
DBQLExplainTbl
DBQLModeMacro
DBQLogTbl
IS a …
macro
(empty) table macro
(empty) table
THAT …
controls authority of users to execute the
Teradata SQL BEGIN/REPLACE/FLUSH/
END QUERY LOGGING statements.
contains the explain information in an unformatted string without line breaks.
controls the authority of users to execute the MODE option of the BEGIN/
REPLACE QUERY LOGGING statements is the main table containing information about queries being logged.
AND is created by…
the DIP utility when it runs
DIPVIEWSV.
Tableinit and only populated if you specify the EXPLAIN option.
the DIP utility when it runs
DIPVIEWSV.
Tableinit (invoked by the Sysinit utility), during installation.
280 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
DBQL Tables, Views, and Macros
THIS dictionary object …
DBQLObjTbl
DBQLRuleCountTbl
DBQLRules[V]
DBQLRuleTbl
DBQLSQLTbl
DBQLStepTbl
DBQLSummaryTbl
DBQLUtilityTbl
DBQLXMLLOCKTbl
IS a …
table table view table table table table table table
THAT …
stores information on the target objects of the query being logged. One row is logged for each object referenced in the query.
stores the cardinality of DBQLRuleTbl
(for internal use only).
stores information about each processing step used to satisfy the query. One row is logged for each step stores information about queries that meet the criteria for a rule specifying the
SUMMARY or THRESHOLD option stores information about load/export and
Data Stream Architecture utilities at the job level logs lock delays from the Lock Manager in the AMPs in XML format
AND is created by…
Tableinit and only populated if you specify the OBJECT option.
displays the current rules in
DBC.DBQLRuleTbl (to a user with DBC or SystemFE privileges).
stores the rules resulting from each BEGIN
QUERY LOGGING or REPLACE QUERY statements.
One row exists for each set of username, account string and application name specified in the BEGIN QUERY
LOGGING and REPLACE QUERY
LOGGING statements.
The END QUERY LOGGING statement removes rows from the rule table.
This table is for internal use only.
stores the full SQL text of the query. One query string may require more than one row.
Tableinit (invoked by the Sysinit utility).
This table is only populated if you specify BEGIN QUERY LOGGING statements.
the DIP utility when it runs
DIPVIEWSV.
Tableinit (invoked by the Sysinit utility).
This table is only populated if you specify BEGIN QUERY LOGGING or REPLACE QUERY LOGGING statements.
Tableinit and only populated if you specify the SQL option.
Note: If a Teradata dynamic workload management software exception occurs, the Teradata dynamic workload management software causes rows to be written to the DBQLSqlTbl.
Tableinit and only populated if you specify the STEPINFO option.
Tableinit (invoked by the Sysinit utility), during installation.
the WITH UTILITYINFO option of
BEGIN QUERY LOGGING or
REPLACE QUERY LOGGING. the WITH LOCK option of BEGIN
QUERY LOGGING or REPLACE
QUERY LOGGING.
Database Administration 281
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
About the Presentation of Tables and Views in This Book
THIS dictionary object …
DBQLXMLTbl
IS a …
table
QryLockLogXML[V] view
QryLogExplainDoc[V] view
QryLogExplain[V]
QryLogObjects[V] view view
QryLogSQLDoc[V]
QryLogSQL[V]
QryLogSteps[V]
QryLogSummary[V] view view view view
QryLogTDWM[V]
QryLog[V]
QryLogUtilityV
QryLogXMLDocV
QryLogXML[V] view view view view view
THAT …
stores the XML plan for all DML statements.
accesses the DBQLXMLLOCKTbl access the DBQLExplainTbl accesses the DBQLExplainTbl accesses the DBQLObjTbl access the DBQLSQLTbl accesses the DBQLSQLTbl accesses the DBQLStepTbl accesses the DBQLSummaryTbl accesses the DBQLogTbl accesses the DBQLogTbl accesses the DBQLUtilityTbl accesses the DBQLXMLTbl accesses the DBQLXMLTbl
AND is created by…
Tableinit and only populated if you specify the XMLPLAN option.
the DIP utility when it runs
DIPVIEWSV.
The DBQL views are predefined for ease of use. However, database administrators can create their own customized views by taking data from one or more DBQL tables and displaying the information in the format they need.
About the Presentation of Tables and Views in
This Book
The DBQL tables and views in this book are presented in alphabetical order for quick reference to the meaning of individual fields. The actual DBQL tables and view fields do not appear in alphabetical order.
About Querying DBQL Tables
You can query the main table (DBQLogTbl) alone, but to query other DBQL tables you need to create a join with DBQLogtbl. Use the QueryID field to create the join.
282 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
Main Log Table: DBQLogTbl
Default rows are stored in the DBC.DBQLogTbl table, the foundation of the DBQL feature.
The default is one default row per query.
Use the LIMIT SUMMARY and LIMIT THRESHOLD options to control whether a default row is logged for a query. Using:
•
LIMIT THRESHOLD reduces the amount of logged rows to one row for each query above the threshold limit. Other queries are counted in the DBQLSummaryTbl.
•
LIMIT SUMMARY eliminates default row logging and counts all queries in four “buckets” based on the limit criteria.
In addition to the default row logging, you may specify a WITH option to log OBJECTS, SQL,
STEPINFO and/or EXPLAIN. The LIMIT THRESHOLD can accept SQL, STEPINFO and/or
OBJECTS combination.
Note: If you specify the WITH options that result in more detailed information, a default row is still generated in DBQLogTbl.
The Default Row
The fields of the default row provide general query information that is usually adequate for investigating a query that is interfering with performance. When no options are specified, a default row includes:
•
User ID and user name under which the session being logged was initiated
•
Unique ID for the process, session, and host (client) connection
•
Account string, expanded as appropriate, that was current when the query completed
•
First 200 characters of the query SQL statement
•
CPU and I/O statistics
•
Default database name that was current when the query completed
Row Values
The available fields in a DBQLogTbl row are listed in the following table.
DBC.DBQLogTbl Field
AbortFlag
Description
If the query being logged was aborted, the collected data could be incomplete.
Note: A query consisting of a multistatement or iterated request with simple INSERT statements can be successful and yet have statement errors, as indicated by the ErrorCode column.
Database Administration 283
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
DBC.DBQLogTbl Field
AcctString
AcctStringDate
AcctStringHour
AcctStringTime
AMPCPUTime
AMPCPUTimeNorm
AppID
CacheFlag
CalendarName
CallNestingLevel
CheckpointNum
Description
The account string at the time the row was committed. Because accounts can be changed dynamically at the query or session level, this may not be the same account that:
•
DBQL verified for the user at logon time
•
Was in effect when processing of the query began
Value of the expanded &D ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Even if ASE truncates the expanded account string, DBQL will report the full value. For example, if &D is expanded into position
26 or higher in the account definition and truncation occurs, DBQL will still report the entire value of the date in the AcctStringDate field.
Value of the expanded &H ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Value of the expanded &T ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Even if ASE truncates the expanded account string, DBQL will report the full value.
Total AMP time used for the query in CPU seconds.
Note: The I/O, CPU, and spool usage may not be accurate for aborted queries.
Normalized AMP CPU seconds for co-existence systems.
Application (executable) or utility name; for example, BTEQ from
LogonSource for network clients.
This field is blank if the query is not found in the request cache. It could also be one of the following:
•
“T” if the query plan is generic and in the request cache.
•
“G” if the query plan is generic and not in the request cache but is a candidate for caching in subsequent runs of the same query.
•
“S” if the query plan is specific (either a static or dynamic plan).
•
“A” if a Specific Always decision is taken. That is, for each query
USING values are peeked during request parsing. The optimizer may choose a dynamic plan with results feedback.
For more information on specific plans and generic plans, see
“Peeking at Parameterized Values in the Data Parcel” in SQL Request
and Transaction Processing.
The name of the business calendar the session uses. Possible values include ISO, COMPATIBLE, and TERADATA (the default).
The level of nesting when running in stored procedures.
The checkpoint number when loading data into an MLOADX staging table. The default is zero.
284 Database Administration
Database Administration
DBC.DBQLogTbl Field
ClientAddr
ClientID
CollectTimeStamp
CPUDecayLevel
DataCollectAlg
DBQLStatus
DefaultDatabase
DelayTime
DisCPUTime
DisCPUTimeNorm
ErrorCode
ErrorText
EstMaxRowCount
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
Description
The client IP address of the submitted query as obtained from
Logonsource from a network client.
The network client user name under which this session is logged
(for example, ADMIN) and the PK/FK for the table.
A date and time unique to each buffer cache, which changes for each new buffer. Part of the multi-column NUPI for the table (see also
ProcID).
This time will be set when the cache is ready to receive new data, not when rows are written to the database.
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381
.
Contains the most severe decay level reached for CPU usage on any one node for requests using the default timeshare method in
Priority Scheduler for SUSE Linux Enterprise Server 11 or above.
Values are 0, 1, or 2. A value of 0 indicates that the request is running at the default priority. A value of 1 indicates that the priority level is reduced and the resource allotment is halved. A value of 2 indicates that the priority level is reduced again, the resource allotment is halved again, and it will not be reduced more.
CPU/IO collection algorithm used by DBQL. Possible values include:
•
1 = Use the classic algorithm with step adjustments.
•
2 = Use AMP algorithm 2 (diagnostic only).
•
3 = Use AMP algorithm 3 (includes information on aborted and parallel steps).
Internal DBQL logging status. Zero indicates no status conditions.
This field is NULL if you are not using DBQL CPU/IO Collection algorithm 3.
Name of the default database for the query.
The seconds a query was delayed by the Teradata dynamic workload management software.
Dispatcher CPU time in seconds.
Dispatcher normalized CPU time in seconds.
If greater than 0, this field is the number of the error message in
ErrorText. This field contains a code if the query caused a Parser syntax error.
If not null, this field provides processing errors and Parser errors.
For example, the text would explain that the query was not processed due to a Parser syntax error.
The largest number of rows handled by a single step in the query, as estimated by the Optimizer.
285
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
DBC.DBQLogTbl Field
EstProcTime
EstResultRows
ExceptionValue
ExpandAcctString
ExtraField20
ExtraField22
ExtraField23
FinalWDID
FirstRespTime
FirstStepTime
InternalRequestNum
IODecayLevel
IterationCount
Description
Sum of minimum estimated processing time for steps in the query as generated by the Optimizer.
Note: This time is reported in seconds.
Estimated result rows as generated by the Optimizer.
Specifies what type of Teradata dynamic workload management software exception occurred.
The expanded account name under which the query is submitted if account expansion is invoked.
Note: The &I, which gives the host, session, and request number for the query is different for a stored procedure call. For stored procedure calls, the request number is the client request number which is the request of the CALL statement.
The persistent portion of the DBQLogTbl.SpoolUsage column.
The number of seconds the query was held to meet the TASM minimum response time.
The total time (in seconds) from when the request was submitted to the time the first response was sent.
Indicates the workload definition in which the query completed execution.
The timestamp when the first response packet is sent to the client unless the request is held by TASM to meet the minimum response time. In this case, it is the timestamp when the first response packet is ready to be sent to the client.
The timestamp when the first step is dispatched.
The internal request number.
For commands, other than those within a stored procedure, the internal request number and the number in the RequestNum field will be the same.
For stored procedures invoked within a session, the internal request number increments by 1 for every request made by the stored procedure. This means that RequestNum will be the request number of the CALL and the value in InternalRequestNum will continue to increment for all other queries issued by the session.
Contains the most severe decay level reached for I/O usage on any one node for requests using the default timeshare method in
Priority Scheduler for SUSE Linux Enterprise Server 11 or above.
Values are 0, 1, or 2. A value of 0 indicates that the request is running at the default priority. A value of 1 indicates that the priority level is reduced and the resource allotment is halved. A value of 2 indicates that the priority level is reduced again, the resource allotment is halved again, and it will not be reduced more.
The number of data rows used by a parameterized request.
286 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
DBC.DBQLogTbl Field
KeepFlag
LastRespTime
LastStateChange
LockDelay
LockLevel
Description
Whether the response parcel should be kept until the client responds that it is no longer needed. The value is either Y or N.
The timestamp of the last response. This field is NULL if you are not using DBQL CPU/IO Collection algorithm 3.
Time of the last State change by the Teradata dynamic workload management software.
Maximum wait time to get a lock on an object in centiseconds. The maximum value is 32,767 centiseconds.
The highest lock level (Access/Read/Write/Exclusive) for locks associated with this request. For explicit transactions, the lock level for the transaction. Does not include row hash locks.
LogicalHostID
LogonDateTime
LogonSource
LSN
MaxAMPCPUTime
MaxAMPCPUTimeNorm
MaxAmpIO
Unique identifier of the logon source for the logged query. A value of 0 indicates an internal session.
The timestamp of the start of the session.
The logon source string text.
MaxCPUAmpNumber
MaxIOAmpNumber
MaxStepMemory
The number of the AMP with the highest CPU activity.
MaxCPUAmpNumberNorm Number of the AMP with the maximum normalized CPU seconds for co-existence systems.
The number of the AMP with the highest I/O usage for this step.
The maximum amount of memory in MBs used by any one AMP in any step in this request. This field can be compared with the TDWM estimated memory usage field, TDWMEstMemUsage, also in
DBQLogTbl. This field does not apply to INSERT/UPDATE/
DELETE requests.
MaxStepsInPar
MinAmpCPUTime
MinAmpCPUTimeNorm
The Logon Sequence Number used for a load utility.
CPU seconds of the highest CPU utilized AMP in the query.
Normalized maximum CPU seconds for an AMP.
Logical I/O count of the highest utilized AMP in the query.
The maximum number of Level 2 steps done in parallel for the query.
CPU seconds of the lowest CPU utilized AMP in the query.
Normalized minimum CPU seconds for an AMP.
Note: Normalized columns, such as MinAmpCPUTimeNorm, are for co-existence systems only.
MinAmpIO
NoClassification
Logical I/O count of the lowest logical I/O utilized AMP in the query.
Indicates if the query was not classified.
287
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
DBC.DBQLogTbl Field
NumFragments
NumOfActiveAmps
NumRequestCtx
NumResultRows
NumSteps
NumStepswPar
OpEnvID
ParserCPUTime
ParserCPUTimeNorm
ParserExpReq
ProcID
ProfileID
ProxyRole
ProxyUser
ProxyUserID
QueryBand
Description
For a request for which a dynamic plan is generated, this field indicates the number of plan fragments. For a static plan, this field is NULL.
Number of AMPs active for the query. Use it to compute the average
CPU or I/O per AMP.
The number of request contexts associated with the session.
The total number of rows returned for the query.
Total number of Level 1 steps required to process the query. The value is 0 if:
•
The query was aborted or you ended logging for the user before steps were generated.
•
It is a PREPARE statement. The PREPARE statement is logged separately from the corresponding SQL.
Total number of Level 1 steps with parallel steps.
The internal identifier of the Operating Environment / Planned
Environment currently enforced by the Teradata dynamic workload management software.
Total Parser and Dispatcher CPU seconds used for the query.
Normalized parser CPU seconds for co-existence systems.
Seconds the parser waited on an express request.
Unique processor ID of the Dispatcher and part of the multicolumn NUPI for the table (see also CollectTimeStamp).
The name of the profile, if any, under which the user submitted the query.
The rolename of a proxy connection.
The username of a proxy connection.
The ID of the ProxyUser being charged for spool and temp space. If the field is NULL, the session user ID was used for spool and temp space.
The query band under which the query is submitted.
Possible values include:
•
=T> transaction query band pairs
•
=S> session query band pairs
•
=P> profile query band pairs
288 Database Administration
Database Administration
DBC.DBQLogTbl Field
QueryID
QueryRedriven
QueryText
RedriveKind
ReqIOKB
ReqPhysIO
ReqPhysIOKB
RequestMode
RequestNum
ResponseTimeMet
SeqRespTime
SessionID
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
Description
Internally generated identifier of the query and the FK to other
DBQL tables.
Note: QueryID is a system-wide unique field; you can use QueryID to join DBQL tables or the DBC.TdwmExceptionLog table with
DBQL tables without needing ProcID as an additional join field.
Whether the query was redriven if redrive protection was enabled.
Possible values are:
•
R = The request was redriven
•
N = The request was not redriven
•
E = The request was reparsed due to an internal Parser error.
•
M = The request was reparsed due to an internal Parser memory limit.
The first characters of the SQL query. Up to 200 characters are captured by default.
If you use the LIMIT SQLTEXT option, you can specify that the first n characters be captured where n is 0-10000.
Whether the query has redrive protection. Possible values are:
•
' ' = Not participating in redrive
•
MN = memory-based protection, no fallback spool
Total logical I/O usage in kilobytes.
The number of physical I/Os.
Total physical I/O usage in kilobytes.
Request mode from parcel request processing option:
•
Prep: Prepare mode (P)
•
Exec: Execute mode (E)
•
Both: Prepare and execute mode (B)
•
PrepS: Prepare, supporting parameterized SQL (S)
Unique identifier of the query.
Note: The request number for the commands that occur within a stored procedure CALL statement will all appear as the same number of the CALL statement itself. For more information on request numbers for stored procedures, see note for
ExpandAcctString.
Indicates if the query met service level goals.
Sum of the response time of all steps as if they had been executed sequentially, in seconds.
Unique session identifier.
289
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
DBC.DBQLogTbl Field
SessionTemporalQualifier
SessionWDID
SpoolUsage
StartTime
Statements
StatementGroup
Description
When a DML or SELECT request refers to a temporal table but omits a temporal qualifier, the system uses the value of the session temporal qualifier. This is the session temporal qualifier in effect when the query is logged. For more information on the qualifier, see
ANSI Temporal Table Support and Temporal Table Support.
Workload ID class associated with the session. It determines the priority of the session when the TDWM workload classification is active.
If you are using algorithm 3 (the DBQLogTbl DataCollectAlg field=
3), the peak spool usage in bytes for any step of the query.
For other algorithms, the maximum amount of spool used while processing the query.
If SpoolUsage is 0, then there is no usage. If it is null, the data on usage was not valid.
The timestamp when the query is submitted.
Number of statements in a request:
•
1 = single-statement.
•
> 1 = multiple statements.
If there is a DDL statement in a request, StatementGroup reports which type:
•
DDL ALTER
•
DDL CREATE
•
DDL GRANT
If the statement has only one DML statement or multiple DML statements that are all of the same type, StatementGroup will indicate the type. For example if there are three DELETE statements in a request, StatementGroup will report:
DML DELETE
Similarly, for requests with individual or multiple INSERT,
INSERT... SELECT, UPDATE or SELECT statements,
StatementGroup will report:
•
DML INSERT
•
DML INSERT... SELECT
•
DML UPDATE
•
SELECT
In a multistatement request with different types of DML statements, you will see a list showing the number of statements of each type in the request. For example, a request with one insert and two update statements will appear as:
DML Del=0 Ins=1 InsSel=0 Upd=2 Sel=0
290 Database Administration
Database Administration
DBC.DBQLogTbl Field
StatementType
SysConID
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
Description
The type of statement of the query.
In a multistatement request, this is the last statement of the request.
However, this may not accurately describe the request. For more statement information, see StatementGroup.
The internal identifier of the System Condition (SysCon)/ System
Health currently enforced by the Teradata dynamic workload management software.
TacticalCPUException
TacticalIOException
TDWMAllAmpFlag
TDWMConfLevelUsed
TDWMEstLastRows
TDWMEstMaxRows
TDWMEstMemUsage
TDWMEstTotalTime
TDWMRuleID
ThrottleBypassed
TotalIOCount
The number of nodes that had a CPU exception for the request.
The number of nodes that had an I/O exception for the request.
Indicates whether one of the steps of the query is an all-amp step.
This is used to classify a query into a workload definition.
Note: No longer a TASM rule attribute. The letter 'N' will always be logged.
The estimated last row count generated by the Optimizer used to classify a query into a workload definition.
The estimated maximum row count generated by the Optimizer used to classify a query into a workload definition.
The estimated memory used by the request (in megabytes). It represents the largest step in the request.
The total estimated time of a query by the Teradata dynamic workload management software (based on Optimizer estimates) and applied against rules read from the TDWM database.
Note: This time is reported in milliseconds in DBC.DBQLogTbl, but it is reported in seconds in the view DBC.QryLogTDWM[V].
Rule identifier of the query.
Whether an active request is active solely due to the ThrottleBypass ruleset attribute. This attribute overrides the throttle limits if the session owning the request has an object lock higher than the Access level. Possible values include:
•
ThrottleBypass ruleset attribute is set.
•
ThrottleBypass ruleset attribute is not set.
Total logical I/O used by the query.
Note: The I/O, CPU, and spool usage may not be accurate for aborted queries.
291
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Main Log Table: DBQLogTbl
DBC.DBQLogTbl Field
TTGranularity
TxnMode
TnxUniq
UnitySQL
UnityTime
UserID
UserName
UtilityByteCount
UtilityInfoAvailable
UtilityRowCount
VHLogicalIO
VHLogicalIOKB
VHPhysicalIOKB
VHPhysIO
WarningOnly
WDID
Description
Possible values include:
•
LogicalRow: Row is timestamped with the time the row is processed by the AMP.
•
Request: Row is timestamped with the time the request is submitted.
•
Transaction: Row is timestamped with the time when the first non-locking reference is made to a temporal table, or when the built-in function TEMPORAL_TIMESTAMP is first accessed during the transaction.
For more information, see Temporal Table Support.
Transaction mode:
•
ANSI: ANSI transaction mode
•
BTET: Teradata transaction mode
The unique value within the transaction number. Other transaction number components (userid, procid) are already in DBQLogTbl.
Combined with TxnUniq, these fields represent the complete transaction id for the logged request.
Time in seconds the stored procedure request was processed by
Unity.
Time in seconds the stored procedure request was processed by
Unity.
The unique ID of the user whose query is logged.
Name under which the session being logged was initiated.
The number of bytes transferred by MultiLoad or FastLoad insertion, updates, or deletions.
Indicates whether an SQL request has utility information available in DBQLUtilityTbl. Possible values include:
•
Y for SQL requests from DSA and load/export utilities
•
N for SQL requests from other sources
The number of rows inserted, updated, and deleted by MultiLoad or
FastLoad for target tables.
VERYHOT logical I/O count per query.
VERYHOT logical I/Os in KB per query.
VERYHOT physical I/Os in KB per query.
VERYHOT physical I/O count per query (VERYHOT cylinder and data block I/Os that were handled by physical disk I/O).
Indicator if the error was only a warning. That is, warning mode.
The workload definition number assigned to the query.
292 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
DBC.DBQLogTbl Field
WDOverride
Description
Indicates if the workload definition assignment was overridden.
Note: The ExtraFieldXX columns will be assigned to new column names in the next major release of Teradata Database, where XX is the number of the column (for example,
DBC.DBQLStepTbl.ExtraField6).
View of the Main Log Table: QryLog[V]
Each row in the QryLog[V] contains default DBQL information for a query. The following table describes the fields in QryLog[V].
QryLog[V] Field
AbortFlag
AcctString
AcctStringDate
AcctStringHour
AcctStringTime
AMPCPUTime
AMPCPUTimeNorm
Description
If the query being logged was aborted, the collected data could be incomplete.
Note: A query consisting of a multistatement or iterated request with simple INSERT statements can be successful and yet have statement errors, as indicated by the ErrorCode column.
The account string at the time the row was committed. Because accounts can be changed dynamically at the query or session level, this may not be the same account that:
•
DBQL verified for the user at logon time
•
Was in effect when processing of the query began
Value of the expanded &D ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Even if ASE truncates the expanded account string, DBQL will report the full value. For example, if &D is expanded into position
26 or higher in the account definition and truncation occurs,
DBQL will still report the entire value of the date in the
AcctStringDate field.
Value of the expanded &H ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Value of the expanded &T ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Even if ASE truncates the expanded account string, DBQL will report the full value.
Total AMP time used for the query in CPU seconds.
Note: The I/O, CPU, and spool usage may not be accurate for aborted queries.
Normalized AMP CPU seconds for co-existence systems.
Database Administration 293
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
QryLog[V] Field
AppID
CacheFlag
CalendarName
CallNestingLevel
CheckpointNum
ClientAddr
ClientID
CollectTimeStamp
CPUDecayLevel
Description
Application (executable) or utility name; for example, BTEQ from
LogonSource for network clients.
This field is blank if the query is not found in the request cache. It could also be one of the following:
•
“T” if the query plan is generic and in the request cache.
•
“G” if the query plan is generic and not in the request cache but is a candidate for caching in subsequent runs of the same query.
•
“S” if the query plan is specific (either a static or dynamic plan).
•
“A” if a Specific Always decision is taken. That is, for each query
USING values are peeked during request parsing. The
Optimizer may choose a dynamic plan with results feedback.
For more information on specific plans and generic plans, see
“Peeking at Parameterized Values in the Data Parcel” in SQL
Request and Transaction Processing.
The name of the business calendar the session uses. Possible values include ISO, COMPATIBLE, and TERADATA (the default).
The level of nesting when running in stored procedures.
The checkpoint number when loading data into an MLOADX staging table. The default is zero.
The client IP address of the submitted query as obtained from
Logonsource from a network client.
The network client user name under which this session is logged
(for example, ADMIN) and the PK/FK for the table.
A date and time unique to each buffer cache, which changes for each new buffer. Part of the multi-column NUPI for the table (see also ProcID).
This time will be set when the cache is ready to receive new data, not when rows are written to the database.
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
Contains the most severe decay level reached for CPU usage on any one node for requests using the default timeshare method in
Priority Scheduler for SUSE Linux Enterprise Server 11 or above.
Values are 0, 1, or 2.
•
A value of 0 indicates that the request is running at the default priority.
•
A value of 1 indicates that the priority level is reduced and the resource allotment is halved.
•
A value of 2 indicates that the priority level is reduced again, the resource allotment is halved again, and it will not be reduced more.
294 Database Administration
Database Administration
QryLog[V] Field
DataCollectAlg
DBQLStatus
DefaultDatabase
DelayTime
MinRespHoldTime
TotalFirstRespTime
DisCPUTime
DisCPUTimeNorm
ElapsedTime
ErrorCode
ErrorText
EstMaxRowCount
EstProcTime
EstResultRows
ExpandAcctString
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
Description
CPU/IO collection algorithm used by DBQL. Possible values include:
•
1 = Use the classic algorithm with step adjustments.
•
2 = Use AMP algorithm 2 (diagnostic only).
•
3 = Use AMP algorithm 3 (includes information on aborted and parallel steps).
Internal DBQL logging status. Zero indicates no status conditions.
This field is NULL if you are not using DBQL CPU/IO Collection algorithm 3.
Name of the default database for the query.
The number of seconds a request was delayed by workload management.
The number of seconds the request was held to meet the TASM minimum response time.
The total time (in seconds) from when the request was submitted to when the first response was sent. Includes DelayTime and
MinRespHoldTime.
Dispatcher CPU time in seconds.
Dispatcher normalized CPU time in seconds.
The time between when the query is submitted and when the first response is sent to the host.
If greater than 0, this field is the number of the error message in
ErrorText. This field contains a code if the query caused a Parser syntax error.
If not null, this field provides processing errors and Parser errors.
For example, the text would explain that the query was not processed due to a Parser syntax error.
The largest number of rows handled by a single step in the query, as estimated by the Optimizer.
Sum of minimum estimated processing time for steps in the query as generated by the Optimizer.
Note: This time is reported in seconds.
Estimated result rows as generated by the Optimizer.
The expanded account name under which the query is submitted if account expansion is invoked.
Note: The &I, which gives the host, session, and request number for the query is different for a stored procedure call. For stored procedure calls, the request number is the client request number which is the request of the CALL statement.
295
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
QryLog[V] Field
FirstRespTime
FirstStepTime
InternalRequestNum
IODecayLevel
IterationCount
KeepFlag
LastRespTime
LockDelay
LockLevel
LogicalHostID
LogonDateTime
LogonSource
LSN
Description
The timestamp when the first response packet is sent to the client unless the request is held by TASM to meet the minimum response time. In this case, it is the timestamp when the first response packet is ready to be sent to the client.
Note: There is a LastRespTime field, but it is NULL if you are not using DBQL CPU/IO Collection algorithm 3.
The timestamp when the first step is dispatched.
The internal request number.
For commands, other than those within a stored procedure, the internal request number and the number in the RequestNum field will be the same.
For stored procedures invoked within a session, the internal request number increments by 1 for every request made by the stored procedure. This means that RequestNum will be the request number of the CALL and the value in InternalRequestNum will continue to increment for all other queries issued by the session.
Contains the most severe decay level reached for I/O usage on any one node for requests using the default timeshare method in
Priority Scheduler for SUSE Linux Enterprise Server 11 or above.
Values are 0, 1, or 2. A value of 0 indicates that the request is running at the default priority. A value of 1 indicates that the priority level is reduced and the resource allotment is halved. A value of 2 indicates that the priority level is reduced again, the resource allotment is halved again, and it will not be reduced more.
The number of data rows used by a parameterized request.
Whether the response parcel should be kept until the client responds that it is no longer needed. The value is either Y or N.
The timestamp of the last response. This field is NULL if you are not using DBQL CPU/IO Collection algorithm 3.
Maximum wait time to get a lock on an object in centiseconds.
The maximum value is 32,767 centiseconds.
The highest lock level (Access/Read/Write/Exclusive) for locks associated with this request. For explicit transactions, the lock level for the transaction. Does not include row hash locks.
Unique identifier of the logon source for the logged query. A value of 0 indicates an internal session.
The timestamp of the start of the session.
The logon source string text. For more information, see the appendix on LogonSource column fields and examples in Data
Dictionary.
The Logon Sequence Number used for a load utility.
296 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
QryLog[V] Field
MaxAMPCPUTime
MaxAMPCPUTimeNorm
MaxAmpIO
MaxCPUAmpNumber
MaxCPUAmpNumber
Norm
MaxIOAmpNumber
MaxStepMemory
MaxStepsInPar
MinAmpCPUTime
MinAmpCPUTimeNorm
NumFragments
NumOfActiveAMPs
NumRequestCtx
Description
CPU seconds of the highest CPU utilized AMP in the query.
Normalized maximum CPU seconds for an AMP.
Logical I/O count of the highest utilized AMP in the query.
The number of the AMP with the highest CPU activity.
Number of the AMP with the maximum normalized CPU seconds for co-existence systems.
The number of the AMP with the highest I/O usage for this step.
The maximum amount of memory in MBs used by any one AMP in any step in this request. This field can be compared with the
TDWM estimated memory usage field, TDWMEstMemUsage, also in DBQLogTbl. This field does not apply to INSERT/
UPDATE/DELETE requests.
The maximum number of Level 2 steps done in parallel for the query.
CPU seconds of the lowest CPU utilized AMP in the query.
Normalized minimum CPU seconds for an AMP.
Note: Normalized columns, such as MinAmpCPUTimeNorm, are for co-existence systems only.
For a request for which a dynamic plan is generated, this field indicates the number of plan fragments. For a static plan, this field is NULL.
Number of AMPs active for the query. Use it to compute the average CPU or I/O per AMP.
The number of request contexts associated with the session.
297
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
QryLog[V] Field
NumResultRows
NumSteps
NumStepswPar
ParserCPUTime
ParserCPUTimeNorm
ParserExpReq
PersistentSpool
ProcID
ProfileID
ProxyRole
ProxyUser
ProxyUserID
Description
The total number of rows returned for the query.
For an Enhanced Statement Status (ESS) client, the value
18,446,744,073,709,551,615 could mean that number or higher.
Note: The value formatted as a decimal value is
18,446,744,073,709,551,616 due to float arithmetic.
For a non-ESS client, a warning message is returned if the actual activity count exceeds 4294967295.
The same value (modulo 2^32) is logged in the NumResultRows column.
The value 4294967295 along with the warning message indicates the actual counter is one of the following:
•
A multiple of 4294967296 minus 1.
•
Is equal to 18,446,744,073,709,551,615.
•
Is greater than 18,446,744,073,709,551,615.
The maximum internal counter is 18,446,744,073,709,551,615.
The value 4294967295 or less, without the warning message, is the actual row count.
Total number of Level 1 steps required to process the query. The value is 0 if:
•
The query was aborted or you ended logging for the user before steps were generated.
•
It is a PREPARE statement. The PREPARE statement is logged separately from the corresponding SQL.
Total number of Level 1 steps with parallel steps.
Total Parser and Dispatcher CPU seconds used for the query.
Normalized parser CPU seconds for co-existence systems.
Seconds the parser waited on an express request.
Persistent portion of the SpoolUsage column.
Unique processor ID of the Dispatcher and part of the multicolumn NUPI for the table (see also CollectTimeStamp).
The name of the profile, if any, under which the user submitted the query.
The rolename of a proxy connection.
The username of a proxy connection.
The ID of the ProxyUser being charged for spool and temp space.
If the field is NULL, the session user ID was used for spool and temp space.
298 Database Administration
Database Administration
QryLog[V] Field
QueryBand
QueryID
QueryRedriven
QueryText
ReDriveKind
ReqIOKB
ReqPhysIO
ReqPhysIOKB
RequestMode
RequestNum
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
Description
The query band under which the query is submitted.
Possible values include:
•
=T> transaction query band pairs
•
=S> session query band pairs
•
=P> profile query band pairs
Internally generated identifier of the query and the FK to other
DBQL tables.
Note: QueryID is a system-wide unique field; you can use
QueryID to join DBQL tables or the DBC.TdwmExceptionLog table with DBQL tables without needing ProcID as an additional join field.
Whether the query was redriven if redrive protection was enabled.
Possible values are:
•
R = The request was redriven
•
N = The request was not redriven
•
E = The request was reparsed due to an internal Parser error.
•
M = The request was reparsed due to an internal Parser memory limit.
The first characters of the SQL query. Up to 200 characters are captured by default.
If you use the LIMIT SQLTEXT option, you can specify that the first n characters be captured where n is 0-10000.
Whether the query has redrive protection. Possible values are:
•
' ' = Not participating in redrive
•
MN = memory-based protection, no fallback spool
Total logical I/O usage in kilobytes.
The number of physical I/Os.
Total physical I/O usage in kilobytes.
Request mode from parcel request processing option:
•
Prep: Prepare mode (P)
•
Exec: Execute mode (E)
•
Both: Prepare and execute mode (B)
•
PrepS: Prepare, supporting parameterized SQL (S)
Unique identifier of the query.
Note: The request number for the commands that occur within a stored procedure CALL statement will all appear as the same number of the CALL statement itself. For more information on request numbers for stored procedures, see note for
ExpandAcctString.
299
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
QryLog[V] Field
SeqRespTime
SessionID
SessionTemporalQualifier
SpoolUsage
StartTime
StatementGroup
Statements
Description
Sum of the response time of all steps as if they had been executed sequentially, in seconds.
Unique session identifier.
When a DML or SELECT request refers to a temporal table but omits a temporal qualifier, the system uses the value of the session temporal qualifier. This is the session temporal qualifier in effect when the query is logged. For more information on the qualifier, see ANSI Temporal Table Support and Temporal Table Support.
If you are using algorithm 3 (the DBQLogTbl DataCollectAlg field= 3), the peak spool usage in bytes for any step of the query.
For other algorithms, the maximum amount of spool used while processing the query.
If SpoolUsage is 0, then there is no usage. If it is null, the data on usage was not valid.
The timestamp when the query is submitted.
If there is a DDL statement in a request, StatementGroup reports which type:
•
DDL ALTER
•
DDL CREATE
•
DDL GRANT
If the statement has only one DML statement or multiple DML statements that are all of the same type, StatementGroup will indicate the type. For example if there are three DELETE statements in a request, StatementGroup will report:
DML DELETE
Similarly, for requests with individual or multiple INSERT,
INSERT... SELECT, UPDATE or SELECT statements,
StatementGroup will report:
•
DML INSERT
•
DML INSERT... SELECT
•
DML UPDATE
•
SELECT
In a multistatement request with different types of DML statements, you will see a list showing the number of statements of each type in the request. For example, a request with one insert and two update statements will appear as:
DML Del=0 Ins=1 InsSel=0 Upd=2 Sel=0
Number of statements in a request:
•
1 = single-statement.
•
> 1 = multiple statements.
300 Database Administration
Database Administration
QryLog[V] Field
StatementType
TacticalCPUException
TacticalIOException
TDWMEstMemUsage
ThrottleBypassed
TotalIOCount
TTGranularity
TxnMode
TxnUniq
UnitySQL
UnityTime
UserID
UserName
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Main Log Table: QryLog[V]
Description
The type of statement of the query.
In a multistatement request, this is the last statement of the request. However, this may not accurately describe the request. For more statement information, see StatementGroup.
The number of nodes that had a CPU exception for the request.
The number of nodes that had an I/O exception for the request.
The estimated memory used by the request (in megabytes). It represents the largest step in the request.
Whether an active request is active solely due to the
ThrottleBypass ruleset attribute. This attribute overrides the throttle limits if the session owning the request has an object lock higher than the Access level. Possible values include:
•
ThrottleBypass ruleset attribute is set.
•
ThrottleBypass ruleset attribute is not set.
Total logical I/O used by the query.
Note: The I/O, CPU, and spool usage may not be accurate for aborted queries.
Possible values include:
•
LogicalRow: Row is timestamped with the time the row is processed by the AMP.
•
Request: Row is timestamped with the time the request is submitted.
•
Transaction: Row is timestamped with the time when the first non-locking reference is made to a temporal table, or when the built-in function TEMPORAL_TIMESTAMP is first accessed during the transaction.
For more information, see Temporal Table Support.
Transaction mode:
•
ANSI: ANSI transaction mode
•
BTET: Teradata transaction mode
The unique value within the transaction number. Other transaction number components (userid, procid) are already in
DBQLogTbl. Combined with TxnUniq, these fields represent the complete transaction ID for the logged request.
Possible values include:
•
Y = Unity requested the SQL to be modified.
•
N = Unity request has not been made.
Seconds the stored procedure request was processed by Unity.
The unique ID of the user whose query is logged.
Name under which the session being logged was initiated.
301
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Teradata Dynamic Workload Management View of the Main Log Table: QryLogTDWM[V]
QryLog[V] Field
UtilityByteCount
UtilityInfoAvailable
UtilityRowCount
VHLogicalIO
VHLogicalIOKB
VHPhysIO
VHPhysIOKB
WarningOnly
Description
The number of bytes transferred by MultiLoad or FastLoad insertion, updates, or deletions.
Indicates whether an SQL request has utility information available in DBQLUtilityTbl. Possible values include:
•
Y for SQL requests from DSA and load/export utilities
•
N for SQL requests from other sources
The number of rows inserted, updated, and deleted by MultiLoad or FastLoad for target tables.
VERYHOT logical I/O count per query.
VERYHOT logical I/Os in KB per query.
VERYHOT physical I/O count per query (VERYHOT cylinder and data block I/Os that were handled by physical disk I/O).
VERYHOT physical I/Os in KB per query.
Indicator if the error was only a warning. That is, warning mode.
Teradata Dynamic Workload Management View of the Main Log Table: QryLogTDWM[V]
QryLogTDWM[V] is the Teradata Dynamic Workload Management view of the main log table, DBQLLogTbl. The following table describes the fields in QryLogTDWM[V].
QryLogTDWM[V] Field
AcctString
CollectTimeStamp
DefaultDatabase
DelayTime
Description
The account string at the time the row was committed. Because accounts can be changed dynamically at the query or session level, this may not be the same account that:
•
DBQL verified for the user at logon time
•
Was in effect when processing of the query began
A date and time unique to each buffer cache, which changes for each new buffer. Part of the multi-column NUPI for the table (see also ProcID).
This time will be set when the cache is ready to receive new data, not when rows are written to the database.
For more information on CollectTimeStamp, see
CollectTimeStamp Value Between Tables” on page 381
.
Name of the default database for the query.
The seconds a query was delayed by the Teradata dynamic workload management software.
302 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Teradata Dynamic Workload Management View of the Main Log Table: QryLogTDWM[V]
QryLogTDWM[V] Field
ExceptionValue
FinalWDID
InternalRequestNum
LastStateChange
LogicalHostID
LSN
MinRespHoldTime
TotalFirstRespTime
NoClassification
OpEnvID
ProcID
QueryID
RequestNum
ResponseTimeMet
SessionID
SessionWDID
Description
Specifies what type of Teradata dynamic workload management software exception occurred.
Indicates the workload definition in which the query completed execution.
The internal request number.
For commands, other than those within a stored procedure, the internal request number and the number in the RequestNum field will be the same.
For stored procedures invoked within a session, the internal request number increments by 1 for every request made by the stored procedure.
This means that RequestNum will be the request number of the CALL and the value in InternalRequestNum will continue to increment for all other queries issued by the session.
Time of the last State change by the Teradata dynamic workload management software.
Unique identifier of the logon source for the logged query. A value of 0 indicates an internal session.
The Logon Sequence Number used for a load utility.
The number of seconds the request was held to meet the minimum response time.
Total time (in seconds) from when the request was submitted to when the first response was sent. Includes DelayTime and MinRespHoldTime.
Indicates if the query was not classified.
The internal identifier of the Operating Environment/Planned
Environment currently enforced by the Teradata dynamic workload management software.
Unique processor ID of the Dispatcher and part of the multi-column
NUPI for the table (see also CollectTimeStamp).
Internally generated identifier of the query.
Unique identifier of the query.
Note: The request number for the commands that occur within a stored procedure CALL statement will all appear as the same number of the
CALL statement itself. For more information on request numbers for stored procedures, see note for ExpandAcctString.
Indicates if the query met service level goals.
Unique session identifier.
Workload ID class associated with the session. It determines the priority of the session when the TDWM workload classification is active.
303
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Teradata Dynamic Workload Management View of the Main Log Table: QryLogTDWM[V]
QryLogTDWM[V] Field
StatementGroup
SysConID
TDWMAllAmpFlag Indicates whether one of the steps of the query is an all-amp step. This is used to classify a query into a workload definition.
TDWMConfLevelUsed
Note: No longer a TASM rule attribute. The letter 'N' will always be logged.
TDWMEstLastRows
TDWMEstMaxRows
TDWMEstMemUsage
The estimated last row count generated by the Optimizer used to classify a query into a workload definition.
The estimated maximum row count generated by the Optimizer used to classify a query into a workload definition.
The estimated memory used by the request (in megabytes). It represents the largest step in the request.
TDWMEstTotalTime
Description
If there is a DDL statement in a request, StatementGroup reports which type:
•
DDL ALTER
•
DDL CREATE
•
DDL GRANT
If the statement has only one DML statement or multiple DML statements that are all of the same type, StatementGroup will indicate the type. For example if there are three DELETE statements in a request,
StatementGroup will report:
DML DELETE
Similarly, for requests with individual or multiple INSERT, INSERT...
SELECT, UPDATE or SELECT statements, StatementGroup will report:
•
DML INSERT
•
DML INSERT... SELECT
•
DML UPDATE
•
SELECT
In a multistatement request with different types of DML statements, you will see a list showing the number of statements of each type in the request. For example, a request with one insert and two update statements will appear as:
DML Del=0 Ins=1 InsSel=0 Upd=2 Sel=0
The internal identifier of the System Condition (SysCon)/ System Health currently enforced by the Teradata dynamic workload management software.
TDWMRuleID
UserID
The total estimated time of a query by the Teradata dynamic workload management software (based on Optimizer estimates) and applied against rules read from the TDWM database.
Note: This time is reported in milliseconds in DBC.DBQLogTbl, but it is reported in seconds in the view DBC.QryLogTDWM[V].
Rule identifier of the query.
The unique ID of the user whose query is logged.
304 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Explain Log Table: DBQLExplainTbl
QryLogTDWM[V] Field
UserName
WDID
WDOverride
Description
Name under which the session being logged was initiated.
The workload definition number assigned to the query.
Indicates if the workload definition assignment was overridden.
Explain Log Table: DBQLExplainTbl
This table is populated if the WITH EXPLAIN option is requested in addition to the default information row. If the additional Explain text is greater than 31,000 characters, multiple rows are generated.
DBC.DBQLExplainTbl field Description
ProcID Unique processor ID of the Dispatcher and part of the multi-column
NUPI (also see CollectTimeStamp).
CollectTimeStamp Time that the rows were written to the database. This field is also part of the multicolumn NUPI (also see ProcID).
For more information on CollectTimeStamp, see
CollectTimeStamp Value Between Tables” on page 381
.
QueryID
ExpRowNo
ExplainText
Internally generated identifier of the query and the FK to other DBQL tables.
Note: QueryID is a system-wide unique field; You can use QueryID to join DBQL tables or the DBC.TdwmExceptionLog table with DBQL tables without needing ProcID as an additional join field.
Used if Explain text is greater than 31,000 characters and the system generates multiple rows.
Full Explain text segment.
The text is not formatted and appears as a single long string. You may need to program a parsing application to look at the text.
Note: If ExplainText has more than one row per QueryID, use the
QryLogExplainDoc[V] view to see a single, readable document per
QueryID. For more information, see “Improving Explain Text
View of the EXPLAIN Log Table:
QryLogExplainDoc[V]
The QryLogExplainDoc[V] view is populated if the WITH EXPLAIN option is requested in addition to the information row. QryLogExplainDoc[V] provides a view into
Database Administration 305
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the EXPLAIN Log Table: QryLogExplainDoc[V]
DBQLExplainTbl that combines multiple rows of ExplainText with the same QueryID into a single ExplainTextDoc document.
The following table describes the fields in QryLogExplainDoc[V].
QryLogExplainDoc[V] Field
ProcID
CollectTimestamp
QueryID
ExplainTextDoc
Description
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (also see CollectTimeStamp).
Time that the rows were written to the database. This field is also part of the multicolumn NUPI (also see ProcID).
Internally generated identifier of the query and the FK to other
DBQL tables.
Note: QueryID is a system-wide unique field; You can use QueryID to join DBQL tables or the DBC.TdwmExceptionLog table with
DBQL tables without needing ProcID as an additional join field.
ExplainTextDoc combines multiple ExplainText rows with the same
QueryID into a single document as CLOB type.
Improving Explain Text Readability
To combine multiple ExplainText doc rows with the same query ID into one easily readable document, perform a SELECT request on the QryLogExplainDoc[V] view.
Example
sel queryid,explaintextdoc from qrylogexplaindocv where queryid=307188741814475132;
The result looks like the following partial output:
*** Query completed. One row found. 2 columns returned.
*** Total elapsed time was 1 second.
QueryID ExplainTextDoc
307188741814475132 1) First, we lock Test_DB.b12 for read on a reserved rowHash to prevent global deadlock. 2) Next, we lock Test_DB.b3 for read on a reserved rowHash to prevent global deadlock. 3) We lock Test_DB.b12 for read, and we lock Test_DB.b3 for read. 4) We do an all-AMPs RETRIEVE step from Test_DB.b3 by way of an all-rows scan with a condition of ("NOT (Test_DB.b3.c2 IS NULL)") into Spool 2 (all_amps), which is redistributed by the hash code of (Test_DB.b3.c2) to all AMPs. Then we do a SORT to order
Spool 2 by row hash. The size of Spool 2 is estimated with low confidence to be 4 rows
(68 bytes). The estimated time for this step is 0.03 seconds. 5) We execute the following steps in parallel....
Note: Export the result set to a file. On the monitor, some of the lines may truncate.
The memory allocated for the input is 32 MB. To see the memory setting, use the following: cufconfig -o | grep MallocLimit
To add more memory, adjust the MallocLimit setting using cufconfig. For more information about cufconfig, see Utilities.
306 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the EXPLAIN Log Table: QryLogExplain[V]
View of the EXPLAIN Log Table:
QryLogExplain[V]
The QryLogExplain[V] view is populated if the WITH EXPLAIN option is requested in addition to the information row. The following table describes the fields in
QryLogExplain[V].
QryLogExplain[V] Field Description
CollectTimestamp Time that the rows were written to the database. This field is also part of the multicolumn NUPI (also see ProcID).
ExplainText Full Explain text segment.
The text is not formatted and appears as a single long string. You may need to program a parsing application to look at the text.
ExpRowNo
ProcID
QueryID
Used if Explain text is greater than 31,000 characters and the system generates multiple rows.
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (also see CollectTimeStamp).
Internally generated identifier of the query and the FK to other DBQL tables.
Note: QueryID is a system-wide unique field; You can use QueryID to join DBQL tables or the DBC.TdwmExceptionLog table with DBQL tables without needing ProcID as an additional join field.
Object Log Table: DBQLObjTbl
With the WITH OBJECTS option, DBQL logs one row in DBQLObjTbl for each data object referenced by the query. An object can be a database, data table, column, secondary index, join index, or journal table. (If the object is a secondary index, its number is logged rather than a name.) DBQL gets the use counts from the Optimizer and not the SQL statement itself. The system logs a count of the number of times the object was accessed by the Optimizer.
Note: Any DBC database tables and columns used by the system while processing a query are not reflected in the DBQL object rows for that query. This means, for example, that statements like CREATE TABLE or SELECT FROM DBC.xxx will not have objects logged through DBQL because they deal with DBC tables and columns.
Database Administration 307
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Object Log Table: DBQLObjTbl
Row Values
The following table lists the fields populated in an objects row.
DBC.DBQLObjTbl Field
CollectTimeStamp
FreqofUse
ObjectColumnName
ObjectDatabaseName
ObjectID
ObjectNum
ObjectTableName
ObjectType
ProcID
QueryID
TypeOfUse
Description
Time that the rows were written to the database. This field is also part of the multi-column NUPI (see also ProcID).
For more information on CollectTimeStamp, see
CollectTimeStamp Value Between Tables” on page 381
.
Number of times the object was accessed, as determined by the
Optimizer, to process the query.
Name of the column. Or if the ObjectType is 'Idx' or Index, this field returns the name of the column associated with the index.
Some queries will not provide a name, such as COLLECT
STATISTICS.
For multicolumn indexes, there will be one additional row for each column in the index and each row will have the same object number.
Name of the database that owns the target object.
Unique internal identifier of this object.
Number of the column or secondary index.
Name of the table or view.
Character code indicating the type of object targeted.
For the possible values for this column, see
“Chapter 15 ObjectType” on page 261 .
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (see also CollectTimeStamp).
Internally-generated identifier of the query.
Note: QueryID is a system-wide unique field; You can use QueryID to join DBQL tables or the DBC.TdwmExceptionLog table with
DBQL tables without needing ProcID as an additional join field.
Describes the use of the object. TypeOfUse contains the following numerical values:
•
1 = Found in the resolver
•
2 = Accessed during query processing
•
4 = Found in a conditional context
•
8 = Found in inner join condition
•
16 = Found in outer join condition
•
32 = Found in a sum node
•
64 = Found in a full outer join condition
Note: More than one of these values can be used.
308 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
ObjectType
ObjectType
Value
Agg
AgS
Aut
Col
DB
Description
User-defined aggregate function
User-defined aggregate STAT function
Security authorization
Column
Database
Database Administration
Tab
TbF
Tmp
TbO
TbC
Trg
UDF
Idx
JIx
Index. For each index, there is a database name, table name, and column name.
The ObjectId column is the identifier of the table and the ObjectNum column is the number of the index in that table. For multi-column indexes, there is one row for each column of the index that a query used. For example, if an index consists of three columns and the query uses all three, there will be three rows, each with a different column name. The column name will be null for an index for statements such as COLLECT STATISTICS, ALTER PROCEDURE, SHOW PROCEDURE, or
SELECT COUNT(*).
Join index. For each join index, there is a database name and join index name in the ObjectTableName field. For these rows, the ColumnName indicates a column referred to by the join index.
•
ObjectType is ‘JIx.’
•
ObjectId matches the ID of the join index.
•
ObjectNum is 0.
Jrl Journal
Mac Macro
NoT
SP
No type (unknown)
Stored procedure
Table
Table function
Temporary
Table operator
Contract function
Trigger
User-defined function
309
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Object Log Table: QryLogObjects[V]
Value
UDM
UDT
Viw
Vol
XSP
Description
User-defined method
User-defined type
View
Volatile
External stored procedure
View of the Object Log Table:
QryLogObjects[V]
With the WITH OBJECTS option, DBQL logs one row in DBQLObjTbl and the associated view, QryLogObjects[V], for each data object referenced by the query. An object can be a database, data table, column, secondary index, join index, or journal table. (If the object is a secondary index, its number is logged rather than a name.) DBQL gets the use counts from the Optimizer and not the SQL statement itself. The system logs a count of the number of times the Optimizer accessed the object.
Note: Any DBC database tables and columns used by the system while processing a query are not reflected in the DBQL object rows for that query. This means, for example, that statements like CREATE TABLE or SELECT FROM DBC.xxx will not have objects logged through DBQL because they deal with DBC tables and columns.
The following table lists the fields populated in the object log view.
QryLogObjects[V] Field Description
CollectTimeStamp Time that the rows were written to the database.
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
FreqofUse
ObjectColumnName
Number of times the object was accessed, as determined by the Optimizer, to process the query.
Name of the column. Or if the ObjectType is 'Idx' or Index, this field returns the name of the column associated with the index.
Some queries will not provide a name, such as COLLECT STATISTICS.
For multicolumn indexes, there will be one additional row for each column in the index and each row will have the same object number.
ObjectDatabaseName
ObjectID
ObjectNum
Name of the database that owns the target object.
Unique internal identifier of this object.
Number of the column or secondary index.
310 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Parameterized Query Log Table: DBQLParamTbl
QryLogObjects[V] Field Description
ObjectTableName
ObjectType
Name of the table or view.
Character code indicating the type of object targeted.
For the possible values for this column, see
ProcID
QueryID
TypeOfUse
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (see also CollectTimeStamp).
Internally generated identifier of the query.
Describes the use of the object. TypeOfUse contains the following numerical values:
•
1 = Found in the resolver
•
2 = Accessed during query processing
•
4 = Found in a conditional context
•
8 = Found in inner join condition
•
16 = Found in outer join condition
•
32 = Found in a sum node
•
64 = Found in a full outer join condition
More than one of these values can be used.
Parameterized Query Log Table: DBQLParamTbl
The DBQLParamTbl logs the parameter variable name, type, position, and value for each parameter in a parameterized request. The table is populated if you specify the PARAMINFO option for the BEGIN/REPLACE QUERY LOGGING statement.
Note: This table may contain confidential user information and should be accessed only by trusted personnel with access to the user DBC password. For more information about the user
DBC password, see Security Administration.
Row Values
The following table describes the fields in a DBQLParamTbl row.
DBC.DBQLParamTbl field
CollectTimeStamp
Description
Time the query log was generated.
Database Administration 311
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Parameterized Query Log Table: QryLogParam[V]
DBC.DBQLParamTbl field
ParamInfo
ProcID
QueryID
ZoneID
Description
Name, type, position, and value for each parameter in the parameterized request.
Where the parameter marker or positional parameter is “?” in the request, DBQL replaces the variable names with system-generated names
P1 to Pn. The numbering 1 to n stands for the parameter positions from left to right as they appear in the request text.
ID of the processor that handled the request being logged.
Query ID of the parameterized request.
Reserved for future use.
The TD_DBQLParam embedded services function converts the contents of the BLOB type
ParamInfo column into JavaScript Object Notation (JSON) format for use in diagnosing parameterized query problems.
View of the Parameterized Query Log Table:
QryLogParam[V]
The QryLogParam[V] view is populated if parameterized request logging is enabled with the
PARAMINFO option for BEGIN/REPLACE QUERY LOGGING and this view
(QryLogParam[V])is included in a request.
For example:
SELECT QueryID, RowNum, ParamJSON from QryLogParam; converts all the rows in DBQLParamTbl to JSON documents. The output is in CLOB format.
To perform JSON operations on the data, use the QryLogParamJSON view instead.
Row Values
The following table lists the populated fields in the QryLogParam[V] view.
QryLogParam[V] Field
ParamJSON
QueryID
RowNum
Description
Parameter and data rows in CLOB format.
Query ID of the parameterized request.
Row number indicating the sequence of
ParamInfo and data record JSON documents.
312 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Parameterized Query Log Table: QryLogParamJSON[V]
View of the Parameterized Query Log Table:
QryLogParamJSON[V]
The QryLogParamJSON[V] view is populated if parameterized request logging is enabled with the PARAMINFO option for BEGIN/REPLACE QUERY LOGGING and this view
(QryLogParamJSON[V]) is included in a request.
For example:
SELECT QueryID, RowNum, ParamJSON from QryLogParamJSON; converts all the rows in DBQLParamTbl to JSON documents. With this view, users can perform JSON operations on the ParamJSON column data.
Row Values
The following table lists the populated fields in the QryLogParamJSON[V] view.
QryLogParamJSON[V] Field
ParamJSON
QueryID
RowNum
Description
Parameter and data rows in JSON format.
Query ID of the parameterized request.
Row number indicating the sequence of
ParamInfo and data record JSON documents.
SQL Log Table: DBQLSQLTbl
The DBQLSQLTbl table is populated if you specify the WITH SQL option in addition to default information. The WITH SQL option causes DBQL to attempt to log the full statement text, no matter how large, into the DBQLSQLTbl table. Multiple rows are generated as necessary.
Row Values
The following table lists the populated fields in a DBQLSQLTbl row.
DBC.DBQLSQLTbl Field
CollectTimeStamp
ProcID
Description
Time that the rows were written to the database. This field is also part of the multi-column NUPI (see also ProcID).
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (also see CollectTimeStamp).
Database Administration 313
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the SQL Log Table: QryLogSQLDoc[V]
DBC.DBQLSQLTbl Field
QueryID
SQLRowNo
SQLTextInfo
Description
Internally generated identifier of the query.
Note: QueryID is a system-wide unique field; You can use QueryID to join DBQL tables or the DBC.TdwmExceptionLog table with DBQL tables without needing ProcID as an additional join field.
Row number of the statement. Value is 1 unless the statement is large and requires multiple rows.
A full SQL text segment, a string of up to approximately 32,000 characters.
Note: BTEQ has a limited column length. Use Teradata SQL Assistant or Teradata Studio to display longer lengths.
Note: If SQLTextInfo has more than one row per QueryID, use the
QryLogSQLDoc[V] view to see a single, readable document per
QueryID. For more information, see
“Improving SQL Text Readability” on page 315
.
When SQL is Not Recorded
If a query encounters an error or is aborted, the data in the field SQLTextInfo might be:
•
Truncated
•
Set to Null (?)
•
Set to “Unavailable”
The word “Unavailable” indicates that the system failed to translate correctly (for example, if the system must translate to UNICODE).
Also, in some cases, DBQL does not log the entire SQL statement. For example, DBQL logs the
CREATE/REPLACE PROCEDURE statements including the arguments, but not the additional options nor statements making up the body of the stored procedure.
View of the SQL Log Table: QryLogSQLDoc[V]
The QryLogSQLDoc[V] view is populated if you request that SQL is shown for a query along with the default information. The WITH SQL option causes DBQL to attempt to log the full statement text, no matter how large, into the DBQLSQLTbl table.
QryLogSQLDoc[V] combines multiple rows of SQLTextInfo with the same QueryID into a single SQLTextDoc document. The following table lists the populated fields in the
QryLogSQLDoc[V] view.
314 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the SQL Log Table: QryLogSQL[V]
QryLogSQLDoc[V]
Field
CollectTimestamp
ProcID
QueryID
SqlTextDoc
Description
Time that the rows were written to the database. This field is also part of the multi-column NUPI (see also ProcID).
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
Unique processor ID of the Dispatcher and part of the multi-column NUPI
(also see CollectTimeStamp).
Internally generated identifier of the query.
Combines multiple SQLTextInfo rows with the same QueryID into a single
XML document as CLOB type.
Improving SQL Text Readability
To combine multiple SQLTextDoc rows with the same query ID into one easily readable document, perform a SELECT request on the QryLogSQLDoc[V] view.
Example
sel queryid,sqltextdoc from qrylogsqldocv where queryid=977188741514475132;
Note: Export the result set to a file. On the monitor, some of the lines may truncate.
The memory allocated for the input is 32 MB. To see the memory setting, use the following: cufconfig -o | grep MallocLimit
To add more memory, adjust the MallocLimit setting using cufconfig. For more information, see Utilities.
View of the SQL Log Table: QryLogSQL[V]
The QryLogSQL[V] view is populated if you request that SQL is shown for a query along with the default information. The WITH SQL option causes DBQL to attempt to log the full statement text, no matter how large, into the DBQLSQLTbl table. Multiple rows are generated as necessary. The following table lists the populated fields in the QryLogSQL[V] view.
QryLogSQL[V] Field Description
CollectTimestamp Time that the rows were written to the database. This field is also part of the multi-column NUPI (see also ProcID).
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
ProcID Unique processor ID of the Dispatcher and part of the multi-column NUPI
(also see CollectTimeStamp).
Database Administration 315
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
QryLogSQL[V] Field Description
QueryID
SqlRowNo
Internally-generated identifier of the query.
Row number of the statement. Value is 1 unless the statement is large and requires multiple rows.
SqlTextInfo A full SQL text segment, a string of up to approximately 32,000 characters.
Note: BTEQ has a limited column length. Use Teradata SQL Assistant or
Teradata Studio to display longer lengths.
Query Step Information Log Table:
DBQLStepTbl
DBQLStepTbl is populated if you specify the WITH STEPINFO option. When the query completes, the system logs one row for each query step, including parallel steps.
Row Values
The following table lists the populated fields in a DBQLStepTbl row.
DBC.DBQLStepTbl Field
CollectTimeStamp
CPUTime
CPUTimeNorm
DispatchSeq
EstCPUCost
EstHRCost
Description
Time that the rows were written to the database. This field is also part of the multi-column NUPI (see also ProcID).
For more information on CollectTimeStamp, see
CollectTimeStamp Value Between Tables” on page 381
.
This is the CPU seconds for this step.
Note: For requests using algorithm 1, the CPUTime shown in the first parallel step represents usage for the entire set of parallel steps.
This is the normalized AMP CPU seconds for this step (used for coexistent systems).
Note: For requests using algorithm 1, the CPUTimeNorm shown in the first parallel step represents usage for the entire set of parallel steps.
This field is populated only for requests that use algorithm 3 of
DBQL CPU/IO Collection.
Dispatch sequence, the order in which the step was handled by the dispatcher itself or dispatched to the AMPs, beginning with '1'.
An estimate of the milliseconds of CPU time for the step, as determined by the Optimizer.
An estimate of other costs for the step. There is no comparable actual data for this estimate.
316 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
DBC.DBQLStepTbl Field
EstIOCost
EstNetCost
EstProcTime
EstRowCount
EstRowCountSkew
EstRowCountSkewMatch
ExtraField19
ExtraField23
ExtraField24
FragmentNum
IOCount
IOKB
LockDelay
LSN
MaxAmpCPUTime
MaxAmpCPUTimeNorm
MaxAmpIO
Description
An estimate of service time in milliseconds for I/O for this step.
There is no comparable actual data for this estimate.
An estimate of the BYNET service time in milliseconds for the step.
There is no comparable actual data for this estimate.
The estimated processing time as determined by the Optimizer.
Note: This time is reported in seconds.
The estimated row count for this step.
For a Partial Redistribution and Partial Duplication (PRPD) plan split step, EstRowCount is the estimated row counts for all split spools.
Note: A split step is a RETRIEVE or JOIN step with “split into” appearing in the EXPLAIN when target spools are generated.
In PRPD, the estimated row count for the skew split spool (which contains the rows with skewed values of the relation in this step).
In PRPD, the estimated row count for the skew match split spool
(which contains the rows with skewed values of the other relation to be joined with the relation in this step).
The persistent portion of the DBQLStepTbl.SpoolUsage column.
Most significant integer part of the persistent portion of the
DBQLStepTbl.SpoolUsage column.
Least significant integer part of the persistent portion of the
DBQLStepTbl.SpoolUsage column.
Indicates the plan fragment to which the current step belongs. For a static plan, this is set to NULL.
This is the number of logical I/Os used by this step.
Note: For requests using algorithm 1, the IOCount shown in the first parallel step represents the entire set of parallel steps.
Logical I/Os in KB per step.
Maximum wait time to get a lock on an object in centiseconds. The maximum value is 32,767 centiseconds.
The Logon Sequence Number used by load utility.
CPU secondsof the highest CPU utilized AMP in the step.
Normalized maximum CPU seconds for an AMP.
Note: Normalized columns, such as MaxAmpCPUTimNorm, are for co-existence systems only.
The highest logical I/O count on the AMP with the highest logical
I/O usage for the step.
317
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
DBC.DBQLStepTbl Field
MaxAMPSpool
Description
If you are using DBQL CPU/IO Collection algorithm 3,
MaxAMPSpool is the largest peak spool usage of any AMP for this step. For other algorithms, MaxAMPSpool is the highest spool usage on any AMP at the end of the step.
MaxCPUAmpNumber
MaxIOAmpNumber
The number of the AMP with the highest CPU usage for the step.
MaxCPUAmpNumberNorm Number of the AMP with the maximum normalized CPU seconds for co-existence systems.
The number of the AMP with the highest logical I/O usage for the step.
MaxSpoolAmpNumber
MinAmpCPUTime
MinAmpCPUTimeNorm
Number of the AMP with the highest spool usage.
CPU seconds of the lowest CPU utilized AMP in the step.
Normalized minimum CPU seconds for an AMP.
Note: Normalized columns, such as MinAmpCPUTimeNorm, are for co-existence systems only.
MinAmpIO
MinAMPSpool
NumCombinedPartitions
NumContexts
NumCPReferences
NumofActiveAMPs
PhysIO
PhysIOKB
ProcID
QueryID
Logical I/O count of the lowest logical I/O utilized AMP in the query.
If you are using DBQL CPU/IO Collection algorithm 3,
MinAMPSpool is the smallest peak spool usage of any AMP for this step. For other algorithms, MinAMPSpool is the lowest spool usage of any AMP at the end of the step.
If there is static partition elimination for the step or if a noPI table or join index has column partitions, the number of combined partitions accessed (not eliminated). Otherwise, the column is null.
For column-partitioned tables or join indexes: the number of contexts allocated to access partitions at the same time as a source or target relation that has partitioning. Otherwise, the column is null.
For column-partitioned tables or join indexes: the number of column partitions referenced in a column-partitioned source or target. Otherwise, the column is null.
The number of AMPs involved with the step.
Physical I/O count per step.
Physical I/Os in KB per step.
Unique processor ID of the Dispatcher and part of the multicolumn NUPI (see also CollectTimeStamp).
Internally-generated identifier of the query.
Note: QueryID is a system-wide unique field; You can use QueryID to join DBQL tables or the DBC.TdwmExceptionLog table with
DBQL tables without needing ProcID as an additional join field.
318 Database Administration
Database Administration
DBC.DBQLStepTbl Field
RowCount
RowCount2
RowCount3
RowsWComprColumns
SpoolUsage
SSRReceiverCount
StatementNum
StepInstance
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
Description
For:
•
The MRM (Merge Row Multiple) or EXE (MultiLoad) steps,
RowCount is the number of rows inserted.
•
All other steps, RowCount is the actual number of rows the step returns (indicating activity count). For a PRPD plan split step,
RowCount includes the rows from all split spools.
Note: If “split into” appears in the EXPLAIN of a RETRIEVE or
JOIN step, Teradata Database is using PRPD.
For:
•
The MRM (Merge Row Multiple) or EXE (MultiLoad) steps,
RowCount2 is the number of updated rows.
•
An LFI (FastLoad) step, RowCount2 is the number of rows loaded.
•
A PRPD plan split step, RowCount2 is the actual number of rows in the skew split spool.
For:
•
The EXE (MultiLoad) step, RowCount3 is the number of rows deleted.
Note: For the number of rows inserted by the EXE step, see
RowCount. For the number of rows updated by the EXE step, see
RowCount2.
•
A PRPD plan split step, RowCount3 is the actual number of rows in the skew match split spool.
Number of rows with compressed columns.
Spool usage varies during a step. If you are using DBQL CPU/IO
Collection algorithm 3, SpoolUsage is the peak spool usage in bytes for this step. For other algorithms, SpoolUsage is the amount of spool usage for the AMP at the end of the step.
The number of Single-Sender Redistribution (SSR) receiver AMPs that the step uses to redistribute rows. If the step does not use SSR for row redistribution, the value is zero.
Statement number with which the step is associated. If associated with multiple statements (for example, array insert), this is the highest statement number associated with this step.
This field is populated only for requests that use algorithm 3 of
DBQL CPU/IO Collection.
Step instance number for steps that are repeated. If a step is dispatched 3 times, there will be one summarized step logged for the 3 instances of the same step number and the step instance field will be 3, indicating that 3 steps instances are represented. This field will be 1 for non-repeated steps.
319
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
DBC.DBQLStepTbl Field
StepLev1Num
StepLev2Num
StepName
StepStartTime
StepStatus
StepStopTime
StepWD
Description
Step number. If this row is for a parallel step, the number is repeated for as many parallel steps as were used for this step (for examples, see StepLev2Num).
If this row is for a parallel step, this is the second-level number of the step that spawned it.
For example, if the value of StepLevl1Num is 4 and this row logs the first parallel step for step 4, this value is 1 (and would read 04 01). If this row logs the second row generated for step 4, this value is 2 (and would read 04 02), and so forth.
If this row is not a parallel-step row, this value is 0.
Abbreviation of the internal name used to identify this step (for example, DEL for a DELETE step).“RESP” is a pseudo step that may appear after the last response of a DBQL logged request, depending on the setting of the DBS Control utility field DBQLLogLastResp.
Timestamp when the step was sent to the AMP to the nearest micro second.
This field is populated only for requests that use algorithm 3 of
DBQL CPU/IO Collection.
The step status, indicating dispatcher and AMP step disposition.
Values include:
•
Dispatched: Step was dispatched to the AMP.
•
AMPexec: Step was executed in the AMP.
•
AMPokay: Step completed okay in the AMP.
•
AMPerror: Step failed in the AMP and returned an error.
•
AMPReqAB: Step aborted in the AMP for request abort.
•
AMPTxnAB: Step aborted in the AMP for transaction abort.
•
Completed: Step completed by the dispatcher.
•
Skipped: Conditional step was skipped by the dispatcher.
Timestamp when the step returned from the AMP to the nearest micro second.
The Workload Definition in effect for the step.
320 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
DBC.DBQLStepTbl Field
TriggerKind
TriggerNestLevel
Description
TriggerLevelKind: A composite of trigger details.
ExtraField21 = TriggerKind + 100,000 * TriggerNestingLevel. To interpret this field, see the TriggerKind and TriggerNestingLevel fields of the QryLogSteps view.
TriggerKind bit flags:
•
TrigNone: 0
•
GetActivityCnt: 1
•
FetchQualifyingRows: 2
•
BuildTrigUsingRow: 4
•
BeginLoop: 8
•
EndLoop: 16
•
IdColWithRowTrig: 32
•
SkipQualifyingRows: 64
•
Zero: Step was not part of a trigger
•
Nonzero: Step was part of a trigger
The TriggerNestingLevel can range from 0-255.
UtilityTableId
VHLogicalIO
VHLogicalIOKB
VHPhysIO
VHPhysIOKB
The table ID for the utility.
This field is always 0.
VERYHOT logical I/O count per step.
VERYHOT logical I/Os in KB per step.
VERYHOT physical I/O count per step (VERYHOT cylinder index and data block I/Os that were handled by physical disk I/Os.)
VERYHOT physical I/Os in KB per step.
Note: The ExtraFieldXX columns will be assigned to new column names in the next major release of Teradata Database, where XX is the number of the column (for example,
DBC.DBQLStepTbl.ExtraField6).
Step Descriptions
The steps reflected in the DBQLStepTbl are generated by Teradata Database to execute a query. They may not always coincide with steps seen in the Explain because some steps are for internal use only.
The following table lists the possible logged step names and their meanings.
Step Internal Name Description
44-3 Performance Monitor spoil step
AAB Asynchronous abort of a DBC/SQL request
Database Administration 321
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
ALT
DIX
DJT
DRI
DRIDR
DSPRET
DTB
DTmp
EBD
EBE
EDM
EDR
Edt
CSUPD
Ctb
CTRts
CWD
DBQLDC
DEL
DELQT
DIRI
BIL
BLC
BMSMS
CIX
CkNPK
CKP
CRI
CSSUM
Step Internal Name Description
Insert target and originating table IDs into the HUT Control Segment for a
Logon Sequence Number
Begin isolated loading
Block-level compress/uncompress
Bit map set manipulations such as intersect and union
Create secondary index
Check N parent keys
Checkpoint database
Create reference index
Collect Statistics aggregate operations (his step is now obsolete.)
Collect Statistics update
Create table header
Create table privileges
Check workload step
DBQL data collection
Delete row
Delete row from queue table
Drop inconsistent reference index
Delete secondary index
Delete journal table
Delete reference index
Delete RID Row
Dispatcher retrieve step
Drop table
Delete temporary tables
BEGIN EDIT DELETE
BEGIN MLOAD
Prepare for deblocker/application task (DML)
End request
End transaction
322 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
HLA
HUL
ILR
INS
INSLDC
INSQT
InvJHI
JIN
ExpVR
Fail
FCF
FDS
FLGRI
Hcs
ERE
ESD
ESR
EVT
EXE
ExecSP
ExpHR
ExpRL
Step Internal Name Description
EED
EEN
Prepare MLOAD work tables
END edit step
EFE
EIL
EIXSUM
EIXUPD
EPF
End FastExport
End isolated loading
Collect EIX statistics aggregate operations
Collect EIX statistics update
End plan fragment; the presence of this step indicates the end of a plan fragment for a dynamic plan
Release MLOAD work tables
Data acquisition
Sort MLOAD work tables
Create event table row
Execute edit
Execute Stored Procedure
Export Horizontal Redistribution step
Export Release Locks Step
Export Vertical Redistribution step
Operation did not work
Forward configuration
Flush DBSpace accounting table entry
Flag reference index
Add table record info and privileges into the Hut Control Segment for a
Logon Sequence Number
High-Level Large Object Access
Host utility set lock
Contain access log entries
Insert a row
Insert Deferred Lob Constant
Insert a row to a queue table
Invalidate Join/Hash index
Join
323
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
MiUpd
MLK
MRD
MRG
MRL
MRU
MRM
MTB
LobFet
LogAOf
LogAOn
LOT
LOT2VM
MDT
MiIns
MiRet
Step Internal Name Description
JTmp
LBG
Journal temporary tables
Data-Load BEGIN LOADING
LCM
LCP
LFI
LIN
Load config map
Data-Load CHECKPOINT LOADING
Data-Load END LOADING
Data-Load USING ... INSERT
Large object fetch
Logging for online archive off
Logging for online archive on
Large object transport
Forward LOT to Virtual Mailbox
Merge delete tables
Merge-Into insert
Merge-Into retrieve
There is a Merge-Into table reference in a RET (that is, retrieve) query.
MTH
MTmp
MVN
NEXTOK
OAR
Okay
Merge-Into update
Multi-Lock: several LCK Messages in one
Merge delete two tables based on row hash code
Merge two tables based on row hash code
Merge utility locks
Merge update
Merge Row Multiple Operations Step
Modify table. To modify the table internally, the table header must also be modified
Modify table header
Materialize temporary tables
Modify version number in table header
It is okay to send the next step
One AMP reply
Operation worked
324 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Step Information Log Table: DBQLStepTbl
SQB
SQL
SRD
STATFN
StpRLK
SUM
TRET
UPD
Spl012
SplDB
SplFL
SplOUC
SplPSC
SplQL
SplRTS
SplTAB
RAE
RET
RSF
SAMP
SAT
SMS
Spl004
Spl011
Step Internal Name Description
OKELCT Send StpOKElicitData message to notify Dispatcher that it is fine to send next step within the same request. It is required to prevent deadlocks and serve as a form of flow control.
PFN
PKA
PRELOB
QryLgn
Process function
Prime key abort
InsPreLob step
Query logon on step
Remove abort entry from TOABORT list
Retrieve row
Release spool files
Perform sampling from table/spool
Synchronous abort test step from DBC/SQL statement
Set manipulations such as intersect, union, minus
Dictionary data spoil step for role
Dictionary data spoil step for security constraint
Stats cache spoil step
Spoil database information in Data Dictionary
DBQL cache flush spoil step
Object use count spoil step
Spoil Parser session information
Spoil the DBQL Rule cache
Spoil Stored Procedure cache information
Spoil Table information in Data Dictionary
Set Query_Band
SQL step for Request Text storage
Sort a table
Perform statistical functions
Release lock
Perform local aggregate operations
Trigger retrieve step
Update row
325
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Query Step Information Log Table: QryLogSteps[V]
Step Internal Name Description
UpsIns
UpsUpd
Upsert insert
Upsert update
VJT
Warn
YCM
YPM
Validate journal table
Warning response
Ynet conclude merge
Ynet prepare merge
View of the Query Step Information Log Table:
QryLogSteps[V]
The QryLogSteps[V] view of DBQLStepTbl is populated if you specify the WITH STEPINFO option. When the query completes, the system logs one row for each query step, including parallel steps.
QryLogSteps[V] Field
CollectTimestamp
CPUtime
CPUTimeNorm
DispatchSeq
DMLLoadID
ElapsedTime
Description
Time that the rows were written to the database. This field is also part of the multi-column NUPI (see also ProcID).
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
This is the CPU seconds for this step.
Note: For requests using algorithm 1, the CPUTime shown in the first parallel step represents usage for the entire set of parallel steps.
This is the normalized AMP CPU seconds for this step (used for coexistence systems).
Note: For requests using algorithm 1, the CPUTimeNorm shown in the first parallel step represents usage for the entire set of parallel steps.
This field is populated only for requests that use algorithm 3 of
DBQL CPU/IO Collection.
Dispatch sequence, the order in which the step was handled by the dispatcher itself or dispatched to the AMPs, beginning with '1'.
Indicates the load id value used by a modification step on a loadisolated table. The value is NULL for modification steps on nonload-isolated tables.
The difference between the time when the step was sent to the AMP and when the step was returned from the AMP (to the nearest micro second).
326 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Query Step Information Log Table: QryLogSteps[V]
QryLogSteps[V] Field
EstCPUCost
EstHRCost
EstIOCost
EstNetCost
EstProcTime
EstRowCount
EstRowCountSkew
EstRowCountSkewMatch
EstRowCountSkew
EstRowCountSkewMatch
FragmentNum
IOcount
IOKB
LockDelay
LSN
MaxAmpCPUTime
Description
An estimate of the milliseconds of CPU time for the step, as determined by the Optimizer.
An estimate of other costs for the step. There is no comparable actual data for this estimate.
An estimate of service time in milliseconds for
I/O for this step. There is no comparable actual data for this estimate.
An estimate of the BYNET service time in milliseconds for the step.
There is no comparable actual data for this estimate.
The estimated processing time as determined by the Optimizer.
Note: This time is reported in seconds.
The estimated row count for this step.
For a Partial Redistribution and Partial Duplication (PRPD) plan split step, EstRowCount is the estimated row counts for all split spools.
Note: A split step is a RETRIEVE or JOIN step with “split into” appearing in the EXPLAIN when target spools are generated.
In PRPD, the estimated row count for the skew split spool (which contains the rows with skewed values of the relation in this step).
In PRPD, the estimated row count for the skew match split spool
(which contains the rows with skewed values of the other relation to be joined with the relation in this step).
In PRPD, the estimated row count for the skew split spool (which contains the rows with skewed values of the relation in this step).
In PRPD, the estimated row count for the skew match split spool
(which contains the rows with skewed values of the other relation to be joined with the relation in this step).
Indicates the plan fragment to which the current step belongs. For a static plan, this is set to NULL.
This is the number of logical I/Os used by this step.
Note: For requests using algorithm 1, the IOCount shown in the first parallel step represents the entire set of parallel steps.
Logical I/Os in KB per step.
Maximum wait time to get a lock on an object in centiseconds. The maximum value is 32,767 centiseconds.
The Logon Sequence Number used by load utility.
CPU seconds of the highest CPU utilized AMP in the step.
327
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Query Step Information Log Table: QryLogSteps[V]
QryLogSteps[V] Field
MaxAmpCPUTimeNorm
Description
Normalized maximum CPU seconds for an AMP.
Note: Normalized columns, such as MaxAmpCPUTimNorm, are for co-existence systems only.
MaxAmpIO
MaxAMPSpool
The highest logical I/O count on the AMP with the highest logical I/
O usage for the step.
If you are using DBQL CPU/IO Collection algorithm 3,
MaxAMPSpool is the largest peak spool usage of any AMP for this step. For other algorithms, MaxAMPSpool is the highest spool usage on any AMP at the end of the step.
MaxCPUAmpNumber
MaxIOAmpNumber
The number of the AMP with the highest CPU usage for the step.
MaxCPUAmpNumberNorm The number of the AMP with the maximum normalized CPU seconds for co-existence systems.
The number of the AMP with the highest logical I/O usage for the step.
MaxSpoolAmpNumber
MinAmpCPUTime
MinAmpCPUTimeNorm
MinAmpIO
MinAMPSpool
NumCombinedPartitions
Number of the AMP with the highest spool usage.
CPU seconds of the lowest CPU utilized AMP in the step.
Normalized minimum CPU seconds for an AMP.
Note: Normalized columns, such as MinAmpCPUTimeNorm, are for co-existence systems only.
Logical I/O count of the lowest logical I/O utilized AMP in the query.
If you are using DBQL CPU/IO Collection algorithm 3,
MinAMPSpool is the smallest peak spool usage of any AMP for this step. For other algorithms, MinAMPSpool is the lowest spool usage of any AMP at the end of the step.
If there is static partition elimination for the step or if a noPI table or join index has column partitions, the number of combined partitions accessed (not eliminated). Otherwise, the column is null.
NumContexts
NumCPReferences
NumOfActiveAMPs
PersistentSpool
PhysIO
PhysIOKB
For column-partitioned tables or join indexes: the number of contexts allocated to access partitions at the same time as a source or target relation that has partitioning. Otherwise, the column is null.
For column-partitioned tables or join indexes: the number of column partitions referenced in a column-partitioned source or target. Otherwise, the column is null.
The number of AMPs involved with the step.
Persistent portion of the SpoolUsage column.
Physical I/O count per step.
Physical I/Os in KB per step.
328 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Query Step Information Log Table: QryLogSteps[V]
QryLogSteps[V] Field
ProcID
QueryID
RowCount
RowCount2
RowCount3
RowsWComprColumns
SpoolUsage
SSRReceiverCount
StatementNum
Description
Unique processor ID of the Dispatcher and part of the multicolumn NUPI (see also CollectTimeStamp).
Internally generated identifier of the query.
For:
•
The MRM (Merge Row Multiple) or EXE (MultiLoad) steps,
RowCount is the number of rows inserted.
•
All other steps, RowCount is the actual number of rows the step returns (indicating activity count). For a PRPD plan split step,
RowCount includes the rows from all split spools.
Note: If “split into” appears in the EXPLAIN of a RETRIEVE or
JOIN step, Teradata Database is using PRPD.
If the row count for a step is 18,446,744,073,709,551,615, the number is logged in the RowCount column as
1.8446744073709552e+19.
Note: The value formatted as a decimal value is
18,446,744,073,709,551,616 due to float arithmetic.
For:
•
The MRM (Merge Row Multiple) or EXE (MultiLoad) steps,
RowCount2 is the number of updated rows.
•
An LFI (FastLoad) step, RowCount2 is the number of rows loaded.
•
A PRPD plan split step, RowCount2 is the actual number of rows in the skew split spool.
For:
•
The EXE (MultiLoad) step, RowCount3 is the number of rows deleted.
Note: For the number of rows inserted by the EXE step, see
RowCount. For the number of rows updated by the EXE step, see
RowCount2.
•
A PRPD plan split step, RowCount3 is the actual number of rows in the skew match split spool.
Number of rows with compressed columns.
Spool usage varies during a step. If you are using DBQL CPU/IO
Collection algorithm 3, SpoolUsage is the peak spool usage in bytes for this step. For other algorithms, SpoolUsage is the amount of spool usage for the AMP at the end of the step.
The number of Single-Sender Redistribution (SSR) receiver AMPs that the step uses to redistribute rows. If the step does not use SSR for row redistribution, the value is zero.
Statement number with which the step is associated. If associated with multiple statements (for example, array insert), this is the highest statement number associated with this step.
329
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Query Step Information Log Table: QryLogSteps[V]
QryLogSteps[V] Field
StepInstance
StepLev1Num
StepLev2Num
StepName
StepStartTime
StepStatus
StepStopTime
StepWD
Description
This field is populated only for requests that use algorithm 3 of
DBQL CPU/IO Collection.
Step instance number for steps that are repeated. If a step is dispatched 3 times, there will be one summarized step logged for the 3 instances of the same step number and the step instance field will be 3, indicating that 3 steps instances are represented. This field will be 1 for non-repeated steps.
Step number. If this row is for a parallel step, the number is repeated for as many parallel steps as were used for this step (for examples, see StepLev2Num).
If this row is for a parallel step, this is the second-level number of the step that spawned it.
For example, if the value of StepLevl1Num is 4 and this row logs the first parallel step for step 4, this value is 1 (and would read 04 01). If this row logs the second row generated for step 4, this value is 2 (and would read 04 02), and so forth.
If this row is not a parallel-step row, this value is 0.
Abbreviation of the internal name used to identify this step (for example, DEL for a DELETE step).For more information on the value of “RESP,” see the DBQLLogLastResp field in Utilities.
Timestamp when the step was sent to the AMP to the nearest micro second.
This field is populated only for requests that use algorithm 3 of
DBQL CPU/IO Collection.
The step status, indicating dispatcher and AMP step disposition.
Values include:
•
Dispatched: Step was dispatched to the AMP.
•
AMPexec: Step was executed in the AMP.
•
AMPokay: Step completed okay in the AMP.
•
AMPerror: Step failed in the AMP and returned an error.
•
AMPReqAB: Step aborted in the AMP for request abort.
•
AMPTxnAB: Step aborted in the AMP for transaction abort.
•
Completed: Step completed by the dispatcher.
•
Skipped: Conditional step was skipped by the dispatcher.
Timestamp when the step returned from the AMP to the nearest micro second.
The workload definition in effect for the step.
330 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Summary Log Table: DBQLSummaryTbl
QryLogSteps[V] Field
TriggerKind
TriggerNestLevel
UtilityTableId
VHLogicalIO
VHLogicalIOKB
VHPhysIO
VHPhysIOKB
Description
Trigger kind. Space-separated string of abbreviated values for the bit flags.
TriggerKind bit flags:
•
TrigNone: 0
•
GetActivityCnt: 1
•
FetchQualifyingRows: 2
•
BuildTrigUsingRow: 4
•
BeginLoop: 8
•
EndLoop: 16
•
IdColWithRowTrig: 32
•
SkipQualifyingRows: 64
•
Zero: Step was not part of a trigger
•
Nonzero: Step was part of a trigger
Trigger nesting level.
The TriggerNestLevel can range from 0-255.
The table ID for the utility.
This field is always 0.
VERYHOT logical I/O count per step.
VERYHOT logical I/Os in KB per step.
VERYHOT physical I/O count per step (VERYHOT cylinder index and data block I/Os that were handled by physical disk I/Os.)
Physical I/Os in KB per step.
Summary Log Table: DBQLSummaryTbl
DBC.DBQLSummaryTbl holds the counts logged for queries of users with THRESHOLD or
SUMMARY rules. It is only populated if summary information is requested or a threshold value is used.
Rows Generated by the THRESHOLD Option
If a threshold value is used, the DBQLSummaryTbl will log queries based on the criterion of elapsed seconds, elapsed hundredths, normalized time, logical I/O count, or AMP CPU hundredths of a second. The ValueType column in DBQLSummaryTbl tells which criterion are used.
Queries with values less than or equal to the given limit will be counted in the
DBQLSummaryTbl. Queries with more than the given limit will be logged in DBQLogTbl.
Database Administration 331
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Summary Log Table: DBQLSummaryTbl
For example, if the criterion is elapsed seconds, for each session being logged, DBQL with the
THRESHOLD option does the following:
•
Each time a query completes within the threshold time, increments the counters for the session
•
Every 10 minutes, writes the cumulative count for each session as a separate row in
DBQLSummaryTbl
•
DBQL generates a default row in DBQLogTbl for queries that exceed the threshold time
For example, if you specified “LIMIT THRESHOLD=500 CPUTIME ON user1”, you could find out how many queries user1 submitted that required more than 5 CPU seconds.
The following is an example of how to query the DBQLSummaryTbl table, using the
QryLogSummary[V] view, to find the total I/O count and total CPU time:
SELECT collecttimestamp, sessionid, querycount, ampcputime, totaliocount
FROM QryLogSummaryV ORDER BY collecttimestamp;
CollectTimeStamp
------------------
2006-07-29 10:30:05
2006-07-29 11:07:10
2006-07-29 11:07:10
2006-07-29 11:07:10
2006-07-29 11:07:10
2006-07-29 11:17:10
2006-07-29 11:17:10
2006-07-29 11:17:10
2006-07-29 11:17:10
2006-07-29 11:17:10
SessionID
----------
1,001
1,001
1,013
1,000
1,014
1,017
1,014
1,031
1,023
1,026
QueryCount AMPCPUTime TotalIOCount
------------ --------- ------------
5
1
.031
.015
4
0
71
96
1
2
.000
.047
.907
1.234
0
0
2,427
2,983
26
1
94
42
.329
.016
1.093
.578
552
0
2,483
1,196
Rows Generated by the SUMMARY Option
DBQL behavior resulting from the SUMMARY option is unique in that:
•
No default rows are generated to DBQLogTbl for summarized queries.
•
You define the summary criteria and then DBQL summarizes queries into those buckets.
Each bucket counts the number of queries for the session that fall into that bucket and also sums up their elapsed time, I/O count and various CPU times.
•
DBQL maintains summary counters in cache. The contents are committed to the
DBQLSummaryTbl table every 10 minutes, when the cache is flushed.
The following table describes the summary criteria that make up the buckets.
Using the following modifier on the SUMMARY Option…
CPUTIME
CPUTIMENORM
Means the system logs the values specified as…
hundredths of a second of AMP CPU time. So for example:
BEGIN QUERY LOGGING LIMIT SUMMARY=5, 10, 15
CPUTIME ON user1; means counting queries based on CPU time where 5 is 0.05 seconds of CPU time, 10 is 0.10 seconds and 15 is 0.15 seconds of
CPU time.
hundredths of a second of normalized AMP CPU time. This is similar to CPUTIME above, but uses the times from a coexistence system that are normalized.
332 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Summary Log Table: DBQLSummaryTbl
Using the following modifier on the SUMMARY Option…
ELAPSEDSEC
ELAPSEDTIME
IOCOUNT
Means the system logs the values specified as…
Without an modifier to the SUMMARY option:
BEGIN QUERY LOGGING LIMIT SUMMARY=5, 10, 15 ON user1; elapsed seconds is used. You can explicitly set elapsed seconds as a criteria by using the following statement:
BEGIN QUERY LOGGING LIMIT SUMMARY=5, 10, 15
ELAPSEDSEC ON user1; elapsed time where n is specified in hundredths of a second.
the number of logical I/Os used. For example:
BEGIN QUERY LOGGING LIMIT
SUMMARY=1000,5000,10000 IOCOUNT ON ALL;
DBQL gathers information for the collection interval (the default is every 10 minutes). For every collection interval and every active session at the end of the interval, there is as many as four rows per session containing the query count, the number of seconds of elapsed time for those queries, the amount of CPU time and the number of logical I/Os. Each row represents one of four possible buckets of information.
In the default case, where elapsed time is used to qualify a query for one of the four buckets, the following buckets are possible:
•
0 to 5 seconds
•
Longer than 5 seconds to 10 seconds
•
Longer than 10 seconds to 15 seconds
•
Longer than 15 seconds
For example, if during that next logging period for the user, two queries ran under 5 seconds, three queries ran for 7 seconds, and one query ran for over 15 seconds, the following rows would be written to DBQLSummaryTbl for the session:
COUNT SECONDS LOWHIST
-----------------
2
3
1
1
21
200
0
5
HIGHHIST
--------
5
10
15 4294967295
For this example, there were:
•
No queries between 10 and 15 seconds
•
To determine the average time for each query counted, divide SECONDS by COUNT (for example, the two queries in the first row averaged 0.5 seconds each; the three queries in the second row averaged 7 seconds each).
If you specified LIMIT THRESHOLD=500 CPUTIME ON user1, you could find out how many queries user1 submitted that required more than 5 CPU seconds.
333
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Summary Log Table: DBQLSummaryTbl
Row Values
The following table lists the fields available for population in DBQLSummaryTbl rows.
DBC.DBQLSummaryTbl Field
AcctString
AMPCPUTime
AMPCPUTimeNorm
AppID
ClientAddr
ClientID
CollectTimeStamp
HighHist
LogicalHostID
LowHist
ParserCPUTime
ParserCPUTimeNorm
ProcID
ProfileID
Description
The unexpanded account name under which the query is submitted.
Total AMP CPU seconds.
Normalized AMP CPU seconds for co-existence systems only.
The application ID under which the query is submitted.
The client IP address of the submitted query.
The client ID under which the query is submitted.
Time that the row was inserted into the database. This field is also part of the multi-column NUPI (also see ProcID).
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
•
Highest value for the SUMMARY option.
•
Zero for the THRESHOLD option.
•
If the modifiers CPUTIME or IOCOUNT are used, then this value is the highest CPU seconds or I/O counts set by the user.
Use CPUTIMENORM for hundredths of a second of normalized CPU time. To measure subsecond queries, use
ELAPSEDTIME. The default, elapsed time in seconds, can be explicitly specified with ELAPSEDSEC.
Unique identifier of the logon source for the logged query. A value of 0 indicates an internal session.
•
Lowest value for the SUMMARY option.
•
For the THRESHOLD option, this is the threshold time set by the user. The default time is 5.
•
If the modifiers CPUTIME or IOCOUNT are used, then this value is the lowest CPU seconds or I/O counts set by the user.
Use CPUTIMENORM for hundredths of a second of normalized CPU time. To measure subsecond queries, use
ELAPSEDTIME. The default, elapsed time in seconds, can be explicitly specified with ELAPSEDSEC.
CPU seconds to parse queries.
Normalized CPU seconds to parse queries.
Note: Normalized columns, such as ParserCPUTimeNorm, are for co-existence systems only.
Unique processor ID of the Dispatcher and part of the multicolumn NUPI (also see CollectTimeStamp).
The profile ID under which the user is submitting queries.
334 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Summary Log Table: QryLogSummary[V]
DBC.DBQLSummaryTbl Field
QueryCount
QuerySeconds
SessionID
TotalIOCount
UserID
ValueType
Description
Number of queries run in the last 10 minutes.
Total run time of queries in this period for this histogram.
Calculated to the nearest hundredth of a second.
Identifies the session for which queries are being summarized
(counted).
The total logical I/O count.
Note: Logical I/O and CPU may not be accurate for aborted queries.
The unique ID of the user whose query is logged.
The THRESHOLD criterion is as follows:
•
C = AMP CPU seconds
•
H = elapsed hundredths of a second
•
I = I/O count
•
N = Normalized AMP CPU seconds
•
S = elapsed time in seconds.
View of the Summary Log Table:
QryLogSummary[V]
The QryLogSummary[V] view of DBQLSummaryTbl holds the counts logged for queries of users with THRESHOLD or SUMMARY rules. It is only populated if summary information is requested or a threshold value is used.
QryLogSummary[V] Field Description
AcctString The unexpanded account name under which the query is submitted.
AMPCPUTime
AMPCPUTimeNorm
Total AMP CPU seconds.
Normalized AMP CPU seconds for co-existence systems only.
AppID
AverageAmpCPU
AverageAmpCPUNorm
AverageIO
AverageParserCPU
The application ID under which the query is submitted.
The average AMP CPU of the queries that ran in the last 10 minutes.
The average normalized AMP CPU seconds for the queries run in the last 10 minutes on a co-existence system.
The average logical I/O count for the queries run in the last 10 minutes.
The average CPU seconds to parse the queries that ran in the last 10 minutes.
Database Administration 335
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Summary Log Table: QryLogSummary[V]
QryLogSummary[V] Field Description
AverageParserCPUNorm The average normalized CPU seconds to parse the queries run in the last 10 minutes on a co-existence system.
AverageTime The average run time for queries run in the last 10 minutes, calculated to the nearest hundredth of a second.
ClientAddr
ClientID
CollectTimeStamp
The client IP address of the submitted query.
The client ID under which the query is submitted.
HighHist
Time that the row was inserted into the database. This field is also part of the multi-column NUPI (also see ProcID).
For more information on CollectTimeStamp, see
CollectTimeStamp Value Between Tables” on page 381
.
•
Highest value for the SUMMARY option.
•
Zero for the THRESHOLD option.
•
If the modifiers CPUTIME or IOCOUNT are used, then this value is the highest CPU seconds or I/O counts set by the user.
Use CPUTIMENORM for hundredths of a second of normalized
CPU time. To measure subsecond queries, use ELAPSEDTIME. The default, elapsed time in seconds, can be explicitly specified with
ELAPSEDSEC.
LogicalHostID
LowHist
ParserCPUTime
ParserCPUTimeNorm
ProcID
Unique identifier of the logon source for the logged query. A value of 0 indicates an internal session.
•
Lowest value for the SUMMARY option.
•
For the THRESHOLD option, this is the threshold time set by the user. The default time is 5.
•
If the modifiers CPUTIME or IOCOUNT are used, then this value is the lowest CPU seconds or I/O counts set by the user. Use
CPUTIMENORM for hundredths of a second of normalized CPU time. To measure subsecond queries, use ELAPSEDTIME. The default, elapsed time in seconds, can be explicitly specified with
ELAPSEDSEC.
CPU seconds to parse queries.
Normalized CPU seconds to parse queries.
Note: Normalized columns, such as ParserCPUTimeNorm, are for coexistence systems only.
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (also see CollectTimeStamp).
ProfileID
QueryCount
QuerySeconds
The profile ID under which the user is submitting queries.
Number of queries run in the last 10 minutes.
Total run time of queries in this period for this histogram. Calculated to the nearest hundredth of a second.
336 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Utility Job Log Table: DBQLUtilityTbl
QryLogSummary[V] Field Description
SessionID Identifies the session for which queries are being summarized
(counted).
TotalIOCount
UserID
ValueType
The total logical I/O count.
Note: Logical I/O and CPU may not be accurate for aborted queries.
The unique ID of the user whose query is logged.
The THRESHOLD criterion is as follows:
•
C = AMP CPU seconds
•
H = elapsed hundredths of a second
•
I = I/O count
•
N = Normalized AMP CPU seconds
•
S = elapsed time in seconds.
Utility Job Log Table: DBQLUtilityTbl
This table is populated if you use the WITH UTILITYINFO option for BEGIN/REPLACE
QUERY LOGGING. Each row stores information for one completed load/export utility or
Data Stream Architecture job.
Note: The following table describes the different column phases listed below.
Utility Protocols
FastLoad
MLOAD
MLOADX
FastExport
DSA
Phase Zero
Name
Setup
Phase One
Name
Acquisition
Phase Two
Name
Application
Phase Three
Name
Cleanup
Phase Four
Name
N/A
Setup
N/A
Select
Dictionary
Export
Data
Cleanup N/A
Build Postscript
Phase1 and Phase2 columns are used for the required phases. Phase0, Phase3, and Phase4 columns are used for the optional phases.
DBC.DBQLUtilityTbl Field
AcctString
Description
Unexpanded account name under which the utility job is submitted.
Database Administration 337
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Utility Job Log Table: DBQLUtilityTbl
338
DBC.DBQLUtilityTbl Field
AcctStringDate
AcctStringHour
AcctStringTime
AppID
ClientID
ClientAddr
CollectTimeStamp
CPUDecayLevel
DelayTime
DSAOperation
ExpandAcctString
ExtendedMLoad
FastExportNoSpool
Description
Value of the expanded &D ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Even if ASE truncates the expanded account string, DBQL will report the full value. For example, if &D is expanded into position 26 or higher in the account definition and truncation occurs, DBQL will still report the entire value of the date in the AcctStringDate column.
Value of the expanded &H ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Value of the expanded &T ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Application (executable) or utility name; for example,
FASTLOAD from LogonSource for network clients.
Network client user name under which this session is logged
(for example, ADMIN).
Client IP address of the submitted utility job as obtained from Logonsource from a network client.
A date and time when information collection for the job is completed.
Decay level reached for CPU usage per AMP, for requests using the default timeshare method in Priority Scheduler for
SUSE Linux Enterprise Server 11 or above. Values are 0, 1, or
2. A value of 0 indicates that the request is running at the default priority. A value of 1 indicates that the priority level is reduced and the resource allotment is halved. A value of 2 indicates that the priority level is reduced again, the resource allotment is halved again, and it will not be reduced more.
Seconds a query was delayed by the Teradata dynamic workload management software.
•
D indicates DUMP operation.
•
R indicates RESTORE operation.
Expanded account name under which the utility job is submitted if account expansion is invoked.
Note: The &I, which gives the host, session, and request number for the query is different for a stored procedure call.
For stored procedure calls, the request number is the client request number which is the request of the CALL statement.
•
Y indicates MLOADX.
•
N indicates MLOADX is not being used.
For utilities implementing FastExport protocol:
•
Y indicates No Spool mode.
•
N indicates Spool mode.
Database Administration
Database Administration
DBC.DBQLUtilityTbl Field
FinalWDID
IODecayLevel
JobEndTime
JobInstanceID
JobName
JobStartTime
LogicalHostID
LogonDateTime
LogonSource
LSN
MaxDataWaitTime
MaxDataWaitTimeID
NumSesOrBuildProc
OpEnvID
Phase0EndTime
Phase0IOKB
Phase0ParserCPUTime
Phase0ParserCPUTimeNorm
Phase0PhysIO
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Utility Job Log Table: DBQLUtilityTbl
Description
Workload definition under which the main utility work completed execution.
Decay level reached for I/O usage per AMP, for requests using the default timeshare method in Priority Scheduler for
SUSE Linux Enterprise Server 11 or above. Values are 0, 1, or
2. A value of 0 indicates that the request is running at the default priority. A value of 1 indicates that the priority level is reduced and the resource allotment is halved. A value of 2 indicates that the priority level is reduced again, the resource allotment is halved again, and it will not be reduced more.
Job end time.
DSA job execution ID. Not used for other utilities at this time.
DSA job name. Not used for other utilities at this time.
Job start time.
Logical host ID of the control SQL session.
Date (for example, YYYY-MM-DD) and time (for example,
HH:MM:SS) of the control SQL session logon.
Logon source string text.
Logon Sequence Number.
Maximum wait time from external source for input data or output requests.
•
For FastLoad, MultiLoad, and FastExport: The ID of the
AMP owning the utility session with the highest data wait time.
•
For MLOADX: The session number of the session with the highest data wait time.
•
For DSA: The ID of the stream with the highest data wait time.
For load/export utilities, it is the number of utility sessions.
For DSA, it is the number of build processes.
Internal identifier of the Operating Environment / Planned
Environment currently enforced by the Teradata dynamic workload management software.
Phase 0: End time.
Phase 0: Logical I/Os in KB.
Phase 0: Parser and Dispatcher CPU time.
Phase 0: Normalized Parser and Dispatcher CPU time.
Phase 1: Physical I/O count.
339
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Utility Job Log Table: DBQLUtilityTbl
340
DBC.DBQLUtilityTbl Field
Phase0PhysIOKB
Phase0StartTime
Phase0TotalCPUTime
Phase0TotalCPUTimeNorm
Phase0TotalIO
Phase1BlockCount
Description
Phase 0: Physical I/O in KB.
Phase 0: Start time.
Phase 0: Total CPU time.
Phase 0: Total normalized CPU time.
Phase 0: Total logical I/Os.
Phase 1: Block (message) count.
Phase1ByteCount
Phase1EndTime
Phase1IOKB
Phase1MaxAMPMemoryUsage
Phase1MaxAWTUsage Phase 1: Maximum number of AMP worker tasks (AWTs) used. Not available for DSA and FastExport using spool.
Phase 1: ID of the AMP with the highest CPU usage. Phase1MaxCPUAmpNumber
Phase1MaxCPUAmpNumberNorm Phase 1: ID of the AMP with the highest normalized CPU usage.
Phase1MaxCPUTime
Phase1MaxCPUTimeNorm
Phase1MaxIO
Phase1MaxIOAmpNumber
Phase 1: CPU usage of the most-used AMP.
Phase 1: Normalized CPU usage of the most-used AMP.
Phase 1: Logical I/Os of the most-used AMP.
Phase 1: ID of the AMP with the most logical
I/Os.
Phase1MaxRSGMemoryUsage
Phase1ParserCPUTime
Phase1ParserCPUTimeNorm
Phase1PhysIO
Phase1PhysIOKB
Phase1RowCount
Phase1RSGCPUTime
Phase1RSGCPUTimeNorm
Phase 1: Byte count.
Phase 1: End time.
Phase 1: Total logical I/O usage in KB.
Phase 1: Maximum AMP memory usage.
Phase 1: For DSA, the maximum RSG memory usage.
Phase 1: Parser CPU usage.
Phase 1: Normalized parser CPU usage.
Phase 1: Total physical I/O usage.
Phase 1: Total physical I/O usage in KB.
Phase 1: Row count.
Phase 1: RSG CPU usage for Data Stream Architecture
(DSA).
Phase 1: Normalized RSG CPU usage for Data Stream
Architecture (DSA).
Phase1StartTime
Phase1TotalCPUTime
Phase1TotalCPUTimeNorm
Phase 1: Start time.
Phase 1: Total CPU time.
Phase 1: Total normalized CPU usage.
Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Utility Job Log Table: DBQLUtilityTbl
DBC.DBQLUtilityTbl Field
Phase1TotalIO
Phase2BlockCount
Phase2ByteCount
Phase2EndTime
Phase2IOKB
Phase2MaxAMPMemoryUsage
Description
Phase 1: Total logical I/Os.
Phase 2: For DSA, the number of blocks processed.
Phase 2: For DSA, the total count of bytes processed.
Phase 2: End time.
Phase 2: Total logical I/O usage in KB.
Phase 2: Maximum AMP memory usage.
Phase2MaxAWTUsage
Phase2MaxCPUAmpNumber
Phase 2: Maximum number of AWTs used.
Phase 2: ID of the AMP with the highest CPU usage.
Phase2MaxCPUAmpNumberNorm Phase 2: ID of the AMP with the highest normalized CPU usage.
Phase2MaxCPUTime
Phase2MaxCPUTimeNorm
Phase2MaxIO
Phase2MaxIOAmpNumber
Phase 2: CPU usage of the most-used AMP.
Phase 2: Normalized CPU usage of the most-used AMP.
Phase 2: Logical I/Os of the most-used AMP.
Phase 2: ID of the AMP with the most logical
I/Os.
Phase2MaxRSGMemoryUsage
Phase2ParserCPUTime
Phase2ParserCPUTimeNorm
Phase2ParserCPUTimeNorm
Phase2PhysIO
Phase2PhysIOKB
Phase2RSGCPUTime
Phase2RSGCPUTimeNorm
Phase2StartTime
Phase2TotalCPUTime
Phase2TotalCPUTimeNorm
Phase2TotalIO
Phase3EndTime
Phase3IOKB
Phase3MaxAMPMemoryUsage
Phase3MaxAWTUsage
Phase 2: For DSA, Maximum RSG memory usage.
Phase 2: Parser CPU usage.
Phase 2: Normalized parser CPU usage.
Phase 2: Normalized Parser and Dispatcher CPU time.
Phase 2: Total physical I/Os.
Phase 2: Total physical I/O usage in KB.
Phase 2: For DSA, the RSG CPU usage.
Phase 2: For DSA, the normalized RSG CPU usage.
Phase 2: Start time.
Phase 2: Total CPU usage.
Phase 2: Total normalized CPU usage.
Phase 2: Total logical I/Os.
Phase 3: End time.
Phase 3: Total logical I/O usage in KB.
Phase 3: Maximum AMP memory usage.
Phase 3: Maximum number of AWTs used.
341
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Utility Job Log Table: DBQLUtilityTbl
DBC.DBQLUtilityTbl Field
Phase3MaxRSGMemoryUsage
Phase3ParserCPUTime
Phase3ParserCPUTimeNorm
Phase3PhysIO
Phase3PhysIOKB
Phase3RSGCPUTime
Phase3RSGCPUTimeNorm
Phase3StartTime
Phase3TotalCPUTime
Phase3TotalCPUTimeNorm
Phase3TotalIO
Phase4EndTime
Phase4IOKB
Phase4ParserCPUTime
Phase4ParserCPUTimeNorm
Phase4PhysIO
Phase4PhysIOKB
Phase4StartTime
Phase4TotalCPUTime
Phase4TotalCPUTimeNorm
Phase4TotalIO
ProcID
ProfileID
ProxyRole
ProxyUser
QueryBand
RowsDeleted
RowsExported
RowsInserted
Description
Phase 3: For DSA, the maximum RSG memory usage.
Phase 3: Parser CPU usage.
Phase 3: Normalized parser CPU usage.
Phase 3: Total physical I/Os.
Phase 3: Total physical I/O usage in KB.
Phase 3: For DSA, RSG CPU usage.
Phase 3: For DSA, normalized RSG CPU usage.
Phase 3: Start time.
Phase 3: Total CPU usage.
Phase 3: Total normalized CPU usage.
Phase 3: Total logical I/Os.
Phase 4: For DSA, the end of the phase.
Phase 4: For DSA, the total logical I/O usage in KB.
Phase 4: For DSA, the parser CPU usage.
Phase 4: For DSA, the normalized parser CPU usage.
Phase 4: For DSA, the total physical I/Os.
Phase 4: For DSA, the total physical I/O usage in KB.
Phase 4: For DSA, the start of the phase.
Phase 4: For DSA, the total CPU usage.
Phase 4: For DSA, the total normalized CPU usage.
Phase 4: For DSA, the total logical I/Os.
Unique processor ID of the Dispatcher and part of the multicolumn NUPI for the table (see also CollectTimeStamp column).
Profile ID, if any, under which the user submitted the utility job.
Proxy role used for the utility job.
Proxy user for the utility job.
Query band under which the utility job is submitted.
Number of rows deleted.
Number of rows exported.
Number of rows inserted.
342 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Utility Job Log Table: QryLogUtility[V]
DBC.DBQLUtilityTbl Field
RowsUpdated
SessionID
SessionWDID
SysConID
TDWMRuleID
UserID
UserName
Description
Number of rows updated.
Unique session identifier.
Workload ID class associated with the session. It determines the priority of the session when the Teradata dynamic workload software classification is active.
Internal identifier of the System Condition (Syscon) / System
Health currently enforced by the Teradata dynamic workload management software.
ID of the rule that caused a delay.
Unique ID of the user who submitted the utility job.
Name under which the session being logged was initiated.
UtilityRequest
WDID
The main utility SQL request for the job. It contains the target database and/or table names. The possible statements are:
•
BEGIN LOADING (FastLoad)
•
BEGIN MLOAD (MLOAD/MLOADX)
•
SELECT (FastExport)
•
DUMP/RESTORE (DSA)
Workload definition assigned to the main utility work.
For examples of using DBQLUtilityTbl, see “Utility Job Performance Analysis and Capacity
View of the Utility Job Log Table:
QryLogUtility[V]
Database Administration
DBC.QryLogUtility[V] Field
AcctString
AcctStringDate
Description
Unexpanded account name under which the utility job is submitted.
Value of the expanded &D ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Even if ASE truncates the expanded account string, DBQL will report the full value. For example, if &D is expanded into position 26 or higher in the account definition and truncation occurs, DBQL will still report the entire value of the date in the AcctStringDate column.
343
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Utility Job Log Table: QryLogUtility[V]
DBC.QryLogUtility[V] Field
AcctStringHour
AcctStringTime
AppID
ClientID
ClientAddr
CollectTimeStamp
CPUDecayLevel
DelayTime
DSAOperation
ExpandAcctString
ExtendedMLoad
FastExportNoSpool
FinalWDID
Description
Value of the expanded &H ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Value of the expanded &T ASE code in the account string. If none exists, the value is denoted with a “?” and is null.
Application (executable) or utility name; for example,
FASTLOAD from LogonSource for network clients.
Network client user name under which this session is logged
(for example, ADMIN).
Client IP address of the submitted utility job as obtained from Logonsource from a network client.
A date and time when information collection for the job is completed. This includes part of the multi-column NUPI for the table. For more information, see the ProcID column.
Decay level reached for CPU usage per AMP, for requests using the default timeshare method in Priority Scheduler for
SUSE Linux Enterprise Server 11 or above. Values are 0, 1, or
2. A value of 0 indicates that the request is running at the default priority. A value of 1 indicates that the priority level is reduced and the resource allotment is halved. A value of 2 indicates that the priority level is reduced again, the resource allotment is halved again, and it will not be reduced more.
Seconds a query was delayed by the Teradata dynamic workload management software.
•
D indicates DUMP operation.
•
R indicates RESTORE operation.
Expanded account name under which the utility job is submitted if account expansion is invoked.
Note: The &I, which gives the host, session, and request number for the query is different for a stored procedure call.
For stored procedure calls, the request number is the client request number which is the request of the CALL statement.
•
Y indicates MLOADX.
•
N indicates MLOADX is not being used.
For utilities implementing FastExport protocol:
•
Y indicates No Spool mode.
•
N indicates Spool mode.
Workload definition under which the main utility work completed execution.
344 Database Administration
Database Administration
JobEndTime
JobInstanceID
JobName
JobStartTime
LogicalHostID
LogonDateTime
LogonSource
LSN
MaxDataWaitTime
MaxDataWaitTimeID
NumSesOrBuildProc
OpEnvID
Phase0EndTime
Phase0IOKB
Phase0ParserCPUTime
Phase0ParserCPUTimeNorm
Phase0PhysIO
Phase0PhysIOKB
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Utility Job Log Table: QryLogUtility[V]
DBC.QryLogUtility[V] Field
IODecayLevel
Description
Decay level reached for I/O usage per AMP, for requests using the default timeshare method in Priority Scheduler for
SUSE Linux Enterprise Server 11 or above. Values are 0, 1, or
2. A value of 0 indicates that the request is running at the default priority. A value of 1 indicates that the priority level is reduced and the resource allotment is halved. A value of 2 indicates that the priority level is reduced again, the resource allotment is halved again, and it will not be reduced more.
Job end time.
DSA job execution ID. Not used for other utilities at this time.
DSA job name. Not used for other utilities at this time.
Job start time.
Logical host ID of the control SQL session.
Date (for example, YYYY-MM-DD) and time (for example,
HH:MM:SS) of the control SQL session logon.
Logon source string text.
Logon Sequence Number.
Maximum wait time from external source for input data or output requests.
•
For FastLoad, MultiLoad, and FastExport: The ID of the
AMP owning the utility session with the highest data wait time.
•
For MLOADX: The session number of the session with the highest data wait time.
•
For DSA: The ID of the stream with the highest data wait time.
For load/export utilities, it is the number of utility sessions.
For DSA, it is the number of build processes.
Internal identifier of the Operating Environment / Planned
Environment currently enforced by the Teradata dynamic workload management software.
Phase 0: End time.
Phase 0: Logical I/Os in KB.
Phase 0: Parser and Dispatcher CPU time.
Phase 0: Normalized Parser and Dispatcher CPU time.
Phase 1: Physical I/O count.
Phase 0: Physical I/O in KB.
345
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Utility Job Log Table: QryLogUtility[V]
346
DBC.QryLogUtility[V] Field
Phase0StartTime
Phase0TotalCPUTime
Phase0TotalCPUTimeNorm
Phase0TotalIO
Phase1BlockCount
Phase1ByteCount
Description
Phase 0: Start time.
Phase 0: Total CPU time.
Phase 0: Total normalized CPU time.
Phase 0: Total logical I/Os.
Phase 1: Block (message) count.
Phase 1: Byte count.
Phase1EndTime
Phase1IOKB
Phase1MaxAMPMemoryUsage
Phase1MaxAWTUsage
Phase 1: Maximum AMP memory usage.
Phase 1: Maximum number of AMP worker tasks (AWTs) used. Not available for DSA and FastExport using spool.
Phase 1: ID of the AMP with the highest CPU usage. Phase1MaxCPUAmpNumber
Phase1MaxCPUAmpNumberNorm Phase 1: ID of the AMP with the highest normalized CPU usage.
Phase1MaxCPUTime
Phase1MaxCPUTimeNorm
Phase1MaxIO
Phase1MaxIOAmpNumber
Phase 1: End time.
Phase 1: Total logical I/O usage in KB.
Phase 1: CPU usage of the most-used AMP.
Phase 1: Normalized CPU usage of the most-used AMP.
Phase 1: Logical I/Os of the most-used AMP.
Phase 1: ID of the AMP with the most logical
I/Os.
Phase1MaxRSGMemoryUsage
Phase1ParserCPUTime
Phase1ParserCPUTimeNorm
Phase1PhysIO
Phase1PhysIOKB
Phase1RowCount
Phase1RSGCPUTime
Phase1RSGCPUTimeNorm
Phase 1: For DSA, the maximum RSG memory usage.
Phase 1: Parser CPU usage.
Phase 1: Normalized parser CPU usage.
Phase 1: Total physical I/O usage.
Phase 1: Total physical I/O usage in KB.
Phase 1: Row count.
Phase 1: RSG CPU usage for Data Stream Architecture
(DSA).
Phase 1: Normalized RSG CPU usage for Data Stream
Architecture (DSA).
Phase1StartTime
Phase1TotalCPUTime
Phase1TotalCPUTimeNorm
Phase1TotalIO
Phase 1: Start time.
Phase 1: Total CPU time.
Phase 1: Total normalized CPU usage.
Phase 1: Total logical I/Os.
Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Utility Job Log Table: QryLogUtility[V]
DBC.QryLogUtility[V] Field
Phase2BlockCount
Phase2ByteCount
Phase2EndTime
Phase2IOKB
Phase2MaxAMPMemoryUsage
Phase2MaxAWTUsage
Description
Phase 2: For DSA, the number of blocks processed.
Phase 2: For DSA, the total count of bytes processed.
Phase 2: End time.
Phase 2: Total logical I/O usage in KB.
Phase 2: Maximum AMP memory usage.
Phase 2: Maximum number of AWTs used.
Phase2MaxCPUAmpNumber Phase 2: ID of the AMP with the highest CPU usage.
Phase2MaxCPUAmpNumberNorm Phase 2: ID of the AMP with the highest normalized CPU usage.
Phase2MaxCPUTime
Phase2MaxCPUTimeNorm
Phase2MaxIO
Phase2MaxIOAmpNumber
Phase 2: CPU usage of the most-used AMP.
Phase 2: Normalized CPU usage of the most-used AMP.
Phase 2: Logical I/Os of the most-used AMP.
Phase 2: ID of the AMP with the most logical
I/Os.
Phase2MaxRSGMemoryUsage
Phase2ParserCPUTime
Phase2ParserCPUTimeNorm
Phase2PhysIO
Phase2PhysIOKB
Phase2RSGCPUTime
Phase2RSGCPUTimeNorm
Phase2StartTime
Phase2TotalCPUTime
Phase2TotalCPUTimeNorm
Phase2TotalIO
Phase3EndTime
Phase3IOKB
Phase3MaxAMPMemoryUsage
Phase3MaxAWTUsage
Phase3MaxRSGMemoryUsage
Phase3ParserCPUTime
Phase 2: For DSA, Maximum RSG memory usage.
Phase 2: Parser CPU usage.
Phase 2: Normalized parser CPU usage.
Phase 2: Total physical I/Os.
Phase 2: Total physical I/O usage in KB.
Phase 2: For DSA, the RSG CPU usage.
Phase 2: For DSA, the normalized RSG CPU usage.
Phase 2: Start time.
Phase 2: Total CPU usage.
Phase 2: Total normalized CPU usage.
Phase 2: Total logical I/Os.
Phase 3: End time.
Phase 3: Total logical I/O usage in KB.
Phase 3: Maximum AMP memory usage.
Phase 3: Maximum number of AWTs used.
Phase 3: For DSA, the maximum RSG memory usage.
Phase 3: Parser CPU usage.
347
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the Utility Job Log Table: QryLogUtility[V]
DBC.QryLogUtility[V] Field
Phase3ParserCPUTimeNorm
Phase3PhysIO
Phase3PhysIOKB
Phase3RSGCPUTime
Phase3RSGCPUTimeNorm
Phase3StartTime
Phase3TotalCPUTime
Phase3TotalCPUTimeNorm
Phase3TotalIO
Phase4EndTime
Phase4IOKB
Phase4ParserCPUTime
Phase4ParserCPUTimeNorm
Phase4PhysIO
Phase4PhysIOKB
Phase4StartTime
Phase4TotalCPUTime
Phase4TotalCPUTimeNorm
Phase4TotalIO
ProcID
ProfileID
ProxyRole
ProxyUser
QueryBand
RowsDeleted
RowsExported
RowsInserted
RowsUpdated
SessionID
Description
Phase 3: Normalized parser CPU usage.
Phase 3: Total physical I/Os.
Phase 3: Total physical I/O usage in KB.
Phase 3: For DSA, RSG CPU usage.
Phase 3: For DSA, normalized RSG CPU usage.
Phase 3: Start time.
Phase 3: Total CPU usage.
Phase 3: Total normalized CPU usage.
Phase 3: Total logical I/Os.
Phase 4: For DSA, the end of the phase.
Phase 4: For DSA, the total logical I/O usage in KB.
Phase 4: For DSA, the parser CPU usage.
Phase 4: For DSA, the normalized parser CPU usage.
Phase 4: For DSA, the total physical I/Os.
Phase 4: For DSA, the total physical I/O usage in KB.
Phase 4: For DSA, the start of the phase.
Phase 4: For DSA, the total CPU usage.
Phase 4: For DSA, the total normalized CPU usage.
Phase 4: For DSA, the total logical I/Os.
Unique processor ID of the Dispatcher and part of the multicolumn NUPI for the table (see also CollectTimeStamp column).
Profile ID, if any, under which the user submitted the utility job.
Proxy role used for the utility job.
Proxy user for the utility job.
Query band under which the utility job is submitted.
Number of rows deleted.
Number of rows exported.
Number of rows inserted.
Number of rows updated.
Unique session identifier.
348 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
XML Lock Log Table: DBQLXMLLockTbl
DBC.QryLogUtility[V] Field
SessionWDID
SysConID
TDWMRuleID
UserID
UserName
Description
Workload ID class associated with the session. It determines the priority of the session when the Teradata dynamic workload software classification is active.
Internal identifier of the System Condition (Syscon) / System
Health currently enforced by the Teradata dynamic workload management software.
ID of the rule that caused a delay.
Unique ID of the user who submitted the utility job.
Name under which the session being logged was initiated.
UtilityRequest
WDID
The main utility SQL request for the job. It contains the target database and/or table names. The possible statements are:
•
BEGIN LOADING (FastLoad)
•
BEGIN MLOAD (MLOAD/MLOADX)
•
SELECT (FastExport)
•
DUMP/RESTORE (DSA)
Workload definition assigned to the main utility work.
XML Lock Log Table: DBQLXMLLockTbl
This table is populated if the WITH LOCK option is requested in addition to the default information row. Each row stores a subset of the XML document recording lock delays. The table has no primary index (NoPI) and no fallback.
DBC.DBQLXMLLOCKTbl field
CollectTimeStamp
ProcID
Description
Date and time when the lock contention is recorded. This is different from the time when the contention first occurs. A field inside XMLTextInfo represents the start time of the contention.
The lock contention may be recorded minutes after it begins, depending on when the information is flushed to the AMP. The default AMP flush rate is 10 minutes. To find when the lock was finally granted, add the lock delay duration to the start time.
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
Unique processor ID of the AMP vproc that processed the request.
Database Administration 349
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
DBC.DBQLXMLLOCKTbl field
QueryID
XMLRowNo
XMLTextInfo
Description
Internally generated identifier of the query and the foreign key to other DBQL tables.
Note: QueryID is a system-wide unique field. You can use
QueryID to join DBQL tables or join the
DBC.TdwmExceptionLog table with DBQL tables without needing ProcID as an additional join field.
Sequence number for XML plans that span more than one row.
Lock plan represented as an XML document.
Shredding the Lock Plan Information in the
XMLTextInfo Column
You can shred (format as a table) the lock plan information in the XMLTextInfo column to be able to query it and read it. Without shredding, output from that column typically looks like this: sel xmltextinfo from dbc.dbqlxmllocktbl;
*** Query completed. 7 rows found. One column returned.
*** Total elapsed time was 1 second.
XMLTextInfo
<?xml version="1.0" encoding="UTF-8"?> <!--XML row for DBQLLockXML--> <DBQLLockXML xsi:schemaLocation="http://schemas.teradata.com/dbqllockplan dbqllockplan.xsd" xmlns="http://schemas.teradata.com/dbqllockplan" xmlns:xsi="http://www.w3.org/2001/
XMLSchema-instance"> <LockContention QueryID="307191340665578443" CollectTimeStamp="2014-
09-07T23:27:07.16" DelayStartTime="2014-09-07T23:24:01.70" AbortFlag="FALSE"
TransactionCount="0"> <ContentionData LockDelay="1438" LocalDeadLock="false"
GlobalDeadLock="false" MultipleBlocker="false" vproc="1"> <BlockerRef> <RequestRef>
<RequestID RequestID_1="0" RequestID_2="9"/> <Step StepLevel="0" StepNum_1="1"
StepNum_2="0" StatementNo="1"/> <Transaction unique_1="1" unique_2="26210" vproc="30719"
TransactionState="Active" TransactionStartTime="2014-09-07T23:23:07.2" /> </RequestRef>
<SessionRef LogicalHostId="1"> <Session SessionID_1="0" SessionID_2="1139"
SessionID_Combined="1139"/> <User UserName="DBC" AccountName="DBC"/> </SessionRef>
<LockData LockTypeRequested="Read" LockObjectRequested="R" LockKind="RowHash on All
Partitions" > </LockData> <LockObject DatabaseName="MAM" TableName="t1"/> <Job> <Message
MessageClass="21" MessageKind="8" MessageClassName="SYSMSGS2SCLASS"
MessageKindName="TNT"/> <Operation WorkLevel="0" OperationType="TermiNate Transaction
(Spawned StpEDT)" JobType="AmpStep"/> </Job> </BlockerRef> <BlockedRef> <RequestRef> </
RequestRef> <SessionRef LogicalHostId="1"> <Session SessionID_1="0" SessionID_2="1140"
SessionID_Combined="1140"/> </SessionRef> <LockData LockTypeRequested="Write"
LockObjectRequested="R" LockKind="RowHash on All Partitions" StartPartition="0"
StartRowHashInt="4294967295" EndPartition="0" EndRowHashInt="4294967295" > </LockData>
</BlockedRef> </ContentionData> </LockContention> </DBQLLockXML>
Shredding the Data in the DBQLXMLLockTbl
To shred the lock plan data in the XMLTextInfo field, follow this procedure:
350 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
1
Ensure that the DIPPOST script ran on the system. At installation, DIPPOST is the final script executed as part of DIPALL. The script executes quickly if you need to run it. From the Linux command line, start DIP and then run the DIPPOST script: dip
Select the name of the post commands script:
DIPPOST
Executing DIPPOST
Please wait...
DIPPOST is complete
2
3
Would you like to execute another DIP script (Y/N)?
n
Exiting DIP...
Using BTEQ, log on, delete any objects created by previous use of this procedure, and create a user-defined target database:
.logon unzen,dbc; drop database TargetDBName; create database TargetDBName as perm=10000000;
Grant privileges to the new user locklogshredder and log on as locklogshredder.
Note: The DIPPOST script created user locklogshredder.
grant all on TargetDBName to TargetDBName with grant option; grant all on TargetDBName to dbc with grant option; grant all on TargetDBName to locklogshredder with grant option; grant all on locklogshredder to TargetDBName with grant option; grant all on td_sysxml to TargetDBName with grant option; grant all on TargetDBName to td_sysxml with grant option;
4
5
6
7
.logon unzen/locklogshredder;
Drop the target tables in case they remain from a previous shredding operation.
Note: The shredding operation creates the target tables to store the shredded data. The user does not create the SQL DDL. However, the user must prepare space for and accommodate the table. After each shredding operation, clean up the data from the previous shredding operation to avoid accumulating old data. drop table TargetDBName.Shredded_LockTbl; drop table TargetDBName.Shredded_TransTbl;
Prepare the output to be formatted as needed using BTEQ.
Call the stored procedure that creates the target tables with the shredded data. If the tables already exist, a non-zero result is returned.
CALL
LockLogShredder.SP_CreateTable('TargetDBName','Shredded_LockTbl',
'Shredded_TransTbl', resultCode);
Call the stored procedure that populates the target tables.
CALL LockLogShredder.SP_LockLog_Shredder('select random(0,21437899),
CREATEXML(xmltextinfo) from dbc.dbqlxmllocktbl',
'TargetDBName','Shredded_Locktbl', 'Shredded_TransTbl', resultCode);
This stored procedure shreds the XML data in DBQLXMLLockTbl, converting each XML from a long text string into multiple fields and storing them in two target tables. Target
351
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
8
table Shredded_LockTbl holds the shredded output of most of the lock information in the
TextInfo column of DBQLXMLLockTbl. The lock contention information contains only the blocking transaction and the first transaction that is blocked by it. Target table
Shredded_TransTbl expands this to include all blocked transactions waiting behind the blocking transaction.
Display the count of the shredded rows in the target tables, so you know in advance the size of the result:
9
sel count(*) from TargetDBName.Shredded_LockTbl; sel count(*) from TargetDBName.Shredded_TransTbl;
Query the shredded lock table and transaction table to see the list of blocked transactions:
10
sel queryid from TargetDBName.Shredded_LockTbl order by 1;
Query the data to see the primary lock table rows:
11
SELECT
DatabaseName,
TableName,
StepLevel,
StepNum_1,
StepNum_2,
StatementNo,
UserName,
AccountName,
MessageClass,
MessageKind,
MessageClassName,
MessageKindName,
WorkLevel,
OperationType,
JobType,
LockTypeRequested,
LockObjectRequested,
BlockedLockTypeRequested,
BlockedLockObjectRequested,
LockDelay,
LocalDeadLock,
GlobalDeadLock,
MultipleBlocker,
ErrorText,
ErrorSource,
AbortFlag,
ErrorCode
FROM TargetDBName.Shredded_LockTbl;
Query the data to see the transaction table rows:
SELECT
queryid,
DatabaseName,
TableName,
unique_1,
unique_2,
PEVproc,
RowStartKeyrowhash_1,
RowStartKeyrowhash_2,
RowStartKeyrowunique_1,
RowStartKeyrowunique_2,
352 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
RowKeyEndrowhash_1,
RowKeyEndrowhash_2,
RowKeyEndrowunique_1,
RowKeyEndrowunique_2,
LockTypeRequested,
LockObjectRequested
FROM TargetDBName.Shredded_TransTbl;
Result rows with NULL values reflect a simple lock contention with one blocked transaction and one blocker, which are listed in the primary lock table.
12
Join the shredded target tables Shredded_LockTbl and Shredded_TransTbl using the
QueryId field, which is in both tables.
SELECT
a.DatabaseName (TITLE 'Lock_DatabaseName'),
a.TableName (TITLE 'Lock_TableName'),
a.UserName (TITLE 'Lock_UserName'),
a.AccountName (TITLE 'Lock_AccountName'),
a.MessageClass (TITLE 'Lock_MessageClass'),
a.MessageKind (TITLE 'Lock_MessageKind'),
a.MessageClassName (TITLE 'Lock_MessageClassName'),
a.MessageKindName (TITLE 'Lock_MessageKindName'),
a.WorkLevel (TITLE 'Lock_WorkLevel'),
a.OperationType (TITLE 'Lock_OperationType'),
a.JobType (TITLE 'Lock_JobType'),
a.LockTypeRequested (TITLE 'Lock_LockTypeRequested'),
a.LockObjectRequested (TITLE 'Lock_LockObjectRequested'),
a.BlockedLockTypeRequested (TITLE 'Lock_BlockedLockTypeRequested'),
a.BlockedLockObjectRequested (TITLE 'Lock_BlockedLockObjectRequested'),
a.LockDelay (TITLE 'Lock_LockDelay'),
a.LocalDeadLock (TITLE 'Lock_LocalDeadLock'),
a.GlobalDeadLock (TITLE 'Lock_GlobalDeadLock'),
a.MultipleBlocker (TITLE 'Lock_MultipleBlocker'),
a.ErrorText (TITLE 'Lock_ErrorText'),
a.ErrorSource (TITLE 'Lock_ErrorSource'),
a.AbortFlag (TITLE 'Lock_AbortFlag'),
a.ErrorCode (TITLE 'Lock_ErrorCode'),
b.queryid (TITLE 'COMMON QUERYID'),
b.DatabaseName (TITLE 'Trans_DatabaseName'),
b.TableName (TITLE 'Trans_TableName'),
b.unique_1 (TITLE 'Trans_unique_1'),
b.unique_2 (TITLE 'Trans_unique_2'),
b.PEVproc (TITLE 'Trans_PEVproc'),
b.RowStartKeyrowhash_1 (TITLE 'Trans_RowStartKeyrowhash_1'),
b.RowStartKeyrowhash_2 (TITLE 'Trans_RowStartKeyrowhash_2'),
b.RowStartKeyrowunique_1 (TITLE 'Trans_RowStartKeyrowunique_1'),
b.RowStartKeyrowunique_2 (TITLE 'Trans_RowStartKeyrowunique_2'),
b.RowKeyEndrowhash_1 (TITLE 'Trans_RowKeyEndrowhash_1'),
b.RowKeyEndrowhash_2 (TITLE 'Trans_RowKeyEndrowhash_2'),
b.RowKeyEndrowunique_1 (TITLE 'Trans_RowKeyEndrowunique_1'),
b.RowKeyEndrowunique_2 (TITLE 'Trans_RowKeyEndrowunique_2'),
b.LockTypeRequested (TITLE 'Trans_LockTypeRequested'),
b.LockObjectRequested (TITLE 'Trans_LockObjectRequested')
FROM TargetDBName.Shredded_LockTbl a, TargetDBName.Shredded_TransTbl b
WHERE a.QueryId = b.QueryId;
Database Administration 353
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Shredded_LockTbl: Blocking Transaction Lock Shredding Target Table
The TargetDBName.Shredded_LockTbl stores shredded lock contention data for the blocking transaction and the first transaction that is blocked by it. The exact name of the target table is specified by the call to the stored procedure LockLogShredder.SP_CreateTable.
Shredded_LockTbl Field
DatabaseName
TableName
RequestID_1
RequestID_2 unique_1 unique_2
PEVproc
TransactionState
TransactionStartTime
StepLevel
StepNum_1
StepNum_2
StatementNo
SessionID_1
SessionID_2
SessionID_Combined
UserName
AccountName
ExpReqProc seq_1 seq_2
Description
The database containing the object locked by the blocker
The table locked by the blocker
Byte 1 of the request ID of the blocker
Byte 2 of the request ID of the blocker
Part 1 of the transaction ID of the blocker
Part 2 of the transaction ID of the blocker
PE vproc of the blocker
Transaction state of the blocker
Possible values include the following:
•
Inactive
•
Active
•
DontAbort
•
Aborting
•
PKAborted
Transaction start time of the blocker
The dispatcher-recorded step level of the blocker
Part 1 of the step number of the blocker
Part 2 of the step number of the blocker
The statement number of the blocker
Part 1 of the session ID of the blocker
Part 2 of the session ID of the blocker
Combined session ID of blocker, as recorded in SessionTbl
User name of the blocker
Account name of the blocker
Originating vproc of the internal express request lock of the blocker
Part 1 of the logon sequence number of a HUT lock blocker
Part 2 of the logon sequence number of a HUT lock blocker
354 Database Administration
Database Administration
Shredded_LockTbl Field
LogicalHostId
ZoneId
MessageClass
MessageKind
MessageClassName
MessageKindName
WorkLevel
OperationType
JobType
SQLType
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Description
Logical host ID of the blocker
Secure zone ID of the blocker
Internal AMP message class ID of the blocker
Internal AMP message kind ID of the blocker
Internal AMP message class name of the blocker
Possible values include the following:
•
SYSMSGSTPCLASS: Messages containing Parsergenerated steps sent to the AMPs from the Dispatcher
•
SYSMSGS2SCLASS: Messages sent between steps during row redistribution or count cascading
•
SYSMSGHUTCLASS: Messages sent between host utilities and AMPs
•
SYSMSGEXPCLASS: Messages sent as express requests between the Parser and AMPs
Internal AMP message kind name of the blocker
Operation work level number of the blocker
Type of SQL operation being performed by the blocker
Job types
Possible values include the following:
•
AmpStep
•
ExpressRequest
•
Utilities
•
Archive/Restore
•
Undetermined
Type of SQL performed in the operation of the blocker
355
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
356
Shredded_LockTbl Field
LockTypeRequested
LockObjectRequested
LockKindRequested
BlkedTableName
BlkedRequestID_1
BlkedRequestID_2
Blkedunique_1
Description
Type of lock requested by the blocker
Possible values include the following:
•
Access
•
Read
•
Write
•
Exclusive
•
IAccess (intentional access)
•
IRead (intentional read)
•
IWrite (intentional write)
•
IExclusive (intentional exclusive)
•
Read4X (read for exclusive)
•
IRead4X (intentional read for exclusive)
•
Read4W (read for write)
•
ReadHL (read host utility lock)
•
IReadHL (intentional read host utility lock)
Type of object on which the blocker requested a lock
Possible values include the following:
•
D (database)
•
T (table)
•
R (row hash)
•
RK (row hash in one partition)
•
RN (row key range)
•
RP (row hash in partition range)
•
TP (table partition range)
The kind of lock requested by the blocker
Possible values include the following:
•
Rowhash range
•
Rowhash on all partitions
•
Rowkey
•
Rowkey range (locked for a range of rows)
•
Rowhash and partition range
•
Partition range with complete rowhash range
•
Rowkey range for internal use request is waiting for a lock
The table on which the blocked request is waiting for a lock
Byte 1 of the ID of the blocked request
Byte 2 of the ID of the blocked request
Part 1 of the transaction ID of the blocked request
Database Administration
Database Administration
Shredded_LockTbl Field
Blkedunique_2
BlkedPEVproc
BlkedTransactionState
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
BlkedTransactionStartTime
BlkedStepNum_1
BlkedStepNum_2
BlockedSessionID_1
BlockedSessionID_2
BlockedUserName
BlockedAccountName
BlockedExpReqProc
Blockedseq_1
Blockedseq_2
BlockedLogicalHostId
BlockedStartPartition
BlockedStartRowhashInt
BlockedEndPartition
BlockedEndRowHashInt
Description
Part 2 of the transaction ID of the blocked request
PE vproc of the blocked request
Transaction state of the blocked request
Possible values include the following:
•
Inactive
•
Active
•
DontAbort
•
Aborting
•
PKAborted
Start time of the blocked transaction
Part 1 of the step number of the blocked request
Part 2 of the step number of the blocked request
Part 1 of the Session ID of the blocked request
Part 2 of the session ID of the blocked request
SessionTbl
User name for the blocked request
Account name for the blocked request
Originating vproc of the internal express request lock of the blocked request
Part 1 of the logon sequence number for the blocked request
Part 2 of the logon sequence number for the blocked request
Logical host ID of the blocked request
Starting partition number for the partition-level lock of the blocked request
Starting row hash number for the partition-level lock of the blocked request
Ending partition number for the partition-level lock of the blocked request
Ending row hash number for the partition-level lock of the blocked request
357
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Shredded_LockTbl Field
BlockedLockTypeRequested
BlockedLockObjectRequested
BlockedLockKindRequested
LockDelay
LocalDeadLock
GlobalDeadLock
MultipleBlocker
AmpVproc
Description
Type of lock requested by the blocked request
Possible values include the following:
•
Access
•
Read
•
Write
•
Exclusive
•
IAccess (intentional access)
•
IRead (intentional read)
•
IWrite (intentional write)
•
IExclusive (intentional exclusive)
•
Read4X (read for exclusive)
•
IRead4X (intentional read for exclusive)
•
Read4W (read for write)
•
ReadHL (read host utility lock)
•
IReadHL (intentional read host utility lock)
The type of object on which the blocked request needs a lock
Possible values include the following:
•
D (database)
•
T (table)
•
R (row hash)
•
RK (row hash in one partition)
•
RN (row key range)
•
RP (row hash in partition range)
•
TP (table partition range)
The kind of lock requested by the blocked request
Possible values include the following:
•
Rowhash range
•
Rowhash on all partitions
•
Rowkey
•
Rowkey range (locked for a range of rows)
•
Rowhash and partition range
•
Partition range with complete rowhash range
•
Rowkey range for internal use
Delay time of the blocked request in centiseconds
True if a local deadlock is detected, otherwise false
True if a global deadlock is detected, otherwise false
True if there are multiple blocked requests, otherwise false
AMP number where the lock contention occurred
358 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Shredded_LockTbl Field
QueryID
CollectTimeStamp
DelayStartTime
ErrorText
AbortFlag
ErrorCode
TransactionCount
Description
Unique query ID for joining with related lock shredded tables or other DBQL tables
Timestamp of when the lock contention was recorded
Timestamp of when the lock delay began
Error text associated with the lock contention, if any
True if the lock contention has resulted in an abort
Error code associated with the lock contention, if any
Number of additional blocked transactions
Shredded_TransTbl: Blocked Transaction Lock Shredding Target Table
The TargetDBName.Shredded_TransTbl stores additional shredded data for all blocked transactions involved in a lock contention. This table joins with the related table
Shredded_LockTbl. The exact name of the target table is specified by the call to the stored procedure LockLogShredder.SP_CreateTable.
Shredded_TransTbl Field
QueryID
TransactionOrder
Description
Unique query ID for joining with Shredded_LockTbl or other DBQL tables
Sequence order of the blocked transaction
Database Administration
TableName
RequestID_1
RequestID_2 unique_1 unique_2
PEVproc
TransactionState request is waiting for a lock
The table on which the blocked request is waiting for a lock
Byte 1 of the ID of the blocked request
Byte 2 of the ID of the blocked request
Part 1 of the transaction ID of the blocked request
Part 2 of the transaction ID of the blocked request
PE vproc of the blocked request
Transaction state of the blocked request
Possible values include the following:
•
Inactive
•
Active
•
DontAbort
•
Aborting
•
PKAborted
359
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Shredded_TransTbl Field
TransactionStartTime
StepNum_1
StepNum_2
SessionID_1
SessionID_2
SessionID_Combined
LockStatus
LockTypeRequested
LockObjectRequested
Description
Start time of the blocked transaction
Part 1 of the step number of the blocked request
Part 2 of the step number of the blocked request
Part 1 of the session ID of the blocked request
Part 2 of the session ID of the blocked request
Combined session ID of blocked request, as recorded in
SessionTbl
Lock status of the blocked transaction
Possible values include the following:
•
Waiting
•
Granted
Type of lock requested by the blocked request
Possible values include the following:
•
Access
•
Read
•
Write
•
Exclusive
•
IAccess (intentional access)
•
IRead (intentional read)
•
IWrite (intentional write)
•
IExclusive (intentional exclusive)
•
Read4X (read for exclusive)
•
IRead4X (intentional read for exclusive)
•
Read4W (read for write)
•
ReadHL (read host utility lock)
•
IReadHL (intentional read host utility lock)
Type of object on which the blocked request needs a lock
Possible values include the following:
•
D (database)
•
T (table)
•
R (row hash)
•
RK (row hash in one partition)
•
RN (row key range)
•
RP (row hash in partition range)
•
TP (table partition range)
360 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Output of Shredding
After shredding the output of the DBQLXMLLockTbl XMLTextInfo column in
Shredded_LockTbl and joining it with the shredded output of Shredded_TransTbl, the result looks like this.
The result shows 7 shredded rows. Two rows with “false” in the LockMultipleBlocker field show simple lock contentions, with 1 blocking transaction and 1 blocked transaction. Five rows show transactions that are blocked by the same 2 QueryIDs.
*** Query completed. 7 rows found. 39 columns returned.
*** Total elapsed time was 1 second.
----------------------------------------------------------------------------------------
-----------------------
Lock_DatabaseName systemfe
Lock_TableName t1
Lock_UserName ?
Lock_AccountName ?
Lock_MessageClass 8
Lock_MessageKind 5
Lock_MessageClassName SYSMSGEXPCLASS
Lock_MessageKindName EXPRELMSG
Lock_WorkLevel 0
Lock_OperationType Release lock
Lock_JobType ExpressRequest
Lock_LockTypeRequested Exclusive
Lock_LockObjectRequested Database
Lock_BlockedLockTypeRequested Access
Lock_BlockedLockObjectRequested Row
Lock_LockDelay 574.
Lock_LocalDeadLock false
Lock_GlobalDeadLock false
Lock_MultipleBlocker false
Lock_ErrorText
Lock_ErrorSource ?
Lock_AbortFlag false
Lock_ErrorCode 0
COMMON QUERYID 307191700992332723.
Trans_DatabaseName ?
Trans_TableName ?
Trans_unique_1 ?
Trans_unique_2 ?
Trans_PEVproc ?
Trans_RowStartKeyrowhash_1 ?
Trans_RowStartKeyrowhash_2 ?
Trans_RowStartKeyrowunique_1 ?
Trans_RowStartKeyrowunique_2 ?
Trans_RowKeyEndrowhash_1 ?
Trans_RowKeyEndrowhash_2 ?
Trans_RowKeyEndrowunique_1 ?
Trans_RowKeyEndrowunique_2 ?
Trans_LockTypeRequested ?
Trans_LockObjectRequested ?
----------------------------------------------------------------------------------------
-----------------------
Lock_DatabaseName systemfe
Lock_TableName t1
Lock_UserName ?
Database Administration 361
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Lock_AccountName ?
Lock_MessageClass 8
Lock_MessageKind 5
Lock_MessageClassName SYSMSGEXPCLASS
Lock_MessageKindName EXPRELMSG
Lock_WorkLevel 0
Lock_OperationType Release lock
Lock_JobType ExpressRequest
Lock_LockTypeRequested Exclusive
Lock_LockObjectRequested Database
Lock_BlockedLockTypeRequested Read
Lock_BlockedLockObjectRequested Row
Lock_LockDelay 517.
Lock_LocalDeadLock false
Lock_GlobalDeadLock true
Lock_MultipleBlocker true
Lock_ErrorText Aborted due to too many transactions waiting
Lock_ErrorSource 2
Lock_AbortFlag true
Lock_ErrorCode 2631
COMMON QUERYID 163837595253057014.
Trans_DatabaseName systemfe
Trans_TableName t1
Trans_unique_1 4
Trans_unique_2 35082
Trans_PEVproc 16382
Trans_RowStartKeyrowhash_1 2384
Trans_RowStartKeyrowhash_2 0
Trans_RowStartKeyrowunique_1 1024
Trans_RowStartKeyrowunique_2 2384
Trans_RowKeyEndrowhash_1 ?
Trans_RowKeyEndrowhash_2 ?
Trans_RowKeyEndrowunique_1 ?
Trans_RowKeyEndrowunique_2 ?
Trans_LockTypeRequested Write
Trans_LockObjectRequested Row
----------------------------------------------------------------------------------------
-----------------------
Lock_DatabaseName systemfe
Lock_TableName t1
Lock_UserName ?
Lock_AccountName ?
Lock_MessageClass 8
Lock_MessageKind 5
Lock_MessageClassName SYSMSGEXPCLASS
Lock_MessageKindName EXPRELMSG
Lock_WorkLevel 0
Lock_OperationType Release lock
Lock_JobType ExpressRequest
Lock_LockTypeRequested Exclusive
Lock_LockObjectRequested Database
Lock_BlockedLockTypeRequested Read
Lock_BlockedLockObjectRequested Row
Lock_LockDelay 517.
Lock_LocalDeadLock false
Lock_GlobalDeadLock true
Lock_MultipleBlocker true
Lock_ErrorText Aborted due to too many transactions waiting
Lock_ErrorSource 2
362 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Lock_AbortFlag true
Lock_ErrorCode 2631
COMMON QUERYID 163837595253057014.
Trans_DatabaseName systemfe
Trans_TableName t1
Trans_unique_1 3
Trans_unique_2 35085
Trans_PEVproc 16381
Trans_RowStartKeyrowhash_1 ?
Trans_RowStartKeyrowhash_2 ?
Trans_RowStartKeyrowunique_1 ?
Trans_RowStartKeyrowunique_2 ?
Trans_RowKeyEndrowhash_1 ?
Trans_RowKeyEndrowhash_2 ?
Trans_RowKeyEndrowunique_1 ?
Trans_RowKeyEndrowunique_2 ?
Trans_LockTypeRequested Read
Trans_LockObjectRequested Table
----------------------------------------------------------------------------------------
-----------------------
Lock_DatabaseName systemfe
Lock_TableName t1
Lock_UserName ?
Lock_AccountName ?
Lock_MessageClass 8
Lock_MessageKind 5
Lock_MessageClassName SYSMSGEXPCLASS
Lock_MessageKindName EXPRELMSG
Lock_WorkLevel 0
Lock_OperationType Release lock
Lock_JobType ExpressRequest
Lock_LockTypeRequested Exclusive
Lock_LockObjectRequested Database
Lock_BlockedLockTypeRequested Access
Lock_BlockedLockObjectRequested Row
Lock_LockDelay 517.
Lock_LocalDeadLock false
Lock_GlobalDeadLock true
Lock_MultipleBlocker true
Lock_ErrorText Aborted due to too many transactions waiting
Lock_ErrorSource 2
Lock_AbortFlag true
Lock_ErrorCode 2631
COMMON QUERYID 163837595253057013.
Trans_DatabaseName systemfe
Trans_TableName ?
Trans_unique_1 5
Trans_unique_2 35083
Trans_PEVproc 16383
Trans_RowStartKeyrowhash_1 ?
Trans_RowStartKeyrowhash_2 ?
Trans_RowStartKeyrowunique_1 ?
Trans_RowStartKeyrowunique_2 ?
Trans_RowKeyEndrowhash_1 ?
Trans_RowKeyEndrowhash_2 ?
Trans_RowKeyEndrowunique_1 ?
Trans_RowKeyEndrowunique_2 ?
Trans_LockTypeRequested Write
Trans_LockObjectRequested Database
Database Administration 363
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
----------------------------------------------------------------------------------------
-----------------------
Lock_DatabaseName systemfe
Lock_TableName t1
Lock_UserName ?
Lock_AccountName ?
Lock_MessageClass 8
Lock_MessageKind 5
Lock_MessageClassName SYSMSGEXPCLASS
Lock_MessageKindName EXPRELMSG
Lock_WorkLevel 0
Lock_OperationType Release lock
Lock_JobType ExpressRequest
Lock_LockTypeRequested Exclusive
Lock_LockObjectRequested Database
Lock_BlockedLockTypeRequested Access
Lock_BlockedLockObjectRequested Row
Lock_LockDelay 517.
Lock_LocalDeadLock false
Lock_GlobalDeadLock true
Lock_MultipleBlocker true
Lock_ErrorText Aborted due to too many transactions waiting
Lock_ErrorSource 2
Lock_AbortFlag true
Lock_ErrorCode 2631
COMMON QUERYID 163837595253057013.
Trans_DatabaseName systemfe
Trans_TableName t1
Trans_unique_1 4
Trans_unique_2 35082
Trans_PEVproc 16382
Trans_RowStartKeyrowhash_1 2384
Trans_RowStartKeyrowhash_2 0
Trans_RowStartKeyrowunique_1 1124
Trans_RowStartKeyrowunique_2 2584
Trans_RowKeyEndrowhash_1 ?
Trans_RowKeyEndrowhash_2 ?
Trans_RowKeyEndrowunique_1 ?
Trans_RowKeyEndrowunique_2 ?
Trans_LockTypeRequested Exclusive
Trans_LockObjectRequested Row
----------------------------------------------------------------------------------------
-----------------------
Lock_DatabaseName systemfe
Lock_TableName t1
Lock_UserName ?
Lock_AccountName ?
Lock_MessageClass 8
Lock_MessageKind 5
Lock_MessageClassName SYSMSGEXPCLASS
Lock_MessageKindName EXPRELMSG
Lock_WorkLevel 0
Lock_OperationType Release lock
Lock_JobType ExpressRequest
Lock_LockTypeRequested Exclusive
Lock_LockObjectRequested Database
Lock_BlockedLockTypeRequested Access
Lock_BlockedLockObjectRequested Row
Lock_LockDelay 517.
364 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Shredding the Lock Plan Information in the XMLTextInfo Column
Lock_LocalDeadLock false
Lock_GlobalDeadLock true
Lock_MultipleBlocker true
Lock_ErrorText Aborted due to too many transactions waiting
Lock_ErrorSource 2
Lock_AbortFlag true
Lock_ErrorCode 2631
COMMON QUERYID 163837595253057013.
Trans_DatabaseName systemfe
Trans_TableName t1
Trans_unique_1 3
Trans_unique_2 35085
Trans_PEVproc 16382
Trans_RowStartKeyrowhash_1 ?
Trans_RowStartKeyrowhash_2 ?
Trans_RowStartKeyrowunique_1 ?
Trans_RowStartKeyrowunique_2 ?
Trans_RowKeyEndrowhash_1 ?
Trans_RowKeyEndrowhash_2 ?
Trans_RowKeyEndrowunique_1 ?
Trans_RowKeyEndrowunique_2 ?
Trans_LockTypeRequested Read
Trans_LockObjectRequested Table
----------------------------------------------------------------------------------------
-----------------------
Lock_DatabaseName systemfe
Lock_TableName t1
Lock_UserName ?
Lock_AccountName ?
Lock_MessageClass 8
Lock_MessageKind 5
Lock_MessageClassName SYSMSGEXPCLASS
Lock_MessageKindName EXPRELMSG
Lock_WorkLevel 0
Lock_OperationType Release lock
Lock_JobType ExpressRequest
Lock_LockTypeRequested Exclusive
Lock_LockObjectRequested Database
Lock_BlockedLockTypeRequested Access
Lock_BlockedLockObjectRequested Row
Lock_LockDelay 582.
Lock_LocalDeadLock false
Lock_GlobalDeadLock false
Lock_MultipleBlocker false
Lock_ErrorText
Lock_ErrorSource ?
Lock_AbortFlag false
Lock_ErrorCode 0
COMMON QUERYID 307191700992332732.
Trans_DatabaseName ?
Trans_TableName ?
Trans_unique_1 ?
Trans_unique_2 ?
Trans_PEVproc ?
Trans_RowStartKeyrowhash_1 ?
Trans_RowStartKeyrowhash_2 ?
Trans_RowStartKeyrowunique_1 ?
Trans_RowStartKeyrowunique_2 ?
Database Administration 365
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the XML Lock Log Table: QryLockLogXML[V]
Trans_RowKeyEndrowhash_1 ?
Trans_RowKeyEndrowhash_2 ?
Trans_RowKeyEndrowunique_1 ?
Trans_RowKeyEndrowunique_2 ?
Trans_LockTypeRequested ?
Trans_LockObjectRequested
----------------------------------------------------------------------------------------
-----------------------
View of the XML Lock Log Table:
QryLockLogXML[V]
The QryLockLogXML[V] view of DBQLXMLLOCKTbl is populated if the WITH LOCK option is requested in addition to the default information row. Each row stores a subset of the
XML document recording lock delays.
QryLockLogXML[V] Field Description
CollectTimeStamp Date and time when the lock contention is recorded. This is different from the time when the contention first occurs. A field inside
XMLTextInfo represents the start time of the contention.
The lock contention may be recorded minutes after it begins, depending on when the information is flushed to the AMP. The default AMP flush rate is 10 minutes. To find when the lock was finally granted, add the lock delay duration to the start time.
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
ProcID
QueryID
Unique processor ID of the AMP vproc that processed the request.
Internally generated identifier of the query.
XMLRowNo
XMLTextInfo
Sequence number for XML plans that span more than one row.
Lock plan represented as an XML document.
Usage Notes
You cannot shred the contents of the XMLTextInfo column in QryLockLogXML[V]. To shred this column, you must use the underlying table, DBQLXMLockTbl. For details, see
“Shredding the Lock Plan Information in the XMLTextInfo Column” on page 350
.
366 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
XML Plan Log Table: DBQLXMLTbl
XML Plan Log Table: DBQLXMLTbl
This table stores query plans and statistics usage information in an XML document. These
BEGIN QUERY LOGGING or REPLACE QUERY LOGGING options populate the table:
•
WITH XMLPAN
•
WITH STATSUSAGE
DBC.DBQLXMLTbl Field
CollectTimeStamp
ProcID
QueryID
XMLDocType
XMLRowNo
XMLTextInfo
Description
Date and time that the rows were written to the database. This field is also part of the multicolumn NUPI (see also ProcID).
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (also see CollectTimeStamp).
Internally generated unique identifier of the query.
Describes the type of content stored in column XMLTextInfo. This column contains an encoded 32-bit integer value whose bit positions describe the type of data in the associated XML document stored in the
XMLTextInfo column. The XML DocType bit mask values are:
•
0x0000000001: XMLPLAN logging option was enabled
•
0x0000000002: STATSUSAGE logging option was enabled
•
0x0000010000: Data for XMLPLAN was logged
•
0x0000020000: Data for STATSUSAGE was logged
•
0x000000004: VERBOSE logging suboption was enabled
•
0x000000008: DETAILED logging suboption was enabled
•
0x0000040000: Data for VERBOSE was logged
•
0x0000080000: Data for DETAILED was logged
The DBC.QryLogXML[V] view shows the previous information with Y and N values. Accessing this information through
DBC.QryLogXML[V] is recommended because it is easier to use.
The row number in an XML query plan.
Contains an XML document with the query plan and/or statistics usage information.
Note: If the XMLTextInfo has more than one row per QueryID, use the
QryLogXMLDocV view to see a single, readable document per
QueryID. For more information, see
Note: The WITH ALL option does not log the query plan to DBQLXMLTbl.
For information on how to store and retrieve XML data and how to perform XML operations such as validation, query, and transformation in Teradata Database, see Teradata XML.
Database Administration 367
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the XML Plan Log Table: QryLogXMLDocV
View of the XML Plan Log Table:
QryLogXMLDocV
Note: In a future release, the XMLTextInfo column in DBQLXMLTbl will change its data type from VARCHAR to CLOB type casted to Teradata XML type. To ensure your applications remain compatible with this future enhancement, Teradata recommends using the
QryLogXMLDocV view instead of QryLogXMLV.
This view of DBQLXMLTbl shows query plans and statistics usage information in an XML document. This view combines multiple rows of XMLTextInfo with the same QueryID into a single XMLTextDoc document.These BEGIN QUERY LOGGING or REPLACE QUERY
LOGGING options populate the QryLogXMLDocV view:
•
WITH XMLPLAN
•
WITH STATSUSAGE
QryLogXMLDocV Field Description
CollectTimeStamp Date and time that the rows were written to the database. This field is also part of the multicolumn NUPI (see also ProcID).
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
DetailedStatsEnabled
HasDetailedStatsData
Possible values include:
•
Y: Detailed statistics usage logging is enabled.
•
N: Detailed statistics usage logging is not enabled.
Possible values include:
•
Y: Detailed statistics usage data is in the XMLTextInfo column.
•
N: Detailed statistics usage data is not in the XMLTextInfo column.
HasStatsUsageData
HasVerboseData
Possible values include:
•
Y: Statistics usage data is in the XMLTextInfo column.
•
N: Statistics usage data is not in the XMLTextInfo column.
Possible values include:
•
Y: VERBOSE EXPLAIN data is in the XMLTextInfo column.
•
N: VERBOSE EXPLAIN data is not in the XMLTextInfo column.
HasXMLPlanData
ProcID
QueryID
Possible values include:
•
Y: XMLPLAN data is in the XMLTextInfo column.
•
N: XMLPLAN data is not in the XMLTextInfo column.
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (also see CollectTimeStamp).
Internally generated unique identifier of the query.
368 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the XML Plan Log Table: QryLogXMLDocV
QryLogXMLDocV Field Description
StatsUsageEnabled Possible values include:
•
Y: Statistics usage logging is enabled.
•
N: Statistics usage logging is not enabled.
VerboseEnabled
XMLDocType
Possible values include:
•
Y: VERBOSE EXPLAIN logging is enabled.
•
N: VERBOSE EXPLAIN logging is not enabled.
Describes the type of content stored in column XMLTextInfo. This column contains an encoded 32-bit integer value whose bit positions describe the type of data in the associated XML document stored in the
XMLTextInfo column. The XML DocType bit mask values are:
•
0x0000000001: XMLPLAN logging option was enabled
•
0x0000000002: STATSUSAGE logging option was enabled
•
0x0000010000: Data for XMLPLAN was logged
•
0x0000020000: Data for STATSUSAGE was logged
•
0x000000004: VERBOSE logging suboption was enabled
•
0x000000008: DETAILED logging suboption was enabled
•
0x0000040000: Data for VERBOSE was logged
•
0x0000080000: Data for DETAILED was logged
Note: It is easier to use the [Option] Enabled and Has [Option] columns to see this information. Queries against these columns allow you to specify a Y or N value instead of bit-mapped values.
XMLPlanEnabled
XMLTextDoc
Possible values include:
•
Y: The XML query plan logging is enabled.
•
N: The XML query plan logging is not enabled.
Combines multiple XMLTextInfo rows with the same QueryID into a single XML document as Teradata XML type.
Improving XML Text Readability
To combine multiple XML text rows with the same query ID into one easily readable document, perform a SELECT request on the QryLogXMLDocV view.
Example
sel queryid,xmltextdoc from qrylogxmldocv where queryid=588188772714475132;
Note: Export the result set to a file. On the monitor, some of the lines may truncate.
The memory allocated for the input is 32 MB. To see the memory setting, use the following: cufconfig -o | grep MallocLimit
To add more memory, adjust the MallocLimit setting using cufconfig. For more information, see Utilities.
Database Administration 369
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
View of the XML Plan Log Table: QryLogXML[V]
View of the XML Plan Log Table: QryLogXML[V]
This view of DBQLXMLTbl shows query plans and statistics usage information in an XML document. These BEGIN QUERY LOGGING or REPLACE QUERY LOGGING options populate the QryLogXML[V] view:
•
WITH XMLPLAN
•
WITH STATSUSAGE
QryLogXML[V] Field
CollectTimeStamp
DetailedStatsEnabled
HasDetailedStatsData
HasStatsUsageData
HasVerboseData
HasXMLPlanData
ProcID
QueryID
StatsUsageEnabled
VerboseEnabled
Description
Date and time that the rows were written to the database. This field is also part of the multicolumn NUPI (see also ProcID).
For more information on CollectTimeStamp, see “Comparing
CollectTimeStamp Value Between Tables” on page 381 .
Possible values include:
•
Y: Detailed statistics usage logging is enabled.
•
N: Detailed statistics usage logging is not enabled.
Possible values include:
•
Y: Detailed statistics usage data is in the XMLTextInfo column.
•
N: Detailed statistics usage data is not in the XMLTextInfo column.
Possible values include:
•
Y: Statistics usage data is in the XMLTextInfo column.
•
N: Statistics usage data is not in the XMLTextInfo column.
Possible values include:
•
Y: VERBOSE EXPLAIN data is in the XMLTextInfo column.
•
N: VERBOSE EXPLAIN data is not in the XMLTextInfo column.
Possible values include:
•
Y: XMLPLAN data is in the XMLTextInfo column.
•
N: XMLPLAN data is not in the XMLTextInfo column.
Unique processor ID of the Dispatcher and part of the multi-column
NUPI (also see CollectTimeStamp).
Internally generated unique identifier of the query.
Possible values include:
•
Y: Statistics usage logging is enabled.
•
N: Statistics usage logging is not enabled.
Possible values include:
•
Y: VERBOSE EXPLAIN logging is enabled.
•
N: VERBOSE EXPLAIN logging is not enabled.
370 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Query Data Storage and Protection
QryLogXML[V] Field
XMLDocType
XMLPlanEnabled
XMLRowNo
XMLTextInfo
Description
Describes the type of content stored in column XMLTextInfo. This column contains an encoded 32-bit integer value whose bit positions describe the type of data in the associated XML document stored in the
XMLTextInfo column. The XML DocType bit mask values are:
•
0x0000000001: XMLPLAN logging option was enabled
•
0x0000000002: STATSUSAGE logging option was enabled
•
0x0000010000: Data for XMLPLAN was logged
•
0x0000020000: Data for STATSUSAGE was logged
•
0x000000004: VERBOSE logging suboption was enabled
•
0x000000008: DETAILED logging suboption was enabled
•
0x0000040000: Data for VERBOSE was logged
•
0x0000080000: Data for DETAILED was logged
Note: It is easier to use the [Option] Enabled and Has [Option] columns to see this information. Queries against these columns allow you to specify a Y or N value instead of bit-mapped values.
Possible values include:
•
Y: The XML query plan logging is enabled.
•
N: The XML query plan logging is not enabled.
The row number in an XML query plan.
Contains an XML document with the query plan and/or statistics usage information.
Query Data Storage and Protection
Because of its nature, there is no recovery mechanism for the cache in which DBQL row values are collected. Should a Teradata Database restart occur, any rows in cache that have not been sent to the AMPs are lost. However, query logging is not aborted as a result of a restart; DBQL uses the contents of DBQLRuleTbl to continue logging.
For information on flushing the DBQL cache information to the logs, see
Flushing the DBQL Cache” on page 373
and
“Flushing the DBQL Cache Manually” on page 373
.
Protection for a Committed Row
DBQL tables occupy permanent space in database DBC. This means that once a row has been sent to an AMP, its insertion into a DBQL log is safeguarded through the Teradata Database transaction and recovery journaling process.
In addition, the data in every DBQL table is protected with FALLBACK. This means that the data is always available unless two or more AMPs in the same clique fail simultaneously.
Database Administration 371
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Things to Consider When Logging DBQL Data
Note: Because dictionary tables are permanent, the contents of all logs remain until they are explicitly deleted. When you use DBQL, be sure to move rows from the logs to your own tables on a regular basis to avoid running out of PERM space in DBC. For instructions, see
in the following section.
Things to Consider When Logging DBQL Data
•
You must have EXECUTE privilege on the DBC.DBQLAccessMacro macro to log DBQL
data. To grant the EXECUTE privilege to an administrative user, see “Granting DBQL
Administrative Privileges to Other Users” on page 391 .
•
Teradata recommends that you use BTEQ script files to enable DBQL. You can define the rules for logging in these scripts. Just be sure to keep the scripts up to date and saved for future use. You can also create files that disable query logging for your convenience.
Logging Overhead
Before you enable logging, first consider how much and what sort of data you need for adequate analysis. The more information you ask for and the more users you track, the higher the cost to performance and the faster the logs will grow. However, you can specify summary and threshold limits to obtain meaningful data with minimal cost.
The following table lists examples that describe the overhead incurred per user and account.
IF the collection type is …
default (no options specified) individual, because the query ran longer than
THRESHOLD seconds counts of queries that complete within
SUMMARY intervals steps process detail object detail explain detail
SQL text detail
THEN logging is per …
query query that runs in more time, or used more CPU or
I/O limits query that runs in less time, or used less CPU or I/O limits query query query query
AND overhead entails …
one default row per query
IN the following log(s) …
DBQLogTbl one default row per long-running query as many rows as it takes to capture the complete explain text.
The explain text is not formatted. Collecting explain detail has some performance impact.
as many rows as it takes to capture the complete text of the SQL request
DBQLSqlTbl,
DBQLStepTbl,
DBQLObjTbl, and
DBQLogTbl
DBQLSummaryTbl when the query count is >0, one row per logging interval (every 10 minutes).
Possible maximum is four rows every 10 minutes for the duration of the session one row for each step generated DBQLStepTbl one row for each object used to resolve the query DBQLObjTbl
DBQLExplainTbl
DBQLSQLTbl
372 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Things to Consider When Logging DBQL Data
IF the collection type is …
XML query plan detail
XML lock logging
THEN logging is per …
query lock of qualifying duration
AND overhead entails …
as many rows as it takes to capture the complete
XML plan. The XML plan includes the SQL request and other things such as the EXPLAIN text
Note: For more information, see SQL Request
and Transaction Processing.
as many rows as it takes to capture the XML lock plan
IN the following log(s) …
DBQLXMLTbl
DBQLXMLLOCKT bl
Options for Flushing the DBQL Cache
Before Teradata Database writes DBQL data to Data Dictionary tables, it holds the data in cache until either:
•
The cache is full.
•
You end query logging.
•
The number of seconds has elapsed that you define in the DBS Control utility field
DBQLFlushRate.
Note: You can select a rate from 1 to 3,600 seconds. However, Teradata recommends a flush rate of at least 10 minutes (600 seconds), which is the default. Less than 10 minutes can impact performance.
•
You flush the DBQL cache manually.
Flushing the DBQL Cache Manually
If you need to flush DBQL caches more quickly than the time defined by DBQLFlushRate, either:
•
Issue the FLUSH QUERY LOGGING request:
FLUSH QUERY LOGGING WITH
flush_option
;
1101A816 where flush_option can be:
Option Name
ALLDBQL
DEFAULT
Table Cache Flushed
DBQLogTbl, DBQLSQLTbl, DBQLStepTbl,
DBQLObjTbl, DBQLXMLTbl,
DBQLExplainTbl, DBQLXMLLockTbl, and
DBQLParamTbl
DBQLogTbl
Database Administration 373
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Logging Scenarios
Option Name
EXPLAIN
LOCK
OBJECTS
PARAMINFO
SQL
STEPINFO
SUMMARY
XMLPLAN
Table Cache Flushed
DBQLExplainTbl
DBQLXMLLockTbl
DBQLObjTbl
DBQLParamTbl
DBQLSQLTbl
DBQLStepTbl
DBQLSummaryTbl
DBQLXMLTbl
This request:
•
Requires the EXECUTE privilege on DBC.DBQLAccessMacro
•
Is valid in BTET (Teradata) mode only
•
Cannot be part of a multi-statement request
•
Cannot be used in a stored procedure
Note: This request is especially recommended when a system is not busy and caches are not filling up and written to Data Dictionary tables as frequently.
This request can also flush Teradata dynamic workload management software caches.
Only the DBQL options are described here. For the Teradata dynamic workload management software options, see SQL Data Definition Language Syntax and Examples.
•
Or reset the value in the DBQLFlushRate to a faster rate.
Note: Changes to DBQLFlushRate also causes the summary/threshold cache to flush, which causes the summary/threshold collection time interval to change.
Logging Scenarios
The following tables offer examples of the type of data that is logged according to the rules you defined and the behavior of the query.
IF you…
try to drop a user or database with DBQL enabled log on as the authorized user DBC (or your DBA user) and attempt to DROP, UPDATE or ALTER any DBQL object
THEN …
an error 5780 occurs. You cannot drop a user or database that has DBQL logging enabled. First disable logging for that user or database.
the statement fails with an access error:
Failure 3523 (<username> does not have <drop | update | alter> access to <DBQL table name>)
374 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Logging Scenarios
IF you…
log on as an unauthorized user (not DBC or an administrative user) and submit a BEGIN
QUERY LOGGING or REPLACE QUERY
LOGGING statement
THEN …
the BEGIN QUERY LOGGING or REPLACE
QUERY LOGGING statement fails with an access error:
Failure 3523 (username does not have statement permission) disable query logging for a user running a session that is being logged abort a session that is being logged begin query logging for a user and that user runs the same query multiple times during one session want to view all logged rows for a query no more rows are cached for that session or that user after the running query completes.
the AbortFlag value is set to T in the DBQLogTbl row for the query.
multiple rows are logged in DBQLogTbl.
If the query is in the steps cache when it is executed after the first execution, the CacheFlag is set to T. The second time a query is run, it goes into the steps cache. The third and subsequent times, the CacheFlag is set to T.
use the QueryID field to join DBQLogTbl rows with (depending on the rules for the user) other
DBQL table rows.
want to forbid logging for any MULTILOAD job submit the following statement:
BEGIN QUERY LOGGING WITH NONE ON
APPLNAME= 'MULTLOAD';
Note: Valid application names are drawn from the pool of names returned in the LogonSource string, and for MultiLoad, it returns
MULTLOAD (no I). For a list of the commonly seen LogonSource strings, see “Application
Names” in SQL Data Definition Language
Detailed Topics. want to start logging queries for a user who already has a WITH NONE rule in place want to find out if statement errors occurred in the successful multistatement or iterated request submit REPLACE QUERY LOGGING.
REPLACE QUERY LOGGING WITH STEPINFO
ON user5 ACCOUNT = ‘abc’; check the ErrorCode column of the DBQL log table. If the ErrorCode column is populated with error 9259, this means that the query succeeded, but contains statement errors.
375
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Logging Scenarios
Scenarios of Logging Accounts
IF you…
submit one of the following statements:
BEGIN QUERY LOGGING ON ALL;
Or
REPLACE QUERY LOGGING ON ALL; submit one of the following statements:
BEGIN QUERY LOGGING ON ALL ACCOUNT =
'ABC';
Or
REPLACE QUERY LOGGING ON ALL ACCOUNT
= 'ABC'; begin query logging for a specific account for a user, but the account does not exist begin query logging for a specific account
(define a rule) for a user, and that user logs on under an account that does not match the rule want to log SQL for all users except specifically when user2 logs on under account 'ABC'
THEN …
the system creates one rule for the user named
“ALL.” If the DBQL rule for ALL is in the system, each query is logged for every user that logs on.
any user that logs on with the account string
'ABC' will be logged.
the BEGIN QUERY LOGGING statement is accepted (accounts do not need to exist).
DBQLRules[V] shows a rule for the user, but queries run by that user are not logged because the sessions never match the user/account/ application triplet.
no rows are logged for any queries run during that session.
begin query logging for a specific account for a user, and the account includes ASE codes submit the following statements:
BEGIN QUERY LOGGING WITH SQL ON
ALL;
BEGIN QUERY LOGGING WITH NONE ON myuser2 ACCOUNT='abc';
If user2 issues a query under account ABC, the system will not log the query. However, if user2 logs on with another account such as DEF, the system will log the query and DBQLSQLTbl will show the SQL for user2/DEF queries.
Note: To stop these logging rules, submit the
END QUERY statement for each rule. The rule for user2 must be specifically removed with a separate END QUERY LOGGING statement.
END QUERY LOGGING ON ALL;
END QUERY LOGGING ON myuser2
ACCOUNT='ABC'; both the input account string and the expanded account string are logged. (For details on ASE
codes, see “Logging Resource Usage Data with
Account String Variables” on page 162
.)
The system automatically strips away leading blanks and converts all letters in an account string to upper case before storing it in
DBC.DBQLRuleTbl.
376 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Logging Scenarios
IF you…
submit the following:
BEGIN QUERY LOGGING WITH OBJECTS ON
ALL ACCOUNT='WebQry&D&H'; submit the following:
BEGIN QUERY LOGGING WITH STEPINFO ON
ALL ACCOUNT='$HTactQry&D&H'; submit the following:
BEGIN QUERY LOGGING WITH SQL,
STEPINFO, OBJECTS LIMIT THRESHOLD=3
ON ALL ACCOUNT='$LStratQry&D&H';
THEN …
A row in DBQLObjTbl and DBQLogTbl for each object for each query during each
'WebQry&D&H' session.
A row in DBQLStepTbl and DBQLogTbl for each step of each query during each
'$HTactQry&D&H' session.
For each '$LStratQry&D&H' session:
•
One row of count data in DBQLSummaryTbl for all queries that completed in less than three seconds (within the 10-minute logging intervals)
•
For each query that ran longer than three seconds, one row of data is logged in
DBQLSQLTbl, DBQLStepTbl, DBQLObjTbl, and DBQLogTbl
Note: If you define more than one account string in your SQL statement, use a comma to separate the delimited list and enclose the account strings in parentheses. See SQL Data
Definition Language for more information on syntax.
Scenarios of Detailed Logging
IF you… THEN …
Objects
define a rule for a user specifying WITH
OBJECTS define a rule with WITH OBJECTS and the user runs a query that causes the Optimizer to reference the same object twice if the user runs a SELECT that joins two tables owned by the same database:
•
One row for the query is logged in
DBQLogTbl
•
Rows are logged in DBQLObjTbl as follows:
•
A row for the database
•
One row for each table
•
One row for each accessed column, based on the Optimizer plan for the query
•
One row for the object is logged in
DBQLObjTbl.
•
The value in the FreqofUse field is incremented to 2.
SQL
begin query logging with no options for a user, and that user subsequently logs on and runs a query a default row is logged for that user in
DBQLogTbl with 200 characters of the SQL.
Database Administration 377
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Logging Scenarios
IF you…
begin logging for a user with no options and the user runs a query with more than 200 characters create a rule for a user that specifies LIMIT
SQLTEXT= 0 and the user runs a query define a rule with both the WITH SQL option and LIMIT SQLTEXT=1000 define a rule with the WITH SQL option and
LIMIT SQLTEXT=0
THEN …
a row is logged in DBQLogTbl that includes the first 200 characters of the query.
logging depends on the following:
•
If you also specified the WITH ALL or WITH
SQL option, then the SQL text is logged only to DBQLSQLTbl.
•
If you did not also specify WITH ALL or
WITH SQL, then no SQL characters are logged.
a row is logged in DBQLogTbl that includes the first 10,000 SQL characters.
define a rule specifying LIMIT SQLTEXT=10000 and the user runs a query containing 15000 characters create a rule specifying LIMIT SQLTEXT=32000
(or anything larger than 10,000) and the user runs a query comprising >31000 SQL characters define a rule with just the WITH SQL option a row is logged in DBQLogTbl that includes the first 10,000 SQL characters.
The first 200 characters of the SQL statement are logged in DBQLogTbl and the entire statement is logged in as many rows as required in
DBQLSQLTbl.
The first 1,000 characters of the SQL statement are logged in DBQLogTbl and the entire statement is logged in as many rows as required in DBQLSQLTbl.
None of the SQL characters are saved in
DBQLogTbl. The entire statement is logged in as many rows as required in DBQLSQLTbl.
Steps
define a rule with WITH STEPINFO and the user runs a query that does not generate parallel steps define a rule with WITH STEPINFO and the user runs a query that generates parallel steps
One row is logged in DBQLStepTbl for each step used to resolve the query. In each row, the value of StepLev2Num is 0.
One row is logged in DBQLStepTbl for each step used to resolve the query and each parallel step is differentiated by the step level number in
StepLev2Num.
Scenarios of Threshold and Summary Options
IF you…
create rules for a user and specify LIMIT
SUMMARY=5,10,15 and during the next session, and every query takes longer than 5 seconds but less than 10 seconds to complete
THEN …
all queries fall into the second bucket (5 to 10 seconds), so the second group is the only query count logged for the session in
DBQLSummaryTbl.
378 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Logging Scenarios
IF you…
create rules for a user and specify LIMIT
SUMMARY = 5,10,15 and during the next session, every query completes in less than 5 seconds create rules and specify LIMIT
SUMMARY=1,15,10 create rules for UserA and specify SQL,
OBJECTS LIMIT THRESHOLD (without a time value), and UserA then processes four queries, where:
•
One query takes more than 5 seconds to complete
•
Three queries complete in less than 5 seconds create a rule and specify LIMIT
THRESHOLD=100000
THEN …
all queries fall into the first bucket (up to 5 seconds), so the first group is the only query count logged for the session in
DBQLSummaryTbl.
the statement is accepted (no checking is performed on SUMMARY input values) but the results are unpredictable.
the statement is accepted and the default value of 5 seconds is assigned. For the next session of
UserA:
•
The longer query is logged in DBQLSqlTbl,
DBQLObjTbl, and DBQLogTbl, with values in all valid fields of the rows.
•
For each of the three shorter queries:
•
No entries are made in DBQLSqlTbl,
DBQLObjTbl, and DBQLogTbl.
•
DBQLSummaryTbl will have a row with these values:
COUNT = 3
SECONDS = 10
LOWHIST = 5
HIGHHIST = 0
An error is returned; THRESHOLD must be less than 32K.
Example of OBJECT Data for One Query
The following example illustrates the rows in DBQLObjTbl resulting from a query logged with the WITH OBJECT option:
ObjDBName ObjTblName ObjColName ObjID ObjNum ObjType FreqofUse
---------- ---------- ----------- ------- ------- ------- ---------
D_PERSONNEL
D_PERSONNEL
?
DEPARTMENT
? 00001604
? 00009005
0
0
DB
Tab
1
1
D_PERSONNEL
D_PERSONNEL
D_PERSONNEL
D_PERSONNEL
DEPARTMENT
DEPARTMENT
DEPARTMENT
DEPARTMENT
DeptNo 00009005
DeptName 00009005
EmpCount 00009005
Loc 00009005
1,025
1,026
1,027
1,028
Col
Col
Col
Col
2
1
1
1
Database Administration 379
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Examining the Logged Data
Example of STEP Data for One Query
StepLev1
--------
1
2
3
4
4
4
4
4
4
4
5
6
The following example illustrates the rows in DBQLStepTbl resulting from a query logged with the WITH STEPINFO option:
4
5
6
7
0
1
2
3
StepLev2 StepName
---------------
0
0
MLK
MLK
MLK
SAT
SAT
INS
INS
INS
INS
CTRts
0
0
Ctb
Edt
StepStartTime
--------------------------
2004-07-08 20:37:22.770000
2004-07-08 20:37:23.770000
2004-07-08 20:37:23.780000
2004-07-08 20:37:23.790000
2004-07-08 20:37:23.790000
2004-07-08 20:37:23.800000
2004-07-08 20:37:23.800000
2004-07-08 20:37:23.820000
2004-07-08 20:37:23.830000
2004-07-08 20:37:24.110000
2004-07-08 20:37:25.080000
2004-07-08 20:37:25.120000
StepStopTime
--------------------------
2004-07-08 20:37:23.770000
2004-07-08 20:37:23.780000
2004-07-08 20:37:23.790000
2004-07-08 20:37:23.790000
2004-07-08 20:37:23.790000
2004-07-08 20:37:23.800000
2004-07-08 20:37:23.810000
2004-07-08 20:37:24.830000
2004-07-08 20:37:24.840000
2004-07-08 20:37:25.060000
2004-07-08 20:37:25.100000
2004-07-08 20:37:25.130000
RowCount
--------
1
0
1
0
0
1
1
1
1
1
1
1
Examining the Logged Data
First-level information is captured in DBQLogTbl and DBQLSummaryTbl. For shortrunning, high-volume queries, you can request the THRESHOLD option in combination with the SQL, STEPINFO, or OBJECTS logging option to reduce collection overhead. You can further specify the units of elapsed time, CPU usage, or number of I/Os.
Use the data from DBQLogTbl and DBQLSummaryTbl to identify issues such as workloads that do not meet response time service-level agreements. If space is a major concern, you can use threshold logging, but having the detail can provide much more insight.
Best practices for DBQL logging dictate that you log at the Teradata Database system level with the SQL and OBJECTS logging options. If there are specific account strings, or user IDs that only do millions of tactical, subsecond requests, Teradata Database recommends that you use the THRESHOLD logging option with the STEPINFO logging option to log CPU usage.
When used selectively, detailed data can be invaluable. Analyze it to:
•
Optimize queries or your query management or priority strategy; for example, compare:
•
Results of different queries that target the same join index
•
Elapsed times of the same query run on different days or hours
•
Determine reasons for high consumption or poor performance by correlating DBQL data to other data collections with query characterization, including QCF, ResUsage, and
DBC.AMPUsage
•
Make efficient use of existing capacity and plan for future expansion by factoring in exponential growth of query demand
You can also input detail data for Baseline Profiling, Target Level Emulation, and Teradata client tools and utilities such as Meta Data Services, and Teradata Visual Explain. Client tools aid in analysis and present the information in a graphic form that is easily manipulated and understood. For more information, see Teradata Visual Explain User Guide.
380 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Zone-level Query Logging
Comparing CollectTimeStamp Value Between Tables
For all the DBQL tables, CollectTimeStamp is captured at different points in the query processing cycle. Do not use the CollectTimeStamp field value to compare tables for this reason.
Instead, select one or more queries based on the DBQLogTbl.StartTime field or
DBQLogTbl.FirstRespTime and then join that row with the other tables based on the
QueryID field.
Note: Depending on when the DBQL caches are written, all the rows for a specific QueryID may not be flushed to the disk at the same time. This is also true when you load DBQL rows off DBC space and onto temporary, history, or archive database. Some rows for a specific
QueryID may be moved to your database while some will subsequently be logged for the DBC
DBQL tables.
Using QueryID for Joins on DBQL Tables
Because each DBQL table has its own separate cache, the system may fill up the cache for one table and log to that DBQL table before it logs into another DBQL table. This results in slightly different CollectTimeStamp values. Therefore, do not include CollectTimeStamp fields in joins.
If you need to join different DBQL tables, use the QueryID field. If you plan on comparing values between different DBQL tables, you can flush the DBQL caches to the dictionary tables.
See
“Options for Flushing the DBQL Cache” on page 373 .
Note: Although it is rare, you may see orphan rows in join results involving the DBQLObjTbl,
DBQLExplainTbl, or DBQLXMLTbl tables. An orphan row is one that does not appear in the default log table (DBQLogTbl), but has an entry in one of the three other tables.
Teradata recommends that you ignore the orphan rows and do not copy them over when you copy the DBQL tables out of DBC and into temporary or history tables in another database.
Zone-level Query Logging
Secure zones allow you to create one or more exclusive database hierarchies, called zones, within a single Teradata database system. Access to the data in each zone and zone administration is handled separately from the Teradata database system and from other zones.
Each zone can have a DBA for that zone.
Secure zones are useful in situations where the access to data must be tightly controlled and restricted. You can also use secure zones to support some regulatory compliance requirements for the separation of data access from database administration duties. Other uses include separating and securing data by corporate division or by cloud storage tenant.
Notice:
Sites that use Secure Zones should revoke SELECT privileges granted directly on DBC tables and grant access to DBC tables through views only. This prevents users from accessing information not related to their zones.
Database Administration 381
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Relationship Between DBQL Temporary Tables and DBQL History Tables
About Query Logging Privileges in a Zoned System
Zone-level query logging allows zone DBAs to monitor user queries in their zones. Zone members can enable query logging only on objects within their zones. Query logging privileges are as follows:
User Type
•
User DBC
•
Non-zone user with the ZoneBypass privilege and the EXECUTE privilege on
DBC.DBQLAccessMacro
Zone user with the EXECUTE privilege on
DBC.DBQLAccessMacro
Non-zone user with the EXECUTE privilege on
DBC.DBQLAccessMacro
BEGIN/REPLACE/SHOW/END Query Logging
Allowed
System-level. These users can see and act on zone and non-zone objects.
These users cannot end zone-level logging created by zone users.
Zone level. This user can see and act only on objects in his zone. If this user issues BEGIN
QUERY LOGGING ON ALL, only objects in the zone of this user are logged.
Non-zone-level. This user can see and act only on non-zone objects.
For More Information
For more information on secure zones, see Security Administration.
Relationship Between DBQL Temporary Tables and DBQL History Tables
Data from DBC DBQL tables is first loaded in DBQL temporary tables and then into DBQL history tables.
The following table shows the relationship between temporary DBQL tables and DBQL history tables.
Note: You need to create temporary and history tables to archive DBQL data. Temporary and history tables are not system tables automatically included in the Data Dictionary.
SYS_MGMT
Tablename Loaded From
DBQLOGTBL_TMP DBC.DBQLOGTBL
DBQLOGTBL_HST DBQLOGTBL_TMP
Used to Load
DBQLOGTBL_HST
Used to Delete
DBC.DBQLOGTBL
Primary Index
ProcId
CollectTimeStamp
LogDate
QueryID
ProcID
382 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Relationship Between DBQL Temporary Tables and DBQL History Tables
SYS_MGMT
Tablename
DBQLSQLTBL_TMP DBC.DBQLSQLTBL
DBQLSQLTBL_HST
DBQLOBJTBL_TMP
DBQLOBJTBL_HST
DBQLOBJTBL_SUM
DBQLSummaryTBL
_TMP
DBQLSummaryTBL
_HST
Note: The PI of the
DBQLSummaryTBL
_HST is as shown here because there is no good PI that offers a primary index retrieval or AMP local Joins and guarantees good distribution.
Loaded From
DBQLSQLTBL_TMP
DBC.DBQLOBJTBL
DBQLOBJTBL_TMP
DBQLOBJTBL_TMP
DBC.DBQLSummaryTBL
DBQLSummaryTBL_
TMP
Used to Load Used to Delete Primary Index
DBQLSQLTBL_HST DBC.DBQLSQLTBL
ProcID
CollectTimeStamp
DBQLOBJTBL_HST DBC.DBQLOBJTBL
LogDate
QueryID
ProcID
ProcID
CollectTimeStamp
DBQLSummaryTBL
_HST
DBC.DBQL
SummaryTBL
LogDate
QueryID
ProcID
LogDate
UserName
ObjectID
ObjectNum
ProcID
CollectTimeStamp
ProcID
CollectTimeStamp
Database Administration 383
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Daily Maintenance Process
Daily Maintenance Process
Maintenance means the recommended practice of clearing out the DBQL dictionary tables after copying relevant data into the user DBQL temporary and DBQL history tables.
Note: DBQL temporary and DBQL history tables are user tables that the Database
Administrator (DBA) generates. They are not DBQL tables. Rather they are tables that mirror the DBQL tables and serve to simplify the analysis process and the clear the raw data from the
DBQL dictionary tables. DBQL dictionary tables are internal to the DBC.
There are 2 steps to the daily maintenance process. Each step is restartable.
Note: Step 1 must finish before Step 2 can start.
•
Step 1
•
Checks to make sure that DBQLOGTBL_TMP, DBQLSQLTBL_TMP,
DBQLOBJTBL_TMP and DBQLSummaryTBL_TMP are empty.
•
Executes the END QUERY LOGGING statement on one user ID to get the buffers to dump.
•
Executes the macro SYS_MGMT.LoadDBQLTMP.
This macro loads the DBQL temporary tables from the DBC DBQL tables and deletes data from the DBC DBQL tables
•
Executes the BEGIN QUERY LOGGING statement on the user ID for which logging has just ended.
•
Executes the COLLECT STATISTICS statement on the DBQL temporary tables.
•
Step 2
•
Checks to make sure that the DBQL temporary tables have rows in them.
•
Executes the macro SYS_MGMT.LoadDBQLHSTTBLS.
This macro loads DBQLOGTBL_HST, DBQLSQLTBL_HST, DBQLOBJTBL_HST,
DBQLOBJTBL_SUM and DBQLSummaryTBL_HST and deletes data from the DBQL temporary tables.
•
Executes the COLLECT STATISTICS statement on DBQLOGTBL_HST,
DBQLSQLTBL_HST, DBQLOBJTBL_HST, DBQLOBJTBL_SUM and
DBQLSummaryTBL_HST
Monthly Maintenance Process
The monthly maintenance process consists of one step that executes the purge macro
SYS_MGMT.PRGDBQLHSTTBLS.
Retaining data for thirteen months is recommended to facilitate this year/last year comparisons. Thus, once a month the monthly maintenance script should be run to remove the 14th oldest month of data from SYS_MGMT DBQL History tables.
384 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Maintaining the Logs
Maintaining the Logs
You can access the contents of the DBQL logs, but the following DBQL objects are protected:
•
No user, including DBC and SystemFE, can access the DBQLAccessMacro macro or the
DBQLRuleTbl or DBQLRuleCountTbl tables.
•
No user can modify DBQLAccessMacro or alter, drop, or update any of the DBQL table definitions.
•
You cannot delete rows from DBQLRuleTbl or DBQLRuleCountTbl, because they are manipulated by BEGIN QUERY LOGGING, REPLACE QUERY LOGGING, and END
QUERY LOGGING processing.
However, when logged on as user DBC, SystemFE, or administrative user with the proper privileges, you can delete rows in the DBQL logs. This is necessary for controlling their size. It is a best practice to regularly offload the DBQL data from DBC and into another database.
Note: Offload the DBQL logs from DBC as often as possible. Query logging consumes DBC
PERM space and the rows remain in the DBQL tables until you remove them. Even though logging is not designed as a standard operation to be performed against all users all the time, the tables can fill up very quickly. Especially if you are tracking target objects for a large group of users.
Note: Do not offload data during peak busy times as this may block ongoing queries.
When you are ready to use DBQL on your production queries, follow this procedure:
1
2
Create and maintain an executable BTEQ script file to submit (or keep a copy of) your final BEGIN QUERY LOGGING or REPLACE QUERY LOGGING statements in case a
Sysinit is ever required (for disaster recovery or to migrate to a new platform).
After the database has been restored, you can start a BTEQ session and run the script to easily repopulate the DBQLRuleTbl and rebuild your rules cache.
Set up a regular schedule to periodically perform the following maintenance:
a
Regularly summarize the data you want to retain permanently for analysis and planning. You can save it in a user database, external files, applications, or BTEQ reports, and so forth.
b
Then submit the statement
DELETE * FROM DBQL tablename
log tables to empty them and free up DBC PERM space.
;
on each of the DBQL
Schedule clean up activities during off-peak times. You may want to consider disabling DBQL logging before you perform clean up activities on the logs although this is recommended but not necessary. Otherwise, the delete process locks the DBQL table and if DBQL needs to flush a cache to the same table to continue logging queries, the whole system could experience a slow-down.
Database Query Log Setup and Maintenance Scripts are available online at http:// downloads.teradata.com/download .
Database Administration 385
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Archiving DBQL Data
Methods for Minimizing Log Table Size
There are several ways you can minimize the size of your DBQL log tables and therefore avoid running out of disk space in DBC:
•
Regularly clean up DBQL log tables by offloading them to history tables. This is especially necessary if you are also doing detail logging. (That is, logging steps, objects, SQL, or
Explain). You should do this daily if not several times a day.
You can also create temporary tables to move data from the DBQL logs before doing an
INSERT ... SELECT in to your history tables to minimize contention.
•
Use PPIs on your DBQL history tables. If you prefer not to use a PPI, use a SI on the date column for your history table.
•
Add a date column to avoid multiple casting of the time columns.
•
Use MVC on the columns of the copies of DBQL tables.
•
Do not use fallback on your historical copies of DBQL tables. Using both NO FALLBACK and compression will save a lot of disk space.
•
Log only the data you need. For example, if default data in DBQLogTbl is sufficient, do not log WITH SQL or WITH OBJECTS as well.
•
Limit summary logging to just the accounts you want to track.
•
Limit SQL text logging to a specific number of characters.
•
Remember to end logging whenever logging data is no longer needed for a specific user, group of users, account, list of accounts, or applications.
Archiving DBQL Data
If your DBQL table archives are defined as PPI tables, do the following:
1
2
3
Archive DBQL data nightly. This is strongly recommended so that the DBQL logs do not take up space in DBC and impact the system.
Add a log date to each row. The log date is used as the partitioning column and makes querying the archived data much faster.
Change the primary indexes. This is done to avoid table skewing.
You can also separate historical data from junk queries to make your DBQL table archives more concise.
Reviewing or Ending Current Rules
The following sections describe how to review rules and stop DBQL logging.
Reviewing Rules
You can review DBQL rules one of two ways:
386 Database Administration
Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Reviewing or Ending Current Rules
•
Use the SHOW QUERY LOGGING statement to determine which rule would be used for a specified user/account/application. For example:
SHOW QUERY LOGGING ON marketinguser;
•
Use the DBC.DBQLRules[V] view as a window into the contents of the DBC.RuleTbl which stores all DBQL rules in the system.
The following table describes what kind of rules the system checks for depending on the
SHOW QUERY LOGGING statement you submit. If you only list a user name or an account string, the SHOW QUERY LOGGING statement shows the Best Fit Rule.
Note: Only a user with EXECUTE privilege on DBC.DBQLAccessMacro can execute the
SHOW QUERY LOGGING statement.
IF you submit a SHOW QUERY
LOGGING statement for…
ALL user1 acct1 user2 all acct3
APPLNAME='xxx'
THEN the system will show…
the first rule in the hierarchy that applies, looking only at all/all accounts the first rule in the hierarchy that applies, looking at:
•
user1/account1
•
user1/all accounts
•
all/account1
•
all/all accounts the first rule in the hierarchy that applies, looking only at:
•
user2/all accounts
•
all/all accounts the rule hierarchy (in hierarchy order) that would apply for any user that logged on under acct 3. The system will look only at:
•
all/account3
•
all/all accounts
APPLNAME='xxx' which is the only APPLNAME rule that applies.
The DBC.DBQLRules[V] view provides a window into the contents of the DBQLRuleTbl table and shows which rules would be applied by the system. Only a user with SELECT privilege on DBC.DBQLRules[V] can access the view. For Teradata systems configured with secure zones, access to some Data Dictionary views is constrained by zone. DBQLRules[V] is constrained by zone. For more information, see Data Dictionary.
A SELECT on the DBQLRules[V] view displays the rules currently in effect. You can qualify the response by user name, or account string (if it exists), or both. For example:
SELECT * FROM DBC.DBQLRULESV WHERE ACCOUNTSTRING='$LTest&D&H';
387
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Reviewing or Ending Current Rules
The following table describes the fields of DBC.DBQLRules[V]:
DBC.DBQLRules[V]
Fields
AccountString
ApplName
AlgMode
StatsUsageFlag
ExplainFlag
LockDelay
AlgMode
DetailDiag
DetailedStats
ObjFlag
NoColumns
ParamFlag
SqlFlag
StepFlag
SummaryFlag
SummaryVal2
SummaryVal3
TextSizeLimit
Description
The default or specified user account.
The application name for the rule.
The DBQL CPU/IO Collection algorithm to use for this rule.
Indicates whether statistics usage is logged. The possible values are T (true) or F (false).
Whether explain text is stored. Possible values are T (true) or F (false).
An integer indicating in centiseconds the minimum threshold length of a lock contention. Any lock contention greater than this length will be recorded in DBC.DBQLXMLLOCKTbl.
For the BEGIN/REPLACE QUERY LOGGING options:
•
STATSUSAGE: displays if the DETAILED suboption is enabled (T) or disabled (F).
•
XMLPLAN: displays if the VERBOSE suboption is enabled (T) or disabled
(F).
Detailed diagnostic control information represented in bit format.
Whether detailed statistics are being logged. Possible values are T or F.
Whether object data (columns and indexes) are stored. Possible values are T or F.
Whether columns are logged as part of object logging. Possible values are T or F.
Whether parameters and values are stored for parameterized requests.
Possible values are T or F.
Whether SQL text is stored. Possible values are T or F.
Whether step-level data is stored. Possible values are T or F.
Whether summary information is stored in DBC.DBQLSummaryTbl.
Possible values are T or F.
Returns the high value for interval 2 if SummaryFlag is True
The value can be in seconds, I/Os, or hundredths of CPU seconds (see
TypeOfCriterion).
Returns the high value for interval 3 if SummaryFlag is True.
This same value is used for interval 4.
The value can be in seconds, I/Os, or hundredths of CPU seconds (see
TypeOfCriterion).
Indicates the number of characters of SQL text to be stored.
388 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Reviewing or Ending Current Rules
DBC.DBQLRules[V]
Fields
ThresholdFlag
TypeOfCriterion
TypeOfRule
UserName
Verbose
XMLPlanFlag
Description
Possible values are:
T = Count the queries that are shorter or equal to SummaryVal1 in
DBC.DBQLSummaryTbl. Provide detailed data on longer queries in
DBC.DBQLogTbl.
F = Provide detailed data for all queries in DBC.DBQLogTbl unless the
SummaryFlag is True.
Specify one of the following criteria with summary and threshold options:
•
ElapsedSec: Elapsed seconds
•
CPUTime: CPU time (hundredths of a second)
•
IOCount: I/O count
•
CPUTimeNorm: Normalized CPU time (hundredths of a second) for coexistence systems
•
ElapsedTime: Elapsed time (hundredths of a second)
•
No criteria
Whether logging is enabled or disabled by this rule. The value is either:
•
Logging enabled
•
WITH NONE (No logging)
The user name associated with the rule.
Whether verbose EXPLAIN and related information in XML format are provided. Possible values are T or F.
Whether XML plans are stored. Possible values are T or F.
Ending DBQL Logging for Specific Users and/or Accounts
When you are ready to stop logging, specify any accounts on which you enabled DBQL logging. For example, if you submitted the following statement:
BEGIN QUERY LOGGING WITH OBJECTS ON APPLNAME='FASTLOAD';
to stop logging, use:
END QUERY LOGGING ON APPLNAME='FASTLOAD';
Teradata recommends using scripts to enable and disable logging to avoid typographical errors.
Note: The application names you specify with the APPLNAME option are the names the system passes in the LogonSource string. (For a list of the commonly seen LogonSource string application names, see “Application Names” in SQL Data Definition Language.)
Database Administration 389
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Reviewing or Ending Current Rules
Ending All DBQL Logging
Use the END QUERY LOGGING ON ALL RULES statement to eliminate all rules currently in the DBQLRuleTbl table using the ON ALL RULES option. For example:
END QUERY LOGGING ON ALL RULES;
The system will:
•
Delete rules from the DBQLRuleTbl.
•
Set the DBQLRuleCountTbl.NumberOfRules field to 0.
•
Flush all rules caches.
Effects of Dynamically Enabling/Replacing/Disabling Logging on Current
Rules
When you enable, replace, or disable query logging, the currently running query is not affected, but all subsequent queries take on new rule changes.
The following table describes the DBQL behavior as a result of a change to an active session.
IF you…
enable logging (submit a
BEGIN QUERY LOGGING statement) for an active session replace an existing rule (submit a REPLACE QUERY
LOGGING statement) abort a session that is being logged
AND a query for that session is already… THEN …
in process
•
Data for the current query is not collected.
•
Logging begins with receipt of the next query.
in process cached logging begins with receipt of the next query.
disable logging (submit an
END QUERY LOGGING statement) for an active session change a logon account of a user while the user is submitting queries that are under DBQL logging cached in process if a default row is being logged (logging was not just SUMMARY), the AbortFlag is set to T.
•
One or more DBQL rows are written
(but may be incomplete).
•
The current query will be logged
(perhaps in cache if it is in flight).
•
All DBQL caches are flushed.
•
Subsequent queries during that session are not logged.
•
All rules caches are flushed.
whether or not the DBQL logging rule can be applied on the session is re-evaluated on the next query.
390 Database Administration
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Granting DBQL Administrative Privileges to Other Users
Granting DBQL Administrative Privileges to
Other Users
You need the EXECUTE privilege on the special macro DBQLAccessMacro to enable and disable query logging. DBQLAccessMacro is created by DIPVIEWSV of the DIP utility.
4
5
6
1
2
The system users DBC and SystemFE have the EXECUTE privilege on DBQLAccessMacro and can grant it to others when needed. If you want other users, such as your administrative user, to be able to execute DBQLAccessMacro, follow this procedure:
Log on as user DBC (or SystemFE).
List the contents of database DBC to see if the DBQLAccessMacro, DBQLRuleTbl, and
DBQLRuleCountTbl have been created:
3
7
HELP DATABASE DBC ;
The DBQL components listed in
“DBQL Tables, Views, and Macros” on page 280 should
be reported.
Grant the following privileges to your database administrator user:
GRANT EXECUTE ON DBC.DBQLAccessMacro TO DBADMIN ;
GRANT SELECT ON DBC.DBQLRULESV TO DBADMIN ;
Log off the DBC or SystemFE session.
Log on again as user DBADMIN.
Define query logging rules for one or more users, one or more accounts, or applications using BEGIN QUERY LOGGING or REPLACE QUERY LOGGING statements. (For full syntax, see SQL Data Definition Language.)
Check the DBQLRules[V] view to see if the rules are correct:
SELECT * FROM DBC.DBQLRulesV ;
If you find an error, submit an REPLACE QUERY LOGGING statement for that user.
Database Administration 391
Chapter 15: Tracking Query Behavior with Database Query Logging: Operational DBAs
Granting DBQL Administrative Privileges to Other Users
392 Database Administration
CHAPTER 16
Analyzing Requests and Request
Plans: Application DBAs
This chapter describes how to analyze request plans using the EXPLAIN request modifier and the Query Capture Facility.
Using EXPLAIN to Analyze Request Plans
About the EXPLAIN Request Modifier
The EXPLAIN request modifier reports an English language summary of the request plan generated by the Optimizer to process any valid SQL request. The summary describes the
AMP steps the system would perform to resolve the request, but the request is not processed unless the Optimizer uses a dynamic plan and then the request is partially processed.
Note: To EXPLAIN a request, you must have the privileges required to execute the request.
EXPLAIN is useful for evaluating and optimizing SQL requests. It provides a summary of the access and join plans generated by the Optimizer for the specified SQL request, including information such as:
•
The indexes to be used
•
Any intermediate spool files that will be generated
•
The join types to be performed
•
Whether the requests in a transaction will be dispatched in parallel
•
The relative time it will take to complete the request
•
Whether a dynamic or static request plan will be used
Using EXPLAIN allows you to compare the relative performance of the different versions of a request and choose the most efficient SQL.
Recommendation: Always use EXPLAIN to analyze any new requests under development.
Subtle differences in the way a request is structured can produce significant differences in its resource impact and performance, even though the requests may produce identical end results.
Generating an EXPLAIN with SQL Assistant
1
In the SQL Assistant Query window, enter the SQL request to be EXPLAINed.
2
Press F6.
Note: Shortcut F6 always produces a static request plan. For the dynamic plan, use DYNAMIC
EXPLAIN request.
Database Administration 393
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Capturing Request Plan Steps
For more information, see Teradata SQL Assistant for Microsoft Windows User Guide
.
Generating an EXPLAIN with BTEQ
To generate an EXPLAIN for an SQL request, add the word “EXPLAIN” at the beginning of the request.
For example, the following SQL performs an EXPLAIN on a SELECT statement:
EXPLAIN SELECT * FROM Employee;
If you want a dynamic request plan instead, issue DYNAMIC EXPLAIN request. For example,
DYNAMIC EXPLAIN SELECT * FROM Employee;
Reference Information
Information on...
syntax and options for the
EXPLAIN request modifier interpreting the output of the
EXPLAIN request modifier for various request types
Teradata Visual Explain utility
Is available in...
SQL Data Manipulation Language
SQL Request and Transaction Processing
Teradata Visual Explain User Guide
Capturing Request Plan Steps
About the Query Capture Facility
SQL provides a powerful facility for analyzing the request plans the Optimizer creates for various data manipulation statements. This feature, called the Query Capture Facility, permits a DBA to capture access plan information from the Optimizer using the SQL statements
COLLECT DEMOGRAPHICS, DUMP EXPLAIN, INSERT EXPLAIN, and BEGIN QUERY
CAPTURE (see SQL Data Definition Language for more information about these statements) and then analyze the captured data using the Teradata Index Wizard or standard SQL DML statements.
The captured request plans are stored in a relational database called a Query Capture Database
(QCD) that you can create for that purpose. QCD is a user database whose space is drawn from available permanent disk space: it is not a set of system tables found in a fixed location.
Applications of QCF include:
•
Store all request plans for customer requests. You can then compare and contrast requests as a function of software release, hardware platform, and hardware configuration.
•
Provide data so that you can generate your own detailed analyses of captured request steps using standard SQL DML statements and third-party request management tools.
394 Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Relationship Between BEGIN QUERY CAPTURE and BEGIN QUERY LOGGING
•
Provide the input for these utilities: Teradata Index Wizard (which recommends index definitions for improved efficiency), and Teradata VE (which presents a graphical view of the flow of data during request execution).
About Capturing Request Plans Using DBQL
An alternative to using the Query Capture Facility is to capture request plans using the request BEGIN QUERY LOGGING WITH XMLPLAN. This request causes the request plan for every non-cached INSERT, UPDATE, SELECT, DELETE, MERGE, and EXEC statement to be logged as an XML document in DBC.DBQLXMLTbl. For details, see
.
Relationship Between BEGIN QUERY CAPTURE and BEGIN QUERY LOGGING
BEGIN QUERY CAPTURE requests always take precedence over BEGIN QUERY LOGGING requests. If you submit a BEGIN QUERY LOGGING request when there is already an active
BEGIN QUERY CAPTURE request on the same user, the query logging takes place for the user in all the sessions except for the CAPTURE session and the query capture session continues capturing queries.
This is true whether the BEGIN QUERY LOGGING statement was issued in the capture session or from some other session. The opposite is true when you submit a BEGIN QUERY
CAPTURE request when there is already an active BEGIN QUERY LOGGING request on the same user. The query capture takes place for the capture session and the query logging for the user continues for all the other sessions.
DBS Control Parameters for Tuning the Workload Cache Limits for Index and Partition Analysis
Index Wizard processing in Teradata Database allocates workload cache for the purpose of caching information retrieved from the QCD and other temporary working structures and needed during Analysis and Validation phases of recommending indexes and partitioning expressions for various SQL workloads.
Two DBS Control performance group parameters control the respective workload cache sizes for the Analysis and Validation phases.
•
IAMaxWorkloadCache
•
IVMaxWorkloadCache
The following table provides some of the details for these parameters. See Utilities for further information.
Database Administration 395
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About the Query Capture Database
Parameter Purpose
IAMaxWorkloadCache Defines the maximum size of the Index
Wizard workload cache for Analysis operations.
This parameter is applicable to both the
INITIATE INDEX ANALYSIS and
INITIATE PARTITION ANALYSIS statements.
IVMaxWorkloadCache Defines the maximum size of the Index
Wizard workload cache for Validation operations.
This parameter is applicable to all SQL statements issued within a session when
DIAGNOSTIC “Validate Index” has been enabled.
Valid Range
(megabytes)
32 - 256
1 - 32
Default
(megabytes)
8
48
About the Query Capture Database
A query capture database, or QCD, stores the Optimizer query analysis workload data and the statistical and demographic data needed to support query analysis.
The SystemFE DIP creates a default QCD called TDQCD beneath the SystemFE user, but you can also create your own QCD databases, and you can define multiple QCD databases for your systems.
You must be granted the INSERT privilege to use a BEGIN QUERY CAPTURE request to capture query plans in XML format at the session level and write data into TDQCD or any other user-defined QCD.
Teradata Database determines the size of TDQCD based on the number of system AMPs using the following equation.
TDQCD_size = 50MB +
number_of_system_AMPs x 100 x 1,024 bytes
You can use the modify_tdqcd.spl SQL stored procedure to modify the size of the TDQCD database based on the number of AMPs in a test system. The definition of modify_tdqcd.spl is as follows.
REPLACE PROCEDURE TDQCD.MODIFY_TDQCD(IN Basesize int, OUT Success int)
BEGIN
DECLARE TotalAmps INTEGER;
DECLARE CurrentSize INTEGER;
DECLARE TargetSize INTEGER;
SELECT COUNT(DISTINCT Vproc) FROM Dbc.DiskSpaceV INTO TotalAmps ; select ZEROIFNULL(permspace) into :CurrentSize from Dbc.DatabasesV where databasename =
'tdqcd';
396 Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About the Query Capture Database
SET TargetSize = Basesize + (TotalAmps * 100 * 1024);
IF (CurrentSize < TargetSize) then
CALL dbc.sysexecsql('MODIFY DATABASE TDQCD AS PERM = ' ||:TargetSize || 'BYTES');
END IF;
SET Success = 1;
END;
Use the Control Center feature of the Visual EXPLAIN tool (see “Procedure Using the Visual
EXPLAIN Utility” on page 398 ) or BTEQ using the CreateQCF table (see
) to create your query capture databases.QCD tables are populated using various methods, including the following:
•
BEGIN QUERY CAPTURE
•
COLLECT DEMOGRAPHICS
•
COLLECT STATISTICS (QCD Form)
•
INSERT EXPLAIN … WITH STATISTICS
•
BEGIN QUERY CAPTURE
•
Invoking workload macros from the client-based Teradata Index Wizard utility.
Notice:
Because the QCD is often a set of user tables, you can change both their individual structure and the structure of the QCD; however, you should never change any of the QCD structures in any way.
The reason for this constraint is that the SQL statements BEGIN QUERY CAPTURE, DUMP
EXPLAIN, INSERT EXPLAIN, INITIATE INDEX ANALYSIS, RESTART INDEX ANALYSIS,
COLLECT DEMOGRAPHICS, and COLLECT STATISTICS (QCD Form) all assume the default physical model when they capture the specified query, statistical, and demographic information for insertion into the QCD tables.
The results of changing the structure of the QCD tables are unpredictable.
Deleting Query Plans from a QCD
6
7
Use the Control Center option of Visual EXPLAIN to delete query plans from a QCD.
3
4
1
2
5
Start Visual EXPLAIN.
Pull down the Tools menu and select Control Center.
Select the Manage QCD tab.
Select the Clean up QCD button.
Select the database from which query plans are to be removed from the Database scroll bar.
Select the Delete Database Objects radio button.
Select the OK button.
Database Administration 397
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Creating the Query Capture Database Tables
Creating the Query Capture Database Tables
About Creating QCD Tables
Before you can capture request plan information from the Optimizer white tree and statistical and data demographic information from their respective data structures, you must either create and secure the QCD tables or create your own user-defined query capture database.
This topic explains two ways to set up your QCDs.
Procedure Using the Visual EXPLAIN Utility
Perform the following procedure to create the tables and other database objects for the query capture database using Visual EXPLAIN.
If you do not have the Visual EXPLAIN tool, then see
“Procedure Using BTEQ” on page 399 .
1
2
Start Visual EXPLAIN.
Pull down the Tools menu and select Control Center.
7
8
9
10
11
5
6
3
4
Select the Manage QCD tab.
Select the Setup QCD button.
Select the Create all QCF database objects (tables and macros) radio button.
Type the name of the database owner in the Owner selection box.
Specify the Perm Space parameter in the Perm Space selection box.
Specify the Spool Space parameter in the Spool Space selection box.
Check the Fallback check box depending on your requirements.
Select the Create button.
Secure the QCD tables by granting the appropriate privileges to the users who will be analyzing its data.
You can implement additional secured access by creating views on QCD.
12
Populate the QCD tables using the appropriate tools.
•
BEGIN QUERY CAPTURE requests to capture request plans. For the syntax of this statement, see SQL Data Definition Language Syntax and Examples.
•
INSERT EXPLAIN requests to capture query plans. For the syntax of this statement, see SQL Data Manipulation Language.
398 Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Creating the Query Capture Database Tables
•
Any of the following statements, as appropriate, to capture statistical and demographic data:
•
COLLECT DEMOGRAPHICS
•
COLLECT STATISTICS (QCD Form)
•
INSERT EXPLAIN … WITH STATISTICS
•
The appropriate workload macros to create workloads for index analysis.
Procedure Using BTEQ
Perform the following procedure to create the tables for the query capture database using
BTEQ. This procedure does not create the macros that the Visual EXPLAIN utility needs to work with a QCD. You must create these macros from the Visual EXPLAIN Control Center before the utility can visualize or load QCD query plans.
The procedure assumes that you have already created QCD_database_name.
1
2
Start BTEQ.
Change your current database or user to the database in which the QCD tables are to be created as follows.
3
DATABASE QCF_database_name; where QCF_database_name is the name of the database you created for the QCD tables.
Perform the following steps in the indicated order.
a .SET WIDTH 254
b .EXPORT FILE = file_name
c SELECT TabDefinition
FROM systemfe.CreateQCF
ORDER BY SeqNumber;
4
d .EXPORT FILE = file_name
e .RUN FILE = file_name where file_name is the name of the file you create to contain the output of the SELECT request executed in
.
Secure QCD by granting the appropriate privileges to the users who will be analyzing its data.
To use the BEGIN QUERY CAPTURE request to insert data into the QCD tables in
TDQCD, you must have the ALL privilege.
See SQL Data Control Language for information about how to implement column-level and row-level security for your tables.
You can implement additional secured access by creating views on QCD.
Database Administration 399
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Dropping Query Capture Database Tables
5
Populate the QCD tables using the appropriate tools.
•
BEGIN QUERY CAPTURE requests to capture query plans. For the syntax of this statement, see SQL Data Definition Language Syntax and Examples.
•
INSERT EXPLAIN requests to capture request plans. For the syntax of this statement, see SQL Data Manipulation Language.
•
Any of the following statements, as appropriate, to capture statistical and demographic data.
•
COLLECT DEMOGRAPHICS
•
COLLECT STATISTICS (QCD Form)
•
INSERT EXPLAIN … WITH STATISTICS
•
The appropriate workload macros to create workloads for index analysis.
Client tools like Visual EXPLAIN and the Teradata Index Wizard access QCD tables using the views and macros created using the Control Center feature of Visual EXPLAIN.
Dropping Query Capture Database Tables
About Dropping QCD Tables
This topic describes how to drop all the tables and other database objects from a QCD database.
Procedure Using Visual EXPLAIN
Perform the following procedure to drop the tables and other database objects for the query capture database using Visual EXPLAIN.
If you do not have the Visual EXPLAIN tool, then see
below.
5
6
7
3
4
1
2
Start Visual Explain.
Pull down the Tools menu and select Control Center.
Select the Manage QCD tab.
Select the Clean up QCD button.
Select the database to be cleaned up from the Database scroll bar menu.
Select the Delete Database Objects radio button.
Select the OK button.
Procedure Using BTEQ
Perform the following procedure to drop the tables for the query capture database using
BTEQ. The procedure assumes that you have already created QCF_database_name. This procedure does not drop the views and macros on the specified QCD.
400 Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About Querying QCD Tables
1
2
3
Start BTEQ.
Change your current database to the database from which the QCF tables are to be dropped as follows.
DATABASE QCD_database_name; where QCD_database_name is the name of the database you created for the QCD tables.
Perform the following steps in order.
a .EXPORT FILE = file_name
b SELECT DelTable
FROM systemfe.CleanupQCF
ORDER BY SeqNumber;
c EXPORT RESET;
d .RUN FILE = file_name where file_name is the name of the file you create to contain the output of the SELECT request in
About Querying QCD Tables
You can analyze the information in QCD using several different approaches.
The following methods are typical ways to use a QCD to analyze query data.
•
To create and compare graphical EXPLAIN reports, use the Visual EXPLAIN tool.
•
To perform ad hoc requests of QCD, use interactive SQL DML statements.
•
To perform standardized analyses of QCD, create macros, stored procedures, or embedded
SQL applications to perform your standard requests and report the results.
See
“About QCD Request Macros and Views” on page 401 for a set of macros you can use
for this purpose.
•
To run index validation diagnostics for the Teradata Index Wizard, use the DIAGNOSTIC
“Validate Index” statement. See SQL Data Manipulation Language for the syntax of the
DIAGNOSTIC “Validate Index” statement.
About QCD Request Macros and Views
Teradata Database provides a set of views on the QCD tables to restrict their access. Various levels of access to QCD tables are granted to different user categories.
Teradata Database also provides a set of macros for maintaining various aspects of the query capture database. The Control Center feature of the Visual EXPLAIN client utility provides the interface to create the views and macros and to grant privileges to QCD users.
Database Administration 401
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About Querying QCD Tables
QCD Macro and View Versions
Two versions of all the views and macros are created in a QCD.
X
NonX
Version Definition
Access restricted to information inserted by the current user in the QCD.
Access permitted to information inserted by any user of the QCD.
These macros and views are usually performed by means of the Visual EXPLAIN and Teradata
Index Wizard client utilities.
User Categories
The following categories of users are defined on a QCD to enhance its security.
User Category
Normal
Power
Administrator
Description
Loads, views, and deletes own plans or workloads only.
Loads and views plans or workloads inserted by any user.
Deletes own plans or workloads only.
Loads, view, and deletes any plan created by any user.
Drops and deletes QCD tables.
QCD creator has Administrator privileges granted by default.
Specific User Category Privileges
The following table indicates the specific privileges granted to each user category.
User Category
Normal
Database Object Type
QCD tables
X views
X macros
Non-X views
Non-X macros
Privileges Granted
•
INSERT
•
UPDATE
•
SELECT on the following tables.
•
IndexRecommendations
•
SeqNumber
•
TableStatistics
•
DELETE on the AnalysisLog table
SELECT
EXEC none none
402 Database Administration
Database Administration
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
About Querying QCD Tables
User Category
Power
Administrator
Database Object Type
QCD tables
X views
X macros
Non-X views
Non-X macros
QCD tables
X views
X views use the value for
UDB_Key in the Query table to restrict access to request plans to the user who captures them, while the Non-X views do not restrict that access.
X macros
Non-X views
Non-X macros
•
DROP
•
EXECUTE
•
DELETE
•
DROP
•
INSERT
•
SELECT
•
UPDATE
•
DROP
•
EXECUTE
Privileges Granted
•
INSERT
•
UPDATE
•
SELECT on the following tables:
•
IndexRecommendations
•
SeqNumber
•
TableStatistics
•
DELETE on the AnalysisLog table
SELECT
EXEC
SELECT
EXEC (excluding DELETE plan and workload macros)
•
DELETE
•
DROP
•
INSERT
•
SELECT
•
UPDATE
•
DELETE
•
DROP
•
INSERT
•
SELECT
•
UPDATE
403
Chapter 16: Analyzing Requests and Request Plans: Application DBAs
Building Baseline Transaction Profiles
Building Baseline Transaction Profiles
Baseline profiles can provide information on typical resource usage by period and by user, on a daily, weekly, or monthly basis. Building baseline transaction profiles can help you track the effects of software upgrades, introductions of new users or new applications, and help you determine what a “healthy and normal” system should look like.
You can build baseline profiles for:
•
Single operations (such as FastLoad, full table scans, primary index INSERT ... SELECTs, select joins, and so on.)
•
Multiple, concurrently run transactions
Once defined and stored, baseline profiles can help you:
•
Compare current to profiled operations on a real-time basis.
•
Collect data instantaneously for a set interval.
•
Detect and resolve throughput anomalies.
You can gather performance metrics for profiling from:
•
ResUsage reports. See Resource Usage Macros and Tables.
•
DBQL. See Chapter 15: “Tracking Query Behavior with Database Query Logging:
•
Viewpoint.
404 Database Administration
CHAPTER 17
Testing Query Performance Using
Target-Level Emulation:
Application DBAs
This chapter describes using the Target-Level Emulation facility to capture production system information. You can then use this information to generate a query plan identical to the plan that would be produced on the production system.
You can use this tool to validate and verify new queries in a test environment, ensuring that your production work is not disrupted by problematic queries.
About Target Level Emulation
The Target Level Emulation facility permits you to emulate a production environment on a test system by capturing environmental cost parameters, dynamic AMP sample statistics, intermediate IPE feedback of results or statistical data, and Optimizer-relevant DBS Control information from that environment and storing it in the relational tables
SystemFE.Opt_Cost_Table, SystemFE.Opt_RAS_Table, and SystemFE.Opt_DBSCtl_Table.
SystemFE.Opt_XMLPlan_Tbl stores query plans in XML format for use in target-level emulation.
You can then use the information from these tables together with appropriate column and index statistics to make the Optimizer on the test system generate query plans as if it were operating in the target environment rather than the test environment.
This feature produces a query plan for the emulated target system: it does not emulate the performance of that system.
The SQL DIAGNOSTIC statements that support Target Level Emulation are documented in
SQL Data Manipulation LanguageFor additional information about using Target Level
Emulation from a client utility GUI interface, see Teradata System Emulation Tool User Guide.
Benefits
This feature ensures that queries are thoroughly debugged and optimized in a safe, but equivalent, emulated environment prior to introducing them into the production environment. TLE:
•
Models the impact of various environmental changes and DBS Control parameter settings on SQL request performance.
Database Administration 405
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
About Target Level Emulation
•
Provides an environment for determining the source of various Optimizer-based production database query problems using environmental cost data and dynamic AMP sample-based statistical data.
About the Two Forms of Target Level Emulation
There are 2 forms of target level emulation.
•
Cost-based
•
Dynamic AMP sample-based
The DBS Control information stored in SystemFE.Opt_DBSCtl_Table supports both forms of target level emulation. The two forms are definitely not mutually exclusive: they are meant to be used together as a tool for the precise analysis of production system query plans generated on much smaller test systems.
Environmental cost parameters are constant across a system configuration because there is only one set of environmental cost values per system. Table statistics, on the other hand, are different for each base table and do not vary as a function of system configuration. For example, the cardinality of a given table is the same whether that table is on a two AMP system or a 100 AMP system; however, environmental costs vary considerably as a function of system configuration.
These differences account for the sometimes dissimilar syntax of analogous SQL
DIAGNOSTIC statements for handling cost and dynamic AMP statistical information.
Cost-based target level emulation has two possible forms: static and dynamic.
•
Static emulation refers to setting environmental cost parameters at the SYSTEM level.
Once set, the cost parameters persist across system restarts and reboots.
•
Dynamic emulation refers to setting environmental cost parameters at the IFP (referring to the Parsing Engine or PE), REQUEST, or SESSION levels (see SQL Data Manipulation
Language for definitions of what these terms mean in the context of Target Level
Emulation).
When you first set cost parameters, they are read dynamically from
SystemFE.Opt_Cost_Table and copied to a memory-resident data structure. The Optimizer then initializes its cost parameters from this memory-resident structure dynamically. The cost parameters do not persist across system restarts and reboots.
The advantage of dynamic target level emulation is that multiple users in multiple sessions can simultaneously emulate environmental cost parameters from multiple target systems on the same test system.
Dynamic AMP sample-based target level emulation permits you to generate dynamic AMP statistical samples on a production system and then export the captured data to a test system for detailed query analysis.
406 Database Administration
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
Enabling Target Level Emulation
About Related Tools and Utilities
The target level emulation feature is closely related to other Teradata Database support tools, particularly those detailed in the following list:
•
Query Capture Facility.
•
Visual Explain Tool.
See Teradata Visual Explain User Guide.
•
Teradata System Emulation Tool (TSET).
You can perform TLE tasks using the Teradata System Emulation Tool interface. See
Teradata System Emulation Tool User Guide for details.
•
Teradata Index Wizard.
See Teradata Index Wizard User Guide.
•
CHECK STATISTICS option of the INSERT EXPLAIN and DUMP EXPLAIN statements.
See the documentation for these statements in SQL Data Manipulation Language.
Enabling Target Level Emulation
Target Level Emulation is disabled by default and should never be enabled on a production system.
Before you can use target level emulation on a test system, you must enable it.
The following is a high level procedure for performing this task.
1
2
Run the DIP utility script DIPSYSFE to create the optimizer cost values table
SystemFE.Opt_Cost_Table, the dynamic AMP sample statistical values table
SystemFE.Opt_RAS_Table, the XML-formatted query plans table
SystemFE.Opt_XMLPlan_Tbl, and the DBS Control parameters table
SystemFE.Opt_DBSCtl_Table.
Use the DBS Control utility to enable Target Level Emulation on your test system by setting DBS Control general parameter 22, Target Level Emulation, to TRUE.
Target Level Emulation and Cost Profiles
To enable optimizer cost profile system DIAGNOSTIC statements in combination with Target
Level Emulation, you must set the EnableCostProfileTLE parameter of DBS Control to TRUE.
If the DBS Control parameter EnableSetCostProfile is set to 0, Teradata Database automatically sets EnableCostProfileTLE to FALSE. This cannot be changed independently.
To solve the problem, ensure that the EnableSetCostProfile parameter is set to a value other than 0 and then set the EnableCostProfileTLE parameter to TRUE.
Database Administration 407
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
Capturing Target System Optimizer Cost Values and Dynamic AMP Statistical Samples
Capturing Target System Optimizer Cost Values and Dynamic AMP Statistical Samples
Process Phases
The extraction of Optimizer cost value information from a target system and its subsequent application to a test system is a two-phase process, as follows.
1
Data capture phase.
2
Target system optimizer variables are read from the Dictionary and various system tables and written to a staging area. This phase is initiated from the target system.
Application phase.
The captured data in the staging area is exported to the appropriate Dictionary and system tables on the test system. This phase can be initiated from either the target or the test system.
You can use the Teradata System Emulation Tool from a client system to perform this data capture and application. See Teradata System Emulation Tool User Guide for details.
If you do not use Teradata System Emulation Tool, then you must follow a more circuitous path to capture target system optimizer variables and apply them to a test system. This topic presents both procedures.
Procedure Using the Teradata System Emulation Tool
Capturing data from a target production system and applying it to a test system is a two-phase process.
•
The first phase reads the relevant data on the target system and then writes it to a staging area.
•
The second phase exports the staging data to the test system and applies it to the appropriate data tables and system files.
The following is a high level overview of this procedure.
1
Depending on your requirements, perform one or all of the following steps.
a
Populate SystemFE.Opt_Cost_Table and SystemFE.Opt_DBSCtl_Table on the target system by running the DIAGNOSTIC DUMP COSTS statement (see SQL Data
Manipulation Language).
This operation extracts the Optimizer environmental cost value and DBS Control information from the target system and writes it to a relational table so it can be copied and used to drive the Optimizer on a test system.
b
Populate SystemFE.Opt_RAS_Table on the target system by running the DIAGNOSTIC
DUMP SAMPLES statement (see SQL Data Manipulation Language).
This operation extracts dynamic AMP statistics from the header of the specified table on the target system and writes it to a relational table so it can be copied and used to drive the Optimizer on a test system.
408 Database Administration
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
Capturing Target System Optimizer Cost Values and Dynamic AMP Statistical Samples
2
3
4
5
6 c
Populate SystemFE.Opt_XMLPlan_Tbl by running the DIAGNOSTIC DUMP
XMLEXPLAIN statement (see SQL Data Manipulation Language).
Log onto Teradata System Emulation Tool from a client connected with the target system.
Create a target system profile to define the databases, tables, views, macros, and related environmental parameters or dynamic AMP statistics and DBS Control information to be captured for loading onto the test system.
Collect the Optimizer cost value information or dynamic AMP statistics, the DBS Control information, and any necessary object definitions and test queries from the target system and place it in a staging area from which the test system can import it.
To do this, invoke the Teradata System Emulation Tool Export function from the target system. This action writes the data created by DIAGNOSTIC DUMP COSTS or
DIAGNOSTIC DUMP SAMPLES into a staging area.
Invoke the Teradata System Emulation Tool Import function from a client connected with the test system to import the data from the staging area on the target system to
SystemFE.Opt_Cost_Table or SystemFE.Opt_RAS_Table and to
SystemFE.Opt_DBSCtl_Table on the test system.
The Import function permits you to select from several different load utilities to use to transfer the data.
Depending on your requirements, perform one or both of the following steps:
a
Run the DIAGNOSTIC SET COSTS SQL statement (see SQL Data Manipulation
Language) to specify the exact level of cost value information (and for which target configuration) you want to set.
b
Run the DIAGNOSTIC SET SAMPLES SQL statement (see SQL Data Manipulation
Language) to specify the exact level of dynamic AMP statistical sampling (and for which target configuration) you want to set.
You can also use Teradata System Emulation Tool to copy any of the following target system data to an appropriate test system:
•
Object definitions
•
Statistics
•
QCD data
•
Dynamic AMP samples
•
All available target system data, including cost parameters
Procedure Using SQL DIAGNOSTIC Statements
The following list provides a high level procedure for capturing Optimizer environmental cost information from a production system and applying that information to a test system.
1
Depending on your requirements, perform one or both of the following steps.
a
Populate SystemFE.Opt_Cost_Table and SystemFE.Opt_DBSCtl_Table on the target system by running the DIAGNOSTIC DUMP COSTS statement (see SQL Data
Manipulation Language for the syntax and usage information for this statement).
Database Administration 409
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
Capturing Target System Optimizer Cost Values and Dynamic AMP Statistical Samples
2
3 b
Populate SystemFE.Opt_RAS_Table on the target system by running the DIAGNOSTIC
DUMP SAMPLES statement (see SQL Data Manipulation Language for the syntax and usage information for this statement).
This extracts the dynamic AMP statistical samples from the target system and writes them to a relational table so they can be copied and used to drive the Optimizer on a test system.
Depending on your requirements, perform one or both of the following steps.
a
This extracts the Optimizer environmental cost value information from the target system and writes it to a relational table so it can be copied and used to drive the
Optimizer on a test system.
Copy the Optimizer cost value information and any necessary object definitions and test queries from the target system to the test system.
To do this, select rows from SystemFE.Opt_Cost_Table on the production system and export them to a file using an appropriate load utility from the following list:
•
FastExport
•
CLIv2
•
BTEQ SQL script
b
Copy the dynamic AMP statistical samples and any necessary object definitions and test queries from the target system to the test system.
To do this, select rows from SystemFE.Opt_RAS_Table on the production system and export them to a file using an appropriate load utility from the following list.
•
FastExport
•
CLIv2
•
BTEQ SQL script
Depending on your requirements, perform one or both of the following steps.
a
Import the rows into SystemFE.Opt_Cost_Table on the test system using an appropriate load utility from the following list.
•
FastLoad
•
MultiLoad
•
Teradata Parallel Data Pump
•
CLIv2
•
BTEQ SQL script
b
Import the rows into SystemFE.Opt_RAS_Table on the test system using an appropriate load utility from the following list.
•
FastLoad
•
MultiLoad
•
Teradata Parallel Data Pump
•
CLIv2
•
BTEQ SQL script
410 Database Administration
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
Target Level Emulation and Cost Profiles
4
5
6
7
Import the rows into SystemFE.Opt_DBSCtl_Table on the test system using an appropriate load utility from the following list.
•
FastLoad
•
MultiLoad
•
Teradata Parallel Data Pump
•
CLIv2
•
BTEQ SQL script
Import the rows into SystemFE.Opt_XMLPlan_Tbl using an appropriate load utility from the following list.
•
FastLoad
•
MultiLoad
•
Teradata Parallel Data Pump
•
CLIv2
•
BTEQ SQL script
Populate the test system with appropriate test data imported from the production system for which testing is to be performed.
Depending on your requirements, perform one or both of the following steps.
a
Run the DIAGNOSTIC SET COSTS SQL statement (see SQL Data Manipulation
Language) to specify the exact level of cost value information (and for which target configuration) you want to set.
b
Run the DIAGNOSTIC SET SAMPLES SQL statement (see SQL Data Manipulation
Language) to specify the exact level of dynamic AMP statistical sampling (and for which target configuration) you want to set.
For an illustration, see
“About Mapping Target System Files to a Test System” on page 414 .
Target Level Emulation and Cost Profiles
Using Target Level Emulation to Capture Target System Cost Profile Data
You can use Target Level Emulation to obtain all necessary target system cost profile data. The general procedure is to use Teradata System Emulation Tool to save a copy of the
DBC.CostProfiles and DBC.ConstantValues rows for the active cost profile on the target system, and then to install these as variable profiles on the emulation system (see
Emulation and Cost Profiles” on page 407 ).
Target system cost profile information is associated with the TLE cost segment row in
SystemFE.Opt_Cost_Table.
This information consists of the following data.
•
Profile type
•
Profile name
Database Administration 411
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
Target Level Emulation and Cost Profiles
•
Profile category
•
Any rows for that profile from the DBC.ConstantValues table
Capturing Profile Information
You can capture Profile identity information by submitting the following request:
DIAGNOSTIC DUMP COSTS targ-sys-name
The profile type, name, and category are stored in the cost table row.
Teradata System Emulation Tool looks for this profile during export and, if found, captures the result of the following macro, which produces a set of SQL requests that create the actual cost profile:
EXEC DBC.ShowCostProfileAs('profile-name'
, 'targ-sys-name_Profile'
, ');
Teradata System Emulation Tool Import Function
The Teradata System Emulation Tool Import function installs the cost segment row in the
SystemFE.Opt_Cost_Table table on the emulated system and then runs the sequence of SQL requests to create a variable cost profile named targ-sys-name_Profile. It then inserts the constant value rows collected from the target system.
Installing Cost Segments at a Particular Emulation Level
The following diagnostic request installs the cost segment and activates targ-sys-name_Profile at the specified emulation level.
DIAGNOSTIC SET COSTS targ-sys-name,
PROFILE targ-sys-name_Profile ON FOR level; where level is one of the following scopes.
This scope level...
REQUEST
IFP
SESSION
SYSTEM
Applies to the current...
SQL request.
parsing engine.
session.
system, meaning there is no restriction enforced.
412 Database Administration
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
Capturing and Installing Costing Information Including the Active Cost Profile
Capturing and Installing Costing Information
Including the Active Cost Profile
Two Ways to Capture and Install Cost Profile Information
There are 2 ways to capture Target Level Emulation data:
•
Using the Teradata System Emulation Tool Export function (see Teradata System
Emulation Tool User Guide).
•
Using the following SQL DIAGNOSTIC statements:
•
DIAGNOSTIC COSTPRINT
•
DIAGNOSTIC DUMP COSTS
•
DIAGNOSTIC SET COSTS
•
DIAGNOSTIC SET PROFILE
•
DIAGNOSTIC DUMP SAMPLES
•
DIAGNOSTIC SET SAMPLES
See SQL Data Manipulation Language for the syntax and usage notes for these statements.
Procedure Using SQL DIAGNOSTIC Statement Requests
The procedure defined here uses the second method.
1
Determine the name of the session profile.
2
DIAGNOSTIC HELP PROFILE;
Assume the request returns the name prof-name for the session profile.
Capture the cost segment and SQL to generate the emulation cost profile.
DIAGNOSTIC DUMP COSTS Wellington;
.EXPORT FILE=ProfileSQL
EXEC DBC.ShowCostProfileAs ( 'prof-name'
, 'Wellington_Profile'
, 'Active cost profile for Wellington')
3
4
.EXPORT reset
Load the file named ProfileSQL.
To do this, execute the following BTEQ installation command.
.RUN FILE=ProfileSQL
This installs as a Wellington_Profile variable Cost Profile.
Insert the cost segment row for Wellington into SystemFE.Opt_Cost_Table and his DBS
Control and internal parameter rows into SystemFE.Opt_DBSCtl_Table using
DIAGNOSTIC DUMP COSTS (see
“Capturing Target System Optimizer Cost Values and
Dynamic AMP Statistical Samples” on page 408 ).
Database Administration 413
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
About Mapping Target System Files to a Test System
5
6
Insert the internal parameter row for Wellington into SystemFE.Opt_RAS_Table using
).
Run the appropriate SQL DIAGNOSTIC requests to install the Target Level Emulation data.
DIAGNOSTIC SET COSTS Wellington
, PROFILE Wellington_Profile
ON FOR scope_level; where scope_level indicates one of the following.
This scope level...
REQUEST
IFP
SESSION
SYSTEM
Applies to the current...
SQL request.
parsing engine.
session.
system, meaning there is no restriction enforced.
For details, see “SET DIAGNOSTIC COSTS” in SQL Data Manipulation Language.
DIAGNOSTIC SET SAMPLES Wellington
ON FOR scope_level; where scope_level indicates one of the following.
This scope level...
SESSION
SYSTEM
Applies to the current...
session.
system, meaning there is no restriction enforced.
7
For details, see “DIAGNOSTIC SET SAMPLES” in SQL Data Manipulation Language.
This step is optional.
Remove the profile.
EXEC DBC.DeleteCostProfile ('Wellington_Profile');
About Mapping Target System Files to a Test
System
This topic provides an overview of the process of mapping target system files to the Optimizer on a test system using Target Level Emulation.
The following graphic indicates the flow of how target system optimizer environmental cost parameters are mapped to test system structures that, in turn, cause the test system query
414 Database Administration
Database Administration
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
About Mapping Target System Files to a Test System optimizer to generate request plans that are based on the environmental cost parameters and data demographics of the emulated production system.
SystemFE Database
Cost Structure
Environment Structure
System Structure
Cost Table
Target Key
Target
System
Target System
Mapped to
Global PDE
System
Structures
Mapped
Target
Structure
Mapped
Cost Table
Structure
Optimizer Cost GDO
FF07D348
The flow for dynamic AMP statistical samples is essentially identical after making the necessary changes to the node labels in the diagram.
415
Chapter 17: Testing Query Performance Using Target-Level Emulation: Application DBAs
About Mapping Target System Files to a Test System
416 Database Administration
CHAPTER 18
Working with System Information
and Global Defaults: Operational DBAs
This chapter provides an overview of the Teradata Database configuration, the global default controls, the client connection software, and describes tools you can use to view or change system-wide (global) information.
Viewing the Software Release and Version
There are several methods for determining the current software release and version.
•
From any console or any client session, you can view the currently running Teradata
Database version and release level with the following query:
SELECT * FROM DBC.DBCInfoV;
Note: You must have the SELECT privilege on the DBC.DBCInfoV view to use this query.
Typically, this view grants SELECT access to PUBLIC by default.
The query returns the version and release level in the following form:
*** Query completed. 3 rows found. 2 columns returned.
*** Total elapsed time was 3 seconds.
InfoKey InfoData
----------------------------- ---------------------------------------
RELEASE NN.nn.nn.nn
LANGUAGE SUPPORT MODE Standard
VERSION NN.nn.nn.nn
BTEQ -- Enter your DBC/SQL request or BTEQ command:
Where NN.nn.nn.nn is equal to the number of the current release. For example,
15.10.00.00.
•
You can use the GET VERSION command in Supvr window of Database Window to view the current version of the PDE, DBS, GTW, TCHN, RSG, and TDGSS.
For more information on Database Window, see “Database Window (xdbw)” in Utilities.
Reviewing or Changing System-Wide
Parameters
The values for system-wide defaults are distributed across the configuration via the following:
•
If your system is set for Universal hash code, the:
Database Administration 417
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
About International Character Set Settings and Defaults
•
DBC.Hosts system table
•
tdlocaledef.txt file
•
Globally Distributed Objects (GDOs), which store global configuration settings that are automatically propagated to all nodes of an MPP system. The system uses several GDOs internally, each storing configuration settings that are specific to different system functions or utilities. For example, there are GDOs to store the settings managed by the DBS
Control, Cufconfig, Gtwcontrol, and Control GDO Editor utilities. GDOs are used to store configuration settings for Teradata dynamic workload management software and TASM, and many other Teradata Database functions. Use the gdomview command-line tool from the system console to view a history of the changes made to the GDO. You can access the gdomview online documentation from a system console using the ‘man’ and ‘pdehelp’ commands.
About International Character Set Settings and
Defaults
Teradata Database uses internal server character sets to represent character data stored within the database and supports many external client character sets.
For information on how to implement character sets, see International Character Set Support.
International Language Support Mode
During installation, your database is set up with either Standard or Japanese language support mode enabled. All object names are stored in Unicode so that Data Dictionary field definitions across all Teradata platforms are consistent
To determine what language support mode your system is using, submit the following: sel infodata from dbc.dbcinfoV where infokey = 'LANGUAGE SUPPORT MODE';
Default Server Character Sets
Whenever you define CHAR or VARCHAR data as part of defining a table, UDF, macro, or stored procedure and you do not specify a character set, the default is determined by the default character set for the user.
Server character sets include:
•
LATIN
•
UNICODE
•
KANJI1
•
KANJISJIS
Note: In accordance with Teradata internationalization plans, KANJI1 support is deprecated and is to be discontinued in the near future. KANJI1 is not allowed as a default character set; the system changes the KANJI1 default character set to the UNICODE character set. Creation of new KANJI1 objects is highly restricted. Although many KANJI1 queries and applications
418 Database Administration
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
About International Character Set Settings and Defaults may continue to operate, sites using KANJI1 should convert to another character set as soon as For more information, see “KANJI1 Character Set” in International Character Set Support.
Client Character Sets
Each client uses a particular character set during a session to transmit user data. The client character set maps each character in the client to an equivalent character in the server set.
Teradata Database translates data strings received from a client into the server, or internal, character set for storage and processing, and translates it back to the client character set when exporting response data to the client. This translation allows clients using different character sets to access and modify the same stored data.
Teradata Database offers many predefined client character sets (along with the codes for the appropriate collation sequences). However, you can also define and install your own character set translations and collations on the server and flag those currently desired as available for use. You can assign an active character set as the default for a client and also activate up to a maximum of 16 client character sets.
Note: The ASCII, EBCDIC, UTF-8, and UTF-16 character sets are permanently installed and always available.
Viewing the Status of Client Character Sets
Use the views described below to determine the current status of installed character sets.
View Name
DBC.CharTranslationsV
DBC.CharSetsV
DBC.HostsInfoV
Description
Displays the names of installed character sets.
Each name identifies the set of translation tables needed to define the external-internal character mapping for one language. They can be predefined, or created by you.
Note: Because they are permanently loaded and available, the ASCII,
EBCDIC, UTF-8, and UTF-16 character sets are not reported.
Displays the names of the character sets you flagged in the
DBC.Translations table as the ones to activate during a tpareset.
These are currently active and available to users at the session level only if a tpareset was performed after you set the flags.
Note: A name in this view and CharTranslationsV does not prove that it is active. The InstallFlag column in CharTranslationsV is an indication, but can be misleading if the table was changed without a tpareset.
Displays the names of the character sets you assigned as the client defaults, by host.
Client character sets permanently enabled by Teradata Database include:
•
ASCII
•
EBCDIC
Database Administration 419
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
About International Character Set Settings and Defaults
•
UTF-8
•
UTF-16
For more information on mapping and collation codes, see International Character Set
Support. For IBM mainframe clients, see “TDP Functionality” on page 430 and Teradata
Director Program Reference.
For a complete description of the views described in the previous table, see Data Dictionary.
Changing Character Set Defaults
You can change the default client character set and the default server character set at the user level, and the user can choose alternatives during a session.
Default Client Character Set
If you do not define a client character set as the default for a client in the DBC.Hosts table, the automatic default is the character set native to that client.
During a session, you can find out which client character set is in effect with the SQL HELP
SESSION statement and can specify a different (but active) client character set in various ways, depending on the particular client software. For example, with the BTEQ client software, you can use the SESSION CHARSET command:
.SET SESSION CHARSET ASCII
For details on when the SESSION CHARSET command can be used, see Basic Teradata Query
Reference.
Default Server Character Set
To specify the character set for a user, issue a CREATE USER or MODIFY USER request with the DEFAULT CHARACTER SET clause. If you do not specify this clause when you create or modify a user, the setting of the Default Character Set field in the DBS Control utility determines the default character set.
The default value for the Default Character Set field is 0 (Unicode) for Japanese Language
Support mode and 1 (Latin) for Standard Language Support mode.
Notice:
If you change the character set for user DBC, user-defined functions may not work without being recompiled. You must also:
•
Rerun DIPUDT and DIPDEM
•
Recompile any site-defined DBC functions that use SQL_TEXT
Changing Collation Defaults
You can define the default collation sequence for a user with the COLLATION clause of the
CREATE/MODIFY USER statement. Teradata Database provides a number of small but flexible sets of collations to choose from. For example, CHARSET_COLL produces a binary ordering based on the current character set; MULTINATIONAL collation can be tailored as needed.
420 Database Administration
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Changing Time Zones, Currencies, and Formats for Date, Time, and Numbers
If you do not define a default collation for the user, the automatic default is HOST (the collation that is compatible with the logon client). HOST collation gives you EBCDIC collation on mainframe-attached clients and ASCII collation for all others.
During a session, the user can override the user or current session default with the SQL statement SET SESSION COLLATION. For example:
SET SESSION COLLATION JIS_COLL;
For a full explanation of Teradata Database collating conventions and instructions on how to define your own sequences, see International Character Set Support.
Changing Time Zones, Currencies, and Formats for Date, Time, and Numbers
To define or change your time zone or how Teradata Database formats numeric, date, time, and currency output, use the command-line utility Teradata Locale Definition (tdlocaledef).
Tdlocaledef converts a specification for data formatting (SDF) text file into a Teradata
Database globally distributed object (GDO), an internal binary format file that stores configuration information. The GDO is made available simultaneously to all nodes of an MPP system. The utility can also convert the text file to a local, non-distributed, binary file, that can be converted back to text in order to ensure the formatting syntax is valid.
Note: Format changes take effect only after Teradata Database is restarted, and do not affect columns that were created prior to the restart.
Note: A new Teradata system is typically staged with global defaults in the SDF file that are appropriate for the United States. International customers and those who need customized settings, including TimeZoneString, can change the defaults using tdlocaledef.
For more details on the SDF file, see “SDF File” in the description of the Teradata Locale
Definition (tdlocaledef) utility in Utilities.
Changing Time Zone Strings and Rules
Note: Teradata requests that customers open an incident so that the Customer Support
Center can assist you in changing time zone settings on your system.
If you do not need Daylight Saving Time, you can set the time using the DBS Control utility settings SystemTimeZoneHour and SystemTimeZoneMinute.
To redefine your time zone, check the list of available time zone strings and their rules in
/usr/tdbms/etc/tdlocaledef_tzrules.txt.
These rules are also in the
GetTimeZoneDisplacement UDF. To change a time zone string and rules, you must change both the GetTimeZoneDisplacement UDF and tdlocaledef.
For more information, see “Adding or Modifying Time Zone Strings” under the topic
“GetTimeZoneDisplacement” in SQL Functions, Operators, Expressions, and Predicates and
DBS Control utility and Teradata Locale Definition utility in Utilities.
Database Administration 421
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Changing Time Zones, Currencies, and Formats for Date, Time, and Numbers
Daylight Savings Time and the Ambiguous Hour
Locales that observe Daylight Savings Time (DST) set their clocks ahead one hour for a portion of the year for DST, then return their clocks to standard time by setting the clocks back. As a consequence, on the day when DST ends and clocks are turned back an hour, the times within this regained hour occur twice in the same day.
Without a way to distinguish the DST (first) occurrence of a specific time value within this hour from the standard time (second) occurrence, these time values are ambiguous unless a time zone offset is explicitly specified with the time value.
Teradata Database can automatically account for DST time switching through the use of time zone strings. Time zone strings allow you to define named time zones for locales based on their time offset from UTC. For locales that observe DST, time zone strings can include rules that define when DST starts and ends. Teradata Database uses this information to determine appropriate handling of time and date information, and to automatically adjust time zone offsets as appropriate for locales that observe DST.
However, when time zone strings are used for these locales, but time zone offset values are not included with time values, Teradata Database cannot determine whether time values within this ambiguous hour should be interpreted and treated as DST or as standard time. In these cases, the DBS Control setting for DisplacementOnOverlap determines whether these times are assumed by Teradata Database to have a DST or standard time zone offset.
Note: This issue occurs only when the following three conditions are true:
•
Time zone strings are used for DST locales
•
Time values do not include a time zone offset value
•
The DBS Control field TimeDateWZControl is set to 3 (date and time values without time zone offset information are stored using system local time)
For more information on...
Time zone strings
DisplacementOnOverlap
TimeDateWZControl
See...
Database Administration.
SQL Functions, Operators, Expressions, and Predicates.
the Teradata Locale Definition Utility chapter in Utilities. the DBS Control chapter in Utilities.
422 Database Administration
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Client Configuration Overview
Client Configuration Overview
There are two types of Teradata Database clients.
Type
Mainframe-attached
Network-attached
Description
Mainframe computing system (IBM z/OS only).
Windows, Novell Linux, Red Hat Linux, Oracle Solaris, HP-UX, and
IBM AIX
A session is the transfer of data between a user on a client and Teradata Database on the server, including the logical and physical connections that make data transfer possible.
Teradata Director Program for Mainframe-Attached Clients
The mainframe interface enables communication between a mainframe client and Teradata
Database. Teradata Director Program establishes a session with the associated PE and manages communications between the client and Teradata Database. When a user or application submits an SQL request, the TDP transfers it to Teradata Database, receives the response, and transfers the response to the application.
Network-Attached Clients
In a network environment, the client interface product (such as CLIv2 or the Teradata JDBC
Driver) establishes a session with the gateway and manages the routing of user request/server response parcels to and from the gateway. The gateway manages the routing of parcels to and from the PE.
Database Administration 423
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Client Configuration Overview
The following figure provides a functional overview of mainframe-attached and networkattached clients and Teradata Database.
Mainframe-Attached
System
Client
Application
Network-Attached
System
Client
Application
CLI
Channel
TDP
Teradata Database
Client
Interface
Product
TCP/IP
Network
Gateway
Parsing
Engine
Parsing
Engine
Message Passing Layer
AMP AMP AMP AMP
1093-002A
424 Database Administration
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Communicating with Teradata Database
Communicating with Teradata Database
Sessions Defined
A session is a logical connection between the user and Teradata Database. A session permits a user to submit one transaction at a time and receive one response at a time, and the current session can have only one transaction outstanding at any time. A user may communicate through one or more active sessions concurrently.
A session is explicitly logged on and off from Teradata Database and is established when the
Teradata Database server accepts the logon string of the user. When a session is logged off, the system discards the user-session identification and does not accept additional Teradata SQL statements from that session.
Request and Response Messages
When a session has been logged on, Teradata Database communicates with the host through request and response messages containing parcels of information. The maximum response message size limit is 1MB.
How Request Parcels Are Handled
To access Teradata Database from a mainframe client, the user or application program logs on through a Teradata client interface product and submits a request. The request is processed by the interface and directed to a TDP.
Each request is handled as follows:
•
In a mainframe environment, mainframe CLIv2 builds the request parcel and sends it to the TDP, which sends it through the mainframe connection to the PE associated with that
TDP.
•
In a network environment, the client interface product (such as CLIv2 or the Teradata
JDBC Driver) builds the request parcel and sends it over the network to the gateway. The gateway sends the parcel to the PE for the session, which may or may not reside on the same node as the gateway.
Response Parcels
The result of a query or a status becomes a returned answer set. The PE turns the answer sets for a request into one or more response parcels and returns them to:
•
The TDP, in a mainframe environment. The TDP and mainframe CLIv2 returns it to the client utility or program.
•
The gateway, in a network environment. The gateway sends it to the client interface product (such as CLIv2 or the Teradata ODBC Driver). The client interface product returns the response to the client utility or program.
Database Administration 425
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Mainframe Environment
Controlling Session Defaults
You can control session defaults by including the DEFAULT DATABASE, COLLATION, or
ROLE options in the CREATE USER or MODIFY USER statement, or by issuing a SET statement during a session.
You can also control session conditions for particular users by defining a STARTUP clause containing a string of one or more Teradata SQL statements. A STARTUP string is subject to the same restrictions as a macro.
Client-Server Applications Connectivity
Applications performing embedded SQL statements need to preprocessed by the Teradata
Preprocessor2, which runs on a client system.
You can run the preprocessor against an application without connecting to Teradata Database if you specify the SQLCHECK (-sc) option as NOSYNTAX. However, to precompile and run an application, a separate connection to Teradata Database is required for the precompilation and the runtime event.
For instructions on how to prepare, precompile, and run an application on Teradata Database with embedded SQL statements, see SQL Stored Procedures and Embedded SQL and Teradata
Preprocessor2 for Embedded SQL Programmer Guide.
Also see:
•
Teradata Call-Level Interface Version 2 Reference for Workstation-Attached Systems
•
Teradata Call-Level Interface Version 2 Reference for Mainframe-Attached Systems
Mainframe Environment
Teradata utilities and software programs support Teradata Database access in mainframe environments. These utilities and programs run under the operating system of the client and provide the functionality for a user to access the database system.
Background
The Teradata Mainframe Interface enables communication between a mainframe client and a
Teradata Database server using a channel with either a parallel or serial I/O interface. In the case of the Extended Channel Solution (ECS), the Teradata Mainframe Interface uses either a channel plus TCP/IP, or TCP/IP instead of a channel.
The ECS interface includes:
•
The TCHN software on the TPA node, which manages channel devices and communicates with Teradata Database
•
The APSV software on the ECS node, which encapsulates channel write blocks and sends them to the Teradata system via TCP/IP
426 Database Administration
Teradata System (Plano, TX)
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Mainframe Environment
Mainframe A
(Chicago Data Center)
Network
TCP/IP
ECS FICON Channels
Mainframe B
(Chicago Data Center)
Teradata System (Reno, NV)
Example of a Channel Extension with Teradata ECS
The Teradata Channel Interface hardware and the driver may be located on a TPA node or on the Extended Channel Solution (ECS) node. The ECS node provides the following benefits:
•
Reduces exposure to Teradata outages induced by adapter faults
•
Enables improved mainframe connectivity to a greater number of nodes within the
Teradata Database system
•
Creates more options for users connecting Teradata Database to a mainframe
Current platforms can be connected with the PEFA—PCI Bus FICON Host Channel Adapter.
Database Administration 427
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Mainframe Environment
CP and CUA
Currently, each pair of devices, whatever their implementation, is now referred to as a
Channel Processor (CP). The even and odd address pair is also known as a channel unit address (CUA).
Software Components
The following figure illustrates the software components in mainframe-attached clients that play important roles in getting requests to and from Teradata Database.
Mainframe-Attached
System
Client
Application
CLI
Channel
TDP
428
Parsing
Engine
The following table describes these software components.
1093-003A
Component Description
Client application
•
Written by a developer in your company
•
Written under contract to you by Teradata
•
One of the Teradata Database-provided utilities
Users use these applications to submit SQL statements, maintain files, and generate reports.
For details on supported programming languages, the Teradata Preprocessor2, and embedded SQL requirements, see Teradata Preprocessor2 for Embedded
SQL Programmer Guide.
Database Administration
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Mainframe Environment
Component
CLIv2
TDP
Description
A low-level interface to Teradata Database. It consists of system calls that:
•
Create sessions
•
Allocate request and response buffers
•
Create and deblock parcels of information
•
Fetch response information for the requesting client.
For more information, see Teradata Call-Level Interface Version 2 Reference for
Mainframe-Attached Systems or Teradata Call-Level Interface Version 2 Reference
for Workstation-Attached Systems.
Manages communication between mainframe clients and Teradata Database.
Functions include:
•
Session initiation and termination
•
Logging, verification, recovery, and restart of client applications
•
Physical input to or output from PE vprocs
•
Security
•
Session balancing across multiple PEs
•
Control of the default client character set to be used during sessions originating through this TDP (unless overridden by the user with a
BTEQ.SET SESSION CHARSET command)
See
“TDP Functionality” on page 430 and Teradata Director Program Reference.
Mainframe Sessions
A session number uniquely identifies the work stream of a session for a given TDP. A logical client number uniquely identifies each TDP within an z/OS client or multiple clients. A session number and a logical client number identify each session to the z/OS client.
You can request DBCTIME timestamps to record when the:
•
TDP receives the request.
•
Request was queued to the server.
•
Server received the response.
•
Response was queued to the cross memory task.
•
Response was returned to the input (response) buffer of the user.
Session Pools
A session pool is a number of sessions that are logged onto the Teradata Database server as a unit, using the same logon string via a TDP START POOL command.
Note: Session pools can be created only by TDP trusted z/OS system operators and authorized
Time Sharing Option (TSO) users.
Unlike ordinary sessions, pool sessions are automatically assigned to applications that initiate a logon using the same logon string as that established for the pool. Every session in a pool is assigned to a specific PE and stays with that PE until the session ends.
Database Administration 429
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
TDP Functionality
When the pool is established, all sessions are not in use. When an application sends a logon request whose string matches that of the session pool, the application is assigned an available session from the pool.
That session is marked in-use and cannot be reassigned to another application until the current application logs off. Sessions pools remain logged on until you log them off with the
STOP POOL or LOGOFF POOL commands.
For more information on session pools and TDP commands, see Teradata Director Program
Reference.
TDP Functionality
All messages that a mainframe client sends or receives to Teradata Database normally pass through the TDP. In addition to session, packet, and security control, mainframe TDPs are responsible for:
•
Balancing sessions across assigned parsing engines
•
Routing responses back to the originating address space
Also, you can request DBCTIME time stamps to record when:
•
The TDP receives the request.
•
The request was queued to the server.
•
The server received the response.
•
The response was queued to the cross-memory task.
•
The response was returned to the input (response) buffer of the user.
TDP Exits
An exit is a point at which a user request temporarily leaves the existing code to perform a user-specified task before continuing on with normal processing. You can define routines to perform some function or alteration of normal processing.
You can customize the TDP to perform a user-defined exit routine. Customizing the TDP can assist you in collecting information for performance or functional analysis.
The TDP User Transaction Collection Exit (TDPUTCE) is a routine that allows you to collect statistics about all of the requests and responses that pass through the TDP. TDPUTCE is an exit taken from the Transaction Monitor.
Memory Management
To provide for memory acquisition during system operation without incurring the high overhead associated with the operating system memory services, the TDP acquires units of main memory, or cells, from its own more efficient memory management.
During startup, the memory manager pre-allocates a number of cells in sizes that are convenient for use by the TDP.
430 Database Administration
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Network Environment
The sizes of the cells are internal constants. The initial number of cells is an internal default.
If a TDP subtask requests a cell from the memory manager, but other TDP subtasks are using all available cells, the memory manager takes one of the following actions:
•
Obtains a new cell from the operating system
•
Places the requesting subtask into a wait for memory state.
If the requester is placed into a wait state, the wait ends when another TDP subtask releases a cell. The decision to obtain a new cell or wait for an existing cell is based on TDP considerations.
The TDP typically uses a virtual region of about 4 to 5 MB. To avoid overhead calls to the operating system, the TDP divides its work areas in cells. A warning message (TDP0021) displays when 80% of the cells of a certain size are in use.
Also see
“Performance and Memory Management” on page 535
.
Using TDP Commands
Commands you enter from the console are not executed until you execute the RUN command. The TDP accepts operator commands from:
•
The z/OS console
•
z/OS and TSO users
•
Client interface program applications (such as CLIv2 and the Teradata ODBC and JDBC
Drivers)
Messages that result from executing operator commands entered from a console are returned to the console.
For detailed information, see Teradata Director Program Reference.
Network Environment
In a network environment, each workstation has a copy of Teradata client software, including the utilities, programs, and drivers needed to access Teradata Database.
Functionality
In a network environment, the client interface product (such as CLIv2 or the Teradata JDBC
Driver) and gateway software in the node handle the functions that TDP handles in a mainframe-attached environment.
Database Administration 431
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
Network Environment
Software Components
This figure illustrates the software components in network-attached clients that play important roles in getting and executing requests to and from Teradata Database.
Network-Attached
System
Client
Application
Client
Interface
Product
TCP/IP
Network
Gateway
Component
Client application
Teradata CLI
Database connectivity drivers
Parsing
Engine
1093-004A
The following table describes these software components.
Description
Either:
•
Written by a developer in your company
•
Written under contract to you by Teradata
•
One of the Teradata Database-provided utilities
Users use these applications to submit SQL statements, maintain files, and generate reports.
A low-level interface to Teradata Database, CLI consists of routines that perform functions similar to CLI system calls on mainframeattached clients, including:
•
Logging sessions on and off
•
Submitting SQL queries
•
Receiving responses with the answer set
A variety of drivers for open connectivity products, such as ODBC and JDBC.
References
Teradata Preprocessor2 for Embedded
SQL Programmer Guide
Teradata Call-Level Interface Version 2
Reference for Workstation-Attached
Systems
Teradata JDBC Driver User Guide
ODBC Driver for Teradata User Guide
432 Database Administration
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
The Teradata Gateway
The Teradata Gateway
The gateway software is the interface between the network and Teradata Database. It runs on the Teradata Database server as a separate operating system task. Client sessions that communicate through the gateway to Teradata Database are installed and running on workstations.
In contrast, sessions originating from a mainframe-attached system access Teradata Database through mainframe connections and TDP software and bypass the gateway completely.
The following table describes tools available to administer your gateway.
Tool
Gateway Control utility
Gateway Global utility
Data Dictionary table
Name
gtwcontrol gtwglobal
DBC.EventLog
Function
Recommended for use only by Teradata field support engineers to.
•
Determine the format of some user names
•
Configure security features.
•
Control gateway resource usage.
•
Investigate gateway problems
For details, see Security Administration.
•
Monitor network sessions and traffic
•
Disable logons
•
Force off a rogue session
•
Investigate gateway problems
The table logs entries for logon and logoff events found by session control, including logon failures detected external to session control (such as when the Gateway receives an authentication failure).
Displaying Network and Session Information
Using the Gateway Global utility, you can find the information listed in the following table.
Command
DISPLAY NETWORK
DISPLAY GTW
DISPLAY SESSION
DISPLAY DISCONNECT
DISPLAY FORCE
Description
Displays your network configuration.
Displays all sessions connected to the gateway.
Displays information about a specific session on the gateway.
Displays a list of sessions that have disconnected and not yet reconnected.
Displays sessions that have been forced off the system by Gateway
Global or a security violation.
Database Administration 433
Chapter 18: Working with System Information and Global Defaults: Operational DBAs
The Teradata Gateway
Controlling Network Sessions
Use the Gateway Global utility to perform the actions described in the following table.
Command
DISABLE LOGONS
ENABLE LOGONS
KILL USER
KILL SESSION
Description
Disable logons to Teradata Database through the gateway. This includes logons through DBC.
Enable logons to Teradata Database via the gateway.
Terminates all sessions of a specific user.
Terminates a specific session. Must know session number.
Controlling Trace Logs
Use the commands listed in the following table to turn tracing on or off or to flush the trace buffers to the Event log.
Command
ENABLE TRACE
DISABLE TRACE
FLUSH TRACE
Description
Records internal gateway events.
Turns off the writing of event log entries.
Directs the gateway to write the contents of its internal trace buffers to the event log file.
For More Information
For information on the Gateway Control or Gateway Global utilities, see Utilities. For information on how to configure your Gateway, see Parallel Upgrade Tool (PUT) Reference.
434 Database Administration
CHAPTER 19
Troubleshooting:
Operational DBAs
Teradata Vital Infrastructure (TVI) (or Customer Care Link (CCL) on older systems), will send alerts to you and to Teradata when there is a problem in the system. However, there are some areas where you must monitor the system. You can use Teradata Viewpoint, or write custom scripts to monitor blocking, locate idling sessions, and determine resource utilization.
This chapter suggests some tools for resolving problems and points to specific references for information on how to use these tools. Some of these tools can help you find and analyze a problem. For information on how to handle system slowdowns or hangs, see
Query Blocks and Delays” on page 501
.
Note: Access to administrator utilities is usually limited to privileges specifically granted in the database. Users of these utilities should log on using their Teradata Database usernames.
Users that are externally authenticated (with directory or Kerberos usernames, for example) may not have access to administrator utilities. For more information about the privileges of externally authenticated users, see Security Administration.
Tools for Troubleshooting and Administrating
The following table lists some common tools you can use to administrate your system. The table also lists tools intended for use by Teradata field engineers or Teradata Database system developers. If the tools are described as for use by Teradata personnel only, do not use them unless instructed by Teradata Customer Support.
For more information on the respective tool, see the chapter or book referenced. For a list of the utilities by suggested function (such as maintenance, troubleshooting, or installation), see the “Alphabetical Listing of Utilities” in Utilities.
What to Troubleshoot or
Administrate Tool and Description
Administration Workstation The AWS console for an MPP installation displays the status of each physical component, including nodes,
DBW consoles, and the BYNET. It provides many of the functions the System Console would for an SMP.
AMPs and AMP Worker
Tasks
The ampload utility reports AMP bottlenecks due to unavailable free AWTs.
Reference
AWS manuals (Visit www.info.teradata.com
and search for the keyword
“AWS”.)
“AMP Load (ampload)” in
Utilities
Database Administration 435
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
AMPs and AMP Worker
Tasks (continued)
Configuration changes crashdumps
Tool and Description
The AWT Monitor (awtmon) utility quickly views how many AMP worker tasks are active (in use) and determines “hot AMPs” so you can troubleshoot performance problems.
awtmon is a front-end tool to the “puma -c” command and prints AWT inuse count information in a summary format.
By default, awtmon displays the AWT in-use count of the local node. With the -s option, print system-wide information and locate hot AMPs on the nodes of the entire system. When hot AMPs are identified, run awtmon on those hot nodes using pcl or psh.
You can also use the ResUsageSawt table to see AMP worker tasks in-use counts by work type and determine when AMPs enter the state of flow control.
To determine which workload is responsible for I/O skew, use the ResUsageSps table and then use DBQL to identify specific queries.
The Parallel Upgrade Tool (PUT) utility reboots nodes, sets up disk arrays, or changes the configuration of your system. PUT can perform configuration tasks such things as moving AMPs and PEs, changing clustering assignments, or partitioning Logical Unit Numbers
(LUNs).
Use the crashdumps analysis tools such as the
Crashdumps Save Program (CSP) for viewing the crashdumps, csppeek for checking the contents of a saved crashdump, and the error logs.
Reference
•
“AWT Monitor
(awtmon)” in Utilities
•
“ResUsageSawt Table” and “ResUsageSps
Table” in Resource Usage
Macros and Tables
Parallel Upgrade Tool
(PUT) Reference
.
436 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Data corruption and integrity
Defining vprocs and hosts
Tool and Description
Checksums check the integrity of Teradata Database disk
I/O operations. A checksum is a numeric value computed from data. For a given set of data, the checksum value is always the same, provided the data is unchanged.
Because calculating checksums requires system resources and may affect system performance, system-wide checksumming is disabled by default on most platforms.
Contact Teradata Customer Support if you suspect disk corruption.
Use the CHECKSUM option of CREATE TABLE, ALTER
TABLE, CREATE JOIN INDEX, and CREATE HASH
INDEX statements to enable checksums for a table, hash index, or join index.
The SCANDISK command of the Ferret utility checks the integrity of the Teradata Database file system. This includes the master index, cylinder index, and data block structures (including structures associated with WAL logs).
Previously halted SCANDISK operations can be restarted using a Perl script.
Note: If further events occur after SCANDISK has been running for some time, restarting SCANDISK where it left off the last time will not find errors in previously scanned data that were caused by the most recent failures. Therefore, be very careful when deciding to restart a pending SCANDISK operation versus starting the entire operation again from the beginning.
The Configuration utility defines AMPs, PEs, and hosts and their interrelationships for Teradata Database. This utility is used in conjunction with the Reconfiguration utility.
The PUT utility also allows you to define and set up vprocs on the system.
Reference
“Checksum Fields” in
Utilities
•
“Ferret Utility (ferret)” in Utilities
•
For Teradata Customer
Support personnel, see
“Filer Utility (filer)” in
Support Utilities
“Configuration Utility
(config)” in Support
Utilities
Parallel Upgrade Tool
(PUT) Reference
Database Administration 437
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Disk space
Down AMP
Tool and Description
The Ferret utility allows field engineers and Teradata support people to monitor and control disk usage.
The commands of this utility allow the system to combine free sectors on a cylinder, reconfigure contents on a disk leaving a specified percentage of space free for future use, report which tables qualify for packing and display disk cylinder utilization and available free cylinders.
Note: This utility is used by Teradata Database engineers to perform routine and special diagnostics.
Update DBC utility recalculates PermSpace, SpoolSpace, and TempSpace for user DBC in DBase table. Then based on DBase values, Update DBC utility recalculates
MaxPermSpace and MaxSpoolSpace in
DBC.DataBaseSpace for all databases.
Use Update DBC only to correct inconsistency in the
DBase or DataBaseSpace tables, which might occur as a result of rare types of system failures.
Update Space recalculates permanent, temporary, or spool spaces for one database or all databases.
The DBC.DiskSpaceV view provides disk space usage information per AMP. This includes permanent and spool data by database or account for each AMP. Use this view to track large spool usage and available PERM space.
If you receive an error that an AMP is down (for example from a MultiLoad or BTEQ job) use Vprocmanager to reboot a fatal AMP or bring a offline down AMP up.
Reference
“Ferret Utility (ferret)” in
Utilities
“Update DBC
(updatedbc)” in Utilities
“Update Space
(updatespace)” in Utilities
•
.
•
Data Dictionary
“Vproc Manager
(vprocmanager)” in
Utilities
438 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Down Table
Global parameters
Gateway settings
Tool and Description
The system will mark a data or index subtable as down if an error occurs in several regions of the file system structures of the table. This error could be due to hardware or bad I/O. To address or fix a down region in a subtable, do one of the following:
•
Use the DELETE ALL/DROP table statement.
•
Rebuild the table using the Table Rebuild utility.
•
Restore the table from backup.
•
Drop and recreate the index.
•
Drop and recreate the fallback subtable.
After repairing a down table, use the ALTER
TABLE…RESET DOWN statement to reset a down table.
In the case of hardware errors, a fallback table can be rebuilt to copy the rows from the fallback subtable, to repair the rows in the down regions of the primary subtable, and vice versa. If the down table is rebuilt or restored from an earlier back up, the down status flag for the various subtables will be reset, and the rows in them will also be moved to unmarked data blocks and cylinders from the fallback copies.
To change the number of regions per AMP that must fail before a table is marked as down, use the
MaxDownRegions field in DBS Control.
See “Table” in this table for more information.
DBS Control displays and modifies the tunable global parameters in the DBS Control record.
The Gateway Control utility allows you to set Gateway settings such as how to handle encryption, number of session allowed for host group, time out values and more.
You can use options in the Gateway Control utility to select things like connection time out length, external authentication settings, or whether to use the system default or customer-settable defaults after a configuration or addition of new host groups and gateway vprocs, and more.
Note: This utility is used by Teradata Database engineers to perform routine and special diagnostics.
The Gateway Global utility monitors and controls sessions for LAN-connected users. You can monitor traffic, control network sessions, display which users are using which sessions, or abort specific sessions.
Reference
•
“ALTER TABLE” in SQL
Data Definition
Language
•
“MaxDownRegions” in
Utilities
“DBS Control
(dbscontrol)” in Utilities
•
•
“Gateway Control
(gtwcontrol)” in Utilities
•
“Gateway Global
(gtwglobal)” in Utilities
Database Administration 439
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Hangs or slowdowns
Loads into Large Tables
Tool and Description
See the topics “Sessions that are hung”, “AMPs and AMP
Worker Tasks”, and “Resource usage” in this table.
Resource Check Tools detect hangs or slowdowns as follows:
•
mboxchk: Monitors Teradata Database response time. If the response time indicates the database is slow or unresponsive, mboxchk runs a Teradata
Support Center script to gather system information, logs an event, and contacts Teradata Vital
Infrastructure (TVI).
•
nodecheck: Displays local, node-level resources only.
Provides summary data to syscheck for analysis.
•
syscheck: Monitors the system for signs of congestion that might lead to system slowdowns or perceived hangs and notifies you when Warning or Alert thresholds are reached.
A text file called syscheckrc resides on each node. You can set it up to detect and report any resource that falls below a threshold.
The resource check tools are located in the /usr/pde/ bin directory.
To improve the speed of a batch INSERT/SELECT request into a large table, partition the table by transaction date, which groups the inserts into data blocks within the partition instead of throughout the table. This reduces the number of I/Os.
You can also use the DBQL utility job log table,
DBC.DBQLUtilityTbl, to troubleshoot utility job performance.
Reference
mboxchk, nodecheck, syscheck in man pages or pdehelp
•
•
SQL Data Definition
Language Syntax and
Examples
•
Performance Analysis and Capacity Planning” on page 149
Lock contentions or held locks
There are several things that might cause lock contention: a pending lock, a very congested system, a number of transactions exceeding the machine capacity, or a conflict with ARC.
Careful planning of what sessions run in which order or at what time of day can help prevent lock contention.
Use the EXPLAIN. Consider running a request that needs an EXCLUSIVE lock and will take a while to process during off hours.
The Lock Display utility shows all real-time locks in use.
This includes locks for concurrency control. You can identify which transactions are blocked and which transactions are doing the blocking.
“About Lock Contentions” on page 250
“Lock Display (lokdisp)” in
Utilities
440 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate Tool and Description
You may have locking issues on DBC.AccessRights if you are running concurrent jobs with the same userid in the same database and dropping or creating objects.
To reduce locking issues on DBC.AccessRights:
•
Run the jobs with a few different userids and distribute them randomly among the connection strings to reduce the occurrence of two userids running concurrently.
•
Determine if housekeeping needs to be performed on
DBC.AccessRights for the UserID/DatabaseID involved.
•
Determine if statistics need to be collected on the
Data Dictionary tables.
The DBQL Lock Log logs in XML format in
DBQLXMLLOCKTbl any lock contention longer than a user-specified threshold.
You can access the DBQL lock log through the
Viewpoint lock viewer portlet or query the system table
DBC.DBQLXMLLOCKTbl or the view
DBC.QrylockLogXMLV.
Showlocks utility identifies and displays all active host utility (HUT) locks placed on databases and tables by
ARC operations.
HUT locks may interfere with application processing and are normally released after the utility process is complete.
If locks interfere with application processing, you can remove them by invoking the RELEASE LOCK statement. It is available through the ARC utility or as an
SQL statement.
Showlocks can be started from DB Window Supervisor screen or the HUTCNS console.
•
Supervisor: start showlocks
•
HUTCNS: LOCksdisplay (entering LOC is all that is needed)
Reference
Teradata knowledge articles on reducing locking issues with specific applications
.
•
Teradata Archive/
Recovery Utility
Reference
•
“Show Locks
(showlocks)” in Utilities
Database Administration 441
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Memory
(insufficient memory resulting in Error 3710)
Priority of jobs
Queries
Resource usage
Tool and Description
Teradata recommends that you use the default value set for the MaxParseTreeSegs field in DBS Control. If you run into this error, first try simplifying your queries and/ or dropping unnecessary join indexes. If this still does not work, verify the value you have set for
MaxParseTreeSegs and contact Teradata Customer
Support.
Note: The optimal value depends on your system and what release and fix level you are running.
Increasing MaxParseTreeSegs to a value larger than the default may allow a previously failed 3710 error to run but could result in other problems such as a long parsing time or in extreme cases, a Teradata reset.
Priority Scheduler assigns different priorities to different types of jobs. Higher priority work receives access to
CPU and I/O more frequently.
Use the ResUsageSps table to understand how CPU is being consumed by different workloads and, on SLES 10, to validate the relative weights given to workloads. You can also use this table to understand the pattern of database activity across the processing day.
Use BEGIN/REPLACE QUERY LOGGING WITH
XMLPLAN or use the EXPLAIN IN XML request modifier.
Alternately, use the STEPINFO option of BEGIN/
REPLACE QUERY LOGGING to determine which step uses the most resources and if there are any large differences between estimated and actual CPU, I/O, and number of rows.
Manage your queries using Teradata Viewpoint. You can:
•
Filter queries against object-access or query-resource rules.
•
Control the flow of queries coming into the system for concurrency reasons and delay them if necessary.
•
Log any system exception actions.
•
Determine in which Performance Group queries should run (in SLES 10).
Using resource usage macros and tables to obtain information about which nodes or which vprocs are heavily using which resources. This can help you identify bottlenecks.
The DBQL view DBC.QryLogV reports things such as the AMP using the most CPU, the AMP with the most I/
O, or maximum amount of spool used when processing a query.
Reference
“MaxParseTreeSegs” in
Utilities
•
“Priority Scheduler
(schmon)” in Utilities
•
“ResUsageSps Table” in
Resource Usage Macros and Tables
Resource Usage Macros and
Tables
442 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Sessions that are hung
Session information when logged onto multiple sessions
Tool and Description
Use Query Session utility to determine the state of each session and then use Lock Display utility to find lock contentions.
To find the Thread ID or Job ID of the session from which you are querying, first submit a SELECT
SESSION;
Then after that query returns the session number n, use that number in the following query: sel logonsource from sessioninfovx where sessionno = n;
*** Query completed. One row found. One column
returned.
*** Total elapsed time was 1 second.
LogonSource
-----------------------------------------------
--------
(TCP/IP) 09AB 127.0.0.1 DBC
01 LSS
8804 USER BTEQ
Reference
“Troubleshooting Problems with Sessions” on page 157
“SessionInfo[V][X]” in
Data Dictionary
This helps you determine which Client process/Thread
ID belongs to which session. This is especially useful if you want to find out which Thread ID belongs to which session because you are logged on to multiple sessions.
Query Session utility displays the state of load utility and query sessions. Details can include statuses such as
Parsing, Active, Blocked, Response, whether stored procedures are being processed, and so on.
“Query Session (qrysessn)” in Utilities
Database Administration 443
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Skew
Slowdowns
Slow queries to DBC tables
Space Exceeded
Tool and Description
There are many ways to detect skew:
•
Use Teradata Viewpoint to determine which user is using the most CPU. In general, a bad query can be detected by finding users with high CPU relative to I/
O access.
•
Look at the EXPLAIN text to determine the number of steps used for the query and the type of operation.
You may find something like a product join in a poorly written query. You can abort the query and investigate further or fix the query.
•
To identify node-level skew, use the ResUsageSpma table, comparing maximum CPU usage to the average
CPU consumed across all nodes. If parallel efficiency as shown in the ResUsage Spma table is not approaching 100%, look for the reasons for the imbalance.
•
To find AMP-level skew, there are several options. Use the ResCPUByAMP macro to find AMP-level CPU skew. You can look at the ResUsageSpvr table and compare resource usage across different vprocs. You can also look at the number of requests field in the
ResUsageSps table to see if workloads on one node are actively supporting more requests than on other nodes, then drill down to AMPs. Also using the
ResUsageSps table, you can examine queue wait and service time numbers to find backed up queries by workload or allocation group.
•
To find skew at the request level, you can compare
DBQLogTbl fields MaxAMPCPUTime with
MinAMPCPU. Because user logging is often incomplete with DBQL, ResUsage tables and macros may provide more accurate information.
•
Another option is to use PM/API to develop your own application to monitor for high CPU consumers.
See the topic “Hangs or slowdowns” in this table.
If your site uses a lot of custom applications or tools that often query the DBC tables, consider collecting statistics on these tables
If a space-intensive operation, such as a load utility or restoring from an archive, exceeds available perm space, two DBS Control utility fields can be set temporarily to allow operations to complete and avoid out of space errors.
The settings apply to all operations and all instances of space accounting, such as table inserts, so Teradata recommends setting these fields to a non-zero value only temporarily.
Reference
•
“EXPLAIN Request
Modifier” in SQL Data
Manipulation Language
•
“ResUsageSps Table,”
“ResUsageSpma Table,”
“ResUsageSvpr table,” and “ResCPUByAMP
Macros” in Resource
Usage Macros and Tables
Data Dictionary Tables” on page 202
OverflowMaxPermSpace-
Percentage and
OverflowMaxPermSpace-
KByteCount fields of the
DBS Control utility in
Utilities.
444 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Spool space depletion
Tool and Description
There are several reasons that could contribute to limited spool usage:
•
Poorly written queries can sometimes consume excessive spool space. Use Teradata Viewpoint to check queries before allowing them to run. You can check the EXPLAIN to see how much spool is used and prevent the system from getting overloaded with bad queries.
•
Outdated statistics can cause bad queries. The
Optimizer may calculate query plans differently than it ought if statistics it uses does not accurately reflect the system. The Optimizer also determines how much spool it needs or does not need based on collected statistics. Refresh collected statistics regularly.
•
If you are using OLAP statistical functions and you run out of spool, check the syntax of your queries.
For example, when using the PARTITION BY clause, if a large number of identical values in the partitioning column hash to the same AMP, this can result in out-of-spool errors. Choose a column that results in rows being distributed over a broader set of
AMPs.
•
Poorly skewed data. Poor data distribution means one AMP is responsible for a larger proportion of the data. Because the spool space is divided by the number of AMPs, reaching the maximum spool space limit on one AMP means your query will fail even despite the other unused spool space on the other
AMPs.
•
Poorly chosen PI that leads to AMP skew. Because the spool space is divided by the number of AMPs, reaching the maximum spool space limit on one
AMP means your query will fail even despite the other unused spool space on the other AMPs.
•
Adding nodes or AMPs without reassigning spool space means spreading out the spool amongst more
AMPs.
•
The limit for spool space as defined in the profile overrides the user settings. Check the limit for the profile to see if it is lower than the limit defined for the user. You can compare the values of
MaxProfileSpool and MaxSpool in DBC.DiskSpace.V
Limiting spool space for new users helps reduce the impact of possibly bad queries (such as product joins).
With a reasonable limit, the user will get out-of-spool messages before a bad query ties up the system.
Furthermore, adding spool to a bad query will allow the query to run longer but a bad query may still not complete.
Reference
•
•
“EXPLAIN Request
Modifier” in SQL Data
Manipulation Language
•
Teradata Viewpoint
•
“Ordered Analytical
Functions” in SQL
Functions, Operators,
Expressions, and
Predicates
Database Administration 445
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Space allocated to storage
System crashes or failures
System recovery
Table
Tool and Description
Use the Teradata Virtual Allocator Manager (TVAM) utility.
Allows you to display mapping or force migration of cylinders on the system.
Note: Do not use the TVAM utility unless instructed by
Teradata Customer Support.
Screen Debug and Screen Dump controls how and what your system records for crashdumps. Teradata recommends that default settings for all dumps be changed only when requested by a system support representative or Teradata Customer Support.
DBC.Software_Event_LogV is a system view that provides detailed information about errors or system failures and their related node, vproc, partition, task, function, software version, optional backtrace, diagnostic information, and so on.
The Event_Tag field of the view reports the error message number.
DUL utility saves or restores system dump tables onto tape.
Reference
Man page or online help
“Control GDO Editor (ctl)” in Utilities
•
•
Data Dictionary
Messages
Recovery Manager utility monitors and reports the progress of a system recovery after a crash or a user abort. You can use this utility to monitor transaction rollbacks, cancel rollbacks, and more.
The Table Rebuild utility rebuilds tables Teradata
Database cannot recover automatically. Table Rebuild can rebuild the following for an AMP:
•
The primary or fallback portion of a table
•
An entire table (both primary and fallback portions)
•
A particular range of rows of a subtable
•
The corrupted regions of a table
•
A corrupted or missing table header
•
All tables
•
All fallback tables
•
All tables in a database
•
Single tables sequentially or tables of each database in parallel.
“Dump Unload/Load
Utility (dul, dultape)” in
Utilities
“Recovery Manager
(rcvmanager)” in Utilities
•
“Table Rebuild
(rebuild)” in Utilities
•
See also “Table inconsistencies or corruption”
446 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Table inconsistencies or corruption
Tool and Description
CheckTable is a diagnostic tool that checks for:
•
Table and dictionary inconsistencies, such as differing table versions, ParentCount and ChildCount data, and partitioning definitions.
•
Table corruption, such as duplicate rows or unique values and data inconsistency, of primary and fallback data and stored procedure tables.
•
Inconsistencies in internal data structures such as table headers, row identifiers, secondary indexes, and reference indexes.
•
Invalid row hash values and partition numbers.
Options for CheckTable include the following:
•
Four levels of data integrity checking (PENDINGOP,
ONE, TWO, and THREE). Each successive level performs additional checks that are more thorough and consequently more time consuming.
•
The DOWN ONLY option displays status information for down tables only.
Note: The DOWN ONLY option works for all levels except PENDINGOP.
•
The ERROR ONLY option causes CheckTable to display only skipped tables, down tables, and tables with errors and warnings. This allows you to quickly identify and address problems found by CheckTable.
•
The OUTPUT command allows you to redirect
CheckTable output to a log file. The ECHO option for this command allows you to direct output to both the console and a log file.
CHECKTABLEB is a non-interactive batch mode version of CheckTable that can be run through the cnsrun utility.
CHECKTABLEB is identical to CheckTable except that it is meant to run from a script.
The Table Rebuild utility allows you to fix a primary or fallback copy of a table or only specific corrupted parts.
When the rebuild operation successfully completes on an
AMP, TableRebuild will remove the down region information from the table header. If, however, the fallback subtable has any down regions defined on it, the rebuild process aborts and the system returns an error.
You cannot rebuild a table with parts of its fallback down. While you can rebuild stored procedures, UDFs, and UDMS, you cannot rebuild a join index or hash index defined on a table being rebuilt.
Reference
“CheckTable (checktable)” in Utilities
•
“Table Rebuild
(rebuild)” in Utilities
•
See also “Table inconsistencies or corruption”
Database Administration 447
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
What to Troubleshoot or
Administrate
Vprocs
Vprocs and storage
Tool and Description
The Vproc Manager utility allows you to:
•
View or change vproc states
•
Initialize and boot a specific vproc
•
Initialize the storage associated with a specific vproc
•
Force a manual Teradata Database restart
A few of the vprocs statuses include:
•
FATAL: the vproc or its associated storage is not operational. The system marks it fatal because of repeated crashes or if data integrity is in danger.
Multiple crashes are indicated by the crash count and indicate a repeatable condition of data corruption, bad SQL or a software bug. Contact Teradata
Customer Support.
•
FATAL** : this is an “fsgwizard” data integrity situation. Contact Teradata Customer Support.
•
OFFLINE: the vproc is operational, but either set offline purposely by an operator or if, for example, all
AMPs in a clique are OFFLINE, they were probably set OFFLINE by the system because none of the nodes in its clique were up. This usually occurs during a system-wide reboot of all nodes, but nodes come up at different times.
Note: A local crash of an OFFLINE vproc will cause the database to restart.
•
UTILITY Catchup (offline catchup): the vproc was previously down (OFFLINE or FATAL or some other state). If its fallback data was written to another vproc in its cluster, the vproc is now back up (but not online) while it catches up its fallback data.
From the Supervisor screen of the Database Window, enter GET CONFIG to view the current system configuration.
To administrate vprocs and disks, use the Vproc
Manager utility.
The Query Configuration utility reports the status of vprocs managed by node or for the entire Teradata
Database system.
Reference
“Vproc Manager
(vprocmanager)” in
Utilities
•
“Database Window
(xdbw)” in Utilities
•
“Vproc Manager
(vprocmanager)” in
Utilities
“Query Configuration
(qryconfig)” in Utilities
448 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Tools for Troubleshooting Client Connections
What to Troubleshoot or
Administrate
Vproc Manager
(vprocmanager) utility
Tool and Description
If vprocmanager exits with an error...
# vprocmanager
PDE is not running: Operation not permitted
PDE event 10117 reported when task is not attached to PDE or PDE is down. Task should stop on this error.
then you are probably attempting to run it from a non-
TPA node. Run vprocmanager from a TPA node (a node running Teradata Database software).
Reference
“Vproc Manager
(vprocmanager)” in
Utilities
Tools for Troubleshooting Client Connections
Mainframe Connection Tools
The following tools monitor session and TDP activity on mainframe-connected clients.
•
HSI timestamp - Host System Interface (HSI) timestamps tell you when TDP receives a request, when the request parcel is sent to or queued for Teradata Database, and when the response parcel is received from Teradata Database.
•
TDPUTCE - TDP User Transaction Collection Exit (TDPUTCE) collects statistics about all of the requests and responses controlled by the TDP, including user, session/request parcels, timestamps, request type, and request/response parcels.
•
Your site is responsible for processing and analyzing the data collected by TDPUTCE.
•
z/OS System Management Facility (SMF) is a feature in the z/OS operating system that can record accounting and performance information, such as:
•
Statistical information about the processing activity of a PE recorded at shutdown.
•
Log-off session information.
•
Logon violations and security violations records.
•
Statistical data about the processing activity of the TDP, recorded at shutdown.
•
Statistical information about the processing activity of a Channel Processor (CP) either recorded at shutdown or started with the Channel Communication Unit (CCU) operand.
For more information on TDP, see Teradata Director Program Reference.
Database Administration 449
Chapter 19: Troubleshooting: Operational DBAs
Troubleshooting Spool Space Problems
Network Connection Tools
The following tools monitor and control sessions originating from workstation-attached clients.
Tool
Gateway Control
(gtwcontrol)
Teradata Network
Statistics (tdnstat)
Description
Utility that allows you to monitor network and session information. You can do the following:
•
get network configuration information
•
see all sessions connected via the gateway
•
see status information for a selected gateway session
•
force off one or more network sessions
Utility that gives you a snapshot, or a snapshot differences summary, of statistics specific to
Teradata Network Services. You also can clear the current network statistics.
Reference
•
•
Utilities
Man or pdehelp pages
Troubleshooting Spool Space Problems
The system automatically creates and drops spool files which it uses as temporary work tables to execute queries. Sometimes, one of two potential problems can occur with the spool files:
•
Leftover spool. Leftover spool occurs when the system fails to properly drop the spool file after the execution of a query completes. The spool that the query was using appears frozen without a session. This may result in error 2667 which aborts the transaction. Note that with error 2667, the system terminates the query not because the query was faulty, but as a result of the leftover spool problem.
•
“Phantom” spool. Phantom spool occurs when the DBC.DatabaseSpace table is not properly updated to reflect current and actual space usage.
To check for either types of spool problems, run the following query when the system is most quiescent and when there are the least amount of users logged on:
SELECT DATABASENAME, VPROC, CURRENTSPOOL
FROM DBC.DISKSPACE
WHERE DATABASENAME NOT IN (SEL USERNAME FROM DBC.SESSIONINFO)
AND CURRENTSPOOL > 0
ORDER BY 1,2
WITH SUM(currentspool);
Note: If the user whose query unintentionally caused the spool space problem is logged on at the time you run this query, the query will not detect the user because the query is looking for spool usage for a user that is holding spool space but is not logged on.
If the query returns rows, run the Update Space (updatespace) utility to update the spool space accounting in DBC.DatabaseSpace for each of the databases.
450 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Adjusting for Low Available Free Memory
Using the Update Space Utility
The Update Space utility (updatespace) recalculates the permanent, temporary, or spool space used by either of the following:
•
A single database and its individual tables
•
All databases in a system and their individual tables
The Update Space utility accomplishes this by:
•
Examining storage descriptors and adding up space for each table.
•
Setting values in CurrentPermSpace, CurrentTempSpace, or CurrentSpoolSpace in the
DBC.DatabaseSpace table for each table and for the containing database as a whole.
A different utility, Update DBC (updatedbc), recalculates the maximum allowed values for permanent, temporary, and spool space in the DBC.Dbase and DBC.DatabaseSpace tables.
The following are the difference between the Update DBC and Update Space utilities:
•
Update DBC recalculates maximum allowed values for permanent, temporary, and spool space.
•
Update Space recalculates current usage for permanent, temporary, and spool space.
The only reason to use Update Space is to correct inconsistencies in the DBC.DatabaseSpace table, which might occur as the result of rare types of system failures.
The format of the command to use with the “Update Space” utility is:
If the problem is fixed using Update Space, then the problem was phantom spool. If running
Update Space does not fix the problem, you have leftover spool and the only way to fix the problem is to restart the system.
For more details on…
command syntax for the Update Space utility the spool space problems
DBC.DatabaseSpace table
See…
“Update Space (updatespace)” in Utilities.
Knowledge Article SD1000C126E.
Data Dictionary.
For assistance with spool space problems, contact Teradata Customer Support.
Adjusting for Low Available Free Memory
When the amount of available free memory dips too far below 100 MB (25,000 pages), some sites have experienced issues. 40 MB of free memory is the lowest acceptable amount. You can
Database Administration 451
Chapter 19: Troubleshooting: Operational DBAs
Solving Partitioning and RI Validation Errors usually avoid problems if you configure your AMPs to have at least 135 MB of OS-managed memory per AMP. You can adjust the amount of available free memory by performing the following:
•
Use ctl to adjust the FSG Cache percent downward to make more memory available to free memory. If the system takes too much memory for FSG Cache and the OS does not use that memory, the free memory is wasted.
•
If available free memory goes below 100 MB during heavy periods of redistribution (as explained later in this section), lower the value of the RedistBufSize field in the DBS
Control Record (see “RedistBufSize” in Utilities).
Solving Partitioning and RI Validation Errors
You can use the following procedures and tools to detect and correct errors in tables that use partitioning and referencing. (For details, see “Partitioned and Nonpartitioned Primary
Indexes” and “FOREIGN KEY … REFERENCES Constraints” in Database Design.)
IF you want to …
correct a partitioning expression that is causing transaction rollbacks due to an evaluation error (such as divide by zero)
THEN …
do one of the following:
•
Change the partitioning expression
•
Delete the rows causing the problem
•
Remove partitioning from the table
•
Drop the table run the CheckTable utility LEVEL 3 command.
find invalid table states or internal structures regenerate only the headers in a table with partitioning validate a column-partitioned table or join index for a table with partitioning:
•
Regenerate table headers
•
Re-evaluate partition expressions
•
Recalculate row hash values
•
Move rows to proper AMPs and row partitions
•
Update any SI, JI, and HI defined for the table reset the state of RI tables after an
ARC RESTORE operation use the ALTER TABLE … REVALIDATE statement.
use the REVALIDATE option of the ALTER TABLE request. use the ALTER TABLE … REVALIDATE WITH DELETE/
INSERT[INTO] statement.
•
WITH DELETE deletes any rows with a partition number that is null or outside the valid range.
•
WITH INSERT [INTO] deletes any rows with a partition number that is null or outside the valid range and inserts them into save_table.
Note: REVALIDATE changes the table version.
run the ARC utility REVALIDATE REFERENCES FOR command. See Teradata Archive/Recovery Utility Reference for details.
452 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Incident Information Checklist
IF you want to …
find corrupt rows after running an update or delete operation using
WITH NO CHECK OPTION on tables with RI constraints purify a Child table for which corrupt rows were reported by the RI
Validation Query
THEN …
submit the RI Validation Query, structured as:
SELECT DISTINCT childtablename.*
FROM childtablename,parenttablename WHERE childtablename.fkcol NOT IN (SELECT pkcol
FROM parenttablename)
AND childtablename.fkcol IS NOT NULL;
This query reports every row in the Child table with an FK value that does not have a matching PK value. (FK nulls are excluded because it is not possible to determine the values they represent.) delete from the Child table any reported rows as soon as possible to maintain the integrity of your database.
Incident Information Checklist
If you notice your system is having problems and you want to contact Teradata Customer
Support to report an incident, collect as much information about your system as possible. The list below is not exhaustive and is meant only to help you begin gathering details.
Note that not all of the questions may be applicable to your system and that other aspects of performance problems may not be addressed below. It is important to provide any other information that is not listed below but that you feel is important.
1
2
Describe any performance problems and its impact.
•
Is the system hung?
•
Is there a general system-wide slowdown?
•
Is there a slow query or class of queries?
•
Has there been a gradual deterioration?
•
Are there any time-outs?
Example: The entire Teradata Database system runs fine for a while, then performance gradually deteriorates over a span of a few days. Marketing XYZ queries time out more often until eventually the entire system hangs. A Teradata reset clears the problem.
Is performance degraded for the entire system or part of the system?
•
A particular node(s) or clique(s)?
•
A particular subsystem like BYNET or disk array?
•
A particular application or client?
•
The network?
•
A particular group or class of users?
•
A particular job or query?
•
Do you have a performance baseline to compare against?
Database Administration 453
Chapter 19: Troubleshooting: Operational DBAs
Incident Information Checklist
3
4
5
6
Example: The Marketing XYZ queries, when run from a gateway connected session via a
Web interface, had degraded response time from 10 minutes to approximately 30 minutes.
When does the problem occur?
•
All the time?
•
A particular time of day, day of week, week of month, month of year?
•
When did this problem start? Did it start suddenly?
•
Is it getting progressively worse?
•
Is the performance problem intermittent and unpredictable?
•
Has this problem happened before?
•
Are there any other past incidents that might be relevant to the current performance situation at this site?
•
Is the problem reproducible? If so, how?
•
What else is running while this problem occurs?
Other than performance, do you observe any other symptoms? For example:
•
Any unusual errors, warnings or messages, especially lately?
•
MiniCylPacks?
•
File space problems?
•
Memory depletion?
•
Lock contention?
•
Network congestion, collisions, bottlenecks?
•
z/OS errors, cell problems, in particular?
•
Are users having problems logging into the system and the database?
•
Are any third party incidents currently open?
What recent changes have been made to the system?
•
Recent configuration changes?
•
New hardware or new software?
•
New applications or new queries?
•
Different job mix and scheduling?
•
Have you just newly switched to a production environment?
•
Increased or different workload?
•
New or additional users?
•
New or additional data?
•
System tuning changes?
•
Priority Scheduler configuration?
Describe the data processing environment and where Teradata fits into the context.
•
What is your system generally used for? (Decision Support, OLTP, OLAP, Ad-Hoc
Queries, a mix?)
•
What other vendor applications are you using?
454 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Monitoring Transaction Recovery
7
8
•
Which applications are your most critical applications?
•
Who are the users? Are there different groups of users running different applications?
•
What types of data and how often is data loaded?
•
Can you provide a high-level block diagram illustrating the Teradata Database system, interfaces, networks, hosts, data marts, intermediate Servers, clients, sessions, users,
ODBC, and so on?
Describe the Teradata Database System Configuration.
•
What are the versions of DBS, PDE, TCHN, TGTW, TDGSS, BYNET, and storage devices?
•
What is the number of nodes, CPU speed, AMPs per node, memory size per node, and storage per AMP?
•
Has the Priority Scheduler been set up differently than the default settings?
•
Are there any unusual or asymmetrical aspects to the system configuration, such as different node types, different CPU speeds, different size cliques, different numbers of
VPROCs per node, different storage subsystem, BYNET or other I/O components?
What supporting evidence and data do you have to assist Teradata Customer Support in troubleshooting?
•
Do you have logs for: resource usage data, access logging, the Viewpoint Lock Viewer portlet, system accounting, syschk and mboxchk, blmstats, tdnstats, event logs and other error logs.
•
Do you have baseline data?
•
If you are trying to troubleshoot a particular query, have you measured performance of the same query from different clients (that is, BTEQ versus network attached client or have you tried to isolate bottlenecks)?
•
Has there been network congestion or collisions?
Monitoring Transaction Recovery
The Rcvmanager (Recovery Manager) utility lets you monitor the backing out of incomplete transactions on tables that may be locked for access. The resulting journal is called the
Transaction Journal. Rcvmanager also shows the count of records of data presently in the
Down AMP Recovery Journal. This journal represents the data rows an AMP vproc must recover from the other AMP vprocs in the cluster before coming back online to the database.
The Recovery Manager utility runs only when the system is in one of the following states:
•
Logon
•
Logon/Quiet
•
Logoff
•
Logoff/Quiet
•
Startup (if system has completed voting for transaction recovery)
Database Administration 455
Chapter 19: Troubleshooting: Operational DBAs
Monitoring Transaction Recovery
If the system is not in one of the above states, Recovery Manager will terminate immediately after you start it.
Starting Recovery Manager
To start Recovery Manager, enter start rcvmanager in the Supervisor interactive area.
Stopping Recovery Manager
After you start Recovery Manager, you cannot stop it with the Supervisor program STOP command. You must use the Rcvmanager QUIT; command to stop the program.
Note: All Rcvmanager commands end with a semicolon (;).
Recovery Manager Commands
The LIST STATUS command displays information about recovery operations in progress. The processor id option provides additional detailed information about a specific down AMP.
The LIST LOCKS command displays a list of all locks currently held by online transaction recovery.
The LIST ROLLBACK TABLES command displays the list of tables which are currently being rolled back in the system. Separate listings are generated to distinguish between online transaction recovery and system recovery. Table ids can be selected from this listing for executing the CANCEL ROLLBACK ON TABLE command. In case a '*' character follows the table names then they cannot be specified in the CANCEL ROLLBACK ON TABLE command.
The LIST CANCEL ROLLBACK TABLES command displays the list of tables for which rollback processing is pending cancellation during the online transaction recovery. These tables are removed from the list when all the journal rows corresponding to the tables have been skipped on all the AMPs.
The PRIORITY command can be used to either display or set the current priorities for rebuild or recovery. For example:
RECOVERY PRIORITY displays the current recovery priority setting.
RECOVERY PRIORITY LOW sets the recovery priority to Low.
DEFAULT PRIORITY sets the recovery priority to Low and the rebuild priority to Medium.
The CANCEL ROLLBACK ON TABLE command is used to specify a table for which rollback processing is to be cancelled for an online user requested abort or during system recovery. The
DBC password is required to execute this command. Multiple tables can be specified by separating their table ids with commas.
Notice:
The target table will be unusable after this command is issued and will become usable only when the table is dropped and created again, when the table is restored from an archived backup, or when a DELETE ALL operation is performed on that table. The CANCEL
ROLLBACK command should only be used in cases where the rollback will take longer than the restore of the table or where the table is unimportant (such as a temporary table). A single
456 Database Administration
Chapter 19: Troubleshooting: Operational DBAs
Resolving Orphan or Phantom Spool Issues table retrieve operation can be performed on the target table by using the READ OVERRIDE locking modifier on it.
The ROLLBACK SESSION... PERFORMANCE GROUP command can be used to either display or set the current performance group of the rollback for a particular session. The priority associated with the specified performance group is used to change the priority of the rollback for the specified host-id and session number.
Resolving Orphan or Phantom Spool Issues
In rare cases when a request completes (or is aborted), the spool file is not dropped. The leftover spool may be:
•
Orphan: The spool file (tableid) incorrectly remains in existence.
•
Phantom: The spool file tableid is gone, but the spool space is still allocated.
Execute the following query to determine the presence of leftover spool of either type:
SELECT Databasename, Vproc, CurrentSpool
FROM DBC.Diskspace
WHERE Databasename NOT IN (SELECT Username FROM DBC.SessionInfo)
AND CurrentSpool > 0
ORDER BY 1,2
WITH SUM (CurrentSpool);
If this request returns rows, the next step is to run the Update Space (updatespace) utility.
The Update Space utility can also be used to correct inconsistencies in the
DBC.DatabaseSpace table, which might occur because of unusual types of system failures.
From the Supervisor window issue:
START UPDATESPACE
Repairing Data Corruption
Table Rebuild is a utility that repairs data corruption. It does so by rebuilding tables on a specific AMP vproc based on data located on the other AMP vprocs in the fallback cluster.
Table Rebuild can rebuild data in the following subsets:
•
The primary or fallback portion of a table
•
An entire table (both primary and fallback portions)
•
All tables in a database
•
All tables that reside on an AMP vproc
Database Administration 457
Chapter 19: Troubleshooting: Operational DBAs
Repairing Data Corruption
Table Rebuild performs differently based on the type of table (for example, fallback or not).
Type of Table
Fallback Tables
Non-fallback Tables
Permanent Journal Tables
Action
Delete the table header (one row table which defines a user data table)
Delete specified portion of the table being rebuilt
Rebuild the table header
Rebuild the specified portion of the table
Delete the table header.
Rebuild the table header.
Delete data for the table being rebuilt.
Rebuild the table header.
Locks the table pending rebuild.
Note: You must restore non-fallback data.
Note: You can start REBUILD when the failed AMP has been fixed and brought back to the
OFFLINE state.
458 Database Administration
APPENDIX A
How to Read Syntax Diagrams:
All DBAs
This appendix describes the conventions that apply to reading the syntax diagrams used in this book.
Syntax Diagram Conventions
Notation Conventions
Item
Letter
Number
Word
Spaces
Punctuation
Definition / Comments
An uppercase or lowercase alphabetic character ranging from A through Z.
A digit ranging from 0 through 9.
Do not use commas when typing a number with more than 3 digits.
Keywords and variables.
•
UPPERCASE LETTERS represent a keyword.
Syntax diagrams show all keywords for SQL statements in uppercase, unless operating system restrictions require them to be in lowercase.
•
lowercase letters represent a keyword that you must type in lowercase, such as a
Linux command.
•
Mixed Case letters can be used to represent functions, methods, and other non-
SQL keywords that can be entered in uppercase, lowercase or mixed case.
•
lowercase italic letters represent a variable such as a column or table name.
Substitute the variable with a proper value.
•
lowercase bold letters represent an excerpt from the diagram. The excerpt is defined immediately following the diagram that contains it.
•
UNDERLINED LETTERS represent the default value. This applies to both uppercase and lowercase words.
Use one space between items such as keywords or variables.
Type all punctuation exactly as it appears in the diagram.
Paths
The main path along the syntax diagram begins at the left with a keyword, and proceeds, left to right, to the vertical bar, which marks the end of the diagram. Paths that do not have an arrow or a vertical bar only show portions of the syntax.
Database Administration 459
Appendix A: How to Read Syntax Diagrams: All DBAs
Syntax Diagram Conventions
The only part of a path that reads from right to left is a loop.
Continuation Links
Paths that are too long for one line use continuation links. Continuation links are circled letters indicating the beginning and end of a link:
A
A
FE0CA002
When you see a circled letter in a syntax diagram, go to the corresponding circled letter and continue reading.
Required Entries
Required entries appear on the main path:
SHOW
FE0CA003
If you can choose from more than one entry, the choices appear vertically, in a stack. The first entry appears on the main path:
SHOW
CONTROLS
VERSIONS
FE0CA005
Optional Entries
You may choose to include or disregard optional entries. Optional entries appear below the main path:
SHOW
CONTROLS
FE0CA004
460 Database Administration
Appendix A: How to Read Syntax Diagrams: All DBAs
Syntax Diagram Conventions
If you can optionally choose from more than one entry, all the choices appear below the main path:
Strings
READ
SHARE
ACCESS
JC01A010
Some commands and statements treat one of the optional choices as a default value. This value is UNDERLINED. It is presumed to be selected if you type the command or statement without specifying one of the options.
String literals appear in apostrophes:
'msgtext '
JC01A004
Abbreviations
If a keyword or a reserved word has a valid abbreviation, the unabbreviated form always appears on the main path. The shortest valid abbreviation appears beneath.
SHOW
CONTROLS
CONTROL
In the above syntax, the following formats are valid:
•
SHOW CONTROLS
•
SHOW CONTROL
FE0CA042
Loops
Database Administration
A loop is an entry or a group of entries that you can repeat one or more times. Syntax diagrams show loops as a return path above the main path, over the item or items that you can repeat:
(
,
,
cname
4
3
)
JC01B012
461
Appendix A: How to Read Syntax Diagrams: All DBAs
Syntax Diagram Conventions
Read loops from right to left.
The following conventions apply to loops:
IF...
there is a maximum number of entries allowed there is a minimum number of entries required a separator character is required between entries a delimiter character is required around entries
THEN...
the number appears in a circle on the return path.
In the example, you may type cname a maximum of 4 times.
the number appears in a square on the return path.
In the example, you must type at least three groups of column names.
the character appears on the return path.
If the diagram does not show a separator character, use one blank space.
In the example, the separator character is a comma.
the beginning and end characters appear outside the return path.
Generally, a space is not needed between delimiter characters and entries.
In the example, the delimiter characters are the left and right parentheses.
Excerpts
Sometimes a piece of a syntax phrase is too large to fit into the diagram. Such a phrase is indicated by a break in the path, marked by (|) terminators on each side of the break. The name for the excerpted piece appears between the terminators in boldface type.
The boldface excerpt name and the excerpted phrase appears immediately after the main diagram. The excerpted phrase starts and ends with a plain horizontal line:
LOCKING
A
HAVING
con where_cond
excerpt excerpt
A
,
cname
,
col_pos
JC01A014
462 Database Administration
Appendix A: How to Read Syntax Diagrams: All DBAs
Syntax Diagram Conventions
Multiple Legitimate Phrases
In a syntax diagram, it is possible for any number of phrases to be legitimate:
DATABASE
TABLE
VIEW
dbname tname vname
JC01A016
In this example, any of the following phrases are legitimate:
•
dbname
•
DATABASE dbname
•
tname
•
TABLE tname
•
vname
•
VIEW vname
Sample Syntax Diagram
,
CREATE VIEW
CV
A
B
viewname dbname
DATABASE
tname
TABLE
SEL
VIEW
,
expr vname
FROM
cname
FOR
IN
,
tname
AS
LOCKING
LOCK
ACCESS
SHARE
READ
WRITE
EXCLUSIVE
EXCL
MODE
qual_cond
.aname
C
HAVING cond
; qual_cond
WHERE cond
GROUP BY
,
cname
,
col_pos
A
B
C
JC01A018
Database Administration 463
Appendix A: How to Read Syntax Diagrams: All DBAs
Syntax Diagram Conventions
Diagram Identifier
The alphanumeric string that appears in the lower right corner of every diagram is an internal identifier used to catalog the diagram. The text never refers to this string.
464 Database Administration
APPENDIX B
Teradata System Limits: All DBAs
This appendix provides the following Teradata Database limits.
•
System limits
•
Database limits
•
Session limits
•
System-derived and system-generated column data types
The reported limits apply only to the platform software. Platform-dependent client limits are not documented in this appendix.
System Limits
The system specifications in the following tables apply to an entire Teradata Database
configuration.
Miscellaneous System Limits
Parameter
Maximum number of combined databases and users.
Maximum number of database objects per system lifetime.
A database object is any object whose definition is recorded in DBC.TVM.
Because the system does not reuse DBC.TVM IDs, this means that a maximum of
1,073,741,824 such objects can be created over the lifetime of any given system. At the rate of creating one new database object per minute, it would take 2,042 years to use
1,073,741,824 unique IDs.
Maximum size for a table header.
Maximum size of table header cache.
Maximum size of a response spool row.
Maximum number of change logs that can be specified for a system at any point in time.
Maximum number of locks that can be placed on aggregate online archive logging tables, databases, or both per request.
The maximum number of locks that can be placed per LOGGING ONLINE
ARCHIVE ON request is fixed at 25,000 and cannot be altered.
4.2 x 10
9
1,073,741,824
1 MB
8 x 10
6
64KB
bytes
1,000,000
25,000
Value
Database Administration 465
Appendix B: Teradata System Limits: All DBAs
System Limits
Parameter
Maximum number of 64KB parse tree segments allocated for parsing requests.
Maximum number of nodes per system configuration.
Limits on vprocs of each type restrict systems with a large number of nodes to fewer vprocs per node.
Systems with the maximum vprocs per node cannot approach the maximum number of nodes.
12,000
1,024
Message Limits
Value
Parameter
Maximum number of CLIv2 parcels per message
Maximum message size
Value
256
~ 65,000 bytes
This limit applies to messages to and from client systems and to some internal Teradata Database messages.
255 bytes Maximum error message text size in a failure parcel
Storage Limits
Total data capacity
Parameter
Minimum data block size with small cylinders
Value
~ 12 TB/AMP
~9KB
18 512-byte sectors
Default data block size with small cylinders
Maximum data block size with small cylinders
Minimum data block size with large cylinders
~127KB
254 512-byte sectors
~256KB
512 512-byte sectors
~21KB
42 512-byte sectors
Default data block size with large cylinders ~127KB
254 512-byte sectors
Maximum data block size for a large cylinder system that does not use 4KB alignment 1,023.5KB
2,047 sectors
466 Database Administration
Parameter
Maximum data block size for a large cylinder system that uses 4KB alignment
Maximum number of data blocks that can be merged per data block merge
Maximum merge block ratio
Gateway and Vproc Limits
Parameter
Maximum number of sessions per PE.
Maximum number of gateways per node.
Appendix B: Teradata System Limits: All DBAs
System Limits
Value
1,024KB
2,048 sectors
8
100% of the maximum multirow block size for a table.
Maximum number of sessions per gateway.
Maximum number of vprocs per system.
Maximum number of AMP vprocs per system.
Value
120
Multiple.
This is true because the gateway runs in its own vproc on each node. See Utilities for details.
The exact number depends on whether the gateways belong to different host groups and listen on different IP addresses.
Tunable: 1 - 2,147,483,647.
1,200 maximum certified.
The default is 600.
See Utilities for details.
30,720
This includes the sum of all of the following types of vproc for a configuration:
•
AMP
Access Module Processor vprocs
•
GTW
Gateway Control vprocs
•
PE
Parsing Engine vprocs
•
RSG
Relay Services Gateway vprocs
•
TVS
Teradata Virtual Storage allocator vprocs
16,200
Database Administration 467
Appendix B: Teradata System Limits: All DBAs
System Limits
Parameter
Maximum number of GTW vprocs per system.
This is a soft limit that Teradata technical support personnel can reconfigure for you.
Each GTW vproc on a node must be in a different host group. If two GTW vprocs are in the same host group, they must be on different nodes.
The default is one GTW vproc per node with all of them assigned to the same host group.
The maximum depends on:
•
Number of IP addresses assigned to each node.
•
Number of host groups configured in the database.
Each gateway on a node must be assigned to a different host group from any other gateway on the
same node and each gateway needs to be assigned a disjoint set of IP addresses to service.
This does not mean that all gateways in a system must be assigned to a different host group.
It means that each gateway on the
same node must be assigned to a different host group.
Gateways on different nodes can be assigned to the same host group.
2,048
Value
Maximum number of PE vprocs per system.
This is a soft limit that Teradata technical support personnel can reconfigure for you.
Maximum number of TVS allocator vprocs per system.
This is a soft limit that Teradata technical support personnel can reconfigure for you.
Maximum number of vprocs, in any combination, per node.
Maximum number of AMP vprocs per cluster.
Maximum number of external routine protected mode platform tasks per PE or AMP.
This value is derived by subtracting 1 from the maximum total of PE and AMP vprocs per system (because each system must have at least one PE), which is 16,384. This is obviously not a practical configuration.
The valid range is 0 to 20, inclusive. The limit is 20 platform tasks for each platform type, not 20 combined for both. See Utilities for details.
Maximum number of external routine secure mode platform tasks per PE or AMP.
This value is derived by subtracting 1 from the maximum total of PE and AMP vprocs per system (because each system must have at least one PE), which is 16,384. This is obviously not a practical configuration.
Size of a request control block
Default number of lock segments per AMP vproc.
This is controlled by the NumLokSegs parameter in DBS Control.
2,048
127
8
20
20
~ 40 bytes
2
468 Database Administration
Appendix B: Teradata System Limits: All DBAs
System Limits
Parameter
Maximum number of lock segments per AMP vproc.
This is controlled by the NumLokSegs parameter in DBS Control.
Default size of a lock segment.
This is controlled by the LockLogSegmentSize parameter in DBS Control.
Maximum size of a lock segment.
This is controlled by the LockLogSegmentSize parameter in DBS Control.
Default number of locks per AMP
Maximum number of locks per AMP.
Maximum size of the lock table per AMP.
The AMP lock table size is fixed at 2 MB and cannot be altered.
Maximum size of the queue table FIFO runtime cache per PE.
8
64 KB
1 MB
3,200
209.000
2 MB
Value
Maximum number of SELECT AND CONSUME requests that can be in a delayed state per PE.
Amount of private disk swap space required per protected or secure mode server for
C/C++ external routines per PE or AMP vproc.
Amount of private disk swap space required per protected or secure mode server for
Java external routines per node.
•
100 queue table entries
•
1 MB
24
256 KB
30 MB
Hash Bucket Limits
Parameter
Number of hash buckets per system.
This value is user-selectable. See the topic “Hash Maps” in Chapter 8 of Database
Design for details.
Value
The number is user-selectable per system.
The choices are:
•
65,536
•
1,048,576
Bucket numbers range from 0 to the system maximum.
Database Administration 469
Appendix B: Teradata System Limits: All DBAs
System Limits
Size of a hash bucket
Parameter Value
The size depends on the number of hash buckets on the system If the system has:
•
65,536 hash buckets, the size of a hash bucket is 16 bits.
•
1,048,576 hash buckets, the size of a hash bucket is 20 bits.
Set the default hash bucket size for your system using the DBS
Control utility (see Utilities:
Volume 1 (A-K) for details).
Interval Histogram Limits
Parameter
Number of hash values.
Maximum number of intervals per index or column set histogram.
The system-wide maximum number of interval histograms is set using the
MaxStatsInterval parameter of the DBS Control record or your cost profile. For descriptions of the other parameters listed, see SQL Request and Transaction Processing.
Default number of intervals per index or column set histogram.
Maximum number of equal-height intervals per interval histogram.
4.2 x 10
9
500
250
500
Value
470 Database Administration
Appendix B: Teradata System Limits: All DBAs
Database Limits
Database Limits
The database specifications in the following tables apply to a single Teradata database. The values presented are for their respective parameters individually and not in combination.
Name and Title Size Limits
Parameter
Maximum name size for database objects, for example, account, attribute, authorization, column, constraint, database, function, GLOP, index, macro, method, parameter, password, plan directive, procedure, profile, proxy user, query, role, procedure, table, transform group, trigger, UDF, UDM, UDT, using variable name, view.
See SQL Fundamentals for other applicable naming rules.
Maximum system name size.
Used in SQL statements for target level emulation. See SQL Data Manipulation
Language.
Maximum SQL text title size.
Value
128 UNICODE characters
63 characters
256 characters
Table and View Limits
Parameter
Maximum number of journal tables per database.
Maximum number of error tables per base data table.
Maximum number of columns per base data table or view.
Maximum number of columns per error table.
This limit includes 2,048 data table columns plus 13 error table columns.
Maximum number of UDT columns per base data table.
The same limit is true for both distinct and structured UDTs.
The absolute limit is 2,048, and the realizable number varies as a function of the number of other features declared for a table that occupy table header space.
Maximum number of LOB and XML columns per base data table.
This limit includes predefined type LOB columns and UDT LOB columns.
A LOB UDT or XML UDT column counts as one column even if the UDT is a structured type that has multiple LOB attributes.
Maximum number of columns created over the life of a base data table.
Maximum number of rows per base data table.
1
1
2,048
2,061
~1,600
32
Value
2,560
Limited only by disk capacity.
Database Administration 471
Appendix B: Teradata System Limits: All DBAs
Database Limits
Parameter
Maximum number of bytes per table header per AMP.
A table header that is large enough to require more than ~64,000 bytes uses multiple
64KB rows per AMP up to 1 MB.
A table header that requires 64,000 or fewer bytes uses only a single row and does not use the additional rows that are required to contain a larger table header.
The maximum size for a table header is 1 MB.
Maximum number of characters per SQL index constraint.
Maximum non-spool row size.
Maximum internal spool row size.
Maximum size of the queue table FIFO runtime cache per table.
Maximum logical row size.
In this case, a logical row is defined as a base table row plus the sum of the bytes stored in a LOB or XML subtable for that row.
This value is derived by multiplying the maximum number of LOB or XML columns per base table (32) times the maximum size of a LOB or XML column, both of which are 2,097,088,000 8-bit bytes.
Remember that each LOB or XML column consumes 40 bytes of Object ID from the base table, so 1,248 of those 67,106,816,000 bytes cannot be used for data.
Maximum non-LOB column size for a nonpartitioned table.
This limit is based on subtracting the minimum row overhead value for a nonpartitioned table row (12 bytes) from the system-defined maximum row length
(64,256 bytes).
Maximum non-LOB column size for a partitioned table.
This limit is based on subtracting the minimum row overhead value for a partitioned table row (16 bytes) from the system-defined maximum row length (64,256 bytes).
Maximum number of values (excluding NULLs) that can be multivalue compressed per base table column.
Maximum amount of BYTE data per column that can be multivalue compressed
Maximum amount of data per column that can be multivalue compressed for
GRAPHIC, LATIN, KanjiSJIS, and UNICODE server character sets
1 MB
16,000
64,256 bytes
~ 1MB
2,211 row entries
67,106,816,000 bytes
64,244 bytes
64,240 bytes
255
Value
4,093 bytes
8,188 characters
472 Database Administration
Appendix B: Teradata System Limits: All DBAs
Database Limits
Parameter
Maximum number of columns that can be compressed per primary-indexed or primary-AMP-indexed table or join index using multivalue compression or algorithmic compression.
This assumes that the object is not a NoPI table or join index or a global temporary trace table. All other tables, hash indexes, and join indexes must have a primary index or a primary AMP index. Primary indexes and primary AMP indexes cannot be either multivalue compressed or algorithmically compressed. Because of this, the limit is the maximum number of columns that can be defined for a table, which is 2,048, minus 1.
The limit for multivalue compression is far more likely to be reached because of table header overflow, but the amount of table header space that is available for multivalue compressed values is limited by a number of different factors. See Database Design for details.
Join index columns inherit their compression characteristics from their parent tables.
Maximum number of columns that can be compressed per NoPI table using multivalue compression or algorithmic compression
Column-partitioned NoPI join index columns inherit their compression characteristics from their parent tables.
Maximum number of algorithmically compressed values per base table column.
Maximum width of data that can be multivalue compressed for BYTE, BYTEINT,
CHARACTER, GRAPHIC, VARCHAR, and VARGRAPHIC data types
Maximum width of data that can be multivalue compressed for data types other than
BYTE, BYTEINT, CHARACTER, DATE, GRAPHIC, VARCHAR, and VARGRAPHIC
Maximum width of data that can be algorithmically compressed.
The maximum data width is unlimited if you specify only algorithmic compression for a column.
If you specify a mix of multivalue and algorithmic compression for a column, then the limits for multivalue compression also apply for algorithmic compression.
Maximum number of table-level CHECK constraints that can be defined per table.
Maximum number of primary indexes or primary AMP indexes per table, hash index, or join index that is not a NoPI database object.
Minimum number of primary indexes per primary-indexed table, hash index, or join index.
Minimum number of primary AMP indexes per primary-AMP-indexed table or join index.
Maximum number of primary indexes per primary-AMP-indexed or NoPI or join index or global temporary trace table.
Maximum number of columns per primary index or primary AMP index.
Maximum number of column partitions per table, including two columns partitions reserved for internal use).
Minimum number of column partition numbers that must be available for use by an
ALTER TABLE request to alter a column partition.
2,047
2,048
Unlimited
Unlimited
Unlimited
Unlimited
100
1
1
1
0
64
2,050
1
Value
Database Administration 473
Appendix B: Teradata System Limits: All DBAs
Database Limits
Parameter
Maximum partition number for a column partitioning level.
Maximum combined partition number for a single-level column-partitioned table or column-partitioned join index.
Value
Maximum number of column partitions for that level + 1
The same as the maximum partition number for the single partitioning level.
17,592,186,044,415
65,535
Maximum number of rows per hash bucket for a 44-bit uniqueness value.
Maximum combined partition number for a multilevel partitioning for 2-byte partitioning.
Maximum combined partition number for a multilevel partitioning join index for
8-byte partitioning.
Maximum number of ranges, excluding the NO RANGE, UNKNOWN, and NO
RANGE OR UNKNOWN and UNKNOWN partitions, for a RANGE_N partitioning expression for 2-byte partitioning.
This value is limited by the largest possible INTEGER value.
Maximum number of ranges, excluding the NO RANGE, UNKNOWN, and NO
RANGE OR UNKNOWN and UNKNOWN partitions, for a RANGE_N partitioning expression for 8-byte partitioning.
This value is limited by the largest possible BIGINT value.
Minimum value for n in a RANGE#Ln expression.
Maximum value for n in a RANGE#Ln expression for 2-byte partitioning
Maximum value for n in a RANGE#Ln expression for 8-byte partitioning.
Maximum number of partitions, including the NO RANGE, UNKNOWN, and NO
RANGE OR UNKNOWN partitions, for a single-level partitioning expression composed of a single RANGE_N function with INTEGER data type
Maximum number of ranges for a single-level partitioning expression composed of a single RANGE_N function with INTEGER data type that is used as a partitioning expression if the NO RANGE and UNKNOWN partitions are not specified.
Maximum number of ranges for a single-level partitioning expression composed of a single RANGE_N function with BIGINT data type that is used as a partitioning expression if the NO RANGE and UNKNOWN partitions are not specified.
Maximum number of ranges for a single-level partitioning expression composed of a single RANGE_N function with BIGINT data type that is used as a partitioning expression if both the NO RANGE and UNKNOWN partitions are specified.
Maximum value for a partitioning expression that is not based on a RANGE_N or
CASE_N function.
This is allowed only for single-level partitioning.
Maximum number of defined partitions for a column partitioning level
9,223,372,036,854,775,807
65,533
9,223,372,036,854,775,805
1
15
62
2.147.483.647
65,533
9,223,372,036,854,775,805
9,223,372,036,854,775,807
65,535
The number of column partitions specified + 2
The 2 additional partitions are reserved for internal use.
474 Database Administration
Appendix B: Teradata System Limits: All DBAs
Database Limits
Parameter
Maximum number of defined partitions for a row partitioning level if the row partitions specify the RANGE_N or CASE_N function.
Maximum number of defined partitions for a row partitioning level if the row partitions do not specify the RANGE_N or CASE_N function.
Maximum number of partitions for a partitioning level when you specify an ADD clause.
This value is computed by adding the number of defined partitions for the level plus the value of the integer constant specified in the ADD clause.
Maximum number of partitions for a column partitioning level when you do not specify an ADD clause and at least one row partitioning level does not specify an ADD clause
Maximum number of column partitions for a column partitioning level when you do not specify an ADD clause, you also specify row partitioning, and each of the row partitions specifies an ADD clause.
Value
The number of row partitions specified.
65,535
9,223,372,036,854,775,807
The number of column partitions defined + 10.
Maximum number of partitions for each row partitioning level without an ADD clause in level order, if using the number of row partitions defined as the maximum for this and any lower row partitioning level without an ADD clause.
Maximum partition number for a row-partitioning level.
The largest number for the column partitioning level that does not cause the partitioning to be 8-byte partitioning.
The largest number for the column partitioning level that does not cause the partitioning to be 8-byte partitioning.
The same as the maximum number of partitions for the level.
2
2,147,483,647
Minimum number of partitions for a row-partitioning level.
Maximum number of partitions for a CASE_N partitioning expression.
This value is limited by the largest possible INTEGER value.
Maximum value for a RANGE_N function with an INTEGER data type.
Maximum value for a RANGE_N function with a BIGINT data type that is part of a partitioning expression.
Maximum value for a CASE_N function for both 2-byte and 8-byte partitioning.
Maximum number of partitioning levels for 2-byte partitioning.
Other limits can further restrict the number of levels for a specific partitioning.
Maximum number of partitioning levels for 8-byte partitioning.
Other limits can further restrict the number of levels for a specific partitioning.
Maximum value for n for the system-derived column PARTITION#Ln.
Minimum number of partitions per row-partitioning level for a multilevel partitioning primary index.
Minimum number of partitions defined for a row-partitioning level.
Maximum number of partition number ranges from each level that are not eliminated for static row partition elimination for an 8-byte row-partitioned table or join index.
2,147,483,647
9,223,372,036,854,775,805
2,147,483,647
15
62
62
1
2 or greater
8,000
Database Administration 475
Appendix B: Teradata System Limits: All DBAs
Database Limits
Parameter
Maximum number of table-level constraints per base data table.
Maximum size of the SQL text for a table-level index CHECK constraint definition.
Maximum number of referential integrity constraints per base data table.
Maximum number of columns per foreign and parent keys in a referential integrity relationship.
Maximum number of characters per string constant.
Maximum number of row-level security constraints per table, user, or profile.
Maximum number of row-level security statement-action UDFs that can be defined per table.
Maximum number of non-set row-level security constraint encodings that can be defined per constraint.
The valid range is from 1 to 10,000.
0 is not a valid non-set constraint encoding.
Maximum number of set row-level security constraint encodings that can be defined per constraint.
31,000
5
4
10,000
256
Value
100
16,000 characters
64
64
Spool Space Limits
Parameter
Maximum internal spool row size
BLOB, CLOB, XML, and Related Limits
~ 1MB
Value
Maximum BLOB object size
Maximum CLOB object size
Maximum XML object size
Maximum number of LOB rows per rowkey per AMP for NoPI LOB or
XML tables
Parameter
Maximum size of the file name passed to the AS DEFERRED BY NAME option in a USING request modifier
Value
2,097,088,000 8-bit bytes
•
2,097,088,000 single-byte characters
•
1,048,544,000 double-byte characters
2,097,088,000 8-bit bytes
~ 256M
The exact number is 268,435,455 LOB or XML rows per rowkey per AMP.
VARCHAR(1024)
476 Database Administration
Appendix B: Teradata System Limits: All DBAs
Database Limits
User-Defined Data Type, ARRAY Data Type, and VARRAY Data Type Limits
Parameter
Maximum structured UDT size.
This value is based on a table having a 1 byte (BYTEINT) primary index. Because a
UDT column cannot be part of any index definition, there must be at least one non-UDT column in the table for its primary index.
Row header overhead consumes 14 bytes in an NPPI table and 16 bytes in a PPI table, so the maximum structured UDT size is derived by subtracting 15 bytes (for an NPPI table) or 17 bytes (for a PPI table) from the row maximum of 64,256 bytes.
Maximum number of UDT columns per base data table.
The absolute limit is 2,048, and the realizable number varies as a function of the number of other features declared for a table that occupy table header space.
The figure of 1,600 UDT columns assumes a FAT table header.
This limit is true whether the UDT is a distinct or a structured type.
Maximum database, user, base table, view, macro, index, trigger, procedure, UDF,
UDM, UDT, constraint, or column name size.
Other rules apply for Japanese character sets, which might restrict names to fewer than
30 bytes. See SQL Fundamentals for the applicable rules.
Maximum number of attributes that can be specified for a structured UDT per
CREATE TYPE or ALTER TYPE request.
The maximum is platform-dependent, not absolute.
Maximum number of attributes that can be defined for a structured UDT.
While you can specify no more than 300 to 512 attributes for a structured UDT per
CREATE TYPE or ALTER TYPE request, you can submit any number of ALTER TYPE requests with the ADD ATTRIBUTE option specified as necessary to add additional attributes to the type up to the upper limit of approximately 4,000.
Maximum number of levels of nesting of attributes that can be specified for a structured UDT.
Maximum number of methods associated with a UDT.
There is no absolute limit on the number of methods that can be associated with a given
UDT.
Methods can have a variable number of parameters, and the number of parameters directly affects the limit, which is due to Parser memory restrictions.
There is a workaround for this issue. See “ALTER TYPE” in SQL Data Definition
Language Detailed Topics for details.
Maximum number of input parameters with a UDT data type of VARIANT_TYPE that can be declared for a UDF definition.
Minimum number of dimensions that can be specified for a multidimensional ARRAY or VARRAY data type.
Maximum number of dimensions that can be specified for a multidimensional ARRAY or VARRAY data type.
Value
•
64,242 bytes (NoPI table)
•
64,241 bytes (NPPI table)
•
64,239 bytes (PPI table)
~1,600
30 bytes in Latin or Kanji1 internal representation
300 - 512
~4,000
512
~500
8
2
5
Database Administration 477
Appendix B: Teradata System Limits: All DBAs
Database Limits
Macro, UDF, SQL Procedure, and External Routine Limits
Parameter
Maximum number of parameters specified in a macro.
Maximum expanded text size for macros and views.
Maximum number of open cursors per procedure.
Maximum number of result sets a procedure can return
Maximum number of columns returned by a dynamic result table function.
The valid range is from 1 to 2,048. There is no default.
Maximum number of dynamic SQL requests per procedure
Maximum length of a dynamic SQL request in a procedure
This includes its SQL text, the USING data (if any), and the CLIv2 parcel overhead.
Maximum combined size of the parameters for a procedure
2,048
2 MB
15
15
2,048
15
Value
Approximately 1 MB
Maximum size of condition names and UDF names specified in a procedure.
Maximum number of parameters specified in a UDF defined without dynamic UDT parameters.
Maximum number of parameters that can be defined for a constructor method for all types except ARRAY/VARRAY
Maximum number of parameters that can be defined for a constructor method of an
ARRAY/VARRAY type
n
where n is the number of elements defined for the type
Unlimited Maximum number of combined return values and local variables that can be declared in a single UDF.
Maximum number of combined external routine return values and local variables that can be instantiated at the same time per session.
Maximum combined size of the parameters defined for a UDF.
1,000
1 MB for input parameters
64 KB for output parameters
1,144 Maximum number of parameters specified in a UDF defined with dynamic UDT parameters.
The valid range is from 0 to 15. The default is 0.
Maximum number of parameters specified in a method.
Maximum number of parameters specified in an SQL procedure.
Maximum number of parameters specified in an external procedure written in C or
C++.
Maximum number of parameters specified in an external procedure written in Java.
1 MB for input parameters
64 KB for output (and input/ output) parameters
30 bytes
128
128
128
256
256
255
478 Database Administration
Appendix B: Teradata System Limits: All DBAs
Database Limits
Parameter
Maximum size of an ARRAY or VARRAY UDT.
This limit does not include the number of bytes used by the row header and the primary index or primary AMP index of a table.
Maximum length of external name string for an external routine.
An external routine is the portion of a UDF, external procedure, or method that is written in C, C++, or Java (only external procedures can be written in Java). This is the code that defines the semantics for the UDF, procedure, or method.
Maximum package path length for an external routine.
Maximum SQL text size in a procedure.
Maximum number of nested CALL statements in a procedure.
Maximum number of Statement Areas per SQL procedure diagnostics area. See SQL
Stored Procedures and Embedded SQL and SQL External Routine Programming.
Maximum number of Condition Areas per SQL procedure diagnostics area. See SQL
Stored Procedures and Embedded SQL and SQL External Routine Programming.
64 KB
Value
1,000 characters
256 characters
Approximately 1 MB
15
1
16
Query and Workload Analysis Limits
Parameter
Maximum size of the Index Wizard workload cache.
The default is 48 MB and the minimum is 32 MB.
Maximum number of columns and indexes on which statistics can be collected or recollected at one time.
512 or limited by available parse tree memory and amount of spool.
Maximum number of pseudo indexes on which multicolumn statistics can be collected and maintained at one time.
A pseudo index is a file structure that allows you to collect statistics on a composite, or multicolumn, column set in the same way you collect statistics on a composite index.
This limit is independent of the number of indexes on which statistics can be collected and maintained.
Maximum number of sets of multicolumn statistics that can be collected on a table or join index if single-column PARTITION statistics are not collected on the table or index.
Maximum number of sets of multicolumn statistics that can be collected on a table or join index if single-column PARTITION statistics are collected on the table or index.
Maximum size of SQL query text overflow stored in QCD table QryRelX that can be read by the Teradata
Index Wizard.
Value
187 MB
512
32
32
31
1 MB
Database Administration 479
Appendix B: Teradata System Limits: All DBAs
Database Limits
Secondary, Hash, and Join Index Limits
Parameter
Number of tables that can be referenced in a join.
Minimum number of secondary, hash, and join indexes, in any combination, per base data table.
Maximum number of secondary, hash, and join indexes, in any combination, per base data table.
Each composite NUSI defined with an ORDER BY clause counts as 2 consecutive indexes in this calculation.
The number of system-defined secondary and single-table join indexes contributed by PRIMARY
KEY and UNIQUE constraints counts against the combined limit of 32 secondary, hash, and join indexes per base data table.
Maximum number of columns referenced per secondary index.
Maximum number of columns referenced per single table in a hash or join index.
Maximum number of rows per secondary, hash, or join index.
128
0
32
Value
64
64
Limited only by disk capacity.
64 Maximum number of columns referenced in the fixed part of a compressed join index.
Teradata Database implements a variety of different types of user-visible compression in the system.
When describing compression of hash and join indexes, compression refers to a logical row compression in which multiple sets of nonrepeating column values are appended to a single set of repeating column values. This allows the system to store the repeating value set only once, while any nonrepeating column values are stored as logical segmental extensions of the base repeating set.
Maximum number of columns referenced in the repeating part of a compressed join index.
Maximum number of columns in an uncompressed join index.
Maximum number of columns in a compressed join index.
64
2,048
•
64 for repeating
•
64 for nonrepeating
Reference Index Limits
Parameter
Maximum number of reference indexes per base data table.
There is a maximum of 128 Reference Indexes in a table header, 64 from a parent table to child tables and 64 from child tables to a parent table.
64
Value
480 Database Administration
Appendix B: Teradata System Limits: All DBAs
Database Limits
SQL Request and Response Limits
Parameter
Maximum SQL text size per request.
This includes SQL request text, USING data, and parcel overhead.
Maximum number of entries in an IN list.
There is no fixed limit on the number of entries in an IN list; however, other limits such as the maximum SQL text size, place a request-specific upper bound on this number.
Maximum SQL activity count size.
Maximum number of tables and single-table views that can be joined per query block.
This limit is controlled by the MaxJoinTables DBS Control parameter and the Cost Profile flags.
Maximum number of partitions for a hash join operation.
Maximum number of subquery nesting levels per query.
Maximum number of tables or single-table views that can be referenced per subquery.
This limit is controlled by the MaxJoinTables DBS Control parameter and the Cost Profile flags.
Maximum number of fields in a USING row descriptor.
Maximum number of open cursors per embedded SQL program
Maximum SQL text response size.
Maximum number of columns per DML request ORDER BY clause.
Maximum number of columns per DML request GROUP BY clause.
Maximum number of ORed conditions or IN list values per request
Maximum number of fields in a CONSTANT row
Row-Level Security Constraint Limits
1 MB
8 bytes
128
50
64
128
Value
Unlimited
2,543
16
1 MB
64
64
1,048,576
32,768
Parameter
Maximum number of row-level security constraints per table.
Maximum number of hierarchical row-level security constraints per user or profile.
Maximum number of values per hierarchical row-level security constraint.
Maximum number of non-hierarchical row-level security constraints per user or profile.
Maximum number of values per non-hierarchical row-level security constraint.
5
6
10,000
2
256
Value
Database Administration 481
Appendix B: Teradata System Limits: All DBAs
Session Limits
Session Limits
The session specifications in the following table apply to a single session:
Parameter
Maximum number of sessions per PE.
Maximum number of sessions per gateway vproc.
See Utilities for details.
Maximum number of active request result spools per session.
Maximum number of parallel steps per request.
Parallel steps can be used to process a request submitted within a transaction (which can be either explicit or implicit).
Value
120
Tunable: 1 - 2,147,483,647.
1,200 maximum certified.
The default is 600.
16
20
Maximum number of materialized global temporary tables that can be materialized simultaneously per session.
Maximum number of volatile tables that can be instantiated simultaneously per session.
2,000
1,000
Maximum number of SQL procedure diagnostic areas that can be active per session.
1
482 Database Administration
APPENDIX C
Performance Management:
All DBAs
This appendix presents a quick reference guide for the setup, monitoring, and management of database performance.
Implementing Performance Management
•
Follow recommended database and query design practices:
•
For best practice design of database components for optimal performance, see
Database Design.
•
For best practice design of database queries, see SQL Fundamentals, SQL Request and
Transaction Processing, SQL Functions, Operators, Expressions, and Predicates.
•
Set up recommended performance management tools. See
.
•
Manage query performance. See
“Managing Query Performance” on page 493 .
•
Manage the database workload. See
“Managing the Database Workload” on page 507 .
•
Manage database resources. See “Managing Database Resources to Enhance Performance” on page 517
.
•
Do performance-related periodic maintenance. See “Periodic Maintenance and
.
About Performance Management Setup
The following table describes set up tasks that must be completed to use the performance management techniques described elsewhere in this Appendix.
Tool
Resource Usage Data
(ResUsage)
Setup Task
Turn on ResUsage data collectors to enable system data gathering and provide information to various Teradata Viewpoint system management portlets.
Note: In addition to Teradata Viewpoint, you can use ResUsage macros to examine the collected data.
For information, see Resource Usage Macros and Tables.
Database Administration 483
Appendix C: Performance Management: All DBAs
About Performance Management Setup
Tool
Lock Logging
Teradata Viewpoint
Space allocation
Collect Statistics
Database Query Logging
(DBQL)
Setup Task
Use the WITH LOCK =n log option for the BEGIN/REPLACE
QUERY LOGGING statement to log any lock contention longer than
n centiseconds. The minimum acceptable value for n is 5.
You can access the DBQL lock log through the Viewpoint lock viewer portlet or query the system table DBC.DBQLXMLLOCKTbl or the view DBC.QrylockLogXMLV.
For information, see
“WITH Logging Options” on page 269
.
Configure Teradata Viewpoint, the primary Teradata Database system management tool, to:
•
Monitor resource usage, query activity, workload and detect problems.
•
Set thresholds that help to identify performance problems.
•
Set up alerts to inform you when thresholds are exceeded.
For details, see the Teradata Viewpoint User Guide.
Allocate space according to best practice recommendations, including a spool space reserve. See:
•
Chapter 9: “Managing Space: Operational DBAs.”
•
“Working with System-Level Space Allocation,” in Security
Administration.
Collect statistics on table columns and recollect when necessary to provide the Optimizer with the information necessary to create optimal query execution plans.
See Chapter 14: “Improving Query Performance Using COLLECT
STATISTICS: Application DBAs.”
Set up DBQL to collect data about queries for use in query optimization and detection of bad queries. See:
•
“Setting Up Database Query Logging” on page 490
.
•
Chapter 15: “Tracking Query Behavior with Database Query
For information on other useful tools, see Chapter 19: “Troubleshooting: Operational DBAs.”
484 Database Administration
Appendix C: Performance Management: All DBAs
Monitoring the System
Monitoring the System
Measuring System Conditions
The following table summarizes recommended system conditions to measure.
System
Conditions
Response time
Resource utilization
Data growth
Changes in data access
What to Use
Heartbeat queries
Baseline testing
Database
Query Log
(DBQL)
Resource
Usage
AMPUsage
DBQL
Increase in the number of active sessions
LogOnOffV, acctg
What to Collect
•
Response-time samples
•
System-level information
•
•
Application response time patterns
•
Response-time samples
Execution plans
ResNode Macro set
•
SPMA summarized to one row per node
How to Use
Saved samples can be used to:
•
Track trends
•
Monitor and alert
•
Validate Priority Scheduler configuration
•
Check for differences +/- in response time.
•
Check EXPLAINs if degradation is present.
•
Collect / keep information for comparison when there are changes to the system.
Track trends:
•
To identify anomalies
•
To identify performancetuning opportunities
•
For capacity planning
Look for:
•
Peaks
•
Skewing, balance
Use this to quantify heavy users.
CPU seconds and I/O for each unique account
CPU, I/O, spool used, rows returned
Summary row per table once a month
Look at trends.
•
Script row counts
•
Permspace
DBQL Summary row per table and the number of accesses once a month
Monthly and summarized number of sessions
Look for increases in access, trends.
Look for increases in concurrency and active users.
Database Administration 485
Appendix C: Performance Management: All DBAs
Monitoring the System
System
Conditions
Increase in system demand
What to Use
DBQL
What to Collect
Query counts and response times, plus other query information
System demand
How to Use
Look for trends, including growth trends, and measure against goals.
Resource
Usage
AMPUsage Queries with errors or queries that have been canceled
Using Alerts to Monitor the System
The Teradata Viewpoint alerting monitors Teradata Database and automatically invokes actions when critical (or otherwise interesting) events occur.
•
Teradata Viewpoint alerting can generate alerts based on the following events:
•
System
•
Node
•
Vproc
•
Session
•
SQL
•
Manual
•
Canary query
•
System health
•
Teradata Viewpoint alerting can:
•
Send email to an administrator
•
Send an SNMP trap
•
Run a program
•
Run a BTEQ script
•
Write to the alert log
Suggested Alerts and Thresholds
The following table lists key events and the values that constitute either warning or alerts.
Type
CPU saturation
I/O saturation
Event
Average system CPU > x%
CPU+WIO > x% and WIO > y%
Warning
(x = 95)
(x = 90, y = 20)
Critical
486 Database Administration
Appendix C: Performance Management: All DBAs
Monitoring the System
Type
Query blocked
Event
Query or Session blocked on a resource for longer than x minutes, and by whom
Total number of blocked processes > x
Warning
(x = 60)
Entire system blocked
User exceeding normal usage
“Hot Node” or “Hot
AMP” problem
Disk space
Product Join
System restart
Node down
Heartbeat query
Number of sessions per user >x (with an exclusion list, and custom code to roll up sessions by user)
Inter-node or inter-AMP parallelism is less than x% for more than 10 minutes
Disk Use% > x (vproc)
Average BYNET > x% (system)
Restart
Node is down
Timeout
(x = 4)
(x=10)
(x=90)
(x=50)
Critical
(x = 10)
(x=95)
SNMP
SNMP
SNMP
Weekly and/or Daily Reports
Weekly reports provide data on performance trends.
To make weekly and/or daily reporting effective, Teradata recommends that you establish threshold limits. Note that filtering on key windows having a 24-hour aggregate of weekly aggregates that includes low-used weekends and night hours distorts the report.
Furthermore, Teradata recommends that you analyze the longest period possible. This avoids misleading trend data by softening temporary highs and lows.
Below are examples of some of weekly reports and their sources:
•
CPU AV & Max from ResUsage.
This report provides data on the relationship between resources and demand
•
CPU seconds by workload (account) from AMPUsage
•
Throughput by Workload from DBQL
This report provides data on how much demand there is.
•
General Response times from Heartbeat Query Log
This report provides data on the general responsiveness of the system.
•
Response Time by Workload from DBQL
This report provides data on how fast individual queries were responded to.
•
Active Query & Session Counts by Workload
This report provides data on how many users and queries were active and concurrent.
•
CurrentPERM
This report provides data on how data volume is or is not growing.
Database Administration 487
Appendix C: Performance Management: All DBAs
Collecting Statistics
•
Spool
This report provides data on how spool usage is or is not growing.
Note: The above reports can, of course, be run daily.
Collecting Statistics
Guidelines for Collecting Statistics
Use the COLLECT STATISTICS statement to collect statistics on the columns used in predicates or GROUP BY clauses. When the table size grows more than 10%, you should recollect statistics on the table. Recollection covers all columns on which statistics were initially collected
Task
Collect UPI statistics on small tables
Collect NUPI statistics
Collect NUSI statistics on all
NUSIs
Collect NUSI statistics on covering (ALL option) NUSIs
Collect NUSI statistics on
NUSIs with ORDER BY
Collect non-index column statistics on all non-indexed columns used for table selection
Guideline
If you collect no other statistics on the table, collect UPI statistics.
The table is small (that is, 100 rows/AMP).
The NUPI is:
•
Fairly or highly unique, and
•
Used commonly in joins, or
•
Skewed
•
Small
The Optimizer can use the NUSI in range scans (BETWEEN...
AND...).
With statistics available, the system can decide to hash on the values in the range if demographics indicate that to do so would be less costly than a full table scan.
Without statistics, the index will likely not be used and, in some cases, the NUSI may be used when it is not efficient to do so.
If an SI is defined with the intent of covering queries (the ALL option is specified), you should consider collecting statistics even if the indexed columns do not appear in WHERE conditions.
Collecting statistics on a potentially covering NUSI provides the
Optimizer with the total number of rows in the NUSI subtable and allows the Optimizer to make better decisions regarding the cost savings from covering.
If a sort key is specified in a NUSI definition with the ORDER BY option, collect statistics on that column so the Optimizer can compare the cost of using a NUSI-based access path with a range or equality condition on the sort key column.
Collecting statistics on a group of columns allows the Optimizer to estimate the number of qualifying rows for queries that have search conditions on each of the columns or that have a join condition on each of the columns.
488 Database Administration
Appendix C: Performance Management: All DBAs
Collecting Statistics
Task
Refresh statistics after updates When:
•
The number of rows changed is greater than 10%.
•
For row partitioned tables >10% for any partition.
•
The demographics of columns with collected statistics changes.
Drop statistics
Guideline
If statistics are no longer needed because no queries are using them, they should be dropped.
Collecting Statistics on Skewed Data
For collecting statistics on skewed data, the Optimizer can accommodate exceptional values.
Statistics take into account the most nonunique value in a table and the most nonunique value in a value range.
Without statistics, the system relies on dynamic AMP sampling, which can be quite inaccurate when the table is small or unevenly distributed. Small tables often distribute unevenly.
Collecting Statistics on Join Index Columns
•
For non-sparse join and hash indexes, the Optimizer inherits the row count and all histograms from the base tables. Teradata recommends that you collect statistics on base tables only and not on the non-sparse join or the hash indexes.
•
For sparse join indexes, Teradata recommends that you collect statistics on base tables and sparse join indexes separately.
•
For non-compressed join indexes, the Optimizer performs all-AMP sampling to estimate the join index row count accurately.
•
For compressed sparse join indexes, Teradata recommends that you collect PI statistics.
The All-AMP random sampling that the Optimizer performs may not reflect an accurate row count because of the compression.
About PARTITION Statistics
Teradata Database provides the user with the ability to collect with “PARTITION statistics” based on partition numbers rather than column values. This enables the Optimizer to more accurately estimate cost operations involving a partitioned table.
The Optimizer is provided with:
•
The number of partitions that are nonempty
•
How the rows are distributed among the nonempty partitions.
Partition statistics can be collected for just the PARTITION column (single-column partition statistics) or on the PARTITION column and other table columns (multicolumn PARTITION statistics). Collecting statistics on the partitioning columns is also recommended. When the
Optimizer has this information, it can better calculate the relative cost of various methods of optimizing a query for a partitioned table.
Database Administration 489
Appendix C: Performance Management: All DBAs
Setting Up Database Query Logging
Having PARTITION statistics allows the Optimizer to generate a better plan for partitioned tables. For example, the Optimizer can cost joins involving partitioned tables more accurately with PARTITION statistics.
Setting Up Database Query Logging
Database Query Logging (DBQL) provides a series of predefined tables and views that store historical records of queries based on rules you specify.
For more information on DBQL, see Chapter 15: “Tracking Query Behavior with Database
Query Logging: Operational DBAs.”
Performance Effects of Query Logging
Because DBQL operates asynchronously, the logging activity has a much lower impact on the overall response time of given transactions. Furthermore, DBQL writes its information to internal memory buffers or caches. These are flushed to disk when the buffer is full or at the time indicated by the DBS Control Record field DBQLFlushRate. The default rate is every 10 minutes for normal operation, but the Database Administrator (DBA) can change the rate to be more frequent to analyze performance issues.
For information on DBQLFlushRate, see Utilities.
Performance Information in DBQL Tables
DBQL Table
DBQLogTbl
DBQLSummaryTbl
DBQLStepTbl
DBQLObjTbl
DBQLSqlTbl
DBQLExplainTbl
Performance Management Information
Data on individual queries, including:
•
Query origination, start, stop, and other timings
•
CPU and logical I/O usage
•
Error codes
•
SQL (truncated)
•
Step counts
A summary of short-running queries.
For high-volume queries, it provides query origination, response time summaries, query counts, and CPU and logical
I/O usage.
Query step timings, CPU and I/O usage, row counts, and step actions, among other things
Usage information about database objects such as tables, columns, indexes, and databases.
The complete SQL text.
EXPLAIN output.
490 Database Administration
Appendix C: Performance Management: All DBAs
Setting Up Database Query Logging
DBQL Table
DBQLXMLTbl
Performance Management Information
Query plans stored in Extensible Markup Language (XML).
Recommended DBQL Logging Scenarios
User Type 1: Short Subsecond Known Work (Tactical Work)
•
Log as Summary
•
BEGIN QUERY LOGGING LIMIT SUMMARY = 1,3,5 ON ALL ACCOUNT =
'acctname';
The numbers 1,3,5 are clock seconds not CPU seconds
•
No SQL gets logged
•
No Objects get logged
For this workload, high-speed performance and minimal response time are the primary objectives. Typically, this workload tends to be very predictable in nature with queries typically designed to be single AMP retrievals.
Logging of this workload at the request level may be unnecessary for two reasons:
•
The transactions are well-tuned, known, and repeated over and over again.
•
The additional overhead required to record SQL for each request would represent a meaningful portion of the overall work performed on behalf of the transaction, that is, the additional overhead could materially impact request response time.
The objective is to capture only summarized information about these SQL requests. Since the expectation for this workload type is that the work is predictable, repeatable and does not vary much, summary logging is sufficient. If however, there could be unauthorized use of this ID, or on occasion, a longer running query is run, threshold logging could be used.
User Type 2: Mostly Long Running Work
•
Log detail with SQL and objects
•
BEGIN QUERY LOGGING WITH SQL, OBJECTS LIMIT SQLTEXT =0 ON ALL
ACCOUNT = 'acctname';
If there are 10s of thousands of subsecond work, additional overhead will be incurred
Teradata recommends that a high degree of detailed data be captured for analysis of this workload category. The data generated from this DBQL logging option generates the critical detailed information needed for effective performance management and tuning.
The logging captures the entire SQL text for each request along with the objects used in processing the request. The SQL text and Object data is critical for performing query access path analysis. The DBQL detail data provides not only CPU and IO data, but also the data to calculate whether a query is skewed, does a large scan, or has the characteristics of a large product join, the keys to high impact performance tuning.
Database Administration 491
Appendix C: Performance Management: All DBAs
Setting Up Database Query Logging
User Type 3: Mostly Short Subsecond Requests with Occasional
Long Running or Unknown Work
•
Log Threshold
•
BEGIN QUERY LOGGING WITH SQL, OBJECTS LIMIT THRESHOLD =100 CPUTIME
AND SQLTEXT =10000 ON ALL ACCOUNT = 'acctname';
•
The threshold number 100 represents hundredths of CPU seconds and causes queries that require more than one second of CPU time to be logged in DBQLSQLTbl, DBQLObjTbl and DBQLogTbl, with details. Queries with less than one second CPU time are summarized.
With threshold logging, DBQL cannot log to separate Explain and XML tables, even for those queries taking longer than the specified criteria. SQL, STEPINFO, and OBJECTS can be logged during threshold logging, even for those queries taking longer than the specified clock seconds.
This logging scenario captures data at a more detailed level for unknown or long running requests, while still logging the bulk of the work, the subsecond requests, at a summary level.
DBQL Collection Recommendations
•
All usage should be logged in DBQL 24x7.
•
Flush the DBQL caches every 10 minutes. See
“Options for Flushing the DBQL Cache” on page 373.
•
Log usage in DBQL at the user account level:
Recommendation for account string setup is as follows: $H1$WXYZ&S&D&H, where:
•
$H1$ = Performance Group (PG)
•
WXYZ = Work Group Identifier
•
&S&D&H = ASE variables - session, date, and hour
See
“Working with User Accounts” on page 158 .
Using DBQL Tables
You can use DBQL tables to collect and evaluate:
•
System throughput
•
Response time
•
Query details, such as query step level detail, request source, SQL, answer size, rejected queries, resource consumption
•
Objects accessed by the query
Such information enables you to:
•
Identify a workload that does not meet response time Service Level Goals (SLGs)
•
Drill down into DBQL after targeting workloads using:
492 Database Administration
Appendix C: Performance Management: All DBAs
Managing Query Performance
This resource...
AMPUsage
Spool Log
To identify details about...
Top consumers
High spool users
Collecting Useful DBQL Historical Data
DBQL History summaries key DBQL data to 1 row per user / account / application ID / client
ID per time period. Teradata recommends retaining 13 months of copied detail. That is,
Teradata recommends copying detail to another table daily. You should delete the source of copied detail daily to keep online summary collection efficient.
Managing Query Performance
You can monitor queries using the Teradata Viewpoint Query Monitor portlet and other tools and utilities to identify poorly performing queries. Poor query performance is usually due to:
•
Stale statistics. See “Collecting Statistics” on page 488 .
•
Poor query construction. See
“Finding and Fixing Problem Queries” on page 493
.
•
Skewed query processing. See
“Managing Skewed Processing” on page 495
.
•
Skewed table distribution. See “Identifying and Fixing Skewed Tables” on page 497
.
•
Locks that block queries from accessing needed data. See
Blocked Queries” on page 501 .
This section also covers the performance issues related to transaction rollbacks, which occur
when a query fails to complete. See “Working with Transaction Rollbacks” on page 503
.
Finding and Fixing Problem Queries
1
2
You can use the Teradata Viewpoint Query Monitor portlet for regular monitoring of database queries and to investigate whether a reported delay is the result of a bad query.
Note: The Query Monitor portlet has many functions beyond what is shown in this example procedure. See Teradata Viewpoint User Guide for detailed setup and options.
Monitor query activity using the By Session tab.
Look for bad queries using the following Query Monitor parameters:
•
High CPU Use% combined with low REQ I/O accesses. If the CPU divided by the I/0 is
> 100, the query is probably bad.
•
High CPU Skew% with significant Delta CPU.
•
High spool usage.
Note: You can click on and sort the display by a parameter instead of by session number.
Database Administration 493
Appendix C: Performance Management: All DBAs
Finding and Fixing Problem Queries
4
5
3
6
7
8
Select a session row that shows CPU characteristics indicating a bad query to display the
Details View, which provides information about the query, including the user and account that submitted the query.
Select the SQL tab to display the query. Examine the SQL for errors and poor construction.
Select the Explain tab to display an abbreviated version of the step statistics and explain text (generated from a full EXPLAIN request), along with a listing of the active steps, completed steps, and steps yet to run.
Check for the following in the Explain:
•
The steps required to execute the query
•
The type of operation being performed, and whether it includes a product join
Often the query is stuck on the step where the product join occurs.
You can copy the query from the SQL tab, run it from Teradata SQL Assistant or Teradata
Studio, and request a full EXPLAIN to get more details and validate the source of the problem.
If the query appears to be constructed correctly, other problems may be indicated.
•
System conditions may cause uneven processing. See
“Skewed Query Processing Across
and
“Skewed Query Processing Across AMPs” on page 496
.
•
The data table may have skewed distribution. See
“Identifying and Fixing Skewed
.
•
The query may be blocked from accessing the data. See the sections beginning with
“Identifying and Managing Blocked Queries” on page 501 .
If a bad query causes problems in the database, you can:
•
Use Query Monitor to abort the query. See Teradata Viewpoint User Guide.
•
Use the SET SESSION ACCOUNT request to reassign a poorly-performing query to a lower priority account. For information on:
•
Using accounts to set job priorities, see
“Account Strings and Performance Groups” on page 513 .
•
SET SESSION ACCOUNT, see SQL Data Definition Language Syntax and Examples.
494 Database Administration
Appendix C: Performance Management: All DBAs
Managing Skewed Processing
Alternate Query Investigation Methods
Problem
If the system is CPU-bound or disk-bound, or the processing is skewed.
If the processing is skewed, identify the user.
If the processing is skewed, get information on the problematic query.
Action
•
Look at physical resource utilization:
•
Use Teradata Viewpoint. Monitor physical resources to view
CPU/disk usage by node. Monitor virtual resources to view
CPU/disk usage by vproc.
•
Look at DBC.ResusageSpma to view CPU, disk, and BYNET usage.
•
Look at workload utilization:
Use DBQL to find the user with the skewed processing.
•
Look at data distribution:
Use DBC.TableSizeV to identify tables with skewed data distribution.
•
Use Teradata Viewpoint.
•
Check DBC.SessionInfoV or run Query Session to identify the current user.
•
Check DBC.LogOnOffV to view historical users.
•
Check DBC.AMPUsage to identify the heavy resource user.
•
Check DBC.DiskSpaceV to identify large spool user.
•
To identify the query, ask the user or check query log.
•
Run EXPLAIN on the query to determine if a bad join plan exists. Look for unexpected product joins. Verify that large tables are not joined before small tables. Look for estimates that are too high or too low.
Managing Skewed Processing
Skewed Query Processing Across Nodes
Parallel node efficiency is a matter of how evenly the workload is shared among nodes. The more evenly the nodes share the work, the higher the efficiency.
You can calculate parallel node efficiency by dividing average node utilization by maximum node utilization, and identify skewed processing across nodes.
IF parallel node efficiency …
is nearly 100% falls significantly below 100%
THEN …
the better the nodes are working together.
in that time period, one or a few nodes are working harder than the others.
Database Administration 495
Appendix C: Performance Management: All DBAs
Managing Skewed Processing
IF parallel node efficiency …
falls below 60% more often than a couple of sampling periods
THEN …
your installation is not taking advantage of the parallel architecture.
Possible causes of skewed node processing include:
•
Down node
•
Non-Teradata Database application running on a TPA node
•
Co-existence system where the number of AMPs per node is not optimally balanced with different node powers.
•
Poor AMP efficiency
Skewed Query Processing Across AMPs
AMP vprocs always run in parallel, but the way data rows or column values are striped across the disks affect the parallel operation of AMP step processing.
Unbalanced, or skewed, or spiked, disk loads can cause one or a few AMPs to be doing most of the I/Os. For example, when a numeric column allows zeros and/or nulls, the majority of rows might hash to the same AMP.
If your disk loads are poorly balanced, discuss with operations ways to correct the situation.
For example:
•
Perhaps queries or views against a column with zeroes/nulls could use “WHERE NOT
NULL” or “NOT= 0” qualifiers.
•
If the cause is a NUPI, consider redefining the index, especially for a very large table, to achieve a higher percentage of uniqueness.
496 Database Administration
Appendix C: Performance Management: All DBAs
Identifying and Fixing Skewed Tables
Common Skew Issues
Issue
Same row hash value for an excessive number of rows
•
Rows with same row hash value cannot fit in a single data block.
•
Rows spill over into additional data blocks.
Some AMPs have many more rows of a table than do other
AMPs.
Cause
The table is skewed due to a highly nonunique primary index (PI).
As an estimate, more than 1000 occurrences/
NUPI value begin to cause performance degradation problems. This figure is based on all the rows for the same NUPI value spilling over into more than five data blocks.
See
“Identifying and Fixing Skewed Tables” on page 497
.
The data block size is set too low.
See
“Managing Data Block Usage” on page 530
.
Note: Data block size should not be a problem on systems using block-level compression because the size should be set to the maximum.
•
One or a few NUPI values have many more rows than all the other NUPI values.
•
A highly skewed join field in a join or aggregation, for example a join on a city code in a customer table where a large number of customers may be located in a few cities.
•
A highly skewed referencing column (for instance, a city code) when using referential integrity (RI).
Results
•
Increased I/Os for updates.
•
Increased compares for inserts and FastLoad (more Central
Processing Unit (CPU) and I/Os).
•
Performance degradation in the
Restore and Table Rebuild utilities.
•
Poor CPU parallel efficiency on the AMPs during full table scans and bulk inserts.
•
Increased I/O for updates, and increased compares for inserts
(also more I/O).
Identifying and Fixing Skewed Tables
Performance Effects of Skewed Row Distribution
Uneven distribution of table rows among AMPs (skew) can prevent efficient query processing.
Skewed distribution results in:
•
Poor CPU parallel efficiency on full table scans and bulk inserts
•
Increased I/O for updates and inserts of over-represented values
The effects of a skewed table appear in several types of operations. For example:
•
In full table scans, the AMPs with fewer rows of the target table must wait for the AMPs with disproportionately high numbers of rows to finish. Node CPU utilization reflects these differences because a node is only as fast as the slowest AMP of that node.
•
In the case of bulk inserts to a skewed table, consider the extra burden placed on an AMP with a high number of multiple rows for the same NUPI value.
Database Administration 497
Appendix C: Performance Management: All DBAs
Identifying and Fixing Skewed Tables
For example, assume you have a 5 million row table, with 5,000 rows having the same
NUPI value. You are inserting 100,000 rows into that table, with 100 of those insert rows having the same NUPI value. The AMP holding the 5,000 rows with that NUPI value has to perform one half million duplicate row checks (5,000 * 100) for this NUPI. This operation results in poor parallel efficiency.
Identifying Skewed Distribution of Table Data
You can do the following to identify uneven row distribution:
•
Check space usage using Teradata Viewpoint. See
“Using Teradata Viewpoint to Find
•
Run the ResVproc macro to check the AMP parallel efficiency so that you can identify the
AMP affecting that node. See Resource Usage Macros and Tables.
•
Check table distribution information at the AMP level by running an SQL query against
DBC.TableSizeV. See
“Finding Skewed Tables by Querying the TableSizeV View” on page 181 .
•
Check hash distribution among AMPs. See “Finding Uneven Distribution Using Hash
System Tables Subject to Skewing
•
Journal tables take the require spool and other transient storage from free space. A common practice is to allocate a database with a certain amount of PERM, but never use it. This ensures that there is free space for journals and so on. When your journal tables are skewed, you get an error when running queries, letting you know the system needs the journal space but cannot obtain it.
•
The DBC.AccessRights table is notorious for becoming skewed. If it grows too large, it will affect the space limits dramatically. To reduce this skewing, either clean up
DBC.AccessRights by removing redundant rows or convert to using roles and then remove the now redundant rows.
•
DBC.AccLogTbl table. Teradata Database generates at least one entry every time a user defined for logging attempts to access an object. This table can grow very large and consume a lot of disk space.
•
Resource usage logging tables or query logging tables. Consider daily offloading data from the actual DBC tables into temporary staging tables and ultimately history tables for analysis or archive.
Any table can becoming skewed if the primary index is poorly designed or if statistics have not been collected and the Optimizer is using stale data. A usual tell-tale sign is if one or more queries are doing a lot of maintenance on typically a single AMP. If this happens, you need to track that query down and handle it like any other skewed processing.
498 Database Administration
Appendix C: Performance Management: All DBAs
Identifying and Fixing Skewed Tables
Using Teradata Viewpoint to Find Skewed Tables
You can use the Details view in the Teradata Viewpoint Space Usage portlet to examine skew percentage for the tables in a database.
Finding Skewed Tables in DBC.TableSizeV
You can query the DBC.TableSizeV view to determine if data for a given table is evenly distributed across all AMP vprocs.
SELECT vproc,CurrentPerm
FROM DBC.TableSizeV
WHERE Databasename = 'RST'
AND Tablename = 'Message'
ORDER BY 2 desc;
The output lists the AMPs in order of the space they use for the Message table, from the most
to least space used. See “Finding Skewed Tables by Querying the TableSizeV View” on page 181
.
Finding Uneven Distribution Using Hash Functions
Use the following functions to identify uneven hash distribution of data.
Function
HASHAMP
HASHBACKAMP
HASHBUCKET
HASHROW
Definition
AMP that owns the hash bucket
Fallback AMP that owns the hash bucket
Grouping for the specific hash value
32 bits of row hash ID without the uniqueness field
HASHAMP Example
If you suspect distribution problems (skewing) among AMPS, the following is a sample of what you might enter for a three-column PI:
SELECT HASHAMP (HASHBUCKET (HASHROW (col_x, col_y, col_z))), count (*)
FROM hash15
GROUP BY 1
ORDER BY 2 desc;
HASHROW Example
If you suspect collisions in a row hash, the following is a sample of what you might enter for a three-column PI:
SELECT HASHROW (col_x, col_y, col_z), count (*)
FROM hash15
GROUP BY 1
ORDER BY 2 desc
HAVING count(*) > 10;
Database Administration 499
Appendix C: Performance Management: All DBAs
Identifying and Fixing Skewed Tables
Primary Index and Skewed Row Distribution
The hash value of the PI for a row determines where the row is stored on the AMP. In a normal environment, hash values are evenly distributed across the nodes and the AMPs within a node.
The less unique the values are for the index, the less evenly the rows of that table are distributed across the AMPs. If a table has a NUPI with thousands of instances of a single value, the table can become skewed.
Performance Effects of a Primary Index
•
The more unique the PI, the more unique the row hash value.
•
The more unique the row hash value, the more even the data distribution across all the
AMP vprocs. Even data distribution enhances parallel efficiency on full table scan operations.
•
UPIs generate the most even distribution of table rows across all AMP vprocs.
•
NUPIs generate even row distribution to the same degree that values for a column or columns are unique. Rows with the same row hash always go to the same AMP, whether they are from the same table or from different tables.
To determine the best PI for a table, factor in the:
•
Extent of update activity
•
Number of full table scans
•
Join activity against the PI definition
•
Frequency of PI as a selectivity column and, therefore, a potential access path
If a PI causes uneven data distribution, you should re-specify it according to the primary index design best practices shown in Database Design.
Adjusting Skew Allowance
The DBS Control SkewAllowance field specifies a percentage factor value used by the
Optimizer in choosing the size of each hash join partition. SkewAllowance reduces the memory size for the hash join specified by HTMemAlloc. This allows the Optimizer to take into account a potential skew of the data that could make the hash join run slower than a merge join.
If you know your data very well and do not expect skewing at this extreme, you can set this value to 50%, which still allows for a skew that is double the size the Optimizer uses in its estimates.
Values 20 through 74% can improve hash join performance in some single-user situations but should not be used on production systems. Do not use these values whenever more than one user is logged on. Most end users never have a need to use settings in these ranges.
A setting higher than 75% might improve hash join performance when the data is so severely skewed that it degrades performance. In this case, turn SkewAllowance off or try increasing the value to 80%.
500 Database Administration
Appendix C: Performance Management: All DBAs
Identifying and Managing Blocked Queries
Identifying and Managing Blocked Queries
Access to data for a query can be blocked by other queries which are temporarily given a lock on the data as a normal part of database operation. However some blocks, such as those due to a hung query, indicate database problems.
Also see
“Finding and Resolving Lock Contentions” on page 250 .
Locking Overview
When multiple transactions need to perform work that requires a nonshareable lock on the same object, Teradata Database controls concurrency by:
•
Granting a lock to the transaction that requests access to first.
•
Queuing subsequent transactions in order of their arrival, such that they wait indefinitely until preceding query completes and a new lock can be granted, with these exceptions:
•
Requests identified with the LOCKING modifier NOWAIT option immediately abort rather than join the queue.
•
MultiLoad transactions can timeout after waiting for over 50 seconds. See Teradata
MultiLoad Reference.
•
Aborting the younger request(s) among deadlocked requests. Deadlocking occurs when multiple requests need access to several of the same objects, each request has a lock on one of the objects, and is locked out of at least one of the objects, such that none of the requests can complete.
•
Releasing the lock and granting a new lock to the oldest transaction in the queue, when the current transaction completes.
For a complete discussion of locks and locking, see SQL Request and Transaction Processing.
Investigating Query Blocks and Delays
Most blocks are momentary and do not require attention. However, if a block persists, you can investigate it to identify the cause and determine whether further action is required.
1
Use the Teradata Viewpoint Query Monitor portlet to monitor the parameters related to query blocks and delays.
State
Blocked
Description
Indicates that a session query is held up by a lock on an object that the query is attempting to access.
Database Administration 501
Appendix C: Performance Management: All DBAs
Identifying and Managing Blocked Queries
State Description
Blocked Time How long the query has been blocked.
Momentary locks are a normal part of database operation. Persistent locks may indicate:
•
A runaway query or hung query that cannot complete and release its locks, and which should be aborted
•
A long-executing query, such as large-scale update or backup operation, which should be rescheduled to avoid resource contention
Delayed Indicates that the query is in a delay queue caused by a workload rule.
2
3
Click the session ID for the query to access the Details View.
•
If a query is blocked, see the Blocked By tab for details
•
If the query is delayed, the Delay tab provides details about the cause of the delay
You can also use the Lock Viewer portlet to find out additional details about the block:
•
Block Time
•
Database
•
Table
•
Delay
•
Blocked User
•
Blocking Level
•
Blocking User
Additional Tools for Analyzing Lock Problems
The following table provides suggestions for analyzing and solving lock problems.
Tool
Lock Display
Query Session
Show Locks
Analysis
Transaction locks
Blocked session
Host utility (HUT) locks; that is, locks placed by a client-based utility, such as Archive and
Recovery
Solution
Determine which session is holding the lock that blocks others.
Abort the session causing blocked transactions.
Submit RELEASE LOCKS as either an Archive and Recovery command or an SQL statement.
Reducing Deadlocks
•
Except with CREATE INDEX, use LOCKING FOR ACCESS if dirty reads are acceptable.
•
Consider BTEQ handling of transaction processing. After transaction rollback, BTEQ continues the transaction from the point of failure, not at the beginning of the transaction.
502 Database Administration
Appendix C: Performance Management: All DBAs
Working with Transaction Rollbacks
•
Set the DeadLockTimeout field via the DBS Control utility to 30 seconds if you have a mix of DSS and PI updates on fallback tables.
•
Be sure to use RELEASE LOCKS on Archive/Recovery jobs.
•
Use the LOCKING ROW [FOR] WRITE/EXCLUSIVE phrase preceding a transaction.
This phrase does not override any lock already being held on the target table. LOCKING
ROW is appropriate only for single table selects that are based on a PI or SI constraint. For example:
LOCKING ROW FOR WRITE
SELECT y FROM tableA WHERE pi =1;
UPDATE tableA SET y=0 WHERE pi =1;
•
In macros, use multistatement requests instead of Begin Transactions (BT)/End
Transactions (ET) to minimize table-level deadlocking. For example:
LOCKING ROW FOR WRITE
SELECT y FROM tableA WHERE pi =1
; UPDATE tableA SET y=0 WHERE pi =1 ;
This causes all the necessary locks to be applied at the start, which avoids the potential for a deadlock. Use the EXPLAIN modifier to check out the processing sequence.
•
Be aware that when you are updating tables where several rows in the base table share a row in the join index subtable (such as in most aggregate join index cases), there can be considerable collision on the same row.
Handling Blocked Internal DBQL Requests
DBQL uses express requests to do internal work on DBQL log tables. If you are performing work on the same log tables, such as deleting rows, this may block DBQL internal requests because you have a lock on the table. The internal requests will then consume AMP worker tasks until your work finishes or until you disable the DBQL logging.
Note: You cannot abort or otherwise do anything to this internal session record. It exists only to show that internal DBQL requests are blocked.
If DBQL internal requests remain blocked, the system deadlock detection may abort them. If the system aborts these internal sessions, the logging data will be lost. You must submit the appropriate END QUERY LOGGING statement to end logging before you do any kind of maintenance on DBQL tables.
If you notice frequently blocked DBQL internal requests reported by the internal session, consider scheduling your queries (such as table maintenance) on DBQL tables when there is no logging activity. Or use locks that hold the logging tables for a minimal amount of time.
Working with Transaction Rollbacks
A rollback is a reversal of an incomplete database transaction due to problems. A transaction may fail to complete and be aborted due to:
•
A deadlocked query
•
A database restart
Database Administration 503
Appendix C: Performance Management: All DBAs
Working with Transaction Rollbacks
•
An ABORT SESSION command
The rollback removes any partially completed database updates from the affected user tables.
The system uses the TJ (dbc.transientjournal), which contains data about incomplete transactions, including a “before image” of each modified table row, to maintain data integrity.
Also see the rollback topics in
Chapter 11: “Archiving, Restoring, and Recovering Data:
Effects on Performance
Because a rollback can conceivably involve millions or even billions of rows, a rollback can affect the performance and availability of resources in Teradata Database while the rollback is in progress.
The rollback affects the performance of the system because the rollback competes for CPU with other users. Moreover, a rollback can keep locks on affected tables for hours, or even for days, until the rollback is complete.
RollbackPriority
Because rollbacks can involve millions or billions of rows, competing for CPU and other system resources, rollbacks can impact system performance. Rollbacks can keep locks on affected tables for hours or days until the rollback is complete. During a rollback, a trade-off occurs between overall system performance and table availability.
How RollbackPriority affects performance is not always straightforward, and is related to the
Priority Scheduler configuration, job mix, and other processing dynamics. The
RollbackPriority setting should only be changed after full consideration of the performance consequences:
•
When RollbackPriority is set to FALSE, rollbacks are performed at system priority, a special priority higher than any user-assigned priority, that is reserved for critical internal work. As a result, faster rollbacks occur at the expense of other online performance.
The default setting of FALSE is especially appropriate when rollbacks are large, occurring to critical tables that are accessed by many users. It is better to complete these rollbacks as quickly as possible to maximize table availability.
•
When RollbackPriority is set to TRUE, rollbacks are executed within the aborted job's
Performance Group or workload. This isolates the rollback processing to the aborted job's priority, and minimizes the effect on the performance of the rest of the system. However, if the rollback places locks on tables that other users are waiting for, this causes a greater performance impact for those users, especially if the rollback is running at a low priority.
A setting of TRUE is appropriate when rollbacks are typically smaller, occurring to smaller tables that are less critical, and less extensively used.
Regardless of the RollbackPriority setting, rollbacks are never subject to CPU limits:
•
When RollbackPriority is FALSE, rollbacks run in the system performance group where jobs use as much CPU as they require.
504 Database Administration
Appendix C: Performance Management: All DBAs
Working with Transaction Rollbacks
•
When RollbackPriority is TRUE, rollbacks are not constrained by CPU limits at the system, Resource Partition, or Allocation Group level.
Minimizing or Avoiding Rollbacks
If you are doing a lot of deletes on rows from a table, consider using MultiLoad rather than
BTEQ. MultiLoad completely avoids use of the TJ and is restartable.
To minimize the number of TJ rows being generated on an INSERT . . . SELECT request, consider doing a multistatement INSERT . . . SELECT request into an empty table from both of the other tables. This only puts one entry into the transient journal to let the DBS know that the target table is an empty table and to drop all the rows in the target table if the need for a rollback is encountered. After the new table is created, drop the old table and rename the new table to the name of the old table.
More details on both the delete rows and INSERT . . . SELECT request are available online in the Support Link Knowledge Base.
Detecting a Rollback in Progress
Sessions in rollback may appear to have logged off in both DBC.LogOnOffV and
DBC.AccessLogV, but this is not always the case. The logoff would depend on the manner in which the job was aborted. If you specify one of the following:
ABORT SESSION hostid.username LOGOFF
ABORT SESSION *.username LOGOFF
ABORT SESSION hostid.* LOGOFF then the LOGOFF option would terminate the session.
Without it, the session should continue until the abort completes or the Supervisor issues a
LOGOFF request. Unless an SQL job is explicitly coded to do otherwise, a session will also appear to have logged off if the system has undergone a restart.
The rollback or abort is independent of the session. It is actually handled by a completely different mechanism with internally allocated AMP worker tasks.
Example
To activate the RCVmanager, go to the Database Window and type “start rcvmanager”. Then issue the “list rollback tables” command. It will show you each table that is being rolled back at that point in time, how many TJ rows have been rolled back and how many rows are remaining.
If you run this command twice, you can then make an estimate how long it will take the rollback to complete, based on the rows processed and rows remaining and the time between the two snapshots.
Database Administration 505
Appendix C: Performance Management: All DBAs
Working with Transaction Rollbacks list rollback tables;
TABLES BEING ROLLED BACK AT 10:01:26 04/09/20
ONLINE USER ROLLBACK TABLE LIST
Host Session User ID Workload Definition AMP W/Count
---- -------- --------- -----------------------------
1 234324 0000:0001 24
TJ Rows Left TJ Rows Done Time Est.
------------- ------------- ---------
53638 1814 00:09:51
Table ID Name
--------- --------------------------
0000:16A6 "FINANCE_T"."Order_Header"
SYSTEM RECOVERY ROLLBACK TABLE LIST
Host Session TJ Row Count
---- -------- -------------
Table ID Name
--------- ------------- ----
Enter command, "QUIT;" or "HELP;" : list rollback tables;
TABLES BEING ROLLED BACK AT 10:01:37 04/09/20
ONLINE USER ROLLBACK TABLE LIST
Host Session User ID Workload Definition AMP W/Count
---- -------- --------- ------------------- -----------
1 234324 0000:0001 24
TJ Rows Left TJ Rows Done Time Est.
------------- ------------- ---------
52663 2789 00:09:45
Table ID Name
--------- ---------------------------
0000:16A6 "FINANCE_T"."Order_Header"
SYSTEM RECOVERY ROLLBACK TABLE LIST
Host Session TJ Row Count
---- -------- -------------
Table ID Name
--------- ------------------
Enter command, "QUIT;" or "HELP;" :
506 Database Administration
Appendix C: Performance Management: All DBAs
Managing the Database Workload
Canceling Rollbacks
When a transaction fails to complete because the database restarts or the job of the transaction has failed, the system must remove any partially completed database updates and roll back to the pre-transaction condition to assure data integrity.
Because rollbacks can lock affected tables until the rollback completes, you can restore specific tables if it is quicker than waiting for a rollback to complete. If you decide to use the restore method, cancel the rollback using the Recovery Manager utility. See Utilities.
Note: You should list rollback tables using the LIST ROLLBACK TABLES command and then cancel rollback from that list within the same Recovery Manager session.
Noticing a Slow Rollback of an ALTER TABLE Statement
Because Teradata ensures that all transactions must succeed or fail in their entirety, the system does not allow partial results. In most cases, if a transaction fails, Teradata rolls back all the previous actions of a transaction. This requires that the system logs the before state of every action so that the action can, if necessary, be undone later. For instance, when you delete a row, Teradata must first log a copy of the row, so that it can be restored if the transaction fails.
Sometimes, logging is extremely expensive compared to the cost of the action, for example, dropping a large table. If the system does not have to roll the table back later, the system may simply release all table data, deallocating entire cylinders with a single action. In this case,
Teradata delays performing the action until it is certain that the transaction will succeed. In effect, Teradata commits the transaction before executing the deferred action.
During this special post-commit phase, the transaction is still working and holding locks, but it cannot be aborted (since it is already committed). In most cases, this post-commit phase finishes quickly and is unnoticeable.
The exception is the ALTER TABLE statement. It may take longer when used to add or remove columns of a table. ALTER TABLE requires that every row in the table be rebuilt and is, therefore, an expensive operation. It can take hours on a large table, even with the highly optimized, block-oriented method that Teradata uses.
When you notice that a rollback of an ALTER TABLE statement is taking a very long time but you cannot abort it, the system is deferring an abort and will need to complete it.
Managing the Database Workload
Also see
Chapter 12: “Managing Database Resources: Operational DBAs.”
Evaluating Peak Utilization Periods
Once you have identified the peak period, you can look for the bottleneck. Examining
ResUsage data is one technique helpful in finding bottlenecks.
Some possible bottlenecks are:
•
CPU saturation
Database Administration 507
Appendix C: Performance Management: All DBAs
Managing the Database Workload
•
Disk saturation
•
Free and/or FSG cache memory
•
BYNET
•
Vprocs (hot AMPs, coexistence, load balancing)
•
Channel (number of sessions or channel speed)
•
LAN/gateway (number of sessions or network connections)
•
Lock contention
Resource Usage by Priority Scheduler Allocation Group
The ResUsageSps table allows you to see accumulated CPU, number of active processes, and other detail by Priority Scheduler Allocation Group (AG). It carries information that is similar to what is displayed in Priority Scheduler monitor output.
Information carried in the table is organized by:
•
Collection date/time
•
Node
•
Vproc
•
Performance Group
•
Performance Period/AG
For those using TASM, each workload definition is the equivalent of one Performance Group in ResUsageSps.
The CpuTime and the IOBlks columns in ResUsageSps are collections of resource usage over the life of this particular logging interval.
No matter what your configuration size, care must be taken to manage what may be an exceptionally high volume of output from ResUsageSps table logging. Plan to offload
ResUsageSps data frequently or turn on ResUsageSps logging at only limited times of the day, for example during the times of heaviest usage. To minimize the volume of the output, activate Active Row Filter Mode, when logging is enabled for ResUsageSps.
For ResUsageSps table column definitions, see Resource Usage Macros and Tables.
Assessing a Busy System
CPU Saturation
Systems that run frequently at or near capacity are often the subject of assessments in which attempts are made to determine the extent of resource exhaustion.
When CPU is the binding factor and the scarce resource on a system, it is useful to determine the highest level of saturation of the system, that is, the point to which you can drive the system before it becomes so overloaded that it appears to be hanging.
Once a system reaches this level of saturation, the system should still be able to work itself out, but that may require extra time to do so.
508 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Managing the Database Workload
Without appropriate performance monitoring, users may start, once the system appears to be hanging, to:
•
Abort jobs that could cause rollbacks, or
•
Submit duplicate queries that create more work on an already-exhausted system.
While ResUsage data provides the bulk of the information needed to know that a system is at
100% busy with respect to CPU or CPU+I/O Wait, other information may also be needed in order to examine the extent of CPU saturation and system congestion. In other words, one can know that the system is running at capacity, but not the extent of the overload.
When the system becomes so busy that logons become slow or hung, performance monitoring is not able to determine whether the system is actually hung or simply overloaded without using other tools.
1
2
3
Suggested Monitoring Techniques
Since the goal is to be able to drive the system as hard as possible without overloading it, some techniques for assessing the level of busy can be used when CPU usage is high:
Check AWT utilization. If the number is constantly at or near maximum, then
Check the message flow control. If there are tasks apparently in flow control, then
Check run queue ResUsageSawt MailBoxDepth data. If the run queue is growing longer and longer, the system is too busy and will slow down dramatically.
While it is not unusual to see a busy system with high AWT counts, the presence of flow control means that some tasks are currently being blocked from sending more work to busy
AMPs.
Finding a Saturated Resource
Use Resource Check Tools, located in the /usr/pde/bin directory, to check for saturated resources.
IF …
mboxchk is not already running a background task the mboxchk log shows a slow response or timeout no attribute is reported at the WARN level
THEN …
run mboxchk to check current Teradata
Database response time. run syscheck to get a report showing attributes that falls below the specified danger level.
check disk and AMP CPU usage.
For information about mboxchk and other PDE resource check tools, such as nodecheck and syscheck, see man pages or pdehelp.
High I/O Wait
Teradata recommends configurations with CPU to I/O bandwidth ratios according to typical
Teradata Database workload demands in order to avoid CPU starvation. If these guidelines are
509
Appendix C: Performance Management: All DBAs
Managing the Database Workload not followed, or customer workload is unusually heavy with respect to I/O, it is possible that
CPU starvation may still occur, as reflected by high I/O WAIT on the system.
If ResUsage data shows that I/O WAIT is rising while CPU busy is falling, it is an indication that the system is not able to use the available CPU because I/O has become the limiter.
If the onset of high I/O wait is sudden:
1
2
Determine if the high I/O wait is due to disk I/O, waiting for AMPs from other nodes
(because of skewing or coexistence imbalance), low system demand, or BYNET I/O.
CPU+WIO less than 90% may suggest low system demand without a true I/O bottleneck.
Look at node efficiency to determine if the I/O wait is due to node waiting.
Look at the actual disk I/O wait queue using sar –d, and examine:
•
avwait
Average time in milliseconds that transfer requests wait idly on queue for response (in the FibreChannel driver queue or disk queue).
•
avserv
Average time to be serviced (includes seek, rotational latency and data transfer times).
For more information on the sar utility, see the Linux operating system documentation.
Job Scheduling Around Peak Utilization
Rescheduling Jobs
Once you determine your peak system utilization times, you can recommend that some jobs be moved to other time slots.
For example, if peak periods are 9 A.M. to 5 P.M., you can schedule batch and load jobs overnight so they do not interfere with peak daytime demand.
Bound Jobs
Since some jobs tend to be CPU-bound and others I/O- bound, it is a good idea to determine which jobs fit into each category. You can determine this by means of AMPUsage data analysis.
You can schedule a CPU-bound job with an I/O bound job so that the resource underutilized by one job can be used more fully by the other.
TASM and Concurrency
TASM throttle rules can be useful in controlling the concurrency of certain types of queries during peak utilization times.
In addition, TASM filter rules can prevent queries with certain characteristics from even starting to execute during specific windows of time. This can help keep utilization levels under control at times of high contention.
Managing I/O-Intensive Workloads
Below are suggestions for balancing resource usage when the system is I/O-bound:
•
Identify I/O-intensive portions of the total work using AMPUsage reports and DBQL.
510 Database Administration
Appendix C: Performance Management: All DBAs
Managing the Database Workload
•
Reschedule I/O-Intensive work to off-hours.
•
Look for query or database tuning opportunities, including:
•
Collecting/refreshing statistics on all join and selection columns
•
Adding indexes, join indexes, or sparse indexes
•
Using MVC to reduce row size and get more rows per block
•
Using PPI
•
Increasing block sizes
•
Using 3NF data model to obtain narrower rows, more rows / block, fewer I/Os, and then denormalizing as needed
•
Increasing node memory, in order to expand the size of the FSG cache
Using Canary Queries
A canary query a common SQL statement that is run at specific intervals and monitored for data such as response time to characterize the current system workload. Each canary query is fixed to do a consistent amount of work per execution.
You can use Teradata Viewpoint to send canary queries to:
•
Measure response time as an indicator of system demand or system or database hangs.
•
Measure response time for various TASM workloads and performance groups
•
Initiate an alert system if response time degrades so that you can take appropriate action.
•
Establish response time Service Level Agreements (SLAs) based on canary response times.
System Canary Queries
Use system canary queries to check for overall system or database hangs, to take some kind of action when response times reach certain thresholds, or when stalled, such as send alert and/ or capture system level information.
More than just a check, a system canary query should execute diagnostics that capture the state of the system if performance stalls.
System canary queries are intended specifically to focus on the core system of Teradata
Database. They should be short-running (one second), low impact queries on tables that are normally not write locked.
System canary queries are most useful when run frequently. For example, some sites run them every 3 to 5 minutes; other sites find every 5 to 10 minutes adequate.
They should be run on a system node. This will eliminate other factors, such as middle tiers, network connections.
Depending on their makeup, canary queries can add to contention for resources. Use them selectively, where needed, with shorter queries preferable.
Sample System Canary Query
The simplest canary monitor query is the following:
SELECT * from DBC.DBCInfoV;
Database Administration 511
Appendix C: Performance Management: All DBAs
Managing the Database Workload
As the query runs, Teradata Viewpoint can monitor the query, logging start and end times. If the query runs longer than the indicated threshold, an alert and perhaps diagnostic scripts are automatically executed.
Production Canary Queries
Production canary queries may be used to:
•
Take response time samplings, storing them for tracking purposes, or
•
Monitor the expected response times of specific groups of queries, such as short-running tactical queries running in high priority.
Response times are an indicator of system demand. When system demand is high, canary response is high. You can expect all other queries running in the same PG to display similar elongations in response time.
From a user perspective, a sudden deviation in response times would have an immediate impact, since users of consistently short running queries would be the first to notice performance degradation.
Production canary queries have wider uses than system canary queries and can be used in a variety of ways. For example, they:
•
Can be run on production user tables.
•
Could be run from other endpoints in the system architecture, such as a network client PC or z/OS client to expand scope of monitoring.
•
Monitor overall response.
•
Monitor specific area of the job mix.
•
Can be more complex and similar in nature to a particular type of production query, running in the same Priority Scheduler PG.
•
Are run less frequently than system canary queries, usually once every 20 to 60 minutes.
In using a production query from a non-TPA node location, other things, such as network and middle-tier monitoring, are also covered, but when it stalls, you need to investigate further to determine where the bottleneck is located.
Once the response time for a canary query is stored in a table, it can be summarized for use in tracking trends.
ASE Impact on PE and AMP Performance
•
ASE has little impact on PE performance. The cost incurred for analyzing the account string amounts to only a few microseconds.
•
ASE may have impact on AMP performance.
The AMP has the burden of additional DBC.AMPUsage logging. Depending on the number of users and the ASE options selected, the added burden may vary from very slight to enough to degrade performance. In general, the &D, &H, and &L options do not have major effects on performance.
512 Database Administration
Appendix C: Performance Management: All DBAs
Managing the Database Workload
Be cautious where you use the &T option. It can generate an AMPUsage row for virtually every Teradata SQL request. The &T option can have a much greater effect on performance. Therefore, do not use the &T option:
•
In default account ID strings
•
In conjunction with tactical queries
•
With TPump
The &T option should not be a problem for long-running DSS requests, but could be a performance issue if users are running numerous small requests. &T is site-dependent; it should generate no more than 10 to 20 AMPUsage rows per minute.
Note: Because ASE causes the system to write more entries to DBC.AMPUsage, you must manage the table more often.
For more information on ASE, see
Chapter : “Logging Resource Usage Data with Account
AMPUsage Logging with ASE Parameters
AMPUsage logging may have both performance and data storage impacts.
The following table summarizes potential impact.
&L
&I
&S
&T
ASE Parameter
None
&D
&H
&D&H
Performance Impact
Negligible
Negligible
Negligible
Negligible
Data Capacity Impact
1 row per account per AMP
1 row per account per day per AMP
1 row per account per hour per AMP
1 row per account per hour per day per
AMP
Negligible
Negligible
1 row per session pool
1 row per SQL request
Negligible 1 row per session
Potentially non-negligible 1 row per query per AMP
Account Strings and Performance Groups
A Performance Group (PG) must be specified for every account string in use, except for DBC and SYSTEMUSERINFO. The account strings for these user IDs cannot be changed.
Priority Scheduler and Performance Groups
Priority Scheduler (PS) manages the allotment of system resources among concurrently running transactions based on their defined relative priority. This is accomplished through assigning each request to a default PG.
The definition of the PG within PS determines how that request obtains CPU resources over the course of the execution of the request.
Database Administration 513
Appendix C: Performance Management: All DBAs
Managing the Database Workload
Enabled TASM Workloads
When TASM workload definition (Category 3) is enabled, the Priority Scheduler PG portion of the account string is ignored for purposes of priority assignment. Instead, workload classification determines the priority of the request.
However, for the small amount of time that the first request of a session is being parsed prior to classification into a workload, the PG within the account string is used to establish priority.
Charge Back
ASE can be used to implement a simple charge back utility. It provides a way to determine how much system CPU time and disk activity is consumed by a user each day. By adding a few special characters to a user's account name, you can extract detailed information from system tables about what that user has done. Teradata Database expands these special characters into such things as the session number, request number, date or time when the account name is written to a system table.
At the completion of each SQL step, Teradata Database always updates the DBC.Acctg table with statistics about the request. These statistics include the total CPU time in seconds and the number of logical disk I/Os used by the request. This statistical information is summarized by adding it to an existing row that contains the same username and account name.
Configuration
The following example modifies the account string of each user you want to track. You preface each account string with the text CB&D. You can add any additional account information after these four characters if you wish.
When you add a date to the account name, the account name effectively changes each day and a new row is written to the DBC.Acctg table. This row contains the total number of CPU seconds and total number disk I/Os for each request that was submitted on that date.
The &D is an ASE token that expands to the current date in the format YYMMDD. You can use CB to indicate that the account is being tracked for charge back.
You can modify an existing account string for a user using the following SQL command:
MODIFY USER JANETJONES AS ACCOUNT = ('CB&D');
Note: Priority control characters ($R, $H, $M, $L, and so on), if used, must be the first characters in the account string. An example of an account string that contains priority control and account string expansion would be: $MCB&D. The SQL query examples below would need to have the SUBSTR functions modified to account for the new offset of the ASE information.
Example
SELECT ACCOUNTNAME, USERNAME, SUM(CPUTIME), SUM(DISKIO) FROM
DBC.AMPUSAGE
WHERE SUBSTR(ACCOUNTNAME, 1, 2) = 'CB'
GROUP BY USERNAME, ACCOUNTNAME
ORDER BY USERNAME, ACCOUNTNAME;
*** Query completed. 11 rows found. 4 columns returned.
514 Database Administration
Appendix C: Performance Management: All DBAs
Managing the Database Workload
*** Total elapsed time was 2 seconds.
AccountName UserName Sum(CpuTime) Sum(DiskIO)
---------------------------------------------------------
CB060902 JANETJONES 1,498.64
3,444,236
CB060903
CB060904
CB060905
CB060902
JANETJONES
JANETJONES
JANETJONES
JOHNSMITH
934.23
883.74
214.99
440.05
1,588,764
924,262
200,657
396,338
CB060903
CB060904
CB060905
CB060902
JOHNSMITH
JOHNSMITH
JOHNSMITH
SAMOREILLY
380.12
112.17
56.88
340.34
229,730
184,922
99,677
410,178
CB060903
CB060902
SAMOREILLY
WEEKLY
70.74
3,498.03
56,637
7,311,733
If we wanted to charge $0.25 per CPU second and bill for the month of
September 2006, we could use the following query to generate the bill:
SELECT USERNAME, SUM(CPUTIME)*0.25 (FORMAT '$$ZZZ,ZZZ,ZZ9.99')
FROM DBC.AMPUSAGE
WHERE SUBSTR(ACCOUNTNAME, 1, 6) = 'CB0609'
GROUP BY 1
ORDER BY 1
WITH SUM(CPUTIME)*0.25 (FORMAT '$$ZZZ,ZZZ,ZZ9.99', TITLE 'Grand
Total:');
*** Query completed. 4 rows found. 2 columns returned.
*** Total elapsed time was 2 seconds.
UserName (Sum(CpuTime)*0.25)
------------------------------------------------
JANETJONES $882.90
JOHNSMITH
SAMOREILLY
WEEKLY
Grand Total:
$247.33
$102.77
$874.51
-------------------
$2,107.51
Charge Back and Overhead
From a CPU perspective charge back entails very little overhead. The accounting table is already being updated at the completion of each statement. The only cost is the creation of a new row in the table for each user each day. From a space perspective, the accounting table will grow by one row for each user each day. Periodic cleanup can constrain this growth.
Priority Scheduler
Priority Scheduler is a resource management tool that controls the dispersal of computer resources in Teradata Database. This resource management tool uses scheduler parameters that satisfy site-specific requirements and system parameters that depict the current activity level of Teradata Database. You can provide Priority Scheduler parameters to directly define a strategy for controlling computer resources.
You can use Priority Scheduler to:
•
Define a weighting system, based on user logon characteristics or Workload Definition
(WD) rules, which prioritizes the database workload.
Database Administration 515
Appendix C: Performance Management: All DBAs
Managing the Database Workload
•
To define scheduling parameters and to monitor your current system activity.
Note: If TASM workload definition (Category 3) is active, no Priority Scheduler modifications are allowed through schmon.
Design Goals
Priority Scheduler includes default parameters that provide four priority levels; all users are assigned to one level.
To take advantage of Priority Scheduler capabilities:
•
Assign users to one of the several default priority levels based on a priority strategy.
•
Define additional priority levels and assign users to them to provide a more sophisticated priority strategy.
•
Assign user who execute very response-sensitive work to a very high priority level to support Teradata Database applications.
For a description of the structure of and relationships between the scheduling components and parameters of Priority Scheduler, see “Priority Scheduler” in Utilities.
When setting up for Priority Scheduler, Teradata recommends:
•
A low number of active AGs are preferred. 5 or 6 is a good target.
•
1 or 2 Resource Partitions (RPs) to cover all user work.
•
A reasonably higher weight assigned to tactical query components, compared to those supporting other user work.
•
A demotion AG, if needed.
One possible RP setup is the following.
RP
Default
Tactical (optional)
Standard
Weight
40
40
20
Description
Console utilities
Highly-tuned tactical queries only
All nontactical user-assigned work
The recommended setup assumes that tactical queries are highly tuned and that they demonstrate the following characteristics: that all-AMP queries consume low levels of CPU per node.
Bringing together all PGs doing nontactical work into a single RP makes it:
•
Easier to understand priority differences among AGs.
•
Simpler to setup and tune.
•
Less complex when faced with growth.
•
Easier when several PGs to share a single AG.
516 Database Administration
Appendix C: Performance Management: All DBAs
Managing Database Resources to Enhance Performance
Managing Database Resources to Enhance
Performance
Managing these resources efficiently can enhance performance:
•
Space
•
Disk
•
Spool
•
PackDisk
•
FreeSpace FSP
•
Memory
•
Monitoring memory
•
Free Memory
•
Memory consuming features
•
FSGCache
•
Processing Resources
•
Canary queries
•
Monitor ResUsage data (Macros, Viewpoint)
•
DBC.AMPUsage
•
Compression
•
DataBlockSize
Performance and Space Management
Also see
Chapter 9: “Managing Space: Operational DBAs.”
Managing System Disk Space
System disk space is initially allocated as:
•
A spool space reserve, which guarantees a certain amount of space remains unused for data storage so it is available for query execution.
•
Permanent space assigned to a top level administrative user to contain other users, databases, and tables.
You should monitor disk space usage and set up alerts that track usage patterns, for the capacity planning purposes.
Managing Spool Space
Managing spool space allocation for users can be a method to control both space utilization and potentially bad (that is, non-optimized) queries.
Database Administration 517
Appendix C: Performance Management: All DBAs
Performance and Space Management
Spool Space and Perm Space
Spool space is allocated to a user. If several users are active under the same logon and one query is executed that exceeds the limit of the spool space allocated, all active queries for that user that require spool will likewise be denied additional spool and will be aborted.
If space is an issue, it is better to run out of spool space than to run out of permanent space. A user requesting additional permanent space will do so to execute queries that change tables
(inserts or updates, for example). Additional spool requests are almost always done to support a SELECT. Selects are not subject to rollback.
To configure this, see “Cylinders Saved for PERM” in Utilities.
Spool Space Accounting
Spool space for users is updated in the DatabaseSpace table. However, given the possibility of phantom spool, running UPDATESPACE is necessary to clear spool space for users that have logged off, whether they were aborted users or not. After UPDATESPACE has been run, spool space values in DBC.DatabaseSpace reflects actual spool usage.
Phantom spool cases are those in which the DBC.DatabaseSpace table indicates that there is spool, although no spool exists on the disk. Phantom spool cases are not the same as “left-over spool” cases. Left-over spool cases are those in which spool is actually created and exists on the disk, although a request completed its execution.
Using Spool Space as a Trip Wire
Lowering spool space may be a way to catch resource-intensive queries that will never finish or that will run the entire system out of free space if the user is allocated a very high spool space limit.
In the interest of system performance, do not allocate high spool space limits to all users and, in general, be very conservative in setting spool space limits.
Managing Cylinders
Teradata Database disk space is divided into cylinders, with each cylinder being made up of a number of data blocks, each of which contains one or more data rows. The database allocates cylinders as follows:
Usage
Contiguous sectors on a cylinder
Free cylinders
Description
datablocks are stored on adjacent sectors in a cylinder.
If a cylinder has 20 available sectors, but only 10 are contiguous, a 15sector block must be stored on another cylinder.
Teradata Database performs better if permanent data is distributed across multiple cylinders. However, permanent data and spool data cannot share the same cylinder.
Therefore, a system must always have empty cylinders that can be used for spool space.
518 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Performance and Space Management
Teradata Database does not run out of disk space until it allocates and fully utilizes all cylinders.
Low Cylinder Utilization
Performance degradations can occur, however, as soon as the system gets close to exhausting the free cylinder pool. This happens because the system performs MiniCylPacks on cylinders with low utilization in order to reclaim the unused disk space. Therefore, you should be aware if you are running out of space due to a preponderance of under-utilized cylinders.
Low utilization of cylinders can occur when:
•
You FastLoad a table using a small FreeSpacePercent (FSP) and then insert additional data to the table that is greater than the FSP.
•
You delete a significant percent of a table but have not yet run Ferret PACKDISK to reclaim the space.
Frequently Updated Tables
With frequently updated tables, the free space on the cylinder can become so fragmented that it cannot be used.
When this occurs, the system could allocate additional cylinders to the table. To avoid this problem, the system sometimes performs a cylinder defragmentation to make the free space on the cylinder usable again.
AutoCylPack
AutoCylPack (automatic background cylinder packing) manages space on cylinders, finding cylinders with low or high utilization and returning them to their user-defined FSP (Free
Space Percent), that is, the percentage of storage space within a cylinder that AutoCylPack leaves free of data to allow for future table growth.
There is a system wide default for AutoCylPack. It can be overridden on a table-by-table basis by specifying the FSP clause in a CREATE TABLE or ALTER TABLE statement.
It is important for Teradata Database file system to maintain enough free cylinders and to manage the space on the cylinders.
Although AutoCylPack runs as a background task issuing I/Os, you can adjust its level of impact. For the DBS Control fields that support AutoCylPack, see Utilities.
Performance
AutoCylPack helps reduce:
•
The chances that MiniCylPacks run. MiniCylPacks are strictly concerned with reclaiming whole cylinders.
•
The need for you to perform regular Ferret PACKDISK operations.
PACKDISK can be performed on a table, a range of tables or an entire system, but it affects performance of the foreground tasks and thus system performance.
519
Appendix C: Performance Management: All DBAs
Performance and Space Management
If cylinders have higher utilization:
•
System performance improves because there is a higher data block to cylinder index ratio and more effective use of Cylinder Read. There will be less unoccupied sectors read in.
•
A table occupies less cylinders. This leaves more cylinders available for other uses.
MiniCylPack
A MiniCylPack moves data blocks in logical sequence from cylinder to cylinder, stopping when the required number of free cylinders is available. A single MiniCylPack may affect two to 20 cylinders on an AMP.
The process continues until one cylinder is completely emptied. The master index begins the next required MiniCylPack at the location that the last MiniCylPack completed.
Teradata Database file system will start to MiniCylPack when the number of free cylinders drops to the value set by MiniCylPackLowCylProd. The default is 10.
The File Information Block (FIB) keeps a history of the last five cylinders allocated to avoid
MiniCylPacks on them.
Note: Spool files are never cylinder packed.
Use the DBS Control (see “MiniCylPackLowCylProd” in Utilities) to specify the free cylinder threshold that causes a MiniCylPack. If the system needs a free cylinder and none are available, a MiniCylPack occurs spontaneously.
2
3
Migrating Data Blocks
1
If space can be made available either by migrating blocks forward to the next cylinder or backwards to the previous cylinder, choose the direction that would require moving the fewest blocks.
If the number of blocks is the same, choose the direction of the cylinder with the most number of free sectors.
If Step 1 fails to free the desired sectors, try migrating blocks in the other direction.
If space can be made available only by allocating a new cylinder, allocate a new cylinder.
The preference is to add a new cylinder:
•
Before the current cylinder for permanent tables.
•
After the current cylinder for spool tables and while performing FastLoads.
When migrating either forward or backward, the number of blocks may vary because the system considers different blocks for migration.
Because of the restriction on key ranges within a cylinder, the system, when migrating backward, must move tables and rows with the lowest keys. When migrating forward, the system must move tables and rows with the largest keys.
The system follows special rules for migrating blocks between cylinders to cover special uses, such as sort and restore. There are minor variations of these special rules, such as migrating more data blocks than required in anticipation of additional needs, and looking for subtable breaks on a cylinder to decide how many data blocks to attempt to migrate.
520 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Performance and Space Management
Performance
Although cylinder packing itself has a small impact on performance, it often coincides with other performance impacting conditions or events. When Teradata Database file system performs a MiniCylPack, the operation frees exactly one cylinder.
The cylinder packing operation itself runs at the priority of the user whose job needed the free cylinder. The cylinder packing operation is the last step the system can take to recover space in order to perform a write operation, and it is a signal that the system is out of space.
Needing to pack cylinders may be a temporary condition in that a query, or group of queries, with very high spool usage consumes all available free space. This is not a desirable condition.
If space is a problem,
•
Enable AutoCylPack if you have not done so and specify an FSP of 0% for read-only tables using the CREATE TABLE or ALTER TABLE statement
•
Run the Ferret PACKDISK command.
Error Codes
MiniCylPacks are a natural occurrence and serve as a warning that the system may be running short on space. Tightly packed data can encourage future cylinder allocation, which in turn triggers more MiniCylPacks.
The system logs MiniCylPacks in the Software_Event_LogV with the following error codes.
Code
340514100
340514200
340514300
Description
Summary of MiniCylPacks done at threshold set via the DBS Control.
A MiniCylPack occurred during processing and a task was waiting for it to complete.
The system could not free cylinders using MiniCylPack. The MiniCylPack failed.
This means that the system is either getting too full or that the free cylinder threshold is set unreasonably high. Investigate this error code immediately.
Frequent 340514200 or 340514300 messages indicate that the configuration is under stress, often from large spool file requirements on all AMPs. MiniCylPacks tend to occur across all
AMPs until spool requirements subside. This impacts all running requests.
If table data is skewed, you might see MiniCylPacks even if Teradata Database has not used up most of the disk space.
Defragmentation
As random updates occur over time, empty gaps become scattered between data blocks on the cylinder. This is known as fragmentation. When a cylinder is fragmented, total free space may be sufficient for future updates, but the cylinder may not contain enough contiguous sectors to store a particular data block. This can cause cylinder migrates and even new cylinder allocations when new cylinders may be in short supply. To alleviate this problem, the file
521
Appendix C: Performance Management: All DBAs
Performance and Space Management system defragments a fragmented cylinder, which collects all free space into contiguous sectors.
Free
Cylinder
Data Block
Free Space
New Data
Block
KY01A018
Use the DBS Control (see “DefragLowCylProd” in Utilities) to specify the free cylinder threshold that causes defragmentation. When the system reaches this free cylinder threshold, it defragments cylinders as a background task.
Contiguous
Free Space
Data Blocks
Free
Cylinder
New Data
Block
KY01A019
To defragment a cylinder, the file system allocates a new cylinder and copies data from the fragmented cylinder to the new one. The old cylinder eventually becomes free, resulting in a defragmented cylinder with no change in the number of available free cylinders.
Since the copy is done in order, this results in the new cylinder having a single, free-sector entry that describes all the free sectors on the cylinder. New sector requests on this cylinder are completed successfully, whereas before they may have failed.
Ferret Defragmentation
The Ferret Defrag command is a defragmentation command. It can be used to scan the entire set of user cylinders and then to defragment the qualified cylinders.
Running Ferret Defrag command, however on a busy customer system affects the performance of the foreground tasks and thus system performance. Moreover, Ferret Defrag has no way to know if there are enough free cylinders already available in the system, so that no more defragmentation is required.
522 Database Administration
Appendix C: Performance Management: All DBAs
Performance and Space Management
Performance Degradation
While AutoCylPacks, MiniCylPacks and defragmentation help the system reclaim free disk space for further use, they incur a performance degradation. Properly size and tune the system to avoid this overhead.
Freeing Space on Cylinders
In addition to MiniCylPack and AutoCylPack, you can use the following to free space on cylinders or to make more cylinders available to the system:
•
FreeSpacePercent (FSP) (see “PACKDISK and FreeSpacePercent” on page 527 )
•
PACKDISK (see Utilities)
•
Cylinders to Save for Perm (see “Cylinders Saved for PERM” in Utilities)
•
Temporary space limits on users/profiles.
•
Spool space limits on user profile definitions:
•
Set larger spool space limits.
•
Minimize the number for individual users to limit runaway query spool space.
Both of the above will not necessarily stop space management routines such as
MiniCylPack from running, but it could help manage the area.
•
Archival and deletion of aged data
Planned deletion of obsolete rows facilitates space availability. Depending on the nature of your data, you may want to archive before deleting. For more information, see Teradata
Archive/Recovery Utility Reference.
•
Appropriate data compression (see “Data Compression and Performance” on page 529 )
•
Additional disk space
•
Appropriate data block sizing:
•
Maximum block size allocation
•
Minimum block size allocation
•
Journal data block size allocation
Managing I/O with Cylinder Read
Cylinder Read is a method of loading data blocks in a single database I/O operation. Instead of block I/Os for operations such as table scans and joins that process most or all of the data blocks of a table, Cylinder Read issues an I/O of up to 2 MB.
Because Cylinder Read loads the desired data blocks in a single database I/O operation,
Teradata Database incurs I/O overhead only once per cylinder rather than for each data block.
Cylinder Read and I/O
With larger block sizes, it is possible that the frequency of Cylinder Reads will decrease, particularly for moderate or smaller tables. This may increase block-at-a-time I/O requests, driving up I/O demand.
Database Administration 523
Appendix C: Performance Management: All DBAs
Performance and Space Management
To be a candidate for Cylinder Read on an AMP, the cylinder must contain 6 or more data blocks for the table being accessed. When you increase the block size drastically, more rows fit into each data block and fewer data blocks are needed to hold the rows of the table on each
AMP. For large tables this may not be a problem, but you may end up doing more I/O operations on moderate or smaller tables.
Cylinder Read allows you to keep smaller block sizes, while still having the advantage of reading many rows in one I/O operation when scanning.
Using Cylinder Read for WAL Log Maintenance
Setting Cylinder Read to LogOnly using ctl limits cylinder scan to WAL log maintenance only.
With LogOnly, the average size of the WAL log should be reduced. Table scan jobs may run slower than before because individual data blocks are read rather than whole cylinders.
Setting Cylinder Read to LogOnly may reintroduce some Cylinder Read performance anomalies that may have caused you to disable Cylinder Read in the first place. But completely disabling Cylinder Read can cause WAL log maintenance to fall behind, thereby causing a general system slowdown.
But by setting Cylinder Read to LogOnly, users only use Cylinder Read for WAL log maintenance and not for table scans, and because WAL log maintenance only runs in the background once a minute, the number of anomalies should be less than those you may have experienced when Cylinder Read was fully enabled.
Tracking Cylinder Read Resource Usage
To track Cylinder Read behavior if you enable resource usage logging, see the Cylinder Read
Columns in the “ResUsageSvpr Table” in Resource Usage Macros and Tables.
Managing Free Space
Evaluating FreeSpacePercent
Reserved free space allows tables to expand within their currently allocated cylinders. This can prevent or delay the need for additional cylinders to be allocated, which incurs the overhead of moving data to the new cylinders. Avoiding new cylinder allocations can improve overall system performance.
Choosing an appropriate FSP value for a table involves both performance and space considerations, and depends on the growth needs of the table:
•
Reference tables that experience no modifications or growth require no FSP, so FSP can be zero for these types of tables. If the system is primarily these types of tables, set
FreeSpacePercent to zero, and use the FREESPACE option of the CREATE TABLE and
ALTER TABLE SQL statements to set a different FSP value for tables that will experience growth.
•
Tables that are expected to experience large scale growth require higher FSP values than tables that grow to lesser degrees. However, larger FSP values consume more storage, so
FSP choice should balance space and performance considerations.
524 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Performance and Space Management
A table that would require 100 cylinders of storage with 0% FSP, requires 134 cylinders when
FSP is set to 25%. If FSP is 75%, that same table would require 400 cylinders. Ensure that the requisite cylinders are available to satisfy the storage overhead added by the FSP, or performance can suffer.
With time, the available free space may change, due to table modifications that do not honor the FreeSpacePercent setting. The AutoCylPack background task runs periodically to check and restore the FSP for tables. The Ferret PACKDISK command can be run manually to force
FSP to be restored, or to temporarily set a table FSP to a different value. MiniCylPack may change the available free space if there is a shortage of cylinders on the system.
Effects of FreeSpacePercent
FreeSpacePercent (FSP) is a system-wide parameter. FSP does not override a value you specify for an individual table in a CREATE or ALTER TABLE request.
MiniCylPack operations attempt to honor the FSP specified in a CREATE TABLE and ALTER
TABLE statements or, if no table-specific FSP is specified, the system default FSP. If
MiniCylPack cannot reclaim space while honoring those values, it will keep trying to use a degraded (smaller FSP) value until space can be reclaimed. This continues until MiniCylPack attempts to use an FSP value of 0 for all cylinders.
In some situations, Teradata Database runs out of free cylinders even though over 20% of the permanent disk storage space is available. This is due to a:
•
Higher FSP setting than necessary, which causes the system to allocate unneeded space
•
Lower FSP setting than necessary, which causes the system to over-allocate new cylinders
•
Low storage density (utilization) on large tables due to cylinder splits
Operations Honoring the FSP
When adding rows to a table, the file system can choose either to use 100% of the storage cylinders available or to honor the FSP. The following operations honor FSP:
•
FastLoad
•
MultiLoad into empty tables
•
Restore
•
Table Rebuild
•
Reconfiguration
•
SQL to add fallback
•
SQL to create an SI
Operations that Disregard FSP
The following operations disregard FSP:
•
SQL inserts and updates
•
Tpump
•
MultiLoad inserts or updates to populated tables
525
Appendix C: Performance Management: All DBAs
Performance and Space Management
If your system is tightly packed and you want to apply or reapply FSP, you can:
•
Specify the IMMEDIATE clause with the ALTER TABLE statement on your largest tables.
•
DROP your largest tables and FastLoad them.
•
DUMP your largest tables and RESTORE them.
•
In Ferret, set the SCOPE to TABLE and PACKDISK FSP = xxxx
In each case, table re-creation uses utilities that honor the FSP value and fills cylinders to the
FSP in effect. These options are only viable if you have the time window in which to accomplish the processing. Consider the following guidelines:
•
If READ ONLY data, pack tightly (0%).
•
For INSERTs:
•
Estimate growth percentage to get FSP. Add 5% for skewing.
•
After initial growth, FSP has no impact.
•
Reapply FSP with DROP/FASTLOAD, DUMP/RESTORE or PACKDISK operations.
•
Experiment with different FSP values before adding nodes or drives.
Determining a Value for FSP
Use the data in the following table to determine a value for FSP.
IF the majority of tables are…
read-only
NOT read-only
THEN…
set the default system-wide FSP value to 0.
You can override the FSP for the remaining modifiable tables on an individual table basis with the ALTER TABLE statement.
the FSP value depends on the percentage of increase in table size due to the added rows.
Set the FreeSpacePercent parameter to reflect the net growth rate of the data tables (inserts minus deletes). Common settings are 5 to 15%. A value of 0% is appropriate for tables that are not expected to grow after initially being loaded.
For example, if the system keeps a history for 13 weeks, and adds data daily before purging a trailing 14th week, use an FSP of at least 1/13 (8%).
To accommodate minor data skews and any increase in weekly volume, you can add an extra 5% (for a total of 13%) FSP.
Because the system dynamically allocates free cylinder space for storage of new or updated rows, leaving space for this during the initial load allows a table to expand with minimal cylinder splits and migrates. However, if you do not expect table expansion, that is, the majority of tables are read-only, use the lowest value (0%) for FSP.
If the system default FSP is zero, do as shown in the following table to minimize problems.
526 Database Administration
Appendix C: Performance Management: All DBAs
Performance and Space Management
IF you…
use read-only tables add data via
BTEQ or a CLI program
THEN…
change nothing.
At load time, or PACKDISK time, the system stores tables at maximum density.
set the FREESPACE value on the CREATE TABLE statement to an appropriate value before loading the table.
If the table is loaded, use the ALTER TABLE statement to change FREESPACE to an appropriate value before running PACKDISK.
If you set FSP to a value other than 0, tables are forced to occupy more cylinders than necessary. The extra space is not reclaimed until either you insert rows into the table, use the
Ferret utility to initiate PACKDISK on a table, or until MiniCylPack is performed due to a lack of free cylinders.
When the system default FSP is greater than 0, use the information in the following table to minimize problems.
IF you… THEN…
use read-only tables set FREESPACE on the CREATE TABLE statement to 0 before loading the table.
If the table is loaded, use the ALTER TABLE statement to change FREESPACE to 0 before running PACKDISK.
add data via BTEQ or a CLI program change nothing.
The system adds rows at maximum density.
Adjusting FREESPACE for a Table
You can specify the default value for free space left on a cylinder during certain operations using the FREESPACE option in the CREATE TABLE and ALTER TABLE statements.
You can select a different value for tables that are constantly modified versus tables that are only read after they are loaded. To specify the global free space value, use FreeSpacePercent
(see “FreeSpacePercent” in Utilities).
PACKDISK and FreeSpacePercent
When Teradata Database runs out of free cylinders, you can run Ferret PACKDISK, an expensive overhead operation, to compact data to free up more cylinders.
To reduce the frequency of PACKDISK operations:
•
When FastLoading tables to which rows will be subsequently added, set FSP to 5-20% to provide enough free space to add rows.
•
For historical data, when adding and deleting data, provide enough free space to add rows.
For example, you add up to 31 days before deleting on a table with six months history.
Database Administration 527
Appendix C: Performance Management: All DBAs
Performance and Space Management
•
Add one month to six months: 1/7 = 14.3%
•
Safety - plan on 1.5 months, 1.5 / 7.5 = 20%
Set Free Space Percent to 20%.
•
For historical data and fragmented cylinders:
•
For large tables, either set FSP to 20 - 35%, or set MaxBlockSize to a smaller size (for example, 16 KB).
•
Translate free space to the number of data blocks; at least 6-12 blocks of free space.
•
Specify the IMMEDIATE clause with the ALTER TABLE statement.
The table header contains the FSP for each table. If you change the default FSP, the system uses the new default the next time you modify the table. FSP has no effect on block size.
Running Other Utilities with PACKDISK
If you run PACKDISK frequently, use the following tools, two of which are utilities, to determine the amount of free space:
•
DBC.DiskSpaceV
•
SHOWSPACE, a Ferret command, shows you the percent of free space per cylinder.
If this figure is low, it will impact performance by performing dynamic cylinder packs when the system needs contiguous space.
•
SHOWFSP, a Ferret command like SHOWSPACE, helps find tables that need packing.
SHOWFSP shows the number of cylinders that can be freed up for individual tables by specifying a desired free space percent, and is useful in discovering which tables would free the most cylinders if PACKDISK were run on them
Cylinder Splits
A FreeSpacePercent value of 0% indicates that no empty space is reserved on Teradata
Database disk cylinders for future growth when a new table is loaded. That is, the current setting causes each data cylinder to be packed 100% full when a new table is loaded.
Unless data is deleted from the table prior to subsequent row inserts, the 0% value guarantees a cylinder split the first time that a row is inserted into the table (following the initial load).
Cylinder splits consume system I/O overhead and often result in poor use of data cylinders.
PACKDISK, FSP, and Cylinder Allocations
Packing data for tables being modified too tightly can result in too many cylinder allocations, which reduces the number of free cylinders. For example, if the data on a table occupying 10 cylinders is packed to 0% free space, inserting one row on each cylinder (for a total of only 10 rows) may require Teradata Database to allocate up to 10 additional new cylinders.
However, if you run PACKDISK on the table with an FSP of 10%, the original table would occupy 11 cylinders. When the same 10 rows are inserted into this table, there will most likely be space available on the existing cylinders and the table would remain, consuming only 11 cylinders. This leaves the other 9 cylinders that were previously used free and available for future cylinder requests. Another way to accomplish the same goal is to update the table first and then perform the PACKDISK operation to free up any cylinders that are not required.
528 Database Administration
Appendix C: Performance Management: All DBAs
Performance and Space Management
Running the PACKDISK command in this example saved 9 free cylinders, but required manual intervention. Another way to accomplished this is to specify an FSP of 10% in the
CREATE TABLE or ALTER TABLE statement, and then let AutoCylPack adjust the table as part of its background activity. If new rows are added to the table in this example on a regular basis, then the AutoCylPack solution is preferable. If new rows are rarely added, then it is better to leave the table packed with 0% free space and run a PACKDISK when required.
Data Compression and Performance
Compression is useful to free up additional disk space, but it may or may not enhance performance depending on the type of compression used, the workload, the frequency of data access, and the system capabilities. Compression generally affects performance as follows:
•
They cycle of decompressing and recompressing required to access and store data may significantly increase CPU usage compared to uncompressed data.
•
Storing data in dense, compressed form may reduce I/O requirements.
See “Compression” in Chapter 4: “Space.”
Multivalue Compression (MVC)
MVC compresses recurring values within a column into a single value in the table header.
Performance Impact
MVC enhances the performance of high data volume applications, like call record detail and click-stream data, and provides significant performance improvement for general ad hoc workloads and full-table scan applications
Smaller physical row size results in less data blocks and fewer I/Os and improved overall performance, depending upon amount of compression achieved.
Select and delete operations show a proportionate improvement in all cases. Inserts and updates show mixed results. The load utilities benefit from the compressed values.
Algorithmic Compression (ALC)
ALC includes:
•
Teradata-supplied UDFs to compress various types of character data
•
The ability to create custom UDF algorithms
Performance Impact
Teradata-standard ALC UDFs tend to reduce I/O, but can increase CPU usage.
Database Administration 529
Appendix C: Performance Management: All DBAs
Performance and Space Management
Algorithm
•
CAMSET and
DECAMSET (Unicode)
•
CAMSET_L and
DECAMSET_L (Latin)
•
•
LZCOMP and
LZDECOMP (Unicode)
LZCOMP_L and
LZDECOMP_L (Latin)
•
TRANSUNICODETOUTF8
•
TRANSUTF8TOUNICODE
Performance Effects
Relatively low CPU cost for small and medium column widths
(higher than UTF-8, lower than ZLIB BLC), but high cost for large column width.
The decompression cost increases more quickly than UTF-8 and
ZLIB as the column width grows, but for short and medium length columns, the difference is not very significant
LZCOMP requires high CPU usage for compression, many times the CPU cost of TRANSUNICODETOUTF8.
The decompression cost is generally low; somewhat higher than
TRANSUNICODETOUTF8 for short and medium length strings, but somewhat lower than TRANSUNICODETOUTF8 for long strings.
Low CPU cost for both compression and decompression.
Block-Level Compression
BLC compresses data at the data block level.
Because BLC is CPU-intensive, consider the CPU capabilities of the particular Teradata
Database platform when using BLC.
When using BLC on the Teradata appliance systems with very high CPU capabilities, you can:
•
Apply BLC to all allowable types of data.
•
Load and access data at any time because there is enough available CPU to frequently decompress data
When using BLC on other platforms with less CPU capacity, you may need to:
•
Load tables during off-peak hours.
•
Limit access to compressed tables during critical throughput periods.
Some operations (including queries, insert/updates, ARC dump/restores, reconfig, and
CheckTable) can use considerably more CPU while operating on compressed tables. Unless the system is very CPU rich, these operations can impact other workloads and lengthen elapsed response times
Managing Data Block Usage
Global settings for data block size are controlled using DBS Control utility settings:
•
PermDBAllocUnit
•
PermDBSize
You can also adjust the data block size for individual tables using the DATABLOCKSIZE and
MERGEBLOCKRATIO options in CREATE TABLE or ALTER TABLE statements.
530 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Performance and Space Management
PermDBAllocUnit
As tables are modified, rows are added, deleted, and changed. Data blocks grow and shrink dynamically to accommodate their current contents. However, data block sizes can change only in units of PermDBAllocUnit. This means there will nearly always be some unused space left at the end of the data block. If table modifications are relatively even, such incremental changes in data block size result in an average of approximately half an allocation unit of space wasted for every data block. (This is a rough approximation, and will depend on many factors that differ from database to database.)
In environments where new rows are added frequently to tables, or where tables with variable length rows are frequently growing, system performance might be improved slightly by increasing the allocation unit size. With a larger allocation unit, data blocks will not need to be enlarged as frequently, because there will already be room for additional changes. However, in environments where new rows are not added frequently, the additional space in each block can degrade performance by increasing the average I/O size.
Make only small changes to this setting at a time, and carefully evaluate the results before committing the change on a production system. Set the allocation unit to a multiple of the average row size of tables that change frequently, rounded up to the nearest sector.
Because the benefit of larger allocation units is often offset by the consequent increase in average wasted space, Teradata recommends that PermDBAllocUnit be left at the default setting.
PermDBSize
Database performance can be affected by the relationship of data block size to the type of work typically performed by the database:
•
When database queries are tactical in nature, involving one or a few table rows, it is advantageous to have fewer rows stored per data block to speed data access. Online transaction processing (OLTP) is an example of this type of work.
•
Alternatively, when database queries are strategic in nature, involving complex queries that involve many table rows per table, it is advantageous to have many rows stored in each data block, to minimize costly data I/O operations. Decision support software (DSS) and complex report generation are examples of this type of work.
PermDBSize sets the default maximum size used by the system for multirow data blocks in permanent tables. Use a larger value if the database is used primarily for strategic work, and a smaller value if the database is used primarily for tactical work.
In a mixed-work environment, determine a value for PermDBSize based on the kind of work typically performed by the database. For tables involved in other types of work, PermDBSize can be overridden on a table-by-table basis using the DATABLOCKSIZE option of the
CREATE TABLE and ALTER TABLE SQL statements.
For example, if only a few tables are involved in decision support, overall throughput can be improved by up to 50% using the following strategy:
•
Define a data block size of 254 sectors for the decision support tables.
531
Appendix C: Performance Management: All DBAs
Performance and Space Management
•
Set PermDBSize to 42, to serve as the default block size for the remaining tables, which are involved in tactical work.
•
Use the smallest value for JournalDBSize, to expedite UPDATE operations.
Adjusting DATABLOCKSIZE in Tables
You can control the default size for multirow data blocks for individual tables using the:
•
DATABLOCKSIZE option
•
MERGEBLOCKRATIO option in the CREATE TABLE and ALTER TABLE statements, as follows.
IF you specify…
DATABLOCKSIZE in
CREATE TABLE
DATABLOCKSIZE in
ALTER TABLE
MERGEBLOCKRATIO in
CREATE TABLE
MERGEBLOCKRATIO in
ALTER TABLE the IMMEDIATE clause
THEN…
the datablock can grow to the size specified in DATABLOCKSIZE instead of being limited to the global PermDBSize.
For any row, the system uses only the datablock size required to contain the row.
the datablocks can grow to the size specified in DATABLOCKSIZE where the row size requires the growth.
Whether they are adjusted to that new size gradually over a long period of time depends on the use of the IMMEDIATE clause.
it limits attempts to combine blocks if the resulting size is larger than the specified percent of the maximum multirow datablock size. the size of the resulting block when multiple existing blocks are being merged has a upper limit.
The limit depends on whether logically adjacent blocks are deemed mergeable with the single block being modified.
Blocks can still be initially loaded at the PermBDSize specification or the block size specified with the DATABLOCK option. Merges occur only during full table modifications.
MERGEBLOCKRATIO and the default size for multirow datablocks are unrelated.
the rows in all existing datablocks of the table are repacked into blocks using the newly specified size. For large tables, this can be a timeconsuming operation, requiring spool to accommodate two copies of the table while it is being rebuilt.
If you do not specify the IMMEDIATE clause, existing datablocks are not modified. As individual datablocks of the table are modified as a result of user transactions, the new value of DATABLOCKSIZE is used. Thus, the table changes over time to reflect the new block size.
To evaluate actual block sizes of tables, you can run the SHOWBLOCKS command of the
Ferret utility. For more information on running this command, see Utilities. To specify the global data block size, use PermDBSize (see “PermDBSize” in Utilities).
532 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Performance and Space Management
You can also use SQL macros to generate SHOWBLOCKS-like information. The
PopulateFsysInfoTable macro generates file system data to a table created by the
CreateFsysInfoTable macro. The PopulateFsysInfoTable_ANSI macro generates file system data to a table created by the CreateFsysInfoTable_ANSI macro. For more information, see the chapter on file system information macros in SQL Functions, Operators, Expressions, and
Predicates.
Specifying Maximum Data Block Size
You can set maximum block size two ways.
Operation
Set PermDBSize via the
DBS Control
Use the CREATE or
ALTER TABLE command
Comments
When you set maximum block size at the system level, a table utilizes a value only until a new value is set system-wide.
When you set maximum block size at the table level, this value remains the same until you execute an ALTER TABLE command to change it.
Larger block sizes enhance full-table scan operations by selecting more rows in a single I/O.
The goal for DSS is to minimize the number of I/O operations, thus reducing the overall time spent on transactions.
Smaller block sizes are best used on transaction-oriented systems to minimize overhead by only retrieving what is needed.
For more information on data block size, see “PermDBSize” in Utilities.Rows cannot cross block boundaries. If an INSERT or UPDATE causes a block to expand beyond the defined maximum block size, the system splits the block into two or three blocks depending on the following.
IF…
the new or changed row belongs in the beginning or end of a block the new or changed row belongs in the middle of a block
AND…
a block containing only that row is larger than the defined maximum block size for the table a block containing only that row would be larger than the defined maximum size for the table
THEN…
the row is placed in a block by itself.
then a three-way block split is performed.
The existing block is split into two blocks at the point where the row being modified belongs and a new block is created between them containing only the modified row.
533
Appendix C: Performance Management: All DBAs
Performance and Space Management
IF…
the existing block size can be changed to accommodate modification and still not be larger than the defined maximum size for the table the existing block size cannot be changed to accommodate modification and still not be larger than the define maximum size for the table
AND…
case is empty case is empty
THEN…
a single new block with the existing rows and the new or changed row is created.
the modification is applied and then the block is split into as many parts as required (and as few parts as possible) so that each part is not larger than the defined maximum for the table.
Additional special rules exist that take precedence over the preceding rules. For example:
•
Rows of different subtables never coexist in data blocks.
•
Spool tables are almost always created as whole blocks all at once with many rows in them with the maximum size defined for spool (as long as there are enough rows).
Using Data Block Merge
The Datablock Merge operation reduces the number of small data blocks in table(s) by combining them with logically adjacent data blocks. It enables the Teradata file system to automatically merging smaller data blocks into a single large data block. Having fewer blocks reduces the number of I/O operations required to read and modify a large set of rows from disk. The Merge:
•
Runs automatically.
•
Does not require manual investigation of block size histograms.
•
Does not require any specific AMP level locking.
•
Searches continuously for small blocks to merge, even if some of those blocks do not require updates.
•
Only affects the performance of workloads that modify data. Read-only workloads, insert operations into new subtables, and prime key modification requests are not affected.
Moreover, Datablock Merge is only applicable to permanent and permanent journal tables, not to global temporary or volatile tables.
For the DBS Control fields that support datablock merge, see Utilities.
For the option, MergeBlockRatio, which defines the size of the resulting block when multiple existing blocks are being merged, see “CREATE TABLE” and “ALTER TABLE” in
SQL Data Definition
Language Syntax and Examples
.
Effects of Journal Data Block Size
Journal data block sizes may affect I/O usage. A larger journal size may result in less I/O or cause wasted datablock space. You can set the JournalDBSize. See Utilities.
534 Database Administration
Appendix C: Performance Management: All DBAs
Performance and Memory Management
Performance and Memory Management
About Shared Memory
The system determines how memory is shared between the OS and FSGCache. For example, the following figure illustrates how the system calculates FSG Cache for a Teradata Database system with 16 GB shared memory.
Database Administration
3020 MB
Linux - 500 MB
28 Vprocs @
90 MB each
FSG
Cache%
100%
(8687 MB)
65%
FSG
Cache
13364 MB
OS
Managed
FSG
Managed
Total Memory Size 16 GB (16384 MB)
1097A007
When Linux boots, PDE:
•
Determines how much memory the OS is already using.
In the example, memory is shown to be 500 MB. This number may vary depending on the system configuration.
•
Leaves to the OS an amount of memory equal to the number of vprocs times the memory allocated to each vproc.
In the example, this amount is 90 MB for each of the 28 vprocs.
Note: Vprocs include AMPs, PEs, the Gateway, and the Node vproc.
•
Calculates the FSG Cache by (1) subtracting the OS-managed memory from total memory size and then (2) applying a FSG Cache% to that amount.
Note: To calculate the FSG Cache per AMP, divide the FSG Cache by the number of AMPs.
The amount of OS-managed memory is 3020 MB and the FSG Cache, if the FSG Cache% is 100%, would be 13364 MB.
If the FSG Cache% were set to 65%, then the FSG Cache would be 8687 MB, and the additional free memory would be 13364 MB minus 8687 MB, or 4677 MB.
535
Appendix C: Performance Management: All DBAs
Performance and Memory Management
If you want to run applications with memory requirements unknown to Teradata
Database software, reduce the FSG Cache% to leave memory for the applications.
Note: FSG Cache is managed in multiples of 4 KB up to 128 KB, and then in increments of
256 KB, 512 KB, and 1 MB.
Reserving Additional Free Memory for Applications
To reserve 20% of the FSG Cache for applications over and above the 90 MB/vproc reserved for the OS, go to the DBS screen and set FSG Cache percent to 80. The system assigns 80% of the FSG Cache to FSG and leaves the remaining 20% for other applications.
For information on ctl, see Utilities.
About Memory-Consuming Features
Certain features may require more memory in order to show their optimal performance benefit. Of particular mention are:
•
Array INSERT
•
Compression
•
Cylinder read
•
Geospatial data type
•
Global and Persistent Data (GLOP)
•
Hash joins and product joins
•
Join index
•
Large objects (LOBs)
•
Row and column partitioning
•
Table functions
•
Stored procedures
•
User-defined functions (UDFs)
•
XML data type
•
1 MB response buffer
•
1 MB data blocks
•
Larger than 1 MB plan cache
Performance gains from implementing these features may be countered by their impact on memory usage. In turn, you may experience more segment swaps and incur additional swap physical disk I/O. To counter this, you can lower the FSG cache percent to ensure that 135 MB per AMP is allocated in OS memory.
Lowering the FSG cache percent may cause fewer cache hits on table data and instead cause a different type of additional physical disk I/O. In general, additional I/O on table data is a lesser performance issue than swapping I/O, but it can still have an impact on performance.
536 Database Administration
Appendix C: Performance Management: All DBAs
Performance and Memory Management
Investigating the Need to Add Memory
There is a recommended amount of memory for each Teradata Database hardware platform.
See the Release Definition for your Teradata Database release for memory recommendations.
1
2
You can also monitor the use of FSG cache memory to determine if you should add more
memory to assure full performance. See also “Managing FSG Cache” on page 540 .
3
Monitor your system during critical times to determine the ratio of logical to physical I/O.
After the lowering of the FSG cache percent to provide more memory for a memory consuming feature, again monitor your system during critical windows to understand the new ratio of logical to physical I/Os.
If the amount of FSG cache misses increases by more than 20% and the system has become
I/O-bound, then adding more memory, if possible, is recommended.
ResUsage and Cache Hit Rates
ResUsage data can help determine cache hit rates by comparing total logical I/O requests to I/
O requests that resulted in a physical I/O. Comparing these helps clarify any confusion that may arise when aligning DBQL data with ResUsage data, since DBQL data only tracks logical
I/O.
An analogous circumstance occurs when attempting to differentiate between total BYNET message requests and BYNET message requests that result in a physical BYNET message being sent. Smaller systems see a greater difference between the two because of a higher percentage of point-to-point (PTP) messages with the sender and the receiver being the same node.
In ResUsageSpma table, a logical read count, that is, a count of the total number of requested disk segments for the interval, is represented in the FileAcqs column. The physical read count, or the number of requested disk segments that actually resulted in a read by the I/O subsystem, is represented in the AcqReads and the PreReads columns.
There are similar breakdowns for writes. BYNET logical messages are represented in the columns that begin with Msg, and the physical counterparts are represented with the columns that begin with NetMsg.
Monitoring Memory
Use the ResUsage tables to obtain records of free memory usage and FSG Cache.
Memory Type
Free Memory
FSG Cache
Monitor With
ResUsage
ResUsageSpma
ResUsageSvpr
Comments
See Resource Usage Macros and Tables for more information on using ResUsage macros and utilities to monitor free memory and FSG Cache.
Database Administration 537
Appendix C: Performance Management: All DBAs
Performance and Memory Management
Using Memory Effectively
1
2
Regardless of the amount of available memory on your system, you need to use it effectively.
To determine if your system is using memory effectively:
3
Start with a value for FSG Cache percent. See
Consider adjusting other memory-related settings to optimize database function.
•
AggrCacheSize
•
DBSCacheThr
•
DictionaryCacheSize
•
HTMemAlloc
•
IdCol Batch Size
•
PPICasheThrP
•
RedistBufSize
•
SyncScanCacheThr
“Managing FSG Cache” on page 540 .
Adjust the value based on available free memory requirements. See “Reserving Additional
Free Memory for Applications” on page 536 and
“Managing Free Memory” on page 538
.
For information on how to adjust DBS Control memory settings, see Utilities.
Managing Free Memory
Free memory is used by:
•
Administrative programs, such as program text and data, message buffers, kernel resources, and other applications such as FastLoad that require memory use.
•
Vprocs for non-file system activity, such as:
Activity
AWT
Parser tasks
Dispatcher tasks
Scratch segments
Messages
Dictionary cache
Request cache
Description
Pieces of AMP logic used for specific AMP tasks
Pieces of PE logic responsible for parsing SQL
Pieces of PE logic responsible for dispatching work
Temporary work space
Communication between vprocs
Dictionary steps for parsing
Temporary space used when executing steps
Determining Available Free Memory
The ResNode macro displays limited information on free memory. The ResNode report includes the Free Mem% column, which is the percentage of unused memory.
538 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Performance and Memory Management
Adjusting for Low Available Free Memory
When the amount of available free memory dips too far below 100 MB (25,000 pages), some sites have experienced issues. 40 MB of free memory is the lowest acceptable amount. You can usually avoid problems if you configure your AMPs to have at least 135 MB of OS-managed memory per AMP. You can adjust the amount of available free memory by performing the following:
•
Use ctl to adjust the FSG Cache percent downward to make more memory available to free memory. If the system takes too much memory for FSG Cache and the OS does not use that memory, the free memory is wasted.
•
If available free memory goes below 100 MB during heavy periods of redistribution, lower the value of the RedistBufSize field in the DBS Control Record (see “RedistBufSize” in
Utilities).
Assured Minimum Non-FSG Cache Size
Teradata recommends these guidelines for minimum non-FSG Cache size per AMP.
•
135 MB per AMP when all nodes are up.
•
112 MB per AMP when 1 node is down in a clique.
•
90 MB per AMP when the maximum number of nodes allowed down are down in a clique.
The above configuration guidelines help avoid performance issues with respect to memory swaps and paging, memory depletion, and CPU starvation when memory is stressed.
Performance Management Recommendations
Many sites may require more free memory than the default calculated on “About Shared
. Teradata recommends that you provide additional memory per AMP to free memory by setting the FSG Cache percent to a value less than 100%.
Use the following calculation:
FSG Cache percent = (FSG Cache - 39 MB * # AMPs) / FSG Cache
Memory Size
(GB)
6.0
8.0
Memory for Baseboard
Drivers & 10 AMPs/2PE
Vprocs
1854
1914
FSG Cache (MB)
4290
6278
Less 58 MB per
AMP Vprocs
3822
5810
FSG Cache
Percent
89%
92%
For large configurations, consider one or more of the following options to resolve I/O bottlenecks or excessive memory swapping:
•
Consider using aggregate join indexes to reduce calculations during query processing.
•
Set RedistBufSize to an incremental value lower; for example, from 4 KB to 3 KB
•
Set FSG Cache percent to less than 100%, taking into account total memory size.
•
Consider modifying the application to reduce row redistribution.
•
Ask your support representative to reduce the internal redistribution buffer size.
539
Appendix C: Performance Management: All DBAs
Performance and Memory Management
Note: This is an internal tuning parameter, not the user-tunable RedistBufSize.
Potential Problems
A possible problem is when an application on a large configuration generates many messages over the BYNET with concurrent row redistributions involving all nodes.
The following are not a problem:
•
Row duplications
•
Merging of answer sets
Row Redistribution Memory Requirement
To avoid the overhead of sending lots of little messages across the BYNET, buffers are used to batch up individual rows during the row redistribution process. Both load utilities and queries involve such redistribution, but their approach to outbound buffering is different.
Row redistribution for query processing uses separate single buffers per AMP for each node in the system. This means that the amount of memory required for redistribution in a node grows as the system grows.
The DBC Control Record RedistBufSize field controls the size of redistribution buffers. See
.
•
Default query redistribution buffer size = 32 KB per target node
•
Total memory for one sending AMP = 32 KB * number of nodes in system
•
For eight AMPs per node, total memory required per node =
8 * 32 KB * number of nodes in system
Redistribution Buffer Resizing
The following example provides the calculations for resizing the redistribution buffer based on a configuration of 8 nodes at eight AMPs per node. (The system reserves 32MB per AMP).
•
Single node requirement (single user) = 32 KB * 8 = 256 KB
•
Multi-user (20 concurrent users) = 20 * 256 KB = 5 MB (less than the 32 MB default)
The following example provides the calculations for a configuration of 96 nodes at 8 AMPs per node and shows that the requirement exceeds the system default:
•
Single node requirement (single user) = 32 KB * 96 = 3072 KB (3 MB)
•
Multi-user (20 concurrent users) = 20 * 3072 KB = 64 MB (far exceeding 32 MB per AMP)
Performance effects of high-volume redistribution processing include:
•
Excessive memory paging/swapping
•
Possible I/O bottleneck on BYNET I/O
Managing FSG Cache
System Usage of FSGCache
Teradata Database file system manages FSG Cache, which is used by:
540 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Performance and Memory Management
•
AMPs on the node
•
Backup activity for AMPs on other nodes
Teradata Database file system uses FSG Cache for file system segments such as:
•
Permanent data blocks (includes fallback data and SIs)
•
Cylinder Indexes for permanent data blocks
•
Cylinder statistics for Cylinder Read
•
Spool data blocks and Cylinder Indexes for spool
•
WAL space, including Transient Journal (TJ) rows and WAL REDO records
•
Recovery journal rows
Space in FSG Cache
Space in the FSG Cache is not necessarily evenly distributed among AMPs. It is more like a pool of memory; each AMP uses what it needs.
FSG cache contains the most recently used database segments. When Teradata Database needs to read a data block, it checks and reads from cache instead of from disk, whenever possible.
The system performs optimally when FSG Cache is as large as possible, but not so large that not enough memory exists for the database programs, scratch segments, and other operating system programs that run on the node.
Calculating FSG Cache Size Requirements
The FSG Cache percent field controls the percentage of memory to be allocated to FSG Cache.
You can change the value in FSG Cache percent using the ctl utility. See the section on setting variables in “Control GDO Editor (ctl)” in Utilities.
First, configure sufficient operating system memory, using the guidelines discussed in
“Managing Free Memory” on page 538 . Then let the remaining memory be allocated to FSG
Cache.
Calculating FSG Cache Read Misses
To calculate if FSG Cache read misses have increased, use the following formulas:
•
FSG Cache read miss = physical read I/O divided by logical read I/O
Physical read I/O counts can be obtained from ResUsageSpma table by adding
FileAcqReads + FilePreReads.
Logical I/O counts can be obtained from ResUsageSpma table column FileAcqs.
•
Increase in FSG Cache misses = FSGCacheReadMissAfter divided by
FSGCacheReadMissBefore
While Teradata Database cannot guarantee a particular improvement in system performance, experience has shown gains of 2-8% when adding 1GB of memory per node in such instances.
541
Appendix C: Performance Management: All DBAs
Performance and Memory Management
Using DBS Control Settings to Manage Memory
Aggregate Cache Size
The valid range of values for aggregate cache size is from 1 MB to 8 MB. The default cache size is 4MB. DBS Control parameter, AggrCacheSize, controls the size of the aggregate cache.
Considerations Before Increasing Aggregate Cache Size
•
If aggregations typically have few groups and large numbers of rows per group, a larger aggregate cache size is not needed.
•
If aggregations are mostly on the PI, the aggregation algorithm usually keeps only one row in memory at a time, so cache size is less important.
•
If aggregations have large numbers of groups and large numbers of rows per group, a larger cache is recommended.
•
If you are doing lots of aggregations at the same time, defining a larger aggregate cache means less memory for all user combined. Thus, make sure you understand how your memory is being used before you make any changes to aggregate cache size.
DictionaryCacheSize
The default value allows more caching of table header information and reduces the number of
I/Os required. It is especially effective for workloads that access many tables (more than 200) and for those that generate many dictionary seeks.
Increase the size of the dictionary cache to allow the parser to cache additional data dictionary and table header information.
For tactical and Online Complex Processing (OLCP) type workloads, maintaining a consistently short, few-second response time is important. These workloads may benefit from a larger dictionary cache, particularly when their query plans have not been cached in the request cache. A larger dictionary cache will allow more dictionary detail, needed for parsing and optimizing, to remain in memory for a longer period of time. For query workloads with a response time of more than one minute, there may be no measurable difference when this field is set to a higher value.
IdCol Batch Size
The IdCol Batch Size setting involves a trade-off between insert performance and potential gaps in the numbering of rows inserted into tables that have identity columns.
A larger setting results in fewer updates to DBC.IdCol in reserving batches of numbers for a load. This can improve the performance of bulk inserts into an identity column table.
However, because the reserved numbers are kept in memory, unused numbers will be lost if a database restart occurs, resulting in a gap in the numbering of identity columns.
PPICacheThrP
Under most use cases, the default value for PPICacheThrP is adequate, and should not be changed. However, if there are performance issues that might be addressed by adjusting this value, consider the information in this section.
542 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Performance and Memory Management
The current data block for the corresponding partition (or buffer, in the case of inserts to column-partitioned tables) is associated with each context. The current set of data blocks or buffers (one for each context) are kept in memory, if possible, to improve the performance of processing the set of partitions at the same time. If there is a shortage of memory, these blocks or buffers may need to be swapped to disk. Excessive swapping, however, can degrade system performance.
Larger values may improve the performance of partitioning operations, as long as the following occur:
•
Data blocks or AMP buffers for each context can be kept in memory. When they can no longer be kept in memory and must be swapped to disk, performance may degrade.
•
The number of contexts does not exceed the number of nonempty, noneliminated partitions for partitioning operations. (If they do, performance will not improve because each partition can have a context, and additional contexts would be unused.)
In some cases, increasing the value of PPICacheThrP above the default value can provide a performance improvement for individual queries that do these partitioning operations.
However, be aware of the potential for memory contention and running out of memory if too many of these queries are running at the same time.
The default setting of 10 is conservative, and intended to avoid such memory problems. With
80 AMP Worker Tasks (AWTs) on a system with the default setting of 10, the maximum amount of FSG cache memory per AMP that could be used for these partitioning operations is 80% of FSG cache memory, if all AMPs are simultaneously executing partitioning operations, such as sliding-window joins for 80 separate requests. For configurations that have more than 80 AWTs defined as the maximum, the setting is scaled to the number AWTs. For example, at the default setting of 10, a cap of 80% of FSG cache memory per AMP would still be in effect on such systems.
For many sites, the default may be too conservative. All 80 AWTs might not be running partitioning operations at the same time. If, at most, 60 partitioning operations are expected to occur at the same time, the value of PPICacheThrP could possibly be raised to 15. If at most
40 are expected, the value could possibly be raised to 20, and so on. The best value for this parameter is dependent on the maximum concurrent users expected to be on the system and their workload. No one value is appropriate for all systems.
Also, consider that the number of concurrent partitioning operations, such as sliding-window joins, may increase as partition usage is increased. Increasing the value may increase performance now without memory contention or running out of memory but, in the future, as more partitioning operations run concurrently, performance may decrease, or out of memory situations may occur.
If less than 80 concurrent partitioning operations are expected for your site, and you think that better performance may be possible with an increased value for PPICacheThrP, you can experiment with PPICacheThrP settings to determine an increased PPICacheThrP setting that is optimal for your site and safe for your workloads. Measure pre- and post-change performance and degree of memory contention under expected current and future workloads to evaluate the effects of the change. If increasing the value to one that is reasonably safe for your site does not yield adequate performance for partitioning operations such as sliding-
543
Appendix C: Performance Management: All DBAs
Periodic Maintenance and Performance window joins, consider defining partitions with larger granularity, so fewer partitions are involved in the sliding-window join.
RedistBufSize
To avoid the overhead of sending lots of little messages across the BYNET, buffers are used to batch up individual rows during the row redistribution process. RedistBufSize setting determines the size in kilobytes for row redistribution buffers used during load utilities.
The default is 4 (which translates to 4 KB), and each AMP will have one such buffer in memory for each AMP in the configuration during a load job row redistribution.
If a system has relatively few AMPs, a larger redistribution buffer size usually has a positive effect on load performance. However, on larger systems with many AMPs, a large buffer size can consume excessive memory, especially if many load jobs are run concurrently.
Maintain the RedistBufSize at 4 KB for up to 48 nodes with 8 AMPs/node. As the system configuration grows larger, you can compensate by doing one or more of the following:
•
Set RedistBufSize smaller in proportion to the increase in the total number of AMPs, that is, send more smaller messages)
•
Add more memory to increase the total memory in the system to accommodate the redistribution buffers somewhat larger.
•
Increase the amount of free memory available for redistribution buffers by setting
FSGCache percent smaller
To help determine if RedistBufSize is too high, see if the minimum available free memory consistently goes below 100 MB during heavy periods of load utility redistribution. Also, check for significant swapping (more than 10/second) during this period. If this is the case, reduce RedistBufSize an incremental value lower, for example, from 4 KB to 3 KB.
Periodic Maintenance and Performance
Cleaning Up Logs
You should periodically remove old information from the DBC.Acctg table. For example, the following command will delete entries for September 2006:
DELETE FROM DBC.ACCTG WHERE SUBSTR(ACCOUNTNAME, 1, 6) = 'CB0609';
Note: To minimize performance impact, perform any necessary maintenance to DBC tables when the system is quiescent or least busy. Cleaning certain tables, such as DBC.AccLogTbl lock the table and prevent access to the system.
DefragLowCylProd
A low value in this field reduces the performance impact of defragmentation. However, setting the value extremely low might cause cylinders to be consumed more quickly, which could cause more MiniCylPacks to run.
544 Database Administration
Appendix C: Performance Management: All DBAs
Capacity Planning
Set DefragLowCylProd higher than MiniCylPackLowCylProd because defragmentation has a smaller performance impact than cylinder pack.
Capacity Planning
Monitoring CPU Usage
Teradata Database records both normalized and raw CPU measures. Normalized CPU time is derived by applying a CPU scaling factor to the node level raw CPU time.
The standard co-existence scaling factors for all node types are pre-defined in PDE startup parameter files in the Open PDE startup.txt file. The per-node values are added to the vconfig and tosgetpma() structures by PDE startup for use by other components. In this way, Teradata
Database provides accurate performance statistics for mixed node systems, particularly with respect to CPU skewing and capacity planning, that is, usable capacity.
The formula is:
Node_Scaling_Factor * Node_CPU_Time
The scaling factor values are derived from:
•
Measuring the raw capability of the Teradata compute node unconstrained by I/O or environment.
•
Basing values on basic functionality, including data access rate, data load rate, row processing rate, raw transaction rate.
Currently, the 5100 is the base scaling factor value (1.00).
For AMP level reporting in DBQL and AMPUsage, the formula is:
Node_Scaling_Factor * AMP_CPU_Time
ResUsage and DBC.AMPUsage View
The DBC.AMPUsage view displays CPU usage information differently than the way usage information is displayed in ResUsage data.
This facility…
ResUsage
DBC.AMPUsage view
Provides…
metrics on the system, without making distinctions by individual user or account ID.
AMP usage by individual user or account ID.
AMPUsage counts logical I/Os, not physical.
Some CPU seconds used by the system cannot be accounted for in
AMPUsage. Therefore, ResUsage CPU metrics will always be larger than AMPUsage metrics. Typically, AMPUsage captures about 70-
90% of ResUsage CPU time.
For more information on the DBC.AMPUsage view, see Data Dictionary.
Database Administration 545
Appendix C: Performance Management: All DBAs
Capacity Planning
Using the ResNode Macro Set
For capacity planning, generally only ResUsageSpma is required. This is the only table required to make use of the ResNode macro set.
Important information from ResNode includes:
•
CPU utilization
•
Parallel efficiency to show hot nodes or AMPs
•
CPU to I/O balance
•
OS busy versus DBS busy to see the characteristics of the workload
•
Memory utilization
•
Availability and process swapping (paging)
•
Network traffic
Observing Trends
ResUsage data is most useful in seeing trends when there is a reasonably long history (more than a year) available for comparison. Use this data to answer questions, such as:
•
How heavily is the system used at different times of the day or week?
•
When are there peaks or available cycles in utilization?
CPU Busy
The ResNode macro reports CPU busy by second averages. Use it for general system analysis.
The macro contain the Avg CPU Busy column, which is the average CPU utilization for all nodes. Avg CPU Busy % is a measure of how often multiple CPUs in a node were busy during a log period.
•
In a DSS environment, a small number of jobs can easily bring the CPU close to 100% utilization.
•
High CPU utilization consists of a Avg CPU Busy % of 70% over significant periods of time.
•
The CPU is stressed differently in DSS and transaction processing environments.
For ResNode columns, see Resource Usage Macros and Tables.
CPU Tasks in a DSS Environment
The following lists how CPU tasks are carried out on the node during DSS operations.
1
2
Prepare for read:
•
Memory management allocates memory for the data block.
•
Database software communicates with the file system.
•
File system communicates with the disk controller.
Qualify rows. Determine if the row satisfies the WHERE clause condition(s).
546 Database Administration
Database Administration
Appendix C: Performance Management: All DBAs
Capacity Planning
3
4
Most DSS operations require full table scans in which the WHERE clause condition check is relatively time-consuming. Full table scans generally result from SQL statements whose
WHERE clause does not provide a value for an index or partition elimination.
Process rows:
•
Join
•
Sort
•
Aggregate
Format qualifying rows for spool output.
CPU Tasks During Transaction and Batch Maintenance Processing
The following describes how the CPU tasks are carried out on the node during a transaction batch maintenance processing.
Notice that the qualify rows activity is missing from the table. In transaction processing, it is more common for the WHERE clause to provide a value for the PI or USI. The read itself qualifies rows. Transaction processing typically avoids further conditional checks against nonindexed columns. All of these CPU tasks occur on the nodes.
1
2
Prepare for read:
•
Memory management allocates memory for the data block.
•
Database communicates with the file system.
•
File system communicates with the disk controller.
Update row:
•
Database locates row to be updated.
•
Memory management allocates memory for the new data block to be built.
•
Database updates the changed row and copies the old rows.
•
Database communicates with the file system.
•
File system communicates with the disk controller.
Parallel Node Efficiency
Parallel note efficiency is a measure of how evenly the workload is shared among the nodes.
The more evenly the nodes are utilized, the higher the parallel efficiency.
Parallel node efficiency is calculated by dividing average node utilization by maximum node utilization. Parallel node efficiency does not consider the heaviness of the workload. It only looks at how evenly the nodes share that workload.
The closer parallel node efficiency is to 100%, the better the nodes work together. When the percentage falls below 100%, one or a few nodes are working much harder than the others in the time period. If node parallel efficiency is below 60% for more than one or two 10-minute log periods, Teradata Database is not getting the best performance from the parallel architecture.
547
Appendix C: Performance Management: All DBAs
Capacity Planning
Poor Parallel Efficiency
Possible causes of poor node parallel efficiency include:
•
Down node
•
Uneven number of AMPs per node
•
Skewed table distribution
•
Skewed join or aggregation processing
•
Non-Teradata Database application running on a TPA node
•
Coexistence system with different speed nodes
Poor parallel efficiency can also occur at the AMP level. Common causes of poor AMP parallel efficiency include:
•
Poor table distribution (You can check this in DBC.TableSizeV.)
•
Skewed processing of an SQL statement:
•
User CPU in seconds (You can check this in DBC.AMPusage.)
•
Spool (You can check this in DBC.DiskspaceV.)
CPU Use
The following macros provide information on CPU use.
Macro
ResCPUByNode
ResCPUByPE
ResCPUByAMP
Table
SPMA
SVPR
SVPR
Purpose
Report of how each individual node is utilizing
CPUs
Report of how each Parsing Engine (PE) utilizes the CPUs on irrespective node
Report of how each AMP utilizes the CPUs on the respective node
For information on the above macros, see Resource Usage Macros and Tables.
Monitoring Disk Utilization
ResUsage and Disk Utilization
The ResNode macro reports disk utilization averages by second.
Parallel Disk Efficiency
Parallel AMP efficiency does not necessarily match parallel disk efficiency. Monitor parallel disk efficiency using ResUsageSvdsk and ResUsageSpdsk tables.
The following table lists common mistakes that can cause skewing and poor parallel efficiency and suggests solutions.
548 Database Administration
Appendix C: Performance Management: All DBAs
Capacity Planning
Mistake
A user did not define a PI. The system uses the first column of the table as the default P1.
A user used a null as PI for the target table in a left outer join.
Solution
Define a PI with good data distribution.
A user performed a join on column with poor data distribution. For example, the user entered:
SELECT A.colname, B.x, B.y
FROM A, B
WHERE A.colname =
B.colname;
Perform any of the following:
•
Choose a different PI.
•
Handle NULL case separately.
•
Use a multiset table.
•
Identify column value and counts. For example, enter:
SELECT colname, count(*)
FROM T
HAVING count(*) > 1000
GROUP BY 1 ORDER BY 1 Desc;
The following displays: colname count(*) codeXYZ 720,000 codeABC 1,200
•
Break the query into two separate SQL statements.
For example, handle codeXYZ only in one SQL statement; handle all other cases in another SQL statement.
•
Collect statistics on the join column.
Monitoring Traffic
ResUsage and Host Traffic
The ResHostByLinkfmacro analyzes the traffic flow between Teradata Database and the mainframe or workstation client and reports averages by second.
For ResHostByLink columns, see Resource Usage Macros and Tables.
ResUsage and BYNET Data Macro
Macro Description
second averages
Purpose
For ResNode columns, see Resource Usage Macros and Tables.
Database Administration 549
Appendix C: Performance Management: All DBAs
Performance Considerations when Expanding or Upgrading the Database System
Performance Considerations when Expanding or Upgrading the Database System
Expansion involves adding any combination of disk arrays, memory, vprocs, or nodes (with
BYNETs), and then running the Parallel Upgrade Tool (PUT) and Configuration and
Reconfiguration utilities.
The Reconfiguration utility provides an estimate of the duration of an outage. See Support
Utilities.
Upgrade means upgrading the Teradata database software.
Adding Disk Arrays
To determine if the system needs more storage, look at the ResUsageSvpr table for unusual disk activity, such as frequent:
•
MiniCylPacks
•
Defrags
•
Packdisks
You may need to add more storage capacity to existing nodes when:
•
Excessive disk activity (I/O) is impacting performance
•
Application changes require additional spool space
•
Database growth requires additional storage
Adding Vprocs
The following table lists reasons for adding vprocs.
IF you want to …
increase storage capacity reduce single-user response time
THEN add …
disks, probably with AMPs, and perhaps nodes. When you add storage, you normally add AMPs.
The number of AMPs you add depends on the number of ranks assigned to the existing AMPs. For example, if you add two disk cabinets to each of 20 ranks, you would add 10-20 more AMPs to your configuration.
Note: Teradata recommends that you increase memory when you add AMPs.
If your existing CPUs cannot handle the load caused by the additional AMPs, you may also need to add nodes.
AMPs.
•
Optimization of single-user throughput or response time is especially significant when there are fewer AMPs than CPUs per node.
•
In a concurrent workload, you might be able to achieve the desired throughput by adding more AMPs to existing nodes.
•
Be sure your memory is adequate for the additional AMP workload.
550 Database Administration
Appendix C: Performance Management: All DBAs
Performance Considerations when Expanding or Upgrading the Database System
IF you want to …
increase the capacity for concurrent sessions
THEN add …
PEs, perhaps nodes, and perhaps a mainframe connection.
•
For network sessions, add PEs if you have fewer than 10 PEs per node. If you already have 10 PEs per node, add another node.
Note: The session limit is 120 for each PE and 1200 for each gateway.
Because each node runs just one instance of the gateway, more than 10 PEs per node does not provide increased network sessions.
•
The channel driver does not have a session limit, but normally one PE is configured for each channel connection.
If your configuration has less than 4 channel connections, you can add another channel connection and another PE.
•
Verify that there is enough CPU capacity to handle more PEs. If not, add nodes.
Adding Memory
When you add memory, you increase cache to maximize the capability of the CPUs. This is helpful when CPUs are processing faster than the disk contents can be read into memory (that is, when the system is I/O-bound).
The following table lists reasons to add more memory.
Condition
Add vprocs to existing nodes
Description
Each vproc consumes 32 MB of memory. When you add vprocs to existing nodes, you probably should add memory.
Additional vprocs can substantially reduce free memory, which can cause more
I/Os because the system can cache fewer datablocks.
More memory means that more code and data can be cached, achieving less
I/O for paging and swapping.
Excessive paging/ swapping
(thrashing)
Tables in memory Increased memory may reduce I/O by accommodating:
•
Tables that are currently too large to remain in memory during processing
•
More small tables concurrently residing in memory during processing
I/O can be affected by the size of data blocks. See the DATABLOCKSIZE option for the CREATE TABLE statement in SQL Data Definition Language.
Reconfiguring the Database
When you add nodes, AMPs, or disks to the Teradata Database configuration, it is good practice to always first backup all data. Some types of expansion, such as adding a disk array, require row redistribution, so if the system contains very large tables, it may be faster to archive and restore.
Database Administration 551
Appendix C: Performance Management: All DBAs
Performance Considerations when Expanding or Upgrading the Database System
You can use the Reconfiguration Estimator utility to obtain an estimate of elapsed time needed for reconfiguration, based on the number and size of the data tables on your current system.
The Reconfiguration Estimator prompts you for information about the planned upgrade and provides estimates for the following phases:
•
Redistribution
•
Deletion
•
Rebuilding SIs
If you have questions about possible procedures or the reported time estimates, contact
Teradata Customer Support. For an overview of the utilities, issues, and procedures involved in a TPA reconfiguration, see Support Utilities.
Scaling
Your user applications should be scalable.
For example, assume that table BigT is a very large table but its rows are hash distributed based on only 16 unique values. Applications using BigT perform well and with high parallelism on a system with 1 or 2 nodes and 8 to 16 AMPs.
If you then expand the system to 128 AMPs, the rows of BigT still hash to only 16 unique values, and so are still distributed among only 16 AMPs. Thus, applications do not perform any better, and perhaps not as well.
To ensure scalability of your applications, try to make your PI a unique or nearly unique index. If a single column does not provide uniqueness, combine two or more columns to make up the index. You can define up to 64 columns per PI.
The result is many more unique hash combinations, and your applications should continue to perform well as your system expands.
Checking Performance when Upgrading
When upgrading the Teradata Database software, you should monitor system behavior to make sure the upgrade is producing the expected performance increases.
•
Recollect statistics for the existing configuration if they are not current.
•
Create a baseline test suite.
•
Run the baseline on the system before the upgrade.
•
Save the EXPLAINs for the base queries.
•
Run the test bed again after the upgrade, and get new EXPLAINs.
•
Compare the two EXPLAINS.
•
Check for faster/slower response on any of the test queries. If slower, look at the before and after EXPLAIN.Check the statistics.
•
If stale, recollect.
552 Database Administration
Appendix C: Performance Management: All DBAs
Performance Considerations when Expanding or Upgrading the Database System
•
If not, report this as a performance regression problem to Teradata Support immediately.
Performing these simple activities, even with a small group of queries (5-10 minimum are recommended), will help validate your performance results and expectations following any sort of important system change.
Database Administration 553
Appendix C: Performance Management: All DBAs
Performance Considerations when Expanding or Upgrading the Database System
554 Database Administration
APPENDIX D
Handling Teradata Crashdumps:
Operational DBAs
This appendix provides an overview of Teradata crashdumps. If you encounter any problems that result in a Teradata crashdump being generated, contact Teradata Customer Support or your customer support personnel.
Teradata Vital Infrastructure and Crashdumps
Under normal circumstances, Teradata support personnel handle crashdumps directly using the TVI.
Each system at your site must be registered with its own site ID so that Teradata will have all the proper information on the configuration of each specific system on your site.
If you have created and registered a site ID for each system at your site, you should have TVI software already installed so that it can begin monitoring your Teradata Database. When a
3610 error or an 033-12123-00 error occurs, this triggers an incident to become created. (You can log on through Teradata @ Your Service to open incidents as well.)
•
A 3610 error indicates an internal error produced by a request sent to Teradata Database which has caused a snapshot dump and aborts the given request. This is done instead of restarting the system.
•
A 033-12123-00 message represents a task snapshot dump and an event is logged when a program requests a snapshot dump on itself.
TVI, through the TVI Manager and Auto Incident Create (AIC) server, will automatically notify support personnel from Teradata Customer Support (TDCS). The customer support personnel will try to troubleshoot the problem to determine the root cause or may work with field engineers if there is a hardware problem. They will engage your local site team and may also escalate the incident to Teradata developers, if necessary.
Every customer site team is different: some local site teams may start collecting crashdumps and have them already ready in the event that your Teradata Customer Support personnel asks for the crashdump to be sent in to Teradata. Other site teams will wait until instructed by the
Teradata Customer Support to send the crashdump information.
If there are multiple users on the system at once and you are not sure who caused it or where the bad request or error came from, the customer support personnel may need to extract the problem SQL from the crashdump and ask you to run TSET on that SQL and send in the
TSET pack file to continue the investigation.
Database Administration 555
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Crashdumps Overview
TSET File Pack and the DBC.TSETQueryText Table
The TSET pack file is a set of Teradata Customer Support files that can help in troubleshooting a crash. These files are intended for Teradata support personnel so you do not need to create or manage these files.
Part of the TSET file pack includes information that comes from the DBC.TSETQueryText table. This table is a globally available system table created during sysinit. If a 3610 error occurs in the Parser, the system will attempt to log a row into this table. However, depending on the severity of the 3610 error, the system may not be able to populate the TSETQueryText table.
While the information from DBC.TSETQueryText does not directly populate the crashdumps database, the contents of the table can be used to generate TSET information by Teradata support personnel for crash resolution. The DBC.TSETQueryText table captures problematic queries on the system that result in a 3610 error.
After the problematic query is captured, TVI provides Teradata and the DBA with a command to generate and retrieve the necessary TSET information based on the contents of the
DBC.TSETQueryText table. This allows Teradata support analysts to reproduce the query immediately on a test system and saves time of waiting for a crashdump to finish saving as well as the time it takes to send (FTP) the crashdump to Teradata Customer Support.
Note: As an alternative method to the TSET Perl scripts for extracting TSET information based on the DBC.TSETQueryText table, Teradata support personnel can use the external stored procedure named ExportTSET. However, the TSET information produced by the external stored procedure is in the form of BTEQ scripts and is currently not compatible with the TSET GUI client.
Crashdumps Overview
This section is an overview of Teradata crashdumps concepts and terminology. Information on Operating System (OS) dumps is provided only for clarification and differentiation from
Teradata crashdumps.
556 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Crashdumps Overview
Dump Types
The following table describes the types of dumps on your system.
Crashdump Type
Operating System
(OS) dump
Teradata
Crashdump
Snapshot Dump
Description
An OS dump is the unprocessed contents of memory at the time of an operating system crash. The content of the dump depends on your platform
OS.
An OS dump is the result of a single node OS crash and contains the entire contents of the memory of a node. It may also be referred to as a system dump, kernel dump, or node dump. On Linux, an OS dump is called a kernel panic dump.
When an OS crash occurs, the node writes the contents of memory to a system dump area and reboots. An OS dump is typically the size of memory. An OS dump is independent of Teradata or any particular application.
A Teradata Crashdump, also known as a PDE dump or simply crashdump, is a result of an unrecoverable Teradata error. When Teradata crashes, PDE on each node writes selective data to its own PDE dump directory, called a raw PDE dump. How much and which of this data is actually written also depends on the type of error. After the raw PDE dump is captured, Teradata restarts, but the system does not reboot.
After Teradata restarts, the CSP utility saves the raw PDE dump of each node as rows in a crashdump table in the DBC.Crashdumps database. Depending on dump control parameters, CSP may start automatically or you can start it manually. Since Teradata is a parallel system, a full Teradata crashdump is a system-wide entity, in table format, consisting of data from each node.
Specialized tools exist to examine the system-wide crashdump in table format.
A snapshot dump occurs when a process fails on a node and the failure does not cause a Teradata Database restart.
Two utilities record process and backtrace information from snapshot dumps:
•
The Snapshot Dump facility saves the report to the DBC.Crashdumps database.
•
The dmpdiver PDE utility saves the report to /var/opt/teradata/
tdtemp
. Dmpdiver also sends the report to the Teradata Support Center automatically via TVI and links it with an automatically created incident.
Optionally, you can include in the report the SQL responsible for generating the dump. Including the SQL may speed up problem resolution, but it is not the default. To include the SQL in the report or to change any other dmpdiver behavior, see the man page for dmpdiver.
Database Administration 557
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Teradata Crashdumps Processes
Teradata Crashdumps Processes
The procedure for Teradata crashdumps is as follows:
1
2
3
4
An event occurs that triggers a crashdump.
DMP runs at the time the fault or reset is detected and saves a crashdump to the PDE raw dump directory.
After Teradata Database restarts, what happens next depends on the setting of the Save
Dumps field of Screen Debug in the Ctl utility. If the field is set to:
•
On, CSP starts automatically and saves the crashdump from the raw dump directory to a table in the DBC.Crashdumps database. Although Save Dumps is On by default,
Teradata recommends that you set this field to Off to conserve system resources and save crashdumps only when you need to.
•
Off, you start CSP manually and save the crashdump to the DBC.Crashdumps database. Manually saving crashdumps rather than having the system save them automatically is a best practice (See
running the same version of the operating system and Teradata Database. See
“Manually Saving Crashdumps to Another Teradata System” on page 568
Because Teradata Database is a parallel system, a full crashdump is a system-wide entity in a table format consisting of data from each node. All nodes work in parallel to write their piece of the crashdump into a common table in DBC.Crashdumps: they write the contents of memory to internal disk and save the raw PDE dump into the DBC.Crashdumps database on all nodes of the system at approximately the same time.
Note: In some cases, Teradata Customer Support may choose to debug crashdumps directly from the dump area. For more information, see
“Debugging Crashdumps” on page 570 .
Copy the crashdump from the table in DBC.Crashdumps and send the crashdump data to
Teradata Customer Support either:
•
Through FTP services.
•
Or, if you cannot upload your crashdumps using standard FTP services, copy the crashdumps onto external media using DUL or DULTAPE utility. Use DUL to write to
disk or DULTAPE to write to the node that has the internal tape drive. (See “Saving
Crashdumps to Disk or Tape” on page 569 .)
For more detailed information on how to handle crashdumps for your specific OS, see
“Handling Teradata Crashdumps” on page 564
.
558 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Teradata Crashdump Formats
Teradata Crashdump Formats
The PDE first writes the data in a raw PDE dump and this information is saved (or can be saved) into a Teradata table in the DBC.Crashdumps database.
Because a raw PDE dump is written in “dump” format and a crashdump is saved in “table” format, they are both compatible with Teradata's standard crashdump management and analysis tools.
The CSP SAVE dump command requires two options: “source” and “target.”The default format for -source is “dump” (which is the PDE dump directory) and the default -target is
“table” (the DBC.Crashdumps database). There are other formats (such as dbm or flatfile) that can be specified, but these formats are for special purposes and should not be used.
For more information, see the CSP pdehelp. See “Manually Saving Teradata Crashdumps” on page 566
for examples of the SAVE command.
Teradata Crashdump Sizes
An OS dump is different from a Teradata crashdump in not only content but also size.
Typically, a Teradata crashdump is smaller than the sum of the memory for all the nodes in the database configuration.
The size of raw PDE dumps still in “dump” format (before the crashdumps are saved in table format) depends on various factors such as the type of error, number of vprocs, amount of memory, number of processes running when Teradata crashed and so on. So the size of raw
PDE dumps can vary greatly from 10 MB to 4 GB or even larger. For example, on a Linux node that is 4 GB with 10 vprocs, the size of the Teradata crashdump could range from 1.5 to 2
GB.
Because a full crashdump saved to the DBC.Crashdumps database is the collection of the raw
PDE dumps from all the nodes, a full crashdump is approximately twice the size of the raw dump multiplied by the number of nodes for your system. The DBC.Crashdumps database, which is allocated from the permanent space of DBC, should be large enough to hold 4 to 5 crashdumps.
Note: Dump creation fails if the number of dumps saved on a node exceeds the limit in the control GDO. After each dump, Teradata checks the number of dumps saved on the node and writes a warning to the system messages log if there is space for only one more dump or if the saved dump limit has been reached.
To prevent space issues, actively manage dumps by saving raw PDE dumps from the dump directory to DBC.Crashdumps, unloading crashdumps from the DBC.Crashdumps database to tape, and deleting unwanted crashdumps from both the dump directory or
DBC.Crashdumps database.
Database Administration 559
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Time Considerations
Time Considerations
The time it takes to save a crashdump can take anywhere between 45 minutes to several hours.
There are several factors that affect how long saving a crashdump could take:
•
The size of the crashdump (snapshot dumps are much smaller than regular dumps)
•
How many nodes you have
•
How busy the system is
•
Whether you save the crashdump without fallback (the default) or with fallback
Also, if you are using FTP services to send crashdump files to Teradata Customer Support, you must factor in that time as well.
Viewing Teradata Crashdump Messages
Messages are written to the log file located at
/var/log/messages.
Locating Crashdumps
For the most part, if you encounter a problem on your system that results in a crashdump,
Teradata Vital Infrastructure will notify Teradata if you have TVI installed on your system.
Otherwise, you should contact your customer support personnel.
•
Raw PDE crashdumps are saved to /var/opt/teradata/tddump.
Teradata recommends that the TDTEMP and TDDUMP partitions be identified as /var/ opt/teradata/tdtemp and /var/opt/teradata/tddump, respectively. You can use the dmpconfig utility to view or change the default locations. For more information on this utility, type dmpconfig -h.
•
Crashdumps in table format are located at
DBC.Crashdumps.Crash_YYMMDD_HHMMSS_NN.
Note: When you first install Teradata Database on your Linux system, you must partition the disk and save a slice for dumps. See Teradata Tools and Utilities for SUSE Linux Installation
Guide for more information.
Configuring the Crashdumps Database
The DBC.Crashdumps database is an internal Teradata database initially installed by the
DIPCRASH scripts when the system is installed. Because it is a Teradata database, it requires permanent space. Space is allocated and managed similar to other databases and can be modified using Teradata DML.
560 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Configuring the Crashdumps Database
The size of the DBC.Crashdumps database is configurable and depends on available space and your administrative preferences. Just as the PDE dump directory should be large enough to hold approximately 4 to 5 raw dumps, the DBC.Crashdumps database should be large enough to hold 4-5 full crashdumps (a full crashdump is the total of all raw PDE dumps from each node).
By default the database is NO FALLBACK, and no dumps are saved if an AMP goes down. If you specify FALLBACK, you gain fault tolerance, but the crashdump will take significantly more time and resources.
Crashdumps space is allocated from current PERM space for DBC. Reducing crashdumps space makes more free space available in DBC. Increasing Crashdumps space reduces DBC space.
For more information, see “Allocating More Space to DBC.Crashdumps Database” on page 562
.
Ad-Hoc Creation of DBC.Crashdumps with DIPCRASH
Sometimes you may want to run the DIPCRASH script from the DIP utility because the script may not have been properly run and set up during system installation.
To verify that Crashdumps exists, submit the following statement:
HELP DATABASE CRASHDUMPS;
If the database table exists but contains no data, the following message appears:
Empty HELP information returned.
If the database table does not exist, an error message appears warning you that the table does not exist. If you want to create the database table, run the DIP utility and execute the
DIPCRASH script.
You can start the DIP utility from the Supervisor window of DBW.
Note: Make sure DBC has enough space to create Crashdumps and still retain enough for a maximum-sized transient journal plus overhead.
The amount of PERM space allocated to Crashdumps is deducted from the currently available space of user DBC.
To review DBC space, follow the procedure detailed under
DBC.Crashdumps Database” on page 562 .
For instructions on running DIP, see “Database Initialization Program (DIP)” in Utilities.
Checking DBC.Crashdumps Space Allocation and Usage
When your system was first installed with Teradata Database, there was a very small default amount of PERM space allocated to DBC.Crashdumps. You should adjust the space based on what you find is required for your workload and your site. For example, you may have to also increase or decrease the amount of space based on how many crashdumps you want to actually retain at one time. If the starting figures suggested above are not large enough, the best thing to do is run some tests and increase or decrease PERM as required.
Database Administration 561
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Configuring the Crashdumps Database
As described in
“Teradata Crashdump Sizes” on page 559 , a full crashdump is the size of the
raw dump multiplied by the number of nodes for your system. The DBC.Crashdumps database, which is allocated from the PERM space of DBC, should be large enough to hold 4 to 5 crashdumps.
To first check the amount of space used by DBC.Crashdumps, use the following query: sel databasename, sum(maxperm), sum(currentperm) from dbc.diskspaceV
where databasename='crashdumps' with sum(maxperm) (title 'Total'),sum(currentperm) group by 1;
*** Query completed. 2 rows found. 3 columns returned.
*** Total elapsed time was 1 second.
DatabaseName Sum(MaxPerm) Sum(CurrentPerm)
------------------- ---------------- -----------------------------
Crashdumps 21,474,836,424 991,296,000
---------------- -------------------
In this example 21,474,836,424 (21 GB) are allocated to the crashdumps database and
991,296,000 (1 MB) is used.
Allocating More Space to DBC.Crashdumps Database
To increase the permanent space allocated to DBC.Crashdumps, submit a MODIFY USER statement similar to the following where the amount of PERM space equals the amount you would like to increase DBC.Crashdumps: modify user crashdumps as PERM = 245000000;
*** Database/User has been modified.
*** Total elapsed time was 1 second.
Then submit the following statement to verify the new size and confirm that space was properly allocated: sel databasename, sum(maxperm) from dbc.allspaceV where databasename =
'crashdumps' group by 1;
*** Query completed. One row found. 2 columns returned.
*** Total elapsed time was 1 second.
DatabaseName Sum(MaxPerm)
------------------------------ -------------------
Crashdumps 245,000,000
Note: If DBC does not have enough space to increase Crashdumps, you need to free up space or add more disks. Crashdumps are saved in compressed format by default.
You can also use the -compress and -nocompress options on the csp command line to override the default setting. To save space, use the -compress option.
562 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Automatic Versus Forced Crashdump
Automatic Versus Forced Crashdump
The following section discusses when and why automatic crashes occur and why you might want to force a crash to occur. For example, you can force a crash or restart to recover from a system hang. The forced crashdump may provide useful information about why the system hung.
Automatic Teradata Crashdumps
The following table describes the events that cause a Teradata crashdump.
This event …
Teradata Database error restart
Forced Teradata Database restart (with Teradata crashdump)
Can occur …
for many reasons. Most commonly, a restart is due to a fatal error condition in Teradata Database.
The Teradata crashdump capture occurs before the restart on each node to preserve the error information in node memory.
at any time, using restart commands.
Only use restart commands for maintenance, so you do not interrupt jobs that are running. See
“Commands for Forcing a Teradata
.
Forced Teradata Crashdumps
You can control the crashdump type (OS dump or Teradata crashdump) by the way you force the crashdump. Select the type of crashdump you need from the following table.
IF one or more of these conditions exist …
•
The node is not responding
•
The system is not making a Teradata crashdump. (The system does not respond to input while it is making a Teradata crashdump).
•
Teradata Database sessions are not progressing
•
New sessions cannot start
•
The AWS is responding
THEN use …
OS dump.
Teradata crashdump.
Commands for Forcing a Teradata Crashdump
You can restart Teradata Database without causing a reboot using one of the following methods (also see
“Forced Teradata Crashdumps” on page 563 ):
•
In DBW, open the Supervisor window icon and enter the following command: restart tpa dump=yes comment where comment is text explaining why there is going to be a restart.
Database Administration 563
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Handling Teradata Crashdumps
The system performs a Teradata crashdump and a cold or coldwait restart. (For details, see
“RESTART” under “Vproc Manager (vprocmanager)” in Utilities.)
•
You can access the Teradata Command Prompt window and enter the tpareset command: tpareset -d comment
The tpareset command has basically the same effect as the restart command in DBW.
The -d option forces a Teradata crashdump.
The system performs a cold restart and a Teradata crashdump.
Whichever command you choose to execute, the system prompts you for a confirmation.
Handling Teradata Crashdumps
Use the Debug screen of the ctl utility to verify Teradata crashdumps control settings. The
Debug fields should read as the following table describes.
ctl Field Name
Save Dumps
Maximum Dumps 5
Snapshot Crash
Value
Off
Off
Purpose
Although the default setting for this field in On, which enables the automatic transfer of Teradata crashdumps from the dump directory to DBC.Crashdumps database, it is a best practice to set this field to Off.
It is a best practice to set this field to Off because allowing the system to automatically save crashdumps can be resource intensive. You can manually save the crashdumps if you need to by using the -force option of CSP. For more
information, see “Manually Saving Teradata Crashdumps” on page 566
.
Set the maximum number of raw PDE dumps per node that the system will save on the dump directory. This includes snapshot dumps.
•
The value of -1 means that the number of dumps is limited only by the space available on the disk containing the dump directory. This is not recommended.
•
A value of 0 means no dumps will be saved.
•
The default is 5.
If the maximum value has been reach and another dump occurs, the latest dump is not captured and an error message is printed to the messages log.
A snapshot dump logs information about a recoverable error without requiring the system to restart.
This field should only be changed under the direction of
Teradata support personnel for diagnosing errors.
564 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Handling Teradata Crashdumps
Save Considerations
A Teradata crashdump table cannot be automatically saved if:
•
The ctl Save Dumps setting is off.
•
An AMP is down and FALLBACK is not defined for Crashdumps database
•
The Teradata Database system is not up
•
There is not enough space
You can, however, manually save crashdumps using the CSP utility. For more information, see
“Manually Saving Teradata Crashdumps” on page 566
.
Listing Teradata Crashdumps Using CSP
Using the CSP utility, you can list views in either the dump directory or the DBC.Crashdumps database.
•
To list the available crashdumps saved in the DBC.Crashdumps database, use the following
CSP command:
# csp -mode list -source table
•
To list the available raw PDE dumps, use the following CSP command:
# csp -mode list -source dump
Note: The -source parameter defaults to the dump directory. Therefore submitting
“csp -mode list” is equivalent to “csp -mode list -source dump”.
The output will show the fields listed in the following table.
Note: The first line shows if the output is from the dump directory or the database: csp: Searching for dumps in Teradata database Crashdumps on the local system or csp: Searching for dumps in raw dump directory
CSP Output Column
Sel
Description
If an asterisk for the crashdump is listed for this row, the crashdump is eligible or selectable for the specific list, save, or clear operation.
If there is no asterisk, it is not included as part of the specific list, save, or clear operation.
ID (date-time-token) The identifier assigned by CSP to the crashdump. For example, the ID
2008/11/08-15:55:26-24 means the crashdump was taken:
•
On the date of November 8th, 2008
•
At the time of 15:55:26
•
With the token 24
Nodes Number of nodes from which crashdumps information has been collected.
If one of four nodes is down, that node is not included.
Database Administration 565
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Handling Teradata Crashdumps
CSP Output Column
Event
Instigator
Status
Description
The TDBMS/PDE event code which caused the crash.
The trigger for the crashdump. The format is vproc number/Partition ID/
Task ID. In the examples that follow the table:
•
the vproc number is 16384
•
the Partition IDs are 0 and 11
•
the Task IDs that triggered the crashdump are 11480, 15160, 28272 and
6418).
The status of the crashdump. If a crashdump is corrupted or is from the wrong Teradata Database software version, this column will report:
“Required files are missing or corrupted” or “Dump belongs to a different
PDE version.”
If the field is blank, the crashdump is usable and valid.
You can constrain the CSP results with criteria such as listing all dumps from a specific date:
# csp -mode list 2008/09/09 csp: Searching for dumps in raw dump directory /var/opt/teradata/tddump csp: 4 dumps found, 1 dump to process csp: Sel ID (date-time-token) Nodes Event Instigator Status csp: --- ---------------------- ----- ----- --------------- -----csp: * 2008/09/09-17:03:13-02 1 10196 16384/0/(11480) csp: 2008/08/14-11:35:58-01 1 10196 16384/0/(15160) csp: 2008/07/11-14:41:58-01 1 10416 1/11/(28272) csp: 2008/07/01-11:58:42-05 1 10196 16384/0/(6418)
#
Note: Despite the date restriction for searching only for dumps on 2008/09/09, the list command returns all dumps including ones from other days such as 2008/08/14, 2008/07/11, and 2008/07/01. However, the crashdump from the desired date criteria of 2008/09/09 has an asterisk in the Sel field, which shows that it matches the command criteria.
If a node is inaccessible (for example, it is down because the BYNET or the PDE is not running), CSP will not be able to access the dump directory on that node. It will report the raw PDE dumps only one nodes that are up.
# csp -mode list csp: Searching for dumps in raw dump directory /var/opt/teradata/tddump csp: 4 dumps found, 4 dumps to process csp: Sel ID (date-time-token) Nodes Event Instigator Status csp: --- ---------------------- ----- ----- --------------- -----csp: * 2008/09/09-17:03:13-02 1 10196 16384/0/(11480) csp: * 2008/08/14-11:35:58-01 1 10196 16384/0/(15160) csp: * 2008/07/11-14:41:58-01 1 10416 1/11/(28272) csp: * 2008/07/01-11:58:42-05 1 10196 16384/0/(6418)
#
Manually Saving Teradata Crashdumps
CSP is controlled by the SCREEN DEBUG command of the ctl utility. Use CSP to save crashdumps manually:
1
Submit the following command to list the available raw PDE crashdumps:
2
#csp -mode list -source dump
Save the raw PDE dumps into table format in the DBC.Crashdumps database.
•
The basic form of the CSP command for saving crashdumps is:
566 Database Administration
Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Handling Teradata Crashdumps
#csp -mode save -source dump -target stream
CSP reads any files created by the DMP program in the Dump area and writes them into the Crashdumps database.
•
You can also use the following command to save all dumps found by CSP:
#csp -mode save -force
Dump 2007/11/26-09:54:25-01 is about to be saved
Are you sure? [y,n,q,?] >
A question will be displayed for each dump and the user must answer each one with yes, no, or quit (individual letters such as y, n or q can be used).
csp: Searching for dumps in configured PDE dump directory on all nodes csp: 3 dump found, 3 dumps to process csp: Saving dump 1998/11/05-18:55:26-24 to table Crashdumps.Crash_200711
05_185526_24 csp: Writing data rows for dump 2007/11/05-18:55:26-24 csp: Dump 2007/11/05-18:55:26-24 saved csp: Dump 2007/11/05-18:55:26-24 cleared csp: Saving dump 2007/11/04-17:37:53-23 to table Crashdumps.Crash_200711
04_173753_23 csp: Writing data rows for dump 2007/11/04-17:37:53-23 csp: Dump 2007/11/05-18:55:26-24 saved csp: Dump 2007/11/05-18:55:26-24 cleared csp: Saving dump 2007/11/04-17:23:27-22 to table Crashdumps.Crash_200711
04_172327_22 csp: Writing data rows for dump 2007/11/04-17:23:27-22 csp: Dump 2007/11/05-18:55:26-24 saved csp: Dump 2007/11/05-18:55:26-24 cleared
3
If the SCREEN DEBUG setting has the Save Dumps flag set to not save crashdumps, use the -force option to allow CSP to override the Save Dumps setting. For example, if you submit:
#csp -mode save -source dump -target stream -force
By default, the system saves the crashdumps in Last In, First Out (LIFO) order. The most recent crashdumps are saved first. After the crashdump is saved to DBC.Crashdumps, the crashdumps are automatically cleared from the PDE dump directory.
Verify that the crashdumps were properly saved to DBC.Crashdumps by using the following command:
# csp -mode list -source table csp: Searching for dumps in Teradata database Crashdumps on the local system csp: 1 dump found, 1 dump to process csp: Sel DumpName Nodes Event Instigator Status csp: --- ---------------------- ----- ----- ------------ -----csp: * Crash_20080807_142628_01 1 10196 16384/2/(21600)
#
You can run CSP commands from any node in the system, but the utility will read the dump directory on all nodes.
Note: For more information on CSP commands, type
csp -h
at a Teradata Command
Prompt or see the resource file located at /usr/pde/lib/defaults/Libcsp.
567
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Handling Teradata Crashdumps
Manually Saving Crashdumps to Another Teradata System
1
2
After Teradata restarts following a crashdump, you can save the dump to another Teradata system if it is running the same version of the operating system and Teradata Database. To do this:
Copy the contents of the raw dump directory to the other system.
Save all dumps CSP finds by using this command:
3
csp -mode save -force
Verify that the crashdumps were properly saved to DBC.Crashdumps by using the this command:
# csp -mode list -source table csp: Searching for dumps in Teradata database Crashdumps on the local system csp: 1 dump found, 1 dump to process csp: Sel DumpName Nodes Event Instigator csp: ------------------------------------------csp: * Crash_20080807_142628_01 1 10196 16384/2/(21600)
Status
------
#
Handling an Interrupted Dump Save Operation
If a dump save in progress is interrupted, for example the DBS restarts, the dump save is not resumed automatically and the dump does not appear with the csp -mode list command. If a dump save was interrupted, check the messages log to get the name of the crashdump and then use csp -force option to save the crashdump.
If a dump save is run while a node is out of the configuration, the crashdump is saved from the nodes that are currently available but not from the node that is out of the configuration. No error displays if this occurs.
The resulting saved crashdump is eventually a usable crashdump. You can run the following command on the saved crashdump to verify that it is usable:
#csppeek -i -d crashdumpname where crashdumpname is usually “Crash_yyyymmdd_hhmmss_nn”.
However, the final crashdump does not include the piece of the crashdump from the down node. In other words, the final crashdump will be missing the crashdump information from the down node that is not part of the configuration.
Error Logging
Error messages related to saving Teradata crashdumps are logged in /var/log/messages. For more information, see
“Viewing Teradata Crashdump Messages” on page 560
and
“Troubleshooting Crashdumps” on page 575 .
568 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Saving Crashdumps to Disk or Tape
Saving Crashdumps to Disk or Tape
If you cannot use FTP services to send your crashdumps to Teradata and must save your crashdumps to disk or tape, use the DUL or DULTAPE utility. DUL and DULTAPE are separate packages that you must order and install individually. For more information on DUL and DULTAPE, see Utilities.
Use this utility …
DUL
DULTAPE
On a …
node mainframe node
To load/unload a crashdump to/from a …
file on the node.
file or a tape.
Note: You must run DULTAPE on the node that has the internal tape drive. tape.
Note: You must run DULTAPE on the node that has the internal tape drive.
DULTAPE uses the FastLoad utility to copy a crashdump onto a tape. You can run multiple
DULTAPE sessions as long as the total number of sessions does not exceed the limit specified in the MaxLoadTasks field in DBS Control.
You can unload crashdump information to several sequential files, which are:
•
Rows of data dumped from the AMPs
•
Internal load module correspondence information
•
Applicable rows from Software_Event_LogV
By default, crashdump files are named Crash_yyyymmdd_hhmmss_nn where:
Variable
yyyymmdd hhmmss nn
Definition
Year, month, and day of the crashdump
Time (hour, minute, and second) of the crashdump
Sequential number associated with the crashdump
You can change the name using BTEQ, but remember that the timestamps are important. For more information on DUL and DULTAPE, see Utilities.
Database Administration 569
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Sending Crashdumps to Teradata
Sending Crashdumps to Teradata
To use FTP services to send crashdumps directly to Teradata for analysis, contact your
Teradata Support personnel for more information. You can also access the latest instructions from the knowledge articles through Teradata @ Your Service.
The knowledge articles include the latest file paths and server addresses. For example, there are articles with the following titles:
•
How to Upload Teradata Crashdumps to the TGSC
•
How to load a crashdump that is in DUL format
In addition, you may find other crashdumps articles useful. Search for keywords
“crashdumps” and “Linux” to access links to crashdump related articles such as:
•
Options For Database Crashdump Analysis
•
Tips and techniques for managing PDE crashdump
The knowledge articles on crashdumps are located in the Teradata Database Knowledge
Repositories available through Teradata @ Your Service. Sign up for an account at http://tays.teradata.com
.
Debugging Crashdumps
Teradata Customer Support may need to debug crashdumps. Teradata Customer Support can use the system debugger to access the raw dumps before they are saved to the Crashdump table
(even if the PDE and DBS are not up and running). This saves time and helps Teradata personnel identify problems more quickly.
If someone from Teradata Customer Support needs to immediately debug a problem by looking at a raw dump rather than wait for the system to save it to the Crashdumps table, they can use the raw flavor commands of the system debugger.
If your assistance is required, a Teradata Customer Support representative may direct you to use the commands listed in the following table.
Command
set crash-database info rawdumps info dbmdumps
Description
Sets the Teradata system to the one on which you would like to view saved dumps.
Note: This does not set the location for raw dumps.
Displays a list of dumps in the raw dump directory of the node.
Displays a list of dumps in the dbm dump repository of the node.
570 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Debugging Crashdumps
Command
info dbsdumps or info dumps
Description
Displays a list of the dumps on the node.
Note that the “set crash-database” command may have been used to point to a crashdumps database on a different machine than the one on which the debugger utility is running. The info dbsdumps command is only available with the DBS flavor of dumps.
attach rawdump crashdumpname Attaches the named raw dump in the node's raw directory.
attach dbsdump crashdumpname or attach dump crashdumpname
Attaches the named DBS dump or crashdump in the Teradata
DBS crashdumps database. For example,
(gdb) attach dump Crash_20040704_160628_01
Attaching a Raw Dump and Displaying the Backtrace
First type the
csp -mode list
command to allow CSP to display what dumps are available. foosystem:/var/opt/teradata/tddump # csp -mode list csp: Searching for dumps in raw dump directory /var/opt/teradata/tddump csp: 2 dumps found, 2 dumps to process csp: Sel ID (date-time-token) Nodes Event Instigator Status csp: --- ---------------------- ----- ----- --------------- -----csp: * 2008/09/16-14:00:40-05 1 10416 1/11/(20096) csp: * 2008/09/16-13:53:07-04 1 10416 0/11/(14975)
Then run the Teradata system debugger utility (/usr/pde/bin/gdb) to start up a Coroner session to debug one of the raw dumps.
Note: It is not recommended to debug a raw dump if csp is currently saving dumps.
foosystem:/var/opt/teradata/tddump # gdb
GNU gdb 6.0/13.10.00.00
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x8664-suselinux-teradata".
(gdb) i raw
//:
2008/09/16-14:00:40-05
2008/09/16-13:53:07-04
(gdb) at raw 2008/09/16-14:00:40-05
Attaching to crash dump
//: cdbopen: *** Processing data from raw dump, please wait... ***
//: Unloading files to: /var/opt/teradata/tdtemp/crn-daeqV1
//: Unloading file libm.so.6
//: Unloading file libcliv2.so
//: Unloading file libdl.so.2
//: Unloading file libelf.so.0
//: Unloading file libgcc_s.so.1
//: Unloading file libpthread.so.0
//: Unloading file libnsl.so.1
//: Unloading file libc.so.6
//: Unloading file libtdusr.so
//: Unloading file libthread_db.so.1
//: Unloading file ld-linux-x86-64.so.2
//: Unloading file libnss_files.so.2
//: Unloading file emffile
//: Unloading file emffile.idx
//: Unloading file libstdc++.so.6
//: Unloading file libaio.so.1
//: Unloading file librt.so.1
//: Unloading file libbz2.so.1
//: Unloading file libz.so.1
Database Administration 571
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Debugging Crashdumps
//: Unloading file libcrypt.so.1
//: Unloading file libresolv.so.2
//: Unloading file libgssapi_krb5.so
//: Unloading file libkrb5.so.3
//: Unloading file libk5crypto.so.3
//: Unloading file libcom_err.so.2
//: Unloading file libkrb5support.so.0
*** Selecting Aborting task(s), if any
Software Versions:
PDE = 13.10.00.00
TDBMS = 13.10.00.00
PDEGPL = 13.10.00.00
TGTW = 13.10.00.00
TDGSS = 13.10.00.00
Reading symbols for task actmain...Using host libthread_db library "/lib64/ libthread_db.so.1".
('libdbs1.so', 2aaaaabc7000) ('libumm1.so', 2aaaaaee9000)
('libevl1.so', 2aaaaafec000) ('libfil1.so', 2aaaab105000)
('libudf.so', 2aaaab455000) ('libudt.so', 2aaaab558000)
('libuci.so', 2aaaab6c3000) ('libamp1.so', 2aaaab7cf000)
('libamp4.so', 2aaaabbf3000) ('libstp.so', 2aaaabf02000)
('libpdt.so', 2aaaac28d000) ('libtvsasam.so', 2aaaac3c0000)
('libjil.so', 2aaaac50a000) ('libnetpde.so', 2aaaac623000)
('libpde.so', 2aaaac75d000) ('libemf.so', 2aaaac8b1000)
('libpdesym.so', 2aaaac9dd000) ('libbz2.so.1', 2aaaacb06000)
('libpthread.so.0', 2aaaacc17000) ('libelf.so.0', 2aaaacd2e000)
('libnsl.so.1', 2aaaace44000) ('libalgapi.so', 2aaaacf5b000)
('libgeos.so.2', 2aaaad0d4000) ('libgeos_c.so.1', 2aaaad3e0000)
('libproj.so.0', 2aaaad4f6000) ('libgdal.so.1', 2aaaad651000)
('libogrx.so', 2aaaadd1d000) ('libm.so.6', 2aaaade4e000)
('libcrypt.so.1', 2aaaadfa3000) ('libc.so.6', 2aaaae0dc000)
('libstdc++.so.6', 2aaaae30e000) ('libgcc_s.so.1', 2aaaae50c000)
('libaio.so.1', 2aaaae61a000) ('libdl.so.2', 2aaaae71b000)
('libthread_db.so.1', 2aaaae81f000) ('ld-linux-x86-64.so.2', 2aaaaaaab000)
('libz.so.1', 2aaaae928000) ('librt.so.1', 2aaaaea3c000) done.
1//20871: Event number 33-10416-00 (severity 50, category 10), occurred on
1//20871: Tue Sep 16 14:00:40 2008, at 001-01 (Vproc 1, partition 11, task
1//20871: 20871) in system foosystem in Module , version
1//20871: PDE:13.10.00.00,TDBMS:13.10.00.00,PDEGPL:13.10.00.00,TGTW:13.10.
1//20871: 00.00,TDGSS:13.10.00.00
1//20871: Task aborted abnormally.
1//20871:
1//20871:
1//20871:
[Switching to 1//20871]
1//20871: ---Type <return> to continue, or q <return> to quit---
0x00002aaaae16dec1 in __nanosleep_nocancel ()
from /var/opt/teradata/tdtemp/crn-daeqV1/libc.so.6
(gdb) bt
#0 0x00002aaaae16dec1 in __nanosleep_nocancel ()
from /var/opt/teradata/tdtemp/crn-daeqV1/libc.so.6
#1 0x00002aaaae16ddae in sleep ()
from /var/opt/teradata/tdtemp/crn-daeqV1/libc.so.6
#2 0x00002aaaac77368e in tsksuspend (utcbp=0x2aaaaebd7d80)
at tsk/tskport.c:404
#3 0x00002aaaac794407 in logeventreset (logmsg_p=0x2aab1441a260, resetflags=0)
at log/logusrlib.c:959
#4 0x00002aaaabf6e1b2 in awtcheck (MessagePtr=0x2aab1441a260)
at ../../amp/awt/awtmain.c:1244
#5 0x00002aaaac794e19 in logevent_com (eventcode=10416,
location=<value optimized out>, length=<value optimized out>,
dumpkind=0 '\0', resetflags=0) at log/logusrlib.c:658
#6 0x00002aaaac78a315 in prgfaultcatch (signal=339177112,
siginfo=0x2aab12fe1000, contextp=<value optimized out>)
at prg/prgport.c:2268
#7 <signal handler called>
#8 evlfaultgen (a=1) at ../../par/evl/evllib.c:892
#9 0x00002aab144bb678 in ?? ()
#10 0x00002aaaaac4f20f in evlinterp (cm=0x2aab14377f30, context=0x2aab143acdf8)
at ../../par/evl/evlinterp.c:2033
#11 0x00002aaaaac7784c in evlbuild (EvlCtxt=0x2aab143acdf8,
OutRowLoc=46914267187568, OutRowFld5=
{fldptr = {loc = 46914268279680, ptr = 0x2aab144bb380}, statusptr =
0x2aab144bb380, ptr = 0x2aab144bb380, loc = 46914268279680}, Sequent=0x2aab143ad0e8,
DumRowFlag=<value optimized out>) at ../../par/evl/evlbuild.c:222
#12 0x00002aaaabfc05a8 in WalkTbl (LINK=0x2aab14378e10)
at ../../amp/stp/stpret.c:2108
#13 0x00002aaaabfc55d4 in stpret () at ../../amp/stp/stpret.c:3613
#14 0x00002aaaac10427e in awtstphd () at ../../amp/awt/awtstphd.c:264
#15 0x00002aaaabf7106c in awtmain (argc=296775680, argv=0xabf6e615)
572 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Deleting Teradata Crashdumps
at ../../amp/awt/awtmain.c:2844
#16 0x00002aaaac77404a in tsknewthread (threadargp=<value optimized out>)
at tsk/tskport.c:1185
#17 0x00002aaaacc1d143 in start_thread ()
from /var/opt/teradata/tdtemp/crn-daeqV1/libpthread.so.0
#18 0x00002aaaae19b74d in clone ()
from /var/opt/teradata/tdtemp/crn-daeqV1/libc.so.6
(gdb)
Deleting Teradata Crashdumps
Deleting unwanted Teradata crashdumps from your disk and archiving crashdumps you want to keep may help free up space. It may help prevent crashdumps from using up the space on your system and causing problems.
First determine how many crashdumps you want and how long you want to keep the crashdump. Depending on the size of your system and the size of the crashdumps themselves, you can consider keeping some of the crashdumps as historical records and delete them later when they are no longer necessary.
If you are having space problems due to crashdumps, consider limiting the “Max Dumps” parameter in ctl to a smaller number of crashdumps which will help prevent the crashdump area from filling up. If you are unsure about what to do, contact Teradata Customer Support.
If you decide to delete all the crashdump tables in the Crashdumps database, submit:
DELETE DATABASE CRASHDUMPS;
You may also specify a single crashdump to be deleted using:
DROP TABLE CRASHDUMPS.tablename;
The following procedure describes how to use the CSP utility to delete crashdumps from the raw dump directory.
Note: For more information on CSP commands, type csp -h at a Teradata Command Prompt or see the resource file located at /usr/pde/lib/defaults/Libcsp.
1
List the available raw PDE crashdumps.
Use the following command from the Teradata Command Prompt: csp -mode list -source dump
The -source parameter reads the raw crashdumps from the PDE dump directory. The dump directory is also the default source, so it is equivalent to the command csp -mode list (without the -source dump).
csp -mode list -source -table
Use the following command from the Teradata Command Prompt: csp -mode list -source dump
The -source parameter reads the raw crashdumps from the PDE dump device. The dump device is also the default source, so it is equivalent to the command
csp -mode list
(without the
-source dump
).
csp -mode list -source -table
For more options and syntax, type
csp -help
from the Teradata Command Prompt.
Database Administration 573
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Fault Isolation Diagnostics
2
For CSP, the
-nodes <nodelist>
option allows you to choose specific nodes from which to list crashdumps. Use the
-nodes
option to target specific nodes for save, list, or clear operations.
Delete (clear) unwanted raw PDE crashdumps.
Use a command similar to the following: csp -mode clear 2008/12/06-14:19:34-02
The raw PDE crashdumps cleared are those selected with an asterisk in the “Sel” field.
Notice:
If you do not type a name, you will delete all crashdumps. However, you will be prompted to answer ‘y’, ‘n’, or ‘q’ for each crashdump to be cleared.
Note: The csp default setting for -source is dump (the PDE dump directory). The “csp mode clear” command is run from one node but deletes the raw crashdumps on all accessible Teradata nodes.
Fault Isolation Diagnostics
Fault isolation features capture information specific to the fault without requiring a reset.
These failure diagnostics facilitate early analysis of problems before the associated dump is available for examination.
Content and abort handling depends on whether the fault being isolated occurred in the parser, an AMP, or Dispatcher, as described in the following table.
Type of Fault
Isolation
Dispatcher
Parser
Description
The Dispatcher prequalifies a fault for fault isolation.
If the fault qualifies for fault isolation, the Dispatcher captures diagnostic data and then aborts the transaction. If the transaction abort completes successfully, then the system avoids a DBS reset.
The diagnostic data consists of a snapshot dump, user session related data, stack backtrace, and the SQL request if it is available. (The last three items listed are captured in the system error log.)
Note, however, that a DBS reset occurs if the fault fails to qualify for fault isolation, or if the system cannot complete the abort of the transaction successfully.
The parser issues a transaction abort and the Snapshot Dump facility captures the cause of the problem.
The data capture includes the dispatcher worker task containing the query text.
574 Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Troubleshooting Crashdumps
Type of Fault
Isolation
AMP (Spool
Table)
Description
Teradata attempts to recover from the spool error without a reset.
Typically, such an abort is accompanied by an optional Snapshot Dump to help
Teradata Customer Support analyze the reason for the AMP failure.
Aborts generated by the AMP Fault Isolation feature:
•
Return a detailed text description of the AMP failure to the associated client session.
•
Log diagnostic details of the failure to the system error log. AMP Fault
Isolation also performs this error-logging when the system reset cannot be avoided.
Note: If you need to tune AMP Fault Isolation, contact Teradata Customer
Support. They can help modify the settings to:
•
Reduce the cases for which the aborts are enabled.
•
Disable the feature entirely to force a Teradata reset and a dump on the next occurrence of the failure.
Troubleshooting Crashdumps
The following section is intended for Teradata support personnel and lists some common questions regarding how to troubleshoot crashdumps.
1
What are the most common errors encountered while handling crashdumps and why do these occur?
Question
What does an error 2644 indicate?
What does an error “Filesystem full” indicate?
What does the error “Control GDO maxdumps exceeded; no more dumps will be captured” indicate?
Answer
DBC.Crashdumps database is full.
Delete unwanted tables to free up more space or add additional space to the Crashdumps database.
The raw PDE crashdump directory is full.
You should delete unwanted crashdumps from the dump directory using the clear option of the CSP utility. For more information, submit: csp -h at a Teradata command prompt.
This error occurs when the maxdump parameter has been reached and another crashdump occurs.
Database Administration 575
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Troubleshooting Crashdumps
576
Question
I notice a crashdump with the following name:
Crash_yyyymmddhhmmss_nn_Part
What does this mean?
Answer
This is a partial crashdump. If a crashdump is not saved to the Crashdumps database yet, it is considered a partial crashdump. You should simply wait and give the system more time to process the crashdump. If you have waited but are not sure the crashdump is okay to use, run the csppeek utility on the saved crashdump.
If a crashdump is not saved completely in the dump directory, the crashdump is considered an incomplete crashdump.
2
3
You can find the crashdump error messages in the log for your operating system (see
“Viewing Teradata Crashdump Messages” on page 560 .)
What does it mean when there are “.Err” files present?
If you submit the following command in BTEQ:
HELP DATABASE CRASHDUMPS
; you might see .Err files, for example .Err1 and .Err2 files.
This may be normal if the system is still in the process of saving a crashdump. These error files will be cleared when the save completes. However, if the logs show that CSP did not successfully save the crashdump and the .Err files are left over (that is, they remain uncleared), this indicates that something went wrong while saving the dump.
What should I do if I am unable to save crashdumps?
If a CSP save failed, always check the log first. Then check the following items:
a
Are your host entries properly configured? Is the Host Group ID defined on all of the hosts?
The COP entries are needed in the hosts file for a client to login to the database. The copname must be configured in the
/etc/hosts
file.
In addition to the hostname entries, you must also define the hostname COP alias on each node for all the crashdumps tools accessing crashdumps. Set up is normally done during staging or configuration of client utilities. Consider the following for setting up
COP entries:
•
The cop alias entries should follow standard Teradata Network CLI protocols.
•
One of the cop entries in the file must be dbccop1. This name is used as a default for CLI. If you do not have dbccop1 in the hosts file then you get the following error:
MOSI: ER_HOST (117): DBC name not found - possible HOSTS file problem.
*** ERROR 204: Can't Init Things. Exiting!
•
The COP entries must be sequential or CLI will stop looking for COP entries in the first COP entry that does not return successfully. For example, if dbccop1, dbccop2, dbccop3, dbccop5 are entries in the hosts file, CLI will not use dbccop5 to connect to the Teradata Database server, because dbccop4 is missing.
If a Teradata Database system can connect to four COPs, the COP entries in the hosts file should look as follows:
Database Administration
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Troubleshooting Crashdumps
b
[IP address of NODE1] COPNAMEcop1
[IP address of NODE2] COPNAMEcop2
[IP address of NODE3] COPNAMEcop3
[IP address of NODE4] COPNAMEcop4
Note: You can have multiple COP entries per node as long as the COPNAME is different. For example:
39.64.8.9 smp001-4 dbccop1 hotwmacop1 wmahotcop1
39.64.8.10 smp001-5 dbccop2 hotwmacop2 wmahotcop2
39.64.8.11 smp001-6 dbccop3 hotwmacop3 wmahotcop3
39.64.8.12 smp001-7 dbccop4 hotwmacop4 wmahotcop4
Have you made any tuning changes to throttle the number of FastLoad sessions running at any given time? If a user FastLoad is running and crashdumps attempts to log on its session while using a PE-based host group, there may not be enough sessions available.
c
Does the Crashdumps database have enough space to save the dumps? See
“Configuring the Crashdumps Database” on page 560 for information on how to
check how much space Crashdumps has and how to allocate more space. You may also need to delete unwanted crashdumps to make more room.
4
5 d
When CSP cannot allocate more than the minimum number of sessions per node in the specified host group, you are unable to save crashdumps. Find a different time window when the system is more quiescent.
What if CSP runs out of sessions?
To save a crashdump, the system must be able to log on a minimum number of sessions as defined in the resource file for CSP. The default is 4 sessions per node. There is also a maximum number of internal sessions set on the system per PE.
If you notice that CSP runs out of sessions, run CSP at a different time when the system is more quiescent.
How can I tell if a crashdump is okay to use?
•
Use the CSP utility to check the raw PDE crashdumps with the following command: csp -mode list -source dump
The output will appear as follows:
# csp -mode list csp: Searching for dumps in raw dump directory /var/opt/teradata/tddump csp: 4 dumps found, 4 dumps to process csp: Sel ID (date-time-token) Nodes Event Instigator Status csp: --- ---------------------- ----- ----- ------------- -----csp: * 2008/09/09-17:03:13-02 1 csp: dump_20080909_180130 1
10196 16384/0/(11480)
16384 csp: 2008/08/14-11:35:58-01 1 csp: * 2008/07/11-14:41:58-01 1
10196 16384/0/(15160)
10416 1/11/(28272) csp: * 2008/07/01-11:58:42-05 1
#
10196 16384/0/(6418)
Required files are missing or corrupted
If the Status column reports “Required files are missing or corrupted” or “Dump belongs to a different PDE version,” the crashdump should not be used. If the Status column is blank, however, this means that the crashdump is okay to use.
•
Use the csppeek utility by submitting the following command at a Teradata command prompt for a saved crashdump: csppeek -i -d crashdumpname where crashdumpname is in the form of Crash_yyyymmdd_hhmmss_nn.
Database Administration 577
Appendix D: Handling Teradata Crashdumps: Operational DBAs
Troubleshooting Crashdumps
6
If csppeek reports information about the crashdump as follows:
Dump (version 20) contains 1 node, 10 vprocs, 35 pids, 839 tids, 0 memos
nodeids: 16384(I)
Software versions:
PDE 13.10.00.00
TDBMS 13.10.00.00
TGTW 13.10.00.00
TCHN 13.10.00.00
TDGSS 13.10.00.00
NODE 16384 is system noun
Dump token: Crash_080911_080707_01 instigator: 33//3268
Caused by logevent: 10196-60 reported by /32382720 (3268/16384/0) node 16384 contains vprocs 16384(N)
16383(P)
16382(P)
10238(V)
10237(V)
8192(G)
3(A)
2(A)
1(A)
0(A)
This means the crashdump is okay to use. If it does not report anything or reports an error, this indicates a problem with the crashdump because it could not read the crashdump information and you should not use the crashdump.
How do I get CSP to resume automatically saving crashdumps when it reports a 2644 failure (no more space in crashdumps database)?
When CSP reports this failure, CSP and cspslave will go to sleep until they are awakened by a utility named cspwake or by a tpareset. After you add more space to the Crashdumps database, you can run cspwake at a command prompt without any arguments to resume automatic dump saving. If automatic dump saving was not being done, then CSP would have to be manually started with the –force option instead.
578 Database Administration
APPENDIX E
Considerations for Administering
a System with Unity Director: All DBAs
In most cases, database administration is the same when you have Unity Director as it is in any
Teradata system. This appendix summarizes cases in which the presence of Unity Director changes recommended administrative procedures.
Creating a Database or User
Teradata recommends that DBAs create users and privileges through Unity Director. Unity
Director synchronizes passwords and privileges on all Teradata systems it controls. If a DBA creates users directly on the individual database systems, the DBA must make sure that user credentials and privileges are in synch on all Teradata systems involved.
Creating Roles and Profiles
Teradata also recommends creating roles and profiles through Unity Director to avoid having to synchronize roles and profiles on the multiple database systems Unity Director controls.
Creating Tables and Views
Tables and views under Unity Director control need to have the exact same structure on multiple systems. The easiest way to do this is to issue the SQL DDL for tables and views through Unity Director.
Loading and Exporting Data
Unity Director does not currently support FastLoad, MultiLoad, and FastExport. Unity
Loader supports Teradata Parallel Transporter (TPT) load and JDBC FastLoad methods to load data in bulk.
Database Administration 579
Appendix E: Considerations for Administering a System with Unity Director: All DBAs
Querying the AMPUsage and AccountInfoV Views
Querying the AMPUsage and AccountInfoV
Views
Unity Director uses the account string from a logon request as one of the values for determining routing mode across Teradata systems. Because their content is system-specific, query the AMPUsage and AccountInfoV views directly on the Teradata systems and not through Unity Director.
Managing Space
Unity Director interrupts table access when a table cannot be updated because it is out of space. Adding more space, however, is a system-specific activity with no Unity involvement.
Once more space is available, Unity Director automatically continues with table updates.
Restarting Teradata Database
If a Teradata system restarts, Unity Director automatically resubmits any in-progress SELECT requests to another Teradata system it controls. Unity Director makes the restart transparent to the users and applications that submitted SELECT requests.
Investigating Lock Contentions
Requests submitted through Unity Director are locked in both the database and Unity
Director. Consider the Unity Director locking layer also when investigating lock contentions.
To see the locks and blocked sessions related to Unity Director, see the Unity Director portlet in Viewpoint.
Analyzing Queries and Query Plans
Send requests with EXPLAIN modifiers directly to a Teradata system. Sending EXPLAINS through Unity Director sends the request to one system, but the request could execute on another, so the plan and execution may not be consistent.
When you have a desired query execution plan, develop code and optimize performance directly on the individual systems Unity Director controls. You can then send queries through
Unity Director, which submits them to any of the systems.
580 Database Administration
Appendix E: Considerations for Administering a System with Unity Director: All DBAs
Making Global and User-Level Configurations Consistent
Making Global and User-Level Configurations
Consistent
The systems Unity Director controls should have consistent global and user-level configurations. This is required to ensure data consistency when Unity Director submits SQL
DML to multiple systems. For more information, see Chapter 18: “Working with System
Information and Global Defaults: Operational DBAs.”
For More Information
Details about Unity Director are available in the Unity Director User Guide and the orange book Teradata Unity: Operational Uses & Best Practices. Access orange books through Teradata
@ Your Service: http://tays.teradata.com
.
Database Administration 581
Appendix E: Considerations for Administering a System with Unity Director: All DBAs
For More Information
582 Database Administration
Index
A
Access
tracking using query bands 234
Account string
Accumulators
count values, resetting 201 database space, resetting 201 resetting 201
Aggregate cache size, increase in 542
Aggregate results, performing 72
AMP efficiency, balanced data and 496
AMPUsage
ARC utility statements
Archiving
Archiving tables using temperature-based compression 210
ASE
interactions with utilities 169
Automatic privileges
for created users and databases 128
Automatic restarts
AWS console
troubleshooting Teradata Database 435
B
Baseline profiles, benefits of 404
BEGIN QUERY LOGGING statement 268
BYNET data macro, ResUsage and 549
C
Cache hit rates, ResUsage and 537
Capacity planning, ResUsage and 546
Channel connections, adding 551
Channel environment
session pools 429 sessions 429
Channel-attached clients
monitoring session parcels of 449
Character sets
client, viewing the status of 419
restrictions on creation of KANJI1 objects 418
Checking for
table corruption, see CheckTable utility table inconsistencies, see CheckTable utility
CHECKPOINT statement, ARC utility 215
CheckTable utility
ERROR ONLY 447 troubleshooting the Teradata Database with 447
Client character sets 419 viewing the status of 419
COLLECT STATISTICS
COLLECT STATISTICS (QCD Form) 399, 400
Compression
temperature-based and archiving 210
Copying a table using the AS option 81
Database Administration 583
Index
Counting the number of times the system accesses an object
CPU utilization, ResUsage and 546
Crashdumps
configuring DBC.Crashdumps database 560
database, modifying space for 562
manually saving on Linux or Windows 566
Creating QCD tables
CurrentTemp temporary space 183
Cursor logic stored procedure 91
Cylinder read, managing I/O 227
Cylinder space, Free Space Percent and 188
Cylinder splits, FreeSpacePercent and 528
Cylinders
freeing
D
Data
recovery using roll operations 215
uneven
Hash functions and 499 identifying 499
Data block
Data blocks
data blocks
Data dictionary
maintaining system logs 199 maintaining the log files 199
Data disks
permanent space usage 51 spool space usage 51 temporary space usage 51
Data distribution
Data protection
Data types, consistency across tables 73
Database commands
Database hierarchy
changing using GIVE statement 62
Database object use counts
resetting use count fields 237
Database privileges automatic
Database query logging
for performance management 490
Database Window Supervisor
Database Window, troubleshooting with 448
Databases
differences between users and 47
datablocks
DATABLOCKSIZE, performance and 532
DBC.AccessRights table
DBC.Acctg table, clearing with DBC.AMPUsage view 200
DBC.CharSetsV view, use of 419
DBC.CharTranslationsV view, use of 419
DBC.ClearPeakDisk macro
resetting peak space values in DBC.DatabaseSpace 201
DBC.DataBaseSpace table
correcting inconsistencies in 201 resetting peak values 201
DBC.DBase table, correcting inconsistencies in 201
584 Database Administration
DBC.Events_ConfigurationV view 224
DBC.HostsInfoV view, use of 419
DBC.ResUsage tables, maintaining 199
DBC.SecurityDefaultsV view
defining system-level password attributes 113
DBQL
controlling with SQL statements 268
how Teradata checks for rules 265
maintenance process, monthly 384
parameterized query log table 311
protection of committed rows 371
SQL statements
DBQL log tables
doing joins using the QueryID field 381
DBQL macro, DBQLAccessMacro 280
DBQL views
Database Administration
Index
DBS Control
Cylinders Saved for PERM setting 183
DBS Control fields
DBW
Deadlocks, preventing excessive 502
DIP utility
Disk space
DROP TEMPORARY TABLE statement 82
DULTAPE utility, saving PDE crashdumps to tape 569
Dump Unload/Load Tape utility. See DULTAPE
Dump Unload/Load utility. See DUL
E
END QUERY LOGGING statement 268
Events_ConfigurationV view 224
EXECUTE privilege
needed on DBQLAccessMacro for DBQL 391
F
Fallback
FastExport utility 141, 148, 149
FastLoad utility 146, 149 deciding if best for task 146
Ferret utility
finding tables for packing using SHOWFSP 189
making space with PACKDISK 190
585
Index
troubleshooting Teradata Database 437, 438
Free Space Percent
FREESPACE, performance and 527
operations disregarding 525 operations honoring 525
calculating FSG Cache misses 541 space in 541
Functions
database for embedded services 52
G
troubleshooting network connections 450
GIVE statement, transferring ownership 62
H
Heartbeat query. See Canary query
Host traffic, ResUsage and 549
I
International character support
J
Job scheduling
Jobs, bound, performance and 510
K
KANJI1
restriction on creation of objects 418
L
LIMIT SQLTEXT option for DBQL 276
LIMIT SUMMARY option for DBQL 276
Limits
586
setting Free Space Percent 188
LOAD utility, interpreting status 145
Locking
Lockout user duration, controlling 113
Locks, tools for analyzing 502
Login, as member of tdtrusted group 27
M
Macros
DBC.ClearAllDatabaseUseCount 237
Maxima
Memory
adjusting for low available free 451, 539
efficient use of 538 managing free memory 538
memory
MergeBlockRatio, DBS Control 187
MLOAD utility. See MultiLoad utility
N
Network environment 431 functionality 431
No Primary Index tables. See NoPI tables
Node
Non-FSG Cache size, minimum 539
Database Administration
Normalized view for coexistence 545
Null
NULLIF option, designing tables with 72
O
ObjectUseCountCollectRate, DBS Control
for rate of use count collection 236
Optional SQL clauses
Options for DBQL logging
P
PACKDISK command
Ferret utility 190 making space with 190
PACKDISK utility
Parameterized query log table 311
Parser fault isolation dumps 574
Partitioning
Passwords
defining system-level attributes 113
Peak disk space
clearing values from DBC.DatabaseSpace 201
Peak values
Performance
Performance management
Permanent space
increasing by giving ownership 178
Database Administration
Index
Permanent space, spool space and 517
Phantom spool, spool space accounting 518
Primary index
Priority Scheduler
Priority Scheduler data, ResUsage and 508
Privileges
EXECUTE, required for DBQL 391
Q
QCD database
DBS Control parameters for tuning workload cache limits for index and partition analysis 395
privileges for different user categories 402
QCF
capturing query plan information using the Query Capture
QCF and Query Capture Databases (QCD) 394
Queries
counting how often queries access database objects 235
fixing performance problems 493
Query
finding origin of using query bands 234
Query band
finding origin of query using 234
Query bands
Query Session utility, using to troubleshoot 443
587
Index
R
Recovery
Recovery Manager utility, using to troubleshoot 446
REPEAT command
Resetting use count fields 237
Resource usage
managing I/O with cylinder read 227
Resource usage tables
Restoring tables using temperature-based compression 210
ResUsage
priority scheduler data and 508
Rollbacks
canceling 507 noticing slow rollback on ALTER TABLE 507
Row-level security
collecting statistics on tables protected by 258
RSS logging
S
Saturated resources, finding 509
Secure zones
Security
row-level and statistics collection 258
Session
Showlocks utility
troubleshooting Teradata Database 441
Skewed distribution and primary index 500
Snapshot dumps
spool table fault isolation 575
Software
Space
increasing by giving ownership 178
peak disk, clearing values from DBC.DatabaseSpace 201
temporary, defining limits 181
updating with Update DBC utility 438 updating with Update Space utility 438
Specifications
database limits 471 database maxima 471
session limits 482 session maxima 482
system limits 465 system maxima 465
Spool space
Spool table fault isolation dumps 575
SQL and performance
SQL statements
COLLECT STATISTICS (QCD Form) 399, 400
Staging tables. See NoPI tables
Statistics
on join index columns 489 on partitions 489
Stored procedure, creating 91, 92
588 Database Administration
Index
Support tools
Teradata System Emulation Tool (TSET) 407
Query Capture Facility. See also QCF
allocating permanent storage space 51
System
logs
archiving 199 maintaining 199 purging 199
peak periods, ob scheduling and 510
System accounting
System alerts, thresholds for 486
System conditions, measuring
changes in data access 485 data growth 485 increase in active sessions 485
resource utilization 485 response time 485
System planning, charge-back billing 158
System reconfiguration, after expansion 551
System tables
considerations for creating 71
DBC.DataBasesSpace, correcting inconsistencies in 201
DBC.DBase, correcting inconsistencies in 201
maintaining by purging 199 purging the logs 199
System users
System views
troubleshooting with Software_Event_LogV 446
allocating permanent storage space to SYSTEMFE 51
SystemFE user, privileges for DBQL 391
T
Table Rebuild utility
troubleshooting primary or fallback tables 446 troubleshooting Teradata Database 446
Tables
copying AS a global temporary 80
tables
Target Level Emulation Facility
capturing cost data 405 capturing dynamic AMP sample statistical data 405
enabling Target Level Emulation 407
related tools and utilities 407
Target system (production system). See Target Level
tdnstat utility
troubleshooting network connections 450
TDP
exits 430 functionality 430 memory management 430
Temperature-based compression
archiving/restoring tables 210
Temporary space
Teradata crashdumps
deleting
saving 564 setting up the system to save dumps 564
Database Administration 589
Index
Teradata Database
viewing software release levels 417
Teradata Director Program. See TDP
Teradata Parallel Pump. See TPump utility
Tracking system object usage 235
Transaction rollback, and performance 503
Troubleshooting
Troubleshooting tools
channel and client connections 449
Gateway Control utility 450 network connections 450
U
UDT
Unhashed tables. See NoPI tables
Update DBC utility
troubleshooting Teradata Database 438
allocating permanent storage space for user DBC 50
User environment
controlling session defaults 426
User sessions
User-defined function, example of creating 100
Users
controlling lockout duration 113
590
Utilities
DULTAPE, saving PDE crashdumps to tape 569
Ferret and PACKDISK command 190
Ferret and SHOWFSP command 189
V
View
EXPLAIN log table QryLogExplainV 307
object log table QryLogObjectsV 310
parameterized query log 312, 313
query step information table QryLogStepsV 326
SQL log table QryLogSQLV 314, 315
summary log table QryLogSummaryV 335
TDWM-specific of main log table QryLogTDWMV 302
XML lock log table QryLockLogXMLV 366
XML plan log table QryLogXMLV 368, 370
Views
DBC.AMPUsage, clearing DBC.Acctg table 200
Database Administration
DBC.SecurityDefaultsV, defining system-level password attributes 113
Software_Event_Log and troubleshooting 446
W
WITH ALL option for DBQL logging 271
WITH OBJECTS option for DBQL logging 272
WITH SQL option for DBQL logging 273
Index
Database Administration 591
Index
592 Database Administration
Download
Link público atualizado
O link público para seu chat foi atualizado. Você pode visualizar e gerenciar seus links de chat após o registro aqui.
Link público do documento
Anúncio
