Aster Client Guide - Teradata Documentation

Aster Client Guide - Teradata Documentation
Aster Client Guide
Release Number: AC 5.11
Product ID: B700-2001-511K
Revision 1
September 4, 2013
The product or products described in this book are licensed products of Teradata Corporation or its affiliates.
Teradata, Active Data Warehousing, Active Enterprise Intelligence, Applications-Within, Aprimo, Aprimo Marketing Studio, Aster, BYNET,
Claraview, DecisionCast, Gridscale, MyCommerce, Raising Intelligence, Smarter. Faster. Wins., SQL-MapReduce, Teradata Decision Experts,
"Teradata Labs" logo, "Teradata Raising Intelligence" logo, Teradata ServiceConnect, Teradata Source Experts, "Teradata The Best Decision Possible"
logo, The Best Decision Possible, WebAnalyst, and Xkoto are trademarks or registered trademarks of Teradata Corporation or its affiliates in the
United States and other countries.
Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc.
AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc.
Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation
in the United States and/or other countries.
Apple, Mac, and OS X all are registered trademarks of Apple Inc.
Axeda is a registered trademark of Axeda Corporation. Axeda Agents, Axeda Applications, Axeda Policy Manager, Axeda Enterprise, Axeda Access,
Axeda Software Management, Axeda Service, Axeda ServiceLink, and Firewall-Friendly are trademarks and Maximum Results and Maximum
Support are servicemarks of Axeda Corporation.
Data Domain, EMC, PowerPath, SRDF, and Symmetrix are registered trademarks of EMC Corporation.
GoldenGate is a trademark of Oracle.
Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company.
Hortonworks, the Hortonworks logo and other Hortonworks trademarks are trademarks of Hortonworks Inc. in the United States and other
countries.
Intel, Pentium, and XEON are registered trademarks of Intel Corporation.
IBM, CICS, RACF, Tivoli, and z/OS are registered trademarks of International Business Machines Corporation.
Linux is a registered trademark of Linus Torvalds.
LSI is a registered trademark of LSI Corporation.
Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States
and other countries.
NetVault is a trademark or registered trademark of Quest Software, Inc. in the United States and/or other countries.
Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries.
Oracle, Java, and Solaris are registered trademarks of Oracle and/or its affiliates.
QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation.
Quantum and the Quantum logo are trademarks of Quantum Corporation, registered in the U.S.A. and other countries.
Red Hat is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Used under license.
SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc.
SPARC is a registered trademark of SPARC International, Inc.
Symantec, NetBackup, and VERITAS are trademarks or registered trademarks of Symantec Corporation or its affiliates in the United States and
other countries.
Unicode is a registered trademark of Unicode, Inc. in the United States and other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other product and company names mentioned herein may be the trademarks of their respective owners.
THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN "AS-IS" BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR
NON-INFRINGEMENT. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO THE ABOVE EXCLUSION
MAY NOT APPLY TO YOU. IN NO EVENT WILL TERADATA CORPORATION BE LIABLE FOR ANY INDIRECT, DIRECT, SPECIAL, INCIDENTAL,
OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS OR LOST SAVINGS, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
The information contained in this document may contain references or cross-references to features, functions, products, or services that are not
announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions,
products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or
services available in your country.
Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated
without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time
without notice.
To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document.
Please email: teradata-books@lists.teradata.com.
Any comments or materials (collectively referred to as "Feedback") sent to Teradata Corporation will be deemed non-confidential. Teradata
Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform,
create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata
Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including
developing, manufacturing, or marketing products or services incorporating Feedback.
Copyright © 2000-2013 by Teradata Corporation. All Rights Reserved.
Table of Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Conventions Used in This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Typefaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
SQL Text Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Command Shell Text Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Contact Teradata Global Technical Support (GTS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
About Teradata Aster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
About This Document. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Version History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Chapter 1: Installing Clients and Utilities . . . . . . . . . . . . . . . . . . . . . . . 12
Aster Client Support Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Aster Client Platform and OS Support Matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Aster ODBC Driver Support Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Obtaining Aster Client Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Installing the Aster Database Cluster Terminal (ACT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Install ACT on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Install ACT on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Install ACT on MacOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Install ACT on Solaris. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Installing and Configuring ODBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
ODBC Driver Manager Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Install ODBC on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Install ODBC on Linux, Solaris, or MacOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Install ODBC on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Configure DataDirect Driver Manager on Linux and AIX . . . . . . . . . . . . . . . . . . . . . . . . . 23
Install and Configure the unixODBC Driver Manager on Linux,
Solaris, AIX, and MacOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Installing the .NET Data Provider for Aster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Installing the Loader Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Aster Client Guide
3
Installing the Export Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Installing TD Wallet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Download TD Wallet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Install and Configure TD Wallet on Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Install and Configure TD Wallet on Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Chapter 2: Aster Client Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Teradata Wallet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Wallet Contents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
TD Wallet Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
SSL Security for Aster Clients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
SSL-Related Files and Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
SSL Settings on the Queen Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Setting Configuration Parameters on the Queen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Creating Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Setting up SSL for ODBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Encrypting Data Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Adding AD-Based SSO Authentication to ODBC Connections with SSL. . . . . . . . . . . . . 43
Setting SSL Parameters for the ODBC Client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Common SSL Configuration Scenarios for ODBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Setting up SSL for ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Setting Parameters on the Queen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Encrypting Communications from the Queen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Using Single Sign-on (SSO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Scenario 1: Queen Provides a Self-Signed Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate . . . . . . . . . . 54
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate . . . . . . . . . . 55
Scenario 4: Encrypting Communication from the Queen to the Client . . . . . . . . . . . . . . 56
Chapter 3: Aster Database Cluster Terminal (ACT) . . . . . . . . . . . 58
ACT Quick Start. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Launch ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Launch ACT on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Launch ACT on Linux or Solaris. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Launch ACT on Mac. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Launch ACT Directly on the Queen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Log In to ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4
Aster Client Guide
Startup Parameters for ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Using the “on-error-stop” Option in ACT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Use a Configuration File to Pass ACT Startup Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 65
How to Use ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Issue SQL Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Exit ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Page Through Query Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Throttle Query Results in ACT and Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
ACT Utility Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Repeat Previously Typed Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Tab Completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
ACT Commands (at the SQL Prompt) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Database Parameters Set with \set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Troubleshooting ACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
ACT Connection Hangs When Using SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Invalid User Name Error in ACT After Password Change . . . . . . . . . . . . . . . . . . . . . . . . . 79
Misleading Error Message Reports Problem With a Role Instead of With a User . . . . . . 79
Chapter 4: Connect Using Database Drivers . . . . . . . . . . . . . . . . . . . . 80
General Tips for Connecting Clients to Aster Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Recommended Character Set Is UTF-8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Supported Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
When Querying System Tables with ODBC, Set AUTOCOMMIT to 'OFF'. . . . . . . . . . . 81
ODBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Set Up ODBC for Perl Connectivity on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Set up ODBC for PHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
ODBC Usage Notes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
JDBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Aster JDBC Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Differences from the Legacy JDBC Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Before You Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Install the JDBC Driver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Use the JDBC Driver in a Java Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Parameters for Connecting through JDBC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Configuring the JDBC Log Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Behavior and Performance Settings for JDBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
The Copy, Install File, Uninstall File, and Download File Commands . . . . . . . . . . . . . . . 91
Using Client-Side Cursors in JDBC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Test JDBC Connect Program. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Configure Aster Database SQL Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Aster Client Guide
5
SQL Behavior Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Setting the SQL Behavior Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Syntax for ODBC Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Process SQL Statements in JDBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Process a Simple Query in JDBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
JDBC Troubleshooting and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Connect Reporting Tools to Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Connect Aqua Data Studio to Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Connect MicroStrategy to Aster Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Chapter 5: Tools for .NET Environments . . . . . . . . . . . . . . . . . . . . . . . . 104
SSIS Data Loading with the .NET Data Provider for Aster . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Sample Program for .NET. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Features Demonstrated in the Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Steps to Build and Test the Sample Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Source Code of Program to Demonstrate .NET Data Provider for Aster . . . . . . . . . . . . 116
Possible Exceptions and Resolutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Chapter 6: Loading Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Best Practices for Data Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Loading Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Scenario 1: Pre-Production Data Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Scenario 2: Loading in a Production Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Loading Best Practices Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Aster Loader Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Argument Flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Exit Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
Loading Data with the Loader Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
Removing NULLs from data on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Loading from Multiple Files Using a Map File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Hints for Successful Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Error Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Troubleshoot Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6
Aster Client Guide
Running Multiple Loaders Concurrently . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Load Stalls Upon Cancellation or Failure Encountered During a Load . . . . . . . . . . . . . 147
Load Fails on UNIQUE or PRIMARY KEY Violation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Invalid Input Syntax Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Single Quote Character Must be Escaped When Using the -q Option . . . . . . . . . . . . . . 147
Using the -C Option With Uppercase or Special Characters . . . . . . . . . . . . . . . . . . . . . . 148
Uppercase Characters are Passed as Lowercase if not Escape Quoted . . . . . . . . . . . . . . . 148
Issues Using Escape Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Chapter 7: Exporting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Aster Database Export Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Argument Flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Appendix 1: Encoding Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Aster ODBC Encoding Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Aster Client Guide
7
8
Aster Client Guide
Preface
This guide provides instructions for users and administrators of Aster Client, version AC 5.11.
If you’re using a later version, you must download a newer edition of this guide!
The following additional resources are available:
•
Aster Database upgrades, clients and other packages:
http://downloads.teradata.com/download/aster
•
Documentation for existing customers with a Teradata @ Your Service login:
http://tays.teradata.com/
•
Documentation that is available to the public:
http://www.info.teradata.com/
Conventions Used in This Guide
This document assumes that the reader is comfortable working in Windows and Linux/UNIX
environments. Many sections assume you are familiar with SQL.
This document uses the following typographical conventions.
Typefaces
Command line input and output, commands, program code, filenames, directory names, and
system variables are shown in a monospaced font. Words in italics indicate an example or
placeholder value that you must replace with a real value. Bold type is intended to draw your
attention to important or changed items. Menu navigation and user interface elements are
shown using the User Interface Command font.
SQL Text Conventions
In the SQL synopsis sections, we follow these conventions:
Aster Client Guide
•
Square brackets ([ and ]) indicate one or more optional items.
•
Curly braces ({ and }) indicate that you must choose an item from the list inside the braces.
Choices are separated by vertical lines (|).
•
An ellipsis (...) means the preceding element can be repeated.
9
Contact Teradata Global Technical Support (GTS)
•
A comma and an ellipsis (, ...) means the preceding element can be repeated in a commaseparated list.
•
In command line instructions, SQL commands and shell commands are typically written
with no preceding prompt, but where needed the default Aster Database SQL prompt is
shown: beehive=>
Command Shell Text Conventions
For shell commands, the prompt is usually shown. The $ sign introduces a command that’s
being run by a non-root user:
$ ls
The # sign introduces a command that’s being run as root:
# ls
Contact Teradata Global Technical Support
(GTS)
For assistance and updated documentation, contact Teradata Global Technical Support
(GTS):
•
Support Portal: http://tays.teradata.com/
•
International: 212-444-0443
•
US Customers: 877-698-3282
•
Toll Free Number: 877-MyT-Data
About Teradata Aster
Teradata Aster provides data management and advanced analytics for diverse and big data,
enabling the powerful combination of cost-effective storage and ultra-fast analysis of
relational and non-relational data. Teradata Aster is a division of Teradata and is
headquartered in San Carlos, California.
For more information, go to http://www.asterdata.com
About This Document
This is the “Aster Client Guide,” version AC 5.11. This edition covers Aster Clients version AC
5.11 and was first published on June 27, 2012.
Aster Client Guide
10
About This Document
Version History
December 2012: 5.0.3
April 2013: AC5.10
August 2013: AC5.11
Revision History
Aster Client Guide
Date
Description
September 4, 2013
Corrections to Installing Clients and Utilities, including
support matrix for Aster Clients and ODBC. Minor installation
procedure changes.
August 10, 2013
Initial Release AC5.11
11
CHAPTER 1
Installing Clients and Utilities
This section explains how to install various clients and utilities that complement your Aster
Database installation.
Aster Client Guide
•
Aster Client Support Matrices
•
Obtaining Aster Client Packages
•
Installing the Aster Database Cluster Terminal (ACT)
•
Installing and Configuring ODBC
•
Installing the .NET Data Provider for Aster
•
Installing the Loader Tool
•
Installing the Export Tool
•
Installing TD Wallet
12
Installing Clients and Utilities
Aster Client Support Matrices
Aster Client Support Matrices
Aster Client Platform and OS Support Matrix
This table lists the supported platforms and operating systems for each Aster Client. For
ODBC, there are also some additional requirements, listed in the Aster ODBC Driver Support
Matrix (page 15).
.
Table 1 - 1: Aster Client Platform Support Matrix
Client
Supported Platforms
Supported Versions
ACT
Windows 32-bit
Windows Server 2008 R2 Standard
Windows Vista Business
ODBC*
Windows 64-bit
Windows Server 2008 R2 Standard
Linux 32-bit
glibc 2.7 or above is required
Linux 64-bit
glibc 2.7 or above is required
Mac OSX
10.5
Solaris 32-bit
SunOS 5.10 on x86 and SPARC
Windows 32-bit
Windows Server 2008 R2 Standard
Windows Vista Business
Windows 64-bit
Windows Server 2008 R2 Standard
Linux 32-bit
glibc 2.7 or above is required
Linux 64-bit
glibc 2.7 or above is required
Mac OSX
10.5 or 10.6
Solaris 32-bit
SunOS 5.10 on x86 and SPARC
Solaris 64-bit
SunOS 5.10 on SPARC
AIX 32-bit
5.3, 6.1 and 7.1 with required libraries:
libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
AIX 64-bit
5.3, 6.1 and 7.1 with required libraries:
libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
JDBC
13
All Platforms except
Solaris 64-bit
JRE from Oracle(Sun) 1.5 or above
JRE from IBM 1.6
Aster Client Guide
Installing Clients and Utilities
Aster Client Support Matrices
Table 1 - 1: Aster Client Platform Support Matrix (continued)
Client
Supported Platforms
Supported Versions
.NET
Windows 32-bit
Windows Server 2008 R2 Standard
Windows Vista Business
Optional: Visual Studio 2008 Profession Edition or
higher edition (e.g. Premium, Ultimate)
Windows 64-bit
Windows Server 2008 R2 Standard
Optional: Visual Studio 2008 Profession Edition or
higher edition (e.g. Premium, Ultimate)
Loader
Windows 32-bit
Windows Server 2008 R2 Standard
Windows Vista Business
Windows 64-bit
Windows Server 2008 R2 Standard
Linux 32-bit
glibc 2.7 or above is required
Linux 64-bit
glibc 2.7 or above is required
Mac OSX
10.5
Solaris 32-bit
SunOS 5.10 on x86 and SPARC
AIX 32-bit
5.3, 6.1 and 7.1 with required libraries:
libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
AIX 64-bit
5.3, 6.1 and 7.1 with required libraries:
libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
Export
Windows 32-bit
Windows Server 2008 R2 Standard
Windows Vista Business
Windows 64-bit
Windows Server 2008 R2 Standard
Linux 32-bit
glibc 2.7 or above is required
Linux 64-bit
glibc 2.7 or above is required
Solaris 32-bit
SunOS 5.10 on x86 and SPARC
AIX 32-bit
5.3, 6.1 and 7.1 with required libraries:
libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
AIX 64-bit
5.3, 6.1 and 7.1 with required libraries:
libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
Aster Client Guide
14
Installing Clients and Utilities
Aster Client Support Matrices
* See the Aster ODBC Driver Support Matrix (page 15) for required libraries and ODBC driver
managers.
Aster ODBC Driver Support Matrix
This table lists the supported ODBC driver managers and other requirements for the Aster
ODBC driver on each operating system.
Table 1 - 2: Aster ODBC Driver Support Matrix
Operating System
Prerequisites
Windows 32-bit
Microsoft Driver Manager (included with the Windows OS)
Microsoft Visual C++ 2008 Redistributable Package (x86)
Windows 64-bit
Microsoft Driver Manager (included with the Windows OS)
Microsoft Visual C++ 2008 Redistributable Package (x64)
Linux 32-bit
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.2.12 or 2.2.14
With required libraries:
libgcc 3.4.6 or higher
glibc 2.7 or higher
Linux 64-bit
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.2.12 or 2.2.14
• unixODBC Driver Manager 2.3.1
With required libraries:
libgcc 3.4.6 or higher
glibc 2.7 or higher
MacOS
One of these supported ODBC driver managers:
• iODBC Driver Manager 3.52.3
• unixODBC 2.2.12 or 2.2.14
One of these supported libgcc:
• libgcc 4.2 for MacOS 10.5
• libgcc 4.5 for MacOS 10.6
Solaris x86 32-bit
unixODBC Driver Manager 2.2.12 or 2.2.14
libgcc 3.4.6 or higher
15
Solaris SPARC 32bit
unixODBC Driver Manager 2.2.12 or 2.2.14
Solaris SPARC 64bit
unixODBC Driver Manager 2.3.1
libgcc 4.2.0 or higher
libgcc 4.2.0 or higher
Aster Client Guide
Installing Clients and Utilities
Obtaining Aster Client Packages
Table 1 - 2: Aster ODBC Driver Support Matrix (continued)
Operating System
Prerequisites
AIX 32-bit
One of these supported ODBC driver managers:
• DataDirect Driver Manager (Included in the ODBC package)
• unixODBC Driver Manager 2.3.1
With required libraries:
libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
AIX 64-bit
One of these supported ODBC driver managers:
•
•
DataDirect Driver Manager (Included in the ODBC package)
unixODBC Driver Manager 2.2.14
With required libraries:
libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
Obtaining Aster Client Packages
To obtain the Aster Client packages:
1
Find the newest Aster Client package posted on Teradata Developer Exchange:
http://downloads.teradata.com/download/aster
2
Click on the package name for your operating system.
3
Log in to Teradata Developer Exchange, if prompted.
4
Accept the End User License Agreement.
5
Copy the package to the computer you'll use to query your database.
6
Expand the package by unzipping or untarring the file, as appropriate.
7
If necessary, set the correct file permissions to make the Aster Client files executable.
Installing the Aster Database Cluster Terminal
(ACT)
ACT is a terminal-based query tool that connects with Aster Database. ACT lets you type
queries, see the results, and save them to a file. Alternatively, you can source your queries from
a file. ACT supports connection via SSL and SSO and is available for Windows, Linux, Solaris
and Mac. Follow the instructions for your OS:
Aster Client Guide
•
Install ACT on Windows
•
Install ACT on Linux
16
Installing Clients and Utilities
Installing the Aster Database Cluster Terminal (ACT)
•
Install ACT on MacOS
•
Install ACT on Solaris
Install ACT on Windows
To install ACT on Windows:
1
Place the executable act.exe in a directory on your client machine. Teradata Aster
recommends that you create a directory, C:\Program Files\asterdata and place the
act.exe executable there.
2
Set permissions to make the file executable:
3
•
In Windows File Explorer, navigate to find the ACT executable file.
•
Right-click on it, select Properties, click Security.
•
In the To change permissions section, click Edit.
•
Click the name of the group who can use the application, and tick the checkbox for
Allow/Read & Execute in the bottom of the window.
•
Click OK and click OK again.
Test your ACT installation like this:
•
From the windows Start menu, choose Run..., type cmd, and click OK.
•
At the command line, change directories to the folder holding act.exe.
•
Type act --help at the command line. The help page should be printed to the
command line.
•
To connect to your Aster Database, type
act -h <ip address of your queen>
•
When prompted, type the database username and password (The default user/
password credentials are beehive/beehive) The <dbuser>=> prompt indicates you've
logged in successfully (i.e. beehive=> for the default user). Type \q to quit.
•
See the section “Launch ACT” on page 59 for information on running ACT.
Install ACT on Linux
To install ACT on Linux:
1
Place the executable in a directory on your client machine. Teradata Aster recommends
that you use the directory, /home/beehive/clients.
2
Change the file’s permissions to make it executable:
$ chmod 755 act
Tip! ACT for Linux requires glibc version 2.7 or higher. If you do not have glibc version 2.7 or higher, you must use the
IP address instead of the hostname for the -h flag when running ACT. To check the version of glibc, issue the command ldd --version.
3
17
See the section “Launch ACT” on page 59 for information on running ACT.
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
Install ACT on MacOS
To install ACT on MacOS:
1
Double click the .dmg file. A new icon with a name similar to the .dmg file name appears
on your desktop.
2
If a new Finder window doesn’t automatically appear, double click on the new icon that
has appeared on your desktop. A new Finder window appears.
3
Find the application’s icon in the Finder window. Drag and drop it into your Applications
directory.
4
See the section “Launch ACT” on page 59 for information on running ACT.
Install ACT on Solaris
To install ACT on Solaris:
1
Place the executable in a directory on your client machine. Teradata Aster recommends
that you use the directory /export/home/beehive/clients
2
Change the file's permissions to make it executable:
$ chmod 755 act
3
See the section “Launch ACT” on page 59 for information on running ACT.
Installing and Configuring ODBC
Teradata Aster provides a standard ODBC driver for Aster Database. The supported operating
systems for the ODBC driver are listed in the “Aster Client Platform and OS Support Matrix”
on page 13. Additional dependencies and supported driver managers are listed in the “Aster
ODBC Driver Support Matrix” on page 15.
The Aster Database ODBC driver may change in any Aster Database release. For this reason, with each new edition of
Aster Database, you should reinstall the driver.
ODBC Driver Manager Compatibility
The Aster Database ODBC driver requires a driver manager. Different platforms use different
driver managers. For a list of platforms and their supported ODBC driver managers, see the
“Aster ODBC Driver Support Matrix” on page 15.
Install ODBC on Windows
Follow the steps below to install the Aster Database ODBC driver on Windows.
Driver Manager Support
The Windows operating system includes Driver Manager, which allows ODBC applications to
use the correct driver.
Aster Client Guide
18
Installing Clients and Utilities
Installing and Configuring ODBC
Installation Procedure
1
Verify that Microsoft Visual C++ 2008 Redistributable Package (x86) is installed on the
system where you want to install the driver. You can do this by looking at installed
programs from the Control Panel.
Note that Microsoft also offers newer versions of the package, such as Microsoft Visual
C++ 2010 Redistributable Package (x86). Teradata Aster has not tested compatibility with
these later versions! If the supported version is not installed:
a
Download it from Microsoft, choosing the version that fits your architecture:
•
32-bit
•
64-bit
2
If you are upgrading, use the Windows Add/Remove Programs tool to uninstall the old Aster
ODBC version.
3
Double click on the Aster ODBC .msi file to install it. The actual file name depends on
your operating system:
4
•
Windows 32-bit: nClusterODBCInstaller_i386.msi
•
Windows 64-bit: nClusterODBCInstaller_x64.msi
A setup wizard walks you through the installation of Aster Database ODBC as one of the
available data sources on your computer. The default installation directory will be
something like:
•
For 32-bit Windows: C:\Program Files\Teradata Aster\Aster ODBC Driver
(x86)\
•
For 64-bit Windows: C:\Program Files\Teradata Aster\Aster ODBC Driver
(x64)\
For AC 5.0.2 and before, the ODBC driver name is Aster Data ODBC Driver for nCluster
19
5
From the Windows Control Panel, double click Administrative Tools to open the
Administrative Tools window.
6
Double click the Data Sources (ODBC) option to open the ODBC Data Source Administrator
dialog box.
7
Click the System DSN tab.
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
8
Click Add to open the Create New Data Source dialog box.
9
Select the Aster ODBC Driver data source from the list.
10 Click Finish.
The Aster Database Login window appears.
11 In the Aster Database Login window, enter the following information:
•
Data Source: Use this field to give this database connection an easy-to-recognize name.
•
Server: The hostname or IP address of your Aster Database queen.
•
Port: The port on which your Aster Database queen listens for client connections. The
default is 2406.
•
Database: The name of the database in Aster Database you want to connect to. Default
system database is beehive.
•
Aster Client Guide
Username: Database user name.
20
Installing Clients and Utilities
Installing and Configuring ODBC
•
Password: Database user’s password.
•
Fetch Count: See “Throttle Query Results in ACT and Aster Database” on page 69.
•
SSL Settings: See “SSL Security for Aster Clients” on page 39.
•
SSO Settings: See “Adding AD-Based SSO Authentication to ODBC Connections with
SSL” on page 43.
•
enable_quoted_identifiers: See “Quoted-Identifier Handling” on page 97.
•
enable_backslash_escapes: See “Escape Character Handling” on page 97.
•
Map NUMERIC/DECIMAL to DOUBLE: Teradata Aster recommends that you turn this
feature on.
•
Bytea As Varchar: The login window does not provide a check box for specifying that
values in a column of datatype bytea should be retrieved in a character representation
(as opposed to the default binary representation). However, you can set this later in the
Windows registry. See “Bytea-Stored Data Settings on Windows” on page 83.
12 Click OK to save the data source information.
13 Click Test Connection to test the connectivity to the database.
14 If the connection is working, a confirmation window appears. Click OK.
Your ODBC setup is complete. Now, in your applications that will query Aster Database, you
may connect to Aster Database as an ODBC data source.
Install ODBC on Linux, Solaris, or MacOS
To install the Aster Database ODBC driver on Linux, Solaris, or MacOS, follow the
instructions below.
Prerequisites
1
The Aster Database ODBC driver may require a specific version of one or more libraries,
depending on your operating system. Check the “Aster ODBC Driver Support Matrix” on
page 15 and ensure you have the correct libraries installed.
2
Ensure that these two libraries have been added to the LD_LIBRARY_PATH:
•
•
3
libgcc_s.so
libstd++.so
The Aster Database ODBC driver requires that you install a driver manager. Check the
“Aster ODBC Driver Support Matrix” on page 15 and ensure you have the correct ODBC
driver manager for your platform installed.
Instructions for installing are available here:
21
•
For unixODBC Driver Manager: See Install and Configure the unixODBC Driver
Manager on Linux, Solaris, AIX, and MacOS.
•
For DataDirect Driver Manager: You can find the supporting files in the ../stage/
clients-odbc-<your_platform>/DataDirect/ directory created when you
expanding the ODBC driver package. See “Configure DataDirect Driver Manager on
Linux and AIX” on page 23.
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
•
For iODBC driver manager: On MacOS, you have the option of using the iODBC
driver manager, version 3.52.3 instead of unixODBC driver manager. This driver and
its documentation are available from http://www.iodbc.org.
Installation Procedure
Install the Aster Database ODBC driver as shown below.
1
Extract the ODBC package for your operating system into the directory where you want to
install the driver.
2
Change to the Aster Database driver directory that was created when you extracted the
package:
# cd stage/clients-odbc-<your_client_os>
3
On Solaris 64-bit, this directory will contains a compressed file; extract it, too. You will see
two drivers. Choose ..\ODBCDriver\libAsterDriver.so
4
Edit your library path environment variable, to add the Aster ODBC library directory to it.
The Aster ODBC library directory has the path <install location>/stage/clientsodbc-<your_client_os>/Libs.
The library path variable is:
•
LD_LIBRARY_PATH on Linux and Solaris
•
DYLD_LIBRARY_PATH on MacOS
Edit the appropriate environment settings file to do this (for example, edit the ~/.bashrc
file if you want to set it for the current user on a typical Linux environment). To set it for
the current session only, type the command shown below, substituting
DYLD_LIBRARY_PATH for LD_LIBRARY_PATH on MacOS:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/stage
/clients-odbc-<your_client_os>/Libs
5
Add or edit the ODBCSYSINI environment variable, setting it to the directory where your
ODBC connection settings files (odbc.ini and odbcinst.ini) will reside. To follow this
example, let’s assume we are working as user “mjones” and will save the configuration files
to our home directory /home/mjones.
export ODBCSYSINI=/home/mjones
6
Check that the Aster Database ODBC driver library can find all its dependencies.
Assuming we have installed in /usr/local/lib, we would type (on Linux or Solaris):
# cd /usr/local/lib
# ldd stage/clients-odbc-<your_platform>/ODBCDriver/
<your_driver_library>
where <your_platform> is one of:
•
linux-64
•
linux-32
•
solaris-x86
•
solaris-sparc64
•
solaris-sparc (for 32-bit)
and where <your_driver_library> is one of:
Aster Client Guide
22
Installing Clients and Utilities
Installing and Configuring ODBC
•
libAsterDriver_unixODBC.so for unixODBC 2.2.12
•
libAsterDriver.so for unixODBC 2.2.14 or 2.3.1
On MacOS, we would type:
# cd /usr/local/lib
# otool -L stage/clients-odbc-mac/ODBCDriver/
<your_driver_library>
where <your_driver_library> is one of:
•
libAsterDriver_unixODBC.dylib for unixODBC 2.2.12
•
libAsterDriver.dylib for unixODBC 2.2.14 or 2.3.1
If a “not found” message does not appear, then all the required libraries have been linked.
7
Choose the next step, depending on your operating system and ODBC driver manager:
•
Configure DataDirect Driver Manager on Linux and AIX
•
Install and Configure the unixODBC Driver Manager on Linux, Solaris, AIX, and
MacOS
•
For MacOS, go to http://www.iodbc.org to download and configure iODBC driver
manager.
Install ODBC on AIX
Driver Manager Support
Aster ODBC on AIX supports either DataDirect Driver Manager or unixODBC driver
manager. The driver manager allows ODBC applications to use the correct driver (in this case,
the Aster Database ODBC driver) to connect to Aster Database.
Installation Procedure
Install the Aster Database ODBC driver on AIX.
1
Extract the bundle for the ODBC driver:
a
Unzip the file:
$ gunzip clients-odbc-aix.tar.gz
b
Untar the file:
$ tar -xvf clients-odbc-aix.tar
Configure DataDirect Driver Manager on Linux and AIX
Use the following process to configure the DataDirect Driver Manager on Linux and AIX:
1
Find the DataDirect Driver Manager configuration files under the extracted directory:
$ cd stage/clients-odbc-<your_client_os>/DataDirect/setup
2
Copy the odbcinst.ini, odbc.ini, and aster.ini files from the /setup directory to
your home directory:
$ cp -p * ~
3
Change to your home directory:
$ cd
23
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
4
Make backups of the files you moved and rename aster.ini to .aster.ini:
$
$
$
$
5
cp
mv
cp
cp
-p aster.ini aster.ini.backup
aster.ini .aster.ini
-p odbc.ini odbc.ini.backup
-p odbcinst.ini odbcinst.ini.backup
Make the following edits to .aster.ini:
a
Set DriverManagerEncoding to UTF-32.
b
Set ODBCInstLib to <InstallDir>/DataDirect/lib/libodbcinst.so, replacing
<InstallDir> with the folder where the driver is installed.
For example:
[driver]
DriverManagerEncoding=UTF-32
ODBCInstLib=<InstallDir>/DataDirect/lib/libodbcinst.so
DriverLocale=en-US
LogLevel=0
ErrorMessagesPath=<InstallDir>/ErrorMessages
6
Modify odbc.ini as follows:
a
Change the DSN configuration parameters SERVER, UID, PWD, DATABASE and
PORT.
[ODBC Data Sources]
... ...
asterdsn=AsterDriver
[ODBC]
... ...
[asterdsn]
Driver=<InstallDir>/DataDirect/lib/libAsterDriver.so
SERVER=192.206.82.100
PORT=2406
DATABASE=beehive
UID=beehive
PWD=beehive
b
Add this item to the [ODBC] section of odbc.ini:
InstallDir=<InstallDir>/DataDirect
7
8
Add the <InstallDir>/Libs directory to:
•
LD_LIBRARY_PATH for Linux, or
•
LIBPATH for AIX PowerPC.
Export the following values, where <directory_path> is the path to the directory where
the files odbc.ini and odbcinst.ini reside:
export ODBCHOME=<directory_path>
export ODBCINI=$ODBCHOME/odbc.ini
export ODBCINST=$ODBCHOME/odbcinst.ini
9
Edit the odbcinst.ini file, as shown in this example:
[ODBC Drivers]
... ...
AsterDriver=Installed
[ODBC Translators]
OEM to ANSI=Installed
Aster Client Guide
24
Installing Clients and Utilities
Installing and Configuring ODBC
[ODBC]
#This section must contain values for DSN-less connections
#if no odbc.ini file exists. If an odbc.ini file exists,
#the values from that [ODBC] section are used.
[AsterDriver]
Driver=<InstallDir>/DataDirect/lib/libAsterDriver.so
IconvEncoding=UCS-4LE
Install and Configure the unixODBC Driver Manager on Linux, Solaris, AIX,
and MacOS
On Linux, Solaris, AIX, and MacOS, Teradata Aster supports using the unixODBC driver
manager. This section describes how to install and configure unixODBC.
Install the unixODBC driver manager
If you already have the unixODBC manager installed, skip this section and proceed to
“Configure the unixODBC driver manager” on page 25.
1
2
Download the unixODBC driver manager from one of these links, depending on the
version you want to install:
•
For version 2.2.12: http://www.unixodbc.org/unixODBC-2.2.12.tar.gz
•
For version 2.2.14: http://www.unixodbc.org/unixODBC-2.2.14.tar.gz
•
For version 2.3.1: http://www.unixodbc.org/unixODBC-2.3.1.tar.gz
Install unixODBC by running the following commands:
# CFLAGS="-DSIZEOF_LONG=8" ./configure --prefix=/usr \
--sysconfdir=/etc/unixodbc \
--enable-fdb \
--disable-gui &&
make
3
Wait while it builds and installs.
4
Working as root user, run this, substituting your version of the driver where appropriate:
# make install &&
find doc -name "Makefile*" -exec rm {} \; &&
chmod 644 doc/{lst,ProgrammerManual/Tutorial}/* &&
install -v -m755 -d /usr/share/doc/unixODBC-2.2.12 &&
cp -v -R doc/* /usr/share/doc/unixODBC-2.2.12
5
Check the installation by typing the following command. This prints version information
and lists the names and locations of the configuration files you will need to install or set
up.
# odbcinst -j
If your unixODBC installation is not working properly, check the instructions at http://
www.unixodbc.org/ for help.
6
Proceed to the next section, Configure the unixODBC driver manager.
Configure the unixODBC driver manager
If you have chosen to use the unixODBC Driver Manager, configure it as follows:
25
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
1
Get the templates for the ODBC connection settings files. Copy these files from the Aster
Database driver’s Setup directory to the user’s home directory. The files you need are
aster.ini, odbc.ini and odbcinst.ini:
#
#
#
#
cd
cp
cp
cp
/usr/local/lib/stage/clients-odbc-linux64/Setup
odbc.ini ~
odbcinst.ini ~
aster.ini ~/.aster.ini
Note that we have also renamed the aster.ini file, adding a dot at the beginning of the
file name. You must do this.
2
Edit the .aster.ini file as follows:
a
Set DriverManagerEncoding to UTF-32.
b
Set ODBCInstLib to one of the following, depending on your OS:
•
on Linux: <unixODBC_InstallDir>/lib/libodbcinst.so
•
on Solaris: <unixODBC_InstallDir>/lib/libodbcinst.so
•
on MacOS: <unixODBC_InstallDir>/lib/libodbcinst.dylib
replacing <unixODBC_InstallDir> with the directory installed by the unixODBC
driver.
c
Set DriverLocale to en-US (or another locale or your choice).
d
Set LogLevel to 0 (or another level or your choice).
e
Set the ErrorMessagesPath to point to the ErrorMessages subdirectory in the Aster
Database driver directory. For this example, we edit the last line in the file to read:
ErrorMessagesPath=/usr/local/lib/stage/clients-odbc-<your_OS>/
ErrorMessages
Tip! At this point, you can run “odbcinst -j” to find out where the ODBC driver expects to find its configuration files.
3
Aster Client Guide
In a text editor, edit the odbc.ini file, making the following changes:
a
Set SERVER to the hostname or IP address of your Aster Database queen.
b
Set PORT to 2406, the standard port on which your Aster Database queen listens for
client connections.
c
Set DATABASE to the name of the database in Aster Database you want to connect to.
d
Optionally, you may set UID and PWD to your Aster Database SQL username and
password, respectively.
e
Finally, Teradata Aster recommends that you add the setting,
NumericAndDecimalAsDouble=1.
f
If you retrieve bytea-stored data through the ODBC driver, you can specify whether
values in a column of datatype bytea will be retrieved in a character representation, or
in the default binary representation. To have the ODBC driver to retrieve values in
26
Installing Clients and Utilities
Installing and Configuring ODBC
character representation, add the setting, ByteaAsVarchar=1 to your odbc.ini; if
you leave it unset, the driver preserves the binary output representation of bytea data.
g
Optionally, you can set a number of other database connection behavior settings.
These include enable_quoted_identifiers (see “Quoted-Identifier Handling” on
page 97), enable_backslash_escapes: See “Escape Character Handling” on
page 97.
For this example, we set the contents of odbc.ini to read:
[ODBC Data Sources]
Aster ODBC Driver DSN=AsterDriver
[Aster ODBC Driver DSN]
Driver=AsterDriver
SERVER=10.50.52.100
PORT=2406
DATABASE=beehive
NumericAndDecimalAsDouble=1
# UID=<USERNAME>
# PWD=<PASSWORD>
Tip! You can have multiple data sources. The name “Aster ODBC Driver DSN” in the odbc.ini file is just a default
name that Teradata Aster has given to the sample data source. You can rename this source and add more, as shown in
this example:
[ODBC Data Sources]
my_1st_source=AsterDriver
my_2nd_source=AsterDriver
[my_1st_source]
Driver=AsterDriver
SERVER=10.50.52.100
...
[my_2nd_source]
Driver=AsterDriver
SERVER=10.42.43.100
...
4
27
In a text editor, edit the odbcinst.ini file, setting the Driver parameter to the Aster
Database driver directory path. For this example, we set the contents of odbcinst.ini to
read:
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
Table 1 - 3: Driver parameter settings for unixODBC
Operating System
unixODBC Version
Driver Parameter Setting
Linux 32-bit
unixODBC 2.2.14
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-linux32/ODBCDriver/lib/
libAsterDriver.so
IconvEncoding=UCS-4LE
unixODBC 2.2.12
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-linux32/ODBCDriver/lib/
libAsterDriver_unixODBC.so
IconvEncoding=UCS-4LE
unixODBC 2.2.14
or 2.3.1
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-linux64/ODBCDriver/lib/
libAsterDriver.so
IconvEncoding=UCS-4LE
unixODBC 2.2.12
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-linux64/ODBCDriver/lib/
libAsterDriver_unixODBC.so
IconvEncoding=UCS-4LE
unixODBC 2.2.14
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-solaris-x86/ODBCDriver/
libAsterDriver.so
IconvEncoding=UCS-4LE
unixODBC 2.2.12
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-solaris-x86/ODBCDriver/
libAsterDriver_unixODBC.so
IconvEncoding=UCS-4LE
unixODBC 2.2.14
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-solaris-sparc/ODBCDriver/
libAsterDriver.so
IconvEncoding=UCS-4LE
unixODBC 2.2.12
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-solaris-sparc/ODBCDriver/
libAsterDriver_unixODBC.so
IconvEncoding=UCS-4LE
unixODBC 2.3.1
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-solaris-sparc64/ODBCDriver/
libAsterDriver.so
IconvEncoding=UCS-4LE
Linux 64-bit
Solaris 32-bit
x86
Solaris 32-bit
SPARC
Solaris 64-bit
Aster Client Guide
28
Installing Clients and Utilities
Installing and Configuring ODBC
Table 1 - 3: Driver parameter settings for unixODBC
Operating System
unixODBC Version
Driver Parameter Setting
AIX 32-bit
unixODBC 2.3.1
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-aix-ppc32/ODBCDriver/
libAsterDriver.so
IconvEncoding=UCS-4LE
AIX 64-bit
unixODBC 2.2.14
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-aix-ppc64/ODBCDriver/
libAsterDriver.so
IconvEncoding=UCS-4LE
MacOS
unixODBC 2.2.14
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-mac/ODBCDriver/
libAsterDriver.dylib
IconvEncoding=UCS-4LE
unixODBC 2.2.12
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-mac/ODBCDriver/
libAsterDriver_unixODBC.dylib
IconvEncoding=UCS-4LE
5
Save your changes to the odbcinst.ini file.
6
The installation and configuration are now complete.
Test the unixODBC Driver Manager
Once the driver is installed and configured, you should test it:
The unixODBC driver manager comes with a tool called “isql” that you can use to test the
driver. You can test your connection now by connecting to Aster Database using the isql tool.
1
Type the command as shown below. Here, we put the data source name in single quotes
because it contains spaces. The “beehive” and “beehive” are the default username and
password in a new Aster Database installation:
# isql -v 'Aster ODBC Driver DSN' beehive beehive
2
A “Connected!” message indicates that the driver is working correctly:
+---------------------------------------+
| Connected!
|
|
|
| sql-statement
|
| help [tablename]
|
| quit
|
|
|
+---------------------------------------+
SQL>
3
29
Set-up is complete.
Aster Client Guide
Installing Clients and Utilities
Installing the .NET Data Provider for Aster
Troubleshooting
If after installation you cannot connect and you see the error message:
[unixODBC][AsterData][ODBC] (11560) Unable to locate
SQLGetPrivateProfileString function
then do the following procedure:
1
Find the library libodbcinst.so (libodbcinst.dylib on MacOS) and note its path.
2
Set the LD_LIBRARY_PATH environment variable so that it includes the directory that
contains the library. For example, if the library is in /usr/lib64, then you will type:
export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
3
Open the .aster.ini file for editing.
4
Set the ODBCInstLib item to the correct path where the library libodbcinst.so (or
libodbcinst.dylib on MacOS) is located. For example, if the library is in the directory
/usr/lib64, the line should read:
ODBCInstLib=/usr/lib64/libodbcinst.so
Installing the .NET Data Provider for Aster
This section describes how to install the .NET Data Provider for Aster. This driver is a
managed .NET Data Provider for Aster Database and allows .NET client applications (for
example, Console, WinForms, ASP.NET, and Web Services) to send and retrieve data from
Aster Database.
Prerequisites
Make sure the .NET 2.0 framework is installed on your workstation before installing the .NET
Data Provider for Aster. You can download it from: http://www.microsoft.com/downloads
Procedure
Follow the steps below to install the .NET Data Provider for Aster:
1
2
Place the installer on your client machine:
•
For 32-bit, the installer is nClusterDNProviderInstaller_i386.msi
•
For 64-bit, the installer is nClusterDNProviderInstaller_x64.msi
Run the installer.
a
In the Welcome screen, click Next.
b
In the Select Installation Folder screen:
Choose the installation location. The default, depending on your version of Windows
will be something like one of:
C:\Program Files\Teradata Aster\.NET Data Provider for Aster (x86)\
C:\Program Files\Teradata Aster\.NET Data Provider for Aster (x64)\
Aster Client Guide
30
Installing Clients and Utilities
Installing the .NET Data Provider for Aster
Specify who should be allowed to use the driver by choosing Everyone or Just me.
Click Next.
c
In the Confirm window, click Next.
d
When the Installation Complete window appears, click Close.
Note: Before proceeding, make sure that the .NET Data Provider for Aster is installed correctly. There should be several DLL files and the RegisterAsterProvider.exe file in the .NET provider install path (for example, C:\Program Files
(x86)\Teradata Aster\Aster .NET Data Provider (x86)). If not, the .NET Data Provider for Aster will not work. You can
also check the installation from the Control Panel. If the size of Aster .NET Data Provider (x86) is less than 1 MB and
the version is 1.0.x, you need a new driver.
Installing SQLServer and SSAS after installing the .NET Data
Provider for Aster
It is recommended to install SQLServer and SSAS before installing the .NET Data Provider for
Aster. When you install the .NET Data Provider for Aster, the installer checks the Windows
registry to see if SSAS is installed. Then the correct files are installed and configured
automatically. But in cases where the .NET Data Provider for Aster is already installed before
SSAS/SQL Server, you will need to do the following steps before they can work together:
1
Find these four files:
•
Install.cmd
•
Install.vbs
•
Install.reg
•
Aster.xsl
These files can be found in the folder where the .NET Data Provider for Aster is installed,
typically something like C:\Program Files (x86)\Teradata Aster\Aster .NET Data Provider
(x86).
2
31
Install the files manually.
Aster Client Guide
Installing Clients and Utilities
Installing the Loader Tool
When installed, these files enable the .NET Data Provider for Aster to appear as an option
within Microsoft Visual Studio and Microsoft Analysis Services.
To install the files, run the Install.cmd file from the command line with the /codebase
option. The /codebase option should have the full path to the Simba.Net.dll file. The
syntax to run the command is:
Install.cmd /codebase <full path>\Simba.Net.dll
For example, the command will look similar to:
Install.cmd /codebase "C:\Program Files (x86)\Teradata Aster
\Aster .NET Data Provider (x86)\Simba.Net.dll"
If you are installing the .NET Data Provider for Aster to work with SQL Server 2005, you
must use the /vs2005 option, as in:
Install.cmd /vs2005 /codebase <full path>\Simba.Net.dll
3
Put the Aster Database cartridge (the file Aster.xsl) into the corresponding directory for
SSAS. This file should be placed into all the cartridge directories for SSAS, which may
include all of the following directories, though the directories on your Windows machine
may differ slightly:
SQL Server 2008
C:\Program Files\Microsoft Analysis Services\AS OLEDB\10\Cartridges
C:\Program Files\Microsoft SQL Server\MSAS10.MSSQLSERVER\OLAP
\bin\Cartridges
C:\Program Files\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\
IDE\DataWarehouseDesigner\UIRdmsCartridge
C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE
\PrivateAssemblies\DataWarehouseDesigner\UIRdmsCartridge
SQL Server 2005
C:\Program Files\Microsoft SQL Server\MSSQL.3\OLAP\bin\Cartridges
C:\Program Files\Microsoft SQL Server\90\Tools\Binn\VSShell\Common7
\IDE\DataWarehouseDesigner\UIRdmsCartridge
C:\Program Files\Microsoft Visual Studio 8\Common7\IDE
\PrivateAssemblies\DataWarehouseDesigner\UIRdmsCartridge
Installing the Loader Tool
Install the Aster Database Loader Tool on a data staging machine that has fast network
connectivity to your Aster Database loader nodes. The data staging machine is where you will
place the data files to be loaded. Note that if you are upgrading, you can simply replace
ncluster_loader with the newer version.
Install the Loader Tool on Linux
Aster Client Guide
1
Copy ncluster_loader to the client machine.
2
Set permissions to make the ncluster_loader file executable.
3
On Linux, ncluster_loader requires glibc 2.7 or later. Update glibc if needed.
32
Installing Clients and Utilities
Installing the Export Tool
Install the Loader Tool on Windows
After expanding the Aster Client package, there is no further installation required. Just run
ncluster_loader.exe from the command line.
Install the Loader Tool on AIX
After expanding the Aster Client package, there is no further installation required. Just set
permissions to make the ncluster_loader file executable.
Installing the Export Tool
To run the Aster Export Tool, copy it to the client machine and set permissions to make
ncluster_export executable.
Installing TD Wallet
TD Wallet supports SQL-MR functions and the following Aster Clients on all operating
systems except MacOS:
•
ACT
•
Loader Tool
•
Export
•
JDBC
•
ODBC
Download TD Wallet
To use TD Wallet with Aster Clients, you must first download it from http://
downloads.teradata.com/download/tools. The supported version of TD Wallet is version 14.00.
Install and Configure TD Wallet on Linux
ACT
1
Install TD Wallet.
2
Add a name/value pair to the wallet.
$ ./tdwallet add mypassword
Enter desired value for the string named "mypassword":
String named "mypassword" added.
3
Install the latest ACT for Aster Database.
4
Create the symbolic link “tdwalletdir” in the directory where ACT is installed.
For example, these commands add the symbolic link:
cd /home/beehive/work/multibranch/build/bin
ln -s /opt/teradata/client/tdwallet tdwalletdir
33
Aster Client Guide
Installing Clients and Utilities
Installing TD Wallet
JDBC
1
Install TD Wallet.
2
Add a name/value pair to the wallet.
$ ./tdwallet add mypassword
Enter desired value for the string named "mypassword":
String named "mypassword" added.
3
Get the JDBC Driver and tdwalletJNI.so.
4
Copy tdwalletJNI.so into /home/beehive/work/multibranch/builds/buildmain/lib/.
$ javac -classpath /home/beehive/work/multibranch/builds/build-main/
lib/noarch-aster-jdbc-driver.jar com/test/testjdbc/tdwallet.java
$ java -classpath .:/home/beehive/work/multibranch/builds/build-main/
lib/noarch-aster-jdbc-driver.jar -Djava.library.path=/home/beehive/
work/multibranch/builds/build-main/lib/ com.test.testjdbc
You can also find the tdwalletJNI.so file in the same directory as noarch-asterjdbc-driver.jar in these packages:
•
•
5
clients-platform-version-reversion.tar.gz
clients_all/platform...
Create the symbolic link “tdwalletdir” in the directory where tdwalletJNI.so is located.
For example, these commands add the symbolic link:
cd /home/beehive/work/multibranch/builds/build-main/lib/
ln -s /opt/teradata/client/tdwallet tdwalletdir
Install and Configure TD Wallet on Windows
ACT
1
Install TD Wallet.
2
Add a name/value pair to the wallet.
E:\>tdwallet add mypassword
Enter desired value for the string named "mypassword":
String named "mypassword" added.
3
Use the TD Wallet password instead of the real password to log in to Aster Database. For
example:
E:\asterclientWIN\win64\bin>act.exe -h 192.65.197.130 -U beehive
-w $tdwallet(mypassword)
Welcome to act 05.11.00.00, the Aster nCluster Terminal.
...
ODBC
Aster Client Guide
1
Install TD Wallet.
2
Add name/value pairs to the wallet.
3
Install the latest ODBC Driver for Aster Database.
4
Use the TD Wallet password ($tdwallet(mypassword)) instead of the real password to log in
to Aster Database.
34
Installing Clients and Utilities
Installing TD Wallet
$tdwallet(mypassword)
JDBC
1
Install TD Wallet.
2
Add name/value pairs to the wallet.
E:\>tdwallet add mypassword
Enter desired value for the string named "mypassword":
String named "mypassword" added.
3
Get the latest JDBC Driver, and make the dependent library libtdwalletJNI.dll
available in the library path.
E:\>javac -classpath e:\asterclient\lib\noarch-aster-jdbc-driver.jar
com\test\testjdbc\tdwallet.java
E:\>java -classpath e:\asterclient\lib Djava.library.path=e:\asterclient\lib com.test.testjdbc
4
35
Use the TD Wallet password ($tdwallet(mypassword)) instead of the real password to log in
to Aster Database.
Aster Client Guide
CHAPTER 2
Aster Client Security
This section explains the security options for Aster Clients:
•
Teradata Wallet
•
SSL Security for Aster Clients
•
Setting up SSL for ODBC
•
Setting up SSL for ACT
Teradata Wallet
Teradata Wallet (TD Wallet) is a software utility that provides the users of an Aster Database
client system with full and unrestricted access to their stored database passwords on that
system, while at the same time protecting those passwords from being exposed in scripts.
Each user on an Aster Database client has a TD Wallet. The TD Wallet securely stores the
passwords that the user adds to the wallet. However, a user cannot access the passwords stored
in the wallets of other users.
Rather than using the passwords in scripts, you can use their corresponding names defined in
your wallet.
TD Wallet supports SQL-MR functions and the following Aster Clients:
Aster Client Guide
•
ACT
•
Loader Tool
•
Export
•
JDBC
•
ODBC
36
Aster Client Security
Teradata Wallet
Wallet Contents
A wallet contains a set of <name, value> pairs. These pairs consist of Unicode character
sequences.
Table 2 - 4: Wallet contents
Item
Description
name
The name of this password entry. This is the name you use to reference the
password in scripts without exposing it. We recommend using meaningful
names to help you determine what the password is used for. The name is
case-sensitive.
value
The password. The password is not exposed in your scripts. Only the name of
this password is exposed.
The example in Table 2 - 5 shows entries in the TD Wallet of user client system user jdoe.
Table 2 - 5: The Teradata Database passwords of user jdoe
Name
Value
pwd_for_beehive_db
s4t#gp6s_#4
pwd_for_customers_db
nsdho_34f
pwd_for_clickstream_db
oc_m_3nd234
TD Wallet Commands
TD Wallet provides these commands:
Table 2 - 6: TD Wallet commands
37
Command
Description
tdwallet add name
Adds an item to the wallet. When you run this command,
tdwallet prompts you for the value component of the name/value
pair.
tdwallet addsk name
Adds a string with the specified name (saved-key).
tdwallet del name
Deletes the specified item from the wallet.
tdwallet list
Lists the contents of the wallet.
tdwallet chgpwd
Changes or establishes the password of the wallet.
tdwallet suppwd
Supplies the password of the wallet.
tdwallet forgetpwd
Forgets the password of the wallet.
tdwallet chgsavkey
Changes or establishes the saved-key.
tdwallet version
Displays the version information for tdwallet.
tdwallet help [topic] ...
Displays the help information for tdwallet.
Aster Client Guide
Aster Client Security
Teradata Wallet
Usage
After you install TD Wallet you can use the $tdwallet directive in place of your password. The
syntax if the directive is:
$tdwallet(name)
where name is the name of a password entry in the TD Wallet.
TD Wallet Password Processing Rules
When TD Wallet processes $tdwallet directives, it follows these rules:
1
If specified, process name as follows:
•
Replace “\)” with “)”
•
Replace “\$” with “$”
•
Replace “\\” with “\”
2
Query the corresponding current user’s wallet for an item with a name matching the result
of the processing in step 1.
3
Replace name with the value of the item found by the query in step 2.
Example using TD Wallet with Aster Loader Tool
In this example, we load the file sales_fact 2010MarchSales_data.txt into Aster
Database as the user beehive. TD Wallet replaces $tdwallet(beehive) with the
corresponding password stored on the client TD Wallet.
$ ./ncluster_loader -h 10.50.25.100 –w $tdwallet(beehive) -D "~"
sales_fact 2010MarchSales_data.txt
Example using TD Wallet with Aster Export
In this example, we export data from the Aster Database table mytable to the file
mydata.txt as the user mjones. TD Wallet replaces $tdwallet(mjones) with the
corresponding password stored on the client TD Wallet.
$ ./ncluster_export -U mjones -w $tdwallet(mjones) -h 10.50.52.100 -d
mydb mytable mydata.txt
Example using TD Wallet with ACT
In this example, we connect to Aster Database as the user beehive. TD Wallet replaces
$tdwallet(beehive) with the corresponding password stored on the client TD Wallet.
$ act -d beehive -h 10.42.52.100 -U beehive -w $tdwallet(beehive)
Example using TD Wallet in a SQL-MR query
In this query, TD Wallet replaces $tdwallet(abc) with the corresponding password stored
on the client TD Wallet.
select * from cfilter(
on (select 1)
partition by 1
database('beehive')
userid('beehive')
Aster Client Guide
38
Aster Client Security
SSL Security for Aster Clients
PASSWORD('$tdwallet(abc)')
inputtable('cfilter_test')
outputTable('cfilter_test1')
inputColumns('item')
joinColumns('tranid')
droptable('true'));
SSL Security for Aster Clients
You can set up Aster Database to use SSL for secure communications with the following Aster
Clients:
•
ODBC
•
ACT
Port Number
The Aster Database queen port number for SSL connections is the same as the regular client
connection port: 2406. Port 2406 is multiplexed to support both secure sockets layer (SSL)
connections and unencrypted connections.
SSL-Related Files and Settings
The queen has the following files related to SSL communications and set-up:
•
Private key: Default one is /home/beehive/certs/server.key
•
Self-signed PEM certificate (that is, the base64-encoded ASCII representation of the
certificate only; this is the version of the certificate that starts with "-----BEGIN
CERTIFICATE-----" and ends with "-----END CERTIFICATE-----"). The default one is /
home/beehive/certs/server.cert
•
Another copy of the self-signed certificate, also in PEM format, but this time formatted in
X.509 structure. Note! This file may be missing from your Aster Database installation. To
work around this problem, you must create the default PEM file from the server.cert
file, saving it as /home/beehive/certs/server.pem. To create it, log in to the queen as
root, change directories to /home/beehive/certs, and type:
# openssl x509 -in server.cert -text >> server.pem
•
The queen’s SSL-related settings (see “SSL Settings on the Queen Reference” on page 39)
in the Aster Database queen configuration file, /home/beehive/config/
procmgmtConfigs/coordinator.cfg
SSL Settings on the Queen Reference
This section provides a reference to the available SSL settings on the Aster Database queen.
Queen-Side SSL Parameters
Below, we list the configuration flags that can be used on the queen to tune SSL behavior. Most
queen-side flags have a corresponding client-side flag. When you change a flag on one side
(client or server), you will typically have to make appropriate changes to the other side.
39
Aster Client Guide
Aster Client Security
SSL Security for Aster Clients
•
disallowPeerWithoutCertificates: If this flag is set, the client cannot communicate
with its peer (server) without a valid certificate. This flag is defaulted to FALSE.
•
allowSelfSignedPeer: If this flag is set, Aster Database allows connections from clients
with self-signed certificates. Defaults to TRUE.
•
Set either trustedCAFileName or trustedCAPath, depending on whether you have one
or many CA certificates:
•
trustedCAFileName: The pathname of the single PEM-formatted CA certificate that
Aster Database trusts. (You also have the option of trusting multiple CA certificates;
see trustedCAPath, below.) Whenever the queen gets a certificate from the peer, the
queen traverses the certificate chain to verify that the certificate specified by
trustedCAFileName is part of the chain. If so, the peer is allowed to connect.
•
trustedCAPath: Directory containing PEM-formatted CA certificates that Aster
Database trusts. The files inside this directory are looked up based on the CA subject
name hash value.
•
sslCertificatePath: SSL certificate location.
•
sslPrivateKeyPath: SSL private key location.
•
sslFileType: The formatting type of the certificate. Set this to a string value of 1 for
PEM-encoded certs (called “SSL_FILESYSTEM_PEM” on the client side) or 2 for ASN1encoded certs (called “SSL_FILETYPE_ASN1” on the client side). Default is 1.
•
secureMuleServer: If set to true, Aster Database will be configured to use a secure
channel for its communication. If secureMuleServer is enabled, the configuration flags
sslCertificatePath and sslPrivateKeyPath should be appropriately set.
•
secureWrites: If set to true, encrypts communications going out to clients. Required to
be true if the client is set to encrypt reads (e.g. --ssl-encrypt-reads flag is set in ACT).
Setting Configuration Parameters on the Queen
Many procedures for setting up SSL ask you to set configuration parameters on the Aster
Database queen. Below, we explain how to do this.
Setting Persistent Parameters
To edit a setting (and have your edit survive reboots), make the setting as shown below.
Settings you change in this manner will survive reboots and soft restarts but they will not
survive upgrades.
1
Login as root into queen and use a text editor to open the file, /home/beehive/config/
procmgmtConfigs/coordinator.cfg
2
Find the “queenExec” section which looks like:
"taskName": "queenExec",
"nodeIps": "REPLACE_NODE_IP",
"executableLocation": "/home/beehive/bin/exec/queenExec",
"maxTaskRestarts": -1
3
Add the executableArgs section to the above section as shown below:
"executableArgs": "<flags in CSV format>"
Aster Client Guide
40
Aster Client Security
Setting up SSL for ODBC
For example, executableArgs for Scenario 1 (described earlier in this chapter) will look
like:
"taskName": "queenExec",
"nodeIps": "REPLACE_NODE_IP",
"executableLocation": "/home/beehive/bin/exec/queenExec",
"maxTaskRestarts": -1,
"executableArgs": "--disallowPeerWithoutCertificates=false,
--allowSelfSignedPeer=true,--trustedCAFileName=/home/beehive/certs/
server.pem,--sslCertificatePath=/home/beehive/certs/server.cert,
--sslPrivateKeyPath=/home/beehive/certs/server.key,--sslFileType=1"
4
Soft restart and activate the cluster.
You cannot configure the SSL flags on the queen to persist after Aster Database upgrades. The
above coordinator.cfg file will be overwritten the next time you upgrade! Before every
upgrade, make a copy of your configuration changes, and reapply them after the upgrade.
This step is part of the documented upgrade procedure.
Creating Certificates
Creating a Self-Signed Certificate
You may to create your own self-signed certificate. The high-level steps for doing this using
the OpenSSL tool are shown here, assuming you create a private key file called “host.key”:
openssl genrsa 1024 > host.key
chmod 400 host.key
openssl req \-new \-x509 \-nodes \-sha1 \-days 365 \-key host.key >
host.cert
Regenerating SSL Certificates/Private Key During Aster Database
Installation
You may manually regenerate the default private key and certificate using the following
command on the queen:
1
Working as root user, perform a soft shutdown on Aster Database:
# ncli system softshutdown
2
Wait for the system to shut down, and then run the resetCert command:
# /home/beehive/bin/lib/configure/ConfigureNCluster.py --resetCert
This will back up any existing server.key/server.cert files present in /home/beehive/certs
and generate a new private key and certificate file.
Setting up SSL for ODBC
By default, an ODBC client running on Windows or Linux sends all of its communications to
Aster Database over an SSL-encrypted connection. By default, results and other data returned
from Aster Database to the client are NOT encrypted. This default configuration can be
changed to suit your needs, as explained in the sections that follow.
41
Aster Client Guide
Aster Client Security
Setting up SSL for ODBC
On the client side, the files related to SSL and its configuration are:
•
Copy of the queen’s public key in PEM format. This is a copy of the queen’s server.pem.
For example, you might save it as /home/mjones/certs/server.pem
•
The client’s SSL-related settings (see “Client-Side SSL Settings” on page 42), stored:
•
for Linux, in the client’s odbc.ini file; and
•
for Windows, in the ODBC parameter fields of the registry
Client-Side SSL Settings
Below, we list the configuration flags that can be used on the client side to tune SSL behavior.
These settings work with the Aster Database ODBC client. Most client-side flags have a
corresponding queen-side flag. When you change a flag on one side (client or server), you will
typically have to make appropriate changes to the other side.
•
•
EnableSSL: Enables/disables the use of SSL (string value of 0 for false, or 1 for true).
•
0 = Disable SSL (default)
•
1 = Enable SSL
SSLEncryptReads: Determines whether the client expects data returned from database to
be encrypted (string value of 0 for false, or 1 for true).
•
•
0 = query results are unencrypted (the default)
•
1 = query results are encrypted
SSLAllowSelfSignedPeer: Determines whether the client allows peers with self signed
certificates to communicate (string value of 0 for false, or 1 for true; default is 1).
•
SSLFileType: The certificate file type. A string value; one of:
•
SSL_FILETYPE_PEM (the default)
•
SSL_FILETYPE_ASN1
•
SSLPrivateKeyPath: Path to the private key to be used. Optional. (A string value.)
•
SSLCertificatePath: Path to the SSL certificate to be used. (A string value.)
•
Set either SSLTrustedCADir or SSLTrustedCAFilename, depending on whether you
have one or many CA certificates:
•
SSLTrustedCADir: Path to the directory containing CA certificates in PEM format.
(A string value.)
•
SSLTrustedCAFilename: Filename of CA certificate in PEM format. (A string value.)
Encrypting Data Traffic
By default, Aster Database encrypts only control-path communications, such as the login
credentials an ODBC user submits, and the queries he or she submits. For implementations
that demand greater security, Aster Database also gives you the option of SSL-encrypting the
data traffic as the queen returns it to the client.
All other things being equal, switching any network connection from unencrypted to SSLencrypted has the effect of reducing the maximum available rate of data transmission on that
connection.
Aster Client Guide
42
Aster Client Security
Setting up SSL for ODBC
To set this up:
Queen-Side Settings
Make the following settings on the queen:
•
secureWrites=true
Client-Side Settings
Make the following settings on each ODBC client:
•
SSLEncryptReads=1
Adding AD-Based SSO Authentication to ODBC Connections with SSL
If your Aster Database is configured to authenticate users against Active Directory (“AD”), you
can configure your Aster ODBC clients to authenticate against AD, too. With this
configuration in place, each ODBC client will be required to authenticate against AD when it
tries to connect to Aster Database. If the ODBC client authenticates successfully, an SSL
channel is established automatically for communication between Aster Database and the
client. When your system is configured to use SSO, the \password command in ACT will not
be available.
Configuring Single Sign-on (SSO) with SSL
Queen-Side Settings
Make the following settings on the queen:
1
Set up Aster Database user authentication as explained in the Aster Database User Guide.
2
Configure the queen SSL configuration flags as described in one of the scenarios above
which you are planning on implementing. (For example, see “Scenario 1: Allowing
connections from clients without certificates” on page 49
Client-Side Settings
Do the following:
1
Set the flags in the client’s ODBC configuration file or registry as described in the scenario.
2
Set the EnableSSO flag in the client’s ODBC configuration file:
•
3
EnableSSO=1
If EnableSSO is set to 1, you must also:
•
Ensure that ServerIP is set to the fully qualified domain name of the Aster Database
queen and not to an IP address.
•
For 64-bit Linux machines: The ODBC driver assumes that libvas-gssapi.so is
present at /opt/quest/lib64/. If /opt/quest/lib64/libvas-gssapi.so does
not exist, locate libvas-gssapi.so by referring to the VAS documentation and set
the GSSPath parameter to point to the installed location of libvas-gssapi.so. For
example, if libvas-gssapi.so is deployed at /usr/lib64, then the GSSPath
parameter needs to be set to /usr/lib64 in the ODBC.ini config file as shown below:
GSSPath=/usr/lib64
43
Aster Client Guide
Aster Client Security
Setting up SSL for ODBC
•
For 32-bit Linux machines: The ODBC driver assumes that libvas-gssapi.so is
present at /opt/quest/lib/. If /opt/quest/lib/libvas-gssapi.so does not
exist, locate libvas-gssapi.so by referring to the VAS documentation and set the
GSSPath parameter to point to the installed location of libvas-gssapi.so. For
example, if libvas-gssapi.so is deployed at /usr/lib, then the GSSPath parameter
needs to be set to /usr/lib in the ODBC.ini config file as shown below:
GSSPath=/usr/lib
Setting SSL Parameters for the ODBC Client
This section explains how to set up the 2nd-generation Aster ODBC client for SSL
communications with Aster Database. Consult the section that applies to your operating
system:
•
“Linux ODBC DSN Set-Up”, below; or
•
“Windows ODBC DSN Set-Up” on page 45
Linux ODBC DSN Set-Up
On a UNIX-like systems, DSNs are added by setting parameters in the odbc.ini file
Sample ODBC.INI:
This sample assumes your queen machine is called cqueen.asterengqa.com and that you
are following Scenario 1 (outlined earlier in this chapter):
[ODBC Data Sources]
AsterTest=AsterDriverTest
[AsterTest]
Driver=AsterDriverTest
SERVER=cqueen.asterengqa.com
DATABASE=beehive
PORT=2406
UID=testuser13
PWD=testuser133
SQLSupportedConversions=3
NumericAndDecimalAsDouble=1
EnableSSO=0
GSSPath=
EnableSSL=1
SSLEncryptReads=0
SSLAllowSelfSignedPeer=1
SSLFileType=SSL_FILETYPE_PEM
SSLPrivateKeyPath=
SSLCertificatePath=
SSLTrustedCADir=
SSLTrustedCAFilename=
Sample ODBCINST.INI:
This sample assumes you have installed the driver in /Drivers/AsterDriver/ODBCDriver:
[AsterDriverTest]
Driver=/Drivers/AsterDriver/ODBCDriver/libAsterDriver_unixODBC.so
IconvEncoding=UCS-4LE
Aster Client Guide
44
Aster Client Security
Setting up SSL for ODBC
Sample aster.ini:
This sample assumes you want to log error messages in /Drivers/AsterDriver:
[driver]
DriverManagerEncoding=UTF-32
DSILogging=1
ErrorMessagesPath=/Drivers/AsterDriver/ErrorMessages
Windows ODBC DSN Set-Up
On Windows, DSNs are added by modifying the Windows Registry using regedit.exe or
using a .reg file. The standards surrounding key names (such as whether to use “Server” or
“servername”) used in a connection string for ODBC driver are loose, so please take care to
follow the examples we provide.
Details for setting the values in the registry are given below. Choose the section that fits your
needs and client type:
•
“Adding a driver to a Windows 32-bit operating system”, below.
•
“Adding a 64-bit driver to a Windows 64-bit operating system” on page 46
•
“Adding a 32-bit driver to a Windows 64 bit operating system” on page 47
Adding a driver to a Windows 32-bit operating system
Make the following registry settings:
Driver for 32-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\Aster ODBC
Driver (32-bit)]
"Driver"="C:\\AsterDriverWin32\Wow6432Node\ODBCDriver\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver32"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win32\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
The values in the keys above can be modified depending on where the driver is located on the
local machine and what the name of the driver should be. The values above are based on the
assumption that the driver folder is at "C:\AsterDriver-Win32" and the name of the driver is
"AsterDriver32". For an example .reg file that makes these settings, contact Teradata support.
45
Aster Client Guide
Aster Client Security
Setting up SSL for ODBC
DSN for 32-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\AsterDSN32]
"driver"="AsterDriver32"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
"SSLTrustedCAFilename"="\"\""
"EnableSSO"="0"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\ODBC Data Sources]
"AsterDSN32"="AsterDriver32"
Adding a 64-bit driver to a Windows 64-bit operating system
Below we list the registry entries for setting up a 64-bit driver on Windows. Make the registry
settings shown below:
•
64-bit driver for 64-bit Windows
•
64-bit DSN for 64-bit Windows
64-bit driver for 64-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\AsterDriver64]
"Driver"="C:\\AsterDriver-Win64\\ODBCDriver\\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver64"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win64\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
64-bit DSN for 64-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
Aster Client Guide
46
Aster Client Security
Setting up SSL for ODBC
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\AsterDSN64]
"driver"="AsterDriver64"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
"SSLTrustedCAFilename"="\"\""
"EnableSSO"="0"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\ODBC Data Sources]
"AsterDSN64"="AsterDriver64"
Above, the name of the DSN for the 64-bit driver is AsterDSN64. The server being connected
to is 10.51.12.100.
Adding a 32-bit driver to a Windows 64 bit operating system
Some applications running on a Windows 64-bit machine require a 32-bit driver. The 32-bit
operations work on Windows 64 bit machines under a mechanism called “Wow6432Node.”
Below we list the registry entries for setting up 32-bit drivers on Windows. Make the registry
settings shown below:
•
32-bit driver on 64-bit Windows
•
32-bit DSN on 64-bit Windows
32-bit driver on 64-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\AsterDriver32
]
"Driver"="C:\\AsterDriver-Win32\\ODBCDriver\\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver32"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win32\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
The values in the keys above can be modified depending on where the driver is located on the
local machine and what the name of the driver should be. The values above are based on the
assumption that the driver folder is at "C:\AsterDriver-Win32" for 32 bit drivers and the name
47
Aster Client Guide
Aster Client Security
Setting up SSL for ODBC
of the driver is "AsterDriver32". For an example .reg file that makes these settings, contact
Teradata support.
32-bit DSN on 64-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\AsterDSN32]
"driver"="AsterDriver32"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
"SSLTrustedCAFilename"="\"\""
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\ODBC Data
Sources]
"AsterDSN32"="AsterDriver32"
Above, the name of the DSN for the 32-bit driver is AsterDSN32. The server being connected
to is 10.51.12.100.
Common SSL Configuration Scenarios for ODBC
This section presents common SSL configuration scenarios and shows how to set up each one.
The scenarios are:
•
Scenario 1: Allowing connections from clients without certificates
•
Scenario 2: Client has a copy of the queen’s public key
•
Scenario 3: Client has a copy of a certificate you provide
•
Scenario 4: Client must present a valid, CA-signed certificate
In all of the scenarios, we instruct you to set parameters on the queen and on the client. For
instructions, see:
Aster Client Guide
•
For the queen: Setting Configuration Parameters on the Queen; and
•
For clients: Setting up SSL for ODBC
48
Aster Client Security
Setting up SSL for ODBC
Scenario 1: Allowing connections from clients without certificates
In this scenario, the Aster Database SSL configuration is edited to allow connections from
clients that have no certificate. In such a configuration, any Aster Database ODBC client can
establish an SSL-secured connection, provided the user authenticates successfully with Aster
Database. Communications from the client to Aster Database (for example, queries submitted
to the database) are SSL encrypted, and query results travel in the clear.
Queen-Side Settings
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=false
•
allowSelfSignedPeer=true
•
trustedCAFileName=/home/beehive/certs/server.pem
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
Ensure that secureWrites is set to false
•
Ensure that secureMuleServer is set to true
•
There is no need to set the trustedCAPath parameter.
Client-Side Settings
Make the following settings on each ODBC client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
•
There is no need to set the other SSL settings such as SSLPrivateKeyPath.
Scenario 2: Client has a copy of the queen’s public key
In this scenario, we edit Aster Database’s SSL configuration to allow connections only from
clients that have a copy of queen’s public key. This scenario uses the default public key
(/home/beehive/certs/server.pem) that is part of the standard queen installation.
Queen-Side Settings
Make the following settings on the queen:
49
•
disallowPeerWithoutCertificates=true
•
allowSelfSignedPeer=false
•
trustedCAFileName=/home/beehive/certs/server.pem
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
Ensure that secureWrites is set to false.
Aster Client Guide
Aster Client Security
Setting up SSL for ODBC
•
Ensure that secureMuleServer is set to true.
•
There is no need to set the trustedCAPath parameter.
Client-Side Settings
1
Copy the queen’s public key (self-signed certificate), /home/beehive/certs/
server.pem, to the client. For this example, we will assume the client will store the public
key as /home/jbloggs/certs/server.pem.
2
Make the following settings on each ODBC client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLTrustedCADir=/home/jbloggs/certs/server.pem
Scenario 3: Client has a copy of a certificate you provide
This scenario is the same as the preceding one, except that now we replace the default ASter
Database certificate with your site’s own certificate. As before, this configuration allows
connections only from clients that have a copy of queen’s public key. This scenario uses the
public key, /home/beehive/certs/sampleco.pem.
Queen-Side Settings
1
Get your public key file and save it on the queen. For this example, we will save it as
/home/beehive/certs/sampleco.pem on the queen.
2
Save the corresponding private key file on the queen. For this example, we will save it as
/home/beehive/certs/sampleco.key on the queen.
3
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=true
•
allowSelfSignedPeer=false
•
trustedCAFileName=/home/beehive/certs/sampleco.pem
•
sslCertificatePath=/home/beehive/certs/sampleco.pem
•
sslPrivateKeyPath=/home/beehive/certs/sampleco.key
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
There is no need to set the trustedCAPath parameter.
Client-Side Settings
Aster Client Guide
1
Copy the queen’s public key, /home/beehive/certs/sampleco.pem to the client. For
this example, we will assume the client will store the public key as /home/jbloggs/
certs/sampleco.pem.
2
Make the following settings on each ODBC client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
50
Aster Client Security
Setting up SSL for ODBC
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLTrustedCADir=/home/jbloggs/certs/sampleco.pem
Scenario 4: Client must present a valid, CA-signed certificate
This scenario presents a more strict regime that requires all clients to present CA-signed
certificates. In other words, clients cannot connect using a self-signed certificate, nor can they
connect using a copy of the queen’s public key.
Queen-Side Settings
1
Get the root certificate of the CA (certificate authority) that signed your client’s certificate.
Save the root certificate it on the queen. For this example, we will save it as /home/
beehive/certs/ca-cert.pem on the queen.
2
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=true
•
allowSelfSignedPeer=false
•
trustedCAFileName=/home/beehive/certs/ca-cert.pem
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
There is no need to set the trustedCAPath parameter if you use a single root certificate
for all clients.
Variation: If your client’s certificates were not all signed by the same CA, then you must set
Aster Database to recognize all the CA root certificates used to sign you clients’ certificates,
like so:
1
Save the root certificates of all the signing CAs on the queen.
2
Set trustedCAPath to point to the directory that contains the root certificates. For
example:
•
3
trustedCAPath=/home/beehive/certs
Un-set the queen configuration parameter, trustedCAFileName, by setting it to no value
at all. For example:
•
trustedCAFileName=
Client-Side Settings
1
Save the client’s public key on the client. For this example, we will assume the client will
store its public key as /home/jbloggs/certs/my-client-cert.pem.
2
Make the following ODBC settings on the client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=0
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLTrustedCADir=/home/jbloggs/certs/my-client-cert.pem
Repeat the above steps for all ODBC client machines in your environment.
51
Aster Client Guide
Aster Client Security
Setting up SSL for ACT
Setting up SSL for ACT
This section presents common SSL configuration scenarios and shows how to set up each one
when using ACT. For more general information on setting up SSL, see SSL Security for Aster
Clients (page 39).
In addition to the server settings, the command line argument --enable-ssl must be used
when invoking ACT. If you attempt to use an SSL argument (--ssl-self-signed-peer, for
example) without --enable-ssl, a warning message will display. In addition,
secureMuleServer=true must be set on the server if any SSL command line arguments are
used in ACT.
On the client side, the files related to SSL and its configuration are:
•
Copy of the queen’s public key in PEM format. This is a copy of the queen’s server.pem.
For example, you might save it as /home/mjones/certs/server.pem
•
The ACT SSL settings as described in this section.
The common SSL scenarios are:
•
Scenario 1: Queen Provides a Self-Signed Certificate (page 53)
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate (page 54)
•
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate (page 55)
•
Scenario 4: Encrypting Communication from the Queen to the Client (page 56)
Setting Parameters on the Queen
To set up any of the following SSL configurations, the parameter secureMuleServer=true
must be set on the queen. Each scenario also includes some additional SSL settings that must
be made on the queen. See Setting Configuration Parameters on the Queen (page 40) for
information on how to set these parameters.
Encrypting Communications from the Queen
By default, communications from the client to the queen (for example, passwords and queries
submitted to the database) are SSL encrypted, and query results travel in the clear. To encrypt
query results, follow the steps in Scenario 4: Encrypting Communication from the Queen to
the Client in addition to the steps to set up SSL.
Using Single Sign-on (SSO)
For information on setting up SSO on the queen, see Adding AD-Based SSO Authentication
to ODBC Connections with SSL (page 43). Note that when using SSO, the -U and -w options
are not used, because the username and password are passed directly to the host via SSO. Also,
when using SSO, you should specify a fully qualified hostname using the -h option, as in the
example: <hostname>.<domain>.<com|org etc>. If only the hostname is used, ACT will
append the local domain name before attempting to look up the host. Using an IP address
with -h is not supported with SSO.
Aster Client Guide
52
Aster Client Security
Setting up SSL for ACT
The following is an example of a configuration file that uses SSO:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: saturn.asterdata.com
dbname: sampledb
username: sampleuser
# SSL settings
enable-ssl: true
ssl-self-signed-peer: true
# SSO settings
enable-sso: true
Scenario 1: Queen Provides a Self-Signed Certificate
In this scenario, the server provides a self-signed certificate to the client at the time of
authentication. The certificate on the queen will be signed by the server itself, as opposed to by
a CA (certification authority). In such a configuration, any client can establish an SSL-secured
connection, provided the user authenticates successfully with Aster Database.
Queen-Side Settings
Make the following settings on the queen (note that these are the default settings for a new
Aster Database installation):
•
disallowPeerWithoutCertificates=false
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of “1” means SSL_FILETYPE_PEM.)
•
Ensure that secureWrites is set to false
•
Ensure that secureMuleServer is set to true
•
There is no need to set the trustedCAPath and trustedCAFileName parameters.
Client-Side Settings
Use the following command line arguments when executing ACT:
•
--enable-ssl
•
--ssl-self-signed-peer
Or use a config file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: 10.10.10.10
dbname: sampledb
username: sampleuser
53
Aster Client Guide
Aster Client Security
Setting up SSL for ACT
# SSL settings
enable-ssl: true
ssl-self-signed-peer: true
Note that the Client need not have a copy of the server's certificate. Do not use the other SSL
settings such as --ssl-trusted-ca-file, or --ssl-trusted-ca-path.
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate
In this scenario, ACT (the client) needs a CA-signed copy of the queen’s certificate. The
location of this certificate can be specified with --ssl-trusted-ca-file, or as a chain of
trusted certificates using the --ssl-trusted-ca-dir parameter. For more specific
information on using a certificate chain, see Queen-Side SSL Parameters (page 39).
Queen-Side Settings
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=false
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of "1" means SSL_FILETYPE_PEM. A value of “2” means
SSL_FILETYPE_ASN1.)
•
Ensure that secureWrites is set to false.
•
Ensure that secureMuleServer is set to true.
•
There is no need to set the trustedCAPath and trustedCAFileName parameters.
Client-Side Settings
Do the following:
Copy the queen's public key (self-signed certificate), /home/beehive/certs/server.pem,
to the client. For this example, we will assume the client will store the public key as /home/
jbloggs/certs/server.pem.
Use the following command line arguments when executing ACT:
•
--enable-ssl
•
--ssl-trusted-ca-file server.pem
•
--ssl-trusted-ca-dir /home/jbloggs/certs/
Or use a config file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: 10.10.10.10
dbname: sampledb
username: sampleuser
# SSL settings
enable-ssl: true
Aster Client Guide
54
Aster Client Security
Setting up SSL for ACT
ssl-trusted-ca-file: server.pem
ssl-trusted-ca-dir: /home/jbloggs/certs/
Because --ssl-self-signed-peer is not specified, the connection can be made only when
the server provides a CA signed certificate. The identical CA signed certificate must already
exist on the client. However, when --ssl-self-signed-peer is used, the server is able to
supply the certificate at the time of connection and nothing is required on the client.
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate
This scenario presents a stricter regime, where the queen only accepts connections from
clients that provide a CA-signed certificate, for which a copy already exists on the queen at the
time of connection. In other words, clients cannot connect using a self-signed certificate, nor
can they connect using a copy of the queen's public key. The identical signed certificate must
exist on the queen as well. Setting the server flag disallowPeerWithoutCertificates =
true is what forces the client to provide a certificate in order to connect.
Queen-Side Settings
Do the following:
1
Get the root certificate of the CA (certificate authority) that signed your client's certificate.
Save the root certificate on the queen. For this example, we will save it as /home/
beehive/certs/client.pem on the queen.
2
Make the following settings on the queen:
•
disallowPeerWithoutCertificates=true
•
trustedCAFileName=/home/beehive/certs/client.pem
•
sslCertificatePath=/home/beehive/certs/server.cert
•
sslPrivateKeyPath=/home/beehive/certs/server.key
•
sslFileType=1 (A value of "1" means SSL_FILETYPE_PEM. A value of “2” means
SSL_FILETYPE_ASN1.)
•
There is no need to set the trustedCAPath parameter if you use a single root
certificate for all clients.
•
Ensure that secureWrites is set to false.
•
Ensure that secureMuleServer is set to true.
Variation: If your client's certificates were not all signed by the same CA, then you must set
Aster Database to recognize all the CA root certificates used to sign you clients' certificates,
like so:
1
Save the root certificates of all the signing CAs on the queen.
2
Set trustedCAPath to point to the directory that contains the root certificates. For
example:
•
3
Un-set the queen configuration parameter, trustedCAFileName, by setting it to no value
at all. For example:
•
55
trustedCAPath=/home/beehive/certs
trustedCAFileName=
Aster Client Guide
Aster Client Security
Setting up SSL for ACT
Client-Side Settings
Use the following command line arguments when executing ACT. For this example, we will
assume the client will store the certificate as /home/jbloggs/certs/client.cert and the
key as /home/jbloggs/certs/client.key:
•
--enable-ssl
•
--ssl-certificate-path /home/jbloggs/certs/client.cert
•
--ssl-private-key-path /home/jbloggs/certs/client.key
•
--ssl-cert-filetype 1 (A value of "1" means SSL_FILETYPE_PEM. A value of
“2” means SSL_FILETYPE_ASN1.)
Or use a config file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: 10.10.10.10
dbname: sampledb
username: sampleuser
# SSL settings
enable-ssl: true
ssl-certificate-path: /home/jbloggs/certs/client.cert
ssl-private-key-path: /home/jbloggs/certs/client.key
ssl-cert-filetype: 1
Scenario 4: Encrypting Communication from the Queen to the Client
By default, only the queries issued by ACT to the queen are encrypted. It is also possible to
encrypt the communication from the queen to ACT. Note that these changes can be applied to
all of the previous scenarios. To do this, make the following changes in addition to the settings
necessary to enable SSL.
Queen-Side Settings
Make the following setting on the queen:
•
secureWrites=true
Even though you may have set secureWrites=false when setting up SSL, set it to true now
in order to enable encryption of communication from the queen. This permits setting up the
SSL connection before enabling two way encryption, if desired.
Client-Side Settings
Use the following command line arguments when executing ACT:
•
--ssl-encrypt-reads
Or add the following entry to the config file:
•
Aster Client Guide
ssl-encrypt-reads: 1
56
Aster Client Security
Setting up SSL for ACT
57
Aster Client Guide
Aster Database Cluster Terminal
(ACT)
CHAPTER 3
This section explains how to use Aster Database Cluster Terminal (ACT) to query and manage
databases. ACT is a terminal-based query tool that connects with Aster Database. ACT lets you
connect to the database (optionally using SSO and/or SSL), type queries, issue them to Aster
Database, and get query results. Alternatively, you can source your queries from a file. ACT
can return your query results to the command line or to a file, which makes it useful for
extracting data. Meta-commands and shell-like features are provided to facilitate writing
scripts and automating tasks.
Tip! Beginning with ACT version 4.6, the ACT client cannot connect to versions of Aster Database prior to version
4.6. If you attempt to connect to a pre-4.6 version of Aster Database with a 4.6 or later version of ACT, you will see an
error message indicating that there is a version mismatch between Aster Database and the client. You should obtain
the version of ACT that matches the version of Aster Database to which you are attempting to connect.
The following sections explain how to launch and use ACT:
•
ACT Quick Start
•
Launch ACT
•
Startup Parameters for ACT
•
How to Use ACT
•
ACT Commands (at the SQL Prompt)
•
Troubleshooting ACT
ACT Quick Start
You run ACT from the command line. The launch command is:
$ act -d <db name> -h <hostname> -U <username>
[-w <password>] [argument flags]
where
•
Aster Client Guide
db name is your database name,
58
Aster Database Cluster Terminal (ACT)
Launch ACT
•
hostname is the name or IP address of the queen,
•
username is your SQL login name,
•
password is your SQL password in Aster Database (this is optional; ACT prompts you for a
password if you do not pass a -w parameter), and
•
argument flags are any of the parameter/value combinations listed in “Startup Parameters
for ACT” on page 61.
For example:
$ act -d emea_sales -h 10.48.58.100 -U mjones
Tip! When using SSO (single sign-on), the -U and-w options are not used, because the username and password
are passed directly to the host via SSO.
To log in to the default database that is provided in your installation, type this, replacing the IP
address with the hostname or IP address of your Aster Database queen:
$ act -d beehive -h 10.42.52.100 -U beehive -w beehive
To see a list of ACT command line arguments, type:
$ act --help
Launch ACT
See the appropriate section below for instructions on launching ACT:
•
Launch ACT on Windows
•
Launch ACT on Linux or Solaris
•
Launch ACT on Mac
•
Launch ACT Directly on the Queen
Tip! On an Aster Database where LDAP authentication is enabled, if during logon an ACT user gets the error message:
'ERROR: An internal error has occurred.',
make sure the username is present in Aster Database with proper privileges.
•
Log In to ACT
Launch ACT on Windows
59
1
Open a command prompt.
2
Change directories to the folder which contains the act.exe executable.
3
Log In to ACT.
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Launch ACT
Launch ACT on Linux or Solaris
1
Ensure that your path includes the directory where ACT is installed.
2
Log In to ACT.
Tip! ACT for Linux requires glibc version 2.6.18 or higher. If you do not have glibc version 2.6.18 or higher, you must
use the IP address instead of the hostname for the -h flag when running ACT. To check the version of glibc, issue
the command ldd --version.
Launch ACT on Mac
1
Open a terminal.
2
Log In to ACT.
Launch ACT Directly on the Queen
Your Aster Database queen also contains an installation of the ACT client. To run it there,
you’ll need a user account on the queen machine, and you’ll need an SSH client on your
workstation.
To run the ACT on the queen:
1
Open a SSH connection to the queen. If you do not have a user account on the queen, ask
the machine’s administrator for one.
Tip! If you need an SSH client, do one of the following:
• Install the OpenSSH client as explained on the OpenSSH homepage at http://www.openssh.org/
• or, for Windows only, install the PuTTY SSH client.
2
Change directories to the directory where ACT is installed (by default, /home/beehive/
clients.
3
Log in to ACT:
$ act -d <db name> -U <username> -w <password> [argument flags]
Note that if you do not provide the hostname using -h, ACT defaults to the localhost. For
details on the command line options, see “Startup Parameters for ACT” on page 61
Log In to ACT
1
Run ACT by typing a command like:
act -d <db name> -h <hostname> -U <username> [-w <password>]
[argument flags]
For details on the command line options, see “Startup Parameters for ACT” on page 61
2
Aster Client Guide
Provide your database password by:
•
adding –w <password> to the ACT login string, or
•
omitting the -w argument and providing your database password at the prompt.
60
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
3
Choose a database by adding -d <database name> to the ACT login string. If -d is not
used, ACT places you in the system database (with the default name “beehive”).
4
You will see a welcome message, followed by the database prompt, which shows the
database name, followed by “=>”. For example:
Welcome to act AC 5.11, the Aster Database Terminal.
beehive=>
Startup Parameters for ACT
ACT takes a variety of startup parameters when you launch it. (Don’t confuse these with
ACT’s at-the-SQL-prompt flags, which you can read about in “ACT Commands (at the SQL
Prompt)” on page 73.)
You can pass these parameters at the command-line (see “Using the “on-error-stop” Option in
ACT” on page 64) or in a configuration file or “Use a Configuration File to Pass ACT Startup
Parameters” on page 65). The same startup parameter cannot be repeated in a single
invocation of ACT, or an error will be returned.
To list the command line parameters, type act --help.
Descriptions of each parameter are below:
Table 3 - 1: Summary of the most common command-line parameters for ACT
Flag
Description
-d [ --dbname ] DBNAME
Specify database name to connect to
(default: “beehive”).
-h [ --host ] HOSTNAME
Aster Database server host (default:
“localhost”).
-U [ --username ] NAME
Aster Database username (default:
“beehive”). Not used with SSO.
-l (the letter “ell”)
List available databases, then exit.
[ --list-databases ]
Tip! Note the default values for the connection parameters. If you do not specify the parameters -d (database
name), -h (hostname), -U (username), and/or -p (port) in the connect string, ACT will use the default values.
The default values are:
• “beehive” for database name
• “localhost” for hostname
• “beehive” for username, and
• “2406” for port.
If -w is not used, ACT will prompt for a password.
61
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Table 3 - 2: General-purpose command-line parameters for ACT
Flag
Description
-d [ --dbname ] DBNAME
Specify database name to connect to (default:
“beehive”).
--config-file FILENAME
Lads startup parameters from a configuration file
specified by FILENAME. See “Use a Configuration File
to Pass ACT Startup Parameters” on page 65 for more
information.
-c [ --single-command
]COMMAND
Run only single command (SQL or internal) and exit.
For example:
act -c "COPY MyTable FROM stdin;" <
myDataFile.dat
-f [ --input-file ] FILENAME
Execute commands from file, then exit. Run a SQL
script.
-1 (the numeral “one”)
Execute command file as a single transaction.
[ --single-transaction ]
-l (the letter “ell”)
List available databases, then exit.
[ --list-databases ]
-? [ --help ]
Show command line help, then exit.
-V [ --version ]
Output version information, then exit.
--on-error-stop
Enables the “on-error-stop” option, by default this
option is disabled. See “Using the “on-error-stop”
Option in ACT” on page 64 for more information.
or
-E
Table 3 - 3: Input- and output-related command-line parameters for ACT
Flag
Description
-a [ --echo-script-input ]
Echo all input from script.
-e [ --echo-all-input ]
Echo commands sent to server.
-o [ --redirect-query-results ]
FILENAME
Send query results to file (or | pipe).
Table 3 - 4: SSL and SSO related command-line parameters for ACT
Aster Client Guide
Flag
Description
--enable-ssl
Enables Secure Socket Layer (ssl) support. Must be used if
any of the other SSL/SSO arguments are used.
62
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Table 3 - 4: SSL and SSO related command-line parameters for ACT (continued)
Flag
Description
--ssl-encrypt-reads
SSL Encrypt Reads. Must be used if secureWrites=true
on the server. Conversely, must not be used if
secureWrites=false on the server. See Setting
Configuration Parameters on the Queen (page 40) for
information on how to set the secureWrites parameter
on the server.
--ssl-self-signed-peer
Indicates that ACT will connect to a Queen which will
provide a self-signed certificate.
--ssl-private-key-path
PATH
The SSL Private Key Path indicates where the private key is
stored on the client (ACT) machine.
--ssl-certificate-path
PATH
The SSL Certificate Path indicates where the certificate is
stored on the client (ACT) machine.
--ssl-trusted-ca-dir
DIRECTORY
When using a chain of certificates rather than a single
certificate, use the SSL Trusted CA Dir to set the directory on
the client machine where the chain of trusted certificates is
stored.
--ssl-trusted-ca-file
FILENAME
Use SSL Trusted CA Filename to provide the location of the
signed copy of the server’s certificate on the client machine.
--ssl-cert-filetype ARG
SSL Certificate File Type (use 1 for PEM; 2 for ANS1; default:
0).
--enable-sso
Enables Single sign-on (SSO) support.
--gss-lib-path PATH
For Linux, sets the GSS shared library path (default on linux
is /opt/guest/lib32 or /opt/guest/lib64). Ignored on
Windows.
Tip! The SSL settings in ACT have interdependencies, and in most cases they rely on the SSL settings on the queen.
See Setting up SSL for ACT (page 52).
Table 3 - 5: Output format-related command-line parameters for ACT
63
Flag
Description
-q [ --quiet ]
Run quietly and do not print messages, only query
output. Use this for clean query output. Often used
with the -c flag.
-t [ --print-rows-only ]
Print rows only.
-x [ --expanded ]
Turn on expanded table output.
-A [ --unaligned ]
Turn on unaligned table output
-F [ --field-separator ] ARG
Set field separator (default: '|')
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Table 3 - 6: Connection-related command-line parameters for ACT
Flag
Description
-h [ --host ] HOSTNAME
Aster Database queen hostname or IP address
(default: "localhost"). Note that ACT supports glibc
version 2.6.18 or higher. If you do not have glibc
version 2.6.18 or higher, you must use the IP address
instead of the hostname. To check the version of
glibc, issue the command ldd --version.
When using SSO, you should specify a fully qualified
hostname using the -h option, as in the example:
<hostname>.<domain>.<com|org etc>. If
only the hostname is used with SSO, ACT will
append the local domain name before attempting to
look up the host. Using an IP address with -h is not
supported with SSO.
-p [ --port ] PORT
Aster Database server port (default: "2406").
-U [ --username ] USERNAME
Aster Database username (default: "beehive").
-w [ --password ] PASSWORD
Aster Database password. This parameter is
optional; ACT will prompt for a password if you do
not pass a -w parameter. Not used with SSO.
Using the “on-error-stop” Option in ACT
The “on-error-stop” option can be used to stop ACT if an error occurs while running queries.
It discontinues executing the multi-statement SQL or the SQL file.
When the “on-error-stop” option is set, ACT halts query processing and exits as follows:
•
If ACT is in interactive mode, it returns to the ACT prompt when meeting an error.
•
If ACT is executed in the command line, it exits with status “3” when meeting an error.
Note! Unless the “-f” or “-c” options are used, ACT is always in interactive mode once it
is launched. The “\set” command can be used at the command line at any time. For more
details on the \set syntax for the “on-error-stop” option, see “Database Parameters Set
with \set” on page 78.
Setting “on-error-stop” in the command line
To enable the “on-error-stop” option (by default this option is disabled) from the command
prompt, use either of the following parameters:
• --on-error-stop
• -E
For example, to enable the “on-error-stop” in a running session of ACT, type the following
at the ACT prompt:
beehive$ ./act -h 153.65.197.120 -U beehive -w beehive
-E -f test_queries.sql
Aster Client Guide
64
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
beehive$ ./act -h 153.65.197.120 -U beehive -w beehive
--on-error-stop -f test_queries.sql
Use a Configuration File to Pass ACT Startup Parameters
Specifying all the necessary startup parameters for ACT on the command line can become
cumbersome, especially when using SSL. Because of this, Teradata Aster has provided a way to
specify startup parameters in a configuration file, to streamline starting ACT. On Windows
and Linux/UNIX-based operating systems, ACT looks for the configuration file in a specific
location and loads it automatically. Alternatively, the configuration file may be invoked using
the --config-file parameter when launching ACT.
All of the startup parameters in ACT are supported in the config file, except for the following:
•
--config-file
•
-V [ --version ]
•
-? [ --help ]
To use a configuration file, first create a text file of startup parameters. The following rules
apply when creating the config file:
1
Lines starting with a # character are ignored (considered as comments).
2
Blank lines are ignored (including lines containing just spaces).
3
Parameters are entered using the format
flagname: value
where flagname is same as the name of the command line flag without the preceding
hyphens (--) and value is the flag value as it would be provided on the command line.
Note that the short notations of flags are not supported. For example:
host: <ip>
will work but the following:
h: <ip>
will not work.
4
Flags which do not take any argument on the command line should be given a value of
either true or false.
5
Flag names are case-sensitive.
6
If the config file includes invalid flag names or repeated entries, ACT will not launch, and
an error will display.
7
If the config file includes the “on-error-stop” option with the parameters set to enable
this option, ACT will stop if an error occurs while running SQL queries. See Set “on-errorstop” in the ACT config file (page 65) for information on setting this option.
Set “on-error-stop” in the ACT config file
To enable the “on-error-stop” option, add it to the ACT config file: "actconfig.ini" and set
the value to "true". Set the value to "false" to turn off this option. By default, this feature is
disabled or “off ”.
The following is an example from of a config file for ACT:
65
Aster Client Guide
Aster Database Cluster Terminal (ACT)
How to Use ACT
# ACT configuration file example
# Contains setting to enable on-error-stop
host: 10.10.10.10
dbname: sampledb
username: beehive
on-error-stop: true
The default location for the config file on Linux/UNIX-based systems is $(HOME)/
.actconfig. For Windows, the default location for the config file is %HOMEPATH%/
actconfig.ini. Upon launching, ACT will look for the config file in the default location,
and if found, will use it by default.
Command line flags can be specified when starting ACT, in addition to the flags in the
configuration file. If a flag is present in the config file and specified at the command line as
well, the value on the command line will override the value in the config file. ACT will notify
the user that the command line flag was used upon startup.
The following is an example of a config file for ACT:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and
database
host: 10.10.10.10
dbname: sampledb
username: sampleuser
# SSL settings
enable-ssl: true
ssl-self-signed-peer: true
ssl-encrypt-reads: false
To start ACT, explicitly using the config file, issue a command like this example:
$ act --config-file /home/beehive/.act_ssl_config
To start ACT, explicitly using the config file and also specifying an additional parameter to
redirect query results to a file for this session only, issue a command like this example:
$ act --config-file /home/beehive/.act_ssl_config -o /home/beehive/
query_results_file
How to Use ACT
Issue SQL Queries
To use ACT to run a SQL query against Aster Database, you can do one of the following:
•
Enter the query at the ACT command line
•
Run the query from a file
However, before you can run queries, you must first launch ACT, as described in “Launch
ACT” on page 59.
For example, to run queries against the retail_sales database, run this command on the Queen
to launch ACT:
Aster Client Guide
66
Aster Database Cluster Terminal (ACT)
How to Use ACT
[root@localhost ~]# act -d retail_sales –U beehive –w beehive
Welcome to act AC 5.11, the Aster nCluster Terminal.
Type:
\copyright for distribution terms
\h for help with SQL commands
\? for help with act commands
\g or terminate with semicolon to execute query
\q to quit
retail_sales=>
To list the tables in the database, enter \d at the ACT prompt (in this case, retail_sales=>).
For example:
retail_sales=> \d
List of relations
Schema |
Name
| Type | Owner
--------+--------------+-------+--------public | customer_dim | table | beehive
public | date_dim
| table | beehive
public | geo_dim
| table | beehive
public | product_dim | table | beehive
public | region_dim
| table | beehive
public | sales_fact
| table | beehive
public | store_dim
| table | beehive
(7 rows)
Run queries from the command line
To control how ACT handles errors when running single or multi-line SQL queries from the
command line, see “Using the “on-error-stop” Option in ACT” on page 64.
To run a single-line SQL query from the command line:
1
Type the query at the ACT prompt.
Note! The query must end with a semicolon.
2
Press Enter.
For example, to list all the rows of the customer_dim table of the retail_sales database and
order the results based on gender, enter this command at the ACT command-line prompt:
retail_sales=> select customer_id, gender, city_id from customer_dim
order by gender;
customer_id | gender | city_id
-------------+--------+--------743 | F
|
46
2711 | F
|
124
744 | F
|
66
To run a multi-line SQL query from the command line:
1
Type the first line of the query, then press Enter.
The ACT prompt changes from => to -> (for example, retail_sales->).
2
Enter the remaining lines.
You can press Ctrl-C at any time to abandon this command mode without executing the
query.
67
Aster Client Guide
Aster Database Cluster Terminal (ACT)
How to Use ACT
3
On the last line, type a semicolon at the end of the line, then press Enter to run the query.
For example:
retail_sales=> select customer_id, gender, city_id
retail_sales->from customer_dim
retail_sales->order by gender;
or
retail_sales=> select customer_id, gender, city_id
retail_sales->from customer_dim
retail_sales->order by gender
retail_sales->;
Run queries from files
To control how ACT handles errors when running SQL queries from a file, see “Use a
Configuration File to Pass ACT Startup Parameters” on page 65.
To run a query stored in a file:
1
Make sure the filename ends with .sql.
The query does not have to end with a semicolon.
2
At the prompt, enter this command:
\i filename
For example, consider this query stored in the file myQuery.sql:
SELECT *
FROM customer_dim
ORDER BY gender
To run this query, enter this command at the ACT command prompt:
retail_sales=> \i myQuery.sql
You can also run this query using the –f flag of the act command. For example:
[root@localhost ~]# act -d retail_sales -w beehive -f myQuery.sql
Workaround for Multibyte Characters
ACT does not accept multibyte characters from the SQL prompt. For example, the following
statement will produce a syntax error:
beehive=> create database db
The workaround to input multibyte characters is as follows:
1
2
Input the multibyte characters from an SQL file instead of at the SQL prompt:
a
Create a file containing the SQL to insert the multibyte data (for example,
insert_file.sql).
b
Execute it by issuing \i insert_file.sql.
Or, use the SQL command buffer in ACT:
a
Aster Client Guide
Input the data using the query buffer, by issuing the ACT \e command, which brings
up the default text editor.
68
Aster Database Cluster Terminal (ACT)
How to Use ACT
b
Type in the SQL, including the multibyte characters.
c
Execute the SQL by exiting the text editor (for example by typing :wq if your default
text editor is vi.)
Exit ACT
To quit ACT, type \q and hit <Enter>.
Page Through Query Results
When ACT returns results to the screen, it (by default) prints one page of results at a time. Hit
the space bar to display the next page of results, hit <Enter> to display just one more line of
results, and type “q” to quit looking at results and return to the SQL prompt. When you get to
the last page of results, ACT displays the text, “(END)”. Type “q” to return to the SQL prompt.
Throttle Query Results in ACT and Aster Database
To reduce the memory footprint of ACT and other Aster Database clients, you can set a fetchcount that constrains the number of rows returned at one time when you select from a large
table. You can limit the total number of rows returned by using fetch-limit. Additionally, you
can make your query results stream more efficiently by using the server-side cursors feature. In
ACT, you can set these parameters using \set, and in other clients you can typically set them
in your data source definition or parameters file.
Let’s look at fetch-count first.
Reduce Memory Use With Fetch-Count
The purpose of using fetch-count is to reduce the memory footprint of ACT on the server.
Therefore, this type of configuration would normally be done by an administrator, or at least
the setting would be made under an administrator’s guidance. To set a fetch-count, you
specify a value for the fetch-count parameter in ACT. This causes ACT to fetch rows in sets,
with each set containing only the user-specified number of rows (or fewer rows).
ACT uses a fetch count by default, i.e. even when fetch-count is not set explicitly. The
fetch-count (number of rows per fetch) should always be set to greater than 0, The default
value is 1024 rows.
To set the fetch-count in a running session of ACT, use the \set command to set the
fetch-count parameter. To do this, type the following at the ACT prompt:
\set fetch-count n
where n is the maximum number of rows ACT should return at a time.
To enforce the fetch-count, ACT uses server side cursors to fetch results, which can help
prevent the memory footprint of ACT from growing too large.
Note that to the user, the results returned will not be different when using fetch-count. The
purpose is simply to reduce the memory footprint of ACT on the server.
69
Aster Client Guide
Aster Database Cluster Terminal (ACT)
How to Use ACT
Limit Caching With Server-Side Cursors
Setting use-server-cursors has the same effect as declaring a server-side cursor with the
DECLARE CURSOR syntax in SQL. When server-side cursors are activated, one batch of data
is retrieved at a time. This lets the queen avoid caching the entire result set and lets the workers
continue performing computations while the queen is retrieving rows. As the result set
becomes available from a worker, the queen sends it to the client.
Depending on the type of query, results can start streaming back to the client immediately
(e.g. SELECT * WHERE...) or the results may have to be computed in their entirety (e.g.
GROUP BY) before streaming to the client. Regardless of whether the results begin to stream
to the client before the query has finished executing, server-side cursors can improve
performance significantly for large result sets. If you want to start streaming results of a query
as soon as possible, you can benefit from setting use-server-cursors and specifying a
fetch-count. By default, use-server-cursors is set to 0 (off).
Example of Fetch-Count With Server-Side Cursors
The following steps are an example of how to use server-side cursors and fetch-count together:
1
Enable server cursors by issuing the following command inside ACT (a setting of 1 turns
cursors “on” and a setting of 0 means “off ”):
beehive=> \set use-server-cursors 1
2
Set a fetch-count of 100:
beehive=> \set fetch-count 100
3
Issue the following SQL query:
SELECT A,B,COUNT(*) FROM lineitem GROUP BY a, b;
This will actually cause the following statements to be run:
BEGIN;
DECLARE x
FETCH 100
FETCH 100
...
...
FETCH 100
CLOSE x;
END;
CURSOR FOR SELECT A,B,COUNT(*) FROM LINEITEM GROUP BY A,B;
FROM x; // 100
FROM x; // 200
FROM x; // until all the rows have been fetched
Aster Database reports all the fetched rows as the row count for this query.
Set the Maximum Number of Rows Returned With Fetch-Limit
To limit the total number of rows returned by each query, set the fetch-limit in ACT. fetchlimit (the total number of rows fetched for a query) can be set to any value. A value less than
Aster Client Guide
70
Aster Database Cluster Terminal (ACT)
How to Use ACT
0 implies fetch all rows. A value greater than 0 implies fetch-limit rows in total. The default
value is -1 (all rows).
Tip! When fetch-limit is used, the total row count returned for the query will be the total row count returned
by the query or the row count specified by fetch-limit, whichever is smaller. For example, if a query normally
returns 35,453 rows, but you have specified a fetch limit of 1000, the query will return 1000 rows (and it will display
“1000 rows returned”). There will be no indication that there were in fact 35,453 rows that would have been returned
had you not had a fetch limit set.
In ACT, you set the fetch-limit by typing:
\set fetch-limit n
where n is the maximum number of rows a single query will be allowed to return.
The SQL LIMIT Clause vs. Fetch-Limit and Fetch-Count
Setting a fetch-limit is an alternative to specifying a LIMIT clause explicitly in your
SELECT query, with an important difference in how the query will be executed, which can
affect performance. fetch-limit is an ACT feature which controls how many rows are
fetched by the queen from the workers and by the client from the queen. ACT stops fetching
results from the cursor after the specified limit is reached. LIMIT is a SQL clause which
controls how many rows are computed on the worker(s) or the queen.
The following examples illustrate the difference between using fetch-limit and using a
LIMIT clause in the SQL query:
Consider the statement:
SELECT * FROM FOO, BAR where FOO.A = BAR.A;
Example 1: Using fetch-limit and fetch-count
To issue the query in ACT with a fetch-count of 100 and a fetch-limit of 1000:
\set fetch-count 100
\set fetch-limit 1000
SELECT * FROM FOO, BAR where FOO.A = BAR.A;
Here is how the query will execute:
1
ACT opens a cursor on the query.
2
The queen activates Parallel Cursors and passes the query
SELECT * FROM FOO, BAR where FOO.A = BAR.A;
to the workers.
71
3
Each worker then computes the entire equi-join as dictated by the FOO.A = BAR.A
constraint.
4
The queen fetches results from workers in batches of 100 rows, until a limit of 1000 is
reached.
5
ACT fetches results from the queen in batches of 100 rows, until a limit of 1000 is reached.
Aster Client Guide
Aster Database Cluster Terminal (ACT)
How to Use ACT
Example 2: Using a LIMIT clause in the SQL query
To issue the same query in ACT using the SQL LIMIT clause to limit results to 1000 rows:
SELECT * FROM FOO, BAR where FOO.A = BAR.A LIMIT 1000;
Here is how the query will execute:
1
ACT issues the SQL statement to the queen.
2
The queen planner identifies LIMIT 1000 and pushes down the following query to each of
the workers:
SELECT * FROM FOO, BAR where FOO.A = BAR.A LIMIT 1000;
3
Each worker computes the equi-join as dictated by the FOO.A = BAR.A LIMIT 1000
constraint. This allows for optimizations where the entire equi-join computation need not
be done at each worker.
4
The queen fetches results from the workers.
5
ACT fetches results from the queen.
The difference between the two approaches occurs in step 3 of both examples. Using the SQL
LIMIT clause (Example 2), the workers are allowed to compute with the constraint of LIMIT
1000, whereas in Example 1 they have to compute the entire equi-join.
So, as you can see, fetch-limit via server-side cursors does not translate into the workers
doing a LIMIT 1000 on their individual slice of data. Therefore, if the use case calls for it, an
Aster Database power-user should be aware that using the SQL LIMIT clause can speed up
query execution dramatically in Aster Database.
ACT Utility Commands
ACT also offers many utility commands that provide assistance and carry out useful utility
operations such as uploading files and changing output options. These commands start with a
backslash character, and we describe them in the section, “ACT Commands (at the SQL
Prompt)” on page 73.
Repeat Previously Typed Commands
Hit the up arrow on your keyboard to toggle through your history of previously typed
commands. To edit a previously typed command, just use the usual combination left arrow,
right arrow, delete, and backspace keys to make your edits, and type <Enter> to finish editing
the line. If needed, type “;” to issue the command. To list your command history on Linux,
type \s.
Tip! If you receive an “Error writing history to file.” error on Linux when attempting to view command history with \s,
check that the current Linux user has permissions to write to the current working directory.
Tab Completion
The UNIX/Linux version of ACT can tab-complete SQL commands and table names that you
type. Tab-completion is not available in the Windows version of ACT.
Aster Client Guide
72
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
To use this feature, type the first couple of letters of a command and hit the <Tab> key. If the
completion is unambiguous, ACT completes the command. If ACT doesn’t complete the
command, hit <Tab> again and ACT prints all the possible completions. Using the list as a
reference, type enough additional characters to unambiguously identify the desired command
or table, and hit <Tab> again to complete it. Here are a few common uses of tab completion:
•
To complete common SQL commands. For example, type “se” and hit <Tab> to type
SELECT.
•
To list various ACT utility commands. For example, type “\”and hit <Tab> to show all the
commands, or type “\d” and hit <Tab> to show all the commands that start with “d”.
•
To complete a table name. For example, type “SELECT * FROM sa” and hit <Tab> to
complete the table name or hit <Tab> twice show all the table names that start with “sa”.
You can also list the names of all tables in the database by typing “SELECT * FROM ” (note
the trailing space) and hitting <Tab> twice.
ACT Commands (at the SQL Prompt)
The ACT SQL prompt accepts a number of ACT-specific commands that you issue by typing a
backslash followed by a character or combination of characters and arguments. Do not type a
semicolon to conclude these commands! Hitting <Enter> executes the command.
Note! Don’t confuse these with the ACT command-line startup parameters (also known as
“command line flags”), which you type at the shell command line when you launch ACT. For
a list of those, see “Startup Parameters for ACT” on page 61.
We describe the SQL-prompt commands in the tables that follow:
•
General-purpose utility commands in ACT
•
Environment settings in ACT
•
Query Buffer Commands in ACT
•
Input/output-related commands in ACT
•
Installed-function and installed-file management commands in ACT
•
Informational commands in ACT
•
Formatting-related commands in ACT
Table 3 - 7: General-purpose utility commands in ACT
73
Command
Description
\?
prints help for ACT commands.
Aster Client Guide
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 7: General-purpose utility commands in ACT (continued)
Command
\c[onnect]
HOST PORT
\c[onnect]
HOST
\c[onnect]
\c[onnect]
Description
DBNAME USER
DBNAME USER
DBNAME USER
DBNAME
change login credentials and/or connect to a new database.
The parameters must be specified in the order shown, with a
space before each, and parameters may not be skipped. In
other words, if only one parameter is specified, it is
understood to be DBNAME; if a second parameter is also
specified, it is understood to be USER; and so on.
\cd [DIR]
change the current working directory.
\copyright
show ACT usage and distribution terms.
\h
help with SQL commands.
\h [SQL command name]
help with syntax of the specified SQL command, * for all
commands.
\g
or terminate with a semicolon (;) to execute query.
\q
quit ACT.
\! [command]
execute command in shell or start interactive shell.
\password
change the password for the current user.
Table 3 - 8: Environment settings in ACT
Command
Description
\info
display current environment settings.
\set
display current ACT parameter settings.
\set param-name
[param-value]
set ACT parameter setting param-name to value paramvalue. (For example, “\set fetch-count 500” tells ACT
to fetch no more than 500 rows at a time when selecting.) If no
parameter value is supplied, displays the current setting for the
specified parameter.
\timing [on|off]
toggle or set timing of commands.
\pager [on|off]
toggle or set to use pager to enable paging through large result
sets.
Table 3 - 9: Query Buffer Commands in ACT
Aster Client Guide
Command
Description
\e [FILE]
edit the query buffer (or file) with external editor. On most systems, this
launches your default text editor. When you save and exit the editor, the
edited statement is passed back to ACT for running.
\g [FILE]
send query buffer to server (and results to file or | (pipe character)).
\p
show the contents of the query buffer.
74
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 9: Query Buffer Commands in ACT
Command
Description
\r
reset (clear) the query buffer.
\w FILE
write query buffer to file.
Table 3 - 10: Input/output-related commands in ACT
Command
Description
\echo
[STRING]
write string to query output stream (see \o below).
\i [FILE]
execute SQL commands from SQL script file. (Run an SQL script.)
\o [FILE]
redirect all query results to file or | (pipe character).
\o
Type \o with no argument to stop sending results to a file and resume sending
them to the ACT shell.
\s
[FILENAME]
display command history in Linux (optionally, print history to a file specified by
FILENAME) Note that query history includes only the first 2048 characters of
each query.
Table 3 - 11: Installed-function and installed-file management commands in ACT
Command
Description
\dF
list installed files, SQL-MapReduce functions, and other functions in the
current schema. Use a regular expression as an argument to display a subset of
the available functions. For example, to view all installed functions in the
database, issue:
\dF *.*
where the first asterisk means "all schemas" and the second means "all
functions and files."
\dF+
show details for all installed files, SQL-MapReduce functions, and other
functions in the current schema. For each function, the output shows the
name, schema, owner, upload time, and MD5 Hash fingerprint of the
function.
Use a regular expression as an argument to display a subset of the available
functions. For Example, type \dF+ *.* to show details for functions and
files in all schemas in the database.
\dE
75
show all the installed SQL-MR functions for which the current user has
privileges. Use a regular expression as an argument to display a subset of the
available functions. Shows function name, schema, owner, function version
and creation time.
Aster Client Guide
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 11: Installed-function and installed-file management commands in ACT (continued)
Command
Description
\install
<FILE>
[[<SCHEMA>/
]<FILE_ALIAS>
]
install the file or SQL-MapReduce function in Aster Database. The file must
be available on the file system where ACT is running. Note that the database
user running this command must have permission to install files and
functions in the specified schema.
You cannot install two files or functions with the same name. If attempting to
do this, you must follow these steps:
• remove the existing file or function
• install the new file or function
• grant the appropriate privileges on the file or function.
There is a limit of 238MB on the size of the file to be installed. If you try to
install a larger file, you will see an error like:
ERROR: row text exceeds limit of 238MB ...
Note that when installing larger files, the queen may run out of memory. The
queen needs available memory of approximately eight times the size of the file
to be installed, in order to encode, buffer and copy the file.
\download
[[<SCHEMA>/
]<FILE_ALIAS>
] <FILE>
download the specified installed file or function (identified by its FILE or
FILE_ALIAS) to the machine where ACT is running. Note that the database
user running this command must have permission to download files and
functions from the specified schema.
\remove
[[<SCHEMA>/
]<FILE_ALIAS>
]
remove from the cluster the file or SQL-MapReduce function specified by its
FILE_ALIAS. Note that the database user running this command must have
permission to remove files and functions from the specified schema.
Table 3 - 12: Informational commands in ACT
Aster Client Guide
Command
Description
\d
list all tables, indexes and views in the current schema.
\d [PATTERN}
describe table or index.
\dt
list all tables in the current schema.
\dt [PATTERN}
print schema, name, type, and owner of a table or
tables. To see tables in a custom schema, type \dt
schemaname.*
\dv
list views.
\dv [PATTERN]
describe view.
\di
list indexes.
\di [PATTERN}
describe index.
\dg
list groups and roles.
\dg [PATTERN}
describe group.
76
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 12: Informational commands in ACT (continued)
Command
Description
\du
list users.
\du [PATTERN}
describe user.
\dn
list schemas.
\dn [PATTERN}
describe schema.
\l
list all databases.
\extl host=hostname_or_IP
[option_name=option_value,
…]
lists all databases on an external Hadoop systems.
\extd host=hostname_or_IP
database=dbname
[option_name=option_value,
…]
lists all tables in a database or describe a table on an
external Hadoop system.
Tip! In Aster Database 5.0, for the \extd command in ACT, if the optional user argument is not specified, the
command will fail on any but the default database. The error message is not specific about what caused the command
to fail. The workaround is to always specify the argument user when issuing \extd.
Tip! ACT uses the schema search path (search_path) for the database user when displaying lists of tables,
veiws and indexes. The schema search path defaults to the schema search path for the current user in the database.
To set the search_path from ACT, issue the following command:
beehive=> SET session search_path TO <schema>;
Note that multiple schemas are not supported. If multiple schemas are listed in the search_path, the first
schema listed will be used.
To display the current search_path type:
beehive=> SHOW search_path;
Note that you may also set the search_path on the server.
Alternatively, you can specify the schema to use when issuing commands by following the command with a schema
qualified reference. This example shows how to display information on all tables in the schema “myschema”:
\dt myschema.*
Table 3 - 13: Formatting-related commands in ACT
77
Command
Description
\a
toggle between unaligned and aligned output mode.
\f [STRING]
show or set field separator for unaligned query output.
\t [on|off]
show only rows (off by default).
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Database Parameters Set with \set
Table 3 - 13: Formatting-related commands in ACT (continued)
Command
Description
\x [on|off]
set or toggle expanded output mode ON and OFF. With expanded output
mode turned on, each record is split into rows, with one row for each
value, and each new record is introduced with a text label in the form, --[ RECORD 37 ]---. This can help make wide tables readable on a
small screen, and is very useful if you’re trying to read EXPLAIN output.
Note that in expanded mode, the number of rows is not returned at the
end of the table. Because of this, when querying a table with no rows,
you will simply see the ACT prompt again.
Database Parameters Set with \set
The syntax to use the \set command is:
\set [ name [ value [ ... ] ] ]
Table 3 - 14: Database parameters
Parameter
Description
auto-commit [1|0]
When set to 1 (the default, on), each SQL command is automatically
committed upon successful completion. When set to 0 (off), you may
manually commit your changes after each transaction or series of
transactions by issuing the COMMIT command, or undo changes by
issuing ROLLBACK. If you do not issue the COMMIT command, all
transactions that occurred since the last COMMIT will rollback
automatically.
fetch-count [int]
To limit the number of rows returned at a time. ACT uses a fetch count
by default (i.e. even when fetch-count is not set explicitly.) The
fetch-count (number of rows per fetch) should always be set to
greater than 0, The default value is 1024 (1024 rows).
fetch-limit [int]
To set the maximum number of rows returned per query. A value less
than 0 implies fetch all rows. A value greater than 0 implies fetchlimit rows in total. The default value is -1 (all rows).
use-server-cursors
[1|0]
When set to 1, sets the server to use cursors (useful when the result set
is very large). When set to 0 (the default, off), sets the server to not use
cursors.
on-error-stop [1|0]
By default, this feature is disabled or set to off = 0.
When set to 1 (or “on”) ACT will stop and exit if it meets an error
during SQL query processing.
The following are ACT exit messages:
• EXIT_SUCCESS = 0 means ACT finished processing normally.
• EXIT_FAILURE = 1 means an error occurred, such as "file not
found" in the “-f ” option.
• EXIT_USER = 3 means an error occurred in a sql script and the
option “on-error-stop” was on or enabled.
Aster Client Guide
78
Aster Database Cluster Terminal (ACT)
Troubleshooting ACT
Troubleshooting ACT
ACT Connection Hangs When Using SSL
When ACT is configured to use SSL, the queen must also be set up to use SSL. If the queen is
not setup for SSL and ACT tries to connect using SSL, the connection will fail. An error is not
returned; the ACT client just hangs. If you experience this, check to ensure that SSL settings
on the queen and in ACT match.
Invalid User Name Error in ACT After Password Change
When changing the user’s password using ALTER USER <username> PASSWORD
'<new_password>'; in ACT or in a SQL script running through ACT, you will see an ALTER
USER successful message returned. However, ACT does not change the cached password for
the user. Because of this, if you then connect as the user and issue another query, you may see
an error related to invalid username and password. This is because ACT is using the cached
username and password when attempting to establish a new session.
If you encounter this, change the password using the \password command in ACT instead. If
you change the password using \password, your ACT connection will continue to work.
Misleading Error Message Reports Problem With a Role Instead of With a
User
When attempting to connect to a database as a user who does not have connect privileges on
that database, the error incorrectly reports that the problem is with a role instead of with the
user. This example illustrates the error message:
beehive=> create user kris with password 'beehive';
CREATE USER
beehive=> create database foo;
CREATE DATABASE
beehive=> \c foo kris;
Password for "kris":
act: ERROR: role "kris" does not have connect privileges on this
database.
Previous connection kept
79
Aster Client Guide
Connect Using Database Drivers
CHAPTER 4
Aster Database provides Open Database Connectivity (ODBC) and Java Database
Connectivity (JDBC) drivers for connecting business intelligence (BI) tools. This section
explains how to install and use the Aster Database drivers.
General tips:
•
General Tips for Connecting Clients to Aster Database
Setting up database drivers:
•
ODBC Driver
•
JDBC Driver
•
Tools for .NET Environments: .NET Data Provider for Aster for use with SSIS, SSAS, SSRS,
and .NET
Using database drivers:
•
Process SQL Statements in JDBC
•
JDBC Troubleshooting and Limitations
•
Configure Aster Database SQL Settings
Connecting BI and query tools:
•
Connect Reporting Tools to Aster Database
General Tips for Connecting Clients to Aster
Database
Recommended Character Set Is UTF-8
For all tools that you use to connect to databases in Aster, you will typically want to set the
default character set to UTF-8. This is particularly important if you plan to use special
characters (for example, German letters ä and ß) in a char, varchar, or text column, and it is
particularly important if you are connecting from a Windows-based machine.
For example, if you will use an SSH client (for example, putty) to run ACT or ncluster_loader,
make sure you set the SSH client’s default character set to UTF-8.
Aster Client Guide
80
Connect Using Database Drivers
ODBC Driver
Supported Encoding
For a list of encodings supported by Aster ODBC, see “Encoding Support” on page 156.
When Querying System Tables with ODBC, Set AUTOCOMMIT to 'OFF'
Cursors cannot be declared and used with Aster Database system tables. The default mode of
autocommit-on with the ODBC driver declares cursors. Therefore, the ODBC driver should
always query system tables by turning AUTOCOMMIT to 'off '.
ODBC Driver
Set Up ODBC for Perl Connectivity on Linux
To install the Aster Database ODBC driver and configure your environment to allow Perl
scripts to connect to the database through the driver, follow these steps. These instructions
assume you have installed Perl on your workstation, and that the Aster Database queen is
reachable on the network. Follow these steps to provide an Aster Database connection your
Perl scripts can use:
1
Set up your /etc/resolv.conf and /etc/nsswitch.conf for accessing the Aster
Database queen..
2
The server connection information is set in the odbc.ini file, as explained earlier in this
chapter. On Windows, the server connection information is set using the ODBC Data
Source Administrator tool as explained in “Set up ODBC for PHP” on page 82.
3
In /root type
$ perl -eshell -MCPAN
4
Get the latest packages and install them by entering:
$ install Bundle::CPAN
$ install DBI
5
Rebuild and install DBD::ODBC.
$ export LD_LIBRARY_PATH=/home/beehive/toolchain/x86_64-unknownlinux-gnu/unixODBC-2.2.12/lib
$ export PATH=$PATH:/home/beehive/toolchain/x86_64-unknown-linuxgnu/unixODBC-2.2.12/bin
$ which odbc_config
/home/beehive/toolchain/x86_64-unknown-linux-gnu/unixODBC-2.2.12/
bin/odbc_config
$ odbc_config --cflags -DHAVE_UNISTD_H -DHAVE_PWD_H
-DHAVE_SYS_TYPES_H -DHAVE_LONG_LONG -DSIZEOF_LONG=8
$ perl -eshell -MCPAN
cpan[1]> force install DBD::ODBC
6
Run odbcinst -j to see where the .ini files are being picked up:
$ odbcinst -j
7
81
Run the following Perl script to check your installation.
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
If everything is set correctly, it will run without an error. If you encounter problems, check
the data source name (“DSN”) in the connect statement. In the example below, the
username and password of the database are “beehive” and “beehive”:
#!/usr/bin/perl
use DBI;
use DBD::ODBC;
# Connect to ODBC DSN and turn off AutoCommit
my $dbh = DBI->connect('dbi:ODBC:nc',"beehive","beehive",
{AutoCommit => 0});
$dbh->do("BEGIN");
$dbh->do("set random_page_cost to '4'");
$dbh->do("set enable_seqscan to 'off'");
$dbh->disconnect;
Set up ODBC for PHP
To set up PHP to work with ODBC, first install Aster ODBC driver on Linux.
Use unixODBC 2.2.12 and build it with the CFLAG, SIZEOF_LONG=8. The same will apply
to building PHP when compiled to be used with unixODBC. Here are the supported versions:
•
PHP 5.4.10
•
Apache 2.2.23
PHP
To set up PHP:
1
Make sure Apache is installed and make note of the directory. For this example, we will
assume Apache is installed at /usr/local/apache
2
Ensure that unixODBC has been installed as described above.
3
Download the source for PHP 5.4.10 and extract it to the desired directory. The following
setup instructions should be used for PHP:
$ CFLAGS="-DSIZEOF_LONG=8" ./configure --with-apxs2=/usr/local/
apache/bin/apxs --with-zlib --with-unixODBC --with-pdo-odbc=unixODBC
$ make && make install
$ cp -p .libs/libphp5.so /usr/local/apache/modules
Apache and PHP
Apart from the typical PHP-Apache setup, there are two things to keep in mind:
1
Ensure that the Linux system is set up to find the libraries that the ODBC driver requires.
a
Add the <INSTALL-DIR>/Libs and <INSTALL-DIR>/ODBCDriver paths to the /etc/
ld.so.conf file, where <INSTALL-DIR> is the location for the ODBC installation.
b
The following commands should be executed to refresh the ld cache and restart the
Apache web server:
$/sbin/ldconfig
$/etc/init.d/apachectl restart
Aster Client Guide
82
Connect Using Database Drivers
ODBC Driver
2
To make sure that the ODBCSYSINI environment variable is available when PHP calls are
made, use the following line in your PHP code:
#given ODBCSYSINI is to be set to /etc then:
putenv("ODBCSYSINI=/etc")
Tip! You should not set up your own PHP or use /etc/init.d/apachectl on the queen for your own web
pages.
ODBC Usage Notes
Optional ODBC Setting for bytea-Stored Data
If you retrieve bytea-stored data through the ODBC driver, you can specify whether values in a
column of datatype bytea will be retrieved in a character representation, or in the default
binary representation.
To set this up you must set the ByteaAsVarchar setting. Setting ByteaAsVarchar=1 instructs
the ODBC driver to retrieve values in character representation; leaving it unset preserves the
binary output representation.
•
To set this up on Solaris, see the discussion of the odbc.ini file in “Install the unixODBC
driver manager” on page 25.
•
To set this up on Linux or AIX, see “Configure DataDirect Driver Manager on Linux and
AIX” on page 23.
•
To set this up on Windows, see below.
Bytea-Stored Data Settings on Windows
See the section that applies to your operating system and driver:
•
For a 64-bit ODBC driver running on 64-bit Windows, or for a 32-bit driver on 32-bit
Windows:
Set the flag by adding it to the Windows registry entry for the DSN. Using a registry editor,
add this line to the registry, taking care to first replace <DSN-NAME> with your Data
Source name:
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\<DSN-NAME>]
"ByteaAsVarchar"="1"
•
For a 32-bit ODBC driver running on 64-bit Windows
Set the flag by adding it to the Windows registry entry for the DSN. Using a registry editor,
add this line to the registry, taking care to first replace <DSN-NAME> with your Data
Source name:
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\<DSN-name>]
"ByteaAsVarchar"="1"
Avoid using bind_param with SQL_DECIMAL
When using the bind_param method call with decimal fields, you should not explicitly specify
SQL_DECIMAL as the parameter bind type. If you do this, and the parameter value has a
fractional digital part, the factional part changes to 0 after applying bind_param. The
83
Aster Client Guide
Connect Using Database Drivers
ODBC Driver
DBD::ODBC bind_param implementation makes this conversion when specifying
SQL_DECIMAL/SQL_NUMERIC as the parameter bind type. Note that bind_param works
fine when SQL_DECIMAL is not explicitly specified, so you should avoid this when passing
decimal fields.
Bind columns without specifying a SQL datatype
Use the Perl API without specifying a SQL datatype the Aster datatypes, as in the example:
$rc = $sth->bind_col($column_number, \$var_to_bind);
Avoid specifying the SQL datatype to bind to the column, as in:
$rc = $sth->bind_col($column_number, \$var_to_bind, $bind_type);
This will work fine when binding boolean and character columns with DBI::SQL_CHAR, but
for the other datatypes, this will cause two issues:
1
If you specify a numeric SQL type in bind_col, corrupt data may be returned. Numeric
types include SQL_INTEGER, SQL_SMALLINT, SQL_DOUBLE, SQL_REAL,
SQL_NUMERIC/SQL_DECIMAL, SQL_BIT, SQL_TIME, SQL_TIMESTAMP,
SQL_DATE, and SQL_DATETIME.
2
If you specify a SQL_VARCHAR type in bind_col, an error may be returned. The error
looks like:
•
On Unix ODBC: “DBD::ODBC::st fetch failed: [unixODBC][Driver Manager]Invalid
application buffer type (SQL-HY003)”
•
On Windows ODBC: “DBD::ODBC::st fetch failed: [Microsoft][ODBC Driver
Manager] Program type out of range (SQL-HY003)”
This issue has been observed with the following software versions:
Aster Client Guide
•
Perl 5.8.9
•
DBD-ODBC 1.3.1
•
DBI-1.616
84
Connect Using Database Drivers
JDBC Driver
JDBC Driver
JDBC is an API for the Java programming language that provides methods for querying and
updating data in a relational database. The Aster Database JDBC driver enables your Java
applications and reporting tools to retrieve data directly from Aster Databases.
The Aster Database JDBC driver is a Type 4 JDBC driver that implements the JDBC 3
specification.
•
Aster JDBC Driver
•
Differences from the Legacy JDBC Driver
•
Before You Start
•
Install the JDBC Driver
•
Use the JDBC Driver in a Java Application
•
Parameters for Connecting through JDBC
•
Configuring the JDBC Log Settings
•
Behavior and Performance Settings for JDBC
•
Using Client-Side Cursors in JDBC
•
Test JDBC Connect Program
Aster JDBC Driver
This version of the JDBC driver, introduced in Aster Client release 5.0.3, adds these
capabilities to those that were already supported in the legacy JDBC driver:
85
•
Ability to handle larger data sets than the legacy JDBC driver; doesn’t try to load whole
result set onto the client in all cases
•
Support for multibyte characters in data and user metadata UTF-8 encoding
•
Support for Single Sign-On (SSO) authentication
•
Support for Secure Socket Layers (SSL) encrypted communications
•
Support customized connection parameters
•
Support configuring statement FetchSize to limit the data being returned
•
Support server side cursors
•
Support specifying maximum number of rows
•
Support for these commands:
•
Copy—Moves data between Aster Database tables and a remote client (from and to a
file) via the connection between the client and the server.
•
Install File—Installs the data file or SQL-MapReduce function in the specified Aster
Database schema.
•
Uninstall File—Removes the file or SQL-MapReduce function from Aster Database.
•
Download File—Downloads the specified installed file or function.
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
Differences from the Legacy JDBC Driver
These are important differences between this JDBC driver and the legacy (pre-5.0.3) JDBC
driver:
•
JDBC only supports scrollable forward cursors. Only the con.createStatement
(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY) and
connection.createStatement() functions are supported. The Other parameter values like
ResultSet.TYPE_SCROLL_INSENSITIVE and ResultSet.CONCUR_UPDATABLE are not
supported.
•
The executeBatch() function throws a SQL exception (java.sql.SQLException).
•
The result set cannot be accessed after an explicit commit.
•
After running PreparedStatement.executenatch(), you must set all bind values or else the
driver throws an error.
•
The java.sql.PreparedStatement.setObject() function does not throw exceptions for invalid
java.sql.Types.TINYINT values. Instead, the function wraps the values around. This
behavior complies with the Java standard.
•
ResultSetMetaData cannot be accessed after a ResultSet reset.
•
The driver cannot establish a connection to the database when there are white space
characters to the left of the connection string.
•
The register serverConnect to database using JDBC in Aqua Data Studio (ADS) as register
server. Expand the tree view when connection is lost. Register server will prompt an error.
•
The DatabaseMetaData.getBestRowIdentifier() function is not supported.
•
The ResultSet.getArray() function is not supported.
•
You can call com.asterdata.ncluster.Driver.ASTER_BUILD_VERSION() to get JDBC
version information. For detailed version information, check the MANIFEST.MF file.
Before You Start
Prerequisites
The JDBC driver supports these versions of the Java JDK:
•
Oracle JDK 1.5
•
IBM JDK 1.6
Install the JDBC Driver
Follow these steps to install the Aster Database JDBC driver:
1
Obtain the package that contains the Aster Database JDBC driver . You can do this in one
of these ways:
•
To get the newest package, download it from http://downloads.teradata.com/download/
tools
•
Aster Client Guide
On your queen node, you can find the installers in the directory /home/beehive/
clients_all/<your_client_OS>.
86
Connect Using Database Drivers
JDBC Driver
2
Unzip the ZIP package.
The resulting folder contains multiple JAR files.
3
Copy the JAR files to a location in the classpath of the application that uses the driver.
Use the JDBC Driver in a Java Application
Follow these steps to integrate the Aster Database JDBC driver into your Java application:
1
Set the CLASSPATH environment variable to include the paths of the JAR files extracted
from the Aster Database JDBC driver package.
•
Use the Java -classpath flag at the command line to add the paths to the CLASSPATH
environment variable. For example, if you place the JAR files in /usr/jars, then add the
path like this:
java -classpath /usr/my_program /usr/jars/
or
•
Add the paths to the driver to the CLASSPATH environment variable. For example:
export CLASSPATH=/usr/jars/:$CLASSPATH
2
In your application:
a
Import the java.sql package like this:
import java.sql.*;
b
Load the driver using the Class.forName() method:
Class.forName("com.asterdata.ncluster.Driver");
c
Define the username, password, and url (including the host, port, and database)
parameters, which are needed to connect to the database. For example:
String username = "user";
String password = "password";
String url
= "jdbc:ncluster://myhost:2406/database";
For more information about the URL format, see “Required Parameters” on page 88.
d
Get a Connection instance from JDBC using the DriverManager.getConnection()
method:
try
{
Connection conn =
DriverManager.getConnection(url, username, password);
}
catch (SQLException ex)
{
// could not connect
}
NOTE: The getConnection method throws a “No driver available SQLException”
exception if CLASSPATH does not contain the path for the JDBC driver, or if the
parameters are incorrect.
87
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
Parameters for Connecting through JDBC
Required Parameters
To establish a connection to an Aster Database using the Aster Database JDBC driver, you
must provide the driver with the URL to use to connect to the database. The URL has this
format:
jdbc:ncluster://<Host:Port>/<Database>?enable_backslash_escapes=<on_or
off>&enable_quoted_identifiers=<on_or off>
For example:
jdbc:ncluster://192.65.197.90:2406/
beehive?enable_backslash_escapes=on&enable_quoted_identifiers=on
The URL needs three parameters to connect to an Aster Database:
Table 4 - 15: Parameters in URL to connect to an Aster Database
Parameter
Required? Description
Host
Optional
The name of the server where the database resides.
Default is localhost.
Port
Optional
The port number that the database server is listening
on.
Default is 2406.
Database
Required
The database name.
enable_backslash_escapes
Optional
Sets the queen parameter enable_backslash_escapes
for the current session. The two possible values are:
• on—Enables this parameter on the queen (default).
• off—Disables this parameter on the queen.
enable_quoted_identifiers
Optional
Sets the queen parameter enable_quoted_identifiers
for the current session. The two possible values are:
• on—Enables this parameter on the queen (default).
• off—Disables this parameter on the queen.
In addition, to the URL, you must also provide the username and password needed to access
the Aster Database, which you can get from your Aster Database administrator.
Optional Parameters
You can set the Autocommit and fetch_count settings for the connection in the URL by adding
the autocommit and fetch_count parameters. See “Frequently Used JDBC Settings” on
page 89. If your application will query large tables, you should set autocommit to false, and
you should declare a fetch_count for the connection. By doing this, you enable the
connection to use distributed cursors for improved performance.
You can also use the NumericAndDecimalAsDouble parameter to map NUMERIC and DECIMAL
type columns to SQL_DOUBLE. When you set this parameter, its value will be stored in a connection
context. This example shows how to use this parameter to decode the row header:
Aster Client Guide
88
Connect Using Database Drivers
JDBC Driver
if ((sqlType == Types.NUMERIC ||
sqlType == Types.DECIMAL) &&
inSettings.connectionSettings_.numericAsDouble_) {
sqlType = Types.DOUBLE;
}
Configuring the JDBC Log Settings
To enable and configure JDBC logging:
1
Create a file named simba.properties and add it to the folder containing the JDBC driver.
2
In the file, define the LogLevel and LogPath properties.
•
LogLevel—(Optional) Specifies the log level: OFF, FATAL, ERROR, WARNING, INFO,
DEBUG, and TRACE.
•
LogPath—(Optional) Specifies the path to the log file (AsterJDBC.log). If this property
is not defined, the JDBC driver stores the log file in the same folder where the JDBC
driver is located.
For example:
LogLevel=DEBUG
LogPath=D:\\logs
Behavior and Performance Settings for JDBC
When using the Aster Database JDBC driver, you make most behavior and performance
settings in your SQL code using the SET command.
You can also use a number of standard JDBC driver methods for setting behavior and
performance options, such as setAutoCommit() and setPrepareThreshold().
The scope of these variables are limited to the scope of the transaction. After the commit()
call, the transaction variables revert to their original values. In order to limit the transaction
variables to a particular query, the variables need to be saved prior to being set and once the
query is executed the variables must be set back to their original values.
Frequently Used JDBC Settings
•
autocommit: Boolean value that sets the autocommit behavior for the connection, true to
turn on autocommit (each statement, by default concluded with a semicolon, is run
immediately in the database) or false for off (you must COMMIT your transaction to run
it). You can set this in the connection URL with the autocommit parameter (for example,
jdbc:ncluster://10.80.50.100:2406/beehive?autocommit=false) or in the Java
code for your connection with Connection.setAutoCommit(). The default is true.
Unsupported JDBC Settings
Most JDBC option-setting methods in the Aster Database driver exhibit the standard JDBC
behavior, but there are exceptions. Please note:
•
setReadOnly(): Not supported; Aster Database does not allow changing connection type
to read-only.
89
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
•
setTransactionIsolation(): Not supported; Aster Database does not allow changing
transaction isolation levels.
Example Code Snippet That Sets Performance Tuning Variables
This example (PerformanceTuning.java) shows you how you can set some performance
tuning variables for a particular transaction:
package userguide;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class PerformanceTuning {
public static void main(String[] args) {
String username = "db_superuser";
String password = "db_superuser";
String url = "jdbc:ncluster://153.65.197.125:2406/beehive";
Connection conn = null;
try
{
conn =
DriverManager.getConnection(url, username, password);
// Remember the current autocommit state
boolean autoCommit = conn.getAutoCommit();
// Turn off auto commits
conn.setAutoCommit(false);
// create a statement with transaction variables set
// which will be used for this query
Statement stmt = conn.createStatement();
stmt.executeUpdate("SET enable_hashjoin = 'false'");
stmt.executeUpdate("SET enable_mergejoin = 'false'");
stmt.executeUpdate("SET enable_nestloop='true'");
stmt.executeUpdate("SET random_page_cost = '4.0'");
Statement statement = conn.createStatement();
String query="select * from knn_test";
ResultSet resultSet = statement.executeQuery(query);
// Process the result set
while (resultSet.next()) {
// do something
}
// commit the transaction
conn.commit();
resultSet.close();
stmt.close();
// Set the autocommit state back
conn.setAutoCommit(autoCommit);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Aster Client Guide
90
Connect Using Database Drivers
JDBC Driver
}
In the example above, the scope of the SET variables is limited to the commands between the
autoCommit(false) and commit() lines.
The Copy, Install File, Uninstall File, and Download File Commands
Version 5.0.3 of the JDBC driver added support for these commands:
•
Copy
•
Install File
•
Uninstall File
•
Download File
For more information about these commands, see the section describing SQL commands in
the Aster Database User Guide.
Copy
This command moves data between Aster Database tables and a remote client (from and to a
file) via the connection between the client and the server.
This is an example:
public void copyCommandExamples() {
String stmt1 = "COPY simba TO 'd:\\simba.txt' DELIMITER as ','";
String stmt2 = "COPY simba TO 'd:\\simba.csv' with csv QUOTE AS '@';";
try {
Statement s = conn_.createStatement();
s.execute(stmt1);
s.execute(stmt2);
s.close();
} catch (Exception e) {
e.printStackTrace();
fail(e.getMessage());
}
}
Install File
This command installs the data file or SQL-MapReduce function in the specified Aster
Database schema.
INSTALL FILE filename [[schema/]alias]
91
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
This is an example:
public void testInstallWithSchema() {
String stmt1 = "install file 'd:\\odbcng.tar.gz' ";
String stmt2 = "install file 'd:\\odbcng.tar.gz' uploadfile/'odbcng.zip'";
String stmt3 = "install file 'd:\\abc.txt' uploadfile/'abc.txt'";
try {
conn_.setAutoCommit(false);
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
conn_.commit();
s.close();
} catch (Exception e) {
try {
conn_.rollback();
} catch (SQLException e1) {
e1.printStackTrace();
}
fail(e.getMessage());
}
}
Uninstall File
This command removes the file or SQL-MapReduce function from Aster Database.
UNINSTALL FILE 'filename' from schema schemaname
This is an example:
public void testUninstall() {
String stmt1 = "UNINSTALL file 'abc.txt' from schema uploadfile ;";
String stmt2 = "UNINSTALL file 'odbcng.zip' from schema uploadfile ;";
String stmt3 = "UNINSTALL file 'odbcng.tar.gz'
;";
try {
conn_.setAutoCommit(false);
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
conn_.commit();
s.close();
} catch (Exception e) {
try {
conn_.rollback();
} catch (SQLException e1) {
e1.printStackTrace();
}
fail(e.getMessage());
}
}
Aster Client Guide
92
Connect Using Database Drivers
JDBC Driver
Download File
This command downloads the specified installed file or function.
DOWNLOAD FILE [[schema/]alias] filename
This is an example:
public void download_file_examples() {
String stmt1 = "DOWNLOAD file uploadfile/'abc.txt' 'd:\\download-abc.txt'";
String stmt2 = "DOWNLOAD file uploadfile/'odbcng.zip' 'd:\\download-uploadfileodbcng.zip'";
String stmt3 = "DOWNLOAD file 'odbcng.tar.gz' 'd:\\download-public-odbcng.zip'";
try {
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
s.close();
} catch (Exception e) {
fail(e.getMessage());
}
}
Using Client-Side Cursors in JDBC
To give you control over the latency of initial query results, the Aster Database JDBC driver
supports client-side cursors. Client-side cursors let you avoid retrieving all the result rows for
a query at once. Instead, you specify the number of rows that should be retrieved to the client
at a time. When that set is exhausted, the next page of rows is retrieved by repositioning the
cursor.
Observe the following guidelines when working with cursors:
1
Turn off autocommit mode for the Connection object. See “Using Cursors in Your Code”,
below.
2
The ResultSet object that receives the output of your Statement cannot be a scrollable
ResultSet. That is, it must have the type, ResultSet.TYPE_FORWARD_ONLY. This is the
default, so you need not rewrite your code.
3
If your application will query large tables, you should make sure your ResultSet can use
distributed cursors. To do this,
a
make sure the ResultSet is not updatable (ResultSet.CONCUR_READ_ONLY)
b
make sure it’s not scrollable (ResultSet.TYPE_FORWARD_ONLY)
c
make sure it does not have HOLD enabled
(ResultSet.CLOSE_CURSORS_AT_COMMIT).
d
Also, the application should connect with autocommit set to false and the connection
must have a specified fetch_count.
Tip! When working with ResultSets of type ResultSet.TYPE_FORWARD_ONLY, you cannot scroll backwards, nor can you jump to any location in the ResultSet other than the next row.
93
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
4
In the Statement object, you must pass a single query, not multiple queries strung together
with semicolons.
5
You must set the statement’s fetch_count using the Statement.setFetchSize(int
rows) command. This instructs the driver to fetch the specified number of rows at a time
from the database. If the fetch size is not set, the driver fetches the full set of rows that
match the query.
6
To help ensure a quick response when a page of rows is exhausted, the JDBC driver, by
default, pre-fetches and caches ten pages of results from the database. A page is one
fetch_count worth of rows. You can set the number of pages to be pre-fetched, or you can
disable pre-fetching if desired.
Using Cursors in Your Code
The driver fetches fetch_count number of rows (as set with Statement.setFetchSize(int
rows)) from Aster Database and exposes them to the database application as a “rows” data
structure. The default fetch_count is zero, so you must set a fetch_count if you wish to use
cursors.
With ResultSet.next(), your application iterates over the rows data structure. When all
rows in the rows data structure have been processed, the next call to next() will force the
fetching a new set of fetch_count number of rows, and so on.
To use cursors, set the fetch size of your SQL statement using the setFetchSize(int rows)
method. Later, setting the fetch_count back to 0 will cause all rows to be retrieved when the
query runs (the default behavior).
Tip! In the example below, we set the Autocommit setting and the FetchSize setting using the setAutoCommit() and setFetchSize() methods, but you also have the option of setting these in the JDBC connection
parameters when you make the connection. See “Frequently Used JDBC Settings” on page 89.
Example That Uses Client-Side Cursors
Assuming a Connection conn, you can set the FetchSize like this:
// Remember the current autocommit state
boolean autoCommit = conn.getAutoCommit();
// Make sure autocommit is off
conn.setAutoCommit(false);
Statement st = conn.createStatement();
// Turn use of the cursor on by setting FetchSize, expressed in rows
st.setFetchSize(50);
ResultSet rs = st.executeQuery("SELECT * FROM mytable");
// Set the autocommit state back
conn.setAutoCommit(autoCommit);
while (rs.next()) {
System.out.print("a row was returned.");
}
rs.close();
// Close the statement.
Aster Client Guide
94
Connect Using Database Drivers
JDBC Driver
st.close();
Disabling Cursors
Cursors are enabled by default. To turn them off, set the FetchSize to zero. Assuming a
statement st and a ResultSet rs, you would do this as shown here:
st.setFetchSize(0);
rs = st.executeQuery("SELECT * FROM mytable");
while (rs.next()) {
System.out.print("many rows were returned.");
}
Test JDBC Connect Program
This program can be used as a template for your JDBC connectivity:
// JDBC Test Program
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.ResultSet;
public class jdbctest {
static String userid="beehive", password = "beehive";
// Change the IP address to the node you want to connect to.
static String url = "jdbc:ncluster://10.60.3.100:2406/beehive"; //2.0
static Connection con = null;
public static void main(String[] args) throws Exception {
Connection con = getJDBCConnection();
if(con!= null){
try {
int count = 0;
System.out.println("Got Connection.");
DatabaseMetaData meta = con.getMetaData();
System.out.println("getDriverName: "+meta.getDriverName());
System.out.println("getDriverVersion: "+meta.getDriverVersion());
System.out.println("getDriverMajorVersion:
"+meta.getDriverMajorVersion());
System.out.println("getDriverMinorVersion:
"+meta.getDriverMinorVersion());
System.out.println("getDatabaseProductName:
"+meta.getDatabaseProductName());
System.out.println("getDatabaseProductVersion:
"+meta.getDatabaseProductVersion());
System.out.println("getIdentifierQuoteString:
"+meta.getIdentifierQuoteString());
System.out.println("\ngetSQLKeywords: "+meta.getSQLKeywords());
System.out.println("\ngetNumericFunctions:
"+meta.getNumericFunctions());
System.out.println("\ngetStringFunctions
:
"+meta.getStringFunctions());
System.out.println("getSystemFunctions
:
"+meta.getSystemFunctions());
System.out.println("getTimeDateFunctions
:
"+meta.getTimeDateFunctions());
System.out.println("getSearchStringEscape
:
"+meta.getSearchStringEscape());
System.out.println("getExtraNameCharacters
:
95
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
"+meta.getExtraNameCharacters());
System.out.println("getCatalogTerm
:
"+meta.getCatalogTerm());
System.out.println("getCatalogSeparator
:
"+meta.getCatalogSeparator());
System.out.println("getURL
:
"+meta.getURL());
System.out.println("getUserName
:
"+meta.getUserName());
System.out.println("getMaxCursorNameLength
:
"+meta.getMaxCursorNameLength());
System.out.println("getMaxSchemaNameLength
:
"+meta.getMaxSchemaNameLength());
System.out.println("getMaxProcedureNameLength :
"+meta.getMaxProcedureNameLength());
System.out.println("getMaxCatalogNameLength
:
"+meta.getMaxCatalogNameLength());
System.out.println("getMaxColumnsInIndex
:
"+meta.getMaxColumnsInIndex());
System.out.println("supportsSubqueriesInComparisons :
"+meta.supportsSubqueriesInComparisons());
System.out.println("getMaxConnections
:
"+meta.getMaxConnections());
System.out.println("getMaxColumnsInTable
:
"+meta.getMaxColumnsInTable());
System.out.println("isReadOnly
:
"+meta.isReadOnly());
System.out.println("\ngetCatalogs:");
ResultSet res = meta.getCatalogs();
while (res.next()) {
System.out.println(res.getString(1));
}
System.out.println("\ngetTables:");
res.close();
res = meta.getTables(null,null,"%",null);
while (res.next()) {
System.out.println(res.getString(1) +res.getString(2) +res.getString(3)
+res.getString(4));
}
System.out.println("");
res.close();
res = meta.getTableTypes();
System.out.println("\ngetTableTypes:");
while (res.next()) {
System.out.println(res.getString(1));
}
res.close();
Statement stmt=con.createStatement();
ResultSet rs = stmt.executeQuery("select count(*) from page_views");
while (rs.next()) {
System.out.println(rs.getInt(1));
}
rs.close();
stmt.close();
con.close();
} catch(SQLException ex) {
System.err.println("SQLException: " + ex.getMessage());
}
}else{
System.out.println("Could not Get Connection");
}
}
public static Connection getJDBCConnection(){
try {
Aster Client Guide
96
Connect Using Database Drivers
Configure Aster Database SQL Settings
Class.forName("com.asterdata.ncluster.Driver");
} catch(java.lang.ClassNotFoundException e) {
System.err.print("ClassNotFoundException: ");
System.err.println(e.getMessage());
}
try {
con = DriverManager.getConnection(url,userid, password);
} catch(SQLException ex) {
System.err.println("SQLException: " + ex.getMessage());
}
return con;
}
}
Configure Aster Database SQL Settings
This section explains how to set parameters that customize the behavior of Aster Database
SQL.
SQL Behavior Parameters
Escape Character Handling
enable_backslash_escapes: Set to “on” (the default), Aster Database allows backslash
escape strings in your constants. The escape strings are C-like backslash escape sequences,
each introduced with a backslash character (\). Set to “off ”, Aster Database does not recognize
backslash escape strings in your constants unless the constant is prefixed with a letter E before
its opening single quote. In any constant not prefixed with the letter E, a backslash is
interpreted simply as a backslash character.
Quoted-Identifier Handling
enable_quoted_identifiers: This setting affects the way Aster Database processes strings
enclosed in double-quote characters ("..."). With enable_quoted_identifiers='on' (the
default), Aster Database follows the standard behavior of interpreting each double-quoted
string as an identifier (a column, table, schema, or function name). With
enable_quoted_identifiers='off', each double-quoted string is interpreted as a literal
string constant. Any printable character may be represented in a double-quoted string.
97
Aster Client Guide
Connect Using Database Drivers
Configure Aster Database SQL Settings
Setting the SQL Behavior Parameters
At the SQL Prompt
You can set these parameters from the SQL prompt by executing SET commands in the form:
SET SESSION <parameter name> = {'<param setting>'};
For example:
SET SESSION enable_backslash_escapes = {'on'|'off'};
and
SET SESSION enable_quoted_identifiers = {'on'|'off'};
In the Aster Database ODBC Driver
The Aster Database ODBC-NG driver allows you to set Aster Database SQL behavior
parameters like, for example, enable_backslash_escapes. The connection parameters are:
•
enable_backslash_escapes={on|off}
•
enable_quoted_identifiers={on|off}
These can be set in the DSN configuration (i.e., the odbc.ini file or Windows registry).
Alternatively, a connecting application may set these options in the connection string when
connecting without a registered DSN.
Syntax for ODBC Commands
This section lists some ODBC commands for which the syntax differs from the native Aster
Database syntax. Note that comments and parameter markers are not allowed in these
commands. However, the prepare/execute model is supported.
COPY FROM/TO
Semantics and overall flow of execution for COPY is the same as for the Aster Database Loader
Tool. See “Aster Loader Tool” on page 129.
Syntax
COPY table [(column list)] FROM <quoted file name> ...COPY attributes
or
COPY table [(column list)] TO <quoted file name> ...COPY attributes
The COPY command accepts a quoted file name and streams the data into or out of Aster
Database using the Aster Database Loader Tool protocol for maximum throughput.
INSTALL
The INSTALL command is similar to the ACT command “\install <FILE> [[<SCHEMA>/
]<FILE_ALIAS>]” on page 76, and supports SQL-MR security semantics.
Syntax
INSTALL FILE <quoted file name> [[<schema>/]<file alias>]
•
Aster Client Guide
The schema name must be quoted if it contains spaces or mixed case.
98
Connect Using Database Drivers
Process SQL Statements in JDBC
•
The file alias must be quoted.
UNINSTALL
The UNINSTALL command supports SQL-MR security semantics.
Syntax
UNINSTALL FILE <quoted file name> [[<schema>/]<file alias>]
•
The schema name must be quoted if it contains spaces or mixed case.
•
The file alias must be quoted.
DOWNLOAD
The DOWNLOAD command is similar to the ACT command “\download [[<SCHEMA>/
]<FILE_ALIAS>] <FILE>” on page 76, and supports SQL-MR security semantics.
Syntax
DOWNLOAD FILE <quoted file name> [[<schema>/]<file alias>]
•
The schema name must be quoted if it contains spaces or mixed case.
•
The file alias must be quoted.
Process SQL Statements in JDBC
You can execute SQL statements by using a Statement or a PreparedStatement instance.
Sometimes it is more convenient to use a PreparedStatement object for sending SQL
statements to the database. This special type of statement is derived from the more general
class, Statement, that you already know.
If you want to execute a Statement object many times, it normally reduces execution time to
use a PreparedStatement object instead.
The main feature of a PreparedStatement object is that, unlike a Statement object, it is
given an SQL statement when it is created. The advantage to this is that in most cases, this SQL
statement is sent to the DBMS right away, where it is compiled. As a result, the
PreparedStatement object contains not just an SQL statement, but an SQL statement that
has been precompiled. This means that when the PreparedStatement is executed, the
DBMS can just run the PreparedStatement SQL statement without having to compile it
first.
Process a Simple Query in JDBC
This example issueS a simple query and printS out the first column of each row using
a Statement.
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery("SELECT * FROM mytable WHERE columnf =
500");
while (rs.next()) {
System.out.print("Column 1 returned ");
99
Aster Client Guide
Connect Using Database Drivers
JDBC Troubleshooting and Limitations
System.out.println(rs.getString(1));
}
rs.close();
st.close();
This example issues the same query as before but uses a PreparedStatement and a bind
value in the query.
int fvalue = 500;
PreparedStatement st = conn.prepareStatement("SELECT * FROM mytable
WHERE columnf = ?");
st.setInt(1, fvalue);
ResultSet rs = st.executeQuery();
while (rs.next()) {
System.out.print("Column 1 returned ");
System.out.println(rs.getString(1));
}
rs.close();
st.close();
The instance returns a ResultSet containing the results. Aster Database does not support
cursor-based server-side caching of results which would save a small amount of time in
parsing.
To retrieve data from the ResultSet instance, call next(), which returns true if there are
results.
The ResultSet instance can be closed by calling close(), or if another query is issued using
the same Statement instance that was used to create this ResultSet.
Statement stmt=con.createStatement();
ResultSet rs = stmt.executeQuery("select count(*) from page_views");
while (rs.next()) {
System.out.println(rs.getInt(1));
}
rs.close();
stmt.close();
You can also run Update/Insert/Delete statements and Create/Drop Table statements using the
same method.
JDBC Troubleshooting and Limitations
Infinity and Negative Infinity Timestamp Conversion
The JDBC driver converts 'infinity' and '-infinity' to specific TimeStamp values (a very large
and a very tiny TimeStamp) in the ResultSet.getObject API, which can cause unexpected
results on the client side.
Aqua Data Studio and other client tools, like SQL Assistant JAVA edition also have this issue.
These tools do not support 'infinity' and '-infinity' in the timestamp datatype. If they
encounter these values, they will display specific TimeStamp values (a very large and a very
tiny TimeStamp).
Aster Client Guide
100
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
If you want to retain the negative infinity and infinity values, you can use ResultSet.getString
instead. Then you will be able to get the correct values for infinity and negative infinity,
without this conversion to a very small or very large timestamp value.
Thread Safety
The Aster legacy JDBC driver (pre-5.0.3) supports thread safety for all JDBC objects. The new
Aster JDBC Driver (5.0.3 and later), does not support JDBC object thread safety in all cases. If
you have developed code based on the legacy JDBC driver which uses the same instance of a
JDBC object across multiple threads simultaneously, you may have to modify your code to
ensure that JDBC object thread safety is implemented by your application.
Connect Reporting Tools to Aster Database
This section shows how to connect some specific reporting and query tools to Aster Database:
•
Connect Aqua Data Studio to Aster Database
•
Connect MicroStrategy to Aster Database
Connect Aqua Data Studio to Aster Database
ADS support is being deprecated.
AquaFold’s ADS lets you perform DDL operations and query data interactively, and it
provides tools that help you write and manage queries efficiently. ADS is a third-party tool
available for purchase from Aqua Fold directly.
Aster Database is compatible with ADS version 10.0.2 with patch ads-10.0.7_03-patch.zip.
Install ADS
This section explains how to install ADS on your client workstation and connect to an Aster
Database.
1
Download Aqua Data Studio version 10.0.2 or later from http://www.aquafold.com/
downloads.html
2
Install Aqua Data Studio on your client workstation as explained in your version of the
Aqua Data Studio documentation at http://docs.aquafold.com/
aquadatastudio_11_documentation.html
101
3
Start Aqua Data Studio.
4
Select the command Server: Register Server.
5
In the list, select the “Aster nCluster Driver.”
6
Fill in the tabs as required.
Aster Client Guide
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
Apply the ADS Patch
1
Download the patch from AquaFold:
http://dd1.aquafold.com/download/v10.0.0/ads-10.0.7_03-patch.zip
2
Unzip the patch files.
3
Replace the files under C:\Program Files\Aqua Data Studio 10.0 - 64bit\lib with the files
from ads-10.0.7_03-patch.zip.
4
Restart ADS.
Connect MicroStrategy to Aster Database
Observe the following guidelines when connecting MicroStrategy to Aster Database:
Versions
MicroStrategy 9 or later is required, and Aster Database 5.0 or later is required.
Platforms Supported
Aster Database supports Intelligence Server clients running on Windows XP and Windows
Vista with the Aster Database ODBC driver for Windows.
Limitations
Aster Database is only certified as a warehouse with MicroStrategy. Aster Database cannot be
used as a repository.
Set-up Instructions
To connect MicroStrategy to Aster Database, follow the steps below.
Prerequisites
1
Microstrategy supports the ODBC Driver Manager bundled with the Aster Database
ODBC driver and the Windows Driver Manager only.
2
Make sure the following patches are applied to your MicroStrategy installation:
MicroStrategy 8: Contact MicroStrategy Customer support for
•
DATABASE.PDS file that's certified with Aster Database
•
DTMAPPING.PDS file that's certified with Aster Database
MicroStrategy 9: Contact MicroStrategy Customer support for
•
DTMAPPING.PDS file that works with Aster Database
MicroStrategy 9.0.1 and above: No changes required
Install Drivers
On the client machine where MicroStrategy runs, install the database drivers:
Aster Client Guide
1
Install the Aster Database ODBC driver. See “ODBC Driver” on page 81 for installation
instructions.
2
Install the MicroStrategy VLDB driver, version 9 or later.
102
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
Connecting to an Aster Database
1
Open your MicroStrategy project and choose Intelligence Server as your connection option.
2
Use the Configuration Wizard to create a New Project Source.
3
Create a New Database Instance and give it a name.
4
In the Connection Type field, choose Aster Database nCluster.
5
Create a New Database Connection and give it a name.
6
Create a new ODBC System DSN and give it a name.
Once you have saved the System DSN, you can create reports in MicroStrategy that use the
data from your Aster Database. In your Logical Table definitions in MicroStrategy, you can
create queries that use special Aster Database tools such as nPath.
Best Practices
By following the guidelines below, you can avoid common errors in Aster DatabaseMicroStrategy integration.
Schema changes
•
If your schema contains NUMERIC(X,0) type columns, you should replace these with
INT or BIGINT type columns for a higher probability of success with existing
MicroStrategy reports.
•
For your small- to medium-sized tables that have no BIGINT or INT columns, you should
create the tables in Aster Database as replicated dimension tables. This takes more space in
the cluster, but works better with MicroStrategy.
Operational items
When pointing an existing MicroStrategy report from another database to Aster Database, the
warehouse schema should be “UPDATED” the first time when pointed to an Aster Database
Instance. This updates the metadata inside MicroStrategy for correct MicroStrategy columns.
This is particularly important when a schema is ported from some other database to Aster
Database.
SQL query changes
•
103
To use Aster Database’s nPath and other custom SQL-MapReduce functions:
•
Reports must be created as free-form SQL type reports in MicroStrategy.
•
Teradata Aster recommends that you embed the query in an MicroStrategy logical
table and build the reports on top of that.
•
If there are existing reports that need to run on Aster Database, some tweaking may be
needed in the VLDB Settings of MicroStrategy depending on the complexity of the
dynamically generated queries. For example, you may need to turn off TEMP table
creation, and so on. This is very rare, but editing VLDB settings can be useful in some
cases.
•
If the following error appears in queries: “Unable to find function to convert from <x> to
<y>”, and if x and y are datatypes like int, float, and so on, then you may need a
workaround to explicitly cast the column types. Contact Teradata Support for help on this.
Aster Client Guide
CHAPTER 5
Tools for .NET Environments
Teradata Aster provides a suite of tools that enable you to use Aster Database as a data source
in your .NET applications and reports. These tools include:
•
.NET Data Provider for Aster: A managed database driver for .NET that allows Microsoft
SQL Server Integration Services (SSIS) and .NET client applications to connect to an Aster
Database server.
•
Tutorials, including a sample program that shows how to use the Aster Database tools for
.NET in a Microsoft reporting and BI environment.
This section explains how to install these tools and how to start writing applications and
reports that work with Aster Database.
•
SSIS Data Loading with the .NET Data Provider for Aster
•
Sample Program for .NET
SSIS Data Loading with the .NET Data Provider
for Aster
This section is a tutorial that shows you how to create a Data Flow task in SSIS for transferring
data from a table to a flat file via the .NET Data Provider for Aster.
Overview
SSIS is a tool for building extract, transform, and load (ETL) jobs. In .NET environments,
SSIS provides a fast way to extract data from or load data into your Aster Database databases.
To connect to Aster Database, SSIS requires the .NET Data Provider for Aster. This tutorial
shows you how to set up SSIS to use the driver and provides an example for exporting data
from Aster Database to a flat file.
Procedure
To set up SSIS to use .NET Data Provider for Aster, follow these steps:
Aster Client Guide
104
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
1
Run Microsoft SSIS
2
Choose File: New > Project to create a new integration project.
3
In the New Project dialog:
4
a
Select Integration Services Project.
b
Enter your project a name
c
Click OK.
In the Connection Manager tab at the bottom of the window, add a new Connection.
To do this, right-click and select New ADO.NET Connection.
5
In the Configure ADO.NET Connection Manager window, click New.
In the next couple of steps we’ll create a widget (a database connection definition) that
allows SSIS to connect to your Aster Database.
105
Aster Client Guide
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
6
In the Connection Manager dialog, in the Provider drop-down box, choose Aster Data
Provider, and click OK.
7
8
Aster Client Guide
In the Connection Manager dialog, make the following settings:
a
Set the Database to the name of the database in Aster Database you want to connect to.
b
Set the Host to your Aster Database queen IP address.
c
Set Port to the queen port, which is usually 2406.
d
Set the User Id and Password to the credentials of a user with sufficient rights on your
database in Aster Database.
e
Click OK. Note the name of the connection definition you are saving. Click OK again.
Choose View > Toolbox.
106
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
9
Drag and drop the Data Flow Task from the Control Flow Items list into the Control Flow
designer panel.
10 Click the Data Flow tab.
11 From the Data Flow Sources panel, drag an ADO NET Source object into the Data Flow panel.
12 To make this source a connection to your database in Aster Database,
double-click ADO NET Source to configure it.
The ADO.NET Source Editor dialog box appears.
107
Aster Client Guide
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
13 In the ADO.NET Source Editor dialog box, configure the Connection Manager properties:
a
Click Connection Managers.
b
From the Data access mode drop-down menu, choose SQL command.
c
In the SQL command text field, enter a SQL query.
14 In the ADO.NET Source Editor dialog box, check the Columns property values:
Aster Client Guide
a
Click Columns.
b
Check the External Column and Output Column values.
108
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
15 In the ADO.NET Source Editor dialog box, configure the Error Output properties:
a
Click Error Output.
b
For each output column, set its Error property to Redirect row.
16 In the ADO.NET Source Editor dialog box, click OK.
17 From the Data Flow Destinations panel, drag a Flat File Destination object into the Data Flow
panel.
109
Aster Client Guide
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
18 Connect the green arrow of the ADO NET Source object to the Flat File Destination object.
19 Configure the Flat File Destination object.
Aster Client Guide
a
Double-click the Flat File Destination object.
b
In the Flat File Destination Editor, click Connection Manager.
c
Click New.
d
In the Flat File Format dialog box, click Delimited.
e
Click OK.
110
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
111
f
In the Flat File Connection Manager Editor dialog box, in the Connection manager name
field, enter the name of the output file of the connection manager.
g
Click Browse.
h
Select an output file and click Open.
i
Click OK.
j
In the Flat File Destination Editor, click Mappings.
k
Set the correct column mappings for the Flat File Destination object.
l
Click OK.
Aster Client Guide
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
20 From the Data Flow Destinations panel, drag a Flat File Destination object into the Data Flow
panel.
This object is the destination for all error records.
21 Connect the error output (red arrow) of the ADO NET Source object to Flat File
Destination you just added.
22 Create another Flat File Connection Manager for the new Flat File Destination object that serves
as the error destination.
Aster Client Guide
a
Double-click the Flat File Destination object.
b
In the Flat File Destination Editor, click Connection Manager.
c
Click New.
d
In the Flat File Format dialog box, click Delimited.
e
Click OK.
f
In the Flat File Connection Manager Editor dialog box, in the Connection manager name
field, enter the name of the output file to be used to store all error records.
g
Click Browse.
h
Select an output file and click Open.
i
Click OK.
j
In the Flat File Destination Editor, click Mappings.
112
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
k
Set the correct column mappings for the error destination.
l
Click OK.
23 Save the project.
24 Run the project with debugging (Debug > Start Debugging).
You can check the progress of the workflow in the Progress panel.
113
Aster Client Guide
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
When the export is successful, the flat file source and destination objects in the Data Flow
pane turn green.
If the export is not successful, the ADO NET Source object turns red. For example, you
might get an exception like “Error: 0xC0047062 at Data Flow Task, ADO NET Source [16]:
System.ArgumentException: Error loading assembly: C:\Program Files (x86)\Teradata
Aster\Aster .NET Data Provider (x86)\AsterDataC#DSII.dll.”
In this case, configure the 32-bit Runtime in SSIS for the project or you install the 64-bit
NClusterDNProvider.
The Microsoft SQL Server Business Intelligence Studio (BIDS) is a Visual Studio plug in.
For SQL Server 2008, this edition is 32-bit only. This requires using 32-bit drivers,
including for Aster Database.
However, in the 64-bit versions of Windows, there is a project flag that you must set to
allow 32-bit drivers to operate. Select the project, right-click, and choose Properties. Then,
set Run64BitRunTime to False. If not, you get an architecture mismatch and other
connection errors.
Aster Client Guide
114
Tools for .NET Environments
SSIS Data Loading with the .NET Data Provider for Aster
Similarly, to run the package outside of BIDS—such as running a SQL Server Agent job or
running the package by itself—click the tab Execution options and check the check box Use
32 bit run time.
115
Aster Client Guide
Tools for .NET Environments
Sample Program for .NET
Sample Program for .NET
This section provides a sample program, Program.cs that demonstrates some of the features
supported by the .NET Data Provider for Aster.
Features Demonstrated in the Sample
The Program.cs sample shows the following:
•
Insert into int column
•
DataReader
•
DataSet
•
Prepared query execution
•
Serializing to XML
•
CopyIn
•
CopyOut
•
Metadata operations
•
Indefinite timeout
•
Exception handling
Steps to Build and Test the Sample Program
1
Create a new Visual C# Console Application project called “Program.cs”.
2
Replace the generated Program.cs with the Program.cs provided below.
3
Ensure that the reference to the .NET Data Provider for Aster is properly set. If not, then
you can do a References: Add Reference and browse to nClusterDNProvider.dll.
4
In the main() method of Program.cs, edit the connectionString values to use your
Aster Database user credentials.
5
Compile and run the application.
Source Code of Program to Demonstrate .NET Data Provider for Aster
Below is the source code for the sample program, “program.cs” that demonstrates the use of
the .NET Data Provider for Aster.
using
using
using
using
using
using
using
System;
System.Collections.Generic;
System.Text;
System.Data;
System.Data.Common;
System.IO;
System.Threading;
namespace ConsoleApplication2
{
class Program
{
Aster Client Guide
116
Tools for .NET Environments
Sample Program for .NET
// This method expects the following table in the backend:
//
// create table tableout(field_int int, field_numeric numeric)
// distribute by hash(field_int);
//
//
/*
static void AddWithDataSet(DbConnection connection)
{
DataSet ds = new DataSet();
nClusterDataAdapter da = new nClusterDataAdapter("select * from tableout", conn);
da.InsertCommand = new DbCommand("insert into tableout(field_int, field_numeric)
" + " values (:a, :b)", conn);
da.InsertCommand.Parameters.Add(new nClusterParameter("a",DbType.Int32));
da.InsertCommand.Parameters.Add(new nClusterParameter("b",DbType.Decimal));
da.InsertCommand.Parameters[0].Direction = ParameterDirection.Input;
da.InsertCommand.Parameters[1].Direction = ParameterDirection.Input;
da.InsertCommand.Parameters[0].SourceColumn = "field_int";
da.InsertCommand.Parameters[1].SourceColumn = "field_numeric";
da.Fill(ds);
DataTable dt = ds.Tables[0];
DataRow dr = dt.NewRow();
dr["field_int"] = 4;
dr["field_numeric"] = 7.3M;
dt.Rows.Add(dr);
DataSet ds2 = ds.GetChanges();
da.Update(ds2);
ds.Merge(ds2);
ds.AcceptChanges();
}
static void GenerateXmlFromDataSet(DbConnection connection)
{
nClusterDataAdapter da = new nClusterDataAdapter("select * from tableb",
connection);
DataSet ds = new DataSet();
da.Fill(ds);
ds.WriteXml("DataSetFeed.xml");
}
*/
// table schema: create table tableBytea (field_int int, field_bytea bytea)
distribute by hash(field_int);
static void ByteaDataType(DbConnection connection, String filename)
{
int res;
FileStream fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
BinaryReader br = new BinaryReader(new BufferedStream(fs));
Byte[] bytes = br.ReadBytes((Int32)fs.Length);
Console.WriteLine(fs.Length);
br.Close();
fs.Close();
DbCommand cmd = connection.CreateCommand();
cmd.CommandText = "insert into tableBytea values(1, ?)";
DbParameter param = cmd.CreateParameter();
param.ParameterName = "bytesData";
param.DbType = DbType.Binary;
param.Value = bytes;
cmd.Parameters.Add(param);
res = cmd.ExecuteNonQuery();
117
Aster Client Guide
Tools for .NET Environments
Sample Program for .NET
cmd.CommandText = "select field_bytea from tableBytea where field_int = 1;";
Byte[] result = (Byte[])cmd.ExecuteScalar();
fs = new FileStream(filename + ".suffix", FileMode.Create, FileAccess.Write);
BinaryWriter bw = new BinaryWriter(new BufferedStream(fs));
bw.Write(result);
bw.Flush();
fs.Close();
bw.Close();
}
private static void CopyIn(DbConnection connection)
{
Console.WriteLine("Creating command...");
DbCommand cmd = connection.CreateCommand();
cmd.CommandText = "COPY t_test_adonet_in from 'foo.out'";
Console.WriteLine("Command created: " + cmd.CommandText);
Console.WriteLine();
Console.WriteLine("ExecuteNonQuery: " + cmd.CommandText);
cmd.ExecuteNonQuery();
}
private static void CopyOut(DbConnection connection)
{
Console.WriteLine("Creating command...");
DbCommand cmd = connection.CreateCommand();
cmd.CommandText = "COPY t_test_adonet to 'foo.out'";
Console.WriteLine("Command created: " + cmd.CommandText);
Console.WriteLine();
Console.WriteLine("ExecuteNonQuery: " + cmd.CommandText);
cmd.ExecuteNonQuery();
}
// table schema: create table t4 (i int, j char(10)) distribute by hash(i);
public static void SimpleInsert(DbConnection connection)
{
DbCommand command = connection.CreateCommand();
command.CommandText = "insert into t_test_adonet values(11, 'eleven')";
Int32 rowsaffected;
rowsaffected = command.ExecuteNonQuery();
Console.WriteLine("{0} rows added in table t_test_adonet", rowsaffected);
}
// table schema: create table t4 (i int, j char(10)) distribute by hash(i);
public static void DataReader(DbConnection connection)
{
DbCommand command = connection.CreateCommand();
command.CommandText = "select * from t_test_adonet";
DbDataReader reader = command.ExecuteReader();
while (reader.Read())
{
for (int i = 0; i < reader.FieldCount; i++)
{
Console.Write("{0} \t", reader[i]);
}
Console.WriteLine();
}
}
// table schema: create table t4 (i int, j char(10)) distribute by hash(i);
public static void PreparedQuery(DbConnection connection)
{
// Declare the parameter in the query string
DbCommand command = connection.CreateCommand();
Aster Client Guide
118
Tools for .NET Environments
Sample Program for .NET
command.CommandText = "select * from t_test_adonet where i > ?";
// Now add the parameter to the parameter collection of
//the command specifying its type.
DbParameter param = command.CreateParameter();
param.ParameterName = "value1";
param.DbType = DbType.Int32;
command.Parameters.Add(param);
command.Parameters[0].Value = 2;
using (DbDataReader reader = command.ExecuteReader())
{
while (reader.Read())
{
for (int i = 0; i < reader.FieldCount; i++)
{
Console.Write("{0} \t", reader[i]);
}
Console.WriteLine();
}
}
}
// table schema: create table t_test_adonet (i int, j char(10)) distribute by
hash(i);
static void OutputParameter(DbConnection connection)
{
Console.WriteLine("OuputParameter start....");
// Send a query to backend.
DbCommand command = connection.CreateCommand();
command.CommandText = "select * from t_test_adonet where i = 11";
// Now declare an output parameter to receive the first column of
// the tablea.
DbDataReader reader = command.ExecuteReader();
// Now, the firstcolumn parameter will have the value of
// the first column of the resultset.
Print(reader);
Console.WriteLine("OuputParameter end.");
}
static void Print(DataTable tbl)
{
// For each row, print the values of each column.
foreach (DataRow row in tbl.Rows)
{
for (int i = 0; i < tbl.Columns.Count; i++)
{
Console.Write("{0} \t", row[i]);
}
Console.WriteLine();
}
}
static void Print(DbDataReader reader)
{
// For each row, print the values of each column.
for (int i = 0; i < reader.FieldCount; ++i)
{
Console.Write(String.Format("{0,12} |", reader.GetName(i)));
119
Aster Client Guide
Tools for .NET Environments
Sample Program for .NET
}
// Write out the dividing line.
Console.WriteLine();
for (int i = 0; i < reader.FieldCount; ++i)
{
Console.Write("==============");
}
Console.WriteLine();
// Write out the actual data.
while (reader.Read())
{
for (int i = 0; i < reader.FieldCount; ++i)
{
Console.Write("{0,12} |", (reader.IsDBNull(i)) ?
"<null>" : FormatData(reader[i]));
}
Console.WriteLine();
}
}
public static string FormatData(object obj)
{
if (obj is DBNull)
{
return "<null>";
}
if (obj is Byte[])
{
const string hexChars = "0123456789ABCDEF";
string binary
foreach (byte
{
binary +=
binary +=
}
= "0x";
b in (Byte[])obj)
hexChars[(b >> 4) & 0x0F];
hexChars[b & 0x0F];
return binary;
}
else
{
return obj.ToString();
}
}
static void GetMetaDataCollections(DbConnection connection)
{
DataTable tbl = connection.GetSchema("MetaDataCollections");
Print(tbl);
}
static void GetRestrictions(DbConnection connection)
{
DataTable tbl = connection.GetSchema("Restrictions");
Print(tbl);
}
/*
static void GetDatabases(DbConnection connection)
{
DataTable tbl = connection.GetSchema("Databases", new String[] {
"beehive" });
Aster Client Guide
120
Tools for .NET Environments
Sample Program for .NET
Print(tbl);
}
*/
static void GetTables(DbConnection connection)
{
string[] restrictions = new string[4];
restrictions[2] = "tablea";
Print(connection.GetSchema("Tables", restrictions));
}
static void GetColumns(DbConnection connection)
{
string[] restrictions = new string[4];
restrictions[2] = "tablea";
Print(connection.GetSchema("Columns", restrictions));
}
static void GetViews(DbConnection connection)
{
string[] restrictions = new string[4];
restrictions[2] = "tablea_view";
Print(connection.GetSchema("Tables", restrictions));
}
/*
static void GetUsers(DbConnection connection)
{
DataTable tbl = connection.GetSchema("Users", new String[] { "beehive" });
Print(tbl);
}
* */
static void Setup(DbConnection connection)
{
Int32 rowsaffected;
Console.WriteLine("Creating the tables ...");
DbCommand command1 = connection.CreateCommand();
command1.CommandText = "DROP TABLE IF EXISTS tableBytea, tableout";
rowsaffected = command1.ExecuteNonQuery();
DbCommand command2 = connection.CreateCommand();
command2.CommandText = "CREATE TABLE t_test_adonet (i int, j char(10)) distribute
by hash(i)";
rowsaffected = command2.ExecuteNonQuery();
DbCommand command3 = connection.CreateCommand();
command3.CommandText = "CREATE TABLE tableBytea (field_int int, " + " field_bytea
bytea) " + "distribute by hash(field_int)";
rowsaffected = command3.ExecuteNonQuery();
DbCommand command4 = connection.CreateCommand();
command4.CommandText = "create table tableout(field_int int, " + "field_numeric
numeric) distribute by hash(field_int)";
rowsaffected = command4.ExecuteNonQuery();
}
static void Main(string[] args)
{
DbConnection connection = null;
//String connectionString = "Server=153.65.197.90;Port=2406;User
Id=beehive;Password=beehive;Database=beehive;CommandTimeout=-1";
//DbConnection conn = new nClusterConnection(connectionString);
try
121
Aster Client Guide
Tools for .NET Environments
Sample Program for .NET
{
Console.WriteLine("Aster .Net Provider Test Program");
Console.WriteLine();
Console.WriteLine();
Console.WriteLine("Looking up provider factory...");
DbProviderFactory factory = DbProviderFactories.GetFactory("Aster.Net");
Console.WriteLine("Found provider factory.");
connection = factory.CreateConnection();
string connectionString =
"uid=beehive;pwd=beehive;dbname=beehive;ip=153.65.197.90;port=2406;NumericAndDecimalAsDo
uble=1";
connection.ConnectionString = connectionString;
// Connect to the database then retrieve the schema information.
Console.WriteLine("Connecting...");
connection.Open();
Console.WriteLine("Connected.");
Console.WriteLine();
// create the tables
Setup(connection);
// Metadata operations
GetMetaDataCollections(connection);
GetRestrictions(connection);
//GetDatabases(connection);
GetTables(connection);
GetColumns(connection);
GetViews(connection);
//GetUsers(connection);
DataReader(connection);
SimpleInsert(connection);
PreparedQuery(connection);
OutputParameter(connection);
//AddWithDataSet(connection);
// GenerateXmlFromDataSet(connection);
ByteaDataType(connection, "..\\..\\input\\out.txt");
CopyOut(connection);
CopyIn(connection);
}
catch (Exception e)
{
Console.WriteLine();
Console.WriteLine("***********************************************************");
Console.WriteLine("Exception: " + e.Message);
Console.WriteLine("Stack: ");
Console.WriteLine(e.StackTrace);
Console.WriteLine("***********************************************************");
if (connection != null)
connection.Close();
}
Console.WriteLine("Press any key to continue.");
Console.ReadKey();
}
}
}
Aster Client Guide
122
Tools for .NET Environments
Sample Program for .NET
Possible Exceptions and Resolutions
These are resolutions to possible exceptions:
•
•
You might get an exception like “Exception: Unable to find the requested .Net Framework
Data Provider. It may not be installed.”
•
Reason: Your 32-bit provider is not installed or is broken.
•
Resolution: Reinstall your 32-bit provider and make sure that the size and version are
correct.
Loading assembly error. You might get an exception like:
Error -- 1056964601 : Internal error: The operation terminated
unsuccessfully.Error -- 1055784933 : Errors in the high-level
relational engine. The following exception occurred while the managed
IDbConnection interface was being used:
Error loading assembly:
C:\Program Files (x86)\Teradata Aster\Aster .NET Data Provider
(x86)\AsterDataC#DSII.dll.
If this exception occurred in SSAS:
•
123
•
Reason: Your 64-bit provider might not be installed correctly.
•
Resolution: Reinstall your 64-bit provider and make sure that the size and version are
correct.
If this exception occurred in SSIS, then:
•
Reason: The 32-bit Runtime not enabled.
•
Resolution: Right-click the project and choose Properties. Then, set Run64BitRunTime to
False.
•
Note that in the SSAS view, no time zone information is shown for Aster Database
columns of the date data type “time with time zone”. This is a known issue.
Aster Client Guide
CHAPTER 6
Loading Data
To load data into Aster Database, you can use the Aster Loader Tool, the COPY command, the
INSERT command, or a custom-defined SQL-MapReduce data loading function you have
written. This section provides tips for efficient loading and shows how to load using the Aster
Loader Tool. The following sections explain these utilities:
•
Best Practices for Data Loading
•
Aster Loader Tool
•
Troubleshoot Loading
Best Practices for Data Loading
This section describes best practices for efficiently loading data into Aster Database. In
particular, this section explains how to help optimize new loading projects using the tools
provided by the Aster Loader nodes and the Aster Loader Tool (ncluster_loader). The sections
are:
•
Loading Terminology
•
Scenario 1: Pre-Production Data Loading
•
Scenario 2: Loading in a Production Environment
Loading Terminology
We use the following terms in the text that follows:
Aster loader node: In the cluster, a loader node is a node dedicated to data loading. Many
loader nodes can operate in parallel.
Aster Loader Tool (ncluster_loader) is the client application for initiating high-speed bulk
loads.
Aster Database Load Error Logging is a feature in ncluster_loader that allows you to perform
loading that is more tolerant of poorly formatted input data. Load Error Logging sends
malformed rows to an error logging table.
Input data: Source input file(s) which are to be loaded into Aster Database. All source files are
compatible with a format that Aster Database is able to load. Examples include the CSV
Aster Client Guide
124
Loading Data
Best Practices for Data Loading
format (RFC 4180: http://www.rfc-editor.org/rfc/rfc4180.txt) or the tab-delimited format. Each
file must use a consistent newline character throughout. Never mix UNIX and Windows-style
newline characters in the same file; this will cause your load attempt to fail!
Data staging node(s): Nodes where all the input data is present. In a typical setup the input
data is stored on the local filesystem. However, other use cases where all the data is stored on a
network-mounted storage device are possible.
Row(s): In any given input data file individual rows are present. The used format for the input
data describes where row boundaries are. Usually a “row” refers to a logical unit of
information that needs to be stored as a unit in the data warehouse. One example for rows
include call records which consist of source and destination phone numbers, call duration
time, and so on.
The following section highlights three different data loading scenarios and the respective best
practices that should be performed to make the process as a whole as seamless as possible.
Scenario 1: Pre-Production Data Loading
To validate your loading process and data before deploying it to your production system, it is
an accepted industry best practice to load to a pre-production or testing system. You can
perform test data loading using the standard Aster Database command-line, bulk loading tool,
ncluster_loader. For a complete overview of the tool’s command-line options refer to
“Argument Flags” on page 130.
The tool’s error logging features are useful for debugging your loading process. If there is a
possibility that your input data contains malformed rows, you should consider using error
logging for bulk load operations to accomplish the following goals:
1
Make sure that even in the presence of malformed rows a given load operation succeeds.
This can be accomplished by enabling error logging but not setting an error logging limit.
(Set --el-enabled but do not set an --el-limit.) If you are not interested in what
errors are present, malformed input rows can be discarded so that they are not stored in
the cluster (--el-enabled --el-discard-errors).
Tip: A UNIQUE or PRIMARY KEY violation in the data being loaded will always cause the load to abort. So if the
table you are loading into contains these constraints and your load is failing, check the data you are loading to ensure
it complies.
2
Store malformed rows into a separate table for later inspection.
You should specify a custom error logging table to store malformed rows. This is done via
the appropriate option (--el-enabled --el-table = 'my_errorlogging_table').
If the error logging feature of Aster Database is turned on without using --el-table to
specify an error logging table, malformed rows are stored in the default system error
logging tables (nc_errortable_part or nc_errortable_replicated). This is
undesirable because the rows from different loads will be mixed together in the default
tables. Note that any custom error logging table has to inherit from the default system
error logging table.
125
Aster Client Guide
Loading Data
Best Practices for Data Loading
3
Abort data load operation in the presence of too many malformed rows.
This is in particularly useful if you want a given load operation to abort if too many
malformed rows are present in the input data (--el-enabled --el-limit = 100). In
order to preserve atomicity for bulk load operations, the load operation fails as a
transaction when the error limit is reached. When the operation fails, any rows already
written by the transaction to the target table and error logging table are deleted.
4
Label malformed rows in the error logging table.
If multiple operations are loading data into Aster Database at the same time (e.g., the preproduction system is testing integration of two separate data sources), you can label each
load operation to identify which rows belong to which data source (--el-enabled -el-label = 'my_data_source').
The following types of errors in input data files can be detected:
a
Malformed datatypes (e.g., text value for an integer column);
b
Missing column values (e.g., the input data file provides data values for the first two
columns, but not for the third one);
c
Additional column values (e.g., the input data file contains a row with more values
than there are columns in the destination table);
d
Check constraint violations (e.g., the integer value of an imported data field exceeds
the range allowed by the check constraint on the target table);
e
Character set conversion errors (e.g., input data file contains invalid UTF-8
characters);
f
Missing or invalid distribution key (e.g., for your distribution key, you have specified a
column that has a distribution-key-disallowed datatype).
Summary
In an environment where not all operations are run in an automated fashion, where new data
sources are to be integrated, or where existing ETL processes are to be changed, we
recommend that you set an error logging limit to prevent accumulation of too many
malformed rows in the error logging tables. If malformed data rows need to be inspected, we
recommend that you use a custom error logging table or create a separate error logging table
for that load job.
Scenario 2: Loading in a Production Environment
Teradata Aster recommends that, before you bulk load data into your production system, you
first verify your loading procedure on a test system. In this section, we’ll focus on the
differences between a data loading job in a testing environment and one in a production
system.
In a production deployment, we recommend that you follow the steps below to ensure you
take advantage of the bulk loading and error logging tools in Aster Database:
Aster Client Guide
126
Loading Data
Best Practices for Data Loading
1
Define quote and escape characters that don’t conflict with the data.
If you have double-quote marks inside your data, the easiest way to have them kept as
literals is to choose a character that does not occur anywhere in the input data file and use
that as both the quote character and the escape character. Once the double-quote
character is no longer treated as the quote character, it will be treated as a literal and will be
preserved. So, for example, if your file has no tab characters in it (as either data or as a
column separator), then you can preserve embedded double-quote characters by
specifying
-q '\t' -e '\t'
on the ncluster_loader command line.
2
Avoid extra spaces in your data, because they can be significant.
Quotes can be used on any portion of a data field, typically around special characters. For
example, with the default CSV mode, this is the usual way to handle commas within a
string:
1,2,"red,blue,green"
But putting quotation marks around each comma is equivalent:
1,2,red","blue","green
This can introduce problems when working with varchar columns, because many people
put space between the comma and the quote, and that space is considered significant. For
example:
1,2, "red,blue,green"
In this example, the third column will be loaded with a space before the r character.
3
Consider performance when loading autopartitioned tables.
Beginning in Aster Database version 5.0, loading of logically partitioned tables has much
lower overhead and performs better than previously. Specifically, performance is improved
when loading data into a table with many child partitions. The improvement applies to
tables that use either row or columnar storage, but is most pronounced for tables with row
storage.
4
Run ANALYZE after you load each reasonably sized batch.
When bulk loading data, it’s important to run ANALYZE regularly. In ncluster_loader, use
the --auto-analyze (or -z) flag to do this. With this flag present, ncluster_loader will
run ANALYZE on the target table or tables after it loads the data.
While running, ANALYZE requires an amount of disk space on the cluster proportional to
the amount of new data in the table being analyzed. For this reason, Teradata Aster
recommends that your run ANALYZE after every load, rather than waiting for multiple
loads to finish. Waiting too long can result in a large amount of the cluster’s disk space
being consumed during the running of ANALYZE. Daily child tables offer a good example:
When you load to daily child tables, run ANALYZE on each child table after you load the
day’s data into that table.
5
127
Stay informed about cluster activity.
Aster Client Guide
Loading Data
Best Practices for Data Loading
As with any other production system component, it is important to stay informed about
any irregular activity that is going on in the cluster, in this case in particular what is
happening in the data loading path.
If you want to make sure that large loads with too many malformed rows do not complete
successfully, you can set an error logging limit for your load operation (--el-enabled -el-limit = '100000'). You can specify a limit clause with a value that is small enough
to allow most loads to complete successfully while forcing the failure of loads with
unexpected percentages of malformed rows. Exact values will depend on the size of the
files being loaded and the expected malformed row ratios.
You should use an alert system to monitor the size of your error logging tables.
6
Keep an eye on the volume of data in the load error logging tables.
The system does not perform automatic truncation of the error logging tables! The error
logging tables are regular tables in Aster Database. The same operations that an
administrator would apply to regular tables thus need to be applied to error logging tables.
Tasks include the appropriate setup of child table hierarchies for efficient data processing,
regular VACUUM operations to free up disk space occupied by dead rows, and the
monitoring of table growth over time. Custom error logging tables can be used by passing
the appropriate arguments when you run the Aster Loader Tool (for example, --elenabled --el-table = 'el_table_April_2010').
7
Use an error logging file for faster re-try of rows that failed.
Have ncluster_loader create a file containing the row data that failed to load. To do this,
pass the argument, --el-errfile <my_error_file.txt> to the tool. Upon
completion of the load, you can inspect the contents of the error logging file, fix issues you
find there, and reload from that file.
8
Watch your load error statistics.
To monitor error rates, check the system statistics tables, nc_all_errorlogging_stats
and nc_user_errorlogging_stats. These tables show you the malformed and wellformed row count for each load operation. Records from the statistics tables can be
correlated with the malformed rows in the load error logging tables. Both the error logging
and statistics tables contain an error logging label which can be used to identify a
particular loading operation.
9
Trace malformed input data to the data source.
An important aspect of a production system is that it must provide the user with the
means to find the problem source when an operation fails. When loading data, the user
can use the error information about malformed rows (SQL error code and error message,
sqlerrcode and errmessage, respectively, in the load error logging table) to diagnose
the problem in the input data. To do this, you must invoke your data loading operation
with the error logging feature enabled (--el-enabled). As described above, this
information can then be correlated with the error logging statistics table to find the source
of the malformed data. To facilitate this process, you may wish to apply custom error
logging labels to each load operation (--el-enabled --el-label =
'my_data_source').
Aster Client Guide
128
Loading Data
Aster Loader Tool
Loading Best Practices Summary
For any production system, the close monitoring of system components is absolutely mission
critical. Correlating error information present in the error logging tables can reveal possible
problems early in the process.
Aster Loader Tool
The Aster Loader Tool, ncluster_loader, is a command-line application for bulk-loading data
from files into Aster Database. It provides an alternative to the SQL COPY statement. Each
invocation of the Aster Loader Tool is equivalent to a single invocation of the COPY
statement. Each invocation is a single database transaction that loads the contents of the
specified data file or files into the specified table or tables.
This section includes:
•
Syntax
•
Argument Flags
•
Exit Status
•
Loading Data with the Loader Tool
•
Removing NULLs from data on Linux
•
Loading from Multiple Files Using a Map File
•
Examples
•
Hints for Successful Loading
•
Error Logging
Syntax
To run the Aster Loader Tool, you type:
$ ncluster_loader [arguments] [schemaname.]tablename [ filename |
dirname ]
where
•
arguments are the command-line flags that control how the loader runs. The flags are
explained in Argument Flags, below, or you can display the help by typing:
$ ncluster_loader -?
•
schemaname is the optional name of the destination schema. If no schema name is
provided, Aster Database will search the schemas listed in the schema search path.
•
tablename is the name of the destination table (See “Case-Sensitive Handling for Table
Names” on page 130 if you wish to have Aster Database evaluate table names in a casesensitive manner.);
•
filename or dirname indicates the file or directory of files to be loaded. Files to be loaded
can optionally be compressed gzip or bzip2 files. These are extracted and their contents are
then loaded.
129
Aster Client Guide
Loading Data
Aster Loader Tool
•
filename Qualified path of the file containing the data to be loaded. The contents of
the file must be in either CSV or text format, as described for the COPY statement.
Details of the encoding used (such as non-default values for null or delimiter) are
specified using the appropriate options, as described below; or
•
Qualified path of the directory containing one or more data files to be
loaded. All data files found within this directory are expected to be in the same format
and will be loaded as a single transaction. Subdirectories will not be processed.
dirname
If you don’t supply a file or directory name argument, Aster Loader assumes you want to load
from STDIN. See “Loading from STDIN Example” on page 142.
Tip for Windows users: When using Aster Loader Tool on Microsoft Windows, bear in mind:
• If the filename or dirname contains spaces, make sure you enclose it in double quotes.
• When specifying a dirname, you must use a double backslash at the end of the path. For example, use
“c:\temp\loadFiles\\” to specify a directory called “loadFiles”.
• Never mix UNIX and Windows-style newline characters in the same data file. Doing so will cause your load attempt
to fail.
Case-Sensitive Handling for Table Names
To force Aster Loader to treat your table and schema names in a case-sensitive manner, you
must surround the schema and table names in double quote marks when you invoke
ncluster_loader. To pass a double quote mark to Aster Loader, you must prefix it with the
escape character, which is a backslash. That means you type a double quote mark like so: \".
If either the schema name, the table name or both names include capital letters, you must
surround each name in escaped quotation marks, individually.
For example, assuming you want to load to table "Bar" in schema "Foo", then you would
invoke Aster Loader using this form:
ncluster_loader.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"Foo\".\"Bar\"
mydata.csv -c
and in the map file, you would reference the table as:
"table" : "\"Foo\".\"Bar\"",
If you want to load to table "bar" in schema "Foo", you would still need to escape quote the
schema and the table separately as follows:
ncluster_loader.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"Foo\".\"bar\"
mydata.csv -c
and in the map file, as:
"table" : "\"Foo\".\"bar\"",
Argument Flags
In addition to the schemaname.tablename and filename/dirname arguments explained
above, the Aster Loader Tool takes the following argument flags at the command line or in the
map file. (Map files let you load from many input files in a single running of Aster Loader. See
“Troubleshoot Loading” on page 147.)
Aster Client Guide
130
Loading Data
Aster Loader Tool
In the table that follows, the argument flags are sorted based on the long-form, command-line
flag:
•
The left column lists the flag you use at the command line.
•
The middle column lists the flag you can use in a map file (See “Rules for Passing
Arguments in a Map File” on page 140.). If no value appears in the middle column, then
the argument is one that can only be passed at the command line.
Table 6 - 16: Argument Flags for Aster Loader Tool
Flag at Command Line
Flag in Map File
Description
-a [ --auto-partition]
auto-partition
Specifies that ncluster_loader will run in autopatitioning mode,
which automatically routes each row to the appropriate child table in
a parent/child table created through inheritance.
-B [ --begin-script ]
arg
begin-script
Specifies the qualified path of the file containing SQL commands
that should be executed when the transaction starts, i.e. immediately
after the BEGIN command is issued to Aster Database. Note: Datareturning statements such as SELECT are not allowed in scripts
executed by the ncluster_loader.
Pass this argument if you want ncluster_loader to interpret the input
file as being in CSV format. By default ncluster_loader uses a text file
with a tab-delimited format, often called “TSV”.
-c [ --csv ]
-C [ --columns ] arg
columns
An optional comma-separated list of the columns in the target table
that will receive the data being loaded (the default is to assume that
each input row contains exactly one data value for each column in
the target table, in the order in which the table columns were
defined).
By using -C, you can specify that the ordering of the columns in the
input file is different from the ordering in the table. You can also use
-C to specify that this input file contains values for only some of the
columns in the table.
The input data is assumed to contain values for the columns in the
order specified here. For example, to load data into columns 'col1'
and 'col2', one could specify "col1, col2" as the value for this option.
Column names not specified here are expected to get NULL values.
See “Using the -C Option With Uppercase or Special Characters” on
page 148.
-d [ --dbname ] arg
-D [ --delimiter ] arg
131
dbname
The name of the database to connect to (default is "beehive").
The column delimiter character to use when interpreting the input
file (must be a string that represents a valid single character, such as
'd' or '\n'). The default is a tab character ('\t') in text mode, a
comma in CSV mode.
Aster Client Guide
Loading Data
Aster Loader Tool
Table 6 - 16: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
--date-style arg
date-style
Specifies the date format. If you are using a map file, you must
specify a single date-style per map file.
The following formats are supported:
• ISO - This is the default. Uses ISO 8601-style dates and times
(YYYY-MM-DD HH:MM:SS) like, for example,
2010-12-17 07:37:16-08
• SQL - Uses Oracle/Ingres-style dates and times. For example:
12/17/2010 07:37:16.00 PST
• POSTGRES - Uses traditional PostgreSQL format. For example:
Wed Dec 17 07:37:16 2010 PST
• German - Uses dd.mm.yyyy for numeric date representations. For
example:
17.12.2010 07:37:16.00 PST
• SQL,DMY - For example:
17/12/2010 15:37:16.00 CET
• SQL,MDY - For example:
12/17/2010 07:37:16.00 PST
• Postgres,DMY - For example:
Wed 17 Dec 07:37:16 2010 PST
Specifies the escape character for CSV input (must be a string that
represents a valid single character, such as d or \n). The default
escape character is the same as the default quote character - the
double-quote (").
-e [ --escape ] arg
If an explicit quote character (-q) is specified, and no explicit escape
character (-e) is specified, then the escape character will be set to be
the same as the quote character specified.
You may only pass this option when loading a CSV-formatted file (-csv). Warning: ncluster_loader will fail if you pass this argument
when loading a tab-delimited text file.
-E [ --end-script ] arg end-script
Qualified path of the file containing SQL commands that should be
executed when the transaction is about to commit, i.e., immediately
before the END/COMMIT command is issued to Aster Database.
Note: Data-returning statements such as SELECT are not allowed in
scripts executed by the Aster Loader Tool.
--el-discard-errors
discard-errors
Sub-flag that can accompany --el-enabled. If present, specifies
that all errors (malformed rows) be discarded by Aster Database. By
default, errors are not discarded.
--el-enabled
errorlogging
If present, turns on error logging for this invocation of the
ncluster_loader. This needs to be enabled for any other error logging
option to be accepted. The default is disabled. See “Error Logging”
on page 145.
Aster Client Guide
132
Loading Data
Aster Loader Tool
Table 6 - 16: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
--el-label arg
label
Sub-flag that can accompany --el-enabled. The --el-label
flag specifies the label to be used to tag the malformed rows. By
default, Aster Database will use a statement identifier as the label
value. (There’s one statement identifier per COPY command; if there
is one map entry for many input files, then you’ll have a unique
statement identifier per input file.) The label is useful for finding
your failed rows in the error logging table and for finding statistics
about the load attempt in the nc_all_errorlogging_stats
table.
-F --el-errfile arg
errfile
Sub-flag that can accompany --el-enabled. The --elerrfile flag introduces the pathname of the optional error logging
file. If you use error logging, you must have an error logging table,
and you can have an error logging file. Upon completion of the load,
ncluster_loader writes the contents of the error logging table’s
rawdata column (and no other columns) to the error logging file.
Only the contents of this column are written to the file. The filename
will have a numeral appended to it in the form, “_0”.
For this option to work, you must have also specified an el-table.
Regardless of whether or not you specify that an error logging file
should be used, the error logging table will still contain all error rows
upon completion of each load.
-L [ --el-limit ] arg
limit
Sub-flag that can accompany --el-enabled. The --el-limit
flag sets the maximum number of malformed rows allowed for each
file being loaded. By default the limit is unbounded. If the error
count during your load attempt exceeds the specified --el-limit,
the transaction aborts and no rows are inserted into the destination
table, and no rows are inserted into the error logging table.
Supported on Linux only. This flag executes a shell script against the
source file, and uses the output of the script as the new source file for
loading. You can use this feature along with the provided script
remove-null.sh to remove NULL(0x00) values from data to be
loaded using Aster Loader Tool on Linux.
--preprocess-script arg
-S [ --el-schema ] arg
schema
Sub-flag that can accompany --el-enabled. The --el-schema
flag specifies the name of the schema to which the error logging table
belongs. If not specified, the public schema is used.
-T [ --el-table ] arg
table
Sub-flag that can accompany --el-enabled. The --el-table
flag specifies the name of the error logging table that will capture
malformed rows.
If you are loading to a distributed table, the error logging table must
be a distributed table. Likewise, if you are loading to a replicated
table, the error logging table must be a replicated table.
If this flag is not provided, Aster Database uses a default table:
nc_errortable_repl is used if the load-destination table is a
replicated table, or nc_errortable_part is used if the loaddestination table is a distributed table.
Important: The system does not perform automatic truncation of the
error logging tables! You must VACUUM them regularly.
133
Aster Client Guide
Loading Data
Aster Loader Tool
Table 6 - 16: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
-f [ --force-loader ]
force-loader
Instructs the Aster Loader Tool to use the loader node specified with
the -l or --loader option even if the IP address provided is not
known to Aster Database. Note that if this option is specified, the
Aster Loader Tool will only try that single IP address and return an
error status if the connection fails for any reason.
filename (Not a flag; you pass
file
Qualified pathname of the file or directory containing data to be
loaded. The contents of the file must be in either CSV or text format,
as described for the COPY statement.
the file or directory name itself!)
If you pass a directory name, then all data files found in the directory
are loaded in a single transaction. Subdirectories are ignored. All files
must be in the same format.
If you pass no file or directory name, Aster Loader loads from
STDIN.
In a map file, use the file flag. At the command line, there is no flag
for this argument; instead, ncluster_loader assumes that you pass the
named-flag arguments first, then the table name, and finally the
source file name.
-? [ --help ]
Shows online help for Aster Loader Tool.
-h [ --hostname ] arg
Host name or IP address of the Aster Database queen. Default is
'localhost'. This always points to the queen, even if you’re loading
using loader nodes. See “Multiple Loader Nodes” on page 143.
--header-included
Indicates that the first line of the data file to be input is a comma
delimited list of the column names. The number of skip-rows will
be counted beginning after the header, if this option is specified.
Cannot be combined with --columns.
-l [ --loader ] arg
loader
Preferred loader IP address. If a value is provided, the Aster Loader
Tool tries to load to this IP address before trying any other loader
node. Note that the Aster Loader Tool expects this IP address to be
known by Aster Database and will ignore the address provided if this
is not the case. To change this behavior one can use the -f or -force-loader flag.
-m [ --map-file ] arg
Name of the file containing mappings of data files or directories to
target tables. This option allows you to load multiple directories into
a table and also to load data into multiple tables within the same
transaction. For details about the format used for the map file, see
“Troubleshoot Loading” on page 147.
-n [ --null ] arg
String that is used to represent a null value. The default is \N in text
(non-CSV) mode, and an empty value with no quotes in CSV mode.
When loading any non-CSV delimited format (e.g. TSV), you can
easily load files that contain empty strings (that is, files that don’t use
the typical \N to represent nulls). To do this, run
ncluster_loader with the -n flag, followed by two doublequote characters. That is, you type:
ncluster_loader -n ""
See also --null-backslash-escapes.
Aster Client Guide
134
Loading Data
Aster Loader Tool
Table 6 - 16: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
Indicates that backslashes in the '-n/--null' flag should be
treated as special characters, which will be processed according to the
default rules for escaped strings.
--null-backslashescapes
You should use this flag whenever the null string contains a
backslash, to ensure compatibility with future versions of Aster
Database.
TCP port on which Aster Database listens for connections. Default is
2406.
-p [ --port ] arg
-P [ --data-prefix ]
arg
data-prefix
String that should be used as a prefix for each row (each line in a data
file) being loaded into Aster Database.
Important: The Aster Loader Tool will simply concatenate this value
to each row without any parsing. You must insert the proper
delimiter, quote, and escape characters in the prefix string.
Character to be used as the quoting character for CSV input (must
be a string that represents a valid single character, such as d or \n).
The default is the double-quote. This option is only valid when CSV
input has been specified (-c or --csv) and will cause the load to
fail otherwise.
-q [ --quote ] arg
If an explicit quote character (-q) is specified, and no explicit escape
character (-e) is specified, then the escape character will be set to be
the same as the quote character specified.
Note that if using a single quote (') as the quoting character, you
should escape it when specifying it with -q as -q"\'". Otherwise
any input data that includes a comma will cause the load to fail.
Specifies how many rows of the file to skip before starting to load
data. If combined with --header-included, --skip-rows will
not start counting until after the first line in the file. The default is 0.
--skip-rows arg
-t [ --timeout ] arg
timeout
Timeout value in seconds for Aster Database connection attempts.
Default is 30 seconds.
tablename (Not a flag; you
pass the table name itself!)
table
Name of the destination table. In a map file, use the table flag. At
the command line, there is no flag; instead, ncluster_loader assumes
that you pass the named-flag arguments first, then the table name,
and finally the source file name or directory name.
--truncate-table
truncate-table
Performs a TRUNCATE operation to empty the table before
beginning the load. If child tables are present, they will be truncated,
too. By default, this feature is disabled.
-U [ --username ] arg
username
If present, the Aster Loader Tool connects to the database with this
username instead of the default ("beehive"). (The user account must
have permission to do so, of course.)
-v [ --vacuum-table ]
Runs VACUUM on the target table if the load fails. If target child
tables or partitions are present, they are VACUUMed, too. By
default, this feature is disabled.
-V [ --version ]
Print the Aster Loader Tool version and exit.
--verbose
Run in verbose mode.
135
Aster Client Guide
Loading Data
Aster Loader Tool
Table 6 - 16: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
Flag in Map File
Description
-w [ --password ] arg
password
Connect to the database using the specified password (as opposed to
providing a password at the prompt).
-W [--password-prompt]
Forces the Aster Loader Tool to prompt for a password before
connecting to a database. It should automatically prompt for a
password whenever the server requests password authentication. If
no password prompt is issued and the server requires password
authentication, the connection attempt will fail.
-z [ --auto-analyze ]
Runs ANALYZE on the table or tables after loading the data. By
default, this is disabled. If data was loaded to child tables via
autopartitioning, they will be analyzed, as well.
Warning: Analyzing a columnar table may be slow if there are many
columns. To improve the speed of statistics collection, execute a
separate ANALYZE command after the load that only processes the
columns involved in query row filters or grouping.
Exit Status
This table lists the possible return values upon exiting a data load:
Table 6 - 17: The Aster Loader Tool exit values
Exit
Value
Description
0
The Aster Loader Tool terminated successfully.
2
An error internal to the Aster Loader Tool was detected.
3
Another error was detected.
4
Failed to establish a connection to Aster Database.
5
An error was detected while communicating with Aster Database.
6
Malformed input data detected.
Loading Data with the Loader Tool
Connecting
The Aster Loader Tool is a regular Aster Database client application. In order to connect to an
Aster Database, you need to specify the host name or IP address of the queen node via the
command line option -h. If the connection could not be made for any reason (e.g.,
insufficient privileges, server is not running on the targeted host, etc.), the Aster Loader Tool
will return an error and terminate.
Procedure:
To load data with the Aster Loader Tool, do this:
Aster Client Guide
136
Loading Data
Aster Loader Tool
1
Install the Aster Loader Tool on your data staging machine. (See “Installing the Loader
Tool” on page 32.)
2
If you have not already added one or more loader nodes to your Aster Database cluster,
create them now in the Admin>Cluster Management>Nodes tab in the AMC.
3
Prepare the file or files that contain the data you wish to load:
a
For hints on file formatting, see the descriptions of the --csv, --delimiter, and
--null arguments, below. Use a consistent newline character in your input file(s)!
b
Determine your mapping of the input file’s field values to the columns of your target
table. See the description of the --columns argument, below.
c
Determine any special parsing hints you need to provide to the loader routing. See the
descriptions of the --escape, --quote, and --data-prefix arguments, below.
4
Place your data input file(s) on the data staging machine.
5
Figure out how you want to handle records that fail to load. See the section “Error
Logging” on page 145. Teradata Aster recommends that you create an error logging table
that will receive rows that fail to load. Be prepared to query the
nc_all_errorlogging_stats table for statistics about your load attempt.
6
Figure out which advanced options of the Aster Loader Tool you will use:
7
a
Do you need to load from multiple files or insert into multiple tables? If so, see the
description of the --map-file argument and read the section, “Troubleshoot
Loading” on page 147.
b
Do you need to automatically partition data when loading parent child tables with
inheritance? Use the --auto-partition argument and read the section, “Loading
Parent Child Tables with Inheritance” on page 144
c
If you need to run an SQL script before and/or after the data is loaded, read the
descriptions of the --begin-script and --end-script arguments, below.
Load your data by running the Aster Loader Tool on your input file(s). See “Argument
Flags” on page 130 for a list of the command-line options. The command you type will be
similar to:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" customers
input_data.txt
8
Check the results of your load attempt by querying the statistics tables,
nc_all_errorlogging_stats and nc_user_errorlogging_stats.
9
Check your error logging table(s) for rows that failed to load. Be aware that if the load
failed entirely (for example, if the number of errors exceeded the --el-limit), then no
new rows will appear in your target table, and no error rows will appear in the error
logging table.
10 If you wish to import the failed rows, find each row of input data that failed, edit it to fix it,
and combine these fixed rows into a new input file. Re-run the Aster Loader Tool on the
new input file.
137
Aster Client Guide
Loading Data
Aster Loader Tool
Removing NULLs from data on Linux
The option --preprocess-script for Aster Loader Tool on Linux executes a shell script
against the source file, and uses the output of the script as the new source file for loading. You
can use this feature along with a provided script remove-null.sh to remove NULL(0x00)
values from data to be loaded using Aster Loader Tool on Linux.
To install and use the shell script:
1
Copy the script remove_null.sh from the Aster Client package to the client machine.
The script can be found in the directory:
•
For 64-bit Linux: stage/home/beehive/clients-linux64/shell-scripts/
remove-null.sh
•
For 32-bit Linux: stage/home/beehive/clients-linux32/shell-scripts/
remove-null.sh
2
Ensure that the directory where remove-null.sh is located is in your PATH.
3
Set the file permissions on the script to make it executable:
# chmod +x remove-null.sh
4
Use this syntax when invoking Aster Loader Tool:
$ ./ncluster_loader -h 10.0.0.10 -U beehive -w beehive -d beehive
--preprocess-script=remove_null.sh target_table source_file.csv
Loading from Multiple Files Using a Map File
With the Aster Loader Tool you can load from multiple data files and insert into multiple
tables in a single invocation by passing the --map-file or -m option. We refer to this as
loading with a map file. All tables are loaded in a single database transaction.
When loading a very large amount of data, you may choose to created multiple map files that
each load their data files using a different loader. This can help speed up the process of loading
a large amount of data.
Procedure:
Aster Client Guide
1
Prepare your data files for import. Each file should be formatted as usual for the Aster
Loader Tool. All the files you submit in a single running of the Aster Loader Tool must be
in the same format.
2
You can optionally pass connection parameters in the map file. Any connection
parameters specified on the command line supercede those in the map file. The
connection parameters you can specify are:
•
dbname - the name of the database
•
username - the name of the Aster Database user
•
password - the password of the Aster Database user
•
loader - the hostname or IP address of the Aster Loader node
•
force-loader - same as the -f command line option. Instructs the Aster Loader Tool
to use the loader node specified with the loader parameter even if the IP address
provided is not known to Aster Database. Note that if this option is specified, the Aster
138
Loading Data
Aster Loader Tool
Loader Tool will only try that single IP address and return an error status if the
connection fails for any reason.
•
timeout - same as the -t command line option. Specifies the timeout value in seconds
for Aster Database connection attempts. Default is 30 seconds.
3
Prepare your map file. The map file is a text file containing a set of logical text blocks, each
surrounded by curly braces. Each block represents a file or directory to be loaded. The
format is like this:
{
"dbname" : "beehive",
"username" : "beehive",
"password" : "beehive_pwd",
"loader" : "141.206.66.28",
"force-loader" : true,
"timeout" : 5,
"loadconfig" :
[
{
"table" : "schema1.targettable1",
"file" : "data/insert1.txt",
"errorlogging" : { "enabled" : true }
},
{
"table" : "schema1.targettable1",
"file" : "data/insert2.txt",
"begin-script" : "input/mapfile/begin-script.sql",
"end-script" : "input/mapfile/end-script.sql",
"errorlogging" :
{
"enabled" : true,
"discard-errors" : true
"label" : "insert2_log",
"schema" : "nc_system",
"table" : "nc_errortable_part"
}
]
}
In the above example, we assume the current directory (from which we invoke
ncluster_loader) contains a subdirectory, data, which has two files, insert1.txt and
insert2.txt, and we load these both into table targettable1 in schema1. Error
logging is turned on. For the second table, additional error logging parameters are
supplied to log to a system table, label each row and skip errors.
Each block in the map file must contain the following required parameter flags and their
values:
•
table specifies the name of the target table. Typically you will schema-qualify the table
name as shown in the example above. You may omit the schema, in which case the
user’s schema search path determines the schema. If your table name is case sensitive,
you must surround the table name with backslash-escaped double quote marks (\"),
like this:
"table" : "schema1.\"TargetTable\"",
•
file specifies the name of the file or directory to be loaded. See filename in “Syntax”
on page 129.
139
Aster Client Guide
Loading Data
Aster Loader Tool
•
begin-script and end-script specify scripts to run either before or after the
loading of the file. Both are optional. Each map file entry can have a separate beginscript and end-script. For each map file entry, ncluster_loader will run the
begin-script, load the data from the file, then run the end-script.
The begin-script and end-script require each statement to be on a single line. Do
not include commands that begin or end the transaction in the script files. Here is an
example of a valid script file:
DROP table IF EXISTS foo;
CREATE table foo (id int, sometext varchar(40)) DISTRIBUTE BY HASH
(id);
•
errorlogging introduces a block that specifies how to handle malformed rows in the
input file. Inside the errorlogging block (enclosed in curly braces) you pass the
enabled parameter and, optionally, the parameters discard-errors, label, limit,
schema, and table. These parameters correspond, respectively, to the command-line
flags, --el-enabled, --el-discard-errors, --el-label, --el-limit, -schema, --el-table.
Each block in the map file can contain the optional parameter flags listed in the middle
column of the table, “Argument Flags” on page 130.
4
Run the Aster Loader Tool, passing the --map-file or -m option and the name of your
map file. Pass additional command line flags as needed, observing the rules set forth in the
preceding paragraph.
Rules for Passing Arguments in a Map File
Observe the following rules when you pass loader arguments in a map file:
1
Each block in a map file can contain the optional parameter flags listed in the middle
column of the table, “Argument Flags” on page 130. Observe all rules spelled out there.
2
If you wish to use these flags, you must pass them inside the map file, and you cannot use
their command-line equivalents.
3
Those flags that have no value listed in the middle column can also be used when you’re
loading with a map file, but you must pass them at the command line, instead.
Example Map File
The example file reproduced here shows various valid combinations of loading parameters.
Note that the third block in the example (the one inserting into “test3”) instructs the Aster
Loader Tool to load all the files it finds in directory “my_data_dir”.
{
"dbname" : "beehive",
"username" : "beeuser",
"password" : "jw8s0F4",
"loader" : "10.51.6.240",
"force-loader" : true,
"timeout" : 5,
"loadconfig" :
[
{
"table" : "test1",
Aster Client Guide
140
Loading Data
Aster Loader Tool
"file" : "data/insert.text",
"errorlogging" : { "enabled" : true }
},
{
"table" : "test2",
"file" : "data/insert.text.n",
"errorlogging" : { "enabled" : true }
},
{
"table" : "test3",
"file" : "my_data_dir/",
"errorlogging" : { "enabled" : true }
},
{
"table" : "test6",
"file" : "data/insert.text.incorrect",
"errorlogging" :
{
"enabled" : true,
"label" : "vm_test_12-test6"
}
},
{
"table" : "test12",
"file" : "data/insert.text.incorrect",
"errorlogging" :
{
"enabled" : true,
"label" : "vm_test_12-test12",
"discard-errors" : true
}
},
{
"table" : "test13",
"file" : "data/insert.text.incorrect.pk",
"prefix" : "0",
"columns" : "a,b",
"errorlogging" :
{
"enabled" : true,
"label" : "vm_test_12-test13",
"limit" : 100000,
"schema" : "public",
"table" : "nc_errortable_part"
}
}
]
}
Examples
Simple Loading Example
Assume the following:
141
•
File to be loaded: 2010MarchSales_data.txt (assume current working directory)
•
Number of rows: 1000 rows encoded in text format
Aster Client Guide
Loading Data
Aster Loader Tool
•
Delimiter: delimited by character "~"
•
Password: beehive
•
Destination table: sales_fact
•
Destination Aster Database: IP address 10.50.25.100
•
Username: assume default
•
Database: assume default
This is what you would type:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" sales_fact
2010MarchSales_data.txt
After successful completion, you will see the following output indicating 1000 rows loaded:
1000 tuples were successfully loaded into table customers.
Loading from STDIN Example
To load from STDIN, you simply omit the file name/directory name argument. (When
loading from a data file, the last argument to ncluster_loader is the data file name.) For
example, to import our UK sales data and replace the product name “NappyPlus” with
“DiaperPlus” we could use sed and ncluster_loader together like so:
$ sed s/NappyPlus/DiaperPlus/ 2010_UK_sales.csv | ./ncluster_loader -U
mjones -w st4g0l33 sales_fact -c -l 10.51.50.240
Loading from STDIN lets you pipe your data through useful tools like sed, the text stream
editor. Here’s an example of sed at work, ensuring that backslash escape sequences are
properly formatted in the input data before Aster Loader sees the data.
Here’s some sample data:
# cat sampleData-3.tsv
5
How often do back-slash characters ('\') appear in your data?
6
And how often do you think they actually disappear: 1 \ ? 2 \ ? 3 \ ?
7
\W\a\y \t\o\o \o\f\t\e\n\! \! \!
Here’s how we run Aster Loader, piping its input data through sed:
$ cat sampleData-3.tsv \
| sed -e 's_\\_\\\\_g' \
| ncluster_loader -h $QUEEN_IP -d my_db -U beehive -w beehive testo /
dev/stdin
Loading tuples using node '192.168.28.100'.
3 tuples were successfully loaded into table 'test'.
Here are the result rows:
$ act -h $SYSMAN_IP -d my_db -U beehive -w beehive -c 'SELECT * FROM testo ORDER BY id;'
id |
string
----+--------------------------------------------------------------------1 | This is just a line.
5 | How often do back-slash characters ('\') appear in your data?
6 | And how often do you think they actually disappear: 1 \? 2 \? 3 \?
7 | \W\a\y \t\o\o \o\f\t\e\n\! \! \!
(4 rows)
Aster Client Guide
142
Loading Data
Aster Loader Tool
Example with Error Logging
Use the same assumptions as in the previous example, and assume we will log malformed
rows (that is, rows that the loader cannot interpret and therefore cannot load) to a table called
“2010MarchSales_error_table,” tagging each error row with the label
“2010MarchSalesErr”. At the end of the load attempt, the error data will also be copied to
the file, /home/ccrisp/2010MarchSales_error.txt. We’ll set a limit of 100 error rows; if
more than 100 errors are encountered, the load will be cancelled.
To do this:
1
Create the custom error logging table: Run ACT as a user with table creation rights (for
example, a user with the catalog_admin role) and type:
CREATE TABLE 2010MarchSales_error_table () INHERITS
(nc_errortable_part);
2
Exit ACT, return to the command line, and type:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" --el-enabled
--el-label 2010MarchSalesErr --el-limit 100 --el-table
2010MarchSales_error_table --el-errfile /home/ccrisp/
2010MarchSales_error.txt sales_fact 2010MarchSales_data.txt
For more information on logging malformed rows, see “Error Logging” on page 145.
Hints for Successful Loading
Recommended Character Set Is UTF-8
The default character set for Aster databases is UTF-8, and the Aster team recommends that
you load only UTF-8 formatted data when loading to char, varchar, and text columns.
For the tools you use to prepare files and to connect to Aster, make sure you have set the
default character set to UTF-8. This is particularly important if you are loading data from a
Windows-based machine. For example, if you will use an SSH client (e.g., putty) to run
ncluster_loader, make sure you set the SSH client’s default character set to UTF-8.
We recommend that, prior to loading, you convert your text files to UTF-8. For example, if
you’re a Notepad++ user, you can use the command, “Convert to UTF8 without BOM.”
Newline Character
Make sure your data file uses a consistent character to represent newlines. If the file uses \r\n
for newlines, then it should not also use \n for newlines, and vice versa. If your file contains
both UNIX-style \n newlines and Windows-style \r\n newlines, then you must clean the file
before you try to load it. The UNIX command, dos2unix, can be useful for doing this.
Multiple Loader Nodes
The Aster Loader Tool supports the use of many Aster Loader nodes. For most loading tasks,
the queen is sufficient to handle all loading, but for high volume loading, you can add
dedicated loader nodes to your cluster.
To use a loader node, you invoke one or more ncluster_loader instances that will load through
that loader node. You may run many ncluster_loader sessions in parallel against one loader
143
Aster Client Guide
Loading Data
Aster Loader Tool
node, and you may use many loader nodes in parallel (with each node handling loads from a
number of ncluster_loader instances).
To do this, you invoke each ncluster_loader instance with the -l (and optionally -f )
argument to specify the loader node. The required flags are:
•
as always, the --hostname option (-h) provides the queen IP address;
•
the --loader flag (-l) provides the IP address of the desired loader node; and
•
Optionally, the --force-loader flag (-f ) forces the use of the desired loader node.
Loading Parent Child Tables with Inheritance
The --auto-partition option is retained in order to support parent/child tables created
with inheritance. It is not used when working with parent/child tables created with
autopartitioning. Using --auto-partition instructs the Aster Loader Tool to automatically
send each row to the right child table during loading. Each row is directed to a table according
to the check constraints you have set up on the child tables.
For example, if you partition your data into daily child tables based on the contents of a
timestamp column, each ultimate child table in your schema will have a CHECK constraint
that specifies what value of timestamp may be loaded into that child table. When you load
data, the autopartitioning feature will route each row to the appropriate child table, based on
its timestamp value.
Use autopartitioning like this:
1
Set up the parent-child table schema in your database. On each ultimate child table, write
a CHECK constraint that specifies what data may be loaded into that child table.
Warning! Aster Database does not detect overlapping constraints on peer child tables. As a result, the correct placement of a row during loading can be indeterminate.
Workaround: Take care that the constraints you define do not create overlapping logical partitions. A simple mistake
would be to set up range constraints like this:
CHECK ( ymdh BETWEEN '2005-07-01' AND '2005-08-01' );
CHECK ( ymdh BETWEEN '2005-08-01' AND '2005-09-01' );
In this example, it is not clear in which partition the ymdh value '2005-08-01' resides.
2
3
Prepare your data for loading:
a
Your data input file can contain data values that will end up in many different child
tables.
b
To handle rows that do not fit your partitioning scheme, you can rely on the standard
error logging feature of the Aster Loader Tool (see “Error Logging” on page 145) or
create a check constraint that will catch rows that you do not want to include in your
partitions.
Run the Aster Loader Tool with the -a or --auto-partition flag.
Detecting UNIQUE and PRIMARY KEY Violations Before Loading
Detecting UNIQUE and PRIMARY KEY violations in the data to be loaded is not always
straightforward. In many cases the source is not a database you can easily run a query on to
Aster Client Guide
144
Loading Data
Aster Loader Tool
detect non-unique keys. Some techniques you can use to detect these conditions in your
source data:
•
Build a version of the target table in the target database without a UNIQUE or PRIMARY
KEY constraint, load the data, then run a “detect duplicates” query to find the problematic
rows/keys. In some cases only loading a sample of the data is sufficient to provide enough
clues to find and fix the problem in the source data.
•
An alternative step (using an ETL tool) would be to use this “keyless” version of the target
table as a staging/temp table, which would load then check for issues like duplicate keys
and dump them to a second error table. If no issues are found, then transfer the data to the
final destination table.
•
If the source is a database, then run the “detect duplicates” query there.
Using COPY with Columnar Tables
A loading operation using the Aster Loader Tool, COPY, or INSERT can be expensive when
the following conditions exist:
•
the target table uses columnar storage, AND
•
the target table has many logical partitions, AND
•
the loaded data matches many different logical partitions.
In this case, the memory allocated to perform the load may be as large as the amount of source
data. To avoid high memory requirements, it is best to divide a large load into batches. There
are two alternative approaches:
•
Ensure that each batch only loads a small number of logical partitions in the columnar
table. For example, when inserting data into a columnar table with weekly partitions, each
batch may insert data for a single month.
•
Ensure that the size of each batch is a small fraction of system memory available at the
worker nodes. This should only be done if the data being loaded into the columnar table
has a mixture of records matching many different logical partitions. As an example,
suppose that a year's worth of data is being loaded into a columnar fact table with weekly
partitions, and the cluster has four physical worker nodes with 100GB of system memory
in each node. Loading data in 40GB batches will use 10GB memory per physical worker,
which is 10% of the overall available memory.
Error Logging
Enabling Load Error Logging
Use the --el-enabled flag (or the errorlogging flag inside a map file) to run the Aster
Loader Tool in a mode in which it tolerates poorly formatted input rows and logs each bad
row to a table. This differs from Aster Loader’s normal mode of running:
145
•
Running normally, Aster Loader aborts the load immediately if it encounters a bad input
row, and it does not log the malformed input row to a table.
•
Running in --el-enabled mode, Aster Loader logs each malformed input row (that is,
any row it cannot interpret for loading or cannot load due to datatype mismatch or check
Aster Client Guide
Loading Data
Aster Loader Tool
constraint violation) to an error table and continues to load the remaining rows in the
load job. We refer to this as “error logging.”
The --el-enabled flag is a master flag that operates with a set of sub-flags (--el-discarderrors, --el-errfile, --el-label, --el-limit, --el-schema, and --el-table) that
fine-tune your handling of malformed rows. To use any of the sub-flags, you must first have
specified the --el-enabled flag. If you’re using a map file, the syntax is different. The master
flag is errorlogging, and the sub-flags are discard-errors, errfile, label, limit,
schema, and table.
The --el-discard-errors flag discards all malformed rows, the --el-label tags failed
row data, the --el-limit flag sets a maximum number of allowed failed rows for the job, and
the --el-table flag specifies a custom error logging table. See “Argument Flags” on page 130
for explanations. See “Example with Error Logging” on page 143 for example usage.
To perform error logging, the Aster Loader Tool relies on the error handling features of the
Aster Database COPY command in SQL.
If data being loaded will cause duplicate values of a UNIQUE or PRIMARY KEY constraint on
the target table, it is considered an error. This particular error cannot be handled by error
logging, so the loading transaction will be aborted if any record causes a unique or primary
key constraint violation, even when error logging is enabled.
What Errors Get Logged
Error logging is more general than just handling malformed rows. Essentially, any error
related to an individual row can be recorded by error logging and the load can continue,
except for the special case of UNIQUE/PRIMARY KEY violations. If there is such a violation,
the loader will abort the load. Also, any error not related to an individual row (such as
insufficient privileges) will abort the load operation.
Some examples of what errors can be caught by error logging:
•
NOT NULL violations
•
Wrong number of columns
•
Value format errors (e.g. specifying "asdf " for an integer column)
•
No matching child partition
•
CHECK constraint violations
•
Field length overflow (e.g., specifying "asdf " for a varchar(2) column)
•
Text/CSV format errors (e.g. misplaced carriage return, unterminated quoted string, etc.)
Load Error Logging Tables
By default, Aster Database logs error rows to one of its default error logging tables, based on
type of the target table: malformed rows for distributed tables go into table
nc_errortable_part table, and malformed rows for replicated tables go into the
nc_errortable_repl table. Alternatively, you can create your own error logging tables. You
can perform SELECT, UPDATE, and DELETE operations on the error log tables, and you can
inherit from the default error logging tables to create your own error logging tables.
Aster Client Guide
146
Loading Data
Troubleshoot Loading
Statistics About Logged Errors
To see the number of rows that loaded or failed to load, query the load error statistics tables,
nc_all_errorlogging_stats and nc_user_errorlogging_stats. Each loading
command generates a row in the tables. For a given transaction, the totalcount, goodcount,
and malformedcount columns show the total number of rows you tried to load, the number
of rows that successfully loaded, and the number of rows not loaded, respectively.
Troubleshoot Loading
Running Multiple Loaders Concurrently
When running multiple ncluster_loader instances concurrently, eventually Aster Database will
throw errors due to physical limitations imposed by system resources. Due to this, you should
not run more than ten ncluster_loader instances concurrently.
Load Stalls Upon Cancellation or Failure Encountered During a Load
The Aster Loader Tool can take a lot of time to finish after a failure is detected. Likewise, if a
Control+C is issued, the loader can take several minutes to cancel completely.
Load Fails on UNIQUE or PRIMARY KEY Violation
A UNIQUE or PRIMARY KEY violation in the data being loaded will always cause the load to
abort. So if your load is failing, check your data and the schema of the table you are loading
into for these violations.
Invalid Input Syntax Error
If you encounter an error of type:
"Error: Invalid input syntax on partition key for..."
then there may be a problem with the character encoding of your input data. Please check the
encoding of your input data and the datatypes of your destination columns, fix any
mismatches you find, and retry the load.
Single Quote Character Must be Escaped When Using the -q Option
If you encounter an error like:
"ERROR:
extra data after last expected column"
you may have encountered a problem with command line parsing. If you used a single quote
delimiter in your CSV input file, and the input data includes a comma within one of the text
strings to be loaded, you will see this error.
To correct this, you will need to escape the single quote character when specifying it using the
-q option on the command line as -q"\'".
Here is an example:
147
Aster Client Guide
Loading Data
Troubleshoot Loading
Assume the file test.txt, which we are attempting to load into table table1 includes the
following input data row:
1, 25, "Some text with a, comma in it."
The following will not load, returning the "Error: extra data after last expected
column" error:
ncluster_loader -l 10.50.129.103 -U db_superuser -w db_superuser -d
beehive -c -q"'" table1 test.txt
The following will load:
ncluster_loader -l 10.50.129.103 -U db_superuser -w db_superuser -d
beehive -c -q"\'" table1 test.txt
Using the -C Option With Uppercase or Special Characters
When using the -C option where the column list has any uppercase or special characters, you
must put the column list in double quotes. On Windows, this requires escaping the double
quotes:
Example on Linux:
./ncluster_loader -h 10.51.150.100 -l 10.51.150.240 -w db_superuser -U
db_superuser -c -D , -C '"ID","NAME","$value"' test2 test2.csv
On Windows, when using the -C option where the column list has any uppercase or special
characters, you must put the column list in escaped double quotes.
Example on Windows:
ncluster_loader.exe -h 10.51.150.100 -w db_superuser -U db_superuser -c
-C '\"ID\",\"NAME\",\"$value\"' test2 test2.csv
Uppercase Characters are Passed as Lowercase if not Escape Quoted
If your data contains a table or schema with a capital letter in its name, you must escape quote
the table or schema name. If you do not do this, all lower case letters are assumed. Because of
this, the data will load into a schema or table having the same name all in lower case, if it
exists.
For example, if you specify the table Test.Try, Test.try, test.Try or test.try, the data will always
be loaded into test.try unless you specify escape characters with enclosed quotes for the table
name parameter to ncluster_loader as in this example:
# /home/beehive/clients/ncluster_loader -U db_superuser -w db_superuser
-c '\"Test\".\"Try\"' file001
Loading tuples using node 'localhost'.
4 tuples were successfully loaded into table '"Test"."Try"'.
This example loads the data in the expected table (Test.Try).
Issues Using Escape Characters
A few notes on using escape characters may help troubleshoot any issues when using them:
Aster Client Guide
148
Loading Data
Troubleshoot Loading
1
The escape character takes effect only if it is inside a quoted string. Consider this example:
He said \"Who is that?\" when she entered the room.
The first \ is treated as a regular character, and not as an escape character, because it is not
inside a quoted string, and the first " is treated as the beginning of a string. Because there
is an opening quote but no corresponding closing quote on the line, the parser will either
complain, or will scan subsequent lines until it finds a quote that can be treated as a quote
(rather than as a literal). The parser will treat everything up to that point as one data field
with embedded newlines.
To correct this, enclose the entire string in quotes as in this example:
"He said \"Who is that?\" when she entered the room."
2
149
If an explicit quote character (-q) is specified, and no explicit escape character (-e) is
specified, then the escape character will be set to be the same as the quote character
specified.
Aster Client Guide
CHAPTER 7
Exporting Data
Aster Database Export Tool
The Aster Database Export Tool, ncluster_export, is a command-line application for bulkexporting data from Aster Database to a file or to STDOUT. It provides an alternative to the
SQL COPY TO statement. Each invocation of the Aster Database Export Tool is equivalent to
a single invocation of the COPY TO statement. Each invocation is a single database transaction
that exports the contents of the specified table into the specified file or STDOUT.
Related Topic:
“Loading Data” on page 124
Synopsis
To run the Aster Database Export Tool, you type:
$ ./ncluster_export [arguments] [schemaname.]tablename [ filename ]
You can also pipe the results through a standard UNIX command such as gzip. For example:
$ ./ncluster_export -U mjones -w st4g0l33 -h 10.50.52.100 -d mydb mytable
| gzip -c > mytable.gz
In the synopsis above, the arguments are:
•
schemaname is the optional name of the source table’s schema. If no schema name is
provided, Aster Database will search the schemas listed in your current schema search
path.
•
tablename is the name of the source table. See “Case-Sensitive Handling for Table
Names” on page 151 if you wish to have Aster Database evaluate table names in a casesensitive manner. To export from multiple tables, see “Exporting from Multiple Tables” on
page 151.
•
filename indicates the name of the file that will receive the exported data. If you don’t
supply a file or directory name argument, Aster Database Export assumes you want to
export to STDOUT. If the filename contains spaces, make sure you enclose it in double
quotes.
•
arguments are the command-line flags that control how the exporter runs. The flags are
explained in “Argument Flags” on page 152, or you can display the help by typing:
Aster Client Guide
150
Exporting Data
Aster Database Export Tool
$ ncluster_export -?
Case-Sensitive Handling for Table Names
To force Aster Database Export to treat your table and schema names in a case-sensitive
manner, you must surround the schema and table names in double quote marks when you
invoke ncluster_export. To pass a double quote mark to Aster Database Export, you must
prefix it with the escape character, which is a backslash. That means you type a double quote
mark like so: \"
To do this, surround the entire schemaname.tablename string in a single pair of double
quotation marks.
For example, assuming you want to export to table "t" in schema "S", then you would invoke
Aster Database Export in this form:
ncluster_export.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"S\".\"t\"
mydata.txt
Exporting from Multiple Tables
With ncluster_export you can export from multiple tables in a single invocation by creating a
map file of tables to be exported and passing the --map-file or -m option with the filename.
All tables are exported within a single database transaction. To export from multiple tables,
perform the following steps:
1
Prepare your map file. The map file is a text file containing a set of logical text blocks, each
surrounded by curly braces. Each block represents a table to be exported. The format is as
follows:
{
"exportconfig" :
[
{
"table" : "exporttable1",
"fileprefix" : "path/to/dir/file"
},
{
"table" : "exporttable2",
"fileprefix" : "path/to/dir/prefix",
"maxtuplesperfile" : 100,
"columns" : "a,b"
}
]
}
Each block must contain the following required parameters:
•
“table” specifies the name of the table to export.
•
“fileprefix” specifies the output file / file prefix to be used.
Each block can contain the following optional parameters:
151
•
“maxtuplesperfile” can be used to specify how many records will be put in each output
file. This allows outputting multiple files for a single table, with the specified number
of records in each.
•
“columns” is a comma-separated list of the columns to be exported.
Aster Client Guide
Exporting Data
Aster Database Export Tool
2
Run ncluster_export, passing the --map-file or -m option and the name of your map
file.
$ ./ncluster_export -U mjones -w st4g0l33 -h 10.50.52.100 -d mydb -m
"path/to/dir/mapfile"
Argument Flags
Aster Database Export accepts the following flags.
Table 7 - 1: Aster Database Export Flags
Aster Client Guide
Column
Type
-? [ --help ]
Shows the online help.
-B [ --begin-script ] arg
Specifies the qualified path of the file containing SQL
commands that should be executed when the
transaction starts, i.e. immediately after the begin
command is issued to Aster Database. NOTE: Datareturning statements such as SELECT are not allowed.
Each command in the begin script should be on a
separate line. Do not include BEGIN or END
statements.
-c [ --csv ]
Exports the output files in CSV format (the default is to
use Text format).
-C [ --columns ] arg
An optional comma-separated list of columns to be
exported from the source table (the default is to export
all columns).
-d [ --dbname ] arg
Specifies the name of the database to connect to (the
default is 'beehive').
-D [ --delimiter ] arg
Specifies the delimiter character to use when exporting
data (must be a string that represents a valid single
character, such as 'd' or '\n'). The default is a tab
character ('\t') in text mode, a comma (',') in CSV
mode.
-E [ --end-script ] arg
Specifies the qualified path of the file containing SQL
commands that should be executed when the
transaction finishes, i.e. immediately before the end
command is issued to Aster Database. NOTE: Datareturning statements such as SELECT are not allowed.
Each command in the end script should be on a separate
line. Do not include BEGIN or END statements.
-e [ --escape ] arg
Specifies the escape character for CSV output (must be a
string that represents a valid single character, such as 'd'
or '\n'). The default is the quote value - double-quote by
default. This option is only valid when CSV mode is
specified.
152
Exporting Data
Aster Database Export Tool
Table 7 - 1: Aster Database Export Flags (continued)
Column
Type
-f [ --force-loader ]
Instructs ncluster_export to use the loader node
specified with the '-l/--loader' option even if the IP
address provided is not known to Aster Database.
NOTE: ncluster_export will only try that single IP
address and return an error status if the connection fails
for any reason.
filename (Not a flag; you pass
the file or directory name itself!)
filename indicates the name of the file that will receive
-h [ --hostname ] arg
Specifies the hostname or IP address of the machine on
which Aster Database is running. Default is 'localhost'.
-l [ --loader ] arg
Preferred loader IP address. If a value is provided,
ncluster_export will try to export through this IP
address before trying any other loader node. NOTE:
ncluster_export expects this IP address to be known by
Aster Database and will simply ignore the address
provided if this is not the case. To change this behavior
one can use the '-f/--force-loader' option.
-m [ --map-file ] arg
Specifies the name of the file containing mappings of
the tables to be exported. This option allows export of
multiple tables within the same transaction. For details
about the format used for the map file, see “Exporting
from Multiple Tables” on page 151.
-M [ --max-tuples-perfile ] arg
Specifies how many records are exported to a single file.
If the total number of records exceeds this number
multiple output files with the suffix '_N' are produced,
where N is an integer.
-n [ --null ] arg
Specifies a string that represents a NULL value. The
default is '\N' in text mode and an empty value with no
quotes in CSV mode. See also “--null-backslashescapes” on page 153.
--null-backslash-escapes
Indicates that backslashes in the '-n/--null' flag should be
treated as special characters, which will be processed
according to the default rules for escaped strings.
the exported data. If you don’t supply a file or directory
name argument, Aster Database Export assumes you
want to export to STDOUT. If the filename contains
spaces, make sure you enclose it in double quotes.
You should use this flag whenever the null string
contains a backslash, to ensure compatibility with future
versions of Aster Database.
-p [ --port ] arg
153
Specifies the TCP port on which Aster Database is
listening for connections. The default is port 2406.
Aster Client Guide
Exporting Data
Aster Database Export Tool
Table 7 - 1: Aster Database Export Flags (continued)
Column
Type
-q [ --quote ] arg
Specifies the quote character for CSV output (must be a
string that represents a valid single character, such as 'd'
or '\n'). The default is the double-quote. This option is
only valid if CSV mode is specified.
schemaname (Not a flag; you pass
schemaname is the optional name of the source table’s
schema. If no schema name is provided, Aster Database
will search the schemas listed in your current schema
search path.
the schema name itself!)
Aster Client Guide
-t [ --timeout ] arg
Specifies the timeout value (in seconds) to use when
connecting to Aster Database (default is 30).
tablename (Not a flag; you pass
the table name itself!)
tablename is the name of the source table. See “CaseSensitive Handling for Table Names” on page 151 if you
wish to have Aster Database evaluate table names in a
case-sensitive manner. To export from multiple tables,
see “Exporting from Multiple Tables” on page 151.
-U [ --username ] arg
Connects to the database with the specified username
instead of the default ('beehive'). (You must have
permission to do so, of course.)
-V [ --version ]
Prints the version of ncluster_export and exit.
-w [ --password ] arg
Connects to the database with the specified password
(as opposed to providing a password at the prompt).
-W [ --password-prompt ]
Forces ncluster_export to prompt for a password before
connection to the database.
154
Exporting Data
Aster Database Export Tool
155
Aster Client Guide
Appendix 1 Encoding Support
This appendix lists Aster Client encoding support. Note that data being loaded to Aster
Database should use a UTF-8 character set with one of the supported encodings. The
encoding support for JDBC and .NET Data Provider are Unicode (UTF-16). This is because
the character string data type in JAVA and C# are encoded in Unicode (UTF-16).
Aster ODBC Encoding Support
This table lists the encoding support in Aster Clients, which are based on the Simba 9.0 SDK.
Note that the Aster Clients on MacOS use the Simba 8.1.0 SDK, and as such they differ from
the others in that they do not include support for ENC_MSWIN31_KOREAN (Korean
character set) .
Table 1 - 1: Encoding Support in Aster ODBC
Simba Encoding Name
Description
Source
ENC_BIG5
Multibyte character set. Chinese
characters for Taiwanese.
http://www.iana.org/assignments/charset-reg/
Big5-HKSCS
ENC_CP437_US
IBM 437
IBM NLS RM Vol 2 SE09-8002-01, March 1990
ENC_CP850
IBM 850
IBM NLS RM Vol 2 SE09-8002-01, March 1990
ENC_CP1252
Windows Unicode Latin (English,
Spanish, Germanic/Scandinavian)
character set.
ENC_EUC_CNS
Simplified Chinese multibyte
https://en.wikipedia.org/wiki/Extended_Unix_Code
character set for UNIX. Based on ISO
2022.
ENC_EUC_JP
Japanese multibyte character set for
UNIX. Based on ISO 2022.
ENC_EUC_KR
Korean multibyte character set for
UNIX. Based on ISO 2022.
Aster Client Guide
https://en.wikipedia.org/wiki/Extended_Unix_Code
http://tools.ietf.org/html/rfc1557
156
Aster ODBC Encoding Support
Table 1 - 1: Encoding Support in Aster ODBC (continued)
Simba Encoding Name
Description
Source
ENC_GB_2312_80
Simplified Chinese character set.
Official character set of People’s
Republic of China.
ISO-IR: International Register of Escape Sequences
ENC_GBK
Chinese IT standardized Unicode
character set.
http://www.iana.org/assignments/charset-reg/GBK
ENC_HP_ROMAN8
Roman character set for use with HP
printers.
LaserJet IIP Printer User's Manual, HP part no
33471-90901, Hewlett-Packard, June 1989.
ENC_HZ_GB_2312
Printable ASCII Chinese character set. http://tools.ietf.org/html/rfc1842 ,
http://tools.ietf.org/html/rfc1843
ENC_IBM_280
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
ENC_IBM_424
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
ENC_IBM_775
HP PCL 5 Comparison Guide (P/N 5021-0329) pp
B-13, 1996.
ENC_IBM_851
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
ENC_IBM_00858
http://www.iana.org/assignments/charset-reg/
IBM00858
ENC_IBM_866
IBM NLDG Volume 2 (SE09-8002-03) August 1994.
ENC_IBM_871
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
ENC_IBM_918
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
ENC_IBM_1026
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
ENC_IBM_1140
http://www.iana.org/assignments/charset-reg/
IBM01140
ENC_IBM_1141
http://www.iana.org/assignments/charset-reg/
IBM01141
ENC_IBM_1142
http://www.iana.org/assignments/charset-reg/
IBM01142
ENC_IBM_1143
http://www.iana.org/assignments/charset-reg/
IBM01143
ENC_IBM_1144
http://www.iana.org/assignments/charset-reg/
IBM01144
ENC_IBM_1145
http://www.iana.org/assignments/charset-reg/
IBM01145
ENC_IBM_1146
http://www.iana.org/assignments/charset-reg/
IBM01146
ENC_IBM_1147
http://www.iana.org/assignments/charset-reg/
IBM01147
ENC_IBM_1148
http://www.iana.org/assignments/charset-reg/
IBM01148
ENC_IBM_1149
http://www.iana.org/assignments/charset-reg/
IBM01149
ENC_IBM_EBCDIC_8859_1
157
IBM ASCII
IBM Globalization
Aster Client Guide
Aster ODBC Encoding Support
Table 1 - 1: Encoding Support in Aster ODBC (continued)
Simba Encoding Name
Description
Source
ENC_IBM_EBCDIC_ARAB_B IBM Arabic character set
ILINGUAL
IBM Globalization
ENC_IBM_EBCDIC_DENMA IBM Nordic character set.
RK_NORWAY
IBM Globalization
ENC_IBM_EBCDIC_EASTER IBM Latin written Slavic and Central
N_EUROPE
European character set.
IBM Globalization
ENC_IBM_EBCDIC_FINLAN IBM Scandinavian character set.
D_SWEDEN
IBM Globalization
ENC_IBM_EBCDIC_FRANC
E
IBM Globalization
IBM French character set.
ENC_IBM_EBCDIC_GERMA IBM Germanic character set.
NY_AUSTRIA
IBM Globalization
ENC_IBM_EBCDIC_KATAKA IBM Katakana character set.
NA_DB
IBM Globalization
ENC_IBM_EBCDIC_SPAIN_
LATIN_AMERICA
IBM Spanish/Latin character set.
IBM Globalization
ENC_IBM_EBCDIC_UK
IBM United Kingdom character set.
IBM Globalization
ENC_IBM_EBCDIC_W_EUR IBM Western European character set. IBM Globalization
OPE
ENC_ISO_2022_CN
http://tools.ietf.org/html/rfc1922
ENC_ISO_2022_CN_EXT
http://tools.ietf.org/html/rfc1922
ENC_ISO_2022_JP
http://tools.ietf.org/html/rfc1468
ENC_ISO_2022_JP_2
http://tools.ietf.org/html/rfc1554
ENC_ISO_2022_KR
http://tools.ietf.org/html/rfc1557
ENC_ISO_646_IRV
Similar encoding to ASCII. Some
characters from ASCII removed to
make space for national variants.
http://en.wikipedia.org/wiki/ISO/IEC_646
ENC_ISO_8859_2
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_3
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_4
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_5
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_6
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_7
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_8
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_9
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_10
ISO-IR: International Register of Escape Sequences
ENC_ISO_8859_13
http://www.iana.org/assignments/charset-reg/ISO8859-13
Aster Client Guide
158
Aster ODBC Encoding Support
Table 1 - 1: Encoding Support in Aster ODBC (continued)
Simba Encoding Name
Description
Source
ENC_ISO_8859_14
http://www.iana.org/assignments/charset-reg/ISO8859-14
ENC_ISO_8859_15
http://www.iana.org/assignments/charset-reg/ISO8859-15
ENC_JIS_ENCODING
ISO 2022 escape sequences used to
shift code sets.
JIS X 0202-1991.
ENC_KO18_CYRILLIC
8-bit Russian, Bulgarian and
Ukrainian (Cyrillic) encoding.
http://tools.ietf.org/html/rfc1489
ENC_KS_C_5601
ISO-IR: International Register of Escape Sequences
ENC_LATIN1
ISO 8859 1
ISO-IR: International Register of Escape Sequences
ENC_MS_CP932
Windows Japanese character set.
Extends Shift_JIS
JIS X0201:1997. JIS X0208:1997.
ENC_MSWIN31_ARABIC
Arabic character set.
ENC_MSWIN31_BALTIC
Baltic character set.
ENC_MSWIN31_CYRILLIC
Cyrillic character set.
ENC_MSWIN31_EASTERN_
EUROPE
Latin written Slavic and Central
European character set.
ENC_MSWIN31_GREEK
Greek character set
ENC_MSWIN31_HEBREW
Hebrew character set.
ENC_MSWIN31_KOREAN
Korean character set. Not supported
on MacOS.
ENC_MSWIN31_TURKISH
Turkish character set.
ENC_MSWIN31_VIETNAME Vietnamese character set.
SE
ENC_MVS_OPEN_EDITION EBCDIC encoding used for MVS
OpenEdition OS.
http://www.redbooks.ibm.com/redbooks/pdfs/
sg244529.pdf
ENC_PC_ARABIC
ENC_PC_CANADIAN_FREN
CH
ENC_PC_CYRILLIC
ENC_PC_EASTERN_EUROP
E
ENC_PC_GREEK
ENC_PC_ICELANDIC
ENC_PC_NORDIC
ENC_PC_PORTUGUESE
ENC_PC_TURKISH
159
Aster Client Guide
Aster ODBC Encoding Support
Table 1 - 1: Encoding Support in Aster ODBC (continued)
Simba Encoding Name
Description
Source
ENC_SHIFT_JIS
Extends cHalfWidthKatana: adds
graphic characters.
JIS X0201:1997. JIS X0208:1997.
ENC_STD_MAC_ROMAN
8-bit encoding for Mac OS. Used for
English and Western (Latin)
languages.
https://en.wikipedia.org/wiki/Mac_OS_Roman
ENC_TIS_620
Thai Industrial Standards Institute
ENC_US_ASCII
ANSI X3.4-1968.
ENC_UTF8
Unicode 8-bit encoding.
http://tools.ietf.org/html/rfc3629
ENC_UTF16_BE
Big Endian Unicode 16-bit encoding.
http://tools.ietf.org/html/rfc2781
ENC_UTF16_LE
Little Endian Unicode 16-bit
encoding.
http://tools.ietf.org/html/rfc2781
ENC_UTF32_BE
Big Endian Unicode 32-bit encoding.
http://www.unicode.org/unicode/reports/tr19/
ENC_UTF32_LE
Little Endian Unicode 32-bit
encoding.
http://www.unicode.org/unicode/reports/tr19/
Aster Client Guide
160
Aster ODBC Encoding Support
161
Aster Client Guide
Index
Symbols
B
.NET Data Provider for Aster
install 30
backslash
adding to input stream with sed 142
allow backslash escapes 97
backslash escape sequences 142
begin-script argument during batch load 140
begin-script flag in batch loading 131
best practices
loading 124
bulk load 124
autopartition 144
end-script 132
from many files 138
logging bad rows during import 145
map file 138
ncluster_loader 129
bulk loading
arguments 130
flags 130
map file: supported flags 131
parameters 130
bytea
handling bytea data in ODBC 83
A
about this book 10
ACT
check version of ACT 62
command history 75
command reference 73
command-line flag reference 61
file management commands 75
installing 16
launching 59
launching on Linux or Solaris 60
launching on Mac 60
launching on Windows 59
list all databases 77
list all tables in Aster Database 76
parameter, setting with \set 74
parameters 61
parameters at SQL prompt 73
tab completion 72
transpose columns and rows in output 78
using ACT 66
ACT commands 73
command-line options 61
SQL-prompt options 73
Active Directory authentication
SSL connections and 43
AIX
configure ODBC Driver Manager 23
ANALYZE
bulk loading and 127
Aster Data, about 10
Aster Database
checking version of ACT 62
connecting, overview 80
Aster Database configuration settings
SQL behavior 97
Aster Database Export: See ncluster_export.
Aster support portal 10
auto-analyze flag in batch loading 136
auto-partition flag in batch loading 131
autopartitioning
ncluster_loader and 144
Aster Client Guide
C
certificate 41
for ODBC connection 41
character set for loading 143
character set, default 80
characters, special 80
loading 143
child table
automatically route data into 144
client 12
installing ACT client 16
recommended client settings 80
client settings, recommended 80
client software versions 62
client tools 12
client-side cursors in JDBC 93
column
transpose columns and rows 78
column alignment of query output 77
columns argument during batch load 131
command buffer 74
command history 74, 75
162
send to file 74
command reference
ACT command-line flags 61
ACT SQL prompt 73
command-line (in SQL) commands for ACT 73
command-line options for ACT 61
commands
ACT command-line flags 61
ACT commands at SQL prompt 73
configuration
queen system parameters 40
configuration settings
SQL behavior 97
connect 80
JDBC driver 85
Microsoft .NET, drivers for 104
MicroStrategy 102
ODBC driver 81
connecting 80
overview 80
through JDBC 88
conventions 9
coordinator.cfg 40
COPY
in ncluster_loader 129
copyright 10
csv flag in batch loading 131
cursor
ACT, cursors in 69
distributed 93
JDBC, cursors in 93
customer support 10
customizing SQL behavior 97
D
data
autopartition during load 144
bulk load 124
encrypting data traffic 42
data export 150
data loading 124
database
list all tables 76
list the databases in a cluster 77
database drivers 80
JDBC 85
Microsoft .NET, drivers for 104
ODBC 81
ODBC for Perl scripts 81
date
date format in bulk loading 132
date of publication 10
date-style flag in batch loading 132
163
dbname flag in batch loading 131
delimiter flag in batch loading 131
delimiters
setting in ACT 77
discard-errors flag in batch loading 132
distributed cursors 93
documentation conventions 9
documentation version and updates 10
documentation, about 9
Driver Manager
configuration 23
ODBC 23
drivers 80
JDBC 85
Microsoft .NET, drivers for 104
ODBC 81
ODBC for Perl scripts 81
E
E prefix 97
edition 10
el-enabled flag in batch loading 132
ELT 124
enable_quoted_identifiers 97
encryption 39
of query results 42
SSL for ODBC 39
end-script argument during batch load 140
end-script flag in batch loading 132
error file flag in batch loading 133
error limit in bulk loading 133
error logging 145
loading, log of errors for 145
tables to capture load errors, default 146
error logging file 133
error logging table, naming 133
errorlogging argument during batch load 140
escape character 132
allow backslash escapes 97
bulk loading and 132
ncluster_loader and 132
escape flag in batch loading 132
escape sequence
adding to input stream with sed 142
ETL 124
SSIS and 104
expanded output mode 78
export 150
Aster Database export tool 150
installing 33
JDBC driver 85
ODBC driver 81
source table, case-sensitive name 151
Aster Client Guide
tools for 150
export data
other tools 150
export tool 150
connecting through JDBC 88
cursors in 93
driver 85
how to write applications that use JDBC 99
query example 99
sample test program 95
F
FETCH_COUNT 69
FETCH_LIMIT 70
fetch-count 69
fetch-limit 70
field separator, setting in ACT 77
file
redirect command history to file 74
redirect query history to file 74
redirect query results to file 75
file argument during batch load 139
file input to ACT 75
with \i 75
with -f 62
force-loader flag for bulk loading 134
format
dates in bulk loads 132
G
get latest documentation 10
group
list all groups in Aster Database 76
H
help 10
help for ncluster_loader 134
history in ACT 75
hostname flag for bulk loading 134
I
import
from many files 138
logging bad rows during import 145
ncluster_loader 129
index
list all indexes in Aster Database 76
install
ACT client 16
Aster Database Loader Tool 32
export tool 33
TD Wallet 33
J
Java
JDBC statements 99
JDBC 85
Aster Client Guide
L
label argument during batch load 140
label for data loading errors 133
LIMIT
FETCH_LIMIT as alternative to 70
limit argument during batch load 140
limit rows returned at a time 69
limit total rows returned per query 70
line break character 143
list all tables 76
list databases command 77
load 124
autopartition 144
from many files 138
from SDTIN 142
logging bad rows during import 145
map file 138
load data 124
loader
arguments 130
Aster Database loader tool 129
destination table 129
destination table, case-sensitive name 130
flags 130
map file: supported flags 131
parallel loading with 143
parameters 130
specifying a dedicated loader node 143
loader flag for bulk loading 134
loader node 124
parallel loading with 143
loader tool 129
loading 124
arguments 130
Aster Database Export tool 150
Aster Database Loader tool 129
best practices 124
character set for loading 143
flags 130
from SDTIN 142
handling nulls when loading data 134
JDBC driver 85
map file: supported flags 131
ODBC driver 81
parameters 130
SSIS and 104
164
troubleshooting 143
troubleshooting problems encountered when loading 147
loading errors 145
log errors 145
custom error logging tables 146
loading
errors, logging 145
installing for use with Perl scripts 81
installing on AIX 23
installing on Linux 21
installing on Windows 18
SSL settings 42
output
transpose columns and rows 78
M
P
malformed rows during load 145
map file 138
supported flags 131
map file for ncluster_loader 138
map-file flag for bulk loading 134
memory usage in ACT: limit with FETCH_COUNT paging of
query results 69
Microsoft .NET, drivers for 104
Microsoft SSIS 104
MicroStrategy, connections for 102
parameter
setting queen system parameters 40
settings for ACT 74
partitioning
autopartition during load 144
performance tuning
improve ACT performance with FETCH_LIMIT 70
limit ACT memory use with FETCH_COUNT 69
server-side cursors in ACT 69
Perl scripts, ODBC driver for 81
pipe to ncluster_loader 142
pivot column and row output 78
port flag for bulk loading 135
portal 10
preferences
SQL settings 97
prefix argument during batch load 135
PreparedStatement in JDBC 99
pre-process data with sed 142
N
nc_errortable_part 146
nc_errortable_repl 146
ncluster_export 150
source table, case-sensitive name 151
ncluster_loader 129
arguments 130
character set for loading 143
date format 132
destination table 129
destination table, case-sensitive name 130
flags 130
installing 32
loading from many files 138
loading to multiple tables 138
logging bad rows during import 145
map file: supported flags 131
parameters 130
newline character 143
null
handling nulls when loading data 134
null flag for bulk loading 134
O
ODBC 81
bytea handling 83
driver 81
Driver Manager 23
Driver Manager configuration 23
set up SSL 44
ODBC driver
for MicroStrategy 102
165
Q
queen
system parameters, setting 40
query
transpose results 78
typing in ACT 66
via Java JDBC calls 99
query buffer 74
send to file 74
query history 74
send to file 74
query results 75
send to file 75
query tool
installing ACT client 16
quiet mode of ACT 63
quote character flag for bulk loading 135
quoted identifier
enable in Aster Database 97
R
recommended client settings 80
recommended loading settings 143
Aster Client Guide
redirect command history to file 74
redirect query history to file 74
redirect query results to file 75
reporting tools
MicroStrategy 102
results 75
send to file 75
row
transpose columns and rows 78
running SQL scripts 75
with \i 75
with -f 62
S
schema
list all schemas in Aster Database 77
schema argument during batch load 140
scripts 75
run SQL script with \i 75
run SQL script with -f 62
security
SSL for ODBC 39
sed
for backslash sequences 142
sed, using with loader 142
send command history to file 74
send query history to file 74
send query results to file 75
server-side cursors
in ACT 69
set command in ACT 74
settings
enable_backslash_escapes 97
enable_quoted_identifiers 97
SQL settings 97
SSL connection settings 39
SQL
customizing SQL behavior 97
parameters that affect SQL command interaction 73
parameters that set ACT client behavior 61
parameters that set SQL behavior 97
redirect query results to file 75
scripts, running with \i 75
scripts, running with -f 62
SQL commands
utility commands in ACT 73
SQL COPY 129
SQL prompt commands 73
SQL prompt, using in ACT 66
SQL tool, installing ACT client 16
SSH 60
SSH client 60
SSIS 104
Aster Client Guide
SSL 39
set up ODBC client 44
with Active Directory authentication 43
SSL for ODBC 39
certificates for 41
client settings 42
common configuration scenarios 48
encryption of query results 42
how to set up on queen 40
queen settings 39
reference to Aster Database SSL settings 39
SSO for ODBC connections 43
Statement in JDBC 99
STDIN
loading data from 142
support 10
system tables
nc_errortable_part 146
nc_errortable_repl 146
T
tab completion 72
table
list all tables in Aster Database 76
table argument during batch load 139
table argument inside errorlogging block 140
TD Wallet 36
commands 37
configure 33
install 33
installing 33
technical support 10
Teradata Wallet 36
terminal
installing ACT client 16
tools 12
MicroStrategy 102
SSH client 60
troubleshooting
load attempt fails 143
loading, troubleshooting for 147
TRUNCATE
before bulk loading 135
truncate-table flag for bulk loading 135
typeface conventions 9
U
umlauts 80
updated documentation 10
URL 10
Aster Support URL 10
user
list all users in Aster Database 77
166
UTF-8
loading 143
recommended client settings 80
utilities 12
JDBC driver 85
Microsoft .NET, drivers for 104
ncluster_export 150
ncluster_loader 129
ODBC driver 81
V
VACUUM
after bulk loading 135
vacuum-table flag for bulk loading 135
version
check ACT version 62
documentation version 10
ncluster_loader version 135
W
workaround
certificate, missing self-signed cert 39
X
x command 78
167
Aster Client Guide
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising