advertisement
Aster Client Guide
Release Number: 6.10
Product ID: B700-2001-610K
April, 2015
The product or products described in this book are licensed products of Teradata Corporation or its affiliates.
Teradata, Active Data Warehousing, Active Enterprise Intelligence, Applications-Within, Aprimo Marketing Studio, Aster, BYNET, Claraview,
DecisionCast, Gridscale, MyCommerce, QueryGrid, SQL-MapReduce, Teradata Decision Experts, "Teradata Labs" logo, Teradata ServiceConnect,
Teradata Source Experts, WebAnalyst, and Xkoto are trademarks or registered trademarks of Teradata Corporation or its affiliates in the United
States and other countries.
Adaptec and SCSISelect are trademarks or registered trademarks of Adaptec, Inc.
AMD Opteron and Opteron are trademarks of Advanced Micro Devices, Inc.
Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.
Apple, Mac, and OS X all are registered trademarks of Apple Inc.
Axeda is a registered trademark of Axeda Corporation. Axeda Agents, Axeda Applications, Axeda Policy Manager, Axeda Enterprise, Axeda Access,
Axeda Software Management, Axeda Service, Axeda ServiceLink, and Firewall-Friendly are trademarks and Maximum Results and Maximum
Support are servicemarks of Axeda Corporation.
Data Domain, EMC, PowerPath, SRDF, and Symmetrix are registered trademarks of EMC Corporation.
GoldenGate is a trademark of Oracle.
Hewlett-Packard and HP are registered trademarks of Hewlett-Packard Company.
Hortonworks, the Hortonworks logo and other Hortonworks trademarks are trademarks of Hortonworks Inc. in the United States and other countries.
Intel, Pentium, and XEON are registered trademarks of Intel Corporation.
IBM, CICS, RACF, Tivoli, and z/OS are registered trademarks of International Business Machines Corporation.
Linux is a registered trademark of Linus Torvalds.
LSI is a registered trademark of LSI Corporation.
Microsoft, Active Directory, Windows, Windows NT, and Windows Server are registered trademarks of Microsoft Corporation in the United States and other countries.
NetVault is a trademark or registered trademark of Dell Inc. in the United States and/or other countries.
Novell and SUSE are registered trademarks of Novell, Inc., in the United States and other countries.
Oracle, Java, and Solaris are registered trademarks of Oracle and/or its affiliates.
QLogic and SANbox are trademarks or registered trademarks of QLogic Corporation.
Quantum and the Quantum logo are trademarks of Quantum Corporation, registered in the U.S.A. and other countries.
Red Hat is a trademark of Red Hat, Inc., registered in the U.S. and other countries. Used under license.
SAP is the trademark or registered trademark of SAP AG in Germany and in several other countries.
SAS and SAS/C are trademarks or registered trademarks of SAS Institute Inc.
SPARC is a registered trademark of SPARC International, Inc.
Symantec, NetBackup, and VERITAS are trademarks or registered trademarks of Symantec Corporation or its affiliates in the United States and other countries.
Unicode is a registered trademark of Unicode, Inc. in the United States and other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other product and company names mentioned herein may be the trademarks of their respective owners.
T
HE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED ON AN
"
AS
-
IS
"
BASIS
,
WITHOUT WARRANTY OF ANY KIND
,
EITHER
EXPRESS OR IMPLIED
,
INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY
,
FITNESS FOR A PARTICULAR PURPOSE
,
OR
NON
-
INFRINGEMENT
. S
OME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES
,
SO THE ABOVE EXCLUSION
MAY NOT APPLY TO YOU
. I
N NO EVENT WILL
T
ERADATA
C
ORPORATION BE LIABLE FOR ANY INDIRECT
,
DIRECT
,
SPECIAL
,
INCIDENTAL
,
OR CONSEQUENTIAL DAMAGES
,
INCLUDING LOST PROFITS OR LOST SAVINGS
,
EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES
.
The information contained in this document may contain references or cross-references to features, functions, products, or services that are not announced or available in your country. Such references do not imply that Teradata Corporation intends to announce such features, functions, products, or services in your country. Please consult your local Teradata Corporation representative for those features, functions, products, or services available in your country.
Information contained in this document may contain technical inaccuracies or typographical errors. Information may be changed or updated without notice. Teradata Corporation may also make improvements or changes in the products or services described in this information at any time without notice.
To maintain the quality of our products and services, we would like your comments on the accuracy, clarity, organization, and value of this document.
Please email: [email protected].
Any comments or materials (collectively referred to as "Feedback") sent to Teradata Corporation will be deemed non-confidential. Teradata
Corporation will have no obligation of any kind with respect to Feedback and will be free to use, reproduce, disclose, exhibit, display, transform, create derivative works of, and distribute the Feedback and derivative works thereof without limitation on a royalty-free basis. Further, Teradata
Corporation will be free to use any ideas, concepts, know-how, or techniques contained in such Feedback for any purpose whatsoever, including developing, manufacturing, or marketing products or services incorporating Feedback.
Copyright © 2000-2014 by Teradata. All Rights Reserved.
Aster Client Guide 3
Table of Contents
Preface
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Aster Client Guide
Chapter 1: Installing Clients and Utilities
. . . . . . . . . . . . . . . . . . . . . . . 13
Install and Configure the unixODBC Driver Manager on Linux, Solaris, AIX, and Mac OS
4
5
Chapter 2: Authentication and Security
. . . . . . . . . . . . . . . . . . . . . . . . . 45
Adding AD-Based SSO Authentication to ODBC Connections with SSL . . . . . . . . . . . . . 50
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate . . . . . . . . . . 62
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate . . . . . . . . . . 64
Scenario 4: Encrypting Communication from the Queen to the Client . . . . . . . . . . . . . . 67
Aster Client Guide
Chapter 3: Aster Database Cluster Terminal (ACT)
. . . . . . . . . . . 73
Misleading Error Message Reports Problem With a Role Instead of With a User . . . . . 103
Aster Client Guide
Chapter 4: Connect Using Database Drivers
. . . . . . . . . . . . . . . . . . . 105
When Querying System Tables with ODBC, Set AUTOCOMMIT to 'OFF'. . . . . . . . . . 106
6
7
Chapter 5: Tools for .NET Environments
. . . . . . . . . . . . . . . . . . . . . . . . 135
Aster Client Guide
Aster Client Guide
Example: Using Aster Export Source and Aster Loader Destination . . . . . . . . . . . . . . . . 173
Chapter 6: Loading Data
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
Load Stalls Upon Cancellation or Failure Encountered During a Load . . . . . . . . . . . . . 205
Single Quote Character Must be Escaped When Using the -q Option . . . . . . . . . . . . . . 206
Uppercase Characters are Passed as Lowercase if not Escape Quoted . . . . . . . . . . . . . . . 207
Chapter 7: Exporting Data
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
8
Appendix 1: Encoding Support
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
Index
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
9 Aster Client Guide
Preface
This guide provides instructions for users and administrators of Aster Client, version 6.10. If you are using a later version, you must download a newer edition of this guide!
The following additional resources are available:
• Aster Database upgrades, clients and other packages: http://downloads.teradata.com/download/aster
• Documentation for existing customers with a Teradata @ Your Service login: http://tays.teradata.com/
• Documentation that is available to the public: http://www.info.teradata.com/
Conventions Used in This Guide
This document assumes that the reader is comfortable working in Windows and Linux/UNIX environments. Many sections assume you are familiar with SQL.
This document uses the following typographical conventions.
Typefaces
Command line input and output, commands, program code, filenames, directory names, and system variables are shown in a monospaced
font. Words in italics indicate an example or placeholder value that you must replace with a real value. Bold type is intended to draw your attention to important or changed items. Menu navigation and user interface elements are shown using the
User Interface Command
font.
SQL Text Conventions
In the SQL synopsis sections, we follow these conventions:
• Square brackets ([ and ]) indicate one or more optional items.
• Curly braces ({ and }) indicate that you must choose an item from the list inside the braces.
Choices are separated by vertical lines (|).
• An ellipsis (...) means the preceding element can be repeated.
Aster Client Guide 10
Contact Teradata Global Technical Support (GTS)
• A comma and an ellipsis (, ...) means the preceding element can be repeated in a commaseparated list.
• In command line instructions, SQL commands and shell commands are typically written with no preceding prompt, but where needed the default Aster Database SQL prompt is shown: beehive=>
Command Shell Text Conventions
For shell commands, the prompt is usually shown. The
$
sign introduces a command that’s being run by a non-root user:
$ ls
The
#
sign introduces a command that’s being run as root:
# ls
Contact Teradata Global Technical Support
(GTS)
For assistance and updated documentation, contact Teradata Global Technical Support
(GTS):
•
Support Portal:
http://tays.teradata.com/
• International: 212-444-0443
• US Customers: 877-698-3282
• Toll Free Number: 877-MyT-Data
About Teradata Aster
Teradata Aster provides data management and advanced analytics for diverse and big data, enabling the powerful combination of cost-effective storage and ultra-fast analysis of relational and non-relational data. Teradata Aster is a division of Teradata and is headquartered in San Carlos, California.
For more information, go to http://www.asterdata.com
About This Document
This is the “Aster R Enterprise User Guide,” version 6.10. This edition covers Aster Client version 6.10.
Aster Client Guide 11
Version History
Table 2 - 1: Version History Table
Release
AC 6.10
AC 6.00.00.2
AC 6.0
Product ID
B700-2000-610K
B700-2004-600K
B700-2001-600K
Date
April 2015
July 2014
January 2014
About This Document
Aster Client Guide 12
CHAPTER 1
Installing Clients and Utilities
This section explains how to install various clients and utilities that complement your Aster
Database installation.
•
•
Obtaining Aster Client Packages
•
Installing the Aster Database Cluster Terminal (ACT)
•
Installing and Configuring ODBC
•
Installing the .NET Data Provider for Aster
•
•
•
Aster Client Guide 13
Installing Clients and Utilities
Aster Client Support Matrices
Aster Client Support Matrices
Aster Client Platform and OS Support Matrix
This table lists the supported platforms and operating systems for each Aster Client. For
ODBC, there are also some additional requirements, listed in the Aster ODBC Driver Support
.
Table 1 - 1: Aster Client Platform Support Matrix
Client
ACT
Supported Platforms
Windows 32-bit
Windows 64-bit
Linux 32-bit
Linux 64-bit
MacOS X
Solaris x86 32-bit
Solaris SPARC32
Solaris SPARC64
AIX 32-bit
AIX 64-bit
Supported Versions
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8 glibc 2.7 or above is required
RHEL 6.4
RHEL 6.5
glibc 2.7 or above is required
RHEL 6.4
RHEL 6.5
SUSE SLES 11 SP2
10.7 and 10.8
SunOS 5.10 and 5.11
SunOS 5.10 and 5.11
SunOS 5.10 and 5.11
6.1 and 7.1
6.1 and 7.1
14 Aster Client Guide
Aster Client Guide
Installing Clients and Utilities
Aster Client Support Matrices
Table 1 - 1: Aster Client Platform Support Matrix (continued)
Client
ODBC*
Supported Platforms
Windows 32-bit
JDBC
.NET
Windows 64-bit
Linux 32-bit
Linux 64-bit
MacOS X
Solaris x86 32-bit
Solaris SPARC32
Solaris SPARC64
AIX 32-bit
AIX 64-bit
All Platforms except
Solaris SPARC64
Windows 32-bit
Windows 64-bit
Supported Versions
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8 glibc 2.7 or above is required
RHEL 6.4
RHEL 6.5
glibc 2.7 or above is required
RHEL 6.3
RHEL 6.4
SUSE SLES 11 SP2
10.7 or 10.8
SunOS 5.10 and 5.11
Solaris 5.10 and 5.11
Solaris 5.10 and 5.11
6.1 and 7.1 with required libraries: libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
6.1 and 7.1 with required libraries: libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
JRE from Oracle(Sun) 1.5 or above
JRE from IBM 1.6
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8
15
Installing Clients and Utilities
Aster Client Support Matrices
Table 1 - 1: Aster Client Platform Support Matrix (continued)
Client
Loader
Supported Platforms
Windows 32-bit
Windows 64-bit
Linux 32-bit
Linux 64-bit
MacOS X
Solaris x86 32-bit
Solaris 32-bit
AIX 32-bit
AIX 64-bit
Supported Versions
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8 glibc 2.7 or above is required
RHEL 6.4
RHEL 6.5
glibc 2.7 or above is required
RHEL 6.4
RHEL 6.5
SUSE SLES 11 SP2
10.7 or 10.8
SunOS 5.10 and 5.11
SunOS 5.10 and 5.11
6.1 and 7.1 with required libraries: libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
6.1 and 7.1 with required libraries: libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
16 Aster Client Guide
Installing Clients and Utilities
Aster Client Support Matrices
Table 1 - 1: Aster Client Platform Support Matrix (continued)
Client
Exporter
Supported Platforms
Windows 32-bit
Windows 64-bit
Linux 32-bit
Linux 64-bit
Solaris x86 32-bit
Solaris 32-bit
AIX 32-bit
AIX 64-bit
Supported Versions
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8
Windows Server 2008 R2
Windows Server 2012 R2
Windows 8 glibc 2.7 or above is required
RHEL 6.4
RHEL 6.5
glibc 2.7 or above is required
RHEL 6.4
RHEL 6.5
SUSE SLES 11 SP2
SunOS 5.10 and 5.11
Solaris 5.10 and 5.11
6.1 and 7.1 with required libraries: libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
6.1 and 7.1 with required libraries: libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
* See the
Aster ODBC Driver Support Matrix (page 17)
for required libraries and ODBC driver managers.
Aster ODBC Driver Support Matrix
This table lists the supported ODBC driver managers and other requirements for the Aster
ODBC driver on each operating system.
Table 1 - 2: Aster ODBC Driver Support Matrix
Operating System
Windows 32-bit
Prerequisites
Microsoft Driver Manager (included with the Windows OS)
Microsoft Visual C++ 2008 Redistributable Package (x86)
Aster Client Guide 17
Installing Clients and Utilities
Aster Client Support Matrices
Table 1 - 2: Aster ODBC Driver Support Matrix (continued)
Operating System
Windows 64-bit
Linux 32-bit
Linux 64-bit
MacOS 32-bit
MacOs 64-bit
Solaris x86 32-bit
Solaris SPARC 32bit
Prerequisites
Microsoft Driver Manager (included with the Windows OS)
Microsoft Visual C++ 2008 Redistributable Package (x64)
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.3.1
With required libraries: libgcc 3.4.6 or higher glibc 2.7 or higher
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.3.1
With required libraries: libgcc 3.4.6 or higher glibc 2.7 or higher
One of these supported ODBC driver managers:
• iODBC Driver Manager 3.52.3
• unixODBC 2.2.12 or 2.2.14
One of these supported libgcc:
• libgcc 4.2 for MacOS 10.5
• libgcc 4.5 for MacOS 10.6
One of these supported ODBC driver managers:
• iODBC Driver Manager 3.52.3
• unixODBC 2.3.1
One of these supported libgcc:
• libgcc 4.2 for MacOS 10.5
• libgcc 4.5 for MacOS 10.6
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.3.1
libgcc 3.4.6 or higher
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.3.1
libgcc 4.2.0 or higher
18 Aster Client Guide
Installing Clients and Utilities
Obtaining Aster Client Packages
Table 1 - 2: Aster ODBC Driver Support Matrix (continued)
Operating System
Solaris SPARC 64bit
AIX 32-bit
AIX 64-bit
Prerequisites
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.3.1
libgcc 4.2.0 or higher
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.3.1
With required libraries: libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
One of these supported ODBC driver managers:
• DataDirect Driver Manager (included in the ODBC package)
• unixODBC Driver Manager 2.3.1
With required libraries: libstdc++.a (gcc) 4.6.1
libgcc_s.a (gcc) 4.6.1
Obtaining Aster Client Packages
4
5
2
3
6
7
To obtain the Aster Client packages:
1
Find the newest Aster Client package posted on Teradata Developer Exchange: http://downloads.teradata.com/download/aster
Click on the package name for your operating system.
Log in to Teradata Developer Exchange, if prompted.
Accept the End User License Agreement.
Copy the package to the computer you'll use to query your database.
Expand the package by unzipping or untarring the file, as appropriate.
If necessary, set the correct file permissions to make the Aster Client files executable.
Installing the Aster Database Cluster Terminal
(ACT)
ACT is a terminal-based query tool that connects with Aster Database. ACT lets you type queries, see the results, and save them to a file. Alternatively, you can source your queries from
Aster Client Guide 19
Installing Clients and Utilities
Installing the Aster Database Cluster Terminal (ACT) a file. ACT supports connection via SSL and SSO and is available for Windows, Linux, Solaris,
AIX and Mac OS. Follow the instructions for your OS:
•
•
•
•
•
If you will be using ACT commands to interact with the Aster File Store (AFS), after installing
.
Installing ACT on Windows
To install ACT on Windows:
1
2
3
Place the executable act.exe in a directory on your client machine. Teradata Aster recommends that you create a directory,
C:\Program Files\asterdata
and place the act.exe
executable there.
Set permissions to make the file executable:
• In Windows File Explorer, navigate to find the ACT executable file.
• Right-click on it, select
Properties
, click
Security
.
• In the
To change permissions
section, click
Edit
.
• Click the name of the group who can use the application, and tick the checkbox for
Allow/Read & Execute
in the bottom of the window.
• Click
OK
and click
OK
again.
Test your ACT installation like this:
• From the windows
Start
menu, choose
Run...
, type cmd, and click
OK
.
• At the command line, change directories to the folder holding act.exe
.
• Type act --help
at the command line. The help page should be printed to the command line.
• To connect to your Aster Database, type act -h <ip address of your queen>
• When prompted, type the database username and password (The default user/ password credentials are beehive/beehive) The
<dbuser>=>
prompt indicates you've logged in successfully (i.e. beehive=>
for the default user). Type
\q
to quit.
• See the section
for information on running ACT.
Installing ACT on Linux
To install ACT on Linux:
20 Aster Client Guide
Installing Clients and Utilities
Installing the Aster Database Cluster Terminal (ACT)
1
2
Place the executable in a directory on your client machine. Teradata Aster recommends that you use the directory,
/home/beehive/clients
.
Change the file’s permissions to make it executable:
$ chmod 755 act
Tip!
ACT for Linux requires glibc version 2.7 or higher. If you do not have glibc version 2.7 or higher, you must use the
IP address instead of the hostname for the mand ldd --version
.
-h flag when running ACT. To check the version of glibc, issue the com-
3
See the section
for information on running ACT.
Installing ACT on Mac OS
To install ACT on Mac OS:
1
2
3
4
Double click the .dmg file. A new icon with a name similar to the .dmg file name appears on your desktop.
If a new Finder window does not automatically appear, double click on the new icon that has appeared on your desktop. A new Finder window appears.
Find the application’s icon in the Finder window. Drag and drop it into your Applications directory.
See the section
for information on running ACT.
Installing ACT on Solaris
To install ACT on Solaris:
1
2
Place the executable in a directory on your client machine. Teradata Aster recommends that you use the directory
/export/home/beehive/clients
Change the file's permissions to make it executable:
3
$ chmod 755 act
See the section
for information on running ACT.
Installing ACT on AIX
To install ACT on AIX:
The Aster Database Cluster Terminal (ACT) built for version 7.1 of the AIX 64-bit platform is not compatible with version
6.1 of the AIX 64-bit platform.
Aster Client Guide
1
2
Place the executable in a directory on your client machine. Teradata Aster recommends that you use the directory
/export/home/beehive/clients
Change the file's permissions to make it executable:
$ chmod 755 act
21
Installing Clients and Utilities
Installing the Aster Database Cluster Terminal (ACT)
3
See the section
for information on running ACT.
Configuring ACT for the Aster File Store
To configure ACT to support Aster File Store (AFS), perform this setup procedure:
1
If you have not already installed ACT:
2 a
Obtain the client package as shown in “Obtaining Aster Client Packages” on page 19 .
b
Install ACT for your operating system as directed in
“Installing the Aster Database
Cluster Terminal (ACT)” on page 19 .
Install the supported Sun (Oracle) JDK, depending on your operating system:
• On Solaris i386 and Sparc: JDK 1.7, available for download from Oracle at: http://www.oracle.com/technetwork/java/javase/downloads/index.html
Do not install Sun (Oracle) JDK version 1.6 on Solaris i386 and Sparc operating systems. Attempting to install and use
Sun (Oracle) JDK version 1.6 on Solaris i386 and Sparc operating systems will render the ACT AFS shell inoperable.
3
4
• On other platforms: JDK 6u21 for your operating system, available for download from
Oracle at: http://www.oracle.com/technetwork/java/javasebusiness/downloads/java-archivedownloads-javase6-419409.html#jdk-6u21-b07-oth-JPR
Every time you run ACT to access AFS, some environment settings must be used. Make note of what the setting should be for each of these:
a
Set JAVA_HOME to the location of the JDK.
b
Set the JRE_HOME to
$JAVA_HOME/jre
.
c
Add
$JRE_HOME/lib
and
$JAVA_HOME/lib
to the CLASSPATH.
d
Add noarch-aster-adfs-client.jar
to the CLASSPATH. This file is located in the client package.
e
Add
$JAVA_HOME/bin
and
$JRE_HOME/bin
to the PATH.
Use the method of your choice to set the environment variables for the user who will run
ACT:
• When you launch ACT, use a command like:
$ act --afs-port <port> --afs-jar <path_to_jar>
--java-home <path_to_jdk>
• Create or modify the file
/etc/profile.local
.
• Create a new file under the
/etc/profile.d
directory.
• Run the export
commands to set the environment variables in your shell every time before launching ACT for AFS access.
22 Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
5
6
7
For AIX 32-bit and AIX 64-bit platforms, the JDK environment must be set every time before launching ACT for AFS access. Depending on the version of the AIX operating system, use the commands shown here to set the JDK environment:
• On AIX 32-bit: export LIBPATH=/path-to-JDK-32-bit/jre/lib/ppc/:/path-to-
JDK-32-bit/jre/lib/ppc/j9vm/
• On AIX 64-bit: export LIBPATH=/path-to-JDK-64-bit/jre/lib/ppc64/:/pathto-JDK-64-bit/jre/lib/ppc64/j9vm/
If you chose to modify one of the suggested user profiles, edit the appropriate file to set the environment variables as follows: export JAVA_HOME=<path_to_jdk> export JRE_HOME=$JAVA_HOME/jre export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:<path_to_jar>:
$CLASSPATH export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
To check your settings, use the export
command.
Installing and Configuring ODBC
Teradata Aster provides a standard ODBC driver for Aster Database. The supported operating
. Additional dependencies and supported driver managers are listed in the “Aster
ODBC Driver Support Matrix” on page 17
.
The Aster Database ODBC driver may change in any Aster Database release. For this reason, with each new edition of
Aster Database, you should reinstall the driver.
ODBC Driver Manager Compatibility
The Aster Database ODBC driver requires a driver manager. Different platforms use different driver managers. For a list of platforms and their supported ODBC driver managers, see the
“Aster ODBC Driver Support Matrix” on page 17 .
Optional ODBC Setting for varchar Data
For varchar columns without a length limitation (for example, varchar(-1) or text), the ODBC driver truncates the value with a length specified by MaxLenVarchar. By default, the max length returned of varchar is 8192.
If you need to retrieve the whole varchar data through the ODBC driver, you must set the
MaxLenVarchar to a proper value in the odbc.ini
files.
To set this up:
• On Solaris, see the discussion of the odbc.ini
file in Install the unixODBC driver manager (page 33)
.
Aster Client Guide 23
Installing Clients and Utilities
Installing and Configuring ODBC
•
On Linux or AIX, see Configure DataDirect Driver Manager on Linux and AIX (page 29) .
• On Windows, see
varchar Data Settings in Windows (page 24)
.
1
2 varchar Data Settings in Windows
For a 64-bit ODBC driver running on 64-bit Windows or for a 32-bit driver running on 32 bit
Windows:
Set the flag by adding it to the Windows registry entry for the DSN.
Using a registry editor, add this line to the registry. Be sure to first replace <
DSN-NAME
> with your Data Source name:
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\<DSN-NAME>]
"MaxLenVarchar"="30000"
For a 32-bit ODBC driver running on 64-bit Windows:
1
2
Set the flag by adding it to the Windows registry entry for the DSN.
Using a registry editor, add this line to the registry. Be sure to first replace <
DSN-NAME
> with your Data Source name:
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\<DSN-NAME>]
"MaxLenVarchar"="30000"
Install ODBC on Windows
Follow the steps below to install the Aster Database ODBC driver on Windows.
Driver Manager Support
The Windows operating system includes Driver Manager, which allows ODBC applications to use the correct driver.
Installation Procedure
1
Verify that Microsoft Visual C++ 2008 Redistributable Package (x86) is installed on the system where you want to install the driver. You can do this by looking at installed programs from the Control Panel.
Note that Microsoft also offers newer versions of the package, such as Microsoft Visual
C++ 2010 Redistributable Package (x86). Teradata Aster has not tested compatibility with these later versions! If the supported version is not installed:
2
3 a
Download it from Microsoft, choosing the version that fits your architecture:
• 32-bit
• 64-bit
If you are upgrading, use the Windows
Add/Remove Programs
tool to uninstall the old Aster
ODBC version.
Double click on the Aster ODBC .msi file to install it. The actual file name depends on your operating system:
• Windows 32-bit: nClusterODBCInstaller_i386.msi
• Windows 64-bit: nClusterODBCInstaller_x64.msi
24 Aster Client Guide
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
4
5
6
7
8
A setup wizard walks you through the installation of Aster Database ODBC as one of the available data sources on your computer. The default installation directory will be something like:
• For 32-bit Windows:
C:\Program Files\Teradata Aster\Aster ODBC Driver
(x86)\
• For 64-bit Windows:
C:\Program Files\Teradata Aster\Aster ODBC Driver
(x64)\
For AC 5.0.2 and before, the ODBC driver name is Aster Data ODBC Driver for nCluster
From the Windows
Control Panel
, double click
Administrative Tools
to open the
Administrative Tools window.
Double click the
Data Sources (ODBC)
option to open the ODBC Data Source Administrator dialog box.
Click the
System DSN
tab.
Click
Add
to open the Create New Data Source dialog box.
9
10
Select the
Aster ODBC Driver
data source from the list.
Click
Finish
.
The Aster Database Login window appears.
25
Installing Clients and Utilities
Installing and Configuring ODBC
11
In the Aster Database Login window, enter the following information:
26
12
13
•
Data Source
: Use this field to give this database connection an easy-to-recognize name.
•
Server
: The hostname or IP address of your Aster Database queen.
•
Port
: The port on which your Aster Database queen listens for client connections. The default is 2406.
•
Database
: The name of the database in Aster Database you want to connect to. Default system database is beehive.
•
Username
: Database user name.
•
Password
: Database user’s password.
•
Fetch Count
: See
“Throttle Query Results in ACT and Aster Database” on page 84 .
•
SSL Settings
: See “SSL Basics for ODBC” on page 54 .
•
SSO Settings
: See “ODBC with SSO Client-Side Settings” on page 49
.
•
enable_quoted_identifiers
: See
“Quoted-Identifier Handling” on page 128
.
•
enable_backslash_escapes
: See
“Escape Character Handling” on page 128
.
•
Map NUMERIC/DECIMAL to DOUBLE
: Teradata Aster recommends that you turn this feature on.
Click
OK
to save the data source information.
Depending on your Windows operating system:
14
15
•
If you are using a Windows 64-bit system, go to “Specifying the DSN Manager for
• If you are using a Windows 32-bit system, continue with this procedure.
Click
Test Connection
to test the connectivity to the database.
If the connection is working, a confirmation window appears. Click
OK
.
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
Your ODBC setup is complete. Now, in your applications that will query Aster Database, you may connect to Aster Database as an ODBC data source.
7
8
5
6
Specifying the DSN Manager for Windows 64-bit
1
Click the
File DSN
tab.
2
3
Click the
Add...
button.
Select the “Aster ODBC Driver” from the list, and click
Next
..
4
Type in the field for the data source:
• To manage a data source that connects to a 32-bit driver, enter:
C:\windows\sysWOW64\odbcad32.exe
• To manage a data source that connects to a 64-bit driver, enter:
C:\windows\system32\odbcad32.exe
Click
Next
.
Click
Finish
.
Click
Test Connection
to test the connectivity to the database.
If the connection is working, a confirmation window appears. Click
OK
.
Your ODBC setup is complete. Now, in your applications that will query Aster Database, you may connect to Aster Database as an ODBC data source.
Install ODBC on Linux, Solaris, or Mac OS
To install the Aster Database ODBC driver on Linux, Solaris, or Mac OS, follow the instructions below.
Prerequisites
1
The Aster Database ODBC driver may require a specific version of one or more libraries, depending on your operating system. Check the
“Aster ODBC Driver Support Matrix” on page 17
and ensure you have the correct libraries installed.
2
The Aster Database ODBC driver requires that you install a driver manager. Check the
“Aster ODBC Driver Support Matrix” on page 17
and ensure you have the correct ODBC driver manager for your platform installed.
Instructions for installing are available here:
•
For unixODBC Driver Manager: See Install and Configure the unixODBC Driver
Manager on Linux, Solaris, AIX, and Mac OS
.
• For DataDirect Driver Manager: You can find the supporting files in the
../stage/ clients-odbc-<your_platform>/DataDirect/
directory created when you expanding the ODBC driver package. See
“Configure DataDirect Driver Manager on
• For iODBC driver manager: On Mac OS, you have the option of using the iODBC driver manager, version 3.52.3 instead of unixODBC driver manager. This driver and its documentation are available from http://www.iodbc.org
.
Aster Client Guide 27
Installing Clients and Utilities
Installing and Configuring ODBC
3
On Linux, ensure that the LC_ALL environment variable is not empty and has been set correctly:
# export LC_ALL=C
(if you use default locale setting)
# export LC_ALL=en_US.UTF-8
(if you use UTF-8 as your locale setting)
Installation Procedure
Install the Aster Database ODBC driver as shown below.
1
2
Extract the ODBC package for your operating system into the directory where you want to install the driver.
Change to the Aster Database driver directory that was created when you extracted the package:
3
4
# cd stage/clients-odbc-<your_client_os>
On Solaris 64-bit, this directory will contains a compressed file; extract it, too. You will see two drivers. Choose
..\ODBCDriver\libAsterDriver.so
Edit your library path environment variable, to add the required libraries to it.
The library path environment variable is:
• On Linux and Solaris:
LD_LIBRARY_PATH
• On Mac OS:
DYLD_LIBRARY_PATH
An option is to edit an environment settings file (for example, ~/.bashrc file on a typical Linux environment). However, doing so can lead to warning messages from other commands, which have no association with the Aster Database
ODBC driver. An alternative option is to create a setup script to set the appropriate environment variable; the script must be “sourced” immediately before using the Aster Database ODBC driver. If a setup script is created to set the appropriate environment variable, consider creating a companion script to unset the environment variable in order to avoid warning messages when no longer using the driver.
5
6
Use the commands shown below to add the ODBC library, depending on your operating system:
• On Linux or Solaris: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/stage
/clients-odbc-<your_client_os>/Libs
• On Mac OS: export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/usr/local/lib/stage
/clients-odbc-<your_client_os>/Libs
Add or edit the
ODBCSYSINI
environment variable, setting it to the directory where your
ODBC connection settings files ( odbc.ini
and odbcinst.ini
) will reside. To follow this example, assume you are working as user “mjones” and will save the configuration files to home directory
/home/mjones
.
export ODBCSYSINI=/home/mjones
Check that the Aster Database ODBC driver library can find all its dependencies.
Assuming you have installed in
/usr/local/lib
, type (on Mac OS, Linux or Solaris):
# cd /usr/local/lib
# ldd stage/clients-odbc-<your_platform>/<your_odbcdriver>/
28 Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
7
<your_driver_library>
where
<your_platform>
is one of:
• linux-64
• linux-32
• mac
• mac64
• solaris-x86
• solaris-sparc64
• solaris-sparc
(for 32-bit) and where
<your_driver_library>
is libAsterDriver.so
If a “not found” message does not appear, then all the required libraries have been linked.
Choose the next step, depending on your operating system and ODBC driver manager:
•
Configure DataDirect Driver Manager on Linux and AIX or
•
Install and Configure the unixODBC Driver Manager on Linux, Solaris, AIX, and Mac
• For Mac OS, go to http://www.iodbc.org
to download and configure iODBC driver manager.
Install ODBC on AIX
Driver Manager Support
Aster ODBC on AIX supports either DataDirect Driver Manager or unixODBC driver manager. The driver manager allows ODBC applications to use the correct driver (in this case, the Aster Database ODBC driver) to connect to Aster Database.
Installation Procedure
Install the Aster Database ODBC driver on AIX.
1
Extract the bundle for the ODBC driver:
a
Unzip the file:
$ gunzip clients-odbc-aix.tar.gz
b
Untar the file:
$ tar -xvf clients-odbc-aix.tar
Configure DataDirect Driver Manager on Linux and AIX
Use the following process to configure the DataDirect Driver Manager on Linux and AIX:
1
Find the DataDirect Driver Manager configuration files under the extracted directory:
2
$ cd stage/clients-odbc-<your_client_os>/DataDirect/setup
Copy the odbcinst.ini
, odbc.ini
, and aster.ini
files from the
/setup
directory to your home directory:
Aster Client Guide 29
Installing Clients and Utilities
Installing and Configuring ODBC
3
4
5
$ cp -p * ~
Change to your home directory:
$ cd
Make backups of the files you moved and rename aster.ini
to
.aster.ini
:
$ cp -p aster.ini aster.ini.backup
$ mv aster.ini .aster.ini
$ cp -p odbc.ini odbc.ini.backup
$ cp -p odbcinst.ini odbcinst.ini.backup
Make the following edits to
.aster.ini
:
a
Set DriverManagerEncoding to UTF-8.
b
Set ODBCInstLib to
<InstallDir>/DataDirect/lib/libodbcinst.so
, replacing
<
InstallDir
> with the folder where the driver is installed.
For example:
[driver]
DriverManagerEncoding=UTF-8
ODBCInstLib=<InstallDir>/DataDirect/lib/libodbcinst.so
DriverLocale=en-US
LogLevel=0
ErrorMessagesPath=<InstallDir>/ErrorMessages
6
7
8
Modify odbc.ini
as follows:
a
Change the DSN configuration parameters SERVER, UID, PWD, DATABASE and
PORT and replace
<InstallDir>
with the folder where the driver is installed.
[ODBC Data Sources]
... ...
asterdsn=AsterDriver
[ODBC]
... ...
[asterdsn]
Driver=<InstallDir>/DataDirect/lib/libAsterDriver.so
SERVER=192.206.82.100
PORT=2406
DATABASE=beehive
UID=beehive
PWD=beehive b
Add this item to the [ODBC] section of odbc.ini
:
InstallDir=<InstallDir>/DataDirect
c
To support multi-byte characters, add this item to the [ODBC] section of odbc.ini
:
IANAAppCodePage=106
Add the
<InstallDir>/DataDirect/lib
directory to one of:
• On Linux:
LD_LIBRARY_PATH
• On AIX PowerPC:
LIBPATH
Export the following values, where
<directory_path>
is the path to the directory where the files odbc.ini
and odbcinst.ini
reside: export ODBCHOME=<directory_path> export ODBCINI=$ODBCHOME/odbc.ini export ODBCINST=$ODBCHOME/odbcinst.ini
30 Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
9
Edit the odbcinst.ini
file, as shown in this example:
[ODBC Drivers]
... ...
AsterDriver=Installed
[ODBC Translators]
OEM to ANSI=Installed
[ODBC]
#This section must contain values for DSN-less connections
#if no odbc.ini file exists. If an odbc.ini file exists,
#the values from that [ODBC] section are used.
[AsterDriver]
Driver=<InstallDir>/DataDirect/lib/libAsterDriver.so
IconvEncoding=UCS-4LE
Install ODBC on Solaris
Driver Manager Support
Aster ODBC on Solaris supports either DataDirect Driver Manager or unixODBC driver manager. The driver manager allows ODBC applications to use the correct driver (in this case, the Aster Database ODBC driver) to connect to Aster Database.
Installation Procedure
Install the Aster Database ODBC driver on Solaris.
1
Extract the bundle for the ODBC driver:
a
Unzip the file:
$ gunzip clients-odbc-solaris.tar.gz
b
Untar the file:
$ tar -xvf clients-odbc-solaris.tar
Configure DataDirect Driver Manager on Solaris
Use the following process to configure the DataDirect Driver Manager on Solaris:
1
Find the DataDirect Driver Manager configuration files under the extracted directory:
2
$ cd stage/clients-odbc-<your_client_os>/DataDirect/setup
Copy the odbcinst.ini
, odbc.ini
, and aster.ini
files from the
/setup
directory to your home directory:
3
4
$ cp -p * ~
Change to your home directory:
$ cd
Make backups of the files you moved and rename aster.ini
to
.aster.ini
:
$ cp -p aster.ini aster.ini.backup
$ mv aster.ini .aster.ini
$ cp -p odbc.ini odbc.ini.backup
$ cp -p odbcinst.ini odbcinst.ini.backup
Aster Client Guide 31
Installing Clients and Utilities
Installing and Configuring ODBC
5
Make the following edits to
.aster.ini
:
a
Set DriverManagerEncoding to UTF-8.
b
Set ODBCInstLib to
<InstallDir>/DataDirect/lib/libodbcinst.so
, replacing
<
InstallDir
> with the folder where the driver is installed.
For example:
[driver]
DriverManagerEncoding=UTF-8
ODBCInstLib=<InstallDir>/DataDirect/lib/libodbcinst.so
DriverLocale=en-US
LogLevel=0
ErrorMessagesPath=<InstallDir>/ErrorMessages
6
7
8
9
Modify odbc.ini
as follows:
a
Change the DSN configuration parameters SERVER, UID, PWD, DATABASE and
PORT.
[ODBC Data Sources]
... ...
asterdsn=AsterDriver
[ODBC]
... ...
[asterdsn]
Driver=<InstallDir>/DataDirect/lib/libAsterDriver.so
SERVER=192.206.82.100
PORT=2406
DATABASE=beehive
UID=beehive
PWD=beehive b
Add this line to the [ODBC] section of odbc.ini
:
InstallDir=<InstallDir>/DataDirect
c
To support multi-byte characters, add this line to the [ODBC] section of odbc.ini
:
IANAAppCodePage=106
Add the
<InstallDir>/DataDirect/lib
to:
LD_LIBRARY_PATH
Export the following values, where
<directory_path>
is the path to the directory where the files odbc.ini
and odbcinst.ini
reside: export ODBCINI=<directory_path>/odbc.ini
export ODBCINST=<directory_path>/odbcinst.ini export ODBCINSYSINI=<directory_path>
Edit the odbcinst.ini
file, as shown in this example:
[ODBC Drivers]
... ...
AsterDriver=Installed
[ODBC Translators]
OEM to ANSI=Installed
[ODBC]
#This section must contain values for DSN-less connections
#if no odbc.ini file exists. If an odbc.ini file exists,
#the values from that [ODBC] section are used.
32 Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
[AsterDriver]
Driver=<InstallDir>/DataDirect/lib/libAsterDriver.so
IconvEncoding=UCS-4LE
Install and Configure the unixODBC Driver Manager on Linux, Solaris, AIX, and Mac OS
On Linux, Solaris, AIX, and Mac OS, Teradata Aster supports using the unixODBC driver manager. This section describes how to install and configure unixODBC.
3
4
Install the unixODBC driver manager
If you already have the unixODBC manager installed, skip this section and proceed to
“Configure the unixODBC driver manager” on page 33
.
1
2
Download the unixODBC driver manager from one of these links, depending on the version you want to install:
• For version 2.3.1: http://www.unixodbc.org/unixODBC-2.3.1.tar.gz
Install unixODBC by running the following commands:
# CFLAGS="-DSIZEOF_LONG=8" ./configure --prefix=/usr \
--sysconfdir=/etc/unixodbc \
--enable-fdb \
--disable-gui && make
Wait while it builds and installs.
Working as root user, run this, substituting your version of the driver where appropriate:
# make install && find doc -name "Makefile*" -exec rm {} \; && chmod 644 doc/{lst,ProgrammerManual/Tutorial}/* &&
5
6
install -v -m755 -d /usr/share/doc/unixODBC-2.3.1 && cp -v -R doc/* /usr/share/doc/unixODBC-2.3.1
Check the installation by typing the following command. This prints version information and lists the names and locations of the configuration files you will need to install or set up.
# odbcinst -j
If your unixODBC installation is not working properly, check the instructions at http:// www.unixodbc.org/
for help.
Proceed to the next section, Configure the unixODBC driver manager
.
Configure the unixODBC driver manager
If you have chosen to use the unixODBC Driver Manager, configure it as follows:
1
Get the templates for the ODBC connection settings files. Copy these files from the Aster
Database driver’s
Setup
directory to the user’s home directory. The files you need are aster.ini
, odbc.ini
and odbcinst.ini
:
# cd /usr/local/lib/stage/clients-odbc-linux64/Setup
# cp odbc.ini ~
# cp odbcinst.ini ~
Aster Client Guide 33
Installing Clients and Utilities
Installing and Configuring ODBC
2 a b
# cp aster.ini ~/.aster.ini
Note that we have also renamed the aster.ini
file, adding a dot at the beginning of the file name. You must do this.
Edit the
.aster.ini
file as follows:
Set DriverManagerEncoding to UTF-32.
Set ODBCInstLib to one of the following, depending on your OS:
• on Linux:
<unixODBC_InstallDir>/lib/libodbcinst.so
• on Solaris:
<unixODBC_InstallDir>/lib/libodbcinst.so
• on Mac OS:
<unixODBC_InstallDir>/lib/libodbcinst.dylib
replacing
<unixODBC_InstallDir>
with the directory installed by the unixODBC driver.
c
Set DriverLocale to en-US
(or another locale or your choice).
d
Set LogLevel to
0
(or another level or your choice).
e
Set the ErrorMessagesPath to point to the ErrorMessages subdirectory in the Aster
Database driver directory. For this example, we edit the last line in the file to read:
ErrorMessagesPath=/usr/local/lib/stage/clients-odbc-<your_OS>/
ErrorMessages
Tip!
At this point, you can run “odbcinst -j” to find out where the ODBC driver expects to find its configuration files.
34
3 f
In a text editor, edit the odbc.ini
file, making the following changes:
a
Set SERVER to the hostname or IP address of your Aster Database queen.
b
Set PORT to 2406, the standard port on which your Aster Database queen listens for client connections.
c
Set DATABASE to the name of the database in Aster Database you want to connect to.
d
Optionally, you may set UID and PWD to your Aster Database SQL username and password, respectively.
e
Finally, Teradata Aster recommends that you add the setting,
NumericAndDecimalAsDouble=1
.
Optionally, you can set a number of other database connection behavior settings.
These include enable_quoted_identifiers
(see
“Quoted-Identifier Handling” on page 128 ),
enable_backslash_escapes
: See
“Escape Character Handling” on page 128 .
For this example, we set the contents of odbc.ini
to read:
[ODBC Data Sources]
Aster ODBC Driver DSN=AsterDriver
[Aster ODBC Driver DSN]
Driver=AsterDriver
SERVER=10.50.52.100
Aster Client Guide
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
PORT=2406
DATABASE=beehive
NumericAndDecimalAsDouble=1
# UID=<USERNAME>
# PWD=<PASSWORD>
Tip!
You can have multiple data sources. The name “Aster ODBC Driver DSN” in the odbc.ini
file is just a default name that Teradata Aster has given to the sample data source. You can rename this source and add more, as shown in this example:
[ODBC Data Sources] my_1st_source=AsterDriver my_2nd_source=AsterDriver
[my_1st_source]
Driver=AsterDriver
SERVER=10.50.52.100
...
[my_2nd_source]
Driver=AsterDriver
SERVER=10.42.43.100
...
4
In a text editor, edit the odbcinst.ini
file, setting the
Driver
parameter to the Aster
Database driver directory path. For this example, we set the contents of odbcinst.ini
to read:
Table 1 - 3: Driver parameter settings for unixODBC
Operating System
Linux 32-bit
unixODBC Version
unixODBC 2.3.1
Linux 64-bit
Solaris 32-bit x86 unixODBC 2.3.1
unixODBC 2.3.1
Driver Parameter Setting
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-linux32/ODBCDriver/lib/ libAsterDriver.so
IconvEncoding=UCS-4LE
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-linux64/ODBCDriver/lib/ libAsterDriver.so
IconvEncoding=UCS-4LE
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-solaris-x86/ODBCDriver/ libAsterDriver.so
IconvEncoding=UCS-4LE
35
Installing Clients and Utilities
Installing and Configuring ODBC
Table 1 - 3: Driver parameter settings for unixODBC
Operating System
Solaris 32-bit
SPARC
unixODBC Version
unixODBC 2.3.1
Solaris 64-bit
AIX 32-bit
AIX 64-bit
MacOS 32-bit
MacOS 64-bit unixODBC 2.3.1
unixODBC 2.3.1
unixODBC 2.3.1
unixODBC 2.2.14
unixODBC 2.2.12
unixODBC 2.3.1
Driver Parameter Setting
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-solaris-sparc/ODBCDriver/ libAsterDriver.so
IconvEncoding=UCS-4LE
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-solaris-sparc64/ODBCDriver/ libAsterDriver.so
IconvEncoding=UCS-4LE
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-aix-ppc32/ODBCDriver/ libAsterDriver.so
IconvEncoding=UCS-4LE
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-aix-ppc64/ODBCDriver/ libAsterDriver.so
IconvEncoding=UCS-4LE
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-mac/ODBCDriver/ libAsterDriver.dylib
IconvEncoding=UCS-4LE
[AsterDriver]
Driver=/usr/local/lib/stage/clientsodbc-mac/ODBCDriver/ libAsterDriver_unixODBC.dylib
IconvEncoding=UCS-4LE
[AsterDriver]
Driver=/usr/local/lib/stage/ clientsodbc-mac64/ODBCDriver/ libAsterDriver.dylib
IconvEncoding=UCS-4LE
5
6
7
Save your changes to the odbcinst.ini
file.
Add the
<InstallDir>/unixODBC/lib
directory to:
• On Linux:
LD_LIBRARY_PATH
• On AIX PowerPC:
LIBPATH
• On Mac OS:
DYLD_LIBRARY_PATH
The installation and configuration are now complete.
36 Aster Client Guide
Aster Client Guide
Installing Clients and Utilities
Installing and Configuring ODBC
Test the unixODBC Driver Manager
Once the driver is installed and configured, you should test it:
This procedure is only valid for testing unixODBC driver connectivity. Do not use this procedure to test any other type of driver connectivity.
The unixODBC driver manager comes with a tool called “isql” that you can use to test the driver. You can test your connection now by connecting to Aster Database using the isql tool.
1
Type the command as shown below. Here, we put the data source name in single quotes because it contains spaces. The “beehive” and “beehive” are the default username and password in a new Aster Database installation:
2
# isql -v 'Aster ODBC Driver DSN' beehive beehive
A “Connected!” message indicates that the driver is working correctly:
3
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
+---------------------------------------+
SQL>
Set-up is complete.
Troubleshooting
If after installation you cannot connect and you see the error message:
3
4
[unixODBC][AsterData][ODBC] (11560) Unable to locate
SQLGetPrivateProfileString function then do the following procedure:
1
2
Find the library libodbcinst.so
( libodbcinst.dylib
on Mac OS) and note its path.
Set the
LD_LIBRARY_PATH
environment variable so that it includes the directory that contains the library. For example, if the library is in
/usr/lib64
, then you will type: export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
Open the
.aster.ini file for editing.
Set the
ODBCInstLib
item to the correct path where the library libodbcinst.so
(or libodbcinst.dylib
on Mac OS) is located. For example, if the library is in the directory
/usr/lib64
, the line should read:
ODBCInstLib=/usr/lib64/libodbcinst.so
37
Installing Clients and Utilities
Installing the .NET Data Provider for Aster
Installing the .NET Data Provider for Aster
Prerequisites
Make sure the .NET 2.0 framework is installed on your workstation before installing the .NET
Data Provider for Aster. You can download it from: http://www.microsoft.com/downloads
Procedure
This section describes how to install the .NET Data Provider for Aster. This driver is a managed .NET Data Provider for Aster Database and allows .NET client applications (for example, Console, WinForms, ASP.NET, and Web Services) to send and retrieve data from
Aster Database.
Follow the steps below to install the .NET Data Provider for Aster:
1
2
Place the installer on your client machine:
• For 32-bit, the installer is nClusterDNProviderInstaller_i386.msi
• For 64-bit, the installer is nClusterDNProviderInstaller_x64.msi
Run the installer.
a b
In the
Welcome
screen, click
Next
.
In the
Select Installation Folder
screen:
Choose the installation location. The default, depending on your version of Windows will be something like one of:
C:\Program Files\Teradata Aster\.NET Data Provider for Aster (x86)\
C:\Program Files\Teradata Aster\.NET Data Provider for Aster (x64)\
Specify who should be allowed to use the driver by choosing
Everyone
or
Just me
.
Click
Next
.
38 Aster Client Guide
Installing Clients and Utilities
Installing the .NET Data Provider for Aster
c d
In the
Confirm
window, click
Next
.
When the
Installation Complete
window appears, click
Close
.
Note:
Before proceeding, make sure that the .NET Data Provider for Aster is installed correctly. There should be several DLL files and the RegisterAsterProvider.exe file in the .NET provider install path (for example, C:\Program Files
(x86)\Teradata Aster\Aster .NET Data Provider (x86)). If not, the .NET Data Provider for Aster will not work. You can also check the installation from the Control Panel. If the size of Aster .NET Data Provider (x86) is less than 1 MB and the version is 1.0.x, you need a new driver.
Aster Client Guide
Installing SQLServer and SSAS after installing the .NET Data
Provider for Aster
It is recommended to install SQLServer and SSAS before installing the .NET Data Provider for
Aster. When you install the .NET Data Provider for Aster, the installer checks the Windows registry to see if SSAS is installed. Then the correct files are installed and configured automatically. But in cases where the .NET Data Provider for Aster is already installed before
SSAS/SQL Server, you will need to do the following steps before they can work together:
1
Find these four files:
2
•
Install.cmd
•
Install.vbs
•
Install.reg
•
Aster.xsl
These files can be found in the folder where the .NET Data Provider for Aster is installed, typically something like C:\Program Files (x86)\Teradata Aster\Aster .NET Data Provider
(x86).
Install the files manually.
When installed, these files enable the .NET Data Provider for Aster to appear as an option within Microsoft Visual Studio and Microsoft Analysis Services.
39
Installing Clients and Utilities
Installing the Loader Tool
3
To install the files, run the Install.cmd file from the command line with the /codebase option. The /codebase option should have the full path to the Simba.Net.dll file. The syntax to run the command is:
Install.cmd /codebase <full path>\Simba.Net.dll
For example, the command will look similar to:
Install.cmd /codebase "C:\Program Files (x86)\Teradata Aster
\Aster .NET Data Provider (x86)\Simba.Net.dll"
If you are installing the .NET Data Provider for Aster to work with SQL Server 2005, you must use the /vs2005 option, as in:
Install.cmd /vs2005 /codebase <full path>\Simba.Net.dll
Put the Aster Database cartridge (the file
Aster.xsl
) into the corresponding directory for
SSAS. This file should be placed into all the cartridge directories for SSAS, which may include all of the following directories, though the directories on your Windows machine may differ slightly:
SQL Server 2008
C:\Program Files\Microsoft Analysis Services\AS OLEDB\10\Cartridges
C:\Program Files\Microsoft SQL Server\MSAS10.MSSQLSERVER\OLAP
\bin\Cartridges
C:\Program Files\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\
IDE\DataWarehouseDesigner\UIRdmsCartridge
C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE
\PrivateAssemblies\DataWarehouseDesigner\UIRdmsCartridge
SQL Server 2005
C:\Program Files\Microsoft SQL Server\MSSQL.3\OLAP\bin\Cartridges
C:\Program Files\Microsoft SQL Server\90\Tools\Binn\VSShell\Common7
\IDE\DataWarehouseDesigner\UIRdmsCartridge
C:\Program Files\Microsoft Visual Studio 8\Common7\IDE
\PrivateAssemblies\DataWarehouseDesigner\UIRdmsCartridge
Installing the Loader Tool
Install the Aster Database Loader Tool on a data staging machine that has fast network connectivity to your Aster Database loader nodes. The data staging machine is where you will place the data files to be loaded. Note that if you are upgrading, you can simply replace ncluster_loader with the newer version.
Install the Loader Tool on Linux
1
Copy ncluster_loader to the client machine.
2
Set permissions to make the ncluster_loader file executable.
3
On Linux, ncluster_loader requires glibc 2.7 or later. Update glibc if needed.
Install the Loader Tool on Windows
After expanding the Aster Client package, there is no further installation required. Just run ncluster_loader.exe from the command line.
40 Aster Client Guide
Installing Clients and Utilities
Installing the Export Tool
Install the Loader Tool on AIX
After expanding the Aster Client package, there is no further installation required. Just set permissions to make the ncluster_loader file executable.
Installing the Export Tool
To run the Aster Export Tool, copy it to the client machine and set permissions to make ncluster_export
executable.
Installing Teradata Wallet
Teradata Wallet supports SQL-MR functions and the following Aster Clients on Windows and
Linux:
• ACT
• Loader Tool
• Export Tool
• JDBC
• ODBC
• ADO.NET
Download Teradata Wallet
To use Teradata Wallet with Aster Clients, you must first download it from http:// downloads.teradata.com/download/tools
. The supported version of Teradata Wallet is version
15.00.
Install and Configure Teradata Wallet on Linux
3
4
ACT
1
Install Teradata Wallet.
2
Add a name/value pair to the wallet.
$ ./tdwallet add mypassword
Enter desired value for the item named "mypassword":
Item named "mypassword" added.
Install the latest ACT for Aster Database.
Create the symbolic link “tdwalletdir” in the directory where ACT is installed.
For example, these commands add the symbolic link: cd /home/beehive/work/multibranch/build/bin ln -s /opt/teradata/client/tdwallet tdwalletdir
Aster Client Guide 41
Installing Clients and Utilities
Installing Teradata Wallet
5
Use the Teradata Wallet password instead of the real password to log in to the Aster
Database. For example: act -h 192.65.197.130 -U beehive -w \$tdwallet\mypassword\ -d beehive
Welcome to act 06.10.00.00, the Aster nCluster Terminal.
3
4
JDBC
1
Install Teradata Wallet.
2
Add a name/value pair to the wallet.
5
6
$ ./tdwallet add mypassword
Enter desired value for the item named "mypassword":
Item named "mypassword" added.
Get the JDBC Driver and tdwalletJNI.so
.
Copy tdwalletJNI.so
into
/home/beehive/work/multibranch/builds/buildmain/lib/
.
$ javac -classpath /home/beehive/work/multibranch/builds/build-main/ lib/noarch-aster-jdbc-driver.jar com/test/testjdbc/tdwallet.java
$ java -classpath .:/home/beehive/work/multibranch/builds/build-main/ lib/noarch-aster-jdbc-driver.jar -Djava.library.path=/home/beehive/ work/multibranch/builds/build-main/lib/ com.test.testjdbc
You can also find the tdwalletJNI.so
file in the same directory as noarch-asterjdbc-driver.jar
in these packages:
• clients-platform-version-reversion.tar.gz
• clients_all/platform...
Create the symbolic link “tdwalletdir” in the directory where tdwalletJNI.so
is located.
For example, these commands add the symbolic link: cd /home/beehive/work/multibranch/builds/build-main/lib/ ln -s /opt/teradata/client/tdwallet tdwalletdir
Use the Teradata Wallet password
$tdwallet(mypassword)
instead of the real password to log in to the Aster Database.
3
4
Loader Tool or Export Tool
1
Install Teradata Wallet.
2
Add a name/value pair to the wallet.
5
$ ./tdwallet add mypassword
Enter desired value for the item named "mypassword":
Item named "mypassword" added.
Install the latest Loader Tool or Export Tool for nCluster.
Create the symbolic link “tdwalletdir” in the directory where your Loader Tool or Export
Tool is installed.
For example, these commands add the symbolic link: cd /home/beehive/work/multibranch/build/bin ln -s /opt/teradata/client/tdwallet tdwalletdir
Use the Teradata Wallet password
$tdwallet(mypassword)
when you use password.
42 Aster Client Guide
Installing Clients and Utilities
Installing Teradata Wallet
Install and Configure Teradata Wallet on Windows
ACT
1
Install Teradata Wallet.
2
Add a name/value pair to the wallet.
3
E:\>tdwallet add mypassword
Enter desired value for the item named "mypassword":
Item named "mypassword" added.
Use the Teradata Wallet password instead of the real password to log in to Aster Database.
For example:
E:\asterclientWIN\win64\bin>act.exe -h 192.65.197.130 -U beehive
-w $tdwallet(mypassword)
Welcome to act 06.10.00.00, the Aster nCluster Terminal.
...
3
4
ODBC
1
Install Teradata Wallet.
2
Add name/value pairs to the wallet.
E:\>tdwallet add mypassword
Enter desired value for the item named "mypassword":
Item named "mypassword" added.
Install the latest ODBC driver for Aster Database.
Use the Teradata Wallet password (
$tdwallet(mypassword)
) instead of the real password to log in to Aster Database.
Aster Client Guide
$tdwallet(mypassword)
JDBC
1
Install Teradata Wallet.
2
Add name/value pairs to the wallet.
3
E:\>tdwallet add mypassword
Enter desired value for the item named "mypassword":
Item named "mypassword" added.
Get the latest JDBC driver and make the dependent library libtdwalletJNI.dll
available in the library path.
43
Installing Clients and Utilities
Installing Teradata Wallet
4
E:\>javac -classpath e:\asterclient\lib\noarch-aster-jdbc-driver.jar com\test\testjdbc\tdwallet.java
E:\>java -classpath e:\asterclient\lib -
Djava.library.path=e:\asterclient\lib com.test.testjdbc
$tdwallet(mypassword)
Use the Teradata Wallet password (
$tdwallet(mypassword)
) instead of the real password to log in to Aster Database.
3
4
Loader Tool or Export Tool
1
Install Teradata Wallet.
2
Add a name/value pair to the wallet.
5
$ ./tdwallet add mypassword
Enter desired value for the item named "mypassword":
Item named "mypassword" added.
Install the latest Loader or Export Tool for nCluster.
Create the symbolic link “tdwalletdir” in the directory where your Loader Tool or Export
Tool is installed.
For example, these commands add the symbolic link: cd /home/beehive/work/multibranch/build/bin ln -s /opt/teradata/client/tdwallet tdwalletdir
Use the Teradata Wallet password
$tdwallet(mypassword)
when you use password.
ADO.NET
1
Install Teradata Wallet.
2
Add a name/value pair to the wallet.
3
$ ./tdwallet add mypassword
Enter desired value for the item named "mypassword":
Item named "mypassword" added.
Install the latest ADO.NETfor nCluster.
Use the Teradata Wallet password
$tdwallet(mypassword)
when you use password. For example: connectionString =
“uid=db_superuser;pwd=$tdwallet(mypassword);dbname=beehive;ip=153.65.
179.110;port=2406”;
44 Aster Client Guide
CHAPTER 2
Authentication and Security
This section explains the security options for Aster Clients:
•
•
•
•
•
Teradata Wallet
Teradata Wallet is a software utility that provides the users of an Aster Database client system with full and unrestricted access to their stored database passwords on that system, while at the same time protecting those passwords from being exposed in scripts.
Each user on an Aster Database client has a Teradata Wallet. The Teradata Wallet securely stores the passwords that the user adds to the wallet. However, a user cannot access the passwords stored in the wallets of other users.
Rather than using the passwords in scripts, you can use their corresponding names defined in your wallet.
Teradata Wallet supports SQL-MR functions and the following Aster Clients on Windows and
Linux:
• ACT
• Loader Tool
• Export
• JDBC
• ODBC
• ADO.NET
Aster Client Guide 45
Authentication and Security
Teradata Wallet
Wallet Contents
A wallet contains a set of <name, value> pairs. These pairs consist of Unicode character sequences.
Table 2 - 4: Wallet contents
Item
name value
Description
The name of this password entry. This is the name you use to reference the password in scripts without exposing it. We recommend using meaningful names to help you determine what the password is used for. The name is case-sensitive.
The password. The password is not exposed in your scripts. Only the name of this password is exposed.
The example in
Table 2 - 5 shows entries in the Teradata Wallet of user client system user jdoe.
Table 2 - 5: The Teradata Database passwords of user jdoe
Name
pwd_for_beehive_db pwd_for_customers_db pwd_for_clickstream_db
Value
s4t#gp6s_#4 nsdho_34f oc_m_3nd234
Teradata Wallet Commands
Teradata Wallet provides these commands:
Table 2 - 6: Teradata Wallet commands
Command
tdwallet add name tdwallet addsk name tdwallet del name tdwallet list tdwallet chgpwd tdwallet suppwd tdwallet forgetpwd tdwallet chgsavkey tdwallet version tdwallet help [topic] ...
Description
Adds an item to the wallet. When you run this command, tdwallet prompts you for the value component of the name/value pair.
Adds a string with the specified name (saved-key).
Deletes the specified item from the wallet.
Lists the contents of the wallet.
Changes or establishes the password of the wallet.
Supplies the password of the wallet.
Forgets the password of the wallet.
Changes or establishes the saved-key.
Displays the version information for tdwallet.
Displays the help information for tdwallet.
46 Aster Client Guide
Authentication and Security
Teradata Wallet
Usage
Aster Client Guide
After you install Teradata Wallet you can use the
$tdwallet
directive in place of your password. The syntax if the directive is:
$tdwallet(name) where
name
is the name of a password entry in the Teradata Wallet.
Teradata Wallet Password Processing Rules
When Teradata Wallet processes
$tdwallet
directives, it follows these rules:
1
If specified, process
name
as follows:
• Replace “\)” with “)”
• Replace “\$” with “$”
• Replace “\\” with “\”
2
3
Query the corresponding current user’s wallet for an item with a name matching the result of the processing in step 1.
Replace
name
with the value of the item found by the query in step 2.
Example using Teradata Wallet with Aster Loader Tool
In this example, we load the file sales_fact 2010MarchSales_data.txt
into Aster
Database as the user beehive. Teradata Wallet replaces
$tdwallet(beehive)
with the corresponding password stored on the client Teradata Wallet.
$ ./ncluster_loader -h 10.50.25.100 –w $tdwallet(beehive) -D "~" sales_fact 2010MarchSales_data.txt
Example using Teradata Wallet with Aster Export
In this example, we export data from the Aster Database table mytable
to the file mydata.txt
as the user mjones. Teradata Wallet replaces
$tdwallet(mjones)
with the corresponding password stored on the client Teradata Wallet.
$ ./ncluster_export -U mjones -w $tdwallet(mjones) -h 10.50.52.100 -d mydb mytable mydata.txt
Example using Teradata Wallet with ACT
In this example, we connect to Aster Database as the user beehive. Teradata Wallet replaces
$tdwallet(beehive)
with the corresponding password stored on the client Teradata Wallet.
$ act -d beehive -h 10.42.52.100 -U beehive -w $tdwallet(beehive)
Example using Teradata Wallet in a SQL-MR query
In this query, Teradata Wallet replaces
$tdwallet(abc)
with the corresponding password stored on the client Teradata Wallet.
select * from cfilter(
on (select 1)
partition by 1
database('beehive')
userid('beehive')
47
Authentication and Security
Authentication Cascading
PASSWORD('$tdwallet(abc)')
inputtable('cfilter_test')
outputTable('cfilter_test1')
inputColumns('item')
joinColumns('tranid')
droptable('true'));
Authentication Cascading
Authentication Cascading, which was introduced in Aster Database™ 6.0, enables you to omit the username and password when issuing SQL and function calls from the ACT Command
Line (SQL prompt), from SQL scripts, and when connecting through JDBC or ODBC.
The key features of Authentication Cascading include:
• Support for SQL-MR functions.
• The ability to execute SQL-MR functions without including the username and password in the user script.
• The ability to hide username and password from SQL statements that invoke JDBC driver functions in the Aster Analytics Foundation.
• The ability to write driver functions which do not require authentication credentials (this includes SSO, LDAP or username/password).
• The ability to use the authenticated context from which the SQL-MR function was invoked to re-authenticate with when connecting back via the JDBC or ODBC driver.
• Authentication Cascading allows the user to connect to a different database, provided the user is authorized to connect to it, as long as it is in the same cluster.
• When using Authentication Cascading, if you do not specify a database and port, these are taken to be the default on the Aster Database you are currently connected to.
Prerequisites
• The versions of the Aster client and server must be the same, as backward and forward compatibility are not ensured.
• You must be logged in to Aster Database system from ACT, JDBC, or ODBC to maintain the context of your internal connection. When the session ends, your connection ends and you must reauthenticate if you wish to connect again.
• Authentication Cascading is enabled specifically for SQL-MR functions; it is not supported for Stream.
• When using Authentication Cascading, you should omit the username and password.
Authentication Cascading Continuity
Once ACT, the JDBC driver, or ODBC obtains the authentication information, it makes a connection to the queen. Once an external connection is authenticated, the authentication cascades throughout other connections that are part of that session. A session begins once the user logs in to Aster Database and ends when the user logs out or the connection ends.
48 Aster Client Guide
Authentication and Security
Using Single Sign-on (SSO)
Please refer to
“Parameters for Connecting through JDBC” on page 113 for specifics on the
parameters for connecting through JDBC.
Using Single Sign-on (SSO)
This section covers setting up SSO on the queen, for use with JDBC, ODBC and with ACT.
Configuring Single Sign-on (SSO) with SSL on the Queen
Make the following settings on the queen to enable SSO:
1
2
Set up Aster Database user authentication as explained in the Aster Database User Guide.
Configure the queen SSL configuration flags on the queen as described in the SSL scenario
you are using. For a list of scenarios, see “SSL Security Scenarios” on page 59 .
Configuring the Registry Key for JDBC on Windows
By default, Windows does not allow the session key of a TGT to be accessed. The registry key must be configured on the Windows platform if you use JDBC with SSO.
For Windows XP and Windows 2000, the registry key and value should be:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\Kerberos
Value Name: allowtgtsessionkey
ValueType: REG_DWORD
Value: 0x01
For Windows 2003 and Windows Vista, the registry key and value should be:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\Kerberos\Paramet ers
Value Name: allowtgtsessionkey
ValueType: REG_DWORD
Value: 0x01
ODBC with SSO Client-Side Settings
Do the following:
1
2
3
4
Set the SSL flags in the client’s ODBC configuration file or registry as described in the scenario you are using from
“SSL Security Scenarios” on page 59 .
Set the
EnableSSO
flag in the client’s ODBC configuration file:
•
EnableSSO=1
Ensure that
ServerIP
is set to the fully qualified domain name of the Aster Database queen and not to an IP address.
Set up the libraries:
• For 64-bit Linux machines: The ODBC driver assumes that libvas-gssapi.so
is present at
/opt/quest/lib64/
. If
/opt/quest/lib64/libvas-gssapi.so
does not exist, locate libvas-gssapi.so
by referring to the VAS documentation and set the
GSSPath
parameter to point to the installed location of libvas-gssapi.so
. For
Aster Client Guide 49
Authentication and Security
Using Single Sign-on (SSO) example, if libvas-gssapi.so
is deployed at
/usr/lib64
, then the
GSSPath parameter needs to be set to
/usr/lib64
in the
ODBC.ini
config file as shown below:
GSSPath=/usr/lib64
• For 32-bit Linux machines: The ODBC driver assumes that libvas-gssapi.so
is present at
/opt/quest/lib/
. If
/opt/quest/lib/libvas-gssapi.so
does not exist, locate libvas-gssapi.so
by referring to the VAS documentation and set the
GSSPath
parameter to point to the installed location of libvas-gssapi.so
. For example, if libvas-gssapi.so
is deployed at
/usr/lib
, then the
GSSPath parameter needs to be set to
/usr/lib
in the
ODBC.ini
config file as shown below:
GSSPath=/usr/lib
Adding AD-Based SSO Authentication to ODBC Connections with SSL
If your Aster Database is configured to authenticate users against Active Directory (“AD”), you can configure your Aster ODBC clients to authenticate against AD, too. With this configuration in place, each ODBC client will be required to authenticate against AD when it tries to connect to Aster Database. If the ODBC client authenticates successfully, an SSL channel is established automatically for communication between Aster Database and the client. When your system is configured to use SSO, the
\password
command in ACT will not be available.
Using SSO with ACT
Note that when using SSO with ACT, the
-U
and
-w
options are not used, because the username and password are passed directly to the host via SSO. Also, when using SSO, you should specify a fully qualified hostname using the
-h
option, as in the example:
<hostname>.<domain>.<com|org etc>
. If only the hostname is used, ACT will append the local domain name before attempting to look up the host. Using an IP address with
-h
is not supported with SSO.
The following is an example of a configuration file that uses SSO:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and database host: saturn.asterdata.com
dbname: sampledb username: sampleuser
# SSL settings enable-ssl: true ssl-self-signed-peer: true
# SSO settings enable-sso: true
50 Aster Client Guide
Authentication and Security
SSL Security Basics
SSL Security Basics
This section discusses SSL with Aster Clients generally. It lists all available queen SSL settings and provides basic information that applies to all the clients in these sections:
•
•
SSL-Related Files and Settings
•
SSL Settings on the Queen Reference
•
Setting Configuration Parameters on the Queen
•
SSL Port Number
The Aster Database queen port number for SSL connections is the same as the regular client connection port: 2406. Port 2406 is multiplexed to support both secure sockets layer (SSL) connections and unencrypted connections.
SSL-Related Files and Settings
The queen has the following files related to SSL communications and set-up:
• Private key: Default one is
/home/beehive/certs/server.key
• Self-signed PEM certificate (that is, the base64-encoded ASCII representation of the certificate only; this is the version of the certificate that starts with "-----BEGIN
CERTIFICATE-----" and ends with "-----END CERTIFICATE-----"). The default one is
/ home/beehive/certs/server.cert
• Another copy of the self-signed certificate, also in PEM format, but this time formatted in
X.509 structure. Note! This file may be missing from your Aster Database installation. To work around this problem, you must create the default PEM file from the server.cert
file, saving it as
/home/beehive/certs/server.pem.
To create it, log in to the queen as root, change directories to
/home/beehive/certs
, and type:
# openssl x509 -in server.cert -text >> server.pem
• The queen’s SSL-related settings in the Aster Database queen configuration file,
/home/ beehive/config/procmgmtConfigs/coordinator.cfg
SSL Settings on the Queen Reference
This section provides a reference to the available SSL settings on the Aster Database queen.
Queen-Side SSL Parameters
These are the configuration flags that can be used on the queen to tune SSL behavior. Most queen-side flags have a corresponding client-side flag. When you change a flag on one side
(client or server), you will typically have to make appropriate changes to the other side.
• disallowPeerWithoutCertificates
: If this flag is set, the client cannot communicate with its peer (server) without a valid certificate. Defaults to FALSE.
Aster Client Guide 51
Authentication and Security
SSL Security Basics
• allowSelfSignedPeer
: If this flag is set, Aster Database allows connections from clients with self-signed certificates. Defaults to TRUE.
• Set either trustedCAFileName
or trustedCAPath
, depending on whether you have one or many CA certificates:
• trustedCAFileName
: The pathname of the single PEM-formatted CA certificate that
Aster Database trusts. (You also have the option of trusting multiple CA certificates; see trustedCAPath
, below.) Whenever the queen gets a certificate from the peer, the queen traverses the certificate chain to verify that the certificate specified by trustedCAFileName
is part of the chain. If so, the peer is allowed to connect.
• trustedCAPath
: Directory containing PEM-formatted CA certificates that Aster
Database trusts. The files inside this directory are looked up based on the CA subject name hash value.
• sslCertificatePath
: SSL certificate location.
• sslPrivateKeyPath
: SSL private key location.
• sslFileType
: The formatting type of the certificate. Set this to a string value of 1 for
PEM-encoded certs (called “SSL_FILESYSTEM_PEM” on the client side) or 2 for ASN1encoded certs (called “SSL_FILETYPE_ASN1” on the client side). Default is 1.
• secureMuleServer
: If set to true, Aster Database will be configured to use a secure channel for its communication. If secureMuleServer
is enabled, the configuration flags sslCertificatePath
and sslPrivateKeyPath
should be appropriately set.
• secureWrites
: If set to true, encrypts communications going out to clients. Required to be true if the client is set to encrypt reads (e.g.
--ssl-encrypt-reads
flag is set in ACT).
Setting Configuration Parameters on the Queen
Many procedures for setting up SSL ask you to set configuration parameters on the Aster
Database queen. The procedure in this section explains how to do this.
Setting Persistent Parameters
To edit a setting (and have your edit survive reboots), make the setting as shown below.
Settings you change in this manner will survive reboots and soft restarts but they will not survive upgrades.
1
2
Login as root into queen and use a text editor to open the file,
/home/beehive/config/ procmgmtConfigs/coordinator.cfg
Find the “queenExec” section which looks like:
3
"taskName": "queenExec",
"nodeIps": "REPLACE_NODE_IP",
"executableLocation": "/home/beehive/bin/exec/queenExec",
"maxTaskRestarts": -1
Add the executableArgs
section to the above section as shown below:
"executableArgs": "<flags in CSV format>"
For example, executableArgs
for Scenario 1 (described earlier in this chapter) will look like:
"taskName": "queenExec",
52 Aster Client Guide
Authentication and Security
SSL Security Basics
4
"nodeIps": "REPLACE_NODE_IP",
"executableLocation": "/home/beehive/bin/exec/queenExec",
"maxTaskRestarts": -1,
"executableArgs": "--disallowPeerWithoutCertificates=false,
--allowSelfSignedPeer=true,--trustedCAFileName=/home/beehive/certs/ server.pem,--sslCertificatePath=/home/beehive/certs/server.cert,
--sslPrivateKeyPath=/home/beehive/certs/server.key,--sslFileType=1"
Soft restart and activate the cluster.
You cannot configure the SSL flags on the queen to persist after Aster Database upgrades. The above coordinator.cfg
file will be overwritten the next time you upgrade! Before every upgrade, make a copy of your configuration changes, and reapply them after the upgrade.
This step is part of the documented upgrade procedure.
Creating Certificates
Creating a Self-Signed Certificate
You may create your own self-signed certificate. The high-level steps for doing this using the
OpenSSL tool are shown here, assuming you create a private key file called “host.key”: openssl genrsa 1024 > host.key
chmod 400 host.key
openssl req -new -x509 -nodes -sha1 -days 365 -key host.key > host.cert
Generating a Private Key and Certificate
You can use the default files under the path
/home/beehive/certs/
which were generated automatically when Aster Database was installed.
1
2
You can optionally manually regenerate the default private key and certificate by following these steps:
Log in to the queen as root.
Shut down Aster Database:
3
# ncli system softshutdown
Issue the following command:
4
5
# /home/beehive/bin/lib/configure/ConfigureNCluster.py --resetCert
Restart Aster Database:
# ncli system softstartup
Activate Aster Database:
# ncli system activate
This will back up any existing server.key and server.cert files present in
/home/beehive/ certs
and generate a new private key and certificate file.
Aster Client Guide 53
Authentication and Security
SSL Basics for ODBC
SSL Basics for ODBC
On the client side for ODBC, the files related to SSL and its configuration are:
• The queen’s public key in PEM format. This is a copy of the queen’s server.pem
. For example, you might save it as
/home/mjones/certs/server.pem
• The client’s SSL-related settings, stored in one of:
• For Linux, in the client’s odbc.ini
file.
• For Windows, in the ODBC parameter fields of the registry.
Client-Side SSL Settings
There are several configuration flags that can be used on the client side to tune SSL behavior for the Aster ODBC Client. Most client-side flags have a corresponding queen-side flag. When you change a flag on one side (client or server), you will typically have to make appropriate changes to the other side.
•
EnableSSL
: Enables/disables the use of SSL (string value of 0 for false, or 1 for true).
• 0 = Disable SSL (default)
• 1 = Enable SSL
•
SSLEncryptReads
: Determines whether the client expects data returned from database to be encrypted (string value of 0 for false, or 1 for true).
• 0 = query results are unencrypted (the default)
• 1 = query results are encrypted
•
SSLAllowSelfSignedPeer
: Determines whether the client allows peers with self signed certificates to communicate (string value of 0 for false, or 1 for true; default is 1).
•
SSLFileType
: The certificate file type. A string value; one of:
•
SSL_FILETYPE_PEM
(the default)
•
SSL_FILETYPE_ASN1
•
SSLPrivateKeyPath
: Path to the private key to be used. Optional. (A string value.)
•
SSLCertificatePath
: Path to the SSL certificate to be used. (A string value.)
• Set either
SSLTrustedCADir
or
SSLTrustedCAFilename
, depending on whether you have one or many CA certificates:
•
SSLTrustedCADir
: Path to the directory containing CA certificates in PEM format.
(A string value.)
•
SSLTrustedCAFilename
: Filename of CA certificate in PEM format. (A string value.)
Setting SSL Parameters for the ODBC Client
This section explains how to set up the Aster ODBC Client for SSL communications with
Aster Database. Consult the section that applies to your operating system:
•
“Linux ODBC DSN Set-Up” , below; or
•
“Windows ODBC DSN Set-Up” on page 55
54 Aster Client Guide
Aster Client Guide
Authentication and Security
SSL Basics for ODBC
Linux ODBC DSN Set-Up
On a UNIX-like systems, DSNs are added by setting parameters in the odbc.ini
file
Sample ODBC.INI:
This sample assumes your queen machine is called cqueen.asterengqa.com
and that you are following Scenario 1 (outlined earlier in this chapter):
[ODBC Data Sources]
AsterTest=AsterDriverTest
[AsterTest]
Driver=AsterDriverTest
SERVER=cqueen.asterengqa.com
DATABASE=beehive
PORT=2406
UID=testuser13
PWD=testuser133
SQLSupportedConversions=3
NumericAndDecimalAsDouble=1
EnableSSO=0
GSSPath=
EnableSSL=1
SSLEncryptReads=0
SSLAllowSelfSignedPeer=1
SSLFileType=SSL_FILETYPE_PEM
SSLPrivateKeyPath=
SSLCertificatePath=
SSLTrustedCADir=
SSLTrustedCAFilename=
Sample ODBCINST.INI:
This sample assumes you have installed the driver in
/Drivers/AsterDriver/ODBCDriver
:
[AsterDriverTest]
Driver=/Drivers/AsterDriver/ODBCDriver/libAsterDriver.so
IconvEncoding=UCS-4LE
Sample aster.ini:
This sample assumes you want to log error messages in
/Drivers/AsterDriver
:
[driver]
DriverManagerEncoding=UTF-32
DSILogging=1
ErrorMessagesPath=/Drivers/AsterDriver/ErrorMessages
Windows ODBC DSN Set-Up
On Windows, DSNs are added by modifying the Windows Registry using regedit.exe
or using a
.reg
file. The standards surrounding key names (such as whether to use “Server” or
“servername”) used in a connection string for ODBC driver are loose, so please take care to follow the examples we provide.
Details for setting the values in the registry are given below. Choose the section that fits your needs and client type:
•
Adding a driver to a Windows 32-bit operating system
55
Authentication and Security
SSL Basics for ODBC
•
Adding a 64-bit driver to a Windows 64-bit operating system
•
Adding a 32-bit driver to a Windows 64 bit operating system
Adding a driver to a Windows 32-bit operating system
Make the following registry settings:
Driver for 32-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\Aster ODBC
Driver (32-bit)]
"Driver"="C:\\AsterDriver-
Win32\Wow6432Node\ODBCDriver\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver32"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win32\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
The values in the keys above can be modified depending on where the driver is located on the local machine and what the name of the driver should be. The values above are based on the assumption that the driver folder is at "C:\AsterDriver-Win32" and the name of the driver is
"AsterDriver32". For an example .reg file that makes these settings, contact Teradata support.
DSN for 32-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\AsterDSN32]
"driver"="AsterDriver32"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
"SSLTrustedCAFilename"="\"\""
"EnableSSO"="0"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\ODBC Data Sources]
56 Aster Client Guide
Aster Client Guide
Authentication and Security
SSL Basics for ODBC
"AsterDSN32"="AsterDriver32"
Adding a 64-bit driver to a Windows 64-bit operating system
Below we list the registry entries for setting up a 64-bit driver on Windows. Make the registry settings shown below:
•
64-bit driver for 64-bit Windows
•
64-bit driver for 64-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\AsterDriver64]
"Driver"="C:\\AsterDriver-Win64\\ODBCDriver\\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver64"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win64\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
64-bit DSN for 64-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\AsterDSN64]
"driver"="AsterDriver64"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
"SSLTrustedCAFilename"="\"\""
"EnableSSO"="0"
[HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\ODBC Data Sources]
"AsterDSN64"="AsterDriver64"
Above, the name of the DSN for the 64-bit driver is AsterDSN64. The server being connected to is 10.51.12.100.
57
Authentication and Security
SSL Basics for ODBC
Adding a 32-bit driver to a Windows 64 bit operating system
Some applications running on a Windows 64-bit machine require a 32-bit driver. The 32-bit operations work on Windows 64 bit machines under a mechanism called “Wow6432Node.”
Below we list the registry entries for setting up 32-bit drivers on Windows. Make the registry settings shown below:
•
32-bit Driver on 64-bit Windows
•
32-bit Driver on 64-bit Windows
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\AsterDriver32
]
"Driver"="C:\\AsterDriver-Win32\\ODBCDriver\\AsterDataODBCDSII.dll"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBCINST.INI\ODBC Drivers]
"AsterDriver32"="Installed"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Aster]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Aster\Driver]
"DSILogging"="0"
"ErrorMessagesPath"="C:\\AsterDriver-Win32\\ErrorMessages"
"DriverManagerEncoding"="UTF-16"
The values in the keys above can be modified depending on where the driver is located on the local machine and what the name of the driver should be. The values above are based on the assumption that the driver folder is at "C:\AsterDriver-Win32" for 32 bit drivers and the name of the driver is "AsterDriver32". For an example .reg file that makes these settings, contact
Teradata support.
32-bit DSN on 64-bit Windows
To add a DSN for the above driver setup in the registry, make these entries in the registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\AsterDSN32]
"driver"="AsterDriver32"
"SERVER"="10.51.12.100"
"DATABASE"="beehive"
"PORT"="2406"
"UID"="beehive"
"PWD"="beehive"
"SQLSupportedConversions"="3"
"NumericAndDecimalAsDouble"="1"
"EnableSSO"="0"
"SSLKeyFile"="\"\""
"GSSPath"=""
"EnableSSL"="1"
"SSLEncryptReads"="1"
"SSLAllowSelfSignedPeer"="0"
"SSLFileType"="SSL_FILETYPE_PEM"
"SSLPrivateKeyPath"="\"\""
"SSLCertificatePath"="\"\""
"SSLTrustedCADir"="\"\""
58 Aster Client Guide
Authentication and Security
SSL Security Scenarios
"SSLTrustedCAFilename"="\"\""
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\ODBC Data
Sources]
"AsterDSN32"="AsterDriver32"
Above, the name of the DSN for the 32-bit driver is AsterDSN32. The server being connected to is 10.51.12.100.
SSL Security Scenarios
Support for specific, common SSL scenarios is covered in this section. These common scenarios are described along with the required settings on the queen and the clients that support each scenario:
•
Scenario 1: Queen Provides a Self-Signed Certificate
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate
•
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate
•
Scenario 4: Encrypting Communication from the Queen to the Client
•
Scenario 5: Client has a Copy of the Certificate You Provide
This matrix shows which clients support which scenarios:
Table 2 - 7: Aster Client and SSL Scenario Compatibility Matrix
Aster Client
ODBC
JDBC
ACT
Supported SSL Scenarios
•
Scenario 1: Queen Provides a Self-Signed Certificate
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s
•
Scenario 3: Client CA-signed Certificate Must Match the
•
Scenario 4: Encrypting Communication from the Queen to the
•
Scenario 5: Client has a Copy of the Certificate You Provide
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s
•
Scenario 4: Encrypting Communication from the Queen to the
•
Scenario 1: Queen Provides a Self-Signed Certificate
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s
•
Scenario 3: Client CA-signed Certificate Must Match the
•
Scenario 4: Encrypting Communication from the Queen to the
•
Scenario 5: Client has a Copy of the Certificate You Provide
Aster Client Guide 59
Authentication and Security
SSL Security Scenarios
Table 2 - 7: Aster Client and SSL Scenario Compatibility Matrix
Aster Client
.NET
Aster Loader Tool
Aster Export Tool
Informatica Connector
SSIS Connector
Supported SSL Scenarios
•
Scenario 1: Queen Provides a Self-Signed Certificate
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s
•
Scenario 3: Client CA-signed Certificate Must Match the
•
Scenario 4: Encrypting Communication from the Queen to the
•
Scenario 5: Client has a Copy of the Certificate You Provide
•
Scenario 1: Queen Provides a Self-Signed Certificate
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s
•
Scenario 3: Client CA-signed Certificate Must Match the
•
Scenario 4: Encrypting Communication from the Queen to the
•
Scenario 5: Client has a Copy of the Certificate You Provide
•
Scenario 1: Queen Provides a Self-Signed Certificate
•
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s
•
Scenario 3: Client CA-signed Certificate Must Match the
•
Scenario 4: Encrypting Communication from the Queen to the
•
Scenario 5: Client has a Copy of the Certificate You Provide
No SSL support.
No SSL support.
Scenario 1: Queen Provides a Self-Signed Certificate
In this scenario, the Aster Database allows connections from clients that have no certificate.
The server provides a self-signed certificate to the client at the time of authentication. The certificate on the queen will be signed by the server itself, as opposed to by a CA (certification authority). In such a configuration, any client can establish an SSL-secured connection, provided the user authenticates successfully with Aster Database. Communications from the client to Aster Database (for example, queries submitted to the database) are SSL encrypted, and query results travel in the clear.
Scenario 1: Queen-Side Settings
1
Determine if you want to use the default certificate or generate a new one:
• To use the default certificate, continue to the next step.
•
To generate a new certificate, see “Generating a Private Key and Certificate” on page 53 .
60 Aster Client Guide
Aster Client Guide
Authentication and Security
SSL Security Scenarios
2
Make the following settings on the queen (note that these are the default settings for a new
Aster Database installation):
• disallowPeerWithoutCertificates=false
• sslCertificatePath=/home/beehive/certs/server.cert
• sslPrivateKeyPath=/home/beehive/certs/server.key
• sslFileType=1
(A value of “1” means SSL_FILETYPE_PEM.)
• Ensure that secureWrites
is set to false.
• Ensure that secureMuleServer
is set to true.
• There is no need to set the trustedCAPath
and trustedCAFileName
parameters.
Scenario 1: ODBC Client-Side Settings
Make the following settings on each ODBC client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
• There is no need to set the other SSL settings such as
SSLPrivateKeyPath
.
Scenario 1: ACT Client-Side Settings
Use the following command line arguments when executing ACT:
•
--enable-ssl
•
--ssl-self-signed-peer
Or use a configuration file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and database host: 10.10.10.10
dbname: sampledb username: sampleuser
# SSL settings enable-ssl: true ssl-self-signed-peer: true
Note that the client doesn't need to keep a copy of the server's certificate.
Do not use the other SSL settings such as
--ssl-trusted-ca-file
, or
--ssl-trustedca-path
.
Scenario 1: .NET Client-Side Settings
The .NET client connection string must contain
EnableSSL
and
SSLAllowSelfSignedPeer configuration information.
This example adds the SSL connection information to the .NET client connection string:
61
Authentication and Security
SSL Security Scenarios
String connectionString = ""; connectionString +=
"uid=db_superuser;pwd=db_superuser;dbname=beehive;ip=153.65.178.185;port
=2406;EnableSSL=1;SSLAllowSelfSignedPeer=1"
Scenario 1: Aster Loader Tool Client-Side Settings
Use the following command line arguments when executing the Aster Loader Tool:
•
--enable-ssl
•
--ssl-self-signed-peer
Scenario 1: Aster Exporter Tool Client-Side Settings
Use the following command line arguments when using the Aster Exporter Tool:
•
--enable-ssl
•
--ssl-self-signed-peer
Scenario 2: Client Must Have a CA-Signed Copy of the Server’s Certificate
In this scenario, the client needs a CA-signed copy of the queen’s certificate. The location of this certificate can be specified with
SSLTrustedCAFilename
, or as a chain of trusted certificates using the
SSLTrustedCADir
parameter. For more specific information on using a certificate chain, see
Queen-Side SSL Parameters (page 51)
.
Scenario 2: Queen-Side Settings
1
Determine if you want to use the default certificate or generate a new one:
• To use the default certificate files under the path
/home/beehive/certs/
on the queen, continue to the next step.
• To manually generate a new private key and certificate, log in to the queen as root and ensure that the queen is using openssl version 1.0.1c by issuing:
# openssl version
Then issue:
2
# openssl genrsa 1024 > server.key
# chmod 400 server.key
# openssl req -config /home/beehive/config/openssl.cnf -new -x509
-nodes -sha1 -days 365 -key server.key > server.cert
# openssl x509 -in server.cert -text > server.pem
Make the following settings on the queen:
• disallowPeerWithoutCertificates=false
• trustedCAFileName=/home/beehive/certs/server.pem
(or alternatively, use trustedCAPath
to specify multiple files).
• sslCertificatePath=/home/beehive/certs/server.cert
• sslPrivateKeyPath=/home/beehive/certs/server.key
• sslFileType=1
(A value of "1" means SSL_FILETYPE_PEM.)
• Ensure that secureWrites
is set to false.
• Ensure that secureMuleServer
is set to true.
62 Aster Client Guide
Aster Client Guide
Authentication and Security
SSL Security Scenarios
Scenario 2: JDBC Client-Side Settings
1
Generate a truststore for the Aster Database certificate by copying the file
/home/ beehive/certs/server.cert
from queen node to your local client machine.
2
To import the certificate, execute a command like this at the command prompt on the client, substituting the correct paths and the alias of your choice: keytool -import -trustcacerts -alias "asterca" -file
C:\keystore\server.cert -keystore C:\truststore\truststore.jks
3
Issue the following JDBC commands on the client, substituting the correct user, password, database, and path values for the truststore:
Properties props = new Properties(); props.setProperty("user", "beehive"); props.setProperty("password", "beehive_password"); props.setProperty("database", "beehive"); props.setProperty("ENABLESSL", "true"); props.setProperty("SSLTRUSTSTORE","C:\\truststore\\truststore.jks"); props.setProperty("SSLTRUSTSTOREPASSWORD","aster_password");
Connection conn = DriverManager.getConnection(url, props);
Scenario 2: ODBC Client-Side Settings
1
Copy the queen’s public key (self-signed certificate), server.pem
, to the client. For this example, we will assume the client will store the public key as
/home/jbloggs/certs/server.pem
.
/home/beehive/certs/
2
Make the following settings on each ODBC client:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLTrustedCADir=/home/jbloggs/certs/server.pem
Scenario 2: ACT Client-Side Settings
Copy the queen's public key (self-signed certificate),
/home/beehive/certs/server.pem
, to the client. For this example, we will assume the client will store the public key as
/home/ jbloggs/certs/server.pem
.
Use the following command line arguments when executing ACT:
•
--enable-ssl
•
--ssl-trusted-ca-file server.pem
Or use a config file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and database host: 10.10.10.10
dbname: sampledb username: sampleuser
63
Authentication and Security
SSL Security Scenarios
# SSL settings enable-ssl: true ssl-trusted-ca-file: server.pem
Because
--ssl-self-signed-peer
is not specified, the connection can be made only when the server provides a CA signed certificate. The identical CA signed certificate must already exist on the client. However, note that if
--ssl-self-signed-peer
were used, the server would able to supply the certificate at the time of connection and nothing is required on the client.
Scenario 2: .NET Client-Side Settings
The .NET client connection string must contain
EnableSSL
and
SSLTrustedCAFileName configuration information.
This example adds the SSL connection information to the .NET client connection string:
String connectionString = ""; connectionString +=
"uid=db_superuser;pwd=db_superuser;dbname=beehive;ip=153.65.178.185;port
=2406;EnableSSL=1;ssltrustedcafilename=c:\\SSL_Relate\\client_cert server.pem";
Scenario 2: Aster Loader Tool Client-Side Settings
Copy the queen's public key (self-signed certificate),
/home/beehive/certs/server.pem
, to the client. For this example, we will assume the client will store the public key as
/home/ jbloggs/certs/server.pem
.
Use the following command line arguments when executing the Aster Loader Tool:
•
--enable-ssl
•
--ssl-trusted-ca-file server.pem
Scenario 2: Aster Exporter Tool Client-Side Settings
Copy the queen's public key (self-signed certificate),
/home/beehive/certs/server.pem
, to the client. For this example, we will assume the client will store the public key as
/home/ jbloggs/certs/server.pem
.
Use the following command line arguments when executing the Aster Exporter Tool:
•
--enable-ssl
•
--ssl-trusted-ca-file server.pem
Scenario 3: Client CA-signed Certificate Must Match the Queen Certificate
This scenario presents a stricter regime, where the queen only accepts connections from clients that provide a CA-signed certificate, for which a copy already exists on the queen at the time of connection. In other words, clients cannot connect using a self-signed certificate, nor can they connect using a copy of the queen's public key. The identical signed certificate must exist on the queen as well. Setting the server flag disallowPeerWithoutCertificates = true is what forces the client to provide a certificate in order to connect.
64 Aster Client Guide
Aster Client Guide
Authentication and Security
SSL Security Scenarios
Scenario 3: Queen-Side Settings
1
Get the root certificate of the CA (certificate authority) that signed your client's certificate.
Save the root certificate on the queen. For this example, we will save it as
/home/ beehive/certs/client.pem
on the queen.
2
Make the following settings on the queen:
• disallowPeerWithoutCertificates=true
• trustedCAFileName=/home/beehive/certs/client.pem
• sslCertificatePath=/home/beehive/certs/server.cert
• sslPrivateKeyPath=/home/beehive/certs/server.key
• sslFileType=1
(A value of "1" means SSL_FILETYPE_PEM. A value of “2” means
SSL_FILETYPE_ASN1.)
• There is no need to set the trustedCAPath parameter if you use a single root certificate for all clients.
• Ensure that secureWrites
is set to false.
• Ensure that secureMuleServer
is set to true.
1
2
Variation: If your client's certificates were not all signed by the same CA, then you must set
Aster Database to recognize all the CA root certificates used to sign you clients' certificates:
3
Save the root certificates of all the signing CAs on the queen.
Set trustedCAPath
to point to the directory that contains the root certificates. For example:
• trustedCAPath=/home/beehive/certs
Un-set the queen configuration parameter, trustedCAFileName
, by setting it to no value at all. For example:
• trustedCAFileName=
Scenario 3: ODBC Client-Side Settings
1
Generate the files client.pem
using openssl version 1.0.1c:
, client.cert
, and client.key
on each client machine openssl genrsa 1024 > client.key
chmod 400 client.key
openssl req -new -x509 -nodes -sha1 -days 365 -key client.key > client.cert
openssl x509 -in client.cert -text > client.pem
2
Use the following configuration when executing ODBC. For this example, we will assume the client will store the certificate as
/home/jbloggs/certs/client.cert
and the key as
/home/jbloggs/certs/client.key
:
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLCertificatePath=/home/jbloggs/certs/client.cert
65
Authentication and Security
SSL Security Scenarios
•
SSLPrivateKeyPath=/home/jbloggs/certs/client.key
Scenario 3: ACT Client-Side Settings
1
Generate the files client.pem
using openssl version 1.0.1c:
, client.cert
, and client.key
on each client machine openssl genrsa 1024 > client.key
chmod 400 client.key
openssl req -new -x509 -nodes -sha1 -days 365 -key client.key > client.cert
openssl x509 -in client.cert -text > client.pem
2
Use the following command line arguments when executing ACT. For this example, we will assume the client will store the certificate as
/home/jbloggs/certs/client.cert and the key as
/home/jbloggs/certs/ client.key
:
•
--enable-ssl
•
--ssl-self-signed-peer
•
--ssl-certificate-path /home/jbloggs/certs/client.cert
•
--ssl-private-key-path /home/jbloggs/certs/client.key
•
--ssl-cert-filetype 1
(A value of "1" means SSL_FILETYPE_PEM. A value of
“2” means SSL_FILETYPE_ASN1.)
Or use a configuration file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and database host: 10.10.10.10
dbname: sampledb username: sampleuser
# SSL settings enable-ssl: true ssl-certificate-path: /home/jbloggs/certs/client.cert
ssl-private-key-path: /home/jbloggs/certs/client.key
ssl-cert-filetype: 1
Scenario 3: .NET Client-Side Settings
1
Generate the files client.pem
using openssl version 1.0.1c:
, client.cert
, and client.key
on each client machine openssl genrsa 1024 > client.key
chmod 400 client.key
openssl req -new -x509 -nodes -sha1 -days 365 -key client.key > client.cert
openssl x509 -in client.cert -text > client.pem
This example adds the SSL connection information to the .NET client connection string:
String connectionString = ""; connectionString +=
"uid=db_superuser;pwd=db_superuser;dbname=beehive;ip=153.65.178.185;port
=2406;EnableSSL=1;sslcertificatepath=c:\\SSL_Relate\\client_cert\\client
.cert;sslprivatekeypath=c:\\SSL_Relate\\client_cert client.key";
66 Aster Client Guide
Authentication and Security
SSL Security Scenarios
Scenario 3: Aster Loader Tool Client-Side Settings
1
Generate the files client.pem
using openssl version 1.0.1c:
, client.cert
, and client.key
on each client machine openssl genrsa 1024 > client.key
chmod 400 client.key
openssl req -new -x509 -nodes -sha1 -days 365 -key client.key > client.cert
openssl x509 -in client.cert -text > client.pem
2
Use the following command line arguments when executing the Aster Loader Tool. For this example, we will assume the client will store the certificate as
/home/jbloggs/ certs/client.cert and the key as
/home/jbloggs/certs/ client.key
:
•
--enable-ssl
•
--ssl-self-signed-peer
•
--ssl-certificate-path /home/jbloggs/certs/client.cert
•
--ssl-private-key-path /home/jbloggs/certs/client.key
•
--ssl-cert-filetype 1
(A value of "1" means SSL_FILETYPE_PEM. A value of
“2” means SSL_FILETYPE_ASN1.)
Scenario 3: Aster Exporter Tool Client-Side Settings
1
Generate the files client.pem
using openssl version 1.0.1c:
, client.cert
, and client.key
on each client machine openssl genrsa 1024 > client.key
chmod 400 client.key
openssl req -new -x509 -nodes -sha1 -days 365 -key client.key > client.cert
openssl x509 -in client.cert -text > client.pem
2
Use the following command line arguments when executing the Aster Exporter Tool. For this example, we will assume the client will store the certificate as
/home/jbloggs/ certs/client.cert and the key as
/home/jbloggs/certs/ client.key
:
•
--enable-ssl
•
--ssl-self-signed-peer
•
--ssl-certificate-path /home/jbloggs/certs/client.cert
•
--ssl-private-key-path /home/jbloggs/certs/client.key
•
--ssl-cert-filetype 1
(A value of "1" means SSL_FILETYPE_PEM. A value of
“2” means SSL_FILETYPE_ASN1.)
Scenario 4: Encrypting Communication from the Queen to the Client
By default, only the queries issued by the client to the queen are encrypted. It is also possible to encrypt the communication from the queen to the client. Note that these changes can be applied to all of the previous scenarios. To do this, make the following changes in addition to the settings necessary to enable SSL.
Aster Client Guide 67
Authentication and Security
SSL Security Scenarios
All other things being equal, switching any network connection from unencrypted to SSLencrypted has the effect of reducing the maximum available rate of data transmission on that connection.
Scenario 4: Queen-Side Settings
Make the following setting on the queen:
• secureWrites=true
Even though you may have set secureWrites=false
when setting up SSL, set it to true
now in order to enable encryption of communication from the queen. This permits setting up the
SSL connection before enabling two way encryption, if desired.
Scenario 4: ODBC Client-Side Settings
Make the following settings on each ODBC client:
•
SSLEncryptReads=1
Scenario 4: ACT Client-Side Settings
Use the following command line arguments when executing ACT:
•
--ssl-encrypt-reads
Or add the following entry to the configuration file:
• ssl-encrypt-reads: 1
Scenario 4: JDBC Client-Side Settings
Issue the following JDBC commands on the client, substituting the correct user, password, database, and path values for the truststore:
Properties props = new Properties(); props.setProperty("user", "beehive"); props.setProperty("password", "beehive_password"); props.setProperty("database", "beehive"); props.setProperty("ENABLESSL", "true"); props.setProperty("SSLENCRYPTREADS", "true"); props.setProperty("SSLTRUSTSTORE","C:\\truststore\\truststore.jks"); props.setProperty("SSLTRUSTSTOREPASSWORD","aster_password");
Connection conn = DriverManager.getConnection(url, props);
Scenario 4: .NET Client-Side Settings
Add sslencryptreads=1
to the .NET connection string as in this example: connectionString ="...;sslencryptreads=1";
Scenario 4: Aster Loader Tool Client-Side Settings
Use the following command line arguments when executing the Aster Loader Tool:
•
--ssl-encrypt-reads
Scenario 4: Aster Exporter Tool Client-Side Settings
Use the following command line arguments when executing the Aster Exporter Tool:
68 Aster Client Guide
Authentication and Security
SSL Security Scenarios
•
--ssl-encrypt-reads
Scenario 5: Client has a Copy of the Certificate You Provide
This scenario is similar to “Scenario 2: Client Must Have a CA-Signed Copy of the Server’s
Certificate” on page 62 , except that now we replace the default Aster Database certificate with
your site’s own certificate. As before, this configuration allows connections only from clients that have a copy of queen’s public key. This scenario uses the public key, / home/beehive/ certs/sampleco.pem
.
Scenario 5: Queen-Side Settings
1
Determine if you want to use an existing certificate or generate a new one:
• To use an existing certificate, save these files on the queen, substituting the names of your existing files:
/home/beehive/certs/sampleco.cert
/home/beehive/certs/sampleco.pem
/home/beehive/certs/sampleco.key
• To manually generate a new certificate, log in to the queen as root and ensure that the queen is using openssl version 1.0.1c by issuing:
# openssl version
Then issue:
2
# openssl genrsa 1024 > sampleco.key
# chmod 400 sampleco.key
# openssl req -config /home/beehive/config/openssl.cnf -new -x509
-nodes -sha1 -days 365 -key sampleco.key > sampleco.cert
# openssl x509 -in sampleco.cert -text > sampleco.pem
Make the following settings on the queen:
• disallowPeerWithoutCertificates=false
• allowSelfSignedPeer=true
• trustedCAFileName=/home/beehive/certs/sampleco.pem
• sslCertificatePath=/home/beehive/certs/sampleco.cert
• sslPrivateKeyPath=/home/beehive/certs/sampleco.key
• sslFileType=1
(A value of "1" means SSL_FILETYPE_PEM. A value of “2” means
SSL_FILETYPE_ASN1.)
• There is no need to set the trustedCAPath parameter if you use a single root certificate for all clients.
• Ensure that secureWrites
is set to false.
• Ensure that secureMuleServer
is set to true.
Scenario 5: ODBC Client-Side Settings
1
Copy the queen’s public key, this example, we will assume the client will store the public key as
/home/jbloggs/ certs/sampleco.pem
.
/home/beehive/certs/sampleco.pem
to the client. For
2
Make the following settings on each ODBC client:
Aster Client Guide 69
Authentication and Security
SSL Security Scenarios
•
EnableSSL=1
•
SSLEncryptReads=0
•
SSLAllowSelfSignedPeer=1
•
SSLFileType=SSL_FILETYPE_PEM
•
SSLTrustedCADir=/home/jbloggs/certs/
•
SSLTrustedCAFileName=/home/jbloggs/certs/sampleco.pem
Scenario 5: ACT Client-Side Settings
1
Copy the queen’s public key, this example, we will assume the client will store the public key as / home/jbloggs/ certs/sampleco.pem
.
/home/beehive/certs/sampleco.pem
to all clients. For
2
Use the following command line arguments when executing ACT:
•
--enable-ssl
•
--ssl-trusted-ca-file /home/jbloggs/certs/sampleco.pem
Or use a configuration file similar to the following:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and database host: 10.10.10.10
dbname: sampledb username: sampleuser
# SSL settings enable-ssl: true ssl-trusted-ca-file: /home/jbloggs/certs/client.pem
Scenario 5: .NET Client-Side Settings
1
Copy the queen’s public key,
/home/beehive/certs/sampleco.pem
this example, we will assume the client will store the public key as
C:
\
SSL_Relate\client_cert\sampleco.pem
.
to all clients. For
2
The .NET client connection string must contain the
EnableSSL
and
SSLTrustedCAFileName
configuration information. This example adds the SSL connection information to the .NET client connection string:
String connectionString = ""; connectionString +=
"uid=db_superuser;pwd=db_superuser;dbname=beehive;ip=153.65.178.185; port=2406;EnableSSL=1;ssltrustedcafilename=c:\\SSL_Relate\\ client_cert\\sampleco.pem";
Scenario 5: Aster Loader Tool Client-Side Settings
1
Copy the queen’s public key, this example, we will assume the client will store the public key as / home/jbloggs/ certs/sampleco.pem
.
/home/beehive/certs/sampleco.pem
to all clients. For
2
Use the following command line arguments when executing the Aster Loader Tool:
•
--enable-ssl
70 Aster Client Guide
Authentication and Security
SSL Security Scenarios
•
--ssl-trusted-ca-file /home/jbloggs/certs/sampleco.pem
Scenario 5: Aster Exporter Tool Client-Side Settings
1
Copy the queen’s public key, this example, we will assume the client will store the public key as / home/jbloggs/ certs/sampleco.pem
.
/home/beehive/certs/sampleco.pem
to all clients. For
2
Use the following command line arguments when executing the Aster Exporter Tool:
•
--enable-ssl
•
--ssl-trusted-ca-file /home/jbloggs/certs/sampleco.pem
Aster Client Guide 71
Authentication and Security
SSL Security Scenarios
72 Aster Client Guide
CHAPTER 3
Aster Database Cluster Terminal
(ACT)
ACT is a terminal-based query tool that connects with Aster Database. ACT is installed on the queen during a typical Aster Database installation. If you wish to install ACT on another
machine see “Installing the Aster Database Cluster Terminal (ACT)” on page 19
.
This chapter explains how to use Aster Database Cluster Terminal (ACT) to query and manage databases. ACT lets you connect to the database (optionally using SSO and/or SSL), type queries, issue them to Aster Database, and get query results. Alternatively, you can source your queries from a file. ACT can return your query results to the command line or to a file, which makes it useful for extracting data. Meta-commands and shell-like features are provided to facilitate writing scripts and automating tasks.
The following sections explain how to launch and use ACT:
•
•
•
•
•
ACT Commands (at the SQL Prompt)
•
•
•
ACT Quick Start
You run ACT from the command line. The launch command is:
$ act -d <db name> -h <hostname> -U <username>
[-w <password>] [argument flags] where
• db name is your database name,
• hostname is the name or IP address of the queen,
Aster Client Guide 73
Aster Database Cluster Terminal (ACT)
Launching ACT
• username is your SQL login name,
• password is your SQL password in Aster Database (this is optional; ACT prompts you for a password if you do not pass a
-w
parameter), and
•
For example:
$ act -d emea_sales -h 10.48.58.100 -U mjones
Tip!
When using SSO (single sign-on), the
-U
and
-w
options are not used, because the username and password are passed directly to the host via SSO.
To log in to the default database that is provided in your installation, type this, replacing the IP address with the hostname or IP address of your Aster Database queen:
$ act -d beehive -h 10.42.52.100 -U beehive -w beehive
To see a list of ACT command line arguments, type:
$ act --help
Launching ACT
See the appropriate section below for instructions on launching ACT:
•
•
Launching ACT on Linux, Solaris or AIX
•
•
Launching ACT Directly on the Queen
Tip!
On an Aster Database where LDAP authentication is enabled, if during logon an ACT user gets the error message:
'ERROR: An internal error has occurred.'
, make sure the username is present in Aster Database with proper privileges.
•
Launching ACT on Windows
1
2
3
Open a command prompt.
Change directories to the folder which contains the act.exe executable.
Log in to ACT .
74 Aster Client Guide
Aster Database Cluster Terminal (ACT)
Launching ACT
Launching ACT on Linux, Solaris or AIX
1
2
Ensure that your path includes the directory where ACT is installed.
Log in to ACT .
Tip!
ACT for Linux requires glibc version 2.6.18 or higher. If you do not have glibc version 2.6.18 or higher, you must use the IP address instead of the hostname for the the command ldd --version
.
-h flag when running ACT. To check the version of glibc, issue
The Aster Database Cluster Terminal (ACT) built for version 7.1 of the AIX 64-bit platform is not compatible with version
6.1 of the AIX 64-bit platform.
Launching ACT on Mac
1
2
Open a terminal.
Log in to ACT .
Launching ACT Directly on the Queen
Your Aster Database queen also contains an installation of the ACT client. To run it there, you’ll need a user account on the queen machine and an SSH client on your workstation.
To run the ACT on the queen:
1
Open a SSH connection to the queen. If you do not have a user account on the queen, ask the machine’s administrator for one.
Tip!
If you need an SSH client, do one of the following:
•
Install the OpenSSH client as explained on the OpenSSH homepage at http://www.openssh.org/
• or, for Windows only, install the PuTTY SSH client.
2
3
Change directories to the directory where ACT is installed (by default,
/home/beehive/ clients
).
Log in to ACT .
Logging In to ACT
1
Run ACT by typing a command like:
2
act -d <db name> -h <hostname> -U <username> [-w <password>]
[argument flags]
Note that if you do not provide the hostname using
-h
, ACT defaults to the localhost. For details on the command line options, see
“Startup Parameters for ACT” on page 76
Provide your database password by:
• adding the optional
–w <password>
to the ACT login string, or
Aster Client Guide 75
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
3
4
• omitting the
-w
argument and providing your database password at the prompt.
Choose a database by adding
-d <database name>
to the ACT login string. If
-d
is not used, ACT places you in the system database (with the default name “beehive”).
You will see a welcome message, followed by the database prompt, which shows the database name, followed by “=>”. For example:
Welcome to act 6.10, the Aster Database Terminal.
beehive=>
Startup Parameters for ACT
ACT takes a variety of startup parameters when you launch it. (Don’t confuse these with
ACT’s at-the-SQL-prompt flags, which you can read about in “ACT Commands (at the SQL
.)
You can pass these parameters at the command-line (see
“Using the "on-error-stop" Option in
) or in a configuration file or “Using a Configuration File to Pass ACT
Startup Parameters” on page 80
). The same startup parameter cannot be repeated in a single invocation of ACT, or an error will be returned.
To list the command line parameters, type act --help
.
Descriptions of each parameter are below:
Table 3 - 1: Summary of the most common command-line parameters for ACT
Flag
-d [ --dbname ] <dbname>
-h [ --host ] <hostname>
-U [ --username ] <username>
-l
(the letter “ell”)
[ --list-databases ]
Description
Specifies the database name to connect to
(default: “beehive”).
Specifies the Aster Database server host
(default: “localhost”).
Specifies the Aster Database username
(default: “beehive”). Not used with SSO.
Lists all available databases, then exits.
Tip!
Note the default values for the connection parameters. If you do not specify the parameters
-d
(database name),
The default values are:
• “beehive” for database name
• “localhost” for hostname
If
-w
-h
(hostname),
-U
(username), and/or
• “beehive” for username, and
• “2406” for port.
is not used, ACT will prompt for a password.
-p
(port) in the connect string, ACT will use the default values.
76 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Table 3 - 2: General-purpose command-line parameters for ACT
Flag
-d [ --dbname ] <dbname>
--config-file <filename>
-c [ --single-command ]
<command>
Description
Specifies the database name to connect to (default:
“beehive”).
Loads startup parameters from a configuration file specified by <filename>. See
File to Pass ACT Startup Parameters” on page 80
for more information.
Runs only a single command (SQL or internal) and exits. For example: act -c "COPY MyTable FROM stdin;" < myDataFile.dat
Executes commands from file, then exits. Runs a SQL script.
Executes a command file as a single transaction.
-f [ --input-file ]
<filename>
-1
(the numeral “one”)
[ --single-transaction ]
-l
(the letter “ell”)
[ --list-databases ]
-? [ --help ]
-V [ --version ]
--on-error-stop or
-E
Lists available databases, then exits.
Shows command line help, then exits.
Outputs version information, then exits.
Enables the
“on-error-stop”
option; by default this option is disabled. See
Option in ACT” on page 79 for more information.
Table 3 - 3: Input- and output-related command-line parameters for ACT
Flag
-a [ --echo-script-input ]
-e [ --echo-all-input ]
-o [ --redirect-query-results ]
FILENAME
Description
Echoes all input from script.
Echoes commands sent to server.
Sends query results to file (or | pipe).
Table 3 - 4: SSL and SSO related command-line parameters for ACT
Flag
--enable-ssl
Description
Enables Secure Socket Layer (ssl) support. Must be used if any of the other SSL/SSO arguments are used.
77
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Table 3 - 4: SSL and SSO related command-line parameters for ACT (continued)
Flag
--ssl-encrypt-reads
--ssl-self-signed-peer
--ssl-private-key-path
<path>
--ssl-certificate-path
<path>
--ssl-trusted-ca-dir
<directory>
Description
SSL Encrypt Reads. Must be used if secureWrites=true on the server. Conversely, must not be used if secureWrites=false
Configuration Parameters on the Queen (page 52) for
information on how to set the secureWrites parameter on the server.
Indicates that ACT will connect to a Queen which will provide a self-signed certificate.
Indicates where the private key is stored on the client (ACT) machine.
Indicates where the certificate is stored on the client (ACT) machine.
Sets the directory on the client machine where the chain of trusted certificates is stored. Use this parameter when using a chain of certificates rather than a single certificate.
--ssl-trusted-ca-file
<filename>
--enable-sso
Provides the location of the signed copy of the server’s certificate on the client machine.
--ssl-cert-filetype <arg>
Sets the SSL Certificate File Type ( for <arg>, use 1 for PEM;
2 for ANS1; default: 0).
Enables Single sign-on (SSO) support.
--gss-lib-path <path>
For Linux only, sets the GSS shared library path (default on
Linux is /opt/guest/lib32 or /opt/guest/lib64).
Ignored on Windows.
Tip!
The SSL settings in ACT have interdependencies, and in most cases they rely on the SSL settings on the queen.
See SSL Security Scenarios (page 59)
.
Table 3 - 5: Output format-related command-line parameters for ACT
Flag
-q [ --quiet ]
-t [ --print-rows-only ]
-x [ --expanded ]
-A [ --unaligned ]
-F [ --field-separator ] <arg>
Description
Runs quietly and does not print messages, only query output. Use this for clean query output. Often used with the -c flag.
Prints rows only.
Turns on expanded table output.
Turns on unaligned table output.
Sets the field separator (default: '|').
78 Aster Client Guide
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
Table 3 - 6: Connection-related command-line parameters for ACT
Flag
-h [ --host ] <hostname>
-p [ --port ] <port>
-U [ --username ] <username>
-w [ --password ] <password>
Description
Specifies the Aster Database queen hostname or IP address (default: "localhost"). Note that ACT supports glibc version 2.6.18 or higher. If you do not have glibc version 2.6.18 or higher, you must use the
IP address instead of the hostname. To check the version of glibc, issue the command ldd -version
.
When using SSO, you should specify a fully qualified hostname using the -h option, as in the example:
<hostname>.<domain>.<com|org etc>
. If only the hostname is used with SSO, ACT will append the local domain name before attempting to look up the host. Using an IP address with -h is not supported with SSO.
Specifies the Aster Database server port (default:
"2406").
Specifies the Aster Database username (default:
"beehive").
Specifies the Aster Database password. This parameter is optional; ACT will prompt for a password if you do not pass a -w parameter. Not used with SSO.
Table 3 - 7: AFS-related command-line parameters for ACT
Flag
--afs-port <port>
--afs-jar <path_to_jar>
--java-home <path_to_jdk>
Description
Specifies the port to use to access AFS.
Adds the AFS client jar noarch-aster-adfsclient.jar
to the CLASSPATH on the client machine.
Sets JAVA_HOME for AFS to the specified JDK directory.
Using the "on-error-stop" Option in ACT
The “on-error-stop” option can be used to stop ACT if an error occurs while running queries.
It discontinues executing the multi-statement SQL or the SQL file.
When the “on-error-stop” option is set, ACT halts query processing and exits as follows:
• If ACT is in interactive mode, it returns to the ACT prompt when it encounters an error.
• If ACT is executed in the command line, it exits with status 3 when it encounters an error.
Note! Unless the
-f or
-c options are used, ACT is always in interactive mode once it is launched. The
\set
command can be used at the command line at any time. For more details
Aster Client Guide 79
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT on the
\set
syntax for the “on-error-stop” option, see
“Setting Database Parameters” on page 102
.
Setting "on-error-stop" on the Command Line
To enable the “on-error-stop” option (by default this option is disabled) from the command prompt, use either of the following parameters:
• --on-error-stop
• -E
For example, to enable the “on-error-stop” in a running session of ACT, type the following at the ACT prompt: beehive$ ./act -h 153.65.197.120 -U beehive -w beehive
-E -f test_queries.sql
beehive$ ./act -h 153.65.197.120 -U beehive -w beehive
--on-error-stop -f test_queries.sql
Using a Configuration File to Pass ACT Startup Parameters
Specifying all the necessary startup parameters for ACT on the command line can become cumbersome, especially when using SSL. Because of this, Teradata Aster has provided a way to specify startup parameters in a configuration file, to streamline starting ACT. On Windows and Linux/UNIX-based operating systems, ACT looks for the configuration file in a specific location and loads it automatically. Alternatively, the configuration file may be invoked using the
--config-file
parameter when launching ACT.
All of the startup parameters in ACT are supported in the config file, except for the following:
•
--config-file
•
-V [ --version ]
•
-? [ --help ]
1
2
3
To use a configuration file, first create a text file of startup parameters. The following rules apply when creating the config file:
Lines starting with a
# character are ignored (considered as comments).
Blank lines are ignored (including lines containing just spaces).
Parameters are entered using the format
4
flagname: value where flagname
is same as the name of the command line flag without the preceding hyphens (
--
) and value
is the flag value as it would be provided on the command line.
Note that the short notations of flags are not supported. For example: host: <ip> will work but the following: h: <ip> will not work.
Flags which do not take any argument on the command line should be given a value of either true
or false.
80 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Startup Parameters for ACT
5
6
7
Flag names are case-sensitive.
If the config file includes invalid flag names or repeated entries, ACT will not launch, and an error will display.
If the config file includes the on-error-stop option with the parameters set to enable this
for information on setting this option.
Setting "on-error-stop" in the ACT config file
To enable the “on-error-stop” option, add it to the ACT config file actconfig.ini
and set the value to true. Set the value to false to turn off this option. By default, this feature is disabled or “off ”.
The following is an example from of a config file for ACT:
# ACT configuration file example
# Contains setting to enable on-error-stop host: 10.10.10.10
dbname: sampledb username: beehive on-error-stop: true
The default location for the config file on Linux/UNIX-based systems is
$(HOME)/
.actconfig
. For Windows, the default location for the config file is
%HOMEPATH%/ actconfig.ini
. Upon launching, ACT will look for the config file in the default location, and if found, will use it by default.
Command line flags can be specified when starting ACT, in addition to the flags in the configuration file. If a flag is present in the config file and specified at the command line as well, the value on the command line will override the value in the config file. ACT will notify the user that the command line flag was used upon startup.
The following is an example of a config file for ACT:
# ACT configuration file example
# Contains settings for connecting securely to a specific host and database host: 10.10.10.10
dbname: sampledb username: sampleuser
# SSL settings enable-ssl: true ssl-self-signed-peer: true ssl-encrypt-reads: false
To start ACT, explicitly using the config file, issue a command like this example:
$ act --config-file /home/beehive/.act_ssl_config
To start ACT, explicitly using the config file and also specifying an additional parameter to redirect query results to a file for this session only, issue a command like this example:
$ act --config-file /home/beehive/.act_ssl_config -o /home/beehive/ query_results_file
81
Aster Database Cluster Terminal (ACT)
Using ACT
Using ACT
Issuing SQL Queries
There are two ways to use ACT to run a SQL query against Aster Database:
•
Running Queries from the ACT Prompt
•
•
Support for Multibyte Characters
To see a list of supported SQL commands in ACT, enter
\h
at the ACT prompt.
NOTE: You cannot cancel a query when it is in the prepare phase.
Running Queries from the ACT Prompt
To control how ACT handles errors when running single or multi-line SQL queries from the
command line, see “Using the "on-error-stop" Option in ACT” on page 79 .
To run a single-line SQL query from the command line:
1
2
Type the query at the ACT prompt. The query must end with a semicolon.
Press Enter.
For example, to list all the rows of the customer_dim table of the retail_sales database and order the results based on gender, enter this command at the ACT command-line prompt: retail_sales=> select customer_id, gender, city_id from customer_dim order by gender;
customer_id | gender | city_id
-------------+--------+---------
743 | F | 46
2711 | F | 124
744 | F | 66
You can also run commands that return information about the environment, such as: retail_sales=> show sessionid; which returns the session identifier for the current session.
Running Multi-Line SQL Queries in ACT
To run a multi-line SQL query from the command line:
1
2
3
Type the first line of the query, then press Enter.
The ACT prompt changes from
=>
to
->
(for example, retail_sales->
).
Enter the remaining lines.
You can press Ctrl-C at any time to abandon this command mode without executing the query.
On the last line, type a semicolon at the end of the line, then press Enter to run the query.
82 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Using ACT
For example: retail_sales=> select customer_id, gender, city_id retail_sales->from customer_dim retail_sales->order by gender; or retail_sales=> select customer_id, gender, city_id retail_sales->from customer_dim retail_sales->order by gender retail_sales->;
Running Queries from a File
To control how ACT handles errors when running SQL queries from a file, see “Using a
Configuration File to Pass ACT Startup Parameters” on page 80
.
To run a query stored in a file:
1
2
Make sure the filename ends with .sql.
The query does not have to end with a semicolon.
At the prompt, enter this command:
\i <
filename
>
For example, consider this query stored in the file myQuery.sql
:
SELECT *
FROM customer_dim
ORDER BY gender
To run this query, enter this command at the ACT command prompt: retail_sales=> \i myQuery.sql
You can also run this query using the
–f
flag of the ACT command. For example:
[root@localhost ~]# act -d retail_sales -w beehive -f myQuery.sql
Support for Multibyte Characters
ACT 5.10 and later accepts multibyte characters from the SQL prompt, as shown in this example:
83
Aster Database Cluster Terminal (ACT)
Using ACT
Exit ACT
To quit ACT, type
\q
and hit <Enter>.
Page Through Query Results
When ACT returns results to the screen, it (by default) prints one page of results at a time. Hit the space bar to display the next page of results, hit <Enter> to display just one more line of results, and type “q” to quit looking at results and return to the SQL prompt. When you get to the last page of results, ACT displays the text, “(END)”. Type “q” to return to the SQL prompt.
Throttle Query Results in ACT and Aster Database
To reduce the memory footprint of ACT and other Aster Database clients, you can set a fetchcount that constrains the number of rows returned at one time when you select from a large table. You can limit the total number of rows returned by using fetch-limit. Additionally, you can make your query results stream more efficiently by using the server-side cursors feature.
In ACT, you can set these parameters using
\set
, and in other clients you can typically set them in your data source definition or parameters file.
Let’s look at fetch-count first.
Reduce Memory Use With Fetch-Count
The purpose of using fetch-count is to reduce the memory footprint of ACT on the server.
Therefore, this type of configuration would normally be done by an administrator, or at least the setting would be made under an administrator’s guidance. To set a fetch-count, you specify a value for the fetch-count parameter in ACT. This causes ACT to fetch rows in sets, with each set containing only the user-specified number of rows (or fewer rows).
ACT uses a fetch count by default, i.e. even when fetch-count is not set explicitly. The fetchcount (number of rows per fetch) should always be set to greater than 0, The default value is
1024 rows.
To set the fetch-count in a running session of ACT, use the
\set
command to set the fetchcount
parameter. To do this, type the following at the ACT prompt:
\set fetch-count n where n is the maximum number of rows ACT should return at a time.
To enforce the fetch-count, ACT uses server side cursors to fetch results, which can help prevent the memory footprint of ACT from growing too large.
Note that to the user, the results returned will not be different when using fetch-count. The purpose is simply to reduce the memory footprint of ACT on the server.
Limit Caching With Server-Side Cursors
Setting use-server-cursors has the same effect as declaring a server-side cursor with the
DECLARE CURSOR syntax in SQL. When server-side cursors are activated, one batch of data is retrieved at a time. This lets the queen avoid caching the entire result set and lets the workers continue performing computations while the queen is retrieving rows. As the result set becomes available from a worker, the queen sends it to the client.
84 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Using ACT
Depending on the type of query, results can start streaming back to the client immediately
(e.g. SELECT * WHERE...) or the results may have to be computed in their entirety (e.g.
GROUP BY) before streaming to the client. Regardless of whether the results begin to stream to the client before the query has finished executing, server-side cursors can improve performance significantly for large result sets. If you want to start streaming results of a query as soon as possible, you can benefit from setting use-server-cursors and specifying a fetchcount. By default, use-server-cursors is set to 0 (off).
Example of Fetch-Count With Server-Side Cursors
The following steps are an example of how to use server-side cursors and fetch-count together:
1
Enable server cursors by issuing the following command inside ACT (a setting of 1 turns cursors “on” and a setting of 0 means “off ”):
2
3
beehive=> \set use-server-cursors 1
Set a fetch-count of 100: beehive=> \set fetch-count 100
Issue the following SQL query:
SELECT A,B,COUNT(*) FROM lineitem GROUP BY a, b;
This will actually cause the following statements to be run:
BEGIN;
DECLARE x CURSOR FOR SELECT A,B,COUNT(*) FROM LINEITEM GROUP BY A,B;
FETCH 100 FROM x; // 100
FETCH 100 FROM x; // 200
...
...
FETCH 100 FROM x; // until all the rows have been fetched
CLOSE x;
END;
Aster Database reports all the fetched rows as the row count for this query.
Set the Maximum Number of Rows Returned With Fetch-Limit
To limit the total number of rows returned by each query, set the fetch-limit in ACT. fetchlimit (the total number of rows fetched for a query) can be set to any value. A value less than 0 implies fetch all rows. A value greater than 0 implies fetch-limit rows in total. The default value is -1 (all rows).
Tip!
When fetch-limit is used, the total row count returned for the query will be the total row count returned by the query or the row count specified by fetch-limit, whichever is smaller. For example, if a query normally returns 35,453 rows, but you have specified a fetch limit of 1000, the query will return 1000 rows (and it will display “1000 rows returned”). There will be no indication that there were in fact 35,453 rows that would have been returned had you not had a fetch limit set.
In ACT, you set the fetch-limit by typing:
\set fetch-limit n where n is the maximum number of rows a single query will be allowed to return.
85
Aster Database Cluster Terminal (ACT)
Using ACT
The SQL LIMIT Clause vs. Fetch-Limit and Fetch-Count
Setting a fetch-limit is an alternative to specifying a LIMIT clause explicitly in your SELECT query, with an important difference in how the query will be executed, which can affect performance. fetch-limit is an ACT feature which controls how many rows are fetched by the queen from the workers and by the client from the queen. ACT stops fetching results from the cursor after the specified limit is reached. LIMIT is a SQL clause which controls how many rows are computed on the worker(s) or the queen.
The following examples illustrate the difference between using fetch-limit and using a LIMIT clause in the SQL query:
Consider the statement:
SELECT * FROM FOO, BAR where FOO.A = BAR.A;
Example 1: Using fetch-limit and fetch-count
To issue the query in ACT with a fetch-count of 100 and a fetch-limit of 1000:
\set fetch-count 100
\set fetch-limit 1000
SELECT * FROM FOO, BAR where FOO.A = BAR.A;
Here is how the query will execute:
1
2
ACT opens a cursor on the query.
The queen activates Parallel Cursors and passes the query
3
4
5
SELECT * FROM FOO, BAR where FOO.A = BAR.A; to the workers.
Each worker then computes the entire equi-join as dictated by the
FOO.A = BAR.A
constraint.
The queen fetches results from workers in batches of 100 rows, until a limit of 1000 is reached.
ACT fetches results from the queen in batches of 100 rows, until a limit of 1000 is reached.
Example 2: Using a LIMIT clause in the SQL query
To issue the same query in ACT using the SQL LIMIT clause to limit results to 1000 rows:
SELECT * FROM FOO, BAR where FOO.A = BAR.A LIMIT 1000;
Here is how the query will execute:
1
2
ACT issues the SQL statement to the queen.
The queen planner identifies
LIMIT 1000
and pushes down the following query to each of the workers:
SELECT * FROM FOO, BAR where FOO.A = BAR.A LIMIT 1000;
3
Each worker computes the equi-join as dictated by the constraint. This allows for optimizations where the entire equi-join computation need not be done at each worker.
FOO.A = BAR.A LIMIT 1000
4
The queen fetches results from the workers.
86 Aster Client Guide
Aster Database Cluster Terminal (ACT)
Using ACT
5
ACT fetches results from the queen.
The difference between the two approaches occurs in step 3 of both examples. Using the SQL
LIMIT clause (Example 2), the workers are allowed to compute with the constraint of
LIMIT
1000
, whereas in Example 1 they have to compute the entire equi-join.
So, as you can see, fetch-limit via server-side cursors does not translate into the workers doing a
LIMIT 1000 on their individual slice of data. Therefore, if the use case calls for it, an Aster
Database power-user should be aware that using the SQL LIMIT clause can speed up query execution dramatically in Aster Database.
ACT Utility Commands
ACT also offers many utility commands that provide assistance and carry out useful utility operations such as uploading files and changing output options. These commands start with a
backslash character, and we describe them in the section, “ACT Commands (at the SQL
.
Repeat Previously Typed Commands
Hit the up arrow on your keyboard to toggle through your history of previously typed commands. To edit a previously typed command, just use the usual combination left arrow, right arrow, delete, and backspace keys to make your edits, and type <Enter> to finish editing the line. If needed, type “;” to issue the command. To list your command history on Linux, type
\s
.
Tip!
If you receive an “Error writing history to file.” error on Linux when attempting to view command history with \s, check that the current Linux user has permissions to write to the current working directory.
Tab Completion
The UNIX/Linux version of ACT can tab-complete SQL commands and table names that you type. Tab-completion is not available in the Windows version of ACT.
To use this feature, type the first couple of letters of a command and hit the <Tab> key. If the completion is unambiguous, ACT completes the command. If ACT doesn’t complete the command, hit <Tab> again and ACT prints all the possible completions. Using the list as a reference, type enough additional characters to unambiguously identify the desired command or table, and hit <Tab> again to complete it. Here are a few common uses of tab completion:
• To complete common SQL commands. For example, type “se” and hit <Tab> to type
SELECT.
• To list various ACT utility commands. For example, type
\ and hit <Tab> to show all the commands, or type
\d
and hit <Tab> to show all the commands that start with “d”.
• To complete a table name. For example, type “SELECT * FROM sa” and hit <Tab> to complete the table name or hit <Tab> twice show all the table names that start with “sa”.
You can also list the names of all tables in the database by typing “SELECT * FROM ” (note the trailing space) and hitting <Tab> twice.
Aster Client Guide 87
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
ACT Commands (at the SQL Prompt)
The ACT SQL prompt accepts a number of ACT-specific commands that you issue by typing a backslash followed by a character or combination of characters and arguments. Do not type a semicolon to conclude these commands! Hitting <Enter> executes the command.
Note! Don’t confuse these with the ACT command-line startup parameters (also known as
“command line flags”), which you type at the shell command line when you launch ACT. For
a list of those, see “Startup Parameters for ACT” on page 76 .
We describe the SQL-prompt commands in the tables that follow:
•
General Purpose Utility Commands in ACT
•
•
•
Input/Output-Related Commands in ACT
•
Installed Function and Installed File Management Commands in ACT
•
•
Formatting-Related Commands in ACT
Table 3 - 8: General Purpose Utility Commands in ACT
Command
\?
\c[onnect] <dbname>
<user> <host> <port>
\c[onnect] <dbname>
<user> <host>
\c[onnect] <dbname>
<user>
\c[onnect] <dbname>
\cd [<dir>]
\copyright
\h
\h <SQL_command>|*
\g
\password
\q
\! [<command>]
Description
Prints help for ACT commands.
Changes login credentials and/or connects to a new database.
The parameters must be specified in the order shown, with a space before each, and parameters may not be skipped. In other words, if only one parameter is specified, it is understood to be DBNAME; if a second parameter is also specified, it is understood to be USER; and so on.
Changes the current working directory.
Shows ACT usage and distribution terms.
Provides help for SQL commands.
Provides help with syntax for the specified SQL command, * for all commands.
Executes the query. Alternatively, terminate the query with a semicolon (;).
Changes the password for the current user.
Quits ACT.
Executes the command in a shell or starts an interactive shell.
88 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 9: Environment Settings in ACT
Command
\info
\pager [on|off]
\set
\set <param_name>
[<param_value>]
\timing [on|off]
Description
Displays the current environment settings.
Toggles or sets ACT to use the pager to enable paging through large result sets.
Displays the current ACT parameter settings.
Sets the ACT parameter setting <param_name> to the value
<param_value>
. (For example, “\set fetch-count
500
” tells ACT to fetch no more than 500 rows at a time when selecting.) If no parameter value is supplied, displays the current setting for the specified parameter.
Toggles or sets the display for the timing of commands on or off. Note that the output does not include the I/O time.
To view the total query execution time including the output formatting and printing time, run ACT with the UNIX time command (using the ACT command line -c or -f flag).
Table 3 - 10: Query Buffer Commands in ACT
Command
\e [<file>]
\g [<file>]
\p
\r
\w <file>
Description
Edits the query buffer (or file) with an external editor. On most systems, this launches your default text editor. When you save and exit the editor, the edited statement is passed back to ACT for running.
Sends the query buffer to the server (and optionally writes the results to file or | (pipe character)).
Shows the contents of the query buffer.
Resets (clears) the query buffer.
Writes the query buffer to a file.
Table 3 - 11: Input/Output-Related Commands in ACT
Command
\echo
<string>
\i <file>
\o <file>
\o
\s [<file>]
Description
Writes a string to the query output stream (see \o below).
Executes SQL commands from a SQL script file. (Runs a SQL script.)
Redirects all query results to a file or | (pipe character).
Stops sending results to a file and resumes sending them to the ACT shell.
Displays the command history in Linux (optionally, prints the history to a file specified by <file>) Note that query history includes only the first 2048 characters of each query.
89
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 12: Installed Function and Installed File Management Commands in ACT
Command
\dF
\dF+
\dE
[<schema_pattern>].
[<function_pattern>]
Description
Lists installed files, SQL-MapReduce functions, and other functions in the current schema. Use a regular expression as an argument to display a subset of the available functions. For example, to view all installed functions in the database, issue:
\dF *.* where the first asterisk means "all schemas" and the second means
"all functions and files."
Shows details for all installed files, SQL-MapReduce functions, and other functions in the current schema. For each function, the output shows the name, schema, owner, upload time, and MD5 Hash fingerprint of the function.
Use a regular expression as an argument to display a subset of the available functions. For Example, type \dF+ *.* to show details for functions and files in all schemas in the database.
Shows all the installed SQL-MR functions for which the current user has privileges. Use a regular expression as an argument to display a subset of the available functions. Shows function name, schema, owner, function version and creation time.
90 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 12: Installed Function and Installed File Management Commands in ACT (continued)
Command
\dE+
[<schema_pattern>].
[<function_pattern>]
Description
Shows details about all the installed SQL-MR functions for which the current user has privileges. Optionally specify schemas and/or functions to display. You may use a regular expression when specifying the schema and function. If no schema is specified, defaults to the current user’s default schema.
For each matched function, ACT outputs following information:
8
9
2
3
File Name: As-installed name of the file.This is the name you use to access or manage the file with the install, uninstall, and download commands in Aster Database.
1
API Version: Shows the API version used to create the function.
Runner Kind: The kind of function: java or c.
Usage Syntax: Shows usage syntax.
4
Brief Description: Shows a short description of the function.
5
Detailed Description: Shows a long description of the function.
6
Input Columns: Shows input columns the function accepts.
7
Output Columns: Shows output columns the function returns.
Author: Shows the author of the function.
Interfaces Implemented: Row function/Partition function/
Multiple inputs/Graphic function
If the specified pattern matches more than one function, ACT will output the results for each matched function in a list. When running
\dE+
in the ACT shell with multiple functions, ACT will page the output.
91
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 12: Installed Function and Installed File Management Commands in ACT (continued)
Command
\install <file>
[[<schema>]
<file_alias>]
\download
[[<schema>]
<file_alias>] <file>
\remove
[[<schema>]
<file_alias>]
Description
Installs the file or SQL-MapReduce function in Aster Database. The file must be available on the file system where ACT is running. Note that the database user running this command must have permission to install files and functions in the specified schema.
The install file name and the schema name cannot be the same. An error will result if the names are the same.
You cannot install two files or functions with the same name. If attempting to do this, you must follow these steps:
• remove the existing file or function
• install the new file or function
• grant the appropriate privileges on the file or function.
There is a limit of 238MB on the size of the file to be installed. If you try to install a larger file, you will see an error like:
ERROR: row text exceeds limit of 238MB ...
Note that when installing larger files, the queen may run out of memory. The queen needs available memory of approximately eight times the size of the file to be installed, in order to encode, buffer and copy the file.
Downloads the specified installed file or function (identified by its
<file>
or <file_alias>) to the machine where ACT is running. Note that the database user running this command must have permission to download files and functions from the specified schema.
Removes from the cluster the file or SQL-MapReduce function specified by its <file_alias>. Note that the database user running this command must have permission to remove files and functions from the specified schema.
Table 3 - 13: Informational Commands in ACT
Command
\d
\d [<pattern>]
\dt
\dt [<pattern>]
\dv
\dv [<pattern>]
\di
Description
Lists all tables, indexes and views in the current schema.
Describes table(s) or index(es).
Lists all tables in the current schema.
Prints the schema, name, type, and owner of a table or tables. To see tables in a custom schema, type \dt
<schemaname>.*
Lists all views in the current schema. .
Describes view(s).
Lists all indexes in the current schema. .
92 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
ACT Commands (at the SQL Prompt)
Table 3 - 13: Informational Commands in ACT (continued)
Command
\di [<pattern>]
\dg
\dg [<pattern>]
\du
\du [<pattern>]
\dn
\dn [<pattern>]
\l
\extl host=<hostname>
[<option_name>=
<option_value>, …]
\extd host=<hostname> database=<dbname>
[<option_name>=
<option_value>, …]
Description
Describes index(es).
Lists groups and roles.
Describes group(s).
Lists users.
Describes user(s).
Lists schemas.
Describes schema(s).
List all databases.
Lists all databases on an external Hadoop systems.
Lists all tables in a database or describes a table on an external Hadoop system.
Tip!
ACT uses the schema search path ( search_path
) for the database user when displaying lists of tables, veiws and indexes. The schema search path defaults to the schema search path for the current user in the database.
To set the search_path
from ACT, issue the following command: beehive=> SET session search_path TO <schema>;
Note that multiple schemas are not supported. If multiple schemas are listed in the schema listed will be used.
To display the current search_path
type: search_path
, the first beehive=> SHOW search_path;
Note that you may also set the search_path
on the server.
Alternatively, you can specify the schema to use when issuing commands by following the command with a schema qualified reference. This example shows how to display information on all tables in the schema “myschema”:
\dt myschema.*
Table 3 - 14: Formatting-Related Commands in ACT
Command
\a
\f [<string>]
\t [on|off]
Description
Toggles between unaligned and aligned output modes.
Shows or sets the field separator for unaligned query output.
Shows only rows (off by default).
93
Aster Database Cluster Terminal (ACT)
Aster File Store Commands
Table 3 - 14: Formatting-Related Commands in ACT (continued)
Command
\x [on|off]
Description
Sets or toggles expanded output mode ON and OFF. With expanded output mode turned on, each record is split into rows, with one row for each value, and each new record is introduced with a text label in the form, ---[ RECORD 37 ]---. This can help make wide tables readable on a small screen, and is very useful if you’re trying to read
EXPLAIN output. Note that in expanded mode, the number of rows is not returned at the end of the table. Because of this, when querying a table with no rows, you will simply see the ACT prompt again.
Table 3 - 15:
Command
\afs
<afs_command>
<command_param>
\aport
[<afs_port>]
Description
Operates on Aster File Store (AFS). For a list of available commands see
“Aster File Store Commands” on page 94 .
Shows and/or sets the AFS port.
Aster File Store Commands
ACT provides commands for interfacing with the Aster File Store (AFS). You specify these commands using the syntax: beehive=> \afs <afs_command> <command_param>
All of these AFS commands behave similarly to the corresponding UNIX commands; the command descriptions show where behavior is different. Error information is sent to stderr and the output is sent to stdout. Note that these commands must be issued outside of a transaction block (i.e. not between BEGIN and END.)
Before you can use these commands, you’ll need to set up your ACT environment to work
with AFS. To do this, see “Configuring ACT for the Aster File Store” on page 22 .
Specifying a URI or Path
Many of the commands use the argument
<path>
, which represents a Uniform Resource
Identifier (URI) for a file or directory. A URI or path, has the form:
<protocol>://[<host>[:<host_port>]]/<path>
Some valid URI examples for
<path> are:
• adfs://aster-afs/file2
Specifies a file in the root directory of the user in AFS.
• adfs:///file2
Specifies a file in the root directory of the user in AFS.
• adfs:///user/mydirectory
Specifies a directory in AFS.
• file:///myimage.gif
Specifies a file on the local file system.
94 Aster Client Guide
Aster Database Cluster Terminal (ACT)
Aster File Store Commands
•
/user/content.json
Specifies a file in the /user directory (adfs is implied).
If a protocol is not specified adfs
is implied and a URI is expected. For local files, and where a local file is not the only option allowed, the protocol must be specified as file://
. If a URI is supplied, and a host/port pair is not provided, ACT assumes that the client is connecting to a service running on the Aster Database queen that was specified when ACT was invoked.
AFS Command Reference
In the table of AFS commands, an ellipses (
...
) indicates that the prior construct may be specified multiple times. For example
<src> ...
indicates that you may specify as many sources as you like.
Table 3 - 16: AFS Commands in ACT
Parameter
\afs -cat <src>
Description
Fetches all files that match the file pattern <src> and displays their content on stdout.
\afs -chgrp [-R]
<group> <path> ...
\afs -chmod [-R]
<mode>[,<mode>] ...
| <octalmode>
<path> ...
Changes the group that owns a directory or file (recursively if -R is specified).
Changes the permissions on a file or directory (recursively if -R is specified).
<mode>
is same as the modes used for the chmod shell command. It takes the form <role> +|-|= <permission>.
Available <role> options are:
• u for user/owner
• g for group
• o for others
• a for all
If no role is specified, a is assumed, and unlike the shell command, no umask
is applied.
Available <permission> options are:
• r for read
• w for write
• x for execute
Use <octalmode> to specify the mode using three digits. These follow the same rules as the shell command, but unlike the shell command, this usage requires all three digits.
For example. specifying 754 is the same as u=rwx,g=rx,o=r
NOTE: Only octal mode is supported for the Windows 32-bit, Solaris x86, and Solaris SPARC platforms.
Aster Client Guide 95
Aster Database Cluster Terminal (ACT)
Aster File Store Commands
Table 3 - 16: AFS Commands in ACT (continued)
Parameter Description
\afs -chown [-R]
[<owner>]
[:[<group>]]
<path> ...
\afs -copyFromLocal
<localsrc> ... <dst>
\afs -copyToLocal
[-ignoreCrc] [-crc]
<src> <localdst>
Changes the owner (and optionally the group) of a file or directory
(recursively if -R is specified). If only the owner or group is specified, then it alone will be modified.
Note that owner and group names are case sensitive and may only consist of alphanumeric characters, and any of -_.@/.
Avoid using '.' to separate owner name and group. If owner names have dots in them and you are using a local file system, you might see unexpected results since the shell command 'chown' is also used for local files.
Identical to the -put command. Copies file(s) from the local file system into AFS.
Identical to the -get command. Copies all the files that match the source file pattern to the local destination specified. Use the -crc option to perform a checksum (cyclic redundancy check) on the file before copying or the -ignoreCrc option to not perform the check.
The local destination must be a directory. The source file(s) are retained.
\afs -count[-q]
<path>
Counts the number of directories, files and bytes under the paths that match the specified file pattern.
The output columns are:
• DIR_COUNT
• FILE_COUNT
• CONTENT_SIZE
• FILE_NAME
When including the -q option, the output columns are:
• QUOTA
• REMAINING_QUOTA
• SPACE_QUOTA
• REMAINING_SPACE_QUOTA
• DIR_COUNT
• FILE_COUNT
• CONTENT_SIZE
• FILE_NAME
Note that for quota values, the maximum space available will be shown, because AFS does not support quotas per user.
\afs -cp <src> <dst>
Copies all files that match the file pattern <src> to a destination.
When copying multiple files, the destination must be a directory. Also, when copying multiple files, the last file is copied to the destination directory. For example:
\afs -cp source1/file_1 /source2/file_1 /test/
/source2/file_1
is copied to the destination directory /test/.
96 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Aster File Store Commands
Table 3 - 16: AFS Commands in ACT (continued)
Parameter Description
\afs -du <path>
\afs -dus <path>
\afs -expunge
Shows the amount of space, in bytes, used by the file specified. If a directory is specified, shows the sizes of files and directories it contains.
Equivalent to the UNIX commands:
• du -sb <path>/* in case of a directory
• du -b <path> in case of a file.
The output is in the form name (full path) size (in bytes).
Shows a summary of the amount of space, in bytes, used by the files that match the specified pattern. Equivalent to the UNIX command du
-sb
. The output is in the form name (full path) size (in bytes).
Cleans up the trash.
\afs -get
[-ignoreCrc] [-crc]
<src> <localdst>
Copies all the files that match the source file pattern to the local destination specified. Use the -crc option to perform a checksum
(cyclic redundancy check) on the file before copying or the
-ignoreCrc
option to not perform the check. The local destination must be a directory. The source file(s) are retained.
\afs -getmerge <src>
<localdst> [addnl]
Gets all the files that match the source file pattern and merges and sorts them into a single file on the local file system. Specify the addnl flag to insert a line break between the contents of each file in the output file.
The source files remain as they are.
\afs -help
[<afs_cmd>]
Displays help for the given command or all commands if none is specified.
\afs -ls <path> •
Lists information about files that match the specified pattern.
If a path is not specified, the contents of
/user/
<currentUser>
will be listed. Directory entries are of the form dirName (full path)
<dir>
and file entries are of the form fileName(full path)
<r n> size where n is the number of replicas specified for the file and size is the size of the file, in bytes.
\afs -lsr <path>
\afs -mkdir <path>
Recursively lists the information about files that match the specified pattern. Behaves very similarly to Hadoop fs -ls, except that the data is shown for all the entries in the subtree.
Creates a directory in the specified location(s). Similar to UNIX mkdir -p
in that it will create parent directories along the path.
\afs -moveFromLocal
<localsrc> ... <dst>
Same as -put, except that the local source file(s) are deleted after they are copied.
\afs -mv <src> <dst>
Moves files that match the specified file pattern <src> to a destination. When moving multiple files, the destination must be a directory. Also, when moving multiple files, the last file is moved to the destination directory. For example:
\afs -mv source1/file_1 /source2/file_1 /test/
/source2/file_1
is moved to the destination directory /test/.
\afs -put <localsrc>
... <dst>
Copies file(s) from the local file system into AFS.
97
Aster Database Cluster Terminal (ACT)
Aster File Store Commands
Table 3 - 16: AFS Commands in ACT (continued)
Parameter Description
\afs -rm
[-skipTrash] <path>
\afs -rmr
[-skipTrash] <path>
\afs -set
[property[=value]]
\afs -setLogLevel category[=logLevel]
\afs -setrep [-R]
[-w] <rep> <path>
\afs -stat
[<format>] <path>
\afs -tail [-f]
<file>
Deletes all files which match the specified pattern. Equivalent to the
UNIX command rm <src>. Use the -skipTrash option to bypass the trash and immediately delete <src>.
Recursively deletes all directories which match the specified pattern.
Equivalent to the UNIX command rm -rf <src>. Use the
-skipTrash
option to bypass the trash and immediately delete
<path>
.
Sets the specified Java property to the specified value on the client. If only the property is specified, shows its value. If no property is specified, displays all properties and their values.
Valid properties are listed in “Java Properties for AFS Clients” on page 98
.
Defines the minimum set of levels recognized by the system. The levels are OFF, FATAL, EROR, WARN, INFO, DEBUG, TRACE, or ALL. The default level is ERROR.
Sets the replication level of a directory or file (recursively if -R is specified).
Prints statistics about the file or directory at <path> in the specified format. Format accepts filesize in blocks (%b), filename (%n), lock size
(%o), replication (%r), modification date (%y, %Y).
Shows the last 1KB of the file. The -f option shows appended data as the file grows.
\afs -test -[ezd]
<path>
Tests the properties of a file or directory and returns 0 if the test fails or
1 if it succeeds. Specify properties as one or more arguments:
• -e tests if the file exists.
• -z tests if the file has zero length.
• -d tests if the file is a directory.
If no options are specified, tests for all (i.e. if the file exists, has zero length, and is a directory then returns 0, else returns 1).
\afs -text <src>
Takes a source file and outputs the file in text format. The allowed input formats are .zip and TextRecordInputStream.
\afs -touchz <path>
Creates a file of zero length with a timestamp in the format yyyy-MMdd HH:mm:ss at <path>. If the file already exists with a non-zero length, an error is returned.
Java Properties for AFS Clients
You can set the Java properties for the ACT client to use when accessing AFS using the ACT
\afs -set
command.
\afs
shows what is explicitly set. Defaults may exist, but will not show.
98 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Aster File Store Commands
You can set the following properties using this command:
Table 3 - 17: Java Properties for AFS Clients
Property
Basic Client Options
com.asterdata.aster.
adfs.default_mb
Default Value
2048 com.asterdata.aster.
smore.localtmpfile
com.asterdata.aster.
smore.no_verify_crc
com.asterdata.aster.
janus.serverPort
com.asterdata.aster.
aftp.rpcTimeoutSecs
com.asterdata.aster.
janus.rpcTimeoutSecs
com.asterdata.aster.
smore.rpcTimeoutSecs
com.asterdata.aster.
adfs.rpcTimeoutSecs
com.asterdata.aster.
adfs.split_write_buf_size
false false
2601
180
30
300
300
524288
Notes
Dictates how many splits should be reserved when a new file is created
(based on the file size and the size of the splits). This does not limit the size of the files as more splits will be requested as needed.
When set to "true", enables writing split streams to a local temporary file. This adds some overhead but allows for recovery during write errors.
When set to "true", disables checksum verification of each split.
The Hadoop FileSystem setVerifyChecksum
method can be used to configure this value as well.
Sets the Janus server port; typically, the port will be dictated by the
URL.
Sets the default RPC timeout for
AFTP (file transfer) operations.
Sets the default RPC timeout for configuration server operations.
Sets the default RPC timeout for object store operations.
Sets the default RPC timeout for
AFS operations.
Sets the value that is passed as the buffer size and overrides com.asterdata.aster.
smore.write_buf_size
in the case when smore objects are stored as splits. Use this value to modify the setting for AFS purposes, not the smore value, since smore can be used for purposes other than
AFS files.
Increasing this value to 1MB or higher may require a larger heap size. Sizes above 1MB or even
512KB yield no real benefit.
99
Aster Database Cluster Terminal (ACT)
Aster File Store Commands
Table 3 - 17: Java Properties for AFS Clients (continued)
Property
com.asterdata.aster.
adfs.split_read_buf_size
Default Value
524288 com.asterdata.aster.
adfs.expose_uri
false com.asterdata.aster.
adfs.set_credentials_config
Java Runtime Operators
user.name
java.io.tmpdir
JVM Settings
-XX:Use<gc>GC false
Notes
Similar to
.split_write_buf_size
but for reads.
A workaround for a problem with hadoop fs -count
where some calls fail with a permissions issue because the client doesn’t return the username and password in requests. If you encounter this issue, using this setting is a workaround.
) such that new instances from that configuration will use the same credentials. A different workaround for the issue described in com.asterdata.
aster.adfs. expose_uri
.
The default username will be used if not specified.
When com.asterdata.aster.
smore.localtmpfile
is set to true, this value dictates where the local file is written. See this Java documentation link
for information on the typical values for this property.
Sets the Garbage Collection (GC) method. The following values were tested:
• -XX:+UseSerialGC
• -XX:+UseParallelGC
• -XX:+UseConcMarkSweepGC
In some cases (like VMs used for test/dev systems), Parallel yielded better results than Serial, but in higher end hardware no real difference was found.
100 Aster Client Guide
Aster Client Guide
Aster Database Cluster Terminal (ACT)
Aster File Store Commands
Table 3 - 17: Java Properties for AFS Clients (continued)
Property
-Xmx<size>
Hadoop Configuration Options
fs.default.name
fs.adfs.block.size
fs.adfs.impl
Default Value Notes
Sets the memory size. The following values were tested:
• 512MB
• 1GB
• 2GB
Setting this to 512MB is recommended unless Out of
Memory errors are encountered or if com.asterdata.aster.adfs
.split_write_buf_size
or com.asterdata.aster.adfs
.split_read_buf_size
are set to a higher value.
Set this to "adfs://<queen>/" to change the default hadoop filesystem to adfs.
67108864 (64M)
Sets the size to make each split in a file. The following values were tested:
• 32MB
• 64MB
• 128MB
64MB was found to be optimal in basic tests.
Set this value to direct Hadoop to the Aster File Store implementation. The value must be set to: com.asterdata.aster.
adfs.fs.ADFS
Note that the "adfs" part of the Java path must match the scheme in the
URI (adfs://...).
db_superuser fs.adfs.username
fs.adfs.password
fs.adfs.umask
RPC Operational Options
com.asterdata.aster.
adfs.operation_retries
0022
2
Similar to dfs.umask/ dfs.umaskmode
is given.
.
All operations use this default unless a command-specific setting
101
Aster Database Cluster Terminal (ACT)
Setting Database Parameters
Table 3 - 17: Java Properties for AFS Clients (continued)
Property
com.asterdata.aster.
adfs.operation_timeout_secs
Default Value
120 com.asterdata.aster.
smore.operation.retries
2 com.asterdata.aster.
smore.operation.timeout_secs
120
Notes
All operations use this default unless a command-specific setting is given.
All operations use this default unless a command-specific setting is given.
All operations use this default unless a command-specific setting is given.
Setting Database Parameters
You can use the
\set
command to set Aster Database parameters from ACT. The syntax to use the
\set
command is:
\set [ <name> [ <value> [ ... ] ] ]
These are the database parameters you can set from ACT:
Table 3 - 18: Database parameters
Parameter
auto-commit [1|0] fetch-count [int] fetch-limit [int] use-server-cursors
[1|0]
Description
When set to 1 (on, the default), each SQL command is automatically committed upon successful completion. When set to 0 (off), you may manually commit your changes after each transaction or series of transactions by issuing the COMMIT command, or undo changes by issuing ROLLBACK. If you do not issue the COMMIT command, all transactions that occurred since the last COMMIT will rollback automatically.
To limit the number of rows returned at a time. ACT uses a fetch count by default (i.e. even when fetch-count is not set explicitly.) The fetch-count
(number of rows per fetch) should always be set to greater than 0, The default value is 1024 (1024 rows).
To set the maximum number of rows returned per query. A value less than 0 implies fetch all rows. A value greater than 0 implies fetchlimit
rows in total. The default value is -1 (all rows).
When set to 1, sets the server to use cursors (useful when the result set is very large). When set to 0 (the default, off), sets the server to not use cursors.
102 Aster Client Guide
Aster Database Cluster Terminal (ACT)
Troubleshooting ACT
Table 3 - 18: Database parameters (continued)
Parameter Description
on-error-stop [1|0]
By default, this feature is disabled or set to off = 0.
When set to 1 (or “on”) ACT will stop and exit if it meets an error during SQL query processing.
The following are ACT exit messages:
• EXIT_SUCCESS = 0 means ACT finished processing normally.
• EXIT_FAILURE = 1 means an error occurred, such as "file not found" in the “-f ” option.
• EXIT_USER = 3 means an error occurred in a sql script and the option “on-error-stop” was on or enabled.
Troubleshooting ACT
ACT Connection Hangs When Using SSL
When ACT is configured to use SSL, the queen must also be set up to use SSL. If the queen is not setup for SSL and ACT tries to connect using SSL, the connection will fail. An error is not returned; the ACT client just hangs. If you experience this, check to ensure that SSL settings on the queen and in ACT match.
Invalid User Name Error in ACT After Password Change
When changing the user’s password using
ALTER USER <username> PASSWORD
'<new_password>';
in ACT or in a SQL script running through ACT, you will see an ALTER
USER successful message returned. However, ACT does not change the cached password for the user. Because of this, if you then connect as the user and issue another query, you may see an error related to invalid username and password. This is because ACT is using the cached username and password when attempting to establish a new session.
If you encounter this, change the password using the
\password
command in ACT instead. If you change the password using
\password
, your ACT connection will continue to work.
Misleading Error Message Reports Problem With a Role Instead of With a
User
When attempting to connect to a database as a user who does not have connect privileges on that database, the error incorrectly reports that the problem is with a role instead of with the user. This example illustrates the error message: beehive=> create user kris with password 'beehive';
CREATE USER beehive=> create database foo;
CREATE DATABASE beehive=> \c foo kris;
Password for "kris": act: ERROR: role "kris" does not have connect privileges on this
database.
Aster Client Guide 103
Aster Database Cluster Terminal (ACT)
Troubleshooting ACT
Previous connection kept
104 Aster Client Guide
CHAPTER 4
Connect Using Database Drivers
Aster Database provides Open Database Connectivity (ODBC) and Java Database
Connectivity (JDBC) drivers for connecting business intelligence (BI) tools. This section explains how to install and use the Aster Database drivers.
General tips:
•
General Tips for Connecting Clients to Aster Database
Setting up database drivers:
•
•
•
Tools for .NET Environments : .NET Data Provider for Aster for use with SSIS, SSAS,
SSRS, and .NET
Using database drivers:
•
Process SQL Statements in JDBC
•
JDBC Troubleshooting and Limitations
•
Configure Aster Database SQL Settings
Connecting BI and query tools:
•
Connect Reporting Tools to Aster Database
General Tips for Connecting Clients to Aster
Database
Recommended Character Set Is UTF-8
For all tools that you use to connect to databases in Aster, you will typically want to set the default character set to UTF-8. This is particularly important if you plan to use special characters (for example, German letters ä and ß) in a char, varchar, or text column, and it is particularly important if you are connecting from a Windows-based machine.
For example, if you will use an SSH client (for example, putty) to run ACT or ncluster_loader, make sure you set the SSH client’s default character set to UTF-8.
Aster Client Guide 105
Connect Using Database Drivers
ODBC Driver
Supported Encoding
For a list of encodings supported by Aster ODBC, see “Encoding Support” on page 215 .
When Querying System Tables with ODBC, Set AUTOCOMMIT to 'OFF'
Cursors cannot be declared and used with Aster Database system tables. The default mode of autocommit-on with the ODBC driver declares cursors. Therefore, the ODBC driver should always query system tables by turning AUTOCOMMIT to ' off
'.
ODBC Driver
Using an ODBC Configuration File or Connection String
To construct the ODBC connection string or to edit the configuration file odbc.ini
, see
“Installing and Configuring ODBC” on page 23
.
Note that if you specify parameters in the connection string, these will override what is in the
ODBC configuration file.
Enable Authentication Cascading
To enable ODBC to take advantage of authentication cascading, you will use the following parameters in the connection string to configure the ODBC driver to connect to the Aster
Database:
Table 4 - 19: Parameters for the ODBC driver
Parameter
SERVER
Driver
DSN
Database Name
Port Number
Required? Description
Required The exact IP address of the Queen.
Optional Specifies the driver to use. If passing this parameter, specify {AsterDriver}
You must include either the Driver or the DSN.
Optional Specify your DSN as specified in the ODBC configuration file odbc.ini. This example uses the
DSN Beehive: DSN=Beehive;
You must include either the Driver or the DSN.
Optional If not supplied, the system will re-connect to the database you are currently logged into.
Optional Will connect to default port: 2406
1
First, construct the connection string. You will specify AUTOAUTHENTICATE=true in the ODBC connection string as follows:
• If using the Aster Driver: sprintf((char*)connect_str,"Driver={AsterDriver};SERVER=xxx.xxx.xxx.xxx;
AUTOAUTHENTICATE=true
");
106 Aster Client Guide
Connect Using Database Drivers
ODBC Driver
2
• If using the DSN name: sprintf((char*)connect_str,"DSN=Beehive;SERVER=xxx.xxx.xxx.xxx;
AUTOAUTHENTICATE=true
");
Do not change the ODBC configuration file. If you need to specify parameters in the configuration file on the workers, you should contact Teradata Support.
Once you have constructed the connection string, connect using a function call like: rc = SQLDriverConnect(hdbc, (SQLPOINTER) NULL, connect_str, SQL_NTS, buffer1, 255, &outlen, SQL_DRIVER_NOPROMPT)
ODBC Usage Notes
Avoid using bind_param with SQL_DECIMAL
When using the bind_param method call with decimal fields, you should not explicitly specify
SQL_DECIMAL as the parameter bind type. If you do this, and the parameter value has a fractional digital part, the factional part changes to 0 after applying bind_param. The
DBD::ODBC bind_param implementation makes this conversion when specifying
SQL_DECIMAL/SQL_NUMERIC as the parameter bind type. Note that bind_param works fine when SQL_DECIMAL is not explicitly specified, so you should avoid this when passing decimal fields.
Bind columns without specifying a SQL datatype
Use the Perl API without specifying a SQL datatype the Aster datatypes, as in the example:
$rc = $sth->bind_col($column_number, \$var_to_bind);
Avoid specifying the SQL datatype to bind to the column, as in:
$rc = $sth->bind_col($column_number, \$var_to_bind, $bind_type);
This will work fine when binding boolean and character columns with DBI::SQL_CHAR, but for the other datatypes, this will cause two issues:
1
2
If you specify a numeric SQL type in bind_col, corrupt data may be returned. Numeric types include SQL_INTEGER, SQL_SMALLINT, SQL_DOUBLE, SQL_REAL,
SQL_NUMERIC/SQL_DECIMAL, SQL_BIT, SQL_TIME, SQL_TIMESTAMP,
SQL_DATE, and SQL_DATETIME.
If you specify a SQL_VARCHAR type in bind_col, an error may be returned. The error looks like:
• On Unix ODBC: “DBD::ODBC::st fetch failed: [unixODBC][Driver Manager]Invalid application buffer type (SQL-HY003)”
• On Windows ODBC: “DBD::ODBC::st fetch failed: [Microsoft][ODBC Driver
Manager] Program type out of range (SQL-HY003)”
This issue has been observed with the following software versions:
• Perl 5.8.9
• DBD-ODBC 1.3.1
• DBI-1.616
Aster Client Guide 107
Connect Using Database Drivers
ODBC Driver
Set Up ODBC for Perl Connectivity on Linux
To install the Aster Database ODBC driver and configure your environment to allow Perl scripts to connect to the database through the driver, follow these steps. These instructions assume you have installed Perl on your workstation, and that the Aster Database queen is reachable on the network. Follow these steps to provide an Aster Database connection your
Perl scripts can use:
1
2
3
Set up your
/etc/resolv.conf
and
/etc/nsswitch.conf
for accessing the Aster
Database queen.
The server connection information is set in the odbc.ini
file, as explained earlier in this chapter. On Windows, the server connection information is set using the ODBC Data
Source Administrator tool as explained in
“Set up ODBC for PHP” on page 109 .
In
/root
type
$ perl -eshell -MCPAN
4
5
6
Get the latest packages and install them by entering:
$ install Bundle::CPAN
$ install DBI
Rebuild and install DBD::ODBC.
$ export LD_LIBRARY_PATH=/home/beehive/toolchain/x86_64-unknownlinux-gnu/unixODBC-2.3.1/lib
$ export PATH=$PATH:/home/beehive/toolchain/x86_64-unknown-linuxgnu/unixODBC-2.3.1/bin
$ which odbc_config
/home/beehive/toolchain/x86_64-unknown-linux-gnu/unixODBC-2.3.1/ bin/odbc_config
$ odbc_config --cflags -DHAVE_UNISTD_H -DHAVE_PWD_H
-DHAVE_SYS_TYPES_H -DHAVE_LONG_LONG -DSIZEOF_LONG=8
$ perl -eshell -MCPAN cpan[1]> force install DBD::ODBC
Run odbcinst -j
to see where the .ini files are being picked up:
$ odbcinst -j
7
Run the following Perl script to check your installation.
If everything is set correctly, it will run without an error. If you encounter problems, check the data source name (“DSN”) in the connect statement. In the example below, the username and password of the database are “beehive” and “beehive”:
#!/usr/bin/perl use DBI; use DBD::ODBC;
# Connect to ODBC DSN and turn off AutoCommit my $dbh = DBI->connect('dbi:ODBC:nc',"beehive","beehive",
{AutoCommit => 0});
$dbh->do("BEGIN");
$dbh->do("set random_page_cost to '4'");
$dbh->do("set enable_seqscan to 'off'");
$dbh->disconnect;
108 Aster Client Guide
Connect Using Database Drivers
ODBC Driver
Set up ODBC for PHP
To set up PHP to work with ODBC, first install Aster ODBC driver on Linux.
Use unixODBC 2.3.1 and build it with the CFLAG, SIZEOF_LONG=8. The same will apply to building PHP when compiled to be used with unixODBC. Here are the supported versions:
• PHP 5.4.19
• Apache 2.2.25
2
3
PHP
To set up PHP:
1
Make sure Apache is installed and make note of the directory. For this example, we will assume Apache is installed at
/usr/local/apache
Ensure that unixODBC has been installed as described above.
Download the source for PHP 5.4.19 and extract it to the desired directory. The following setup instructions should be used for PHP:
$ CFLAGS="-DSIZEOF_LONG=8" ./configure --with-apxs2=/usr/local/ apache/bin/apxs --with-zlib --with-unixODBC --with-pdo-odbc=unixODBC
$ make && make install
$ cp -p .libs/libphp5.so /usr/local/apache/modules
Apache and PHP
Apart from the typical PHP-Apache setup, there are two things to keep in mind:
1
Ensure that the Linux system is set up to find the libraries that the ODBC driver requires.
a
Add the <INSTALL-DIR>/Libs and <INSTALL-DIR>/ODBCDriver paths to the /etc/ ld.so.conf file, where <INSTALL-DIR> is the location for the ODBC installation.
2 b
The following commands should be executed to refresh the ld cache and restart the
Apache web server:
$/sbin/ldconfig
$/etc/init.d/apachectl restart
To make sure that the ODBCSYSINI environment variable is available when PHP calls are made, use the following line in your PHP code:
#given ODBCSYSINI is to be set to /etc then: putenv("ODBCSYSINI=/etc")
Tip!
You should not set up your own PHP or use
/etc/init.d/apachectl
on the queen for your own web pages.
Aster Client Guide 109
Connect Using Database Drivers
JDBC Driver
JDBC Driver
JDBC is an API for the Java programming language that provides methods for querying and updating data in a relational database. The Aster Database JDBC driver enables your Java applications and reporting tools to retrieve data directly from Aster Databases.
The Aster Database JDBC driver is a Type 4 JDBC driver that implements the JDBC 3 specification.
•
•
Differences from the Legacy JDBC Driver
•
•
•
Use the JDBC Driver in a Java Application
•
Parameters for Connecting through JDBC
•
Configuring the JDBC Log Settings
•
Behavior and Performance Settings for JDBC
•
Using Client-Side Cursors in JDBC
•
Aster JDBC Driver
This version of the JDBC driver adds these capabilities to those that were already supported in the legacy JDBC driver:
• Ability to handle larger data sets than the legacy JDBC driver; doesn’t try to load whole result set onto the client in all cases
• Support for multibyte characters in data and user metadata UTF-8 encoding
• Support for Single Sign-On (SSO) authentication
• Support for Secure Socket Layers (SSL) encrypted communications
• Support customized connection parameters
• Support configuring statement FetchSize to limit the data being returned
• Support server side cursors
• Support specifying maximum number of rows
• Support for these commands:
• Copy—Moves data between Aster Database tables and a remote client (from and to a file) via the connection between the client and the server.
• Install File—Installs the data file or SQL-MapReduce function in the specified Aster
Database schema.
• Uninstall File—Removes the file or SQL-MapReduce function from Aster Database.
• Download File—Downloads the specified installed file or function.
• Display—Obtains the DDL of a single database object and returns the DDL as a string.
110 Aster Client Guide
Connect Using Database Drivers
JDBC Driver
• Generate—Returns the DDL of all objects in a database.
Differences from the Legacy JDBC Driver
These are important differences between this JDBC driver and the legacy (pre-5.0.3) JDBC driver:
• JDBC only supports scrollable forward cursors. Only the con.createStatement
(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY) and connection.createStatement() functions are supported. The Other parameter values like
ResultSet.TYPE_SCROLL_INSENSITIVE and ResultSet.CONCUR_UPDATABLE are not supported.
• The executeBatch() function throws a SQL exception (java.sql.SQLException).
• The result set cannot be accessed after an explicit commit.
• After running PreparedStatement.executenatch(), you must set all bind values or else the driver throws an error.
• The java.sql.PreparedStatement.setObject() function does not throw exceptions for invalid java.sql.Types.TINYINT values. Instead, the function wraps the values around. This behavior complies with the Java standard.
• ResultSetMetaData cannot be accessed after a ResultSet reset.
• The driver cannot establish a connection to the database when there are white space characters to the left of the connection string.
• The register serverConnect to database using JDBC in Aqua Data Studio (ADS) as register server. Expand the tree view when connection is lost. Register server will prompt an error.
• The DatabaseMetaData.getBestRowIdentifier() function is not supported.
• The ResultSet.getArray() function is not supported.
• You can call com.asterdata.ncluster.Driver.ASTER_BUILD_VERSION() to get JDBC version information. For detailed version information, check the MANIFEST.MF file.
Before You Start
Prerequisites
The JDBC driver supports these versions of the Java JDK:
• Oracle JDK 1.5
• IBM JDK 1.6
Install the JDBC Driver
Follow these steps to install the Aster Database JDBC driver:
1
Obtain the package that contains the Aster Database JDBC driver. You can do this in one of these ways:
• To get the newest package, download it from http://downloads.teradata.com/download/ tools
Aster Client Guide 111
Connect Using Database Drivers
JDBC Driver
2
3
• On your queen node, you can find the installers in the directory
/home/beehive/ clients_all/<your_client_OS>
.
Unzip the ZIP package.
The resulting folder contains multiple JAR files.
Copy the JAR files to a location in the classpath of the application that uses the driver.
Use the JDBC Driver in a Java Application
Follow these steps to integrate the Aster Database JDBC driver into your Java application:
1
2
Set the
CLASSPATH
environment variable to include the paths of the JAR files extracted from the Aster Database JDBC driver package.
• Use the Java
-classpath
flag at the command line to add the paths to the
CLASSPATH environment variable. For example, if you place the JAR files in /usr/jars, then add the path like this: java -classpath /usr/my_program /usr/jars/ or
• Add the paths to the driver to the
CLASSPATH
environment variable. For example: export CLASSPATH=/usr/jars/:$CLASSPATH
In your application:
a
Import the java.sql package like this: import java.sql.*;
b
Load the driver using the
Class.forName()
method:
Class.forName("com.asterdata.ncluster.Driver");
c
Define the username, password, and url (including the host, port, and database) parameters, which are needed to connect to the database. For example:
String username = "user";
String password = "password";
String url = "jdbc:ncluster://myhost:2406/database";
For more information about the URL format, see “Parameters for Connecting through
d
Get a Connection instance from JDBC using the
DriverManager.getConnection() method: try
{
} catch (SQLException ex)
{
// could not connect
}
Connection conn =
DriverManager.getConnection(url, username, password);
NOTE: The getConnection
method throws a “No driver available SQLException” exception if
CLASSPATH
does not contain the path for the JDBC driver, or if the parameters are incorrect.
112 Aster Client Guide
Connect Using Database Drivers
JDBC Driver
Parameters for Connecting through JDBC
To establish a connection to an Aster Database using the Aster Database JDBC driver, you must provide the driver with the URL to use to connect to the database. The URL has this format: jdbc:ncluster://<Host:Port>/<Database>?enable_backslash_escapes=<on_or
off>&enable_quoted_identifiers=<on_or off>[&AUTOAUTHENTICATE=true]
For example: jdbc:ncluster://192.65.197.90:2406/ beehive?enable_backslash_escapes=on&enable_quoted_identifiers=on&AUTOAUT
HENTICATE=true
The URL needs three parameters to connect to an Aster Database:
Table 4 - 20: Parameters in URL to connect to an Aster Database
Parameter
Host
Port
Database enable_backslash_escapes enable_quoted_identifiers
AUTOAUTHENTICATE=true
Required? Description
Optional The name of the server where the database resides.
Default is localhost.
Optional The port number that the database server is listening on.
Default is 2406.
Required The database name.
Optional Sets the queen parameter enable_backslash_escapes for the current session. The two possible values are:
• on—Enables this parameter on the queen (default).
• off—Disables this parameter on the queen.
Optional Sets the queen parameter enable_quoted_identifiers for the current session. The two possible values are:
• on—Enables this parameter on the queen (default).
• off—Disables this parameter on the queen.
Optional Sets the JDBC driver to use the authentication token rather than requiring the username and password to be sent for each connection.
In addition, to the URL, you must also provide the username and password needed to access the Aster Database, which you can get from your Aster Database administrator.
You can add AUTOAUTHENTICATE=true to the URL to use the existing connection token rather than reauthenticating on every connection the JDBC driver makes within the context of a session. See
“Enable Authentication Cascading” on page 106 for more details.
You can set the Autocommit and fetch_count settings for the connection in the URL by adding the autocommit
and fetch_count
parameters. See “Frequently Used JDBC Settings” on page 115
. If your application will query large tables, you should set autocommit
to false, and you should declare a fetch_count
for the connection. By doing this, you enable the connection to use distributed cursors for improved performance.
Aster Client Guide 113
Connect Using Database Drivers
JDBC Driver
You can also use the NumericAndDecimalAsDouble parameter to map NUMERIC and DECIMAL type columns to SQL_DOUBLE. When you set this parameter, its value will be stored in a connection context. This example shows how to use this parameter to decode the row header: if ((sqlType == Types.NUMERIC || sqlType == Types.DECIMAL) && inSettings.connectionSettings_.numericAsDouble_) { sqlType = Types.DOUBLE;
}
Enable Authentication Cascading
To enable your JDBC driver to connect to an Aster Database using the Aster Database JDBC driver, make the following adjustments to the usual Aster Database JDBC configuration.
1
Set the following parameters to configure the JDBC driver to connect to the Aster
Database:
Table 4 - 21: Parameters for the JDBC driver
Parameter
Domain Name
JDBC URL
Database Name
Port Number
Required? Description
Required The exact IP address of the Queen.
Required
(jdbc:ncluster://domain/ database?AUTOAUTHENTICATE=true)
Optional If not supplied, the system will re-connect to the database you are currently logged into.
Optional Will connect to default port: 2406
2
3
The new argument "
?AUTOAUTHENTICATE=true
" (included in the JDBC URL) enables the Authentication Cascading functionality. conn=DriverManager.getConnection(jdbc:ncluster://domain/
database?AUTOAUTHENTICATE=true)
// Authentication Cascading
NOTE: In the code example above, the section in bold is the entire JDBC URL that contains the "
?AUTOAUTHENTICATE=true
" argument to enable Authentication Cascading.
Enable Authentication Cascading by setting "
AUTOAUTHENTICATE
" to "
TRUE
".
The username and password are now hidden from SQL statements that invoke driver functions in the Analytic Foundation. Passwords are not visible on ACT Command Line, client SQL scripts, database logs, AMC, and Viewpoint portlets.
Configuring the JDBC Log Settings
To enable and configure JDBC logging:
1
2
Create a file named simba.properties and add it to the folder containing the JDBC driver.
In the file, define the LogLevel and LogPath properties.
• LogLevel—(Optional) Specifies the log level: OFF, FATAL, ERROR, WARNING, INFO,
DEBUG, and TRACE.
114 Aster Client Guide
Connect Using Database Drivers
JDBC Driver
• LogPath—(Optional) Specifies the path to the log file (AsterJDBC.log). If this property is not defined, the JDBC driver stores the log file in the same folder where the JDBC driver is located.
For example:
LogLevel=DEBUG
LogPath=D:\\logs
Behavior and Performance Settings for JDBC
When using the Aster Database JDBC driver, you make most behavior and performance settings in your SQL code using the SET command.
You can also use a number of standard JDBC driver methods for setting behavior and performance options, such as setAutoCommit()
and setPrepareThreshold()
.
The scope of these variables are limited to the scope of the transaction. After the commit() call, the transaction variables revert to their original values. In order to limit the transaction variables to a particular query, the variables need to be saved prior to being set and once the query is executed the variables must be set back to their original values.
Performance Tuning
If you are running your client application over a slow network, you can enable message compression to tune performance. When message compression is enabled, the queen compresses the query result set before it is delivered to the client application, thereby transmitting a smaller data size over the network.
Enabling compression on a high bandwidth network could result in poor performance. The best results are achieved when enabling compression on a network with slow bandwidth.
To enable compression, make the following setting on the queen node: allowMessageCompression=true
For information on setting configuration parameters on the queen, refer to the Aster Database
User Guide.
Frequently Used JDBC Settings
• autocommit
: Boolean value that sets the autocommit behavior for the connection, true to turn on autocommit (each statement, by default concluded with a semicolon, is run immediately in the database) or false for off (you must COMMIT your transaction to run it). You can set this in the connection URL with the autocommit
parameter (for example, jdbc:ncluster://10.80.50.100:2406/beehive?autocommit=false
) or in the Java code for your connection with
Connection.setAutoCommit()
. The default is true.
Unsupported JDBC Settings
Most JDBC option-setting methods in the Aster Database driver exhibit the standard JDBC behavior, but there are exceptions. Please note:
• setReadOnly()
: Not supported; Aster Database does not allow changing connection type to read-only.
Aster Client Guide 115
Connect Using Database Drivers
JDBC Driver
• setTransactionIsolation()
: Not supported; Aster Database does not allow changing transaction isolation levels.
Example Code Snippet That Sets Performance Tuning Variables
This example (PerformanceTuning.java) shows you how you can set some performance tuning variables for a particular transaction: package userguide; import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.SQLException; import java.sql.Statement; public class PerformanceTuning { public static void main(String[] args) {
String username = "db_superuser";
String password = "db_superuser";
String url = "jdbc:ncluster://153.65.197.125:2406/beehive";
Connection conn = null; try
{ conn =
DriverManager.getConnection(url, username, password);
// Remember the current autocommit state boolean autoCommit = conn.getAutoCommit();
// Turn off auto commits conn.setAutoCommit(false);
// create a statement with transaction variables set
// which will be used for this query
Statement stmt = conn.createStatement(); stmt.executeUpdate("SET enable_hashjoin = 'false'"); stmt.executeUpdate("SET enable_mergejoin = 'false'"); stmt.executeUpdate("SET enable_nestloop='true'"); stmt.executeUpdate("SET random_page_cost = '4.0'");
Statement statement = conn.createStatement();
String query="select * from knn_test";
ResultSet resultSet = statement.executeQuery(query);
// Process the result set while (resultSet.next()) {
// do something
}
// commit the transaction conn.commit(); resultSet.close(); stmt.close();
// Set the autocommit state back conn.setAutoCommit(autoCommit);
} catch (SQLException e) {
// TODO Auto-generated catch block e.printStackTrace();
}
}
116 Aster Client Guide
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
}
In the example above, the scope of the
SET
variables is limited to the commands between the autoCommit(false)
and commit()
lines.
Server and Client-Side Parameter Recommendations for
Improving JDBC Performance
To improve JDBC performance, Teradata recommends that you review this information and then implement the server and client-side parameter settings:
•
MuleFetchMessageSize Parameter (Server-Side)
•
Setting the Server-Side MuleFetchMessageSize Parameter:
•
FetchCount Parameter (Client-Side)
•
Setting the Client-Side FetchCount Parameter:
MuleFetchMessageSize Parameter (Server-Side)
The recommended setting for the server-side MuleFetchMessageSize parameter differs depending on the speed of the network:
• A high speed network connection between the client and server. For example, a 1GE /
10GE LAN.
Because of the high network speed, the data is transferred quickly between the server and client without introducing any significant delay.
In this case, the default parameter values are:
• MuleFetchMessageSize 100KB
• FetchCount = 32K
You can use the default MuleFetchMessageSize of 100KB or try increasing it to 1MB for better performance. While increasing the value of the MuleFetchMessageSize parameter reduces the number of data transfer connections between the client and server, it adds overhead by filling the buffer on the server. Therefore, the exchange of several small messages over a high speed network performs better than adding the additional overhead of filling a single large server message buffer while the client waits for the data.
The average performance degradation as compared to the Legacy is:
• 1.5x when using the default MuleFetchMessageSize of 100KB.
• 1.3x when using the recommended MuleFetchMessageSize of 1MB.
The performance improvement when using the recommended MuleFetchMessageSize is
0.2x.
• A low speed network connection between the client and server.
Because of the low network speed, the transfer of data over the network must be reduced.
Since the clients will receive the data after a longer period, it is best to fill the server message buffer instead of making numerous data transfer connections between the client and server over the network.
In the case of a single query execution, set the MuleFetchMessageSize parameter to 10MB.
117
Connect Using Database Drivers
JDBC Driver
In the case of high concurrency queries, set the MuleFetchMessageSize parameter to 1MB in order to avoid server thrashing.
The average performance degradation as compared to the Legacy is:
• Approximately 20x when using the default MuleFetchMessageSize of 100KB.
• Approximately 2x when using the recommended MuleFetchMessageSize 10MB.
The performance improvement when using the recommended MuleFetchMessageSize is
18x.
Setting the Server-Side MuleFetchMessageSize Parameter:
To set the MuleFetchMessageSize parameter, use this link: http://<queen-IP-address>:2407/std/ configflags
.
1
2
Occasionally, old parameter values are cached. If that happens, the web page that is already open will show the old values even after the page is refreshed. The workaround is:
Open a new tab to see the new values of the config flags.
Clear the browser cache.
FetchCount Parameter (Client-Side)
The recommended setting for the client-side FetchCount parameter is the maximum number of rows that can fit in the Mule Fetch message, which is based on the MuleFetchMessageSize parameter and the row size.
FetchCount = floor (MuleFetchMessageSize / row size)
Setting the FetchCount parameter to a larger value causes the client to wait in a loop until the requested number of rows are fetched by using multiple round trips between the client and server.
Setting the FetchCount parameter to a smaller value does not make use of the full capacity of
MuleFetchMessageSize.
This sample case provides an example of how to determine the best value for the FetchCount parameter:
Table Schema: "tab_1m_rows_fixed_column_small"
Column | Type | Modifiers
-----------+-------------------+----------t_char | character(1024) |
• The size of each row: 1KB
• The number of rows to be fetched: 1,000,000
• The MuleFetchMessageSize parameter value: 1MB (1024KB)
Calculating the FetchCount:
FetchCount = floor (MuleFetchMessageSize / row size)
FetchCount = floor (1024 KB / 1KB)
118 Aster Client Guide
Cancel
Connect Using Database Drivers
JDBC Driver
FetchCount = 1024 (Therefore, the value of the FetchCount parameter should be set to a value of 1000 rows.)
This table shows that a value of 1000 for the FetchCount parameter provides the best total time result for this particular sample case:
Table 4 - 22:
FetchCount
10
50
100
500
1000
5000
10000
50000
100000
1M
56
58
57
58
58
60
58
ExecuteQuery Time
(ms)
60
60
ResultFetch Time
(ms)
128048
28524
23362
18424
14865
15645
15553
15986
16043
Total Time (ms)
128108
28584
23420
18482
14925
15703
15609
16044
16100
Setting the Client-Side FetchCount Parameter:
Each client sets client parameters differently:
• In the JDBC script, the FetchCount parameter is set during getConnection as shown in this sample code:
Properties sysProps = new Properties();
sysProps.put("user", username_);
sysProps.put("password", passwd_);
sysProps.put("FETCH_COUNT", fetchCountValue);
sysProps.putAll(sysProps);
Connection con = DriverManager.getConnection(url_, sysProps);
• In the ACT client, the FetchCount parameter is set using the ACT option shown in this sample code:
-C [ --fetch-count ] arg
Example: act -h 10.80.173.90 -U beehive --fetch-count 1000
The cancel()
method cancels currently executing queries. The cancel request is sent from the client to the server. In some cases, the server might not cancel the query. For example, the server will not cancel the query if the query is not cancellable or if the query has already completed. Note that the COPY command is not cancellable.
Aster Client Guide 119
Connect Using Database Drivers
JDBC Driver
For detailed documentation on the cancel()
method, see the Java documentation on this web site: http://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#cancel%28%29
The following is a code example that uses the cancel()
method:
// This demonstrates how to run a query in one thread and cancel it in
// a different thread.
//
// You will need to customize the IP address of the queen in the variable
// "DB_URL" later in this code.
// You'll also need to add a table named "demo1", or modify the SQL statement
// to use a table that you already have.
import java.sql.*;
// This class allows us to run a query in a separate thread from the main
// thread (the main thread will cancel this query).
class MyThread extends Thread {
Statement myStatement;
// The SQL statement that we want to execute.
String mySql = "";
MyThread(Statement myStmt, String pSql) {
myStatement = myStmt;
mySql = pSql;
}
public void run(){
try {
System.out.println("Running query in different thread");
myStatement.executeQuery(mySql);
}
catch(Exception e) {
e.printStackTrace();
}
}
} public class CancelQuery { static final String JDBC_DRIVER = "com.asterdata.ncluster.Driver";
// Customize this URL to access your own server!
static final String DB_URL = "jdbc:ncluster://10.80.169.10:2406/beehive";
public static void main(String[] args) throws Exception
{
Connection conn = null;
Statement stmt = null;
try{
// Register JDBC driver
Class.forName(JDBC_DRIVER);
// Open a connection to the database server.
System.out.println("Connecting to database...");
// Customize the database user and password if you wish.
conn = DriverManager.getConnection(DB_URL,"beehive","beehive");
120 Aster Client Guide
Connect Using Database Drivers
JDBC Driver
// Execute a query
System.out.println("Creating statement...");
stmt = conn.createStatement();
// If the table "demo1" is a large table, this statement might run long
// enough that it will be practical to cancel it.
String sql = "SELECT COUNT(*) FROM demo1;";
ResultSet rs = null;
// Run the query in a separate thread.
MyThread thread = new MyThread(stmt, sql);
thread.start();
Thread.sleep(5000); // 5,000 milliseconds, in this case
System.out.println("Invoking cancel");
stmt.cancel();
// Wait until the thread running the query finishes.
thread.join();
// Clean up.
stmt.close();
conn.close();
} catch(SQLException se) {
//Handle errors for JDBC
se.printStackTrace();
} catch(Exception e){
//Handle errors for Class.forName
e.printStackTrace();
} finally{
//finally block used to close resources
try{
if (stmt!=null)
stmt.close();
} catch(SQLException se2){
}// nothing we can do
try{
if(conn!=null)
conn.close();
} catch(SQLException se){
se.printStackTrace();
}//end finally try
}//end try
System.out.println("Goodbye!");
}
}
Supported SQL Commands
The JDBC driver supports these commands:
•
•
•
•
•
•
Aster Client Guide 121
Connect Using Database Drivers
JDBC Driver
For more information about these commands, see the section describing SQL commands in the Aster Database User Guide.
Copy
This command moves data between Aster Database tables and a remote client (from and to a file) via the connection between the client and the server.
This is an example: public void copyCommandExamples() {
String stmt1 = "COPY simba TO 'd:\\simba.txt' DELIMITER as ','";
String stmt2 = "COPY simba TO 'd:\\simba.csv' with csv QUOTE AS '@';";
try {
Statement s = conn_.createStatement();
s.execute(stmt1);
s.execute(stmt2);
s.close();
} catch (Exception e) {
e.printStackTrace();
fail(e.getMessage());
}
}
Install File
This command installs the data file or SQL-MapReduce function in the specified Aster
Database schema. The filename and the schema name cannot be the same. An error will result if the names are the same.
INSTALL FILE filename [[schema/]alias]
This is an example: public void testInstallWithSchema() {
String stmt1 = "install file 'd:\\odbcng.tar.gz' ";
String stmt2 = "install file 'd:\\odbcng.tar.gz' uploadfile/'odbcng.zip'";
String stmt3 = "install file 'd:\\abc.txt' uploadfile/'abc.txt'";
try {
conn_.setAutoCommit(false);
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
conn_.commit();
s.close();
} catch (Exception e) {
try {
conn_.rollback();
} catch (SQLException e1) {
e1.printStackTrace();
}
fail(e.getMessage());
}
}
122 Aster Client Guide
Connect Using Database Drivers
JDBC Driver
Uninstall File
This command removes the file or SQL-MapReduce function from Aster Database.
UNINSTALL FILE 'filename' from schema schemaname
This is an example: public void testUninstall() {
String stmt1 = "UNINSTALL file 'abc.txt' from schema uploadfile ;";
String stmt2 = "UNINSTALL file 'odbcng.zip' from schema uploadfile ;";
String stmt3 = "UNINSTALL file 'odbcng.tar.gz' ;";
try {
conn_.setAutoCommit(false);
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
conn_.commit();
s.close();
} catch (Exception e) {
try {
conn_.rollback();
} catch (SQLException e1) {
e1.printStackTrace();
}
fail(e.getMessage());
}
}
Download File
This command downloads the specified installed file or function.
DOWNLOAD FILE [[schema/]alias] filename
This is an example: public void download_file_examples() {
String stmt1 = "DOWNLOAD file uploadfile/'abc.txt' 'd:\\download-abc.txt'";
String stmt2 = "DOWNLOAD file uploadfile/'odbcng.zip' 'd:\\download-uploadfileodbcng.zip'";
String stmt3 = "DOWNLOAD file 'odbcng.tar.gz' 'd:\\download-public-odbcng.zip'";
try {
Statement s = conn_.createStatement();
s.executeUpdate(stmt1);
s.executeUpdate(stmt2);
s.executeUpdate(stmt3);
s.close();
} catch (Exception e) {
fail(e.getMessage());
}
}
Display
This command obtains the DDL of a single database object and returns the DDL as a string.
DISPLAY [IN JSON] {TABLE | VIEW | INDEX | DATABASE | SCHEMA | USER | ROLE
| INSTALLFILE} <database object name>;
Aster Client Guide 123
Connect Using Database Drivers
JDBC Driver
This is an example: display in json table myschema.mytable;
Generate
This command returns the DDL of all objects in a database.
GENERATE [IN JSON] DATABASE <database name> [[WITH][GLOBALS |
NOGLOBALS][OWNERSHIPS | NOOWNERSHIPS][PRIVILEGES | NOPRIVILEGES]]
This is an example: generate database database_name with globals with ownerships;
Using Client-Side Cursors in JDBC
To give you control over the latency of initial query results, the Aster Database JDBC driver supports client-side cursors. Client-side cursors let you avoid retrieving all the result rows for a query at once. Instead, you specify the number of rows that should be retrieved to the client at a time. When that set is exhausted, the next page of rows is retrieved by repositioning the cursor.
Observe the following guidelines when working with cursors:
1
2
3
Turn off below.
autocommit
mode for the Connection object. See “Using Cursors in Your Code”
The ResultSet object that receives the output of your Statement cannot be a scrollable
ResultSet. That is, it must have the type,
ResultSet.TYPE_FORWARD_ONLY
. This is the default, so you need not rewrite your code.
If your application will query large tables, you should make sure your ResultSet can use distributed cursors. To do this,
a
make sure the ResultSet is not updatable (
ResultSet.CONCUR_READ_ONLY
)
b
make sure it’s not scrollable (
ResultSet.TYPE_FORWARD_ONLY
)
c
make sure it does not have HOLD enabled
(
ResultSet.CLOSE_CURSORS_AT_COMMIT
).
d
Also, the application should connect with must have a specified fetch_count
.
autocommit
set to false and the connection
Tip!
When working with ResultSets of type
ResultSet.TYPE_FORWARD_ONLY
, you cannot scroll backwards, nor can you jump to any location in the ResultSet other than the next row.
4
5
In the Statement object, you must pass a single query, not multiple queries strung together with semicolons.
You must set the statement’s fetch_count using the
Statement.setFetchSize(int rows)
command. This instructs the driver to fetch the specified number of rows at a time from the database. If the fetch size is not set, the driver fetches the full set of rows that match the query.
124 Aster Client Guide
Aster Client Guide
Connect Using Database Drivers
JDBC Driver
6
To help ensure a quick response when a page of rows is exhausted, the JDBC driver, by default, pre-fetches and caches ten pages of results from the database. A page is one fetch_count worth of rows. You can set the number of pages to be pre-fetched, or you can disable pre-fetching if desired.
Using Cursors in Your Code
The driver fetches fetch_count number of rows (as set with
Statement.setFetchSize(int rows)
) from Aster Database and exposes them to the database application as a “ rows
” data structure. The default fetch_count is zero, so you must set a fetch_count if you wish to use cursors.
With
ResultSet.next()
, your application iterates over the rows
data structure. When all rows in the rows
data structure have been processed, the next call to next()
will force the fetching a new set of fetch_count number of rows, and so on.
To use cursors, set the fetch size of your SQL statement using the setFetchSize(int rows) method. Later, setting the fetch_count back to 0 will cause all rows to be retrieved when the query runs (the default behavior).
Tip!
In the example below, we set the Autocommit setting and the FetchSize setting using the setAutoCommit()
and setFetchSize()
methods, but you also have the option of setting these in the JDBC connection
parameters when you make the connection. See “Frequently Used JDBC Settings” on page 115
.
Example That Uses Client-Side Cursors
Assuming a Connection conn
, you can set the FetchSize like this:
// Remember the current autocommit state boolean autoCommit = conn.getAutoCommit();
// Make sure autocommit is off conn.setAutoCommit(false);
Statement st = conn.createStatement();
// Turn use of the cursor on by setting FetchSize, expressed in rows st.setFetchSize(50);
ResultSet rs = st.executeQuery("SELECT * FROM mytable");
// Set the autocommit state back conn.setAutoCommit(autoCommit); while (rs.next()) {
System.out.print("a row was returned.");
} rs.close();
// Close the statement.
st.close();
Disabling Cursors
Cursors are enabled by default. To turn them off, set the FetchSize to zero. Assuming a statement st
and a ResultSet rs
, you would do this as shown here: st.setFetchSize(0); rs = st.executeQuery("SELECT * FROM mytable");
125
Connect Using Database Drivers
JDBC Driver while (rs.next()) {
System.out.print("many rows were returned.");
}
Test JDBC Connect Program
This program can be used as a template for your JDBC connectivity:
// JDBC Test Program import java.sql.Connection; import java.sql.DatabaseMetaData; import java.sql.DriverManager; import java.sql.SQLException; import java.sql.Statement; import java.sql.ResultSet; public class jdbctest { static String userid="beehive", password = "beehive";
// Change the IP address to the node you want to connect to.
static String url = "jdbc:ncluster://10.60.3.100:2406/beehive"; //2.0
static Connection con = null; public static void main(String[] args) throws Exception {
Connection con = getJDBCConnection(); if(con!= null){
try {
int count = 0;
System.out.println("Got Connection.");
DatabaseMetaData meta = con.getMetaData();
System.out.println("getDriverName: "+meta.getDriverName());
System.out.println("getDriverVersion: "+meta.getDriverVersion());
System.out.println("getDriverMajorVersion:
"+meta.getDriverMajorVersion());
System.out.println("getDriverMinorVersion:
"+meta.getDriverMinorVersion());
System.out.println("getDatabaseProductName:
"+meta.getDatabaseProductName());
System.out.println("getDatabaseProductVersion:
"+meta.getDatabaseProductVersion());
System.out.println("getIdentifierQuoteString:
"+meta.getIdentifierQuoteString());
System.out.println("\ngetSQLKeywords: "+meta.getSQLKeywords());
System.out.println("\ngetNumericFunctions: "+meta.getNumericFunctions());
System.out.println("\ngetStringFunctions :
"+meta.getStringFunctions());
System.out.println("getSystemFunctions : "+meta.getSystemFunctions());
System.out.println("getTimeDateFunctions :
"+meta.getTimeDateFunctions());
System.out.println("getSearchStringEscape :
"+meta.getSearchStringEscape());
System.out.println("getExtraNameCharacters :
"+meta.getExtraNameCharacters());
System.out.println("getCatalogTerm : "+meta.getCatalogTerm());
System.out.println("getCatalogSeparator :
"+meta.getCatalogSeparator());
System.out.println("getURL : "+meta.getURL());
System.out.println("getUserName : "+meta.getUserName());
System.out.println("getMaxCursorNameLength :
"+meta.getMaxCursorNameLength());
126 Aster Client Guide
Connect Using Database Drivers
JDBC Driver
System.out.println("getMaxSchemaNameLength :
"+meta.getMaxSchemaNameLength());
System.out.println("getMaxProcedureNameLength :
"+meta.getMaxProcedureNameLength());
System.out.println("getMaxCatalogNameLength :
"+meta.getMaxCatalogNameLength());
System.out.println("getMaxColumnsInIndex :
"+meta.getMaxColumnsInIndex());
System.out.println("supportsSubqueriesInComparisons :
"+meta.supportsSubqueriesInComparisons());
System.out.println("getMaxConnections : "+meta.getMaxConnections());
System.out.println("getMaxColumnsInTable :
"+meta.getMaxColumnsInTable());
System.out.println("isReadOnly : "+meta.isReadOnly());
System.out.println("\ngetCatalogs:");
ResultSet res = meta.getCatalogs();
while (res.next()) {
System.out.println(res.getString(1));
}
System.out.println("\ngetTables:");
res.close();
res = meta.getTables(null,null,"%",null);
while (res.next()) {
System.out.println(res.getString(1) +res.getString(2) +res.getString(3)
+res.getString(4));
}
System.out.println("");
res.close();
res = meta.getTableTypes();
System.out.println("\ngetTableTypes:");
while (res.next()) {
System.out.println(res.getString(1));
}
res.close();
Statement stmt=con.createStatement();
ResultSet rs = stmt.executeQuery("select count(*) from page_views");
while (rs.next()) {
System.out.println(rs.getInt(1));
}
rs.close();
stmt.close();
con.close();
} catch(SQLException ex) {
System.err.println("SQLException: " + ex.getMessage());
}
}else{
System.out.println("Could not Get Connection");
}
} public static Connection getJDBCConnection(){ try {
Class.forName("com.asterdata.ncluster.Driver");
} catch(java.lang.ClassNotFoundException e) {
System.err.print("ClassNotFoundException: ");
System.err.println(e.getMessage());
} try { con = DriverManager.getConnection(url,userid, password);
} catch(SQLException ex) {
Aster Client Guide 127
Connect Using Database Drivers
Configure Aster Database SQL Settings
}
}
System.err.println("SQLException: " + ex.getMessage());
} return con;
Configure Aster Database SQL Settings
This section explains how to set parameters that customize the behavior of Aster Database
SQL.
SQL Behavior Parameters
Escape Character Handling
enable_backslash_escapes
: Set to “on” (the default), Aster Database allows backslash escape strings in your constants. The escape strings are C-like backslash escape sequences, each introduced with a backslash character (\). Set to “off ”, Aster Database does not recognize backslash escape strings in your constants unless the constant is prefixed with a letter E before its opening single quote. In any constant not prefixed with the letter E, a backslash is interpreted simply as a backslash character.
Quoted-Identifier Handling
enable_quoted_identifiers
: This setting affects the way Aster Database processes strings enclosed in double-quote characters ("..."). With enable_quoted_identifiers='on'
(the default), Aster Database follows the standard behavior of interpreting each double-quoted string as an identifier (a column, table, schema, or function name). With enable_quoted_identifiers='off'
, each double-quoted string is interpreted as a literal string constant. Any printable character may be represented in a double-quoted string.
Setting the SQL Behavior Parameters
At the SQL Prompt
You can set these parameters from the SQL prompt by executing SET commands in the form:
SET SESSION <parameter name> = {'<param setting>'};
For example:
SET SESSION enable_backslash_escapes = {'on'|'off'}; and
SET SESSION enable_quoted_identifiers = {'on'|'off'};
In the Aster Database ODBC Driver
The Aster Database ODBC-NG driver allows you to set Aster Database SQL behavior parameters like, for example, enable_backslash_escapes. The connection parameters are:
• enable_backslash_escapes={on|off}
128 Aster Client Guide
Connect Using Database Drivers
Configure Aster Database SQL Settings
• enable_quoted_identifiers={on|off}
These can be set in the DSN configuration (i.e., the odbc.ini file or Windows registry).
Alternatively, a connecting application may set these options in the connection string when connecting without a registered DSN.
Syntax for ODBC Commands
This section lists some ODBC commands for which the syntax differs from the native Aster
Database syntax. Note that comments and parameter markers are not allowed in these commands. However, the prepare/execute model is supported.
COPY FROM/TO
Semantics and overall flow of execution for COPY is the same as for the Aster Database Loader
Tool. See
“Aster Loader Tool” on page 186
.
Syntax
COPY table [(column list)] FROM <quoted file name> ...COPY attributes or
COPY table [(column list)] TO <quoted file name> ...COPY attributes
The COPY command accepts a quoted file name and streams the data into or out of Aster
Database using the Aster Database Loader Tool protocol for maximum throughput.
INSTALL
The INSTALL command is similar to the ACT command
, and supports SQL-MR security semantics.
Syntax
INSTALL FILE <quoted file name> [[<schema>/]<file alias>]
• The schema name must be quoted if it contains spaces or mixed case.
• The file alias must be quoted.
UNINSTALL
The UNINSTALL command supports SQL-MR security semantics.
Syntax
UNINSTALL FILE <quoted file name> [[<schema>/]<file alias>]
• The schema name must be quoted if it contains spaces or mixed case.
• The file alias must be quoted.
DOWNLOAD
The DOWNLOAD command is similar to the ACT command
<file_alias>] <file>” on page 92 , and supports SQL-MR security semantics.
Syntax
DOWNLOAD FILE <quoted file name> [[<schema>/]<file alias>]
Aster Client Guide 129
Connect Using Database Drivers
Process SQL Statements in JDBC
• The schema name must be quoted if it contains spaces or mixed case.
• The file alias must be quoted.
Process SQL Statements in JDBC
You can execute SQL statements by using a
Statement
or a
PreparedStatement
instance.
Sometimes it is more convenient to use a
PreparedStatement
object for sending SQL statements to the database. This special type of statement is derived from the more general class,
Statement
, that you already know.
If you want to execute a
Statement
object many times, it normally reduces execution time to use a
PreparedStatement
object instead.
The main feature of a
PreparedStatement
object is that, unlike a
Statement
object, it is given an SQL statement when it is created. The advantage to this is that in most cases, this SQL statement is sent to the DBMS right away, where it is compiled. As a result, the
PreparedStatement
object contains not just an SQL statement, but an SQL statement that has been precompiled. This means that when the
PreparedStatement
is executed, the
DBMS can just run the
PreparedStatement
SQL statement without having to compile it first.
Process a Simple Query in JDBC
This example issues a simple query and prints out the first column of each row using a
Statement
.
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery("SELECT * FROM mytable WHERE columnf =
500"); while (rs.next()) {
System.out.print("Column 1 returned ");
System.out.println(rs.getString(1));
} rs.close(); st.close();
This example issues the same query as before but uses a
PreparedStatement
and a bind value in the query.
int fvalue = 500;
PreparedStatement st = conn.prepareStatement("SELECT * FROM mytable
WHERE columnf = ?"); st.setInt(1, fvalue);
ResultSet rs = st.executeQuery(); while (rs.next()) {
System.out.print("Column 1 returned ");
System.out.println(rs.getString(1));
} rs.close(); st.close();
130 Aster Client Guide
Connect Using Database Drivers
JDBC Troubleshooting and Limitations
The instance returns a
ResultSet
containing the results. Aster Database does not support cursor-based server-side caching of results which would save a small amount of time in parsing.
To retrieve data from the
ResultSet
instance, call next()
, which returns true
if there are results.
The
ResultSet
instance can be closed by calling close()
, or if another query is issued using the same
Statement
instance that was used to create this
ResultSet
.
Statement stmt=con.createStatement();
ResultSet rs = stmt.executeQuery("select count(*) from page_views");
while (rs.next()) {
System.out.println(rs.getInt(1));
}
rs.close();
stmt.close();
You can also run Update/Insert/Delete statements and Create/Drop Table statements using the same method.
JDBC Troubleshooting and Limitations
Cancellation Limitations
There are several limitations to JDBC cancellations:
• The
COPY
command is not cancellable.
• A query cannot be cancelled when it is in the prepare phase.
• Cancelling a SELECT can be done but the selected data will be returned before you get the canceled status.
• When two cancels are executed at almost the same time, only one query will be canceled.
Infinity and Negative Infinity Timestamp Conversion
The JDBC driver converts 'infinity' and '-infinity' to specific TimeStamp values (a very large and a very tiny TimeStamp) in the ResultSet.getObject API, which can cause unexpected results on the client side.
Aqua Data Studio and other client tools, like SQL Assistant JAVA edition also have this issue.
These tools do not support 'infinity' and '-infinity' in the timestamp datatype. If they encounter these values, they will display specific TimeStamp values (a very large and a very tiny TimeStamp).
If you want to retain the negative infinity and infinity values, you can use ResultSet.getString instead. Then you will be able to get the correct values for infinity and negative infinity, without this conversion to a very small or very large timestamp value.
Thread Safety
The Aster legacy JDBC driver (pre-5.0.3) supports thread safety for all JDBC objects. The new
Aster JDBC Driver (5.0.3 and later), does not support JDBC object thread safety in all cases. If
Aster Client Guide 131
Connect Using Database Drivers
Connect Reporting Tools to Aster Database you have developed code based on the legacy JDBC driver which uses the same instance of a
JDBC object across multiple threads simultaneously, you may have to modify your code to ensure that JDBC object thread safety is implemented by your application.
Connect Reporting Tools to Aster Database
This section shows how to connect some specific reporting and query tools to Aster Database:
•
Connect Aqua Data Studio to Aster Database
•
Connect MicroStrategy to Aster Database
Connect Aqua Data Studio to Aster Database
ADS support is being deprecated.
AquaFold’s ADS lets you perform DDL operations and query data interactively, and it provides tools that help you write and manage queries efficiently. ADS is a third-party tool available for purchase from Aqua Fold directly.
Aster Database is compatible with ADS version 10.0.2 with patch ads-10.0.7_03-patch.zip.
5
6
3
4
Install ADS
This section explains how to install ADS on your client workstation and connect to an Aster
Database.
1
2
Download Aqua Data Studio version 10.0.2 or later from http://www.aquafold.com/ downloads.html
Install Aqua Data Studio on your client workstation as explained in your version of the
Aqua Data Studio documentation at http://docs.aquafold.com/ aquadatastudio_11_documentation.html
Start Aqua Data Studio.
Select the command
Server: Register Server
.
In the list, select the “Aster nCluster Driver.”
Fill in the tabs as required.
2
3
Apply the ADS Patch
1
Download the patch from AquaFold:
4
http://dd1.aquafold.com/download/v10.0.0/ads-10.0.7_03-patch.zip
Unzip the patch files.
Replace the files under C:\Program Files\Aqua Data Studio 10.0 - 64bit\lib with the files from ads-10.0.7_03-patch.zip.
Restart ADS.
132 Aster Client Guide
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
Connect MicroStrategy to Aster Database
Observe the following guidelines when connecting MicroStrategy to Aster Database:
Versions and Platforms Supported
To get the latest information on supported releases of MicroStrategy with Aster Client releases, do one of:
• For Teradata customers: Refer to the "Partner Interoperability Matrix" by signing in to your Teradata at Your Service (T@YS) Account and navigating to Support Information >
Support Information > Software Compatibility > Partner Software Compatibility. Then select the document "Partner Interoperability Matrix".
• For Teradata employees: Go to the Teradata Asset Repository and locate the Asset ID:
KM025885.
Limitations
Aster Database is only certified as a warehouse with MicroStrategy. Aster Database cannot be used as a repository.
Set-up Instructions
To connect MicroStrategy to Aster Database, follow the steps below.
Prerequisites
Microstrategy supports the ODBC Driver Manager bundled with the Aster Database ODBC driver and the Windows Driver Manager only.
Make sure the following patches are applied to your MicroStrategy installation, depending on your version:
• MicroStrategy 8: Contact MicroStrategy Customer support to obtain:
• The
DATABASE.PDS
file that's certified with Aster Database.
• The
DTMAPPING.PDS
file that's certified with Aster Database.
• MicroStrategy 9: Contact MicroStrategy Customer support to obtain:
• The
DTMAPPING.PDS
file that's certified with Aster Database.
• MicroStrategy 9.0.1 and above: No changes required.
Install Drivers
On the client machine where MicroStrategy runs, install the database drivers:
1
2
Install the Aster Database ODBC driver. See “ODBC Driver” on page 106 for installation
instructions.
Install the MicroStrategy VLDB driver, version 9 or later.
Connecting to an Aster Database
1
Open your MicroStrategy project and choose
Intelligence Server
as your connection option.
2
3
Use the Configuration Wizard to create a
New Project Source
Create a
New Database Instance
and give it a name.
.
Aster Client Guide 133
Connect Using Database Drivers
Connect Reporting Tools to Aster Database
4
5
6
In the
Connection Type
field, choose Aster Database nCluster.
Create a
New Database Connection
and give it a name.
Create a new
ODBC System DSN
and give it a name.
Once you have saved the System DSN, you can create reports in MicroStrategy that use the data from your Aster Database. In your Logical Table definitions in MicroStrategy, you can create queries that use special Aster Database tools such as nPath.
Best Practices
By following the guidelines below, you can avoid common errors in Aster Database-
MicroStrategy integration.
Schema changes
• If your schema contains NUMERIC(X,0) type columns, you should replace these with
INT or BIGINT type columns for a higher probability of success with existing
MicroStrategy reports.
• For your small- to medium-sized tables that have no BIGINT or INT columns, you should create the tables in Aster Database as replicated dimension tables. This takes more space in the cluster, but works better with MicroStrategy.
Operational items
When pointing an existing MicroStrategy report from another database to Aster Database, the warehouse schema should be “UPDATED” the first time when pointed to an Aster Database
Instance. This updates the metadata inside MicroStrategy for correct MicroStrategy columns.
This is particularly important when a schema is ported from some other database to Aster
Database.
SQL query changes
• To use Aster Database’s nPath and other custom SQL-MapReduce functions:
• Reports must be created as free-form SQL type reports in MicroStrategy.
• Teradata Aster recommends that you embed the query in an MicroStrategy logical table and build the reports on top of that.
• If there are existing reports that need to run on Aster Database, some tweaking may be needed in the VLDB Settings of MicroStrategy depending on the complexity of the dynamically generated queries. For example, you may need to turn off TEMP table creation, and so on. This is very rare, but editing VLDB settings can be useful in some cases.
• If the following error appears in queries: “Unable to find function to convert from <x> to
<y>”, and if x and y are datatypes like int, float, and so on, then you may need a workaround to explicitly cast the column types. Contact Teradata Support for help on this.
134 Aster Client Guide
CHAPTER 5
Tools for .NET Environments
Teradata Aster provides a suite of tools that enable you to use Aster Database as a data source in your .NET applications and reports. These tools include:
• .NET Data Provider for Aster: A managed database driver for .NET that allows Microsoft
SQL Server Integration Services (SSIS) and .NET client applications to connect to an Aster
Database server.
• SSIS Connector, which lets you import and export data between Aster Database and external SQL servers.
This section explains how to install these tools and how to start writing applications and reports that work with Aster Database.
•
Loading Data with the SSIS .NET Data Provider for Aster
•
Using the Teradata Aster Connector for SSIS
Loading Data with the SSIS .NET Data Provider for Aster
This section is a tutorial that shows you how to create a Data Flow task in SSIS for transferring data from a table to a flat file via the .NET Data Provider for Aster.
Overview
SSIS is a tool for building extract, transform, and load (ETL) jobs. In .NET environments,
SSIS provides a fast way to extract data from or load data into your Aster Database databases.
To connect to Aster Database, SSIS requires the .NET Data Provider for Aster. This tutorial shows you how to set up SSIS to use the driver and provides an example for exporting data from Aster Database to a flat file.
Procedure
To set up SSIS to use .NET Data Provider for Aster, follow these steps:
Aster Client Guide 135
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
1
2
3
Run Microsoft SSIS
Choose
File: New > Project
to create a new integration project.
In the New Project dialog:
a
Select
Integration Services Project
.
b c
Enter your project a name
Click
OK
.
4
In the
Connection Manager
tab at the bottom of the window, add a new Connection.
To do this, right-click and select
New ADO.NET Connection
.
136
5
In the Configure ADO.NET Connection Manager window, click
New
.
In the next couple of steps we’ll create a widget (a database connection definition) that allows SSIS to connect to your Aster Database.
Aster Client Guide
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
6
7 b c
In the Connection Manager dialog, in the Provider drop-down box, choose
Aster Data
Provider
, and click
OK
.
In the Connection Manager dialog, make the following settings:
a
Set the Database to the name of the database in Aster Database you want to connect to.
Set the Host to your Aster Database queen IP address.
Set Port to the queen port, which is usually 2406.
d e
Set the User Id and Password to the credentials of a user with sufficient rights on your database in Aster Database.
Click
OK
. Note the name of the connection definition you are saving. Click
OK
again.
Aster Client Guide
8
Choose
View > Toolbox
.
137
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
9
Drag and drop the
Data Flow Task
from the
Control Flow Items
list into the
Control Flow
designer panel.
10
11
Click the
Data Flow
tab.
From the
Data Flow Sources
panel, drag an
ADO NET Source
object into the
Data Flow
panel.
138
12
To make this source a connection to your database in Aster Database, double-click
ADO NET Source
to configure it.
The ADO.NET Source Editor dialog box appears.
Aster Client Guide
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
13
In the ADO.NET Source Editor dialog box, configure the
Connection Manager
properties:
a
Click
Connection Managers
.
b c
From the
Data access mode
drop-down menu, choose
In the
SQL command text
field, enter a SQL query.
SQL command
.
Aster Client Guide
14
In the ADO.NET Source Editor dialog box, check the
Columns
property values:
a
Click
Columns
.
b
Check the External Column and Output Column values.
139
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
15
In the ADO.NET Source Editor dialog box, configure the
Error Output
properties:
a
Click
Error Output
.
b
For each output column, set its Error property to
Redirect row
.
16
17
In the ADO.NET Source Editor dialog box, click
OK
.
From the
Data Flow Destinations
panel, drag a
Flat File Destination
object into the
Data Flow
panel.
140 Aster Client Guide
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
18
Connect the green arrow of the ADO NET Source object to the
Flat File Destination
object.
Aster Client Guide
19
Configure the Flat File Destination object.
a b
Double-click the
Flat File Destination
object.
In the Flat File Destination Editor, click
Connection Manager
.
c d
Click
New
.
In the Flat File Format dialog box, click
Delimited
.
e
Click
OK
.
141
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
f
In the Flat File Connection Manager Editor dialog box, in the
Connection manager name
field, enter the name of the output file of the connection manager.
j i g h k
Click
Browse
.
Select an output file and click
Open
.
Click
OK
.
In the Flat File Destination Editor, click
Mappings
.
Set the correct column mappings for the Flat File Destination object.
142
l
Click
OK
.
Aster Client Guide
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
20
From the
Data Flow Destinations
panel, drag a
Flat File Destination
object into the
Data Flow
panel.
This object is the destination for all error records.
21
Connect the error output (red arrow) of the ADO NET Source object to Flat File
Destination you just added.
Aster Client Guide
22
Create another
Flat File Connection Manager
for the new
Flat File Destination
object that serves as the error destination.
a b
Double-click the
Flat File Destination
object.
In the Flat File Destination Editor, click
Connection Manager
.
c f d e
Click
New
.
In the Flat File Format dialog box, click
Delimited
.
Click
OK
.
In the Flat File Connection Manager Editor dialog box, in the
Connection manager name
field, enter the name of the output file to be used to store all error records.
j i g h
Click
Browse
.
Select an output file and click
Open
.
Click
OK
.
In the Flat File Destination Editor, click
Mappings
.
143
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
k
Set the correct column mappings for the error destination.
l
Click
OK
.
23
24
Save the project.
Run the project with debugging (
Debug > Start Debugging
).
You can check the progress of the workflow in the Progress panel.
144 Aster Client Guide
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
When the export is successful, the flat file source and destination objects in the Data Flow pane turn green.
If the export is not successful, the ADO NET Source object turns red. For example, you might get an exception like “Error: 0xC0047062 at Data Flow Task, ADO NET Source [16]:
System.ArgumentException: Error loading assembly: C:\Program Files (x86)\Teradata
Aster\Aster .NET Data Provider (x86)\AsterDataC#DSII.dll.”
Aster Client Guide
In this case, configure the 32-bit Runtime in SSIS for the project or you install the 64-bit
NClusterDNProvider.
The Microsoft SQL Server Business Intelligence Studio (BIDS) is a Visual Studio plug in.
For SQL Server 2008, this edition is 32-bit only. This requires using 32-bit drivers, including for Aster Database.
However, in the 64-bit versions of Windows, there is a project flag that you must set to allow 32-bit drivers to operate. Select the project, right-click, and choose
Properties
. Then, set
Run64BitRunTime
to
False
. If not, you get an architecture mismatch and other connection errors.
145
Tools for .NET Environments
Loading Data with the SSIS .NET Data Provider for Aster
Similarly, to run the package outside of BIDS—such as running a SQL Server Agent job or running the package by itself—click the tab
Execution options
and check the check box
Use
32 bit run time
.
146 Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Using the Teradata Aster Connector for SSIS
Teradata Aster provides SSIS Connector, a high-performance interface to Aster database for importing and exporting data between Aster Database and external SQL servers.
SSIS is a platform for building enterprise-level data integration and transformations solutions.
Using SSIS, you can update data warehouses, clean and mine data, and manage SQL Server objects and data.
With the Teradata Aster SSIS Connector, you can use ncluster_export to export data from
Aster Database and ncluster_loader to load data into Aster Database.
Teradata Aster SSIS Connector Features
Teradata Aster SSIS Connector has these features:
• Supports SSIS features and capabilities
• Supports create and edit data flow components
• Supports property access via built-in and custom editors
• Provides SSIS error handling
• Supports all Aster Database data types
• Lets you map Aster Database data types to SSIS data types
• Lets you perform data value transforms with special values (NaN, +/-Infinity) and out-of-range values
• Supports international characters
• Supports UTF-8(Aster) and Unicode (UTF-16, SSIS)
• Provides locale support
• Supports these SSIS platforms:
• SQL Server 2008 R2
• SQL Server 2012
Connection Managers
The Teradata Aster SSIS Connector includes these connection managers:
•
Aster Export Source Connection Manager (AsterExport)
•
Aster Loader Destination Connection Manager (AsterLoader)
•
Aster ODBC Connection Manager (AsterODBC)
Aster Export Source Connection Manager (AsterExport)
The Teradata Aster Source Connection Manager lets you:
• Configure the ncluster_export settings, which are used to export data from Aster
Database.
• Share the connectivity information from Aster ODBC Connection Manager.
Aster Client Guide 147
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Aster Loader Destination Connection Manager (AsterLoader)
The Teradata Aster Destination Connection Manager lets you:
• Configure the ncluster_loader settings, which are used to load data into Aster Database.
• Share the connectivity information from Aster ODBC Connection Manager.
Aster ODBC Connection Manager (AsterODBC)
The Teradata Aster ODBC Connection Manager lets you:
• Get metadata (tables, columns, and so on)
• Store connectivity information (host, port, uid, password, and so on)
Data Type Mapping
•
•
Teradata Aster Data Types vs. SSIS Data Types
•
Data Flow Components
The Teradata Aster SSIS Connector uses these two mappers to map Aster Database and SSIS data types:
•
•
Aster Export Source
This component exports data from Aster Database to SSIS via ncluster_export.
Aster Loader Destination
This component loads data from SSIS to Aster Database via ncluster_loader.
Teradata Aster Data Types vs. SSIS Data Types
Some of the Teradata Aster data types differ from the corresponding SSIS data types as described in the examples in this section.
Different length, precision, and scale limit
In Aster Database, for example, the Numeric data type supports up to 1000 digits.
The corresponding SSIS types are:
• DT_DECIMAL (28 digits)
• DT_NUMERIC (38 digits)
Different range and special values
In Aster Database, for example, the Date data type supports values from 4713 BC to
5874897 AD.
The corresponding SSIS type is DT_DBDATE. It supports values from 0001-01-01 AD to
9999-12-31 AD.
148 Aster Client Guide
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Unsupported Data Types
SSIS does not have corresponding data types for these Aster Database types, so the SSIS
Connector maps them to the SSIS DT_WSTR data type:
• Ip
• Ip4range
• timetz
• UUID
Data Mapping Solutions
The Teradata Aster SSIS Connector provides these types of data mapping between Aster
Database and SSIS:
•
Primary and Alternative Mapping
•
•
Special-Value Handling (Aster Export Source)
Primary and Alternative Mapping
Primary mapping maps compatible data types, but there could be range, length, precision, or scale mismatches in the corresponding data values.
For incompatible data types, the Teradata Aster SSIS Connector provides alternative mappings:
• Maps the Aster Database data type to the DT_WSTR or DT_NTEXT SSIS types (Aster
Export Source).
• Maps the SSIS types DT_WSTR and DT_NTEXT back to the Aster Database data type
(Aster Loader Destination).
When performing alternative mapping, the Teradata Aster SSIS Connector does not change the data values.
Table 5 - 23 shows the primary vs. alternative Aster-to-SSIS mappings.
Table 5 - 23: Primary vs. Alternative Mapping (Aster Export Source)
Aster Data Type
smallint integer bigint
Conditions Mapping
Primary
Primary
Primary real Primary
Alternative double precision Primary
Alternative
SSIS Data Type
DT_I2
DT_I4
DT_I8
DT_R4
DT_WSTR
DT_R8
DT_WSTR
149
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 23: Primary vs. Alternative Mapping (Aster Export Source)
Aster Data Type
numeric [p[, s]] boolean
Bytea
Character/bit [(n)]
Character/bit varying [(n)]
Date time [(p)] timetz [(p)] timestamp [(p)] timestamptz [(p)] serial {local | global} bigserial {local | global}
Ip4/Ip4range/uuid/interval
Conditions
p => 29
Mapping
Primary
Alternative p < 29 Primary
Alternative length > 8000
Primary
Primary length <= 8000 Primary length > 4000 Primary length <= 4000 Primary
Primary
Alternative
Primary
Primary
Primary
Alternative
Primary
Alternative
Primary
Primary
Primary
SSIS Data Type
DT_R8
DT_WSTR
DT_DECIMAL
DT_NUMERIC
DT_BOOL
DT_IMAGE
DT_BYTES
DT_NTEXT
DT_WSTR
DT_DBDATE
DT_WSTR
DT_DBTIME2
DT_WSTR
DT_TIMESTAMP2
DT_WSTR
DT_DATETIMESTAMPOFFSET
DT_WSTR
DT_I4
DT_I8
DT_WSTR
Table 5 - 24 shows the primary vs. alternative SSIS-to-Aster mappings.
Table 5 - 24: Primary vs. Alternative Mapping (Aster Loader Destination)
SSIS Data Type
DT_BOOL
DT_BYTES
DT_DBDATE
DT_DBTIME2
DT_DBTIMESTAMP2
DT_DBTIMESTAMPOFFSET
Mapping
Primary
Primary
Primary
Primary
Primary
Primary
Aster Data Type
bool bytea date time without time zone timestamp without time zone timestamp with time zone
150 Aster Client Guide
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 24: Primary vs. Alternative Mapping (Aster Loader Destination)
SSIS Data Type
DT_DECIMAL
DT_I2
DT_I4
DT_I8
DT_NUMERIC
DT_R4
DT_R8
DT_WSTR
DT_IMAGE
DT_NTEXT
Mapping
Primary
Primary
Primary
Alternative
Primary
Alternative
Primary
Primary
Primary
Primary
Alternative
Alternative
Alternative
Alternative
Alternative
Alternative
Primary
Primary
Alternative
Aster Data Type
numeric smallint integer serial bigint bigserial numeric real double precision char varying [(p)]
Char/bit/varbit [(p)]
Ip4/ip4range/uuid/interval timestamp with/without time zone time with/without time zone
Real/double/numeric
Date
Bytea char varying [(p)]
Char/bit/varbit [(p)]
151
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Mapping Limitations
The Aster Database data type timestamptz does not support dates before 1883-11-18 in the
US time zone. If this is an issue, map timestamptz to the DT_WSTR SSIS data type.
Mapping the Aster Database data type double to DT_WSTR with a special value of
1.7976931348623157E+308 is not supported. The SSIS Connector only transfers 14 digits after the floating point. In this case, the value of DT_WSTR is the string
“1.79769313486232E+308”, which is the rounded value of 1.7976931348623157E+308.
You cannot map the Aster Database data type bytea with length > 8000 bytes to DT_BYTES. If you configure error handling, a truncation error is generated.
Mapping the Aster Database data type decimal to DT_NUMERIC might result in precision loss, because DT_NUMERIC is not an absolute accurate data type. For example,
9999999999999999999999999999.99 maps to 10000000000000000000000000000.00.
Special-Value Handling (Aster Export Source)
The Teradata Aster SSIS Connector uses Aster Export Source to provide special-value handling when mapping certain Aster Database types to SSIS (see
• NaN
• -/+Infinity
• Out-of-range values
Table 5 - 25: Special-value handling (Aster Export Source)
Out-of-Range Aster
Real
Double
Numeric
Date
Timestamp
Timestamptz
SSIS
DT_R4
Special Values
NaN, +/- Infinity
DT_R8 NaN, +/- Infinity
DT_R8/DT_Decimal/DT_Numeric NaN
DT_DBDATE
DT_TIMESTAMP2
DT_DATETIMESTAMPOFFSET
+/- Infinity
+/- Infinity
+/- Infinity
< 0001-01-01 (with “BC”) > 9999-12-31
< 0001-01-01 (with “BC”) > 9999-12-31
< 0001-01-01 (with “BC”) > 9999-12-31
For more information, see Configuring Special Value Settings
.
Installing the Teradata Aster SSIS Connector
System Requirements
To install Aster SSIS Connector, you need one of the following:
• A system running SQL Server 2008 R2 with Business Intelligence Development Studio
(BIDS). When installing on this platform, the Aster SSIS Connector installer also installs a light version of Visual Studio 2008.
• A system running SQL Server 2012 with BIDS. When installing on this platform, the Aster
SSIS Connector installer also installs a light version of Visual Studio 2010.
152 Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Installing the Connector
The Teradata Aster Client package provides several Aster SSIS Connector installers.
To install the SSIS Connector in a 32-bit environment, run one of these installers:
• AsterConnector_SSIS2008_i386.msi (SQL Server 2008 with BIDS)
• AsterConnector_SSIS2012_i386.msi (SQL Server 2012 with BIDS)
To install the SSIS Connector in a 64-bit environment, run one of these installers:
• AsterConnector_SSIS2008_x64.msi (SQL Server 2008 with BIDS)
• AsterConnector_SSIS2012_x64.msi (SQL Server 2012 with BIDS)
Creating an Integration Services Project
To create an Integration Services Project in Visual Studio 2008:
1
2
3
Open Visual Studio.
Choose
File > New > Project
.
Create a Integration Services Project.
Aster Client Guide 153
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
4
Add a Data Flow Task.
5
Double-click the Data Flow Task, right-click in the Toolbox, and choose
Choose Items
from the drop-down menu.
The Choose Items dialog appears.
154 Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
6
Check the check boxes for
Aster Export Source
and
Aster Loader Destination
.
7
Add Aster Export Source and/or Aster Loader Components.
8
Start designing.
For more information, see Using SSIS Connector and
Example: Using Aster Export Source and Aster Loader Destination
.
To create an Integration Services Project in Visual Studio 2010:
1
If using Visual Studio 2010, add a Data Flow task, then, in the SSIS Toolbox, right-click
Common
, and choose
Refresh Toolbox
.
Aster Client Guide
2
Double-click the Data Flow task.
155
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
3
4
Add Aster Export Source and/or Aster Loader Components.
Start designing.
For more information, see Using SSIS Connector and
Example: Using Aster Export Source and Aster Loader Destination
.
Using SSIS Connector
•
•
Configuring Aster Export Source Editor Settings
•
Configuring Aster Loader Destination Editor Settings
•
Configuring Aster Export Connection Manager Settings
•
Configuring Aster Loader Connection Manager Settings
•
Configuring Aster ODBC Connection Manager Settings
Adding Connection Managers
In Business Intelligence Development Studio, you can add AsterExport, AsterLoader, and
AsterODBC connection managers via the Connection Managers panel, as described in this section.
Tip:
You can also add connection managers using the Aster Export Source Editor (
Configuring Aster Export Source Editor Settings ) and the Aster Loader Destination Editor.
156 Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
To add a connection manager via the Connection Managers panel:
1
Right-click inside the Connection Manager panel and choose drop-down menu.
New Connection
from the
2
In the Add SSIS Connection Manager window, choose the type of connection manager to add:
• AsterExport
• AsterLoader
• AsterODBC.
Aster Client Guide
• To add an AsterExport connection manager, select
AsterExport
and click
OK
. Then, to configure the manager, see
Configuring Aster Export Connection Manager Settings
.
• To add an AsterLoader connection manager, select
AsterLoader
and click
OK
. Then, to configure the manager, see
Configuring Aster Loader Connection Manager Settings .
• To add an AsterODBC connection manager, select
AsterODBC
and click
OK
. Then, to configure the manager, see
Configuring Aster ODBC Connection Manager Settings .
Configuring Aster Export Source Editor Settings
•
•
•
Configuring Error Output Settings
•
•
Configuring Special Value Settings
157
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Configuring General Settings
Table 5 - 26: General settings
Name
Adapter Name
Description
Aster ODBC Connection
Manager
Aster Export Connection
Manager
Select Aster Table
Select Aster table columns
Description
Name of the Aster Export Source component.
Short description of the component.
Name of the ODBC connection manager. You can select an existing manager or create a new one. To create a new connection manager, choose <New> and configure its settings. For more information, see
Configuring Aster ODBC Connection Manager Settings .
Name of the Aster Export connection manager. You can select an existing manager or create a new one. To create a new connection manager, choose <New> and configure its settings. For more
information, see Configuring Aster Export Connection Manager
Name of the table to export from Aster Database.
The table columns to export. Clicking Preview displays the data that will be exported.
Configuring Columns Settings
The Columns pane is where you specify the Aster-to-SSIS mapping.
Table 5 - 27: Columns settings
Name
External Column
Output Column
DT_Type
Length
Precision
Scale
Description
Name of the column in Aster Database. This is a read-only field.
Name of the exported column. It is the same as External Column, but you can choose to ignore this column.
The SSIS data type to map the exported column to. In some cases, only one option is available. In other cases, you can select from multiple types.
Depending on the specified DT_Type, you can specify length, precision, and/ or scale.
If applicable, the length of the DT_WSTR string.
If applicable, the number of digits in a number.
If applicable, the number of digits to the right of the decimal point in a number.
158 Aster Client Guide
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Configuring Error Output Settings
The Error Output pane is where you specify what happens if an export error occurs.
Table 5 - 28: Error Output settings
Name
Output Column
Error
Truncation
Description
Name of the exported column. It is the same as External Column, but you can choose to ignore this column.
What to do in if a value is not supported by SSIS. Examples of unsupported values are Null, infinity and negative infinity values.
•Fail Component: Causes the Aster Export Source component to fail.
•Ignore: Ignores the error.
•Redirect Row: Redirects the row with the unsupported value to the error output of the component, which can be a text file.
What to do in case exported values are truncated.
•Fail Component: Causes the Aster Export Source component to fail.
•Ignore: Ignores the error.
•Redirect Row: Redirects the row with the truncated value to the error output of the component, which can be a text file.
Configuring Advanced Settings
Table 5 - 29: Advanced settings
Name
Error Handling
•SSIS Redirection Error
Limit
•Total Error Row Limit
Description
Maximum numbers of rows that are redirected before failing the component. If you do not specify a value, no rows are redirected.
Total number of rows containing errors before failing the component.
If you do not specify a value, the component fails if one error occurs.
Logging
•Include Thread
•Include TimeStamp
•Log Performance
Whether to include the thread number in the error record.
Whether to include the thread number in the error record. SSIS transfers data buffer by buffer, and each buffer might be handled by a different thread.
Whether to include a timestamp in the error record.
Whether to log the performance statistics as Information, including elapsed time for calling ncluster_loader or ncluster_export, and data transformation inside Aster SSIS Connector.
•LogFile
•File Name Name of the log file to output.
159
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 29: Advanced settings (continued)
Name
•Location Type
•Environment Variable
•Folder Path
•Logging Level
•LoggingCaptureType
Description
Location type. Choose Environment Variable if the path to the file is stored in an environment variable. Choose Folder Path if you want to specify the folder where the file is stored.
The environment variable that contains the path to where the file is stored.
The path to the folder where the file is stored. Either enter the path or click the “...” button and select the folder.
OFF: Nothing is logged.
FATAL: Only issues that cause component failure are logged.
ERROR: Only error are logged.
WARN: Only warnings are logged.
INFO: Only information messages are logged.
ALL: All logging levels are included in the log file.
toMemoryStream: Send the log file to the screen output.
toDisk: Store the log file on the local disk.
Performance
•CSVFileReadBufferSize Read buffer size of the data file.
Configuring Special Value Settings
The Special Value pane is where you specify special value handling.
Table 5 - 30: Special Value settings
Name
Output Column
External Column
NaN Method
NaN UD Value
Description
Name of the exported column. It is the same as External Column, but you can choose to ignore this column.
Name of the column in Aster Database. This is a read-only field.
If the column contains NaN, you can choose one of these handling methods:
•Not Used: Pass the value directly to SSIS without modification. In this case, mapping errors and value truncation is reported to SSIS, which applies the appropriate error or truncation handling method (Redirect, Ignore, or FailComponent).
•Insert Null: Insert Null in place of NaN.
•User defined: Insert a user-defined value in place of NaN.
The user-defined value to use in place of NaN.
160 Aster Client Guide
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 30: Special Value settings (continued)
Name
PosInf Method
PosInf UD Value
NegInf Method
NegInf UD Value
GreaterRange
(>Max) Method
Description
If the column contains positive infinity, you can choose one of these handling methods:
•Not Used: Pass the value directly to SSIS without modification. In this case, mapping errors and value truncation is reported to SSIS, which applies the appropriate error or truncation handling method (Redirect, Ignore, or FailComponent).
•Insert Null: Insert Null in place of positive infinity.
•User defined: Insert a user-defined value in place of positive infinity.
The user-defined value to use in place of positive infinity.
If the column contains negative infinity, you can choose one of these handling methods:
•Not Used: Pass the value directly to SSIS without modification. In this case, mapping errors and value truncation is reported to SSIS, which applies the appropriate error or truncation handling method (Redirect, Ignore, or FailComponent).
•Insert Null: Insert Null in place of negative infinity.
•User defined: Insert a user-defined value in place of negative infinity.
The user-defined value to use in place of negative infinity.
If the column contains an out-of-range value that is greater than the maximum acceptable value, you can choose one of these handling methods:
•Not Used: Pass the value directly to SSIS without modification. In this case, mapping errors and value truncation is reported to SSIS, which applies the appropriate error or truncation handling method (Redirect, Ignore, or FailComponent).
•Insert Null: Insert Null in place of the exported value.
•User defined: Insert a user-defined value in place of exported.
The user-defined value to use in place of the exported value.
GreaterRange
(>Max) Value
LessRange (<Min)
Method
LessRange (<Min)
Value
If the column contains an out-of-range value that is less than the minimum acceptable value, you can choose one of these handling methods:
•Not Used: Pass the value directly to SSIS without modification. In this case, mapping errors and value truncation is reported to SSIS, which applies the appropriate error or truncation handling method (Redirect, Ignore, or FailComponent).
•Insert Null: Insert Null in place of the exported value.
•User defined: Insert a user-defined value in place of the exported value.
The user-defined value to use in place of the exported value.
161
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Configuring Aster Loader Destination Editor Settings
•
•
•
Configuring Error Output Settings
•
Configuring General Settings
Table 5 - 31: General settings
Name
Adapter Name
Description
Aster ODBC Connection
Manager
Aster Export Connection
Manager
Select Aster Table
Input Locale
Description
Name of the Aster Loader Destination component.
Short description of the component.
Name of the ODBC connection manager. You can select an existing manager or create a new one. To create a new connection manager, choose <New> and configure its settings. For more information, see
Configuring Aster ODBC Connection Manager Settings .
Name of the Aster Export connection manager. You can select an existing manager or create a new one. To create a new connection manager, choose <New> and configure its settings. For more
information, see Configuring Aster Export Connection Manager
Name of the Aster Database table to load data into.
The input locale to use. Check the Is Default check box to make your selection the default locale.
Configuring Columns Settings
The Columns pane is where you specify the SSIS-to-Aster mapping.
Table 5 - 32: Columns settings
Name
Input Column
External Column
Description
The input columns are drop boxes, which can be chosen
From the drop down menu, choose the input column to Aster Loader
Destination to map to an external column in the target table in Aster
Database.
(Read only) The columns (name and data type) of the target table in Aster
Database.
162 Aster Client Guide
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
The example below shows how to switch mapping columns: from (id to id and val to val) to
(id to val and val to id):
1
From the drop-down menu, change the mapping value for the external column “val
(integer)” by choosing
<ignore>
from the drop-down menu.
This makes the input column type “val(DT_I4)” available for mapping to other columns.
2
Map the external column “id (integer)” to “val (DT_I4).”
3
Map the external column “val (integer)” to “id (DT_I4).”
163
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Configuring Error Output Settings
The Error Output pane is where you specify what happens if an loading error occurs.
Table 5 - 33: Error Output settings
Name
Column
Handler
Error
Truncation
Description
Name of the imported column.
Whether malformed values are checked by Aster Database or SSIS:
• Choose
Error/Truncation
to use SSIS checking. SSIS can handle the malformed values inside SSIS, but the performance is impacted.
• Choose
Aster
to use Aster checking, which has better performance than SSIS checking, but the malformed values are passed to Aster
Database and cannot not be send back to SSIS for additional processing.
What to do in if a value is not supported by SSIS. Examples of unsupported values are Null, infinity and negative infinity values.
•Fail Component: Causes the Aster Loader Destination component to fail.
•Ignore: Ignores the error.
•Redirect Row: Redirects the row with the unsupported value to the error output of the component, which can be a text file.
What to do in case loaded values are truncated.
•Fail Component: Causes the Aster Loader Destination component to fail.
•Ignore: Ignores the error.
•Redirect Row: Redirects the row with the truncated value to the error output of the component, which can be a text file.
Configuring Advanced Settings
Table 5 - 34: Advanced settings
Name Description
Error Handling
•Advanced Malformed
Row Log File
•File Name
•Location Type
Name of the log file to output.
Location type. Choose Environment Variable if the path to the file is stored in an environment variable. Choose Folder Path if you want to specify the folder where the file is stored.
•Environment Variable Environment variable that contains the path to where the file is stored.
•Folder Path Path to the folder where the file is stored. Either enter the path or click the “...” button and select the folder.
164 Aster Client Guide
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 34: Advanced settings (continued)
Name
•SSIS Redirection Error
Limit
•Total Aster Error Row
Limit
Description
Maximum numbers of rows that are redirected before failing the component. If you do not specify a value, no rows are redirected.
Number of malformed rows reported by ncluster_loader before the component fails. This option is relevant when you choose Aster for the handling of malformed values.
This option is useful when you choose to execute ncluster_loader at the end of each buffer (CLIExecuteMode). In this case, the ncluster_loader is called many times, and the number of malformed rows are accumulated for each ncluster_loader call until that number reaches the value specified by this parameter.
•Total Error Row Limit Total number of rows containing errors before failing the component. If you do not specify a value, the component fails if one error occurs.
Logging
•Include Thread
•Include TimeStamp
•LogPerformance
Whether to include the thread number in the error record.
Whether to include the thread number in the error record. SSIS transfers data buffer by buffer, and each buffer might be handled by a different thread.
Whether to include a timestamp in the error record.
Whether to log the performance statistics as Information, including elapsed time for calling ncluster_loader and data transformation inside
Aster SSIS Connector.
•LogFile
•File Name
•Location Type
Name of the log file to output.
Location type. Choose Environment Variable if the path to the file is stored in an environment variable. Choose Folder Path if you want to specify the folder where the file is stored.
•Environment Variable Environment variable that contains the path to where the file is stored.
•Folder Path
•Logging Level
•LoggingCaptureType
The path to the folder where the file is stored. Either enter the path or click the “...” button and select the folder.
OFF: Nothing is logged.
FATAL: Only issues that cause component failure are logged.
ERROR: Only error are logged.
WARN: Only warnings are logged.
INFO: Only information messages are logged.
ALL: All logging levels are included in the log file.
toMemoryStream: Send the log file to the screen output.
toDisk: Store the log file on the local disk.
165
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 34: Advanced settings (continued)
Name Description
Performance
•CLIExecuteMode
•MaxCLIThreadNumbe r
Execute mode of ncluster_loader at the end of processing all rows
(AtEndOfRowSet) or at each buffer (AtEachBuffer).
•AtEndOfRowSet: Treats the entire loading task as a single transaction, but performance is impacted.
•AtEachBuffer: Divides the loading task into several transactions (one transaction per buffer), which results in better performance.
Maximum number of parallel ncluster_loader threads. Default is 1.
Configuring Aster Export Connection Manager Settings
•
•
Configuring Component settings
•
Configuring Component Name
In the Name field, enter the name of the connection manager.
Configuring Component settings
Table 5 - 35: Component settings
Name
KeepTempFile
Description
Whether to keep temporary files after the data export. This setting is configured automatically based on your selections below.
nCluster_export
•File Name
•Location Type
Name of the nCluster_export utility. By default, the name of the utility is nCluster_export.exe.
Type of the location setting. Choose Environment Variable if the path to the utility is stored in an environment variable. Choose Folder Path if you want to specify the folder where the utility is stored.
•Environment Variable Environment variable that contains the path to the utility.
•Folder Path Path to the folder where the utility is stored. Either enter the path or click the “...” button and select the folder.
Temp File Location
•Location Type Type of the location setting. Choose Environment Variable if the path to the temporary files is stored in an environment variable. Choose Folder
Path if you want to specify the folder where the files are stored.
•Environment Variable Environment variable that contains the path to the temporary files.
166 Aster Client Guide
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 35: Component settings (continued)
Name
•Folder Path
Description
Path to the folder where the temporary files are stored. Either enter the path or click the “...” button and select the folder.
Configuring Export settings
Table 5 - 36: Export settings
Name Description
Begin Script
•File Name
•Location Type
Name of the script file to run before exporting data.
Type of the location setting. Choose Environment Variable if the path to the file is stored in an environment variable. Choose Folder Path if you want to specify the folder where the file is stored.
•Environment Variable Environment variable that contains the path to the file.
•Folder Path The path to the folder where the file is stored. Either enter the path or click the “...” button and select the folder.
End Script
•File Name
•Location Type
Name of the script file to run after exporting data.
Type of the location setting. Choose Environment Variable if the path to the file is stored in an environment variable. Choose Folder Path if you want to specify the folder where the file is stored.
•Environment Variable The environment variable that contains the path to the file.
•Folder Path Path to the folder where the file is stored. Either enter the path or click the “...” button and select the folder.
ForceLoader Force the use of the preferred loader, which is specified by the LoaderIP property. This is the same as using the -f option with ncluster_export
(ncluster_export -f).
LoaderIP
Timeout
IP address of the preferred loader.
Aster Database connection timeout in seconds. Default is 30.
Configuring Aster Loader Connection Manager Settings
•
•
Configuring Component settings
•
Configuring Component Name
In the Name field, enter the name of the connection manager.
167
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Configuring Component settings
Table 5 - 37: Component settings
Name
KeepTempFile
Description
Whether to keep temporary files after loading the data. This setting is configured automatically based on your selections below.
nCluster_loader
•File Name
•Location Type
Name of the nCluster_loader utility. By default, the name of the utility is nCluster_loader.exe.
Type of the location setting. Choose Environment Variable if the path to the utility is stored in an environment variable. Choose Folder Path if you want to specify the folder where the utility is stored.
•Environment Variable Environment variable that contains the path to the utility.
•Folder Path Path to the folder where the utility is stored. Either enter the path or click the “...” button and select the folder.
Temp File Location
•Location Type Type of the location setting. Choose Environment Variable if the path to the temporary files is stored in an environment variable. Choose Folder
Path if you want to specify the folder where the files are stored.
•Environment Variable Environment variable that contains the path to the temporary files.
•Folder Path Path to the folder where the temporary files are stored. Either enter the path or click the “...” button and select the folder.
Tip:
The Temp File Location parameter lets you specify the location in which to store the temporary files generated by ncluster_loader and ncluster_export.
Configuring Loader settings
Table 5 - 38: Loader settings
Name
Auto Analyse
Auto Partition
Begin Script
•File Name
•Location Type
•Environment Variable
Description
Whether to analyze data after the loading is completed.
Whether to partition data after the loading is completed.
Name of the script file to run after loading the data.
Type of the location setting. Choose Environment Variable if the path to the file is stored in an environment variable. Choose Folder Path if you want to specify the folder where the file is stored.
Environment variable that contains the path to the file.
168 Aster Client Guide
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 38: Loader settings (continued)
Name
•Folder Path
Data Prefix
Date Format
End Script
•File Name
•Location Type
•Environment Variable
•Folder Path
Name of the script file to run after exporting data.
Type of the location setting. Choose Environment Variable if the path to the file is stored in an environment variable. Choose Folder Path if you want to specify the folder where the file is stored.
The environment variable that contains the path to the file.
The path to the folder where the file is stored. Either enter the path or click the “...” button and select the folder.
Error Logging
•Loader Error Logging
•Discard Errors
Whether to enable error logging on malformed rows.
Whether to discard malformed rows.
•Error Log Limit Maximum number of rejected rows before the loader aborts.
•Error Log Table Schema Schema of the custom error log table. This is a table that the loader creates in Aster Database. The loader logs loading errors into this table. If you do not specify A schema, the public schema is used.
This is the same as using the --el-schema option of ncluster_loader.
•Error Log Table
•Session Label
Name of the custom error log table.
Session label or identifier with which to tag rows in the custom error log table. The date and time are automatically appended to this label.
Error Log File
•File Name
•Location Type
Description
Path to the folder where the file is stored. Either enter the path or click the “...” button and select the folder.
CSV-compliant string to add to the start of each log.
ISO, SQL, POSTGRES, German, SQL_DMY, SQL_MDY, or
Postgres_DMY.
•Environment Variable
•Folder Path
Name of the log file in which to log errors.
Type of the location setting. Choose Environment Variable if the path to the file is stored in an environment variable. Choose Folder Path if you want to specify the folder where the file is stored.
Environment variable that contains the path to the file.
Path to the folder where the file is stored. Either enter the path or click the “...” button and select the folder.
169
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Table 5 - 38: Loader settings (continued)
Name
ForceLoader
LoaderIP
Timeout
Truncate Table
Vacuum Table
Description
Force the use of the preferred loader, which is specified by the
LoaderIP property. This is the same as using the -f option with ncluster_loader (ncluster_loader -f).
IP address of the preferred loader.
Aster Database connection timeout in seconds. Default is 30.
Whether to truncate empty tables before loading.
Whether to “vacuum” the target table in Aster Database if loading fails. In other words, if loading fails, the target table is returned to its state before loading.
If you set Session Label to
testLabel
and enable error logging as shown above, error messages are tagged with testLabel<date and time>
, as show in this example:
Loading tuples using node '10.65.197.95'.
35 tuples were loaded into table '"astertargetdb"."t_datetime_time"'.
Statistics for table: "astertargetdb"."t_datetime_time" with label
testLabel-2013-12-03-11-51-04-7283_Thrd_0_0
Total Tuples: 37
Loaded Tuples: 35
Malformed Tuples:2
35 tuples were successfully loaded into 1 tables.
You could then run a query on the error logging table using ACT to ret rive the malformed rows: ssistestdb=> select * from nc_errortable_repl where label like 'testLabel%';
key | tupletimestamp | label | targettable | dmltype | errmessage | sqlerrcode | rawdata
| linenumber | columnname
170 Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
------------+-------------------------------+-------------------------------------------
--+-------------------------------+---------+-------------------------------------------
------+------------+------------------+------------+------------
4294967297 | 2013-11-21 18:41:02.127619-08 | testLabel-2013-12-03-11-51-04-7283_Thrd_0_0
| astertargetdb.t_datetime_time | C | invalid input syntax for type time: "infinity"
| 22007 | "-1","infinity" | 1 | val
4294967297 | 2013-11-21 18:41:02.213283-08 | testLabel-2013-12-03-11-51-04-7283_Thrd_0_0
| astertargetdb.t_datetime_time | C | invalid input syntax for type time: "-infinity"
| 22007 | "-2","-infinity" | 26 | val
(2 rows)
Configuring Aster ODBC Connection Manager Settings
•
•
Configuring Data Source Specification
•
Configuring Connection String Properties
•
Testing Aster Database Connection
Aster Client Guide
Tip:
C:\Windows\System32\odbcad32.exe is for Windows 64-bit, while C:\Windows\SysWOW64\odbcad32.exe is for
Windows 32-bit. (WOW64 is Windows 32-bit running on Windows 64-bit).
Configuring Component Name
In the Name field, enter the name of the connection manager.
171
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Configuring Data Source Specification
Table 5 - 39: Data Source Specifications settings
Name
Use User or System Data
Source Name (DSN)
Use Connection String
Description
From the drop-down menu, choose a DSN defined in “ODBC Data
Source Administrator.”
Click this radio button to use a connection string to connect to Aster
Database. You can enter the string in the corresponding field or enter the connection string parameters in the Connection String Properties.
Configuring Connection String Properties
Table 5 - 40: Connection String Properties
Name
Server
Port
UID
Password
Database
Description
IP address or host name of the Aster Database server.
Server port number. Default is 2406.
Aster Database user ID.
Aster Database user password.
Aster Database name.
Testing Aster Database Connection
After configuring your Aster ODBC connection manager, click
Test Connection
to make sure that SSIS can access the specified Aster Database.
Internationalization and Locale support
Each component in the Teradata Aster SSIS Connector has a locale ID:
• Aster Export Source
The default ComponentLocale is “EN-US.”
• Aster Loader Destination
• The ComponentLocale is “EN-US.”
• The InputLocale is the input columns’ locale.
• Mapping DT_WSTR/NTEXT to any Aster data type.
• Transform the data from input locale to component locale.
• Only enabled when using SSIS checking.
172 Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Example: Using Aster Export Source and Aster Loader Destination
In this example, you create Data Flow Task that exports data to SSIS and loads it back to
Aster Database.
Create a New Project
1
Open SQL Server Business Intelligence Development Studio.
2
3
Choose
File > Project
Select
Integration Services Project
and click
OK
.
Add a Data Flow Task
1
From the Toolbox panel, drag a
Data Flow Task
into the Control Flow pane.
2
Double-click the task you just added.
Add an Aster Export Source Component
1
From the Data Flow Sources section of the Toolbox panel, drag an the Data Flow pane.
Aster Export Source
into
Aster Client Guide
2
Double-click the data flow source you just added.
173
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
3
In the Aster Export Source Editor, configure the Aster ODBC Connection Manager settings:
a
From the Aster ODBC Connection Manager drop-down menu, choose
<New>
.
4
5 b
In the Aster ODBC Connection Manager window, configure the Connection String
Properties fields to point to your Aster Database.
c
Click
Test Connection
to make sure that you can access the Aster Database and click
OK
.
Click
OK
to close the Aster ODBC Connection Manager window.
In the Aster Export Source Editor, configure the Aster Export Connection Manager settings:
a
From the Aster Export Connection Manager drop-down menu, choose
<New>
.
b
In the Aster Export Connection Manager window, specify the location of the ncluster_export.exe utility on your system.
If the path to it is stored in an environment variable, set the Location Type to
Environment Variable
. Then, from the Environment Variable drop-down menu, choose the variable that contains the path.
If you prefer specifying the path, set the Location Type to
Folder Path
. Then, in the
Folder Path field, specify the path.
c
Click
OK
to close the Aster Export Connection Manager window.
In the Aster Export Source Editor window, from the “Select Aster table” drop-down menu, choose a table from your Aster Database:
174
a
In the “Select Aster table columns” section, select or deselect the columns to export by checking the corresponding check box in the Include column.
Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
b
Click
Preview
to view what gets exported. Then click
Done
.
Aster Client Guide
6
In the Aster Export Source Editor window, click Columns.
a
Map Aster Database types to SSIS data types.
7 b
Click
OK
.
In the Aster Export Source Editor window, click
Error Output
.
a
Choose to redirect rows containing errors to the Error Output of the Aster Export
Source component.
b
Click
OK
.
175
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Add an Aster Loader Destination Component
1
From the Data Flow Sources section of the Toolbox panel, drag an component into the Data Flow pane.
Aster Loader Destination
2
Connect the output of the Aster Export Source component to the Aster Loader
Destination component.
3
4
5
6
7
8
Double-click the Aster Loader Destination component.
In the Aster Loader Destination Editor, From the Aster ODBC Connection Manager dropdown menu, choose Aster ODBC Connection Manager you configured earlier.
In the Aster Loader Destination Editor, configure the Aster Loader Connection Manager settings:
a
From the Aster Loader Connection Manager drop-down menu, choose
<New>
.
b
In the Aster Loader Connection Manager window, specify the location of the ncluster_loader.exe utility on your system.
c
Click
OK
to close the Aster Loader Connection Manager window.
In the Aster Loader Destination Editor window, from the “Select Aster table” drop-down menu, choose a table from your Aster Database into which to import the data.
Click Columns.
Map input column types to the external columns.
176 Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Redirect Rows with Errors to a Flat File
1
Add a Flat File Destination component to the Data Flow pane.
2
Connect the Error Output of the Aster Export Source component to the Flat File
Destination component.
3
If needed, configure the Error Output.
Aster Client Guide
Run the Package
To run a package:
1
In the Data Flow pane, right-click and choose
Execute Task
.
In this example, the execution succeeds. The export component exported 27 rows from
Aster Database to SSIS. Only one row was redirected to Error Output and stored in a flat file.
Only one row was redirected to Error Output and stored in a flat file.
2
Build a solution and try running it from the files system.
You can also use SQL Server Management Studio to run the solution. For more
information about running SSIS Connector packages, see Executing SSIS Connector
.
177
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Working with SSIS Connector Solution Packages
•
•
Executing SSIS Connector Packages
•
Optimized SSIS Connector Setting
Saving Packages as 64-bit
1
In SQL Server Business Intelligence Development Studio, Choose
<project_name> Property Pages
.
Integration Services
2
Set Run64BitRuntime to
True
.
3
Save a new version of the package.
Executing SSIS Connector Packages
After building SSIS Connection Solution packages, you can run the packages in two ways:
•
Executing Packages Using the Execute Package Utility
•
Executing Packages Using the DTExec.exe
178 Aster Client Guide
Tools for .NET Environments
Using the Teradata Aster Connector for SSIS
Executing Packages Using the Execute Package Utility
You can use the Execute Package Utility to run SSIS Connector packages.
Aster Client Guide
Executing Packages Using the DTExec.exe
You can use the DTExec.exe program to run SSIS Connector packages:
• For x86, use C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn\DTExec.exe.
• For x64, use D:\Program Files\Microsoft SQL Server\100\DTS\Binn\DTExec.exe.
For example:
C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn>DTExec.exe
-De aster4da ta -Rep V
-F D:\SSIS\PerformancePackages\x86\Aster2Aster_Default_200k.dtsx
Optimized SSIS Connector Setting
This optimized SSIS Connector setting results in improved performance:
• Execute Method:
• Execute Package Utility
• Aster Export Connection Manager:
• KeepTempFile = False
• Aster Loader Connection Manager:
• KeepTempFile = False
• ErrorLogging=False
179
Tools for .NET Environments
Possible Exceptions and Resolutions for .NET
• Aster Export Source:
• CSVFileReadBufferSize = 40960
• Aster Loader Destination:
• CLIExecuteModer = AtEndOfRowset
• ErrorCheck = Aster Check
• Control Flow Properties:
• EngineThreads = 10
• DefaultBufferSize = 104857600
• DefaultBufferMaxRows = 50000
• RunOptimizedMode = true
• MaxConcurrentExecutables = -1
Possible Exceptions and Resolutions for .NET
These are resolutions to possible exceptions:
• You might get an exception like “Exception: Unable to find the requested .Net Framework
Data Provider. It may not be installed.”
• Reason: Your 32-bit provider is not installed or is broken.
• Resolution: Reinstall your 32-bit provider and make sure that the size and version are correct.
• Loading assembly error. You might get an exception like:
Error -- 1056964601 : Internal error: The operation terminated unsuccessfully.Error -- 1055784933 : Errors in the high-level relational engine. The following exception occurred while the managed
IDbConnection interface was being used: Error loading assembly:
C:\Program Files (x86)\Teradata Aster\Aster .NET Data Provider
(x86)\AsterDataC#DSII.dll.
If this exception occurred in SSAS:
• Reason: Your 64-bit provider might not be installed correctly.
• Resolution: Reinstall your 64-bit provider and make sure that the size and version are correct.
• If this exception occurred in SSIS, then:
• Reason: The 32-bit Runtime not enabled.
• Resolution: Right-click the project and choose
Properties
. Then, set
Run64BitRunTime
to
False
.
• Note that in the SSAS view, no time zone information is shown for Aster Database columns of the date data type “time with time zone”. This is a known issue.
180 Aster Client Guide
CHAPTER 6
Loading Data
To load data into Aster Database, you can use the Aster Loader Tool, the COPY command, the
INSERT command, or a custom-defined SQL-MapReduce data loading function you have written. This section provides tips for efficient loading and shows how to load using the Aster
Loader Tool. The following sections explain these utilities:
•
Best Practices for Data Loading
•
•
Best Practices for Data Loading
This section describes best practices for efficiently loading data into Aster Database. In particular, this section explains how to help optimize new loading projects using the tools provided by the Aster Loader nodes and the Aster Loader Tool (ncluster_loader). The sections are:
•
•
Scenario 1: Pre-Production Data Loading
•
Scenario 2: Loading in a Production Environment
You can also load data using Teradata Studio, but loading data will be slower than with the
Aster Loader Tool. The Aster Loader Tool is a bulk loader, and as such it will perform faster.
Loading Terminology
We use the following terms in the text that follows:
Aster loader node: In the cluster, a loader node is a node dedicated to data loading. Many loader nodes can operate in parallel.
Aster Loader Tool (ncluster_loader) is the client application for initiating high-speed bulk loads.
Aster Database Load Error Logging is a feature in ncluster_loader that allows you to perform loading that is more tolerant of poorly formatted input data. Load Error Logging sends malformed rows to an error logging table.
Aster Client Guide 181
Loading Data
Best Practices for Data Loading
Input data: Source input file(s) which are to be loaded into Aster Database. All source files are compatible with a format that Aster Database is able to load. Examples include the CSV format (RFC 4180: http://www.rfc-editor.org/rfc/rfc4180.txt
) or the tab-delimited format. Each file must use a consistent newline character throughout. Never mix UNIX and Windows-style
newline characters in the same file; this will cause your load attempt to fail!
Data staging node(s): Nodes where all the input data is present. In a typical setup the input data is stored on the local filesystem. However, other use cases where all the data is stored on a network-mounted storage device are possible.
Row(s): In any given input data file individual rows are present. The used format for the input data describes where row boundaries are. Usually a “row” refers to a logical unit of information that needs to be stored as a unit in the data warehouse. One example for rows include call records which consist of source and destination phone numbers, call duration time, and so on.
The following section highlights three different data loading scenarios and the respective best practices that should be performed to make the process as a whole as seamless as possible.
Scenario 1: Pre-Production Data Loading
To validate your loading process and data before deploying it to your production system, it is an accepted industry best practice to load to a pre-production or testing system. You can perform test data loading using the standard Aster Database command-line, bulk loading tool, ncluster_loader. For a complete overview of the tool’s command-line options refer to
.
The tool’s error logging features are useful for debugging your loading process. If there is a possibility that your input data contains malformed rows, you should consider using error logging for bulk load operations to accomplish the following goals:
1
Make sure that even in the presence of malformed rows a given load operation succeeds.
This can be accomplished by enabling error logging but not setting an error logging limit.
(Set
--el-enabled
but do not set an
--el-limit
.) If you are not interested in what errors are present, malformed input rows can be discarded so that they are not stored in the cluster (
--el-enabled --el-discard-errors
).
Tip:
A UNIQUE or PRIMARY KEY violation in the data being loaded will always cause the load to abort. So if the table you are loading into contains these constraints and your load is failing, check the data you are loading to ensure it complies.
2
Store malformed rows into a separate table for later inspection.
You should specify a custom error logging table to store malformed rows. This is done via the appropriate option (
--el-enabled --el-table = 'my_errorlogging_table'
).
If the error logging feature of Aster Database is turned on without using
--el-table
to specify an error logging table, malformed rows are stored in the default system error logging tables ( nc_errortable_part
or nc_errortable_replicated
). This is undesirable because the rows from different loads will be mixed together in the default
182 Aster Client Guide
Loading Data
Best Practices for Data Loading
3
4 f
tables. Note that any custom error logging table has to inherit from the default system error logging table.
Abort data load operation in the presence of too many malformed rows.
This is in particularly useful if you want a given load operation to abort if too many malformed rows are present in the input data (
--el-enabled --el-limit = 100
). In order to preserve atomicity for bulk load operations, the load operation fails as a transaction when the error limit is reached. When the operation fails, any rows already written by the transaction to the target table and error logging table are deleted.
Label malformed rows in the error logging table.
If multiple operations are loading data into Aster Database at the same time (e.g., the preproduction system is testing integration of two separate data sources), you can label each load operation to identify which rows belong to which data source (
--el-enabled -el-label = 'my_data_source'
).
The following types of errors in input data files can be detected:
a
Malformed datatypes (e.g., text value for an integer column);
b
Missing column values (e.g., the input data file provides data values for the first two columns, but not for the third one);
c
Additional column values (e.g., the input data file contains a row with more values than there are columns in the destination table);
d
Check constraint violations (e.g., the integer value of an imported data field exceeds the range allowed by the check constraint on the target table);
e
Character set conversion errors (e.g., input data file contains invalid UTF-8 characters);
Missing or invalid distribution key (e.g., for your distribution key, you have specified a column that has a distribution-key-disallowed datatype).
Summary
In an environment where not all operations are run in an automated fashion, where new data sources are to be integrated, or where existing ETL processes are to be changed, we recommend that you set an error logging limit to prevent accumulation of too many malformed rows in the error logging tables. If malformed data rows need to be inspected, we recommend that you use a custom error logging table or create a separate error logging table for that load job.
Scenario 2: Loading in a Production Environment
Teradata Aster recommends that, before you bulk load data into your production system, you first verify your loading procedure on a test system. In this section, we’ll focus on the differences between a data loading job in a testing environment and one in a production system.
In a production deployment, we recommend that you follow the steps below to ensure you take advantage of the bulk loading and error logging tools in Aster Database:
Aster Client Guide 183
Loading Data
Best Practices for Data Loading
1
2
3
4
5
Define quote and escape characters that don’t conflict with the data.
If you have double-quote marks inside your data, the easiest way to have them kept as literals is to choose a character that does not occur anywhere in the input data file and use that as both the quote character and the escape character. Once the double-quote character is no longer treated as the quote character, it will be treated as a literal and will be preserved. So, for example, if your file has no tab characters in it (as either data or as a column separator), then you can preserve embedded double-quote characters by specifying
-q '\t' -e '\t' on the ncluster_loader command line.
Avoid extra spaces in your data, because they can be significant.
Quotes can be used on any portion of a data field, typically around special characters. For example, with the default CSV mode, this is the usual way to handle commas within a string:
1,2,"red,blue,green"
But putting quotation marks around each comma is equivalent:
1,2,red","blue","green
This can introduce problems when working with varchar columns, because many people put space between the comma and the quote, and that space is considered significant. For example:
1,2, "red,blue,green"
In this example, the third column will be loaded with a space before the r
character.
Consider performance when loading autopartitioned tables.
Beginning in Aster Database version 5.0, loading of logically partitioned tables has much lower overhead and performs better than previously. Specifically, performance is improved when loading data into a table with many child partitions. The improvement applies to tables that use either row or columnar storage, but is most pronounced for tables with row storage.
Run ANALYZE after you load each reasonably sized batch.
When bulk loading data, it’s important to run ANALYZE regularly. In ncluster_loader, use the
--auto-analyze
(or
-z
) flag to do this. With this flag present, ncluster_loader will run ANALYZE on the target table(s) after it loads the data. This flag also sets the hint bits on the table(s).
While running, ANALYZE requires an amount of disk space on the cluster proportional to the amount of new data in the table being analyzed. For this reason, Teradata Aster recommends that your run ANALYZE after every load, rather than waiting for multiple loads to finish. Waiting too long can result in a large amount of the cluster’s disk space being consumed during the running of ANALYZE. Daily child tables offer a good example:
When you load to daily child tables, run ANALYZE on each child table after you load the day’s data into that table.
Stay informed about cluster activity.
184 Aster Client Guide
Aster Client Guide
Loading Data
Best Practices for Data Loading
6
7
8
9
As with any other production system component, it is important to stay informed about any irregular activity that is going on in the cluster, in this case in particular what is happening in the data loading path.
If you want to make sure that large loads with too many malformed rows do not complete successfully, you can set an error logging limit for your load operation (
--el-enabled -
-el-limit = '100000'
). You can specify a limit clause with a value that is small enough to allow most loads to complete successfully while forcing the failure of loads with unexpected percentages of malformed rows. Exact values will depend on the size of the files being loaded and the expected malformed row ratios.
You should use an alert system to monitor the size of your error logging tables.
Keep an eye on the volume of data in the load error logging tables.
The system does not perform automatic truncation of the error logging tables! The error logging tables are regular tables in Aster Database. The same operations that an administrator would apply to regular tables thus need to be applied to error logging tables.
Tasks include the appropriate setup of child table hierarchies for efficient data processing, regular VACUUM operations to free up disk space occupied by dead rows, and the monitoring of table growth over time. Custom error logging tables can be used by passing the appropriate arguments when you run the Aster Loader Tool (for example,
--elenabled --el-table = 'el_table_April_2010'
).
Use an error logging file for faster re-try of rows that failed.
Have ncluster_loader create a file containing the row data that failed to load. To do this, pass the argument,
--el-errfile <my_error_file.txt> to the tool. Upon completion of the load, you can inspect the contents of the error logging file, fix issues you find there, and reload from that file.
Watch your load error statistics.
To monitor error rates, check the system statistics tables, nc_all_errorlogging_stats and nc_user_errorlogging_stats
. These tables show you the malformed and wellformed row count for each load operation. Records from the statistics tables can be correlated with the malformed rows in the load error logging tables. Both the error logging and statistics tables contain an error logging label which can be used to identify a particular loading operation.
Trace malformed input data to the data source.
An important aspect of a production system is that it must provide the user with the means to find the problem source when an operation fails. When loading data, the user can use the error information about malformed rows (SQL error code and error message, sqlerrcode and errmessage
, respectively, in the load error logging table) to diagnose the problem in the input data. To do this, you must invoke your data loading operation with the error logging feature enabled (
--el-enabled
). As described above, this information can then be correlated with the error logging statistics table to find the source of the malformed data. To facilitate this process, you may wish to apply custom error logging labels to each load operation (
--el-enabled --el-label =
'my_data_source'
).
185
Loading Data
Aster Loader Tool
Loading Best Practices Summary
For any production system, the close monitoring of system components is absolutely mission critical. Correlating error information present in the error logging tables can reveal possible problems early in the process.
Aster Loader Tool
The Aster Loader Tool, ncluster_loader, is a command-line application for bulk-loading data from files into Aster Database. It provides an alternative to the SQL COPY statement. Each invocation of the Aster Loader Tool is equivalent to a single invocation of the COPY statement. Each invocation is a single database transaction that loads the contents of the specified data file or files into the specified table or tables.
This section includes:
•
•
•
•
Loading Data with the Loader Tool
•
Removing Nulls from Data with Aster Loader Tool on Linux
•
•
Loading from Multiple Files Using a Map File
•
•
•
Syntax
To run the Aster Loader Tool, you type:
$ ncluster_loader [arguments] [schemaname.]tablename [ filename |
dirname ] where
•
arguments
are the command-line flags that control how the loader runs. The flags are explained in
, below, or you can display the help by typing:
$ ncluster_loader -?
•
schemaname
is the optional name of the destination schema. If no schema name is provided, Aster Database will search the schemas listed in the schema search path.
•
tablename
is the name of the destination table (See
Table Names” on page 187 if you wish to have Aster Database evaluate table names in a
case-sensitive manner.);
186 Aster Client Guide
Aster Client Guide
Loading Data
Aster Loader Tool
•
filename
or
dirname
indicates the file or directory of files to be loaded. Files to be loaded can optionally be compressed gzip or bzip2 files. These are extracted and their contents are then loaded.
•
filename
Qualified path of the file containing the data to be loaded. The contents of the file must be in either CSV or text format, as described for the COPY statement.
Details of the encoding used (such as non-default values for null or delimiter) are specified using the appropriate options, as described below; or
•
dirname
Qualified path of the directory containing one or more data files to be loaded. All data files found within this directory are expected to be in the same format and will be loaded as a single transaction. Subdirectories will not be processed.
If you don’t supply a file or directory name argument, Aster Loader assumes you want to load from STDIN. See
“Loading from STDIN Example” on page 200 .
Tip for Windows users:
When using Aster Loader Tool on Microsoft Windows, bear in mind:
•
If the filename
or dirname
contains spaces, make sure you enclose it in double quotes.
•
When specifying a dirname
, you must use a double backslash at the end of the path. For example, use
“c:\temp\loadFiles\\” to specify a directory called “loadFiles”.
• Never mix UNIX and Windows-style newline characters in the same data file. Doing so will cause your load attempt to fail.
Case-Sensitive Handling for Table Names
To force Aster Loader to treat your table and schema names in a case-sensitive manner, you must surround the schema and table names in double quote marks when you invoke ncluster_loader. To pass a double quote mark to Aster Loader, you must prefix it with the escape character, which is a backslash. That means you type a double quote mark like so:
\"
.
If either the schema name, the table name or both names include capital letters, you must surround each name in escaped quotation marks, individually.
For example, assuming you want to load to table "Bar" in schema "Foo", then you would invoke Aster Loader using this form: ncluster_loader.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"Foo\".\"Bar\" mydata.csv -c and in the map file, you would reference the table as:
"table" : "\"Foo\".\"Bar\"",
If you want to load to table "bar" in schema "Foo", you would still need to escape quote the schema and the table separately as follows: ncluster_loader.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"Foo\".\"bar\" mydata.csv -c and in the map file, as:
"table" : "\"Foo\".\"bar\"",
187
Loading Data
Aster Loader Tool
Argument Flags
In addition to the schemaname.tablename
and filename/dirname
arguments explained above, the Aster Loader Tool takes the following argument flags at the command line or in the map file. (Map files let you load from many input files in a single running of Aster Loader. See
“Troubleshoot Loading” on page 205
.)
In the table that follows, the argument flags are sorted based on the long-form, command-line flag:
• The left column lists the flag you use at the command line.
•
The middle column lists the flag you can use in a map file (See “Rules for Passing
Arguments in a Map File” on page 198
.). If no value appears in the middle column, then the argument is one that can only be passed at the command line.
Note that you can always specify a loader node using its IP address. If you wish to specify it by hostname, you first need to add the loader node to the Aster Database hosts file on all nodes through the Hosts tab in the AMC.
Table 6 - 41: Argument Flags for Aster Loader Tool
Flag at Command Line Flag in Map File Description
-a [ --auto-partition] auto-partition
Specifies that ncluster_loader will run in autopatitioning mode, which automatically routes each row to the appropriate child table in a parent/child table created through inheritance.
-B [ --begin-script ]
arg
-c [ --csv ] begin-script
Specifies the qualified path of the file containing SQL commands that should be executed when the transaction starts, i.e. immediately after the BEGIN command is issued to Aster Database. Note: Datareturning statements such as SELECT are not allowed in scripts executed by the ncluster_loader.
Pass this argument if you want ncluster_loader to interpret the input file as being in CSV format. By default ncluster_loader uses a text file with a tab-delimited format, often called “TSV”.
-C [ --columns ] arg
-d [ --dbname ] arg columns dbname
An optional comma-separated list of the columns in the target table that will receive the data being loaded (the default is to assume that each input row contains exactly one data value for each column in the target table, in the order in which the table columns were defined).
By using -C, you can specify that the ordering of the columns in the input file is different from the ordering in the table. You can also use
-C
to specify that this input file contains values for only some of the columns in the table.
The input data is assumed to contain values for the columns in the order specified here. For example, to load data into columns 'col1' and 'col2', one could specify "col1, col2" as the value for this option.
Column names not specified here are expected to get NULL values.
See
“Using the -C Option With Uppercase or Special Characters” on page 206
.
The name of the database to connect to (default is "beehive").
188 Aster Client Guide
Loading Data
Aster Loader Tool
Table 6 - 41: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
-D [ --delimiter ] arg
--date-style arg
Flag in Map File
date-style
Description
The column delimiter character to use when interpreting the input file (must be a string that represents a valid single character, such as
'd'
or '\n'). The default is a tab character ('\t') in text mode, a comma in CSV mode.
Specifies the date format. If you are using a map file, you must specify a single date-style per map file.
The following formats are supported:
• ISO - This is the default. Uses ISO 8601-style dates and times
(YYYY-MM-DD HH:MM:SS) like, for example,
2010-12-17 07:37:16-08
• SQL - Uses Oracle/Ingres-style dates and times. For example:
12/17/2010 07:37:16.00 PST
• POSTGRES - Uses traditional PostgreSQL format. For example:
Wed Dec 17 07:37:16 2010 PST
• German - Uses dd.mm.yyyy for numeric date representations. For example:
17.12.2010 07:37:16.00 PST
• SQL,DMY - For example:
17/12/2010 15:37:16.00 CET
• SQL,MDY - For example:
12/17/2010 07:37:16.00 PST
• Postgres,DMY - For example:
Wed 17 Dec 07:37:16 2010 PST
-e [ --escape ] arg
-E [ --end-script ] arg end-script
--el-discard-errors
--el-enabled
Specifies the escape character for CSV input (must be a string that represents a valid single character, such as d or \n). The default escape character is the same as the default quote character - the double-quote (").
If an explicit quote character (-q) is specified, and no explicit escape character (-e) is specified, then the escape character will be set to be the same as the quote character specified.
You may only pass this option when loading a CSV-formatted file (-
-csv
). Note that ncluster_loader will fail if you pass this argument when loading a tab-delimited text file.
Qualified path of the file containing SQL commands that should be executed when the transaction is about to commit, i.e., immediately before the END/COMMIT command is issued to Aster Database.
Note: Data-returning statements such as SELECT are not allowed in scripts executed by the Aster Loader Tool.
discard-errors
Sub-flag that can accompany --el-enabled. If present, specifies that all errors (malformed rows) be discarded by Aster Database. By default, errors are not discarded.
errorlogging
If present, turns on error logging for this invocation of the ncluster_loader. This needs to be enabled for any other error logging option to be accepted. The default is disabled. See
Aster Client Guide 189
Loading Data
Aster Loader Tool
Table 6 - 41: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
--el-label arg
Flag in Map File
label
-F --el-errfile arg
-L [ --el-limit ] arg
--preprocess-script arg
-S [ --el-schema ] arg errfile limit schema
Description
Sub-flag that can accompany --el-enabled. The --el-label flag specifies the label to be used to tag the malformed rows. By default, Aster Database will use a statement identifier as the label value. (There’s one statement identifier per COPY command; if there is one map entry for many input files, then you’ll have a unique statement identifier per input file.) The label is useful for finding your failed rows in the error logging table and for finding statistics about the load attempt in the nc_all_errorlogging_stats table.
Sub-flag that can accompany --el-enabled. The --elerrfile
flag introduces the pathname of the optional error logging file. If you use error logging, you must have an error logging table, and you can have an error logging file. Upon completion of the load, ncluster_loader writes the contents of the error logging table’s rawdata
column (and no other columns) to the error logging file.
Only the contents of this column are written to the file. The filename will have a numeral appended to it in the form, “_0”.
For this option to work, you must have also specified an el-table.
Regardless of whether or not you specify that an error logging file should be used, the error logging table will still contain all error rows upon completion of each load.
Sub-flag that can accompany --el-enabled. The --el-limit flag sets the maximum number of malformed rows allowed for each file being loaded. By default the limit is unbounded. If the error count during your load attempt exceeds the specified --el-limit, the transaction aborts and no rows are inserted into the destination table, and no rows are inserted into the error logging table.
Supported on Linux only. This flag executes a shell script against the source file, and uses the output of the script as the new source file for loading. You can use this feature along with the provided script remove-null.sh
to remove NULL(0x00) values from data to be loaded using Aster Loader Tool on Linux. For non-Linux operating systems, use the workaround
“Removing NULLs from Data” on page 196
.
Sub-flag that can accompany --el-enabled. The --el-schema flag specifies the name of the schema to which the error logging table belongs. If not specified, the public schema is used.
190 Aster Client Guide
Loading Data
Aster Loader Tool
Table 6 - 41: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
-T [ --el-table ] arg
Flag in Map File
table
-f [ --force-loader ] force-loader
filename
(Not a flag; you pass the file or directory name itself!)
-? [ --help ]
-h [ --hostname ] arg
--header-included
-l [ --loader ] arg file loader
Description
Sub-flag that can accompany --el-enabled. The --el-table flag specifies the name of the error logging table that will capture malformed rows.
If you are loading to a distributed table, the error logging table must be a distributed table. Likewise, if you are loading to a replicated table, the error logging table must be a replicated table.
If this flag is not provided, Aster Database uses a default table: nc_errortable_repl
is used if the load-destination table is a replicated table, or nc_errortable_part is used if the loaddestination table is a distributed table.
Important: The system does not perform automatic truncation of the error logging tables! You must VACUUM them regularly.
Instructs the Aster Loader Tool to use the loader node specified with the -l or --loader option even if the IP address provided is not known to Aster Database. Note that if this option is specified, the
Aster Loader Tool will only try that single IP address and return an error status if the connection fails for any reason. If this options is not used, Aster Loader Tool will attempt to choose the least busy loader node available.
Qualified pathname of the file or directory containing data to be loaded. The contents of the file must be in either CSV or text format, as described for the COPY statement.
If you pass a directory name, then all data files found in the directory are loaded in a single transaction. Subdirectories are ignored. All files must be in the same format.
If you pass no file or directory name, Aster Loader loads from
STDIN.
In a map file, use the file flag. At the command line, there is no flag for this argument; instead, ncluster_loader assumes that you pass the named-flag arguments first, then the table name, and finally the source file name.
Shows online help for Aster Loader Tool.
Host name or IP address of the Aster Database queen. Default is
'localhost'. This always points to the queen, even if you’re loading using loader nodes. See
“Multiple Loader Nodes” on page 202 .
Indicates that the first line of the data file to be input is a comma delimited list of the column names. The number of skip-rows will be counted beginning after the header, if this option is specified.
Cannot be combined with --columns.
Preferred loader IP address. If a value is provided, the Aster Loader
Tool tries to load to this IP address before trying any other loader node. Note that the Aster Loader Tool expects this IP address to be known by Aster Database and will ignore the address provided if this is not the case. To change this behavior one can use the -f or -force-loader
flag.
Aster Client Guide 191
Loading Data
Aster Loader Tool
Table 6 - 41: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
-m [ --map-file ] arg
Flag in Map File
-n [ --null ] arg
--null-backslashescapes
-p [ --port ] arg
-P [ --data-prefix ]
arg
-q [ --quote ] arg
--skip-rows arg
-t [ --timeout ] arg data-prefix timeout
Description
Name of the file containing mappings of data files or directories to target tables. This option allows you to load multiple directories into a table and also to load data into multiple tables within the same transaction. For details about the format used for the map file, see
“Troubleshoot Loading” on page 205 .
String that is used to represent a null value. The default is \N in text
(non-CSV) mode, and an empty value with no quotes in CSV mode.
When loading any non-CSV delimited format (e.g. TSV), you can easily load files that contain empty strings (that is, files that don’t use the typical \N to represent nulls). To do this, run ncluster_loader with the -n flag, followed by two doublequote characters. That is, you type: ncluster_loader -n ""
See also
.
Indicates that backslashes in the '-n/--null' flag should be treated as special characters, which will be processed according to the default rules for escaped strings.
You should use this flag whenever the null string contains a backslash, to ensure compatibility with future versions of Aster
Database.
TCP port on which Aster Database listens for connections. Default is
2406.
String that should be used as a prefix for each row (each line in a data file) being loaded into Aster Database.
Important: The Aster Loader Tool will simply concatenate this value to each row without any parsing. You must insert the proper delimiter, quote, and escape characters in the prefix string.
Character to be used as the quoting character for CSV input (must be a string that represents a valid single character, such as d or \n).
The default is the double-quote. This option is only valid when CSV input has been specified (-c or --csv) and will cause the load to fail otherwise.
If an explicit quote character (-q) is specified, and no explicit escape character (-e) is specified, then the escape character will be set to be the same as the quote character specified.
Note that if using a single quote (') as the quoting character, you should escape it when specifying it with -q as -q"\'". Otherwise any input data that includes a comma will cause the load to fail.
Specifies how many rows of the file to skip before starting to load data. If combined with --header-included, --skip-rows will not start counting until after the first line in the file. The default is 0.
Timeout value in seconds for Aster Database connection attempts.
Default is 30 seconds.
192 Aster Client Guide
Loading Data
Aster Loader Tool
Table 6 - 41: Argument Flags for Aster Loader Tool (continued)
Flag at Command Line
tablename
(Not a flag; you pass the table name itself!)
--truncate-table
-U [ --username ] arg
-v [ --vacuum-table ]
-V [ --version ]
--verbose
-w [ --password ] arg
-W [--password-prompt]
-z [ --auto-analyze ]
Flag in Map File Description
table
Name of the destination table. In a map file, use the table flag. At
the command line, there is no flag; instead, ncluster_loader assumes that you pass the named-flag arguments first, then the table name, and finally the source file name or directory name. truncate-table
Performs a TRUNCATE operation to empty the table before beginning the load. If child tables are present, they will be truncated, too. By default, this feature is disabled.
username
If present, the Aster Loader Tool connects to the database with this username instead of the default ("beehive"). (The user account must have permission to do so, of course.)
Runs VACUUM on the target table if the load fails. If target child tables or partitions are present, they are VACUUMed, too. By default, this feature is disabled.
Print the Aster Loader Tool version and exit.
Run in verbose mode.
password
Connect to the database using the specified password (as opposed to providing a password at the prompt).
Forces the Aster Loader Tool to prompt for a password before connecting to a database. It should automatically prompt for a password whenever the server requests password authentication. If no password prompt is issued and the server requires password authentication, the connection attempt will fail.
Runs ANALYZE on the table(s) after loading the data and sets the hint bits on the table(s). By default, this is disabled. If data was loaded to child tables via autopartitioning, they will be analyzed, as well.
Note that analyzing a columnar table may be slow if there are many columns. To improve the speed of statistics collection, execute a separate ANALYZE command after the load that only processes the columns involved in query row filters or grouping.
Exit Status
This table lists the possible return values upon exiting a data load:
Table 6 - 42: The Aster Loader Tool exit values
2
3
Exit
Value
0
4
Description
The Aster Loader Tool terminated successfully.
An error internal to the Aster Loader Tool was detected.
Another error was detected.
Failed to establish a connection to Aster Database.
Aster Client Guide 193
Loading Data
Aster Loader Tool
Table 6 - 42: The Aster Loader Tool exit values (continued)
Exit
Value
5
6
Description
An error was detected while communicating with Aster Database.
Malformed input data detected.
Loading Data with the Loader Tool
Connecting
The Aster Loader Tool is a regular Aster Database client application. In order to connect to an
Aster Database, you need to specify the host name or IP address of the queen node via the command line option
-h
. If the connection could not be made for any reason (e.g., insufficient privileges, server is not running on the targeted host, etc.), the Aster Loader Tool will return an error and terminate.
4
5
Procedure:
To load data with the Aster Loader Tool, do this:
1
2
3
Install the Aster Loader Tool on your data staging machine. (See “Installing the Loader
.)
If you have not already added one or more loader nodes to your Aster Database cluster, create them now in the
Admin>Cluster Management>Nodes
tab in the AMC.
Prepare the file or files that contain the data you wish to load:
a
For hints on file formatting, see the descriptions of the
--csv
,
--delimiter
, and
--null
arguments, below. Use a consistent newline character in your input file(s)!
6 b c
Determine any special parsing hints you need to provide to the loader routing. See the descriptions of the
--escape
,
--quote
, and
--data-prefix
arguments, below.
Place your data input file(s) on the data staging machine.
Figure out how you want to handle records that fail to load. See the section “Error
. Teradata Aster recommends that you create an error logging table that will receive rows that fail to load. Be prepared to query the nc_all_errorlogging_stats
table for statistics about your load attempt.
Figure out which advanced options of the Aster Loader Tool you will use:
a
Determine your mapping of the input file’s field values to the columns of your target table. See the description of the
--columns
argument, below.
Do you need to load from multiple files or insert into multiple tables? If so, see the description of the
--map-file
argument and read the section,
b
Do you need to automatically partition data when loading parent child tables with inheritance? Use the
--auto-partition argument and read the section,
Parent Child Tables with Inheritance” on page 202
194 Aster Client Guide
Loading Data
Aster Loader Tool
7 c
If you need to run an SQL script before and/or after the data is loaded, read the descriptions of the
--begin-script
and
--end-script
arguments, below.
Load your data by running the Aster Loader Tool on your input file(s). See “Argument
Flags” on page 188 for a list of the command-line options. The command you type will be
similar to:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" customers input_data.txt
8
Check the results of your load attempt by querying the statistics tables, nc_all_errorlogging_stats
and nc_user_errorlogging_stats
.
9
Check your error logging table(s) for rows that failed to load. Be aware that if the load failed entirely (for example, if the number of errors exceeded the
--el-limit
), then no new rows will appear in your target table, and no error rows will appear in the error logging table.
10
If you wish to import the failed rows, find each row of input data that failed, edit it to fix it, and combine these fixed rows into a new input file. Re-run the Aster Loader Tool on the new input file.
Removing Nulls from Data with Aster Loader Tool on Linux
A SQL error occurs when data containing ASCII null (embedded in a non-empty VARCHAR field) is inserted into the Aster Database. This error can arise when data is inserted into the
Aster database through any means, including ncluster_loader and the Teradata Connector.
Note that the ASCII null character is unrelated to the concept of NULL values in the database.
When Aster encounters ASCII null in the data that is being inserted, it treats the null character as invalid UTF-8.
The option
--preprocess-script
for Aster Loader Tool on Linux executes a shell script against the source file, and uses the output of the script as the new source file for loading. You can use this feature along with a provided script remove-null.sh
to remove NULL(0x00) values from data to be loaded using Aster Loader Tool on Linux.
2
3
To install and use the shell script:
1
Copy the script remove_null.sh
from the Aster Client package to the client machine.
The script can be found in the directory:
• For 64-bit Linux: stage/home/beehive/clients-linux64/shell-scripts/ remove-null.sh
• For 32-bit Linux: stage/home/beehive/clients-linux32/shell-scripts/ remove-null.sh
Ensure that the directory where remove-null.sh
is located is in your PATH.
Set the file permissions on the script to make it executable:
4
# chmod +x remove-null.sh
Use this syntax when invoking Aster Loader Tool:
$ ./ncluster_loader -h 10.0.0.10 -U beehive -w beehive -d beehive
--preprocess-script=remove_null.sh target_table source_file.csv
Aster Client Guide 195
Loading Data
Aster Loader Tool
Removing NULLs from Data
A SQL error occurs when data containing ASCII null (embedded in a non-empty VARCHAR field) is inserted into the Aster Database. This error can arise when data is inserted into the
Aster database through any means, including ncluster_loader and the Teradata Connector.
Note that the ASCII null character is unrelated to the concept of NULL values in the database.
When Aster encounters ASCII null in the data that is being inserted, it treats the null character as invalid UTF-8. The following workaround applies to data being loaded from Teradata, but the same concepts (though not the same syntax) apply when loading data from other systems.
When loading from a Teradata system, the easiest workaround is to identify the CHAR or
VARCHAR field with the ASCII nulls and then add a construct like this to your SELECT statement:
WHERE fieldname NOT
LIKE ''%''||''00''XC||''%'')
The ‘00’XC is Teradata syntax to specify the character with the ASCII value 00.
If you can't afford to exclude those records, then you will need to update the values to replace the ASCII nulls with spaces or some other valid ASCII/UTF-8 character, either in the table or in the query that is being run on the Teradata system so that the ASCII nulls are filtered out before they reach the Aster Database.
Loading from Multiple Files Using a Map File
With the Aster Loader Tool you can load from multiple data files and insert into multiple tables in a single invocation by passing the
--map-file
or
-m
option. We refer to this as loading with a map file. All tables are loaded in a single database transaction.
When loading multiple files with the
--map-file
option, the Loader Tool checks for invalid
UTF-8 characters in the first line or in the second line (if there is a header). If an error is found, file loading is stopped but the Loader Tool does not cancel the transaction. This means that if there are other files to be loaded in the map file, the Loader Tool can continue gracefully attempting to load the other files in the same transaction. The Loader Tool will indicate that the file was skipped.
Note that the Loader Tool only checks the first line of the input files or in the second line (if there is a header). Other lines are checked by the server. If errors are found by the server, they are reported and the transaction is aborted.
If there are invalid UTF-8 characters in the input file, you may have to manually convert the data files into UTF-8 encoding. The following is an example of using iconv
to convert the encoding: iconv --from-code=WINDOWS-1252 --to-code=UTF-8 <input.dat>
--output utf8.dat
When loading a very large amount of data, you may choose to create multiple map files that each load their data files using a different loader. This can help speed up the process of loading a large amount of data. Note that a map file loads files serially.
196 Aster Client Guide
Aster Client Guide
Loading Data
Aster Loader Tool
Procedure:
1
Prepare your data files for import. Each file should be formatted as usual for the Aster
Loader Tool. All the files you submit in a single running of the Aster Loader Tool must be in the same format.
2
You can optionally pass connection parameters in the map file. Any connection parameters specified on the command line supercede those in the map file. The connection parameters you can specify are:
3
• dbname
- the name of the database
• username
- the name of the Aster Database user
• password
- the password of the Aster Database user
• loader
- the hostname or IP address of the Aster Loader node
• force-loader
- same as the
-f
command line option. Instructs the Aster Loader Tool to use the loader node specified with the loader
parameter even if the IP address provided is not known to Aster Database. Note that if this option is specified, the Aster
Loader Tool will only try that single IP address and return an error status if the connection fails for any reason.
• timeout
- same as the
-t
command line option. Specifies the timeout value in seconds for Aster Database connection attempts. Default is 30 seconds.
Prepare your map file. Note that a map file loads files serially. The map file is a text file containing a set of logical text blocks, each surrounded by curly braces. Each block represents a file or directory to be loaded. The format is like this:
{
"dbname" : "beehive",
"username" : "beehive",
"password" : "beehive_pwd",
"loader" : "141.206.66.28",
"force-loader" : true,
"timeout" : 5,
"loadconfig" :
[
{
"table" : "schema1.targettable1",
"file" : "data/insert1.txt",
"errorlogging" : { "enabled" : true }
},
{
"table" : "schema1.targettable1",
"file" : "data/insert2.txt",
"begin-script" : "input/mapfile/begin-script.sql",
"end-script" : "input/mapfile/end-script.sql",
"errorlogging" :
{
"enabled" : true,
"discard-errors" : true
"label" : "insert2_log",
"schema" : "nc_system",
"table" : "nc_errortable_part"
}
]
}
197
Loading Data
Aster Loader Tool
198
4
In the above example, we assume the current directory (from which we invoke ncluster_loader) contains a subdirectory, data
, which has two files, insert1.txt
and insert2.txt
, and we load these both into table targettable1
in schema1
. Error logging is turned on. For the second table, additional error logging parameters are supplied to log to a system table, label each row and skip errors.
Each block in the map file must contain the following required parameter flags and their values:
• table
specifies the name of the target table. Typically you will schema-qualify the table name as shown in the example above. You may omit the schema, in which case the user’s schema search path determines the schema. If your table name is case sensitive, you must surround the table name with backslash-escaped double quote marks (
\"
), like this:
"table" : "schema1.\"TargetTable\"",
• file
specifies the name of the file or directory to be loaded. See
filename
in
• begin-script
and end-script
specify scripts to run either before or after the loading of the file. Both are optional. Each map file entry can have a separate beginscript
and end-script
. For each map file entry, ncluster_loader will run the begin-script
, load the data from the file, then run the end-script
. Do not include commands that begin or end the transaction in the script files. Empty lines in the scripts will be ignored.
Here is an example of a valid script file:
DROP table IF EXISTS foo;
CREATE table foo (id int, sometext varchar(40)) DISTRIBUTE BY HASH
(id);
• errorlogging
introduces a block that specifies how to handle malformed rows in the input file. Inside the errorlogging
block (enclosed in curly braces) you pass the enabled
parameter and, optionally, the parameters discard-errors
, label
, limit
, schema
, and table
. These parameters correspond, respectively, to the command-line flags,
--el-enabled
,
--el-discard-errors
,
--el-label
,
--el-limit
,
-schema
,
--el-table
.
Each block in the map file can contain the optional parameter flags listed in the middle
column of the table, “Argument Flags” on page 188 .
Run the Aster Loader Tool, passing the
--map-file
or
-m
option and the name of your map file. Pass additional command line flags as needed, observing the rules set forth in the preceding paragraph.
Rules for Passing Arguments in a Map File
Observe the following rules when you pass loader arguments in a map file:
1
2
Each block in a map file can contain the optional parameter flags listed in the middle
column of the table, “Argument Flags” on page 188 . Observe all rules spelled out there.
If you wish to use these flags, you must pass them inside the map file, and you cannot use their command-line equivalents.
Aster Client Guide
Aster Client Guide
Loading Data
Aster Loader Tool
3
Those flags that have no value listed in the middle column can also be used when you’re loading with a map file, but you must pass them at the command line, instead.
Example Map File
The example file reproduced here shows various valid combinations of loading parameters.
Note that the third block in the example (the one inserting into “ test3
”) instructs the Aster
Loader Tool to load all the files it finds in directory “ my_data_dir
”.
{
"dbname" : "beehive",
"username" : "beeuser",
"password" : "jw8s0F4",
"loader" : "10.51.6.240",
"force-loader" : true,
"timeout" : 5,
"loadconfig" :
[
{
"table" : "test1",
"file" : "data/insert.text",
"errorlogging" : { "enabled" : true }
},
{
"table" : "test2",
"file" : "data/insert.text.n",
"errorlogging" : { "enabled" : true }
},
{
"table" : "test3",
"file" : "my_data_dir/",
"errorlogging" : { "enabled" : true }
},
{
"table" : "test6",
"file" : "data/insert.text.incorrect",
"errorlogging" :
{
"enabled" : true,
"label" : "vm_test_12-test6"
}
},
{
"table" : "test12",
"file" : "data/insert.text.incorrect",
"errorlogging" :
{
"enabled" : true,
"label" : "vm_test_12-test12",
"discard-errors" : true
}
},
{
"table" : "test13",
"file" : "data/insert.text.incorrect.pk",
"prefix" : "0",
"columns" : "a,b",
"errorlogging" :
199
Loading Data
Aster Loader Tool
{
"enabled" : true,
"label" : "vm_test_12-test13",
"limit" : 100000,
"schema" : "public",
"table" : "nc_errortable_part"
}
}
]
}
Examples
200
Simple Loading Example
Assume the following:
• File to be loaded:
2010MarchSales_data.txt
(assume current working directory)
• Number of rows: 1000 rows encoded in text format
• Delimiter: delimited by character "~"
• Password: beehive
• Destination table: sales_fact
• Destination Aster Database: IP address 10.50.25.100
• Username: assume default
• Database: assume default
This is what you would type:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" sales_fact
2010MarchSales_data.txt
After successful completion, you will see the following output indicating 1000 rows loaded:
1000 tuples were successfully loaded into table customers.
Loading from STDIN Example
To load from STDIN, you simply omit the file name/directory name argument. (When loading from a data file, the last argument to ncluster_loader is the data file name.) For example, to import our UK sales data and replace the product name “NappyPlus” with
“DiaperPlus” we could use sed and ncluster_loader together like so:
$ sed s/NappyPlus/DiaperPlus/ 2010_UK_sales.csv | ./ncluster_loader -U mjones -w st4g0l33 sales_fact -c -l 10.51.50.240
Loading from STDIN lets you pipe your data through useful tools like sed, the text stream editor. Here’s an example of sed at work, ensuring that backslash escape sequences are properly formatted in the input data before Aster Loader sees the data.
Here’s some sample data:
# cat sampleData-3.tsv
5 How often do back-slash characters ('\') appear in your data?
6 And how often do you think they actually disappear: 1 \ ? 2 \ ? 3 \ ?
7 \W\a\y \t\o\o \o\f\t\e\n\! \! \!
Aster Client Guide
Loading Data
Aster Loader Tool
Here’s how we run Aster Loader, piping its input data through sed:
$ cat sampleData-3.tsv \
| sed -e 's_\\_\\\\_g' \
| ncluster_loader -h $QUEEN_IP -d my_db -U beehive -w beehive testo / dev/stdin
Loading tuples using node '192.168.28.100'.
3 tuples were successfully loaded into table 'test'.
Here are the result rows:
$ act -h $SYSMAN_IP -d my_db -U beehive -w beehive -c 'SELECT * FROM testo ORDER BY id;'
id | string
----+---------------------------------------------------------------------
1 | This is just a line.
5 | How often do back-slash characters ('\') appear in your data?
6 | And how often do you think they actually disappear: 1 \? 2 \? 3 \?
7 | \W\a\y \t\o\o \o\f\t\e\n\! \! \!
(4 rows)
Example with Error Logging
Use the same assumptions as in the previous example, and assume we will log malformed rows (that is, rows that the loader cannot interpret and therefore cannot load) to a table called
“
2010MarchSales_error_table
,” tagging each error row with the label
“
2010MarchSalesErr
”. At the end of the load attempt, the error data will also be copied to the file,
/home/ccrisp/2010MarchSales_error.txt
. We’ll set a limit of 100 error rows; if more than 100 errors are encountered, the load will be cancelled.
To do this:
1
Create the custom error logging table: Run ACT as a user with table creation rights (for example, a user with the catalog_admin role) and type:
CREATE TABLE 2010MarchSales_error_table () INHERITS
(nc_errortable_part);
2
Exit ACT, return to the command line, and type:
$ ./ncluster_loader -h 10.50.25.100 –w beehive -D "~" --el-enabled
--el-label
2010MarchSalesErr --el-limit 100 --el-table
2010MarchSales_error_table --el-errfile /home/ccrisp/
2010MarchSales_error.txt sales_fact 2010MarchSales_data.txt
For more information on logging malformed rows, see
.
Hints for Successful Loading
Recommended Character Set Is UTF-8
The default character set for Aster databases is UTF-8, and the Aster team recommends that you load only UTF-8 formatted data when loading to char, varchar, and text columns.
For the tools you use to prepare files and to connect to Aster, make sure you have set the default character set to UTF-8. This is particularly important if you are loading data from a
Windows-based machine. For example, if you will use an SSH client (e.g., putty) to run ncluster_loader, make sure you set the SSH client’s default character set to UTF-8.
Aster Client Guide 201
Loading Data
Aster Loader Tool
We recommend that, prior to loading, you convert your text files to UTF-8. For example, if you’re a Notepad++ user, you can use the command, “Convert to UTF8 without BOM.”
Newline Character
Make sure your data file uses a consistent character to represent newlines. If the file uses
\r\n for newlines, then it should not also use
\n
for newlines, and vice versa. If your file contains both UNIX-style
\n
newlines and Windows-style
\r\n
newlines, then you must clean the file before you try to load it. The UNIX command, dos2unix
, can be useful for doing this.
Multiple Loader Nodes
The Aster Loader Tool supports the use of many Aster Loader nodes. For most loading tasks, the queen is sufficient to handle all loading, but for high volume loading, you can add dedicated loader nodes to your cluster.
To use a loader node, you invoke one or more ncluster_loader instances that will load through that loader node. You may run many ncluster_loader sessions in parallel against one loader node, and you may use many loader nodes in parallel (with each node handling loads from a number of ncluster_loader instances).
To do this, you invoke each ncluster_loader instance with the
-l
(and optionally
-f
) argument to specify the loader node. The required flags are:
• as always, the
--hostname
option (
-h
) provides the queen IP address;
• the
--loader
flag (
-l
) provides the IP address of the desired loader node; and
• Optionally, the
--force-loader
flag (
-f
) forces the use of the desired loader node.
Loading Parent Child Tables with Inheritance
The
--auto-partition
option is retained in order to support parent/child tables created with inheritance. It is not used when working with parent/child tables created with autopartitioning. Using
--auto-partition
instructs the Aster Loader Tool to automatically send each row to the right child table during loading. Each row is directed to a table according to the check constraints you have set up on the child tables.
For example, if you partition your data into daily child tables based on the contents of a timestamp
column, each ultimate child table in your schema will have a CHECK constraint that specifies what value of timestamp
may be loaded into that child table. When you load data, the autopartitioning feature will route each row to the appropriate child table, based on its timestamp
value.
Use autopartitioning like this:
202 Aster Client Guide
Aster Client Guide
Loading Data
Aster Loader Tool
1
Set up the parent-child table schema in your database. On each ultimate child table, write a CHECK constraint that specifies what data may be loaded into that child table.
Notice!
Aster Database does not detect overlapping constraints on peer child tables. As a result, the correct placement of a row during loading can be indeterminate.
Workaround: Take care that the constraints you define do not create overlapping logical partitions. A simple mistake would be to set up range constraints like this:
CHECK ( ymdh BETWEEN '2005-07-01' AND '2005-08-01' );
CHECK ( ymdh BETWEEN '2005-08-01' AND '2005-09-01' );
In this example, it is not clear in which partition the ymdh value '2005-08-01' resides.
2
3
Prepare your data for loading:
a
Your data input file can contain data values that will end up in many different child tables.
b
To handle rows that do not fit your partitioning scheme, you can rely on the standard
error logging feature of the Aster Loader Tool (see “Error Logging” on page 204
) or create a check constraint that will catch rows that you do not want to include in your partitions.
Run the Aster Loader Tool with the
-a or
--auto-partition
flag.
Detecting UNIQUE and PRIMARY KEY Violations Before Loading
Detecting UNIQUE and PRIMARY KEY violations in the data to be loaded is not always straightforward. In many cases the source is not a database you can easily run a query on to detect non-unique keys. Some techniques you can use to detect these conditions in your source data:
• Build a version of the target table in the target database without a UNIQUE or PRIMARY
KEY constraint, load the data, then run a “detect duplicates” query to find the problematic rows/keys. In some cases only loading a sample of the data is sufficient to provide enough clues to find and fix the problem in the source data.
• An alternative step (using an ETL tool) would be to use this “keyless” version of the target table as a staging/temp table, which would load then check for issues like duplicate keys and dump them to a second error table. If no issues are found, then transfer the data to the final destination table.
• If the source is a database, then run the “detect duplicates” query there.
Using COPY with Columnar Tables
A loading operation using the Aster Loader Tool, COPY, or INSERT can be expensive when the following conditions exist:
• the target table uses columnar storage, AND
• the target table has many logical partitions, AND
• the loaded data matches many different logical partitions.
203
Loading Data
Aster Loader Tool
In this case, the memory allocated to perform the load may be as large as the amount of source data. To avoid high memory requirements, it is best to divide a large load into batches. There are two alternative approaches:
• Ensure that each batch only loads a small number of logical partitions in the columnar table. For example, when inserting data into a columnar table with weekly partitions, each batch may insert data for a single month.
• Ensure that the size of each batch is a small fraction of system memory available at the worker nodes. This should only be done if the data being loaded into the columnar table has a mixture of records matching many different logical partitions. As an example, suppose that a year's worth of data is being loaded into a columnar fact table with weekly partitions, and the cluster has four physical worker nodes with 100GB of system memory in each node. Loading data in 40GB batches will use 10GB memory per physical worker, which is 10% of the overall available memory.
Error Logging
Enabling Load Error Logging
Use the
--el-enabled
flag (or the errorlogging
flag inside a map file) to run the Aster
Loader Tool in a mode in which it tolerates poorly formatted input rows and logs each bad row to a table. This differs from Aster Loader’s normal mode of running:
• Running normally, Aster Loader aborts the load immediately if it encounters a bad input row, and it does not log the malformed input row to a table.
• Running in
--el-enabled
mode, Aster Loader logs each malformed input row (that is, any row it cannot interpret for loading or cannot load due to datatype mismatch or check constraint violation) to an error table and continues to load the remaining rows in the load job. We refer to this as “error logging.”
The
--el-enabled
flag is a master flag that operates with a set of sub-flags (
--el-discarderrors
,
--el-errfile
,
--el-label
,
--el-limit
,
--el-schema
, and
--el-table
) that fine-tune your handling of malformed rows. To use any of the sub-flags, you must first have specified the
--el-enabled
flag. If you’re using a map file, the syntax is different. The master flag is errorlogging
, and the sub-flags are discard-errors
, errfile
, label
, limit
, schema
, and table
.
The
--el-discard-errors
flag discards all malformed rows, the
--el-label
tags failed row data, the
--el-limit
flag sets a maximum number of allowed failed rows for the job, and the
--el-table
flag specifies a custom error logging table. See “Argument Flags” on page 188
for explanations. See “Example with Error Logging” on page 201 for example usage.
To perform error logging, the Aster Loader Tool relies on the error handling features of the
Aster Database COPY command in SQL.
If data being loaded will cause duplicate values of a UNIQUE or PRIMARY KEY constraint on the target table, it is considered an error. This particular error cannot be handled by error logging, so the loading transaction will be aborted if any record causes a unique or primary key constraint violation, even when error logging is enabled.
204 Aster Client Guide
Loading Data
Troubleshoot Loading
What Errors Get Logged
Error logging is more general than just handling malformed rows. Essentially, any error related to an individual row can be recorded by error logging and the load can continue, except for the special case of UNIQUE/PRIMARY KEY violations. If there is such a violation, the loader will abort the load. Also, any error not related to an individual row (such as insufficient privileges) will abort the load operation.
Some examples of what errors can be caught by error logging:
• NOT NULL violations
• Wrong number of columns
• Value format errors (e.g. specifying "asdf " for an integer column)
• No matching child partition
• CHECK constraint violations
• Field length overflow (e.g., specifying "asdf " for a varchar(2) column)
• Text/CSV format errors (e.g. misplaced carriage return, unterminated quoted string, etc.)
Load Error Logging Tables
By default, Aster Database logs error rows to one of its default error logging tables, based on type of the target table: malformed rows for distributed tables go into table nc_errortable_part
table, and malformed rows for replicated tables go into the nc_errortable_repl
table. Alternatively, you can create your own error logging tables. You can perform SELECT, UPDATE, and DELETE operations on the error log tables, and you can inherit from the default error logging tables to create your own error logging tables.
Statistics About Logged Errors
To see the number of rows that loaded or failed to load, query the load error statistics tables, nc_all_errorlogging_stats
and nc_user_errorlogging_stats
. Each loading command generates a row in the tables. For a given transaction, the totalcount
, goodcount
, and malformedcount
columns show the total number of rows you tried to load, the number of rows that successfully loaded, and the number of rows not loaded, respectively.
Troubleshoot Loading
Running Multiple Loaders Concurrently
When running multiple ncluster_loader instances concurrently, eventually Aster Database will throw errors due to physical limitations imposed by system resources. Due to this, you should not run more than ten ncluster_loader instances concurrently.
Load Stalls Upon Cancellation or Failure Encountered During a Load
The Aster Loader Tool can take a lot of time to finish after a failure is detected. Likewise, if a
Control+C is issued, the loader can take several minutes to cancel completely.
Aster Client Guide 205
Loading Data
Troubleshoot Loading
Load Fails on UNIQUE or PRIMARY KEY Violation
A UNIQUE or PRIMARY KEY violation in the data being loaded will always cause the load to abort. So if your load is failing, check your data and the schema of the table you are loading into for these violations.
Invalid Input Syntax Error
If you encounter an error of type:
"Error: Invalid input syntax on partition key for..." then there may be a problem with the character encoding of your input data. Please check the encoding of your input data and the datatypes of your destination columns, fix any mismatches you find, and retry the load.
Single Quote Character Must be Escaped When Using the -q Option
If you encounter an error like:
"ERROR: extra data after last expected column" you may have encountered a problem with command line parsing. If you used a single quote delimiter in your CSV input file, and the input data includes a comma within one of the text strings to be loaded, you will see this error.
To correct this, you will need to escape the single quote character when specifying it using the
-q
option on the command line as
-q"\'"
.
Here is an example:
Assume the file test.txt
, which we are attempting to load into table table1
includes the following input data row:
1, 25, "Some text with a, comma in it."
The following will not load, returning the
"Error: extra data after last expected column"
error: ncluster_loader -l 10.50.129.103 -U db_superuser -w db_superuser -d beehive -c -q"'" table1 test.txt
The following will load: ncluster_loader -l 10.50.129.103 -U db_superuser -w db_superuser -d beehive -c -q"\'" table1 test.txt
Using the -C Option With Uppercase or Special Characters
When using the
-C
option where the column list has any uppercase or special characters, you must put the column list in double quotes. On Windows, this requires escaping the double quotes:
Example on Linux:
./ncluster_loader -h 10.51.150.100 -l 10.51.150.240 -w db_superuser -U db_superuser -c -D , -C '"ID","NAME","$value"' test2 test2.csv
206 Aster Client Guide
Loading Data
Troubleshoot Loading
On Windows, when using the
-C
option where the column list has any uppercase or special characters, you must put the column list in escaped double quotes.
Example on Windows: ncluster_loader.exe -h 10.51.150.100 -w db_superuser -U db_superuser -c
-C '\"ID\",\"NAME\",\"$value\"' test2 test2.csv
Uppercase Characters are Passed as Lowercase if not Escape Quoted
If your data contains a table or schema with a capital letter in its name, you must escape quote the table or schema name. If you do not do this, all lower case letters are assumed. Because of this, the data will load into a schema or table having the same name all in lower case, if it exists.
For example, if you specify the table Test.Try, Test.try, test.Try or test.try, the data will always be loaded into test.try unless you specify escape characters with enclosed quotes for the table name parameter to ncluster_loader as in this example:
# /home/beehive/clients/ncluster_loader -U db_superuser -w db_superuser
-c '\"Test\".\"Try\"' file001
Loading tuples using node 'localhost'.
4 tuples were successfully loaded into table '"Test"."Try"'.
This example loads the data in the expected table (Test.Try).
Issues Using Escape Characters
A few notes on using escape characters may help troubleshoot any issues when using them:
1
The escape character takes effect only if it is inside a quoted string. Consider this example:
2
He said \"Who is that?\" when she entered the room.
The first
\
is treated as a regular character, and not as an escape character, because it is not inside a quoted string, and the first
"
is treated as the beginning of a string. Because there is an opening quote but no corresponding closing quote on the line, the parser will either complain, or will scan subsequent lines until it finds a quote that can be treated as a quote
(rather than as a literal). The parser will treat everything up to that point as one data field with embedded newlines.
To correct this, enclose the entire string in quotes as in this example:
"He said \"Who is that?\" when she entered the room."
If an explicit quote character (
-q
) is specified, and no explicit escape character (
-e
) is specified, then the escape character will be set to be the same as the quote character specified.
Aster Client Guide 207
Loading Data
Troubleshoot Loading
208 Aster Client Guide
CHAPTER 7
Exporting Data
Aster Export Tool
The Aster Export Tool, ncluster_export, is a command-line application for bulk-exporting data from Aster Database to a file or to STDOUT. It provides an alternative to the SQL COPY TO statement. Each invocation of the Aster Export Tool is equivalent to a single invocation of the
COPY TO statement. Each invocation is a single database transaction that exports the contents of the specified table into the specified file or STDOUT.
Related Topic:
Synopsis
To run the Aster Export Tool, you type:
$ ./ncluster_export [arguments] [schemaname.]tablename [ filename ]
You can also pipe the results through a standard UNIX command such as gzip. For example:
$ ./ncluster_export -U mjones -w st4g0l33 -h 10.50.52.100 -d mydb mytable
| gzip -c > mytable.gz
In the synopsis above, the arguments are:
•
schemaname
is the optional name of the source table’s schema. If no schema name is provided, Aster Database will search the schemas listed in your current schema search path.
•
tablename
is the name of the source table. See “Case-Sensitive Handling for Table
if you wish to have Aster Database evaluate table names in a case-
sensitive manner. To export from multiple tables, see “Exporting from Multiple Tables” on page 210 .
•
filename
indicates the name of the file that will receive the exported data. If you don’t supply a file or directory name argument, Aster Export assumes you want to export to
STDOUT. If the filename
contains spaces, make sure you enclose it in double quotes.
•
arguments
are the command-line flags that control how the exporter runs. The flags are explained in
, or you can display the help by typing:
$ ncluster_export -?
Aster Client Guide 209
Exporting Data
Aster Export Tool
Case-Sensitive Handling for Table Names
To force Aster Export to treat your table and schema names in a case-sensitive manner, you must surround the schema and table names in double quote marks when you invoke ncluster_export. To pass a double quote mark to Aster Export, you must prefix it with the escape character, which is a backslash. That means you type a double quote mark like so:
\"
To do this, surround the entire schemaname.tablename string in a single pair of double quotation marks.
For example, assuming you want to export to table "t" in schema "S", then you would invoke
Aster Export in this form: ncluster_export.exe -h 10.51.3.100 -U mjones -w st4g0l33 \"S\".\"t\" mydata.txt
Exporting from Multiple Tables
With ncluster_export you can export from multiple tables in a single invocation by creating a map file of tables to be exported and passing the
--map-file
or
-m
option with the filename.
All tables are exported within a single database transaction. To export from multiple tables, perform the following steps:
1
Prepare your map file. The map file is a text file containing a set of logical text blocks, each surrounded by curly braces. Each block represents a table to be exported. The format is as follows:
{
"exportconfig" :
[
{
"table" : "exporttable1",
"fileprefix" : "path/to/dir/file"
},
{
"table" : "exporttable2",
"fileprefix" : "path/to/dir/prefix",
"maxtuplesperfile" : 100,
"columns" : "a,b"
}
]
}
Each block must contain the following required parameters:
• “table” specifies the name of the table to export.
• “fileprefix” specifies the output file / file prefix to be used.
Each block can contain the following optional parameters:
• “maxtuplesperfile” can be used to specify how many records will be put in each output file. This allows outputting multiple files for a single table, with the specified number of records in each.
• “columns” is a comma-separated list of the columns to be exported.
210 Aster Client Guide
Exporting Data
Aster Export Tool
2
Run ncluster_export, passing the
--map-file
or
-m
option and the name of your map file.
$ ./ncluster_export -U mjones -w st4g0l33 -h 10.50.52.100 -d mydb -m
"path/to/dir/mapfile"
Argument Flags
Aster Export accepts the following flags.
Table 7 - 1: Aster Export Flags
Column Type
-? [ --help ]
Shows the online help.
-B [ --begin-script ] arg
Specifies the qualified path of the file containing SQL commands that should be executed when the transaction starts, i.e. immediately after the begin command is issued to Aster Database. NOTE: Datareturning statements such as SELECT are not allowed.
Each command in the begin script should be on a separate line. Do not include BEGIN or END statements.
-c [ --csv ]
-C [ --columns ] arg
Exports the output files in CSV format (the default is to use Text format).
An optional comma-separated list of columns to be exported from the source table (the default is to export all columns).
-d [ --dbname ] arg
-D [ --delimiter ] arg
-E [ --end-script ] arg
-e [ --escape ] arg
Specifies the name of the database to connect to (the default is 'beehive').
Specifies the delimiter character to use when exporting data (must be a string that represents a valid single character, such as 'd' or '\n'). The default is a tab character ('\t') in text mode, a comma (',') in CSV mode.
Specifies the qualified path of the file containing SQL commands that should be executed when the transaction finishes, i.e. immediately before the end command is issued to Aster Database. NOTE: Datareturning statements such as SELECT are not allowed.
Each command in the end script should be on a separate line. Do not include BEGIN or END statements.
Specifies the escape character for CSV output (must be a string that represents a valid single character, such as 'd' or '\n'). The default is the quote value - double-quote by default. This option is only valid when CSV mode is specified.
Aster Client Guide 211
Exporting Data
Aster Export Tool
Table 7 - 1: Aster Export Flags (continued)
Column
-f [ --force-loader ]
filename
(Not a flag; you pass the file or directory name itself!)
-h [ --hostname ] arg
-l [ --loader ] arg
-m [ --map-file ] arg
-M [ --max-tuples-perfile ] arg
-n [ --null ] arg
--null-backslash-escapes
-p [ --port ] arg
Type
Instructs ncluster_export to use the loader node specified with the '-l/--loader' option even if the IP address provided is not known to Aster Database.
NOTE: ncluster_export will only try that single IP address and return an error status if the connection fails for any reason.
filename
indicates the name of the file that will receive the exported data. If you don’t supply a file or directory name argument, Aster Export assumes you want to export to STDOUT. If the filename contains spaces, make sure you enclose it in double quotes.
Specifies the hostname or IP address of the machine on which Aster Database is running. Default is 'localhost'.
Preferred loader IP address. If a value is provided, ncluster_export will try to export through this IP address before trying any other loader node. NOTE: ncluster_export expects this IP address to be known by
Aster Database and will simply ignore the address provided if this is not the case. To change this behavior one can use the '-f/--force-loader' option.
Specifies the name of the file containing mappings of the tables to be exported. This option allows export of multiple tables within the same transaction. For details about the format used for the map file, see
“Exporting from Multiple Tables” on page 210 .
Specifies how many records are exported to a single file.
If the total number of records exceeds this number multiple output files with the suffix '_N' are produced, where N is an integer.
Specifies a string that represents a NULL value. The default is '\N' in text mode and an empty value with no quotes in CSV mode. See also
“--null-backslashescapes” on page 212 .
Indicates that backslashes in the '-n/--null' flag should be treated as special characters, which will be processed according to the default rules for escaped strings.
You should use this flag whenever the null string contains a backslash, to ensure compatibility with future versions of Aster Database.
Specifies the TCP port on which Aster Database is listening for connections. The default is port 2406.
212 Aster Client Guide
Aster Client Guide
Exporting Data
Aster Export Tool
Table 7 - 1: Aster Export Flags (continued)
Column
-q [ --quote ] arg
schemaname
(Not a flag; you pass the schema name itself!)
-t [ --timeout ] arg
tablename
(Not a flag; you pass the table name itself!)
-U [ --username ] arg
-V [ --version ]
-w [ --password ] arg
-W [ --password-prompt ]
Type
Specifies the quote character for CSV output (must be a string that represents a valid single character, such as 'd' or '\n'). The default is the double-quote. This option is only valid if CSV mode is specified.
schemaname
is the optional name of the source table’s schema. If no schema name is provided, Aster Database will search the schemas listed in your current schema search path.
Specifies the timeout value (in seconds) to use when connecting to Aster Database (default is 30).
tablename
is the name of the source table. See
Sensitive Handling for Table Names” on page 210
if you wish to have Aster Database evaluate table names in a case-sensitive manner. To export from multiple tables,
see “Exporting from Multiple Tables” on page 210
.
Connects to the database with the specified username instead of the default ('beehive'). (You must have permission to do so, of course.)
Prints the version of ncluster_export and exit.
Connects to the database with the specified password
(as opposed to providing a password at the prompt).
Forces ncluster_export to prompt for a password before connection to the database.
213
Exporting Data
Aster Export Tool
214 Aster Client Guide
Appendix 1 Encoding Support
This appendix lists Aster Client encoding support. Note that data being loaded to Aster
Database should use a UTF-8 character set with one of the supported encodings. The encoding support for JDBC and .NET Data Provider are Unicode (UTF-16). This is because the character string data type in JAVA and C# are encoded in Unicode (UTF-16).
Aster ODBC Encoding Support
This table lists the encoding support in Aster Clients, which are based on the Simba 9.0 SDK.
Note that the Aster Clients on Mac OS use the Simba 8.1.0 SDK, and as such they differ from the others in that they do not include support for ENC_MSWIN31_KOREAN (Korean character set).
Table 1 - 1: Encoding Support in Aster ODBC
Simba Encoding Name
ENC_BIG5
ENC_CP437_US
ENC_CP850
ENC_CP1252
ENC_EUC_CNS
ENC_EUC_JP
ENC_EUC_KR
Description
Multibyte character set. Chinese characters for Taiwanese.
IBM 437
IBM 850
Source
http://www.iana.org/assignments/charset-reg/
Big5-HKSCS
Windows Unicode Latin (English,
Spanish, Germanic/Scandinavian) character set.
Simplified Chinese multibyte character set for UNIX. Based on ISO
2022. https://en.wikipedia.org/wiki/Extended_Unix_Code
Japanese multibyte character set for
UNIX. Based on ISO 2022.
Korean multibyte character set for
UNIX. Based on ISO 2022.
IBM NLS RM Vol 2 SE09-8002-01, March 1990
IBM NLS RM Vol 2 SE09-8002-01, March 1990 https://en.wikipedia.org/wiki/Extended_Unix_Code http://tools.ietf.org/html/rfc1557
Aster Client Guide 215
Aster ODBC Encoding Support
Table 1 - 1: Encoding Support in Aster ODBC (continued)
Simba Encoding Name
ENC_GBK
ENC_HP_ROMAN8
ENC_HZ_GB_2312
ENC_IBM_280
ENC_IBM_424
ENC_IBM_775
ENC_IBM_851
ENC_IBM_00858
ENC_IBM_866
ENC_IBM_871
ENC_IBM_918
ENC_IBM_1026
ENC_IBM_1140
ENC_IBM_1141
ENC_IBM_1142
ENC_IBM_1143
ENC_IBM_1144
ENC_IBM_1145
ENC_IBM_1146
ENC_IBM_1147
ENC_IBM_1148
ENC_IBM_1149
Description Source
I
SO-IR: International Register of Escape Sequences
Official character set of People’s
Republic of China.
Chinese IT standardized Unicode character set.
Roman character set for use with HP printers. http://www.iana.org/assignments/charset-reg/GBK
LaserJet IIP Printer User's Manual, HP part no
33471-90901, Hewlett-Packard, June 1989.
Printable ASCII Chinese character set. http://tools.ietf.org/html/rfc1842
, http://tools.ietf.org/html/rfc1843
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
HP PCL 5 Comparison Guide (P/N 5021-0329) pp
B-13, 1996.
IBM NLS RM Vol 2 SE09-8002-01, March 1990. http://www.iana.org/assignments/charset-reg/
IBM00858
IBM NLDG Volume 2 (SE09-8002-03) August 1994.
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
IBM NLS RM Vol 2 SE09-8002-01, March 1990.
IBM NLS RM Vol 2 SE09-8002-01, March 1990. http://www.iana.org/assignments/charset-reg/
IBM01140 http://www.iana.org/assignments/charset-reg/
IBM01141 http://www.iana.org/assignments/charset-reg/
IBM01142 http://www.iana.org/assignments/charset-reg/
IBM01143 http://www.iana.org/assignments/charset-reg/
IBM01144 http://www.iana.org/assignments/charset-reg/
IBM01145 http://www.iana.org/assignments/charset-reg/
IBM01146 http://www.iana.org/assignments/charset-reg/
IBM01147 http://www.iana.org/assignments/charset-reg/
IBM01148 http://www.iana.org/assignments/charset-reg/
IBM01149
IBM Globalization
216 Aster Client Guide
Aster ODBC Encoding Support
Table 1 - 1: Encoding Support in Aster ODBC (continued)
Simba Encoding Name Description Source
ENC_IBM_EBCDIC_ARAB_B
ILINGUAL
IBM Arabic character set
ENC_IBM_EBCDIC_DENMA
RK_NORWAY
IBM Nordic character set.
IBM Globalization
IBM Globalization
ENC_IBM_EBCDIC_EASTER
N_EUROPE
IBM Latin written Slavic and Central
European character set.
ENC_IBM_EBCDIC_FINLAN
D_SWEDEN
IBM Scandinavian character set.
IBM Globalization
IBM Globalization
ENC_IBM_EBCDIC_FRANC
E
ENC_IBM_EBCDIC_GERMA
NY_AUSTRIA
IBM French character set.
IBM Germanic character set.
ENC_IBM_EBCDIC_KATAKA
NA_DB
IBM Katakana character set.
IBM Spanish/Latin character set. ENC_IBM_EBCDIC_SPAIN_
LATIN_AMERICA
ENC_IBM_EBCDIC_UK IBM United Kingdom character set.
IBM Globalization
IBM Globalization
IBM Globalization
IBM Globalization
IBM Globalization
ENC_IBM_EBCDIC_W_EUR
OPE
ENC_ISO_2022_CN
ENC_ISO_2022_CN_EXT
ENC_ISO_2022_JP
ENC_ISO_2022_JP_2
ENC_ISO_2022_KR
IBM Western European character set.
IBM Globalization http://tools.ietf.org/html/rfc1922 http://tools.ietf.org/html/rfc1922 http://tools.ietf.org/html/rfc1468 http://tools.ietf.org/html/rfc1554 http://tools.ietf.org/html/rfc1557
ENC_ISO_646_IRV Similar encoding to ASCII. Some characters from ASCII removed to make space for national variants. http://en.wikipedia.org/wiki/ISO/IEC_646
ENC_ISO_8859_13
Aster Client Guide http://www.iana.org/assignments/charset-reg/ISO-
8859-13
217
Aster ODBC Encoding Support
Table 1 - 1: Encoding Support in Aster ODBC (continued)
Simba Encoding Name
ENC_ISO_8859_14
Description
ENC_ISO_8859_15
ENC_JIS_ENCODING ISO 2022 escape sequences used to shift code sets.
Source
http://www.iana.org/assignments/charset-reg/ISO-
8859-14 http://www.iana.org/assignments/charset-reg/ISO-
8859-15
JIS X 0202-1991. http://tools.ietf.org/html/rfc1489
Ukrainian (Cyrillic) encoding.
ENC_LATIN1
ENC_MS_CP932
ISO 8859 1
Windows Japanese character set.
Extends Shift_JIS
ENC_MSWIN31_ARABIC Arabic character set.
ENC_MSWIN31_BALTIC Baltic character set.
ISO-IR: International Register of Escape Sequences
JIS X0201:1997. JIS X0208:1997.
ENC_MSWIN31_EASTERN_
EUROPE
Latin written Slavic and Central
European character set.
ENC_MSWIN31_HEBREW Hebrew character set.
ENC_MSWIN31_KOREAN Korean character set. Not supported on Mac OS.
ENC_MSWIN31_TURKISH Turkish character set.
ENC_MSWIN31_VIETNAME
SE
Vietnamese character set.
ENC_MVS_OPEN_EDITION EBCDIC encoding used for MVS
OpenEdition OS. http://www.redbooks.ibm.com/redbooks/pdfs/ sg244529.pdf
ENC_PC_ARABIC
ENC_PC_CANADIAN_FREN
CH
ENC_PC_CYRILLIC
ENC_PC_EASTERN_EUROP
E
ENC_PC_GREEK
ENC_PC_ICELANDIC
ENC_PC_NORDIC
ENC_PC_PORTUGUESE
ENC_PC_TURKISH
218 Aster Client Guide
Aster ODBC Encoding Support
Table 1 - 1: Encoding Support in Aster ODBC (continued)
Simba Encoding Name Description Source
JIS X0201:1997. JIS X0208:1997.
ENC_STD_MAC_ROMAN graphic characters.
8-bit encoding for Mac OS. Used for
English and Western (Latin) languages.
ENC_TIS_620 Thai Industrial Standards Institute
ENC_US_ASCII ANSI X3.4-1968. https://en.wikipedia.org/wiki/Mac_OS_Roman
ENC_UTF16_BE
ENC_UTF16_LE
ENC_UTF32_BE
ENC_UTF32_LE
Big Endian Unicode 16-bit encoding. http://tools.ietf.org/html/rfc2781
Little Endian Unicode 16-bit encoding. http://tools.ietf.org/html/rfc2781
Big Endian Unicode 32-bit encoding. http://www.unicode.org/unicode/reports/tr19/
Little Endian Unicode 32-bit encoding. http://www.unicode.org/unicode/reports/tr19/
Aster Client Guide 219
Aster ODBC Encoding Support
220 Aster Client Guide
Symbols
.NET Data Provider for Aster
A
ACT
command-line flag reference 76
launching on Linux, Solaris or AIX 75 launching on Mac 75
list all tables in Aster Database 92
parameter, setting with \set 89
transpose columns and rows in output 94
Active Directory authentication
AFS
ACT commands 94 commands in ACT 94 set port in ACT 94
AIX
configure ODBC Driver Manager 29
ANALYZE
Aster Database
Aster Database configuration settings
Aster Client Guide
Index
Aster Database Export: See ncluster_export.
auto-analyze flag in batch loading 193
auto-partition flag in batch loading 188
autopartitioning
B
backslash
adding to input stream with sed 200
backslash escape sequences 200
begin-script argument during batch load 198
begin-script flag in batch loading 188
best practices
logging bad rows during import 204
bulk loading
arguments 188 flags 188 map file: supported flags 188 parameters 188
C
certificate 53 for ODBC connection 53
character set, default 105 characters, special 105
child table
automatically route data into 202
recommended client settings 105 client settings, recommended 105
221
client-side cursors in JDBC 124
column
column alignment of query output 93
columns argument during batch load 188
command buffer 89 command history 89 send to file 89
command reference
ACT SQL prompt 88 command-line (in SQL) commands for ACT 88
command-line options for ACT 76
commands
configuration
configuration settings
Microsoft .NET, drivers for 135
COPY
cursor
D
data
database
list the databases in a cluster 93
Microsoft .NET, drivers for 135
222
date
date format in bulk loading 189
date-style flag in batch loading 189
dbname flag in batch loading 188
delimiter flag in batch loading 189
delimiters
discard-errors flag in batch loading 189
documentation version and updates 11
Driver Manager
Microsoft .NET, drivers for 135
E
el-enabled flag in batch loading 189
end-script argument during batch load 198
end-script flag in batch loading 189
error file flag in batch loading 190 error limit in bulk loading 190
error logging 204 loading, log of errors for 204
tables to capture load errors, default 205
error logging table, naming 191
errorlogging argument during batch load 198
bulk loading and 189 ncluster_loader and 189 escape flag in batch loading 189
escape sequence
adding to input stream with sed 200
Aster Client Guide
Aster Database export tool 209
source table, case-sensitive name 210
export data
other tools 209 export tool 209
F
field separator, setting in ACT 93
file
file argument during batch load 198
file input to ACT 89 with \i 89
force-loader flag for bulk loading 191
format
G
group
list all groups in Aster Database 93
H
hostname flag for bulk loading 191
I
import
logging bad rows during import 204
index
list all indexes in Aster Database 92
install
Aster Client Guide
J
Java
how to write applications that use JDBC 130
L
label argument during batch load 198
label for data loading errors 190
LIMIT
FETCH_LIMIT as alternative to 85
limit argument during batch load 198
limit rows returned at a time 84
limit total rows returned per query 85
logging bad rows during import 204
loader
Aster Database loader tool 186 destination table 186
destination table, case-sensitive name 187
flags 188 map file: supported flags 188
specifying a dedicated loader node 202
loader flag for bulk loading 191
Aster Database Export tool 209
Aster Database Loader tool 186
handling nulls when loading data 192
223
troubleshooting problems encountered when loading 205
loading errors 204 log errors 204
custom error logging tables 205
loading
M
malformed rows during load 204
map file for ncluster_loader 196
map-file flag for bulk loading 192
memory usage in ACT: limit with FETCH_COUNT paging of query results 84
Microsoft .NET, drivers for 135
MicroStrategy, connections for 133
N
nc_errortable_part 205 nc_errortable_repl 205
source table, case-sensitive name 210
destination table, case-sensitive name 187
loading from many files 196 loading to multiple tables 196
logging bad rows during import 204
map file: supported flags 188 parameters 188
null
handling nulls when loading data 192 null flag for bulk loading 192
O
Driver Manager configuration 29, 31
ODBC driver
installing for use with Perl scripts 108
output
P
parameter
setting queen system parameters 52
partitioning
improve ACT performance with FETCH_LIMIT 85
limit ACT memory use with FETCH_COUNT 84 server-side cursors in ACT 84
Perl scripts, ODBC driver for 108
pivot column and row output 94
port flag for bulk loading 192
preferences
prefix argument during batch load 192
Q
queen
query
query buffer 89 send to file 89 query history 89 send to file 89 query results 89 send to file 89
query tool
quote character flag for bulk loading 192
quoted identifier
224 Aster Client Guide
R
recommended client settings 105
recommended loading settings 201
reporting tools
row
running SQL scripts 89 with \i 89
S
schema
list all schemas in Aster Database 93
schema argument during batch load 198
scripts 89 run SQL script with \i 89
security
sed
for backslash sequences 200 sed, using with loader 200
send command history to file 89 send query history to file 89 send query results to file 89
server-side cursors
settings
enable_backslash_escapes 128 enable_quoted_identifiers 128
Solaris
configure ODBC Driver Manager 31
SQL
parameters that affect SQL command interaction 88
parameters that set ACT client behavior 76
parameters that set SQL behavior 128
redirect query results to file 89 scripts, running with \i 89
SQL commands
Aster Client Guide
SQL tool, installing ACT client 19
with Active Directory authentication 50
SSL for ODBC
queen settings 51 reference to Aster Database SSL settings 51
STDIN
system tables
nc_errortable_part 205 nc_errortable_repl 205
T
table
list all tables in Aster Database 92
table argument during batch load 198 table argument inside errorlogging block 198
configure 41 install 41 installing 41
terminal
troubleshooting
loading, troubleshooting for 205
TRUNCATE
before bulk loading 193 truncate-table flag for bulk loading 193
U
225
user
list all users in Aster Database 93
UTF-8
recommended client settings 105
Microsoft .NET, drivers for 135
V
VACUUM
after bulk loading 193 vacuum-table flag for bulk loading 193
version
W
workaround
certificate, missing self-signed cert 51
X
226 Aster Client Guide
advertisement
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
advertisement