Vertica Analytics Platform 7.2.x Using the Vertica Analytics Platform

Vertica on Microsoft Azure
HPE Vertica Analytic Database
Software Version: 7.2.x
Document Release Date: 8/29/2016
Legal Notices
Warranty
The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty
statements accompanying such products and services. Nothing herein should be construed as constituting an
additional warranty. HPE shall not be liable for technical or editorial errors or omissions contained herein.
The information contained herein is subject to change without notice.
Restricted Rights Legend
Confidential computer software. Valid license from HPE required for possession, use or copying. Consistent with
FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data
for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.
Copyright Notice
© Copyright 2006 - 2016 Hewlett Packard Enterprise Development LP
Trademark Notices
Adobe™ is a trademark of Adobe Systems Incorporated.
Apache® Hadoop® and Hadoop are either registered trademarks or trademarks of the Apache Software
Foundation in the United States and/or other countries.
Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation.
UNIX® is a registered trademark of The Open Group.
This product includes an interface of the 'zlib' general purpose compression library, which is Copyright © 19952002 Jean-loup Gailly and Mark Adler.
HPE Vertica Analytic Database (7.2.x)
Page 2 of 25
Contents
Overview of Vertica on Microsoft Azure
5
Recommended VM Types
6
More Information
6
Deploying Vertica from the Azure Marketplace
7
Installing and Running Vertica on Azure
9
Configuring and Launching a Virtual Machine (VM)
10
Configure Your VM
10
Add Storage
10
Configure Network Security Group
10
Configure Credentials
10
Assign a Public IP
10
Create Additional VMs
11
Network Security Group Configurations
12
Inbound Settings
12
Outbound Settings
12
More Information
13
Connecting to a Virtual Machine (VM)
14
Connect to Your VM
14
Connect to an Instance from Windows Using PuTTY
14
Connect to Other VMs
14
Preparing Virtual Machines (VMs)
15
Add Vertica License and Private Key
15
Install Dependencies
15
Configuring Storage
16
Attach Storage Volumes to Virtual Machines (VMs)
16
Combine Volumes for Storage
16
Create a Swap File
17
HPE Vertica Analytic Database (7.2.x)
Page 3 of 25
Vertica on Microsoft Azure
More Information
17
Downloading Vertica
18
Forming a Cluster
19
Combine Virtual Machines (VMs)
19
Use the install_vertica or update_vertica Scripts
20
After Your Cluster Is Up and Running
21
Safe Shutdown
22
Using Management Console (MC) on Azure
23
Limitations
23
Version Compatibility
23
Learning More About MC
23
MC User Accounts
23
Send Documentation Feedback
HPE Vertica Analytic Database (7.2.x)
25
Page 4 of 25
Vertica on Microsoft Azure
Overview of Vertica on Microsoft Azure
Overview of Vertica on Microsoft Azure
Vertica clusters on Microsoft Azure operate on Virtual Machines (VMs) within a Virtual
Network. These instructions apply to VMs built with Vertica Version 7.2.x.
HPE Vertica Analytic Database (7.2.x)
Page 5 of 25
Vertica on Microsoft Azure
Recommended VM Types
Recommended VM Types
Vertica supports a range of Microsoft Azure virtual machine (VM) types, each optimized
for different purposes. Choose the VM type that best matches your performance and
price needs as a user.
For the best performance in most common scenarios, Hewlett Packard Enterprise
recommends the following VMs:
Optimization
Type
General Purpose
DS13_V2
DS14_V2
DS15_V2
High Memory and I/O Throughput GS3
GS4
GS5
Note: Data stored on the temporary drives of a VM exists only while that VM is
powered on. After powering off a VM, data on temporary drives is lost.
More Information
For more information about Azure VMs and their limitations, see the Azure
documentation.
HPE Vertica Analytic Database (7.2.x)
Page 6 of 25
Vertica on Microsoft Azure
Deploying Vertica from the Azure Marketplace
Deploying Vertica from the Azure
Marketplace
You can deploy standard Vertica clusters from the Azure Marketplace. Azure
Marketplace deployment allows you to:
l
Create a 1–5 node Vertica cluster.
l
Specify a VM type.
l
Specify storage size.
If you deploy a standard Vertica cluster from the Azure Marketplace, a template created
by Hewlett Packard Enterprise automatically configures and creates your chosen
cluster, based on your specifications.
To obtain the one-click VM, search for Vertica in the Azure Marketplace.
HPE Vertica Analytic Database (7.2.x)
Page 7 of 25
Vertica on Microsoft Azure
Deploying Vertica from the Azure Marketplace
HPE Vertica Analytic Database (7.2.x)
Page 8 of 25
Vertica on Microsoft Azure
Installing and Running Vertica on Azure
Installing and Running Vertica on Azure
Start creating your cluster by creating your first VM. During the VM creation process, you
can create and configure the various other resources required for your cluster, which will
then be available for additional VMs you create. Follow these procedures to install and
run Vertica on Azure:
l
Configuring and Launching a VM
l
Network Security Group Configuration
l
Connecting to a VM
l
Preparing VMs
l
Configuring Storage
l
Downloading Vertica
l
Forming a Cluster
l
After Your Cluster Is Up and Running
l
Safe Shutdown
l
Using Management Console (MC) on Azure
HPE Vertica Analytic Database (7.2.x)
Page 9 of 25
Vertica on Microsoft Azure
Configuring and Launching a Virtual Machine (VM)
Configuring and Launching a Virtual
Machine (VM)
An Azure VM is similar to a traditional host. Just as with an on-premises cluster, you
must prepare and configure your cluster and network at the hardware level before you
can install Vertica.
Configure Your VM
1. From the Azure marketplace, select an operating system that Vertica supports.
2. Select a VM type.
3. Choose a deployment model. Hewlett Packard Enterprise recommends the
resource manager deployment model.
Add Storage
Create an Azure storage account, which will later contain your cluster storage volumes.
For the best performance, Hewlett Packard Enterprise recommends Premium Storage.
For more information about what a Storage Account is, and how to create one, refer to
the Azure Documentation.
Configure Network Security Group
Vertica has specific network security group requirements.
Create and name your own network security group with the basic security group settings
outlined in the Network Security Group Configuration section. You may make additional
modifications, based on your specific needs.
Configure Credentials
Create a password or assign a key pair you intend to use with Vertica.
For information about what a key pair is, and how to create one, refer to the
Azure documentation.
Assign a Public IP
Create a public IP address.
A public IP is an IP address that you can use to connect to your cluster externally.
Hewlett Packard Enterprise recommends you assign a single static public IP to a node
in your cluster. You can then connect to other nodes in your cluster from your primary
node using their internal IP addresses dictated by your Virtual Network settings.
HPE Vertica Analytic Database (7.2.x)
Page 10 of 25
Vertica on Microsoft Azure
Configuring and Launching a Virtual Machine (VM)
For information about public IPs and how to create them, refer to the
Azure documentation.
Create Additional VMs
Create additional VMs using the resources you configured in the steps above.
HPE Vertica Analytic Database (7.2.x)
Page 11 of 25
Vertica on Microsoft Azure
Network Security Group Configurations
Network Security Group Configurations
Inbound Settings
Name
Protocol
Source Destination
Port
Port Range
Range
Source Destination
SSH
TCP
*
22
Any
Any
HTTP
TCP
*
80
Any
Any
HTTPS
TCP
*
443
Any
Any
DNS (UDP)
UDP
*
53
Any
Any
Spread
UDP
*
4803–4805
Any
Any
Spread
TCP
*
4803–4805
Any
Any
VSQL/SQL
TCP
*
5433
Any
Any
Inter-node
Communication
TCP
*
5434
Any
Any
TCP
*
5444
Any
Any
TCP
*
5450
Any
Any
TCP
*
8080
Any
Any
TCP
*
48073
Any
Any
TCP
*
50000
Any
Any
MC
rsync
Outbound Settings
Name
Protocol
Source
Destination Source Destination
Port Range Port Range
All TCP
TCP
0–65535
*
Any
Any
All ICMP
ICMP
0–65535
*
Any
Any
All UDP
UDP
0–65535
*
Any
Any
HPE Vertica Analytic Database (7.2.x)
Page 12 of 25
Vertica on Microsoft Azure
Network Security Group Configurations
More Information
For information about what a network security group is, and how to create one, refer to
the Azure documentation.
HPE Vertica Analytic Database (7.2.x)
Page 13 of 25
Vertica on Microsoft Azure
Connecting to a Virtual Machine (VM)
Connecting to a Virtual Machine (VM)
Before you can connect to any of the VMs you created, you must first make your network
externally accessible. To do so, you must attach the public IP address you created
during network configuration to one of your VMs.
Connect to Your VM
To connect to your VM, complete the following tasks: 1. Attach your public IP address to one of your VMs.
2. Connect to your VM using SSH with the public IP address you created in the
configuration steps.
3. Authenticate using the credentials and authentication method you specified during
the VM creation process.
Connect to an Instance from Windows Using PuTTY
You can connect to the instance from the Windows operating system with PuTTY or
WinSCP over the public IP. To do so, you must use your key.
Connect to Other VMs
Connect to other virtual machines in your virtual network by first using SSH to connect to
your publicly connected VM. Then, use SSH again from that VM to connect through the
private IP addresses of your other VMs.
If you are using private key authentication, you may need to move your key file to the
root directory of your publicly connected VM. Then, use PuTTY or WinSCP to connect to
other VMs in your Virtual Network.
HPE Vertica Analytic Database (7.2.x)
Page 14 of 25
Vertica on Microsoft Azure
Preparing Virtual Machines (VMs)
Preparing Virtual Machines (VMs)
After you create your VMs, you need to prepare them for cluster formation.
Add Vertica License and Private Key
Prepare your instances by adding your private key, if you are using one, and your
Vertica license.
1. As the dbadmin user, copy your private key file (from where you saved it locally)
onto your primary instance.
Depending upon the procedure you use to copy the file, the permissions on the file
may change. If permissions change, the install_vertica script fails with a
message similar to the following:
Failed Login Validation 10.0.2.158, cannot resolve or connect to host as root.
If you receive a failure message, enter the following command to correct
permissions on your private key file:
chmod 600 /<name-of-key>.pem
2. Copy your Vertica license over to your primary VM, placing it in your home directory
or other known location.
Install Dependencies
In addition to the Vertica standard dependencies, you must install the following
packages specific to Vertica on Azure:
l
microsoft-hyper-v
l
pstack
l
mcelog
l
sysstat
HPE Vertica Analytic Database (7.2.x)
Page 15 of 25
Vertica on Microsoft Azure
Configuring Storage
Configuring Storage
Use a dedicated Azure storage account for node storage.
Caution: Hewlett Packard Enterprise recommends that you do not store your
information on the root volume, especially your data and catalog directories. Storing
information on the root volume may result in data loss.
When configuring your storage, make sure to use a supported file system.
Attach Storage Volumes to Virtual Machines (VMs)
Using your previously created storage account, attach storage volumes to your VMs that
are appropriate to your needs.
For best performance, you can combine multiple storage volumes into RAID-0. Hewlett
Packard Enterprise recommends 6 storage volumes per VM for most RAID-0
implementations.
Combine Volumes for Storage
If your are using RAID, follow these steps to create a RAID-0 drive on your VMs. The
following examples show how you can create a RAID-0 volume named "md10" from 6
individual volumes, named "sdc", "sdd", "sde", "sdf", "sdg", and "sdh":
1. Form a RAID-0 volume using the mdadm utility.
mdadm --create /dev/md10 --level 0 --raid-devices=6 /dev/sdc /dev/sdd /dev/sde /dev/sdf
/dev/sdg /dev/sdh
2. Format the file system to be one that Vertica supports.
mkfs.ext4 /dev/md10
3. Find the UUID on the newly formed RAID volume.
ls -l /dev/disk/by-uuid
4. Copy the UUID, and place it in the FSTAB file.
sudo vi /etc/fstab
UUID=<uuid here> /dev/md10
ext4
defaults
,
errors=remount-ro 0
1
5. Create a directory for your Vertica data and catalog folders.
mkdir vertica
mkdir vertica/data
HPE Vertica Analytic Database (7.2.x)
Page 16 of 25
Vertica on Microsoft Azure
Configuring Storage
6. Mount the RAID volume to your data and catalog directory.
mount /dev/md10 /home/dbadmin/vertica/data
Create a Swap File
In addition to storage volumes to store your data, Vertica requires a swap volume or
swap file to operate.
Create a swap file or swap volume of at least 2 GB. The following steps show how you
can create a swap file within Vertica on Azure:
1. Install devnull and swapfile:
install -o root -g root -m 0600 /dev/null /swapfile
2. Create the swap file:
dd if=/dev/zero of=/swapfile bs=1024 count=2048k
3. Prepare the swap file using mkswap:
mkswap /swapfile
4. Use swapon to instruct Linux to swap on the swapfile:
swapon /swapfile
5. Persist the swapfile in FSTAB:
echo "/swapfile
swap
swap
auto
0
0" >> /etc/fstab
Repeat the volume attachment, combination, and swap file creation procedures on each
VM in your cluster.
More Information
l
l
For more information about Azure storage accounts, refer to the Azure
documentation.
For more information about Vertica storage configuration, refer to the Prepare Disk
Storage Locations section of the Vertica documentation.
HPE Vertica Analytic Database (7.2.x)
Page 17 of 25
Vertica on Microsoft Azure
Downloading Vertica
Downloading Vertica
Prepare to install Vertica by downloading the vertica server file from my.vertica.com.
Choose the Vertica server file appropriate for your operating system and license type.
HPE Vertica Analytic Database (7.2.x)
Page 18 of 25
Vertica on Microsoft Azure
Forming a Cluster
Forming a Cluster
Use the install_vertica script to combine two or more individual VMs and create a
cluster.
Check the Virutal Network page for a list of current VMs and their associated private IP
addresses. You need these IP addresses when you run the install_vertica script.
Combine Virtual Machines (VMs)
The following example shows how you can combine VMs using the install_vertica
script.
1. While connected to your primary instance, enter the following command to combine
your instances into a cluster.
a. Substitute the IP addresses for your VMs, and include your root key file name, if
applicable.
b. Include the --point-to-point parameter to configure spread to use direct
point-to-point communication between all Vertica nodes, as required for clusters
on Azure when installing or updating Vertica. If you do not use this parameter,
you receive an error telling you that you must use point-to-point communication
on Azure.
sudo /opt/vertica/sbin/install_vertica --hosts 10.2.0.164,10.2.0.165,10.2.0.166 --dbauser-password-disabled --point-to-point --data-dir /home/dbadmin/vertica/data --sshidentity ~/<name-of-private-key>.pem --license <license.file>
Note: If you are using Community Edition, which limits you to three
instances, you can simply specify -L CE with no license file.
After you combine your instances, Hewlett Packard Enterprise recommends that
you reduce security risks by keeping your key file in a secure place, separate
from your cluster and deleting your on-cluster key with the shred command.
shred examplekey.pem
Important: You will need your key file to perform future Vertica updates.
2. Reboot your cluster to complete cluster formation.
After you form a cluster, you can create a database.
HPE Vertica Analytic Database (7.2.x)
Page 19 of 25
Vertica on Microsoft Azure
Forming a Cluster
Use the install_vertica or update_vertica Scripts
By default, the installer assumes that you have mounted your storage to
/vertica/data. To specify another location, use the --data-dir argument.
Caution: Hewlett Packard Enterprise recommends that you do not store your data
on the root drive.
For complete information on the install_vertica script and its parameters, see the
Installation Guide, specifically the section, About the install_vertica Script.
HPE Vertica Analytic Database (7.2.x)
Page 20 of 25
Vertica on Microsoft Azure
After Your Cluster Is Up and Running
After Your Cluster Is Up and Running
After your cluster is configured and running, you can:
1. Create a database. When you installed Vertica, a database administrator user was
created, dbadmin. You can use this pre-created dbadmin user to create and start a
database. Refer to the Vertica Installation Guide for information on the dbadmin
administrator role.
2. Configure a database. Refer to the Vertica Administrator’s Guide for information on
configuring a database.
HPE Vertica Analytic Database (7.2.x)
Page 21 of 25
Vertica on Microsoft Azure
Safe Shutdown
Safe Shutdown
Shut the database down first to make sure that Vertica is not in the process of writing to
disk when you shutdown.
To safely stop or reboot your cluster:
1. Stop the database.
2. Stop or reboot one or more VMs.
Caution: If you stop or reboot a VM (or the cluster) without shutting the database
down first, disk or database corruption could result.
Refer to the Vertica Administrator’s Guide for information on stopping a database.
HPE Vertica Analytic Database (7.2.x)
Page 22 of 25
Vertica on Microsoft Azure
Using Management Console (MC) on Azure
Using Management Console (MC) on Azure
Management Console (MC) is a database management tool that provides a way for you
to view and manage aspects of your Vertica cluster. If you are running Vertica Release
6.1.2 or later, you can install and run MC.
Limitations
This release of MC on Azure includes the following limitations:
l
l
l
You cannot create or import a cluster on Azure using the MC.
You cannot monitor an Azure cluster using MC on a node that is outside of your
cluster.
You must install MC on a VM within the Azure cluster environment.
Version Compatibility
Each version of Vertica Management Console (MC) is compatible only with the
matching version of the Vertica server. Version numbers must match to three digits. For
example, Vertica 7.1.2 server is supported by Vertica 7.1.2 MC only. This general MC
requirement applies to all cases and is not specific to MC on Azure.
Learning More About MC
Use the following reading path for learning more about MC:
l
l
l
l
For an overview of MC, where you can get it, and what you can do with it, refer to the
Concepts Guide, specifically, Management Console.
For information on installing and configuring MC, refer to the Installation Guide,
specifically, Installing and Configuring Management Console (MC).
For information on the differences between what you can do with MC in comparison
with what you can do with admintools, refer to the Administration Guide, specifically
the section, Administration Tools and Management Console.
For information on creating a database using MC, refer to the Getting Started Guide,
specifically the section, Create the Example Database Using Management Console.
MC User Accounts
When you first configure MC, during the configuration process you create an MC
superuser (a Linux account). Issuing a Factory Reset on the MC does not create a new
MC superuser, nor does it delete the existing MC superuser. When initializing after a
Factory Reset, you must log on using the original MC superuser account.
HPE Vertica Analytic Database (7.2.x)
Page 23 of 25
Vertica on Microsoft Azure
Using Management Console (MC) on Azure
For information on setting MC to its original state (Factory Reset), see the Administration
Guide, specifically the section, Resetting MC to Pre-Configured state.
After you configure MC, you can add users that are specific to MC. Users created
through the MC interface are MC specific. When you subsequently change a password
through the MC, you only change the password for the specific MC user. Passwords
external to MC (for example, system Linux user and Vertica database passwords)
remain unchanged.
For information on MC users, refer to the Administration Guide, specifically the sections,
Creating an MC User and MC configuration privileges.
HPE Vertica Analytic Database (7.2.x)
Page 24 of 25
Send Documentation Feedback
If you have comments about this document, you can contact the documentation team by
email. If an email client is configured on this system, click the link above and an email
window opens with the following information in the subject line:
Feedback on Vertica on Microsoft Azure (Vertica Analytic Database 7.2.x)
Just add your feedback to the email and click send.
If no email client is available, copy the information above to a new message in a web
mail client, and send your feedback to vertica-docfeedback@hpe.com.
We appreciate your feedback!
HPE Vertica Analytic Database (7.2.x)
Page 25 of 25