EPIC User/Admin Guide

EPIC User/Admin Guide
EPIC
User & Administrator Guide
version 1.5.0
EPIC User & Administrator Guide
Notice
This book or parts thereof may not be reproduced in any form
without the written permission of the publishers.
BlueData Software, Inc. believes that the information in this publication is accurate as of its publication date. However, the information is subject to change without notice. THE INFORMATION
IN THIS PUBLICATION IS PROVIDED “AS IS.” BLUEDATA
SOFTWARE, INC. MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS
IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
EPIC, EPIC Lite, and BlueData are trademarks of BlueData Software, Inc. All other trademarks are the property of their respective owners.
Use, copying, or distribution of any BlueData software described
in this publication requires an applicable software license.
Contact Information
BlueData Software, Inc.
148 Castro Street, Suite A-1
Mountain View, CA 94041
Email: info@bluedata.com
Website: www.bluedata.com
For the most up-to-date regulatory document for your product
line, please refer to your specific agreements or contact BlueData Technical Support at support@bluedata.com.
The information in this document is subject to change. This manual is believed to be complete and accurate at the time of publication and no responsibility is assumed for any errors that may
appear. In no event shall BlueData Software, Inc. be liable for
incidental or consequential damages in connection with or arising from the use of this manual and its accompanying related
materials.
Copyrights and Trademarks
Published June, 2015. Printed in the United States of America.
Copyright 2015 by BlueData Software, Inc. All rights reserved.
ii
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
Table of Contents
1 - Preface ................................................................................. 1
1.1 - About This Manual ............................................................. 3
1.1.1 - Formatting Conventions ......................................... 3
1.1.2 - Organization ............................................................. 4
1.2 - Additional Information ..................................................... 5
1.2.1 - Related Documentation ......................................... 5
1.2.2 - Contact Information ............................................... 5
1.2.3 - Support ...................................................................... 5
1.2.4 - End User License Agreement ............................. 5
2 - Launching and Logging In .............................................. 7
3 - Using a Tenant ................................................................. 11
3.1 - The Tenant Member Interface ......................................13
3.1.1 - Toolbar (1-5) ..............................................................14
3.1.2 - Main Menu (6) .........................................................15
3.1.3 - Menu Expand/Collapse (7) ...................................15
3.1.4 - Work Area (8) ..........................................................15
3.2 - Viewing the Member Dashboard ................................ 17
3.3 - Jobs ....................................................................................18
3.3.1 - Creating a New Job .............................................20
3.3.2 - Editing a Job .........................................................24
3.3.3 - Viewing the Job Setup Log ...............................26
3.3.4 - Viewing the Job Output Log .............................26
3.3.5 - Viewing the Job History Log ............................ 27
3.3.6 - Viewing Job Output ............................................ 27
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3.4 - Persistent Clusters ........................................................ 28
3.4.1 - Viewing Cluster Details ...................................... 30
3.4.1.1 - Node(s) Info Tab .......................................... 30
3.4.1.2 - Job(s) Info Tab ............................................ 30
3.4.1.3 - Charts Tab ..................................................... 31
3.4.1.4 - Service(s) Status Tab ................................. 33
3.4.2 - Viewing the Cluster Setup Log ........................ 34
3.4.3 - Editing a Persistent Cluster ............................... 35
3.4.4 - Creating a New Persistent Cluster ...................37
3.5 - Viewing DataTaps .......................................................... 41
3.6 - Viewing Virtual Nodes ................................................. 42
3.7 - Uploading and Downloading Files ........................... 43
3.8 - Status Messages ........................................................... 45
3.8.1 - Virtual Node Statuses ......................................... 45
3.8.2 - Cluster Statuses .................................................. 45
3.8.3 - Job Statuses ......................................................... 45
3.9 - Changing Your Password ............................................47
4 - Tenant Administration ...................................................49
4.1 - The Tenant Administrator Interface ............................ 51
4.1.1 - Toolbar (1-5) ............................................................ 52
4.1.2 - Main Menu (6) ....................................................... 53
4.1.3 - Menu Expand/Collapse (8) ................................. 53
4.1.4 - Work Area (8) ......................................................... 53
4.2 - Viewing the Tenant Admin Dashboard ................... 55
iii
EPIC Installation Guide
4.3 - DataTaps ..........................................................................56
4.3.1 - Editing a DataTap ................................................. 57
4.3.2 - Creating a New DataTap ...................................59
4.4 - Users .................................................................................62
4.4.1 - Assigning Users to a Tenant ..............................63
5 - Site Administration .........................................................65
5.1 - The Site Administrator Interface ................................. 67
5.1.1 - Toolbar (1-5) .............................................................68
5.1.2 - Main Menu (6) ........................................................69
5.1.3 - Menu Expand/Collapse (7) .................................69
5.1.4 - Work Area (8) .........................................................69
5.2 - Viewing the Site Admin Dashboard ........................... 71
5.2.1 - Current Load Tab .................................................. 71
5.2.2 - Resource Usage Tab .......................................... 72
5.2.3 - Service Status Tab .............................................. 73
5.2.4 - Alerts Tab .............................................................. 76
5.3 - Tenants ............................................................................. 77
5.3.1 - Editing an Existing Tenant .................................. 78
5.3.2 - Creating a New Tenant ...................................... 79
5.3.3 - Viewing Users Assigned to a Tenant ..............81
5.3.4 - Assigning Users to a Tenant .............................82
5.3.5 - Deleting a Tenant ................................................83
5.4 - Managing Users and Sessions ...................................84
5.4.1 - Viewing User Details ...........................................85
5.4.2 - Creating a New User .......................................... 87
5.5 - Viewing Physical Nodes ..............................................88
5.5.1 - Node(s) Info Tab ...................................................88
5.5.2 - Charts Tab .............................................................89
5.5.3 - DataTap Charts Tab .............................................91
iv
5.6 - Managing the EPIC Installation .................................. 92
5.6.1 - Installation Tab ..................................................... 92
5.6.1.1 - Worker(s) Status Tab .................................. 93
5.6.1.2 - Installed Workers Tab ............................... 93
5.6.2 - Installing Worker Nodes .................................... 94
5.6.2.1 - Step 1: Selecting the Node(s) .................. 94
5.6.2.2 - Step 2: Installing EPIC on the Node(s) 96
5.6.2.3 - Step 3: Selecting Hard Drives ................97
5.6.2.4 - Step 4: Entering Lockdown Mode ........ 98
5.6.2.5 - Step 5: Adding the Node(s) to the EPIC Platform ............................................................................... 98
5.6.3 - Available Upgrades Tab .................................... 99
5.6.4 - Upgrade History Tab .........................................100
5.6.5 - Upgrading the EPIC Installation .....................100
5.7 - EPIC Catalog ..................................................................103
5.7.1 - Images Tab ...........................................................103
5.7.2 - Add-On Images Tab ..........................................105
5.7.3 - Flavor Tab ............................................................ 107
5.7.4 - Creating a New Flavor ......................................108
5.8 - EPIC Settings ..................................................................110
5.8.1 - System Storage Tab ............................................110
5.8.1.1 - Kerberos Security ........................................111
5.8.2 - User Authentication Tab ................................... 114
5.8.3 - HA Settings Tab ..................................................116
5.8.3.1 - Troubleshooting Node Failures ............. 117
5.8.4 - Other Settings Tab .............................................118
5.9 - Support/Troubleshooting ...........................................120
5.9.1 - Generating an SOS Report ................................121
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5.10 - Lockdown Mode ......................................................... 122
5.10.1 - Entering Lockdown Mode ............................... 122
5.10.2 - Exiting Lockdown Mode ................................. 122
A - Troubleshooting ............................................................ A-1
A.1 - Hardware Errors ............................................................ A-3
A.1.1 - General Problems ............................................... A-3
A.2 - Runtime .......................................................................... A-4
A.2.1 - General ................................................................. A-4
A.2.2 - DataTaps ............................................................. A-4
A.2.3 - DataTap URIs ..................................................... A-4
A.2.4 - Common Job/Cluster Errors ........................... A-5
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
v
EPIC Installation Guide
This page intentionally left blank.
vi
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
1 - Preface
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
1
EPIC User & Administrator Guide
EPIC by BlueData Software, Inc. enables users to provide big
data as a service within their own secure private environments.
Every day, we create 2.5 quintillion bytes of data—a volume so
large that 90% of the data in the world today has been created in
the last two years alone. This data contains key business intelligence and insights that enterprises must fully leverage in order
to expand and maintain their competitive advantages.
Hadoop is a leading big data technology that makes it possible
to extract, process, and present large amounts of data in meaningful ways. The key challenge is that Hadoop relies on physical
clusters and a legacy model of data collocation and computing
methods. Deployments are complex, difficult to manage, inflexible, and inefficient. On average, resource utilization is typically
less than 30%, IT costs are high, and the time-to-value for the
enterprise is often measured in months… this in an environment
with dynamic data and resource needs that can change on a
moments’ notice. Enterprises typically address such problems by
creating multiple clusters, which compounds costs and IT management overhead.
Virtualization and private-cloud platforms solved similar challenges for enterprise data centers by providing flexibility, ease of
management, and reduced costs. EPIC expands these benefits
to big data by allowing enterprises to create, modify, and remove
virtual clusters on demand without sacrificing performance. With
EPIC, enterprises can simultaneously run hundreds of workloads
with automated policy-based scheduling and self-provisioning.
Distributed applications are efficient and elastic, thanks to EPIC’s
proprietary application-sensitive caching, data-path and network
optimization, and sophisticated policy-based automation and
2
management. IT administrators use a single interface to monitor
clusters, jobs, and infrastructure status. EPIC also automates routine tasks such as provisioning, updates and monitoring.
EPIC dramatically reduces deployment complexity while improving business agility by providing an elastic self-service infrastructure that reduces the time-to-value from months to days while
reducing overall costs by 50%-70% compared to traditional, nonvirtualized Hadoop deployments. Users create virtual clusters on
demand and execute jobs without ever having to worry about
the underlying infrastructure.
Please see the About EPIC Guide for detailed information about
EPIC, including:
•
Key features and benefits
•
Hadoop and application support
•
Definitions
•
Architecture
•
Storage
•
High Availability
•
Users
•
System requirements
Note: Most of the information in this Guide pertains
to both EPIC (full) and EPIC Lite (evaluation). The key
differences between the two versions are outlined in
the About EPIC Guide.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
1 - Preface
1.1 - About This Manual
This section describes the formatting conventions and information contained in this manual.
1.1.1 - Formatting Conventions
Specific commands appear in standard Courier font.
Sequences of commands appear in the order in which you
should execute them and include horizontal or vertical spaces
between commands. The following additional formatting also
applies when discussing command-line commands:
This manual uses several formatting conventions to present
information of special importance.
Plain-text responses from the system appear in bold Courier
font.
Lists of items, points to consider, or procedures that do not need
to be performed in a specific order appear in bullet format:
This manual also contains important safety information and
instructions in specially formatted callouts with accompanying
graphic symbols. These callouts and their symbols appear as follows throughout the manual:
•
Item 1
•
Item 2
Procedures that must be followed in a specific order appear in
numbered steps:
1. Perform this step first.
CAUTION: CAUTIONS ALERT YOU TO THE POSSIBILITY OF A SERIOUS ERROR, DATA LOSS, OR
OTHER ADVERSE CONDITION.
Note: Notes provide helpful information.
2. Perform this step second.
Specific keyboard keys are depicted in square brackets and are
capitalized, for example: [ESC]. If more than one key should be
pressed simultaneously, the notation will appear as [KEY1]+[KEY
2], for example [ALT]+[F4].
The Note and Caution icons are blue in the main chapter, and
red in the appendices.
Interface elements such as document titles, fields, windows,
tabs, buttons, commands, options, and icons appear in bold text.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3
EPIC User & Administrator Guide
1.1.2 - Organization
This manual contains the following chapters:
•
1 - Preface: Describes how this manual is formatted and organized.
•
2 - Launching and Logging In: Describes how to log into
EPIC and access the correct tenant.
•
3 - Using a Tenant: Describes the EPIC Tenant Member interface and functionality in detail.
•
4 - Tenant Administration: Describes the EPIC Tenant
Administrator interface and functionality in detail.
•
5 - Site Administration: Describes the EPIC Site Administrator interface and functionality in detail.
This manual also contains the following appendix:
•
A - Troubleshooting: Helps you resolve issues that may arise
while using EPIC. Please see the Installation Guide for information on troubleshooting during installation.
4
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
1 - Preface
1.2 - Additional Information
This section lists related documentation and provides information on contacting BlueData, Inc.
1.2.1 - Related Documentation
Please refer to the following documents for additional information:
•
About EPIC Guide: This guide explains the EPIC architecture,
features, and benefits. It also contains the End User License
Agreement.
•
EPIC Lite Installation Guide: This guide helps you install
EPIC Lite, the free demonstration version of EPIC, on a single
physical node.
•
EPIC Installation Guide: This guide contains instructions for
installing the full version of EPIC on your network.
•
Quick Start Guide: This guide helps you get EPIC up and running with minimal effort.
•
Running Applications in EPIC: This guide provides a brief
overview of how to input data, run jobs, and access job output within EPIC.
•
Deployment Guide: Certain platforms have additional
requirements and/or procedures for installing/running EPIC.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
•
Edge Node Authoring Guide: Describes how to configure
third-party tools to run on EPIC Edge nodes.
1.2.2 - Contact Information
You may contact BlueData Software, Inc. at the following
address:
BlueData Software, Inc.
148 Castro Street, Suite A-1
Mountain View, CA 94041
Email: support@bluedata.com
Website: www.bluedata.com
1.2.3 - Support
Please see the Installation Guide for information on obtaining
technical support from BlueData Software, Inc.
1.2.4 - End User License Agreement
Your use of EPIC is subject to the terms and conditions described
in the End User License Agreement (EULA).
5
EPIC User & Administrator Guide
This page intentionally left blank.
6
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
2 - Launching and Logging In
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
7
EPIC User & Administrator Guide
To launch and log into EPIC:
1. Open a Web browser and navigate to http://<A.B.C.D>,
where <A.B.C.D> is either:
-
The IP address of the EPIC Controller node.
-
The cluster IP address, which will automatically route you
to either the EPIC Controller node (under normal circumstances) or the Shadow Controller node (if EPIC platform
High Availability is enabled and the Controller node has
failed). Please see “HA Settings Tab” on page 116 and
the About EPIC Guide for information on enabling High
Availability for the EPIC platform.
Alternatively, if you have a DNS service on the network that
maps the Controller IP address to the Controller hostname,
then you can go to http://hostname.
2. Enter your username and password in the appropriate fields
and then either click the [Login] button or press [ENTER].
EPIC will log you into the tenant you last accessed before logging out of your previous session and display the Dashboard
screen appropriate to the role you have in that tenant, as follows:
•
If you are a Member of the current tenant, you will see the
Tenant Member Dashboard screen and have Member privileges. See “Using a Tenant” on page 11.
•
If you are an Administrator of the current tenant, you will see
the Tenant Administrator Dashboard screen and have Tenant
Administrator privileges. See “Tenant Administration” on
page 49.
•
If you are a Site Administrator, you will see the Site Administrator Dashboard screen and have Site Administrator privileges. See “Site Administration” on page 65.
The EPIC Login screen appears.
You may switch to any tenant that
you have access to by clicking the
User button at the right of the
Toolbar at the top of the screen
and then selecting the desired
tenant in the pull-down menu.
•
Tenants where you have Member privileges have a yellow
icon next to them.
Figure 2.1: Login screen
8
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
2 - Launching and Logging In
•
Tenants where you have Tenant Administrator privileges
have a green icon next to them.
•
If you can access the Site Admin tenant, that will appear in
the list of tenants with a blue icon and you will have Site
Administrator privileges.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
9
EPIC User & Administrator Guide
This page intentionally left blank.
10
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
11
EPIC User & Administrator Guide
This chapter describes the EPIC Tenant Member interface. A
Member can perform the following functions:
•
Login to EPIC: See “Launching and Logging In” on page 7.
•
View the Tenant Member Dashboard screen: See “The
Tenant Member Interface” on page 13.
•
View, add, edit, and remove Hadoop jobs. See “Jobs” on
page 18.
•
Add, edit, and remove persistent clusters: See “Persistent
Clusters” on page 28.
•
View basic DataTap information: See “Viewing DataTaps”
on page 41.
•
View virtual nodes: See “Viewing DataTaps” on page 41.
12
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.1 - The Tenant Member Interface
The Tenant Member interface contains the following elements:
Figure 3.1: The Tenant Member interface
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
13
EPIC User & Administrator Guide
The following sections describe the numbered callouts in detail.
3.1.1 - Toolbar (1-5)
-
Create Job: Opens the Create New Job screen, which
allows you to create a new Hadoop job (see “Creating a
New Job” on page 20).
-
Create Cluster: Opens the Create New Cluster screen,
which allows you to create a new virtual cluster (see “Creating a New Persistent Cluster” on page 37).
The Toolbar contains the following information/functions from
left to right:
•
Quick Status (1): This area of the Toolbar displays (from left
to right) the number of jobs, clusters, DataTaps, and virtual
nodes. Clicking an icon opens the appropriate page, as follows:
-
Jobs: Clicking this icon opens the Job Management
screen. See “Jobs” on page 18.
-
Clusters: Clicking this icon opens the Cluster Management screen. See “Persistent Clusters” on page 28.
-
DataTaps: Clicking this icon opens the DataTaps screen.
See “Viewing DataTaps” on page 41.
-
Virtual Nodes: Clicking this icon opens the Virtual Nodes
screen. See “Viewing DataTaps” on page 41.
•
User Guide (4): Clicking this link opens this User & Administrator Guide in Adobe Acrobat (PDF) format.
•
User button (5): Clicking this button opens a pull-down menu
with the following options:
-
Tenant: List of tenants that you have access to, based on
your EPIC login credentials (only appears if you can
access more than one tenant). Clicking a tenant opens
either the Tenant Administrator Dashboard screen (see
“The Tenant Member Interface” on page 13) or the
Member Dashboard screen (see “Viewing the Member
Dashboard” on page 17, depending on the role you
have for the selected tenant.
-
Change Password: Opens the Change Password popup
window, which allows you to modify your EPIC password.
See “Changing Your Password” on page 47.
-
Logout: Logs you out of EPIC. Once you log out, you will
need to log back as described in “Launching and Logging In” on page 7 to access EPIC.
The currently selected icon appears highlighted in green.
•
•
Login details (2): This area displays the following info:
-
Tenant you are currently viewing
-
Your username
-
Your role
Quick Access button (3): Clicking this button opens a pulldown menu with the following options:
14
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.1.2 - Main Menu (6)
The Member main menu contains the following options:
•
Dashboard: Opens the Dashboard screen. See “Viewing the
Member Dashboard” on page 17.
•
Jobs: Opens the Job Management screen, which allows you
to view, add, edit, and delete jobs. You may also view various
job logs. See “Jobs” on page 18.
•
Clusters: Opens the Cluster Managment screen, which
allows you to create, edit, and delete persistent virtual clusters. See “Persistent Clusters” on page 28.
•
DataTaps: Opens the DataTaps screen, which allows you to
view basic DataTap information. See “Viewing DataTaps” on
page 41.
•
Virtual Nodes: Opens the Virtual Nodes screen, which
allows you to view the virtual nodes that have been created
for this tenant. See “Viewing Virtual Nodes” on page 42.
3.1.3 - Menu Expand/Collapse (7)
Clicking the Expand/Collapse arrow at the bottom of the main
menu toggles the menu between wide (expanded) and narrow
(collapsed). This feature maximizes the amount of screen space
available for the work area on smaller monitors (such as mobile
devices).
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3.1.4 - Work Area (8)
The work area is where EPIC screens appear. Various generic
functions will be available here, depending on the screen you
are accessing. These generic functions may include some or all
of the following:
•
Use the Display... records pull-down menu to
select how many records you want to see displayed on a single screen.
•
Clicking an up arrow in a table header collapses that table.
•
Clicking a down arrow in a table header
expands that table.
•
Clicking a checkbox in a table selects that
item. You may select one or more items and
then perform an action on the selected item(s).
•
Clicking the checkbox in a table header
selects all of the items in that table.
•
Clicking the arrows in a table column sorts
the table by the information in that column.
For example, clicking the arrows in the Tenant Description
column sorts the list of tenants by the description. Repeatedly clicking a column header toggles the display between
ascending (A-Z) and descending (Z-A) order.
•
Entering one or more keyword(s) in the
Search field and then pressing [ENTER]
returns all records containing the supplied keyword(s).
15
EPIC User & Administrator Guide
Searches happen in real time; the work area refreshes as you
type.
•
•
If a screen contains too many records to display on a single page, you may use the page
numbers and arrows to move between pages, as follows:
-
Clicking a page number opens the selected page of the
current screen.
-
Clicking the First Page (<<) button takes you to the first
page of the current screen.
-
Clicking the Last Page (>>) button takes you to the last
page of the current screen.
Clicking the Back to Top icon at the bottom of
a screen scrolls you back to the top of the
current screen.
16
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.2 - Viewing the Member Dashboard
Clicking Dashboard in the main menu opens the Tenant Member
Dashboard screen, which presents a high-level overview of current activity within this tenant. This screen refreshes every 30
seconds.
The Dashboard screen contains the following information:
•
Jobs Charts: The following information appears for each current job within the tenant:
-
•
Run Time: Yellow bar showing how long the job took to
run, in seconds.
Jobs: The following information appears for each current job:
-
Job Name: Name of the job.
-
Job Status: Status of the job. See “Job Statuses” on
page 45.
Setup Time: Blue bar showing how long the job took to
set up, in seconds.
Figure 3.2: Tenant Member Dashboard screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
17
EPIC User & Administrator Guide
3.3 - Jobs
Selecting Jobs in the main menu opens the Job Management
screen.
This screen contains the following buttons:
•
Create: Clicking the green Create button opens the Create
New Job screen. See “Creating a New Job” on page 20.
•
Delete: Clicking the red Delete button deletes the selected
job(s) from the tenant. Deleting a job that has completed with
or without errors will only remove the job record. A popup
warning appears asking you to confirm or cancel the action.
Click OK to proceed, or Cancel to exit without deleting the
job.
CAUTION: YOU CANNOT UNDELETE A JOB.
Figure 3.3: Job Management screen
18
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
The table on this screen contains the following information and
functions:
•
•
•
Name: Name of the job. Clicking a job name opens the <Job>
Output popup, where <Job> is the name of the job. This
popup displays the current job output and refreshes every 15
seconds. You may view the job output at any time during or
after running the job. See “Viewing Job Output” on page 27.
-
ClusterFS: While a transient job is running, clicking the
ClusterFS link opens the <Cluster> Cluster FS Browser
screen for the transient cluster dedicated to that job. See
“Uploading and Downloading Files” on page 43.
-
Cluster Details: While a transient job is running, clicking
the Cluster Details link opens the <Cluster> screen for
the transient cluster dedicated to that job. See “Viewing
Cluster Details” on page 30.
Distribution: Hadoop distribution being used for the job. See
“Images Tab” on page 103.
•
Started at: Date and time the job started.
Details: Lists the following information for each job:
•
Completed at: Date and time the job completed.
-
Job type: (such as Hadoop Custom Jar).
•
-
App Name: Name of the Hadoop application running the
job (such as teragen).
Status: Current status of the job. See “Job Statuses” on
page 45.
•
Actions: The following actions are available for each job:
-
Worker Count: For transient jobs, number of Worker
nodes assigned to the transient cluster for that job.
-
Controller Node Flavor: For transient jobs, displays the
name and definition of the flavor used for the Controller
node. See “Flavor Tab” on page 107.
-
Worker Node Flavor: For transient jobs, displays the
name and definition of the flavor used for the Worker
node(s). See “Flavor Tab” on page 107.
-
Persistent Cluster: For jobs assigned to a persistent cluster, the name of the cluster to which the job is assigned.
Clicking a cluster name opens the <Cluster> screen for
the selected cluster. See “Viewing Cluster Details” on
page 30.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
-
Clone: Clicking the gray Clone icon (two sheets) in the
Actions column opens the Create New Job screen prepopulated with all of the existing parameters from the job
you are cloning, except that the job will be named <jobname>_n, where n is the number of the current copy. See
“Creating a New Job” on page 20.
-
Delete: Clicking the red Delete icon (trash can) in the
Actions column deletes the job from the tenant. Deleting
a running job will immediately end the job. Deleting a job
that has completed with or without errors will only
remove the job record. A popup warning appears asking
you to confirm or cancel the action. Click OK to proceed,
or Cancel to exit without deleting the job. Please see the
caution on the previous page.
19
EPIC User & Administrator Guide
-
-
-
-
Edit: Clicking the blue Edit icon (pencil) in the Actions column opens the Edit Job screen. See “Editing a Job” on
page 24.
Setup Log: Clicking the gray Setup Log icon (down arrow
in a circle) in the Actions column opens the setup log for
the selected job. This icon appears for jobs run on transient cluster only. See “Viewing the Cluster Setup Log”
on page 34.
Job Output: Clicking the orange Job Output icon (down
arrow in a circle) in the Actions column opens the job output log for the selected job. See “Viewing the Job Setup
Log” on page 26.
Job History: Clicking the purple Job History icon (down
arrow in a circle) in the Actions column opens the job history log for the selected job. See “Viewing the Job History Log” on page 27.
3.3.1 - Creating a New Job
Clicking the green Create button in the Job Management
screen or selecting Create Job in the Quick Access button menu
opens the Create New Job screen. If you clicked the Clone icon
next to an existing job, then this screen will be pre-populated
with the information cloned from the selected job, which you
may accept or edit as needed.
To create a new job:
1. Enter a name for this job in the Job Name field.
2. Select the type of job you are creating using the Job Type
pull-down menu. Your available options are as follows:
-
Hadoop Custom Jar
-
Hadoop Streaming
-
Pig Script
-
Hive Script
-
Impala Script
-
HBase Script
-
Spark - Scala Jar
-
Spark - Java Jar
-
Spark - Python Script
3. Select the .jar or script file(s), as follows:
-
If you selected Hadoop Custom Jar, Spark - Scala Jar, or
Spark - Java Jar, then enter the .jar file and the application name to use in the Jar File and App Name fields,
respectively.
Note: You cannot create a new job if EPIC is in Lockdown mode. See “Lockdown Mode” on page 122.
20
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
Figure 3.4: Create New Job screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
21
EPIC User & Administrator Guide
-
If you selected Hadoop Streaming, then enter the mapper and reducer scripts to use in the Mapper Script and
Reducer Script fields, respectively.
-
If you selected Pig Script, Hive Script, HBase Script, or
Impala Script, then enter the script path to use in the
Script Path field.
-
If you selected Spark - Python Script, then enter the
script to use in the Script File field.
location. See “System Storage Tab” on page 110 and
the About EPIC Guide for more information about system
storage.
Clicking a Choose button opens a File Upload popup, which
allows you to upload a script or .jar file.
Please see the Running Applications in EPIC Guide for stepby-step instructions and examples for each supported job
type.
-
To specify a custom path to use for the cluster file system,
you may type the complete path in the Cluster FS Path
field and/or click the Insert button to open the DataTap
Browser window and select a DataTap to use as the cluster file system.
-
You may drill down into existing directories under a particular DataTap root, and then double-click a specific file
or directory to have its URI copied into your arguments
list.
-
If you need to supply a URI for a file or directory that does
not yet exist, you can drill down as far as possible in the
existing directories, double-click to transfer that URI into
the argument list, and then manually edit the arguments
to complete the URI with your desired path.
-
The location of the cluster file system does not change
once the virtual cluster is created. This is true regardless
of whether the cluster file system resides in system storage or elsewhere. If you originally specified the location
of the file system using a DataTap, that location remains
constant even if the DataTap you used is later changed or
deleted.
-
The Insert widget is a helper. You may modify the Cluster
FS Path URI as needed or manually type in a complete
URI from scratch.
4. Select the cluster type to use for this job by checking the
appropriate Cluster Type radio button. You may select either:
-
-
Transient: A transient cluster is created based on your
Worker Count, Controller Node Flavor, and Worker
Node Flavor settings (see Step 8, below) when you create the job and is removed (destroyed) when the job completes. Proceed to Step 5.
Persistent: A persistent cluster remains in existence independent of individual jobs and can be used again and
again. Skip to Step 9.
5. Select the cluster file system as follows:
-
If you leave the Cluster FS Path field blank, then the cluster file system will use the current EPIC system storage
22
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
6. Select the correct Hadoop or Spark distribution to use for this
job using the Distribution pull-down menu. The available
options will depend on the distribution(s) you currently have
installed. See “Images Tab” on page 103.
7. Select your desired option(s) for the selected distribution in
any field(s) that appear below the Distribution pull-down
menu, such as MR Type.
The Insert widget appears next to these fields as a helper.
When using this widget:
-
You may modify the URIs in the argument list as needed
or manually type in complete URIs from scratch.
-
This widget assists with creating DataTap URIs. If an argument is a path within the cluster file system, then the widget can assist with finding/constructing such a path if the
job is being dispatched to a persistent cluster. For jobs
running on transient clusters, you must enter such paths
manually.
8. If you are running this job on a transient cluster, then:
-
Select the appropriate number of Worker nodes to use for
this job using the Worker Count menu.
-
Select the flavors to use for the nodes using the Controller Node Flavor and Worker Node Flavor pull-down
menus. See the About EPIC Guide and “Flavor Tab” on
page 107 for more information about flavors.
Skip to Step 10.
9. If you selected Persistent for the Cluster Type, then use the
Cluster pull-down menu to select the cluster on which you
want to run the job.
10. Enter the following information as needed:
-
-
11. If you selected one of the Hadoop or Spark options as the
job type in Step 2, you will see a read-only Command Line
field that allows you to verify the job you are about to run.
12. Depending on the Hadoop distribution and option(s) you
selected in Steps 6 and 7, you may see an Advanced Settings link. Clicking this link expands the Advanced Settings
area, which allows you edit various configuration options.
This step is optional and intended for advanced users only.
When using advanced settings:
-
If you selected Hadoop Custom Jar or any of the Spark
options as the job type in Step 2, then you may add or
edit arguments in the Edit Arguments field.
The available settings depend on the options you
selected in the previous steps.
-
If you selected Hadoop Streaming as the job type in Step
2, then you must enter an input and output path in the
Input Path and Output Path fields, respectively.
The default values for each setting are selected by EPIC
based on the job settings and Hortonworks best practices.
-
Changing a default value displays an orange Edited notification at the top of the Advanced Settings area.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
23
EPIC User & Administrator Guide
-
Clicking the red Reset to Defaults button replaces any
edited values with the default values.
-
You may search for values by entering your desired
search term in the Search field.
Figure 3.5: Advanced Settings tools
When you have finished inputting the job parameters, click Submit to save your changes and run the job. If job creation fails with
an error, then please see “Common Job/Cluster Errors” on
page A-5 for a list of common errors and how to resolve them.
3.3.2 - Editing a Job
In the Job Management screen, clicking the blue Edit icon (pencil) for a job opens the Edit Job screen for the selected job.
Note: You cannot edit a job if EPIC is in Lockdown
mode. See “Lockdown Mode” on page 122.
This screen contains the same fields that appeared on the Create New Job screen fields. The fields that appear will vary
depending on whether the job is on a transient or persistent
cluster, type of job, etc. These fields are read-only when the job
is running. When the job is not running, you may edit some of the
fields, depending on the job type, type of cluster, etc.
Click the Submit button when you have finished editing the job
to save your changes.
24
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
Figure 3.6: Edit Job screen (persistent cluster)
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
25
EPIC User & Administrator Guide
3.3.3 - Viewing the Job Setup Log
In the Job Management screen, clicking the gray Setup Log icon
(down arrow in a circle) in the Actions column opens the job
setup log for the selected transient job.
Figure 3.7: Job Setup Log
3.3.4 - Viewing the Job Output Log
In the Job Management screen, clicking the orange Job Output
icon (down arrow in a circle) in the Actions column opens the job
output log for the selected job.
Figure 3.8: Job Output log
26
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.3.5 - Viewing the Job History Log
In the Job Management screen, clicking the purple Job History
icon (down arrow in a circle) in the Actions column opens the job
history log for the selected job.
Figure 3.9: Job History log
3.3.6 - Viewing Job Output
Figure 3.10: Job Output popup (below)
At any time, you may click the name of the job in the Job Name
column of the Job Management screen to open the <Job> Output popup, where <Job> is the name of the selected job. This
popup refreshes every 15 seconds while the job is running, and
displays the output being generated by this job. This is the same
output that you will receive when retrieving the job output as
described in the Running Applications Guide.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
27
EPIC User & Administrator Guide
3.4 - Persistent Clusters
Selecting Clusters in the main menu opens the Cluster Management screen for the tenant, which displays the persistent cluster(s). This screen contains the following buttons:
•
•
•
Create: Clicking this button opens the Create New Cluster
screen. See “Creating a New Persistent Cluster” on
page 37.
Delete: Clicking this button deletes the selected cluster(s)
from the tenant. A popup warning appears asking you to confirm or cancel the action. Click OK to proceed, or Cancel to
exit without deleting the cluster(s).
CAUTION: YOU CANNOT UNDELETE A CLUSTER.
DELETING A CLUSTER IMMEDIATELY ENDS ANY
JOB(S) RUNNING ON THE CLUSTER.
Tenant KeyPair: Clicking this button icon opens an OSdefault Download window that allows you to retrieve the
tenant SSL certificate to allow SSH connections to the virtual
nodes in a tenant. Certificates are provided in .pem format.
You cannot connect to the default Site Admin tenant via SSH.
The table on this screen contains the following information and
functions:
•
Name: Name of the cluster. Clicking a cluster name opens
the <Cluster> screen for that cluster, where <Cluster> is the
name of the cluster. See “Viewing Cluster Details” on
page 30.
•
Type: Type of cluster (such as Hadoop or HBase).
•
Distribution: Distribution and MR type used for the cluster
(such as Hortonworks HDP 1.3/MRv1).
Figure 3.11: Cluster Management screen
28
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
•
Flavor Details: Flavor of the Controller and Worker nodes.
See the About EPIC Guide and “Flavor Tab” on page 107
for more information about node flavors.
Note: Click the Edit icon (pencil) for a cluster to view
the flavor of any Edge node(s) in the cluster. See
“Editing a Persistent Cluster” on page 35.
•
Workers Number of Worker nodes under the Controller node
for the cluster.
•
Details: Lists the following information for each cluster:
-
Apps Installed: Hadoop application(s) installed on this
cluster, such as Pig, Hive, Impala, and/or Hue.
-
Spark Installed: Whether (true) Spark is installed on the
cluster. This does not appear if Spark is not installed.
-
Cluster FS: Clicking this link opens the <Cluster> Cluster
FS Browser screen for the selected cluster, which allows
you to browse the cluster file system. If the cluster file
system is using local storage, then you may also upload
and download files within EPIC. See “Uploading and
Downloading Files” on page 43.
-
High Availability: If YARN high availability has been
enabled for the cluster, then the notation High Availability: On will appear.
-
Edge Nodes: Number of Edge nodes in the cluster. See
“Creating a New Persistent Cluster” on page 37 and
“Add-On Images Tab” on page 105.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
•
Status: Current status of the cluster. See “Cluster Statuses”
on page 45.
•
Actions: The following actions are available for each cluster:
-
Clone: Clicking the gray Clone button (two sheets) in the
Actions column opens the Create Cluster screen with
the information from the cluster being cloned pre-populated in the fields. You may either accept this information
as-is or edit to suit your needs. See “Creating a New Persistent Cluster” on page 37.
-
Delete: Clicking the red Delete icon (trash can) in the
Actions column deletes the cluster from the tenant. A
popup warning appears asking you to confirm or cancel.
Please see the caution on the previous page.
-
Setup Log: Clicking the gray Setup Log icon (down arrow
in a circle) in the Actions column opens the setup log for
the cluster. See “Viewing the Cluster Setup Log” on
page 34.
-
Edit: Clicking the blue Edit icon (pencil) in the Actions column opens the Edit Cluster screen. See “Editing a Persistent Cluster” on page 35.
-
Hosts File Info: Clicking the purple Hosts file info icon
(monitor) opens the hosts file for the selected cluster in a
new browser tab/window.
29
EPIC User & Administrator Guide
3.4.1 - Viewing Cluster Details
Clicking a cluster name in the Cluster Management screen or
clicking the Cluster Details link in the Job Management screen
for a transient job opens the <Cluster> screen for that persistent,
where <Cluster> is the name of the cluster. This screen has the
following tabs:
•
Node(s) Info: See “Node(s) Info Tab” on page 30.
•
Job(s) Info: See “Job(s) Info Tab” on page 30.
•
Charts: See “Charts Tab” on page 31.
•
Service(s) Status: This tab appears for clusters that use an
image based on CentOS. See “Service(s) Status Tab” on
page 33.
•
Node Name: Name of the node.
•
Distribution: Distribution type used for the node.
•
Role: Whether the node is a Controller (controller), Worker
(worker), or Edge (edge) node. If YARN High Availability is
enabled, this column may also say arbiter or standby, as
appropriate.
•
Public IP: IP address used to access the node.
You may also view the cluster file system by clicking the Cluster
FS Browser link at the top of this screen. See “Uploading and
Downloading Files” on page 43.
3.4.1.1 - Node(s) Info Tab
The Node(s) Info tab of the of the <Cluster> screen contains the
Node List table.
This table displays the following information for each of the
nodes in the selected cluster:
•
HBase Client Configuration: This link appears for HBase
clusters only. Clicking this link opens the Client Config Information popup, which displays configuration information for
the cluster.
30
Figure 3.12: Client Config Information popup
•
Process list: List of currently running Hadoop services. Clicking an item in this list opens the Web interface for the
selected service (including third-party tools such as Cloudera
Manager or Ambari, if included in the cluster), if your network
includes a route from the computer you are using to view
EPIC to the Controller node. See the About EPIC Guide for
requirements.
3.4.1.2 - Job(s) Info Tab
The Job(s) Info tab of the of the <Cluster> screen contains the
Running/Finished Jobs table.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
This table displays the following information for available jobs:
•
Job Name: Name of the job.
•
Job Details: Lists the following information for each job:
-
Job type: (such as Hadoop Custom Jar).
-
App Name: Name of the Hadoop application running the
job (such as teragen).
•
Started at: Date and time the job started.
•
Completed at: Date and time the job completed.
•
Job Status: Current job status. See “Job Statuses” on
page 45.
3.4.1.3 - Charts Tab
The Charts tab of the <Cluster> screen contains a series of dials
and charts that provide performance information for the cluster.
•
CPU %: The dial indicates the current percentage of physical
CPU utilization across all cluster processes that are currently
running, and the accompanying chart displays the average
CPU percentage over the selected time period.
•
Memory (GB): The dial indicates the current use of physical
memory across all cluster processes, and the accompanying
chart displays the memory usage over the selected time
period.
•
Network: The dial indicates the current amount of physical
network bandwidth being used by the cluster, and the
accompanying chart displays network bandwidth usage over
the selected time period.
The tabs underneath the dials contain tables that display historical metrics for the load, CPU usage, memory usage, and network
usage for the selected time period.
The top of this screen contains a pull-down menu that lets you
select the time period to review. The available options are Last
Hour (default), 2 Hours, 4 Hours, Day, Week, Month, and Year.
This tab contains displays the following information for the
selected time period:
•
Load: The dial shows the current average percentage of
physical CPU cores used by the cluster (defined as the number of CPU cores in use vs. the total number of available CPU
cores), and the accompanying chart displays the average
load over the selected time period.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
31
EPIC User & Administrator Guide
Figure 3.13: Node(s) Info tab.
Figure 3.14: Job(s) Info tab
32
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
Figure 3.15: Charts tab
3.4.1.4 - Service(s) Status Tab
Clusters using EPIC images based on CentOS will also include a
Service(s) Status tab after the Charts tab. This tab has two or
more groups of information. The two primary groups are:
•
GANGLIA: This group monitors the health of services that
deliver the info for the Charts tab.
•
MAPREDUCE: This group monitors the JobTracker and TaskTracker services that run Hadoop jobs.
•
Additional groups may also appear, depending on the cluster
configuration. These are beyond the scope of this manual.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
The status of a service can be either OK (green dot), CRITICAL
(red dot), or DISABLED (intentionally not running; gray dot). Hovering the mouse over the status button opens a popup with additional information. In general:
•
The Controller node for the cluster must not display any red
dots. If the Controller node has one or more error(s), then the
cluster may not function properly.
•
If all of the dots for a Worker node are red, then that node will
not be able to participate in jobs that run on the cluster; however, the cluster as a whole should still perform correctly with
33
EPIC User & Administrator Guide
a reduced capacity. This situation usually occurs because the
physical host is powered off or has lost network connectivity.
Figure 3.16: Service(s) Status tab
•
A Worker node with some red and some green dots may
cause some jobs to fail, unless the errors are transient conditions caused by the physical host powering on or regaining
network connectivity.
•
The host(s) has/have network connectivity.
See “Troubleshooting” on page A-1 for more information about
resolving errors.
Please contact BlueData Technical Support if a cluster that is
reporting service errors meets all of the following criteria:
3.4.2 - Viewing the Cluster Setup Log
•
The EPIC software is completely installed on the affected
physical host(s).
•
The host(s) is/are powered on.
In the Cluster Management screen, clicking the gray Setup Log
icon (down arrow in a circle) in the Actions column opens the
setup log for the selected persistent cluster.
34
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.4.3 - Editing a Persistent Cluster
•
Clicking the blue Edit icon (pencil) for a cluster in the Cluster
Management screen opens the Edit Cluster screen for that persistent cluster.
CAUTION: EDITING THE ADVANCED SETTINGS
WILL INTERRUPT ALL JOBS CURRENTLY RUNNING ON THE CLUSTER, INCLUDING BOTH JOBS
SUBMITTED VIA EPIC AND JOBS CREATED BY A
USER WHO IS DIRECTLY LOGGED INTO THE
CLUSTER VIRTUAL NODES.
Note: You cannot edit a cluster when EPIC is in Lockdown mode. See “Lockdown Mode” on page 122.
To edit a cluster, you may do one or more of the following:
•
Rename the cluster by entering a new name in the Cluster
Name field.
•
Increase the number of Worker nodes in the cluster by either
entering a number in the Worker Count field or using the up/
down arrows.
You may also edit advanced settings by clicking the
Advanced Settings link.
•
Additional properties may also be available depending on
how the cluster is set up. Please see “Creating a New Persistent Cluster” on page 37 for information on the field(s)
and menu(s).
Figure 3.17: Cluster Setup Log
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
35
EPIC User & Administrator Guide
When you have finished editing the cluster setting(s), click Submit to save your changes to the cluster settings or Reset to clear
your changes without editing the cluster.
Note: The Select Cluster Type, Hadoop Distribution, Controller Node Flavor, and Worker Node Flavor menus are read-only, as are Edge node settings.
You cannot edit these settings.
Figure 3.18: Edit Cluster screen
36
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.4.4 - Creating a New Persistent Cluster
Clicking the green Create button in the Cluster Management
screen or selecting Create Cluster in the Quick Access button
menu opens the Create New Cluster screen. If you clicked the
Clone icon next to an existing persistent cluster, then this screen
will be pre-populated with the information cloned from the
selected cluster, which you may accept or edit as needed.
Note: You cannot create a new cluster if EPIC is in
Lockdown mode. See “Lockdown Mode” on
page 122.
-
If you leave the Cluster FS Path field blank, then the cluster file system will use the current EPIC system storage
location. See “System Storage Tab” on page 110 and
the About EPIC Guide for more information about system
storage.
-
To specify a custom path to use for the cluster file system,
you may type the complete path in the Cluster FS Path
field and/or click the Insert button to open the DataTap
Browser window and select a DataTap to use as the cluster file system.
-
You may drill down into existing directories under a particular DataTap root, and then double-click a specific file
or directory to have its URI copied into your arguments
list.
-
If you need to supply a URI for a file or directory that does
not yet exist, you can drill down as far as possible in the
existing directories, double-click to transfer that URI into
the argument list, and then manually edit the arguments
to complete the URI with your desired path.
-
The location of the cluster file system does not change
once the virtual cluster is created. This is true regardless
of whether the cluster file system resides in system storage or elsewhere. If you originally specified the location
of the file system using a DataTap, that location remains
constant even if the DataTap you used is later changed or
deleted.
To create a new persistent cluster:
1. Enter a name for the new cluster in the Cluster Name field.
2. Select the cluster type using the Cluster Type pull-down
menu. The following options are available:
-
Hadoop: Installs Hadoop on the cluster.
-
HBase: Installs both Hadoop and HBase on the cluster.
-
Spark: Installs Spark on the cluster.
-
Utility: Creates a blank CentOS 6.4 image with an empty
(virtual) Controller node and the specified number of
Worker nodes.
Note: Hadoop clusters that use the Cloudera CDH 5
distribution can also support running Spark jobs.
3. Select the cluster file system as follows:
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
37
EPIC User & Administrator Guide
-
EPIC Guide and “Flavor Tab” on page 107 for more information about node flavors.
The Insert widget is a helper. You may modify the Cluster
FS Path URI as needed or manually type in a complete
URI from scratch.
4. Select the correct Hadoop or Spark distribution to use for this
cluster using the Distribution pull-down menu. If your
selected distribution includes an additional application such
as Cloudera Manager or Ambari, then that application will be
installed when you create the cluster and accessible by clicking the appropriate link in the Process List column of the
<Cluster> screen for that cluster. The available options will
depend on the distribution(s) you currently have installed.
See “Images Tab” on page 103.
Note: If you select a distribution that includes Cloudera Manager, then you must set the Controller Node
Flavor to one with a minimum of 4vCPU cores and
12GB of RAM. The default Large flavor included with
EPIC meets this requirement. Selecting a smaller flavor will cause cluster creation to fail.
5. Additional checkboxes may appear if you select a distribution
that has one or more eligible add-on images available. Check
the appropriate checkbox(es) to install the desired application(s) on one or more Edge node(s).
6. Select your desired option(s) for the selected distribution in
the field(s) that appear below the Distribution pull-down
menu, such as MR Type.
8. Select the number of Worker nodes to use for this cluster by
either entering a number in the Worker Count field or using
the up/down arrows.
9. Select the desired flavor for the Worker node(s) using the
Worker Node Flavor pull-down menu. Each Worker node in
the cluster will have the same flavor. See the About EPIC
Guide and “Flavor Tab” on page 107 for more about node
flavors.
10. Checking the YARN High Availability checkbox enables
YARN resource manager high availability within the virtual
cluster. This allows the cluster to remain usable if the Controller node fails. This option is only be available for certain distributions and has the following prerequisites:
-
You must select YARN in the MR Type pull-down menu.
-
The cluster must have 2 or more Worker nodes.
-
The EPIC platform must contain at least 3 physical hosts.
Enabling YARN high availability also creates additional placement constraints for the virtual nodes, which may cause cluster creation to fail if the EPIC platform is low on available
resources.
7. Select the desired flavor for the Controller node using the
Controller Node Flavor pull-down menu. See the About
38
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
CAUTION: ENABLING HIGH AVAILABILITY FOR A
CLUSTER PROTECTS AGAINST VIRTUAL NODE
FAILURE. THIS IS INDEPENDENT OF EPIC PLATFORM HIGH AVAILABILITY FUNCTIONALITY,
WHICH PROTECTS AGAINST PHYSICAL CONTROLLER NODE FAILURE. PLEASE SEE THE About
EPIC Guide FOR MORE INFORMATION.
11. Check the Applications checkbox if you want the additional
application(s) listed to be installed and configured in your
cluster. The available options will vary depending on the
Hadoop distribution you selected in Step 4. For example, if
you select one of the Cloudera CDH distributions, then this
checkbox will allow you to install the Pig, Hive, Impala, Hue,
and Spark application(s).
12. Depending on the Hadoop distribution and option(s) you
selected in Steps 4 through 6, you may see an Advanced
Settings link. Clicking this link expands the Advanced Settings area, which allows you edit various configuration
options. This step is optional and intended for advanced
users only. When using advanced settings:
-
The available settings depend on the options you
selected in the previous steps.
-
The default values for each setting are selected by EPIC
based on the job settings and Hortonworks best practices.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
-
Changing a default value displays an orange Edited notification at the top of the Advanced Settings area.
-
Clicking the red Reset to Defaults button replaces any
edited values with the default values.
-
You may search for values by entering your desired
search term in the Search field.
Figure 3.19: Advanced Settings tools
13. If you selected cluster parameters that meet the requirements of one or more installed third-party data visualization
and analysis application(s) (see “Add-On Images Tab” on
page 105), then one or more checkbox(es) will appear for the
compatible application(s). You can enable any combination of
available applications by checking the appropriate checkbox(es) and then using the pull-down menu to select an Edge
node flavor to use with that application (see “Flavor Tab” on
page 107).
Note: Edge nodes are virtual nodes like the Controller and any Worker node(s) in the cluster and consume the same resources as any other node with
the same flavor. For example, if your cluster contains
one Controller node, three Worker nodes, and two
Edge nodes, then the cluster will consist of six virtual
nodes.
39
EPIC User & Administrator Guide
Note: You can access any Edge node application by
clicking the appropriate link in the Process List column of the <Cluster> screen for the selected cluster.
When you have finished inputting the parameters for the new
cluster, click Submit to create the new persistent cluster. If cluster creation fails with an error, then please see “Common Job/
Cluster Errors” on page A-5 for a list of common errors and how
to resolve them.
Figure 3.20: Create New Cluster screen
40
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.5 - Viewing DataTaps
Selecting DataTaps in the main menu opens the DataTaps
screen. This screen displays the following information and is
read-only; you cannot edit any of these parameters:
•
Name: Name of the DataTap. Clicking a name in this column
opens the <DataTap> Browser screen for the selected
DataTap. See “Uploading and Downloading Files” on
page 43.
•
Description: Brief description of the DataTap that was
entered when the Tenant Administrator created that DataTap.
See “DataTaps” on page 56.
Figure 3.21: DataTaps screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
41
EPIC User & Administrator Guide
3.6 - Viewing Virtual Nodes
Selecting Virtual Nodes in the main menu opens the Virtual
Nodes screen.
•
Distribution: Type of Hadoop running on the virtual node.
See “Images Tab” on page 103.
This screen displays the following information for your information only; you cannot edit any of these parameters:
•
Cluster type: Whether the cluster was created for a specific
job only (Transient) or whether it exists independently of any
single job (Persistent).
•
Status: Current status of the virtual node. See “Virtual Node
Statuses” on page 45.
•
Name: Name of the virtual node.
•
Public IP: IP address used to access that node.
•
Flavor: Flavor of the virtual node. See the About EPIC Guide
for more information about flavors and “Flavor Tab” on
page 107 for information about customizing flavors.
Figure 3.22: Virtual Nodes screen
42
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.7 - Uploading and Downloading Files
EPIC offers a convenient way to upload and download data to
and from cluster file systems and DataTaps.
•
•
In the DataTaps screen, clicking the name of a DataTap
opens the <DataTap> Browser screen, where <DataTap> is
the name of the selected DataTap.
In the Cluster Management screen, clicking a Cluster FS link
in the Cluster Details column opens the <Cluster> Cluster
FS Browser screen, where <Cluster> is the name of the
selected cluster.
Upload and download functionality is identical for both DataTaps
and cluster file systems. This link is always available for persistent clusters; however, for transient clusters, it is only available
as long as the job is running.
The bottom of this screen contains an expandable tree view of
the directories underneath the root directory of the selected
DataTap or cluster file system. In this view:
•
Clicking a plus sign (+) next to a directory expands that directory to display the file(s) and sub-diectories (if any) under the
selected directory.
•
Clicking a minus sign (-) next to a directory collapses the view
of the file(s) and sub-directories (if any) under the selected
directory.
When you are browsing locations within a locally-created system
storage service, the buttons across the top of this screen allow
you to perform the following actions (from left to right):
•
Selecting a directory and then clicking the green Create
Directory button (plus sign) opens the Create new
directory under <dir> window, where <dir> is the name of the
currently selected directory. Entering a name in the field and
then clicking OK creates a new sub-directory.
Figure 3.23: <Cluster> Cluster FS Browser screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
43
EPIC User & Administrator Guide
directory structure and select paths for various UI purposes. In
this case, you will need to upload/download files and/or create/
remove directories from outside the EPIC interface using some
native client appropriate for the storage service. For certain
operations (like creating a directory), it may also be useful to
access the DataTap from within a virtual node and then manually
perform hadoop fs operations on it.
•
Selecting a directory or file and then clicking the blue
Rename button (notepad) opens the Rename <item>
window, where <item> is the name of the currently selected
directory or file. Entering a name in the field and then clicking
OK renames the selected directory or file.
•
Selecting a directory and then clicking the gray Upload
button (up arrow) opens a standard Upload dialog,
which allows you to navigate to, select, and upload a file to
the selected directory. The dialog appearance will vary
based on your OS and browser settings.
•
Selecting a file and then clicking the purple Download
button (down arrow) opens a standard Save As dialog,
which allows you to save the selected file to a directory on
either your local hard drive or any network storage that you
have access to.
•
Selecting a directory or file and then clicking the red
Delete button (trash can) deletes the selected directory
or file. Deleting a directory also deletes all of the sub-directories and files within that directory.
CAUTION: DO NOT RENAME OR DELETE A DIRECTORY OR FILE THAT IS IN USE, AS THIS COULD
CAUSE JOB FAILURES AND OTHER ERRORS.
THERE IS NO UNDO FUNCTION WHEN DELETING
A DIRECTORY OR FILE.
As noted above, these buttons are only when browsing locations
within a locally-created system storage service (see “System
Storage Tab” on page 110). For any other DataTap, the <Cluster> Cluster FS Browser screen will allow you to view the file/
44
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.8 - Status Messages
•
Deleting: Indicates that EPIC has accepted a command to
delete the cluster and is processing that request. Deleting a
cluster that has Error status will immediately remove that
cluster; otherwise, the cluster will remain in the Deleting
state for some time while EPIC shuts down its virtual nodes.
EPIC displays the following status messages for virtual nodes, as
appropriate for each node:
•
Updating: The cluster is applying a change to its configuration or its number of virtual nodes.
•
Job Name: Name of the job that the virtual node is currently
processing, if the node is part of a transient cluster.
•
Error: Cluster creation or expansion has failed. See “Common Job/Cluster Errors” on page A-5.
•
Cluster name: Name of the virtual cluster, if the node is part
of a persistent cluster.
EPIC displays status messages for virtual nodes, clusters, and
jobs.
3.8.1 - Virtual Node Statuses
3.8.2 - Cluster Statuses
EPIC displays the following status messages for clusters, as
appropriate for each cluster:
•
Created: EPIC has accepted a command to create or configure the cluster but has not started work yet.
•
Starting: EPIC is in the process of creating and/or configuring
the cluster and virtual nodes. Hovering the mouse over the
status button opens a popup with additional progress information.
•
Ready: The cluster is ready to accept and process jobs.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3.8.3 - Job Statuses
EPIC displays the following status messages for jobs, as appropriate for each job:
•
Created: EPIC has accepted a command to start the job but
has not started work yet.
•
Starting: EPIC is in the process of creating and/or configuring
the job and virtual nodes. Hovering the mouse over the status button opens a popup with additional progress information.
•
Deleting: Indicates that EPIC has accepted a command to
delete the job and is processing that request. Deleting a job
that has either the Complete or Error status will immediately
remove that job; otherwise, the job will remain in the Delet-
45
EPIC User & Administrator Guide
ing state for some time while EPIC shuts down its virtual
nodes.
•
Running: The job is currently in progress.
•
Complete: The job has successfully finished running.
•
Error: The job failed to run. See “Common Job/Cluster
Errors” on page A-5.
Note: EPIC displays the Complete and Error job statuses based solely on whether or not EPIC was able
to set up and execute the job. Errors that occur
within the application itself (such as by having an
improper argument and/or bad data) are not necessarily reflected as errors in the EPIC job status display. Be sure to review all job output to verify that
you obtained the expected results, and rerun the job
with any problem(s) fixed if this is not the case.
46
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
3 - Using a Tenant
3.9 - Changing Your Password
Clicking the User button in the Toolbar and then selecting
Change Password opens the Update User Password popup.
When you have finished entering your new password, click Submit to save your changes or Cancel to clear your changes without changing your password.
Figure 3.24: Update User Password popup
To change your password:
1. Enter your current (old) password in the Current Password
field.
2. Enter your new password in the New Password field. Passwords are case-sensitive.
3. Confirm your new password in the Confirm Password field.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
47
EPIC User & Administrator Guide
This page intentionally left blank.
48
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4 - Tenant Administration
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
49
EPIC User & Administrator Guide
This chapter describes the EPIC Tenant Administrator interface.
A tenant administrator can perform the following functions:
•
Login to EPIC: See “Launching and Logging In” on page 7.
•
View the Tenant Administrator Dashboard screen: See
“Viewing the Tenant Admin Dashboard” on page 55.
•
View, add, edit, and remove Hadoop jobs: See “Jobs” on
page 18.
•
Add, edit, and remove persistent clusters: See “Persistent
Clusters” on page 28.
•
Add, edit, and remove DataTaps: See “DataTaps” on
page 56.
•
View virtual nodes: See “Viewing Virtual Nodes” on
page 42.
•
Assign and revoke user roles: See “Users” on page 62.
50
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4 - Tenant Administration
4.1 - The Tenant Administrator Interface
The Tenant Administrator interface contains the following elements:
Figure 4.1: The Tenant Administrator interface
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
51
EPIC User & Administrator Guide
The following sections describe the numbered callouts in detail.
•
4.1.1 - Toolbar (1-5)
The Toolbar contains the following information/functions from
left to right:
•
Quick Status (1): This area of the Toolbar displays (from left
to right) the number of jobs, clusters, DataTaps, virtual nodes,
and users. Clicking an icon opens the appropriate page, as
follows:
Quick Access button (3): Clicking this button opens a pulldown menu with the following options:
-
Create Job: Opens the Create New Job screen, which
allows you to create a new Hadoop job (see “Creating a
New Job” on page 20).
-
Create Cluster: Opens the Create New Cluster screen,
which allows you to create a new cluster (see “Creating a
New Persistent Cluster” on page 37).
-
Create DataTap: Opens the Create New DataTap screen,
which allows you to create a new DataTap (see “Creating
a New DataTap” on page 59).
-
Assign Users: Opens the Assign Users screen, which
allows you to grant roles to EPIC users within the current
tenant (see “Assigning Users to a Tenant” on page 63).
-
Jobs: Clicking this icon opens the Job Management
screen. See “Jobs” on page 18.
-
Clusters: Clicking this icon opens the Cluster Management screen. See “Persistent Clusters” on page 28.
-
DataTaps: Clicking this icon opens the DataTaps screen.
See “DataTaps” on page 56.
•
-
Virtual Nodes: Clicking this icon opens the Virtual Nodes
screen. See “Viewing Virtual Nodes” on page 42.
User Guide (4): Clicking this link opens this User & Administrator Guide in Adobe Acrobat (PDF) format.
•
User button (5): Clicking this button opens a pull-down menu
with the following options:
-
Users: Clicking this icon opens the <Tenant> screen. See
“Users” on page 62.
The currently selected icon appears highlighted in green.
•
Your role
Login details (2): This area displays the following information:
-
Tenant you are currently viewing
-
Your username
52
-
Tenant: List of tenants that you have access to, based on
your EPIC login credentials. Clicking a tenant opens
either the Tenant Administrator Dashboard screen (see
“Viewing the Tenant Admin Dashboard” on page 55) or
the Member Dashboard screen (see “Viewing the Member Dashboard” on page 17, depending on the role you
have for the selected tenant.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4 - Tenant Administration
-
Change Password: Opens the Change Password popup
window, which allows you to modify your EPIC password.
See “Changing Your Password” on page 47.
-
Logout: Logs you out of EPIC. Once you log out, you will
need to log back as described in “Launching and Logging In” on page 7 to access EPIC.
4.1.2 - Main Menu (6)
The Tenant Administrator main menu contains the following
options:
•
Dashboard: Opens the Dashboard screen. See “Viewing the
Tenant Admin Dashboard” on page 55.
•
Jobs: Opens the Job Management screen, which allows you
to view, add, edit, and delete jobs. You may also view various
job logs. See “Jobs” on page 18.
•
Clusters: Opens the Cluster Management screen, which
allows you to create, edit, and delete persistent clusters. See
“Persistent Clusters” on page 28.
•
•
DataTaps: Opens the DataTaps screen, which allows you to
add, edit, and remove DataTaps. See “DataTaps” on
page 56.
Virtual Nodes: Opens the Virtual Nodes screen, which
allows you to view the virtual nodes that the Site Administrator created for this tenant. See “Viewing Virtual Nodes” on
page 42.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
•
Users: Opens the <Tenant> screen, which allows you to
assign and revoke user access to the current tenant. See
“Users” on page 62.
4.1.3 - Menu Expand/Collapse (8)
Clicking the Expand/Collapse arrow at the bottom of the main
menu toggles the menu between wide (expanded) and narrow
(collapsed). This feature maximizes the amount of screen space
available for the work area on smaller monitors (such as mobile
devices).
4.1.4 - Work Area (8)
The work area is where EPIC screens appear. Various generic
functions will be available here, depending on the screen you
are accessing. These generic functions may include some or all
of the following:
•
Use the Display... records pull-down menu to
select how many records you want to see displayed on a single screen.
•
Clicking an up arrow in a table header collapses that table.
•
Clicking a down arrow in a table header
expands that table.
•
Clicking a checkbox in a table selects that
item. You may select one or more items and
then perform an action on the selected item(s).
53
EPIC User & Administrator Guide
•
Clicking the checkbox in a table header
selects all of the items in that table.
•
Clicking the arrows in a table column sorts
the table by the information in that column.
For example, clicking the arrows in the Tenant Description
column sorts the list of tenants by the description. Repeatedly clicking a column header toggles the display between
ascending (A-Z) and descending (Z-A) order.
•
Entering one or more keyword(s) in the
Search field and then pressing [ENTER]
returns all records containing the supplied keyword(s).
Searches happen in real time; the work area refreshes as you
type.
•
If a screen contains too many records to display on a single page, you may use the page
numbers and arrows to move between pages, as follows:
•
-
Clicking a page number opens the selected page of the
current screen.
-
Clicking the First Page (<<) button takes you to the first
page of the current screen.
-
Clicking the Last Page (>>) button takes you to the last
page of the current screen.
Clicking the Back to Top icon at the bottom of
a screen scrolls you back to the top of the
current screen.
54
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4 - Tenant Administration
4.2 - Viewing the Tenant Admin Dashboard
Clicking Dashboard in the main menu opens the Tenant Administrator Dashboard screen, which presents a high-level overview
of current activity within this tenant. This screen refreshes every
30 seconds.
The Dashboard screen displays the following information:
•
Jobs Charts: Displays the job setup and run times in seconds
for each of the jobs shown in the Statistics section.
•
Resource Usage: Displays graphs showing the following:
-
Cores Used: Displays the percentage of virtual CPU
cores used and the number of virtual CPU cores available
for use in EPIC virtual clusters.
-
Memory Used (GB): Displays the percentage of virtual
RAM used (in GB) and the amount of virtual GB of RAM
available for use in EPIC virtual clusters.
Figure 4.2: Tenant Administrator Dashboard screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
55
EPIC User & Administrator Guide
4.3 - DataTaps
Selecting DataTaps in the main menu opens the Manage DataTaps screen for the tenant.This screen contains the following buttons:
•
Create: Clicking this button opens the Create New DataTap
screen. See “Creating a New DataTap” on page 59.
•
Delete: Clicking this button deletes the selected DataTap(s)
from the tenant. Deleting a DataTap that is being used by a
currently-running job will cause file access errors within that
job. A popup warning appears asking you to confirm or cancel the action. Click OK to proceed, or Cancel to exit without
deleting the DataTap(s).
Note: Deleting a DataTap does not affect your data.
If you accidentally delete a DataTap, create a new
one that points to the same location.
The table on this screen contains the following information and
functions:
•
Name: Name of the DataTap. Clicking a name in this column
opens the <DataTap> Browser screen for the selected
DataTap. See “Uploading and Downloading Files” on
page 43.
•
Description: Brief description of the DataTap.
•
Host: DNS name or IP address of the service providing
access to the shared storage resource associated with the
DataTap. For example, the namenode of an HDFS cluster.
•
Type: Type of file system used by the storage resource (GlusterFS, HDFS, or NFS).
Figure 4.3: DataTaps screen
56
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4 - Tenant Administration
•
•
•
Additional Info: If appropriate, the NFS share or GlusterFS
volume used on the storage resource. This column will also
display Kerberos Protected for an HDFS DataTap with Kerberos protection enabled.
Path: Location of the root directory of the DataTap within the
specified NFS share, GlusterFS volume, or HDFS file system.
This field is blank if the DataTap points to the root of the specified share/volume/file system.
Actions: The following actions are available for each
DataTap:
-
-
Edit: Clicking the blue Edit icon (pencil) in the Actions column opens the Edit DataTap screen. Editing a DataTap
that is being used by a currently-running job may cause
file access errors within that job. See “Editing a DataTap”
on page 57.
Delete: Clicking the red Delete icon (trash can) in the
Actions column deletes the DataTap from the tenant.
Deleting a DataTap that is being used by a currently-running job may cause file access errors within that job. A
popup warning appears asking you to confirm or cancel
the action. Click OK to proceed, or Cancel to exit without
deleting the DataTap. Please see the note on this page.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4.3.1 - Editing a DataTap
Clicking the Edit icon (blue pencil) in the Actions column of the
Manage DataTaps screen opens the Edit DataTap screen for the
selected DataTap.
Note: You cannot edit a DataTap if EPIC is in Lockdown mode. See “Lockdown Mode” on page 122.
To edit a DataTap, you may modify some or all of the following:
•
Name: Rename the DataTap by entering a new name in the
Name field. This name may contain letters (A-Z or a-z), digits
(0-9) and punctuation characters (-_.!~*'()$,;:@&= +), but may
not contain spaces.
•
Description: Update the description of the DataTap by providing a new description in the Description field.
•
Select the storage device type (Gluster, HDFS, or NFS) using
the Select Type pull-down menu.
•
If you selected Gluster, then you may update the following
parameters:
-
Host: Enter the name or IP address of the host in the Host
field.
-
Port: Enter the host port number in the Port field. Leave
blank to use the default GlusterFS port.
-
Volume: Enter the name of the volume containing the
desired directory in the Volume field.
57
EPIC User & Administrator Guide
-
•
Path: Enter the directory under the volume to use for the
DataTap in the Path field. You may also click the Browse
button to open an explorer window to navigate to the
desired directory. You can leave this field blank if you
intend the DataTap to point the root of the specified volume.
-
Port: Enter the namenode port number in the Port field.
Leave blank to use the default HDFS namenode port.
-
Path: Enter the HDFS directory under the share to use for
the DataTap in the Path field. You may also click the
Browse button to open an explorer window to navigate to
the desired directory. You can leave this field blank if you
intend the DataTap to point the root of the specified file
system.
-
Kerberos Protection: You can enable or disable Kerberos
protection for the selected DataTap by checking or clearing the Kerberos Protected checkbox, as appropriate.
See “Kerberos Security” on page 111.
If you selected HDFS, then you may update the following
parameters:
-
Host: Enter either the hostname or IP address of the
HDFS namenode in the Host field.
-
Standby NameNode Host: Enter the hostname or IP
address of the HDFS standby namenode, if any, in the
Standby NameNode Host field.
58
Figure 4.4: Edit DataTap screen (below)
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4 - Tenant Administration
•
If you selected NFS, then you may update the following
parameters:
-
Host: Enter either the hostname or IP address of the file
system host in the Host field.
-
Share: Enter the name of the share in the Share field.
-
Path: Enter the directory under the share to use for the
DataTap in the Path field. You may also click the Browse
button to open an explorer window to navigate to the
desired directory. You can leave this field blank if you
intend the DataTap to point the root of the specified
share.
-
When you have finished modifying the parameters for the
DataTap, click Submit to modify that DataTap or Reset to
clear changes without editing the DataTap.
4.3.2 - Creating a New DataTap
Clicking the green Create button in the DataTaps screen opens
the Create New DataTap screen.
Note: You cannot create a new DataTap if EPIC is in
Lockdown mode. See “Lockdown Mode” on
page 122.
DataTaps are created on a per-tenant basis. This means that a
DataTap created in Tenant A is not available to Tenant B. You can,
however, create two DataTaps (one for each tenant) that point to
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
the identical shared directory. This will allow both tenants to use
the same data.
CAUTION: CREATING MULTIPLE DATATAPS TO
THE SAME DIRECTORY CAN LEAD TO CONFLICTS
AND POTENTIAL DATA LOSS.
To create a DataTap:
1. Enter a name for the DataTap in the Name field. This name
may contain letters (A-Z or a-z), digits (0-9) and punctuation
characters (-_.!~*'()$,;:@&=+), but may not contain spaces.
2. Enter a brief description for the DataTap in the Description
field.
3. Select the file system type using the Select Type pull-down
menu. The available options are:
-
Gluster: If you select this option, proceed to Step 4.
-
HDFS: If you select this option, then skip to Step 5.
-
NFS: If you select this option, then skip to Step 6.
4. If you selected Gluster in Step 3, then enter the following
parameters:
-
Host: Enter the name or IP address of the host in the Host
field.
-
Port: Enter the host port number in the Port field. Leave
blank to use the default GlusterFS port.
-
Volume: Enter the name of the volume containing the
desired directory in the Volume field.
59
EPIC User & Administrator Guide
-
Path: Enter the directory under the volume to use for the
DataTap in the Path field. If Kerberos protection is not
enabled, then you may also click the Browse button to
open an explorer window to navigate to the desired
directory. You can leave this field blank if you intend the
DataTap to point the root of the specified volume.
Skip to Step 7 after entering the Gluster parameters.
5. If you selected HDFS in Step 3, then enter the following
parameters:
-
Host: Enter either the hostname or IP address of the
HDFS namenode in the Host field.
-
Standby NameNode Host: Enter the hostname or IP
address of the HDFS standby namenode, if any, in the
Standby NameNode Host field.
-
Port: Enter the namenode port number in the Port field.
Leave blank to use the default HDFS namenode port.
-
Path: Enter the HDFS directory under the share to use for
the DataTap in the Path field. You may also click the
Browse button to open an explorer window to navigate to
the desired directory. You can leave this field blank if you
intend the DataTap to point the root of the specified file
system.
Figure 4.5: Create New DataTap screen (below)
60
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4 - Tenant Administration
-
Kerberos Protection: You can enable or disable Kerberos
protection for the selected DataTap by checking or clearing the Kerberos Protected checkbox, as appropriate.
See “Kerberos Security” on page 111.
Skip to Step 7 after entering the HDFS parameters.
6. If you selected NFS in Step 3, then you enter the following
parameters:
-
Host: Enter either the hostname or IP address of the file
system host in the Host field.
-
Share: Enter the name of the share in the Share field.
-
Path: Enter the directory under the share to use for the
DataTap in the Path field. You may also click the Browse
button to open an explorer window to navigate to the
desired directory. You can leave this field blank if you
intend the DataTap to point the root of the specified
share.
7. Review the entries you made in Steps 1-6 to make sure they
are accurate.
When you have finished modifying the parameters for the
DataTap, click Submit to create the new DataTap.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
61
EPIC User & Administrator Guide
4.4 - Users
Selecting Users in the main menu opens the <Tenant> screen for
that tenant (where <Tenant> is the name of the tenant).
The table on this screen contains the following information and
functions:
This screen contains the following buttons:
•
Login Name: Login name of the user.
•
Full Name: Full name of the user.
•
Role: Role of the user within the tenant. A user may have one
role per tenant.
•
Revoke: Clicking the red Revoke icon (minus sign) in the
Actions column revokes the selected user’s access to the
tenant. A popup warning appears asking you to confirm or
cancel the action. Click OK to proceed, or Cancel to exit without revoking the user’s role for the tenant. Please see the
note on the left.
•
•
Assign: Clicking this button opens the Assign Users screen.
See “Assigning Users to a Tenant” on page 63.
Revoke: Clicking this button revokes the selected user’s
access to the tenant. A popup warning appears asking you to
confirm or cancel the action. Click OK to proceed, or Cancel
to exit without revoking the user’s role for the tenant.
Note: If you revoke a user by mistake, you can reassign them to the tenant using the Assign Users
screen. See “Assigning Users to a Tenant” on
page 63.
Figure 4.6: <Tenant> screen
62
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
4 - Tenant Administration
4.4.1 - Assigning Users to a Tenant
Clicking the Assign button in the <Tenant> screen or selecting
Assign Users in the Quick Access button menu opens the
Assign Users screen, which allows you to assign, change, ore
revoke user access to the current tenant.
To assign a user to the current tenant or change the user’s role
within the current tenant (such as from Member to Tenant Administrator or vice versa):
1. On the left side of the screen, select the user you want to
assign by selecting that user in the USERS list. You may also
start typing the username into the Search field, and the list of
users will update in real time based on your entry.
-
An A icon appears by each user who has the Tenant
Administrator role assigned to them for the current
tenant. A tenant may have multiple administrators. You
may either downgrade the role of that user to Tenant
Member or remove access to this tenant altogether.
-
An M icon appears by each user who has the Tenant
Member role assigned to them for the current tenant. A
tenant may have multiple members. You may either
upgrade the role of that user to Tenant Administrator or
remove access to this tenant altogether.
-
Users who do not have any role in the current tenant may
be granted either the Tenant Member or Tenant Administrator role.
2. Selecting a user enables the <user> section on the right side
of the screen, where <user> is the username you selected.
The name of the tenant to which you are assigning the user
also appears below the username.
3. Check the appropriate radio button to assign a role to the
selected user. The available options are:
Figure 4.7: Assign Users screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
63
EPIC User & Administrator Guide
-
Member: Makes the user a non-administrative member of
the selected tenant.
-
Admin: Makes the user a Tenant Administrator of the current tenant.
Note: A user may have one role per tenant.
Note: Please see “Assigning Users to a Tenant” on
page 82 for instructions on assigning the Site
Administrator role to a user.
4. Click Save to save your changes. A confirmation dialog
appears. Click OK to return to the <Tenant> screen.
Note: This function does not store user passwords.
The built-in user database or your existing external
authentication server will handle user passwords.
If the selected user already has Member or Tenant Administrator
access to the current tenant, you will see a Remove from this
Tenant button at the bottom right of the <Tenant> screen. Clicking this button revokes the user’s role and prevents them from
being able to access the current tenant. A confirmation dialog
appears; click OK to proceed with the revocation or Cancel to
cancel.
64
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
65
EPIC User & Administrator Guide
This chapter describes the EPIC Site Administrator interface. A
site administrator can perform the following functions:
•
Login to EPIC: See “Launching and Logging In” on page 7.
•
View the Site Administrator Dashboard screen: See “Viewing the Site Admin Dashboard” on page 71.
•
Add, edit, and remove tenants: See “Tenants” on page 77.
•
Add, edit, and delete users and sessions: See “Managing
Users and Sessions” on page 84.
•
View physical node status: See “Viewing Physical Nodes”
on page 88.
•
View virtual node status: See “Viewing Virtual Nodes” on
page 42.
•
Manage the EPIC Installation: See “Viewing Physical
Nodes” on page 88.
•
Modify the EPIC Catalog: See “EPIC Catalog” on page 103.
•
Modify EPIC settings: See “EPIC Catalog” on page 103.
•
Generate, download, and delete SOS reports: See “EPIC
Catalog” on page 103.
66
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.1 - The Site Administrator Interface
The Site Administrator interface contains the following elements:
Figure 5.1: The Site Administrator interface
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
67
EPIC User & Administrator Guide
The following sections describe the numbered callouts in detail.
•
User Guide (3): Clicking this link opens this User & Administrator Guide in Adobe Acrobat (PDF) format.
5.1.1 - Toolbar (1-5)
•
Quick Access button (4): Clicking this button opens a pulldown menu with the following options:
The Toolbar contains the following information/functions from
left to right:
•
Quick Status (1): This area of the Toolbar displays (from left
to right) the number of tenants, users, physical nodes, and
virtual nodes, as follows:
Create Tenant: Opens the Create New Tenant screen,
which allows you to create a new EPIC tenant (see “Creating a New Tenant” on page 79).
-
Create User: Opens the Create New User screen, which
allows you to add a new user to the local EPIC user database (see “EPIC Catalog” on page 103).
-
Tenants: Clicking this icon opens the Tenant Management screen. See “Tenants” on page 77.
-
-
Users: Clicking this icon opens the User Management
screen. See “Managing Users and Sessions” on
page 84.
Assign Users: Opens the Assign Users screen, which
allows you to grant roles to EPIC users (see “Assigning
Users to a Tenant” on page 82).
-
-
Physical Nodes: Clicking this icon opens the Physical
Nodes screen. See “Viewing Physical Nodes” on
page 88.
Enter/Exit site lockdown: When enabled, Lockdown
mode prevents users from making any changes to the
EPIC platform (see “Lockdown Mode” on page 122).
-
Virtual Nodes: Clicking this icon opens the Virtual Nodes
screen. See “Viewing Virtual Nodes” on page 42.
The currently selected icon appears highlighted in green.
•
-
Login details (2): This area displays the following information:
-
Tenant you are currently viewing
-
Your username
-
Your role
68
•
User button (5): Clicking this button opens a pull-down menu
with the following options:
-
Tenant: List of tenants that you have access to, based on
your EPIC login credentials. Clicking a tenant opens
either the Tenant Administrator Dashboard screen (see
“The Tenant Member Interface” on page 13) or the
Member Dashboard screen (see “Viewing the Member
Dashboard” on page 17, depending on the role you
have for the selected tenant. The Site Administrator interface opens when you return to the default SIte Admin
tenant.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
-
Change Password: Opens the Change Password popup
window, which allows you to modify your EPIC password.
See “Changing Your Password” on page 47.
•
Installation: Opens the BlueData Cluster Installation screen,
which allows you manage physical Worker nodes. See “Managing the EPIC Installation” on page 92.
-
Logout: Logs you out of EPIC. Once you log out, you will
need to log back as described in “Launching and Logging In” on page 7 to access EPIC.
•
Catalog: Opens the Catalog screen, which allows you to add
Hadoop distribution images and work with node flavor settings. See “EPIC Catalog” on page 103.
•
Settings: Opens the System Settings screen, which allows
you to modify various EPIC settings. See “EPIC Catalog” on
page 103.
•
Support: Opens the Support/Troubleshooting screen, which
allows you to generate, download, and delete SOS reports.
See “EPIC Catalog” on page 103.
5.1.2 - Main Menu (6)
The Site Administrator main menu contains the following options:
•
Dashboard: Opens the Dashboard screen. See “Viewing the
Site Admin Dashboard” on page 71.
•
Tenants: Opens the Tenant Managemenrt screen, which
allows you to add, modify, and delete EPIC tenants. See
“Tenants” on page 77.
•
Users: Opens the User Management screen, which allows
you to view, create, assign, and delete users and sessions.
See “Managing Users and Sessions” on page 84.
•
Physical Nodes: Opens the Physical Nodes screen, which
allows you to view statuses and statistics for the physical
nodes in your EPIC installation. See “Viewing Physical
Nodes” on page 88.
•
Virtual Nodes: Opens the Virtual Nodes screen, which
allows you to view the virtual nodes that have been created
for all tenants and the host(s) those nodes are running on.
See “Viewing Virtual Nodes” on page 42.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5.1.3 - Menu Expand/Collapse (7)
Clicking the Expand/Collapse arrow at the bottom of the main
menu toggles the menu between wide (expanded) and narrow
(collapsed). This feature maximizes the amount of screen space
available for the work area on smaller monitors (such as mobile
devices).
5.1.4 - Work Area (8)
The work area is where EPIC screens appear. Various generic
functions will be available here, depending on the screen you
are accessing. These generic functions may include some or all
of the following:
69
EPIC User & Administrator Guide
•
Use the Display... records pull-down menu to
select how many records you want to see displayed on a single screen.
•
Clicking an up arrow in a table header collapses that table.
•
Clicking a down arrow in a table header
expands that table.
•
Clicking a checkbox in a table selects that
item. You may select one or more items and
then perform an action on the selected item(s).
•
Clicking the checkbox in a table header
selects all of the items in that table.
•
Clicking the arrows in a table column sorts
the table by the information in that column.
For example, clicking the arrows in the Tenant Description
column sorts the list of tenants by the description. Repeatedly clicking a column header toggles the display between
ascending (A-Z) and descending (Z-A) order.
•
Entering one or more keyword(s) in the
Search field and then pressing [ENTER]
returns all records containing the supplied keyword(s).
Searches happen in real time; the work area refreshes as you
type.
•
If a screen contains too many records to display on a single page, you may use the page
numbers and arrows to move between pages, as follows:
70
•
-
Clicking a page number opens the selected page of the
current screen.
-
Clicking the First Page (<<) button takes you to the first
page of the current screen.
-
Clicking the Last Page (>>) button takes you to the last
page of the current screen.
Clicking the Back to Top icon at the bottom of a screen
scrolls you back to the top of the current screen.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.2 - Viewing the Site Admin Dashboard
Clicking Dashboard in the main menu opens the Site Administrator Dashboard screen, which presents a high-level overview of
current EPIC activity. This screen refreshes every 30 seconds.
The following tabs are available:
•
Current Load: This tab displays load statistics for the CPU,
memory, and network resources within the EPIC installation.
See “Current Load Tab” on page 71.
•
Resource Usage: This tab displays usage information on a
per-tenant basis. See “Resource Usage Tab” on page 72.
•
Service Status: This section displays the health status for
each component service within the EPIC platform for each
physical node. See “Service Status Tab” on page 73.
•
•
CPU%: The dial indicates the current percentage of physical
CPU utilization across all cluster processes that are currently
running. Clicking the CPU sub-tab beneath the dials opens
an accompanying chart that displays the average CPU percentage over the selected time period.
•
Memory (GB): The dial indicates the current use of physical
memory across all cluster processes. Clicking the Memory
sub-tab beneath the dials opens an accompanying chart that
displays the memory usage over the selected time period.
•
Network Mbps: The dial indicates the current amount of
physical network bandwidth being used by the cluster. Clicking the Network sub-tab beneath the dials opens an accompanying chart that displays network bandwidth usage over
the selected time period.
Alerts: This tab displays any alert messages generated by
the system. See “Alerts Tab” on page 76.
5.2.1 - Current Load Tab
The Current Load tab displays a series of dials and charts that
show the following information for the selected time period:
•
dials opens an accompanying chart that displays the average
load over the selected time period.
Load: The dial shows the current average percentage of
physical CPU cores used by the EPIC platform (defined as the
number of CPU cores in use vs. the total number of available
CPU cores). Clicking the Load Average sub-tab beneath the
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
You may adjust the time period for which results appear using
the green pull-down menu at the right side of the Current Load
tab. The available options are:
•
Last Hour (default)
•
2 Hours
•
4 Hours
•
Day
71
EPIC User & Administrator Guide
•
Week
•
Month
•
Year
•
Memory Used (GB): Percentage of available RAM being
used by all of the tenants in the EPIC platform.
•
Usage by Tenants: Relative amount of available resources
used by each tenant in the EPIC platform.
5.2.2 - Resource Usage Tab
The bottom of this tab contains a table that displays the following
information for each tenant:
The top of the Resource Usage tab displays dials showing the
following information:
•
Tenant Name: Name of the tenant.
•
Users: Number of EPIC users with permission to access the
tenant (does not include transient clusters).
•
Cores Used: Percentage of available CPU cores being used
by all of the tenants in the EPIC platform.
Figure 5.2: Dashboard screen - Current Load tab
72
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
•
Cores: Number of virtual CPU cores allocated to that tenant.
This is blank if the tenant has no specified virtual CPU quota.
•
VM Count: Number of EPIC virtual nodes running on that
physical node.
•
Memory (GB): Amount of RAM allocated to that tenant in GB.
This is blank if the tenant has no specified quota of RAM.
•
BlueData: The BlueData group in this tab displays the following information:
•
Clusters: Number of persistent virtual clusters assigned to
the tenant.
•
Active Jobs: Number of jobs the tenant is processing.
5.2.3 - Service Status Tab
-
Caching Node: Status of the EPIC data service, which
communicates with the storage services referenced by
DataTaps. This service provides an accelerated I/O channel between those storage services and the applications
running in virtual nodes. On the Controller node, this service also provides the back end for DataTap browsing.
The Service Status tab displays the following information for
each physical node in the EPIC platform:
•
Name: Name of the physical node.
Figure 5.3: Dashboard screen - Resource Usage tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
73
EPIC User & Administrator Guide
•
-
Data Server: Status of the EPIC data service agent, which
acts as an intermediary between the file system and other
entities. This service establishes communication between
a host's virtual nodes and the Caching Node service. On
the Controller node, the Data Server also receives
DataTap browsing queries from the Management Service.
-
Management: Status of the EPIC management service,
which handles back-end administration tasks.
appears as yellow if one of these three nodes has failed
to indicate that the High Availability cluster has been
degraded and that the EPIC platform is not protected
against any further node failure. If the dot is red, then
High Availability protection is not currently functional.
Note: Only one node (either the Controller or
Shadow Controller) functions as the Controller at
any given time.
HA: This group only appears if High Availability is enabled for
the EPIC platform as described in the About EPIC Guide,
“Installing Worker Nodes” on page 94, and “HA Settings
Tab” on page 116. When enabled, this group displays the
following information:
-
-
-
DRBD: Health of the shared storage service health. This
services provides High Availability protection for the EPIC
cluster metadata and must be running on both the Controller and Shadow Controller nodes.
HA Engine: Health of the High Availability Engine, which
is the central High Availability state transitions executing
unit. The HA Engine performs various tasks specific to
High Availability in response to requests from other services and must be running on both the Controller and
Shadow Controller nodes.
HA Status: Status of the High Availability service. The
node for which this service appears is functioning as the
Controller node. If this dot is green, then the High Availability node is functioning normally and all nodes (Controller, Shadow Controller, and Arbiter) are up. This dot
74
-
Pacemaker: Health of the High Availability cluster polling
service. This service periodically polls the High Availability cluster and, in the event of a failure, triggers failover
state transition in the HA Engine. This service must be
running on both the Controller and Shadow Controller
nodes.
-
Cluster Management: Health of the cluster manager daemon that helps the Pacemaker service perform its periodic polling and trigger failover/failback. This daemon
must be running on both the Controller and Shadow Controller nodes.
•
HDFS: This group only appears if a local HDFS service was
established for use as system storage when you installed
EPIC. Describing the individual items in this group is beyond
the scope of this manual.
•
OpenStack: Describing the individual items in this group is
beyond the scope of this manual.
•
Actions: The Actions column of the table includes a Check
Now icon (gears) for each physical node. Clicking this icon
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
refreshes the status of all listed services for the selected
node.
•
A Worker node with some red and some green dots may
cause some EPIC operations to fail, unless the errors are
transient conditions caused by the host powering on or
regaining network connectivity.
The status of a service can be either OK (green dot), CRITICAL
(red dot), or DISABLED (intentionally not running; gray dot). Hovering the mouse over the status button opens a popup with additional information. In general:
Please contact BlueData Technical Support if a host that is
reporting service errors meets all of the following criteria:
•
•
The EPIC software is completely installed.
•
The host is powered on.
•
The host has network connectivity.
•
The Controller must not display any red dots. If the Controller
has one or more error(s), then the EPIC platform may not
function properly.
If all of the dots for a Worker node are red, then that host will
not be able to provide resources to the EPIC platform. This
situation typically occurs because the host has been powered off, has lost network connectivity, or because the EPIC
software is not properly installed.
See “Troubleshooting” on page A-1 for more information about
resolving errors.
Figure 5.4: Dashboard screen - Service Status tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
75
EPIC User & Administrator Guide
5.2.4 - Alerts Tab
The Alerts tab displays any alert messages from the Caching
Node, Data Server, and Management services. An alert is triggered if one or more these services goes into a critical state.
Figure 5.5: Dashboard screen - Alerts tab
76
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.3 - Tenants
Selecting Tenants in the main menu opens the Tenant Management screen.
Click OK to proceed, or Cancel to exit without deleting the
tenant(s). See “Deleting a Tenant” on page 83.
This screen contains the following buttons:
CAUTION: YOU CANNOT UNDELETE A TENANT.
•
Create: Clicking this button opens the Create New Tenant
screen. See “Creating a New Tenant” on page 79.
•
Delete: Clicking this button deletes the selected tenant(s),
except that you cannot delete the Site Administrator tenant
that EPIC creates during the installation process. A popup
warning appears asking you to confirm or cancel the action.
The table on this screen contains the following information and
functions:
•
Tenant Name: Name of the tenant. Clicking a tenant name
opens the <Tenant> screen for that tenant, which allows you
assign and revoke user roles. See “Assigning Users to a
Tenant” on page 82.
Figure 5.6: Tenant Management screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
77
EPIC User & Administrator Guide
•
Tenant Description: Brief description of the tenant.
•
Tenant Quota: Number of virtual CPU cores and amount of
virtual RAM assigned to the tenant, compared to the total
amount of virtual CPU cores and RAM available to that tenant
(such as Cores=8/16). If the tenant has no quota assigned
to it, then the number of virtual CPU cores and RAM being
used by that tenant will appear with the resources being
used (such as Cores=8) and the message (No Quota).
•
Actions: The following actions are available for each tenant:
-
Edit: Clicking the blue Edit icon (pencil) in the Actions column opens the Edit Tenant Screen for the tenant. See
“Assigning Users to a Tenant” on page 82. You cannot
edit the default Site Admin tenant.
-
Tenant KeyPair: Clicking the gray Download icon (down
arrow in a circle) opens an OS-default Download window
that allows you to retrieve the tenant SSL certificate to
allow SSH connections to the virtual node(s) of a tenant.
Certificates are provided in .pem format. You cannot connect to the default Site Admin tenant via SSH.
-
Details: Clicking the green Details icon (letter “i”) opens
the <Tenant> screen for that tenant, which allows you to
assign and revoke user roles. See “Assigning Users to a
Tenant” on page 82.
5.3.1 - Editing an Existing Tenant
In the Tenant Management screen, clicking the blue Edit icon
(pencil) for a tenant opens the Edit Tenant screen for that tenant.
Figure 5.7: Edit Tenant screen
78
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
This screen contains the following fields:
•
Tenant Name: Enter a new name for the selected tenant in
this field.
•
Tenant Description: Enter a brief description of the tenant in
this field.
•
Maximum Cores: Enter the maximum number of virtual CPU
cores that EPIC should make available for use by this tenant.
The total number of virtual CPU cores available to all tenants
is equal to four times the number of physical CPU cores. For
example, if your physical nodes have a total of 32 physical
CPU cores, EPIC will report 128 total available virtual CPU
cores. Please see the About EPIC Guide for information
about how EPIC uses vCPU cores. Leave this field blank if
you do not want to specify a quota.
6. Maximum Memory (GB): Enter the maximum amount of virtual RAM in GB that EPIC should make available for use by
this tenant. The total amount of virtual RAM available to all
tenants is equal to the number of physical GB of RAM on the
EPIC platform. For example, if your physical nodes have a
total of 512GB of RAM, EPIC will report 512GB of total available RAM. Please see the About EPIC Guide for more information about how EPIC uses virtual RAM. Leave this field
blank if you do not want to specify a quota.
When you have finished editing the tenant, click Submit to save
your changes.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5.3.2 - Creating a New Tenant
Clicking the green Create button in the Tenant Management
screen opens the Create New Tenant screen.
To create a new tenant:
1. Enter a name for the new tenant in the Tenant Name field.
2. Enter a brief description for the new tenant in the Tenant
Description field.
3. Maximum Cores: Enter the maximum number of virtual CPU
cores that EPIC should make available for use by this tenant.
By default, this field will display a value equal to 25% of the
virtual CPU cores in the EPIC platform, but you can specify
any number you want. Please see the About EPIC Guide for
important information about how EPIC uses virtual CPU
cores. Leave this field blank if you do not want to specify a
quota.
4. Maximum Memory (GB): Enter the maximum amount of virtual RAM in GB that EPIC should make available for use by
this tenant. The total amount of virtual RAM available to all
tenants is equal to the number of physical GB of RAM on the
EPIC platform. For example, if your physical nodes have a
total of 512GB of RAM, EPIC will report 512GB of total available RAM. Leave this field blank if you do not want to specify
a quota.
By default, this field will display a value equal to 25% of the
available memory in the EPIC platform, but you can specify
any number you want. Please see the About EPIC Guide for
more information about how EPIC uses virtual RAM.
79
EPIC User & Administrator Guide
5. When you have finished creating the tenant, click Submit to
create the new tenant.
EPIC automatically creates a default DataTap for the new
tenant. See the About EPIC Guide.
6. After creating a tenant, you must assign at least one user to
the tenant with the Tenant Administrator role. See “Assigning
Users to a Tenant” on page 82.
The Tenant Administrator for the newly created tenant may now
add resources (virtual clusters and DataTaps) to the tenant. See
“Tenant Administration” on page 49.
Figure 5.8: Create New Tenant screen
80
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.3.3 - Viewing Users Assigned to a
Tenant
confirm or cancel the action. Click OK to proceed, or Cancel
to exit without revoking the user’s role for the tenant.
Note: If you revoke a user by mistake, you can reassign them to the tenant using the Assign Users
screen. See “Assigning Users to a Tenant” on
page 82.
Clicking the name of a tenant or the green Details icon (letter “i”)
in the Tenant Management screen opens the <Tenant> screen
for that tenant (where <Tenant> is the name of the tenant).
This screen contains the following buttons:
•
•
Assign: Clicking this button opens the Assign Users screen.
See “Assigning Users to a Tenant” on page 82.
Revoke: Clicking this button revokes the selected user’s
access to the tenant. A popup warning appears asking you to
The table on this screen contains the following information and
functions:
•
Login Name: Login name of the user.
•
Full Name: Full name of the user.
•
Role: Role of the user within the tenant. A user may have one
role per tenant.
Figure 5.9: <Tenant> screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
81
EPIC User & Administrator Guide
•
Revoke: Clicking the red Revoke icon (minus sign) in the
Actions column revokes the selected user’s access to the
tenant. A popup warning appears asking you to confirm or
cancel the action. Click OK to proceed, or Cancel to exit without revoking the user’s role for the tenant.
1. On the left side of the screen, select the user you want to
assign by selecting that user in the USERS list. You may also
start typing the username into the Search field, and the list of
users will update in real time based on your entry.
-
The number of tenants each user has access to appears
as a numbered icon in each user listing. A user with no
roles will not have an icon in their listing.
-
A star icon indicates that the user is a Site Administrator.
Clicking this user expands the listing and displays a
checked Site Admin box. You may remove the user’s Site
Administrator privileges by clearing the checkbox and
then clicking the Save button.
-
You may make any user a Site Administrator by selecting
that user, checking the Site Admin checkbox, and then
clicking the Save button.
5.3.4 - Assigning Users to a Tenant
Clicking the Assign button in the <Tenant> screen, or selecting
Assign Users in the Quick Access button menu opens the
Assign Users screen, which allows you to assign, change, or
revoke user access to any existing tenant within the site.
To assign a user to a tenant, create or remove a Site Administrator, or change the user’s role within the current tenant (such as
from Member to Tenant Administrator or vice-versa):
Figure 5.10: Assign Users screen
82
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
-
Users who do not have any role in the current tenant may
be granted either the Tenant Member or Tenant Administrator role.
2. In the middle of the screen, select the tenant to which you
want to add/upgrade/downgrade/remove the user in the
TENANTS list. You may also start typing the tenant into the
Search field, and the list of users will update in real time
based on your entry.
If the selected user already has Member or Tenant Administrator
access to the current tenant, you will see a Remove from this
Tenant button at the bottom right of the <Tenant> screen. Clicking this button revokes the user’s role and prevents them from
being able to access the current tenant. A confirmation dialog
appears; click OK to proceed or Cancel to cancel.
5.3.5 - Deleting a Tenant
3. Selecting both a user and a tenant enables the <user> section on the right side of the screen, where <user> is the username you selected. The name of the tenant to which you are
assigning the user also appears below the username.
You can only delete a tenant if that tenant does not have any
resources ( jobs, clusters, users, or DataTaps) assigned to it. To
delete a tenant:
4. Check the appropriate radio button to assign a role to the
selected user. The available options are:
1. Assign yourself as the Tenant Administrator for the tenant
you are going to delete, using the Assign Users screen (see
“Assigning Users to a Tenant” on page 82).
-
Member: Makes the user a non-administrative member of
the selected tenant.
-
Admin: Makes the user a Tenant Administrator of the current tenant.
Note: A user may have one role per tenant.
5. Click Save to save your changes. A confirmation dialog
appears. Click OK to return to the <Tenant> screen.
Note: This function does not store user passwords.
The built-in user database or your existing external
authentication server will handle user passwords.
2. Switch to the new tenant, and then delete all of the resources
assigning to that tenant. Some resources, such as jobs and
clusters, will take some time to be removed completely.
3. Switch to the Site Admin tenant and then open the Tenants
screen by selecting Tenants in the main menu.
4. Click the name of the tenant you wish to delete in the Tenant
Name column to open the <Tenant> screen for that tenant
and then revoke all users for that tenant.
5. Return to the Tenants screen to select the tenant(s) you want
to delete, and then click the Delete button.
A confirmation dialog appears.
6. Click OK to confirm the deletion.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
83
EPIC User & Administrator Guide
5.4 - Managing Users and Sessions
Selecting Users in the main menu opens the User Management
screen.
The Users section of this screen (upper) contains the following
information/functions for each user:
•
Clicking the green Create button at the top of the Users section of this screen opens the Create New User screen. See
“Creating a New User” on page 87.
•
Login Name: Login name of the user. Clicking a user name
opens the User Details screen for the selected user. See
“Viewing User Details” on page 85.
•
Clicking the red Delete button at the top of the Sessions section of this screen deletes the selected session(s). The
affected user(s) will have to log back in to EPIC with their
username and password. Jobs, data, etc. are preserved.
•
Full Name: Full name of the user.
•
Assigned Tenants: Number of tenants in which the user has
a role.
Figure 5.11: User Management screen
84
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
•
•
Authentication Type: Type of authentication used when the
user logs in. This will be either Internal (if you are using the
internal EPIC user database to handle user authentication) or
will specify the type of external authorization being used
(LDAP or Active Directory).
The Sessions section of this screen (lower) contains the following information/functions for every active session:
•
User: Name of the user running the session.
•
Tenant: Tenant the user is using for this session. A user with
access to multiple tenants may run more than one session.
•
Role: Role of the user (Site Admin, Admin, or Member)
•
Expiry: Date and time the current session will expire if the
user takes no actions. EPIC allows up to 24 hours of inactivity
per session, to allow running jobs to complete.
•
Delete: Clicking the red Delete icon (trash can) in the Actions
column deletes the selected session. A popup warning
appears asking you to confirm or cancel the action. Click OK
to proceed, or Cancel to exit without deleting the session(s).
Actions: The following actions are available for each user:
-
-
Details: Clicking the green Details icon (letter i) in the
Actions column opens the User Details screen for that
user. See “Viewing User Details” on page 85.
Delete: Clicking the red Delete icon (trash can) in the
Actions column deletes the selected user. A popup warning appears asking you to confirm or cancel the action.
Click OK to proceed, or Cancel to exit without deleting
the user.
CAUTION: YOU CANNOT UNDELETE A USER.
DELETING A USER REMOVES ALL EPIC ROLES.
Note: Deleting a user only removes them from the
user database. If you are using an external authentication server, then you will need to remove or disable the user’s account on the authentication server.
-
Reset Password: Clicking the gray Reset Password icon
(key) for a user opens the Reset User Password popup
for the selected user. Enter and confirm the new password in the New Password and Confirm Password fields,
and then click Submit to save your changes and close
the popup.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5.4.1 - Viewing User Details
Clicking a user name in the Users table on the User Management screen or clicking the green Details icon for a user opens
the <User> screen, where <User> is the username of the
selected user.
This screen contains the following buttons:
•
Assign: Clicking this button opens the Assign Users screen.
See “Assigning Users to a Tenant” on page 82.
•
Revoke: Clicking this button revokes the selected user(s)
access to the tenant. A popup warning appears asking you to
85
EPIC User & Administrator Guide
confirm or cancel the action. Click OK to proceed, or Cancel
to exit without revoking the user’s role for the tenant.
•
Tenant Name: Name of each tenant the user is currently
assigned to. Each user may have one role per tenant.
Note: If you revoke a user by mistake, you can reassign them to the tenant using the Assign Users
screen. See “Assigning Users to a Tenant” on
page 82.
•
Tenant Description: Brief description of each tenant.
•
Role: Role the user has within that tenant.
•
Revoke: Clicking the red Revoke icon (minus sign) in the
Actions column revokes the selected user’s access to the
tenant. A popup warning appears asking you to confirm or
cancel the action. Click OK to proceed, or Cancel to exit without revoking the user’s role for the tenant. Please see the
note on the left.
The table on this screen contains the following information and
functions:
Figure 5.12: <User> screen
86
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.4.2 - Creating a New User
Clicking the green Create button in the User Management
screen opens the Create New User screen.
To create a new user:
1. Enter a unique user name in the Login Name field. This name
is case sensitive.
2. Enter the full name of the user in the Full Name field.
3. Enter a password in the Password field, and then reenter the
same password in the Confirm Password screen. Passwords
are case sensitive. The user may change her or his password
as described in “Changing Your Password” on page 47.
When you have finished entering the information for the new
user, click Submit to save your changes.
Note: You may only create a new user via this function if your EPIC platform is configured to use the
local user database. If you use an external authentication server to manage EPIC logins, then you need
to create the user account on that authentication
server.
Figure 5.13: Create New User screen
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
87
EPIC User & Administrator Guide
5.5 - Viewing Physical Nodes
Selecting Physical Nodes in the main menu opens the Physical
Nodes screen.
The top of this screen contains the following tabs:
•
Node(s) Info: This tab contains a table with basic information
about each physical node in the EPIC platform. See “Node(s)
Info Tab” on page 88.
•
Charts: This tab contains several charts that display performance information for all physical nodes or a selected node.
See “Charts Tab” on page 89.
•
DataTap Charts: This tab contains several charts that display
data throughput information for all physical nodes or a
selected node. See “DataTap Charts Tab” on page 91.
5.5.1 - Node(s) Info Tab
The Node(s) Info tab displays the following information for each
physical node in the EPIC platform:
Figure 5.14: Physical Nodes screen - Node(s) Info tab
88
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
•
Name: Host name of the physical node.
•
Role: Role of the node in the EPIC platform (Controller or
Worker). This can be one of the following:
-
Controller: Physical node that manages the other physical nodes while also serving as a Worker node in the EPIC
platform.
-
Worker: Physical node that is managed by a Controller.
Note: If you have High Availability enabled for the
EPIC platform, you may see which nodes have the
Shadow Controller and Arbiter roles assigned to
them using the HA Settings tab, which is described
in“HA Settings Tab” on page 116 .
•
VM Count: Number of virtual nodes running on the physical
node.
•
Memory (GB): This is a fraction consisting of the amount of
physical RAM being used by EPIC on the physical node over
the total amount of physical RAM available to EPIC on that
physical node.
•
VCPUs: This is a fraction consisting of the number of virtual
CPU cores being used by EPIC on the physical node over the
total number of virtual CPU cores available to EPIC on that
physical node. EPIC allows over-provisioning, which means
that the number of virtual cores may be larger than the number of physical cores.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5.5.2 - Charts Tab
The Charts tab contains the following pull-down menus:
•
Hosts (blue): The Hosts pull-down menu allows you to select
whether you want to view charts for a single physical node or
all physical node(s) in the EPIC platform. Select either the
desired physical node to view that node or All Host(s) to
view all nodes.
•
Time period (green): The Time Period pull-down menu
allows you to select the time period to review. The available
options are Last Hour (default), 2 Hours, 4 Hours, Day,
Week, Month, and Year.
A series of dials and charts appears to show the following information for the selected node(s) and time period:
•
Load Average: The dial shows the current average percentage of physical CPU cores used by EPIC (defined as the number of CPU cores in use vs. the total number of available CPU
cores), and the accompanying chart displays the historical
load over the selected time period.
•
CPU%: The dial indicates the current percentage of physical
CPU utilization across all EPIC processes that are currently
running, and the accompanying chart displays the historical
CPU percentage over the selected time period.
•
Memory Usage (GB): The dial indicates the current use of
physical memory across all EPIC processes, and the accompanying chart displays the historical memory usage over the
selected time period.
89
EPIC User & Administrator Guide
•
Network Usage: The dial indicates the current amount of
physical network bandwidth being used by EPIC, and the
accompanying chart displays historical network bandwidth
usage over the selected time period.
Figure 5.15: Physical Nodes screen - Charts tab
90
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.5.3 - DataTap Charts Tab
The DataTap Charts tab contains the following pull-down menus:
•
•
Hosts (blue): The Hosts pull-down menu allows you to select
whether you want to view charts for a single physical node or
all physical node(s) in the EPIC platform. Select either the
desired physical node to view that node or All Host(s) to
view all nodes.
options are Last Hour (default), 2 Hours, 4 Hours, Day,
Week, Month, and Year.
A series of charts appears to show the following information for
the selected node(s) and time period:
•
DataTap Read/Write Rate: These charts display the data
read and write rates, in megabytes per second.
•
DataTap Bytes Read/Written: Amount of bytes read/written.
Time period (green): The Time Period pull-down menu
allows you to select the time period to review. The available
Figure 5.16: Physical Nodes screen - DataTap Charts tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
91
EPIC User & Administrator Guide
5.6 - Managing the EPIC Installation
Selecting Installation in the main menu opens the BlueData
Cluster Installation screen.
•
The top of this screen contains the following tabs:
•
•
Installation: This tab lists all of the Worker nodes that have
been added to the EPIC platform. It also allows you to install
EPIC on one or more Worker node(s) and then add the
Worker(s) to your EPIC cluster. See “Installation Tab” on
page 92.
Available Upgrades: This tab allows you to search for EPIC
upgrades. See “Available Upgrades Tab” on page 99.
Upgrade History: This tab displays the upgrade history for
your EPIC installation. See “Upgrade History Tab” on
page 100.
5.6.1 - Installation Tab
The Installation tab lists the Worker node(s) in your EPIC platform and allows you to install additional nodes. This top of this
screen contains the following functions:
•
Worker IP address: Enter the IP address(es) for one or more
Worker node(s) in the Worker IP field.
Figure 5.17: BlueData Cluster Installation screen - Installation tab
92
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
•
Credentials: This is where you add either the root password
or SSH key in order to access the Worker node(s) being
added to the EPIC platform.
•
Actions: While you are installing a Worker node, the following
functions are available:
-
Edit: Clicking the blue Edit icon (pencil) opens the
Advanced Worker settings popup, which allows you to
specify one or more hard drive(s) to add to the local HDFS
system storage. See “Step 3: Selecting Hard Drives” on
page 97.
-
Delete: Clicking the red Delete icon (trash can) removes
the selected Worker node from the EPIC platform before
finalizing the installation. You may also delete multiple
nodes at once by selecting them and then clicking the
red Delete button.
The lower portion of this screen contains the following tabs:
•
Worker(s) Status tab: Shows the status of the Worker node(s)
being added to the EPIC platform, if any. See “Worker(s) Status Tab” on page 93.
•
Installed Workers tab: Lists all of the Worker node(s) that
have already been added to the EPIC platform. See
“Installed Workers Tab” on page 93.
5.6.1.1 - Worker(s) Status Tab
The Worker(s) Status tab lists the following information for each
Worker that is in the process of being added to the EPIC platform:
•
IP Address: IP address of the Worker node.
•
Status: Status of the installation.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5.6.1.2 - Installed Workers Tab
The Installed Workers tab lists the IP address and (if available)
host name of each Worker node in the EPIC platform.
Figure 5.18: Installation screen - Worker(s) Status tab (below)
93
EPIC User & Administrator Guide
5.6.2 - Installing Worker Nodes
Installing Worker node(s) uses the following basic process:
1. Select the node(s) to install as Worker(s).
2. Install EPIC on the node(s).
3. If you will be enabling High Availability for the EPIC platform
and have not selected a Shadow Controller node yet, then
you may assign the role to a single Worker node. See “HA
Settings Tab” on page 116 and the About EPIC Guide for
information about EPIC platform High Availability.
Note: The Shadow Controller node cannot be transferred or assigned later. If you do not assign the
Shadow Controller role to a Worker node during the
installation process, then you will not be able to
assign this role to that Worker after installation. If
you have already assigned this role to a Worker
node, then you cannot remove that role from that
Worker node.
CAUTION: ENABLING HIGH AVAILABILITY FOR
THE EPIC PLATFORM PROTECTS AGAINST CONTROLLER NODE FAILURE. THIS IS INDEPENDENT
OF PERSISTENT CLUSTER HIGH AVAILABILITY
FUNCTIONALITY, WHICH PROTECTS AGAINST
VIRTUAL NODE FAILURE. PLEASE SEE THE About
EPIC Guide FOR MORE INFORMATION ABOUT
HIGH AVAILABILITY.
4. Add the Worker node(s) to your EPIC platform.
94
Before adding one or more Worker node(s), be sure that the
node(s) conform to the requirements described in the About
EPIC Guide and any applicable Deployment Guide. For best
results, it is recommended that all nodes share the same configuration (CPU, RAM, storage, OS, etc.)
CAUTION: INSTALLING EPIC ON ANY PHYSICAL
HOST THAT DOES NOT MEET ALL APPLICABLE
REQUIREMENTS MAY LEAD TO UNPREDICTABLE
BEHAVIOR AND/OR DATA LOSS.
CAUTION: EPIC PERFORMS NUMEROUS CONFIGURATION CHANGES TO THE PHYSICAL HOST
DURING INSTALLATION THAT ARE REQUIRED IN
ORDER FOR THE PLATFORM TO FUNCTION.
THESE CHANGES ARE NOT COMPLETELY
REVERSIBLE AND MAY IMPACT ANY OTHER
APPLICATIONS AND PROCESSES THAT ARE CURRENTLY RUNNING ON THE PHYSICAL HOST. IT IS
STRONGLY RECOMMENDED THAT YOU INSTALL
THE EPIC LITE ON A PHYSICAL HOST THAT IS
NOT BEING USED FOR ANY OTHER PURPOSE IN
ORDER TO AVOID POSSIBLE DISRUPTIONS TO
YOUR BUSINESS PROCESSES.
5.6.2.1 - Step 1: Selecting the Node(s)
The first step in adding Worker node(s) to your EPIC platform is
to select the node(s) to add using the IP address(es) of the
node(s). You must then supply the administrator (root) password
for each node. To select the node(s):
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
1. Enter the IP address(es) of the node(s) in the Worker IP field.
You may select one or more nodes as follows:
-
Single IP address: Enter a properly formatted IP address,
such as 10.10.1.1. This will select a single node.
-
Multiple IP addresses: Enter the first three octets of the
IP addresses, and then separate each digit of the fourth
octet with a commas, such as 10.10.1.1,2,5,8. In this
example, EPIC will install four Worker nodes with IP
addresses of 10.10.1.1, 10.10.1.2, 10.10.1.5, and 10.10.1.8.
-
Multiple IP addresses: Enter multiple IP addresses separated by commas, such as 10.10.1.1, 10.10.1.2,
10.10.1.5, 10.10.1.8. In this example, EPIC will
install four Worker nodes with the same IP addresses as
the previous example.
-
IP address range: Enter an IP address range, such as
10.10.1.1-8. In this example, EPIC will install eight
Worker nodes with IP addresses from 10.10.1.1 to 10.10.1.8.
Figure 5.19: Installation Screen - Installed Workers tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
95
EPIC User & Administrator Guide
-
Combination: Use a combination of the above methods,
such as 10.10.1.1, 10.10.1.2,5,8, 10.10.1.912.
key will only be used for initial access to the worker hosts
and will not be permanently stored.
Note: You may only perform one set of Worker node
installations to one or more node(s) at once. To save
time, consider adding all of the Worker nodes at
once by entering multiple IP addresses as described
above.
2. Select how to access the Worker node(s). Your available
options are:
-
Root access: Check the Root Access radio button and
then enter the root password for the Worker node(s) you
are installing in the Root Password field. SSH key access
is not yet enabled.
Figure 5.21: SSH Key information
3. Click the Add Workers button to install EPIC on the selected
node(s).
5.6.2.2 - Step 2: Installing EPIC on the Node(s)
Clicking the Add Workers button installs EPIC on the selected
node(s). The node(s) will appear in the Worker(s) Status table,
which displays the following information for each node being
added:
Figure 5.20: Root Access information
-
SSH Key: If the Worker node(s) have a public key installed
to allow password-free access, then you may check the
SSH Key based Access radio button. Upload the private
key by clicking the Private Key field to open a standard
File Upload dialog that allows you to browse for and
select the key file. If the key requires a pass phrase, enter
that phrase in the Passphrase field. The uploaded private
96
•
IP address: IP address and (if available), hostname of the
node.
•
Status: Current status of the EPIC installation, which updates
as the installation progresses. This will appear as one of the
following:
-
Connecting: EPIC is attempting to connect to the listed
Worker node(s) using the private network interface that
you selected earlier.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
-
Running bundle: EPIC has successfully connected to the
listed Worker node(s) and installing on the node(s).
-
Bundle completed: EPIC has completed installing on the
listed Worker node(s), which are ready to be added to the
EPIC platform. If you added the node(s) by mistake, you
may remove them by clicking the red Delete icon (trash
can).
-
To remove the node(s), click the red Delete icon (trash
can).
CAUTION: YOU WILL NOT BE ABLE TO REMOVE A
MISTAKENLY-ADDED WORKER NODE OR NODES
FROM THE EPIC PLATFORM ONCE YOU PROCEED
TO THE NEXT STEP.
5.6.2.3 - Step 3: Selecting Hard Drives
Note: If you do not need to add hard drives to local
HDFS system storage, then you may proceed
directly to “Step 5: Adding the Node(s) to the EPIC
Platform” on page 98.
Figure 5.22: Installation progress
•
Actions: Once EPIC finishes installing on the node(s), you will
have the opportunity to either add or remove the node(s).
-
Clicking the blue Edit icon (pencil) for a Worker node
opens the Advanced Worker settings popup for that
worker, which allows you to select one or more hard
drive(s) to add to the local HDFS system storage. See
below.
-
To install EPIC on the selected node(s), click the green
Install button.
Clicking the blue Edit icon (pencil) opens the Advanced Worker
settings popup for that worker.
Figure 5.23: Advanced Worker settings popup
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
97
EPIC User & Administrator Guide
To add one or more hard drive(s) to the local HDFS system storage
1. Selecting one or more node(s) in the table, and then click the
green Install button.
1. Click the drive(s) that you want to add.
A confirmation dialog will appear to inform you that EPIC will
need to reboot the nod(s).
2. Click the Set button to close the popup and add the selected
drive(s) to the EPIC platform.
2. Click OK to proceed.
The blue Install Scheduled and then the green Installing bar
appear in the Worker(s) Status table for the selected Worker
node(s) while EPIC finishes adding the node(s) to the EPIC
platform. This status then changes to Rebooting while EPIC
reboots the Worker node to complete the addition.
5.6.2.4 - Step 4: Entering Lockdown Mode
Before proceeding with installing the Worker node(s), you will
need to place EPIC into Lockdown mode, as follows:
1. Open the Quick Access menu and then select Enter site
lockdown.
Figure 5.24: Entering Lockdown mode
Figure 5.25: Completing the Worker installation
The Lockdown the system popup appears.
2. Enter a descriptive reason for the lockdown in the Enter Reason field.
3. Click the Submit button to place EPIC into Lockdown mode.
5.6.2.5 - Step 5: Adding the Node(s) to the EPIC Platform
3. When the installation completes, exit Lockdown mode by
opening the Quick Access menu and then selecting Exit site
lockdown.
The newly added Worker node(s) will appear in the Installed
Workers tab (see “Installed Workers Tab” on page 93).
Once EPIC has been installed on the selected Worker node(s),
you may finish the node(s) to your EPIC platform, as follows:
98
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.6.3 - Available Upgrades Tab
The Available Upgrades tab lists any available updates to your
EPIC platform software and allows you to install additional
nodes. The table lists the following information for each available
upgrade:
•
Upgrade Package: The identifier of the upgrade package.
This will be the same as the filename of the bundle file (without the .bin extension), unless the file has been renamed.
•
Upgrade From: Lists the valid EPIC versions (and possibly
specific version/build combinations) that can be used as the
starting software version for an upgrade using this package.
The current version and build of EPIC appears at the bottom
right corner of every EPIC screen; this information must
match one of the versions or version/build combinations
listed in a package's Upgrade From field in order to perform
an upgrade using that package.
Figure 5.26: BlueData Cluster Installation screen - Available Upgrades tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
99
EPIC User & Administrator Guide
•
Version: EPIC software version that will be in effect once the
upgrade completes.
•
End Version: If the upgrade was successful, this is the EPIC
software version after the upgrade completed.
•
Build Number: EPIC build number that will be in effect once
the upgrade completes.
•
Start Time: Date and time the upgrade process began.
•
End Time: Date and time the upgrade process ended.
•
SKU: BlueData Software, Inc. product variant identifier for the
new EPIC build and version, such as Lite or Enterprise.
•
Status: Overall status of the upgrade attempt:
•
Actions: Clicking the green Upgrade button for a listed
upgrade begins upgrading to that upgrade. See “Upgrading
the EPIC Installation” on page 100.
Note: Upgrading EPIC will also upgrade your
installed image(s) if image updates are available.
Please see “Images Tab” on page 103 for more
information.
5.6.4 - Upgrade History Tab
The Upgrade History tab displays all previous upgrade attempts,
successful or otherwise. The table on this tab lists the following
information for each previous upgrade attempt:
•
Upgrade: The identifier of the upgrade package used for the
upgrade attempt. Clicking this link opens a popup that displays the final per-host status from that upgrade process.
•
Notes: Any notes entered by the Site Administrator when the
upgrade was initiated.
•
Start Version: The EPIC software version that was installed
on the EPIC platform when the upgrade was initiated.
100
-
Complete: The upgrade completed successfully.
-
Error: The upgrade did not complete successfully.
5.6.5 - Upgrading the EPIC Installation
The EPIC upgrade works as follows:
•
The Site Administrator must have root login access to the
Controller node.
•
All physical node(s) in the EPIC platform will be upgraded.
•
All of the nodes must be both powered on and accessible.
•
The upgrade must succeed on all nodes. If the upgrade fails
on one or more node(s), then the entire upgrade will be rolled
back on all node(s). If the upgrade process fails for any physical node, then all remaining host upgrades will be canceled,
and any node that was already modified will be reverted
back to the original EPIC version.
•
EPIC upgrades the Controller node near the beginning of the
upgrade process. At this point, the management service will
be momentarily interrupted in order to change over to the
new version; thus, explicitly refreshing the web UI during this
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
interval may result in a browser error. If this occurs, try
refreshing the page after a brief wait.
To upgrade the EPIC installation:
1. Place the bundle file for the newer software in the /srv/
bluedata/bundles/ directory of the Controller host. This
bundle will be visible as an upgrade package in the Available
Upgrades tab.
2. Place the EPIC platform into Lockdown status by opening the
Quick Access menu and then selecting Enter site lockdown.
The Lockdown the system popup appears.
3. Enter a descriptive reason for the lockdown in the Enter Reason field, and then click the Submit button to place EPIC into
Lockdown mode.
4. In the Available Upgrades tab, click the green Upgrade button for the build that you want to upgrade to. EPIC will verify
that Lockdown mode is in effect and confirm the upgrade
compatibility with your current version, and then either proceed or display an error message.
Figure 5.27: BlueData Cluster Installation screen - Upgrade History tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
101
EPIC User & Administrator Guide
The Confirm Upgrade popup appears.
5. Enter a brief note in the Notes field if you like, and then click
the Proceed button to continue with the upgrade. Any notes
you enter will be saved and can be viewed later.
overall upgrade process continues on other nodes (meaning
that the node will be rolled back if an error occurs).
6. The Upgrade Progress section appears on the Available
Upgrades tab to display the status of the upgrade progress.
The available statuses are:
-
Pending: The upgrade has not yet started.
-
Upgrading: The upgrade is taking place. Additional
details appear during this phase, as EPIC extracts the
upgrade package, upgrades the Controller node and
Worker nodes, and finishes the upgrade. The package
version being applied also appears.
-
Finalizing: EPIC is performing post-upgrade cleanup.
-
Complete: The upgrade has completed successfully.
-
Rolling_Back: Upgrade has encountered an error and is
reverting the EPIC platform back to the original version.
-
Error: The upgrade did not complete successfully.
Figure 5.28: Upgrade Status window
You may view the completed upgrade in the Upgrade History
tab. See “Upgrade History Tab” on page 100.
7. Clicking the Show Upgrade Details link at any time during
the upgrade process displays the Upgrade Status window.
Per-node status values should be interpreted in the same
way as the status values of the overall upgrade process.
There is also an additional upgraded status, which indicates
that the individual node has upgraded successfully while the
102
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.7 - EPIC Catalog
Selecting Catalog in the main menu opens the Catalog screen.
Note: Some images include additional tools, such as
Cloudera Manager or Ambari. The Details column
includes information about any such tool(s) included
with each image. Installing an image that includes
one or more of these tool(s) will automatically install
the tool(s) as well. If you do not want to install these
tool(s), then select an image that does not include
the tool(s). You can access these tools by clicking
the appropriate link in the Process List column in
Node(s) Info tab of the <Cluster> screen. See
“Node(s) Info Tab” on page 30.
The top of this screen contains the following tabs:
•
Images: This tab allows you to download and install additional Hadoop distributions. See “Images Tab” on page 103.
•
Add-On Images: This tab allows you to download and install
third-party data analysis and visualization tools. See “AddOn Images Tab” on page 105.
•
Flavors: This tab allows you to add, edit, and remove node
flavors, which specify the resources allocated to each virtual
node that is created for a transient or persistent cluster. See
“Flavor Tab” on page 107.
5.7.1 - Images Tab
The Images tab lists all of the Hadoop and Spark distribution
images offered by BlueData Software, Inc. These distributions
are used by persistent and temporary clusters to load and run
applications. This tab also displays the image(s) you currently
have installed and allows you to download and install additional
images, if desired.
Note: Images average approximately 1.0 to 1.2GB in
size and may take some time to download and
install, depending on your network, traffic, and other
variables.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
If you already have an image installed (such as Hortonworks
HDP 2.1 Version 1.1) and a new version of that image becomes
available, then the new version of that image will be downloaded
and installed when you upgrade your EPIC installation as
described in “Available Upgrades Tab” on page 99. When an
upgrade to the EPIC platform includes an upgrade to a previously installed image:
•
All persistent clusters that are running the previous version of
the same image will continue running the older version. You
must remove these clusters and then recreate them in order
to apply the newer version of the image. EPIC does not automatically upgrade the images for these clusters to avoid any
potential interruptions or other unpredictable results that may
occur from an automatic upgrade.
103
EPIC User & Administrator Guide
•
After installing the new version of the image, all persistent
clusters and transient jobs created using that image will use
the new version.
•
Version: Version of the image. This references revisions to
the image as a whole, and is distinct from the version of any
software available on the image.
•
To completely remove the older version of the image from
your EPIC platform, you will need to remove any clusters that
are currently using the older version.
•
Details: Operating system and application support information, as well as any additional tool(s) that are included with
the image. This column also lists the size of the root disk
associated with each image. This is important when creating
or editing flavors as described in “Flavor Tab” on page 107.
•
State: Current status of the image, which can be one of the
following:
Note: Consider building any new persistent cluster(s)
using the newer version of the image and testing
your applications on the newer image to ensure
proper functioning before deleting the old clusters.
The table contains the following information and functions:
•
Image Name: Name of the image (Hadoop or Spark distribution).
104
-
Available: The image is not installed on your system and
is available for download from BlueData Software, inc.
Figure 5.29: Catalog screen - Images tab (below)
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
•
-
Verifying: EPIC is verifying the selected image prior to
downloading.
-
Downloading: EPIC is downloading the selected image.
A percentage indicator displays progress.
-
Installing: Download is complete and EPIC is installing
the image.
-
Installed: The image is downloaded, installed, and ready
for use.
requirements for one or more application(s). For example, a business intelligence tool may be compatible with a cluster running
Cloudera 5.2 with YARN and will appear as an option when you
create this type of cluster. Third-party applications run on Edge
nodes that are separate and distinct from the Controller node
and any Worker node(s) in a cluster. You can select the number
and flavor of Edge nodes when creating a cluster that includes
one or more of these tool(s). See “Creating a New Persistent
Cluster” on page 37.
-
Error: An error has occurred during download or installation. Please try reinstalling the image. If the problem persists, please see the Installation Guide for instructions on
contacting BlueData Technical Support for assistance.
Note: EPIC does not ship with any third-party tools;
however, you may configure tools to run on EPIC
Edge nodes. Please refer to the Edge Node Authoring Guide for additional information.
Actions: The following actions are available for each available (not installed) distribution:
-
-
Download: Clicking the blue Download icon (down
arrow) in the Actions column downloads and installs the
selected image.
Cancel: Clicking the red Cancel icon (letter “X”) in the
Actions column at any point during the download/installation process cancels the image installation.
The table on this screen contains the following information and
functions:
•
Image Name: Name of the third-party tool.
•
Version: Version of the tool.
•
Description: Description of the tool.
•
State: Current status of the third-party image, which can be
one of the following:
-
Available: The third-party image is not installed on your
system and is available for download from BlueData Software, inc.
-
Starting: EPIC is beginning the download process for the
selected third-party image.
5.7.2 - Add-On Images Tab
The Add-On Images tab displays a list of third-party data analysis and visualization applications that are available for download
and installation. Installed applications will appear as one or more
checkbox(es) when you create a persistent cluster that meets the
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
105
EPIC User & Administrator Guide
-
Downloading: EPIC is downloading the selected thirdparty image. A percentage indicator displays progress.
-
Verifying: Download is complete and EPIC is verifying the
integrity of the downloaded third-party tool.
-
Installing: EPIC is installing the third-party image.
-
Installed: The third-party image is downloaded, installed,
and ready for use.
-
Error: An error has occurred during download or installation. Please try reinstalling the third-party image. If the
problem persists, please see the Installation Guide for
instructions on contacting BlueData Technical Support for
assistance.
•
Actions: The following actions are available for each available (not installed) third-party tool:
-
Download: Clicking the blue Download icon (down
arrow) in the Actions column downloads and installs the
selected third-party tool.
-
Cancel: Clicking the red Cancel icon (letter “X”) in the
Actions column at any point during the download/installation process cancels the third-party tool installation.
Figure 5.30: Add-On Images tab
106
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
5.7.3 - Flavor Tab
•
Root disk size (GB): Size of the root disk associated with that
flavor, if any.
The Flavor tab allows the Site Administrator to create, edit, and/
or delete virtual node flavors (see the About EPIC Guide) for use
in persistent clusters and transient jobs.
•
Status: Status of the flavor. This can be one of the following:
This tab contains a green Add New Flavor button, which opens
the Add Flavor Information popup. The table on this screen contains the following information and functions:
•
Flavor Name: Name of the flavor.
•
Flavor Description: Brief description of the flavor.
•
Cores: Number of virtual CPU cores defined for that flavor.
•
Memory (GB): Amount of memory (GB) defined for that flavor.
•
-
Ready: The flavor is ready for use.
-
Error: EPIC could not create the flavor; delete it and try
again.
Actions: The following actions are available for each flavor:
-
Edit: Clicking the blue Edit icon (pencil sheets) in the
Actions column opens the Edit Flavor Information
popup, which is populated with the selected flavor’s current information. To modify the flavor, enter the updated
value(s) in the appropriate field(s) and then click Submit.
Figure 5.31: Flavor tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
107
EPIC User & Administrator Guide
Editing an existing flavor will not affect any current cluster(s) or job(s) that currently use that flavor; even adding a
virtual node to an existing cluster will use the old definition of that flavor. Only clusters and jobs created after
editing the flavor will use the updated values.
-
Delete: Clicking the red Delete icon (trash can) in the
Actions column deletes the selected flavor. Deleting an
existing flavor will not affect any current cluster(s) or job(s)
that currently use that flavor; even adding a virtual node
to an existing cluster will still use the deleted flavor. A
popup warning appears asking you to confirm or cancel
the action. Click OK to proceed, or Cancel to exit without
deleting the job.
CAUTION: EPIC WILL ALLOW YOU TO DELETE ALL
OF THE FLAVORS; HOWEVER, YOU MUST HAVE
AT LEAST ONE FLAVOR DEFINED IN ORDER TO
CREATE A TRANSIENT JOB OR PERSISTENT
CLUSTER.
5.7.4 - Creating a New Flavor
To create a new flavor:
1. In the Main Menu, select Catalog to open the Catalog
screen, and then select the Flavor tab to open that tab.
2. Click the green Add New Flavor button to open the Add Flavor Information popup.
Enter the following information:
108
•
Flavor Name: Name of the flavor.
•
Flavor Description: Brief description of the flavor.
•
Cores: Number of virtual CPU cores defined for that flavor.
•
Memory (GB): Amount of memory in GB defined for that flavor.
•
Root Disk (GB): Enter the size of the root disk to use for this
flavor, if any. Leaving this field blank allows the flavor to use
the image root disk size (see “Images Tab” on page 103).
This is ideal for most situations; however, you may specify a
custom root disk size that will override the image root disk
size when creating a cluster or job that uses this flavor. When
specifying a root disk size for a flavor:
-
You cannot specify a flavor root disk size that is smaller
than the root disk size of the image you will be using
when you create a cluster or job, as doing this will prevent you from using your desired image with this flavor.
-
You may specify a flavor root disk size that is larger than
the root disk size of the image you will be using when you
create a cluster or job. This may be necessary if you will
be running an application that requires a local file system;
however, creating a larger root disk size will cause cluster
creation to take longer because EPIC will need to create
a custom partition and then expand the image into that
partition.
Note: For optimal performance, leave the Root Disk
(GB) field blank and use DataTaps (see “DataTaps”
on page 56) wherever possible.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
3. Click Submit to finish creating the flavor, or Cancel to exit
without creating the flavor.
Figure 5.32: Add Flavor Information popup
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
109
EPIC User & Administrator Guide
5.8 - EPIC Settings
Selecting Settings in the main menu opens the System Settings
screen.
affect tenants created after the change is made; the existing
tenant(s) will retain their existing DataTaps in their current
directories.
The top of this screen contains the following tabs:
•
System Storage: This tab allows you to specify the root
directories for automatically-created tenant DataTaps and
temporary cluster file systems. See “System Storage Tab” on
page 110.
•
User Authentication: This tab allows you to modify user
authentication settings. See “User Authentication Tab” on
page 114.
•
•
•
To change the root directory for DataTaps and/or temporary cluster file systems:
1. Check the appropriate checkbox(es) to select the root directory or directories that you want to change.
HA Settings: This tab allows you to manage High Availability
protection for the EPIC platform. See “HA Settings Tab” on
page 116.
2. If you are changing the root directory for automatically created tenant DataTaps, then enter a default name for the
DataTaps in the Name field. All newly created tenants will
have a DataTap with this name when they are created.
Other Settings: This tab allows you to manage IP addresses,
network interfaces, and the disk to use for High Availability
protection. See “Other Settings Tab” on page 118.
3. If you are changing the root directory for automatically created tenant DataTaps, enter a brief description for those
DataTaps in the Description field.
5.8.1 - System Storage Tab
4. Select the file system type to use (HDFS, Gluster, or NFS).
The System Storage tab allows you to change the root directories for one or both of the following:
•
-
DataTaps that EPIC creates automatically when you create a
new tenant. This is the root directory for all automatically created DataTaps. Changing the system storage location will
110
Temporary cluster file systems. Changing this directory will
affect clusters created after the change is made; the existing
cluster(s) will retain their existing file system settings and
directories.
If you selected HFDS, then enter the host, standby
namenode host, port, and path for the root tenant
DataTap directory in the appropriate fields. You may also
enable or disable Kerberos security by checking or clearing the Kerberos Protected checkbox, as appropriate.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
See “Kerberos Security” on page 111. If you are
enabling Kerberos protection then the Host must be a
Fully Qualified Domain Name (FQDN); IP addresses are
not acceptable.
-
-
-
If you selected Gluster, then enter the host, port, volume,
and path for the root tenant DataTap directory in the
appropriate fields.
KDC Host: Name or IP address of the Kerberos host(s).
You may enter multiple hosts separated by commas. If
you enter more than one host, then EPIC will use the first
host in the list as the primary Kerberos host and will
attempt to use the other host(s) if the primary host is
unreachable.
-
If you selected NFS, then enter the host, share, and path
for the root tenant DataTap directory in the appropriate
fields.
KDC Port: Post used by the Kerberos host(s). Leave this
field blank if not known. If you enter a value in this field,
then all of the Kerberos hosts must use the same port.
-
Keytab File: Obtain this file from your Kerberos administrator. If needed (and if you have root access to the EPIC
Controller node), then you can place this file on your local
computer and check the Yes radio button under Upload
Keytab File and then click the Choose button in the Keytab File field to browse to the file and securely upload it
to EPIC. Alternatively, you can check the No radio button
and then enter the name of a file in the Keytab File field.
This file must previously have been uploaded to EPIC,
either via the interface or manually.
5. Click Submit to make your change(s).
5.8.1.1 - Kerberos Security
EPIC supports Kerberos protection for automatically-created
tenant DataTaps and temporary cluster file systems. You can also
enable or disable Kerberos protection for external DataTaps. To
enable Kerberos protection:
1. If you are editing system storage settings, then check the
appropriate Change the root directory... checkbox(es) for
automatically-create DataTaps and/or temporary cluster file
systems, as appropriate. If you are editing a DataTap, then
proceed to Step 2.
2. Check the appropriate Kerberos Protected checkbox(es).
3. Enter the following parameters. If you are enabling Kerberos
protection for both DataTaps and temporary cluster file systems, then you will need to enter these parameters separately for each root directory:
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
Note: Your organization security policies may not
allow you to upload keytab files via the EPIC interface. If you need to manually upload keytab files,
then place keytab files used for system storage in
the /srv/bluedata/keytab/site_admin directory on
the Controller node. Keytabs used in DataTap definitions are in subdirectories associated with the
tenant ID, such as /srv/bluedata/keytab/3.
111
EPIC User & Administrator Guide
-
Client Principal: This is a unique identity to which Kerberos can assign tickets. Enter the appropriate value in
this field.
-
HDFS Service ID: Name of the service, as defined by
your Kerberos administrator.
-
Realm: Name space that helps define access permissions. Obtain this from your Kerberos administrator.
.
4. Click Submit to make your changes.
Note: To disable Kerberos protection, clear the
appropriate Kerberos Protected checkbox(es) and
then click Submit.
Figure 5.33: Kerberos settings for HDFS storage
112
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
Figure 5.34: System Settings screen - System Storage tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
113
EPIC User & Administrator Guide
5.8.2 - User Authentication Tab
-
The User Authentication tab allows you to manage how EPIC
integrates with your existing external authentication server (if
any). To change the user authentication method:
Note: These fields must match your existing LDAP
parameters exactly. Contact your LDAP administrator for assistance.
1. Select the type of authentication to use with EPIC using the
Authentication Type pull-down menu. The available options
are:
-
-
Skip to Step 4.
None: Selecting this option configures EPIC to use the
built-in user database for user authentication. See “Managing Users and Sessions” on page 84 for information
on managing the local EPIC user database. Skip to Step 4.
LDAP: Selecting this option configures EPIC to use an
existing external LDAP server. Proceed to Step 2.
2. Active Directory: Selecting this option configures EPIC to
use an existing external Active Directory server. Skip to Step
3
3. If you selected LDAP in the Authentication Options pulldown menu, you must supply the following parameters:
-
Enter the hostname or IP address of the external LDAP
host in the LDAP Host field.
-
Enter the port of the external LDAP host in the Port field.
-
Enter the LDAP attribute used to retrieve user profiles
(such as cn or uid) in the User Attribute field.
114
Enter the LDAP user group(s) that will be used when
searching for EPIC users in the User Subtree DN field.
4. If you selected Active Directory in the Authentication
Options pull-down menu, you must supply the following
parameters:
-
Enter the hostname or IP address of the external LDAP
host in the AD Host field.
-
Enter the port of the external Active Directory host in the
Port field.
-
Enter the Active Directory attribute used to retrieve user
profiles (such as cn or uid) in the User Attribute field.
-
Enter the Active Directory subtree that will be used when
searching for EPIC users in the Subtree DN field.
Note: These fields must match your existing Active
Directory parameters exactly. Contact your Active
Directory administrator for assistance.
5. When you have finished configuring your authorization
options, click Submit to save your changes.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
Figure 5.35: System Settings screen - User Authentication tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
115
EPIC User & Administrator Guide
5.8.3 - HA Settings Tab
The HA Settings tab allows you to enable High Availability protection for the EPIC platform, which protects against the failure of
a single physical node (Controller, Shadow Controller, or Arbiter).
Note: Enabling High Availability protection for the
EPIC platform protects the EPIC platform; it does not
protect any virtual clusters that are running on the
EPIC platform. Please see the About EPIC Guide
and the EPIC Installation Guide for more information, and “Installing Worker Nodes” on page 94 for
information about enabling cluster High Availability.
CAUTION: YOU CANNOT DISABLE EPIC PLATFORM HIGH AVAILABILITY AFTER IT HAS BEEN
ENABLED.
116
To enable High Availability for the EPIC platform:
1. Ensure that the EPIC platform meets all of the requirements
listed in the About EPIC Guide by having at least three physical nodes with one Worker node assigned as the Shadow
Controller, as described in the EPIC Installation Guide and
“Installing Worker Nodes” on page 94.
Note: All existing clusters and jobs must be removed
before enabling High Availability protection. In general, you should enable this feature when you first
install the EPIC platform.
2. Enter site lockdown as described in “Lockdown Mode” on
page 122.
3. Check the Enable HA checkbox.
Figure 5.36: System Settings screen - HA Settings tab (below)
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
4. Enter an available IP address to use as the cluster IP address
in the Cluster IP field. This IP address must be in the same
subnet as the physical nodes in the EPIC platform. After
enabling HA, you should use this IP address to log into EPIC,
because this will automatically connect you to the Controller
node (during normal operation) or the Shadow Controller
node (when HA protection has been triggered by Controller
node failure). If the Controller node fails, then you will not be
able to access EPIC using the IP address of that node.
Note: The external switch connecting the physical
nodes to the network must support gratuitous arp in
order for the cluster IP address to function correctly.
5. The Shadow Controller node field displays the IP address
and hostname of the physical node to which this role has
been assigned. This field is read only.
6. Select the physical node to use as the Arbiter node using the
Arbiter Node pull-down menu. If the EPIC platform has three
physical nodes, then the node remaining after assigning the
Controller and Shadow Controller nodes will be the Arbiter. If
there are more than three physical nodes, then you may
select one of the remaining Worker nodes to be the Arbiter.
You cannot remove or modify the Arbiter Node after enabling
High Availability protection.
7. Click Submit to proceed with enabling High Availability protection. A message appears to inform you that the process
has begun, and the HA Settings tab displays the message
Configuring HA. This process may take up to 30 minutes to
complete depending on a number of factors.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
8. A message appears in the upper right corner of the EPIC
interface once the process completes informing you have
EPIC is now running in High Availability mode and reminding
you to begin using the cluster IP address that you entered in
Step 3 to log into EPIC going forward. Clicking the link in this
message logs you out of EPIC and returns you to the Login
screen using the cluster IP address.
9. Log into EPIC using your normal username and password.
10. Exit site lockdown as described in “Lockdown Mode” on
page 122.
If enabling High Availability fails, then the fields in the HA Settings tab will reappear, and EPIC will continue running in its previous non-High Availability state. Please contact BlueData
Technical Support for assistance.
5.8.3.1 - Troubleshooting Node Failures
If a High Availability node (Controller, Shadow Controller, or Arbiter) fails:
•
You can access the Service Status tab of the Site Administrator Dashboard (see “Service Status Tab” on page 73) to see
which node has failed. EPIC will automatically attempt to
recover the node.
•
If the node has not come back up within 20 minutes and EPIC
has not resumed full High Availability operation, then you
must manually reboot the failed node. EPIC will resume High
Availability operation if the manual reboot resolves the issue.
117
EPIC User & Administrator Guide
•
is available in the System Storage tab (see “System Storage
Tab” on page 110).
If manually rebooting the failed node fails to resolve the
problem, then you will need to contact BlueData Technical
Support for assistance.
5.8.4 - Other Settings Tab
The Other Settings tab displays the following information:
•
Floating IP Address/CIDR: These fields display the CIDR to
use in the Floating IP Addr and CIDR fields. This range of
addresses allows network access from outside the EPIC platform to the virtual nodes that EPIC will create as part of future
clusters. If needed, you may enter new values in these fields.
This address cannot be changed if there are any existing persisting clusters or transient jobs.
•
Public network interface: Name of the network interface
connected to the external network. See the About EPIC
Guide. This field is read only.
•
Private network interface: Name of the network interface
connected to the internal management network. See the
About EPIC Guide. This field is read only.
•
Disk for HA: Disk being used for High Availability protection.
See the EPIC Installation Guide. This field is read-only.
The Other Settings tab also contains information about whether
a local HDFS service was created when you installed EPIC. This
information is read-only. The presence of a local HDFS service
does not necessarily mean that the EPIC platform is currently
using local HDFS for system storage purposes. This information
118
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
Figure 5.37: System Settings screen - Other Settings tab
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
119
EPIC User & Administrator Guide
5.9 - Support/Troubleshooting
Selecting Support in the main menu opens the Support/Troubleshooting screen.
•
This screen allows you to create, delete, and download SOS
reports to help you troubleshoot problems with your EPIC platform. If you contact BlueData Technical Support, you may be
requested to forward an SOS report for support purposes. To
upload an SOS report, your EPIC platform must have both of the
following:
The SOS Report(s) table contains the following information/functions for all currently available SOS reports:
•
Internet access, either directly or through a proxy.
DNS service that can resolve references to Internet
addresses.
•
File Name: Name of the SOS report file.
•
File Size (MB): Size of the file in megabytes.
•
Status: Current status of the SOS report. This may be either:
-
Generating: EPIC is currently generating the SOS report.
Figure 5.38: Support/Troubleshooting screen
120
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5 - Site Administration
-
Ready: The SOS report is ready for download or upload
-
Uploading: The SOS report is being uploaded to BlueData Software, Inc. The report can be downloaded, but
not deleted.
-
Uploaded: The most recent attempt to upload the SOS
report to BlueData Software, Inc. succeeded. You may
download, re-upload, or delete this report.
-
Upload failed: The most recent attempt to upload the
SOS report to BlueData Software, Inc. failed. You may
download, re-upload, or delete this report.
-
Error: The SOS report is in an unknown state that is probably corrupted. You may delete this report.
You can perform the following actions for each available SOS
report:
•
Delete: Clicking the red Delete icon (trash can) deletes the
selected SOS report. A popup warning appears asking you to
confirm or cancel the action. Click OK to proceed, or Cancel
to exit without deleting the SOS report.
•
Download: Clicking the gray Download icon (down arrow in
a circle) opens an OS-default Download window that allows
you to retrieve the selected SOS report.
•
Upload: Clicking the Upload icon (up arrow in a circle)
uploads (or re-uploads) the selected SOS report to BlueData
Software, Inc.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
5.9.1 - Generating an SOS Report
To generate an SOS report:
1. If you want to upload the SOS report to BlueData Software,
Inc. then leave the Upload to BlueData checkbox checked;
otherwise, clear this checkbox to prevent the report from
being uploaded.
2. Select the desired level of detail using the SOS Report Level
pull-down menu. You may select a number from 1-3, where 1 sysinfo & primary logs is the least detailed and 3 is the most
detailed. Selecting 3 - large files may generate a report that
is tens or even hundreds of megabytes in size, depending on
your installation and the circumstances.
3. Click the blue Generate SOS button.
4. You may download and/or re-upload the new SOS report
once the Status changes from Generating to Ready.
Note: You may generate and/or upload a single SOS
report at a time.
121
EPIC User & Administrator Guide
5.10 - Lockdown Mode
Lockdown mode prevents all users who do not have Site Administrator privileges from making any changes to the EPIC platform,
such as creating/editing jobs, clusters, or DataTaps. This mode
ensures that the EPIC platform will remain stable while the Site
Administrator makes changes to the EPIC platform configuration,
such as:
•
•
Adding Worker node(s) (see “Managing the EPIC Installation” on page 92)
Upgrading EPIC (see “Installed Workers Tab” on page 93)
You may also use Lockdown mode to quiesce the platform
during other maintenance activities outside of the EPIC interface;
however, Lockdown mode does not prevent users from logging
into any virtual nodes that already exist and performing activities
within the virtual nodes.
Entering Lockdown mode will happen immediately if no tasks are
running. If one or more task(s) are in progress when the Site
Administrator enters Lockdown mode, then EPIC will complete
those tasks and prevent additional changes. For example:
•
Jobs will finish running.
•
New clusters will finish installing.
122
5.10.1 - Entering Lockdown Mode
To enter Lockdown mode:
1. Open the Quick Access menu and then select Enter site
lockdown.
The Lockdown the system popup appears.
2. Enter a descriptive reason for the lockdown in the Enter Reason field and then click Submit.
The red Entering site lockdown indicator appears in the
Toolbar while EPIC finshes any currently-running jobs. This
indicator changes to Site lockdown once EPIC has completed all running jobs and finished entering Lockdown
mode. Any user who attempts to make any changes will
receive a popup warning that this mode is in effect.
5.10.2 - Exiting Lockdown Mode
To exit Lockdown mode, open then Quick Access menu and
then click Exit site lockdown. Exiting from Lockdown mode is
instantaneous and allows normal usage of the EPIC platform to
resume.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
A - Troubleshooting
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
A-1
EPIC User & Administrator Guide
This chapter contains basic troubleshooting instructions, as follows:
•
Hardware Errors: This section assists you in the event of a
hardware failure in either the Controller node or Worker
node(s). See “Hardware Errors” on page A-3.
•
Runtime Errors: This section assists you with various errors
that may occur while running EPIC. See “Runtime” on
page A-4.
Please see the Installation Guide if you need to perform troubleshooting while installing EPIC.
A-2
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
A - Troubleshooting
A.1 - Hardware Errors
This section contains instructions that may help you if you run
into hardware problems while running EPIC.
•
If a Hadoop job spans multiple physical nodes and one of
those nodes fails, the job will fail and the Controller node will
remove the failed Worker from the EPIC platform.
A.1.1 - General Problems
•
EPIC does not include any redundancy. If a disk failure
occurs, you will lose all data that has not been backed up. In
general, you should use an external resource for default storage and ensure that it is backed up in accordance with your
IT policies.
•
An OpenStack limitation prevents you from being able to
delete a Worker node. If you need to delete a Worker node,
then you can blacklist the affected node, but that is not the
same as truly deleting it.
•
If a Worker node fails, the Controller node will automatically
drop the affected Worker node. You can replace a failed
Worker node by setting up the new physical node with the
same hostname, IP address, and MAC address as the old
Worker node. Boot the replacement Worker node and register it with the desired Controller node. If this replacement
involved swapping out a data drive (such as if you have local
default storage), then you will need to recover the data from
your existing backups according to your existing disaster
recovery plan.
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
A-3
EPIC User & Administrator Guide
A.2 - Runtime
automatically create the indicated subdirectory if necessary,
whenever any writes are done to that DataTap.
This section contains instructions that may help you if you run
into software problems while using EPIC.
•
While one or more running jobs are accessing a DataTap,
deleting or editing that DataTap may cause errors in those
jobs.
•
Tenant admins may choose to create DataTaps in different
tenants that point to the same storage path; in this situation,
jobs in different tenants can access the same storage simultaneously. Also, multiple jobs within a tenant may use a given
DataTap simultaneously. While such sharing can be useful, be
aware that the same cautions and restrictions apply to these
use cases as for other types of shared storage: multiple jobs
modifying files at the same location may lead to file access
errors and/or unexpected job results.
A.2.1 - General
•
If one of the services listed in the Node(s) Status tab of the
Physical Nodes screen shows a status of Critical, then use
the Support/Troubleshooting screen to generate an SOS
report, as described in “EPIC Catalog” on page 103.
•
If you see any other software error, then either run an SOS
report (see above) or contact BlueData Technical Support as
described in the Installation Guide.
•
Remember that a Hadoop application may experience an
internal error that does not affect EPIC operation. In this case,
the job status will appear as completed. EPIC only returns a
job execution error if EPIC is unable to run the job.
A.2.2 - DataTaps
•
The Path field of a DataTap specifies where the top of the
DataTap's file system is rooted. For manually created
DataTaps, this field must either be empty, or it must point to
an existing subdirectory of the indicated storage system. For
an automatically created tenant default DataTap, EPIC will
A-4
A.2.3 - DataTap URIs
Given a valid DataTap, you can use its name to compose DataTap
URIs that you give to applications as arguments.
Each such URI maps to some path on the storage system that the
DataTap points to. The path indicated by a URI might or might not
exist at the time you start a job, depending on what the application wants to do with that path. Sometimes the path must indicate a directory or file that already exists, because the
application intends to use it as input. Sometimes the path must
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
A - Troubleshooting
three physical hosts with enough available resources on
them, then creating a cluster with YARN High Availability will
fail.
NOT be something that currently exists, because the application
intends to create it.
The semantics of these paths are entirely application-dependent, and are identical to their behavior when running the application on a physical Hadoop platform.
A DataTap URI can also be used to specify the location of a cluster file system. In this usage, the path that the URI maps to does
not need to already exist; it will be created if necessary.
Note: YARN High Availability is not currently available for transient clusters.
If a job/cluster creation attempt fails for lack of resources, you
can attempt the following solutions:
•
Use a smaller number of nodes, or a flavor with smaller
VCPU/memory requirements, if that is still acceptable for the
application requirements.
•
Free up resources by either:
A.2.4 - Common Job/Cluster Errors
Lack of sufficient available resources is one of the most common
reasons why creating a job or cluster may fail. Some of the specific resource issues that commonly occur can include:
•
There are not enough free resources remaining within the
current tenant (a violation of tenant quotas) to create all of
the requested virtual nodes for the job or cluster.
•
The EPIC platform as a whole does not have enough free
resources available, even if the tenant quota is not violated.
•
An attempt to create at least one of the virtual nodes was
unable to find enough free resources provided by any single
physical host. This may occur because a virtual node must
“fit” within a single host; it cannot span across or use
resources drawn from multiple physical hosts.
•
Persistent clusters with YARN High Availability enabled
require the first three virtual nodes of a cluster on different
physical hosts. If your EPIC platform does not have at least
Copyright © 2015, BlueData Software, Inc. ALL RIGHTS RESERVED.
-
Deleting unused or lower-priority clusters in the tenant.
-
Waiting for current transient jobs in the tenant to complete.
-
Deleting unneeded or lower-priority transient jobs.
•
If you selected YARN High Availability for the cluster but can
make do without, try creating the cluster again without
enabling YARN High Availability.
•
Contact the Tenant Administrator about raising the tenant
quota (if tenant quota violation caused the error).
•
Contact the Site Administrator about adding more physical
hosts to the platform.
A-5
EPIC User & Administrator Guide, version 1.5.0 (06/2015)
This book or parts thereof may not be reproduced in any form
without the written permission of the publishers. Printed in the
United States of America. Copyright 2015 by BlueData Software,
Inc. All rights reserved.
Contact Information:
BlueData Software, Inc.
148 Castro Street, Suite A-1
Mountain View, CA 94041
Email: info@bluedata.com
Website: www.bluedata.com
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising