Troubleshooting Actifio Systems - Actifio Public Documentation

Troubleshooting
Actifio Systems
Troubleshooting Actifio Systems
Copyright, Trademarks, and other Legal Matter
Copyright © 2010 - 2017 Actifio, Inc. All rights reserved.
Actifio®, AnyIT®, Dedup Async®, OnVault®, and VDP® are registered trademarks of Actifio, Inc.
Actifio Sky™, Actifio One™, CDS™, Copy Data Storage Platform™, Manage Data Simply™, Protection and
Availability Storage Platform™, PAS™, and Virtual Data Pipeline™ are trademarks of Actifio, Inc.
All other brands, product names, goods and/or services mentioned herein are trademarks or property of their
respective owners.
Actifio, Inc., is a provider of data protection and availability products. Actifio’s technology is used in products sold
by the company and products and services sold and offered by its commercial partners. The current list of Actifio
patents is available online at: http://www.actifio.com/patents/
Actifio believes the information in this publication is accurate as of its publication date. Actifio reserves the right
to make changes to information published in this document, including without limitation specifications and
product descriptions, at any time and without notice. This document supersedes and replaces all information
supplied prior to the publication hereof.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” ACTIFIO, INC. MAKES NO
REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS
PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
This software and the associated documentation are proprietary and confidential to Actifio. Use, copying, and
distribution of any Actifio software described in this publication requires an applicable software license. Any
unauthorized use or reproduction of this software and the documentation may be subject to civil and/or criminal
liability.
Actifio strives to produce quality documentation and welcomes your feedback. Please send comments and
suggestions to docs@actifio.com.
ii
Troubleshooting Actifio Systems | actifio.com |
Contents
Preface ......................................................................................................................................... v
Actifio Appliances ..............................................................................................................................................................v
The ActifioNOW Customer Portal .....................................................................................................................................v
Actifio Support Centers .....................................................................................................................................................v
Chapter 1 - Introduction to Troubleshooting ........................................................................... 1
Chapter 2 - Using the Actifio Knowledge Base ....................................................................... 3
Chapter 3 - Using Actifio Logs .................................................................................................. 5
What You Can Learn from Actifio Logs ............................................................................................................5
Downloading Logs from the Actifio Appliance ...............................................................................................7
Downloading Connector Logs from One of Your Hosts .................................................................................8
Troubleshooting with Actifio Logs ...................................................................................................................9
Sending Logs to Support By Email................................................................................................................10
Chapter 4 - Issues Related to Networking and Permissions ................................................ 11
Actifio CDS Generation-3 Appliances ............................................................................................................12
Actifio CDS Generation-4 Appliances ............................................................................................................13
Chapter 5 - Getting Help .......................................................................................................... 15
Creating a Support Case .................................................................................................................................15
Managing Open Cases....................................................................................................................................17
| actifio.com | Troubleshooting Actifio Systems
iii
iv
Troubleshooting Actifio Systems | actifio.com |
Preface
This guide provides strategies and step-by-step instructions on how to troubleshoot your Actifio system. It assumes
you have read Getting Started with Actifio Copy Data Management, are familiar with the components of the
Actifio Desktop, and have a good understanding of the concepts associated with an Actifio appliance.
Your Actifio appliance’s Documentation Library contains detailed instructions on how to configure your Actifio
environment. The following guides will be of particular interest:
•
Network Administrator’s Guide to Actifio Copy Data Management
•
Configuring Actifio Event Alerting
•
Connecting Hosts to Actifio Appliances
•
Planning and Developing Service Level Agreements
•
Actifio Authentication to VMware vCenter
•
An Oracle DBA’s Guide to Actifio Copy Data Management
•
An SQL Server DBA’s Guide to Actifio Copy Data Management
Actifio Appliances
Unless otherwise specified, all features and functions described in this document apply to all Actifio appliances.
The ActifioNOW Customer Portal
During the configuration and initialization of your Actifio appliance your Actifio representative provided you with a user
name and password for the ActifioNOW customer portal.
From the ActifioNOW customer portal you can obtain detailed reports about your Actifio appliance, access the Actifio
product documentation, including release notes, and search the knowledge base for answers to specific questions.
To log into the ActifioNOW customer portal:
1.
Go to: https://now.actifio.com.
2.
When prompted, enter the user name and password provided by your Actifio representative.
Actifio Support Centers
To contact an Actifio support representative, you can:
•
Send email to: support@actifio.com
•
Call:
From anywhere: +1.315.261.7501
US Toll-Free: +1.855.392.6810
Australia: 0011 800-16165656
Germany: 00 800-16165656
New Zealand: 00 800-16165656
UK: 0 800-0155019
| actifio.com | Troubleshooting Actifio Systems
v
vi
Troubleshooting Actifio Systems | actifio.com |
1
Introduction to Troubleshooting
Troubleshooting is an important, complex, and frequently frustrating activity. Here is a process to help you
troubleshoot effectively and with as little frustration as possible.
1.
If you have an Actifio error message, see: Chapter 2, Using the Actifio Knowledge Base.
2.
You can gain valuable context from your Actifio logs as detailed in: Chapter 3, Using Actifio Logs.
3.
If it’s something else, start with: Chapter 4, Issues Related to Networking and Permissions.
4.
If the problem is more complex, then go to: Chapter 5, Getting Help.
The Best Solution is Prevention
A system configured according to best practices is less likely to run into trouble. It is a good idea to make sure your
system adheres as closely as possible to the best practices described in:
Actifio Product Documentation Resources
Proper Configuration
Network Administrator’s Guide to Actifio Copy Data Management
Configuring Actifio Event Alerting
Connecting Hosts to Actifio Appliances
Planning and Developing Service Level Agreements
Best Practices
Daily Tasks for Administrators
Using the Report Manager
Most Common Error Codes in Actifio 7.x
Whom You Rely On
A VMware vCenter Administrator’s Guide to Actifio Copy Data Management
An Oracle DBA’s Guide to Actifio Copy Data Management
An SQL Server DBA’s Guide to Actifio Copy Data Management
| actifio.com | Troubleshooting Actifio Systems
1
2
Troubleshooting Actifio Systems | actifio.com |
2
Using the Actifio Knowledge
Base
Your best tool for resolving events with error messages is the knowledge base on the ActifioNOW portal. To search
the knowledge base:
1.
Log into ActifioNOW at: https://now.actifio.com.
2.
Click the Learn tab at the top of the screen and select Knowledge Base.
Accessing the Actifio Knowledge Base from ActifioNOW
3.
Enter the Event ID in the Search field.
4.
Under Articles, select the Event ID filter, and under Product Versions select Any, then click Search.
Search for Event ID 43901-175
| actifio.com | Troubleshooting Actifio Systems
3
5.
The entry for the Event ID describes the problem and how to fix it or where to look for additional clues.
Error Code 43901-175 Knowledge Base Entry
6.
4
If the knowledge base does not provide a solution, continue to Chapter 3, Using Actifio Logs.
Troubleshooting Actifio Systems | actifio.com |
3
Using Actifio Logs
Logs can provide valuable context for an issue, if you know what to look for.
This section describes the Actifio logs and their contents, as well as:
What You Can Learn from Actifio Logs on page 5
Downloading Logs from the Actifio Appliance on page 7
Downloading Connector Logs from One of Your Hosts on page 8
Troubleshooting with Actifio Logs on page 9
Sending Logs to Support By Email on page 10
What You Can Learn from Actifio Logs
The Actifio services generate many logs, some of which are useful for troubleshooting. This section describes how all
of the logs work together. You do not need to use all logs for troubleshooting; to see the most useful logs for
troubleshooting, skip to Troubleshooting with Actifio Logs on page 9.
Each Actifio service has its own log. This is how the different services relate to each other.
Component and Log Relationships
| actifio.com | Troubleshooting Actifio Systems
5
Downloadable Actifio Logs
Service
Log File Name
What the Logs Include
adhd
adhd.log
Local and remote deduplication, replication and
communications with remote systems.
adhd.log-<date>
adhd.log-<date>.bz2
appliance
<HostID>.trc
ethernet.<HostID>.trc
Primary trace logs for reviewing failovers and node
reboots, ethernet status, etc.
node.trc and ntp.trc
dds
dds.log
A set of counters for ongoing system processes.
database
postgresql-<Timestamp>.log
The Actifio internal PostGRES database
flasher
flasher.log
Creation of snapshots.
install
install.log
Events during the installation process.
omd
omd.log
Creation and deletion of VDisks before invoking the
Flasher.
patch
autoupdatepatch-<Version>-update.log
Events during the patch update process.
uds-patch.log
uds-patch-<Timestamp>.log
psrv
psrv.log.0
Communications with Tomcat, Actifio database,
VMware, CLI, system health, scheduling GC.
streamsnapd
streamsnapd.log
Disk to remote disk replication operations.
systemlog
command
/var/log/command – CLI command history
fusevdisk.error.log
Events relating the creation of VDisks.
messages
/var/log/messages – Can be used along with node.trc
to correlate issues with the Linux kernel
ssd-config.log
Events that occurred when configuring the SSD.
tomcat
catalina.log.<sequence>
Communications with the Actifio Desktop and the
Actifio Resource Center. The sequence number
increases as you go back in time.
udpmqd
udpmqd.log
Multiplexes udpengine access to snap storage.
udppm
udppm.log
Scheduling and operation of jobs.
udppm.log-<Date>
udppm.log-<Date>.bz2
6
Troubleshooting Actifio Systems | actifio.com |
Downloadable Actifio Logs
Service
Log File Name
What the Logs Include
connector
A variety of different logs depending
upon the host, but always including
UDSAgent.log.
The Actifio Connector log includes information such as
host OS version/patch revision, HBA make/model/type,
firmware revision, CPU/Memory utilization, connector
logs, RMAN, syslog, etc.
openvpn
openvpn.log
Actifio SecureConnect communications.
openvpn.log-<date>
openvpn.log-<date>.bz2
Downloading Logs from the Actifio Appliance
Log files are a great tool for troubleshooting. You can use the Downloads tab in the Logs section of the Domain
Manager to download log files. Each downloaded log is bundled as a zip file in the target destination download folder.
You can then unzip the downloaded log files and use a program such as Notepad to view the contents of each log.
To download logs from your Actifio CDS or Sky appliance:
1.
Open the Domain Manager to System > Configuration > Logs.
2.
Click the Download tab to access the Download Logs page.
3.
By default, the Actifio appliance downloads logs from the past 3 days (starting with the current date).
To specify a particular date range from which you want to download logs, click the calendar icon and select
a start date in the From Date field and an end date in the To Date fields. The calendar icon displays the
Calendar view where you make your date selections.
4.
Select the types of logs to download from Log Type. By default, all log types are selected. If you wish to
choose the logs to download, click None to deselect all logs and then click the check boxes to select the
logs to download. Click All if you want to select all logs.
5.
Click Download Logs to download the selected log file(s), then specify the location where you want to
download the files. The Waiting to Download message indicates that the download process has started.
Log types are detailed in What You Can Learn from Actifio Logs on page 5.
| actifio.com | Troubleshooting Actifio Systems
7
Downloading Connector Logs from One of Your Hosts
Connector logs are a great tool for troubleshooting. You can use the Downloads tab in the Logs section of the Domain
Manager to download log files. Each downloaded log is a zip file in the target destination download folder.
To download logs from your Actifio CDS or Sky appliance:
1.
Open the Domain Manager to System > Configuration > Logs.
2.
Click the Download tab to access the Download Logs.
3.
By default, the Actifio appliance downloads logs from the past 3 days (starting with the current date).
To specify a particular date range from which you want to download logs, click the calendar icon and select
a start date in the From Date field and an end date in the To Date fields.
8
4.
Click None to deselect all logs, then select Connector at the bottom of the Log Type list.
5.
To download logs from the Actifio Connector installed on a specific host, click the Select Hosts drop-down
and choose from the list of available hosts.
6.
Click Download Logs to download the selected log file(s), then specify the location where you want to
download the files. The Waiting to Download message indicates that the download process has started.
Troubleshooting Actifio Systems | actifio.com |
Troubleshooting with Actifio Logs
Each service has its own log, but many of them have little value for troubleshooting. This is a simpler view of how the
critical components relate to each other.
Simplified Component Relationships Diagram for Troubleshooting
Here is an abbreviated version of the comprehensive list of logs, showing the most useful ones for troubleshooting.
Useful Actifio Logs for Troubleshooting
Service
Log File Name
What the Logs Include
adhd
adhd.log
Local and remote deduplication, replication and
communications with remote systems.
adhd.log-<date>
adhd.log-<date>.bz2
flasher
flasher.log
Creation of snapshots.
omd
omd.log
Creation and deletion of VDisks before invoking
the Flasher.
psrv
psrv.log.0
Communications with Tomcat, Actifio database,
VMware, CLI, system health, scheduling GC.
GUI/tomcat
catalina.log.<Sequence>
Communications with the Actifio Desktop and
the Actifio Resource Center. The Sequence
number increases as you go back in time.
udppm
udppm.log
Scheduling and operation of jobs.
udppm.log-<Date>
udppm.log-<Date>.bz2
host/
connector
A variety of different logs depending
upon the host, but always including
UDSAgent.log.
| actifio.com | Troubleshooting Actifio Systems
The Actifio Connector log includes information
such as host OS version/patch revision, HBA
make/model/type, firmware revision, CPU/
Memory utilization, connector logs, RMAN,
syslog, etc.
9
Sending Logs to Support By Email
To send Actifio appliance logs to Actifio Support:
1.
Open the Domain Manager to System > Configuration > Logs.
2.
Click the Email tab to access the Log Settings. If Email Notification is not configured, configure the setting
before proceeding. See the procedure in Configuring the Appliance to Communicate with an SMTP Server
on page 8.
3.
Select the types of logs to send from Log Type. To select all categories of logs, click All.
Optionally, you can specify the number of log files to be sent to the Actifio Support team in File Limit. the
Actifio appliance creates one log file for each type of log every day in the /act/logs folder. To email the log
files of the past ten days for the selected type, specify 10 in File Limit. If you do not specify the file limit, all
logs files from the /act/logs folders are sent. Set the value to 1 to receive the latest log file of the selected
type.
4.
Click Send Log Through Email to save the configuration.
Selecting Logs to Send to Actifio Support
10
Troubleshooting Actifio Systems | actifio.com |
Issues Related to Networking
and Permissions
4
To isolate a tricky problem, first rule out networking issues and issues relating to permissions on VMs.
•
The first place to look for best practices for Fibre Channel zoning and multipathing, for iSCSI networking,
and for required network ports is Network Administrator’s Guide to Actifio Copy Data Management.
•
vCenter required permissions are detailed in A VMware vCenter Administrator’s Guide to Actifio Copy
Data Management.
•
If you use Oracle, then be sure to review An Oracle DBA’s Guide to Actifio Copy Data Management.
•
If you use Microsoft SQL Server, then be sure to review An SQL Server DBA’s Guide to Actifio Copy
Data Management.
If the network and permissions are good, then check the system architecture against the tables in Reference
Architectures for Actifio Appliances, below. If the architecture is good, then go to Chapter 5, Getting Help.
Reference Architectures for Actifio Appliances
Actifio Sky and CDS appliances can be configured for different levels of security and high availability depending on
available network resources.For best results, appliances should be configured according to the following tables:
Actifio Sky Reference Architectures on page 11
Actifio CDS Generation-3 Reference Architectures on page 12
Actifio CDS Generation-4 Reference Architectures on page 13
Actifio Sky Reference Architectures
Sky
Using
Network
Security
High
Availability
Sky-1
Eth0 (1G) for all traffic
1G only virtual network
Low
Sky-2
Eth0 (1G) for management
1/10G mixed virtual
network
Medium
1/10G mixed virtual
network
High
The Sky
appliance
uses the
hypervisor’s
High
Availability
features.
Eth1 (1/10G) for backup/restore/replication
Sky-4
Eth0 (1G) for management
Eth1 (10G) for backup
Eth2 (1/10G) for replication
More Eth* for backups only if required.
| actifio.com | Troubleshooting Actifio Systems
11
Actifio CDS Generation-3 Appliances
The Actifio CDS Generation-3 appliance includes the two nodes in the middle and the batteries above and below.
An Actifio CDS Generation-3 Appliance
These are the most reliable network architectures for a CDS Generation-3 appliance:
Actifio CDS Generation-3 Reference Architectures
Type
Using
Network
Security
High Availability
3CDS-1
Eth0 (1G) for all traffic
1G only
Low
No
3CDS-2
Eth0 (1G) for management
1G only
Medium
No
1/10G
mixed
Medium
No
1/10G
mixed
High
No
1/10G
mixed
High
Yes
Eth1 (1G) for backup/restore/replication
3CDS-3
Eth0 (1G) for management
Eth2 (10G) for backup/restore/replication
3CDS-4
Eth0 (1G) for management
Eth2 (10G) for backup
Eth3 (10G) replication
3CDS-5
Eth0 (1G) for management
Eth1 (1G) for replication
Eth2/3 (10G & HA) for backup
12
Troubleshooting Actifio Systems | actifio.com |
Actifio CDS Generation-4 Appliances
The Actifio CDS Generation-4 appliance looks like this:
These are the most reliable network architectures for a CDS Generation-4 appliance:
Actifio CDS Generation-4 Reference Architectures
Type
Using
Network
Security
High Availability
4CDS-1
Eth0 (1G) for all traffic
1G only
Low
No
4CDS-2
Eth0 (1G) for management
1G only
Medium
No
1G only
Medium
No
1/10G
mixed
Medium
No
1/10G
mixed
High
No
1/10G
mixed
High
Yes
1/10G
mixed
High
Yes
1/10G
mixed
High
Yes
Eth1 (1G) for backup/restore/replication
4CDS-3
Eth0 (1G) for management
Eth1 (1G) for backup/restore
Eth2 (1G) for replication
4CDS-4
Eth0 (1G) for management
Eth2 (10G) for backup/restore/replication
4CDS-5
Eth0 (1G) for management
Eth3 (10G) for backup
Eth5 (10G) replication
4CDS-6
Eth0 (1G) for management
Eth1 (1G) for replication
Eth3/4 (10G & HA) for backup
4CDS-7
Eth0 (1G) for management
Eth3/4 (10G & HA) for backup
Eth5 (10G) for replication
4CDS-8
Eth0 (1G) for management
Eth3/4 (10G & HA) for backup
Eth5/6 (10G & HA) for replication
| actifio.com | Troubleshooting Actifio Systems
13
14
Troubleshooting Actifio Systems | actifio.com |
5
Getting Help
If you are unable to resolve the problem with the information that you have learned, then open a Support Case. This
section describes:
Creating a Support Case on page 15
Managing Open Cases on page 17
Creating a Support Case
To create a case:
1.
Log into ActifioNOW at: https://now.actifio.com.
Creating a Support Case in ActifioNOW
2.
Select the Get Help tab and then select Create a Case. Fill in the form as needed and click Submit Case.
| actifio.com | Troubleshooting Actifio Systems
15
Entering Support Case Details
16
Troubleshooting Actifio Systems | actifio.com |
Managing Open Cases
To create a case:
1.
Log into ActifioNOW at: https://now.actifio.com.
Accessing an Open Support Case in ActifioNOW
2.
Select the Get Help tab in the top banner, and then select Manage Cases. Review information about your
existing cases. Your current cases are visible under the Open Cases heading.
Managing a Case
| actifio.com | Troubleshooting Actifio Systems
17
18
Troubleshooting Actifio Systems | actifio.com |
Download PDF
Similar pages