advertisement
Troubleshooting the Avaya Pod Fx ™
Release 3.1
NN47204-700
Issue 05.01
October 2017
© 2012-2017, Avaya Inc.
All Rights Reserved.
Notice
While reasonable efforts have been made to ensure that the information in this document is complete and accurate at the time of printing, Avaya assumes no liability for any errors. Avaya reserves the right to make changes and corrections to the information in this document without the obligation to notify any person or organization of such changes.
Documentation disclaimer
“Documentation” means information published in varying mediums which may include product information, operating instructions and performance specifications that are generally made available to users of products. Documentation does not include marketing materials.
Avaya shall not be responsible for any modifications, additions, or deletions to the original published version of Documentation unless such modifications, additions, or deletions were performed by or on the express behalf of Avaya. End User agrees to indemnify and hold harmless Avaya, Avaya's agents, servants and employees against all claims, lawsuits, demands and judgments arising out of, or in connection with, subsequent modifications, additions or deletions to this documentation, to the extent made by End User.
Link disclaimer
Avaya is not responsible for the contents or reliability of any linked websites referenced within this site or Documentation provided by
Avaya. Avaya is not responsible for the accuracy of any information, statement or content provided on these sites and does not necessarily endorse the products, services, or information described or offered within them. Avaya does not guarantee that these links will work all the time and has no control over the availability of the linked pages.
Warranty
Avaya provides a limited warranty on Avaya hardware and software.
Refer to your sales agreement to establish the terms of the limited warranty. In addition, Avaya’s standard warranty language, as well as information regarding support for this product while under warranty is available to Avaya customers and other parties through the Avaya
Support website: https://support.avaya.com/helpcenter/ getGenericDetails?detailId=C20091120112456651010 under the link
“Warranty & Product Lifecycle” or such successor site as designated by Avaya. Please note that if You acquired the product(s) from an authorized Avaya Channel Partner outside of the United States and
Canada, the warranty is provided to You by said Avaya Channel
Partner and not by Avaya.
“Hosted Service” means an Avaya hosted service subscription that
You acquire from either Avaya or an authorized Avaya Channel
Partner (as applicable) and which is described further in Hosted SAS or other service description documentation regarding the applicable hosted service. If You purchase a Hosted Service subscription, the foregoing limited warranty may not apply but You may be entitled to support services in connection with the Hosted Service as described further in your service description documents for the applicable
Hosted Service. Contact Avaya or Avaya Channel Partner (as applicable) for more information.
Hosted Service
THE FOLLOWING APPLIES ONLY IF YOU PURCHASE AN AVAYA
HOSTED SERVICE SUBSCRIPTION FROM AVAYA OR AN AVAYA
CHANNEL PARTNER (AS APPLICABLE), THE TERMS OF USE
FOR HOSTED SERVICES ARE AVAILABLE ON THE AVAYA
WEBSITE, HTTPS://SUPPORT.AVAYA.COM/LICENSEINFO UNDER
THE LINK “Avaya Terms of Use for Hosted Services” OR SUCH
SUCCESSOR SITE AS DESIGNATED BY AVAYA, AND ARE
APPLICABLE TO ANYONE WHO ACCESSES OR USES THE
HOSTED SERVICE. BY ACCESSING OR USING THE HOSTED
SERVICE, OR AUTHORIZING OTHERS TO DO SO, YOU, ON
BEHALF OF YOURSELF AND THE ENTITY FOR WHOM YOU ARE
DOING SO (HEREINAFTER REFERRED TO INTERCHANGEABLY
AS “YOU” AND “END USER”), AGREE TO THE TERMS OF USE. IF
YOU ARE ACCEPTING THE TERMS OF USE ON BEHALF A
COMPANY OR OTHER LEGAL ENTITY, YOU REPRESENT THAT
YOU HAVE THE AUTHORITY TO BIND SUCH ENTITY TO THESE
TERMS OF USE. IF YOU DO NOT HAVE SUCH AUTHORITY, OR IF
YOU DO NOT WISH TO ACCEPT THESE TERMS OF USE, YOU
MUST NOT ACCESS OR USE THE HOSTED SERVICE OR
AUTHORIZE ANYONE TO ACCESS OR USE THE HOSTED
SERVICE.
Licenses
THE SOFTWARE LICENSE TERMS AVAILABLE ON THE AVAYA
WEBSITE, HTTPS://SUPPORT.AVAYA.COM/LICENSEINFO ,
UNDER THE LINK “AVAYA SOFTWARE LICENSE TERMS (Avaya
Products)” OR SUCH SUCCESSOR SITE AS DESIGNATED BY
AVAYA, ARE APPLICABLE TO ANYONE WHO DOWNLOADS,
USES AND/OR INSTALLS AVAYA SOFTWARE, PURCHASED
FROM AVAYA INC., ANY AVAYA AFFILIATE, OR AN AVAYA
CHANNEL PARTNER (AS APPLICABLE) UNDER A COMMERCIAL
AGREEMENT WITH AVAYA OR AN AVAYA CHANNEL PARTNER.
UNLESS OTHERWISE AGREED TO BY AVAYA IN WRITING,
AVAYA DOES NOT EXTEND THIS LICENSE IF THE SOFTWARE
WAS OBTAINED FROM ANYONE OTHER THAN AVAYA, AN AVAYA
AFFILIATE OR AN AVAYA CHANNEL PARTNER; AVAYA
RESERVES THE RIGHT TO TAKE LEGAL ACTION AGAINST YOU
AND ANYONE ELSE USING OR SELLING THE SOFTWARE
WITHOUT A LICENSE. BY INSTALLING, DOWNLOADING OR
USING THE SOFTWARE, OR AUTHORIZING OTHERS TO DO SO,
YOU, ON BEHALF OF YOURSELF AND THE ENTITY FOR WHOM
YOU ARE INSTALLING, DOWNLOADING OR USING THE
SOFTWARE (HEREINAFTER REFERRED TO
INTERCHANGEABLY AS “YOU” AND “END USER”), AGREE TO
THESE TERMS AND CONDITIONS AND CREATE A BINDING
CONTRACT BETWEEN YOU AND AVAYA INC. OR THE
APPLICABLE AVAYA AFFILIATE (“AVAYA”).
Avaya grants You a license within the scope of the license types described below, with the exception of Heritage Nortel Software, for which the scope of the license is detailed below. Where the order documentation does not expressly identify a license type, the applicable license will be a Designated System License. The applicable number of licenses and units of capacity for which the license is granted will be one (1), unless a different number of licenses or units of capacity is specified in the documentation or other materials available to You. “Software” means computer programs in object code, provided by Avaya or an Avaya Channel Partner, whether as stand-alone products, pre-installed on hardware products, and any upgrades, updates, patches, bug fixes, or modified versions thereto. “Designated Processor” means a single stand-alone computing device. “Server” means a Designated Processor that hosts a software application to be accessed by multiple users.
“Instance” means a single copy of the Software executing at a particular time: (i) on one physical machine; or (ii) on one deployed software virtual machine (“VM”) or similar deployment.
Licence types
Designated System(s) License (DS). End User may install and use each copy or an Instance of the Software only on a number of
Designated Processors up to the number indicated in the order.
Avaya may require the Designated Processor(s) to be identified in the order by type, serial number, feature key, Instance, location or other specific designation, or to be provided by End User to Avaya through electronic means established by Avaya specifically for this purpose.
Concurrent User License (CU). End User may install and use the
Software on multiple Designated Processors or one or more Servers, so long as only the licensed number of Units are accessing and using the Software at any given time. A “Unit” means the unit on which
Avaya, at its sole discretion, bases the pricing of its licenses and can be, without limitation, an agent, port or user, an e-mail or voice mail account in the name of a person or corporate function (e.g., webmaster or helpdesk), or a directory entry in the administrative database utilized by the Software that permits one user to interface with the Software. Units may be linked to a specific, identified Server or an Instance of the Software.
Database License (DL). End User may install and use each copy or an Instance of the Software on one Server or on multiple Servers provided that each of the Servers on which the Software is installed communicates with no more than one Instance of the same database.
CPU License (CP). End User may install and use each copy or
Instance of the Software on a number of Servers up to the number
indicated in the order provided that the performance capacity of the
Server(s) does not exceed the performance capacity specified for the
Software. End User may not re-install or operate the Software on
Server(s) with a larger performance capacity without Avaya’s prior consent and payment of an upgrade fee.
Named User License (NU). You may: (i) install and use each copy or
Instance of the Software on a single Designated Processor or Server per authorized Named User (defined below); or (ii) install and use each copy or Instance of the Software on a Server so long as only authorized Named Users access and use the Software. “Named
User”, means a user or device that has been expressly authorized by
Avaya to access and use the Software. At Avaya’s sole discretion, a
“Named User” may be, without limitation, designated by name, corporate function (e.g., webmaster or helpdesk), an e-mail or voice mail account in the name of a person or corporate function, or a directory entry in the administrative database utilized by the Software that permits one user to interface with the Software.
Shrinkwrap License (SR). You may install and use the Software in accordance with the terms and conditions of the applicable license agreements, such as “shrinkwrap” or “clickthrough” license accompanying or applicable to the Software (“Shrinkwrap License”).
Heritage Nortel Software
“Heritage Nortel Software” means the software that was acquired by
Avaya as part of its purchase of the Nortel Enterprise Solutions
Business in December 2009. The Heritage Nortel Software is the software contained within the list of Heritage Nortel Products located at https://support.avaya.com/LicenseInfo under the link “Heritage
Nortel Products” or such successor site as designated by Avaya. For
Heritage Nortel Software, Avaya grants Customer a license to use
Heritage Nortel Software provided hereunder solely to the extent of the authorized activation or authorized usage level, solely for the purpose specified in the Documentation, and solely as embedded in, for execution on, or for communication with Avaya equipment.
Charges for Heritage Nortel Software may be based on extent of activation or use authorized as specified in an order or invoice.
Copyright
Except where expressly stated otherwise, no use should be made of materials on this site, the Documentation, Software, Hosted Service, or hardware provided by Avaya. All content on this site, the documentation, Hosted Service, and the product provided by Avaya including the selection, arrangement and design of the content is owned either by Avaya or its licensors and is protected by copyright and other intellectual property laws including the sui generis rights relating to the protection of databases. You may not modify, copy, reproduce, republish, upload, post, transmit or distribute in any way any content, in whole or in part, including any code and software unless expressly authorized by Avaya. Unauthorized reproduction, transmission, dissemination, storage, and or use without the express written consent of Avaya can be a criminal, as well as a civil offense under the applicable law.
Virtualization
The following applies if the product is deployed on a virtual machine.
Each product has its own ordering code and license types. Note that each Instance of a product must be separately licensed and ordered.
For example, if the end user customer or Avaya Channel Partner would like to install two Instances of the same type of products, then two products of that type must be ordered.
Third Party Components
“Third Party Components” mean certain software programs or portions thereof included in the Software or Hosted Service may contain software (including open source software) distributed under third party agreements (“Third Party Components”), which contain terms regarding the rights to use certain portions of the Software
(“Third Party Terms”). As required, information regarding distributed
Linux OS source code (for those products that have distributed Linux
OS source code) and identifying the copyright holders of the Third
Party Components and the Third Party Terms that apply is available in the products, Documentation or on Avaya’s website at: https:// support.avaya.com/Copyright or such successor site as designated by Avaya. The open source software license terms provided as Third
Party Terms are consistent with the license rights granted in these
Software License Terms, and may contain additional rights benefiting
You, such as modification and distribution of the open source software. The Third Party Terms shall take precedence over these
Software License Terms, solely with respect to the applicable Third
Party Components to the extent that these Software License Terms impose greater restrictions on You than the applicable Third Party
Terms.
The following applies only if the H.264 (AVC) codec is distributed with the product. THIS PRODUCT IS LICENSED UNDER THE AVC
PATENT PORTFOLIO LICENSE FOR THE PERSONAL USE OF A
CONSUMER OR OTHER USES IN WHICH IT DOES NOT RECEIVE
REMUNERATION TO (i) ENCODE VIDEO IN COMPLIANCE WITH
THE AVC STANDARD (“AVC VIDEO”) AND/OR (ii) DECODE AVC
VIDEO THAT WAS ENCODED BY A CONSUMER ENGAGED IN A
PERSONAL ACTIVITY AND/OR WAS OBTAINED FROM A VIDEO
PROVIDER LICENSED TO PROVIDE AVC VIDEO. NO LICENSE IS
GRANTED OR SHALL BE IMPLIED FOR ANY OTHER USE.
ADDITIONAL INFORMATION MAY BE OBTAINED FROM MPEG LA,
L.L.C. SEE HTTP://WWW.MPEGLA.COM
.
Service Provider
THE FOLLOWING APPLIES TO AVAYA CHANNEL PARTNER’S
HOSTING OF AVAYA PRODUCTS OR SERVICES. THE PRODUCT
OR HOSTED SERVICE MAY USE THIRD PARTY COMPONENTS
SUBJECT TO THIRD PARTY TERMS AND REQUIRE A SERVICE
PROVIDER TO BE INDEPENDENTLY LICENSED DIRECTLY FROM
THE THIRD PARTY SUPPLIER. AN AVAYA CHANNEL PARTNER’S
HOSTING OF AVAYA PRODUCTS MUST BE AUTHORIZED IN
WRITING BY AVAYA AND IF THOSE HOSTED PRODUCTS USE
OR EMBED CERTAIN THIRD PARTY SOFTWARE, INCLUDING
BUT NOT LIMITED TO MICROSOFT SOFTWARE OR CODECS,
THE AVAYA CHANNEL PARTNER IS REQUIRED TO
INDEPENDENTLY OBTAIN ANY APPLICABLE LICENSE
AGREEMENTS, AT THE AVAYA CHANNEL PARTNER’S EXPENSE,
DIRECTLY FROM THE APPLICABLE THIRD PARTY SUPPLIER.
WITH RESPECT TO CODECS, IF THE AVAYA CHANNEL
PARTNER IS HOSTING ANY PRODUCTS THAT USE OR EMBED
THE G.729 CODEC, H.264 CODEC, OR H.265 CODEC, THE
AVAYA CHANNEL PARTNER ACKNOWLEDGES AND AGREES
THE AVAYA CHANNEL PARTNER IS RESPONSIBLE FOR ANY
AND ALL RELATED FEES AND/OR ROYALTIES. THE G.729
CODEC IS LICENSED BY SIPRO LAB TELECOM INC. SEE
WWW.SIPRO.COM/CONTACT.HTML
. THE H.264 (AVC) CODEC IS
LICENSED UNDER THE AVC PATENT PORTFOLIO LICENSE FOR
THE PERSONAL USE OF A CONSUMER OR OTHER USES IN
WHICH IT DOES NOT RECEIVE REMUNERATION TO: (I) ENCODE
VIDEO IN COMPLIANCE WITH THE AVC STANDARD (“AVC
VIDEO”) AND/OR (II) DECODE AVC VIDEO THAT WAS ENCODED
BY A CONSUMER ENGAGED IN A PERSONAL ACTIVITY AND/OR
WAS OBTAINED FROM A VIDEO PROVIDER LICENSED TO
PROVIDE AVC VIDEO. NO LICENSE IS GRANTED OR SHALL BE
IMPLIED FOR ANY OTHER USE. ADDITIONAL INFORMATION
FOR H.264 (AVC) AND H.265 (HEVC) CODECS MAY BE
OBTAINED FROM MPEG LA, L.L.C. SEE HTTP://
WWW.MPEGLA.COM
.
Compliance with Laws
You acknowledge and agree that it is Your responsibility for complying with any applicable laws and regulations, including, but not limited to laws and regulations related to call recording, data privacy, intellectual property, trade secret, fraud, and music performance rights, in the country or territory where the Avaya product is used.
Preventing Toll Fraud
“Toll Fraud” is the unauthorized use of your telecommunications system by an unauthorized party (for example, a person who is not a corporate employee, agent, subcontractor, or is not working on your company's behalf). Be aware that there can be a risk of Toll Fraud associated with your system and that, if Toll Fraud occurs, it can result in substantial additional charges for your telecommunications services.
Avaya Toll Fraud intervention
If You suspect that You are being victimized by Toll Fraud and You need technical assistance or support, call Technical Service Center
Toll Fraud Intervention Hotline at +1-800-643-2353 for the United
States and Canada. For additional support telephone numbers, see the Avaya Support website: https://support.avaya.com
or such successor site as designated by Avaya.
Security Vulnerabilities
Information about Avaya’s security support policies can be found in the Security Policies and Support section of https:// support.avaya.com/security .
Suspected Avaya product security vulnerabilities are handled per the
Avaya Product Security Support Flow ( https:// support.avaya.com/css/P8/documents/100161515 ).
Downloading Documentation
For the most current versions of Documentation, see the Avaya
Support website: https://support.avaya.com
, or such successor site as designated by Avaya.
Contact Avaya Support
See the Avaya Support website: https://support.avaya.com
for product or Hosted Service notices and articles, or to report a problem with your Avaya product or Hosted Service. For a list of support telephone numbers and contact addresses, go to the Avaya Support website: https://support.avaya.com
(or such successor site as designated by Avaya), scroll to the bottom of the page, and select
Contact Avaya Support.
Trademarks
Avaya Aura ® is a registered trademark of Avaya Inc.
The trademarks, logos and service marks (“Marks”) displayed in this site, the Documentation, Hosted Service(s), and product(s) provided by Avaya are the registered or unregistered Marks of Avaya, its affiliates, its licensors, its suppliers, or other third parties. Users are not permitted to use such Marks without prior written consent from
Avaya or such third party which may own the Mark. Nothing contained in this site, the Documentation, Hosted Service(s) and product(s) should be construed as granting, by implication, estoppel, or otherwise, any license or right in and to the Marks without the express written permission of Avaya or the applicable third party.
Avaya is a registered trademark of Avaya Inc.
All non-Avaya trademarks are the property of their respective owners.
Linux ® is the registered trademark of Linus Torvalds in the U.S. and other countries.
Contents
Avaya Pod Orchestration Suite Release 3.1
.............................................................................. 9
Avaya Pod Fx upgrades to Release 3.1
................................................................................... 10
Configuring EMC VNXe3200 monitoring and EMC Secure Remote Support ............................... 14
Configuring EMC VNX5300 monitoring and EMC Secure Remote Support ................................ 15
®
Communication Manager in an Avaya Aura
®
Recovering from a failed VCSA 5.5 upgrade
............................................................................ 42
............................................................................................ 43
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
5
Contents
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
6
Chapter 1: Introduction
Purpose
This document contains advice, solutions, and procedures for dealing with common issues encountered while using the Avaya Pod Fx. This document is intended for the following audience:
• end users
• Avaya certified technicians
• Avaya partners
• Avaya Sales Engineer (SE)
• Avaya System Integrator (SI)
Product registration
To prevent service interruption, you must register your Avaya Pod Fx products.
Following are the available methods for product registration:
• Implementation as a service. If Avaya Professional Services provided implementation services on site, Avaya Professional Services also performs product registration on your behalf.
• Avaya Partner and Customer implementation. For information about the step-by-step registration process, see the Avaya Classic Global Registration Process Help Document on the
Avaya Product Support Registration page. The document ID is 100162279.
Product registration is a required element for effective Avaya customer support. You must follow the
Partner and Customer Guidance in the Avaya Global Registration Process to help ensure the seamless support you have come to expect from Avaya.
Warranty
Avaya provides a limited warranty on its Hardware and Software (“Product(s)”).
Refer to your sales agreement to establish the terms of the limited warranty.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
7
Introduction
The standard warranty language for Avaya, as well as information regarding support for this Product while under warranty, is available to Avaya customers and other parties through the Avaya Support website: http://support.avaya.com
.
Note:
If you acquired the Product(s) from an authorized Avaya reseller outside of the United States and Canada, the warranty is provided to you by said Avaya reseller and not by Avaya.
Example
1. Go to the Avaya Support website at http://support.avaya.com
.
2. In the Global search field, type warranty , to search the Avaya Knowledge Base for warranty topics.
The system displays a list of all warranty topics.
3. Click the relevant warranty topic.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
8
Chapter 2: New in this document
The following sections outline what is new in Troubleshooting the Avaya Pod Fx (NN47204–700).
VMware vSphere 6.0 Update 3B
Avaya Pod Fx Release 3.1 uses VMware vSphere 6.0 vCenter Server Appliance Update 3B for all
Avaya Pod Fx platforms.
Compute servers are the physical servers, operating as VMware ESXi hosts. Avaya Pod Fx component applications are virtualized and run on a VMware High Availability (HA) cluster of ESXi hosts.
VMware vCenter Server Appliance (VCSA) provides management for the HA clusters, vMotion, and
Dynamic Resource Schedule (DRS) rules for the ESXi hosts in the Avaya Pod Fx.
Avaya Pod Orchestration Suite Release 3.1
Avaya Pod Orchestration Suite (POS) is a bundle of visualization and management applications required to configure, orchestrate, manage, and monitor your Avaya Pod Fx. POS Release 3.1
applications include the following enhancements:
• Support for Avaya Aura ® 7.1 applications.
• Updates to overall platform security.
New Avaya Pod Fx Release 3.1 system builds provide the following POS application versions predeployed. Existing systems require an upgrade to Avaya Pod Fx Release 3.1, and the following
POS versions:
• Pod Visualization Manager (PVM) Release 3.1.
• Visualization Performance and Fault Monitoring (VPFM) Release 3.1.
• Management Server Console (MSC) Release 3.1
Important:
The following applications have been removed from the POS software starting in Release 3.1:
• Configuration and Orchestration Manager (COM)
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
9
New in this document
• Virtual Provisioning System (VPS)
• IP Flow Manager (IPFM)
• Pod Utility Manager (PUM)
These applications are end of sale and end of support in Release 3.1.
Avaya Pod Fx upgrades to Release 3.1
Preexisting Avaya Pod Fx platforms can upgrade to Release 3.1 software. The following software upgrade scenarios are supported:
• One step upgrade path for all Avaya Pod Fx platforms from Release 2.0.1, 2.1.1, 3.0, 3.0.1, or
3.0.2 to Release 3.1.
Important:
The Avaya Pod Utility Module has been removed from the Management Server Console software in Release 3.1. All upgrades are now conducted using the procedures documented in
Using the Management Server Console for Avaya Pod Fx (NN47204–303).
Avaya Pod Fx upgrades must be performed by Avaya Professional Services, or an Avaya Pod Fx
Select certified Business Partner. They must plan and prepare to perform the upgrades, such as downloading and transferring all the upgrade files required before starting any component upgrades.
To perform an Avaya Pod Fx upgrade, update the following components to the Release 3.1 software baseline:
• Avaya Aura ® System Manager
• Avaya Pod Orchestration Suite (POS) software
• VMware vCenter and ESXi software
• Avaya network switches software and firmware
• EMC storage array software and firmware
• HPE Nimble storage array software and firmware
• HPE or Lenovo compute servers BIOS and firmware
• ServerTech Power Distribution Unit (PDU) firmware
• Remove Unisphere Remote and deploy Unisphere Central if applicable
Warning:
Avaya Pod Fx Release 3.1 requires an Avaya Aura 7.1 System Manager. Before performing any upgrades, verify if your existing solution applications are supported on Avaya Aura 7.1 using the product compatibility matrix available at https://secureservices.avaya.com/compatibility-matrix/ menus/product.xhtml
.
You can upgrade ACM, and ANAV to the xCaaS 2.0 supported versions before performing an upgrade of the Avaya Pod Fx infrastructure to Release 3.1.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
10
HPE Nimble CS1000 Array
For more information about performing Avaya Pod Fx infrastructure upgrades, see Using the
Management Server Console for Avaya Pod Fx (NN47204–303) (Available for Avaya authorized professionals and Avaya Pod Fx Select certified Business Partners only).
It is recommended that you perform the Avaya Pod Fx Configurator Design Review process in the event of an upgrade. This is recommended to validate the updated software line up for the applications does not require additional resources in the Avaya Pod Fx such as additional storage or servers. To schedule a design review, email [email protected]
.
HPE Nimble CS1000 Array
Release 3.1 introduces the HPE Nimble CS1000 hybrid storage array as the storage solution for all new Avaya Pod Fx platforms.
The HPE Nimble CS1000 storage array contains redundant disk controllers, redundant power, networking ports, and expansion shelves for the storage system. The CS1000 Array can accommodate up to 2 ES2–H expansion shelves in Release 3.1.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
11
Chapter 3: Configuring monitoring and alerts
This section contains information about the configuration of monitoring and alerts for the elements of the Avaya Pod Fx.
Related links
Configuring Nimble CS1000 monitoring
on page 12
on page 13
Configuring EMC VNXe3200 monitoring and EMC Secure Remote Support on page 14
Configuring EMC VNX5300 monitoring and EMC Secure Remote Support
on page 15
Configuring alarm notifications on page 17
Configuring Nimble CS1000 monitoring
About this task
Use the following procedure to configure diagnostic monitoring on the Nimble CS1000.
Procedure
1. Open a new browser window or tab.
2. Enter the management IP address of the storage array.
3. Log in to the management interface with the Administrator credentials.
4. Select Administration > Alerts and Monitoring > Diagnostics from the menu.
5. Select Allow Nimble Storage Support to collect analytics data automatically from the array .
6. Select Enable Secure Tunnel .
7. Click Test Diagnostics for Nimble Analytics Settings .
A test is performed on the diagnostic settings. A checkmark should appear beside every item in the test window. If the test fails, ensure that your firewall is allowing traffic from the
Management IP and Controller Support IP addresses. Run the test once again. Troubleshoot and run the test until all items in the test pass.
8. Click Send Diagnostics for Nimble Analytics .
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
12
Nimble CS1000 and VPFM alarms
A test email will be sent to the address configured for the array. Confirm the email configuration settings if an email is not received at the anticipated address.
9. Click Save .
Related links
Configuring monitoring and alerts
on page 12
Nimble CS1000 and VPFM alarms
About this task
The Nimble CS1000 does not currently send alarms or notifications to VPFM. Configure email alerts to receive alarms and notifications from the device.
Use the following procedure to configure email alerts.
Procedure
1. Open a new browser window or tab.
2. Enter the management IP address of the storage array.
3. Log in to the management interface with the Administrator credentials.
4. Select Administration > Alerts and Monitoring > Email from the menu.
5. Enter a reporting email address in the Send From Address section.
Note:
This address is displayed in the From field of an email when an alarm or notification is sent from the array. This does not have to be a valid email address. However, the address should provide enough information to be clearly identifiable to the recipient.
For example, a corporation has 2 Pods named ExampleCorpCallCenter and
ExampleCorpSAN each with one storage array. The administrator could enter [email protected]
when configuring each device to identify where an alarm or notification originates.
6. Click Add in the Send To Addresses section.
7. Enter an email address to send alarms and notifications.
8. Repeat the previous two steps for each address to receive alarms and notifications.
9. Select Send event data to Nimble Storage Support if you wish to also send this information directly to Nimble Support.
10. Enter the information for your corporate SMTP server SMTP Server section.
11. Set the level at which alarms and notifications are sent in the Alert Options section.
12. Click Test to test the settings.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
13
Configuring monitoring and alerts
13. Click Save .
Related links
Configuring monitoring and alerts
on page 12
Configuring EMC VNXe3200 monitoring and EMC Secure
Remote Support
About this task
Use the following procedure to configure EMC monitoring and EMC Secure Remote Support
(ESRS) on the EMC VNXe3200.
Before you begin
• Work closely with the customer and local EMC support team to implement ConnectEMC and
ESRS and to discuss the implementation and Internet security issues. The customer system must be registered in the EMC database by the EMC support team.
• Work with the customer to ensure the port requirements described here are met: https:// support.emc.com/
DOCU55301_EMC_SECURE_REMOTE_SERVICES_RELEASE_3.08_PORT_REQUIREMEN
TS.PDF?LANGUAGE=EN_US&LANGUAGE=EN_US .
• Valid EMC support credentials.
• Customer’s SMTP IP address (alias).
Important:
You must schedule EMC support to complete these tasks.
Procedure
1. Access EMC Unisphere by opening the IP address of the device in a browser tab or from the
PVM storage component view.
2. Enter the username and password in the provided fields.
3. Click Login .
4. On the dashboard, navigate to Settings > Alert Settings .
5. Select the checkboxes entitled Enable ConnectEMC and Include Customer Contact
Information .
6. In the ConnectEMC SMTP Server field, enter the customer’s SMTP IP address.
7. In the Email Alerts section, add the email addresses of the individuals that will receive email notifications.
8. In the Severity level of alerts to send field, set the severity level that will trigger emails to be sent.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
14
Configuring EMC VNX5300 monitoring and EMC Secure Remote Support
9. Click Apply .
10.
(Optional) On the dashboard, navigate to Settings > EMC Secure Remote Support .
11.
(Optional) Click Add Credentials .
12.
(Optional) Enter the EMC support credentials.
13.
(Optional) Click Apply .
14.
(Optional) Select the Enable ESRS check box.
15.
(Optional) Configure a proxy server or policy manager, as required.
Related links
Configuring monitoring and alerts
on page 12
Configuring EMC VNX5300 monitoring and EMC Secure
Remote Support
About this task
Use the following procedure to configure EMC monitoring and EMC Secure Remote Support
(ESRS) on the EMC VNX5300.
Before you begin
• Work closely with the customer and local EMC support team to implement ConnectEMC and
ESRS and to discuss the implementation and Internet security issues. The customer system must be registered in the EMC database by the EMC support team.
• Work with the customer to ensure the port requirements described here are met: HTTPS://
SUPPORT.EMC.COM/
DOCU55301_EMC_SECURE_REMOTE_SERVICES_RELEASE_3.08_PORT_REQUIREMEN
TS.PDF?LANGUAGE=EN_US&LANGUAGE=EN_US .
• Valid EMC support credentials.
• Customer’s SMTP IP address (alias).
• Work with EMC and the customer to decide which version of ESRS should be used:
- ESRS in Unisphere for a single system.
- ESRS VE for multiple EMC VNX systems. An HA option for ESRS VE is also available.
Important:
You must schedule EMC support to complete these tasks.
Procedure
1. Access EMC Unisphere by opening the IP address of the device in a browser tab or from the
PVM storage component view.
2. Log in with the required username and password credentials.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
15
Configuring monitoring and alerts
3. Click Login .
4. Select the system serial number to open the dashboard.
5. On the dashboard, navigate to System > Service Tasks .
6. Click Manage Customer Contact to set the customer contact information.
7. Update the contact information as required. Mandatory fields are noted with an asterisk.
8. Click Update .
9. Click Manage ConnectEMC .
10. In the Email section, complete the fields as follows: a. Set the Email Priority field to Primary .
b. Set the Email Server (SMTP) field to the customer SMTP server IP address.
c. Set the Subject field to ConnectEMC .
d. Set the Recipient Address(es) field to [email protected]
.
e. Set the Send Email Address field to
<System_Serial_Number>@<customer_domain> .
f. Click OK .
11. Click Manage ESRS .
12. In the Support Credentials section, enter the customer EMC credentials.
13. Click OK.
14. Ensure the Credentials Type is set to EMC Online Support .
15. Click OK .
16. Click Yes when prompted to continue with the operation.
17. On the dashboard, navigate to System > Notifications .
18. Click the Notification Templates tab.
19. Press CTRL+SHIFT+F12 on the keyboard.
20. Enter the password messner when prompted.
21. Click OK .
22. Select the latest version of the Call_Home_Template from the provided list.
23. Select Explicit Events from the Events section.
24. Select the ConnectEMC tab.
25. Select Connect Home .
26. Click OK .
27. Select the Distributed Monitors tab.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
16
Configuring alarm notifications
28. Select a device from the list.
29. Click Use Template .
30. Select the template you previously edited.
31. Click Send Test Event .
32. Verify with EMC support personnel the test event was received successfully.
Related links
Configuring monitoring and alerts
on page 12
Configuring alarm notifications
The following section contains procedures required to allow Avaya Pod Fx element events to trigger a response in VPFM. In this example, the response will be to send an email message to the Avaya
Pod Fx operator(s) when IP availability is lost to an Avaya Pod Fx element. The Avaya Pod Fx operator can specify which element events will trigger an alert, and configure the trigger requirements.
Related links
Configuring monitoring and alerts
on page 12
Configuring VPFM to monitor Avaya Pod Fx elements
on page 17
Creating an email action on page 18
Configuring SNMP traps for the SAL Gateway on page 21
Configuring VPFM to monitor Avaya Pod Fx elements
About this task
Use this procedure to configure VPFM to monitor network elements on the domain.
Before you begin
Ensure that Avaya Pod Fx elements have been discovered by VPFM and that the VPFM Domain holding those elements is enabled for Monitoring. If new elements are being added to the Avaya
Pod Fx solution, you must rediscover your network so they will be monitored.
Procedure
1. Click Configurations > Monitoring .
2.
Click Add a new monitoring configuration .
3. Enter the name Avaya CPod Managed Devices and click OK .
4. In These elements click Everything .
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
17
Configuring monitoring and alerts
5.
The Choose a Scope screen appears.
Click Show Private Scopes .
6. Type Avaya Pod Managed Devices in the search box, select the scope then click OK .
7. In Monitor for these information types select Monitored Information .
All boxes should now display a green arrow.
Note:
By default the monitoring data will be retained for 30 days, and the polling interval will be once every ten minutes. You can adjust this as needed and the smallest polling interval is 1 minute. However, Avaya recommends leaving the polling interval at the default value.
8.
Click Apply your changes
Related links
Configuring alarm notifications on page 17
.
Creating an email action
About this task
Use this procedure to create an email action that will inform the Avaya Pod Fx operator of any alarms that are triggered during VPFM monitoring of the Avaya Pod Fx network elements.
Before you begin
Configure VPFM to monitor network elements.
Procedure
1. Navigate to Configurations > Actions.
. Highlight Sample Email Action and click Clone selected action . Enter the name Pod Fx IP Availability Email Action then click
OK .
2. Enter an email alias address in the To: field. You can customize the alias in the From: field or leave it as default.
You can enter multiple email addresses, each separated by a semi-colon.
3. Click the Primary SMTP tab, enter the SMTP hostname or IP address for your environment.
If necessary for your environment, enter the optional information for SMTP username, SMTP password, and SMTP Port, if not default port 25.
4. Select Use SSL if SSL is used in your environment.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
18
Configuring alarm notifications
5. If you have a backup SMTP server, complete the configuration for that tab. Click Apply your changes when finished.
Next steps
To test the email action, click Test action (green right-facing arrow). If the connection to your SMTP server is successful, the system displays a pop-up window stating Action executed successfully . The alias email account(s) will also receive a generic email message from the test.
Related links
Configuring alarm notifications on page 17
Creating a response configuration
About this task
Use this procedure to create a response configuration for all Avaya Pod Fx managed devices.
Procedure
1. Navigate to Actions > By Event Response . Click Add a new response . Enter the name
Pod Fx IP Availability Response . Click OK .
2. In These Elements click Everything .
The system displays the Choose a Scope screen.
3. Click Show Private Scopes
4. Enter Avaya Pod Managed Devices in the search box, select the scope then click OK .
5. On the Domain Elements & Event Types tab, select Response applies to Event Types .
6. Expand Availability Event . Expand Availability Problem . Expand Availability Failure .
Expand IP Availability Failure . Select IP Availability Failure . Clear Phone IP Disconnect .
The tree should look like the following diagram:
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
19
Configuring monitoring and alerts
7. Click the Actions to Execute tab and then click Response is triggered when . Select An event is posted .
8. In Execute the following actions , select Pod Fx IP Availability Email Action .
9. Click Apply your changes button .
Example
After you have completed all of the previous configuration steps, if an Avaya Pod Fx managed element is polled and does not respond, the operator(s) will receive an email similar to the following example:
From: [email protected] [mailto:[email protected]]
Sent: Thursday, April 21, 2016 2:16 PM
To: Pod Fx Operators
Subject: IPAvailabilityFailure on vnx-avstor-1
Domain: tpod_4200
Network Element: vnx-avstor-1
Address: 10.10.8.8
Event Type: IPAvailabilityFailure
Time: Thu Apr 21 14:09:00 PDT 2016
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
20
Configuring alarm notifications
Trigger: posted
Message: unknown IP availability failure on vnx-avstor-1.
Error Description: unknown IP availability failure on vnx-avstor-1.
Configuring SNMP traps for the SAL Gateway
About this task
Use this task to configure elements of the Avaya Pod Fx to send SNMP traps to the SAL Gateway.
Procedure
1. Configure Avaya Virtual Services Platform 4000 Series switches to send SNMP traps.
a. Start an SSH session with the switch.
b. Log in as the admin user.
c. Use the enable command to enter Privileged mode.
d. Use the config t command to enter Configuration mode.
e. Use the snmp-server host <gateway_ip_address> v2c readview command to add the SAL Gateway as a trap receiver.
f. Use the save config command to save the configuration.
g. End the SSH session.
h. Logout.
2. Configure Avaya Virtual Services Platform 7200 Series switches to send SNMP traps.
a. Start an SSH session with the switch.
b. Log in as the admin user.
c. Use the enable command to enter Privileged mode.
d. Use the config t command to enter Configuration mode.
e. Use the snmp-server host <gateway_ip_address> v2c readview command to add the SAL Gateway as a trap receiver.
f. Use the save config command to save the configuration.
g. End the SSH session.
h. Logout.
3. Configure Avaya Virtual Services Platform 7000 Series switches to send SNMP traps.
a. Start an SSH session with the switch.
b. Log in as the admin user.
c. Use the config t command to enter Configuration mode.
d. Use the snmp-server host <gateway_ip_address> avaya123 command to add the SAL Gateway as a trap receiver.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
21
Configuring monitoring and alerts e. Use the save config command to save the configuration.
f. End the SSH session.
g. Logout.
4. Configure PDUs to send SNMP traps.
a. Log in as the admin user to the HTTPS web interface.
b. Select Configuration > SNMP/Thresholds from the menu.
c. Locate the field Trap Destination 2 .
d. Enter the SAL Gateway IP address in the field.
e. Click Apply .
f. Logout.
5. Configure the vCenter Server Appliance to send SNMP traps.
a. Log in to the VCSA web client located at: https://<VCSA-IP-ADDRESS>:9443/vsphereclient/ .
b. Select the applicable vCenter instance.
c. Select Manage > Settings > General from the menu.
d. Click Edit .
e. Select SNMP receivers .
f. Enter the SAL Gateway IP address in the next available receiver field.
g. Verify that it is enabled.
h. Click OK .
i. Logout.
6. Configure the EMC VNXe3200 to send SNMP traps.
a. Log in as the admin user to the HTTPS web interface. The web interface is located: https://EMC_MGMT_IP_ADDRESS .
b. Select Settings > More configuration > Alert Settings from the menu.
c. Click Add under SNMP alerts .
d. Enter the IP address of the SAL Gateway.
e. Enter the user name avaya123 .
f. Select None for the Authentication Protocol .
g. Click OK .
h. Logout.
7. Configure HP iLO to send SNMP traps.
a. Log in to the web interface of the HP iLO. The web interface is located at: https://
HP_iLO_IP_ADDRESS .
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
22
Configuring alarm notifications b. Select Adminstration > Management from the menu.
c. Enter the SAL GW IP address as a new SNMP Alert Destination .
d. Click Apply .
e. Logout.
8. Configure Lenovo TMM to send SNMP traps.
a. Log in to the HTTPS web interface.
b. Select Server Information > Event Management > Platform Events from the menu.
c. Select the Global Alerting Enable checkbox.
d. Select Generate PET for each of the filter names.
e. Click Apply Changes .
f. Select Server Information > Event Management > Trap Settings from the menu.
g. Enter and enable the next available destination.
Note:
A test trap can be sent to verify it is enabled.
h. Click Apply Changes .
i. Logout.
Related links
Configuring alarm notifications on page 17
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
23
Chapter 4: Troubleshooting general issues
This section contains general information and tips for troubleshooting issues.
Related links
on page 24
on page 28
Troubleshooting with PVM on page 31
General troubleshooting
VSP 4000 unit failure
In the event of a VSP 4000 Series switch unit failure, do not perform a hot swap. For assistance, contact Avaya Support.
Note:
There is no emergency shutdown procedure for VSP 4000 Series switches.
SMLT Basic Troubleshooting Checks
Begin any SMLT troubleshooting with the following checks:
• Check that the IST is up and running.
• Check that SMLT is up and running.
• Check the autotopology table for possible loops.
EMC storage array
EMC recommends that you enable Connect EMC support service. This feature automatically sends an email message to EMC support when problems occur that affect service or require user action.
You can enable Connect EMC functionality in the Unisphere Configuration wizard Product Support
Option by selecting the Yes option and specifying the network address of an SMTP email server.
You can also enable the service from the Alert Settings page.
Tip:
Booting up and shutting down the EMC VNXe 3200 can take approximately 30 minutes. Please be patient.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
24
General troubleshooting
Pod Orchestration Suite Applications
Ensure your FQDN does not exceed a maximum of 35 characters to avoid a system error message during logon.
If a blank page appears during login ensure that you are using one of the following supported browsers:
• Internet Explorer 8.x, 9.x, or 10.x
• Firefox 19.0, 20.0, or 21.0
Note:
This problem is seen only when upgrading POS applications.
Virtualization Performance Fault Manager
If VPFM is installed properly, the network was discovered and the trap viewer shows that traps were received all events appear on the events browser. If no events appear, ensure that monitoring is turned on.
MLT configuration problem
The VPFM event browser displays a MLT configuration problem message after a discovery is run on a VPFM domain. The message is a low priority (level 4) and warns the NOC operator of potential
MLT misconfigurations. To understand the reason for the message click the Attribute tab. The
Attribute tab displays the details.
MLT configuration problem messages can mean that a port is configured as an MLT but the link is not physically there or VPFM could not find a device on the other end.
If this occurs, ensure that the MLT links for the devices is configured and connected correctly, and then acknowledge the event.
Managed device warning
As a general warning that an IP address was found in the SONMP, LLDP, or CDP table of a discovered device, but the IP object did not respond, VPFM displays the following message: potential managed device <IPO: 172.30.10.23> was not explored during
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
25
Troubleshooting general issues discovery . When this occurs check that the credentials are correct and then use the MIB browser to query the device.
VPFM monitoring
When the domain state of an agent is set to monitoring the agents enter a running state and receive VPFM monitoring requests. If the domain monitoring state and the agent run state are green the agent is monitoring both.
If a user stops an agent, VPFM monitoring details will be stopped on the agent and can only be restarted by stopping and restarting monitoring for the whole domain.
Note:
It is not recommended to start and stop individual agents.
Pod Utility Module
All upgrade files must be provided in order for the VMs to be upgraded.
Note:
Upgrades are supported for the POS vApps, for all other vApps, manual upgrade is recommended.
See Using the Management Server Console for Avaya Pod Fx for more information on prerequisites.
Pod Visualization Manager
Pod Visualization Manager (PVM) discovers all physical components and builds a view of the Avaya
Pod Fx. If you change a component or physically move a component to another rack, ensure that you update the PVM profile to reflect those changes and then use the Rediscover function. This updates the profile with the current component information.
Tip:
Always have a current profile Inventory backup. A backup can be used at a later date to recover from accidental data loss or administrator error.
Note:
A red indicator on the PVM dashboard indicates the value of the component has surpassed the threshold value set for this component in the Threshold Table on the PVM Preferences page.
High Availability warnings in vSphere client
Right-click each Host in the cluster and select Reconfigure for HA if High Availability warnings are encountered in vSphere client.
Unexpected Virtual Machine behavior
Check Datastore access for the Virtual Machine host in vSphere client if the Virtual Machine behaves in an unexpected manner. If the Application Datastore cannot be reached or the information displayed is incorrect, select Rescan All . You should also check for snapshots and remove any that are present.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
26
General troubleshooting
Note:
At least one path between the Virtual Machine and storage must be maintained at all times. It may be necessary to take the additional step of resetting the Virtual Machine if the Rescan All option does not fix the issue.
Unregistered network connections
Disruptions in network connectivity to the iSCSI ports on the ESXi hosts can cause ports to lose their registration to the target on the storage system. Select Host > Configuration > Storage adapters > iSCSI software adapter > Rescan All from the vCenter menu to rescan the datastore.
Rescanning the datastore should reestablish the port connections.
Recovering the MSC Administrator password
If the Administrator password is changed from the default and subsequently lost, the only way to recover access to the MSC is to reinstall the MSC OVA as documented in Using the Management
Server Console .
vCenter does not boot with error
See the following troubleshooting procedures if the vCenter Server Appliance does not boot with a fsck failed error: https://kb.vmware.com/selfservice/microsites/search.do?
language=en_US&cmd=displayKC&externalId=2081464#EnableBASH .
vCenter inventory is empty
Check the size of the ldapmessages file if the vCenter inventory is empty. The ldapmessages file does not have any limits on the file size and can grow until it is the size of the partition. Delete the file and restart all services if this file has consumed all resources on the partition. The vCenter inventory will return to normal after the restart. Monitor the size of this file going forward and delete as necessary to prevent the issue from happening in the future.
Performance and latency issues
The following are common causes of performance and latency issues on the Avaya Pod Fx. Perform these troubleshooting tasks before contacting support.
• Resource reservations
Search for and remove all resource reservations.
• Snapshots
Search for and remove all snapshots.
• Deployment details do not match initial sizing
Performance and latency can be effected if the applications currently deployed on the Avaya
Pod Fx do not match the approved design. Obtain the Lifecycle Customer Workbook and approved design documentation. This represents the agreed design of the Avaya Pod Fx and is the documentation used to build it. If the applications currently running on the Avaya Pod Fx outstrip the originally designed capabilities of it, performance will be effected.
HP DL360 10GB vmnic in half-duplex mode
There are instances when a HP DL360 server starts up with a 10GB vmnic in half-duplex mode when it is connected to a VSP 7254XSQ.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
27
Troubleshooting general issues
Access the vCenter command line interface and use the following commands to fix this issue.
esxcli network nic down -n vmnic<X> esxcli network nic up -n vmnic<X>
Where <X> is the number of the vmnic to be fixed.
Related links
Troubleshooting general issues on page 24
Quick Fixes
Application
Pod Visualization Manager
Problem Description
The application does not open when launched or produces a “SA not accessible” warning.
Solution or Workaround
1. The jboss application server may be down. Check for status and restart jboss using the following commands as root user:
• On the PVM VM check the jboss status using the command service jboss status .
•
• Restart jboss using the command service jboss restart .
2. The Serviceability Agent may be down. Check for status and restart spirit agent using the following commands as root user:
• On the PVM VM check the spirit agent status using the command service spiritAgent status .
• Restart spirit agent using the following commands: service spiritAgent stop service snmpd restart service spiritAgent start
Table continues…
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
28
Application
Pod Visualization Manager
Pod Visualization Manager
Quick Fixes
Problem Description
CPU/Temperature/Memory on the
PVM Dashboard shows “—” for some IP addresses.
PVM discovery is not operating as expected.
Solution or Workaround
The SNMP credentials or ssh username or password combination is incorrect.
1. Ensure that PVM is able to reach the devices and SNMP credentials are configured.
2. Check that the ssh username and password are entered correctly in the Credentials
Database.
3. Ensure that the polling interval setting for the device is set to “No Polling” and “N” seconds in the preferences screen.
You can stop or restart the
Serviceability Agent (SA) using the
Administrative toolbar
Diagnostics > Stop SA and
Diagnostics > Restart SA functions.
If required, you can manually start or restart a service using the following procedure:
1. Open a console to the PVM
VM.
2. Login to the guest OS.
3. From the PVM VM desktop, open a terminal window.
4. In the Terminal window: a. Restart jboss:
• # service jbossd
{start|stop|status| restart} b. Restart Serviceability
Agent:
• service spiritAgent stop service snmpd restart service spiritAgent start
Table continues…
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
29
Troubleshooting general issues
Application
Pod Visualization Manager
ESXi hosts
Problem Description
Unable to add a device using
Profile Editor.
EVC mode are disabled on the
ESXi host clusters by default.
Avaya Pod Fx that have been expanded with additional compute servers of different make and type can experience issues migrating
VMs between hosts in the cluster with EVC mode disabled.
Solution or Workaround
5. Inspect the logs to confirm the services started correctly.
a.
/opt/spirit/ logging/spirit.log
6. Verify SA inventory: a.
/opt/spirit/ xmlData/
Inventory.xml
b.
/opt/spirit/ xmlData/Backup
The device being added may not be reachable from the PVM virtual machine.
1. Ping the device from the PVM
Virtual Machine. If the device responds it is reachable.
2. Check if the SNMP ports are reachable from the PVM
Virtual Machine.
3. Check if the SNMP credentials are configured properly in the Credentials
Editor.
Mitigation of this requires creation of a new host cluster with the EVC mode enabled, moving hosts to the EVC mode cluster, manual migration of the vCenter to new
EVC cluster, powering down VMs and moving them to the EVC cluster, and cleaning up the rest of the solution.
The following must be done if this situation is encountered:
1. Create a new host cluster with
EVC mode enabled.
2. Move hosts to the new EVC mode enabled cluster.
3. Manual migration of vCenter to the new cluster.
Table continues…
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
30
Troubleshooting with PVM
Application vSphere client
Problem Description
The vSphere client prompts you to accept an SSL certificate the first time you access a Nimble device on the Nimble management tab.
Solution or Workaround
4. Power down VMs and move them to the new cluster.
5. Take any other steps necessary to use this new cluster.
See https://kb.vmware.com/ selfservice/microsites/search.do?
language=en_US&cmd=displayKC
&externalId=1013111&src=vmw_s o_vex_pbrar_70 for additional information and procedures on this topic.
Accept the SSL certificate.
Related links
Troubleshooting general issues on page 24
Troubleshooting with PVM
Pod Visualization Manager (PVM) provides access to the visualization and management applications required to configure, orchestrate, manage, and monitor your Avaya Pod Fx. The following table lists some common troubleshooting information displayed in PVM.
Symptom
A component displays
Unreachable and a green LED is not displayed.
HP Server: an amber LED is displayed.
HP Server: no LED displayed on power supply.
HP Server: no LED on the iLO port.
Lenovo Server: an amber LED is displayed on the power supply.
Nimble CS1000 Storage Server
Disk LED: an orange LED
Troubleshooting Issue or Action
Check the physical connectivity of the component.
Faulty disk.
The power supply is missing or faulty.
The iLO port is not configured.
Faulty power supply.
Faulty disk.
Table continues…
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
31
Troubleshooting general issues
Symptom
Nimble CS1000 Storage Server
Disk LED: EMPTY Disk.
Nimble CS1000 Storage Power
LED: an orange LED.
Nimble CS1000 Storage Ethernet
Links LED: on orange LED.
Nimble CS1000 Storage Fiber
Links LED: on orange LED.
EMC VNXe 3200 Storage Server
Disk LED: an orange LED next to a blinking LED.
EMC VNXe 3200 Storage Server
Disk LED: no LED shown.
EMC VNX 5300 Storage Server
Disk LED: an orange LED
Power Consumption Chart: unusual under or over consumption of power is shown.
Red thresholds are reached in
CPU usage, memory usage, or
Troubleshooting Issue or Action
Note:
Hovering the mouse over an LED presents more information.
The disk is missing.
Note:
Hovering the mouse over an LED presents more information.
An orange LED can mean any of the following:
• Power is off and AC power connected.
• The power supply is not properly seated.
• The array is powered off.
• Power supply internal fault.
Note:
Hovering the mouse over an LED presents more information.
Ethernet link is down.
Note:
Hovering the mouse over an LED presents more information.
Fiber link is down.
Note:
Hovering the mouse over an LED presents more information.
Faulty disk.
Note:
Hovering the mouse over an LED presents more information.
The disk is missing.
Note:
Hovering the mouse over an LED presents more information.
Faulty disk.
Note:
Hovering the mouse over an LED presents more information.
Power consumption should be monitored on a regular basis using the tools provided by PVM. Unusual power fluctuations are a sign of a malfunctioning device or too much power is drawn by the Avaya Pod
Fx. Either circumstance requires further investigation.
The Pod Dashboard displays information on CPU and memory usage and temperature for all Avaya Pod Fx components. A red threshold
Table continues…
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
32
Troubleshooting with PVM
Symptom temperature on the Pod
Dashboard.
Troubleshooting Issue or Action indicator for any of these metrics indicates a situation that requires immediate attention.
Note:
Threshold values are initially set to default values. They are configurable through the PVM to suit individual situations and preferences.
PVM is a consolidated launch pad for all elements in the Avaya Pod Fx solution.
Related links
Troubleshooting general issues on page 24
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
33
Chapter 5: Troubleshooting software issues
This section contains information and procedures for troubleshooting software issues.
Related links
Troubleshooting Avaya Aura Communication Manager in an Avaya Aura Virtualized Environment on
page 34
ESXi host alarm after VMware 6.0 Update 3A upgrade
on page 36
vCenter Server inaccessible after server failure or unexpected server reboot on page 37
Reconfiguring a cluster on page 37
Timeout error configuring High Availability in vCenter
on page 39
Backing up and restoring vCenter configuration on page 41
Recovering from a failed VCSA 5.5 upgrade
on page 42
Avaya Aura VMs do not recover on page 43
Troubleshooting Avaya Aura
®
Communication Manager in an Avaya Aura
®
Virtualized Environment
The issues in the following section are encountered when running Avaya Aura ®
Manager in an Avaya Aura ®
Communication
Virtualized Environment such as the Avaya Pod Fx.
Servers are temporarily in an active / active state
Condition
Avaya Aura ® Communication Manager servers temporarily run in an active / active state in a duplicated server configuration. The active server becomes unresponsive, Avaya Aura ®
Communication Manager interchanges to the standby server, and the unresponsive server returns to an active state. Avaya Aura ® Communication Manager then runs in an active / active state until the arbiter resolves the dual state and returns one server to standby status.
Cause
Avaya Aura ® Communication Manager is working as designed. There is an underlying cause in the overall solution configuration that needs to be addressed.
Solution
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
34
Troubleshooting Avaya Aura ® Communication Manager in an Avaya Aura ® Virtualized Environment
The following actions should be taken to address the underlying cause of the dual active state:
• Datastore architecture should be designed to tolerate drive failures, controller failures, and connectivity issues without disruption. The active and standby servers should be located on separate datastores.
• Avaya Aura ® Communication Manager CPU reservations should be used if ESXi hosts are oversubscribed. See Deploying Avaya Aura ® Communication Manager on VMware
Virtualized Environment (7.x VE and AVP) for reservation guidelines.
® in
Virtualized Environment (6.x VE only) and Deploying Avaya Aura ® Communication Manager in
• VMware recommends only allocating to a virtual machine as many virtual resources as it requires. Allocating more resources than necessary can, in some cases, reduce VM performance as well as the performance of other VMs on the same host. See the section ESXi
CPU Considerations in Performance Best Practices for VMware vSphere 6.0
in the VMware
6.0 documentation for more information.
Link bandwidth or connectivity issues cause non-deterministic behavior
Condition
Active / active states, filesync failures, server interchanges, server resets, or other non-deterministic behavior.
Cause
Duplication link bandwidth or connectivity issues might cause active / active states, filesync failures, server interchanges, server resets, or other non-deterministic behavior.
Solution
The duplication link must have a total capacity of 1Gbps that is dedicated to CM duplication. The duplication link interface should be a separate and dedicated interface. See PSN003556u on the
Avaya Support website for additional information. See Duplicated Avaya Aura should be referenced for any duplicated CM VE deployment.
® Communication
Manager on VMware for best practices on deploying duplicated CM servers in a VE configuration. It
Snapshots can adversely impact service
Condition
Use of snapshots on Avaya Aura snapshots with Avaya Aura ®
® Communication Manager can cause interchanges or adversely impact service in other ways. Extreme caution must be exercised when using virtual machine
Communication Manager deployed on VMware.
Solution
The following must be understood if you are considering the use of snapshots:
• Snapshot operations should never be performed on the active VM. The application must be stopped and placed out of service before performing snapshot operations.
• The application should be started and brought back into service after the snapshot operation is complete.
• If the duplex Open Virtual Application (OVA) is in use, snapshot operations should be performed on the standby CM VM after ensuring the standby is refreshed. If a snapshot is taken on the active VM a service impacting interchange is likely to occur.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
35
Troubleshooting software issues
• A snapshot should never remain on a system for more than 48 to 72 hours before being deleted or reverted. Additional information on snapshot removal can be found in KB1002836 on the VMware Knowledge Base website.
• Third party based snapshots are not supported.
• Best practices for snapshots are documented in the Deploying Avaya Aura ® Communication
Manager on VMware ® in Virtualized Environment document.
Important:
Snapshots should not be used as a backup and recovery mechanism. They are only intended to test an action that may result in an unpredictable outcome.
Using vMotion with real time applications can impact service
Condition
Use of vMotion with real time applications such as Avaya Aura ® service impacting and is not recommended.
Communication Manager can be
Solution
If vMotion is used, it is recommended that manual migrations with vMotion be done only during a maintenance window since the migration could be service impacting. If vMotion DRS automation is used, it is recommended that the DRS automation level be set to the most conservative level possible, such as level 2 or level 1. See PSN20151u on the Avaya Support website for more information.
Related links
Troubleshooting software issues
on page 34
ESXi host alarm after VMware 6.0 Update 3A upgrade
Condition
The alarm message System logs on host are stored on non-persistent storage is displayed after upgrading an ESXi host to VMware 6.0 Update 3A.
Cause
The upgrade changes the System logs location is changed to a location that would not save them if the server was rebooted. This is a default action of the upgrade process and cannot be changed before the upgrade begins.
Solution
Use the following procedure to fix this issue: https://kb.vmware.com/selfservice/microsites/ search.do?language=en_US&cmd=displayKC&externalId=2032823 .
Related links
Troubleshooting software issues
on page 34
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
36
vCenter Server inaccessible after server failure or unexpected server reboot
vCenter Server inaccessible after server failure or unexpected server reboot
The VMware vCenter Server status shows Stopped and fails to Start. In normal operation, status shows Running.
Occurrence of issue: very rare
Overall completion: approximately 1 hour
Cause
A host server failure or unexpected reboot of the host server with the vCenter Appliance and the vCenter database virtual machines, which causes the vCenter appliance and the vCenter database server to be migrated to a new host. This results in a temporary interruption in access to vCenter. If vCenter cannot be reached or accessed after 15 minutes, the vCenter appliance can have issues with the services starting up. The message states vpxd failed to start .
Contact Avaya Support for advanced troubleshooting and repair.
Solution
1. Log in to the vCenter web interface located at: https://<vCenter_ip_address:5480> .
2. Click Summary .
3. In the vCenter Server section, click Start to restart the Server process.
4. Verify that the status changes to Running .
vCenter is back online. Proceed to
on page 37 for the procedure to configure the cluster.
Related links
Troubleshooting software issues
on page 34
Reconfiguring a cluster
About this task
This procedure is used to reconfigure a cluster after an unexpected shutdown of the Avaya Pod Fx.
Procedure
1. On the Management Server Console, launch the vCenter vSphere Client, and log in.
Important:
Administrator must have advanced knowledge of VMware and vCenter.
The vCenter screens do not show any administration.
2. Use the wizard to create a Data Center and Cluster.
3. Add the hosts to the Cluster.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
37
Troubleshooting software issues
Note:
The hosts preserve all configuration and vCenter picks up all administration.
4. In vCenter licensing, add missing licenses for vCenter and hosts.
5. Assign the licenses to the vCenter and hosts, as applicable.
6. In vCenter, click the address for the Avaya Pod Fx settings you want to administer, select the
Turn On vSphere HA and Turn On vSphere DRS check boxes, and click OK .
7. In the Advanced Options (vSphere HA) dialog box, use the data in the following table to restore the VMware HA advanced settings, and click OK .
Option das.isolationaddress0 to das.isolationaddress9
Value
Enter the Default
Gateway IP addresses.
das.usedefaultisolationaddress
Enter false.
Description
Changes the default isolation IP address.
The default address is the Default Gateway address for the ESXi host because it is a highly available IP address.
The das.isolationaddress is queried to determine if an isolation response has occurred on the host. If you change this parameter, you must use a reliable, highly available IP address with as few failure points as possible between it and the ESXi host.
Disables the use of the Default Gateway as an isolation address. Change the value to false. By default, the value is true. This parameter is generally used in conjunction with the das.isolationaddress0 to das.isolationaddress9 parameters.
The following example shows the default Gateway IP addresses for das.isolationaddress1
and das.isolationaddress2. The das.usedefaultisolationaddress is set to false.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
38
Timeout error configuring High Availability in vCenter
8. Click vSphere DRS , and click Fully automated .
9. Move the threshold slider to the right and apply priority 1 and 2 recommendations, and click
OK .
Next steps
Important:
After vCenter is back to normal operation, remove any snapshots that you took to preserve data.
Snapshots can adversely affect VM performance if left in place.
You must then configure HA/DRS.
Related links
Troubleshooting software issues
on page 34
Timeout error configuring High Availability in vCenter
Condition
A timeout might occur during the configuration of High Availability in VMware vCenter. An error similar to the following appears in the /var/log/fdm.log
file.
[ClusterManagerImpl::ConnectToMaster] Couldn't connect to master
N7Vmacore16TimeoutExceptionE(Operation timed out)
[ClusterManagerImpl::AddBadIP] IP 10.x.x.x marked bad for reason Unreachable IP
[ClusterElection::ChangeState] SlaveConnecting => Startup : SlaveConnectingStateFunc
[ClusterManagerImpl::IsBadIP] 10.x.x.x is bad ip
The following situations are observed as a result of the timeout:
• High Availability configuration fails.
• High Availability cannot be configured on a cluster.
• High Availability configuration completes on the master but fails on all slaves.
Cause
This issue occurs if Jumbo Frames are enabled on the host Management Network (the VMkernel port used for host management) and the network configuration is not configured for jumbo frames.
This issue is resolved by ensuring that the vSwitch and Management Network MTU values are configured to 1500.
Solution
1. Right-click the cluster host in vSphere Client.
2. Click the Configuration tab.
3. Click Networking .
4. Click Properties for the Management Network on vSwitch0 on Release 3.0 Pods and vSwitch1 on pre-Release 3.0 Pods.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
39
Troubleshooting software issues
5. Click the Ports tab.
6. Click vSwitch ,
7. Click Edit .
8. Change the MTU value to 1500 if it is not already configured to that value.
9. Click OK .
10. Click the Ports tab.
11. Click Management Network .
12. Click Edit .
13. Change the MTU value to 1500 if it is not already configured to that value.
14. Click OK .
15. Repeat steps 1–14 for all hosts in the cluster.
16. Right-click the cluster and attempt to configure High Availability.
Verify the MTU settings of the Avaya Virtual Services Platform 4850GTS switches if you continue to encounter issues with High Availability configuration. Both devices must be configured to have an
MTU setting of 1950 to ensure proper operation.
1. Connect to the switch.
2. Enter Global Configuration mode.
enable configure terminal
3. Display the current switch MTU setting.
show sys mtu
4. Change the MTU setting to 1950 if it is not already that value.
sys mtu 1950
5. Confirm the new MTU setting.
show sys mtu
6. Save the new configuration.
save config
7. Repeat steps 1–6 for all applicable Avaya Virtual Services Platform 4850GTS switches.
Related links
Troubleshooting software issues
on page 34
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
40
Backing up and restoring vCenter configuration
Backing up and restoring vCenter configuration
About this task
This procedure provides steps for backing up an existing vCenter server and restoring the configuration to a new server. See https://kb.vmware.com/selfservice/microsites/search.do?
language=en_US&cmd=displayKC&externalId=2034505 for additional information about this procedure.
Procedure
1. Deploy the new vCenter server.
2. Navigate to the new server using a web browser and the URL https://
<vcenter_ip_address>:5480 .
3. Accept the certificate when prompted.
4. Log in to the new server.
5. Log in to the old server using PuTTY,
6. Stop the VMware vCenter Server service running on the old server using the command vmware-vpxd stop .
7. Navigate to the vPostgres utility directory on the old server using the command cd /opt/ vmware/vpostgres/current/bin .
8. Display the vPostgres database configuration file using the command cat /etc/vmwarevpx/embedded_db.cfg
.
9. Note the value of EMB_DB_INSTANCE and EMB_DB_USER .
10. Create the vCenter backup using the command ./pg_dump <EMB_DB_INSTANCE> -U
<EMB_DB_USER> -Fp -c > /tmp/VCDBBackupFile where <EMB_DB_INSTANCE> and
<EMB_DB_USER> are the values noted in the previous step.
11. Copy the VCDBBackupFile to the local system using SCP.
12. Create an SCP session with the new vCenter Server.
13. Copy the VCDBBackupFile to /tmp on new vCenter Server.
14. Create a SSH session to the new vCenter Server using PuTTY.
15. Display the new vPostgres database configuration file on new vCenter Server using the command cat /etc/vmware-vpx/embedded_db.cfg
.
16. Note the value of EMB_DB_PASSWORD and EMB_DB_INSTANCE .
17. Navigate to the vPostgres utility directory using the command cd /opt/vmware/ vpostgres/current/bin .
18. Stop the VMware vCenter Server service using the command service vmware-vpxd stop .
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
41
Troubleshooting software issues
19. Restore the vCenter Server vPostgres database from the backup using the command
PGPASSWORD=<EMB_DB_PASSWORD> ./psql -d <EMB_DB_INSTANCE> -Upostgres
-f VCDBBackupFile where <EMB_DB_PASSWORD> and <EMB_DB_USER> are the values noted in the previous step.
20. Restart the vCenter Server using the command service vmware-vpxd start .
Related links
Troubleshooting software issues
on page 34
Recovering from a failed VCSA 5.5 upgrade
About this task
The following procedure provides steps for recovering from a failed upgrade to vCenter Server
Appliance (VCSA) Release 5.1 to 5.5. This procedure presents a high level overview of the process.
Refer to the appropriate documentation for procedures such as:
• Upgrading VCSA deployments.
• vPostgres database file backup and restoration.
• VCSA deployment.
• Host connection.
• Reverting to VCSA snapshots.
Procedure
1. Backup and restore the failed upgrade to a new 5.5 instance.
a. Backup the vPostgres database from the failed 5.5 upgrade.
b. Note the IP address, DNS settings, and Hostname of the failed upgrade.
c. Shutdown the failed 5.5 upgrade.
d. Deploy a new 5.5 instance with the same IP address, DNS settings, and Hostname as the failed upgrade.
e. Restore the vPostgres database backup file to the new 5.5 instance.
f. Log in to the new vCenter instance.
g. Connect any hosts that display as disconnected using root credentials.
h. Proceed to the next step if the 5.5 upgrade does not complete successfully.
2. Perform a fresh upgrade from 5.1 to 5.5.
a. Revert back to the 5.1 snapshot.
b. Perform a fresh upgrade from 5.1 to 5.5.
c. Proceed to the next step if the 5.5 upgrade does not complete successfully.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
42
Avaya Aura ® VMs do not recover
3. Backup and restore the 5.1 instance to a new 5.1 instance and then upgrade to a new 5.5
instance.
a. Revert back to the 5.1 snapshot.
b. Backup the vPostgres database from the 5.1 instance.
c. Note the IP address, DNS settings, and Hostname of the 5.1 instance.
d. Shutdown the 5.1 instance.
e. Deploy a new 5.1 instance with the same IP address, DNS settings, and Hostname as the 5.1 instance just shut down.
f. Restore the vPostgres database file to the new 5.1 instance.
g. Upgrade the new 5.1 instance to 5.5.
Related links
Troubleshooting software issues
on page 34
Avaya Aura
®
VMs do not recover
Condition
Avaya Aura ® VMs may not recover successfully after a sudden and complete power outage when systems are not able to gracefully shut down.
Cause vCenter attempts to migrate the VMs to a different host when the power outage starts but cannot complete the action when all hosts lose power. When the hosts are powered on again, vCenter attempts to restart all hosts but cannot successfully reconnect the VMs to the network. This issue impacts SMGR, AMS, CMS, and Session Manager instances. The following symptoms are found on the Avaya Pod Fx when a power outage has occurred:
• vCenter is not able to complete the process of powering on the VM after the outage.
• An event is displayed indicating the Ethernet interfaces associated with the Avaya Aura ® will start disconnected.
VM
• An error message is displayed if an attempt is made to manually reconnect the network adapter.
Solution
1. Remove network adapters of affected VMs.
2. Add back network adapters of affected VMs.
3. Assign the MAC address originally used with the VM so the correct HWADDR value is assigned to the Eth0 interface.
4. Install any necessary authentication files associated with the affected VMs.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
43
Troubleshooting software issues
Related links
Troubleshooting software issues
on page 34
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
44
Chapter 6: Troubleshooting hardware issues
This section contains information and procedures for troubleshooting hardware issues. It also contains procedures commonly used in hardware troubleshooting such as firmware upgrades.
Related links
Lenovo compute server power supply alarm troubleshooting
on page 45
Upgrading HP ProLiant Gen8 server BIOS for POST errors
on page 50
Upgrading Lenovo server BIOS for fan alarm
on page 52
Checklist for replacing a Power Distribution Unit
on page 53
Lenovo compute server power supply alarm troubleshooting
Condition
The system error LED and Power Supply Unit (PSU) alert on the front panel might stay lit even if the power supplies are operating normally and there are no errors in the System Event logs. In this scenario, the issue is cosmetic and does not affect power supply or system functionality.
Cause
False PSU alert errors are a known issue with Lenovo BMC firmware prior to version 20.00.
Solution
1. Check the LED on the rear of the PSU when a system error LED and PSU alert is shown on the front panel.
• If the LED on the PSU is green or blinking green, the errors and alerts are false and the
PSU is indicating a good state.
• If the LED on the PSU is amber, the errors and alerts are valid and the PSU might be failing and require a replacement.
2. Go to the Lenovo support website and search for keywords BMC update Web GUI .
3. Select the BMC Update utility for Web GUI that is applicable to your ThinkServer model.
4. Verify that the update version shown is version 20.00 or higher. Download both of the update files (the .tgz update utility and the readme).
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
45
Troubleshooting hardware issues
5. Using the Management Server Console on the Avaya Pod Fx, open vCenter and place the host that requires the BMC update into maintenance mode.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
46
Lenovo compute server power supply alarm troubleshooting
Important:
Only perform the Lenovo BMC firmware update procedure when the host server is in
VMware maintenance mode.
6. Open the readme file that you downloaded from the Lenovo website. Perform the steps in the INSTALLATION INSTRUCTIONS section to extract the bmc_xxx.ast file, upload the file to the server, and apply the update.
Note:
The default username and password for the ThinkServer Management Module (TMM) is typically:
• Username: lenovo
• Password: len0vO
(len<zero>v<capital O>)
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
47
Troubleshooting hardware issues
7. After you perform the installation instruction steps in the readme, the firmware update proceeds. You can check the status by watching the status column.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
48
Lenovo compute server power supply alarm troubleshooting
8. Open a new browser session to the ThinkServer Management Module (TMM) and login.
Confirm the firmware update by viewing the Properties page.
9. Using the Management Server Console on the Avaya Pod Fx, open vCenter and exit the host from maintenance mode.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
49
Troubleshooting hardware issues
10. Repeat all steps for any additional Lenovo servers that require the BMC firmware update to fix false PSU alerts.
Related links
Troubleshooting hardware issues
on page 45
Upgrading HP ProLiant Gen8 server BIOS for POST errors
Condition
HP ProLiant Gen8 servers may experience unexpected errors during a reboot. This involves the server displaying POST errors 236 or 239. The server may also reboot unexpectedly and display
POST error 207.
Cause
These POST error conditions are a known issue with some HP ProLiant Gen8 servers. Use the following procedure to upgrade the server BIOS to the latest version to correct this error.
Important:
The host server must be in maintenance mode to perform a BIOS upgrade. Ensure that the server is in maintenance mode before beginning this procedure.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
50
Upgrading HP ProLiant Gen8 server BIOS for POST errors
You must be logged in as root to apply any updates. If you are not logged in as this user, log out, and log in as the root user.
Solution
1. Go to the HP support website ( HTTP://SUPPORT.HP.COM/ ).
2. Select Software and Driver downloads .
3. Enter HP ProLiant DL360p Gen8 Server .
4. Select Find my Product .
5. Select Servers .
6. Select HP ProLiant DL360p Gen8 Server .
7. Select the applicable version of VMWare OS from the drop down list.
8. Select BIOS (Entitlement Required)- System ROM .
9. Select HP BIOS version 2014.11.01 (CP025384.scexe) or higher for VMware ESXI and download it to a temporary directory such as /tmp .
Note:
A valid HP account with a valid contract and warranties is required to access the necessary downloads.
10. Perform the following steps to enable Tech Support Mode on the server: a. Access the server console using the Integrated Lights-Out (ILO) interface or by directly attaching a monitor and keyboard to the server b. Log in using root credentials.
c. Navigate to Troubleshooting Options using the keyboard.
d. Enable ESXi Shell and SSH Support .
11. Copy the downloaded file to a temporary directory on the server to be upgraded.
Note:
HP recommends verifying the results of your download with the MD5 checksum value found on the Installation Instructions tab to ensure the integrity of your download.
12. Locate the temporary directory that contains the file you copied in step 9 and execute the file.
Note:
Ensure that the versions listed are accurate before proceeding.
13. Upgrade the software to the newer version by typing y when prompted.
14. A message is displayed when the upgrade is complete.
15. Reboot from vCenter or from the server.
16. Use vCenter client to log in directly to the server when the update completes.
17. Confirm the BIOS was upgraded, the server is fully functional, and there are no major hardware alarms.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
51
Troubleshooting hardware issues
18. Remove the server from Maintenance Mode.
Related links
Troubleshooting hardware issues
on page 45
Upgrading Lenovo server BIOS for fan alarm
Condition
Lenovo servers may experience the fan alarm Fan sensor, warning event was asserted, reading value: 0RPM (Threshold : 2496RPM) when it is not applicable.
Cause
This is a known issue with Lenovo servers. Use the following procedure to upgrade the server BIOS to the latest version to correct this error.
Important:
The host server must be in maintenance mode to perform a BIOS upgrade. Ensure that the server is in maintenance mode before beginning this procedure.
Solution
1. Go to the Lenovo support website ( HTTP://SUPPORT.LENOVO.COM/ ).
2. Select Servers > ThinkServer rack servers .
3. Select the applicable server from the drop down list.
4. Select BIOS in the Component list.
5. Select BIOS Update Utility for BMC Web GUI-ThinkServer RDXXX where RDXXX represents the specific server model.
6. Verify that the update version shown is version A1TS8BA or higher.
7. Download the TGZ update utility and readme files.
8. Decompress the TGZ update utility and confirm it contains a file with an RBU extension.
9. Confirm the BIOS version is A1TS8BA or higher.
10. Save the decompressed files to a location that is reachable by the server to be upgraded.
11. Place the host in maintenance mode using vCenter on the Management Server Console.
12. Connect to the IP address of the BMC port on the server using a web browser.
13. Log into the ThinkServer Management Module (TMM).
Note:
The default username and password for the TMM is typically:
• Username: lenovo
• Password: len0VO (len<zero>v<capital O>) or lenovo
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
52
Checklist for replacing a Power Distribution Unit
14. Select Update .
15. Select BIOS from the Firmware Type drop down list.
16. Select Browse .
17. Browse to the location of the RBU file from step 8 .
18. Select Upload .
19. Verify that the correct file is present when the upload completes.
20. Select Update .
21. Reboot from vCenter or from the server.
22. Use vCenter client to log in directly to the server when the update completes.
23. Confirm the BIOS was upgraded, the server is fully functional, and there are no major hardware alarms.
24. Remove the server from Maintenance Mode.
Related links
Troubleshooting hardware issues
on page 45
Checklist for replacing a Power Distribution Unit
Use the following general checklist for the field replacement of a Power Distribution Unit. Refer to the following manufacturer documentation for specific instructions on installation and use:
• ServerTech — https://www.servertech.com/support
No.
Important:
Ensure there is no strain when dressing the RJ45 Ethernet cables connecting the PDU to the switch. These cables have to make a right turn through the chassis to get to their connected pieces of equipment. Always allow adequate strain relief at the PDU end of the cables.
Task Description Notes
1
2
3
PDU arrives on site.
Mark defective PDU.
Note shipped PDU serial number.
PDU arrives at the site of
Avaya Pod Fx with faulty
PDU.
Defective PDU is marked in the support ticket.
The technician or installer must write down the serial number of the shipped
PDU.
Table continues…
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
53
Troubleshooting hardware issues
No.
4
5
6
7
8
9
10
11
12
Task Description
Verify redundant equipment power.
Troubleshoot nonredundant equipment power.
Remove PDU power.
Remove all cables.
Remove PDU from enclosure.
Verify that all equipment has redundant power by ensuring that both power supply LEDs are green on all devices.
If not all equipment has redundant power, confirm that the Power Supply with the unlit LED is connected to the faulty PDU. See the
Customer Lifecycle
Workbook for information on which equipment is connected to which PDU.
Remove power from the faulty PDU by removing the cord that connects it to the customer power supply.
Remove all cables from the faulty PDU.
Remove the PDU from the
Avaya Pod Fx enclosure, ensuring not to bump or disconnect any cords from active PDUs.
Locate PDU serial number.
Locate the faulty PDU serial number in the
Customer Lifecycle
Workbook . Replace it with the serial number of the replacement PDU.
Install PDU in enclosure.
Install the replacement
PDU in the Avaya Pod Fx enclosure.
Power replacement PDU. Plug the replacement PDU into the customer power source.
Configure PDU.
See
on page 55 or
Configuring switched PDUs over
Notes
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
Table continues…
54
Checklist for replacing a Power Distribution Unit
No.
Task Description Notes
13
14
15
16
Attach all cables.
page 58 for configuration procedures.
Attach equipment cables to the PDU in the same location they were attached in the faulty PDU.
See the Customer
Lifecycle Workbook for cable locations.
Confirm PDU operation.
Inspect all equipment to ensure it is now displaying two green power supply
LEDs for each piece of equipment.
Confirm equipment operation.
Confirm that all equipment is operational, including the replacement PDU. The
PDU LED display should be showing the same current level as the corresponding paired
PDU.
Return faulty PDU.
Installer or technician must return the faulty PDU to
Avaya using standard equipment return procedures.
Related links
Troubleshooting hardware issues
on page 45
Configuring switched PDUs over Ethernet
on page 55
Configuring switched PDUs over a Console connection on page 58
Configuring switched PDUs over Ethernet
About this task
This procedure provides the information necessary to configure a switched PDU using the internal
Avaya Pod Fx network. PDU configuration is required after installing a new PDU. See
PDU using a DB9 connection.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
55
Troubleshooting hardware issues
Before you begin
You will require the following items to complete this procedure:
• A laptop with an Ethernet port.
• RJ45 to RJ45 cable. Use a crossover cable if the laptop in use does not support autosensing.
Procedure
1. Configure the IP settings on the service laptop to the following:
• IP Address — 192.168.1.253
• Subnet Mask — 255.255.255.0
• Default Gateway — None
2. Connect the service laptop directly to the PDU Ethernet port label NET using an appropriate
RJ45 UTP cable.
Note:
The PDU Ethernet port labeled SER cannot be used for this task.
3. Open a terminal session and ping the PDU IP address of 192.168.1.254.
If there is a ping response, the network connection can be used to configure the PDU.
If there is no response from the ping, verify the laptop network settings and try again. If there is still no response, see
Configuring switched PDUs over Console Connection on page 58
for a procedure to use a Console connection to configure the PDU.
4. Open a browser window and connect to the PDU management software at http://
192.168.1.254.
5. Log in to the management software with the username and password of admn .
6. Select Configuration > Users from the menu.
7. Select Edit for the ADMN user.
8. Consult the Customer Lifecycle Workbook for the PDU password.
9. Change the PDU password to the password contained in the workbook.
Use the password CPod2013!!
if there is no password in the workbook.
10. Select Configuration > Telnet/SSH from the menu.
11. Select Disabled from the Server dropdown list.
12. Click Apply .
13. Perform the following steps if the PDU is vertically mounted with the exit line and cord in the
UP position: a. Select Configuration > System from the menu.
b. Select Inverted from the Display Orientation dropdown list.
c. Click Apply .
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
56
Checklist for replacing a Power Distribution Unit
14. Select Configuration > Network from the menu.
15. Deselect DHCP .
16. Consult the Customer Lifecycle Workbook for the PDU IPv4 address information.
17. Use the information from the workbook to configure the IPv4 address options of the PDU.
18. Click Apply .
19. Select Configuation > SNMP/Thresholds from the menu.
20. Change the GET Community (Read-Only) field to avaya123 .
21. Change the SysName field to Sentry3_SMART_PDU_#_type where # is the PDU number and type is either MAIN or EXT . MAIN denotes a PDU in a main Avaya Pod Fx and EXT denotes a PDU in an extension Avaya Pod Fx.
Examples
PDU 2 on a main Avaya Pod Fx would be labelled Sentry3_SMART_PDU_2_MAIN .
PDU 1 on an extention Avaya Pod Fx would be labelled Sentry3_SMART_PDU_1_EXT .
22. Change the SysLocation field to Main Pod or Extension Pod depending on the installation type.
23. Click Apply .
24. Select Tools > Restart from the menu.
25. Select Restart from the dropdown list.
26. Click Apply .
27. Click OK .
The PDU will now restart.
28. Disconnect the Ethernet cable used to connect the laptop to the PDU.
29. Connect the PDU to the correct port on the switch. See the Customer Lifecycle Workbook for port information.
30. Use the browser provided with the MSC to connect to the PDU management software.
31. Log into the management software to confirm connectivity.
32. Repeat these steps for the next PDU if more than one requires configuration.
Related links
Checklist for replacing a Power Distribution Unit
on page 53
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
57
Troubleshooting hardware issues
6
7
4
5
8
2
3
Pin
1
Configuring switched PDUs over a Console connection
Before you begin
You will require the following items to complete this procedure:
• A laptop with a serial port or a laptop with no serial port and a USB to DB9 adapter.
• RJ45 to DB9 RS-232 (DTE) cable.
PDUs are equipped with an RJ45 DTE RS-232c serial port. RJ45 to DB9 RS-232 (DTE) cable must have the pinout configuration outlined in the following table.
DTE Signal Name
Request to Send
Data Terminal Ready
Transmit Data
Signal Ground
Signal Ground
Receive Data
Data Set ready
Clear to Send
RTS
DTR
TD
RD
DSR
CTS
Input/Output
Output
Output
Output
Input
Input
Input
About this task
This procedure provides the information necessary to configure a switched PDU using a Console connection. PDU configuration is required after installing a new PDU. See
PDUs over Ethernet on page 55 for the procedure to configure a switched PDU using an Ethernet
connection.
Procedure
1. Connect the Console cable to the laptop and the PDU service port.
2. Connect to the PDU with a terminal program such as PuTTY.
3. Log in with the username and password of admn .
4. Use the command set ipv4 address to enter the new IPv4 address.
5. Use the command set ipv4 subnet to enter the new IPv4 subnet mask.
6. Use the command set ipv4 gateway to enter the new IPv4 default gateway.
7. Use the command restart to restart the administrative interface.
Note:
This action only restarts the administrative interface. It does not reboot the power outlets or remove power from the outlets.
8. Confirm by pressing Y when prompted.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
58
Checklist for replacing a Power Distribution Unit
9. Log back in to the device once the application restarts.
10. Use the command show network to confirm the new network information was applied.
11. If the network information was applied properly, close the Console connection.
12. Disconnect the Console cable used to connect the laptop to the PDU.
13. Connect the PDU to the correct port on the switch. See the Customer Lifecycle Workbook for port information.
14. Use the browser provided with the MSC to connect to the PDU management software.
15. Log into the management software to confirm connectivity.
16. Repeat these steps for the next PDU if more than one requires configuration.
Related links
Checklist for replacing a Power Distribution Unit
on page 53
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
59
Chapter 7: Resources
This section contains information on additional available resources.
Related links
on page 62
Avaya Mentor videos on page 63
Searching a documentation collection
on page 64
Subscribing to e-notifications on page 65
Support
The Avaya Pod Fx is an integrated solution with integrated support. Integrated support means that
Avaya support personnel can isolate and resolve any issues with hardware and software applications packaged within the Avaya Pod Fx.
Note:
Avaya is responsible only for support of the Avaya and third-party hardware and software within the Avaya Pod Fx that has been purchased through Avaya as part of the Avaya Pod Fx, or that has been explicitly approved in writing by the Avaya Pod Fx product management team.
Documentation for specific Avaya products can be found on the Avaya website at http:// support.avaya.com
.
Important:
If you deviate from an Avaya Pod Fx release software baseline, change physical equipment or virtual machines, modify network configurations, or add custom applications to Avaya Pod Fx in any way, without express consent from Avaya, you will void warranty and service contracts for
Avaya Pod Fx. The approved application baseline deployed in the Avaya Pod Fx is part of the design review process. It is documented in the LCM workbook. You must stay within that defined baseline to remain in a supported configuration. If application types or counts must change, the new requirements must be reviewed by the Avaya Pod Fx Go To Market team in a design review through your account team or sales authorized business partner.
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
60
Support
Avaya is not responsible for any unapproved third-party applications or hardware, or the direct or indirect impact of same, if installed by the customer.
To ensure the integrity of an Avaya Pod Fx solution and product warranties, any changes must be approved in writing by the Avaya Pod Fx product management team through the Avaya Pod
Fx Configurator Design Review process. To request a design change review, send an email to [email protected].
Important:
The initial software baseline can require software patches and updates after initial installation.
You are not permitted to update or upgrade Avaya Pod Fx Infrastructure applications or Avaya
Aura ® System Manager. This can only be done as a whole, with upgrade from one Avaya Pod
Fx baseline release to another, unless approved by a PSN or PCN issued by the Avaya Pod Fx product team. This is as per the defined Lifecycle Policy for Avaya Pod Fx release management. Applications outside of the Avaya Pod Fx Infrastructure baseline or Avaya Aura ®
System Manager can be upgraded at the discretion of the system owner by following the Avaya
Product Compatibility Matrix at https://support.avaya.com/CompatibilityMatrix/Index.aspx
.
You must ensure that automatic software updates are disabled for applications. Avaya tests updates to determine compatibility and Avaya will advise you about updates as they are verified and approved for use.
Note:
Avaya recommends that you install important Windows updates on the Windows Virtual
Machines running on an Avaya Pod Fxas a best practice. See and follow your corporation security policies regarding Windows updates.
Contacting Avaya Pod Fx Support
The following support mailing lists are available for specific product queries. Use the mailing list appropriate to your query type.
• Avaya Pod Fx sales and sales-related queries — [email protected].
• Avaya Pod Fx support — [email protected].
• Avaya Pod Fx partner support — [email protected].
Avaya associates can access the LCM repository at https://products.share.avaya.com/sites/podlcm/ default.aspx
to obtain the latest copy of the appropriate workbook. Business Partners and customers should send a request for the latest copy of the workbook to [email protected].
About this task
Avaya Support includes:
• Products
• Downloads and documents
• Service requests
• Parts replacement
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
61
Resources
• Tools
• Community support forum and DevConnect
• Training
• Help and policies
• Alerts and reports
Procedure
1. Go to the Avaya Support home page, www.avaya.com/support
Note:
Site registration is required to access Self Service Tools and Downloads. Registration instructions are available on the Support home page.
2. From the Support home page:
• Enter a keyword in the Avaya Support search dialog box.
• Click any of the topic-based links on the page.
Related links
Documentation reference
The following provides a high-level list of links to the Avaya support website and third-party websites. Go to the following websites to obtain documentation for the components supported on
Avaya Pod Fx.
• Avaya: support.avaya.com
–Contains documentation and downloads for networking, management, and applications for Avaya Pod Fx.
• VMware: www.vmware.com
• Compute servers:
- HPE ProLiant DL360 Generation 9 servers: www.hp.com
- HPE ProLiant DL360p and DL360e Generation 8 servers: www.hp.com
- Lenovo ThinkServer RD540 and RD340: www.lenovo.com
• EMC² VNX storage array: www.emc.com
• HPE Nimble storage array: www.nimblestorage.com
• ServerTech Power Distribution Units: www.servertech.com
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
62
Training
For a detailed list of documentation for Avaya Pod Fx and specific components, see the following document:
• Avaya Pod Fx Documentation Reference (NN47204–113)
Related links
Training
Product training is available on the Avaya Learning website. For more information or to register, see http://avaya-learning.com
.
Related links
Avaya Mentor videos
Avaya Mentor videos are available on an Avaya-run Internet channel dedicated to technical content.
Playlist categories include:
• Unified Communications (tested on Internet Explorer and Firefox).
• Contact Centers.
• Networking.
• Small and Midsize Business.
• Uploaded videos. A composite of all available Avaya Mentor videos.
Before you begin
You must have a valid Internet browser installed and working on your device.
About this task
The Avaya Mentor videos include the following content categories:
• How to install Avaya products.
• How to configure Avaya products.
• How to troubleshoot Avaya products.
Procedure
To go to Avaya Mentor videos, click the following link: http://www.youtube.com/avayamentor
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
63
Resources and perform one of the following actions:
• Enter a key word or words in the Search channel dialog box to search for a specific product or topic.
• Click the name of a playlist to scroll through the available videos.
Related links
Searching a documentation collection
On the Avaya Support website, you can download the documentation library for a specific product and software release to perform searches across an entire document collection. For example, you can perform a single, simultaneous search across the collection to quickly find all occurrences of a particular feature. Use this procedure to perform an index search of your documentation collection.
Before you begin
• Download the documentation collection zip file to your local computer.
• You must have Adobe Acrobat or Adobe Reader installed on your computer.
Procedure
1. Extract the document collection zip file into a folder.
2. Navigate to the folder that contains the extracted files and open the file named
< product_name_release >.pdx.
3. In the Search dialog box, select the option In the index named
<product_name_release>.pdx
.
4. Enter a search word or phrase.
5. Select any of the following to narrow your search:
• Whole Words Only
• Case-Sensitive
• Include Bookmarks
• Include Comments
6. Click Search .
The search results show the number of documents and instances found. You can sort the search results by Relevance Ranking, Date Modified, Filename, or Location. The default is
Relevance Ranking.
Related links
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
64
Subscribing to e-notifications
Subscribing to e-notifications
Subscribe to e-notifications to receive an email notification when documents are added to or changed on the Avaya Support website.
About this task
You can subscribe to different types of general notifications, for example, Product Correction Notices
(PCN), which apply to any product or a specific product. You can also subscribe to specific types of documentation for a specific product, for example, Application & Technical Notes for Virtual Services
Platform 7000.
Procedure
1. In an Internet browser, go to https://support.avaya.com
.
2. Type your username and password, and then click Login .
3. Under My Information , select SSO login Profile .
4. Click E-NOTIFICATIONS .
5. In the GENERAL NOTIFICATIONS area, select the required documentation types, and then click UPDATE .
6. Click OK .
7. In the PRODUCT NOTIFICATIONS area, click Add More Products .
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
65
Resources
8. Scroll through the list, and then select the product name.
9. Select a release version.
10. Select the check box next to the required documentation types.
11. Click Submit .
October 2017 Troubleshooting the Avaya Pod Fx ™
Comments on this document? [email protected]
66
advertisement
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Related manuals
advertisement