Cisco UCS Integrated Infrastructure for SAP HANA

Cisco UCS Integrated Infrastructure for SAP HANA
White Paper
Cisco UCS Integrated Infrastructure for SAP HANA
Operations Guide
December 2016
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 1 of 53
Contents
Configuration Summary
Solution Summary
Physical Topology
Operation
Start the Appliance
Stop the Appliance
Scaling
Extend the MapR Converged Data Platform Cluster
Extend the SAP HANA Cluster
Monitoring
MapR Control System
Syslog
Simple Network Management Protocol Traps
Backup and Recovery
File-Based Backup
Storage Snapshot
Mirroring
SAP HANA QA System Refresh
Hardware Maintenance
Infrastructure Components
MapR Converged Data Platform Cluster
SAP HANA Cluster
OS Maintenance
Prerequisites
RHEL Online Update
Configuration Sheets
Infrastructure Components
MapR Cluster
Node Roles
SAP HANA Cluster
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 2 of 53
Configuration Summary
The base configuration for the design discussed in this document can be found in the Cisco® Validated Design
titled Cisco UCS Integrated Infrastructure for SAP HANA:
http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/ucsii_saphana_mapr.html
Solution Summary
Cisco UCS® Integrated Infrastructure for SAP HANA with the MapR Converged Data Platform provides an end-toend architecture with Cisco hardware that supports multiple SAP HANA workloads with high availability and server
redundancy. The solution supports up to 16 Cisco UCS B460 M4 Blade Servers for SAP HANA and up to 8 Cisco
UCS C240 M4 Rack Servers for storage. The next-generation Cisco UCS 6332 Fabric Interconnect supports 40
Gigabit Ethernet network bandwidth for server management and network connectivity. The Cisco UCS C240 M4
server provides persistent storage with the MapR Converged Data Platform, which is a modern Network File
System (NFS) mountable distributed file system. Cisco Nexus® 3000 Series Ethernet switches are used for failover.
If a Cisco UCS 2304 Fabric Extender or a link between the server and fabric interconnect fails, the data traffic path
will use the Cisco Nexus 3000 Series Switch, providing high availability and redundancy. Figure 1 shows the Cisco
UCS Integrated Infrastructure for SAP HANA.
Note:
The reference architecture documented in the Cisco Validated Design consists of eight Cisco UCS B460
M4 Blade Servers for SAP HANA and four Cisco UCS C240 M4 Rack Servers for storage.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 3 of 53
Figure 1.
Cisco UCS Integrated Infrastructure for SAP HANA
Alternatively, the solution can be designed for a pair of Cisco Nexus 9000 Series Switches instead of a single
Cisco Nexus 3000 Series Switch. Two Cisco Nexus 9000 Series Switches can be configured with a virtual port
channel (vPC) between the network switch and the Cisco UCS fabric interconnect, as shown in Figure 2.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 4 of 53
Figure 2.
Cisco UCS Integrated Infrastructure for SAP HANA with a Pair of Cisco Nexus Switches
Physical Topology
Cisco UCS Integrated Infrastructure for SAP HANA with the MapR Converged Data Platform provides an end-toend architecture with Cisco hardware that supports multiple SAP HANA workloads with high availability and server
redundancy. The architecture uses Cisco UCS Manager with Cisco UCS B-Series Blade Servers and C-Series
Rack Servers and Cisco UCS fabric interconnects. The uplink from the fabric interconnect is connected to a Cisco
Nexus 3548 Switch for high availability and failover. The Cisco UCS C-Series servers are connected directly to the
fabric interconnect with the single-wire management feature, and the data traffic between SAP HANA servers and
storage is contained in the fabric interconnect. This infrastructure is deployed to provide IP-based storage access
using NFS protocols with file-level access to shared storage.
Figure 3 shows the Cisco UCS Integrated Infrastructure for SAP HANA described in this Cisco Validated Design. It
shows the Cisco UCS Integrated Infrastructure hardware components and the network connections for a
configuration with IP-based shared storage.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 5 of 53
Figure 3.
Cisco UCS Integrated Infrastructure for SAP HANA
Cisco Unified Computing System
The base reference architecture includes the following Cisco Unified Computing System™ (Cisco UCS)
components:
●
Two Cisco UCS 6332 32-port fabric interconnects
●
Two Cisco UCS 5108 Blade Server Chassis with two Cisco UCS 2304 Fabric Extenders with four 40 Gigabit
Ethernet interfaces
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 6 of 53
●
Eight Cisco UCS B460 M4 high-performance blade servers with two Cisco UCS Virtual Interface Card (VIC)
1380 and two Cisco UCS VIC 1340 adapters
●
Four Cisco UCS C240 M4 high-performance blade servers with one Cisco UCS VIC 1385
Cisco Network
The base design includes one Cisco Nexus 3548 Switch for 10 Gigabit Ethernet connectivity between the two
Cisco UCS fabric interconnects, for failover scenarios.
The solution can be configured for the enterprise data center using a pair for Cisco Nexus 9000 Series Switches.
With two Cisco Nexus switches, vPC is configured between the Cisco Nexus 9000 Series Switches and the Cisco
UCS fabric interconnect, as shown in Figure 4.
The enterprise data center design includes two Cisco Nexus 9000 Series Switches for 10 Gigabit Ethernet
connectivity between the two Cisco UCS fabric interconnects.
Figure 4.
Cisco UCS Integrated Infrastructure for SAP HANA with Data Center Switching
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 7 of 53
Although this is the base design, each of the components can be scaled easily to support specific business
requirements. Additional servers or even blade chassis can be deployed to increase computing capacity without
additional network components. Two Cisco UCS 6332 32-port fabric interconnects can support up to:
●
16 Cisco UCS B460 M4 servers with 8 blade server chassis
●
8 Cisco UCS C240 M4 servers
A minimum of three Cisco UCS C240 M4 servers are required to install the MapR Converged Data Platform with
high availability. Three Cisco UCS C240 M4 servers can support up to six active SAP HANA servers. The scaling
of storage to the computer node is linear, so for every two Cisco UCS servers for SAP HANA, one Cisco UCS
C240 M4 server is needed for storage to meet the SAP HANA storage performance defined by SAP.
The solution is designed to meet the SAP HANA performance requirements defined by SAP. The SAP HANA
server and storage server are connecting to the Cisco UCS fabric interconnect, and all data traffic between SAP
HANA node and the storage node is contained in the fabric interconnect. Each SAP HANA server and storage
server is equipped with two 40 Gigabit Ethernet–capable Cisco UCS VICs, and the storage network provides
dedicated bandwidth between SAP HANA servers and storage servers. Traffic shaping and quality of service
(QoS) are configured and managed by the Cisco UCS fabric interconnect. For the SAP HANA node-to-node
network, 40-Gbps of dedicated network bandwidth is provided in nonblocking mode. Cisco UCS Integrated
Infrastructure for SAP HANA quarantines the bandwidth and latency for best performance to run SAP HANA.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 8 of 53
Operation
The SAP HANA appliance consists mainly of two parts: the MapR Converged Data Platform cluster and the SAP
HANA cluster. All server nodes of the two clusters are connected to the fabric interconnect, and therefore they can
be operated by Cisco UCS Manager. For information about general operation tasks such as powering up a server,
please refer to the Cisco UCS Manager documentation:
http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-manager/products-installation-andconfiguration-guides-list.html
Start the Appliance
Power up all servers for the MapR Converged Data Platform cluster first. After the MapR cluster is up and running,
power up the servers in the SAP HANA cluster. If you power up all the servers at the same time, the HANA cluster
nodes may hang during bootup because of missing NFS shares.
Start the MapR Converged Data Platform
Follow these steps to start the MapR Converged Data Platform:
1.
Log in to Cisco UCS Manager through Secure Shell (SSH); then power up the servers:
UCS-A# scope org HANA
UCS-A /org* # scope service-profile MapR-01
UCS-A /org/service-profile* # power up
(repeat the last two commands for every MapR Service Profile)
UCS-A /org/service-profile* # commit-buffer
2.
On the nodes on which the mapr-zookeeper service is installed (typically the last three nodes in the cluster),
start zookeeper:
service mapr-zookeeper start
3.
Verify that a quorum has been successfully established:
service mapr-zookeeper qstatus
4.
On the nodes on which the mapr-cldb service is installed (typically the first two nodes in the cluster), start
warden:
service mapr-warden start
5.
Verify that a container location database (CLDB) master is running:
maprcli node cldbmaster
6.
Start warden on all remaining nodes (including zookeeper nodes):
service mapr-warden start
7.
Verify that all nodes are up and that running services match configured services:
maprcli node list -columns csvc,svc
Start SAP HANA
Before powering up the servers, make sure that MapR Converged Data Platform is up and running.
1.
Log in to Cisco UCS Manager using SSH; then power up the servers:
UCS-A# scope org HANA
UCS-A /org* # scope service-profile HANA-Server01
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 9 of 53
UCS-A /org/service-profile* # power up
(repeat the last two commands for every HANA-Server Service Profile)
UCS-A /org/service-profile* # commit-buffer
2.
On one node in the SAP HANA cluster, use the SAP start service (hostagent) to start the SAP HANA system.
Run as <sid>adm:
sapcontrol -nr <instance_nr> -function StartSystem
3.
Verify that the SAP HANA database is up (green) on every node:
sapcontrol -nr <instance_nr> -function GetSystemInstanceList
Stop the Appliance
When stopping the appliance, always stop SAP HANA first. If MapR Converged Data Platform will be stopped
before stopping SAP HANA, data loss can occur.
Stop SAP HANA
Follow these steps to stop the SAP HANA appliance:
1.
On one node in the SAP HANA cluster, use the SAP start service (hostagent) to stop the SAP HANA system.
Run as <sid>adm:
sapcontrol -nr <instance_nr> -function StopSystem
2.
Verify that the SAP HANA database is shut down (gray) on every node:
sapcontrol -nr <instance_nr> -function GetSystemInstanceList
3.
Run as root on all nodes, powering off the servers gracefully:
halt -p
Stop the MapR Converged Data Platform
Now stop the MapR Converged Data Platform:
1.
On all nodes, stop the warden:
service mapr-warden stop
2.
On the nodes on which mapr-zookeeper is installed, stop zookeeper:
service mapr-zookeeper stop
3.
On all nodes, power off the servers gracefully:
halt -p
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 10 of 53
Scaling
Cluster extension is performed in two phases:
●
On the MapR Converged Data Platform cluster:
◦ Prepare the OS
◦ Add a node
◦ Create volumes
◦ Add virtual IP addresses
●
On the SAP HANA cluster:
◦ Prepare the OS
◦ Mount volumes
◦ Add a node
Extend the MapR Converged Data Platform Cluster
First assign the Cisco UCS service profile and install and prepare Linux on the new node according to the Cisco
Validated Design.
A standard node will not contain zookeeper or cldb packages. Only the file server (mapr-fileserver) and NFS
gateway (mapr-nfs) packages with their dependencies are required.
Create Volumes
Create as many data and log volumes as new SAP HANA nodes (needed for SAP HANA storage partitions).
Data volume:
maprcli volume create -name <volumename> -path
/apps/hana/<SID>/<datamountpoint_storage_partition> -replicationtype
high_throughput -replication 2 -minreplication 2
Log volume:
maprcli volume create -name <volumename> -path
/apps/hana/<SID>/<logmountpoint_storage_partition> -replicationtype low_latency replication 2 -minreplication 2
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 11 of 53
Add Virtual IP Addresses
Add as many virtual IP addresses as new SAP HANA nodes.
Storage
Partition
Virtual IP Address
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Add the virtual IP address range of the new virtual IP addresses:
maprcli virtualip add -macs <mac_addr_node1> <mac_addr_node2> <mac_addr_nodeN>
-virtualipend <<last_virtual_ip>> -netmask <<subnet_mask> -virtualip
<<first_virtual_ip>>
Extend the SAP HANA Cluster
First assign the service profile and install and prepare Linux on the new node according to the Cisco Validated
Design.
Mount the Volumes
Extend /etc/fstab on every node (including the one that will be added) to reflect the new storage partition that will
be added:
<virtual_IP>:/mapr/<clustername>/apps/hana/data0000<no>
/hana/data/<SID>/mnt0000<no> nfs nolock,hard,timeo=600 0 0
<virtual_IP>:/mapr/<clustername>/apps/hana/log0000<no>
/hana/log/<SID>/mnt0000<no> nfs nolock,hard,timeo=600 0 0
Example:
maprvip08:/mapr/mapr500/apps/hana/data00008 /hana/data/ANA/mnt00008 nfs
nolock,hard,timeo=600 0 0
maprvip08:/mapr/mapr500/apps/hana/log00008 /hana/log/ANA/mnt00008 nfs
nolock,hard,timeo=600 0 0
Mount the volumes:
mount -a
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 12 of 53
Add a Node
Log in to any existing SAP HANA node and use the SAP HANA Lifecycle Manager to add a node.
Run as <sid>adm:
hdblcm --action=add_hosts --remote_execution=saphostagent --sid=<SID> -addhosts=<hostname>:role=worker:storage_partition=<storage_partition>
Verify the Node
Use sapstart framework to verify all instances of the system.
Run as <sid>adm:
sapcontrol -nr <instance_no> -function GetSystemInstanceList
Alternatively, you can use the system landscape view of SAP HANA Studio:
Monitoring
A number of tools are available to monitor the system, including the MapR Control System (MCS), syslog, and
Simple Network Management Protocol (SNMP) traps.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 13 of 53
MapR Control System
MCS gives MapR administrators a single place for configuring, monitoring, and managing their MapR clusters. To
access MCS, simply point your browser at https://<webserver node>:8443 and provide your credentials to log in.
MCS exposes many aspects of the MapR Converged Data Platform for configuration and management, including:
●
Volumes: The MapR Converged Data Platform allows users to logically divide available storage into units
called volumes. User permissions and quotas can be applied on a per-volume basis, making volumes
enablers of multitenant clusters. In addition, data protection features such as snapshots and mirrors are
configured in the MCS on a per-volume basis, allowing precise control over all data in a cluster. Figure 5
shows the volume management page on the MCS. It shows the used and raw capacity of the data and log
volumes for SAP HANA. It also shows the replication factor and quota and ownership information.
Figure 5.
MapR Control System Volume Management Page
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 14 of 53
●
NFS: The MapR Converged Data Platform allows simplified data import and export with NFS protocol
support. Administrators can view and modify the NFS configuration in the MCS, as well as configure and
modify virtual IP addresses used for redundancy. NFS high availability can also be set up with virtual IP
addresses, to protect against network failure. Figure 6 shows the NFS virtual IP address. In this example,
the physical connection between the SAP HANA and MapR nodes has two virtual IP addresses assigned to
it for redundancy.
Figure 6.
●
NFS Virtual IP Addresses on the MapR Control System
Alarms: The MCS continuously monitors the status of each node in the MapR cluster, proactively raising
alarms if problems such as disk failures are found. Alarms are descriptive, indicating to administrators
actions for remediation. Figure 7 shows the MCS dashboard displaying the overall health of the cluster, with
alarm information as well as the infrastructure metrics located in the right pane.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 15 of 53
Figure 7.
●
MapR Control System Dashboard Displaying Cluster Health Information
Performance monitoring: The MCS also provides a wide range of metrics for performance
troubleshooting. To access these metrics, choose Cluster > Nodes in the left pane. Then in the Overview
drop-down menu, choose Performance. See Figure 8 for information. You can also click the wrench icon in
the top-right corner to select only the metrics that you are mostly interested in viewing.
Figure 8.
MapR Control System Performance Metrics
Alternatively, you can use the MCS representational state transfer (REST) API to collect performance metrics in a
log file for later analysis. For example, this command uses the REST API to collect all metrics and output them in
JavaScript Object Notation (JSON) format:
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 16 of 53
curl -k -s -u <MCS username>:<password> 'https://<webserver
node>:8443/rest/node/list?limit=50&start=0' | python -mjson.tool
Syslog
An important part of network server and device management is regular review of log messages. Log messages can
also be used forensically to troubleshoot network problems. On many types of systems including Linux and UNIX
servers and various networking devices such as switches and routers, system message logging is performed using
a standardized format known as syslog messages. One way to improve IT management and administration is to
centralize syslog messages from all the diverse devices on a corporate network on a single syslog server or log
host. Centralization allows the use of automated log analysis tools to alert and search for specific message types,
improving the tools available to system administrators to manage networks. This section describes how to
configure a Red Hat Enterprise Linux (RHEL) 6 server to act as a simple centralized log host, and how to configure
RHEL servers to log system messages to that host over the network.
Syslog-ng
Syslog-ng is the next generation of syslog, the logger that has been part of UNIX and Linux for many years. It is
designed to allow flexible logging of system messages from various systems in different formats, including text
files, databases, email messages, and more. It also has sophisticated filtering mechanisms that allow different
system messages for a given host to be routed to different logging mechanisms depending on type or severity
level. For example, messages with a low severity level could be logged to file, while messages with a higher
severity level could be logged to a file and emailed to the system administrator’s mobile phone for immediate
action.
On the syslog server, syslog-ng is installed to consolidate the syslog information from all appliance components in
one place. Before configuring any Cisco device to send syslog messages, make sure that the devices are
configured with the right date, time, and time zone. Syslog data is useless for troubleshooting if it uses the wrong
date and time.
The syslog-ng software at the server accepts syslog information from the network and use the /var/log/HOSTS
directory to store all the data. The syslog-ng software allows creation of a directory structure in the base directory
to make the information more readable. If the syslog agent used in the enterprise monitoring software does not
support this feature, the configuration can be changed.
Here is a sample structure:
/var/log/HOSTS/$YEAR-$MONTH/$HOST/$FACILITY-$YEAR-$MONTH-$DAY"
/etc/syslog-ng/syslog-ng.conf
…
…
…
#source my_src { .... };
#
source src {
#
# include internal syslog-ng messages
# note: the internal() source is required!
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 17 of 53
#
internal();
#
# the default log socket for local logging:
#
unix-dgram("/dev/log");
#
# uncomment to process log messages from network:
#
udp(ip("0.0.0.0") port(514));
};
…
…
…
#
# Enable this, if you want to keep all messages in one file:
# (don't forget to provide logrotation config)
#
#destination allmessages { file("/var/log/allmessages"); };
#log { source(src); destination(allmessages); };
# this is for separating out network hosts into individual log files.
destination std {
file ("/var/log/HOSTS/$HOST/$FACILITY-$YEAR-$MONTH-$DAY"
owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)
};
};
log {
source(src);
destination(std);
};
…
…
…
Be sure to check that syslog is working locally:
srv01:~ # ls /var/log/HANA/*/srv01/
kern-2012-05-09 syslog-2012-05-09
srv01:~ # cat /var/log/HANA/*/srv01/kern-2012-05-09
May 9 14:47:58 srv01 kernel: klogd 1.4.1, log source = /proc/kmsg started.
srv01:~ # cat /var/log/HANA/*/srv01/syslog-2012-05-09
May 9 14:47:53 srv01 syslog-ng[10731]: syslog-ng starting up; version='2.0.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 18 of 53
Syslog on Cisco UCS
The syslog provides a central point for collecting and processing system logs that you can use to troubleshoot and
audit a Cisco UCS domain. Cisco UCS Manager relies on the Cisco NX-OS Software syslog mechanism and API
and on the syslog feature of the primary fabric interconnect to collect and process the syslog entries. Cisco UCS
Manager manages and configures the syslog collectors for a Cisco UCS domain and deploys the configuration to
the fabric interconnect or fabric interconnects. This configuration affects all syslog entries generated in a Cisco
UCS domain by NX-OS or by Cisco UCS Manager.
You can configure Cisco UCS Manager to do one or more of the following with the syslog and syslog entries:
●
Display the syslog entries in the console or on the monitor.
●
Store the syslog entries in a file.
●
Forward the syslog entries to up to three external log collectors at the location at which the syslog for the
Cisco UCS domain is stored.
Syslog Entry Format
Each syslog entry generated by a Cisco UCS component is formatted as follows:
Year month date hh:mm:ss hostname %facility-severity-MNEMONIC
description
For example: 2007 Nov 1 14:07:58 excal-113 %MODULE-5-MOD_OK: Module 1 is online
Syslog Entry Severity Levels
A syslog entry is assigned a Cisco UCS severity level by Cisco UCS Manager. Table 1 shows how the Cisco UCS
severity levels map to the syslog severity values.
Table 1.
Syslog Severity Levels
cisco ucs severity level
syslog severity level
crit
crit
major
Err
minor
Warning
warning
Notice
info
Info
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 19 of 53
Syslog Entry Parameters
Table 2 presents the information contained in each syslog entry.
Table 2.
Syslog Parameters
Name
Description
Facility
Logging facility that generated and sent the syslog entry. The facilities
are broad categories that are represented by integers. These sources
can be one of the following standard Linux facilities:
● local0
● local1
● local2
● local3
● local4
● local5
● local6
● local7
Severity
Severity of the event, alert, or issue that caused the syslog entry to be
generated. The severity value can be one of the following:
● Emergency
● Critical
● Alert
● Error
● Warning
● Information
● Notification
● Debugging
Hostname
Host name included in the syslog entry that depends on the component
in which the entry originated, as follows:
● The fabric interconnect, Cisco UCS Manager, or the host name of
the Cisco UCS domain
● For all other components, the host name associated with the virtual
interface (VIF)
Timestamp
Date and time when the syslog entry was generated.
Message
Description of the event, alert, or issue that caused the syslog entry to
be generated.
Syslog Services
The following Cisco UCS components use the NX-OS syslog services to generate syslog entries for system
information and alerts:
●
I/O module: All syslog entries are sent by the syslog daemon (syslogd) to the fabric interconnect to which it
is connected.
●
Cisco Integrated Management Controller (IMC): All syslog entries are sent to the primary fabric interconnect
in a cluster configuration.
●
Adapter: All syslog entries are sent by NIC-Tools/Syslog to both fabric interconnects.
●
Cisco UCS Manager: Self-generated syslog entries are logged according to the syslog configuration.
Syslog Configuration
A sample syslog configuration is shown here.
scope monitoring
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 20 of 53
set syslog remote-destination server-1 hostname 192.168.76.6 level notification
facility local7
commit-buffer
enable syslog remote-destination server-1
commit-buffer
set syslog remote-destination server-2 hostname 192.168.76.7 level notification
facility local7 commit-buffer
enable syslog remote-destination server-2
commit-buffer
exit
Syslog Severity-Level Parameters
Table 3 describes the severity levels used in system messages. When you configure the severity level, the system
outputs messages at that level and below.
Table 3.
Syslog Severity-Level Entries
Level
Description
0 – Emergency
System unusable
1 – Alert
Immediate action needed
2 – Critical
Critical condition
3 – Error
Error condition
4 – Warning
Warning condition
5 – Notification
Normal but significant condition
6 – Informational
Informational message only
7 – Debugging
Appears during debugging only
The switch logs the most recent 100 messages of severity-level 0, 1, or 2 to the nonvolatile RAM (NVRAM) log.
You cannot configure logging to NVRAM. You can configure which system messages should be logged based on
the facility that generated the message and its severity level.
Configure Syslog-ng on the SAP HANA Nodes
Now that the log host is ready to receive log messages from hosts on the network, you have to configure your
hosts to send messages to it.
1.
Change the syslog-ng.conf file as follows:
/etc/syslog-ng/syslog-ng.conf
…
#
# Enable this and adopt IP to send log messages to a log server.
#
destination logserver1 { udp("192.168.76.6" port(514)); };
log { source(src); destination(logserver1); };
destination logserver2 { udp("192.168.76.7" port(514)); };
log { source(src); destination(logserver2); };
2.
Restart the syslog daemon:
cishanar02:~# /etc/init.d/syslog restart
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 21 of 53
Simple Network Management Protocol Traps
This section explains how Cisco UCS SNMP traps work and how Cisco UCS faults are reported.
Configure SNMP Traps
Perform these steps to configure Cisco UCS SNMP services:
1.
Enable SNMP.
2.
Create the trap receiver (configure the trap host). The trap receiver typically is the IP address of the event
monitoring services (EMS) tool (for example, HP OpenView).
3.
Create an SNMPv3 user.
Cisco UCS Fault Trap Attribute
Cisco UCS SNMP trap attributes provide all the fault details to identify the nature and cause of the fault that Cisco
UCS Manager detects.
Cisco UCS faults are reported through SNMP traps (Figure 9).
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 22 of 53
Figure 9.
Cisco UCS Faults Reported Through SNMP Traps
1.
In the Navigation pane, click the Equipment tab.
2.
On the Equipment tab, expand Equipment > Chassis > Chassis Number > Servers.
3.
Choose the server for which you want to view the fault results.
4.
In the work pane, click the Faults tab.
5.
In the Details area, view all the Cisco UCS fault trap attributes for fault details.
Cisco UCS Fault Types
The fault type tells you where the problem is located. It tells you which component, equipment, or server has
detected a failure.
Nine fault types are available:
●
FSM: The finite state machine (FSM) failed to complete successfully, or Cisco UCS Manager is retrying one
of the FSM stages.
●
Equipment: Cisco UCS Manager has detected that a physical component is inoperable.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 23 of 53
●
Server: Cisco UCS Manager is unable to complete a server task. For example, it is unable to associate a
service profile with a blade.
●
Configuration: Cisco UCS Manager is unable to successfully configure a component.
●
Environment: Cisco UCS Manager has detected a power problem, thermal problem, voltage problem, or
loss of CMOS settings.
●
Management: Cisco UCS Manager has detected a serious management problem. For example, it failed to
start critical services, it failed to elect the primary switch, the VIF is down (the virtual interface 705 link state
is down), or it detected software version incompatibilities.
●
Connectivity: Cisco UCS Manager has detected a connectivity problem. For example, the adapter is
unreachable.
●
Network: Cisco UCS Manager has detected a network problem. For example, a link is down.
●
Operational: Cisco UCS Manager has detected an operational problem, such as a log capacity issue or
discovery failure. For example, a log data store has reached its maximum capacity and is unable to transfer
files, or server (service profile) discovery failed.
Refer to the Cisco UCS Faults Reference for a complete list of faults in Cisco UCS.
The example in Figure 10 shows a connectivity problem.
Figure 10.
Cisco UCS Connectivity Fault: Example
The fault type is listed as “connectivity.” The root cause is an unreachable adapter.
To verify that the Cisco UCS fault management information base (MIB) is working, enter the snmpwalk command.
You can integrate Cisco UCS Manager with HP OpenView. OpenView can be configured to load the Cisco UCS
fault MIB to verify MIB presence, SNMP traps, and device discovery.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 24 of 53
Backup and Recovery
You can use file-based backup, storage snapshots, and mirroring for backup and recovery.
File-Based Backup
For details about file-based backup, refer to the SAP HANA administration guide:
http://help.sap.com/hana/SAP_HANA_Administration_Guide_en.pdf.
Prepare for File-Based Backup
Before beginning a file-based backup operation, do the following:
1.
Create a MapR volume as the backup destination:
maprcli volume create -name <volumename> -path
/apps/hana/<SID>/<backup_mountpoint> -replicationtype high_throughput replication 2 -minreplication 2
2.
Add the volume to /etc/fstab:
<virtual_IP>:/mapr/<clustername>/apps/hana/<SID>/<backup_mountpoint>
<path_of_backup_destination> nfs nolock,hard,timeo=600 0 0
3.
Change the data and log backup destination in the SAP HANA Backup Console:
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 25 of 53
Perform File-Based Data Backup
Follow these steps to perform the backup operation:
1.
Right-click the system that you want to back up and choose Backup and Recovery > Back Up System.
2.
Check the parameters. To perform a regular file backup, use the following settings:
●
Backup Type: Complete Data Backup
●
Destination Type: File
●
Backup Destination: <backup_directory>
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 26 of 53
●
3.
Backup Prefix: COMPLETE_DATA_BACKUP
Click Finish.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 27 of 53
4.
The backup process starts now. Monitor the progress:
5.
If every backup process reaches 100%, the backup is concluded.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 28 of 53
6.
After finishing the file backup, look at the Backup Catalog:
Storage Snapshot
You can use a storage snapshot to restore SAP HANA data files. The process described here does not capture the
log area, automated log backups, or binary files. You can use the captured data files to restore data after a disaster
or to clone a system.
These are the overall steps to take a snapshot:
●
Prepare for the storage snapshot in the SAP HANA database.
●
Take a snapshot of the actual data volumes on the MapR Converged Data Platform.
●
Confirm the storage snapshot in the SAP HANA database.
After you have taken a snapshot, you can copy the data files in the snapshot directory to a safe location. The
content of the snapshot directory is read-only and cannot be changed. Therefore, as long as the snapshot
procedure was performed correctly, the content in the snapshot directory is always application consistent.
However, there is no consistency check at the block level.
These are the overall steps to restore data from a snapshot:
●
Stop the SAP HANA system.
●
Copy the files from the snapshot directory to the actual location of the data files.
●
Start the recovery process in SAP HANA.
This process recovers the database to the point at which the snapshot in SAP HANA was triggered. It clears the
log area and does not include log backups. You may be able to recover to a specific point in time from a snapshot
if sufficient log backups are provided. For information about this procedure, refer to the SAP HANA administration
guide.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 29 of 53
Create a Snapshot
You can create a storage snapshot from the SAP HANA Studio and MapR MCS GUI or from the command line.
GUI Process
1.
Connect to SAP HANA Studio.
2.
Right-click the system for which a storage snapshot should be performed. Choose Backup and Recovery >
Manage Storage Snapshot.
3.
There should be no active snapshot. Therefore, the status should be "Currently no snapshot prepared.” For
Actions, select Prepare.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 30 of 53
4.
Add a comment to make it easier to find the snapshot in the snapshot database for later tasks. A good practice
is to use the external snapshot ID as the comment. The external snapshot ID can be chosen freely in the
MapR Converged Data Platform. This example uses the YYYYMMDD-NNN format.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 31 of 53
5.
Click OK. On the Overview tab of the SAP HANA Studio Backup Console, the time stamp when SAP HANA
was prepared for storage backup is shown in the status information for the currently active data backup.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 32 of 53
6.
Now the system is ready to take a snapshot on the storage layer. Log in to the MapR MCS GUI and select
Volumes in the navigation pane. Select all data volumes of the SAP HANA system that just has been prepared
for a storage snapshot. Click Volume Actions and choose Snapshot <n> Volumes.
7.
As the snapshot name, a good practice is to use the same name as in SAP HANA. The snapshot name will
also be the name of the read-only snapshot directory in which the content of the snapshot can be found.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 33 of 53
8.
Verify the creation of the storage snapshot by browsing to Snapshots in the navigation pane.
9.
After a storage snapshot is successful, you need to confirm it in SAP HANA Studio. The time between taking a
snapshot and confirming a snapshot should be as short as possible. Otherwise, the snapshot delta information
becomes very large and additional space is allocated. Right-click the system in which a storage snapshot
should be confirmed. Choose Backup and Recovery > Manage Storage Snapshot.
10. If the storage snapshot on the MapR Converged Data Platform was not successful, or if you want to abort the
creation of the snapshot, select Abandon.
11. If the storage snapshot on the MapR Converged Data Platform was successful, select Confirm.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 34 of 53
12. As a good practice, you should match the external backup ID with the snapshot name on the MapR
Converged Data Platform.
The storage snapshot is now created.
Command-Line Process
Use these steps to create a snapshot from the command line:
1.
Create a snapshot in SAP HANA:
echo "BACKUP DATA CREATE SNAPSHOT COMMENT '<snap_id>'" |
/usr/sap/<SID>/HDB<instance_no>/exe/hdbsql -i <instance_no> -n <hostname> -u
system -p <password>
2.
Create a snapshot on MapR for every data volume:
maprcli volume snapshot create -volume <datavolume_no> -snapshotname <snap_id>
3.
Fetch the backup ID of the running snapshot in SAP HANA:
echo "select BACKUP_ID from M_BACKUP_CATALOG where COMMENT = '<snap_id>'" |
/usr/sap/<SID>/HDB<instance_no>/exe/hdbsql -i <instance_no> -n <hostname> -u
system -p <password>
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 35 of 53
4.
Close the current snapshot in SAP HANA:
echo "BACKUP DATA CLOSE SNAPSHOT BACKUP_ID <backup_id> SUCCESSFUL '<snap_id>'" |
/usr/sap/<SID>/HDB<instance_no>/exe/hdbsql -i <instance_no> -n <hostname> -u
system -p <password>
Access Snapshot Data
To access the snapshot data, browse the snapshot directory on the MapR Converged Data Platform:
ls -la /mapr/<clustername>/apps/hana/<mountpoint>/.snapshot/<snap_id>/
This directory is read-only. The data in here will not change. Only discarding the snapshot will remove the data.
Recover Using a Snapshot
Before overwriting the data files at the destination, shut down the SAP HANA system.
Before recovering a SAP HANA system from a snapshot, copy the data files from the SAP HANA system from the
snapshot directory to the original destination:
hadoop fs -cp -f
/mapr/<clustername>/apps/hana/<SID>/data0000<no>/.snapshot/<snap_id>/*
/mapr/<clustername>/apps/hana/<SID>/data0000<no>/
This process will overwrite the files in the destination folder.
The file ownership and permissions are not preserved when you use the hadoop fs -cp command. For every data
volume, you must change the ownership and permissions to match the files in the snapshot. To do so, use chown
and chmod with the --reference option:
find /mapr/<clustername>/apps/hana/<SID>/data0000<no>/.snapshot/<snap_id> -printf
'%P\n' | xargs -I {} chown --reference
/mapr/<clustername>/apps/hana/<SID>/data0000<no>/.snapshot/<snap_id>/{}
/mapr/<clustername>/apps/hana/<SID>/data0000<no>/{}
find /mapr/<clustername>/apps/hana/<SID>/data0000<no>/.snapshot/<snap_id> -printf
'%P\n' | xargs -I {} chmod --reference
/mapr/<clustername>/apps/hana/<SID>/data0000<no>/.snapshot/<snap_id>/{}
/mapr/<clustername>/apps/hana/<SID>/data0000<no>/{}
Now the snapshot files are restored with the correct ownership and permissions in the actual data directory of SAP
HANA. The SAP HANA system is now ready to trigger the recovery process.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 36 of 53
GUI Process
To recover data using a snapshot, follow these steps at the GUI:
1.
In SAP HANA Studio, right-click the system. Choose Backup and Recovery > Recover System.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 37 of 53
2.
Specify the recovery type. Here, select “Recover the database to a specific data backup or storage snapshot.”
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 38 of 53
3.
Specify the backup location. Here, select “Specify backup without catalog.”
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 39 of 53
4.
Specify the backup to recover. For Destination Type, choose Snapshot.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 40 of 53
5.
Review the settings.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 41 of 53
6.
Observe the recovery progress.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 42 of 53
7.
The recovery process is concluded.
Command-Line Process
You can also restore the data from a snapshot using the command line.
As <sid>adm, enter the following command:
HDBSettings.sh recoverSys.py --wait --command="RECOVER DATA USING SNAPSHOT CLEAR
LOG"
Mirroring
You can use a MapR mirror instead of a snapshot for backup. When you use a mirror, the SAP HANA data is
copied immediately to another volume, which allows you to recover your data instantaneously. No time-consuming
copying of data from the snapshot to the original volume is required.
Create a Mirror
You can create a storage snapshot from the SAP HANA Studio and MapR MCS GUI or from the command line.
echo "BACKUP DATA CREATE SNAPSHOT COMMENT '<snap_id>'" |
/usr/sap/ANA/HDB00/exe/hdbsql -i 00 -n cishana01 -u system -p SAPhana10!
echo "select BACKUP_ID from M_BACKUP_CATALOG where COMMENT = '<snap_id>'" |
/usr/sap/ANA/HDB00/exe/hdbsql -i 00 -n cishana01 -u system -p SAPhana10!
On a MapR node, enter the following:
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 43 of 53
CLUSTER=mapr500 (or whatever the cluster name is)
maprcli volume create -type mirror -source ANAdata0000$i@$CLUSTER -name
ANAdata0000$i.1
maprcli volume mirror start -name ANAdata0000$i.1
echo "BACKUP DATA CLOSE SNAPSHOT BACKUP_ID <backup_id> SUCCESSFUL '<snap_id>'" |
/usr/sap/ANA/HDB00/exe/hdbsql -i 00 -n cishana01 -u system -p SAPhana10!
Recover Using a Mirror
The following command will return a mirror-percent-complete value of 100 if the mirror copy is complete and
available to be restored. If this command returns a value less than 100, repeat the command until the mirror is
completed.
maprcli volume info -name ANAdata0000$i.1 -columns mirror-percent-complete
maprcli volume unmount -name ANAdata0000$i
maprcli volume modify -type rw -name ANAdata0000$i.1
maprcli volume mount -name ANAdata0000$i.1 -path /apps/hana/ANA/data0000$i
Rather than create a new mirror volume for the next mirror operation, you can modify the original volume to make it
the mirror volume:
maprcli volume modify -type mirror -source ANAdata0000$i.1@$CLUSTER -name
ANAdata0000$i
Then you can use a maprcli mirror start command to start a mirror operation back to the original volume. This
way, you can toggle between a pair of volumes for mirror backups. Subsequent mirror operations will copy only
blocks that have been modified.
maprcli volume mirror start -name ANAdata0000$i
maprcli volume info -columns mirror-percent-complete
maprcli volume unmount -name ANAdata0000$i.1
maprcli volume modify -type rw -name ANAdata0000$i
maprcli volume mount -name ANAdata0000$i -path /apps/hana/ANA/data0000$i
Mirror for Disaster Recovery
Creating a mirror for disaster recovery is similar to creating a mirror for backup. Instead of using the local cluster
name in the -source parameter of the maprcli commands, use the name of the remote cluster. Note that this
cluster and its CLDB nodes must be specified in /opt/mapr/conf/mapr-clusters.conf.
SAP HANA QA System Refresh
The worth of a QA system mainly consists in its data actuality. In order to keep it up to date, the productive system
can be cloned without interruption and a new QA system can be created. The HANA Lifecycle Manager is able to
configure the cloned volumes, which contain all data from the productive system, and create a new system with a
new SID on a new host.
In our example below we clone system ANA on host cishana05 to system ANB on host cishana06. The MapR
cluster is named mapr500.
Prepare Mirror Volumes on MapR
First, mirror volumes can be created. The source volumes are the volumes of the original HANA system. Mirroring
does not start immediately, so this step does not really clone data, but provides empty volumes where mirroring
action can be triggered at a later point in time.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 44 of 53
maprcli volume create -type mirror -source ANAdata00001@mapr500 -name
ANBdata00001 -path /apps/hana/ANB/data00001
maprcli volume create -type mirror -source ANAlog00001@mapr500 -name ANBlog00001
-path /apps/hana/ANB/log00001
maprcli volume create -type mirror -source ANAshared@mapr500 -name ANBshared path /apps/hana/ANB/shared
Prepare HANA System
Before mirroring can take place, HANA should be in a consistent state. The storage snapshot preparation of HANA
is the correct way to achieve integrity of the data volumes.
echo "BACKUP DATA CREATE SNAPSHOT COMMENT 'snapshot20161201001'" |
/usr/sap/ANA/HDB00/exe/hdbsql -i 00 -n cishana05 -u system -p SAPhana10!
echo "select BACKUP_ID from M_BACKUP_CATALOG where COMMENT =
'snapshot20161201001'" | /usr/sap/ANA/HDB00/exe/hdbsql -i 00 -n cishana05 -u
system -p SAPhana10!
Write down the BACKUP_ID. In our example, the BACKUP_ID is 1479984013513.
Mirror HANA system
Once HANA is in a storage snapshot state, the mirror action of the volumes can be triggered. Execute on a MapR
cluster node:
maprcli volume mirror start -name ANBdata00001
maprcli volume mirror start -name ANBlog00001
maprcli volume mirror start -name ANBshared
You don't have to wait until the actual mirroring action completes before you proceed. The source volumes will be
snapshotted internally, so integrity of the source volumes is given even when HANA storage snapshot will be
closed in the next step.
To close the storage snapshot on the HANA system, execute at the HANA node:
echo "BACKUP DATA CLOSE SNAPSHOT BACKUP_ID 1479984013513 SUCCESSFUL
'snapshot20161201001'" | /usr/sap/ANA/HDB00/exe/hdbsql -i 00 -n cishana05 -u
system -p SAPhana10!
Break Mirror
Before you break the mirror, you have to wait until the actual mirror action is completed. Remember that this type
of mirroring in MapR is not an ongoing process, but a copy of data from the source to the mirror volume at one
point in time only. Once this copy (mirroring) process is done, you can modify the mirror volume to become
writeable:
maprcli volume modify -name ANBdata00001 -type rw
maprcli volume modify -name ANBlog00001 -type rw
maprcli volume modify -name ANBshared -type rw
Configure destination system
Create directories where the mirrored volumes can be mounted to:
mkdir -p /hana/shared/ANB
mkdir -p /hana/data/ANB/mnt00001
mkdir -p /hana/log/ANB/mnt00001
Edit /etc/fstab and mount the directories:
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 45 of 53
mapr04nfs:/mapr/mapr500/apps/hana/ANB/shared
nolock,hard,timeo=600
0 0
/hana/shared/ANB
nfs
mapr04nfs:/mapr/mapr500/apps/hana/ANB/data00001
nolock,hard,timeo=600
0 0
/hana/data/ANB/mnt00001 nfs
mapr05nfs:/mapr/mapr500/apps/hana/ANB/log00001
nolock,hard,timeo=600
0 0
/hana/log/ANB/mnt00001
nfs
Execute hdblcm:
[root@cishana06 hdblcm]# /hana/shared/ANB/hdblcm/hdblcm
SAP HANA Lifecycle Management - SAP HANA 1.00.110.00.1447753075
***************************************************************
Choose an action to perform
Index | Action to be performed | Description
---------------------------------------------------------------------------1
| add_hosts
2
| register_rename_system | Register and Rename SAP HANA System
| Add Additional Hosts to the SAP HANA System
3
| exit
| Exit (do nothing)
Enter selected action index [3]: 2
Target installation directory '/hana/shared/ANB' already exists
Own Host Name: cishana06
The HANA system is installed with SID mountpoint. Based on the file system
layout, the system id will be changed from ANA to ANB
Enter Target Host Name for Source Host 'cishana05' [cishana06]:
The hdblcm detected that we already mounted the cloned volumes to the file system layout of the new SID which is
ANB. Therefore, it chooses this as the new SID automatically. Because the own host name is cishana06, this host
name will be suggested as target host name.
After some more queries (like backup folders, passwords, etc.), hdblcm starts the renaming of the instance. Once
hdblcm has successfully finished the renaming process, the target system is ready to use.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 46 of 53
Hardware Maintenance
This section discusses hardware maintenance for the infrastructure components, MapR Converged Data Platform
cluster, and SAP HANA cluster.
Infrastructure Components
If an infrastructure component, such as a Cisco Nexus switch, fabric interconnect, or chassis or I/O module fails,
open a ticket with Cisco Technical Assistance Center (TAC) Solution Support to get a replacement and for
assistance with the replacement process.
Because the configuration is fully redundant, the SAP HANA system should be fully functional but a little slower
because half of the I/O throughput may be missing in some failure scenarios.
MapR Converged Data Platform Cluster
If a MapR Converged Data Platform cluster fails, use the procedures described here.
Replace Failed Disk: Single-Disk Failure
A failure of a single disk is covered by the RAID 5 configuration of the server. Remember that a storage node
consists of four virtual disks in RAID 5 with six physical disks each. So if one disk fails, just replace it. The rebuild
starts automatically as long as the disk is shown as “unconfigured good.” For details, see
http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c/hw/C240M4/install/C240M4.html.
Replace Failed Disk: Multiple-Disk Failure
A multiple-disk failure in the same RAID group destroys the virtual disk, and therefore also the storage pool in
MapR, which consists of only one virtual disk. In this case, re-create the virtual disk with six healthy physical disks
and follow procedures in the MapR guide to bring back the storage pool on the node:
http://doc.mapr.com/display/MapR/Handling+Disk+Failures.
Replace Failed Node
When a node failure occurs, all the data in the node is not accessible. These steps are required to replace the
node:
●
Remove the node from the cluster by issuing the maprcli node remove –nodes <failed node name>
command. Wait several minutes until the node disappears from the MCS dashboard. This process will also
force the cluster to begin data replication to the surviving nodes to meet the minimum replication
requirements.
●
If the node is repairable and the all disks are okay, fix the node and then add it back to the cluster by issuing
the command /opt/mapr/server/configure.sh –R. Then start warden services (or zookeeper service if this
is a zookeeper node).
●
If the node is not repairable, install a new node and reinstall the required MapR services that existed in the
failed node on the new node by following the procedures in the installation guide. These steps are the same
as if you were installing the node from scratch.
After the node has been added to the cluster (usually after you issue a configure.sh command according to the
installation guide), the lost replicas will be rereplicated on the new node, and data distribution will take place
according to the topology configuration.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 47 of 53
SAP HANA Cluster
If a SAP HANA cluster fails, use the procedures described here.
Replace Failed Node
When a SAP HANA node fails, SAP HANA scale-out host autofailover promotes the standby node to a worker
node. In such a scenario, your SAP HANA cluster is fully functional, but it is not protected against additional node
outages.
If the blade server experienced a hardware failure, but the OS disks are still good, the disks can be installed in the
replacement server blade, and the old service profile can be assigned. In this case, the server boots with the old
configuration. No additional steps are needed.
If the OS is corrupt, you need to install a new OS and add it to the SAP HANA cluster as a standby node according
to the installation guide. If the server is given the same host name, the service profile of the failed server can be
used, preserving identity information such as MAC addresses. If it is given a new host name, you should create a
new service profile from the appropriate service profile template (HANA-A for server 3, or HANA-B for server 7)
and assign this new profile to the server.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 48 of 53
OS Maintenance
The customer is responsible for implementing security updates and patches and adding software components and
OS settings that may be requested by SAP for future SAP HANA versions or required by Red Hat to help ensure
system security and stability. See related SAP operating system notes for required OS settings.
This section describes how to update the OS and the implications of updating OS components. It is not meant to
replace a Linux administration document, but to address topics that arise from the nature of a SAP HANA–ready
OS.
Prerequisites
Whenever you change the OS or parts of the OS such as drivers and kernel parameters, you should have at least
a backup of your SAP HANA system, preferably stored somewhere other than on the appliance. You also should
check the related OS notes and Cisco support channels.
Some changes may require a reboot and should be applied when SAP HANA is down.
RHEL Online Update
Refer to the installation guide for information about how to register the OS. A properly registered OS can be
updated by entering:
yum update
At the time of writing this guide, RHEL 6.7 is the preferred RHEL version for SAP HANA. Use caution when
performing any package updates using yum because it may upgrade the kernel files beyond Release 6.7 as a
perceived dependency and in effect make it a Release 6.8 system, which would be an unsupported configuration.
Refer to the Red Hat knowledgebase for details: https://access.redhat.com/solutions/1243453.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 49 of 53
Configuration Sheets
Complete the customer information questionnaire before beginning the installation process. All the information
needed for the installation should be listed here.
After successful deployment of the solution, complete the configuration sheet. List all names and identities as well
as the MapR and SAP HANA cluster configurations. This information is helpful for troubleshooting.
Use the forms provided here.
Infrastructure Components
Mgmt
Client
VLAN
Network
Subnet Mask
Nexus 3k
FI A
FI B
UCSM
UCSM KVM
MapR Cluster
Cluster IDs
Cluster Name: ___________________________________________
Networks and IP Addresses
Management
HANA-Storage
MapR 01
MapR 02
MapR 03
VLAN
MTU
QoS
Network CP
Fabric
Network
Subnet Mask
IP Range
Node #01
Node #02
Node #03
Node #04
Node #05
Node #06
Node #07
Node #08
MapR-FS and NFS are applied to every MapR node; therefore, these roles are not reflected in the table.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 50 of 53
Node Roles
Node #
Hostname
Zookeeper
CLDB
Webserver
01
02
03
04
05
06
07
08
SAP HANA Cluster
Cluster IDs
System ID (SID): _________________________________
Instance Number: _________________________________
<sid>adm Password: _________________________________
Database Password: _________________________________
Networks and IP Addresses
HANAStorage
Management NFS IPMI
HANAInternal
HANABackup
HANAClient
HANAAppServer
HANADataSource
HANAReplication
VLAN
MTU
QoS
Network CP
Fabric
Network
Subnet
Mask
IP Range
Node #01
Node #02
Node #03
Node #04
Node #05
Node #06
Node #07
Node #08
Node #09
Node #10
Node #11
Node #12
Node #13
Node #14
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 51 of 53
HANAStorage
Management NFS IPMI
HANAInternal
HANABackup
HANAClient
HANAAppServer
HANADataSource
HANAReplication
Node #15
Node #16
Node #17
Node Roles
Node #
Hostname
Worker / Standby
Storage Partition
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
Virtual IP Addresses
Storage
Partition
Virtual IP Address
Virtual IP hostname
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 52 of 53
MAC Addresses Hosting Virtual IP Addresses
Node #
Hostname
HWADDR
01
02
03
04
05
06
07
08
Printed in USA
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
C11-737719-01
12/16
Page 53 of 53
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising