Installing Cisco Elastic Services Controller on OpenStack. Cisco Elastic Services Controller
Quảng cáo
Quảng cáo
P A R T
I
Installing Cisco Elastic Services Controller on
OpenStack
•
•
Installing Cisco Elastic Services Controller on OpenStack, on page 7
•
Installing High Availability Active/Standby, on page 21
•
Cisco Elastic Services Controller Active/Active High Availability Overview, on page 33
•
Installing Active/Active High Availability Cluster, on page 35
•
Managing Cluster in ESC Active/Active High Availability, on page 41
•
Configuring GEO in Active/Active High Availability, on page 43
•
DRBD Encryption for ESC Active/Standby and Active/Active HA Data Replication , on page 49
•
Upgrading ESC Active/Active High Availability, on page 51
C H A P T E R
2
Prerequisites
The following sections detail the prerequisites for installing Cisco Elastic Services Controller:
•
Virtual Resource Requirements, on page 5
•
Software Requirements, on page 5
•
Preparing for the Installation, on page 6
Virtual Resource Requirements
The following table lists the virtual resource requirements for Cisco Elastic Services Controller:
ESC Deployment Virtual CPU per
ESC VM
Virtual Memory
(GB) per ESC VM
Virtual Hard
Disk (GB)per
ESC VM
ESC Standalone 4 8 30
Number of total
VMs Supported
VIM
1000 All supported
VIMs
ESC
Active/Standby
HA
4
ESC
Active/Active
HA/ GEO HA
8
8
16
30
60
2500
5000
All supported
VIMs
OpenStack Only
Software Requirements
The following table lists the software requirements on OpenStack:
Requirements
Supported Browsers
Description
Google Chrome 44.x or later
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
5
Installing Cisco Elastic Services Controller on OpenStack
Preparing for the Installation
Requirements
OpenStack Version
Description
Any of the following:
• Train
• Ocata (V2 and V3)
• Queens (Keystone V3)
Preparing for the Installation
Before you perform the installation, ensure that you are prepared by reviewing this checklist:
Your Information/ Notes Requirements
For Pre-installation
Configuration
QCOW image location
QCOW image
VM per Instance
IP address
Subnet mask
Hostname
Domain name
Gateway IP address
Admin password
ESC Release Package
ESC.qcow2
bootvm.py
An image file for booting up the ESC instance
The installation script compatible with python 2.7.6 and Python 3.4
6
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
C H A P T E R
3
Installing Cisco Elastic Services Controller on
OpenStack
This chapter describes how to install Cisco Elastic Services Controller on OpenStack and includes the following sections:
•
Installation Scenarios, on page 7
•
Main Components of Cisco Elastic Services Controller Setup, on page 8
•
Installing Cisco Elastic Services Controller Using the QCOW Image, on page 8
•
Managing Root Certificates in Cisco Elastic Services Controller, on page 15
•
Managing Keystore in Cisco Elastic Services Controller, on page 17
•
Using a Bootable Volume in ESC Installation, on page 18
Installation Scenarios
The following sections briefly describe some of the common deployment scenarios that are addressed by ESC.
Cisco Elastic Services Controller can be installed in different modes as per the requirement. These different modes are configured during installation. The following sections briefly describe some of the common deployment scenarios that are addressed by ESC.
ESC Standalone
In the standalone scenario, a single active VM is deployed for ESC.
ESC with HA
ESC supports High Availability (HA) in the form of Active/Standby and Active/Active models. For
Active/Standby model, two ESC instances are deployed in the network to prevent ESC failure and to provide
ESC service with minimum interruption. If the primary ESC instance fails, the standby instance automatically takes over the ESC services. ESC HA resolves the following single point failures of ESC:
• Network failures
• Power failures
• Dead VM instance
• Scheduled downtime
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
7
Installing Cisco Elastic Services Controller on OpenStack
Main Components of Cisco Elastic Services Controller Setup
• Hardware issues
• Internal application failures
Note From ESC 5.0, the name, Active/Passive model is changed to Active/Standby model.
Note Installing or upgrading software on the ESC VM is not supported. For further assistance, contact Cisco TAC.
For more information on deploying ESC HA Active/Standby, see 'Configuring High-Availability
Active/Standby' in Installing ESC on OpenStack and Installing ESC on VMware chapters.
For more information on the Active/Active model, see ESC HA Active/Active Overview Chapter.
Main Components of Cisco Elastic Services Controller Setup
The Cisco Elastic Service Controller (ESC) setup has the following components:
• Virtual Infrastructure Manager —Elastic Services Controller (ESC) and its VNFs are deployed in a
Virtual Infrastructure Manager (VIM). This might have one or more underlying physical nodes.
• ESC Virtual Machine —The ESC VM is a VM that contains all the services and processes used to register and deploy services. This includes the ESC Manager and all other services. ESC provides Netconf
API, REST API, and Portal as north bound interfaces to communicate with ESC. ESC VM contains CLI to interact with ESC VM. There are two CLI, one uses the REST API and the other uses Netconf API.
Installing Cisco Elastic Services Controller Using the QCOW
Image
You can install Cisco Elastic Services Controller (ESC) on OpenStack by using a QCOW image. ESC will be deployed in the OpenStack as running VM instance to manage VNFs. Therefore, ESC need OpenStack environment parameters to be installed in OpenStack. The installation time varies from 10 to 20 minutes, depending on the host and the storage area network load. This procedure describes how to create ESC Virtual
Machine (VM) in OpenStack.
Before you begin
• All system requirements are met as specified in
.
• You have the information identified in
Preparing for the Installation, on page 6 .
• Copy the ESC image file on the system where you want to install ESC.
• This system must be accessible by OpenStack.
8
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Installing Cisco Elastic Services Controller Using the QCOW Image
Procedure
Step 1
Step 2
Step 3
Log in to the system where you want to install ESC.
Check the compatibility of the bootvm.py and the ESC image:
./bootvm.py --version
For more information on the ESC installer arguments, see Appendix: A Cisco Elastic Services Controller
Installer Arguments .
In a text editor, create a file named PROJECT-openrc.sh file and add the following authentication information.
The following example shows the information for a project called admin, where the OpenStack username is also admin, and the identity host is located at controller node.
Note To set the required environment variables for the OpenStack command-line clients, you must create an environment file called an OpenStack rc file, or openrc.sh file. This project-specific environment file contains the credentials that all OpenStack services use. The ESC installation script requires these OpenStack environment parameters to perform authentication and installation on OpenStack.
If all the OpenStack credentials are passed through its own arguments, the bootvm.py script doesn't require these parameters.
Step 4
Step 5 export OS_NO_CACHE=true export OS_TENANT_NAME=admin export OS_USERNAME=admin export OS_PASSWORD=admin_pass export OS_AUTH_URL= http://controller node:35357/v2.0
The other OpenStack parameters required for installation are: --os_auth_url, --os_username, --os_password
, -os_tenant_name, --bs_os_user_domain_name, --bs_os_project_domain_name, --bs_os_identity_api_version,
--bs_os_auth_url, --bs_os_username, --bs_os_password , -bs_os_tenant_name, --bs_os_user_domain_name,
--bs_os_project_domain_name, --bs_os_identity_api_version.
For OpenStack V2 API, you need following items to be defined in your global environment variables:
--os_password, --os_auth_url, --os_username, --os_tenant_name.
For OpenStack V3 API, set --os_identity_api_version=3 . Other parameters required for OpenStack V3 API are: --os_user_domain_name, --os_project_domain_name, --os_project_name, --os_password, --os_auth_url,
--os_username, --os_identity_api_version, --os_ca_cert, --requests_ ca_bundle.
Note The arguments, --os_tenant_name, --os_username, --os_password, --os_auth_url will also by default configure the VIM connector. If you want to skip configuring the VIM connector, pass the parameter
(--no_vim_credentials) with the bootvm.py. When no_vim_credentials parameter is provided, the bootvm.py arguments (os_tenant_name, os_username, os_password,os_auth_url) are ignored. For more information on configuring VIM connectors after installation, and managing VIM connectors, see Managing VIM Connectors in the Cisco Elastic Services Controller User Guide .
Note --os_ca_cert and --requests_ca_bundle arguments are only required for https connection.
On any shell from which you want to run OpenStack commands, source the PROJECT-openrc.sh
file for the respective project. In this example, you source the admin-openrc.sh
file for the admin project .
$ source admin-openrc.sh
Check the environment variables.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
9
Installing Cisco Elastic Services Controller on OpenStack
Installing Cisco Elastic Services Controller Using the QCOW Image
Step 6
Step 7
Step 8
$ env | grep OS_
Register ESC image file in the OpenStack image using the glance command:
$ glance image-create \
--name <image_name> \
--is-public=<true or false> or --visibility public or private\
--disk-format <disk_format> \
--container-format <container_format> \
--file <file> \
--progress
An example configuration is shown below:
$ glance image-create \
--name esc-1_0_01_11_2011-01-01 \
--is-public=<true or false> or --visibility public or private \
--disk-format qcow2 \
--container-format bare \
--file esc-1_0_01_11_2011-01-01.qcow2 \
--progress
The glance image-create command is used to create a new image. The command takes the following arguments:
Note The 'is-public' argument is applicable only for OpenStack Kilo release.
Arguments name is-public disk-format container-format file progress
Description
Name of the image.
(Optional) Makes the image accessible to the public.
Disk format of the image. ESC uses a qcow2 disk format.
Container format of image. ESC uses a bare container format.
Local file that contains disk image to be uploaded during creation.
(Optional) Shows upload progress bar.
To verify whether the image has been registered successfully: a) Using OpenStack Controller dashboard:
• Log into OpenStack using your credentials.
• Navigate to Admin > Image .
Verify if the image appears in the list.
b) Using nova CLI:
$ nova image-show <image_name>
The standard resource requirement of ESC is 4vCPU, 8G RAM, and 30GB disk space. ESC installation script takes the pre-defined "m1.large" flavor which has the definition of 4vCPU, 8G RAM and 80G disk space. To use 30GB disk space, create a flavor with the minimum disk space requirement.
$ nova flavor-create ESC_FLAVOR_NAME ESC_FLAVOR_ID 8192 30 4
To deploy ESC VM, do the following:
10
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Additional Installation Options a.
Ensure that the existing network have connectivity to OpenStack controller. To verify the network connectivity using the nova CLI use:
$ nova net-list b.
Record the ID of the network that ESC connects to boot the ESC VM by with image and flavor created earlier. The bootvm.py command requires at least one --user_pass argument to create an admin account for linux (ssh/console access) and at least one --user_confd_pass to create an admin account for ConfD
(netconf/cli access). The following are the syntax for these mandatory user credential arguments:
--user_pass admin:'PASSWORD-OR-HASH'[:OPTIONAL-PUBLIC-KEY-FILE][:OPTIONAL-ROLE]
--user_confd_pass admin:'PASSWORD-OR-HASH'[:OPTIONAL-PUBLIC-KEY-FILE]
Generating hashed password is optional. You can select a plain password as and when required.
To generate a hashed password on Ubuntu OS, use the following command: mkpasswd --method=SHA-512 --salt XyZ123 <<< <Password>
The following is an example to install ESC with an authorized public key. In the following example, single quotes are used to avoid conflict with shell reserved characters:
--user_pass admin:'$algorithm$salt$hash-of-salt-password':$HOME/.ssh/esc_rsa.pub
--user_confd_pass admin:'$algorithm$salt$hash-of-salt-password':$HOME/.ssh/esc_rsa.pub
The public key are generated as part of a key pair with, such as: ssh-keygen -t rsa -b 1024 -C "esc" -N "" -f ~/.ssh/esc_rsa
Your public key and identification key are saved in /home/username /.ssh/esc_rsa and esc_rsa.pub files.
For more examples on the user credential arguments, see Appendix A: Cisco Elastic Services Controller
Installer Arguments.
c.
To check the details of ESC VM and get the information including the IP address(es) of the ESC VM, use the following command:
$ nova show <esc_vm_name>
Additional Installation Options
• Deploying ESC in an OpenStack IPv6 environment: Before deploying ESC instance in IPv6, make sure to source openrc that supports ipv6 addresses. To deploy ESC in IPv6 environment, use the following bootvm arguments:
./bootvm.py
<esc_vm_name-ipv6> --poll --user_rest_pass <username> : <password> --image
<image_name>
--net <ipv6_network> --ipaddr <ipv6_ip_address> --enable-http-rest --user_pass
<username> : <password>
--user_confd_pass <username> : <password> --etc_hosts_file <hosts-file-name> --route
<default routing configuration>
• Deploying ESC in DHCP mode: If you use the bootvm.py argument without --ipaddr, then the ESC instance will be deployed in a DHCP mode. To deploy ESC in a DHCP network, use the following configuration:
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
11
Installing Cisco Elastic Services Controller on OpenStack
Additional Installation Options
./bootvm.py
<esc_vm_name> --image <image_name> --net <IPv6 network> <IPv4 network>
--flavor <flavor_name>
--user_pass <username> : <password>
--user_confd_pass <username> : <password>
Note By default, ESC only support DHCP in IPv4 networks. If IPv6 is used, you need to log in the ESC VM and run "dhclient -6 ethX" (ethX is the V6 interface name) manually to enable V6 DHCP.
When deploying ESC with multiple network interfaces, with one or more type of DHCP, use bootvm.py --defroute N to specify the interface index to be assigned default route and gateway IP.
By default,
N = 0
. So the first network interface on the bootvm.py
command line is default.
For Example:
--defroute 1 will assign <network2> with <default_gateway_ip_address>
./bootvm.py <esc_vm_name> --image <image_id> --net <network1>
<network2> --defroute 1 --gateway_ip <default_gateway_ip_address>
• Using a Bootable Volume in ESC Installation: You can attach a volume to an ESC instance and launch an instance from inside the volume. For more information, see the section
• Assigning Floating IP to the ESC: If you want to associate a floating IP with the ESC instance , do the following:
1.
Check for an available floating IP address and assign it to the ESC VM:
$ nova floating-ip-list
$ nova floating-ip-associate esc_vm_name <ip_address>
2.
Or create a new floating IP address and assign it to the ESC VM:
$ nova floating-ip-create <FLOATING_NETWORK - ID>
$ nova floating-ip-associate esc_vm_name <ip_address> or neutron floatingip-create FLOATING_NETWORK neutron floatingip-associate floating-ip-ID port-ID
• Deploying ESC with static IPs: To use ESC in a specific network with static IPs, for example,
192.168.0.112 at network1, specify --ipaddr and --gateway_ip to the bootvm command line , as shown below:
Note Before assigning static IP, make sure the static IP is available and is not being used on other machine.
./bootvm.py
<esc_vm_name> --image <image_id> --net <network>
--ipaddr <ip_address> --gateway_ip <default_gateway_ip_address> --user_pass
<username> : <password>
--user_confd_pass <username> : <password>
12
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Additional Installation Options
• Deploying ESC with multiple network interfaces: To use multiple networks for ESC, for example,
192.168.0.112 at network1 and 10.20.0.112 at network2, specify both the IP addresses and the network names of the interfaces in the --net and --ipaddr arguments in the following command line. In addition, also choose the default gateway for ESC from the gateways of these networks. Specify the default gateway for ESC through the --gateway_ip argument.
./bootvm.py
<esc_vm_name> --image <image_id> --net <network1>
<network2> --ipaddr <ip_address1> <ip_address2> --gateway_ip <default_gateway_ip_address>
--user_pass <username> : <password>
--user_confd_pass <username> : <password>
Note If --flavor is not specified, bootvm.py will use the default flavor "m1.large" in
OpenStack.
• Deploy ESC with log forwarding options: To forward ESC logs to an rsyslog server, specify the IP address of the rsyslog server while creating an ESC VM. Optionally, you can also specify the port and protocol to use.
For example, if the IP address of the rsyslog server is 172.16.0.0 the port on the server to forward logs is 514, and the protocol used is UDP, the ESC installation could be
./bootvm.py
<esc_vm_name> --image <image_id> --net network1 --rsyslog_server 172.16.0.0
--rsyslog_server_port 514 --rsyslog_server_protocol udp --user_pass <username> : <password>
--user_confd_pass <username> : <password>
• Disabling the ESC GUI: To boot up ESC VM with the graphical user interface disabled, modify the
--esc_ui_startup argument value, as shown in the command line below:
./bootvm.py
<esc_vm_name> --image <image_id> --net <network> --user_pass
<username> : <password>
--user_confd_pass <username> : <password>
--esc_portal_startup=False
• Enabling REST interface for ESC: To support the REST interface, specify --enable-https-rest argument.
You can activate REST interface on both https or http:
./bootvm.py
<esc_vm_name> --image <image_id> --net <network> --user_pass
<username> : <password>
--user_confd_pass <username> : <password> --enable-https-rest
OR
./bootvm.py
<esc_vm_name> --image <image_id> --net <network> --user_pass
<username> : <password>
--user_confd_pass <username> : <password> --enable-http-rest
•
• Enabling REST interface for ETSI: To support the ETSI REST intetrface, specify --enable-http-etsi to activate the interface over http, or --enable-https-etsi to activate the interface over https.
./bootvm.py
<esc_vm_name> --image <image_id> --net <network> --user_etsi_pass
<username> : <password> --enable-https-rest . . .
OR
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
13
Installing Cisco Elastic Services Controller on OpenStack
Additional Installation Options
./bootvm.py
<esc_vm_name> --image <image_id> --net <network> --user_etsi_pass
<username> : <password>
--user_confd_pass <username> : <password> --enable-http-etsi-rest...
Note Only the https REST and ETSI interfaces should be enabled in a production environment.
• Deploying ESC with global parameters: To set the global configurations through the esc_params_file during the installation, use the arguments as shown below. These global configurations can also be changed through REST API after the installation.
Note The default security group is applied to the tenant during tenant creation. By default, the ESC configuration parameter for the security group, openstack.DEFAULT_SECURITY_GROUP_TO_TENANT
is set to true. The configuration parameter must be set at the time of installation. You can query or update the parameter on ESC VM through the REST API. If the parameter is set to true, you can create and assign default security group during tenant creation.
If the parameter is set to false, you cannot create or assign default security group during tenant creation. For details on the parameters that can be configured through esc_params_file, see Appendix A: Cisco Elastic Services Controller Installer
Arguments.
./bootvm.py <esc_vm_name> --image <image_id> --net <network> --flavor <flavor_name>
--user_pass <username>:<password>:<public key file> --user_confd_pass
<username> : <password>
--esc_params_file <esc parameter configuration file>
• Deploying two instances of ESC to build an ESC HA (Active/Standby) pair : For more information on deploying ESC HA (Active/Standby), see Configuring High Availability in Installing ESC on
OpenStack and Installing ESC on VMware chapters.
• Adding a Dynamic Mapping File : In Cisco ESC Release 2.1 and earlier, mapping the actions and metrics defined in the datamodel to the valid actions and metrics available in the monitoring agent was enabled using the dynamic_mappings.xml
file. The file was stored in the ESC VM and was modified using a text editor. ESC 2.2 and later do not have an esc-dynamic-mapping directory and dynamic_mappings.xml
file. If you want to add an existing dynamic_mapping xml file to the ESC VM, do the following:
1.
Backup this file to a location outside of ESC, such as, your home directory.
2.
Create esc-dynamic-mapping directory on your ESC VM. Ensure that the read permissions are set.
3.
Install on your ESC VM using the following bootvm argument:
--file root:root: /opt/cisco/esc/esc-dynamic-mapping/dynamic_mappings.xml
:< path-to-local-copy-of-dynamic-mapping.xml
>
The CRUD operations for mapping the actions and the metrics is available through REST API. To update an existing mapping, delete and add a new mapping through the REST API.
• Changing the confd password on an ESC VM : As an administrator, you can configure the confd password through bootvm.py, during the installation time:
14
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Managing Root Certificates in Cisco Elastic Services Controller
./bootvm.py --user_pass <username> : <password> --user_confd_pass admin:'PASSWORD-OR-HASH':OPTIONAL-PUBLIC-KEY
To reconfigure this password after the installation, execute the following commands:
$ /opt/cisco/esc/confd/bin/confd_cli -u admin
$ configure
$ set aaa authentication users user admin password <your_password>
$ commit
$ exit
Note For the ease of future upgrades, make sure that you keep a copy of all the commands and arguments that are used while installing ESC using the bootvm.py file.
Managing Root Certificates in Cisco Elastic Services Controller
Cisco Elastic Services Controller (ESC) provides a mechanism to enable verification of SSL certificates.
Currently, this feature is supported only on OpenStack. Certificate validation is enabled by default during the initial ESC boot up. However, ESC also allows you to configure these SSL certificates. This section describes how to enable/disable certificate validation, add/remove, or list the certificates for Cisco Elastic Service
Controller on OpenStack. You can add a root certificate during the ESC bootup or even after ESC bootup is completed.
Enabling/Disabling the Root Certificate Validation
Cisco Elastic Services Controller by default enables certificate validation. You can also enable or disable by modifying the parameter, DISABLE_CERT_VALIDATION, available under the Openstack category in the esc_params.conf file, or through the REST interface, or using the escadm tool.
On ESC master node, use the command, sudo escadm enable-certificate or sudo escadm disable-certificate to enable and disable the certificate validation, respectively.
Adding a Root Certificate
You can add a root certificate during the ESC bootup or even after ESC bootup is completed. Before adding certificates, ensure the OpenStack environment file, OpenStack RC file has parameters to perform authentication and installation on OpenStack. The --os_auth_url must be specified while passing the parameters. --os_auth_url specifies the secure (https) or unsecured (http) keystone URL used by OpenStack for authentication.
• Add certificate for standalone (only) during the bootup time, i.e, during the ESC VM installation:
./bootvm.py test-vm --image <image_name> --net <network> [--cert_file CERT_FILE]
[--confd_aes_key CONFD_AES_KEY]
/home/cisco/openstack.crt
--user_pass <username>:<password> --user_confd_pass <username>:<password>
Note Currently, ESC does not support adding a certificate for HA Active/Standby during the installation as the keepalived service is not running when a certificate is added.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
15
Installing Cisco Elastic Services Controller on OpenStack
Removing a Root Certificate
• Add certificate for standalone/HA Active/Standby after booting up the ESC instance. The escadm tool has an truststore add option which has the following arguments: The --file argument refers to the CA certificate file. Using this argument you can import any file format supported by the java keytool: X.509
v1, v2, and v3 certificates, and PKCS#7. The --alias argument is unique and refers to the name this specific CA certificate is given.
1.
Copy/Transfer CA Certificate file to ESC master VM .
2.
Add certificate to ESC truststore. To do this, execute the following command: sudo escadm truststore add --alias [ca cert alias] --file [file path] or sudo escadm truststore truststore add --alias [ca cert alias] --file [file path]
3.
Verify the certificate is added.
sudo escadm truststore show or sudo escadm truststore show
Removing a Root Certificate
The escadm tool has a 'truststore delete' option which only takes --alias argument. The --alias argument refers to the name of the CA certificate to be deleted. Use this argument on the standalone/HA Active/Standby ESC
VM:
Procedure
Step 1
Step 2
On (master) ESC use escadm to delete certificate from ESC truststore.
sudo escadm truststore delete --alias [ca cert alias] or sudo escadm truststore truststore delete --alias [ca cert alias]
Verify the certificate is removed.
sudo escadm truststore show or sudo escadm truststore show
Managing Root Certificates During the Upgrade
• Image Upgrade : If you are backing up the ESC DB for upgrade, then no other action is required, the
ESC truststore will be restored once the ESC DB is restored. If you are not backing up the ESC DB for upgrade, then each CA certificate needs to be added again to the ESC truststore.
• RPM Upgrade : This upgrade method keeps the ESC truststore as is, i.e. all ca certificates in the ESC truststore should remain there after upgrade.
16
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Managing Keystore in Cisco Elastic Services Controller
Managing Keystore in Cisco Elastic Services Controller
The keystore is a storage for certificates and keys that an application uses to authenticate itself with different clients.
ESC Keystore holds only one certificate, that is used by all the applications for example, ESCManager,
VIMManager, MONA, and so on.
ESCADM provides multiple commands to manage the keystore.
When deploying ESC for the first time, a self signed certificate is created and stored in the keystore by default.
important Notes:
• In an Active/Active mode, the default certificate Common Name (CN) is db.service.consul
. If you want to set a new certificate, you must set it with the same CN, otherwise, you must add the following configuration to the heat template of all the nodes to disable the certificate validation in MONA when deploying the ESC: mona: certificate_validation: false
• All the escadm keystore commands require root privileges to be executed on the ESC.
ESCADM Keystore Commands
• escadm keystore show
The escadm keystore show command displays information about the certificate currently stored in the keystore. It shows informations like, the creation date, Alias, and certificate fingerprint.
For example:
$ sudo escadm keystore show
Keystore type: PKCS12
Keystore provider: SUN
Your keystore contains 1 entry esc, Apr 13, 2020, PrivateKeyEntry,
Certificate fingerprint (SHA1):
FF:11:66:3E:93:DD:3A:0B:9A:72:40:16:35:34:D2:22:E1:25:07:80
• escadm keystore export [--out <file path>]
The export command displays the full content of the certificate. If the out option is specified, the content is saved in a file whose path is specified in the previous option.
• escadm keystore set-- file <file path>
The set command replaces the current certificate with a new one residing inside the file whose path is specified in the option file
.
Ensure that the file is in PEM format, containing both the certificate and a private key.
The following example shows how to generate a new self-signed certificate and set it to the keystore:
1.
Use the following command to generate the certificate: openssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -keyout key.pem -out cert.pem
The previous commands gives the following two files:
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
17
Installing Cisco Elastic Services Controller on OpenStack
Using a Bootable Volume in ESC Installation key.pem
and cert.pem
2.
Use the following commands to combine the two files: cat key.pem > server.pem
cat cert.pem >> server.pem
3.
To set the new certificate to the keystore, use the following command:
$ sudo escadm keystore set --file server.pem
Service "keystore" successfully updated ESC keystore and will take effect once ESC services are restarted by running "sudo escadm restart"
Note Ensure that you delete any left over files containing private keys and certificate after the setting is done.
It is mandatory to restart your system for the change to take effect. Use the following command to restart in standalone mode or H/A mode: sudo escadm restart
If the ESC is running in the Active/Active mode, you must restart all the nodes in the same cluster.
However, if the ESC is running in an Active/Active GEO mode, you must stop the GEO service on all the clusters, to make sure a GEO switchover does not take place.
Only in GEO Active/Active mode, to stop GEO service, log in to any node in each cluster and use the following command:
$ sudo escadm geo stop --cluster
You can login to any node in the cluster where the certificate is originally set. Use the following command:
$ sudo escadm stop --cluster
$ sudo escadm start --cluster
If you are on a GEO Active/Active mode, once all the node are up and running in passing state, login again to any node to start the GEO service. Use the folowing command to start the service:
$ sudo escadm geo start --cluster
Using a Bootable Volume in ESC Installation
A volume in OpenStack is a detachable block storage device that can be attached to an ESC instance. You can store and also run ESC instances from a volume.
Note • Only one ESC instance can be launched from one volume at a time.
• ESC installation with a combination of bootable volume and high availability (Active/Standby and
Active/Active) on cinder is not supported.
18
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Using a Bootable Volume in ESC Installation
To launch an ESC instance from a bootable volume, do the following:
Procedure
Step 1
Step 2
Create a bootable volume in OpenStack based on an ESC image or from a bootable volume. The bootable volume must be at least of 30 GB disk size. For more information, see OpenStack documentation.
Deploy ESC VM using the bootvm.py
command and choose the
--boot_volume argument instead of the
--image argument, as shown below:
./bootvm.py <esc_vm_name> --boot_volume <volume_name_or_id> --net <network> --user_pass
<username>:<password>
--user_confd_pass <username>:<password> --flavor <flavor_name>
Note • Only one of these arguments,
--image or
--boot_volume must be passed to the bootvm.py
command. The installation will fail, if both or none of the arguments are used.
• When launching an ESC instance from a bootable volume, volume disk size is considered over the flavor disk size.
• If an ESC instance is deleted, the volume attached to it will not be deleted, as the volume was created out-of-band.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
19
Using a Bootable Volume in ESC Installation
Installing Cisco Elastic Services Controller on OpenStack
20
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
C H A P T E R
4
Installing High Availability Active/Standby
This chapter contains the following sections:
•
High Availability Active/Standby Overview, on page 21
•
How High Availability Works, on page 22
•
Deploying ESC High Availability Active/Standby, on page 23
•
Configuring the Northbound Interface Access, on page 26
•
•
Troubleshooting High Availability Active/Standby, on page 32
High Availability Active/Standby Overview
ESC supports High Availability (HA) in the form of Active/Standby and Active/Active models. For
Active/Standby model, two ESC instances are deployed in the network to prevent ESC failure and provide
ESC service with minimum service interruption. If the primary ESC instance fails, the standby instance automatically takes over the ESC services. ESC HA Active/Standby resolves the following single point failures:
• Network failures
• Power failures
• Dead VM instance
• Scheduled downtime
• Hardware issues
• Internal application failures
Note From ESC 5.0, the name, Active/Passive model is changed to Active/Standby model.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
21
ESC Active/Standby Architecture
ESC Active/Standby Architecture
Figure 1: Cisco Elastic Services Controller Active/Standby Architecture
Installing Cisco Elastic Services Controller on OpenStack
How High Availability Works
ESC HA Active/Standby network can be either set up as a single installation of a ESC HA Active/Standby pair or deployed as two standalone ESC nodes that are converted into HA pair after re-configuring these nodes post deployment. The HA deployment consists of two ESC instances: a primary and a standby. Under normal circumstances, the primary ESC instance provides the service. The corresponding standby instance is passive.
The standby instance is in constant communication with the primary instance and monitors the primary instances' status. If the primary ESC instance fails, the standby instance automatically takes over the ESC services to provide ESC service with minimum interruption.
The standby also has a complete copy of the database of the primary, but it does not actively manage the network until the primary instance fails. The KeepAliveD service monitors both primary and standby instances activity status. When the primary instance fails, the standby takes over automatically. The standby instance takes over primary instance to manage the services while primary instance restoration is taking place.
When the failed instance is restored, if required you can manually initiate a switch-over and resume network management via the primary instance.
Both primary and standby ESC instances are connected to the northbound orchestration system through an
IPv4 or IPv6 network. For the northbound system, a unique virtual IP address is assigned to access the current primary ESC High Availability Active/Standby instance. The deployed VNFs are connected to both ESC primary and standby instances through another IPv6 network.
ESC HA Active/Standby nodes are managed by KeepAliveD and DRBD (Replication tool to keep the ESC database synchronized) sync network services. While the KeepAliveD service monitors both primary and standby instances status, the DRBD service monitors primary instance DB and sync the changes to the standby instance DB. These two services can be co-located on same VIP network or in two separate networks. VM handshake between ESC instances occurs through the KeepAliveD over the IPv4 or IPv6 network.
22
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Deploying ESC High Availability Active/Standby
Deploying ESC High Availability Active/Standby
To deploy Cisco Elastic Services Controller (ESC) High Availability (HA) Active/Standby, ESC standalone instances can be installed on two separate nodes - Primary and Standby. For more information see,
Availability Works, on page 22
. You can connect the Primary and Standby instances to either a Cinder volume or Replication based volume (DRBD).
The following deployment mechanisms can be used to deploy ESC HA Active/Standby:
• Internal Storage—When ESC HA Active/Standby is configured with Internal storage, the Primary and the Standby instances have individual databases which are always synchronized. In this solution, ESC
HA Active/Standby is designed with database replication and DRBD is used as the tool for disk-level replication. The database in the Primary instance simultaneously propagates the data to the database in the Standby instance thus requiring no external storage. In the event of a Primary instance failing, the
Standby instance get assigned the role of the Primary instance along with its own synchronized database.
ESC HA Active/Standby is deployed using Internal storage, the ESC instances reply on the virtual IP address (that is kad_vip argument), and the interface of vrrp instance (that is kad_vif argument) to select the Primary ESC instance. To establish a reliable heartbeat network, it is recommended that the Primary and Standby ESC instances are on different physical hosts. The reliability of the physical links between the ESC instances (such as, network interface bonding) can also be taken into consideration.
• Replicate External-Storages — In this type of architecture, ESC HA Active/Standby is configured with
DRBD and both Primary and Standby instance store their data in two external storages (OpenStack
Cinder volumes). Each ESC node is attached by a Cinder volume and ESC data files are stored in the cinder volume. The data in two ESC node are synchronized through the database replication mechanism provided by DRBD.
The table lists the differences between the HA Active/Standby options :
Data sharing method
Installation Method
Internal Storage Based ESC HA
Active/Standby
Replicate External Storage Based ESC HA
Active/Standby
Data replication between HA
Active/Standby nodes
Post-installation Configuration
Bootvm Installation
Data replication between two external storages (cinder volume)
Bootvm Installation
VIM Support
Dependency
Advantages
OpenStack, VMware, KVM
VIM independent
• No dependency on specific
VIM components.
• Flexible to build of HA
Active/Standby clusters from commodity hardware, without the requirement for shared-storage.
OpenStack only
Rely on OpenStack cinder
• Use database replication mechanism for data synchronization
• Two cinder volumes are used as external storage and are attached to ESC node.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
23
Installing Cisco Elastic Services Controller on OpenStack
Deploying ESC in High Availability Active/Standby Mode on Internal Storage
Limitations The data consistency may be affected in a double fault condition
(occurs when both ESC nodes have problems).
The data consistency may be affected in a double fault condition (occurs when both
ESC nodes have problems).
Deploying ESC in High Availability Active/Standby Mode on Internal Storage
When you boot ESC instances on Primary and Standby instances, you need to specify the following bootvm.py
command arguments to deploy ESC HA Active/Standby on an internal storage:
• kad_vip
Note When ESC HA Active/Standby is deployed, the kad_vip argument allows end users to access the Primary ESC instance.
• kad_vif
• ha_node_list
These arguments enable the bootvm.py
command to automatically set up the internal storage on the OpenStack.
For more information on using the bootvm.py
command arguments, see Appendix A: Cisco Elastic Services
Controller Installer Arguments.
To deploy ESC HA Active/Standby instances, use the bootvm script on both the nodes with the following arguments:
ON HA NODE 1:
$ ./bootvm.py <ESC_HA_Node1>\
--user_pass <username>:<password>\
--user_confd_pass <username>:<password>\
--gateway_ip <default gateway IP address>\
--net <network name1>\
--ipaddr <static ip address>
--image <image_name>\
--avail_zone nova:<openstack zone>\
--ha_node_list=<ESC_HA_NODE1_IP> <ESC_HA_NODE2_IP>\
--db_volume_id <cinder volume id>\
--kad_vip <virtual IP address>\
--kad_vif <VRRP_Interface_Instance>\
--ha_mode drbd
ON HA NODE 2:
$ ./bootvm.py <ESC_HA_Node2>\
--user_pass <username>:<password>\
--user_confd_pass <username>:<password>\
--gateway_ip <default gateway IP address>\
--net <network name1>\
--ipaddr <static ip addresses>\
--image <image_name>\
--avail_zone nova:<openstack zone>\
--ha_node_list=<ESC_HA_NODE1_IP> <ESC_HA_NODE2_IP>\
--db_volume_id <cinder volume id>\
--kad_vip <virtual IP address>\
24
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Deploying ESC in High Availability Active/Standby Mode on Replicate External Storage
--kad_vif <VRRP_Interface_Instance>\
--ha_mode drbd
OR
You can also use escadm tool to re-configure ESC HA Active/Standby parameters on each of the standalone
ESC VMs. Three parameters "--ha_node_list , --kad_vip, --kad_vif" are all required to configure ESC HA
Active/Standby.
Note You should make sure that both the standalone ESC VMs health check is passed before running the below commands to perform HA Active/Standby configuration.
For example:
$ sudo bash
$ escadm ha set --ha_node_list='<ESC_HA_NODE1_IP> <ESC_HA_NODE2_IP>' --kad_vip <virtual IP address> --kad_vif <VRRP_Interface_Instance>
$ sudo escadm reload
$ sudo escadm restart
Deploying ESC in High Availability Active/Standby Mode on Replicate External Storage
Replicate external storage ESC HA Active/Standby requires two cinder volumes for database storage.
Before you begin
• Networks and IP addresses that both ESC instances will connect to
• Keepalived interface and virtual IP for HA Active/Standby switchover
Procedure
Step 1
Step 2
Step 3
Create two cinder volumes in OpenStack. The configured cinder volume size should be 3GB.
$ cinder create --display-name cindervolume_name_a[SIZE]
$ cinder create --display-name cindervolume_name_b[SIZE]
Check the status of the created cinder volume and find the uuids for deployment.
$ cinder list
Deploy ESC HA Active/Standby instances. Use the bootvm script on both the nodes with the following arguments:
ON HA NODE 1:
$ ./bootvm.py <ESC_HA_Node1>\
--user_pass <username>:<password>\
--user_confd_pass <username>:<password>\
--gateway_ip <default gateway IP address>\
--net <network name1>\
--ipaddr <static ip address>\
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
25
Installing Cisco Elastic Services Controller on OpenStack
Configuring the Northbound Interface Access
Step 4
--image <image_name>\
--avail_zone nova:<openstack zone>\
--kad_vip <virtual IP address>\
--kad_vif <VRRP_Interface_Instance>\
--ha_node_list=<ESC_HA_NODE1_IP> <ESC_HA_NODE2_IP>\
--db_volume_id <cinder volume id>\
--ha_mode drbd_on_cinder
ON HA NODE 2:
$ ./bootvm.py <ESC_HA_Node2>\
--user_pass <username>:<password>\
--user_confd_pass <username>:<password>\
--gateway_ip <default gateway IP address>\
--net <network name1>\
--ipaddr <static ip address>\
--image <image_name>\
--avail_zone nova:<openstack zone>\
--kad_vip <virtual IP address>\
--kad_vif <VRRP_Interface_Instance>\
--ha_node_list=<ESC_HA_NODE1_IP> <ESC_HA_NODE2_IP>\
--db_volume_id <cinder volume id>\
--ha_mode drbd_on_cinder
After both VMs are rebooted; the keepalived state on one of ESC VM should be one of ESC VM should be in MASTER state and the other one should be in BACKUP state. You can check ESC HA Active/Standby state by using following command: $ sudo escadm status --v .
Configuring the Northbound Interface Access
When you configure ESC HA Active/Standby, you can also specify a virtual Anycast IP address to the HA
Active/Standby pair. The northbound interface as well as the service portal uses virtual Anycast IP address to access the ESC Primary HA Active/Standby instance. When deploying ESC HA Active/Standby, use the following arguments with the ./bootvm.py
script.
• --ha_node_list
• --kad_vip
• --kad_vif
For more details on these arguments, see section Appendix A: Cisco Elastic Services Controller Installer
Arguments .
The following section explains how to configure ESC HA Active/Standby with multiple interfaces and to configure the virtual Anycast IP address.
Configuring ESC HA Active/Standby with Multiple Interfaces
You can configure ESC HA Active/Standby with DRDB synchronization and VRRP heartbeat broadcasting on a network interface for data synchronization and VNF monitoring. You can use an additional network interface to allocate Virtual IP for the northbound access. To configure the multiple interfaces on ESC HA
Active/Standby nodes, use --ha_node_list, --kad_vip, --kad_vif arguments to specify these multiple network interfaces configuration. For details on these arguments, see section Appendix A: Cisco Elastic Services
Controller Installer Arguments .
26
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Configuring the ESC HA Active/Standby Virtual IP Address
Note KeepAlived doesn’t support single IPv4 VIP address with a IPv6 VRRP instance.
Example configuration steps are shown below:
./bootvm.py <esc_ha1> \
--user_pass <username>:<password>
--user_confd_pass <username>:<password>
--image <image_id> \
--net <net-name> \
--gateway_ip <default_gateway_ip_address> \
--ipaddr <ip_address1> <ip_address2> \
--ha_node_list < IP addresses HA nodes1> < IP addresses for HA nodes2> \
--kad_vip <keepaliveD VIP of the HA nodes and the interface for keepaliveD VIP> \ (for example: --kad_vip 192.0.2.254:eth2)
--kad_vri <virtual router id of vrrp instance>
--kad_vif <virtual IP of the HA nodes or the interface of the keepalived VRRP> \ (for example: --kad_vif eth1 )
--ha_mode <HA installation mode> \
--route <routing configuration> \ (for example:192.0.2.254/24:192.168.0.1:eth1 )
--avail_zone nova:<openstack zone> \
Similarly, a three network interface can be configured for ESC HA Active/Standby nodes. An example three interfaces configuration is shown below with the following assumptions :
• Network 1 is an IPv6 network used for northbound connection. ESC VIP is allocated in this network and the Orchestrator send requests to ESC through ESC VIP.
• Network 2 is an IPv4 network used for ESC sync traffic (DRDB synchronization) and VRRP heartbeat.
This network is also used for OpenStack connection and VNF monitoring.
• Network 3 is another IPv4 network used for management. The SA, rsyslog, etc. can use this network to manage ESC.
./bootvm.py esc-ha-0 --image ESC-2_2_x_yyy --net network-v6 network --gateway_ip 192.168.0.1 --ipaddr
2001:cc0:2020::fa 192.168.0.239 192.168.5.239 --ha_node_list 192.168.0.239 192.168.0.243 --kad_vip
[2001:cc0:2020::fc/48]:eth0 --kad_vif eth1 --ha_mode drbd --route 172.16.0.0:eth1 --avail_zone nova: zone name
./bootvm.py esc-ha-1 --image ESC-2_2_x_yyy --net network-v6 network lab-net-0 --gateway_ip 192.168.0.1
--ipaddr 2001:cc0:2020::fa 192.168.0.239 192.168.5.239 --ha_node_list 192.168.0.239 192.168.0.243 --kad_vip
[2001:cc0:2020::fc/48]:eth0 --kad_vif eth1 --ha_mode drbd --route 172.16.0.0:eth1 --avail_zone nova: zone name
Configuring the ESC HA Active/Standby Virtual IP Address
In this option, the value of kad_vip argument should be a virtual IP, which allows the service portal and the northbound to access the Primary ESC and send requests to ESC HA Active/Standby service through virtual
IP (VIP).
If northbound and both ESC HA Active/Standby nodes are located in the same network, you can connect directly through the virtual IP (VIP). If northbound doesn't sit on the same network as ESC HA Active/Standby, assign a floating IP to ESC HA Active/Standby VIP using the procedure below:
1.
Create a port with the VIP address (kad_vip) in the same network as ESC's kad_vip connects.
neutron port-create network --name network_vip --fixed-ip subnet_id=network-subnet,ip_address=192.168.0.87
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
27
Installing Cisco Elastic Services Controller on OpenStack
Configuring the ESC L3 HA Active/Standby With BGP
2.
Deploy ESC HA Active/Standby . See Configuring High-Availability Active/Standby section in Installing
ESC on OpenStack.
Note Make sure the kad_vip using the same IP address as the port created above.
3.
Associate a floating IP with the port created above. The first uuid is the floating ip id and the second one is the port id.
neutron floatingip-associate <floating IP> <port ID>
Access ESC HA Active/Standby through the floating IP and it will connect to the ESC Primary node.
4.
For the portal access, make sure the keepalive network is accessible by your browser and the virtual IP is the IP address to access the portal of the Primary node.
For example, if the VIP is 192.0.2.254, access ESC HA Active/Standby portal with https://192.0.2.254:9001/.
Configuring the ESC L3 HA Active/Standby With BGP
To configure BGP for ESC HA Active/Standby, there are two options:
1.
Directly booting ESC HA Active/Standby L3 with BGP
2.
Using post configuration from existing ESC HA Active/Standby pair
To configure BGP for ESC HA Active/Standby, the following network parameters are required:
• BGP remote IP
• IP of the interface for BGP anycast routing
• BGP local AS number for routing configuration
• BGP remote AS number for routing configuration
• BGP routing configuration
• --bgp_local_ip
• --bgp_local_router_id
Note You must configure BGP router with neighbors, and restart it. Verify that the router is able to ping the AnyCast
IP.
On the BGP router, set two neighbors. The below BGP configuration is designed for Bird router. The configuration is router specific. For each types of router, the procedure is different:
The below configurations are given according to the bootvm command : protocol bgp E3 from EXABGP { neighbor 198.18.42.222 as 65012;
}
28
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Configuring the ESC L3 HA Active/Standby With BGP protocol bgp E4 from EXABGP { neighbor 198.18.61.222 as 65011;
}
Booting an ESC VM with BGP options
####################################################################
# ESC on bgp-001.novalocal is in PRIMARY state.
####################################################################
[admin@bgp-001 ~]$ health.sh
============== ESC HA (PRIMARY) with DRBD ================= vimmanager (pgid 4007) is running monitor (pgid 4135) is running mona (pgid 4167) is running drbd (pgid 0) is primary snmp (pgid 5375) is running etsi is disabled at startup pgsql (pgid 4586) is running keepalived (pgid 3068) is running portal (pgid 5315) is running confd (pgid 4417) is running filesystem (pgid 0) is running escmanager (pgid 4615) is running
=======================================
ESC HEALTH PASSED
[admin@bgp-001 ~]$
####################################################################
# ESC on bgp-002.novalocal is in BACKUP state.
####################################################################
[admin@bgp-002 ~]$ health.sh
============== ESC HA (BACKUP) with DRBD ================= vimmanager is stopped monitor is stopped mona is stopped drbd (pgid 0) is backup snmp is stopped etsi is disabled at startup pgsql is stopped keepalived (pgid 3069) is running portal is stopped confd is stopped filesystem is stopped escmanager is stopped
=======================================
ESC HEALTH PASSED
[admin@bgp-002 ~]$
Use below values for BGP post configuration:
./bootvm.sh <NETWORK_VM_name> \
--image <ESC_image> \
--ipaddr <static_IP_address1> <IP_address2> <IP_address_3>\
--gateway_ip <gateway IP address of NETWORK> \
--net <net_id1> <net_id2> <net_id3> \
--network_params_file <network_params_file> \
--host_mapping_file <host_mapping_file> \
--avail_zone <openStack_zone> \
--bgp_remote_ip <BGP_remote_IP_address> \
--bgp_local_as <BGP_local_AS_#> \
--bgp_remote_as <BGP_remote_AS_#>\
--bgp_local_router_id <local_BGP_reouter_id> \
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
29
Installing Cisco Elastic Services Controller on OpenStack
Configuring the ESC L3 HA Active/Standby With BGP
--bgp_anycast_ip <BGP_anycast_IP> \
--bgp_md5 <BGP_MD5>
Where,
--ip_addr: ----> the local IP address of the ESC VM
--net: ----> the network id(s) in OpenStack that ESC will connect to.
--bgp_anycast_ip: ----> the IP address that NCS will communicate with
--bgp_remote_ip: ----> this IP address of the external router that ESC will peer with
--bgp_local_as: ----> local AS for the ESC "router"
--bgp_remote_as: ----> AS number for the external router ESC will peer with
--bgp_local_router_id: ----> id for the esc "router"
--bgp_md5: ----> optional - md5 to be used to pair with external router
Configuring BGP HA Active/Standby Post Configuration
1.
For each HA Active/Standby instance, create the network interface file:
# cat /etc/sysconfig/network-scripts/ifcfg-lo:2
IPV6INIT='no'
IPADDR='10.0.124.124'
BROADCAST='10.0.124.255'
<------------------ bgp anycast IP
NETWORK='10.0.124.0'
NETMASK='255.255.255.0'
DEVICE='lo:2'
ONBOOT='yes'
NAME='loopback'
2.
For each HA Active/Standby instance:
Bring lo:2 up
# ifup lo:2
To configure BGP for ESC HA Active/Standby, use the escadm tool in ESC Virtual Machine, as shown below:
$ sudo bash
# escadm bgp set --local_ip LOCAL_IP --anycast_ip ANYCAST_IP --remote_ip REMOTE_IP --local_as
LOCAL_AS --remote_as REMOTE_AS
--local_router_id LOCAL_ROUTER_ID
# escadm reload
# reboot
Example:
[root@bgp-001 admin]# escadm bgp set --local_ip 198.18.42.124 --anycast_ip 10.0.124.124
--remote_ip 192.168.0.2 --local_as 65124 --remote_as 65000 --local_router_id 198.18.42.124
[root@bgp-002 admin]# escadm bgp set --local_ip 198.18.42.125 --anycast_ip 10.0.124.124
--remote_ip 192.168.0.2 --local_as 65114 --remote_as 65000 --local_router_id 198.18.42.125
Configuring a BGP Router
To configure a BGP router, log in to the BGP router to configure BGP Anycast routing. The required parameters are:
<Router_AS_#> same as --bgp_remote_as above
<Esc_ip_address> must be the ESC VM's IP address configured for BGP advertisement.
<ESC_AS_#> same as --bgp_local_as shown above configure router bgp <Router_AS_#> neighbor <ESC_IP_address>
30
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Important Notes remote-as <ESC_AS_#> address-family ipv6 unicast route-policy anycast-in in route-policy anycast-out out route-policy anycast-in pass end-policy route-policy anycast-out drop end-policy commit
Important Notes
• ESC HA Active/Standby
• An HA Active/Standby failover takes about 2 to 5 minutes. The ESC service will not be available during the switchover time.
• When the switchover is triggered during transactions, all incomplete transactions will be dropped.
The requests should be re-sent by northbound if it does not receive any response from ESC.
• External Storage
• If the Primary ESC instance is suspended by OpenStack command, the switch over will be triggered but the cinder volume won't be attached to the new Primary ESC instance. This is not a valid use case for ESC HA Active/Standby.
• Internal Storage
• Two ESC instances have to be deployed to establish the HA Active/Standby solution. The ESC HA
Active/Standby will start to work when both ESC instances are successfully deployed and are able to connect to each other. If you just deploy one ESC instance with HA Active/Standby parameters, the ESC instance keeps Switching-to-Master state and will not be able to provide any service until it reaches its peer.
• Split-brain scenario can still happen in this ESC HA Active/Standby solution, although the chance is very low.
• ETSI-specific Notes
ESC supports ETSI MANO northbound API defined by the European Telecommunications Standards
Institute (ETSI) for NFV Management and Orchestration. The ETSI MANO API is another programmatic interface based on the REST architecture. For more information, see ETSI MANO Compliant Lifecycle
Operations in the Cisco Elastic Services Controller User Guide . Consider the following notes while enabling ETSI service on ESC which is in HA Active/Standby mode:
• The server.address value in the etsi-vnfm.properties
file must be set to a Virtual IP (VIP) address.
This IP address can be used to communicate back to the ETSI services using API callbacks. If the virtual IP address is not specified, the ETSI service startup may fail.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
31
Installing Cisco Elastic Services Controller on OpenStack
Troubleshooting High Availability Active/Standby
• The ETSI VNFM service and the escadm script generate and maintain the security.user.name and security.user.password property values. You should not change it manually.The
security.user.password is encoded.
Troubleshooting High Availability Active/Standby
• Check for network failures. If a network problem occurs, you must check the following details:
• The IP address assigned is correct, and is based on the OpenStack configuration.
• The gateway for each network interface must be pingable.
• Check the logs for troubleshooting:
• The ESC Admin logs at /var/log/esc/escadm.log
• The ESC manager log at /var/log/esc/escmanager.log
• The AA elector log at /var/log/esc/elector-{pid}.log
• Check for DRBD (Replication based ESC HA Active/Standby) for Internal Storage solution:
• Check the DRBD configuration file at
/etc/drbd.d/esc.res
• Access the DRBD log
•
/var/log/messages|grep drbd
• To collect log files through CLI, use the following command on all ESC nodes: sudo escadm log collect
32
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
C H A P T E R
5
Cisco Elastic Services Controller Active/Active
High Availability Overview
This chapter contains the following sections:
•
Cisco Elastic Services Controller Active/Active HA Overview, on page 33
•
ESC Active/Active Architecture, on page 34
Cisco Elastic Services Controller Active/Active HA Overview
ESC supports High Availability (HA) in the form of an Active/Active model. ESC Active/Active HA has three VMs as a cluster in one datacenter.. There are two datacenters. Between the two datacenters, one datacenter acts as active, another acts as standby. ESC Active/Active HA uses Openstack heat template to deploy the three VM cluster in a datacenter.
In a datacenter, ESC service runs on each VM; however, there is only one ESC on a datacenter that runs as cluster leader. DB service runs only on the leader. The ESC services on the other two ESC VMs run as cluster follower. The DB service is only active on ESC services leader VM.
DRBD synchronises the data among ESC VMs. The ESC service on the three ESC VMs connect to the active
DB service. When a leader switchover happens, all the ESC service connects to the newly active DB service.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
33
ESC Active/Active Architecture
ESC Active/Active Architecture
Figure 2: Cisco Elastic Services Controller Active/Active Architecture
Installing Cisco Elastic Services Controller on OpenStack
34
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
C H A P T E R
6
Installing Active/Active High Availability Cluster
This chapter contains the following sections:
•
Installing Active/Active High Availability Cluster , on page 35
•
Validating Active/Active High Availability Cluster Post Installation, on page 37
•
Adding Default VIM Connector to the Active/Active High Available Cluster, on page 38
•
Adding BGP in an Active/Active Cluster, on page 38
Installing Active/Active High Availability Cluster
To configure the Active/Active HA cluster, you can run the following commands on OpenStack, where openrc my-server is openrc for the OpenStack, and test is the stack name.
source ~/elastic-services-controller/esc-bootvm-scripts/openrc my-server-42 openstack stack create test -t aa.yaml
To check the status of the stack, use the following commands:
• openstack stack list
• openstack stack show test
• openstack stack event list test
Once the stack is in
CREATE_COMPLETE status, you can ssh into the VMs. The openstack stack show test command lists the IP addresses for the 3 VMs, you can use them to access to the VMs.
Note The Active/Active HA Cluster is deployed on OpenStack only.
Setting up the User Configuration
You can configure some of the parameters based on your settings, such as network, subnet, using either static
IP or DHCP to allocate IP address, flavor, image, password, etc. The configurable parameters are available in aa-param.yaml
, Openstack heat environment file, when the ESC cluster is instantiated via heat template
( aa.yaml
).
To instantiate ESC cluster, use the following example:
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
35
Installing Cisco Elastic Services Controller on OpenStack
Setting up the User Configuration openstack stack create name -t aa.yaml -e aa-params.yaml
The following example shows how to use a static IP address to configure the port while using environment template: sample@my-server-39:~/aa4.5/apr15$ more aa-params.yaml
parameters: network_1_name: esc-net subnet_name: esc-subnet esc_1_ip: 172.23.0.228
esc_2_ip: 172.23.0.229
esc_3_ip: 172.23.0.230
Following are the user configurable parameters available in aa.yaml : parameters: network_1_name: type: string description: Name of the image default: esc-net subnet_name: type: string description: subnet name esc_1_ip: type: string description: static IP address of esc-1 VM.
esc_2_ip: type: string description: static IP address of esc-2 VM.
esc_3_ip: type: string description: static IP address of esc-3 VM.
resources: esc_1_port: type: OS::Neutron::Port properties: network_id: { get_param: network_1_name } fixed_ips: [ { "subnet": { get_param: subnet_name}, "ip_address": { get_param: esc_1_ip
} } ] esc_2_port: type: OS::Neutron::Port properties: network_id: { get_param: network_1_name } fixed_ips: [ { "subnet": { get_param: subnet_name}, "ip_address": { get_param: esc_2_ip
} } ] esc_3_port: type: OS::Neutron::Port properties: network_id: { get_param: network_1_name } fixed_ips: [ { "subnet": { get_param: subnet_name}, "ip_address": { get_param: esc_3_ip
} } ]
...omitting...
The following example shows how to use configurable image, flavor, and vm name prefix from environment template: sample@my-server-39:~/aa4.5/apr15$ more aa-params.yaml
parameters:
36
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Validating Active/Active High Availability Cluster Post Installation nameprefix: abc image_name: ESC-5_0_DEV_4 flavor_name: m1.large
sample@my-server-39:~/aa4.5/apr15$
The following example shows how to use configurable image, flavor, and vmnameprefix from heat template: parameters: nameprefix: type: string description: Name prefix of vm default: helen image_name: type: string description: Name of the image default: ESC-5_0_DEV_4 flavor_name: type: string description: Name of the image default: m1.large
esc-1: type: OS::Nova::Server properties: name: str_replace: template: $nameprefix-esc-1 params:
$nameprefix : { get_param: nameprefix } image: { get_param: image_name } flavor: { get_param: flavor_name }
... omitting...
Validating Active/Active High Availability Cluster Post
Installation
To verify all the ESC nodes use the following commands. Here, all ESC nodes implies each VMs.
sample@my-server-39:~$ openstack --insecure server list | grep abc
| 5ea6fc79-2b2a-4064-9c6a-a83d6b06c225 | abc-test-esc-3
| ACTIVE | esc-net=172.23.7.203
| ESC-5_0_DEV_13 | m1.large
| 10e165d9-5015-4b64-88fe-19e874e6e7c1 | abc-test-esc-1
| ACTIVE | esc-net=172.23.7.205
| ESC-5_0_DEV_13 | m1.large
| 35f6bad1-865f-4155-8411-d37e2616e079 | abc-test-esc-2
| ACTIVE | esc-net=172.23.7.204
| ESC-5_0_DEV_13 | m1.large
|
|
|
To find out the leader node, ssh into one of the nodes/VMs and run the following:
[admin@sample-test-esc-1 ~]$ sudo escadm elector dump
{
"[email protected]:42143": {
"state": "FOLLOWER",
"location": "[email protected]:42143",
"service": "esc_service"
},
"[email protected]:50474": {
"state": "FOLLOWER",
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
37
Installing Cisco Elastic Services Controller on OpenStack
Adding Default VIM Connector to the Active/Active High Available Cluster
}
"location": "[email protected]:50474",
"service": "esc_service"
},
"[email protected]:59514": {
"state": "LEADER",
"location": "[email protected]:59514",
"service": "esc_service"
}
Adding Default VIM Connector to the Active/Active High
Available Cluster
You can add a default vim connector to the 3 ESC VM cluster in the following two different ways:
1.
Once the 3 ESC VM cluster boots up, use netconf interface to add a default vim connector by using the following command. The vim.xml
is the default vim connector deployment file.
[admin@name-esc-1 ~]$ esc_nc_cli --host db.service.consul --user admin --password
<admin_password> edit-config vim.xml
2.
To configure a default vim connector, you must add the default vim connector configuration inside the heat template day0 file. Add the following block in the write_files section under the cloud-config in aa-day0.yaml file. Once the 3 ESC VM cluster boots up, it creates a default vim connector by its own.
The following example shows how to configure default vim connector in heat template day0 file:
- path: /opt/cisco/esc/esc-config/esc_params.conf
content: | openstack.os_auth_url=http://10.85.103.38:35357/v3 openstack.os_project_name=admin openstack.os_tenant_name=admin openstack.os_user_domain_name=default openstack.os_project_domain_name=default openstack.os_identity_api_version=3 openstack.os_image_api_version=2 openstack.os_username=admin openstack.os_password=password1
Adding BGP in an Active/Active Cluster
To initiate the BGP process, add the anycast IP to the lo device. You can configure it in sys-cfg.yaml
.
For example:
#cloud-config write_files:
- path: /etc/cloud/cloud.cfg.d/sys-cfg.yaml
content: | network: version: 1 config:
- type: physical name: lo subnets:
- type: static
38
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Adding BGP in an Active/Active Cluster address: 172.23.188.188/23
You must specify the advertise IP for consul. In esc-config.yaml
, add the following: consul: advertise_addr: 172.23.1.149
Following is the example for adding the BGP section: bgp: depend_on: elector:leader anycast_ip: 172.23.188.188/23 local_as: '65001' local_ip: 192.168.1.11
local_router_id: 192.168.1.11
remote_as: '65000' remote_ip: 192.168.1.12
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
39
Adding BGP in an Active/Active Cluster
Installing Cisco Elastic Services Controller on OpenStack
40
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
C H A P T E R
7
Managing Cluster in ESC Active/Active High
Availability
This chapter contains the following section:
•
Managing Cluster in ESC Active/Active High Availability, on page 41
Managing Cluster in ESC Active/Active High Availability
To do cluster management in ESC Active/Active HA, you can call some escadm commands on any ESC node and execute it on all nodes in Active/Active cluster.
The following are the supported commands for cluster level calling:
• escadm start
• escadm stop
• escadm geo start/stop
• escadm vim show
Add the option
--cluster to execute any of the previous commands to be executed on all nodes in
Active/Active local cluster:
For example: sudo escadm geo start --v –cluster
The output of the command and exit code of each node are displayed with the IP address of that node to distinguish the execution output, and the output on the local node.
Example for escadm geo start --cluster
:
[root@name-geo-2-1 admin]# escadm geo start --cluster
192.168.1.13 # remote host exit status : 0
Starting geo service: [OK]
192.168.1.12 # remote host exit status : 0
Starting geo service: [OK]
Starting geo service: [OK] # output of the local node
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
41
Managing Cluster in ESC Active/Active High Availability
Installing Cisco Elastic Services Controller on OpenStack
42
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
C H A P T E R
8
Configuring GEO in Active/Active High
Availability
•
Configuring GEO in Active/Active High Availability, on page 43
•
Verifying GEO Services, on page 45
•
Active/Active GEO HA Failure Injection Limitations, on page 46
Configuring GEO in Active/Active High Availability
ESC Active/Active HA has three VMs as a cluster in one datacenter. The second datacenter consists of
GEO-HA.
Following are the 6 pre-defined roles in GEO:
1.
init
: initial role of geo service
2.
pre_primary
3.
primary
4.
pre_secondary
5.
secondary
6.
unknown
: used when consul is not reachable
GEO can change one role to another. Transitions are defined in esc-config.yaml
. Each transition is divided into the following 3 parts:
• from
: current role
• goto
: destination role
• condition
: when GEO changes the role
Transition Conditions
When A/A HA Geo comes up, the primary datacenter has to go through init
, pre_primary
, and primary states; meanwhile, the secondary datacenter has to go through init, pre-secondary, and secondary states. When
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
43
Installing Cisco Elastic Services Controller on OpenStack
Transition Conditions all ESC VMs' health check pass on both primary and secondary datacenters, the ESC A/A HA GEO is up and running. It is ready for use.
Condition Functions
The following are all supported condition functions:
1.
return
: do nothing but return the argument
2.
and
: return true if all arguments are true
3.
or
: return true if any of the argument is true
4.
len
: return the length of the argument
5.
equals
: return true if all arguments are equal
6.
true
: return true if args can be tested for truth value in python
7.
false
: opposite to 'true'
The following are samples for GEO configurations on primary datacenter: on_init: consul start on_primary: start on_secondary: stop on_stop: consul stop startup: manual transitions:
- condition: return: and:
- equals:
- len: service1
- 3
- equals:
- len: service2
- 3 rise: 3 service1: dc: dc1 name: consul_agent passing: true type: service service2: dc: dc2 name: geo passing: true type: service from: init goto: primary
- condition: fall: 2 return: equals:
- len: service
- 3 service: dc: dc1 name: consul_agent from: primary goto: secondary
The following are samples for GEO configurations on secondary datacenter:
44
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Verifying GEO Services on_init: consul start on_primary: start on_secondary: stop on_stop: consul stop startup: manual transitions:
- condition: return: and:
- equals:
- len: service1
- 3
- equals:
- len: service2
- 3 rise: 3 service1: dc: dc1 name: consul_agent passing: true type: service service2: dc: dc2 name: geo passing: true type: service from: init goto: secondary
- condition: fall: 2 return: equals:
- len: service
- 3 service: dc: dc1 name: consul_agent from: secondary goto: primary
Verifying GEO Services
To start Active/Active GEO-HA, run the following command: escadm geo start
To verify the GEO status, use the following command:
[root@test-geo3-ha-1 esc-scripts]# escadm geo status geo (pgid 3745) is primary
The verify the GEO services in current datacenter, use the following command:
[root@test-geo3-ha-1 esc-scripts]# escadm geo dump
{
"[email protected]:44793": {
"role": "primary",
"location": "[email protected]:44793",
"service": "geo"
},
"[email protected]:52459": {
"role": "primary",
"location": "[email protected]:52459",
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
45
Installing Cisco Elastic Services Controller on OpenStack
Active/Active GEO HA Failure Injection Limitations
"service": "geo"
},
"[email protected]:38841": {
"role": "primary",
"location": "[email protected]:38841",
"service": "geo"
}
}
To verify all the GEO services in the datacenters, use the following command:
[root@test-geo4-ha-1 admin]# escadm geo dump --all
{
"[email protected]:36760": {
"role": "primary",
"location": "[email protected]:36760",
"service": "geo"
},
"[email protected]:42362": {
"role": "secondary",
"location": "[email protected]:42362",
"service": "geo"
},
"[email protected]:51936": {
"role": "primary",
"location": "[email protected]:51936",
"service": "geo"
},
"[email protected]:37604": {
"role": "secondary",
"location": "[email protected]:37604",
"service": "geo"
},
"[email protected]:44450": {
"role": "primary",
"location": "[email protected]:44450",
"service": "geo"
},
"[email protected]:34875": {
"role": "secondary",
"location": "[email protected]:34875",
"service": "geo"
}
}
Active/Active GEO HA Failure Injection Limitations
The ESC Active/Active GEO HA is enhanced to support a one-way GEO HA failover with a maintenance window to move it back to healthy state.
If a GEO failover happens and the ESC VMs move to the unhealthy state, use the following steps to bring
ESC A/A GEO HA back to the healthy state through manual intervention:
Procedure
Step 1 Resolve any issues in Datacenter1 (DC1), that may have caused the failures and enabled the GEO switch to
Datacenter2 (DC2).
46
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Active/Active GEO HA Failure Injection Limitations
Step 2
Step 3
Step 4
Step 5
Ensure that the consul is running in at least 2 nodes in DC1 and DC2.
Run the sudo escadm geo replicate –all command on DC2 when DC1 has at least two nodes with consul in the running state.
Run the sudo escadm stop command on all the 6 ESC VMs.
Run the sudo escadm geo restart command on all the 6 ESC VMs.
What to do next
Note ESC does not support any operations on ESC VMs in DC1 after the GEO HA fails over to DC2.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
47
Active/Active GEO HA Failure Injection Limitations
Installing Cisco Elastic Services Controller on OpenStack
48
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
C H A P T E R
9
DRBD Encryption for ESC Active/Standby and
Active/Active HA Data Replication
This chapter contains the following sections:
•
DRBD Encryption for ESC HA Data Replication , on page 49
•
ESC HA with DRBD Encryption, on page 49
DRBD Encryption for ESC HA Data Replication
ESC uses DRBD for data replication across different nodes in an HA cluster environment. DRBD layers logical block device over existing local block devices on cluster nodes.
The written data to the primary node is transferred to the lower-level block device and simultaneously propagated to the secondary node(s). Currently ESC mounts DRBD device directly on
/opt/cisco/esc/esc_database
.
For Example:
# df
Filesystem devtmpfs tmpfs tmpfs
…
1K-blocks
2961760
2972164
2972164
Used Available Use% Mounted on
0
4
8748
2961760
2972160
2963416
0% /dev
1% /dev/shm
1% /run tmpfs 594436 0 594436 0% /run/user/1004
/dev/mapper/esc_crypt 3028620 57212 2797848 3% /opt/cisco/esc/esc_database
Block device encryption encrypts or decrypts the data transparently as it is written/read from block devices, the underlying block device sees only encrypted data.
Security is enhanced with the dm-crypt/LUKS layer to encrypt the data in DRBD partition, between filesystem and DRBD device. LUKS (Linux Unified Key Setup) is a specification for block device encryption.
ESC HA with DRBD Encryption
The following bootvm commands boot ESC HA with DRBD encrypted:
Select DRBD LUKS encryption through bootvm.py. There are 4 variations that result in the equivalent ESC day-0 user-data/esc-config.yaml when passed to the ESC VM instance.
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
49
Installing Cisco Elastic Services Controller on OpenStack
ESC HA with DRBD Encryption bootvm.py --fs_encryption_type luks --fs_luks_key_prompt bootvm.py --fs_encryption_type luks --fs_luks_key 'LuksKeyValue'
=> injects the luks key into default file location /opt/cisco/esc/esc-config/luks_key bootvm.py --fs_encryption_type luks --file root:0400:/opt/cisco/esc/esc-config/luks_key:path-to-local-luks-key-file
=> injects a local file containing the luks key
The following command shows an advanced usage to manage the luks key file at a different path on the ESC
VM filesystem: bootvm.py --fs_encryption_type luks --fs_luks_key_file path-on-esc-vm-luks-key-file
--fs_luks_key_prompt bootvm.py --fs_encryption_type luks --fs_luks_key_file path-on-esc-vm-luks-key-file
--fs_luks_key 'LuksKeyValue'
=> injects the luks key into a different file location bootvm.py --fs_encryption_type luks --fs_luks_key_file path-on-esc-vm-luks-key-file --file root:0400:path-on-esc-vm-luks-key-file:path-to-local-luks-key-file
=> injects the luks key as read from a local file into a different file location
Use the following commands if you are installing ESC with a custom user-data, such as ESC Active/Active deployment with heat templates:
The luks key is specified as a day-0 file and an attribute under esc-config.yaml / filesystem.
Encode the luks key as base64: base64 <<<'LuksKeyValue'
THVrc0tleVZhbHVlCg==
Then, insert the previous luks key into the user-data / cloud-config file: write_files:
- path: /opt/cisco/esc/esc-config/luks_key owner: root:root permissions: '400' encoding: b64 content: THVrc0tleVZhbHVlCg==
- path: /opt/cisco/esc/esc-config/esc-config.yaml
owner: root:esc-user permissions: '0640' content: | resources: filesystem: depend_on: drbd:master encryption_type: luks luks_key_file: /opt/cisco/esc/esc-config/luks_key
50
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
C H A P T E R
10
Upgrading ESC Active/Active High Availability
This chapter contains the following sections:
•
Upgrading ESC Active/Active High Availability, on page 51
Upgrading ESC Active/Active High Availability
Cisco Elastic Service Controller Active/Active HA supports local Active/Active to Active/Active simple upgrade.
Local Active/Active to Active/Active Simple Upgrade
Procedure
Step 1
Step 2
Step 3
Step 4
Back up the database. For more information, see the section,
Backing up the Database, on page 51
.
Remove old VMs. For more information, see the section,
Removing the Old VMs, on page 52 Removing
Old VMs .
Install new ESC Active/Active VMs. For more information, see the section,
Installing a New ESC Active/Active
Restore the ESC database. For more information, see the section,
Restoring the ESC Database, on page 53 .
Backing up the Database
Before upgrade, take the back up of database using the following procedure.
Procedure
Step 1
Step 2
Put the ESC leader VM in the maintenance mode, by running the following command: escadm op_mode set --mode=maintenance
Wait until all the ESC VMs stop processing transactions. To verify, run the following command: escadm ip_trans
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
51
Installing Cisco Elastic Services Controller on OpenStack
Removing the Old VMs
Step 3
Step 4
Create a backup of database on the ESC leader by running the following command: escadm backup --file dbback.tar, scp <dbback.tar>
Collect logs from all the ESC VMs by running the following command: escadm log collect scp
Removing the Old VMs
Procedure
Step 1
Step 2
Shutdown all the ESC follower VMs and the ESC leader VM running the following command: nova stop
Remove old ESC Active/Active VMs from the OpenStack by running the following command: openstack stack delete {stack name}
Installing a New ESC Active/Active VM
After backing up the database and shutting down of the old ESC Active/Active VMs, a new/upgraded (based on new ESC package) Active/Active ESC VM must be installed.
Procedure
Step 1
Step 2
Step 3
Step 4
For OpenStack, register a new image by running the following command: glance image-create
Install the new ESC Active/Active VMs by running the following command: openstack stack create {stack name} --template {location of the template file}
Check all the ESC VMs health, and stop the escadm service in the follower VMs by running the following command: sudo escadm stop for all followers VMs
Once the escadm service is stopped in all the follower VMs, stop the escadm service in the leader VM by running the following command: sudo escadm stop
52
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Installing Cisco Elastic Services Controller on OpenStack
Restoring the ESC Database
Restoring the ESC Database
Restore the ESC database on the new ESC instance, using the following procedure:
Procedure
Step 1
Step 2
Step 3
Step 4
Step 5
Copy the backup file to the new leader by running the folowing command: scp
Restore the database on the ESC leader by running the following command: sudo escadm restore --file <dbback.tar>
After restoring, the restore process starts the escadm service in the leader VM. However, the escadm service remains to be stopped in all the follower VMs.
Verify that the ESC leader VM is running all the services without any interruption.
Put the ESC leader VM into the operation mode by running the following command: sudo escadm op_mode set --mode=operation
Start the ESC services on the follower VMs by running the following command: sudo escadm start
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
53
Restoring the ESC Database
Installing Cisco Elastic Services Controller on OpenStack
54
Cisco Elastic Services Controller 5.2 Install and Upgrade Guide
Tải xuống
Đã cập nhật liên kết công khai
Liên kết công khai đến cuộc trò chuyện của bạn đã được cập nhật. You may view and manage your chat links after registration here.
Liên kết công khai tài liệu
Quảng cáo