ASA Cluster
ASA Cluster
Clustering lets you group multiple ASAs together as a single logical device. A cluster provides all the
convenience of a single device (management, integration into a network) while achieving the increased
throughput and redundancy of multiple devices.
Note
Some features are not supported when using clustering. See Unsupported Features with Clustering, on
page 28.
• About ASA Clustering, page 1
• Licensing for ASA Clustering, page 36
• Prerequisites for ASA Clustering, page 36
• Guidelines for ASA Clustering, page 37
• Defaults for ASA Clustering, page 41
• Configure ASA Clustering, page 41
• Manage ASA Cluster Members, page 65
• Monitoring the ASA Cluster, page 69
• Examples for ASA Clustering, page 75
• History for ASA Clustering, page 90
About ASA Clustering
This section describes the clustering architecture and how it works.
How the ASA Cluster Fits into Your Network
The cluster consists of multiple ASAs acting as a single unit. To act as a cluster, the ASAs need the following
infrastructure:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
1
ASA Cluster
Performance Scaling Factor
• Isolated, high-speed backplane network for intra-cluster communication, known as the cluster control
link.
• Management access to each ASA for configuration and monitoring.
When you place the cluster in your network, the upstream and downstream routers need to be able to
load-balance the data coming to and from the cluster using one of the following methods:
• Spanned EtherChannel (Recommended)—Interfaces on multiple members of the cluster are grouped
into a single EtherChannel; the EtherChannel performs load balancing between units.
• Policy-Based Routing (Routed firewall mode only)—The upstream and downstream routers perform
load balancing between units using route maps and ACLs.
• Equal-Cost Multi-Path Routing (Routed firewall mode only)—The upstream and downstream routers
perform load balancing between units using equal cost static or dynamic routes.
Related Topics
Licensing for ASA Clustering, on page 36
Cluster Control Link, on page 5
ASA Cluster Management, on page 11
Spanned EtherChannel (Recommended), on page 4
Policy-Based Routing (Routed Firewall Mode Only), on page 16
Equal-Cost Multi-Path Routing (Routed Firewall Mode Only), on page 17
Performance Scaling Factor
When you combine multiple units into a cluster, you can expect a performance of approximately:
• 70% of the combined throughput
• 60% of maximum connections
• 50% of connections per second
For example, for throughput, the ASA 5585-X with SSP-40 can handle approximately 10 Gbps of real world
firewall traffic when running alone. For a cluster of 8 units, the maximum combined throughput will be
approximately 70% of 80 Gbps (8 units x 10 Gbps): 56 Gbps.
Cluster Members
Cluster members work together to accomplish the sharing of the security policy and traffic flows. This section
describes the nature of each member role.
Bootstrap Configuration
On each device, you configure a minimal bootstrap configuration including the cluster name, cluster control
link interface, and other cluster settings. The first unit on which you enable clustering typically becomes the
primary unit. When you enable clustering on subsequent units, they join the cluster as secondaries.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
2
ASA Cluster
Cluster Members
Primary and Secondary Unit Roles
One member of the cluster is the primary unit. The primary unit is determined by the priority setting in the
bootstrap configuration; the priority is set between 1 and 100, where 1 is the highest priority. All other members
are secondary units. Typically, when you first create a cluster, the first unit you add becomes the primary unit
simply because it is the only unit in the cluster so far.
You must perform all configuration (aside from the bootstrap configuration) on the primary unit only; the
configuration is then replicated to the secondary units. In the case of physical assets, such as interfaces, the
configuration of the primary unit is mirrored on all secondary units. For example, if you configure
GigabitEthernet 0/1 as the inside interface and GigabitEthernet 0/0 as the outside interface, then these interfaces
are also used on the secondary units as inside and outside interfaces.
Some features do not scale in a cluster, and the primary unit handles all traffic for those features.
Related Topics
Centralized Features for Clustering, on page 28
Primary Unit Election
Members of the cluster communicate over the cluster control link to elect a primary unit as follows:
1 When you enable clustering for a unit (or when it first starts up with clustering already enabled), it broadcasts
an election request every 3 seconds.
2 Any other units with a higher priority respond to the election request; the priority is set between 1 and
100, where 1 is the highest priority.
3 If after 45 seconds, a unit does not receive a response from another unit with a higher priority, then it
becomes primary.
Note
If multiple units tie for the highest priority, the cluster unit name and then the serial number is used to
determine the primary.
4 If a unit later joins the cluster with a higher priority, it does not automatically become the primary unit;
the existing primary unit always remains as the primary unless it stops responding, at which point a new
primary unit is elected.
Note
You can manually force a unit to become the primary. For centralized features, if you force a primary unit
change, then all connections are dropped, and you have to re-establish the connections on the new primary
unit.
Related Topics
Centralized Features for Clustering, on page 28
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
3
ASA Cluster
Cluster Interfaces
Cluster Interfaces
You can configure data interfaces as either Spanned EtherChannels or as Individual interfaces. All data
interfaces in the cluster must be one type only.
Spanned EtherChannel (Recommended)
You can group one or more interfaces per chassis into an EtherChannel that spans all chassis in the cluster.
The EtherChannel aggregates the traffic across all the available active interfaces in the channel. A Spanned
EtherChannel can be configured in both routed and transparent firewall modes. In routed mode, the
EtherChannel is configured as a routed interface with a single IP address. In transparent mode, the IP address
is assigned to the BVI, not to the bridge group member interface. The EtherChannel inherently provides load
balancing as part of basic operation.
Individual Interfaces (Routed Firewall Mode Only)
Individual interfaces are normal routed interfaces, each with their own Local IP address. Because interface
configuration must be configured only on the primary unit, the interface configuration lets you set a pool of
IP addresses to be used for a given interface on the cluster members, including one for the primary. The Main
cluster IP address is a fixed address for the cluster that always belongs to the current primary unit. The Main
cluster IP address is a secondary IP address for the primary unit; the Local IP address is always the primary
address for routing. The Main cluster IP address provides consistent management access to an address; when
a primary unit changes, the Main cluster IP address moves to the new primary unit, so management of the
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
4
ASA Cluster
Cluster Control Link
cluster continues seamlessly. Load balancing, however, must be configured separately on the upstream switch
in this case.
Note
We recommend Spanned EtherChannels instead of Individual interfaces because Individual interfaces
rely on routing protocols to load-balance traffic, and routing protocols often have slow convergence during
a link failure.
Related Topics
Load Balancing Methods, on page 12
Cluster Control Link
Each unit must dedicate at least one hardware interface as the cluster control link.
Cluster Control Link Traffic Overview
Cluster control link traffic includes both control and data traffic.
Control traffic includes:
• Primary election.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
5
ASA Cluster
Cluster Control Link
• Configuration replication.
• Health monitoring.
Data traffic includes:
• State replication.
• Connection ownership queries and data packet forwarding.
Related Topics
Cluster Members, on page 2
Configuration Replication, on page 10
Unit Health Monitoring, on page 8
Data Path Connection State Replication, on page 10
Rebalancing New TCP Connections Across the Cluster, on page 27
Cluster Control Link Interfaces and Network
You can use any data interface(s) for the cluster control link, with the following exceptions:
• You cannot use a VLAN subinterface as the cluster control link.
• You cannot use a Management x/x interface as the cluster control link, either alone or as an EtherChannel.
• For the ASA 5585-X with an ASA IPS module, you cannot use the module interfaces for the cluster
control link; you can, however, use interfaces on the ASA 5585-X Network Module.
You can use an EtherChannel or redundant interface.
For the ASA 5585-X with SSP-10 and SSP-20, which include two Ten Gigabit Ethernet interfaces, we
recommend using one interface for the cluster control link, and the other for data (you can use subinterfaces
for data). Although this setup does not accommodate redundancy for the cluster control link, it does satisfy
the need to size the cluster control link to match the size of the data interfaces.
Each cluster control link has an IP address on the same subnet. This subnet should be isolated from all other
traffic, and should include only the ASA cluster control link interfaces.
For a 2-member cluster, do not directly-connect the cluster control link from one ASA to the other ASA. If
you directly connect the interfaces, then when one unit fails, the cluster control link fails, and thus the remaining
healthy unit fails. If you connect the cluster control link through a switch, then the cluster control link remains
up for the healthy unit.
Related Topics
Cluster Control Link Redundancy, on page 7
Size the Cluster Control Link, on page 6
Size the Cluster Control Link
If possible, you should size the cluster control link to match the expected throughput of each chassis so the
cluster-control link can handle the worst-case scenarios. For example, if you have the ASA 5585-X with
SSP-60, which can pass 14 Gbps per unit maximum in a cluster, then you should also assign interfaces to the
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
6
ASA Cluster
Cluster Control Link
cluster control link that can pass at least 14 Gbps. In this case, you could use 2 Ten Gigabit Ethernet interfaces
in an EtherChannel for the cluster control link, and use the rest of the interfaces as desired for data links.
Cluster control link traffic is comprised mainly of state update and forwarded packets. The amount of traffic
at any given time on the cluster control link varies. The amount of forwarded traffic depends on the
load-balancing efficacy or whether there is a lot of traffic for centralized features. For example:
• NAT results in poor load balancing of connections, and the need to rebalance all returning traffic to the
correct units.
• AAA for network access is a centralized feature, so all traffic is forwarded to the primary unit.
• When membership changes, the cluster needs to rebalance a large number of connections, thus temporarily
using a large amount of cluster control link bandwidth.
A higher-bandwidth cluster control link helps the cluster to converge faster when there are membership changes
and prevents throughput bottlenecks.
Note
If your cluster has large amounts of asymmetric (rebalanced) traffic, then you should increase the cluster
control link size.
Related Topics
Inter-Site Clustering, on page 17
Cluster Control Link Redundancy
We recommend using an EtherChannel for the cluster control link, so that you can pass traffic on multiple
links in the EtherChannel while still achieving redundancy.
The following diagram shows how to use an EtherChannel as a cluster control link in a Virtual Switching
System (VSS) or Virtual Port Channel (vPC) environment. All links in the EtherChannel are active. When
the switch is part of a VSS or vPC, then you can connect ASA interfaces within the same EtherChannel to
separate switches in the VSS or vPC. The switch interfaces are members of the same EtherChannel port-channel
interface, because the separate switches act like a single switch. Note that this EtherChannel is device-local,
not a Spanned EtherChannel.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
7
ASA Cluster
High Availability Within the ASA Cluster
Cluster Control Link Reliability
To ensure cluster control link functionality, be sure the round-trip time (RTT) between units is less than 20
ms. This maximum latency enhances compatibility with cluster members installed at different geographical
sites. To check your latency, perform a ping on the cluster control link between units.
The cluster control link must be reliable, with no out-of-order or dropped packets; for example, for inter-site
deployment, you should use a dedicated link.
Cluster Control Link Failure
If the cluster control link line protocol goes down for a unit, then clustering is disabled; data interfaces are
shut down. After you fix the cluster control link, you must manually rejoin the cluster by re-enabling clustering.
Note
When the ASA becomes inactive, all data interfaces are shut down; only the management-only interface
can send and receive traffic. The management interface remains up using the IP address the unit received
from the cluster IP pool. However if you reload, and the unit is still inactive in the cluster, the management
interface is not accessible (because it then uses the Main IP address, which is the same as the primary
unit). You must use the console port for any further configuration.
Related Topics
Rejoining the Cluster, on page 9
High Availability Within the ASA Cluster
ASA Clustering provides high availability by monitoring unit and interface health and by replicating connection
states between units.
Unit Health Monitoring
The primary unit monitors every secondary unit by sending keepalive messages over the cluster control link
periodically (the period is configurable). Each secondary unit monitors the primary unit using the same
mechanism. If the unit health check fails, the unit is removed from the cluster.
Interface Monitoring
Each unit monitors the link status of all named hardware interfaces in use, and reports status changes to the
primary unit.
• Spanned EtherChannel—Uses cluster Link Aggregation Control Protocol (cLACP). Each unit monitors
the link status and the cLACP protocol messages to determine if the port is still active in the EtherChannel.
The status is reported to the primary unit.
• Individual interfaces (Routed mode only)—Each unit self-monitors its interfaces and reports interface
status to the primary unit.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
8
ASA Cluster
High Availability Within the ASA Cluster
When you enable health monitoring, all physical interfaces (including the main EtherChannel and redundant
interface types) are monitored by default; you can optionally disable monitoring per interface. Only named
interfaces can be monitored. For example, the named EtherChannel must fail to be considered failed, which
means all member ports of an EtherChannel must fail to trigger cluster removal (depending on your minimum
port bundling setting).
A unit is removed from the cluster if its monitored interfaces fail. The amount of time before the ASA removes
a member from the cluster depends on the type of interface and whether the unit is an established member or
is joining the cluster. For EtherChannels (spanned or not), if the interface is down on an established member,
then the ASA removes the member after 9 seconds. The ASA does not monitor interfaces for the first 90
seconds that a unit joins the cluster. Interface status changes during this time will not cause the ASA to be
removed from the cluster. For non-EtherChannels, the unit is removed after 500 ms, regardless of the member
state.
Status After Failure
When a unit in the cluster fails, the connections hosted by that unit are seamlessly transferred to other units;
state information for traffic flows is shared over the control cluster link.
If the primary unit fails, then another member of the cluster with the highest priority (lowest number) becomes
the primary.
The ASA automatically tries to rejoin the cluster, depending on the failure event.
Note
When the ASA becomes inactive and fails to automatically rejoin the cluster, all data interfaces are shut
down; only the management-only interface can send and receive traffic. The management interface remains
up using the IP address the unit received from the cluster IP pool. However if you reload, and the unit is
still inactive in the cluster, the management interface is disabled. You must use the console port for any
further configuration.
Related Topics
Rejoining the Cluster, on page 9
Rejoining the Cluster
After a cluster member is removed from the cluster, how it can rejoin the cluster depends on why it was
removed:
• Failed cluster control link when initially joining—After you resolve the problem with the cluster control
link, you must manually rejoin the cluster by re-enabling clustering at the console port by entering cluster
group name, and then enable.
• Failed cluster control link after joining the cluster—The ASA automatically tries to rejoin every 5
minutes, indefinitely. This behavior is configurable.
• Failed data interface—The ASA automatically tries to rejoin at 5 minutes, then at 10 minutes, and finally
at 20 minutes. If the join is not successful after 20 minutes, then the ASA disables clustering. After you
resolve the problem with the data interface, you have to manually enable clustering at the console port
by entering cluster group name, and then enable. This behavior is configurable.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
9
ASA Cluster
Configuration Replication
• Failed unit—If the unit was removed from the cluster because of a unit health check failure, then rejoining
the cluster depends on the source of the failure. For example, a temporary power failure means the unit
will rejoin the cluster when it starts up again as long as the cluster control link is up and clustering is
still enabled with the enable command. The ASA attempts to rejoin the cluster every 5 seconds.
See Configure the Primary Unit Bootstrap Settings, on page 51.
Data Path Connection State Replication
Every connection has one owner and at least one backup owner in the cluster. The backup owner does not
take over the connection in the event of a failure; instead, it stores TCP/UDP state information, so that the
connection can be seamlessly transferred to a new owner in case of a failure.
If the owner becomes unavailable, the first unit to receive packets from the connection (based on load balancing)
contacts the backup owner for the relevant state information so it can become the new owner.
Some traffic requires state information above the TCP or UDP layer. See the following table for clustering
support or lack of support for this kind of traffic.
Table 1: Features Replicated Across the Cluster
Traffic
State Support
Notes
Up time
Yes
Keeps track of the system up time.
ARP Table
Yes
Transparent mode only.
MAC address table
Yes
Transparent mode only.
User Identity
Yes
Includes AAA rules (uauth) and identity firewall.
IPv6 Neighbor database
Yes
—
Dynamic routing
Yes
—
SNMP Engine ID
No
—
VPN (Site-to-Site)
No
VPN sessions will be disconnected if the primary
unit fails.
Configuration Replication
All units in the cluster share a single configuration. You can only make configuration changes on the primary
unit, and changes are automatically synced to all other units in the cluster.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
10
ASA Cluster
ASA Cluster Management
ASA Cluster Management
One of the benefits of using ASA clustering is the ease of management. This section describes how to manage
the cluster.
Management Network
We recommend connecting all units to a single management network. This network is separate from the cluster
control link.
Management Interface
For the management interface, we recommend using one of the dedicated management interfaces. You can
configure the management interfaces as Individual interfaces (for both routed and transparent modes) or as a
Spanned EtherChannel interface.
We recommend using Individual interfaces for management, even if you use Spanned EtherChannels for your
data interfaces. Individual interfaces let you connect directly to each unit if necessary, while a Spanned
EtherChannel interface only allows remote connection to the current primary unit.
Note
If you use Spanned EtherChannel interface mode, and configure the management interface as an Individual
interface, you cannot enable dynamic routing for the management interface. You must use a static route.
For an Individual interface, the Main cluster IP address is a fixed address for the cluster that always belongs
to the current primary unit. For each interface, you also configure a range of addresses so that each unit,
including the current primary, can use a Local address from the range. The Main cluster IP address provides
consistent management access to an address; when a primary unit changes, the Main cluster IP address moves
to the new primary unit, so management of the cluster continues seamlessly. The Local IP address is used for
routing, and is also useful for troubleshooting.
For example, you can manage the cluster by connecting to the Main cluster IP address, which is always
attached to the current primary unit. To manage an individual member, you can connect to the Local IP address.
For outbound management traffic such as TFTP or syslog, each unit, including the primary unit, uses the
Local IP address to connect to the server.
For a Spanned EtherChannel interface, you can only configure one IP address, and that IP address is always
attached to the primary unit. You cannot connect directly to a secondary unit using the EtherChannel interface;
we recommend configuring the management interface as an Individual interface so that you can connect to
each unit. Note that you can use a device-local EtherChannel for management.
Primary Unit Management Vs. Secondary Unit Management
All management and monitoring can take place on the primary unit. From the primary unit, you can check
runtime statistics, resource usage, or other monitoring information of all units. You can also issue a command
to all units in the cluster, and replicate the console messages from secondary units to the primary unit.
You can monitor secondary units directly if desired. Although also available from the primary unit, you can
perform file management on secondary units (including backing up the configuration and updating images).
The following functions are not available from the primary unit:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
11
ASA Cluster
Load Balancing Methods
• Monitoring per-unit cluster-specific statistics.
• Syslog monitoring per unit (except for syslogs sent to the console when console replication is enabled).
• SNMP
• NetFlow
RSA Key Replication
When you create an RSA key on the primary unit, the key is replicated to all secondary units. If you have an
SSH session to the Main cluster IP address, you will be disconnected if the primary unit fails. The new primary
unit uses the same key for SSH connections, so that you do not need to update the cached SSH host key when
you reconnect to the new primary unit.
ASDM Connection Certificate IP Address Mismatch
By default, a self-signed certificate is used for the ASDM connection based on the Local IP address. If you
connect to the Main cluster IP address using ASDM, then a warning message about a mismatched IP address
appears because the certificate uses the Local IP address, and not the Main cluster IP address. You can ignore
the message and establish the ASDM connection. However, to avoid this type of warning, you can enroll a
certificate that contains the Main cluster IP address and all the Local IP addresses from the IP address pool.
You can then use this certificate for each cluster member.
Load Balancing Methods
Available load balancing methods depend on the firewall mode and interface type.
Spanned EtherChannel (Recommended)
You can group one or more interfaces per unit into an EtherChannel that spans all units in the cluster. The
EtherChannel aggregates the traffic across all the available active interfaces in the channel.
Spanned EtherChannel Benefits
The EtherChannel method of load-balancing is recommended over other methods for the following benefits:
• Faster failure discovery.
• Faster convergence time. Individual interfaces rely on routing protocols to load-balance traffic, and
routing protocols often have slow convergence during a link failure.
• Ease of configuration.
Related Topics
EtherChannels
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
12
ASA Cluster
Load Balancing Methods
Guidelines for Maximum Throughput
To achieve maximum throughput, we recommend the following:
• Use a load balancing hash algorithm that is “symmetric,” meaning that packets from both directions will
have the same hash, and will be sent to the same ASA in the Spanned EtherChannel. We recommend
using the source and destination IP address (the default) or the source and destination port as the hashing
algorithm.
• Use the same type of line cards when connecting the ASAs to the switch so that hashing algorithms
applied to all packets are the same.
Load Balancing
The EtherChannel link is selected using a proprietary hash algorithm, based on source or destination IP
addresses and TCP and UDP port numbers.
Note
On the ASA, do not change the load-balancing algorithm from the default. On the switch, we recommend
that you use one of the following algorithms: source-dest-ip or source-dest-ip-port (see the Cisco Nexus
OS or Cisco IOS port-channel load-balance command). Do not use a vlan keyword in the load-balance
algorithm because it can cause unevenly distributed traffic to the ASAs in a cluster.
The number of links in the EtherChannel affects load balancing.
Symmetric load balancing is not always possible. If you configure NAT, then forward and return packets will
have different IP addresses and/or ports. Return traffic will be sent to a different unit based on the hash, and
the cluster will have to redirect most returning traffic to the correct unit.
Related Topics
Customize the EtherChannel
NAT and Clustering, on page 31
EtherChannel Redundancy
The EtherChannel has built-in redundancy. It monitors the line protocol status of all links. If one link fails,
traffic is re-balanced between remaining links. If all links in the EtherChannel fail on a particular unit, but
other units are still active, then the unit is removed from the cluster.
Connecting to a VSS or vPC
You can include multiple interfaces per ASA in the Spanned EtherChannel. Multiple interfaces per ASA are
especially useful for connecting to both switches in a VSS or vPC.
Depending on your switches, you can configure up to 32 active links in the spanned EtherChannel. This feature
requires both switches in the vPC to support EtherChannels with 16 active links each (for example the Cisco
Nexus 7000 with F2-Series 10 Gigabit Ethernet Module).
For switches that support 8 active links in the EtherChannel, you can configure up to 16 active links in the
spanned EtherChannel when connecting to two switches in a VSS/vPC.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
13
ASA Cluster
Load Balancing Methods
If you want to use more than 8 active links in a spanned EtherChannel, you cannot also have standby links;
the support for 9 to 32 active links requires you to disable cLACP dynamic port priority that allows the use
of standby links. You can still use 8 active links and 8 standby links if desired, for example, when connecting
to a single switch.
The following figure shows a 32 active link spanned EtherChannel in an 8-ASA cluster and a 16-ASA cluster.
The following figure shows a 16 active link spanned EtherChannel in a 4-ASA cluster and an 8-ASA cluster.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
14
ASA Cluster
Load Balancing Methods
The following figure shows a traditional 8 active/8 standby link spanned EtherChannel in a 4-ASA cluster
and an 8-ASA cluster. The active links are shown as solid lines, while the inactive links are dotted. cLACP
load-balancing can automatically choose the best 8 links to be active in the EtherChannel. As shown, cLACP
helps achieve load balancing at the link level.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
15
ASA Cluster
Load Balancing Methods
Policy-Based Routing (Routed Firewall Mode Only)
When using Individual interfaces, each ASA interface maintains its own IP address and MAC address. One
method of load balancing is Policy-Based Routing (PBR).
We recommend this method if you are already using PBR, and want to take advantage of your existing
infrastructure. This method might offer additional tuning options vs. Spanned EtherChannel as well.
PBR makes routing decisions based on a route map and ACL. You must manually divide traffic between all
ASAs in a cluster. Because PBR is static, it may not achieve the optimum load balancing result at all times.
To achieve the best performance, we recommend that you configure the PBR policy so that forward and return
packets of a connection are directed to the same physical ASA. For example, if you have a Cisco router,
redundancy can be achieved by using Cisco IOS PBR with Object Tracking. Cisco IOS Object Tracking
monitors each ASA using ICMP ping. PBR can then enable or disable route maps based on reachability of a
particular ASA. See the following URLs for more details:
http://www.cisco.com/c/en/us/solutions/data-center-virtualization/intelligent-traffic-director/index.html
http://www.cisco.com/en/US/products/ps6599/products_white_paper09186a00800a4409.shtml
Note
If you use this method of load-balancing, you can use a device-local EtherChannel as an Individual
interface.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
16
ASA Cluster
Inter-Site Clustering
Equal-Cost Multi-Path Routing (Routed Firewall Mode Only)
When using Individual interfaces, each ASA interface maintains its own IP address and MAC address. One
method of load balancing is Equal-Cost Multi-Path (ECMP) routing.
We recommend this method if you are already using ECMP, and want to take advantage of your existing
infrastructure. This method might offer additional tuning options vs. Spanned EtherChannel as well.
ECMP routing can forward packets over multiple “best paths” that tie for top place in the routing metric. Like
EtherChannel, a hash of source and destination IP addresses and/or source and destination ports can be used
to send a packet to one of the next hops. If you use static routes for ECMP routing, then an ASA failure can
cause problems; the route continues to be used, and traffic to the failed ASA will be lost. If you use static
routes, be sure to use a static route monitoring feature such as Object Tracking. We recommend using dynamic
routing protocols to add and remove routes, in which case, you must configure each ASA to participate in
dynamic routing.
Note
If you use this method of load-balancing, you can use a device-local EtherChannel as an Individual
interface.
Nexus Intelligent Traffic Director (Routed Firewall Mode Only)
When using Individual interfaces, each ASA interface maintains its own IP address and MAC address.
Intelligent Traffic Director (ITD) is a high-speed hardware load-balancing solution for Nexus 5000, 6000,
7000, and 9000 switch series. In addition to fully covering the functional capabilities of traditional PBR, it
offers a simplified configuration workflow and multiple additional features for a more granular load distribution.
ITD supports IP stickiness, consistent hashing for bi-directional flow symmetry, virtual IP addressing, health
monitoring, sophisticated failure handling policies with N+M redundancy, weighted load-balancing, and
application IP SLA probes including DNS. Due to the dynamic nature of load-balancing, it achieves a more
even traffic distribution across all cluster members as compared to PBR. In order to achieve bi-directional
flow symmetry, we recommend configuring ITD such that forward and return packets of a connection are
directed to the same physical ASA. See the following URL for more details:
http://www.cisco.com/c/en/us/solutions/data-center-virtualization/intelligent-traffic-director/index.html
Inter-Site Clustering
For inter-site installations, you can take advantage of ASA clustering as long as you follow these guidelines.
Sizing the Data Center Interconnect
You should reserve bandwidth on the data center interconnect (DCI) for cluster control link traffic equivalent
to the following calculation:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
17
ASA Cluster
Inter-Site Clustering
If the number of members differs at each site, use the larger number for your calculation. The minimum
bandwidth for the DCI should not be less than the size of the cluster control link for one member.
For example:
• For 4 members at 2 sites:
◦4 cluster members total
◦2 members at each site
◦5 Gbps cluster control link per member
Reserved DCI bandwidth = 5 Gbps (2/2 x 5 Gbps).
• For 6 members at 3 sites, the size increases:
◦6 cluster members total
◦3 members at site 1, 2 members at site 2, and 1 member at site 3
◦10 Gbps cluster control link per member
Reserved DCI bandwidth = 15 Gbps (3/2 x 10 Gbps).
• For 2 members at 2 sites:
◦2 cluster members total
◦1 member at each site
◦10 Gbps cluster control link per member
Reserved DCI bandwidth = 10 Gbps (1/2 x 10 Gbps = 5 Gbps; but the minimum bandwidth should not
be less than the size of the cluster control link (10 Gbps)).
• For 8 members at 2 sites:
◦8 cluster members total
◦4 members at each site
◦5 Gbps cluster control link per member
Reserved DCI bandwidth = 10 Gbps (4/2 x 5 Gbps).
Site IDs and Site-Specific MAC Addresses
You can configure each cluster chassis to belong to a separate site ID.
Site IDs work with site-specific MAC addresses. Packets sourced from the cluster use a site-specific MAC
address, while packets received by the cluster use a global MAC address. This feature prevents the switches
from learning the same global MAC address from both sites on two different ports, which causes MAC
flapping; instead, they only learn the site MAC address. Site-specific MAC addresses are supported for routed
mode using Spanned EtherChannels only.
Site IDs are also used to enable flow mobility using LISP inspection.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
18
ASA Cluster
Inter-Site Clustering
LISP Inspection for Flow Mobility
You can inspect LISP traffic to enable flow mobility between sites.
About LISP
Data center virtual machine mobility such as VMware VMotion enables servers to migrate between data
centers while maintaining connections to clients. To support such data center server mobility, routers need to
be able to update the ingress route towards the server when it moves. Cisco Locator/ID Separation Protocol
(LISP) architecture separates the device identity, or endpoint identifier (EID), from its location, or routing
locator (RLOC), into two different numbering spaces, making server migration transparent to clients. For
example, when a server moves to a new site and a client sends traffic to the server, the router redirects traffic
to the new location.
LISP requires routers and servers in certain roles, such as the LISP egress tunnel router (ETR), ingress tunnel
router (ITR), first hop routers, map resolver (MR), and map server (MS). When the first hop router for the
server senses that the server is connected to a different router, it updates all of the other routers and databases
so that the ITR connected to the client can intercept, encapsulate, and send traffic to the new server location.
ASA LISP Support
The ASA does not run LISP itself; it can, however, inspect LISP traffic for location changes and then use this
information for seamless clustering operation. Without LISP integration, when a server moves to a new site,
traffic comes to an ASA cluster member at the new site instead of to the original flow owner. The new ASA
forwards traffic to the ASA at the old site, and then the old ASA has to send traffic back to the new site to
reach the server. This traffic flow is sub-optimal and is known as “tromboning” or “hair-pinning.”
With LISP integration, the ASA cluster members can inspect LISP traffic passing between the first hop router
and the ETR or ITR, and can then change the flow owner to be at the new site.
LISP Guidelines
• The ASA cluster members must reside between the first hop router and the ITR or ETR for the site. The
ASA cluster itself cannot be the first hop router for an extended segment.
• Only fully-distributed flows are supported; centralized flows, semi-distributed flows, or flows belonging
to individual units are not moved to new owners. Semi-distributed flows include applications, such as
SIP, where all child flows are owned by the same ASA that owns the parent flow.
• The cluster only moves Layer 3 and 4 flow states; some application data might be lost.
• For short-lived flows or non-business-critical flows, moving the owner may not be worthwhile. You
can control the types of traffic that are supported with this feature when you configure the inspection
policy, and should limit flow mobility to essential traffic.
ASA LISP Implementation
This feature includes several inter-related configurations (all of which are described in this chapter):
1 (Optional) Limit inspected EIDs based on the host or server IP address—The first hop router might send
EID-notify messages for hosts or networks the ASA cluster is not involved with, so you can limit the EIDs
to only those servers or networks relevant to your cluster. For example, if the cluster is only involved with
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
19
ASA Cluster
Inter-Site Clustering
2 sites, but LISP is running on 3 sites, you should only include EIDs for the 2 sites involved with the
cluster.
2 LISP traffic inspection—The ASA inspects LISP traffic on UDP port 4342 for the EID-notify message
sent between the first hop router and the ITR or ETR. The ASA maintains an EID table that correlates the
EID and the site ID. For example, you should inspect LISP traffic with a source IP address of the first hop
router and a destination address of the ITR or ETR. Note that LISP traffic is not assigned a director, and
LISP traffic itself does not participate in cluster state sharing.
3 Service Policy to enable flow mobility on specified traffic—You should enable flow mobility on
business-critical traffic. For example, you can limit flow mobility to only HTTPS traffic, and/or to traffic
to specific servers.
4 Site IDs—The ASA uses the site ID for each cluster unit to determine the new owner.
5 Cluster-level configuration to enable flow mobility—You must also enable flow mobility at the cluster
level. This on/off toggle lets you easily enable or disable flow mobility for a particular class of traffic or
applications.
Inter-Site Clustering Guidelines
See the following guidelines for inter-site clustering:
• Supports inter-site clustering in the following interface and firewall modes:
Interface Mode
Firewall Mode
Routed
Transparent
Individual Interface
Yes
N/A
Spanned EtherChannel
Yes
Yes
• For individual interface mode, when using ECMP towards a multicast Rendezvous Point (RP), we
recommend that you use a static route for the RP IP address using the Main cluster IP address as the
next hop. This static route prevents sending unicast PIM register packets to secondary units. If a secondary
unit receives a PIM register packet, then the packet is dropped, and the multicast stream cannot be
registered.
• The cluster control link latency must be less than 20 ms round-trip time (RTT).
• The cluster control link must be reliable, with no out-of-order or dropped packets; for example, you
should use a dedicated link.
• Do not configure connection rebalancing; you do not want connections rebalanced to cluster members
at a different site.
• The cluster implementation does not differentiate between members at multiple sites for incoming
connections; therefore, connection roles for a given connection may span across sites. This is expected
behavior.
• For transparent mode, if the cluster is placed between a pair of inside and outside routers (AKA
North-South insertion), you must ensure that both inside routers share a MAC address, and also that
both outside routers share a MAC address. When a cluster member at site 1 forwards a connection to a
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
20
ASA Cluster
Inter-Site Clustering
member at site 2, the destination MAC address is preserved. The packet will only reach the router at site
2 if the MAC address is the same as the router at site 1.
• For transparent mode, if the cluster is placed between data networks and the gateway router at each site
for firewalling between internal networks (AKA East-West insertion), then each gateway router should
use a First Hop Redundancy Protocol (FHRP) such as HSRP to provide identical virtual IP and MAC
address destinations at each site. The data VLANs are extended across the sites using Overlay Transport
Virtualization (OTV), or something similar. You need to create filters to prevent traffic that is destined
to the local gateway router from being sent over the DCI to the other site. If the gateway router becomes
unreachable at one site, you need to remove any filters so traffic can successfully reach the other site’s
gateway.
• For routed mode using Spanned EtherChannel, configure site-specific MAC addresses. Extend the data
VLANs across the sites using OTV, or something similar. You need to create filters to prevent traffic
that is destined to the global MAC address from being sent over the DCI to the other site. If the cluster
becomes unreachable at one site, you need to remove any filters so traffic can successfully reach the
other site’s cluster units. Dynamic routing is not supported when an inter-site cluster acts as the first hop
router for an extended segment.
Related Topics
Rebalancing New TCP Connections Across the Cluster, on page 27
Connection Roles, on page 25
Inter-Site Examples
The following examples show supported cluster deployments.
Individual Interface Routed Mode North-South Inter-Site Example
The following example shows 2 ASA cluster members at each of 2 data centers placed between inside and
outside routers (North-South insertion). The cluster members are connected by the cluster control link over
the DCI. The inside and outside routers at each data center use OSPF and PBR or ECMP to load balance the
traffic between cluster members. By assigning a higher cost route across the DCI, traffic stays within each
data center unless all ASA cluster members at a given site go down. In the event of a failure of all cluster
members at one site, traffic goes from each router over the DCI to the ASA cluster members at the other site.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
21
ASA Cluster
Inter-Site Clustering
Spanned EtherChannel Routed Mode Example with Site-Specific MAC Addresses
The following example shows 2 cluster members at each of 2 data centers placed between the gateway router
and an inside network at each site (East-West insertion). The cluster members are connected by the cluster
control link over the DCI. The cluster members at each site connect to the local switches using spanned
EtherChannels for both the inside and outside networks. Each EtherChannel is spanned across all chassis in
the cluster.
The data VLANs are extended between the sites using Overlay Transport Virtualization (OTV) (or something
similar). You must add filters blocking the global MAC address to prevent traffic from traversing the DCI to
the other site when the traffic is destined for the cluster. If the cluster units at one site become unreachable,
you must remove the filters so traffic can be sent to the other site’s cluster units. You should use VACLs to
filter the global MAC address.Be sure to disable ARP inspection. See OTV Configuration for Routed Mode
Inter-Site Clustering, on page 88 for more information.
The cluster acts as the gateway for the inside networks. The global virtual MAC, which is shared across all
cluster units, is used only to receive packets. Outgoing packets use a site-specific MAC address from each
DC cluster. This feature prevents the switches from learning the same global MAC address from both sites
on two different ports, which causes MAC flapping; instead, they only learn the site MAC address.
In this scenario:
• All egress packets sent from the cluster use the site MAC address and are localized at the data center.
• All ingress packets to the cluster are sent using the global MAC address, so they can be received by any
of the units at both sites; filters at the OTV localize the traffic within the data center.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
22
ASA Cluster
Inter-Site Clustering
For a sample OTV configuration and best practices, see OTV Configuration for Routed Mode Inter-Site
Clustering, on page 88.
Spanned EtherChannel Transparent Mode North-South Inter-Site Example
The following example shows 2 cluster members at each of 2 data centers placed between inside and outside
routers (North-South insertion). The cluster members are connected by the cluster control link over the DCI.
The cluster members at each site connect to the local switches using spanned EtherChannels for the inside
and outside. Each EtherChannel is spanned across all chassis in the cluster.
The inside and outside routers at each data center use OSPF, which is passed through the transparent ASAs.
Unlike MACs, router IPs are unique on all routers. By assigning a higher cost route across the DCI, traffic
stays within each data center unless all cluster members at a given site go down. The lower cost route through
the ASAs must traverse the same bridge group at each site for the cluster to maintain asymmetric connections.
In the event of a failure of all cluster members at one site, traffic goes from each router over the DCI to the
cluster members at the other site.
The implementation of the switches at each site can include:
• Inter-site VSS/vPC—In this scenario, you install one switch at Data Center 1, and the other at Data
Center 2. One option is for the cluster units at each Data Center to only connect to the local switch, while
the VSS/vPC traffic goes across the DCI. In this case, connections are for the most part kept local to
each datacenter. You can optionally connect each unit to both switches across the DCI if the DCI can
handle the extra traffic. In this case, traffic is distributed across the data centers, so it is essential for the
DCI to be very robust.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
23
ASA Cluster
Inter-Site Clustering
• Local VSS/vPC at each site—For better switch redundancy, you can install 2 separate VSS/vPC pairs
at each site. In this case, although the cluster units still have a spanned EtherChannel with Data Center
1 chassis connected only to both local switches, and Data Center 2 chassis connected to those local
switches, the spanned EtherChannel is essentially “split.” Each local VSS/vPC sees the spanned
EtherChannel as a site-local EtherChannel.
Spanned EtherChannel Transparent Mode East-West Inter-Site Example
The following example shows 2 cluster members at each of 2 data centers placed between the gateway router
and two inside networks at each site, the App network and the DB network (East-West insertion). The cluster
members are connected by the cluster control link over the DCI. The cluster members at each site connect to
the local switches using spanned EtherChannels for both the App and DB networks on the inside and outside.
Each EtherChannel is spanned across all chassis in the cluster.
The gateway router at each site uses an FHRP such as HSRP to provide the same destination virtual MAC
and IP addresses at each site. A good practice to avoid unintended MAC address flapping is to statically add
the gateway routers real MAC addresses to the ASA MAC address table using the mac-address-table static
outside_interface mac_address command. Without these entries, if the gateway at site 1 communicates with
the gateway at site 2, that traffic might pass through the ASA and attempt to reach site 2 from the inside
interface and cause problems. The data VLANs are extended between the sites using Overlay Transport
Virtualization (OTV) (or something similar). You must add filters to prevent traffic from traversing the DCI
to the other site when the traffic is destined for the gateway router. If the gateway router at one site becomes
unreachable, you must remove the filters so traffic can be sent to the other site’s gateway router.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
24
ASA Cluster
How the ASA Cluster Manages Connections
See Spanned EtherChannel Transparent Mode North-South Inter-Site Example, on page 23 for information
about vPC/VSS options.
How the ASA Cluster Manages Connections
Connections can be load-balanced to multiple members of the cluster. Connection roles determine how
connections are handled in both normal operation and in a high availability situation.
Connection Roles
There are 3 different roles defined for each connection:
• Owner—The unit that initially receives the connection. The owner maintains the TCP state and processes
packets. A connection has only one owner. If the original owner fails, then when new units receive
packets from the connection, the director chooses a new owner from those units.
• Director—The unit that handles owner lookup requests from forwarders and also maintains the connection
state to serve as a backup if the owner fails. When the owner receives a new connection, it chooses a
director based on a hash of the source/destination IP address and TCP ports, and sends a message to the
director to register the new connection. If packets arrive at any unit other than the owner, the unit queries
the director about which unit is the owner so it can forward the packets. A connection has only one
director. If a director fails, the owner chooses a new director.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
25
ASA Cluster
How the ASA Cluster Manages Connections
• Forwarder—A unit that forwards packets to the owner. If a forwarder receives a packet for a connection
it does not own, it queries the director for the owner, and then establishes a flow to the owner for any
other packets it receives for this connection. The director can also be a forwarder. Note that if a forwarder
receives the SYN-ACK packet, it can derive the owner directly from a SYN cookie in the packet, so it
does not need to query the director. (If you disable TCP sequence randomization, the SYN cookie is not
used; a query to the director is required.) For short-lived flows such as DNS and ICMP, instead of
querying, the forwarder immediately sends the packet to the director, which then sends them to the
owner. A connection can have multiple forwarders; the most efficient throughput is achieved by a good
load-balancing method where there are no forwarders and all packets of a connection are received by
the owner.
When a connection uses Port Address Translation (PAT), then the PAT type (per-session or multi-session)
influences which member of the cluster becomes the owner of a new connection:
• Per-session PAT—The owner is the unit that receives the initial packet in the connection.
By default, TCP and DNS UDP traffic use per-session PAT.
• Multi-session PAT—The owner is always the primary unit. If a multi-session PAT connection is initially
received by a secondary unit, then the secondary unit forwards the connection to the primary unit.
By default, UDP (except for DNS UDP) and ICMP traffic use multi-session PAT, so these connections
are always owned by the primary unit.
You can change the per-session PAT defaults for TCP and UDP so connections for these protocols are handled
per-session or multi-session depending on the configuration. For ICMP, you cannot change from the default
multi-session PAT. For more information about per-session PAT, see the firewall configuration guide.
New Connection Ownership
When a new connection is directed to a member of the cluster via load balancing, that unit owns both directions
of the connection. If any connection packets arrive at a different unit, they are forwarded to the owner unit
over the cluster control link. For best performance, proper external load balancing is required for both directions
of a flow to arrive at the same unit, and for flows to be distributed evenly between units. If a reverse flow
arrives at a different unit, it is redirected back to the original unit.
Related Topics
Load Balancing Methods, on page 12
Sample Data Flow
The following example shows the establishment of a new connection.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
26
ASA Cluster
ASA Features and Clustering
1 The SYN packet originates from the client and is delivered to one ASA (based on the load balancing
method), which becomes the owner. The owner creates a flow, encodes owner information into a SYN
cookie, and forwards the packet to the server.
2 The SYN-ACK packet originates from the server and is delivered to a different ASA (based on the load
balancing method). This ASA is the forwarder.
3 Because the forwarder does not own the connection, it decodes owner information from the SYN cookie,
creates a forwarding flow to the owner, and forwards the SYN-ACK to the owner.
4 The owner sends a state update to the director, and forwards the SYN-ACK to the client.
5 The director receives the state update from the owner, creates a flow to the owner, and records the TCP
state information as well as the owner. The director acts as the backup owner for the connection.
6 Any subsequent packets delivered to the forwarder will be forwarded to the owner.
7 If packets are delivered to any additional units, it will query the director for the owner and establish a flow.
8 Any state change for the flow results in a state update from the owner to the director.
Rebalancing New TCP Connections Across the Cluster
If the load balancing capabilities of the upstream or downstream routers result in unbalanced flow distribution,
you can configure overloaded units to redirect new TCP flows to other units. No existing flows will be moved
to other units.
ASA Features and Clustering
Some ASA features are not supported with ASA clustering, and some are only supported on the primary unit.
Other features might have caveats for proper usage.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
27
ASA Cluster
ASA Features and Clustering
Unsupported Features with Clustering
These features cannot be configured with clustering enabled, and the commands will be rejected.
• Unified Communication features that rely on TLS Proxy
• Remote access VPN (SSL VPN and IPsec VPN)
• The following application inspections:
◦CTIQBE
◦H323, H225, and RAS
◦IPsec passthrough
◦MGCP
◦MMP
◦RTSP
◦SCCP (Skinny)
◦WAAS
◦WCCP
• Botnet Traffic Filter
• Auto Update Server
• DHCP client, server, and proxy. DHCP relay is supported.
• VPN load balancing
• Failover
• ASA CX module
Centralized Features for Clustering
The following features are only supported on the primary unit, and are not scaled for the cluster. For example,
you have a cluster of eight units (5585-X with SSP-60). The Other VPN license allows a maximum of 10,000
site-to-site IPsec tunnels for one ASA 5585-X with SSP-60. For the entire cluster of eight units, you can only
use 10,000 tunnels; the feature does not scale.
Note
Traffic for centralized features is forwarded from member units to the primary unit over the cluster control
link.
If you use the rebalancing feature, traffic for centralized features may be rebalanced to non-primary units
before the traffic is classified as a centralized feature; if this occurs, the traffic is then sent back to the
primary unit.
For centralized features, if the primary unit fails, all connections are dropped, and you have to re-establish
the connections on the new primary unit.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
28
ASA Cluster
ASA Features and Clustering
• Site-to-site VPN
• The following application inspections:
◦DCERPC
◦ESMTP
◦IM
◦NetBIOS
◦PPTP
◦RADIUS
◦RSH
◦SNMP
◦SQLNET
◦SUNRPC
◦TFTP
◦XDMCP
• Dynamic routing (Spanned EtherChannel mode only)
• Multicast routing (Individual interface mode only)
• Static route monitoring
• IGMP multicast control plane protocol processing (data plane forwarding is distributed across the cluster)
• PIM multicast control plane protocol processing (data plane forwarding is distributed across the cluster)
• Authentication and Authorization for network access. Accounting is decentralized.
• Filtering Services
Related Topics
Size the Cluster Control Link, on page 6
Rebalancing New TCP Connections Across the Cluster, on page 27
Features Applied to Individual Units
These features are applied to each ASA unit, instead of the cluster as a whole or to the primary unit.
• QoS—The QoS policy is synced across the cluster as part of configuration replication. However, the
policy is enforced on each unit independently. For example, if you configure policing on output, then
the conform rate and conform burst values are enforced on traffic exiting a particular ASA. In a cluster
with 3 units and with traffic evenly distributed, the conform rate actually becomes 3 times the rate for
the cluster.
• Threat detection—Threat detection works on each unit independently; for example, the top statistics is
unit-specific. Port scanning detection, for example, does not work because scanning traffic will be
load-balanced between all units, and one unit will not see all traffic.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
29
ASA Cluster
ASA Features and Clustering
• Resource management—Resource management in multiple context mode is enforced separately on each
unit based on local usage.
• LISP traffic—LISP traffic on UDP port 4342 is inspected by each receiving unit, but is not assigned a
director. Each unit adds to the EID table that is shared across the cluster, but the LISP traffic itself does
not participate in cluster state sharing.
• ASA Firepower module—There is no configuration sync or state sharing between ASA Firepower
modules. You are responsible for maintaining consistent policies on the ASA Firepower modules in the
cluster using Firepower Management Center. Do not use different ASA-interface-based zone definitions
for devices in the cluster.
• ASA IPS module—There is no configuration sync or state sharing between IPS modules. Some IPS
signatures require IPS to keep the state across multiple connections. For example, the port scanning
signature is used when the IPS module detects that someone is opening many connections to one server
but with different ports. In clustering, those connections will be balanced between multiple ASA devices,
each of which has its own IPS module. Because these IPS modules do not share state information, the
cluster may not be able to detect port scanning as a result.
AAA for Network Access and Clustering
AAA for network access consists of three components: authentication, authorization, and accounting.
Authentication and accounting are implemented as centralized features on the clustering primary with replication
of the data structures to the cluster secondaries. If a primary is elected, the new primary will have all the
information it needs to continue uninterrupted operation of the established authenticated users and their
associated authorizations. Idle and absolute timeouts for user authentications are preserved when a primary
unit change occurs.
Accounting is implemented as a distributed feature in a cluster. Accounting is done on a per-flow basis, so
the cluster unit owning a flow will send accounting start and stop messages to the AAA server when accounting
is configured for a flow.
FTP and Clustering
• If FTP data channel and control channel flows are owned by different cluster members, the data channel
owner will periodically send idle timeout updates to the control channel owner and update the idle
timeout value. However, if the control flow owner is reloaded, and the control flow is re-hosted, the
parent/child flow relationship will not longer be maintained; the control flow idle timeout will not be
updated.
• If you use AAA for FTP access, then the control channel flow is centralized on the primary unit.
Identity Firewall and Clustering
Only the primary unit retrieves the user-group from the AD and the user-ip mapping from the AD agent. The
primary unit then populates the user information to secondaries, and secondaries can make a match decision
for user identity based on the security policy.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
30
ASA Cluster
ASA Features and Clustering
Multicast Routing and Clustering
Multicast routing behaves differently depending on the interface mode.
Multicast Routing in Spanned EtherChannel Mode
In Spanned EtherChannel mode, the primary unit handles all multicast routing packets and data packets until
fast-path forwarding is established. After the connection is established, each secondary can forward multicast
data packets.
Multicast Routing in Individual Interface Mode
In Individual interface mode, units do not act independently with multicast. All data and routing packets are
processed and forwarded by the primary unit, thus avoiding packet replication.
NAT and Clustering
NAT can impact the overall throughput of the cluster. Inbound and outbound NAT packets can be sent to
different ASA in the cluster because the load balancing algorithm relies on IP addresses and ports, and NAT
causes inbound and outbound packets to have different IP addresses and/or ports. When a packet arrives at
the ASA that is not the connection owner, it is forwarded over the cluster control link to the owner, causing
large amounts of traffic on the cluster control link.
If you still want to use NAT in clustering, then consider the following guidelines:
• No Proxy ARP—For Individual interfaces, a proxy ARP reply is never sent for mapped addresses. This
prevents the adjacent router from maintaining a peer relationship with an ASA that may no longer be in
the cluster. The upstream router needs a static route or PBR with Object Tracking for the mapped
addresses that points to the Main cluster IP address. This is not an issue for a Spanned EtherChannel,
because there is only one IP address associated with the cluster interface.
• No interface PAT on an Individual interface—Interface PAT is not supported for Individual interfaces.
• No PAT with Port Block Allocation—This feature is not supported for the cluster.
• PAT with Port Block Allocation—See the following guidelines for this feature:
◦Maximum-per-host limit is not a cluster-wide limit, and is enforced on each unit individually.
Thus, in a 3-node cluster with the maximum-per-host limit configured as 1, if the traffic from a
host is load-balanced across all 3 units, then it can get allocated 3 blocks with 1 in each unit.
◦Port blocks created on the backup unit from the backup pools are not accounted for when enforcing
the maximum-per-host limit.
◦When a PAT IP address owner goes down, the backup unit will own the PAT IP address,
corresponding port blocks, and xlates. But it will not use these blocks to service new requests. As
the connections eventually time out, the blocks get freed.
◦On-the-fly PAT rule modifications, where the PAT pool is modified with a completely new range
of IP addresses, will result in xlate backup creation failures for the xlate backup requests that were
still in transit while the new pool became effective. This behavior is not specific to the port block
allocation feature, and is a transient PAT pool issue seen only in cluster deployments where the
pool is distributed and traffic is load-balanced across the cluster units.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
31
ASA Cluster
ASA Features and Clustering
• NAT pool address distribution for dynamic PAT—The primary unit evenly pre-distributes addresses
across the cluster. If a member receives a connection and they have no addresses left, the connection is
dropped, even if other members still have addresses available. Make sure to include at least as many
NAT addresses as there are units in the cluster to ensure that each unit receives an address. Use the show
nat pool cluster command to see the address allocations.
• No round-robin—Round-robin for a PAT pool is not supported with clustering.
• Dynamic NAT xlates managed by the primary unit—The primary unit maintains and replicates the xlate
table to secondary units. When a secondary unit receives a connection that requires dynamic NAT, and
the xlate is not in the table, it requests the xlate from the primary unit. The secondary unit owns the
connection.
• Per-session PAT feature—Although not exclusive to clustering, the per-session PAT feature improves
the scalability of PAT and, for clustering, allows each secondary unit to own PAT connections; by
contrast, multi-session PAT connections have to be forwarded to and owned by the primary unit. By
default, all TCP traffic and UDP DNS traffic use a per-session PAT xlate, whereas ICMP and all other
UDP traffic uses multi-session. You can configure per-session NAT rules to change these defaults for
TCP and UDP, but you cannot configure per-session PAT for ICMP. For traffic that benefits from
multi-session PAT, such as H.323, SIP, or Skinny, you can disable per-session PAT for the associated
TCP ports (the UDP ports for those H.323 and SIP are already multi-session by default). For more
information about per-session PAT, see the firewall configuration guide.
• No static PAT for the following inspections—
◦FTP
◦PPTP
◦RSH
◦SQLNET
◦TFTP
◦XDMCP
◦SIP
Dynamic Routing and Clustering
This section describes how to use dynamic routing with clustering.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
32
ASA Cluster
ASA Features and Clustering
Dynamic Routing in Spanned EtherChannel Mode
In Spanned EtherChannel mode: The routing process only runs on the primary unit, and routes are learned
through the primary unit and replicated to secondaries. If a routing packet arrives at a secondary, it is redirected
to the primary unit.
Figure 1: Dynamic Routing in Spanned EtherChannel Mode
After the secondary members learn the routes from the primary unit, each unit makes forwarding decisions
independently.
The OSPF LSA database is not synchronized from the primary unit to secondary units. If there is a primary
unit switchover, the neighboring router will detect a restart; the switchover is not transparent. The OSPF
process picks an IP address as its router ID. Although not required, you can assign a static router ID to ensure
a consistent router ID is used across the cluster. See the OSPF Non-Stop Forwarding feature to address the
interruption.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
33
ASA Cluster
ASA Features and Clustering
Dynamic Routing in Individual Interface Mode
In Individual interface mode, each unit runs the routing protocol as a standalone router, and routes are learned
by each unit independently.
Figure 2: Dynamic Routing in Individual Interface Mode
In the above diagram, Router A learns that there are 4 equal-cost paths to Router B, each through an ASA.
ECMP is used to load balance traffic between the 4 paths. Each ASA picks a different router ID when talking
to external routers.
You must configure a cluster pool for the router ID so that each unit has a separate router ID.
EIGRP does not form neighbor relationships with cluster peers in individual interface mode.
Note
If the cluster has multiple adjacencies to the same router for redundancy purposes, asymmetric routing
can lead to unacceptible traffic loss. To avoid asymmetric routing, group all of these ASA interfaces into
the same traffic zone. See Configure a Traffic Zone.
SCTP and Clustering
When you enable clustering, the following SCTP features are not available:
• SCTP Layer 4 stateful inspection, including protocol validation and state tracking.
• You cannot configure NAT for SCTP traffic.
• You cannot configure SCTP state bypass.
• You cannot implement Diameter application layer inspection on SCTP traffic. However, Diameter
inspection of TCP traffic is supported.
You can use the following SCTP features with clustering enabled:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
34
ASA Cluster
ASA Features and Clustering
• SCTP protocol/port specifications in access rules, extended ACLs, and service objects and groups.
• SCTP application layer inspection, including payload protocol identifier (PPID) filtering.
SIP Inspection and Clustering
A control flow can be created on any unit (due to load balancing); its child data flows must reside on the same
unit.
TLS Proxy configuration is not supported.
SNMP and Clustering
An SNMP agent polls each individual ASA by its Local IP address. You cannot poll consolidated data for
the cluster.
You should always use the Local address, and not the Main cluster IP address for SNMP polling. If the SNMP
agent polls the Main cluster IP address, if a new primary is elected, the poll to the new primary unit will fail.
Syslog and NetFlow and Clustering
• Syslog—Each unit in the cluster generates its own syslog messages. You can configure logging so that
each unit uses either the same or a different device ID in the syslog message header field. For example,
the hostname configuration is replicated and shared by all units in the cluster. If you configure logging
to use the hostname as the device ID, syslog messages generated by all units look as if they come from
a single unit. If you configure logging to use the local-unit name that is assigned in the cluster bootstrap
configuration as the device ID, syslog messages look as if they come from different units.
• NetFlow—Each unit in the cluster generates its own NetFlow stream. The NetFlow collector can only
treat each ASA as a separate NetFlow exporter.
Related Topics
Include the Device ID in Non-EMBLEM Format Syslog Messages
Cisco TrustSec and Clustering
Only the primary unit learns security group tag (SGT) information. The primary unit then populates the SGT
to secondaries, and secondaries can make a match decision for SGT based on the security policy.
VPN and Clustering
Site-to-site VPN is a centralized feature; only the primary unit supports VPN connections.
Note
Remote access VPN is not supported with clustering.
VPN functionality is limited to the primary unit and does not take advantage of the cluster high availability
capabilities. If the primary unit fails, all existing VPN connections are lost, and VPN users will see a disruption
in service. When a new primary is elected, you must reestablish the VPN connections.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
35
ASA Cluster
Licensing for ASA Clustering
When you connect a VPN tunnel to a Spanned EtherChannel address, connections are automatically forwarded
to the primary unit. For connections to an Individual interface when using PBR or ECMP, you must always
connect to the Main cluster IP address, not a Local address.
VPN-related keys and certificates are replicated to all units.
Licensing for ASA Clustering
Cluster units do not require the same license on each unit. Typically, you buy a license only for the primary
unit; secondary units inherit the primary license. If you have licenses on multiple units, they combine into a
single running ASA cluster license.
There are exceptions to this rule. See the following table for precise licensing requirements for clustering.
Model
License Requirement
ASA 5585-X
Cluster License, supports up to 16 units.
Note
ASA 5516-X
Base license, supports 2 units.
Note
ASA 5512-X
Each unit must have the same encryption license; each unit must have the same 10 GE I/O/Security
Plus license (ASA 5585-X with SSP-10 and -20).
Each unit must have the same encryption
license.
Security Plus license, supports 2 units.
Note
Each unit must have the same encryption
license.
ASA 5515-X,
ASA 5525-X,
ASA 5545-X,
ASA 5555-X
Base License, supports 2 units.
ASA on the FXOS
Chassis
See ASA Cluster for the FXOS Chassis.
All other models
No support.
Note
Each unit must have the same encryption
license.
Prerequisites for ASA Clustering
ASA Hardware and Software Requirements
All units in a cluster:
• Must be the same model with the same DRAM. You do not have to have the same amount of flash
memory.
• Must run the identical software except at the time of an image upgrade. Hitless upgrade is supported.
• Must be in the same security context mode, single or multiple.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
36
ASA Cluster
Guidelines for ASA Clustering
• (Single context mode) Must be in the same firewall mode, routed or transparent.
• New cluster members must use the same SSL encryption setting (the ssl encryption command) as the
primary unit for initial cluster control link communication before configuration replication.
• Must have the same cluster, encryption and, for the ASA 5585-X, 10 GE I/O licenses.
Switch Prerequisites
• Be sure to complete the switch configuration before you configure clustering on the ASAs.
• For a list of supported switches, see Cisco ASA Compatibility.
ASA Prerequisites
• Provide each unit with a unique IP address before you join them to the management network.
◦See the Getting Started chapter for more information about connecting to the ASA and setting the
management IP address.
◦Except for the IP address used by the primary unit (typically the first unit you add to the cluster),
these management IP addresses are for temporary use only.
◦After a secondary joins the cluster, its management interface configuration is replaced by the one
replicated from the primary unit.
• To use jumbo frames on the cluster control link (recommended), you must enable Jumbo Frame
Reservation before you enable clustering.
Other Prerequisites
We recommend using a terminal server to access all cluster member unit console ports. For initial setup, and
ongoing management (for example, when a unit goes down), a terminal server is useful for remote management.
Related Topics
Guidelines for ASA Clustering, on page 37
Enable Jumbo Frame Support
Bootstrap Configuration, on page 2
Guidelines for ASA Clustering
Context Mode
The mode must match on each member unit.
Firewall Mode
For single mode, the firewall mode must match on all units.
Failover
Failover is not supported with clustering.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
37
ASA Cluster
Guidelines for ASA Clustering
IPv6
The cluster control link is only supported using IPv4.
Models
Supported on:
• ASA 5585-X—For the ASA 5585-X with SSP-10 and SSP-20, which include two Ten Gigabit Ethernet
interfaces, we recommend using one interface for the cluster control link, and the other for data (you
can use subinterfaces for data). Although this setup does not accommodate redundancy for the cluster
control link, it does satisfy the need to size the cluster control link to match the size of the data interfaces.
• ASA 5516-X
• ASA 5512-X, ASA 5515-X, ASA 5525-X, ASA 5545-X, and ASA 5555-X
Switches
• For the ASR 9006, if you want to set a non-default MTU, set the ASR interface MTU to be 14 bytes
higher than the device MTU. Otherwise, OSPF adjacency peering attempts may fail unless the mtu-ignore
option is used. Note that the device MTU should match the ASR IPv4 MTU.
• On the switch(es) for the cluster control link interfaces, you can optionally enable Spanning Tree PortFast
on the switch ports connected to the cluster unit to speed up the join process for new units.
• When you see slow bundling of a Spanned EtherChannel on the switch, you can enable LACP rate fast
for an Individual interface on the switch.
• On the switch, we recommend that you use one of the following EtherChannel load-balancing algorithms:
source-dest-ip or source-dest-ip-port (see the Cisco Nexus OS and Cisco IOS port-channel
load-balance command). Do not use a vlan keyword in the load-balance algorithm because it can cause
unevenly distributed traffic to the devices in a cluster. Do not change the load-balancing algorithm from
the default on the device.
• If you change the load-balancing algorithm of the EtherChannel on the switch, the EtherChannel interface
on the switch temporarily stops forwarding traffic, and the Spanning Tree Protocol restarts. There will
be a delay before traffic starts flowing again.
• You should disable the LACP Graceful Convergence feature on all cluster-facing EtherChannel interfaces
for Cisco Nexus switches.
• Some switches do not support dynamic port priority with LACP (active and standby links). You can
disable dynamic port priority to provide better compatibility with spanned EtherChannels.
• Network elements on the cluster control link path should not verify the L4 checksum. Redirected traffic
over the cluster control link does not have a correct L4 checksum. Switches that verify the L4 checksum
could cause traffic to be dropped.
• Port-channel bundling downtime should not exceed the configured keepalive interval.
• On Supervisor 2T EtherChannels, the default hash distribution algorithm is adaptive. To avoid asymmetric
traffic in a VSS design, change the hash algorithm on the port-channel connected to the device to fixed:
router(config)# port-channel id hash-distribution fixed
Do not change the algorithm globally; you may want to take advantage of the adaptive algorithm for the
VSS peer link.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
38
ASA Cluster
Guidelines for ASA Clustering
EtherChannels
• In Catalyst 3750-X Cisco IOS software versions earlier than 15.1(1)S2, the cluster unit did not support
connecting an EtherChannel to a switch stack. With default switch settings, if the cluster unit EtherChannel
is connected cross stack, and if the master switch is powered down, then the EtherChannel connected
to the remaining switch will not come up. To improve compatibility, set the stack-mac persistent timer
command to a large enough value to account for reload time; for example, 8 minutes or 0 for indefinite.
Or, you can upgrade to more a more stable switch software version, such as 15.1(1)S2.
• Spanned vs. Device-Local EtherChannel Configuration—Be sure to configure the switch appropriately
for Spanned EtherChannels vs. Device-local EtherChannels.
◦Spanned EtherChannels—For cluster unit Spanned EtherChannels, which span across all members
of the cluster, the interfaces are combined into a single EtherChannel on the switch. Make sure
each interface is in the same channel group on the switch.
◦Device-local EtherChannels—For cluster unit Device-local EtherChannels including any
EtherChannels configured for the cluster control link, be sure to configure discrete EtherChannels
on the switch; do not combine multiple cluster unit EtherChannels into one EtherChannel on the
switch.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
39
ASA Cluster
Guidelines for ASA Clustering
Additional Guidelines
• When significant topology changes occur (such as adding or removing an EtherChannel interface,
enabling or disabling an interface on the ASA or the switch, adding an additional switch to form a VSS
or vPC) you should disable the health check feature and also disable interface monitoring for the disabled
interfaces. When the topology change is complete, and the configuration change is synced to all units,
you can re-enable the interface health check feature.
• When adding a unit to an existing cluster, or when reloading a unit, there will be a temporary, limited
packet/connection drop; this is expected behavior. In some cases, the dropped packets can hang your
connection; for example, dropping a FIN/ACK packet for an FTP connection will make the FTP client
hang. In this case, you need to reestablish the FTP connection.
• If you use a Windows 2003 server connected to a Spanned EtherChannel, when the syslog server port
is down and the server does not throttle ICMP error messages, then large numbers of ICMP messages
are sent back to the ASA cluster. These messages can result in some units of the ASA cluster experiencing
high CPU, which can affect performance. We recommend that you throttle ICMP error messages.
• We do not support VXLAN in Individual Interface mode. Only Spanned EtherChannel mode supports
VXLAN.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
40
ASA Cluster
Defaults for ASA Clustering
Related Topics
Size the Cluster Control Link, on page 6
Bootstrap Configuration, on page 2
Unsupported Features with Clustering, on page 28
Configure an EtherChannel
Guidelines for EtherChannels and Redundant Interfaces
Defaults for ASA Clustering
• When using Spanned EtherChannels, the cLACP system ID is auto-generated and the system priority
is 1 by default.
• The cluster health check feature is enabled by default with the holdtime of 3 seconds. Interface health
monitoring is enabled on all interfaces by default.
• The cluster auto-rejoin feature for a failed cluster control link is unlimited attempts every 5 minutes.
• The cluster auto-rejoin feature for a failed data interface is 3 attempts every 5 minutes, with the increasing
interval set to 2.
• Connection rebalancing is disabled by default. If you enable connection rebalancing, the default time
between load information exchanges is 5 seconds.
• Connection replication delay of 5 seconds is enabled by default for HTTP traffic.
Configure ASA Clustering
To configure clustering, perform the following tasks.
Note
To enable or disable clustering, you must use a console connection (for CLI) or an ASDM connection.
Cable the Cluster Units and Configure Upstream and Downstream Equipment
Before configuring clustering, cable the cluster control link network, management network, and data networks.
Note
At a minimum, an active cluster control link network is required before you configure the units to join
the cluster.
You should also configure the upstream and downstream equipment. For example, if you use EtherChannels,
then you should configure the upstream and downstream equipment for the EtherChannels.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
41
ASA Cluster
Cable the Cluster Units and Configure Upstream and Downstream Equipment
Examples
Note
This example uses EtherChannels for load-balancing. If you are using PBR or ECMP, your switch
configuration will differ.
For example on each of 4 ASA 5585-Xs, you want to use:
• 2 Ten Gigabit Ethernet interfaces in a device-local EtherChannel for the cluster control link.
• 2 Ten Gigabit Ethernet interfaces in a Spanned EtherChannel for the inside and outside network; each
interface is a VLAN subinterface of the EtherChannel. Using subinterfaces lets both inside and outside
interfaces take advantage of the benefits of an EtherChannel.
• 1 Management interface.
You have one switch for both the inside and outside networks.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
42
ASA Cluster
Configure the Cluster Interface Mode on Each Unit
Purpose
Connect Interfaces on each of 4 ASAs To Switch Ports
Cluster control link
TenGigabitEthernet 0/6 and
TenGigabitEthernet 0/7
8 ports total
For each TenGigabitEthernet 0/6
and TenGigabitEthernet 0/7 pair,
configure 4 EtherChannels (1 EC
for each ASA).
These EtherChannels must all be
on the same isolated cluster control
VLAN, for example VLAN 101.
Inside and outside interfaces
TenGigabitEthernet 0/8 and
TenGigabitEthernet 0/9
8 ports total
Configure a single EtherChannel
(across all ASAs).
On the switch, configure these
VLANs and networks now; for
example, a trunk including VLAN
200 for the inside and VLAN 201
for the outside.
Management interface
Management 0/0
4 ports total
Place all interfaces on the same
isolated management VLAN, for
example VLAN 100.
Related Topics
Leave the Cluster, on page 66
Configure Spanned EtherChannels, on page 47
Cluster Interfaces, on page 4
Configure the Cluster Interface Mode on Each Unit
You can only configure one type of interface for clustering: Spanned EtherChannels or Individual interfaces;
you cannot mix interface types in a cluster.
Before You Begin
• You must set the mode separately on each ASA that you want to add to the cluster.
• You can always configure the management-only interface as an Individual interface (recommended),
even in Spanned EtherChannel mode. The management interface can be an Individual interface even in
transparent firewall mode.
• In Spanned EtherChannel mode, if you configure the management interface as an Individual interface,
you cannot enable dynamic routing for the management interface. You must use a static route.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
43
ASA Cluster
Configure Interfaces on the Primary Unit
• In multiple context mode, you must choose one interface type for all contexts. For example, if you have
a mix of transparent and routed mode contexts, you must use Spanned EtherChannel mode for all contexts
because that is the only interface type allowed for transparent mode.
Procedure
Step 1
Show any incompatible configuration so that you can force the interface mode and fix your configuration
later; the mode is not changed with this command:
cluster interface-mode {individual | spanned} check-details
Example:
ciscoasa(config)# cluster interface-mode spanned check-details
Step 2
Set the interface mode for clustering:
cluster interface-mode {individual | spanned} force
Example:
ciscoasa(config)# cluster interface-mode spanned force
There is no default setting; you must explicitly choose the mode. If you have not set the mode, you cannot
enable clustering.
The force option changes the mode without checking your configuration for incompatible settings. You need
to manually fix any configuration issues after you change the mode. Because any interface configuration can
only be fixed after you set the mode, we recommend using the force option so that you can at least start from
the existing configuration. You can re-run the check-details option after you set the mode for more guidance.
Without the force option, if there is any incompatible configuration, you are prompted to clear your
configuration and reload, thus requiring you to connect to the console port to reconfigure your management
access. If your configuration is compatible (rare), the mode is changed and the configuration is preserved. If
you do not want to clear your configuration, you can exit the command by typing n.
To remove the interface mode, enter the no cluster interface-mode command.
Configure Interfaces on the Primary Unit
You must modify any interface that is currently configured with an IP address to be cluster-ready before you
enable clustering. For other interfaces, you can configure them before or after you enable clustering; we
recommend pre-configuring all of your interfaces so that the complete configuration is synced to new cluster
members.
This section describes how to configure interfaces to be compatible with clustering. You can configure data
interfaces as either Spanned EtherChannels or as Individual interfaces. Each method uses a different
load-balancing mechanism. You cannot configure both types in the same configuration, with the exception
of the management interface, which can be an Individual interface even in Spanned EtherChannel mode.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
44
ASA Cluster
Configure Interfaces on the Primary Unit
Configure Individual Interfaces (Recommended for the Management Interface)
Individual interfaces are normal routed interfaces, each with their own IP address taken from a pool of IP
addresses. The Main cluster IP address is a fixed address for the cluster that always belongs to the current
primary unit.
In Spanned EtherChannel mode, we recommend configuring the management interface as an Individual
interface. Individual management interfaces let you connect directly to each unit if necessary, while a Spanned
EtherChannel interface only allows connection to the current primary unit.
Before You Begin
• Except for the management-only interface, you must be in Individual interface mode.
• For multiple context mode, perform this procedure in each context. If you are not already in the context
configuration mode, enter the changeto context name command.
• Individual interfaces require you to configure load balancing on neighbor devices. External load balancing
is not required for the management interface.
• (Optional) Configure the interface as a device-local EtherChannel interface, a redundant interface, and/or
configure subinterfaces.
◦For an EtherChannel, this EtherChannel is local to the unit, and is not a Spanned EtherChannel.
◦Management-only interfaces cannot be redundant interfaces.
Procedure
Step 1
Configure a pool of Local IP addresses (IPv4 and/or IPv6), one of which will be assigned to each cluster unit
for the interface:
(IPv4)
ip local pool poolname first-address — last-address [mask mask]
(IPv6)
ipv6 local pool poolname ipv6-address/prefix-length number_of_addresses
Example:
ciscoasa(config)# ip local pool ins 192.168.1.2-192.168.1.9
ciscoasa(config-if)# ipv6 local pool insipv6 2001:DB8::1002/32 8
Include at least as many addresses as there are units in the cluster. If you plan to expand the cluster, include
additional addresses. The Main cluster IP address that belongs to the current primary unit is not a part of this
pool; be sure to reserve an IP address on the same network for the Main cluster IP address.
You cannot determine the exact Local address assigned to each unit in advance; to see the address used on
each unit, enter the show ip[v6] local pool poolname command. Each cluster member is assigned a member
ID when it joins the cluster. The ID determines the Local IP used from the pool.
Step 2
Enter interface configuration mode:
interface interface_id
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
45
ASA Cluster
Configure Interfaces on the Primary Unit
Example:
ciscoasa(config)# interface tengigabitethernet 0/8
Step 3
(Management interface only) Set an interface to management-only mode so that it does not pass through
traffic:
management-only
By default, Management type interfaces are configured as management-only. In transparent mode, this
command is always enabled for a Management type interface.
This setting is required if the cluster interface mode is Spanned.
Step 4
Name the interface:
nameif name
Example:
ciscoasa(config-if)# nameif inside
The name is a text string up to 48 characters, and is not case-sensitive. You can change the name by reentering
this command with a new value.
Step 5
Set the Main cluster IP address and identify the cluster pool:
(IPv4)
ip address ip_address [mask] cluster-pool poolname
(IPv6)
ipv6 address ipv6-address/prefix-length cluster-pool poolname
Example:
ciscoasa(config-if)# ip address 192.168.1.1 255.255.255.0 cluster-pool ins
ciscoasa(config-if)# ipv6 address 2001:DB8::1002/32 cluster-pool insipv6
This IP address must be on the same network as the cluster pool addresses, but not be part of the pool. You
can configure an IPv4 and/or an IPv6 address.
DHCP, PPPoE, and IPv6 autoconfiguration are not supported; you must manually configure the IP addresses.
Step 6
Set the security level, where number is an integer between 0 (lowest) and 100 (highest):
security-level number
Example:
ciscoasa(config-if)# security-level 100
Step 7
Enable the interface:
no shutdown
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
46
ASA Cluster
Configure Interfaces on the Primary Unit
Examples
The following example configures the Management 0/0 and Management 0/1 interfaces as a device-local
EtherChannel, and then configures the EtherChannel as an Individual interface:
ip local pool mgmt 10.1.1.2-10.1.1.9
ipv6 local pool mgmtipv6 2001:DB8:45:1002/64 8
interface management 0/0
channel-group 1 mode active
no shutdown
interface management 0/1
channel-group 1 mode active
no shutdown
interface port-channel 1
nameif management
ip address 10.1.1.1 255.255.255.0 cluster-pool mgmt
ipv6 address 2001:DB8:45:1001/64 cluster-pool mgmtipv6
security-level 100
management-only
Related Topics
Management Interface, on page 11
Cable the Cluster Units and Configure Upstream and Downstream Equipment, on page 41
Load Balancing Methods, on page 12
Configure an EtherChannel
Enable Jumbo Frame Support
Configure Spanned EtherChannels, on page 47
Configure Spanned EtherChannels
A Spanned EtherChannel spans all ASAs in the cluster, and provides load balancing as part of the EtherChannel
operation.
Before You Begin
• You must be in Spanned EtherChannel interface mode.
• For multiple context mode, start this procedure in the system execution space. If you are not already in
the System configuration mode, enter the changeto system command.
• For transparent mode, configure the bridge group.
• Do not specify the maximum and minimum links in the EtherChannel—We recommend that you do not
specify the maximum and minimum links in the EtherChannel (The lacp max-bundle and port-channel
min-bundle commands) on either the ASA or the switch. If you need to use them, note the following:
◦The maximum links set on the ASA is the total number of active ports for the whole cluster. Be
sure the maximum links value configured on the switch is not larger than the ASA value.
◦The minimum links set on the ASA is the minimum active ports to bring up a port-channel interface
per unit. On the switch, the minimum links is the minimum links across the cluster, so this value
will not match the ASA value.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
47
ASA Cluster
Configure Interfaces on the Primary Unit
• Do not change the load-balancing algorithm from the default (see the port-channel load-balance
command). On the switch, we recommend that you use one of the following algorithms: source-dest-ip
or source-dest-ip-port (see the Cisco Nexus OS and Cisco IOS port-channel load-balance command).
Do not use a vlan keyword in the load-balance algorithm because it can cause unevenly distributed
traffic to the ASAs in a cluster.
• The lacp port-priority and lacp system-priority commands are not used for a Spanned EtherChannel.
• When using Spanned EtherChannels, the port-channel interface will not come up until clustering is fully
enabled. This requirement prevents traffic from being forwarded to a unit that is not an active unit in
the cluster.
Procedure
Step 1
Specify the interface you want to add to the channel group:
interface physical_interface
Example:
ciscoasa(config)# interface gigabitethernet 0/0
The physical_interface ID includes the type, slot, and port number as type slot/port. This first interface in the
channel group determines the type and speed for all other interfaces in the group.
Step 2
Assign this interface to an EtherChannel:
channel-group channel_id mode active [vss-id {1 | 2}]
Example:
ciscoasa(config-if)# channel-group 1 mode active
The channel_id is between 1 and 48. If the port-channel interface for this channel ID does not yet exist in the
configuration, one will be added automatically:
interface port-channel channel_id
Only active mode is supported for Spanned EtherChannels.
If you are connecting the ASA to two switches in a VSS or vPC, then configure the vss-id keyword to identify
to which switch this interface is connected (1 or 2). You must also use the port-channel span-cluster
vss-load-balance command for the port-channel interface in Step 6.
Step 3
Enable the interface:
no shutdown
Step 4
(Optional) Add additional interfaces to the EtherChannel by repeating the process.
Example:
ciscoasa(config)# interface gigabitethernet 0/1
ciscoasa(config-if)# channel-group 1 mode active
ciscoasa(config-if)# no shutdown
Multiple interfaces in the EtherChannel per unit are useful for connecting to switches in a VSS or vPC. Keep
in mind that by default, a spanned EtherChannel can have only 8 active interfaces out of 16 maximum across
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
48
ASA Cluster
Configure Interfaces on the Primary Unit
all members in the cluster; the remaining 8 interfaces are on standby in case of link failure. To use more than
8 active interfaces (but no standby interfaces), disable dynamic port priority using the clacp static-port-priority
command. When you disable dynamic port priority, you can use up to 32 active links across the cluster. For
example, for a cluster of 16 ASAs, you can use a maximum of 2 interfaces on each ASA, for a total of 32
interfaces in the spanned EtherChannel.
Step 5
Specify the port-channel interface:
interface port-channel channel_id
Example:
ciscoasa(config)# interface port-channel 1
This interface was created automatically when you added an interface to the channel group.
Step 6
Set this EtherChannel as a Spanned EtherChannel:
port-channel span-cluster [vss-load-balance]
Example:
ciscoasa(config-if)# port-channel span-cluster
If you are connecting the ASA to two switches in a VSS or vPC, then you should enable VSS load balancing
by using the vss-load-balance keyword. This feature ensures that the physical link connections between the
ASAs to the VSS (or vPC) pair are balanced. You must configure the vss-id keyword in the channel-group
command for each member interface before enabling load balancing (see Step 2).
Step 7
(Optional) You can set the Ethernet properties for the port-channel interface to override the properties set on
the Individual interfaces.
This method provides a shortcut to set these parameters because these parameters must match for all interfaces
in the channel group.
Step 8
(Optional) If you are creating VLAN subinterfaces on this EtherChannel, do so now.
Example:
ciscoasa(config)# interface port-channel 1.10
ciscoasa(config-if)# vlan 10
The rest of this procedure applies to the subinterfaces.
Step 9
(Multiple Context Mode) Allocate the interface to a context. Then enter:
changeto context name
interface port-channel channel_id
Example:
ciscoasa(config)# context admin
ciscoasa(config)# allocate-interface port-channel1
ciscoasa(config)# changeto context admin
ciscoasa(config-if)# interface port-channel 1
For multiple context mode, the rest of the interface configuration occurs within each context.
Step 10 Name the interface:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
49
ASA Cluster
Configure Interfaces on the Primary Unit
nameif name
Example:
ciscoasa(config-if)# nameif inside
The name is a text string up to 48 characters, and is not case-sensitive. You can change the name by reentering
this command with a new value.
Step 11 Perform one of the following, depending on the firewall mode.
• Routed Mode—Set the IPv4 and/or IPv6 address:
(IPv4)
ip address ip_address [mask]
(IPv6)
ipv6 address ipv6-prefix/prefix-length
Example:
ciscoasa(config-if)# ip address 10.1.1.1 255.255.255.0
ciscoasa(config-if)# ipv6 address 2001:DB8::1001/32
DHCP, PPPoE, and IPv6 autoconfig are not supported.
• Transparent Mode—Assign the interface to a bridge group:
bridge-group number
Example:
ciscoasa(config-if)# bridge-group 1
Where number is an integer between 1 and 100. You can assign up to 4 interfaces to a bridge group.
You cannot assign the same interface to more than one bridge group. Note that the BVI configuration
includes the IP address.
Step 12 Set the security level:
security-level number
Example:
ciscoasa(config-if)# security-level 50
Where number is an integer between 0 (lowest) and 100 (highest).
Step 13 Configure a global MAC address for a Spanned EtherChannel to avoid potential network connectivity problems:
mac-address mac_address
Example:
ciscoasa(config-if)# mac-address 000C.F142.4CDE
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
50
ASA Cluster
Configure the Primary Unit Bootstrap Settings
With a manually-configured MAC address, the MAC address stays with the current primary unit. In multiple
context mode, if you share an interface between contexts, you should enable auto-generation of MAC addresses
so that you only need to set the MAC address manually for a shared interface if you disable auto-generation.
Note that you must manually configure the MAC address for non-shared interfaces.
The mac_address is in H.H.H format, where H is a 16-bit hexadecimal digit. For example, the MAC address
00-0C-F1-42-4C-DE is entered as 000C.F142.4CDE.
The first two bytes of a manual MAC address cannot be A2 if you also want to use auto-generated MAC
addresses.
Step 14 (Routed mode) For inter-site clustering, configure a site-specific MAC address for each site:
mac-address mac_address site-id number
Example:
ciscoasa(config-if)#
ciscoasa(config-if)#
ciscoasa(config-if)#
ciscoasa(config-if)#
ciscoasa(config-if)#
mac-address
mac-address
mac-address
mac-address
mac-address
aaaa.1111.1234
aaaa.1111.aaaa
aaaa.1111.bbbb
aaaa.1111.cccc
aaaa.1111.dddd
site-id
site-id
site-id
site-id
1
2
3
4
The site-specific MAC address used by a unit depends on the site ID you specify in each unit’s bootstrap
configuration.
Related Topics
Configure the Cluster Interface Mode on Each Unit, on page 43
Configure the Bridge Virtual Interface (BVI)
Configure an EtherChannel
Guidelines for EtherChannels and Redundant Interfaces
Connecting to a VSS or vPC, on page 13
Enable the Physical Interface and Configure Ethernet Parameters
Configure a Security Context
Configure the Primary Unit Bootstrap Settings
Each unit in the cluster requires a bootstrap configuration to join the cluster. Typically, the first unit you
configure to join the cluster will be the primary unit. After you enable clustering, after an election period, the
cluster elects a primary unit. With only one unit in the cluster initially, that unit will become the primary unit.
Subsequent units that you add to the cluster will be secondary units.
Before You Begin
• Back up your configurations in case you later want to leave the cluster, and need to restore your
configuration.
• For multiple context mode, complete these procedures in the system execution space. To change from
the context to the system execution space, enter the changeto system command.
• We recommend enabling jumbo frame reservation for use with the cluster control link.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
51
ASA Cluster
Configure the Primary Unit Bootstrap Settings
• You must use the console port to enable or disable clustering. You cannot use Telnet or SSH.
• With the exception of the cluster control link, any interfaces in your configuration must be configured
with a cluster IP pool or as a Spanned EtherChannel before you enable clustering, depending on your
interface mode. If you have pre-existing interface configuration, you can either clear the interface
configuration (clear configure interface), or convert your interfaces to cluster interfaces before you
enable clustering.
• When you add a unit to a running cluster, you may see temporary, limited packet/connection drops; this
is expected behavior.
• Pre-determine the size of the cluster control link. See Size the Cluster Control Link, on page 6.
Procedure
Step 1
Enable the cluster control link interface before you join the cluster.
You will later identify this interface as the cluster control link when you enable clustering.
We recommend that you combine multiple cluster control link interfaces into an EtherChannel if you have
enough interfaces. The EtherChannel is local to the ASA, and is not a Spanned EtherChannel.
The cluster control link interface configuration is not replicated from the primary unit to secondary units;
however, you must use the same configuration on each unit. Because this configuration is not replicated, you
must configure the cluster control link interfaces separately on each unit.
• You cannot use a VLAN subinterface as the cluster control link.
• You cannot use a Management x/x interface as the cluster control link, either alone or as an EtherChannel.
• For the ASA 5585-X with an ASA IPS module, you cannot use the module interfaces for the cluster
control link.
a) Enter interface configuration mode:
interface interface_id
Example:
ciscoasa(config)# interface tengigabitethernet 0/6
b) (Optional, for an EtherChannel) Assign this physical interface to an EtherChannel:
channel-group channel_id mode on
Example:
ciscoasa(config-if)# channel-group 1 mode on
The channel_id is between 1 and 48. If the port-channel interface for this channel ID does not yet exist in
the configuration, one will be added automatically:
interface port-channel channel_id
We recommend using the On mode for cluster control link member interfaces to reduce unnecessary traffic
on the cluster control link. The cluster control link does not need the overhead of LACP traffic because it
is an isolated, stable network. Note: We recommend setting data EtherChannels to Active mode.
c) Enable the interface:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
52
ASA Cluster
Configure the Primary Unit Bootstrap Settings
no shutdown
You only need to enable the interface; do not configure a name for the interface, or any other parameters.
d) (For an EtherChannel) Repeat for each additional interface you want to add to the EtherChannel:
Example:
ciscoasa(config)# interface tengigabitethernet 0/7
ciscoasa(config-if)# channel-group 1 mode on
ciscoasa(config-if)# no shutdown
Step 2
(Optional) Specify the maximum transmission unit for the cluster control link interface:
mtu cluster bytes
Example:
ciscoasa(config)# mtu cluster 9000
Set the MTU between 1400 and 9198 bytes. The default MTU is 1500 bytes.
We suggest setting the MTU to 1600 bytes or greater, which requires you to enable jumbo frame reservation
before continuing with this procedure. Jumbo frame reservation requires a reload of the ASA.
This command is a global configuration command, but is also part of the bootstrap configuration that is not
replicated between units.
Step 3
Name the cluster and enter cluster configuration mode:
cluster group name
Example:
ciscoasa(config)# cluster group pod1
The name must be an ASCII string from 1 to 38 characters. You can only configure one cluster group per
unit. All members of the cluster must use the same name.
Step 4
Name this member of the cluster:
local-unit unit_name
Use a unique ASCII string from 1 to 38 characters. Each unit must have a unique name. A unit with a duplicated
name will be not be allowed in the cluster.
Example:
ciscoasa(cfg-cluster)# local-unit unit1
Step 5
Specify the cluster control link interface, preferably an EtherChannel:
cluster-interface interface_id ip ip_address mask
Example:
ciscoasa(cfg-cluster)# cluster-interface port-channel2 ip 192.168.1.1 255.255.255.0
INFO: Non-cluster interface config is cleared on Port-Channel2
Subinterfaces and Management interfaces are not allowed.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
53
ASA Cluster
Configure the Primary Unit Bootstrap Settings
Specify an IPv4 address for the IP address; IPv6 is not supported for this interface. This interface cannot have
a nameif configured.
For each unit, specify a different IP address on the same network.
Step 6
If you use inter-site clustering, set the site ID for this unit so it uses a site-specific MAC address:
site-id number
Example:
ciscoasa(cfg-cluster)# site-id 1
The number is between 1 and 8.
Step 7
Set the priority of this unit for primary unit elections:
priority priority_number
Example:
ciscoasa(cfg-cluster)# priority 1
The priority is between 1 and 100, where 1 is the highest priority.
Step 8
(Optional) Set an authentication key for control traffic on the cluster control link:
key shared_secret
Example:
ciscoasa(cfg-cluster)# key chuntheunavoidable
The shared secret is an ASCII string from 1 to 63 characters. The shared secret is used to generate the key.
This command does not affect datapath traffic, including connection state update and forwarded packets,
which are always sent in the clear.
Step 9
(Optional) Disable dynamic port priority in LACP:
clacp static-port-priority
Some switches do not support dynamic port priority, so this command improves switch compatibility. Moreover,
it enables support of more than 8 active spanned EtherChannel members, up to 32 members. Without this
command, only 8 active members and 8 standby members are supported. If you enable this command, then
you cannot use any standby members; all members are active.
Step 10 (Optional) Manually specify the cLACP system ID and system priority:
clacp system-mac {mac_address | auto} [system-priority number]
Example:
ciscoasa(cfg-cluster)# clacp system-mac 000a.0000.aaaa
When using Spanned EtherChannels, the ASA uses cLACP to negotiate the EtherChannel with the neighbor
switch. ASAs in a cluster collaborate in cLACP negotiation so that they appear as a single (virtual) device to
the switch. One parameter in cLACP negotiation is a system ID, which is in the format of a MAC address.
All ASAs in the cluster use the same system ID: auto-generated by the primary unit (the default) and replicated
to all secondaries; or manually specified in this command in the form H.H.H, where H is a 16-bit hexadecimal
digit. (For example, the MAC address 00-0A-00-00-AA-AA is entered as 000A.0000.AAAA.) You might
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
54
ASA Cluster
Configure the Primary Unit Bootstrap Settings
want to manually configure the MAC address for troubleshooting purposes, for example, so that you can use
an easily identified MAC address. Typically, you would use the auto-generated MAC address.
The system priority, between 1 and 65535, is used to decide which unit is in charge of making a bundling
decision. By default, the ASA uses priority 1, which is the highest priority. The priority needs to be higher
than the priority on the switch.
This command is not part of the bootstrap configuration, and is replicated from the primary unit to the secondary
units. However, you cannot change this value after you enable clustering.
Step 11 Enable clustering:
enable [noconfirm]
Example:
ciscoasa(cfg-cluster)# enable
INFO: Clustering is not compatible with following commands:
policy-map global_policy
class inspection_default
inspect skinny
policy-map global_policy
class inspection_default
inspect sip
Would you like to remove these commands? [Y]es/[N]o:Y
INFO: Removing incompatible commands from running configuration...
Cryptochecksum (changed): f16b7fc2 a742727e e40bc0b0 cd169999
INFO: Done
When you enter the enable command, the ASA scans the running configuration for incompatible commands
for features that are not supported with clustering, including commands that may be present in the default
configuration. You are prompted to delete the incompatible commands. If you respond No, then clustering is
not enabled. Use the noconfirm keyword to bypass the confirmation and delete incompatible commands
automatically.
For the first unit enabled, a primary unit election occurs. Because the first unit should be the only member of
the cluster so far, it will become the primary unit. Do not perform any configuration changes during this
period.
To disable clustering, enter the no enable command.
Note
If you disable clustering, all data interfaces are shut down, and only the management-only interface
is active.
Examples
The following example configures a management interface, configures a device-local EtherChannel for the
cluster control link, and then enables clustering for the ASA called “unit1,” which will become the primary
unit because it is added to the cluster first:
ip local pool mgmt 10.1.1.2-10.1.1.9
ipv6 local pool mgmtipv6 2001:DB8::1002/32 8
interface management 0/0
nameif management
ip address 10.1.1.1 255.255.255.0 cluster-pool mgmt
ipv6 address 2001:DB8::1001/32 cluster-pool mgmtipv6
security-level 100
management-only
no shutdown
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
55
ASA Cluster
Configure Secondary Unit Bootstrap Settings
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
cluster group pod1
local-unit unit1
cluster-interface port-channel1 ip 192.168.1.1 255.255.255.0
priority 1
key chuntheunavoidable
enable noconfirm
Configure Secondary Unit Bootstrap Settings
Perform the following procedure to configure the secondary units.
Before You Begin
• You must use the console port to enable or disable clustering. You cannot use Telnet or SSH.
• Back up your configurations in case you later want to leave the cluster, and need to restore your
configuration.
• For multiple context mode, complete this procedure in the system execution space. To change from the
context to the system execution space, enter the changeto system command.
• We recommend enabling jumbo frame reservation for use with the cluster control link.
• If you have any interfaces in your configuration that have not been configured for clustering (for example,
the default configuration Management 0/0 interface), you can join the cluster as a secondary unit (with
no possibility of becoming the primary in a current election).
• When you add a unit to a running cluster, you may see temporary, limited packet/connection drops; this
is expected behavior.
Procedure
Step 1
Configure the same cluster control link interface as you configured for the primary unit.
Example:
ciscoasa(config)# interface tengigabitethernet 0/6
ciscoasa(config-if)# channel-group 1 mode on
ciscoasa(config-if)# no shutdown
ciscoasa(config)# interface tengigabitethernet 0/7
ciscoasa(config-if)# channel-group 1 mode on
ciscoasa(config-if)# no shutdown
Step 2
Specify the same MTU that you configured for the primary unit:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
56
ASA Cluster
Configure Secondary Unit Bootstrap Settings
Example:
ciscoasa(config)# mtu cluster 9000
Step 3
Identify the same cluster name that you configured for the primary unit:
Example:
ciscoasa(config)# cluster group pod1
Step 4
Name this member of the cluster with a unique string:
local-unit unit_name
Example:
ciscoasa(cfg-cluster)# local-unit unit2
Specify an ASCII string from 1 to 38 characters.
Each unit must have a unique name. A unit with a duplicated name will be not be allowed in the cluster.
Step 5
Specify the same cluster control link interface that you configured for the primary unit, but specify a different
IP address on the same network for each unit:
cluster-interface interface_id ip ip_address mask
Example:
ciscoasa(cfg-cluster)# cluster-interface port-channel2 ip 192.168.1.2 255.255.255.0
INFO: Non-cluster interface config is cleared on Port-Channel2
Specify an IPv4 address for the IP address; IPv6 is not supported for this interface. This interface cannot have
a nameif configured.
Each unit must have a unique name. A unit with a duplicated name will not be allowed in the cluster.
Step 6
If you use inter-site clustering, set the site ID for this unit so it uses a site-specific MAC address:
site-id number
Example:
ciscoasa(cfg-cluster)# site-id 1
The number is between 1 and 8.
Step 7
Set the priority of this unit for primary unit elections, typically to a higher value than the primary unit:
priority priority_number
Example:
ciscoasa(cfg-cluster)# priority 2
Set the priority between 1 and 100, where 1 is the highest priority.
Step 8
Set the same authentication key that you set for the primary unit:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
57
ASA Cluster
Customize the Clustering Operation
Example:
ciscoasa(cfg-cluster)# key chuntheunavoidable
Step 9
Enable clustering:
enable as-slave
You can avoid any configuration incompatibilities (primarily the existence of any interfaces not yet configured
for clustering) by using the enable as-slave command. This command ensures the secondary joins the cluster
with no possibility of becoming the primary in any current election. Its configuration is overwritten with the
one synced from the primary unit.
To disable clustering, enter the no enable command.
Note
If you disable clustering, all data interfaces are shut down, and only the management interface is
active.
Examples
The following example includes the configuration for a secondary unit, unit2:
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
cluster group pod1
local-unit unit2
cluster-interface port-channel1 ip 192.168.1.2 255.255.255.0
priority 2
key chuntheunavoidable
enable as-slave
Customize the Clustering Operation
You can customize clustering health monitoring, TCP connection replication delay, flow mobility and other
optimizations.
Perform these procedures on the primary unit.
Configure Basic ASA Cluster Parameters
You can customize cluster settings on the primary unit.
Before You Begin
• For multiple context mode, complete this procedure in the system execution space on the primary unit.
To change from the context to the system execution space, enter the changeto system command.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
58
ASA Cluster
Customize the Clustering Operation
Procedure
Step 1
Enter cluster configuration mode:
cluster group name
Step 2
(Optional) Enable console replication from secondary units to the primary unit:
console-replicate
This feature is disabled by default. The ASA prints out some messages directly to the console for certain
critical events. If you enable console replication, secondary units send the console messages to the primary
unit so that you only need to monitor one console port for the cluster.
Step 3
Set the minimum trace level for clustering events:
trace-level level
Set the minimum level as desired:
• critical—Critical events (severity=1)
• warning—Warnings (severity=2)
• informational—Informational events (severity=3)
• debug—Debugging events (severity=4)
Configure Health Monitoring and Auto-Rejoin Settings
This procedure configures unit and interface health monitoring.
You might want to disable health monitoring of non-essential interfaces, for example, the management
interface. You can monitor any port-channel ID, redundant ID, or single physical interface ID, or the software
or hardware module, such as the ASA Firepower module. Health monitoring is not performed on VLAN
subinterfaces or virtual interfaces such as VNIs or BVIs. You cannot configure monitoring for the cluster
control link; it is always monitored.
Procedure
Step 1
Enter cluster configuration mode:
cluster group name
Step 2
Customize the cluster unit health check feature:
health-check [holdtime timeout] [vss-enabled]
Example:
ciscoasa(cfg-cluster)# health-check holdtime 5
The holdime determines the amount of time between unit keepalive status messages, between .8 and 45
seconds; The default is 3 seconds.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
59
ASA Cluster
Customize the Clustering Operation
To determine unit health, the ASA cluster units send keepalive messages on the cluster control link to other
units. If a unit does not receive any keepalive messages from a peer unit within the holdtime period, the peer
unit is considered unresponsive or dead. If you configure the cluster control link as an EtherChannel
(recommended), and it is connected to a VSS or vPC pair, then you might need to enable the vss-enabled
option. For some switches, when one unit in the VSS/vPC is shutting down or booting up, EtherChannel
member interfaces connected to that switch may appear to be Up to the ASA, but they are not passing traffic
on the switch side. The ASA can be erroneously removed from the cluster if you set the ASA holdtime timeout
to a low value (such as .8 seconds), and the ASA sends keepalive messages on one of these EtherChannel
interfaces. When you enable vss-enabled, the ASA floods the keepalive messages on all EtherChannel
interfaces in the cluster control link to ensure that at least one of the switches can receive them.
When any topology changes occur (such as adding or removing a data interface, enabling or disabling an
interface on the ASA or the switch, or adding an additional switch to form a VSS or vPC) you should disable
the health check feature and also disable interface monitoring for the disabled interfaces (no health-check
monitor-interface). When the topology change is complete, and the configuration change is synced to all
units, you can re-enable the health check feature.
Step 3
Disable the interface health check on an interface:
no health-check monitor-interface [interface_id | service-module]
Example:
ciscoasa(cfg-cluster)# no health-check monitor-interface management0/0
The interface health check monitors for link failures. If all physical ports for a given logical interface fail on
a particular unit, but there are active ports under the same logical interface on other units, then the unit is
removed from the cluster. The amount of time before the ASA removes a member from the cluster depends
on the type of interface and whether the unit is an established member or is joining the cluster.
Health check is enabled by default for all interfaces. You can disable it per interface using the no form of this
command.
You might want to disable health monitoring of non-essential interfaces, for example, the management
interface. You can specify any port-channel ID, redundant ID, or single physical interface ID, or the
service-module keyword for a hardware or software module, such as the ASA FirePOWER module. Note
that for the ASA 5585-X, if you disable monitoring of the service module, you may also want to disable
monitoring of the interfaces on the module, which are monitored separately.. Health monitoring is not performed
on VLAN subinterfaces or virtual interfaces such as VNIs or BVIs. You cannot configure monitoring for the
cluster control link; it is always monitored.
When any topology changes occur (such as adding or removing a data interface, enabling or disabling an
interface on the ASA or the switch, or adding an additional switch to form a VSS or vPC) you should disable
the health check feature (no health-check) and also disable interface monitoring for the disabled interfaces.
When the topology change is complete, and the configuration change is synced to all units, you can re-enable
the health check feature.
Step 4
Customize the auto-rejoin cluster settings after a health check failure:
health-check {data-interface | cluster-interface} auto-rejoin {unlimited | auto_rejoin_max}
[auto_rejoin_interval [auto_rejoin_interval_variation]]
Example:
ciscoasa(cfg-cluster)# health-check data-interface auto-rejoin 10 3 3
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
60
ASA Cluster
Customize the Clustering Operation
Define the number of attempts at rejoining the cluster by setting unlimited or the auto-rejoin-max between
0 and 65535. 0 disables auto-rejoining. The default value is unlimited for the cluster-interface and 3 for the
data-interface.
Define the interval duration in minutes between rejoin attempts by setting the auto_rejoin_interval between
2 and 60. The default value is 5 minutes. The maximum total time that the unit attempts to rejoin the cluster
is limited to 14400 minutes (10 days) from the time of last failure.
Define if the interval duration increases by setting the auto_rejoin_interval_variation between 1 and 3: 1 (no
change); 2 (2 x the previous duration), or 3 (3 x the previous duration). For example, if you set the interval
duration to 5 minutes, and set the variation to 2, then the first attempt is after 5 minutes; the 2nd attempt is
10 minutes (2 x 5); the 3rd attempt 20 minutes (2 x 10), and so on. The default value is 1 for the cluster-interface
and 2 for the data-interface.
Configure Connection Rebalancing and the Cluster TCP Replication Delay
You can configure connection rebalancing. For more information, see Rebalancing New TCP Connections
Across the Cluster, on page 27
Enable the cluster replication delay for TCP connections to help eliminate the “unnecessary work” related to
short-lived flows by delaying the director/backup flow creation. Note that if a unit fails before the
director/backup flow is created, then those flows cannot be recovered. Similarly, if traffic is rebalanced to a
different unit before the flow is created, then the flow cannot be recovered. You should not enable the TCP
replication delay for traffic on which you disable TCP randomization.
Procedure
Step 1
Enable the cluster replication delay for TCP connections:
cluster replication delay seconds {http | match tcp {host ip_address | ip_address mask | any | any4 | any6}
[{eq | lt | gt} port] {host ip_address | ip_address mask | any | any4 | any6} [{eq | lt | gt} port]}
Example:
ciscoasa(config)# cluster replication delay 15 match tcp any any eq ftp
ciscoasa(config)# cluster replication delay 15 http
Set the seconds between 1 and 15. The http delay is enabled by default for 5 seconds.
In multiple context mode, configure this setting within the context.
Step 2
Enter cluster configuration mode:
cluster group name
Step 3
(Optional) Enable connection rebalancing for TCP traffic:
conn-rebalance [frequency seconds]
Example:
ciscoasa(cfg-cluster)# conn-rebalance frequency 60
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
61
ASA Cluster
Customize the Clustering Operation
This command is disabled by default. If enabled, ASAs exchange load information periodically, and offload
new connections from more loaded devices to less loaded devices. The frequency, between 1 and 360 seconds,
specifies how often the load information is exchanged. The default is 5 seconds.
Do not configure connection rebalancing for inter-site topologies; you do not want connections rebalanced
to cluster members at a different site.
Configure Cluster Flow Mobility
You can inspect LISP traffic to enable flow mobility when a server moves between sites.
Before You Begin
• Assign each cluster unit to a site ID according to Configure the Primary Unit Bootstrap Settings, on
page 51 and Configure Secondary Unit Bootstrap Settings, on page 56.
• LISP traffic is not included in the default-inspection-traffic class, so you must configure a separate class
for LISP traffic as part of this procedure.
Procedure
Step 1
(Optional) Configure a LISP inspection map to limit inspected EIDs based on IP address, and to configure
the LISP pre-shared key:
a) Create an extended ACL; only the destination IP address is matched to the EID embedded address:
access list eid_acl_name extended permit ip source_address mask destination_address mask
Both IPv4 and IPv6 ACLs are accepted. See the command reference for exact access-list extended syntax.
b) Create the LISP inspection map, and enter parameters mode:
policy-map type inspect lisp inspect_map_name
parameters
c) Define the allowed EIDs by identifying the ACL you created:
allowed-eid access-list eid_acl_name
The first hop router or ITR/ETR might send EID-notify messages for hosts or networks that the ASA
cluster is not involved with, so you can limit the EIDs to only those servers or networks relevant to your
cluster. For example, if the cluster is only involved with 2 sites, but LISP is running on 3 sites, you should
only include EIDs for the 2 sites involved with the cluster.
d) If necessary, enter the pre-shared key:
validate-key key
Example:
ciscoasa(config)# access-list TRACKED_EID_LISP extended permit ip any 10.10.10.0 255.255.255.0
ciscoasa(config)# policy-map type inspect lisp LISP_EID_INSPECT
ciscoasa(config-pmap)# parameters
ciscoasa(config-pmap-p)# allowed-eid access-list TRACKED_EID_LISP
ciscoasa(config-pmap-p)# validate-key MadMaxShinyandChrome
Step 2
Configure LISP inspection for UDP traffic between the first hop router and the ITR or ETR on port 4342:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
62
ASA Cluster
Customize the Clustering Operation
a) Configure the extended ACL to identify LISP traffic:
access list inspect_acl_name extended permit udp source_address mask destination_address mask eq
4342
You must specify UDP port 4342. Both IPv4 and IPv6 ACLs are accepted. See the command reference
for exact access-list extended syntax.
b) Create a class map for the ACL:
class-map inspect_class_name
match access-list inspect_acl_name
c) Specify the policy map, the class map, enable inspection using the optional LISP inspection map, and
apply the service policy to an interface (if new):
policy-map policy_map_name
class inspect_class_name
inspect lisp [inspect_map_name]
service-policy policy_map_name {global | interface ifc_name}
If you have an existing service policy, specify the existing policy map name. By default, the ASA includes
a global policy called global_policy, so for a global policy, specify that name. You can also create one
service policy per interface if you do not want to apply the policy globally. LISP inspection is applied to
traffic bidirectionally so you do not need to apply the service policy on both the source and destination
interfaces; all traffic that enters or exits the interface to which you apply the policy map is affected if the
traffic matches the class map for both directions.
Example:
ciscoasa(config)# access-list LISP_ACL extended permit udp host 192.168.50.89 host
192.168.10.8 eq 4342
ciscoasa(config)# class-map LISP_CLASS
ciscoasa(config-cmap)# match access-list LISP_ACL
ciscoasa(config-cmap)# policy-map INSIDE_POLICY
ciscoasa(config-pmap)# class LISP_CLASS
ciscoasa(config-pmap-c)# inspect lisp LISP_EID_INSPECT
ciscoasa(config)# service-policy INSIDE_POLICY interface inside
The ASA inspects LISP traffic for the EID-notify message sent between the first hop router and the ITR or
ETR. The ASA maintains an EID table that correlates the EID and the site ID.
Step 3
Enable Flow Mobility for a traffic class:
a) Configure the extended ACL to identify business critical traffic that you want to re-assign to the most
optimal site when servers change sites:
access list flow_acl_name extended permit udp source_address mask destination_address mask eq port
Both IPv4 and IPv6 ACLs are accepted. See the command reference for exact access-list extended syntax.
You should enable flow mobility on business-critical traffic. For example, you can limit flow mobility to
only HTTPS traffic, and/or to traffic to specific servers.
b) Create a class map for the ACL:
class-map flow_map_name
match access-list flow_acl_name
c) Specify the same policy map on which you enabled LISP inspection, the flow class map, and enable flow
mobility:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
63
ASA Cluster
Customize the Clustering Operation
policy-map policy_map_name
class flow_map_name
cluster flow-mobility lisp
Example:
ciscoasa(config)# access-list IMPORTANT-FLOWS extended permit tcp any 10.10.10.0 255.255.255.0
eq https
ciscoasa(config)# class-map IMPORTANT-FLOWS-MAP
ciscoasa(config)# match access-list IMPORTANT-FLOWS
ciscoasa(config-cmap)# policy-map INSIDE_POLICY
ciscoasa(config-pmap)# class IMPORTANT-FLOWS-MAP
ciscoasa(config-pmap-c)# cluster flow-mobility lisp
Step 4
Enter cluster group configuration mode, and enable flow mobility for the cluster:
cluster group name
flow-mobility lisp
This on/off toggle lets you easily enable or disable flow mobility.
Examples
The following example:
• Limits EIDs to those on the 10.10.10.0/24 network
• Inspects LISP traffic (UDP 4342) between a LISP router at 192.168.50.89 (on inside) and an ITR or
ETR router (on another ASA interface) at 192.168.10.8
• Enables flow mobility for all inside traffic going to a server on 10.10.10.0/24 using HTTPS.
• Enables flow mobility for the cluster.
access-list TRACKED_EID_LISP extended permit ip any 10.10.10.0 255.255.255.0
policy-map type inspect lisp LISP_EID_INSPECT
parameters
allowed-eid access-list TRACKED_EID_LISP
validate-key MadMaxShinyandChrome
!
access-list LISP_ACL extended permit udp host 192.168.50.89 host 192.168.10.8 eq 4342
class-map LISP_CLASS
match access-list LISP_ACL
policy-map INSIDE_POLICY
class LISP_CLASS
inspect lisp LISP_EID_INSPECT
service-policy INSIDE_POLICY interface inside
!
access-list IMPORTANT-FLOWS extended permit tcp any 10.10.10.0 255.255.255.0 eq https
class-map IMPORTANT-FLOWS-MAP
match access-list IMPORTANT-FLOWS
policy-map INSIDE_POLICY
class IMPORTANT-FLOWS-MAP
cluster flow-mobility lisp
!
cluster group cluster1
flow-mobility lisp
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
64
ASA Cluster
Manage ASA Cluster Members
Manage ASA Cluster Members
After you deploy the cluster, you can change the configuration and manage cluster members.
Become an Inactive Member
To become an inactive member of the cluster, disable clustering on the unit while leaving the clustering
configuration intact.
Note
When an ASA becomes inactive (either manually or through a health check failure), all data interfaces
are shut down; only the management-only interface can send and receive traffic. To resume traffic flow,
re-enable clustering; or you can remove the unit altogether from the cluster. The management interface
remains up using the IP address the unit received from the cluster IP pool. However if you reload, and the
unit is still inactive in the cluster, the management interface is disabled. You must use the console port
for any further configuration.
Before You Begin
• You must use the console port; you cannot enable or disable clustering from a remote CLI connection.
• For multiple context mode, perform this procedure in the system execution space. If you are not already
in the System configuration mode, enter the changeto system command.
Procedure
Step 1
Enter cluster configuration mode:
cluster group name
Example:
ciscoasa(config)# cluster group pod1
Step 2
Disable clustering:
no enable
If this unit was the primary unit, a new primary election takes place, and a different member becomes the
primary unit.
The cluster configuration is maintained, so that you can enable clustering again later.
Related Topics
Leave the Cluster, on page 66
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
65
ASA Cluster
Inactivate a Member
Inactivate a Member
To inactivate a member from any unit, perform the following steps.
Note
When an ASA becomes inactive, all data interfaces are shut down; only the management-only interface
can send and receive traffic. To resume traffic flow, re-enable clustering; or you can remove the unit
altogether from the cluster. The management interface remains up using the IP address the unit received
from the cluster IP pool. However if you reload, and the unit is still inactive in the cluster, the management
interface is disabled. You must use the console port for any further configuration.
Before You Begin
For multiple context mode, perform this procedure in the system execution space. If you are not already in
the System configuration mode, enter the changeto system command.
Procedure
Remove the unit from the cluster:
cluster remove unit unit_name
Example:
ciscoasa(config)# cluster remove unit ?
Current active units in the cluster:
asa2
ciscoasa(config)# cluster remove unit asa2
WARNING: Clustering will be disabled on unit asa2. To bring it back
to the cluster please logon to that unit and re-enable clustering
The bootstrap configuration remains intact, as well as the last configuration synched from the primary unit,
so that you can later re-add the unit without losing your configuration. If you enter this command on a secondary
unit to remove the primary unit, a new primary unit is elected.
To view member names, enter cluster remove unit ?, or enter the show cluster info command.
Leave the Cluster
If you want to leave the cluster altogether, you need to remove the entire cluster bootstrap configuration.
Because the current configuration on each member is the same (synced from the primary unit), leaving the
cluster also means either restoring a pre-clustering configuration from backup, or clearing your configuration
and starting over to avoid IP address conflicts.
Before You Begin
You must use the console port; when you remove the cluster configuration, all interfaces are shut down,
including the management interface and cluster control link. Moreover, you cannot enable or disable clustering
from a remote CLI connection.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
66
ASA Cluster
Change the Primary Unit
Procedure
Step 1
For a secondary unit, disable clustering:
cluster group cluster_name no enable
Example:
ciscoasa(config)# cluster group cluster1
ciscoasa(cfg-cluster)# no enable
You cannot make configuration changes while clustering is enabled on a secondary unit.
Step 2
Clear the cluster configuration:
clear configure cluster
The ASA shuts down all interfaces including the management interface and cluster control link.
Step 3
Disable cluster interface mode:
no cluster interface-mode
The mode is not stored in the configuration and must be reset manually.
Step 4
If you have a backup configuration, copy the backup configuration to the running configuration:
copy backup_cfg running-config
Example:
ciscoasa(config)# copy backup_cluster.cfg running-config
Source filename [backup_cluster.cfg]?
Destination filename [running-config]?
ciscoasa(config)#
Step 5
Save the configuration to startup:
write memory
Step 6
If you do not have a backup configuration, reconfigure management access. Be sure to change the interface
IP addresses, and restore the correct hostname, for example.
Change the Primary Unit
Caution
The best method to change the primary unit is to disable clustering on the primary unit, waiting for a new
primary election, and then re-enabling clustering. If you must specify the exact unit you want to become
the primary, use the procedure in this section. Note, however, that for centralized features, if you force a
primary unit change using this procedure, then all connections are dropped, and you have to re-establish
the connections on the new primary unit.
To change the primary unit, perform the following steps.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
67
ASA Cluster
Execute a Command Cluster-Wide
Before You Begin
For multiple context mode, perform this procedure in the system execution space. If you are not already in
the System configuration mode, enter the changeto system command.
Procedure
Set a new unit as the primary unit:
cluster master unit unit_name
Example:
ciscoasa(config)# cluster master unit asa2
You will need to reconnect to the Main cluster IP address.
To view member names, enter cluster master unit ? (to see all names except the current unit), or enter the
show cluster info command.
Related Topics
Become an Inactive Member, on page 65
Centralized Features for Clustering, on page 28
Execute a Command Cluster-Wide
To send a command to all members in the cluster, or to a specific member, perform the following steps.
Sending a show command to all members collects all output and displays it on the console of the current unit.
Other commands, such as capture and copy, can also take advantage of cluster-wide execution.
Procedure
Send a command to all members, or if you specify the unit name, a specific member:
cluster exec [unit unit_name] command
Example:
ciscoasa# cluster exec show xlate
To view member names, enter cluster exec unit ? (to see all names except the current unit), or enter the show
cluster info command.
Examples
To copy the same capture file from all units in the cluster at the same time to a TFTP server, enter the following
command on the primary unit:
ciscoasa# cluster exec copy /pcap capture: tftp://10.1.1.56/capture1.pcap
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
68
ASA Cluster
Monitoring the ASA Cluster
Multiple PCAP files, one from each unit, are copied to the TFTP server. The destination capture file name is
automatically attached with the unit name, such as capture1_asa1.pcap, capture1_asa2.pcap, and so on. In
this example, asa1 and asa2 are cluster unit names.
The following sample output for the cluster exec show port-channel summary command shows EtherChannel
information for each member in the cluster:
ciscoasa# cluster exec show port-channel summary
primary(LOCAL):***********************************************************
Number of channel-groups in use: 2
Group Port-channel Protocol Span-cluster Ports
------+-------------+-----------+----------------------------------------------1
Po1
LACP
Yes Gi0/0(P)
2
Po2
LACP
Yes Gi0/1(P)
secondary:******************************************************************
Number of channel-groups in use: 2
Group Port-channel Protocol Span-cluster
Ports
------+-------------+-----------+----------------------------------------------1
Po1
LACP
Yes
Gi0/0(P)
2
Po2
LACP
Yes
Gi0/1(P)
Monitoring the ASA Cluster
You can monitor and troubleshoot cluster status and connections.
Monitoring Cluster Status
See the following commands for monitoring cluster status:
• show cluster info [health ]
With no keywords, the show cluster info command shows the status of all members of the cluster.
The show cluster info health command shows the current health of interfaces, units, and the cluster
overall.
See the following output for the show cluster info command:
ciscoasa# show cluster info
Cluster stbu: On
This is "C" in state SLAVE
ID
: 0
Site ID
: 1
Version
: 9.4(1)
Serial No.: P3000000025
CCL IP
: 10.0.0.3
CCL MAC
: 000b.fcf8.c192
Last join : 17:08:59 UTC Sep 26 2011
Last leave: N/A
Other members in the cluster:
Unit "D" in state SLAVE
ID
: 1
Site ID
: 1
Version
: 9.4(1)
Serial No.: P3000000001
CCL IP
: 10.0.0.4
CCL MAC
: 000b.fcf8.c162
Last join : 19:13:11 UTC Sep 23 2011
Last leave: N/A
Unit "A" in state MASTER
ID
: 2
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
69
ASA Cluster
Capturing Packets Cluster-Wide
Site ID
: 2
Version
: 9.4(1)
Serial No.: JAB0815R0JY
CCL IP
: 10.0.0.1
CCL MAC
: 000f.f775.541e
Last join : 19:13:20 UTC Sep 23 2011
Last leave: N/A
Unit "B" in state SLAVE
ID
: 3
Site ID
: 2
Version
: 9.4(1)
Serial No.: P3000000191
CCL IP
: 10.0.0.2
CCL MAC
: 000b.fcf8.c61e
Last join : 19:13:50 UTC Sep 23 2011
Last leave: 19:13:36 UTC Sep 23 2011
• show cluster history
Shows the cluster history.
Capturing Packets Cluster-Wide
See the following command for capturing packets in a cluster:
cluster exec capture
To support cluster-wide troubleshooting, you can enable capture of cluster-specific traffic on the primary unit
using the cluster exec capture command, which is then automatically enabled on all of the secondary units
in the cluster.
Monitoring Cluster Resources
See the following command for monitoring cluster resources:
show cluster {cpu | memory | resource} [options]
Displays aggregated data for the entire cluster. The options available depends on the data type.
Monitoring Cluster Traffic
See the following commands for monitoring cluster traffic:
• show conn [detail], cluster exec show conn
The show conn command shows whether a flow is a director, backup, or forwarder flow. Use the cluster
exec show conn command on any unit to view all connections. This command can show how traffic for
a single flow arrives at different ASAs in the cluster. The throughput of the cluster is dependent on the
efficiency and configuration of load balancing. This command provides an easy way to view how traffic
for a connection is flowing through the cluster, and can help you understand how a load balancer might
affect the performance of a flow.
The show conn detail command also shows which flows are subject to flow mobility.
The following is sample output for the show conn detail command:
ciscoasa/ASA2/slave# show conn detail
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
70
ASA Cluster
Monitoring Cluster Traffic
12 in use, 13 most used
Cluster stub connections: 0 in use, 46 most used
Flags: A - awaiting inside ACK to SYN, a - awaiting outside ACK to SYN,
B - initial SYN from outside, b - TCP state-bypass or nailed,
C - CTIQBE media, c - cluster centralized,
D - DNS, d - dump, E - outside back connection, e - semi-distributed,
F - outside FIN, f - inside FIN,
G - group, g - MGCP, H - H.323, h - H.225.0, I - inbound data,
i - incomplete, J - GTP, j - GTP data, K - GTP t3-response
k - Skinny media, L - LISP triggered flow owner mobility,
M - SMTP data, m - SIP media, n - GUP
O - outbound data, o - offloaded,
P - inside back connection,
Q - Diameter, q - SQL*Net data,
R - outside acknowledged FIN,
R - UDP SUNRPC, r - inside acknowledged FIN, S - awaiting inside SYN,
s - awaiting outside SYN, T - SIP, t - SIP transient, U - up,
V - VPN orphan, W - WAAS,
w - secondary domain backup,
X - inspected by service module,
x - per session, Y - director stub flow, y - backup stub flow,
Z - Scansafe redirection, z - forwarding stub flow
ESP outside: 10.1.227.1/53744 NP Identity Ifc: 10.1.226.1/30604, , flags c, idle 0s,
uptime
1m21s, timeout 30s, bytes 7544, cluster sent/rcvd bytes 0/0, owners (0,255) Traffic
received
at interface outside Locally received: 7544 (93 byte/s) Traffic received at interface
NP
Identity Ifc Locally received: 0 (0 byte/s) UDP outside: 10.1.227.1/500 NP Identity
Ifc:
10.1.226.1/500, flags -c, idle 1m22s, uptime 1m22s, timeout 2m0s, bytes 1580, cluster
sent/rcvd bytes 0/0, cluster sent/rcvd total bytes 0/0, owners (0,255) Traffic received
at
interface outside Locally received: 864 (10 byte/s) Traffic received at interface NP
Identity
Ifc Locally received: 716 (8 byte/s)
To troubleshoot the connection flow, first see connections on all units by entering the cluster exec show
conn command on any unit. Look for flows that have the following flags: director (Y), backup (y), and
forwarder (z). The following example shows an SSH connection from 172.18.124.187:22 to
192.168.103.131:44727 on all three ASAs; ASA 1 has the z flag showing it is a forwarder for the
connection, ASA3 has the Y flag showing it is the director for the connection, and ASA2 has no special
flags showing it is the owner. In the outbound direction, the packets for this connection enter the inside
interface on ASA2 and exit the outside interface. In the inbound direction, the packets for this connection
enter the outside interface on ASA 1 and ASA3, are forwarded over the cluster control link to ASA2,
and then exit the inside interface on ASA2.
ciscoasa/ASA1/master# cluster exec show conn
ASA1(LOCAL):**********************************************************
18 in use, 22 most used
Cluster stub connections: 0 in use, 5 most used
TCP outside 172.18.124.187:22 inside 192.168.103.131:44727, idle 0:00:00, bytes
37240828, flags z
ASA2:*****************************************************************
12 in use, 13 most used
Cluster stub connections: 0 in use, 46 most used
TCP outside 172.18.124.187:22 inside 192.168.103.131:44727, idle 0:00:00, bytes
37240828, flags UIO
ASA3:*****************************************************************
10 in use, 12 most used
Cluster stub connections: 2 in use, 29 most used
TCP outside 172.18.124.187:22 inside 192.168.103.131:44727, idle 0:00:03, bytes 0,
flags Y
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
71
ASA Cluster
Monitoring Cluster Traffic
• show cluster info [conn-distribution | packet-distribution | loadbalance | flow-mobility counters]
The show cluster info conn-distribution and show cluster info packet-distribution commands show
traffic distribution across all cluster units. These commands can help you to evaluate and adjust the
external load balancer.
The show cluster info loadbalance command shows connection rebalance statistics.
The show cluster info flow-mobility counters command shows EID movement and flow owner
movement information. See the following output for show cluster info flow-mobility counters:
ciscoasa# show cluster info flow-mobility counters
EID movement notification received : 4
EID movement notification processed : 4
Flow owner moving requested
: 2
• show cluster {access-list | conn | traffic | user-identity | xlate} [options]
Displays aggregated data for the entire cluster. The options available depends on the data type.
See the following output for the show cluster access-list command:
ciscoasa# show cluster access-list
hitcnt display order: cluster-wide aggregated result, unit-A, unit-B, unit-C, unit-D
access-list cached ACL log flows: total 0, denied 0 (deny-flow-max 4096) alert-interval
300
access-list 101; 122 elements; name hash: 0xe7d586b5
access-list 101 line 1 extended permit tcp 192.168.143.0 255.255.255.0 any eq www
(hitcnt=0, 0, 0, 0, 0) 0x207a2b7d
access-list 101 line 2 extended permit tcp any 192.168.143.0 255.255.255.0 (hitcnt=0,
0, 0, 0, 0) 0xfe4f4947
access-list 101 line 3 extended permit tcp host 192.168.1.183 host 192.168.43.238
(hitcnt=1, 0, 0, 0, 1) 0x7b521307
access-list 101 line 4 extended permit tcp host 192.168.1.116 host 192.168.43.238
(hitcnt=0, 0, 0, 0, 0) 0x5795c069
access-list 101 line 5 extended permit tcp host 192.168.1.177 host 192.168.43.238
(hitcnt=1, 0, 0, 1, 0) 0x51bde7ee
access list 101 line 6 extended permit tcp host 192.168.1.177 host 192.168.43.13
(hitcnt=0, 0, 0, 0, 0) 0x1e68697c
access-list 101 line 7 extended permit tcp host 192.168.1.177 host 192.168.43.132
(hitcnt=2, 0, 0, 1, 1) 0xc1ce5c49
access-list 101 line 8 extended permit tcp host 192.168.1.177 host 192.168.43.192
(hitcnt=3, 0, 1, 1, 1) 0xb6f59512
access-list 101 line 9 extended permit tcp host 192.168.1.177 host 192.168.43.44
(hitcnt=0, 0, 0, 0, 0) 0xdc104200
access-list 101 line 10 extended permit tcp host 192.168.1.112 host 192.168.43.44
(hitcnt=429, 109, 107, 109, 104)
0xce4f281d
access-list 101 line 11 extended permit tcp host 192.168.1.170 host 192.168.43.238
(hitcnt=3, 1, 0, 0, 2) 0x4143a818
access-list 101 line 12 extended permit tcp host 192.168.1.170 host 192.168.43.169
(hitcnt=2, 0, 1, 0, 1) 0xb18dfea4
access-list 101 line 13 extended permit tcp host 192.168.1.170 host 192.168.43.229
(hitcnt=1, 1, 0, 0, 0) 0x21557d71
access-list 101 line 14 extended permit tcp host 192.168.1.170 host 192.168.43.106
(hitcnt=0, 0, 0, 0, 0) 0x7316e016
access-list 101 line 15 extended permit tcp host 192.168.1.170 host 192.168.43.196
(hitcnt=0, 0, 0, 0, 0) 0x013fd5b8
access-list 101 line 16 extended permit tcp host 192.168.1.170 host 192.168.43.75
(hitcnt=0, 0, 0, 0, 0) 0x2c7dba0d
To display the aggregated count of in-use connections for all units, enter:
ciscoasa# show cluster conn count
Usage Summary In Cluster:*********************************************
200 in use (cluster-wide aggregated)
cl2(LOCAL):***********************************************************
100 in use, 100 most used
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
72
ASA Cluster
Monitoring Cluster Routing
cl1:******************************************************************
100 in use, 100 most used
• show asp cluster counter
This command is useful for datapath troubleshooting.
Monitoring Cluster Routing
See the following commands for cluster routing:
• show route cluster
• debug route cluster
Shows cluster information for routing.
• show lisp eid
Shows the ASA EID table showing EIDs and site IDs.
See the following output from the cluster exec show lisp eid command.
ciscoasa# cluster exec show lisp eid
L1(LOCAL):************************************************************
LISP EID
Site ID
33.44.33.105
2
33.44.33.201
2
11.22.11.1
4
11.22.11.2
4
L2:*******************************************************************
LISP EID
Site ID
33.44.33.105
2
33.44.33.201
2
11.22.11.1
4
11.22.11.2 4
• show asp table classify domain inspect-lisp
This command is useful for troubleshooting.
Configuring Logging for Clustering
See the following command for configuring logging for clustering:
logging device-id
Each unit in the cluster generates syslog messages independently. You can use the logging device-id command
to generate syslog messages with identical or different device IDs to make messages appear to come from the
same or different units in the cluster.
Related Topics
Include the Device ID in Non-EMBLEM Format Syslog Messages
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
73
ASA Cluster
Monitoring Cluster Interfaces
Monitoring Cluster Interfaces
See the following commands for monitoring cluster interfaces:
• show cluster interface-mode
Shows the cluster interface mode.
• show port-channel
Includes information about whether a port-channel is spanned.
• show lacp cluster {system-mac | system-id}
Shows the cLACP system ID and priority.
• debug lacp cluster [all | ccp | misc | protocol]
Shows debug messages for cLACP.
• show interface
Shows the use of the site MAC address when in use:
ciscoasa# show interface port-channel1.3151
Interface Port-channel1.3151 "inside", is up, line protocol is up
Hardware is EtherChannel/LACP, BW 1000 Mbps, DLY 10 usec
VLAN identifier 3151
MAC address aaaa.1111.1234, MTU 1500
Site Specific MAC address aaaa.1111.aaaa
IP address 10.3.1.1, subnet mask 255.255.255.0
Traffic Statistics for "inside":
132269 packets input, 6483425 bytes
1062 packets output, 110448 bytes
98530 packets dropped
Debugging Clustering
See the following commands for debugging clustering:
• debug cluster [ccp | datapath | fsm | general | hc | license | rpc | transport]
Shows debug messages for clustering.
• debug cluster flow-mobility
Shows events related to clustering flow mobility.
• debug lisp eid-notify-intercept
Shows events when the eid-notify message is intercepted.
• show cluster info trace
The show cluster info trace command shows the debug information for further troubleshooting.
See the following output for the show cluster info trace command:
ciscoasa# show cluster info trace
Feb 02 14:19:47.456 [DBUG]Receive CCP message: CCP_MSG_LOAD_BALANCE
Feb 02 14:19:47.456 [DBUG]Receive CCP message: CCP_MSG_LOAD_BALANCE
Feb 02 14:19:47.456 [DBUG]Send CCP message to all: CCP_MSG_KEEPALIVE from 80-1 at
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
74
ASA Cluster
Examples for ASA Clustering
MASTER
Examples for ASA Clustering
These examples include all cluster-related ASA configuration for typical deployments.
Sample ASA and Switch Configuration
The following sample configurations connect the following interfaces between the ASA and the switch:
ASA Interface
Switch Interface
GigabitEthernet 0/2
GigabitEthernet 1/0/15
GigabitEthernet 0/3
GigabitEthernet 1/0/16
GigabitEthernet 0/4
GigabitEthernet 1/0/17
GigabitEthernet 0/5
GigabitEthernet 1/0/18
ASA Configuration
Interface Mode on Each Unit
cluster interface-mode spanned force
ASA1 Primary Bootstrap Configuration
interface GigabitEthernet0/0
channel-group 1 mode on
no shutdown
!
interface GigabitEthernet0/1
channel-group 1 mode on
no shutdown
!
interface Port-channel1
description Clustering Interface
!
cluster group Moya
local-unit A
cluster-interface Port-channel1 ip 10.0.0.1 255.255.255.0
priority 10
key emphyri0
enable noconfirm
ASA2 Secondary Bootstrap Configuration
interface GigabitEthernet0/0
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
75
ASA Cluster
Sample ASA and Switch Configuration
channel-group 1 mode on
no shutdown
!
interface GigabitEthernet0/1
channel-group 1 mode on
no shutdown
!
interface Port-channel1
description Clustering Interface
!
cluster group Moya
local-unit B
cluster-interface Port-channel1 ip 10.0.0.2 255.255.255.0
priority 11
key emphyri0
enable as-slave
Primary Interface Configuration
ip local pool mgmt-pool 10.53.195.231-10.53.195.232
interface GigabitEthernet0/2
channel-group 10 mode active
no shutdown
!
interface GigabitEthernet0/3
channel-group 10 mode active
no shutdown
!
interface GigabitEthernet0/4
channel-group 11 mode active
no shutdown
!
interface GigabitEthernet0/5
channel-group 11 mode active
no shutdown
!
interface Management0/0
management-only
nameif management
ip address 10.53.195.230 cluster-pool mgmt-pool
security-level 100
no shutdown
!
interface Port-channel10
port-channel span-cluster
mac-address aaaa.bbbb.cccc
nameif inside
security-level 100
ip address 209.165.200.225 255.255.255.224
!
interface Port-channel11
port-channel span-cluster
mac-address aaaa.dddd.cccc
nameif outside
security-level 0
ip address 209.165.201.1 255.255.255.224
Cisco IOS Switch Configuration
interface GigabitEthernet1/0/15
switchport access vlan 201
switchport mode access
spanning-tree portfast
channel-group 10 mode active
!
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
76
ASA Cluster
Firewall on a Stick
interface GigabitEthernet1/0/16
switchport access vlan 201
switchport mode access
spanning-tree portfast
channel-group 10 mode active
!
interface GigabitEthernet1/0/17
switchport access vlan 401
switchport mode access
spanning-tree portfast
channel-group 11 mode active
!
interface GigabitEthernet1/0/18
switchport access vlan 401
switchport mode access
spanning-tree portfast
channel-group 11 mode active
interface Port-channel10
switchport access vlan 201
switchport mode access
interface Port-channel11
switchport access vlan 401
switchport mode access
Firewall on a Stick
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
77
ASA Cluster
Firewall on a Stick
Data traffic from different security domains are associated with different VLANs, for example, VLAN 10 for
the inside network and VLAN 20 for the outside network. Each ASA has a single physical port connected to
the external switch or router. Trunking is enabled so that all packets on the physical link are 802.1q
encapsulated. The ASA is the firewall between VLAN 10 and VLAN 20.
When using Spanned EtherChannels, all data links are grouped into one EtherChannel on the switch side. If
an ASA becomes unavailable, the switch will rebalance traffic between the remaining units.
Interface Mode on Each Unit
cluster interface-mode spanned force
ASA1 Primary Bootstrap Configuration
interface tengigabitethernet 0/8
no shutdown
description CCL
cluster group cluster1
local-unit asa1
cluster-interface tengigabitethernet0/8 ip 192.168.1.1 255.255.255.0
priority 1
key chuntheunavoidable
enable noconfirm
ASA2 Secondary Bootstrap Configuration
interface tengigabitethernet 0/8
no shutdown
description CCL
cluster group cluster1
local-unit asa2
cluster-interface tengigabitethernet0/8 ip 192.168.1.2 255.255.255.0
priority 2
key chuntheunavoidable
enable as-slave
ASA3 Secondary Bootstrap Configuration
interface tengigabitethernet 0/8
no shutdown
description CCL
cluster group cluster1
local-unit asa3
cluster-interface tengigabitethernet0/8 ip 192.168.1.3 255.255.255.0
priority 3
key chuntheunavoidable
enable as-slave
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
78
ASA Cluster
Firewall on a Stick
Primary Interface Configuration
ip local pool mgmt 10.1.1.2-10.1.1.9
ipv6 local pool mgmtipv6 2001:DB8::1002/64 8
interface management 0/0
nameif management
ip address 10.1.1.1 255.255.255.0 cluster-pool mgmt
ipv6 address 2001:DB8::1001/32 cluster-pool mgmtipv6
security-level 100
management-only
no shutdown
interface tengigabitethernet 0/9
channel-group 2 mode active
no shutdown
interface port-channel 2
port-channel span-cluster
interface port-channel 2.10
vlan 10
nameif inside
ip address 10.10.10.5 255.255.255.0
ipv6 address 2001:DB8:1::5/64
mac-address 000C.F142.4CDE
interface port-channel 2.20
vlan 20
nameif outside
ip address 209.165.201.1 255.255.255.224
ipv6 address 2001:DB8:2::8/64
mac-address 000C.F142.5CDE
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
79
ASA Cluster
Traffic Segregation
Traffic Segregation
You may prefer physical separation of traffic between the inside and outside network.
As shown in the diagram above, there is one Spanned EtherChannel on the left side that connects to the inside
switch, and the other on the right side to outside switch. You can also create VLAN subinterfaces on each
EtherChannel if desired.
Interface Mode on Each Unit
cluster interface-mode spanned force
ASA1 Primary Bootstrap Configuration
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
interface port-channel 1
description CCL
cluster group cluster1
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
80
ASA Cluster
Traffic Segregation
local-unit asa1
cluster-interface port-channel1 ip 192.168.1.1 255.255.255.0
priority 1
key chuntheunavoidable
enable noconfirm
ASA2 Secondary Bootstrap Configuration
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
interface port-channel 1
description CCL
cluster group cluster1
local-unit asa2
cluster-interface port-channel1 ip 192.168.1.2 255.255.255.0
priority 2
key chuntheunavoidable
enable as-slave
ASA3 Secondary Bootstrap Configuration
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
interface port-channel 1
description CCL
cluster group cluster1
local-unit asa3
cluster-interface port-channel1 ip 192.168.1.3 255.255.255.0
priority 3
key chuntheunavoidable
enable as-slave
Primary Interface Configuration
ip local pool mgmt 10.1.1.2-10.1.1.9
ipv6 local pool mgmtipv6 2001:DB8::1002/64 8
interface management 0/0
nameif management
ip address 10.1.1.1 255.255.255.0 cluster-pool mgmt
ipv6 address 2001:DB8::1001/32 cluster-pool mgmtipv6
security-level 100
management-only
no shutdown
interface tengigabitethernet 0/8
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
81
ASA Cluster
Spanned EtherChannel with Backup Links (Traditional 8 Active/8 Standby)
channel-group 2 mode active
no shutdown
interface port-channel 2
port-channel span-cluster
nameif inside
ip address 10.10.10.5 255.255.255.0
ipv6 address 2001:DB8:1::5/64
mac-address 000C.F142.4CDE
interface tengigabitethernet 0/9
channel-group 3 mode active
no shutdown
interface port-channel 3
port-channel span-cluster
nameif outside
ip address 209.165.201.1 255.255.255.224
ipv6 address 2001:DB8:2::8/64
mac-address 000C.F142.5CDE
Spanned EtherChannel with Backup Links (Traditional 8 Active/8 Standby)
The maximum number of active ports in a traditional EtherChannel is limited to 8 from the switch side. If
you have an 8-ASA cluster, and you allocate 2 ports per unit to the EtherChannel, for a total of 16 ports total,
then 8 of them have to be in standby mode. The ASA uses LACP to negotiate which links should be active
or standby. If you enable multi-switch EtherChannel using VSS or vPC, you can achieve inter-switch
redundancy. On the ASA, all physical ports are ordered first by the slot number then by the port number. In
the following figure, the lower ordered port is the “primary” port (for example, GigabitEthernet 0/0), and the
other one is the “secondary” port (for example, GigabitEthernet 0/1). You must guarantee symmetry in the
hardware connection: all primary links must terminate on one switch, and all secondary links must terminate
on another switch if VSS/vPC is used. The following diagram shows what happens when the total number of
links grows as more units join the cluster:
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
82
ASA Cluster
Spanned EtherChannel with Backup Links (Traditional 8 Active/8 Standby)
The principle is to first maximize the number of active ports in the channel, and secondly keep the number
of active primary ports and the number of active secondary ports in balance. Note that when a 5th unit joins
the cluster, traffic is not balanced evenly between all units.
Link or device failure is handled with the same principle. You may end up with a less-than-perfect load
balancing situation. The following figure shows a 4-unit cluster with a single link failure on one of the units.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
83
ASA Cluster
Spanned EtherChannel with Backup Links (Traditional 8 Active/8 Standby)
There could be multiple EtherChannels configured in the network. The following diagram shows an
EtherChannel on the inside and one on the outside. An ASA is removed from the cluster if both primary and
secondary links in one EtherChannel fail. This prevents the ASA from receiving traffic from the outside
network when it has already lost connectivity to the inside network.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
84
ASA Cluster
Spanned EtherChannel with Backup Links (Traditional 8 Active/8 Standby)
Interface Mode on Each Unit
cluster interface-mode spanned force
ASA1 Primary Bootstrap Configuration
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/8
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/9
channel-group 1 mode on
no shutdown
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
85
ASA Cluster
Spanned EtherChannel with Backup Links (Traditional 8 Active/8 Standby)
interface port-channel 1
description CCL
cluster group cluster1
local-unit asa1
cluster-interface port-channel1 ip 192.168.1.1 255.255.255.0
priority 1
key chuntheunavoidable
enable noconfirm
ASA2 Secondary Bootstrap Configuration
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/8
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/9
channel-group 1 mode on
no shutdown
interface port-channel 1
description CCL
cluster group cluster1
local-unit asa2
cluster-interface port-channel1 ip 192.168.1.2 255.255.255.0
priority 2
key chuntheunavoidable
enable as-slave
ASA3 Secondary Bootstrap Configuration
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/8
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/9
channel-group 1 mode on
no shutdown
interface port-channel 1
description CCL
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
86
ASA Cluster
Spanned EtherChannel with Backup Links (Traditional 8 Active/8 Standby)
cluster group cluster1
local-unit asa3
cluster-interface port-channel1 ip 192.168.1.3 255.255.255.0
priority 3
key chuntheunavoidable
enable as-slave
ASA4 Secondary Bootstrap Configuration
interface tengigabitethernet 0/6
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/7
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/8
channel-group 1 mode on
no shutdown
interface tengigabitethernet 0/9
channel-group 1 mode on
no shutdown
interface port-channel 1
description CCL
cluster group cluster1
local-unit asa4
cluster-interface port-channel1 ip 192.168.1.4 255.255.255.0
priority 4
key chuntheunavoidable
enable as-slave
Primary Interface Configuration
ip local pool mgmt 10.1.1.2-10.1.1.9
interface management 0/0
channel-group 2 mode active
no shutdown
interface management 0/1
channel-group 2 mode active
no shutdown
interface port-channel 2
nameif management
ip address 10.1.1.1 255.255.255.0 cluster-pool mgmt
security-level 100
management-only
interface tengigabitethernet 1/6
channel-group 3 mode active vss-id 1
no shutdown
interface tengigabitethernet 1/7
channel-group 3 mode active vss-id 2
no shutdown
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
87
ASA Cluster
OTV Configuration for Routed Mode Inter-Site Clustering
interface port-channel 3
port-channel span-cluster vss-load-balance
nameif inside
ip address 10.10.10.5 255.255.255.0
mac-address 000C.F142.4CDE
interface tengigabitethernet 1/8
channel-group 4 mode active vss-id 1
no shutdown
interface tengigabitethernet 1/9
channel-group 4 mode active vss-id 2
no shutdown
interface port-channel 4
port-channel span-cluster vss-load-balance
nameif outside
ip address 209.165.201.1 255.255.255.224
mac-address 000C.F142.5CDE
OTV Configuration for Routed Mode Inter-Site Clustering
The success of inter-site clustering for routed mode with Spanned EtherChannels depends on the proper
configuration and monitoring of OTV. OTV plays a major role by forwarding the packets across the DCI.
OTV forwards unicast packets across the DCI only when it learns the MAC address in its forwarding table.
If the MAC address is not learned in the OTV forwarding table, it will drop the unicast packets.
Sample OTV Configuration
//Sample OTV config:
//3151 – Inside VLAN, 3152 – Outside VLAN, 202 – CCL VLAN
//aaaa.1111.1234 – ASA inside interface global vMAC
//0050.56A8.3D22 – Server MAC
feature ospf
feature otv
mac access-list ALL_MACs
10 permit any any
mac access-list HSRP_VMAC
10 permit aaaa.1111.1234 0000.0000.0000 any
20 permit aaaa.2222.1234 0000.0000.0000 any
30 permit any aaaa.1111.1234 0000.0000.0000
40 permit any aaaa.2222.1234 0000.0000.0000
vlan access-map Local 10
match mac address HSRP_VMAC
action drop
vlan access-map Local 20
match mac address ALL_MACs
action forward
vlan filter Local vlan-list 3151-3152
no ip igmp snooping optimise-multicast-flood
vlan 1,202,1111,2222,3151-3152
otv site-vlan 2222
mac-list GMAC_DENY seq 10 deny aaaa.aaaa.aaaa ffff.ffff.ffff
mac-list GMAC_DENY seq 20 deny aaaa.bbbb.bbbb ffff.ffff.ffff
mac-list GMAC_DENY seq 30 permit 0000.0000.0000 0000.0000.0000
route-map stop-GMAC permit 10
match mac-list GMAC_DENY
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
88
ASA Cluster
OTV Configuration for Routed Mode Inter-Site Clustering
interface Overlay1
otv join-interface Ethernet8/1
otv control-group 239.1.1.1
otv data-group 232.1.1.0/28
otv extend-vlan 202, 3151
otv arp-nd timeout 60
no shutdown
interface Ethernet8/1
description uplink_to_OTV_cloud
mtu 9198
ip address 10.4.0.18/24
ip igmp version 3
no shutdown
interface Ethernet8/2
interface Ethernet8/3
description back_to_default_vdc_e6/39
switchport
switchport mode trunk
switchport trunk allowed vlan 202,2222,3151-3152
mac packet-classify
no shutdown
otv-isis default
vpn Overlay1
redistribute filter route-map stop-GMAC
otv site-identifier 0x2
otv flood mac 0050.56A8.3D22 vlan 3151
OTV Filter Modifications Required Because of Site Failure
If a site goes down, the filters need to be removed from OTV because you do not want to block the global
MAC address anymore. There are some additional configurations required.
You need to add a static entry for the ASA global MAC address on the OTV switch in the site that is functional.
This entry will let the OTV at the other end add these entries on the overlay interface. This step is required
because if the server and client already have the ARP entry for the ASA, which is the case for existing
connections, then they will not send the ARP again. Therefore, OTV will not get a chance to learn the ASA
global MAC address in its forwarding table. Because OTV does not have the global MAC address in its
forwarding table, and per OTV design it will not flood unicast packets over the overlay interface, then it will
drop the unicast packets to the global MAC address from the server, and the existing connections will break.
//OTV filter configs when one of the sites is down
mac-list GMAC_A seq 10 permit 0000.0000.0000 0000.0000.0000
route-map a-GMAC permit 10
match mac-list GMAC_A
otv-isis default
vpn Overlay1
redistribute filter route-map a-GMAC
no vlan filter Local vlan-list 3151
mac address-table static aaaa.1111.1234 vlan 3151 interface Ethernet8/3
//Static entry required only in the OTV in the functioning Site
When the other site is restored, you need to add the filters back again and remove this static entry on the OTV.
It is very important to clear the dynamic MAC address table on both the OTVs to clear the overlay entry for
the global MAC address.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
89
ASA Cluster
History for ASA Clustering
MAC Address Table Clearing
When a site goes down, and a static entry for the global MAC address is added to OTV, you need to let the
other OTV learn the global MAC address on the overlay interface. After the other site comes up, these entries
should be cleared. Make sure to clear the mac address table to make sure OTV does not have these entries in
its forwarding table.
cluster-N7k6-OTV# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+-----------------G d867.d900.2e42 static - F F sup-eth1(R)
O 202 885a.92f6.44a5 dynamic - F F Overlay1
* 202 885a.92f6.4b8f dynamic 5 F F Eth8/3
O 3151 0050.5660.9412 dynamic - F F Overlay1
* 3151 aaaa.1111.1234 dynamic 50 F F Eth8/3
OTV ARP Cache Monitoring
OTV maintains an ARP cache to proxy ARP for IP addresses that it learned across the OTV interface.
cluster-N7k6-OTV# show otv arp-nd-cache
OTV ARP/ND L3->L2 Address Mapping Cache
Overlay Interface Overlay1
VLAN MAC Address Layer-3 Address Age Expires In
3151 0050.5660.9412 10.0.0.2 1w0d 00:00:31
cluster-N7k6-OTV#
History for ASA Clustering
Feature Name
Platform Feature Information
Releases
ASA Clustering for the
ASA 5580 and 5585-X
9.0(1)
ASA Clustering lets you group multiple ASAs together as a single logical device. A cluster
provides all the convenience of a single device (management, integration into a network) while
achieving the increased throughput and redundancy of multiple devices. ASA clustering is
supported for the ASA 5580 and the ASA 5585-X; all units in a cluster must be the same model
with the same hardware specifications. See the configuration guide for a list of unsupported
features when clustering is enabled.
We introduced or modified the following commands: channel-group, clacp system-mac,
clear cluster info, clear configure cluster, cluster exec, cluster group, cluster
interface-mode, cluster-interface, conn-rebalance, console-replicate, cluster master unit,
cluster remove unit, debug cluster, debug lacp cluster, enable (cluster group), health-check,
ip address, ipv6 address, key (cluster group), local-unit, mac-address (interface),
mac-address pool, mtu cluster, port-channel span-cluster, priority (cluster group), prompt
cluster-unit, show asp cluster counter, show asp table cluster chash-table, show cluster,
show cluster info, show cluster user-identity, show lacp cluster, and show running-config
cluster.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
90
ASA Cluster
History for ASA Clustering
Feature Name
Platform Feature Information
Releases
ASA 5500-X support for 9.1(4)
clustering
The ASA 5512-X, ASA 5515-X, ASA 5525-X, ASA 5545-X, and ASA 5555-X now support
2-unit clusters. Clustering for 2 units is enabled by default in the base license; for the ASA
5512-X, you need the Security Plus license.
We did not modify any commands.
Improved VSS and vPC 9.1(4)
support for health check
monitoring
If you configure the cluster control link as an EtherChannel (recommended), and it is connected
to a VSS or vPC pair, you can now increase stability with health check monitoring. For some
switches, such as the Cisco Nexus 5000, when one unit in the VSS/vPC is shutting down or
booting up, EtherChannel member interfaces connected to that switch may appear to be Up to
the ASA, but they are not passing traffic on the switch side. The ASA can be erroneously
removed from the cluster if you set the ASA holdtime timeout to a low value (such as .8
seconds), and the ASA sends keepalive messages on one of these EtherChannel interfaces.
When you enable the VSS/vPC health check feature, the ASA floods the keepalive messages
on all EtherChannel interfaces in the cluster control link to ensure that at least one of the
switches can receive them.
We modified the following command: health-check [vss-enabled].
Support for cluster
members at different
geographical locations
(inter-site); Individual
Interface mode only
9.1(4)
Support for cluster
members at different
geographical locations
(inter-site) for
transparent mode
9.2(1)
Static LACP port
priority support for
clustering
9.2(1)
You can now place cluster members at different geographical locations when using Individual
Interface mode.
We did not modify any commands.
You can now place cluster members at different geographical locations when using Spanned
EtherChannel mode in transparent firewall mode. Inter-site clustering with spanned
EtherChannels in routed firewall mode is not supported.
We did not modify any commands.
Some switches do not support dynamic port priority with LACP (active and standby links).
You can now disable dynamic port priority to provide better compatibility with spanned
EtherChannels. You should also follow these guidelines:
• Network elements on the cluster control link path should not verify the L4 checksum.
Redirected traffic over the cluster control link does not have a correct L4 checksum.
Switches that verify the L4 checksum could cause traffic to be dropped.
• Port-channel bundling downtime should not exceed the configured keepalive interval.
We introduced the following command: clacp static-port-priority.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
91
ASA Cluster
History for ASA Clustering
Feature Name
Platform Feature Information
Releases
Support for 32 active
links in a spanned
EtherChannel
9.2(1)
ASA EtherChannels now support up to 16 active links. With spanned EtherChannels, that
functionality is extended to support up to 32 active links across the cluster when used with
two switches in a vPC and when you disable dynamic port priority. The switches must support
EtherChannels with 16 active links, for example, the Cisco Nexus 7000 with F2-Series 10
Gigabit Ethernet Module.
For switches in a VSS or vPC that support 8 active links, you can now configure 16 active
links in the spanned EtherChannel (8 connected to each switch). Previously, the spanned
EtherChannel only supported 8 active links and 8 standby links, even for use with a VSS/vPC.
If you want to use more than 8 active links in a spanned EtherChannel, you cannot
also have standby links; the support for 9 to 32 active links requires you to disable
cLACP dynamic port priority that allows the use of standby links.
We introduced the following command: clacp static-port-priority.
Note
Support for 16 cluster
members for the ASA
5585-X
9.2(1)
BGP support for ASA
clustering
9.3(1)
The ASA 5585-X now supports 16-unit clusters.
We did not modify any commands.
We added support for BGP with ASA clustering.
We introduced the following command: bgp router-id clusterpool.
Inter-site deployment in 9.3(2)
transparent mode with
the ASA cluster
firewalling between
inside networks
You can now deploy a cluster in transparent mode between inside networks and the gateway
router at each site (AKA East-West insertion), and extend the inside VLANs between sites.
We recommend using Overlay Transport Virtualization (OTV), but you can use any method
that ensures that the overlapping MAC Addresses and IP addresses of the gateway router do
not leak between sites. Use a First Hop Redundancy Protocol (FHRP) such as HSRP to provide
the same virtual MAC and IP addresses to the gateway routers.
Enable and disable ASA 9.4(1)
cluster health monitoring
per interface
You can now enable or disable health monitoring per interface. Health monitoring is enabled
by default on all port-channel, redundant, and single physical interfaces. Health monitoring is
not performed on VLAN subinterfaces or virtual interfaces such as VNIs or BVIs. You cannot
configure monitoring for the cluster control link; it is always monitored. You might want to
disable health monitoring of non-essential interfaces, for example, the management interface.
We introduced the following command: health-check monitor-interface.
ASA clustering support 9.4(1)
for DHCP relay
You can now configure DHCP relay on the ASA cluster. Client DHCP requests are
load-balanced to the cluster members using a hash of the client MAC address. DHCP client
and server functions are still not supported.
We did not modify any commands.
SIP inspection support
in ASA clustering
9.4(1)
You can now configure SIP inspection on the ASA cluster. A control flow can be created on
any unit (due to load balancing), but its child data flows must reside on the same unit. TLS
Proxy configuration is not supported.
We introduced the following command: show cluster service-policy.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
92
ASA Cluster
History for ASA Clustering
Feature Name
Platform Feature Information
Releases
Site-specific MAC
9.5(1)
addresses for inter-site
clustering support for
Spanned EtherChannel
in Routed firewall mode
You can now use inter-site clustering for Spanned EtherChannels in routed mode. To avoid
MAC address flapping, configure a site ID for each cluster member so that a site-specific MAC
address for each interface can be shared among a site’s units.
ASA cluster
9.5(1)
customization of the
auto-rejoin behavior
when an interface or the
cluster control link fails
You can now customize the auto-rejoin behavior when an interface or the cluster control link
fails.
The ASA cluster
supports GTPv1 and
GTPv2
9.5(1)
The ASA cluster now supports GTPv1 and GTPv2 inspection.
Disable health
monitoring of a
hardware module in
ASA clustering
9.5(1)
We introduced or modified the following commands: site-id, mac-address site-id, show
cluster info, show interface
We introduced the following command: health-check auto-rejoin
We did not modify any commands.
Cluster replication delay 9.5(1)
for TCP connections
By default when using clustering, the ASA monitors the health of an installed hardware module
such as the ASA FirePOWER module. If you do not want a hardware module failure to trigger
failover, you can disable module monitoring.
We modified the following command: health-check monitor-interface service-module
This feature helps eliminate the “unnecessary work” related to short-lived flows by delaying
the director/backup flow creation.
We introduced the following command: cluster replication delay
ASA 5516-X support for 9.5(2)
clustering
The ASA 5516-X now supports 2-unit clusters. Clustering for 2 units is enabled by default in
the base license.
We did not modify any commands.
LISP Inspection for
9.5(2)
Inter-Site Flow Mobility
Cisco Locator/ID Separation Protocol (LISP) architecture separates the device identity from
its location into two different numbering spaces, making server migration transparent to clients.
The ASA can inspect LISP traffic for location changes and then use this information for
seamless clustering operation; the ASA cluster members inspect LISP traffic passing between
the first hop router and the egress tunnel router (ETR) or ingress tunnel router (ITR), and then
change the flow owner to be at the new site.
We introduced or modified the following commands: allowed-eid, clear cluster info
flow-mobility counters, clear lisp eid, cluster flow-mobility lisp, debug cluster
flow-mobility, debug lisp eid-notify-intercept, flow-mobility lisp, inspect lisp, policy-map
type inspect lisp, site-id, show asp table classify domain inspect-lisp, show cluster info
flow-mobility counters, show conn, show lisp eid, show service-policy, validate-key
Carrier Grade NAT
9.5(2)
enhancements now
supported in failover and
ASA clustering
For carrier-grade or large-scale PAT, you can allocate a block of ports for each host, rather
than have NAT allocate one port translation at a time (see RFC 6888). This feature is now
supported in failover and ASA cluster deployments.
We modified the following command: show local-host
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
93
ASA Cluster
History for ASA Clustering
Feature Name
Platform Feature Information
Releases
Configurable level for
clustering trace entries
9.5(2)
By default, all levels of clustering events are included in the trace buffer, including many low
level events. To limit the trace to higher level events, you can set the minimum trace level for
the cluster.
We introduced the following command: trace-level
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.5
94
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising