Testing and Troubleshooting

Testing and Troubleshooting
This chapter describes how to troubleshoot the Cisco ASA and test basic connectivity.
• View Debugging Messages, page 1
• Packet Capture, page 1
• View the Crash Dump, page 6
• View the Coredump, page 7
• vCPU Usage in the ASAv, page 7
• Test Your Configuration, page 8
• Monitoring Connections, page 20
• History for Testing and Troubleshooting , page 21
View Debugging Messages
Because debugging output is assigned high priority in the CPU process, it can render the system unusable.
For this reason, use debug commands only to troubleshoot specific problems or during troubleshooting sessions
with Cisco TAC. Moreover, it is best to use debug commands during periods of less network traffic and fewer
users. Debugging during these periods decreases the likelihood that increased debug command processing
overhead will affect system use. To enable debugging messages, see the debug commands in the command
reference.
Packet Capture
Capturing packets may be useful when troubleshooting connectivity problems or monitoring suspicious
activity. We recommend that you contact Cisco TAC if you want to use the packet capture service.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
1
Testing and Troubleshooting
Guidelines for Packet Capture
Guidelines for Packet Capture
Context Mode
• You can configure captures on the cluster control link within a context; only the packet that is associated
with the context sent in the cluster control link is captured.
• You can only configure one capture for a shared VLAN; if you configure a capture in multiple contexts
on the shared VLAN, then only the last capture that was configured is used.
• If you remove the last-configured (active) capture, no captures become active, even if you have previously
configured a capture in another context; you must remove the capture and add it again to make it active.
• All traffic that enters the interface to which the capture is attached is captured, including traffic to other
contexts on the shared VLAN. Therefore, if you enable a capture in Context A for a VLAN that is also
used by Context B, both Context A and Context B ingress traffic are captured.
• For egress traffic, only the traffic of the context with the active capture is captured. The only exception
is when you do not enable the ICMP inspection (therefore the ICMP traffic does not have a session in
the accelerated path). In this case, both ingress and egress ICMP traffic for all contexts on the shared
VLAN is captured.
Additional Guidelines
• If the ASA receives packets with an incorrectly formatted TCP header and drops them because of the
invalid-tcp-hdr-length ASP drop reason, the show capture command output on the interface where
those packets are received does not show those packets.
• You can only capture IP traffic; you cannot capture non-IP packets such as ARPs.
• For inline SGT tagged packets, captured packets contain an additional CMD header that your PCAP
viewer might not understand.
Capture Packets
To capture packets, perform the following steps.
Procedure
Step 1
Enable packet capture capabilities for packet sniffing and network fault isolation:
capture capture_name [type {asp-drop [all | drop-code] | tls-proxy | raw-data | isakmp [ikev1 | ikev2] |
inline-tag [tag] | webvpn user webvpn-user}] [access-list access_list_name] {interface {interface_name |
asa_dataplane | asa_mgmt_plane | cplane} } [buffer buf_size] [ethernet-type type] [reeinject-hide]
[packet-length bytes ] [circular-buffer] [trace [trace-count number]] [real-time [dump] [detail]] [match
protocol {host source-ip | source-ip mask | any} [operator src_port] {host dest_ip |dest_ip mask | any}
[operator dest_port]]
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
2
Testing and Troubleshooting
Capture Packets
Example:
ciscoasa# capture captest interface inside
You must configure an interface for any packets to be captured. Use the same capture_name on multiple
capture statements to capture multiple types of traffic.
The type asp-drop keyword captures packets dropped by the accelerated security path. In a cluster, dropped
forwarded data packets from one unit to another are also captured. In multiple context mode, when this option
is issued in the system execution space, all dropped data packets are captured; when this option is issued in
a context, only dropped data packets that enter from interfaces belonging to the context are captured.
The type raw-data keywords capture inbound and outbound packets. This setting is the default.
The inline-tag tag keyword-argument pair specifies a tag for a particular SGT value or leaves it unspecified
to capture a tagged packet with any SGT value.
The buffer keyword defines the buffer size used to store the packet. When the byte buffer is full, packet
capture stops. When used in a cluster, this is the per-unit size, not the sum of all units. The circular-buffer
keyword overwrites the buffer, starting from the beginning, when the buffer is full.
The interface keyword sets the name of the interface on which to use packet capture.
To capture packets on the dataplane, use the asa_dataplane keyword. To filter packets captured on an add-on
module backplane, use the asa_dataplane option and follow these guidelines: In single mode, the backplane
control packets bypass the access list and are captured. In multiple context mode, only control packets are
captured in the system execution space. Data packets are captured in the context.
The match keyword captures matching the protocol and source and destination IP addresses and optional
ports. You can use this keyword up to three times in one command. The operator can be one of the following:
• lt—less than
• gt—greater than
• eq—equal to
The real-time keyword displays the captured packets continuously in real-time.
The reinject-hide keyword specifies that no reinjected packets will be captured and applies only in a clustering
environment.
If ACL optimization is configured, you cannot use the access-list command in capture. You can only
use the access-group command. An error appears if you try to use the access-list command in this
case.
Capture cluster control-link traffic:
capture capture_name {type lacp interface interface_id [buffer buf_size] [packet-length bytes ]
[circular-buffer] [real-time [dump] [detail]]
Note
Step 2
capture capture_name interface cluster [buffer buf_size] [ethernet-type type] [packet-length bytes ]
[circular-buffer] [trace [trace-count number]] [real-time [dump] [detail]] [trace] [match protocol {host
source-ip | source-ip mask | any} [operator src_port] {host dest_ip |dest_ip mask | any} [operator dest_port]]
Example:
ciscoasa# capture ccl type lacp interface GigabitEthernet0/0
ciscoasa# capture ccl interface cluster match udp any eq 49495 any
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
3
Testing and Troubleshooting
Capture Packets
ciscoasa# capture ccl interface cluster match udp any any eq 49495
You can capture cluster control link traffic in two ways: to capture all the traffic on the cluster control link,
use the cluster keyword for the interface name. To capture only cLACP packets, specify type lacp, and specify
the physical interface ID instead of the interface name. There are two types of packets on the cluster control
link: control plane packets and data plane packets, which both include forwarded data traffic and cluster LU
messages. The TTL field in the IP address header is encoded to differentiate between these two types of
packets. When forwarded data packets are captured, their clustering trailers are included in the capture file
for debugging purposes.
Step 3
Capture packets cluster-wide:
cluster exec capture capture_name arguments
Step 4
Stop the packet capture:
no capture capture_name
To terminate a real-time packet capture, enter Ctrl + c. To permanently remove the capture, use the no form
of this command. The real-time option applies only to raw-data and asp-drop captures.
Step 5
To manually stop the packet capture without removing packets from the buffer:
capture name stop
Step 6
To start the capture again:
no capture name stop
Step 7
Clear the capture:
clear capture capture_name
Examples
Control Plane Packets
All packets to and from the control plane have a TTL of 255, and port number 49495 is used for the clustering
control-plane listen port. The following example shows how to create a LACP capture for the clustering
environment:
ciscoasa# capture lacp type lacp interface GigabitEthernet0/0
The following example shows how to create a capture for control path packets in the clustering link:
ciscoasa# capture cp interface cluster match udp any eq 49495 any
ciscoasa# capture cp interface cluster match udp any any eq 49495
Data Plane Packets
Data packets include those forwarded from one unit to another unit (its connection owner) and cluster LU
messages. Regular cluster LU update messages have a TTL of 254, and there is a special LU packet that has
a TTL of 253. This special LU packet is only for TCP, and it only happens when the director elects a new
flow owner; the director sends back the requesting packet along with the CLU_FULL update packet. The LU
packet is filled with the original packet's L3/L4 header to avoid a potential race condition at the receiver side.
Forwarded data packets have a TTL of less than 4. The following example shows how to create a capture for
data path packets in the cluster control link. To capture all inter-cluster dataplane "flow logical update"
messages, use port 4193.
ciscoasa# access-list cc1 extended permit udp any any eq 4193
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
4
Testing and Troubleshooting
View a Packet Capture
ciscoasa# access-list cc1 extended permit udp any eq 4193 any
ciscoasa# capture dp interface cluster access-list ccl
View a Packet Capture
You can view a packet capture at the CLI, in a browser, or download a capture to a server of your choice.
Procedure
Step 1
View the capture at the CLI:
[cluster exec] show capture [capture_name] [access-list access_list_name] [count number] [decode] [detail]
[dump] [packet-number number]
Example:
ciscoasa# show capture capin
8 packets captured
1:
2:
3:
4:
5:
6:
7:
8:
03:24:35.526812
03:24:35.527224
03:24:35.528247
03:24:35.528582
03:24:35.529345
03:24:35.529681
03:24:57.440162
03:24:57.440757
192.168.10.10
203.0.113.3 >
192.168.10.10
203.0.113.3 >
192.168.10.10
203.0.113.3 >
192.168.10.10
203.0.113.3 >
> 203.0.113.3:
192.168.10.10:
> 203.0.113.3:
192.168.10.10:
> 203.0.113.3:
192.168.10.10:
> 203.0.113.3:
192.168.10.10:
icmp:
icmp:
icmp:
icmp:
icmp:
icmp:
icmp:
icmp:
echo
echo
echo
echo
echo
echo
echo
echo
request
reply
request
reply
request
reply
request
reply
The access-list keyword displays information for packets that are based on IP or higher fields for the specific
access list identification.
The cluster exec keyword lets you issue the show capture command in one unit and run the command in all
the other units at the same time.
The count keyword displays the number of packets specified data.
The decode keyword is useful when a capture of type isakmp is applied to an interface. All ISAKMP data
flowing through that interface will be captured after decryption and shown with more information after
decoding the fields.
The detail keyword displays additional protocol information for each packet.
The dump keyword displays a hexadecimal dump of the packets that are transported over the data link.
The packet-number keyword starts the display at the specified packet number.
Step 2
View the packet capture with your browser:
https://ip_of_asa/admin/capture/capture_name/pcap
If you leave out the pcap keyword, then only the equivalent of the show capture capture_name command
output is provided.
In multiple context mode, the copy capture command is available only in the system execution space.
Step 3
Copy the packet capture to a server. This example shows FTP.
[cluster exec] copy /pcap capture:[context-name/]capture_name ftp://username:password@server_ip/path
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
5
Testing and Troubleshooting
View the Crash Dump
If you leave out the pcap keyword, then only the equivalent of the show capture capture_name command
output is provided.
Examples
The following example shows a type asp-drop capture:
ciscoasa# capture asp-drop type asp-drop acl-drop
ciscoasa# show capture asp-drop
2 packets captured
1: 04:12:10.428093
192.168.10.10.34327 >
2669456341:2669456341(0) win 4128 <mss 536>
Flow is denied by configured rule
2: 04:12:12.427330
192.168.10.10.34327 >
2669456341:2669456341(0) win 4128 <mss 536>
Flow is denied by configured rule
2 packets shown
10.94.0.51.15868: S
Drop-reason: (acl-drop)
10.94.0.51.15868: S
Drop-reason: (acl-drop)
ciscoasa# show capture asp-drop
2 packets captured
1: 04:12:10.428093
192.168.10.10.34327 >
2669456341:2669456341(0) win 4128 <mss 536>
Flow is denied by configured rule
2: 04:12:12.427330
192.168.10.10.34327 >
2669456341:2669456341(0) win 4128 <mss 536>
Flow is denied by configured rule
2 packets shown
10.94.0.51.15868: S
Drop-reason: (acl-drop)
10.94.0.51.15868: S
Drop-reason: (acl-drop)
The following example shows an ethernet-type capture:
ciscoasa# capture arp ethernet-type arp interface inside
ciscoasa# show cap arp
22 packets captured
1:
2:
3:
4:
5:
6:
7:
05:32:52.119485
05:32:52.481862
05:32:52.481878
05:32:53.409723
05:32:53.772085
05:32:54.782429
05:32:54.784695
arp
arp
arp
arp
arp
arp
arp
who-has
who-has
who-has
who-has
who-has
who-has
who-has
10.10.3.13 tell 10.10.3.12
192.168.10.123 tell 192.168.100.100
192.168.10.50 tell 192.168.100.10
10.106.44.135 tell 10.106.44.244
10.106.44.108 tell 10.106.44.248
10.106.44.135 tell 10.106.44.244
10.106.44.1 tell 11.11.11.112:
View the Crash Dump
If the ASA or ASAv crashes, you can view the crash dump information. We recommend that you contact
Cisco TAC if you want to interpret the crash dump. See the show crashdump command in the command
reference.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
6
Testing and Troubleshooting
View the Coredump
View the Coredump
A coredump is a snapshot of the running program when the program has terminated abnormally or crashed.
Coredumps are used to diagnose or debug errors and save a crash for future off-site analysis. Cisco TAC may
request that you enable the coredump feature to troubleshoot application or system crashes on the ASA or
ASAv. See the coredump command in the command reference.
vCPU Usage in the ASAv
The ASAv vCPU usage shows the amount of vCPUs used for the data path, control point, and external
processes.
The vSphere reported vCPU usage includes the ASAv usage as described plus:
• ASAv idle time
• %SYS overhead used for the ASAv VM
• Overhead of moving packets between vSwitches, vNICs, and pNICs. This overhead can be quite
significant.
CPU Usage Example
The following is an example in which the reported vCPU usage is substantially different:
• ASAv reports: 40%
• DP: 35%
• External Processes: 5%
• vSphere reports: 95%
• ASA (as ASAv reports): 40%
• ASA idle polling: 10%
• Overhead: 45%
The overhead is used to perform hypervisor functions and to move packets between NICs and vNICs using
the vSwitch.
Usage can exceed 100% because the ESXi server can use additional compute resources for overhead on behalf
of the ASAv.
VMware CPU Usage Reporting
In vSphere, click the VM Performance tab, then click Advanced to display the Chart Options drop-down
list, which shows vCPU usage for each state (%USER, %IDLE, %SYS, and so on) of the VM. This information
is useful for understanding VMware’s perspective on where CPU resources are being used.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
7
Testing and Troubleshooting
ASAv and vCenter Graphs
On the ESXi server shell (you access the shell by using SSH to connect to the host), esxtop is available. Esxtop
has a similar look and feel to the Linux top command and provides VM state information for vSphere
performance, including the following:
• Details on vCPU, memory, and network usage
• vCPU usage for each state of each VM.
• Memory (type M while running) and network (type N while running), as well as statistics and the number
of RX drops
ASAv and vCenter Graphs
There are differences in the CPU % numbers between the ASAv and vCenter:
• The vCenter graph numbers are always higher than the ASAv numbers.
• vCenter calls it %CPU usage; the ASAv calls it %CPU utilization.
The terms “%CPU utilization” and “%CPU usage” mean different things:
• CPU utilization provides statistics for physical CPUs.
• CPU usage provides statistics for logical CPUs, which is based on CPU hyperthreading. But because
only one vCPU is used, hyperthreading is not turned on.
vCenter calculates the CPU % usage as follows:
Amount of actively used virtual CPUs, specified as a percentage of the total available CPUs
This calculation is the host view of the CPU usage, not the guest operating system view, and is the average
CPU utilization over all available virtual CPUs in the virtual machine.
For example, if a virtual machine with one virtual CPU is running on a host that has four physical CPUs and
the CPU usage is 100%, the virtual machine is using one physical CPU completely. The virtual CPU usage
calculation is Usage in MHz / number of virtual CPUs x core frequency
When you compare the usage in MHz, both the vCenter and ASAv numbers match. According to the vCenter
graph, MHz % CPU usage is calculated as 60/(2499 x 1 vCPU) = 2.4
Test Your Configuration
This section describes how to test connectivity for the single mode ASA or for each security context, how to
ping the ASA interfaces, and how to allow hosts on one interface to ping through to hosts on another interface.
Test Basic Connectivity: Pinging Addresses
Ping is a simple command that lets you determine if a particular address is alive and responsive. The following
topics explain more about the command and what types of testing you can accomplish with it.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
8
Testing and Troubleshooting
Test Basic Connectivity: Pinging Addresses
What You Can Test Using Ping
When you ping a device, a packet is sent to the device and the device returns a reply. This process enables
network devices to discover, identify, and test each other.
You can use ping to do the following tests:
• Loopback testing of two interfaces—You can initiate a ping from one interface to another on the same
ASA, as an external loopback test to verify basic “up” status and operation of each interface.
• Pinging to an ASA—You can ping an interface on another ASA to verify that it is up and responding.
• Pinging through an ASA—You can ping through an intermediate ASA by pinging a device on the other
side of the ASA. The packets will pass through two of the intermediate ASA’s interfaces as they go in
each direction. This action performs a basic test of the interfaces, operation, and response time of the
intermediate unit.
• Pinging to test questionable operation of a network device—You can ping from an ASA interface to a
network device that you suspect is functioning incorrectly. If the interface is configured correctly and
an echo is not received, there might be problems with the device.
• Pinging to test intermediate communications—You can ping from an ASA interface to a network device
that is known to be functioning correctly. If the echo is received, the correct operation of any intermediate
devices and physical connectivity is confirmed.
Choosing Between ICMP and TCP Ping
The ASA includes the traditional ping, which sends ICMP Echo Request packets and gets Echo Reply packets
in return. This is the standard tool and works well if all intervening network devices allow ICMP traffic. With
ICMP ping, you can ping IPv4 or IPv6 addresses, or host names.
However, some networks prohibit ICMP. If this is true of your network, you can instead use TCP ping to test
network connectivity. With TCP ping, the ping sends TCP SYN packets, and considers the ping a success if
it receives a SYN-ACK in response. With TCP ping, you can ping IPv4 addresses or host names, but you
cannot ping IPv6 addresses.
Keep in mind that a successful ICMP or TCP ping simply means that the address you are using is alive and
responding to that specific type of traffic. This means that basic connectivity is working. Other policies running
on a device could prevent specific types of traffic from successfully getting through a device.
Enable ICMP
By default, you can ping from a high security interface to a low security interface. You just need to enable
ICMP inspection to allow returning traffic through. If you want to ping from low to high, then you need to
apply an ACL to allow traffic.
When pinging an ASA interface, any ICMP rules applied to the interface must allow Echo Request and Echo
Response packets. ICMP rules are optional: if you do not configure them, all ICMP traffic to an interface is
allowed.
This procedure explains all of ICMP configuration you might need to complete to enable ICMP pinging of
ASA interfaces, or for pinging through an ASA.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
9
Testing and Troubleshooting
Test Basic Connectivity: Pinging Addresses
Procedure
Step 1
Ensure ICMP rules allow Echo Request/Echo Response.
ICMP rules are optional and apply to ICMP packets sent directly to an interface. If you do not apply ICMP
rules, all ICMP access is allowed. In this case, no action is required.
However, if you do implement ICMP rules, ensure that you include at least the following on each interface,
replacing “inside” with the name of an interface on your device.
ciscoasa(config)# icmp permit 0.0.0.0 0.0.0.0 echo inside
ciscoasa(config)# icmp permit 0.0.0.0 0.0.0.0 echo-reply inside
Step 2
Ensure access rules allow ICMP.
When pinging a host through an ASA, access rules must allow ICMP traffic to leave and return. The access
rule must at least allow Echo Request/Echo Reply ICMP packets. You can add these rules as global rules.
Assuming you already have access rules applied to interfaces or applied globally, simply add these rules to
the relevant ACL, for example:
ciscoasa(config)#
ciscoasa(config)#
access-list outside_access_in extendedpermit icmp any anyecho
access-list outside_access_in extendedpermit icmp any anyecho-reply
Alternatively, just allow all ICMP:
ciscoasa(config)#
access-list outside_access_in extendedpermit icmp any any
If you do not have access rules, you will need to also allow the other type of traffic you want, because applying
any access rules to an interface adds an implicit deny, so all other traffic will be dropped. Use the access-group
command to apply the ACL to an interface or globally.
If you are simply adding the rule for testing purposes, you can use the no form of the access-list command
to remove the rule from the ACL. If the entire ACL is simply for testing purposes, use the no access-group
command to remove the ACL from the interface.
Step 3
Enable ICMP inspection.
ICMP inspection is needed when pinging through the ASA, as opposed to pinging an interface. Inspection
allows returning traffic (that is, the Echo Reply packet) to return to the host that initiated the ping, and also
ensures there is one response per packet, which prevents certain types of attack.
You can simply enable ICMP inspection in the default global inspection policy.
ciscoasa(config)# policy-map global_policy
ciscoasa(config-pmap)# class inspection_default
ciscoasa(config-pmap-c)# inspect icmp
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
10
Testing and Troubleshooting
Test Basic Connectivity: Pinging Addresses
Ping Hosts
To ping any device, you simply enter ping with the IP address or host name, such as ping 10.1.1.1 or ping
www.example.com. For TCP ping, you include the tcp keyword and the destination port, such as ping tcp
www.example.com 80. That is usually the extent of any test you need to run.
Example output for a successful ping:
Sending 5, 100-byte ICMP Echos to out-pc, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
If the ping fails, the output indicates ? for each failed attempt, and the success rate is less than 100 percent
(complete failure is 0 percent):
Sending 5, 100-byte ICMP Echos to 10.132.80.101, timeout is 2 seconds:
?????
Success rate is 0 percent (0/5)
However, you can also add parameters to control some aspects of the ping. Following are your basic options:
• ICMP ping.
ping [if_name] host [repeat count] [timeout seconds] [data pattern] [size bytes] [validate]
Where:
◦if_name is the name of the interface by which the host is accessible. If you do not include a name,
the routing table is used to determine the interface to use.
◦host is the IPv4, IPv6, or host name of the host you are pinging.
◦repeat count is how many packets to send. The default is 5.
◦timeout seconds is the number of seconds for each packet to time out if no response occurs. The
default is 2.
◦data pattern is the hexadecimal pattern to use in the packets sent. The default is 0xabcd.
◦size bytes is the length of the packet sent. The default is 100 bytes.
◦validate indicates that you want reply data validated.
• TCP ping.
ping tcp [if_name] host [port] [repeat count] [timeout seconds] [source host [ports]
Where:
◦if_name is the interface through which the source sends the ping. If you do not include a name,
the routing table is used.
◦host is the IPv4 address or host name of the destination you are pinging. You cannot use TCP ping
with IPv6 addresses.
◦port is the TCP port on the host you are pinging.
◦repeat and timeout have the same meaning as above.
◦source host port indicates the source host and port for the ping. Use port 0 to get a random port.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
11
Testing and Troubleshooting
Test Basic Connectivity: Pinging Addresses
• Interactive ping.
ping
By entering ping without parameters, you are prompted for interface, destination, and other parameters,
including extended parameters not available as keywords. Use this method if you have need for extensive
control over the ping packets.
Test ASA Connectivity Systematically
If you want to do a more systematic test of ASA connectivity, you can use the following general procedure.
Before You Begin
If you want to see the syslog messages mentioned in the procedure, enable logging (the logging enable
command, or Configuration > Device Management > Logging > Logging Setup in ASDM).
Although unnecessary, you can also enable ICMP debug to see messages on the ASA console as you ping
ASA interfaces from external devices (you will not see debug messages for pings that go through the ASA).
We recommend that you only enable pinging and debugging messages during troubleshooting, as they can
affect performance. The following example enables ICMP debugging, sets syslog messages to be sent to
Telnet or SSH sessions and sends them to those sessions, and enables logging. Instead of using the logging
monitor debug command, you can alternately use the logging buffer debug command to send log messages
to a buffer, and then view them later using the show loggingcommand.
ciscoasa(config)#
ciscoasa(config)#
ciscoasa(config)#
ciscoasa(config)#
debug icmp trace
logging monitor debug
terminal monitor
logging enable
With this configuration, you would see something like the following for a successful ping from an external
host (209.165.201.2) to the ASA outside interface (209.165.201.1):
ciscoasa(config)# debug icmp trace
Inbound ICMP echo reply (len 32 id
Outbound ICMP echo request (len 32
Inbound ICMP echo reply (len 32 id
Outbound ICMP echo request (len 32
Inbound ICMP echo reply (len 32 id
Outbound ICMP echo request (len 32
Inbound ICMP echo reply (len 32 id
1 seq 256) 209.165.201.1 > 209.165.201.2
id 1 seq 512) 209.165.201.2 > 209.165.201.1
1 seq 512) 209.165.201.1 > 209.165.201.2
id 1 seq 768) 209.165.201.2 > 209.165.201.1
1 seq 768) 209.165.201.1 > 209.165.201.2
id 1 seq 1024) 209.165.201.2 > 209.165.201.1
1 seq 1024) 209.165.201.1 > 209.165.201.2
The output shows the ICMP packet length (32 bytes), the ICMP packet identifier (1), and the ICMP sequence
number (the ICMP sequence number starts at 0, and is incremented each time that a request is sent).
When you are finished testing, disable debugging. Leaving the configuration in place can pose performance
and security risks. If you enabled logging just for testing, you can disable it also.
ciscoasa(config)#
ciscoasa(config)#
ciscoasa(config)#
ciscoasa(config)#
no
no
no
no
debug icmp trace
logging monitor debug
terminal monitor
logging enable
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
12
Testing and Troubleshooting
Test Basic Connectivity: Pinging Addresses
Procedure
Step 1
Draw a diagram of your single-mode ASA or security context that shows the interface names, security levels,
and IP addresses. The diagram should also include any directly connected routers and a host on the other side
of the router from which you will ping the ASA.
Figure 1: Network Diagram with Interfaces, Routers, and Hosts
Step 2
Ping each ASA interface from the directly connected routers. For transparent mode, ping the BVI IP address.
This test ensures that the ASA interfaces are active and that the interface configuration is correct.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
13
Testing and Troubleshooting
Test Basic Connectivity: Pinging Addresses
A ping might fail if the ASA interface is not active, the interface configuration is incorrect, or if a switch
between the ASA and a router is down (see the following figure). In this case, no debugging messages or
syslog messages appear, because the packet never reaches the ASA.
Figure 2: Ping Failure at the ASA Interface
Figure 3: Ping Failure Because of IP Addressing Problems
If the ping reply does not return to the router, then a switch loop or redundant IP addresses might exist (see
the following figure).
Step 3
Ping each ASA interface from a remote host. For transparent mode, ping the BVI IP address. This test checks
whether the directly connected router can route the packet between the host and the ASA, and whether the
ASA can correctly route the packet back to the host.
A ping might fail if the ASA does not have a return route to the host through the intermediate router (see the
following figure). In this case, the debugging messages show that the ping was successful, but syslog message
110001 appears, indicating a routing failure has occurred.
Figure 4: Ping Failure Because the ASA Has No Return Route
Step 4
Ping from an ASA interface to a network device that you know is functioning correctly.
• If the ping is not received, a problem with the transmitting hardware or interface configuration may
exist.
• If the ASA interface is configured correctly and it does not receive an echo reply from the “known good”
device, problems with the interface hardware receiving function may exist. If a different interface with
“known good” receiving capability can receive an echo after pinging the same “known good” device, the
hardware receiving problem of the first interface is confirmed.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
14
Testing and Troubleshooting
Trace Routes to Hosts
Step 5
Ping from the host or router through the source interface to another host or router on another interface. Repeat
this step for as many interface pairs as you want to check. If you use NAT, this test shows that NAT is operating
correctly.
If the ping succeeds, a syslog message appears to confirm the address translation for routed mode (305009
or 305011) and that an ICMP connection was established (302020). You can also enter either the show xlate
or show conns command to view this information.
The ping might fail because NAT is not configured correctly. In this case, a syslog message appears, showing
that the NAT failed (305005 or 305006). If the ping is from an outside host to an inside host, and you do not
have a static translation, you get message 106010.
Figure 5: Ping Failure Because the ASA is Not Translating Addresses
Trace Routes to Hosts
If you are having problems sending traffic to an IP address, you can trace the route to the host to determine
if there is a problem on the network path.
Procedure
Step 1
Step 2
Make the ASA Visible on Trace Routes, on page 15.
Determine Packet Routes, on page 17.
Make the ASA Visible on Trace Routes
By default, the ASA does not appear on traceroutes as a hop. To make it appear, you need to decrement the
time-to-live on packets that pass through the ASA, and increase the rate limit on ICMP unreachable messages.
Procedure
Step 1
Create an L3/L4 class map to identify the traffic for which you want to customize connection settings.
class-map name
match parameter
Example:
ciscoasa(config)#
class-map CONNS
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
15
Testing and Troubleshooting
Trace Routes to Hosts
ciscoasa(config-cmap)#
match any
For information on matching statements, see the Service Policy chapter in the firewall configuration guide.
Step 2
Add or edit a policy map that sets the actions to take with the class map traffic, and identify the class map.
policy-map name class name
Example:
ciscoasa(config)# policy-map global_policy
ciscoasa(config-pmap)# class CONNS
In the default configuration, the global_policy policy map is assigned globally to all interfaces. If you want
to edit the global_policy, enter global_policy as the policy name. For the class map, specify the class you
created earlier in this procedure.
Step 3
Decrement time-to-live (TTL) on packets that match the class.
set connection decrement-ttl
Step 4
If you are editing an existing service policy (such as the default global policy called global_policy), you can
skip this step. Otherwise, activate the policy map on one or more interfaces.
service-policy policymap_name {global
|
interface interface_name
}
Example:
ciscoasa(config)#
service-policy global_policy global
The global keyword applies the policy map to all interfaces, and interface applies the policy to one interface.
Only one global policy is allowed. You can override the global policy on an interface by applying a service
policy to that interface. You can only apply one policy map to each interface.
Step 5
Increase the rate limit on ICMP Unreachable messages so that the ASA will appear on trace route output.
icmp unreachable rate-limit rate burst-size size
Example:
ciscoasa(config)#
icmp unreachable rate-limit 50 burst-size 1
The rate limit can be 1-100, with 1 being the default. The burst size is meaningless, but must be 1-10.
Example
The following example decrements TTL for all traffic globally and increase the ICMP unreachable limit to
50.
ciscoasa(config)# class-map global-policy
ciscoasa(config-cmap)# match any
ciscoasa(config-cmap)# exit
ciscoasa(config)# policy-map global_policy
ciscoasa(config-pmap)# class global-policy
ciscoasa(config-pmap-c)# set connection decrement-ttl
ciscoasa(config-pmap-c)# exit
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
16
Testing and Troubleshooting
Trace Routes to Hosts
ciscoasa(config)# icmp unreachable rate-limit 50 burst-size 6
Determine Packet Routes
Use Traceroute to help you to determine the route that packets will take to their destination. A traceroute
works by sending UDP packets or ICMPv6 echo to a destination on an invalid port. Because the port is not
valid, the routers along the way to the destination respond with an ICMP or ICMPv6 Time Exceeded Message,
and report that error to the ASA.
The traceroute shows the result of each probe sent. Every line of output corresponds to a TTL value in increasing
order. The following table explains the output symbols.
Output Symbol
Description
*
No response was received for the probe within the timeout period.
U
No route to the destination.
nn msec
For each node, the round-trip time (in milliseconds) for the specified number of probes.
!N.
ICMP network unreachable. For ICMPv6, address is out of scope.
!H
ICMP host unreachable.
!P
ICMP unreachable. For ICMPv6, port not reachable.
!A
ICMP administratively prohibited.
?
Unknown ICMP error.
Before You Begin
Procedure
Trace the route to a destination:
traceroute [destination_ip | hostname} [source {source_ip | source-interface}] [numeric] [timeout
timeout_value] [probe probe_num] [ttl min_ttl max_ttl] [port port_value] [use-icmp]
Example:
ciscoasa# traceroute 209.165.200.225
Type escape sequence to abort.
Tracing the route to 209.165.200.225
1
2
3
4
5
6
7
8
10.83.194.1 0 msec 10 msec 0 msec
10.83.193.65 0 msec 0 msec 0 msec
10.88.193.101 0 msec 10 msec 0 msec
10.88.193.97 0 msec 0 msec 10 msec
10.88.239.9 0 msec 10 msec 0 msec
10.88.238.65 10 msec 10 msec 0 msec
172.16.7.221 70 msec 70 msec 80 msec
209.165.200.225 70 msec 70 msec 70 msec
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
17
Testing and Troubleshooting
Using the Packet Tracer to Test Policy Configuration
ciscoasa# traceroute 2002::130
Type escape sequence to abort.
Tracing the route to 2002::130
1
2
5000::2 0 msec 0 msec 0 msec
2002::130 10 msec 0 msec 0 msec
Normally, you simply include the destination IP address or hostname, such as traceroute www.example.com.
However, you can adjust the characteristics of the trace if desired:
• source {source_ip | source-interface}—Specifies the interface to use as the source of the trace. You can
specify the interface by name or by IP address. For IPv6, you cannot specify the source interface; you
can only specify the source IP address. An IPv6 address is valid only if you enabled IPv6 on an ASA
interface. In transparent mode, you must use the management address.
• numeric—Indicates that only the IP addresses should be shown in the trace route. Without this keyword,
the trace route does DNS lookups for addresses and includes DNS names, assuming that you configure
DNS.
• timeout timeout_value—How long to wait for a response before timing out. The default is 3 seconds.
• probe probe_num—How many probes to send at each TTL level. The default is 3.
• ttl min_ttl max_ttl—The minimum and maximum time-to-live values for the probes. The minimum
default is one, but you can set it to a higher value to suppress the display of known hops. The maximum
default is 30. The traceroute terminates when the packet reaches the destination or when the maximum
value is reached.
• port port_value—The UDP port to use. The default is 33434.
• use-icmp—Send ICMP packets instead of UDP packets for probes.
Using the Packet Tracer to Test Policy Configuration
You can test your policy configuration by modeling a packet based on source and destination addressing and
protocol characteristics. The trace does policy lookup to test access rules, NAT, routing, and so forth, to see
if the packet would be permitted or denied.
By testing packets this way, you can see the results of your policies and test whether the types of traffic you
want to allow or deny are handled as desired. Besides verifying your configuration, you can use the tracer to
debug unexpected behavior, such as packets being denied when they should be allowed.
Procedure
Step 1
The command is complicated, so we divided it into parts. Start by choosing the interface and protocol for the
trace:
packet-tracer input ifc_name [vlan-idvlan_id ] {icmp | tcp | udp | rawip | sctp} [inline-tag tag] ...
Where:
• input ifc_name—The name of the interface from which to start the trace. For a bridge group, specify
the bridge group member interface name.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
18
Testing and Troubleshooting
Using the Packet Tracer to Test Policy Configuration
• vlan-id vlan_id—(Optional.) The virtual LAN, where packet tracer enters a parent interface, which is
later redirected to a sub-interface. VLAN identity is available only when the input interface is not a
sub-interface. Valid values range from 1 - 4096.
• icmp, tcp, udp, rawip, sctp—The protocol to use. “rawip” is raw IP, that is, IP packets that are not
TCP/UDP.
• inline-tag tag—(Optional.) The security group tag value embedded in the Layer 2 CMD header. Valid
values range from 0 - 65533.
Step 2
Next, type in the source address and protocol criteria.
...{src_ip | user username | security-group {name name | tag tag} | fqdn fqdn-string}...
Where:
• src_ip—The source IPv4 or IPv6 address for the packet trace.
• user username—The user identity in the format of domain\user. The most recently mapped address for
the user (if any) is used in the trace.
• security-group {name name | tag tag}—The source security group based on the IP-SGT lookup for
Trustsec. You can specify a security group name or a tag number.
• fqdn fqdn-string—The fully qualified domain name of the source host, IPv4 only.
Step 3
Next, type in the protocol characteristics.
• ICMP—Enter the ICMP type (1-255), ICMP code (0-255), and optionally, the ICMP identifier. You
must use numbers for each variable, for example, 8 for echo.
type code... [ident]...
• TCP/UDP/SCTP—Enter the source port number.
...src_port ...
• Raw IP—Enter the protocol number, 0-255.
... protocol ...
Step 4
Finally, type in the destination address criteria, destination port for TCP/UDP traces, and optional keywords,
and press Enter.
...dmac {dst_ip | security-group {name name | tag tag} | fqdn fqdn-string} dst_port [detailed] [xml]
Where:
• dst_ip—The destination IPv4 or IPv6 address for the packet trace.
• security-group {name name | tag tag}—The destination security group based on the IP-SGT lookup
for Trustsec. You can specify a security group name or a tag number.
• fqdn fqdn-string—The fully qualified domain name of the destination host, IPv4 only.
• dst_port—The destination port for TCP/UDP/SCTP traces. Do not include this value for ICMP or raw
IP traces.
• dmac— The Destination MAC address.
Note
The Destination MAC address is enabled or disabled depending on the selected interface (L2,
L3, VTEP, Management) and the firewall mode.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
19
Testing and Troubleshooting
Monitoring Connections
• detailed—Provides detailed trace results information in addition to the normal output.
• xml—Displays the trace results in XML format.
Step 5
Step 6
If you want packet tracer to enter a parent interface, which is later redirected to a sub-interface, check VLAN
ID and enter the identity number, 1- 4096. VLAN identity is available only when the input interface is not a
sub-interface.
Specify the Destination MAC Address.
Note
The Destination MAC address field is enabled or disabled depending on the selected interface (L2,
L3, VTEP, Management) and the firewall mode.
Example
The following example traces a TCP packet for the HTTP port from 10.100.10.10 to 10.100.11.11. The result
indicates that the packet will be dropped by the implicit deny access rule.
ciscoasa(config)#
packet-tracer input outside tcp 10.100.10.10 80 10.100.11.11 80
Phase: 1
Type: ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
found next-hop 10.86.116.1 using egress ifc
outside
Phase: 2
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
Implicit Rule
Additional Information:
Result:
input-interface: outside
input-status: up
input-line-status: up
output-interface: NP Identity Ifc
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule
Monitoring Connections
To view current connections with information about source, destination, protocol, and so forth, use the show
conn all detail command.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
20
Testing and Troubleshooting
History for Testing and Troubleshooting
History for Testing and Troubleshooting
Feature Name
Platform
Releases
Description
IPv6 support for traceroute
9.7(1)
Traceroute feature was updated to accept IPv6
address.
Support for packet capture
9.7(1)
The stop and start commands were added to the
capture feature.
Support for packet tracer
9.9(1)
Support for clustering persistent tracing was
introduced.
Support for packet capture
9.9(1)
Support for capturing persistent tracing on cluster
units and decrypted packets.
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
21
Testing and Troubleshooting
History for Testing and Troubleshooting
CLI Book 1: Cisco ASA Series General Operations CLI Configuration Guide, 9.7
22