null  null
PERFORMANCE MEASUREMENTS
WITH CISCO DEVICES
SESSION NMS-2042
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
1
Objectives
• This is about
Performance management within FCAPS
Outline features inside Cisco IOS and the value they bring
to performance management
Case studies for performance measures to be applied to
the device, to the network or to the service
How it overlaps performance with fault and accounting
management measures
• This is NOT about
In-depth description of the Cisco IOS technologies used for
performance measurements
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
2
Agenda
• Cisco Strategy for Performance Management
• Technologies
• Performance Measurements—Case Studies
CPU and Memory Utilization
Bandwidth Utilization
Availability
Packet Loss
Delay
Delay Variation (Jitter)
• Different Scenarios
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
3
CISCO STRATEGY FOR
PERFORMANCE MANAGEMENT
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
4
Reasons for Performance Management
• Increases network reliability
• Better understanding and control of network
Deploy new application and services with complete
confidence
• SP differentiator to maintain price premium
• Enterprise justification for IT department costs
• Users can be assured that applications and
services will be there when they need them
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
5
Performance Management
• Main steps for performance management
Data collection
Baseline the network
Set thresholds
Traffic profiles for each class of user
Tuning
• Measurement examples
Line utilization, network throughput,
user response times,…
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
6
Classifying Technologies
Sampling
Passive
Active
Source
Embedded
External Probe
Scope
Device/Link
End-to-End/Path
Perspective
User
NMS-2042
9700_05_2004_c2
Network
© 2004 Cisco Systems, Inc. All rights reserved.
7
Cisco Performance Measurement Strategy
Availability, Response Time,
Packet Loss, …
• Different technologies
SNMP Polling, EEM, SAAgent, RMON, ART MIB, NBAR, QoS, and NetFlow
• With different focus
Device wide, network wide, service wide
Note: No License Fee (Netflow Is the Exception for Some Platforms)
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
8
TECHNOLOGIES
NMS-2042
9700_05_2004_c2
9
© 2004 Cisco Systems, Inc. All rights reserved.
Performance Measurement Technologies
SNMP MIBs and Embedded Event
Manager (EEM)
MEASURES: CPU/Memory Utilization,
Availability
Sampling:
Passive
Collection:
Embedded
Scope:
Device/Link
Perspective:
User/Network
RMON/ART/NBAR
MEASURES: Response Time of Live
Application Traffic to Server Device
Sampling: Passive
Collection: External Probe/Embedded
Scope:
Link/End-to-End
Perspective: User/Network
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
Cisco Service Assurance Agent
(SAAgent)
MEASURES: Latency and Jitter Between
Source Router and Specified Target
Sampling:
Active
Collection:
Embedded
Scope:
Link/End-to-End
Perspective:
User/Network
NetFlow
MEASURES: Device Interface Traffic Rate
by S/D IP Address, Port Number or AS
Sampling:
Collection:
Scope:
Perspective:
Passive
Embedded
Link/End-to-End
Network
10
Performance Measurement Technologies
SNMP MIBs and Embedded
Event Manager (EEM)
MEASURES: CPU/Memory Utilization,
SNMP MIBs and Embedded
Availability
Event Manager (EEM)
CPU/Memory Utilization,
Sampling: MEASURES:
Passive
Availability
Collection:
Embedded
Sampling:
Passive
Collection:
Embedded
Scope:
Device/Link
Scope:
Device/Link
Perspective: Perspective:
User/Network
User/Network
Cisco Service Assurance Agent
(SAAgent)
MEASURES: Latency and Jitter Between
Source Router and Specified Target
Sampling:
Active
Collection:
Embedded
Scope:
Link/End-to-End
Perspective:
User/Network
NetFlow
RMON/ART/NBAR
MEASURES: Response Time of Live
Application Traffic to Server Device
Sampling:
Collection:
Scope:
Perspective:
NMS-2042
9700_05_2004_c2
MEASURES: Device Interface Traffic Rate
by S/D IP Address, Port Number or AS
Passive
External Probe/Embedded
Link/End-to-End
User/Network
Sampling:
Collection:
Scope:
Perspective:
Passive
Embedded
Link/End-to-End
Network
© 2004 Cisco Systems, Inc. All rights reserved.
11
SNMP Possibilities for
Performance Measurements
• OLD-CISCO-CPU MIB, CISCO-PROCESS-MIB and ENTITY-MIB
Collect statistics on the CPU utilization
• CISCO-MEMPOOL-MIB, CISCO-ENHANCED-MEMPOOL-MIB
Collect statistics on the memory utilization
• CISCO-ENVMON-MIB
Status of the environmental monitor on those devices which
support it
• IF-MIB and OLD-CISCO-INTERFACES-MIB
Collect bandwith or link utilization and packet loss
• CISCO-OUTAGE-MONITOR-MIB and CISCO-PING-MIB
Measure Cisco device availability
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
12
Other SNMP Possibilities for
Performance Measurements
• EXPRESSION-MIB
Allows you to create new SNMP objects based upon
formulas
• NBAR-PROTOCOL-DISCOVERY-MIB, CBQoS-MIB
QoS performance measurements
• Other more specific MIBs:
CISCO-FRAME-RELAY-MIB, CISCO-MODEM-MGMT-MIB, …
Recommendation is to use the standard MIBs where
Possible.
NMS-2042
9700_05_2004_c2
13
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco IOS EEM: The Framework
Syslog Event
Health Monitoring
Threshold Crossed
Application
Specific
SNMP ED
Future EDs ..
Syslog ED
Event Detectors
System
Information
Embedded Event
Manager 1.0
EEM Policies
Actions
• Central clearing house for
events detected in a Cisco
IOS device
• Distributes event information
to interested applications
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
Notify
Switch Over
Reload
14
Embedded Event Manager (EEM) 1.0
Background
• Three level conceptual structure
Event detectors—information sources about what is happening
Policy decision—define specific “events” based on ED info, define action
to take when event occurs
Actions—list of possible actions to take
• EEM 1.0 originally developed to support IOS high availability;
applicable to more general situation
SNMP and Syslog Event Detectors
3 actions: force stateful switchover, reboot, send customized message
(SYSLOG or CNS Event Bus)
Introduced in 12.0(26)S
• EEM 2.0 will deliver: other event detectors, ability to invoke
TCL scripts
NMS-2042
9700_05_2004_c2
15
© 2004 Cisco Systems, Inc. All rights reserved.
SNMP Event Detector Highlights
SNMP ED
• Monitor a standard SNMP MIB object and generate
events when the object matches specified values or
crosses specified thresholds
SNMP objects specified via OIDs
• Supports self-health monitoring in Cisco IOS device
providing proactive fault notifications
Monitor key system parameters such as CPU utilization,
processor and IO memory utilization, buffer utilization,
interface load, interface errors etc.
• Health monitoring threshold violations kick off
policy action
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
16
Performance Measurement Technologies
Cisco Service Assurance Agent
(SAAgent)
SNMP MIBs and Embedded
Event Manager (EEM)
MEASURES: CPU/Memory Utilization,
Availability
Sampling:
Passive
Collection:
Embedded
Scope:
Device/Link
Perspective:
User/Network
MEASURES: Latency and Jitter Between
Source Router and Specified Target
Sampling:
Active
Collection:
Embedded
Scope:
Link/End-to-End
Perspective:
User/Network
RMON/ART/NBAR
RMON/ART/NBAR
MEASURES: Response Time of Live
MEASURES: Response
Time of Live
Application Traffic to Server Device
Application Traffic to Server Device
Sampling:
Collection:
Scope:
Perspective:
Passive
NetFlow
MEASURES: Device Interface Traffic Rate
by S/D IP Address, Port Number or AS
Passive
External Probe/Embedded
Link/End-to-End
User/Network
Sampling:
Collection:
Ext. Probe/Embedded
Scope:
Link/End-to-End
Perspective: User/Network
NMS-2042
9700_05_2004_c2
Sampling:
Collection:
Scope:
Perspective:
Passive
Embedded
Link/End-to-End
Network
© 2004 Cisco Systems, Inc. All rights reserved.
17
Measurement Technology
Remote MONitoring (RMON) Background
• RMON is a set of standard MIBs
• RMON is based on IETF RFCs
• Analyzes every frame on a segment
• RMON1 is for data link layer
• RMON2 is for the network layer to
application layer
• RMON2 supported on Network Analysis Module
(NAM) for Catalyst 6000 and 5000
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
18
Measurement Technology
How Does RMON Work?
SQL
Layers 4–7
RMON2 Application Layer
E-Mail
Lotus Notes
WWW
Layers 3
RMON2 Network Layer
S2
C1
S1
C2
Layers 2
RMON1 MAC Layer
Layers 1
Hub Physical Layer
NMS-2042
9700_05_2004_c2
C1
S3
Other Nets
2
1
3
19
© 2004 Cisco Systems, Inc. All rights reserved.
Measurement Technology
RMON1 Groups (RFC 2819 and 1513)
statistics
history
Real-Time—Current Statistics
Statistics over Time
alarm
Predetermined Threshold Watch
host
Tracks Individual Host Statistics
hostTopN
“N” Statistically Most Active Hosts
matrix
A < > B—Conversation Statistics
filters
Packet Structure and Content Matching
capture
event
tokenRing
Collection for Subsequent Analysis
Reaction to Predetermined Conditions
Token Ring—RMON Extensions
mini-RMON Groups
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
20
Measurement Technology
RMON2 Groups (RFC 2021)
protocolDir
Probe’s Master List of Protocols
protocolDist
Segment Protocol Statistics
addressMap
Host-to-MAC Address Matching List
nlhost
nlMatrix
alHost
alMatrix
usrHistory
probeConfig
NMS-2042
9700_05_2004_c2
Host In/Out—Network Layer Statistics
A < > B3—Network Layer Statistics
Host In/Out—Application Layer Statistics
A < > B—Application Layer Statistics
Data Logging—User-Specified Variables
Probe Configuration Standards
© 2004 Cisco Systems, Inc. All rights reserved.
21
Measurement Technology
ART MIB Background
• Application Response Time (ART) MIB extends
RMON2 standards
• Measures delays between request/response
sequences in application flows e.g. http and ftp
• Supports any application that uses well-known
TCP ports
• Probe is needed at both client and server ends with
the ART software option enabled
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
22
Measurement Technology
How Does ART MIB Work?
Application Level Response Time
• TCP protocols only
Client Latency
• Supported protocols
include:
Server Latency
C
Network Flight Time
COMPUSRV
DLSW_RD
DLSW_WR
DNS_TCP
FTP-CTRL
FTP-DATA
HTTP
HTTPS
NB_DGM_T
NB_NS_T
NB_SSN_T
NEWS_TCP
NMS-2042
9700_05_2004_c2
NNTP
NOTESTCP
ORACLSQL
REALAUD
SMTP
SNA_TCP
SOCKET
SQLNET_N
SUNRPC_T
TELNET
XWINDOW
S
Identify Application
Example: HTTP
Response Time
SEQ 101
Packet Level Measurement
ACK 101
SEQ 102
SEQ 103
SEQ 104
ACK 104
SEQ 105
SCCP
ACK 105
© 2004 Cisco Systems, Inc. All rights reserved.
23
Measurement Technology
RMON and ART Implementation
Cisco 26/36/3700 Series Routers
HTTP
P
TT
H
Cisco 6500/7600
NAM Integrated
Traffic Analyser
•
•
•
•
•
•
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
RMON I,II
HCRMON
SMON
DSMON
ART
Voice Analysis
24
Measurement Technology
Network-Based Application Recognition-NBAR
• Examines data from Layers 3 through 7
• Uses Layers 3 and 4 plus packet inspection for
classification
• Stateful inspection of dynamic-port traffic
• Packet Description Language Modules (PDLMs)
define applications recognizable by NBAR
• Customers can specify their own match criteria
to identify TCP- or UDP-based applications
Voice Traffic
Data Traffic
•
•
•
•
•
Packet classification
Packet marking
Bandwidth management
Congestion avoidance
Measurement
Video Traffic
NMS-2042
9700_05_2004_c2
P2P
Over
85 Protocols
Supported
Internet
© 2004 Cisco Systems, Inc. All rights reserved.
25
Measurement Technology
Network-Based Application Recognition-NBAR
• NBAR allows to monitor in REAL TIME
Application per interface
Protocol discovery per interface
Modular QoS traffic classification
Multiple top-n tables listing protocols by bandwidth usage
Thresholds based on protocols
• All platforms that currently support NBAR
Cisco 1700 Series Router since release 12.2(2)T
Cisco 2600, 3600, 7100, 7200 Series Routers since release 12.1(5)T
Cisco 3700 and 7500 Series Routers since release 12.2(8)T
Cisco 6500 platforms since 12.1(3)
• Release 12.2(15)T began to support NBAR-ProtocolDiscovery-MIB
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
26
Measurement Technology
NBAR: Performance
• Depends on the traffic MIX
Classification order is: known-flow, well-known destination ports,
well-known source ports, heuristic.
• 7200VXR (NPE300)
Incremental CPU overhead due to NBAR classifying EMIX traffic
at T3 (90Mbps) rates is in the order of 13%.
I.E if CPU without NBAR=56%, then CPU with NBAR=69%
• Historically been software-based but is gaining hardwareassisted execution in 2004.
Hardware assistance is already available for the 7600/6500
platforms with Multi-processor WAN application module (MWAM).
With this card installed (6x1GP processors) this device should
support upto 6 x 1,700,000 = 10,200,000 stateful IP flow per per
line card.
NMS-2042
9700_05_2004_c2
27
© 2004 Cisco Systems, Inc. All rights reserved.
Performance Measurement Technologies
Cisco Service Assurance Agent
(SAAgent)
SNMP MIBs and Embedded
Event Manager (EEM)
MEASURES: Latency, Connectivity, Packet
Cisco
Service
Loss,
and
JitterAssurance
BetweenAgent
Source Router and
(SAAgent)
Specified Target
MEASURES: CPU/Memory Utilization,
Availability
Sampling:
Passive
Collection:
Embedded
Scope:
Device/Link
Perspective:
User/Network
MEASURES:
Latency and Jitter
Between
Sampling:
Active
Source Router and Specified Target
Collection: Active Embedded
Sampling:
Collection:
Embedded
Scope:
Link/End-to-End
Scope:
Link/End-to-End
Perspective:
Perspective:User/Network
User/Network
RMON/ART/NBAR
MEASURES: Response Time of Live
Application Traffic to Server Device
Sampling:
Collection:
Scope:
Perspective:
NMS-2042
9700_05_2004_c2
Passive
External Probe/Embedded
Link/End-to-End
User/Network
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
NetFlow
MEASURES: Device Interface Traffic Rate
by S/D IP Address, Port Number or AS
Sampling:
Collection:
Scope:
Perspective:
Passive
Embedded
Link/End-to-End
Network
28
SAAgent
Cisco IOS Deployment
11.2
12.0(3)T
ICMP Ping
ICMP Echo Path
X
X
X
X
SSCP (SNA)
X
12.0(5)T
12.1(1)T 12.2(2)T
12.0(8)S
X
X
X
X
X
X
Eng2
12.2(11)T
12.2(11)T
X
X
X
X
UDP Echo
X
X
X
X
TCP Connect
X
X
X
X
12.2(11)T
UDP Jitter
X
X
X
HTTP
X
X
X
12.2(11)T
12.2(11)T
DNS
X
X
X
12.2(11)T
DHCP
DLSw+
X
X
X
X
X
X
12.2(11)T
One-Way Latency with UDP
X
X
12.2(11)T
FTP Get
X
X
12.2(11)T
MPLS VPN Aware
X
12.2(11)T
Frame Relay (CLI)
X
12.2(11)T
ICMP Path Jitter
X
12.2(11)T
App. Performance Monitoring
X
VoIP UDP
NMS-2042
VoIP Proactive ©Monitoring
9700_05_2004_c2
2004 Cisco Systems, Inc. All rights reserved.
12.2(11)T
12.3(4)T
12.3(7)T
29
Measurement Technology
Cisco Service Assurance Agent (SAAgent)
• Cisco IOS feature available on most platforms and
Cisco IOS versions
Introduced in Cisco IOS 11.2 as Response Time Reporter (RTR)
Cisco SAAgent since 12.0(5)T new name, new notion of services
IP SLA 12.2S (RLS6) August 2004
• Provides real-time performance metrics
Device, network and service availability,
Network and service delay,
Packet loss,
Network delay variation (Jitter)
• Uses the Response Time Monitoring (RTTMON) MIB
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
30
Measurement Technology SAAgent
Application Performance Monitor (APM)
• Emulates and measures performance of network
applications
• TCL scripting language management interface
• Emulation scripts currently supported:
SMTP, POP3, IP/TV, LDAP, LotusSend, NNTP, PATTERN,
and SAP
• Initially supporting measurements between
two APM nodes
• Goal is to extend the measurements between APM
node(s) into the real application server(s)
NMS-2042
9700_05_2004_c2
31
© 2004 Cisco Systems, Inc. All rights reserved.
Performance Measurement Technologies
SNMP MIBs and Embedded
Event Manager (EEM)
MEASURES: CPU/Memory Utilization,
Availability
Sampling:
Passive
Collection:
Embedded
Scope:
Device/Link
Perspective:
User/Network
RMON/ART/NBAR
MEASURES: Response Time of Live
Application Traffic to Server Device
Sampling:
Collection:
Scope:
Perspective:
NMS-2042
9700_05_2004_c2
Passive
External Probe/Embedded
Link/End-to-End
User/Network
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
Cisco Service Assurance Agent
(SAAgent)
MEASURES: Latency and Jitter Between
Source Router and Specified Target
Sampling:
Active
Collection:
Embedded
Scope:
Link/End-to-End
Perspective:
User/Network
NetFlow
NetFlow
MEASURES: Device Interface Traffic Rate
MEASURES:
Device
by
S/D IP Address, Port
NumberInterface
or AS
Traffic Rate
by S/D IP Address, Port Number or AS
Sampling:
Passive
Collection:
Scope:
Perspective:
Embedded
Sampling: Link/End-to-End
NetworkPassive
Collection:
Embedded
Scope:
Link/End-to-End
Perspective: Network
32
Measurement Technology
NetFlow Background
• NetFlow is the IP accounting de-facto technology
• When to use NetFlow?
How many users are on the network at any given time?
When will upgrades effect the least number of users?
Who are my top N talkers? Which percentage?
How long do my users surf?
Are users staying with in an acceptable usage policy?
DOS/DDoS attack detections!
Traffic Engineering is now one of more popular NetFlow
applications
NMS-2042
9700_05_2004_c2
33
© 2004 Cisco Systems, Inc. All rights reserved.
Measurement Technology
How Does NetFlow Work?
• Flows are unidirectional
• Flows are enabled on a per
input-interface basis
• Flows are defined by 7 keys:
Source IP address
Destination IP address
Source port
7 Identifiers Other Data
Destination port
Layer 3 protocol
TOS byte (DSCP)
Input interface (ifIndex)
Flow Data
Exported to Management Application
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
34
Measurement Technology
NetFlow Versions
• Version 1—initial one
• Version 5—enhanced version 1
• Version 7—in connection with MultiLayer
Switching (MLS)
• Version 8—router-based aggregation
• Version 9—it can do all the previous versions
above and it has been recently chosen as basis for
the IETF standard IPFIX
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
35
Measurement Technology
NetFlow Highlights
• Run on top of CEF or Fast Switching
• 7 flow identifiers
• For ingress/egress traffic
(also traffic terminated on the router)
• IP only
• Previously only unicast, now with v9 multicast
traffic is also tracked
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
36
PERFORMANCE
MEASUREMENTS: CASE STUDIES
NMS-2042
9700_05_2004_c2
37
© 2004 Cisco Systems, Inc. All rights reserved.
Performance Measurements
Case Studies
• Case Studies: X.Y.Z
Measurement:
(1) CPU utilization
(2) Memory utilization
(3) Bandwith utilization
(4) Availability
(5) Packet loss
(6) Delay
(7) Jitter
NMS-2042
9700_05_2004_c2
Focus:
(1)Device wide
(2)Network wide
(3)Service wide
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
Technology:
(1) CLI
(2) SNMP
(3) RMON/ART
(4) EEM
(5) NBAR
(6) SAAgent
(7) APM
(8) NetFlow
38
Performance Measurements
Measurement
(Case Studies)
Scope
Device
1. CPU Utilization
X
2. Memory Utilization
X
3. Bandwidth Utilization
Network
Service
X
X
4. Availability
X
X
X
5. Packet Loss
X
X
X
6. Delay
(X)
X
X
7. Jitter
(X)
X
X
NMS-2042
9700_05_2004_c2
39
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 1.1.1
CPU Utilization: CLI Commands
• Routers running constantly at high utilization level can affect
the overall performance of forwarding and processing packets
Router#sh proc cpu
CPU utilization for five seconds: 0%/0%; one minute: 1%; five minutes: 1%
PID Runtime(ms)
Invoked
uSecs
5Sec
1Min
1
0
1
0
0.00%
0.00%
0.00%
5Min TTY Process
0 Chunk Mgr
2
368
274108
1
0.00%
0.00%
0.00%
0 Load Meter
3
32940
716632
45
0.00%
0.00%
0.00%
0 OSPF Hello
Router#sh proc cpu sorted ?
1min
Sort based on 1 minute utilization
5min
Sort based on 5 minutes utilization
5sec
Sort based on 5 seconds utilization
|
Output modifiers
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
40
Case Study 1.1.2
CPU Utilization: SNMP
• CPU utilization using OLD-CISCO-CPU MIB
• Supported since 10.2
• As of 12.0 all OLD-CISCO-*-MIBs are “deprecated”
busyPer
Router#sh proc cpu
avgBusy5
avgBusy1
CPU utilization for five seconds: 0%/0%; one minute: 1%; five minutes: 1%
PID Runtime(ms)
Invoked
uSecs
5Sec
1Min
1
0
1
0
0.00%
0.00%
0.00%
5Min TTY Process
0 Chunk Mgr
2
368
274108
1
0.00%
0.00%
0.00%
0 Load Meter
3
32940
716632
45
0.00%
0.00%
0.00%
0 OSPF Hello
• OLD-CISCO-CPU MIB only applies to RP CPU
• OLD-CISCO-CPU MIB doesn’t apply to CPU
utilization for VIP cards (7500) or LC (GSR)
NMS-2042
9700_05_2004_c2
41
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 1.1.2
CPU Utilization: SNMP
• CISCO-PROCESS-MIB: New MIB introduced in
12.0T train; provides information on CPU utilization
and running processes
Router#sh proc cpu
cpmCPUTotal1min
cpmCPUTotal5sec
cpmCPUTotal5min
CPU utilization for five seconds: 0%/0%; one minute: 1%; five minutes: 1%
PID Runtime(ms)
Invoked
uSecs
5Sec
1Min
1
0
1
0
0.00%
0.00%
0.00%
5Min TTY Process
0 Chunk Mgr
2
368
274108
1
0.00%
0.00%
0.00%
0 Load Meter
3
32940
716632
45
0.00%
0.00%
0.00%
0 OSPF Hello
• Solution for VIP cards(7500) and LC(GSR):
CISCO-PROCESS-MIB + ENTITY-MIB
cpmProcessEntry
cpmProcessExtRevEntry
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
42
Case Study 1.1.2
CPU Utilization: SNMP for VIP and LC
• The ENTITY-MIB provides an inventory of the chassis, cpu
card(s), line cards, fans, power supplies etc.; this MIB is the
industry-standard replacement to the OLD-CISCO-CHASSIS-MIB
• Which MIB variables to use for VIP and LC?
cpmCPUTotalTable
Assigned Arbitrarily and Is
Not Saved Over Reboots
cpmCPUTotalIndex Unsigned32,
CISCO-PROCESS
MIB
cpmCPUTotalPhysicalIndex EntPhysicalIndexOrZero,
cpmCPUTotal5sec Gauge32,
cpmCPUTotal1min Gauge32,
cpmCPUTotal5min Gauge32,
Defined in Another
Variable entPhysicalEntry
in the ENTITY-MIB
INDEX { cpmCPUTotalIndex }
NMS-2042
9700_05_2004_c2
43
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 1.1.2: CPU Utilization in VIP and LC
ENTITY and CISCO-PROCESS MIBs Together
• CISCO-PROCESS-MIB:
cpmCPUTotalTable.cpmCPUTotalEntry.cpmCPUTotalPhysicalIndex.1 :
INTEGER: 0
cpmCPUTotalTable.cpmCPUTotalEntry.cpmCPUTotalPhysicalIndex.2 :
INTEGER: 28
cpmCPUTotalTable.cpmCPUTotalEntry.cpmCPUTotal5sec.1 : Gauge32:
cpmCPUTotalTable.cpmCPUTotalEntry.cpmCPUTotal5sec.2 : Gauge32:
cpmCPUTotalTable.cpmCPUTotalEntry.cpmCPUTotal1min.1 : Gauge32:
cpmCPUTotalTable.cpmCPUTotalEntry.cpmCPUTotal1min.2 : Gauge32:
cpmCPUTotalTable.cpmCPUTotalEntry.cpmCPUTotal5min.1 : Gauge32:
cpmCPUTotalTable.cpmCPUTotalEntry.cpmCPUTotal5min.2 : Gauge32:
12
9
10
5
8
4
• There are 2 CPUs displayed
• The second CPU is a VIP identified by an index number of 2
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
44
Case Study 1.1.2: CPU Util in VIP and LC
ENTITY and CISCO-PROCESS MIBs Together
• ENTITY-MIB:
entPhysicalDescr.28 : OCTET STRING- (ascii): Versatile Interface Processor (VIP2-50)
entPhysicalVendorType.28 : OBJECT IDENTIFIER:
.iso.org.dod.internet.private.enterprises.cisco.ciscoModules.3.1.9.7.26
entPhysicalContainedIn.28 : INTEGER: 8
entPhysicalClass.28 : INTEGER: module
entPhysicalParentRelPos.28 : INTEGER: 6
entPhysicalName.28 : OCTET STRING- (ascii):
entPhysicalHardwareRev.28 : OCTET STRING- (ascii):
2.00
entPhysicalFirmwareRev.28 : OCTET STRING- (ascii):
entPhysicalSoftwareRev.28 : OCTET STRING- (ascii):
22.20
entPhysicalSerialNum.28 : OCTET STRING- (ascii):
13944617
entPhysicalMfgName.28 : OCTET STRING- (ascii):
CISCO
entPhysicalTable.entPhysicalEntry.entPhysicalModelName.28 : OCTET STRING- (ascii):VIP2-50
entPhysicalTable.entPhysicalEntry.entPhysicalAlias.28 : OCTET STRING- (ascii):
entPhysicalTable.entPhysicalEntry.entPhysicalAssetID.28 : OCTET STRING- (ascii):
entPhysicalTable.entPhysicalEntry.entPhysicalIsFRU.28 : INTEGER: true
NMS-2042
9700_05_2004_c2
45
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 1.1.3
CPU Utilization: RMON Reaction Condition
• Configure RMON to generate a trap if CPU utilization reaches
or exceeds 80%, and rearm the trap if utilization drops to 40%
or less, sampling interval is 20 seconds
Falling Condition
Rising Condition
Router(config)#rmon alarm 1 cpmCPUTotalEntry.3.0 20 absolute
rising-threshold 80 1 falling-threshold 40 2 owner me
Router(config)#rmon event 1 log trap public description "cpu
busy" owner me
Router(config)#rmon event 2 log description "cpu not too busy"
Threshold
Violation
Rising
Falling
Event 1
Threshold
Violation
Event 1
80%
40%
Threshold
Violation Event 2
Time
NMS-2042
9700_05_2004_c2
No Alert
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
46
Case Study 1.1.4
CPU Utilization: EEM
• The following configuration example generate a syslog
message to be logged when
CPU utilization goes above 90%
Event monitoring will resume when the CPU utilization falls below 80% or
when 20 minutes have passed
Event Manager applet totcpualert
event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.6.1 get-type
exact entry-op ge entry-val 90 exit-comb or exit-op le
exit-val 80 exit-time-sec 1200
action syslog priority alert msg *** Warning*** CPU
utilization is above 90%
• More flexible than RMON event and alarm
Please refer to the NMS-3011 presentation for more info on EEM
“Getting the Right Fault Events from Network Elements”
NMS-2042
9700_05_2004_c2
47
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 2.1.1
Memory Utilization: CLI Commands
• The amount of main memory left on a router’s processor has
significant impact on performance
Router# show memory
Head
Processor 60DB19C0
Fast 60D919C0
Total(b)
Used(b)
Free(b)
Lowest(b)
Largest(b)
119858752
1948928
117909824
117765180
117903232
131072
69560
61512
61512
6146 …
• Buffers are allocated from memory into different memory pools
Router# show buffers
Buffer elements:
499 in free list (500 max allowed)
124485689 hits, 0 misses, 0 created
Public buffer pools:
Small buffers, 104 bytes (total 120, permanent 120:
112 in free list (20 min, 250 max allowed)…
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
48
Case Study 2.1.2
Memory Utilization: SNMP
• Initially OLD-CISCO-MEMORY MIB
• CISCO-MEMPOOL-MIB only supports RP
ciscoMemoryPoolFree
ciscoMemoryPoolUsed
Router# show memory
Head
Processor 60DB19C0
Fast 60D919C0
ciscoMemoryPoolLargestFree
Total(b)
Used(b)
Free(b)
Lowest(b)
Largest(b)
119858752
1948928
117909824
117765180
117903232
131072
69560
61512
61512
6146
• Solution for VIP cards(7500) and LC(GSR): CISCOENHANCED-MEMPOOL MIB
• CISCO-ENHANCED-MEMPOOL MIB is supported from
12.0(21)S for VIP cards and 12.0(20)ST for GSR LCs
NMS-2042
9700_05_2004_c2
49
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 2.1.2
Memory Utilization in VIP and LC: SNMP
• CISCO-ENHANCED-MEMPOOL MIB
"A table of memory pool monitoring entries for all physical
entities on a managed system.“
cempMemPoolTable
INDEX
NMS-2042
9700_05_2004_c2
cempMemPoolIndex
CempMemPoolIndex,
cempMemPoolType
CempMemPoolTypes,
cempMemPoolName
SnmpAdminString,
cempMemPoolPlatformMemory
AutonomousType,
cempMemPoolAlternate
CempMemPoolIndexOrNone,
cempMemPoolValid
TruthValue,
cempMemPoolUsed
Gauge32,
cempMemPoolFree
Gauge32,
cempMemPoolLargestFree
Gauge32,
cempMemPoolLowestFree
Gauge32
{ entPhysicalIndex, cempMemPoolIndex }
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
50
Case Study 3: Performance Measurements
Bandwidth or Link Utilization
• There are multiple methods for collecting BW and
link utilization
• Router can report only on traffic that is either
addressed directly to or passing through itself
• MIBs to be used are
IF-MIB
CBQoS-MIB
Gives you Utilization per class of service
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
51
Case Study 3.2.1
Bandwidth Utilization: CLI Commands
Router# show interfaces serial 0/1
Serial0/1 is up, line protocol is up
Internet address is 192.1.1.105/30
MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec, rely 255/255, load
1/255
Default value
…
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
345817 packets input, 27998005 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
1 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 1 abort
277596 packets output, 20203129 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 output buffer failures, 0 output buffers swapped out
3 carrier transitions
RTS up, CTS up, DTR up, DCD up, DSR up
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
52
Case Study 3.2.2
Bandwidth Utilization—SNMP
• ifInOctets—Total number of octets received on the interface,
including framing characters
• ifOutOctets—Total number of octets transmitted out of the
interface, including framing characters
• ifSpeed—An estimate of the interface’s current bandwidth in
bits per second; for interfaces which do not vary in bandwidth
or for those where no accurate estimation can be made, this
object should contain the nominal bandwidth
Input Util = [(delta (ifInOctets)) * 8 *100] / [(number of seconds in delta)
* ifSpeed ]
Output Util = [delta(ifOutOctets) ) * 8 *100] / [(number of seconds in
delta) * ifSpeed ]
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
53
Case Study 3.2.2
Bandwidth Utilization per COS
• Cisco-Class-Based-QoS-MIB (CiscoCBQosMIB)
Primary Accounting Mechanism for QoS, when configured
through the Modular QoS Command Line Interface (MQC), which
is the de-facto Cisco standard for configuring QoS
• Provides:
Read access and stats for active MQC configuration
NOTE: active means any configured policy that has been
attached to one or more interfaces
• Supported since Cisco IOS 12.1(5)T → Cisco IOS12.2
GSR 12.0(22)S partial support
10720 since 12.0(x)S
10000 ESR since 12.0(24)S
7200, 7500 since 12.0(12)S
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
54
Case Study 3.2.2
Bandwidth Utilization per CoS: CiscoCBQosMIB
After QoS Policies
Have Been Applied
Before QoS
CMPrePolicyPkt
CMPostPolicyByte
CMPrePolicyByte
CMDropPkt
CMDropByte
CMNoBufDropPkt
CMNoBufDropPkt
Bronze
Bronze
Silver
Drop=Pre - Post
Silver
Bronze
Gold
Gold
NMS-2042
9700_05_2004_c2
Silver
© 2004 Cisco Systems, Inc. All rights reserved.
55
Case Study 3.2.2
Bandwidth Utilization per CoS: CiscoCBQosMIB
• Navigating the CBQosMIB is not an easy step!
Identify polices attached to interfaces (cbQosServicePolicyTable)
Identify interfaces with service policies (IF-MIB)
Get policy names (cbQosPolicyMapCfg)
Select a policy
Match configuration information (Cfg tables) with policyindex.object index
(ObjectsTable)
Collect statistics using the policyindex.QosObectindex
• Deployment
Per hop behaviour
At network edge CE-PE: Classification is complex and traffic will be
policed/discarded
In the core: Classification is simple (DSCP) and traffic should not be
discarded
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
56
Case Study 3.2.2
ciscoCBQosMIB Index Challenge
3
Bronze
SNMP
{
1
5
1 6
7
Bronze
4
Bronze
2
Physical Interfaces
Logical Interfaces
MIB Index to reference the interface physical or Logical
Statistics are collected for each interface eg In/Out octets.
A single MQC command (class-map Bronze) applied to multiple interfaces.
How do we reference each occurrence of Bronze?
How can the management system read the statistics?
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
57
Case Study 3.2.2
CBQoS Index Solution
• 3 Indices
Configuration Index
QoS Object Index (Class bronze, Match, Police)
Object will have a different index for each policy they
are attached to.
QoS Policy Index (Policy-map attached to interfaces)
If the same policy is attached to ‘n’ interfaces then the
policy will have ‘n’ policy indices.
in/out on the same interface generates 2 policy indices.
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
58
Case Study 3.2.2
CBQoS Index
• Points to note
No entry in the MIB until Policy is attached to an interface.
Object and Policy indices are generated by the device and are
dynamic.
Statistics are referenced using PolicyIndex.Objectindex
Navigation is complex (Reference Config to stats).
• Reusing MQC statements.
Each time an MQC statement is used it creates a new object with
a unique ‘Objectindex’ reference
Example: if Class Bronze is applied to ‘N’ interfaces there will be
‘N’×Bronze objects in CBQoSmib, each with a unique
Objectindex.
NMS-2042
9700_05_2004_c2
59
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 3.2.2
CISCO-CLASS-BASED-QOS-MIB
1
2
3
4
5
:
:
:
:
:
cbQosServicePolicy
cbQosInterfacePolicy
cbQosFrameRelayVCPolicy
cbQosATMPVCPolicy
cbQosObjects
6 :
7 :
8 :
9 :
10:
11:
12:
13:
14:
21:
cbQosPolicyMapCfg
cbQosClassMapCfg
cbQosMatchStmtCfg
cbQosQueueingCfg
cbQosREDCfg
cbQosREDClassCfg
cbQosPoliceCfg
cbQosTSCfg
cbQosSetCfg
cbQosPoliceActionCfg
15:
16:
17:
18:
19:
20:
cbQosClassMapStats
cbQosMatchStmtStats
cbQosPoliceStats
cbQosQueueingStats
cbQosTSStats
cbQosREDClassStats
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
Maps QoS objects to Interface/PVC
and the direction policy is applied.
Maps configuration to statistics
Configindex to objectindex.policyindex
Configuration Information
New table required for multiple
policing actions in MQC
CB QoS Statistics
60
Case Study 3.2.2
Bandwidth Utilization: Multicast Traffic
• Originally ifTable (RFC 1213) only gave unicast and
non-unicast traffic:
ifIncastPkts, ifOutUcastPkts, ifInNUcastPkts, and
ifOutNUcastPkts
• RFC 1573 and RFC 2863 defines new counters in
the ifXTable
ifInMulticastPkts, ifInBroadcastPkts, ifOutMulticastPkts,
and ifOutBroadcastPkts
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
61
Case Study 3.3.3
Service Bandwidth Utilization: RMON2
• RMON2 allows to monitor:
Applications: Protocol distribution to determine network usage
Hosts: Top N hosts to determine top talkers and top conversations
DSMON: Provides QoS Monitoring.
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
62
Case Study 3.3.5
Service Bandwidth Utilization: NBAR CLI
• Top-N for all interfaces
with Cisco NBAR
Protocol Discovery
enabled
• User can set thresholds
on individual protocols
on an interface, or on a
statistic regardless of
protocol
Thresholds for any
combination of supported
protocols/and or all
protocols
NMS-2042
9700_05_2004_c2
Router#sh ip nbar protocol-discovery top-n 5
Serial0/0
Input
Output
Protocol
Packet Count
Packet Count
Byte Count
Byte Count
5 min bit rate
5 min bit rate
------------------------ --------------------custom-01
40565
40565
2596160
2596160
3000
3000
telnet
395
75
28539
6415
0
0
icmp
101
100
7360
6860
0
0
snmp
28
0
1988
0
0
0
netbios
9
0
738
0
0
0
unknown
205
204
14976
10404
0
0
Total
41304
40944
2649809
2619839
3000
3000
63
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 3.3.5
Service Bandwidth Utilization: NBAR MIB
D rilld o w n fo r Tro u b le S h o o ting
F ro m P o rt to
a p p lic atio n
in o n e e as y
c lic k
L au n ch in d ivid u al
re p o rts to
in ve s tig a te
tro u b le, b y
A p p licatio n
NBAR PD drilldown
-1-
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
64
Case Study 3.3.9
NBAR v NAM
• NBAR
Embedded within IOS
Stateful classification of protocols
Protocol volumes only
Used with MQC
• NAM
A probe embedded within a device
Monitors traffic only
Uses RMON II for protocols + applications.
Relies on Well know port numbers no heuristic classification.
Protocol volumes + Response + Flows
Detailed protocol analysis + packet capture.
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
65
Case Study 3.3.8
Service Bandwidth Utilization: NetFlow
NFC
•
•
•
•
•
•
Graphical display of flow data
Spreadsheet data export
Uses timestamps vs. specific polling (RMON)
RMON has limited interface support, and scalability issue
Detail information about source and destination vs. NBAR
Limited to IP traffic
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
66
Case Study 3.3.8
Service Bandwidth Utilization: NetFlow Collector
IP Flows
NMS-2042
9700_05_2004_c2
Netflow Records
V9 UDP port 9991
© 2004 Cisco Systems, Inc. All rights reserved.
67
Case Study 3.3.8
Service Bandwidth Utilization: NFC Aggregation
Fullv9
Full BGP Next Hop
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
68
Case Study 3.3.8
Service Bandwidth Utilization: NFC Reports
Can filter on keys selected
when creating report
Select additional keys (drill down) to aggregate on.
These are the key not selected when Creating Report
NMS-2042
9700_05_2004_c2
69
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 3.3.8
Service Bandwidth Utilization: NFC Reports
Aggregating on 3 keys
Can continue to drill down until all 8
keys (detailCallRecord) are selected
NMS-2042
9700_05_2004_c2
Drilled down to
Protocol for all flows
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
70
Uniqueness and Strengths
of NetFlow and NBAR
NBAR
NetFlow
• IPv6, MPLS, Multicast, BGP NH
technology integration
• Deep & Stateful Packet Inspection
• Billing, Capacity Planning,
Traffic Engineering
• Protocol Discovery with
application statistics
• Internet Access Monitoring:
Peering & Traffic
• Enables precise classification
& QoS treatment
• IETF Standard for Data Sampling
and Export
• Pre-defined protocol & application
recognition
• Security DDOS Monitoring Tool New
• User-Defined Custom Application
Classification New
• Flow timers, timing of network
traffic types
• Who what where when in the
network
• Large NMS partner community
& open source tools
NMS-2042
9700_05_2004_c2
• New application signatures w/o
software upgrade
• Integration with IP Services
(QoS, NAT, Firewall, IDS) New
71
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 4
Availability
Source
Target
• Availability is the metric used to determine uptime
and downtime
Availability = (Uptime)/(Total Time)
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
72
Case Study 4
Availability (Cont.)
• Device availability
Responding to SNMP packets and ICMP Echo
• Network availability
Packet to reach its destination
• Service availability
Service being measured
• Current practice:
SNMP, ICMP ping, traceroute, SAAgent, and RMON
NMS-2042
9700_05_2004_c2
73
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 4.1.1:
Device Availability PING from NMS
NMS
1.1.1.1
• Ping gives you availability details from your network
management station
NMS% ping 1.1.1.1
1.1.1.1 is alive
Note: Only Target Device Is Involved
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
74
Case Study 4.1.1: Device Availability
Ping within the Network
• Ping command successful only if:
The echo request gets to the destination, and the destination is able to get
an echo reply back to the source
Source
12.0.0.0/24
.2
.3
23.0.0.0/24
.2
.3
34.0.0.0/24
.3
Target
.4
Source# debug ip packet IP packet debugging is on
Source# ping 34.0.0.4
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 34.0.0.4, timeout
5d21h: IP: s=12.0.0.1 (local), d=34.0.0.4, Len 100,
5d21h: IP: s=12.0.0.1 (local), d=34.0.0.4, Len 100,
5d21h: IP: s=12.0.0.1 (local), d=34.0.0.4, Len 100,
5d21h: IP: s=12.0.0.1 (local), d=34.0.0.4, Len 100,
5d21h: IP: s=12.0.0.1 (local), d=34.0.0.4, Len 100,
Success rate is 0 percent (0/5)
NMS-2042
9700_05_2004_c2
is 2 seconds:
unroutable.
unroutable.
unroutable.
unroutable.
unroutable.
75
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 4.2.1: Network Availability
Traceroute Command
12.0.0.0/24
Source
.2
.3
23.0.0.0/24
.2
.3
34.0.0.0/24
.3
Target
.4
Source# traceroute 34.0.0.4
Type escape sequence to abort.
Tracing the route to 34.0.0.4
1 12.0.0.2 4 msec 4 msec 4 msec
2 23.0.0.3 20 msec 16 msec 16 msec
3 34.0.0.4 16 msec * 16 msec
5d01h: IP: s=12.0.0.1 (local), d=34.0.0.4 (Serial0/0), Len 28, sending
5d01h: UDP src=33976, DST=33434
5d01h: IP: s=12.0.0.2 (Serial0/0), d=12.0.0.1 (Serial0/0), Len 56, rcvd 3
5d01h: ICMP type=11, code=0
…
This is the first sequence of packets we send with a TTL=1. The first router, in
this case Router2 (12.0.0.2), drops the packet and sends back to the source
(12.0.0.1) a type=11 ICMP message. This corresponds to the Time Exceeded
Message.
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
76
Case Study 4.1.2: Device Availability
Ping within the Network: SNMP
Source
Target
ftp://ftp.cisco.com/pub/mibs/v2/CISCO-PING-MIB.my
destroy(6), creatAndWait(5), active(1)
snmpset -c public Source ciscoPingEntryStatus.333 integer 6
snmpset
snmpset
snmpset
snmpset
snmpset
-c
-c
-c
-c
-c
public
public
public
public
public
Source
Source
Source
Source
Source
ciscoPingEntryStatus.333 integer 5
ciscoPingEntryOwner.333 octetstring Owner_Name
ciscoPingProtocol.333 integer 1
ip(1)
ciscoPingAddress.333 octetstringhex ab447667
ciscoPingPacketCount.333 integer 20
snmpset -c public Source ciscoPingEntryStatus.333 integer 1
Hex
snmpwalk -c public Source ciscoPingEntry
Row Created in Table
NMS-2042
9700_05_2004_c2
77
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 4.1.2: Device Availability
SNMP Polling
NMS
1.1.1.1
• SNMP polling gives you availability details from your network
management station
NMS% snmpwalk -c public 1.1.1.1 ifEntry
Timeout: No Response from 1.1.1.1
Note: Only Target Device Is Involved
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
78
Case Study 4.2.6: Network Availability
ICMP Path Echo SAAgent Operation
ICMP Path Echo
Operation
Hop 1
Hop 2
Source
SAAgent
• Discovers the path using traceroute
Target
IP Device
• Ping determines response time and availability per
hop in the path
• Options in IP packets: Loose Source Routing (LSR) and QoS
(ToS bits)
• Isolates hop that causes the SLA violation
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
79
Case Study 4.2.6: Network Availability
ICMP Path Echo SAAgent Operation Example
Source#
rtr 1
IP Address of the Target Device
type pathEcho protocol ipIcmpEcho 10.0.0.1
frequency 10
Frequency in Sec (default is 60)
rtr schedule 1 start-time now
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
80
Case Study 4.3
Service Availability
Two Levels of Availability:
• IP connectivity
If the user can reach the IP endpoint the service is available
Can be calculated using basic availability equation
Availability = 1 - Probes with No Response
Total Probes Sent
• Bounded IP connectivity
The user can reach the IP endpoint within some bounded criteria agreed
upon between the service provider and customer
IP connectivity is a requirement for bounded IP connectivity
NMS-2042
9700_05_2004_c2
81
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 4.3
Service Availability Example
• SLA states response time must be within 200ms
• Network probe is an ICMP ping
• 10000 probes are sent between management
system and managed device
• 1 probe fails to respond
• 9 probes have a response time >200ms
IP Connectivity =1 SLA Availability =1 -
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
1 .
10000
1+ 9 .
10000
= 0.9999
= 0.999
82
Case Study 4.3.3: Service Availability
ART MIB
• Find out ‘where’
the application
delays are occurring
Detailed data on
request-response
exchanges between
clients and servers
Application
Server
Server
Latency
Total
Time
• Server visibility
Monitor servers
for protocols,
application usage
and top talkers
IP WAN
NM-NAM
Application
Clients
NM-NAM
NMS-2042
9700_05_2004_c2
Total Time
–Server Latency
Network Flight Time
© 2004 Cisco Systems, Inc. All rights reserved.
83
Case Study 4.3.6: Service Availability
SAAgent Operations
• Different criteria for bounded IP connectivity using
SAAgent:
TCP probe
FTP get probe
DNS probe
HTTP probe
DLSW+ probe
Application Performance Monitor (APM)
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
84
Case Study 4.3.6—Service Availability
DNS Operation
mplsce1 (172.17.246.77)
Source
SAAgent
DNS Server
172.17. 246.224
Source#
rtr 8
type dns target-addr 172.17.246.77 name-server 172.17.246.224
rtr schedule 8 start-time now
Source# RTR 8: Starting An Echo Operation - IP RTR Probe 8
2d03h: DNS Query return code: no error
2d03h: hostname = mplsce1.cisco.com
2d03h: responseTime = 5 (ms)
NMS-2042
9700_05_2004_c2
85
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 4: Availability
Interface/Network Down
• Different methods can be used to REPORT back availability to
the NMS:
SNMP Notifications (traps or informs) and syslog messages do not have
query capability
RMON has the ability to generate traps when MIB variable changes:
ifAdminStatus, ifOperStatus
SAAgent generates traps, or triggers an action for timeout, threshold
violation, connection loss and verify error
Reaction Condition
X
NMS
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
Router
Only RMON and SAAgent Have
Threshold Capability
86
Case Study 5
Packet Loss
• Packet count between the outbound interface of the
source device and the inbound interface of the
destination device
• Difficult task, it requires massive data collection,
and correlation of data across the WAN link
• Depending on where the measurement is taken it
will be possible to get device, network and service
packet loss
• Current practice:
SNMP, SAAgent, NetFlow (or other accounting method)
NMS-2042
9700_05_2004_c2
87
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 5.1.1: Device Packet
Loss SNMP
• OLD-CISCO-INTERFACES-MIB
locIfInputQueueDrops—OID .1.3.6.1.4.1.9.2.2.1.1.26
“The number of packets dropped because the input queue was full.”
• IF-MIB
ifInDiscards—OID .1.3.6.1.2.1.2.2.1.13
"The number of inbound packets which were chosen to be discarded even
though no errors had been detected to prevent their being deliverable to a
higher-layer protocol; one possible reason for discarding such a packet
could be to free up buffer space
Discontinuities in the value of this counter can occur at
re-initialization of the management system, and at other times
as indicated by the value of ifCounterDiscontinuityTime”
From “show interface” command
Output queue 0/40, 0 drops; input queue 0/75, 0 drops, …
locIfOutQueueDrops/
ifOutDiscards
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
locIfInQueueDrops/
ifInDiscards
88
Case Study 5.2.1
Packet Loss: SAAgent Operations
• ICMP path jitter probe
Device/network packet loss
• UDP jitter probe
Network packet loss
• Frame Relay probe
Network packet loss
NMS-2042
9700_05_2004_c2
89
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 5.1.6: Device Packet Loss
PathJitter Operation
Path Jitter
Operation (*)
[N Packets, S Size,
Each T ms]
…
Hop 1
Hop 2
Source
SAAgent
• 2 Phases
Target
IP Device
Discovers the path to target device using traceroute
Evaluates each hop one by one
(*) Requires Cisco IOS Version 12.2(2)T or Later
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
90
Case Study 5.1.6: Device Packet Loss
PathJitter Operation (Cont.)
• Sends a specified number of packets to each hop
along the traced path
Default values for all jitter operations:
N(number of packets) = 10, T(inter-packet delay) = 20ms,
S(size) = 10 Bytes/packet
• Measures:
Per hop average response time delay
Per hop packet loss
Per hop cumulated jitter with noise reduction (RFC 1889)
• Use ICMP packet to measure jitter
• Specific to VoIP environment
(*) Requires Cisco IOS Version 12.2(2)T or Later
NMS-2042
9700_05_2004_c2
91
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 5.1.6: Device Packet Loss
PathJitter Operation Example
Source# sh rtr operational-state 2
---- Path Jitter Statistics ---Source IP
Destination IP
Number of Echos
Interval between Echos Target Only
-
172.17.246.5
172.17.246.20
50
30 ms
Enabled (default)
Device PacketLoss =
Network PacketLoss
For Hop1
Hop IP 172.17.246.2:
RTT:1
PacketLoss:0
MinRTT:1
MaxRTT:2
MinPosJitter:1
MaxPosJitter:1
MinNegJitter:0
MaxNegJitter:0
OutOfSequence:0
DiscardedSamples:0
Hop IP 172.17.246.20:
RTT:1
PacketLoss:0
MaxRTT:3
For Target MinRTT:1
MinPosJitter:2
MaxPosJitter:2
MinNegJitter:1
MaxNegJitter:1
OutOfSequence:0
DiscardedSamples:0
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
Jitter:0
SumRTT:19
SumPos:1
SumNeg:0
Sum2RTT:37
Sum2Pos:1
Sum2Neg:0
Jitter:0
SumRTT:14
SumPos:2
SumNeg:2
Sum2RTT:24
Sum2Pos:4
Sum2Neg:2
92
Case Study 5.y.8: Packet Loss
NetFlow
• NetFlow will need external intelligence from NMS to
calculate packet loss
• Enable NetFlow in input interfaces on strategic
points in our network for a particular traffic flow
• Compare the exported flows in the NMS
12.0.0.0/24
23.0.0.0/24
34.0.0.0/24
Source
Target
NetFlow
Collector
NMS-2042
9700_05_2004_c2
93
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 5.(1/2).8
Device and Network Packet Loss: NetFlow
Router2# sh ip cache flow
IP packet size distribution (94442 total packets):
1-32
64
96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .199 .342 .300 .094 .028 .012 .005 .013 .000 .001 .000 .000 .000 .000
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
IP Flow Switching Cache, 4456704 bytes
3 active, 65533 inactive, 25320 added
525312 ager polls, 0 flow alloc failures
last clearing of statistics never
Protocol
Total
Flows
Packets Bytes Packets Active(Sec) Idle(Sec)
-------Flows
/Sec
/Flow /Pkt
/Sec
/Flow
/Flow
TCP-BGP
7
0.0
2
41
0.0
1.6
7.5
UDP-TFTP
1
0.0
1
67
0.0
0.0
15.1
UDP-other
19880
0.0
3
111
0.1
5.6
15.4
ICMP
5429
0.0
3
41
0.0
0.9
15.5
Total:
25317
0.0
3
97
0.2
4.6
15.4
SrcIf
Se0/0
Se0/1
Se0/1
NMS-2042
9700_05_2004_c2
SrcIPaddress
12.0.0.1
193.1.1.3
193.1.1.3
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
DstIf
Se0/1
Se0/0
Se0/0
DstIPaddress
34.0.0.2
172.17.246.225
172.17.246.228
Pr
11
11
11
SrcP
C2E5
00A1
00A1
DstP
00A1
C2E5
C628
Pkts
13
13
2
94
Case Study 5.3.8
Service Packet Loss: NetFlow
Router2# sh ip cache verbose flow
IP packet size distribution (94452 total packets):
1-32
64
96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .199 .342 .300 .094 .028 .012 .005 .013 .000 .001 .000 .000 .000 .000
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
IP Flow Switching Cache, 4456704 bytes
1 active, 65535 inactive, 25322 added
525430 ager polls, 0 flow alloc failures
last clearing of statistics never
Protocol
Total
Flows
Packets Bytes Packets Active(Sec) Idle(Sec)
-------Flows
/Sec
/Flow /Pkt
/Sec
/Flow
/Flow
TCP-BGP
7
0.0
2
41
0.0
1.6
7.5
UDP-TFTP
1
0.0
1
67
0.0
0.0
15.1
UDP-other
19884
0.0
3
111
0.1
5.6
15.4
ICMP
5429
0.0
3
41
0.0
0.9
15.5
Total:
25321
0.0
3
97
0.2
4.6
15.4
SrcIf
Port Msk AS
Se0/1
00A1 /24 193
NMS-2042
9700_05_2004_c2
SrcIPaddress
12.0.0.1
DstIf
Port Msk AS
Se0/0
C628 /0 0
DstIPaddress
NextHop
34.0.0.2
0.0.0.0
Pr TOS Flgs Pkts
B/Pk Active
11 00 10
5
84
39.7
© 2004 Cisco Systems, Inc. All rights reserved.
95
Case Study 6
Delay
• Time that takes for a packet to traverse from one
endpoint to another
• Delay = Latency + Serialization Delay
Latency (propagation delay), time for an electrical impulse to
travel some physical medium from one endpoint to another
Serialization delay, caused by hopping intermediate
networking devices; it includes queuing, processing and
switching time in the intermediaries
• Doesn’t make much sense to talk about device delay
• Current practice:
SAAgent, Ping, and Cisco Ping MIB
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
96
Case Study 6
Delay
• Factors that affect point to point delay:
Packet size
Constant path characteristics
Traffic load
• Extreme packet delay:
Transient routing loop
Measuring Errors
Delay
BestEffort
Business
Voice
Target
0%
NMS-2042
9700_05_2004_c2
α % β % 100% Utilization
© 2004 Cisco Systems, Inc. All rights reserved.
97
Case Study 6.2: Network Delay
Ping vs. SAAgent
• Accuracy
• Granularity
Ping/traceroute time granularity request to be increased
from 4 to 1 ms Wish-bug CSCea34009
• One way delay measurement
Only with SAAgent UDP Jitter probe
Source and Target router needs to be synchronized
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
98
Case Study 6.2
Network Delay Accuracy in SAAgent
Target
Source
Ts
Tn
Tt
• Round Trip Time (RTT) = Tn –Ts – Tt
Tn = Total processing delay
Ts = Processing delay at Source router
Tt = Processing delay at Target router
Note: Ts And Tt Affect only to CPU Processing Delay
Queuing Delay Is Included in the RTT
NMS-2042
9700_05_2004_c2
99
© 2004 Cisco Systems, Inc. All rights reserved.
Accuracy UDP vs.. ICMP SAAgent Probes
Target
(CPU at 90%)
Source
Pagent
• For round trip time accuracy calculation, always use UDP
echo with SAAgent responder
• In this case, processing time spent on the target and source
routers will be subtracted
• Results more accurate regardless of the sender and receiver
CPU load
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
100
Case Study 6.3.7: Service Delay
SAAgent APM
Management
Station
FTP
Files Are Initially
Located in the NMS +
Output File
FTP
SNMP
APM
APM
Client
Lotus_snd.cf
Master.sch
Lotus_snd.scr
Lotus_snd.dat
Lotus_snd-rem.dat
Lotus_snd-rem.scr
Application
Server
E-Mail SIZE
• LotusNotes send e-mail emulation script; TCPIP port
for LotusNotes is 1352
• Configuration file can be download from CCO
http://www.cisco.com/cgi-bin/tablebuild.pl/saa-apm
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
101
Case Study 6.3.7: Service Delay
SAAgent APM (Cont.)
• Config file
Definition of all the variables that are part of the control
script file
OPER_FREQ, OPER_DIST, OPER_COUNT, …
• Data file for client and server
Parameters used by the script for emulating the
application; user configurable
BUF_SIZE, NUM_OF_BUF, DEST/SRC_IP, DEST/SRC_PORT
• Script file (digital signature defined)
TCL script which emulates LotusNotes send Email
• Scheduler file (digital signature defined)
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
102
Case Study 6.3.7: Service Delay
SAAgent APM (Cont.)
Server# sh saa apm cache
Cache Size (bytes) 100000
Server, It can also be Issued on the Client
Cache used (bytes) 1194
TimeCreated TimeAccessed Size Ref Loc Type SBit FileName
03/01 014412 03/01 014716 72 0 1 DAT 0 tmp/apm/data/lotus_snd-rem.dat
03/01 000114 03/01 014716 1122 0 1 SCR 0 tmp/apm/scripts/lotus_snd-rem.scr
Client# sh saa apm operation
Oper ID 100
ControlFile URL ftp//[email protected]/tmp/apm/config/lotus_snd.cf
…
• Format of the result is CSV: “show saa apm results”
NMS /tmp/apm/output % ls
Lotus_snd.out.100.1.90052
<Oper id>, <upload_id>, <upload_time>
[email protected] 25 % more tcp.out.100.1.90052
100,0,32
<Oper id>, <status oper/error code>, <rtt msec>
100,0,32
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
103
Case Study 7
Delay Variation (Jitter)
• Variation delay over time from point to point
• The amount of jitter tolerable is affected by the
depth of jitter buffer in the network equipment In
the voice path
• Dejitter buffer transforms the variable delay into a
fixed delay, holding the 1st sample received for a
period of time before playing it out
• Doesn’t make much sense talking about
device delay
• Current practice:
SAAgent: UDP jitter operation, and path jitter operation
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
104
Case Study 7
Network Delay Variation (Jitter)
Source
Target
PBX
B
A
PBX
C
D
E
Source Transmits
Time
Ti
Target
Receives
A
B
C
Ti1
Ti1 < Ti
Ti2
Ti2 > Ti
Negative Jitter
Positive Jitter
NMS-2042
9700_05_2004_c2
E
D
Time
Ti3
Ti4
Ti3 = Ti Ti4 – Ti
Zero Jitter
JitterSD
105
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 7.2.6
UDP Jitter SAAgent Operation Example
Packet Flow
Router
Router
GE
GE
Fixed:
Coder
Delay
GE
GE
Fixed:
Serialization Fixed:
Delay
Switch
χ1
σ1
Fixed:
Packetization
Delay
π1
Variable:
Output
Queuing
Delay
σ2
Delay
ω1
β1
ω2
β2
Fixed:
De-Jitter
Buffer
∆4
ω3
β3
β4
• Select the proper operation
• Select the proper test pair
• Select the proper payload, sampling interval, threshold
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
106
Case Study 7.2.6
UDP Jitter SAAgent Operation
• Simulating G.711 VoIP Call
Source#
Use RTP/UDP Ports 16384 and above
rtr 5
type jitter dest-ipaddr 10.52.130.68 dest-port 16384 numpackets 1000 interval 20
Packets Sent Every 20 ms
tos 0x2E
request-data-size 200
TOS Value of 46
rtr schedule 5 life forever start-time now
ntp server 10.0.0.2
Target#
ntp server 10.0.0.2
rtr responder
200B Packet Size
(160B of Payload + 40B of Header)
NTP Setup
NOTE: Use the CCO Tool Voice Codec Bandwith Calculator
http://tools.cisco.com/Support/VBC/do/CodecCalc1.do
NMS-2042
9700_05_2004_c2
107
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 7.2.6
VoIP UDP SAAgent Operation
New
!!
• Support for the speech codes since 12.3(4)T:
G.711 A Law, G.711 mu Law and G.729A
• New output for voice quality scoring: MOS and ICPIF for the
Jitter (codec) operation type was added
• Enhanced SAAgent reaction configuration for VoIP jitter
monitoring in 12.3(7)T
SAAgent can generate Syslog Messages when a violation is generated for
unidirectional packet loss, unidirectional jitter and MOS
• VoIP MOS score granularity in 12.3(7)T
A decimal value is now computed for MOS values in the output of the
show rtr operational-state and show rtr collection-statistics commands;
prior to this release, MOS values were returned as whole numbers only
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
108
Case Study 7.2.6
VoIP UDP SAAgent Operation Example
• Simulating G.711 A-Law codec (64 kbps transmission) VoIP Call
Source #
rtr 10
type jitter dest-ipaddr 209.165.200.225 dest-port 16384
codec g711alaw advantage-factor 2
set default values for:
owner admin
codec-numpackets,
tag jitter-with-voice-scores
codec-size, and
rtr schedule 10 start-time now codec-interval
rtr reaction-configuration 10 react mos threshold-type
immediate threshold-value 490 250 action-type trapOnly
rtr logging traps
connectionLoss,
jitterAvg,
snmp-server host 10.10.10.10 version 2c public jitterDSAvg,
jitterSDAvg,
snmp-server enable traps syslog
Mos,
PacketLossDS,
PacketLossSD
Rtt,
Timeout,
verifyError
To Translate Syslog into Traps
Note: Logging Commands Are Needed Only if You
Want to Send the Syslog Message to a Logging Server
NMS-2042
9700_05_2004_c2
109
© 2004 Cisco Systems, Inc. All rights reserved.
Case Study 7.2.6
UDP Jitter SAAgent Operation (Cont.)
Source# show rtr operation-state 5
Current Operational State
…
SD: Source to Destination
DS: Destination to Source
OW: One Way Delay
Voice Scores:
ICPIF Value: 20 MOS score: 3.20
RTT Values:
NumOfRTT: 11
RTTSum: 28422
RTTAvg: 2583
RTTMin: 711
RTTSum2: 92644272
RTTMax: 4699
Packet Loss Values:
PacketLossSD: 0 PacketLossDS: 0
PacketOutOfSequence: 0 PacketMIA: 989
InternalError: 0
Busies: 0
PacketLateArrival: 56
Jitter Values:
MinOfPositivesSD: 1
MaxOfPositivesSD: 249
NumOfPositivesSD: 197
SumOfPositivesSD: 8792 Sum2PositivesSD:
MinOfNegativesSD: 1
MaxOfNegativesSD: 158
NumOfNegativesSD: 761
SumOfNegativesSD: 8811 Sum2NegativesSD:
MinOfPositivesDS: 1
MaxOfPositivesDS: 273
NumOfPositivesDS: 317
SumOfPositivesDS: 7544 Sum2PositivesDS:
MinOfNegativesDS: 1
MaxOfNegativesDS: 183
NumOfNegativesDS: 603
SumOfNegativesDS: 6967 Sum2NegativesDS:
Interarrival jitterout: 16
Interarrival jitterin: 35
One Way Values:
NumOfOW: 0
OWMinSD: 0
OWMaxSD: 0
OWSumSD: 0
OWSum2SD: 0
OWMinDS: 0
OWMaxDS: 0
OWSumDS: 0
OWSum2DS: 0
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
794884
139299
581458
336135
110
Case Study 7.2.6
UDP vs PathJitter Operations
• VoIP UDP and UDP Jitter
(requires SAAgent Responder)
Measures:
Round-trip delay, one-way delay (requires time sync),
jitter, and packet loss
• PathJitter
Uses ICMP compared to UDP
Measures hop-by-hop:
Round-trip delay, packet loss, and jitter
Note: Use the CCO tool Voice Codec Bandwith Calculator
http://tools.cisco.com/Support/VBC/do/CodecCalc1.do
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
111
SCENARIO
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
112
Performance Measurements: Scenario
• When and what to monitor?
Which day and time of the day?
Data collection
• Baseline the network
Assesment of traffic pattern for each class and user
• Select the proper measurements
• Select the proper technology
• Set thresholds
• Select the reporting tool
• Tuning
NMS-2042
9700_05_2004_c2
113
© 2004 Cisco Systems, Inc. All rights reserved.
Typical Scenario
NAM, NBAR,
SAAgent: ICMP, HTTP, VoIP
Frame Relay
256kbps
6K6K-NAM
Web Server
Branch 1
NMNM-NAM
SNMP
Remote
Office
ISP
VPN Access
NAM
NMNM-NAM
NetFlow
NAM
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
114
Technology Overview
Technology
SNMP/EEM RMON/ART
Attribute
Notification
X
X
SAAgent
NetFlow
X
Trending
X
X
Accuracy
X
X
X
X
Push/Pull
Push
Embedded in IOS
X
Push/Pull Model
Push/Pull
Push/Pull
Scalability
NMS-2042
9700_05_2004_c2
X
X
© 2004 Cisco Systems, Inc. All rights reserved.
115
Performance Monitoring Advice
• SNMP and SAAgent are the most complete solutions for
performance management
• Most of the features are embedded within Cisco IOS
• Leverage probing features inside all of your Cisco
devices, with no additional cost*
• Utilize existing equipment and network management
applications
CiscoWorks, ISC(VPNSC), HP OpenView, InfoVista, Concord
eHealth, Agilent Firehunter, Portal, Arbor Networks…
• Some impact on device performance when enabled
*Note: Netflow Does Have a License Fee when Enabled
on 7200 and 7500 for All other Platforms It Is Free
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
116
Success Factors
• Baseline network and application traffic over long
period of time
• Implement exception reporting
• Ongoing tuning and reactivity
• Monitor multiple SLAs simultaneously
• Make sure you’re prepared to provide information
on an ongoing basis
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
117
Q AND A
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
118
Other Network Management Sessions
• Network Management Online Sessions
NMS-1N01 Introduction to Network Management
NMS-1N02 Introduction to SNMP and MIBs
NMS-1N04 Introduction to Service Assurance Agent
NMS-1N41 Introduction to Performance Management
• Fault
NMS-1011
Principles of Fault Management
NMS-3011
Getting the Right Fault Events from NE
• Accounting
NMS-2031
Traffic Accounting Scenarios
NMS-2032
NetFlow for Accounting, Analysis and Attack
• Performance
NMS-4043
NMS-2042
9700_05_2004_c2
Advanced Service Assurance Agent
© 2004 Cisco Systems, Inc. All rights reserved.
119
Complete Your Online Session Evaluation!
WHAT:
Complete an online session evaluation
and your name will be entered into a
daily drawing
WHY:
Win fabulous prizes! Give us your feedback!
WHERE: Go to the Internet stations located
throughout the Convention Center
HOW:
NMS-2042
9700_05_2004_c2
Winners will be posted on the onsite
Networkers Website; four winners per day
© 2004 Cisco Systems, Inc. All rights reserved.
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
120
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
121
NMS-2042
9700_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
122
© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.
Presentation_ID.scr
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement