Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide

Cisco MDS 9000 Family NX-OS Interfaces
Configuration Guide
Cisco MDS NX-OS Release 6.2(15)
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO
CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS
MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY
PRODUCTS.
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
Text Part Number: OL-29284-01
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT
SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE
OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public
domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH
ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT
LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF
DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING,
WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO
OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this
URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership
relationship between Cisco and any other company. (1110R)
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display
output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in
illustrative content is unintentional and coincidental.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
© 2013-2014 Cisco Systems, Inc. All rights reserved.
CONTENTS
New and Changed Information
Preface
13
xv
Audience
xv
Organization
xv
Document Conventions
xvi
Related Documentation xvii
Release Notes xvii
Regulatory Compliance and Safety Information xvii
Compatibility Information xvii
Hardware Installation xvii
Software Installation and Upgrade xvii
Cisco NX-OS xvii
Command-Line Interface xviii
Intelligent Storage Networking Services Configuration Guides
Troubleshooting and Reference xviii
Obtaining Documentation and Submitting a Service Request
CHAPTER
1
Interfaces Overview
1-xix
Fibre Channel Port Rate Limiting
Maximum NPIV Limit
CHAPTER
2
1-xix
1-xx
1-xx
N Port Virtualization
FlexAttach
xviii
1-xix
Trunks and PortChannels
Extended Credits
xviii
1-xx
1-xxi
Configuring Fibre Channel Interfaces
2-1
Information About Fibre Channel Interfaces 2-1
Generations of Modules and Switches 2-1
Port Groups 2-3
Port Rate Modes 2-5
Dedicated Rate Mode 2-7
Shared Rate Mode 2-8
Dedicated Rate Mode Configurations for the 8-Gbps Modules
2-8
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
1
Contents
Port Speed 2-9
Forward Error Correction 2-10
Dynamic Bandwidth Management 2-10
Out-of-Service Interfaces 2-11
Oversubscription Ratio Restrictions 2-11
Bandwidth Fairness 2-17
Upgrade or Downgrade Scenario 2-18
Guidelines and Limitations 2-18
Combining Generation 1, Generation 2, Generation 3, and Generation 4 Modules
Local Switching Limitations 2-19
Port Index Limitations 2-19
PortChannel Limitations 2-22
Default Settings
2-18
2-24
Configuring Fibre Channel Interfaces 2-25
Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode
Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode
Task Flow for Configuring 12-Port 4-Gbps Module Interfaces 2-27
Task Flow for Configuring 4-Port 10-Gbps Module Interfaces 2-28
Configuring Port Speed 2-28
Configuring FEC 2-30
Configuring Rate Mode 2-32
Displaying the Rate Mode Configuration for Interfaces 2-32
Configuring Local Switching 2-37
Disabling Restrictions on Oversubscription Ratios 2-38
Enabling Restrictions on Oversubscription Ratios 2-41
Enabling Bandwidth Fairness 2-42
Disabling Bandwidth Fairness 2-42
Taking Interfaces out of Service 2-42
Releasing Shared Resources in a Port Group 2-44
Disabling ACL Adjacency Sharing for System Image Downgrade 2-44
2-26
2-27
Verifying Fibre Channel Interfaces Configuration 2-45
Displaying Interface Capabilities 2-45
Displaying SFP Diagnostic Information 2-46
Configuration Examples for Fibre Channel Interfaces 2-47
Configuration Example for FEC Module Interfaces 2-47
Configuration Example for 48-Port 8-Gbps Module Interfaces 2-47
Configuration Example for 24-Port 8-Gbps Module Interfaces 2-48
Configuration Example for 4/44-Port 8-Gbps Module Interfaces 2-49
Configuration Example for 48-Port 4-Gbps Module Interfaces 2-50
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2
OL-29284-01, Release 6.x
Contents
Configuration Example for 24-Port 4-Gbps Module Interfaces
CHAPTER
3
Configuring Interfaces
2-51
3-1
Finding Feature Information
3-1
Feature Information for Interfaces
Prerequisites for Interfaces
3-1
3-4
Guidelines and Limitations for Interfaces 3-4
Guidelines for Configuring Port Monitor Interval 3-4
Guidelines for Local Switching 3-5
Guidelines for 10-Gbps Fibre Channel Mode 3-5
Guidelines for VSAN Interface Configuration 3-6
Default Settings for Interface Parameters
3-6
Information About Interfaces 3-7
Interface Description 3-7
Interface Modes 3-7
E Port 3-9
F Port 3-9
FL Port 3-9
NP Ports 3-9
TE Port 3-9
TF Port 3-10
TNP Port 3-10
SD Port 3-10
ST Port 3-10
Fx Port 3-10
B Port 3-11
Auto Mode 3-11
Interface States 3-11
Administrative States 3-11
Operational States 3-11
Reason Codes 3-12
Graceful Shutdown 3-14
10-Gbps Fibre Channel Mode 3-15
Benefits of 10-Gbps Fibre Channel Mode
Supported Modules and Switches 3-15
Port Administrative Speeds 3-17
Auto Sensing 3-17
Frame Encapsulation 3-17
Debounce Timer 3-18
3-15
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
3
Contents
Bit Error Rate Threshold 3-18
Disabling the Bit Error Rate Action 3-18
SFP Transmitter Types 3-19
Port Guard 3-19
Port-Level Port Guard 3-20
Port Monitor Port Guard 3-20
Port Monitor 3-21
Warning Threshold 3-24
Check Interval 3-26
Port Group Monitor 3-26
Local Switching 3-27
Slow-Drain Device Detection and Congestion Avoidance
Interface Types 3-28
Management Interface 3-28
VSAN Interfaces 3-28
3-27
Configuring Interfaces 3-28
Configuring a Fibre Channel Interface 3-29
Setting the Interface Administrative State 3-30
Configuring an Interface Mode 3-30
Configuring the MAX NPIV Limit 3-31
Configuring the System Default F Port Mode 3-31
Configuring ISL Between Two Switches 3-32
Configuring the 10-Gbps Fibre Channel Mode via the CLI 3-33
Configuring the 10-Gbps Fibre Channel Mode via the Device Manager
Configuring the Port Administrative Speed 3-34
Configuring the Interface Description 3-34
Specifying a Port Owner 3-35
Configuring a Beacon Mode 3-35
Configuring a Switch Port Attribute Default Value 3-36
Configuring the Port-Level Port Guard 3-37
Configuring a Port Monitor 3-38
Enabling a Port Monitor 3-38
Configuring the Check Interval 3-38
Configuring a Port Monitor Policy 3-39
Activating a Port Monitor Policy 3-40
Configuring a Port Monitor Port Guard 3-40
Configuring a Port Group Monitor 3-41
Enabling a Port Group Monitor 3-41
Configuring a Port Group Monitor Policy 3-41
Reverting to the Default Value for a Specific Counter 3-42
3-34
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4
OL-29284-01, Release 6.x
Contents
Turning Off Specific Counter Monitoring 3-43
Activating a Port Group Monitor Policy 3-43
Configuring the Management Interface 3-43
Creating a VSAN Interface 3-44
Configuring Slow-Drain Device Detection and Congestion Avoidance 3-45
Configuring the Congestion Frame Timeout Value for FCoE 3-46
Configuring Pause Frame Timeout Value for FCoE 3-46
Configuring the Congestion Drop Timeout Value for Fibre Channel 3-47
Configuring the No-Credit Frame Timeout Value for Fibre Channel 3-47
Configuring the Slow-Port Monitor Timeout Value for Fibre Channel 3-48
Displaying Credit Loss Recovery Actions 3-50
Configuring the Transmit Average Credit-Not-Available Duration Threshold and
Action 3-50
Verifying Interfaces Configuration 3-52
Displaying Interface Information 3-52
Displaying the Port-Level Port Guard 3-61
Displaying the Port Monitor Status and Policies 3-61
Displaying Port Group Monitor Status and Policies 3-65
Displaying the Management Interface Configuration 3-66
Displaying VSAN Interface Information 3-66
Displaying the Congestion Frame Timeout Value for FCoE 3-67
Displaying the Pause Frame Timeout Value for FCoE 3-67
Displaying the Congestion Drop Timeout Value for Fibre Channel
Displaying the No-Credit Frame Timeout Value for Fibre Channel
Displaying Slow-Port Monitor Events 3-68
Transmit-Wait History Graph 3-72
CHAPTER
4
Configuring Interface Buffers
3-67
3-67
4-1
Information About Interface Buffers 4-1
Buffer-to-Buffer Credits 4-1
Performance Buffers 4-2
Buffer Pools 4-2
BB_Credit Buffers for Switching Modules 4-5
Configuring Buffer Credits on a 4-Gbps, 8-Gbps, or Advanced 8-Gbps Module 4-5
48-Port 16-Gbps Fibre Channel Module BB_Credit Buffers 4-6
48-Port 8-Gbps Advanced Fibre Channel Module BB_Credit Buffers 4-7
48-Port 8-Gbps Fibre Channel Module BB_Credit Buffers 4-8
24-Port 8-Gbps Fibre Channel Module BB_Credit Buffers 4-9
4/44-Port 8-Gbps Host-Optimized Fibre Channel Module BB_Credit Buffers 4-10
48-Port 4-Gbps Fibre Channel Module BB_Credit Buffers 4-11
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5
Contents
24-Port 4-Gbps Fibre Channel Module BB_Credit Buffers 4-13
18-Port Fibre Channel/4-Port Gigabit Ethernet Multiservice Module BB_Credit
Buffers 4-14
12-Port 4-Gbps Switching Module BB_Credit Buffers 4-14
4-Port 10-Gbps Switching Module BB_Credit Buffers 4-16
BB_Credit Buffers for Fabric Switches 4-16
Cisco MDS 9396s Fabric Switch BB_Credit Buffers 4-17
Cisco MDS 9250i and Cisco MDS 9148s Fabric Switch BB_Credit_Buffers 4-17
Cisco MDS 9148 Fabric Switch BB_Credit Buffers 4-18
Cisco MDS 9134 Fabric Switch BB_Credit Buffers 4-18
Cisco MDS 9124 Fabric Switch BB_Credit Buffers 4-19
Cisco MDS 9222i Multiservice Modular Switch BB_Credit Buffers 4-19
Extended BB_Credits 4-20
Extended BB_credits on Generation 1 Switching Modules 4-20
Extended BB_credits on 4-Gbps and 8-Gbps Switching Modules 4-21
Buffer-to-Buffer Credit Recovery 4-22
Buffer-to-Buffer State Change Number 4-22
Receive Data Field Size 4-23
Configuring Interface Buffers 4-23
Configuring Buffer-to-Buffer Credits 4-23
Configuring Performance Buffers 4-24
Configuring Extended BB_credits 4-25
Enabling Buffer-to-Buffer Credit Recovery 4-25
Enabling the Buffer-to-Buffer State Change Number
Configuring Receive Data Field Size 4-26
Verifying BB_Credit Configuration
CHAPTER
5
Configuring Trunking
4-26
4-27
5-1
Information About Trunking 5-1
Trunking E Ports 5-2
Trunking F Ports 5-2
Key Concepts 5-3
Trunking Protocols 5-4
Trunk Modes 5-5
Trunk-Allowed VSAN Lists and VF_IDs
5-5
Guidelines and Limitations 5-7
General Guidelines and Limitations 5-7
Upgrade and Downgrade Limitations 5-8
Difference Between TE Ports and TF-TNP Ports
5-8
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6
OL-29284-01, Release 6.x
Contents
Trunking Misconfiguration Examples
Default Settings
5-10
5-11
Configuring Trunking 5-11
Enabling the Cisco Trunking and Channeling Protocols
Enabling the F Port Trunking and Channeling Protocol
Configuring Trunk Mode 5-12
Configuring an Allowed-Active List of VSANs 5-12
Verifying Trunking Configuration
6
Configuring PortChannels
5-12
5-13
Configuration Example for F Port Trunking
CHAPTER
5-11
5-14
6-1
Information About PortChannels 6-1
PortChannels Overview 6-1
E PortChannels 6-2
F and TF PortChannels 6-3
PortChanneling and Trunking 6-3
Load Balancing 6-4
PortChannel Modes 6-6
PortChannel Deletion 6-7
Interfaces in a PortChannel 6-7
Interface Addition to a PortChannel 6-8
Forcing an Interface Addition 6-9
Interface Deletion from a PortChannel 6-9
PortChannel Protocols 6-9
Prerequisites for PortChannels
6-10
Guidelines and Limitations 6-10
General Guidelines and Limitations 6-11
Generation 1 PortChannel Limitations 6-11
F and TF PortChannel Limitations 6-11
Valid and Invalid PortChannel Examples 6-12
Default Settings
6-13
Configuring PortChannels 6-14
Configuring PortChannels Using the WizardCreating a PortChannel
Configuring the PortChannel Mode 6-14
Deleting PortChannels 6-15
Adding an Interface to a PortChannel 6-15
Forcing an Interface Addition 6-16
Deleting an Interface from a PortChannel 6-16
6-14
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7
Contents
Verifying PortChannel Configuration
6-16
Configuration Examples for F and TF PortChannels
6-19
Configuring Ethernet PortChannels 6-22
Configuring Ethernet PortChannels 6-22
Configuring the Ethernet PortChannel Mode 6-22
Deleting Ethernet PortChannels 6-23
Adding an Interface to a Ethernet PortChannel 6-23
Forcing an Interface Addition to an Ethernet PortChannel 6-24
Deleting an Interface from an Ethernet PortChannel 6-24
Verifying Ethernet PortChannel Configuration
6-24
Configuration Examples for F and TF Ethernet PortChannels
6-32
Configuring Virtual Interfaces 6-34
Mapping a VSAN to a VLAN 6-34
Before You Begin 6-34
Creating an Explicit Virtual Fibre Channel Interface 6-35
Before You Begin 6-35
Creating an Implicit Virtual Fibre Channel Port Channel Interface
Before You Begin 6-36
Associating a Virtual Fibre Channel Interface to a VSAN 6-37
Before You Begin 6-37
CHAPTER
7
Configuring N Port Virtualization
6-36
7-1
Information About N Port Virtualization 7-1
NPV Overview 7-1
N Port Identifier Virtualization 7-2
N Port Virtualization 7-3
NPV Mode 7-4
NP Ports 7-5
NP Links 7-5
Internal FLOGI Parameters 7-6
Default Port Numbers 7-7
NPV CFS Distribution over IP 7-7
NPV Traffic Management 7-7
Auto 7-8
Traffic Map 7-8
Disruptive 7-8
Multiple VSAN Support 7-8
Guidelines and Limitations 7-8
NPV Guidelines and Requirements
7-9
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
8
OL-29284-01, Release 6.x
Contents
NPV Traffic Management Guidelines 7-9
DPVM Configuration Guidelines 7-10
NPV and Port Security Configuration Guidelines 7-10
Connecting an NPIV-enabled Cisco MDS 9396S Multilayer Fabric Switch to an NPV
switch running Cisco MDS NX-OS version earlier than 6.2(13) 7-10
Configuring N Port Virtualization 7-11
Enabling N Port Identifier Virtualization 7-11
Configuring NPV 7-11
Configuring NPV Traffic Management 7-13
Configuring List of External Interfaces per Server Interface 7-13
Enabling the Global Policy for Disruptive Load Balancing 7-13
Verifying NPV Configuration 7-14
Verifying NPV 7-14
Verifying NPV Traffic Management
CHAPTER
8
Configuring FlexAttach Virtual pWWN
7-16
8-1
Information About FlexAttach Virtual pWWN 8-1
FlexAttach Virtual pWWN 8-1
Difference Between San Device Virtualization and FlexAttach Port Virtualization
FlexAttach Virtual pWWN CFS Distribution 8-2
Security Settings for FlexAttach Virtual pWWN 8-3
Guidelines and Limitations
8-2
8-3
Configuring FlexAttach Virtual pWWN 8-3
Automatically Assigning FlexAttach Virtual pWWN 8-3
Manually Assigning FlexAttach Virtual pWWN 8-4
Mapping pWWN to Virtual pWWN 8-4
Verifying FlexAttach Virtual pWWN Configuration
Verifying the End Device 8-6
Monitoring FlexAttach Virtual pWWN
CHAPTER
9
Configuring Port Tracking
Default Settings
8-6
9-1
Information About Port Tracking
Guidelines and Limitations
8-5
9-1
9-2
9-2
Configuring Port Tracking 9-3
Enabling Port Tracking 9-3
Information About Configuring Linked Ports
Binding a Tracked Port Operationally 9-4
9-3
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
9
Contents
Information About Tracking Multiple Ports 9-4
Tracking Multiple Ports 9-5
Information About Monitoring Ports in a VSAN 9-5
Monitoring Ports in a VSAN 9-5
Information AboutForceful Shutdown 9-6
Forcefully Shutting Down a Tracked Port 9-6
Displaying Port Tracking Information
9-6
9-8
I NDEX
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
10
OL-29284-01, Release 6.x
New and Changed Information
As of Cisco MDS NX-OS Release 4.2(1), software configuration information is available in new
feature-specific configuration guides for the following information:
•
System management
•
Interfaces
•
Fabric
•
Quality of service
•
Security
•
IP services
•
High availability and redundancy
The information in these new guides previously existed in the Cisco MDS 9000 Family CLI
Configuration Guide and in the Cisco MDS 9000 Family Fabric Manager Configuration Guide. Those
configuration guides remain available on Cisco.com and should be used for all software releases prior
to Fabric Manager Release 5.0(1a). Each guide addresses the features introduced in or available in a
particular release. Select and view the configuration guide that pertains to the software installed in your
switch.
For a complete list of document titles, see the list of Related Documentation in the “Preface.”
http://www.cisco.com/en/US/products/ps5989/prod_release_notes_list.htm
About This Guide
Table 1 lists the New and Changed features for this guide, starting with MDS NX-OS Release 6.2(9).
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01
-13
Table 1
New and Changed Features
Feature
Description
Warning Threshold and
Check Interval
Added two enhancements to the
port monitor feature:
Changed
in Release Where Documented
6.2(15)
Chapter 3, “Configuring
Interfaces”
6.2(13)
Chapter 3, “Configuring
Interfaces”
Internal CRC Detection
and Isolation
6.2(13)
Added information about CRC
detection and isolation, to detect
CRC errors and isolate the source of
these errors originating within the
switch.
Chapter 3, “Configuring
Interfaces”
MAX NPIV Limit
Maximum NPIV limit functionality 6.2(9)
is not supported currently for
interfaces connecting NPIV core to
NPV switches has been removed.
Chapter 3, “Configuring
Interfaces”
Slow drain enhancement to Added information about the
no-credit timeout value and
leverage HW based
slow-port-monitor timeout value.
capabilities
6.2(9)
Chapter 3, “Configuring
Interfaces”
Enable Forward Error
Correction (FEC) for 16
Gbps ISL connections
6.2(7)
Chapter 2, “Configuring Fibre
Channel Interfaces”
Limit the number of NPIV Added information about MAX
NPIV Limit.
logins to allow customer
not to exceed known scale
limits on MDS 9513
6.2(7)
Chapter 3, “Configuring
Interfaces”
MDS 9710
6.2(1)
•
A new warning threshold is
introduced to allow syslog
notification for lower level
events than the existing alert
level.
•
Both warnings and alerts can be
triggered before the full poll
interval expires.
Slow drain enhancements Added three new counters to the
with slowport monitoring Port monitor policy.
supported on 8-Gbps and
advanced 8-Gbps modules.
Added information about Forward
Error Correction (FEC).
Added a note.
Chapter 1, “Interfaces
Overview”
(DS-X9448-768k9)
Port Monitor
Enhancements
5.2(2a)
Added information about the
feature Port Monitor Port Guard and
three new counters for the port
monitor command.
Chapter 4, “Configuring
Interface Buffers”
Chapter 3, “Configuring
Interfaces”
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
-14
OL-29284-01
Table 1
New and Changed Features (continued)
Changed
in Release Where Documented
Feature
Description
FlexAttach
Added information about disabling 5.0(1a)
FlexAttach.
Chapter 8, “Configuring
FlexAttach Virtual pWWN”
Port Group Monitoring
Enhancements
Added information about
monitoring a selected port group.
Chapter 3, “Configuring
Interfaces”
5.0(1a)
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01
-15
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
-16
OL-29284-01
Preface
This preface describes the audience, organization, and conventions of the Cisco MDS 9000 Family
NX-OS Interfaces Configuration Guide. It also provides information on how to obtain related
documentation.
Audience
This guide is for experienced network administrators who are responsible for configuring and
maintaining the Cisco MDS 9000 Family of multilayer directors and fabric switches.
Organization
The Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide is organized as follows:
Chapter
Title
Description
Chapter 1
Interfaces Overview
Provides an overview of all the features in this
guide.
Chapter 3
Configuring Interfaces
Explains Generation 1 and Generation 2
module port and operational state concepts in
Cisco MDS 9000 Family switches and provides
details on configuring ports and interfaces.
Chapter 2
Configuring Fibre Channel Interfaces Explains configuration concepts for Fibre
Channel module ports and interfaces.
Chapter 4
Configuring Interface Buffers
Explains configuration concepts for Interface
Buffers.
Chapter 5
Configuring Trunking
Explains TE ports and trunking concepts.
Chapter 6
Configuring PortChannels
Explains PortChannels and load balancing
concepts and provides details on configuring
PortChannels, adding ports to PortChannels,
and deleting ports from PortChannels.
Cisco MDS 9000 Family I/O Accelerator Configuration Guide
OL-29284-01
-xv
Chapter
Title
Description
Chapter 7
Configuring N Port Virtualization
Provides an overview of N Port Virtualization
and includes quidelines and requirements for
configuring and verifying NPV.
Chapter 8
Configuring FlexAttach Virtual
pWWN
FlexAttach virtual pWWN feature facilitates
server and configuration management. In a
SAN environment, the server installation or
replacement, requires interaction and
coordination among the SAN and server
administrators.
Document Conventions
Command descriptions use these conventions:
boldface font
Commands and keywords are in boldface.
italic font
Arguments for which you supply values are in italics.
[ ]
Elements in square brackets are optional.
[x|y|z]
Optional alternative keywords are grouped in brackets and separated by
vertical bars.
Screen examples use these conventions:
screen font
Terminal sessions and information the switch displays are in screen font.
boldface screen font
Information you must enter is in boldface screen font.
italic screen font
Arguments for which you supply values are in italic screen font.
< >
Nonprinting characters, such as passwords, are in angle brackets.
[ ]
Default responses to system prompts are in square brackets.
!, #
An exclamation point (!) or a pound sign (#) at the beginning of a line of code
indicates a comment line.
This document uses the following conventions:
Note
Caution
Means reader take note. Notes contain helpful suggestions or references to material not covered in the
manual.
Means reader be careful. In this situation, you might do something that could result in equipment
damage or loss of data.
Cisco MDS 9000 Family I/O Accelerator Configuration Guide
-xvi
OL-29284-01
Related Documentation
The documentation set for the Cisco MDS 9000 Family includes the following documents. To find a
document online, use the Cisco MDS NX-OS Documentation Locator at:
http://www.cisco.com/en/US/docs/storage/san_switches/mds9000/roadmaps/doclocater.htm
Release Notes
•
Cisco MDS 9000 Family Release Notes for Cisco MDS NX-OS Releases
•
Cisco MDS 9000 Family Release Notes for MDS SAN-OS Releases
•
Cisco MDS 9000 Family Release Notes for Cisco MDS 9000 EPLD Images
•
Cisco DCNM Release Notes
Regulatory Compliance and Safety Information
•
Regulatory Compliance and Safety Information for the Cisco MDS 9000 Family
Compatibility Information
•
Cisco Data Center Interoperability Support Matrix
•
Cisco MDS 9000 NX-OS Hardware and Software Compatibility Information and Feature Lists
•
Cisco MDS 9000 Family Switch-to-Switch Interoperability Configuration Guide
Hardware Installation
•
Cisco MDS 9500 Series Hardware Installation Guide
•
Cisco MDS 9200 Series Hardware Installation Guide
•
Cisco MDS 9100 Series Hardware Installation Guide
•
Cisco MDS 9124 and Cisco MDS 9134 Multilayer Fabric Switch Quick Start Guide
Software Installation and Upgrade
•
Cisco MDS 9000 NX-OS Software Upgrade and Downgrade Guide
•
Cisco MDS 9000 Family NX-OS Licensing Guide
•
Cisco MDS 9000 Family NX-OS Fundamentals Configuration Guide
•
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
•
Cisco MDS 9000 Family NX-OS Fabric Configuration Guide
Cisco NX-OS
Cisco MDS 9000 Family I/O Accelerator Configuration Guide
OL-29284-01
-xvii
•
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
•
Cisco MDS 9000 Family NX-OS Security Configuration Guide
•
Cisco MDS 9000 Family NX-OS IP Services Configuration Guide
•
Cisco MDS 9000 Family NX-OS Intelligent Storage Services Configuration Guide
•
Cisco MDS 9000 Family NX-OS High Availability and Redundancy Configuration Guide
•
Cisco MDS 9000 Family NX-OS Inter-VSAN Routing Configuration Guide
•
Cisco MDS 9000 Family Cookbook for Cisco MDS SAN-OS
Command-Line Interface
•
Cisco MDS 9000 Family Command Reference
Intelligent Storage Networking Services Configuration Guides
•
Cisco MDS 9000 Family I/O Acceleration Configuration Guide
•
Cisco MDS 9000 Family SANTap Deployment Guide
•
Cisco MDS 9000 Family Data Mobility Manager Configuration Guide
•
Cisco MDS 9000 Family Storage Media Encryption Configuration Guide
Troubleshooting and Reference
•
Cisco MDS 9000 Family and Nexus 7000 Series System Messages Reference
•
Cisco MDS 9000 Family SAN-OS Troubleshooting Guide
•
Cisco MDS 9000 Family NX-OS MIB Quick Reference
•
Cisco DCNM for SAN Database Schema Reference
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, submitting a service request, and gathering additional
information, see the monthly What’s New in Cisco Product Documentation, which also lists all new and
revised Cisco technical documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
•
Subscribe to the What’s New in Cisco Product Documentation as a Really Simple Syndication (RSS)
feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds
are a free service and Cisco currently supports RSS version 2.0.
Cisco MDS 9000 Family I/O Accelerator Configuration Guide
-xviii
OL-29284-01
CHAPTER
1
Interfaces Overview
•
Trunks and PortChannels, page 1-xix
•
Fibre Channel Port Rate Limiting, page 1-xix
•
Maximum NPIV Limit, page 1-xx
•
Extended Credits, page 1-xx
•
N Port Virtualization, page 1-xx
•
FlexAttach, page 1-xxi
Trunks and PortChannels
Trunking, also known as VSAN trunking, is a feature specific to switches in the Cisco MDS 9000
Family. Trunking enables interconnect ports to transmit and receive frames in more than one VSAN, over
the same physical link. Trunking is supported on E ports and F ports.
PortChannels aggregate multiple physical ISLs into one logical link with higher bandwidth and port
resiliency for both Fibre Channel and FICON traffic. With this feature, up to 16 expansion ports
(E-ports) or trunking E-ports (TE-ports) can be bundled into a PortChannel. ISL ports can reside on any
switching module, and they do not need a designated master port. If a port or a switching module fails,
the PortChannel continues to function properly without requiring fabric reconfiguration.
Cisco NX-OS software uses a protocol to exchange PortChannel configuration information between
adjacent switches to simplify PortChannel management, including misconfiguration detection and
autocreation of PortChannels among compatible ISLs. In the autoconfigure mode, ISLs with compatible
parameters automatically form channel groups; no manual intervention is required.
PortChannels load balance Fibre Channel traffic using a hash of source FC-ID and destination FC-ID,
and optionally the exchange ID. Load balancing using PortChannels is performed over both Fibre
Channel and FCIP links. Cisco NX-OS software also can be configured to load balance across multiple
same-cost FSPF routes.
Fibre Channel Port Rate Limiting
The Fibre Channel port rate-limiting feature for the Cisco MDS 9100 Series controls the amount of
bandwidth available to individual Fibre Channel ports within groups of four host-optimized ports.
Limiting bandwidth on one or more Fibre Channel ports allows the other ports in the group to receive a
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
1-xix
Chapter 1
Interfaces Overview
Maximum NPIV Limit
greater share of the available bandwidth under high-utilization conditions. Port rate limiting is also
beneficial for throttling WAN traffic at the source to help eliminate excessive buffering in Fibre Channel
and IP data network devices.
Maximum NPIV Limit
The maximum number of NPIV logins is not configurable at the port level on edge switches operating
in NPV mode. The maximum NPIV limit feature was introduced in Cisco MDS 9000 Release 5.2(8d)
for the 5.2(x) releases, and 6.2(7) for the 6.2(x) releases on core NPIV switches, which include Cisco
MDS 9513, MDS 9710, and MDS 9250i switches. The maximum NPIV limit feature allows you to
configure a per interface limit. If a maximum limit is configured, whenever FDISCs are received, it
checks if the maximum NPIV limit is exceeded, then it will reject the FDISCs. The reason code is
displayed as rjt_reason.reason_code = R_UNABLE; rjt_reason.reason_code_expl =
EX_NORESOURCES. If the maximum NPIV limit is not exceeded, it will process the FDISCs. The
trunk-max-npiv-limit command is used for F ports in trunking mode with multiple VSANs.
Note
The trunk-max-npiv-limit command is applied when trunk is enabled and it is per entire trunk.
Extended Credits
Full line-rate Fibre Channel ports provide at least 255 standard buffer credits . Adding credits lengthens
distances for the Fibre Channel SAN extension. Using extended credits, up to 4095 buffer credits from
a pool of more than 6000 buffer credits for a module can be allocated to ports as needed to greatly extend
the distance for Fibre Channel SANs.
Note
This feature is supported on all Cisco MDS Director Class Fabric Switches and it is not supported on
any Cisco MDS Fabric switches.
N Port Virtualization
Cisco NX-OS software supports industry-standard N port identifier virtualization (NPIV), which allows
multiple N port fabric logins concurrently on a single physical Fibre Channel link. HBAs that support
NPIV can help improve SAN security by enabling zoning and port security to be configured
independently for each virtual machine (OS partition) on a host. In addition to being useful for server
connections, NPIV is beneficial for connectivity between core and edge SAN switches.
N port virtualizer (NPV) is a complementary feature that reduces the number of Fibre Channel domain
IDs in core-edge SANs. Cisco MDS 9000 family fabric switches operating in the NPV mode do not join
a fabric; they only pass traffic between core switch links and end devices, which eliminates the domain
IDs for these switches. NPIV is used by edge switches in the NPV mode to log in to multiple end devices
that share a link to the core switch. This feature is available only for Cisco MDS Blade Switch Series,
the Cisco MDS 9124 Multilayer Fabric Switch, Cisco MDS 9134 Multilayer Fabric Switch, Cisco MDS
9148 Multilayer Fabric Switch, Cisco MDS 9148S Multilayer Fabric Switch, and Cisco MDS 9396S
Multilayer Fabric Switch.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
1-xx
OL-29284-01, Release 6.x
Chapter 1
Interfaces Overview
FlexAttach
FlexAttach
One of the main problems in a SAN environment is the time and effort required to install and replace
servers. The process involves both SAN and server administrators, and the interaction and coordination
between them can make the process time consuming. To alleviate the need for interaction between SAN
and server administrators, the SAN configuration should not be changed when a new server is installed
or an existing server is replaced. FlexAttach addresses these problems by reducing configuration
changes and the time and coordination required by SAN and server administrators when installing and
replacing servers. This feature is available only for Cisco MDS 9000 Blade Switch Series, the Cisco
MDS 9124, Cisco MDS 9134, Cisco MDS 9148 Multilayer Fabric Switch, Cisco MDS 9148S Multilayer
Fabric Switch, and Cisco MDS 9396S switches when NPV mode is enabled.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
1-xxi
Chapter 1
Interfaces Overview
FlexAttach
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
1-xxii
OL-29284-01, Release 6.x
CHAPTER
2
Configuring Fibre Channel Interfaces
•
Information About Fibre Channel Interfaces, page 2-1
•
Guidelines and Limitations, page 2-18
•
Default Settings, page 2-24
•
Configuring Fibre Channel Interfaces, page 2-25
•
Verifying Fibre Channel Interfaces Configuration, page 2-45
•
Configuration Examples for Fibre Channel Interfaces, page 2-47
Information About Fibre Channel Interfaces
This section includes the following topics:
•
Generations of Modules and Switches, page 2-1
•
Port Groups, page 2-3
•
Port Rate Modes, page 2-5
•
Port Speed, page 2-9
•
Dynamic Bandwidth Management, page 2-10
•
Out-of-Service Interfaces, page 2-11
•
Oversubscription Ratio Restrictions, page 2-11
•
Bandwidth Fairness, page 2-17
Generations of Modules and Switches
Cisco MDS 9000 Family hardware modules and switches are categorized into generations based on the
time of introduction, capabilities, features, and compatibilities:
•
Generation 1—Modules and switches with a maximum port speed of 2 Gbps.
•
Generation 2—Modules and switches with a maximum port speed of 4 Gbps.
•
Generation 3—Modules and switches with a maximum port speed of 8 Gbps.
•
Generation 4—Modules with a maximum port speed of 8-Gbps or 10-Gbps.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-1
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
The Cisco MDS 9500 Series switches, Cisco MDS 9222i, Cisco MDS 9216A, and Cisco MDS 9216i
switches support the Generation 2 modules. Each module or switch can have one or more ports in port
groups that share common resources such as bandwidth and buffer credits.
In addition to supporting Generation 2 modules, the Cisco MDS 9500 Series switches and the Cisco
MDS 9222i switch support the Generation 3 modules. Similar to Generation 2, each Generation 3 or
Generation 4 module can have one or more ports in port groups that share common resources such as
bandwidth and buffer credits.
Generation 3 modules are supported on the Cisco MDS 9506 and 9509 switches with Supervisor-2
modules. The MDS 9513 Director supports 4/44-port Host-Optimized Fibre Channel switching module
with either Fabric 1 or Fabric 2 modules, but requires Fabric 2 module for support of the 48-port and the
24-port 8-Gbps Fibre Channel switching modules. The MDS 9222i switch supports the 4/44-port
Host-Optimized Fibre Channel switching module.
The Cisco 9500 Series switches support the following Generation 4 modules: the 48-port 8-Gbps
Advanced Fibre Channel switching module (DS-X9248-256K9) and the 32-port 8-Gbps Advanced Fibre
Channel module (DS-X9232-256K9). Cisco MDS NX-OS Release 5.2(1) or higher is required to support
the Generation 4 modules.
Table 2-1 identifies the Generation 2, Generation 3, and Generation 4 modules, as well as the Fabric
switches.
Table 2-1
Fibre Channel Modules and Fabric Switches
Part Number
Product Name and Description
Generation 4 Modules
DS-X9248-256K9
48-port 8-Gbps Advanced Fibre Channel switching module.
DS-X9232-256K9
32-port 8-Gbps Advanced Fibre Channel switching module.
DS-X9530-SF2A-K9
Supervisor-2A module for Cisco MDS 9500 Series switches.
DS-13SLT-FAB3
Fabric 3 module that enables the 32-port and the 48-port 8-Gbps Advanced
Fibre Channel switching module to use the full 96-Gbps or 256-Gbps
backplane crossbar bandwidth.
Generation 3 Modules
DS-X9248-96K9
48-port 8-Gbps Fibre Channel switching module.
DS-X9224-96K9
24-port 8-Gbps Fibre Channel switching module.
DS-X9248-48K9
4/44-port 8-Gbps Host-Optimized Fibre Channel switching module
DS-13SLT-FAB2
Fabric 2 module that enables the 24-port and the 48-port 8-Gbps Fibre Channel
switching module to use the full 96-Gbps backplane bandwidth with
any-to-any connectivity.
Generation 3 Fabric Switch
DS-C9148-K9
Cisco MDS 9148 Fabric switch.
48-port 8-Gbps Fabric switch.
Generation 2 Modules
DS-X9148
48-port 4-Gbps Fibre Channel switching module.
DS-X9124
24-port 4-Gbps Fibre Channel switching module.
DS-X9304-18K9
18-port 4-Gbps Fibre Channel switching module with 4-Gigabit Ethernet
ports.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-2
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Table 2-1
Fibre Channel Modules and Fabric Switches (continued)
Part Number
Product Name and Description
DS-X9112
12-port 4-Gbps Fibre Channel switching module.
DS-X9704
4-port 10-Gbps Fibre Channel switching module.
DS-X9530-SF2-K9
Supervisor-2 module for Cisco MDS 9500 Series switches.
Generation 2 Fabric Switches
DS-C9134-K9
Cisco MDS 9134 Fabric switch.
32-port 4-Gbps Fabric switch with 2 additional 10-Gbps ports.
DS-C9124-K9
Cisco MDS 9124 Fabric switch.
24-port 4-Gbps Fabric switch.
DS-C9222i-K9
Cisco MDS 9222i Multiservice Modular switch.
18-port 4-Gbps switch with 4-Gigabit Ethernet IP storage services ports, and a
modular expansion slot to host Cisco MDS 9000 Family switching and services
modules.
Note
Generation 2 Fibre Channel switching modules are not supported on the Cisco MDS 9216 switch;
however, they are supported by both the Supervisor-1 module and the Supervisor-2 module.
For detailed information about the installation and specifications for these modules and switches, refer
to the hardware installation guide for your switch.
Port Groups
Each module or switch can have one or more ports in port groups that share common resources such as
bandwidth and buffer credits. Port groups are defined by the hardware consisting of sequential ports. For
example, ports 1 through 6, ports 7 through 12, ports 13 through 18, ports 19 through 24, ports 25
through 30, 31 through 36, and ports 37 through 42, 43 through 48 are the port groups on the 48-port
8-Gbps Advanced Fibre Channel switching modules.
Table 2-2 shows the bandwidth and number of ports per port group for the Generation 2, Generation 3,
and Generation 4 Fibre Channel modules, and Generation 2 and Generation 3 Fabric switches.
Table 2-2
Bandwidth and Port Groups for the Fibre Channel Modules and Fabric Switches
Part Number
Product Name/ Description
Number of Ports
Per Port Group
Bandwidth
Per Port
Group (Gbps)
6
32.41 or 12.82
Maximum Bandwidth Per
Port (Gbps)
Generation 4 Modules
DS-X9248-256K9
48-port 8-Gbps Advanced
Fibre Channel switching
module.
8 or 10 Gbps—depending on
the configuration
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-3
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Table 2-2
Bandwidth and Port Groups for the Fibre Channel Modules and Fabric Switches (continued)
Number of Ports
Per Port Group
Bandwidth
Per Port
Group (Gbps)
Maximum Bandwidth Per
Port (Gbps)
Part Number
Product Name/ Description
4
32.43 or 12.84
8 or 10 Gbps—depending on
the configuration
DS-X9232-256K9
32-port 8-Gbps Advanced
Fibre Channel switching
module.
DS-X9248-96K9
48-port 8-Gbps Fibre
Channel switching module
6
12.8
8 Gbps
DS-X9224-96K9
24-port 8-Gbps Fibre
Channel switching module
3
12.8
8 Gbps
DS-X9248-48K9
4/44-port 8-Gbps
Host-Optimized Fibre
Channel switching module
12
12.8
8/4 Gbps5
48-port 8-Gbps Fabric switch 4
32
8 Gbps
DS-X9148
48-port 4-Gbps Fibre
Channel switching module
12
12.8
4 Gbps
DS-X9124
24-port 4-Gbps Fibre
Channel switching module
6
12.8
4 Gbps
DS-X9304-18K9
18-port 4-Gbps Fibre
6
Channel switching module
with 4-Gigabit Ethernet ports
12.8
4 Gbps
DS-X9112
12-port 4-Gbps Fibre
Channel switching module
3
12.8
4 Gbps
DS-X9704
4-port 10-Gbps Fibre
Channel switching module
1
10
10 Gbps
DS-C9134-K9
32-port 4-Gbps Fabric switch 4
16
4 Gbps
(Cisco MDS 9134
Fabric switch)
2-port 10-Gbps Fabric switch 1
10
10 Gbps
DS-C9124K9 (Cisco
MDS 9124 Fabric
switch)
24-port 4-Gbps Fabric switch 4
16
4 Gbps
DS-C9222i-K9
18-port 4-Gbps, 4 Gigabit
6
Ethernet ports and a modular
expansion slot.
12.8
4 Gbps
Generation 3 Modules
Generation 3 Fabric Switches
DS-C9148-K9
(Cisco MDS 9148
Fabric switch)
Generation 2 Modules
(MSM-18/4
Multiservice module)
Generation 2 Fabric Switches
(Cisco MDS 9222i
Multiservice Modular
switch)
1. This bandwidth is available with the Fabric 3 module (DS-13SLT-FAB3) in the MDS 9513 switch.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-4
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
2. This bandwidth is available with the Fabric 2 module (DS-13SLT-FAB2) in the MDS 9513 switch, and with the Supervisor-2 (DS-X9530-SF2-K9) or
Supervisor-2A module (DS-X9530-SF2AK9) in the MDS 9509 switch or MDS 9506 switch.
3. This bandwidth is available with the Fabric 3 module (DS-13SLT-FAB3) in the MDS 9513 switch.
4. This bandwidth is available with the Fabric 2 module (DS-13SLT-FAB2) in the MDS 9513 switch, and with the Supervisor-2 (DS-X9530-SF2-K9) or
Supervisor-2A module (DS-X9530-SF2AK9) in the MDS 9509 switch or MDS 9506 switch.
5. A maximum of four ports (one per port group) in a 4/44-port 8-Gbps switching module can operate at 8-Gbps bandwidth in dedicated or shared mode.
All the other ports can operate at a maximum of 4-Gbps in shared mode or dedicated mode.
Port Rate Modes
In Generation 2, Generation 3, and Generation 4 modules, you can configure the port rate modes. The
port rate mode configuration is used to determine the bandwidth allocation for ports in a port group. Two
port rate modes are supported:
•
Dedicated Rate Mode—A port is allocated required fabric bandwidth to sustain line traffic at the
maximum operating speed configured on the port. For more information, see the “Dedicated Rate
Mode” section on page 2-7.
•
Shared Rate Mode—Multiple ports in a port group share data paths to the switch fabric and share
bandwidth. For more information, see the “Shared Rate Mode” section on page 2-8.
Note
In Generation 1 modules, you cannot configure the port rate modes. The mode is determined implicitly
based on the port mode and line card type.
Note
Port rate modes are not supported on the Cisco Fabric Switch for HP c-Class Blade System, and the
Cisco Fabric Switch for IBM Blade Center.
Table 2-3 shows the modules that support dedicated, shared, and the default rate modes.
Table 2-3
Port Rate Mode Support on Generation 2, Generation 3, and Generation 4 Modules and Switches
Part Number
Supports
Dedicated
Rate Mode
Supports
Shared
Rate Mode
Default Speed
Mode and Rate
Mode on All Ports
48-port 8-Gbps Advanced
Fibre Channel switching
module
Yes
Yes
Auto, Shared
32-port 8-Gbps Advanced
Fibre Channel switching
module
Yes
Yes
Auto, Shared
Product Name/
Description
Generation 4 Modules
DS-X9248-256K9
DS-X9232-256K9
Generation 3 Modules
DS-X9248-96K9
48-Port 8-Gbps Fibre
Channel switching module
Yes
Yes
Auto, Shared
DS-X9224-96K9
24-Port 8-Gbps Fibre
Channel switching module
Yes
Yes
Auto, Shared
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-5
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Table 2-3
Port Rate Mode Support on Generation 2, Generation 3, and Generation 4 Modules and Switches
Part Number
DS-X9248-48K9
Product Name/
Description
4/44-Port 8-Gbps
Host-Optimized Fibre
Channel switching module
Supports
Dedicated
Rate Mode
Supports
Shared
Rate Mode
Default Speed
Mode and Rate
Mode on All Ports
Yes
Yes
Auto Max 4 Gbps,
Shared
Yes
No
Auto, Dedicated
Generation 3 Fabric Switches
DS-C9148-K9
(Cisco MDS 9148
Fabric switch)
48-port 8-Gbps Fabric
switch
Generation 2 Modules
DS-X9148
48-port 4-Gbps Fibre
Channel switching module
Yes
Yes
Auto, Shared
DS-X9124
24-port 4-Gbps Fibre
Channel switching module
Yes
Yes
Auto, Shared
DS-X9304-18K9
18-port 4-Gbps Fibre
Channel switching module
with 4-Gigabit Ethernet
ports
Yes
Yes
Auto, Shared
DS-X9112
12-port 4-Gbps Fibre
Channel switching module
Yes
No
Auto, Dedicated
DS-X9704
4-port 10-Gbps Fibre
Channel switching module
Yes
No
Auto, Dedicated
32-port 4-Gbps Fabric
switch
Yes
Yes
Auto, Shared
2-port 10-Gbps Fabric
switch
Yes
No
Auto, Dedicated
24-port 4-Gbps Fabric
switch
Yes
No
Auto, Dedicated
Yes
18-port 4-Gbps Fibre
Channel switch with
4-Gigabit Ethernet IP
storage services ports, and
a modular expansion slot to
host Cisco MDS 9000
Family Switching and
Services Modules
Yes
Auto, Shared
(MSM-18/4
Multiservice
module)
Generation 2 Fabric Switches
DS-C9134-K9
(Cisco MDS 9134
Fabric switch)
DS-C9124-K9
(Cisco MDS 9124
Fabric switch)
DS-C9222i-K9
(Cisco MDS 9222i
Multiservice
Modular switch)
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-6
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Dedicated Rate Mode
When port rate mode is configured as dedicated, a port is allocated required fabric bandwidth and related
resources to sustain line rate traffic at the maximum operating speed configured for the port. In this
mode, ports do not use local buffering and all receive buffers are allocated from a global buffer pool.
Table 2-4 shows the bandwidth provided by the various port speed configurations on the 8-Gbps
Advanced Fibre Channel switching modules.
Table 2-4
Bandwidth Reserved for the Port Speeds on Generation 4 Switching Modules
Configured Speed
Reserved Bandwidth
Auto
8 Gbps
8-Gbps
Auto with 4-Gbps maximum
4 Gbps
4-Gbps
Auto with 2-Gbps maximum
2 Gbps
2-Gbps
1-Gbps
1 Gbps
Table 2-5 shows the bandwidth provided by the various port speed configurations on the 8-Gbps Fibre
Channel switching modules.
Table 2-5
Bandwidth Reserved for the Port Speeds on Generation 3 Switching Modules
Configured Speed
Reserved Bandwidth
Auto
8 Gbps
8-Gbps
Auto with 4-Gbps maximum
4 Gbps
4-Gbps
Auto with 2-Gbps maximum
2 Gbps
2-Gbps
1-Gbps
1 Gbps
Table 2-6 shows the amount of bandwidth reserved for a configured port speed on 4-Gbps switching
modules.
Table 2-6
Bandwidth Reserved for the Port Speeds on Generation 2 Switching Modules
Configured Speed
Reserved Bandwidth
Auto
4 Gbps
4-Gbps
Auto with 2-Gbps maximum
2 Gbps
2-Gbps
1-Gbps
1 Gbps
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-7
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Note
The 4-Port 10-Gbps Fibre Channel module ports in auto mode only support auto speed mode at 10 Gbps.
Shared Rate Mode
When port rate mode is configured as shared, multiple ports within a port group share data paths to the
switch fabric so that fabric bandwidth and related resources are shared. Often, the available bandwidth
to the switch fabric may be less than the negotiated operating speed of a port. Ports in this mode use local
buffering for the BB_credit buffers.
All ports in 8-Gbps Advanced Fibre Channel switching modules where bandwidth is shared support
1-Gbps, 2-Gbps, 4-Gbps, or 8 Gbps traffic. However, it is possible to configure one or more ports in a
port group to operate in dedicated rate mode with 1-Gbps, 2-Gbps, 4-Gbps, or 8 Gbps operating speed.
All ports in 4-Gbps Fibre Channel switching modules where bandwidth is shared support 1-Gbps,
2-Gbps, or 4-Gbps traffic. However, it is possible to configure one or more ports in a port group to
operate in dedicated rate mode with 1-Gbps, 2-Gbps, or 4-Gbps operating speed.
All ports in the 32-Port or 48-Port 8-Gbps Advanced Fibre Channel modules where bandwidth is shared
support 1-Gbps, 2-Gbps, 4-Gbps, or 8-Gbps traffic in a maximum or 32 or 48 ports.
All ports in the 48-Port and 24-Port 8-Gbps Fibre Channel switching modules where bandwidth is shared
support 1-Gbps, 2-Gbps, 4-Gbps, or 8-Gbps traffic.
In the 4/44-Port 8-Gbps Host-Optimized Fibre Channel switching module, all the ports where bandwidth
is shared support 1-Gbps, 2-Gbps, 4-Gbps in a maximum of 44 ports, or 8 Gbps in a maximum of 4 ports.
Dedicated Rate Mode Configurations for the 8-Gbps Modules
Table 2-7 shows the maximum possible dedicated rate mode configuration scenarios for the Generation
4 Fibre Channel modules.
Table 2-7
Dedicated Rate Mode Bandwidth Reservation for Generation 4 Fibre Channel Modules
Part Number
DS-X9248-256K9
DS-X9232-256K9
Product Name/
Description
48-port 8-Gbps
Advanced Fibre
Channel switching
module
32-port 8-Gbps
Advanced Fibre
Channel switching
module
Maximum
Dedicated Allowed Ports
Bandwidth That Can Come
Up
per Port
10 Gbps
24 Ports
8 Gbps
32 Ports
4 Gbps
48 Ports
2 Gbps
48 Ports
1 Gbps
48 Ports
10 Gbps
24 Ports
8 Gbps
32 Ports
4 Gbps
32 Ports
2 Gbps
32 Ports
1 Gbps
32 Ports
Ports in Shared Mode
All the remaining ports
are 8 Gbps shared.
All the remaining ports
are 8 Gbps shared.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-8
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Table 2-8 shows the maximum possible dedicated rate mode configuration scenarios for the Generation
3 Fibre Channel modules.
Table 2-8
Dedicated Rate Mode Bandwidth Reservation for Generation 3 Fibre Channel Modules
Part Number
DS-X9224-96K9
DS-X9248-96K9
DS-X9248-48K9
Product Name/
Description
Maximum
Dedicated Allowed Ports
Bandwidth That Can Come
Up
per Port
24-port 8-Gbps
Fibre Channel
switching module
8 Gbps
8 Ports
4 Gbps
24 Ports
48-port 8-Gbps
Fibre Channel
switching module
8 Gbps
8 Ports
4 Gbps
24 Ports
2 Gbps
48 Ports
4/44-port 8-Gbps
Host-Optimized
Fibre Channel
switching module
8 Gbps
4 Ports
4 Gbps
12 Ports
2 Gbps
24 Ports
1 Gbps
48 Ports
Ports in Shared Mode
All the remaining ports
are 8 Gbps shared.
All the remaining ports
are 8 Gbps shared.
All the remaining ports
are 4 Gbps shared (8 Gbps
of bandwidth can be
provided only to one port
per port group in
dedicated or shared rate
mode).
Port Speed
The port speed on an interface, combined with the rate mode, determines the amount of shared resources
available to the ports in the port group on a 48-port, 24-port 4-Gbps, or any 8-Gbps Fibre Channel
switching module. With the dedicated rate mode, the port group resources are reserved even though the
bandwidth is not used. For example, on Generation 2 modules, if an interface is configured for
autosensing (auto) and dedicated rate mode, then 4 Gbps of bandwidth is reserved even though the
maximum operating speed is 2 Gbps. For the same interface, if autosensing with a maximum speed of 2
Gbps (auto max 2000) is configured, then only 2 Gbps of bandwidth is reserved and the unused 2 Gbps
is shared with the other interface in the port group.
Note
•
The Generation 2, 4-port 10-Gbps switching module supports 10-Gbps traffic only.
•
On Generation 2, 4-Gbps modules, setting the port speed to auto enables autosensing, which
negotiates to a maximum speed of 4 Gbps.
•
On Generation 3, 8-Gbps modules, setting the port speed to auto enables autosensing, which
negotiates to a maximum speed of 8 Gbps.
•
On Generation 4, 8-Gbps modules, setting the port speed to auto enables autosensing, which
negotiates to a maximum speed of 8 Gbps.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-9
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Forward Error Correction
The Transmitter Training Signal (TTS) is defined in FC-FS-4(Clause 5.5). It provides the capability for
FC ports to negotiate the following two capabilities:
1.
Transmitter training, which enables a receiver to send feedback to a transmitter to assist the
transmitter in adapting to the characteristics of the link that connects them.
2.
FEC usage.
The TTS is not used by 4 and 8-gigabit FC ports. From 32-gigabit speed and higher, its use is mandatory.
For 16-gigabit FC ports, EA variants must transmit the TTS during the link speed negotiation, but the
use of it by the receiver is optional, and EL variants must not use TTS.
Forward Error Correction (FEC) is defined in IEEE 802.3TM clause 74 and is implemented in FC
without modification. FEC is not supported on 4, 8 and 16-gigabit EL ports. Its use is optional on
16-gigabit EA ports. The TTS is the mechanism that allows FEC negotiation on such ports.
For more information on configuring FEC and TTS, see the Configuring FEC section.
Dynamic Bandwidth Management
On port switching modules where bandwidth is shared, the bandwidth available to each port within a port
group can be configured based on the port rate mode and speed configurations. Within a port group, some
ports can be configured in dedicated rate mode while others operate in shared mode.
Ports configured in dedicated rate mode are allocated the required bandwidth to sustain a line rate of
traffic at the maximum configured operating speed, and ports configured in shared mode share the
available remaining bandwidth within the port group. Bandwidth allocation among the shared mode
ports is based on the operational speed of the ports. For example, if four ports operating at speeds 1 Gbps,
1 Gbps, 2 Gbps, and 4 Gbps share bandwidth of 8 Gbps, the ratio of allocation would be 1:1:2:4.
Unutilized bandwidth from the dedicated ports is shared among only the shared ports in a port group as
per the ratio of the configured operating speed. A port cannot be brought up unless the reserved
bandwidth is quaranteed for the shared ports (see Table 2-9). For dedicated ports, configured bandwidth
is taken into consideration while calculating available bandwidth for the port group. This behavior can
be changed using bandwidth fairness by using the rate-mode bandwidth-fairness module number
command.
For example, consider a 48-port 8-Gbps module. This module has 6 ports per port group with 12.8 Gbps
bandwidth. Ports 3 to 6 are configured at 4 Gbps. If the first port is configured at 8 Gbps dedicated rate
mode, and the second port is configured at 4-Gbps dedicated rate mode, then no other ports can be
configured at 4 Gbps or 8 Gbps because the left over bandwidth of 0.8 Gbps (12.8-(8+4)) cannot meet
the required 0.96 Gbps for the remaining four ports. A minimum of 0.24 Gbps reserved bandwidth is
required for the for the rest of the four ports. However, if the two ports (for example, 5 and 6) are taken
out of service (which is not same as shutdown), required reserved bandwidth for the two ports (3 and 4)
is 0.48 and port 2 can be configured at 4 Gbps in dedicated rate mode. This behavior can be overridden
by the bandwidth fairness command in which case reserved bandwidth is not enforced. Once the port is
up, ports 3 and 4 can share the unutilized bandwidth from ports 1 and 2.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-10
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Out-of-Service Interfaces
On supported modules and fabric switches, you might need to allocate all the shared resources for one
or more interfaces to another interface in the port group or module. You can take interfaces out of service
to release shared resources that are needed for dedicated bandwidth. When an interface is taken out of
service, all shared resources are released and made available to the other interface in the port group or
module. These shared resources include bandwidth for the shared mode port, rate mode, BB_credits, and
extended BB_credits. All shared resource configurations are returned to their default values when the
interface is brought back into service. Corresponding resources must be made available in order for the
port to be successfully returned to service.
Caution
If you need to bring an interface back into service, you might disrupt traffic if you need to release shared
resources from other interfaces in the same port group.
Oversubscription Ratio Restrictions
The 48-port and 24-port 4-Gbps, and all 8-Gbps Fibre Channel switching modules support
oversubscription on switches with shared rate mode configurations. By default, all 48-port and 24-port
4-Gbps, and 8-Gbps Fibre Channel switching modules have restrictions on oversubscription ratios
enabled. As of Cisco SAN-OS Release 3.1(1) and NX-OS Release 4.1(1), you can disable restrictions
on oversubscription ratios.
Table 2-9 describes the bandwidth allocation for oversubscribed interfaces configured in shared mode
on the 4-Gbps and 8-Gbps modules.
Table 2-9
Bandwidth Allocation for Oversubscribed Interfaces
Reserved Bandwidth (Gbps)
Switching
Module
Maximum
Bandwidth
(Gbps)
Configured Speed
Ratios enabled
Ratios
disabled
24-Port 8-Gbps
Fibre Channel
Module
Auto 8 Gbps
0.8
0.8
8
Auto Max 4 Gbps
0.4
0.4
4
Auto Max 2 Gbps
0.2
0.2
2
4/44-Port
8-Gbps
Host-Optimized
Fibre Channel
Module
8 Gbps
0.87
0.16
8
Auto Max 4 Gbps
0.436
0.08
4
Auto Max 2 Gbps
0.218
0.04
2
1 Gbps
0.109
0.02
1
48-port 4-Gbps
Fibre Channel
switching
module
Auto 4 Gbps
0.8
0.09
4
Auto Max 2 Gbps
0.4
0.045
2
1 Gbps
0.2
0.0225
1
24-port 4-Gbps
Fibre Channel
switching
module
Auto 4 Gbps
1
0.27
4
Auto Max 2 Gbps
0.5
0.135
2
1 Gbps
0.25
0.067
1
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-11
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
All ports in the 48-port and 24-port 4-Gbps modules can be configured to operate at 4 Gbps in shared
mode even if other ports in the port group are configured in dedicated mode, regardless of available
bandwidth. However, when oversubscription ratio restrictions are enabled, you may not have all shared
4-Gbps module ports operating at 4 Gbps.
All ports in the 48-port, 32-Port, and 24-port 8-Gbps modules can be configured to operate at 8 Gbps in
shared mode even if other ports in the port group are configured in dedicated mode, regardless of
available bandwidth. However, when oversubscription ratio restrictions are enabled you may not have
all shared 8-Gbps module ports operating at 8 Gbps.
On the 48-port, 32-Port, and 24-port 8-Gbps modules, if you have configured one 8-Gbps dedicated port
in one port group, no other ports in the same port group can be configured to operate at 8-Gbps dedicated
mode. You can have any number of 8-Gbps shared and 4-Gbps dedicated or shared ports. On the
4/44-port 8-Gbps module, only one port per port group can be configured in 8-Gbps dedicated or shared
mode.
In the following example, each of the 4 ports already have 500 credits and you have 2150 remaining for
the port group. So, in all 4150 credits:
switch(config-if)# show port-resources module 1
Module 1
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
Available dedicated buffers for global buffer
#0 [port-group 1] are 2150
#1 [port-group 2] are 2150
#2 [port-group 3] are 2150
#3 [port-group 4] are 2148
#4 [port-group 5] are 2150
#5 [port-group 6] are 2150
#6 [port-group 7] are 2150
#7 [port-group 8] are 650
#8 [port-group 9] are 2150
#9 [port-group 10] are 2150
#10 [port-group 11] are 2150
#11 [port-group 12] are 2150
Port-Group 1
Total bandwidth is 64.0 Gbps
Allocated dedicated bandwidth is 64.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc1/1
500
16.0 dedicated
fc1/2
500
16.0 dedicated
fc1/3
500
16.0 dedicated
fc1/4
500
16.0 dedicated
Port-Group 6
Total bandwidth is 64.0 Gbps
Allocated dedicated bandwidth is 64.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/21
4090
16.0 dedicated
fc4/22
10
16.0 dedicated
fc4/23
10
16.0 dedicated
fc4/24
10
16.0 dedicated
In the following example, a 24-port 4-Gbps module has oversubscription ratios enabled and three
dedicated ports in one port group operating at 4-Gbps. No other ports in the same port group can be
configured to operate at 4 Gbps.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-12
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
switch# show port-resources module 8
Module 8
Available dedicated buffers are 5478
Port-Group 1
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 0.8 Gbps
Allocated dedicated bandwidth is 12.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc8/1
16
4.0 dedicated
fc8/2
16
4.0 dedicated
fc8/3
16
4.0 dedicated
fc8/4 (out-of-service)
fc8/5 (out-of-service)
fc8/6 (out-of-service)
For dedicated ports, oversubscription ratio restrictions do not apply to the shared pool in port groups. So
if oversubscription ratio restrictions are disabled, and you have configured three 4-Gbps dedicated ports
in one port group, then you can configure all other ports in the same port group to operate at a shared
rate of 4 Gbps.
In the following example, a 48-port module has a group of six ports, four dedicated ports are operating
at 8 Gbps, and the two shared ports are also operating at 8 Gbps:
switch# show port-resources module 5
Module 5
Available dedicated buffers for global buffer #0 [port-group 1] are 3970
Available dedicated buffers for global buffer #1 [port-group 2] are 3970
Available dedicated buffers for global buffer #2 [port-group 3] are 3970
Available dedicated buffers for global buffer #3 [port-group 4] are 3970
Available dedicated buffers for global buffer #4 [port-group 5] are 3058
Available dedicated buffers for global buffer #5 [port-group 6] are 3058
Available dedicated buffers for global buffer #6 [port-group 7] are 3970
Available dedicated buffers for global buffer #7 [port-group 8] are 3970
Port-Group 1
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc5/1
32
8.0 shared
fc5/2
32
8.0 shared
fc5/3
32
8.0 shared
fc5/4
32
8.0 shared
Port-Group 2
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc5/5
32
8.0 shared
fc5/6
32
8.0 shared
fc5/7
32
8.0 shared
fc5/8
32
8.0 shared
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-13
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
Port-Group 3
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc5/9
32
8.0 shared
fc5/10
32
8.0 shared
fc5/11
32
8.0 shared
fc5/12
32
8.0 shared
Port-Group 4
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc5/13
32
8.0 shared
fc5/14
32
8.0 shared
fc5/15
32
8.0 shared
fc5/16
32
8.0 shared
Port-Group 5
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 16.4 Gbps
Allocated dedicated bandwidth is 16.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc5/17
32
8.0 shared
fc5/18
32
8.0 shared
fc5/19
500
8.0 dedicated
fc5/20
500
8.0 dedicated
Port-Group 6
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 16.4 Gbps
Allocated dedicated bandwidth is 16.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc5/21
500
8.0 dedicated
fc5/22
500
8.0 dedicated
fc5/23
32
8.0 shared
fc5/24
32
8.0 shared
Port-Group 7
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc5/25
32
8.0 shared
fc5/26
32
8.0 shared
fc5/27
32
8.0 shared
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-14
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
fc5/28
32
8.0
shared
Port-Group 8
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc5/29
32
8.0 shared
fc5/30
32
8.0 shared
fc5/31
32
8.0 shared
fc5/32
32
8.0 shared
Isola-13# show port-resources
Module 13
Available dedicated buffers
Available dedicated buffers
Available dedicated buffers
Available dedicated buffers
Available dedicated buffers
Available dedicated buffers
Available dedicated buffers
Available dedicated buffers
module 13
for
for
for
for
for
for
for
for
global
global
global
global
global
global
global
global
buffer
buffer
buffer
buffer
buffer
buffer
buffer
buffer
#0
#1
#2
#3
#4
#5
#6
#7
[port-group
[port-group
[port-group
[port-group
[port-group
[port-group
[port-group
[port-group
1]
2]
3]
4]
5]
6]
7]
8]
are
are
are
are
are
are
are
are
3880
3880
3880
3056
3880
3880
3880
3880
Port-Group 1
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc13/1
32
8.0 shared
fc13/2
32
8.0 shared
fc13/3
32
8.0 shared
fc13/4
32
8.0 shared
fc13/5
32
8.0 shared
fc13/6
32
8.0 shared
Port-Group 2
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc13/7
32
8.0 shared
fc13/8
32
8.0 shared
fc13/9
32
8.0 shared
fc13/10
32
8.0 shared
fc13/11
32
8.0 shared
fc13/12
32
8.0 shared
Port-Group 3
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-15
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
-------------------------------------------------------------------fc13/13
32
8.0 shared
fc13/14
32
8.0 shared
fc13/15
32
8.0 shared
fc13/16
32
8.0 shared
fc13/17
32
8.0 shared
fc13/18
32
8.0 shared
Port-Group 4
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 0.4 Gbps
Allocated dedicated bandwidth is 32.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc13/19
250
8.0 dedicated
fc13/20
250
8.0 dedicated
fc13/21
250
8.0 dedicated
fc13/22
250
8.0 dedicated
fc13/23
32
8.0 shared
fc13/24
32
8.0 shared
Port-Group 5
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc13/25
32
8.0 shared
fc13/26
32
8.0 shared
fc13/27
32
8.0 shared
fc13/28
32
8.0 shared
fc13/29
32
8.0 shared
fc13/30
32
8.0 shared
Port-Group 6
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc13/31
32
8.0 shared
fc13/32
32
8.0 shared
fc13/33
32
8.0 shared
fc13/34
32
8.0 shared
fc13/35
32
8.0 shared
fc13/36
32
8.0 shared
Port-Group 7
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc13/37
32
8.0 shared
fc13/38
32
8.0 shared
fc13/39
32
8.0 shared
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-16
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Information About Fibre Channel Interfaces
fc13/40
fc13/41
fc13/42
32
32
32
8.0
8.0
8.0
shared
shared
shared
Port-Group 8
Total bandwidth is 32.4 Gbps
Total shared bandwidth is 32.4 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc13/43
32
8.0 shared
fc13/44
32
8.0 shared
fc13/45
32
8.0 shared
fc13/46
32
8.0 shared
fc13/47
32
8.0 shared
fc13/48
32
8.0 shared
...
When disabling restrictions on oversubscription ratios, all ports in shared mode on 48-port and 24-port
4-Gbps or any 8-Gbps Fibre Channel switching modules must be shut down. When applying restrictions
on oversubscription ratios, you must take shared ports out of service.
Note
When restrictions on oversubscription ratios are disabled, the bandwidth allocation among the shared
ports is proportionate to the configured speed. If the configured speed is auto on Generation 2 modules,
then bandwidth is allocated assuming a speed of 4 Gbps. For example, if you have three shared ports
configured at 1, 2, and 4 Gbps, then the allocated bandwidth ratio is 1:2:4.
As of Cisco SAN-OS Release 3.0 and NX-OS Release 4.1(1) or when restrictions on oversubscription
ratios are enabled, the port bandwidths are allocated in equal proportions, regardless of port speed, so,
the bandwidth allocation for the same three ports mentioned in the example would be 1:1:1.
Bandwidth Fairness
This feature improves fairness of bandwidth allocation among all ports and provides better throughput
average to individual data streams. Bandwidth fairness can be configured per module.
As of Cisco SAN-OS Release 3.1(2), all 48-port and 24-port 4-Gbps Fibre Channel switching modules,
as well as 18-port Fibre Channel/4-port Gigabit Ethernet Multiservice modules, have bandwidth fairness
enabled by default. As of Cisco NX-OS Release 4.1(1), all the 8-Gbps Fibre Channel switching modules
have bandwidth fairness enabled by default.
Caution
When you disable or enable bandwidth fairness, the change does not take effect until you reload the
module.
Use the show module bandwidth-fairness command to check whether ports in a module are operating
with bandwidth fairness enabled or disabled.
switch# show module 2 bandwidth-fairness
Module 2 bandwidth-fairness is enabled
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-17
Chapter 2
Configuring Fibre Channel Interfaces
Guidelines and Limitations
Note
This feature is supported only on the 48-port and 24-port 4-Gbps modules, the 8-Gbps modules, and the
18/4-port Multiservice Module (MSM).
Upgrade or Downgrade Scenario
When you are upgrading from a release earlier than Cisco SAN-OS Release 3.1(2), all modules operate
with bandwidth fairness disabled until the next module reload. After the upgrade, any new module that
is inserted has bandwidth fairness enabled.
When you are downgrading to a release earlier than Cisco SAN-OS Release 3.1(2), all modules keep
operating in the same bandwidth fairness configuration prior to the downgrade. After the downgrade,
any new module that is inserted has bandwidth fairness disabled.
Note
After the downgrade, any insertion of a module or module reload will have bandwidth fairness disabled.
Guidelines and Limitations
This section includes the following topics:
•
Combining Generation 1, Generation 2, Generation 3, and Generation 4 Modules, page 2-18
•
Local Switching Limitations, page 2-19
•
Port Index Limitations, page 2-19
•
PortChannel Limitations, page 2-22
Combining Generation 1, Generation 2, Generation 3, and Generation 4
Modules
Cisco MDS NX-OS Release 5.2(x) and later supports combining Generation 1, Generation 2, Generation
3, and Generation 4 modules and switches with the following considerations:
•
MDS NX-OS Release 4.1(1) and later features are not supported on the following Generation 1
switches and modules:
– Supervisor 1 module
– 4-Port IP Storage Services module
– 8-Port IP Storage Services module
– MDS 9216 switch
– MDS 9216A switch
– MDS 9020 switch
– MDS 9120 switch
– MDS 9140 switch
•
Supervisor-1 modules must be upgraded to Supervisor-2 modules on the MDS 9506 and MDS 9509
Directors.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-18
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Guidelines and Limitations
•
IPS-4 and IPS-8 modules must be upgraded to the MSM-18/4 Multiservice modules.
•
Fabric 1 modules must be upgraded to Fabric 2 modules on the MDS 9513 Director to use the
48-port or the 24-port 8-Gbps module.
•
Fabric 2 modules must be upgraded to Fabric 3 modules on the MDS 9513 Director to get the
maximum backplane bandwidth of 256 Gbps.
•
Cisco Fabric Manager Release 4.x supports MDS SAN-OS Release 3.x and NX-OS 4.x in mixed
mode through Interswitch Link (ISL) connectivity.
Local Switching Limitations
•
All ports in the module must be in shared mode. Use the switchport ratemode shared command to
ensure that all the ports in the module are in shared mode.
•
No E ports are allowed in the module because E ports must be in dedicated mode.
Port Index Limitations
Cisco MDS 9000 switches allocate index identifiers for the ports on the modules. These port indexes
cannot be configured. You can combine Generation 1, Generation 2, Generation 3, and Generation 4
switching modules, with either Supervisor-1 modules or Supervisor-2 modules. However, combining
switching modules and supervisor modules has the following port index limitations:
Note
•
Supervisor-1 modules only support a maximum of 252 port indexes, regardless of the type of
switching modules.
•
Supervisor-2 modules support a maximum of 1020 port indexes when all switching modules in the
chassis are Generation 2 or Generation 3.
•
Supervisor-2 modules only support a maximum of 252 port indexes when only Generation 1
switching modules, or a combination of Generation 1, Generation 2, Generation 3, or Generation 4
switching modules are installed in the chassis.
On a switch with the maximum limit of 252 as port index, any new module that exceeds the limit does
not power up when installed.
You can use the show port index-allocation command to display the allocation of port indexes on the
switch.
switch# show port index-allocation
Module index distribution:
------------------------------------------------------+
Slot| Allowed |
Allotted indices info
|
| range | Total |
Index values
|
----|---------|-------|------------------------------|
1
| ----- |
| (None)
|
2
| ----- |
| (None)
|
3
| ----- |
| (None)
|
4
| ----- |
| (None)
|
5
|
0-1023|
32 | 0-31
|
6
| ----- |
| (None)
|
9
| ----- |
| (None)
|
10 | ----- |
| (None)
|
11 | ----- |
| (None)
|
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-19
Chapter 2
Configuring Fibre Channel Interfaces
Guidelines and Limitations
12 | ----- |
13 |
0-1023|
SUP | 253-255 |
48
3
| (None)
| 32-79
| 253-255
|
|
|
Generation 1 switching modules have specific numbering requirements. If these requirements are not
met, the module does not power up. The port index numbering requirements include the following:
Note
•
If port indexes in the range of 256 to 1020 are assigned to operational ports, Generation 1 switching
modules do not power up.
•
A block of contiguous port indexes is available. If this block of port indexes is not available,
Generation 1 modules do not power up. Table 2-10 shows the port index requirements for the
Generation 1 modules.
If the switch has Supervisor-1 modules, the block of 32 contiguous port indexes must begin on the slot
boundary. The slot boundary for slot 1 is 0, for slot 2 is 32, and so on. For Supervisor-2 modules, the
contiguous block can start anywhere.
Table 2-10
Port Index Requirements for Generation 1 Modules
Number of Port Indexes Required
Generation 1 Module
Supervisor-1 Module Supervisor-2 Module
16-port 2-Gbps Fibre Channel module
16
16
32-port 2-Gbps Fibre Channel module
32
32
8-port Gigabit Ethernet IP Storage Services module
32
32
4-port Gigabit Ethernet IP Storage Services module
32
16
32-port 2-Gbps Fibre Channel Storage Services
Module (SSM).
32
32
14-port Fibre Channel/2-port Gigabit Ethernet
Multiprotocol Services (MPS-14/2) module
32
22
The allowed mix of Generation 1 and Generation 2 switching modules in a chassis is determined at
run-time, either when booting up the switch or when installing the modules. In some cases, the sequence
in which switching modules are inserted into the chassis determines if one or more modules is powered
up.
When a module does not power up because of a resource limitation, you can display the reason by using
the show module command.
switch# show module
Mod Ports Module-Type
--- ----- ----------------------------------5
32
1/2/4/8/10 Gbps Advanced FC Module
7
0
Supervisor/Fabric-2
13
48
1/2/4/8/10 Gbps Advanced FC Module
Mod
--5
7
13
Sw
-------------5.2(2)
5.2(2)
5.2(2)
Hw
-----0.207
1.9
0.212
Model
-----------------DS-X9232-256K9
DS-X9530-SF2-K9
DS-X9248-256K9
Status
---------ok
active *
ok
World-Wide-Name(s) (WWN)
-------------------------------------------------21:01:00:0d:ec:b7:28:c0 to 21:20:00:0d:ec:b7:28:c0
-23:01:00:0d:ec:b7:28:c0 to 23:30:00:0d:ec:b7:28:c0
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-20
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Guidelines and Limitations
Mod
--5
7
13
MAC-Address(es)
-------------------------------------68-ef-bd-a8-45-cc to 68-ef-bd-a8-45-d0
00-24-c4-60-00-f8 to 00-24-c4-60-00-fc
68-ef-bd-a8-40-00 to 68-ef-bd-a8-40-04
Xbar
--1
2
Ports
----0
0
Xbar
--1
2
Sw
-------------NA
NA
Xbar
--1
2
MAC-Address(es)
-------------------------------------NA
NA
Serial-Num
---------JAF1450CHQT
JAE141502L2
JAF1450BMBP
Module-Type
----------------------------------Fabric Module 3
Fabric Module 3
Hw
-----0.4
0.4
Model
-----------------DS-13SLT-FAB3
DS-13SLT-FAB3
Status
---------ok
ok
World-Wide-Name(s) (WWN)
----------------------------------------------------
Serial-Num
---------JAF1451AMHG
JAF1451AMHN
* this terminal session
The running configuration is updated when modules are installed. If you save the running configuration
to the startup configuration (using the copy running-config startup-config command), during reboot
the switch powers up the same set of modules as before the reboot regardless of the sequence in which
the modules initialize. You can use the show port index-allocation startup command to display the
index allocation the switch uses at startup.
switch# show port index-allocation startup
Startup module index distribution:
------------------------------------------------------+
Slot | Allowed |
Alloted indices info
|
| range | Total |
Index values
|
-----|---------|-------|------------------------------|
1
| ----- |
34 | 0-31,80-81
|
2
| ----- |
32 | 32-63
|
3
| ----- |
16 | 64-79
|(Slot 1 shares 80-81)
4
| ----- |
48 | 96-127,224-239
|
SUP | 253-255 |
3
| 253-255
|
Note
The output of the show port index-allocation startup command does not display anything in the
Allowed range column because the command extracts the indices from the persistent storage service
(PSS) and displaying an allowed range for startup indices is meaningless.
If a module fails to power up, you can use the show module slot recovery-steps command to display
the reason.
For information on recovering a module powered-down because port indexes are not available, refer to
the Cisco MDS 9000 Family Troubleshooting Guide.
Tip
Whenever using mixed Generation 1 and Generation 2 modules, power up the Generation 1 modules
first. During a reboot of the entire switch, the Generation 1 modules power up first (default behavior).
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-21
Chapter 2
Configuring Fibre Channel Interfaces
Guidelines and Limitations
PortChannel Limitations
PortChannels have the following restrictions:
Note
•
The maximum number of PortChannels allowed is 256 if all switching modules are Generation 2 or
Generation 3, or both.
•
The maximum number of PortChannels allowed is 128 whenever there is a Generation 1 switching
module in use with a Generation 2 or Generation 3 switching module.
•
Ports need to be configured in dedicated rate mode on the Generation 2 and Generation 3 switching
module interfaces to be used in the PortChannel.
The number of PortChannels allowed does not depend on the type of supervisor module. However,
Generation 3 modules require the Supervisor 2 module on the MDS 9506 and 9509 switches.
The Generation1, Generation 2, and Generation 3 modules have the following restrictions for
PortChannel configuration:
Note
•
Generation 1 switching module interfaces do not support auto speed with a maximum of 2 Gbps.
•
Generation 1 and Generation 2 module interfaces do not support auto speed with maximum of 4
Gbps.
•
Generation 2 and Generation 3 switching module interfaces cannot be forcefully added to a
PortChannel if sufficient resources are not available.
Before adding a Generation 2 or Generation 3 interface to a PortChannel, use the show port-resources
module command to check for resource availability.
When configuring PortChannels on switches with Generation 1, Generation 2, and Generation 3
switching modules, follow one of these procedures:
•
Configure the PortChannel, and then configure the Generation 2 and Generation 3 interfaces to auto
with a maximum of 2 Gbps.
•
Configure the Generation 1 switching modules followed by the Generation 2 switching modules,
and then the Generation 3 switching modules, and then configure the PortChannel.
When configuring PortChannels on switches with only Generation 2 and Generation 3 switching
modules, follow one of these procedures:
•
Configure the PortChannel, and then configure the Generation 3 interfaces to auto with a maximum
of 4 Gbps.
•
Configure the Generation 2 switching modules, followed by the Generation 3 switching modules,
and then configure the PortChannel.
Table 2-11 describes the results of adding a member to a PortChannel for various configurations.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-22
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Guidelines and Limitations
Table 2-11
PortChannel Configuration and Addition Results
PortChannel Configured Speed
Members
PortChannel
New Member
New Member
Type
No members
Generation 1
interfaces
Generation 2
interfaces
Addition Type
Result
Any
Any
Generation 1 or Force
Generation 2 or
Generation 3 or
Generation 4
Pass
Auto
Auto
Generation 1 or Normal or force
Generation 2 or
Generation 3 or
Generation 4
Pass
Auto
Auto max 2000 Generation 2 or Normal
Generation 3 or Force
Generation 4
Auto
Auto max 4000 Generation 3 or
Generation 4
Auto max 2000
Auto
Generation 2 or Normal
Generation 3 or Force
Generation 4
Auto max 2000
Auto max 4000 Generation 3 or
or Generation 4
Auto max 4000
Auto
Auto max 4000
Auto max 2000 Generation 2 or
Generation 3 or
or Generation 4
Auto
Auto
Fail
Pass or fail
Fail
Pass
Generation 2 or
Generation 3 or
or Generation 4
Generation 2 or Normal
Generation 3
Force
Fail
Pass
Auto max 2000
Auto
Generation 1
Normal or force
Auto max 2000
Auto
Generation 2 or Normal
Generation 3
Force
Pass
Fail
Pass or fail
Auto max 4000
Auto
Generation 1 or
Generation 2
Auto max 4000
Auto
Generation 3
Auto
Auto
Generation 1
Normal or force
Fail
Auto max 2000
Auto
Generation 1
Normal or force
Pass
Auto max 2000
Auto
Generation 2 or Normal
Generation 3
Force
Fail
Auto max 2000 Generation 2 or Normal
Generation 3
Force
Fail
Auto
Pass
Pass
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-23
Chapter 2
Configuring Fibre Channel Interfaces
Default Settings
Table 2-11
PortChannel Configuration and Addition Results (continued)
PortChannel Configured Speed
Members
PortChannel
New Member
New Member
Type
Addition Type
Result
Generation 3
interfaces
Auto
Auto
Generation 1
Normal or force
Fail
Auto max 2000
Auto
Generation 1
Normal or force
Pass
Auto max 2000
Auto
Generation 2
Normal
Fail
Force
Pass
Normal
Fail
Force
Pass
Normal
Fail
Force
Pass
Normal
Fail
Force
Pass
Auto
Auto max 2000
Auto
Generation 4
interfaces
Auto max 2000 Generation 2
Auto
Generation 3
Auto max 2000 Generation 3
Auto
Auto
Generation 1
Normal or force
Fail
Auto max 2000
Auto
Generation 1
Normal or force
Pass
Auto max 2000
Auto
Generation 2
Normal
Fail
Force
Pass
Normal
Fail
Force
Pass
Auto
Auto max 2000
Auto
Auto max 2000 Generation 2
Auto
Generation 3 or Normal
Generation 4
Force
Fail
Auto max 2000 Generation 3 or Normal
Generation 4
Force
Fail
Pass
Pass
Use the show port-channel compatibility parameters command to obtain information about
PortChannel addition errors.
Default Settings
Table 2-12 lists the default settings for Generation 2 interface parameters.
Table 2-12
Default Generation 2 Interface Parameters
Default
Parameter
48-Port 4-Gbps
Switching Module
24-Port 4-Gbps
Switching Module
12-Port 4-Gbps
Switching Module
4-Port 10-Gbps
Switching Module
Speed mode
auto
auto
auto
auto
Rate mode
shared
shared
dedicated
dedicated
Port mode
Fx
Fx
auto
auto
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-24
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Table 2-12
Default Generation 2 Interface Parameters (continued)
Default
48-Port 4-Gbps
Switching Module
24-Port 4-Gbps
Switching Module
12-Port 4-Gbps
Switching Module
4-Port 10-Gbps
Switching Module
BB_credit
buffers
16
16
250
250
Performance
buffers
–
–
145
1455
Parameter
Table 2-13 lists the default settings for Generation 3 interface parameters.
Table 2-13
Default Generation 3 Interface Parameters
Default
Parameter
48-Port 8-Gbps
Switching Module
24-Port 8-Gbps
Switching Module
4/44-Port 8-Gbps
Host-Optimized Switching
Module
Speed mode
auto
auto
auto_max_4G
Note
Rate mode
shared
shared
shared
Port mode
Fx
Fx
Fx
BB_credit buffers
32
32
32
Auto_max_4G speed
mode on the 4/44-port
8-Gbps switching
module negotiates to a
maximum speed of 4
Gbps.
Table 2-14 lists the default settings for Generation 4 interface parameters.
Table 2-14
Default Generation 4 Interface Parameters
Default
Parameter
48-Port 8-Gbps Advanced Fibre 32-Port 8-Gbps Advanced Fibre
Channel Switching Module
Channel Switching Module
Speed mode
auto
auto1
Rate mode
shared
shared
Port mode
Fx
Fx
BB_credit buffers
32
32
Configuring Fibre Channel Interfaces
This section includes the following topics:
•
Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode, page 2-26
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-25
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
•
Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode, page 2-27
•
Task Flow for Configuring 12-Port 4-Gbps Module Interfaces, page 2-27
•
Task Flow for Configuring 4-Port 10-Gbps Module Interfaces, page 2-28
•
Configuring Port Speed, page 2-28
•
Configuring FEC, page 2-30
•
Configuring Local Switching, page 2-37
•
Disabling Restrictions on Oversubscription Ratios, page 2-38
•
Enabling Restrictions on Oversubscription Ratios, page 2-41
•
Enabling Bandwidth Fairness, page 2-42
•
Disabling Bandwidth Fairness, page 2-42
•
Taking Interfaces out of Service, page 2-42
•
Releasing Shared Resources in a Port Group, page 2-44
•
Disabling ACL Adjacency Sharing for System Image Downgrade, page 2-44
Task Flow for Migrating Interfaces from Shared Mode to Dedicated Mode
The 48-Port, 24-Port, and 4/44-Port 8-Gbps Fibre Channel switching modules support the following
features:
•
1-Gbps, 2-Gbps, 4-Gbps, and 8-Gbps speed traffic
•
Shared and dedicated rate mode
•
ISL and Fx port modes
•
Extended BB_credits
The 48-port and 24-port 4-Gbps Fibre Channel switching modules support the following features:
Note
•
1-Gbps, 2-Gbps, and 4-Gbps speed traffic
•
Shared and dedicated rate mode
•
ISL (E or TE) and Fx (F or FL) port modes
•
Extended BB_credits
If you change the port bandwidth reservation parameters on a 48-port or 24-port 4-Gbps module, the
change affects only the changed port. No other ports in the port group are affected.
To configure the 4-Gbps and 8-Gbps Fibre Channel switching modules when starting with the default
configuration or when migrating from shared rate mode to dedicated rate mode, follow these steps:
Step 1
Take unused interfaces out of service to release resources for other interfaces, if necessary.
See the “Taking Interfaces out of Service” section on page 2-42.
Step 2
Configure the traffic speed to use (1 Gbps, 2 Gbps, 4 Gbps, 8 Gbps, or autosensing with a maximum of
2 Gbps or 4 Gbps).
See the “Configuring Port Speed” section on page 2-28.
Step 3
Configure the rate mode (dedicated or shared).
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-26
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
See the “Configuring FEC” section on page 2-30.
Step 4
Configure the port mode.
See the “Configuring an Interface Mode” section on page 3-30.
Note
Step 5
ISL ports cannot operate in shared rate mode.
Configure the BB_credits and extended BB_credits, as necessary.
See the “Configuring Buffer-to-Buffer Credits” section on page 4-23and the “Configuring Extended
BB_credits” section on page 4-25.
Task Flow for Migrating Interfaces from Dedicated Mode to Shared Mode
To configure the 4-Gbps and 8-Gbps Fibre Channel switching modules migrating from dedicated rate
mode to shared rate mode, follow these steps:
Step 1
Take unused interfaces out of service to release resources for other interfaces, if necessary.
See the “Taking Interfaces out of Service” section on page 2-42.
Step 2
Configure the BB_credits and extended BB_credits, as necessary.
See the “Configuring Buffer-to-Buffer Credits” section on page 4-23, and the “Extended BB_Credits”
section on page 4-20.
Step 3
Configure the port mode.
See the “Configuring an Interface Mode” section on page 3-30.
Note
Step 4
ISL ports cannot operate in shared rate mode.
Configure the rate mode (dedicated or shared) to use.
See the “Configuring FEC” section on page 2-30.
Step 5
Configure the traffic speed (1 Gbps, 2 Gbps, 4 Gbps, or autosensing with a maximum of 2 Gbps or 4
Gbps) to use.
See the “Configuring Port Speed” section on page 2-28.
Task Flow for Configuring 12-Port 4-Gbps Module Interfaces
The 12-port 4-Gbps switching modules support the following features:
•
1-Gbps, 2-Gbps, and 4-Gbps speed traffic
•
Only dedicated rate mode
•
ISL (E or TE) and Fx (F or FL) port modes
•
Extended BB_credits
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-27
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
•
Performance buffers
To configure 4-port 10-Gbps switching modules when starting with the default configuration, follow
these steps:
Step 1
Configure the traffic speed (1 Gbps, 2 Gbps, 4 Gbps, or autosensing with a maximum of 2 Gbps or 4
Gbps) to use.
See the “Configuring Port Speed” section on page 2-28.
Step 2
Configure the port mode.
See the “Configuring an Interface Mode” section on page 3-30.
Step 3
Configure the BB_credits, performance buffers, and extended BB_credits, as necessary.
See the “Configuring Buffer-to-Buffer Credits” section on page 4-23, and the “Configuring Extended
BB_credits” section on page 4-25.
Task Flow for Configuring 4-Port 10-Gbps Module Interfaces
The 4-port 10-Gbps switching modules support the following features:
•
Only 10-Gbps speed traffic
•
Only dedicated rate mode
•
ISL (E or TE) and F port modes
•
Extended BB_credits
•
Performance buffers
To configure 4-port 10-Gbps switching modules when starting with the default configuration, follow
these steps:
Step 1
Configure the port mode.
See the “Configuring an Interface Mode” section on page 3-30.
Step 2
Configure the BB_credits, performance buffers, and extended BB_credits, as necessary.
See the “Configuring Buffer-to-Buffer Credits” section on page 4-23, and the “Configuring Extended
BB_credits” section on page 4-25.
Configuring Port Speed
Restrictions
•
Changing port speed and rate mode disrupts traffic on the port. Traffic on other ports in the port
group is not affected.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-28
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Detailed Steps
To configure the port speed on an interface on a 4-Gbps or 8-Gbps switching module, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc 1/1
switch(config-if)#
Selects the interface and enters interface
configuration submode.
Step 3
switch(config-if)# switchport speed 4000
Configures the port speed in megabits per
second. Valid values are 1000, 2000, 4000 and
auto. The auto parameter enables autosensing on
the interface.
switch(config-if)# switchport speed 8000
(8-Gbps modules only) Configures the port
speed in megabits per second to 8-Gbps. Valid
values are 1000, 2000, 4000, 8000 and auto. The
auto parameter enables autosensing on the
interface.
Note
switch(config-if)# switchport speed auto
The 8000 and auto max 4000 speed
configurations are available only for the
8-Gbps modules.
On 4-Gbps modules, configures autosensing for
the interface with 4 Gbps of bandwidth
reserved.
On 8-Gbps modules, configures autosensing for
the interface with 8 Gbps of bandwidth
reserved. 2
switch(config-if)# switchport speed auto max
2000
Configures autosensing with a maximum of
2 Gbps of bandwidth reserved.
switch(config-if)# switchport speed auto max
4000
(8-Gbps modules only) Configures autosensing
with a maximum of 4Gbps of bandwidth
reserved.
Note
switch(config-if)# no switchport speed
The 8000 and auto max 4000 speed
configurations are available only for the
8-Gbps modules.
Reverts to the default speed for the interface
(auto).
Note
The default speed on 48-port and
24-port 4-Gbps modules is 4000. The
default speed on 48-port and 24-port
8-Gbps modules is 8000. The default
speed on the 4/44-port 8-Gbps module is
auto max 4000.
Use the show interface command to verify the port speed configuration for an interface on a 4-Gbps or
8-Gbps Fibre Channel switching module.
switch# show interface fc 9/1
fc9/1 is up
Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-29
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Port WWN is 22:01:00:05:30:01:9f:02
Admin port mode is F
snmp traps are enabled
Port mode is F, FCID is 0xeb0002
Port vsan is 1
Speed is 2 Gbps
Rate mode is shared
Transmit B2B Credit is 64
Receive B2B Credit is 16
Receive data field Size is 2112
Beacon is turned off
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
226 frames input, 18276 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
326 frames output, 21364 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 1 NOS, 0 loop inits
3 output OLS, 2 LRR, 0 NOS, 0 loop inits
16 receive B2B credit remaining
64 transmit B2B credit remaining
Configuring FEC
Restrictions
FEC has the following restrictions:
Note
•
FEC is supported on the DS-X9448-768K9 Generation 5 module in the Cisco MDS 9700 Series
switch with Cisco NX-OS 6.2(7) and later releases. It is not supported on the MDS 9500 series, MDS
9148, MDS 9148S or the MDS 9200 series switches. FEC is also supported on the Cisco MDS 9396S
Multilayer Fabric Switch at Cisco NX-OS 6.2(13) and later releases.
•
Configuring FEC briefly disrupts traffic on the port.
•
Supported only on fixed speed 16-gigabit FC ports. FEC is not supported for ports configured with
2000/4000/8000/auto/auto-max maximum speed.
•
FEC with Transmitter Training Signal (TTS) is supported on the DS-X9448-768K9 Generation 5
module in Cisco MDS NX-OS Release 6.2(11c) and 6.2(15) and later. It is not supported in Cisco
MDS NX-OS Release 6.2(13).
•
FEC with TTS feature is not supported in Simple Network Management Protocol (SNMP) or in
Device Manager (DM).
•
FEC is supported in DM.
•
DM uses SNMP to get switch updates.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-30
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Detailed Steps
To configure FEC on an interface on a 48-port 16-Gbps Fibre Channel switching module, follow these
steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc 1/1
switch(config-if)#
Selects the interface and enters interface
configuration submode.
Step 3
switch(config-if)# switchport speed 16000
switch(config-if)# switchport fec
Enables FEC for the 16-Gbps interface.
•
FEC is active if it is configured on both
local and peer switches.
•
FEC is not active if it is configured on the
local switch but not on the peer switch.
Step 4
switch(config-if)# switchport fec tts
Note
The switchport fec tts command can be used only after configuring FEC using the switchport fec
command.
[Optional] Enables TTS allowing negotiation
of FEC capability. This command is only
accepted on ports with fixed 16-gigabit speed
and FEC already enabled.
Use the show interface command to verify the port speed configuration for an interface on a 48-port,
16-Gbps Fibre Channel switching module.
switch# show interface fc3/15
fc3/15 is up
Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
Port WWN is 20:8f:54:7f:ee:ea:3a:00
Admin port mode is auto, trunk mode is off
snmp link state traps are enabled
Port mode is F, FCID is 0xdf0000
Port vsan is 100
Speed is 2 Gbps
Rate mode is dedicated
Transmit B2B Credit is 128
Receive B2B Credit is 32
Receive data field Size is 2112
Beacon is turned off
admin fec state is down
oper fec state is down
5 minutes input rate 0 bits/sec,0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec,0 bytes/sec, 0 frames/sec
16072969258 frames input,34396153854332 bytes
23 discards,45 errors
22 invalid CRC/FCS,0 unknown class
0 too long,0 too short
8040504998 frames output,17206679580576 bytes
1344 discards,0 errors
0 input OLS,0 LRR,0 NOS,0 loop inits
306 output OLS,304 LRR, 1 NOS, 4 loop inits
32 receive B2B credit remaining
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-31
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
128 transmit B2B credit remaining
128 low priority transmit B2B credit remaining
Interface last changed at Wed Mar 12 21:23:36 2014
Last clearing of "show interface" counters
:never
switch#
Configuring Rate Mode
Restrictions
•
Changing port speed and rate mode disrupts traffic on the port.
Detailed Steps
To configure the rate mode (dedicated or shared) on an interface on a 48-port or 24-port 4-Gbps, or any
8-Gbps Fibre Channel switching module, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc 1/1
switch(config-if)#
Selects the interface and enters interface
configuration submode.
Step 3
switch(config-if)# switchport rate-mode
dedicated
Reserves dedicated bandwidth for the
interface.
Note
If you cannot reserve dedicated
bandwidth on an interface, you might
have exceeded the port group
maximum bandwidth. Use the show
port-resources command to
determine what resources are already
allocated.
switch(config-if)# switchport rate-mode shared
Reserves shared (default) bandwidth for the
interface.
switch(config-if)# no switchport rate-mode
Reverts to the default state (shared).
Displaying the Rate Mode Configuration for Interfaces
Use show port-resources module command to verify the rate mode configuration for interfaces on a
48-port or 24-port 4-Gbps, or any 8-Gbps Fibre Channel switching module.
This example shows the port rate mode configuration for interfaces on a 4-Gbps module:
switch# show port-resources module 9
Module 9
Available dedicated buffers are 5400
Port-Group 1
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-32
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc9/1
16
4.0 shared
fc9/2
16
4.0 shared
fc9/3
16
4.0 shared
fc9/4
16
4.0 shared
fc9/5
16
4.0 shared
fc9/6
16
4.0 shared
Port-Group 2
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc9/7
16
4.0 shared
fc9/8
16
4.0 shared
fc9/9
16
4.0 shared
fc9/10
16
4.0 shared
fc9/11
16
4.0 shared
fc9/12
16
4.0 shared
Port-Group 3
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc9/13
16
4.0 shared
fc9/14
16
4.0 shared
fc9/15
16
4.0 shared
fc9/16
16
4.0 shared
fc9/17
16
4.0 shared
fc9/18
16
4.0 shared
Port-Group 4
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc9/19
16
4.0 shared
fc9/20
16
4.0 shared
fc9/21
16
4.0 shared
fc9/22
16
4.0 shared
fc9/23
16
4.0 shared
fc9/24
16
4.0 shared
This example shows the port rate mode configuration for interfaces on a 48-port 8-Gbps module:
switch# show port-resource module 4
Module 4
Available dedicated buffers for global buffer #0 [port-groups 1-4] are 5016
Available dedicated buffers for global buffer #1 [port-groups 5-8] are 5016
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-33
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Port-Group 1
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/1
32
8.0 shared
fc4/2
32
8.0 shared
fc4/3
32
8.0 shared
fc4/4
32
8.0 shared
fc4/5
32
8.0 shared
fc4/6
32
8.0 shared
Port-Group 2
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/7
32
8.0 shared
fc4/8
32
8.0 shared
fc4/9
32
8.0 shared
fc4/10
32
8.0 shared
fc4/11
32
8.0 shared
fc4/12
32
8.0 shared
Port-Group 3
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/13
32
8.0 shared
fc4/14
32
8.0 shared
fc4/15
32
8.0 shared
fc4/16
32
8.0 shared
fc4/17
32
8.0 shared
fc4/18
32
8.0 shared
Port-Group 4
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/19
32
8.0 shared
fc4/20
32
8.0 shared
fc4/21
32
8.0 shared
fc4/22
32
8.0 shared
fc4/23
32
8.0 shared
fc4/24
32
8.0 shared
Port-Group 5
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-34
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/25
32
8.0 shared
fc4/26
32
8.0 shared
fc4/27
32
8.0 shared
fc4/28
32
8.0 shared
fc4/29
32
8.0 shared
fc4/30
32
8.0 shared
Port-Group 6
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/31
32
8.0 shared
fc4/32
32
8.0 shared
fc4/33
32
8.0 shared
fc4/34
32
8.0 shared
fc4/35
32
8.0 shared
fc4/36
32
8.0 shared
Port-Group 7
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/37
32
8.0 shared
fc4/38
32
8.0 shared
fc4/39
32
8.0 shared
fc4/40
32
8.0 shared
fc4/41
32
8.0 shared
fc4/42
32
8.0 shared
Port-Group 8
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc4/43
32
8.0 shared
fc4/44
32
8.0 shared
fc4/45
32
8.0 shared
fc4/46
32
8.0 shared
fc4/47
32
8.0 shared
fc4/48
32
8.0 shared
This example shows the port rate mode configuration for interfaces on a 4/44-port 8-Gbps module:
switch# show port-resources module 7
Module 7
Available dedicated buffers are 3888
Port-Group 1
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-35
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc7/1
32
4.0 shared
fc7/2
32
4.0 shared
fc7/3
32
4.0 shared
fc7/4
32
4.0 shared
fc7/5
32
4.0 shared
fc7/6
32
4.0 shared
fc7/7
32
4.0 shared
fc7/8
32
4.0 shared
fc7/9
32
4.0 shared
fc7/10
32
4.0 shared
fc7/11
32
4.0 shared
fc7/12
32
4.0 shared
Port-Group 2
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc7/13
32
4.0 shared
fc7/14
32
4.0 shared
fc7/15
32
4.0 shared
fc7/16
32
4.0 shared
fc7/17
32
4.0 shared
fc7/18
32
4.0 shared
fc7/19
32
4.0 shared
fc7/20
32
4.0 shared
fc7/21
32
4.0 shared
fc7/22
32
4.0 shared
fc7/23
32
4.0 shared
fc7/24
32
4.0 shared
Port-Group 3
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc7/25
32
4.0 shared
fc7/26
32
4.0 shared
fc7/27
32
4.0 shared
fc7/28
32
4.0 shared
fc7/29
32
4.0 shared
fc7/30
32
4.0 shared
fc7/31
32
4.0 shared
fc7/32
32
4.0 shared
fc7/33
32
4.0 shared
fc7/34
32
4.0 shared
fc7/35
32
4.0 shared
fc7/36
32
4.0 shared
Port-Group 4
Total bandwidth is 12.8 Gbps
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-36
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc7/37
32
4.0 shared
fc7/38
32
4.0 shared
fc7/39
32
4.0 shared
fc7/40
32
4.0 shared
fc7/41
32
4.0 shared
fc7/42
32
4.0 shared
fc7/43
32
4.0 shared
fc7/44
32
4.0 shared
fc7/45
32
4.0 shared
fc7/46
32
4.0 shared
fc7/47
32
4.0 shared
fc7/48
32
4.0 shared
Configuring Local Switching
Note
We recommend that you shut down all of the ports on the module before you execute the local switching
command.
If local switching is enabled, then ports cannot be configured in dedicated mode.
If there are dedicated ports and you enter the local switching command, a warning is displayed and the
operation is prevented.
Detailed Steps
To enable or disable local switching module, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# local-switching module 5
Enables local switching for a module.
If you want to proceed with enabling local
switching, when prompted press Y to continue or N
to exit.
Step 3
switch(config)# no local-switching module
5
Disables local switching for a module.
If you want to proceed with disabling local
switching, when prompted press Y to continue or N
to exit.
Step 4
switch(config)# exit
Exits configuration mode.
Use the show system internal xbar local-switching command to verify the local switching
configuration status on a module.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-37
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
switch# show system internal xbar local-switching
------------------------------------| Slot | Local-switching enabled |
------------------------------------|
1 |
no
|
|
2 |
no
|
|
3 |
no
|
|
4 |
no
|
|
5 |
yes
|
|
6 |
no
|
|
9 |
no
|
| 10 |
no
|
| 11 |
no
|
| 12 |
no
|
| 13 |
no
|
-------------------------------------
Disabling Restrictions on Oversubscription Ratios
Prerequisites
•
Before disabling restrictions on oversubscription ratios, ensure that you have explicitly shut down
shared ports.
Detailed Steps
To disable restrictions on oversubscription ratios on a 48-port or 24-port 4-Gbps, or any 8-Gbps Fibre
Channel switching module, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# no rate-mode
oversubscription-limit module 1
Disables restrictions on oversubscription ratios for a
module.
Note
You must enter this command separately for
each module for which you want to remove
the restrictions.
Step 3
switch(config)# exit
Exits configuration mode.
Step 4
switch# copy running-config startup-config
Saves the new oversubscription ratio configuration
to the startup configuration, and then the new
configuration is enforced upon subsequent reboots
of the module.
Use the show running-config command to view oversubscription ratios for a module. If
oversubscription ratios are enabled, then no restriction appears in the output.
Example 2-1
Module with Restrictions on Oversubscription Ratios Disabled
switch# show running-config
version 3.1(1)
...
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-38
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
no rate-mode oversubscription-limit module 2
interface fc2/1
switchport speed 2000
interface fc2/1
...
Examples
To disable restrictions on oversubscription ratios for ports on a 48-port Generation 2 switch that is
configured with both shared and dedicated ports, follow these steps:
Step 1
To disable restrictions on oversubscription ratios, you must shut down any shared ports. Use the show
port-resources command to view the configuration on a module and to identify shared ports.
switch# show port-resources module 2
Module 2
Available dedicated buffers are 4656
Port-Group 1
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc2/1
16
4.0 shared
fc2/2
16
4.0 shared
fc2/3
16
4.0 dedicated
fc2/4
16
4.0 shared
fc2/5
16
4.0 shared
fc2/6
16
4.0 dedicated
fc2/7
16
4.0 dedicated
fc2/8
16
4.0 shared
fc2/9
16
4.0 shared
fc2/10
16
4.0 shared
fc2/11
16
4.0 shared
fc2/12
16
4.0 shared
...
Port-Group 4
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc2/37
16
4.0 shared
fc2/38
16
4.0 shared
fc2/39
16
4.0 dedicated
fc2/40
16
4.0 dedicated
fc2/41
16
4.0 dedicated
fc2/42
16
4.0 shared
fc2/43
16
4.0 shared
fc2/44
16
4.0 shared
fc2/45
16
4.0 shared
fc2/46
16
4.0 shared
fc2/47
16
4.0 shared
fc2/48
16
4.0 shared
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-39
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Step 2
Shut down all shared ports for which you want to remove restrictions on oversubscription ratios.
switch (config)# interface fc2/1-2, fc2/4-5, fc2/8-38, fc2/43-48
switch (config-if)# shutdown
Step 3
Display the interface status to confirm the shutdown of all shared ports.
switch(config-if)# end
switch# show interface brief
------------------------------------------------------------------------------Interface Vsan
Admin Admin
Status
SFP
Oper Oper
Port
Mode
Trunk
Mode Speed Channel
Mode
(Gbps)
------------------------------------------------------------------------------fc2/1
1
FX
-down
swl
--fc2/2
1
FX
-down
swl
--fc2/3
1
T
-up
swl
--fc2/4
1
FX
-down
swl
--fc2/5
1
FX
-down
swl
--fc2/6
1
TE
-up
swl
--fc2/7
1
TE
-up
swl
--fc2/8
1
FX
-down
swl
--...
fc2/48
1
FX
-down
sw1
---
Step 4
Disable restrictions on oversubscription ratios for the ports.
switch# config t
Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# no rate-mode oversubscription-limit module 2
Step 5
Bring up the ports that you shut down in step 2, and display their status to confirm that they are no longer
shut down.
switch(config)# interface fc2/1-2, fc2/4-5, fc2/8-38, fc2/43-48
switch(config-if)# no shutdown
switch(config-if)# end
switch# show interface brief
------------------------------------------------------------------------------Interface Vsan
Admin Admin
Status
SFP
Oper Oper
Port
Mode
Trunk
Mode Speed Channel
Mode
(Gbps)
------------------------------------------------------------------------------fc2/1
1
FX
-up
swl
--fc2/2
1
FX
-up
swl
--fc2/3
1
T
-up
swl
--fc2/4
1
FX
-up
swl
--fc2/5
1
FX
-up
swl
--fc2/6
1
TE
-up
swl
--fc2/7
1
TE
-up
swl
--fc2/8
1
FX
-up
swl
--...
fc2/48
1
FX
-up
sw1
---
Step 6
Confirm that the ports are now operating with no restrictions on oversubscription ratios.
switch# show running-config | include oversubscription-limit
no rate-mode oversubscription-limit module 2 <---indicates no restrictions on
oversubscription ratios
Step 7
Save the new oversubscription ratio configuration to the startup configuration.
switch# copy running-config startup-config
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-40
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Enabling Restrictions on Oversubscription Ratios
Prerequisites
•
You must enable restrictions on oversubscription ratios before you can downgrade modules to a
previous release.
•
Before enabling restrictions on oversubscription ratios, ensure that you have explicitly configured
shared ports to out-of-service mode.
Detailed Steps
To enable restrictions on oversubscription ratios on a 48-port or 24-port 4-Gbps, or any 8-Gbps Fibre
Channel switching module, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc2/1-2,
fc2/4-5, fc2/8-38, fc2/43-48
Specifies the port interfaces for which you want to
enable restrictions on oversubscription ratios.
Step 3
switch(config-if)# shutdown
Shuts down shared ports.
Step 4
switch(config-if)# out-of-service
Takes shared ports out of service.
Step 5
switch# rate-mode oversubscription-limit
module 1
Enables restrictions on oversubscription ratios for
the module.
Note
You must enter this command separately for
each module for which you want to add the
restriction.
switch# config t
switch(config)# interface fc2/1-2,
fc2/4-5, fc2/8-38, fc2/43-48
switch(config-if)# no out-of-service
switch(config-if)# no shutdown
Returns all shared ports to service.
Step 6
switch(config)# exit
Exits configuration mode.
Step 7
switch# copy running-config startup-config
Saves the new oversubscription ratio configuration
to the startup configuration, and then the new
configuration is enforced upon subsequent reboots
of the module.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-41
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Enabling Bandwidth Fairness
Detailed Steps
To enable bandwidth fairness on a switching module, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# rate-mode
bandwidth-fairness module 1
Enables bandwidth fairness for a module.
Step 3
switch(config)# exit
Note
You must enter this command separately for
each module for which you want to enable
bandwidth fairness. You must reload the
module for the command to take effect.
Exits configuration mode.
Disabling Bandwidth Fairness
Restrictions
•
If you disable bandwidth fairness, up to a 20 percent increase in internal bandwidth allocation is
possible for each port group; however, bandwidth fairness is not guaranteed when there is a mix of
shared and full-rate ports in the same port group.
Detailed Steps
To disable bandwidth fairness on a switching module, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# no rate-mode
bandwidth-fairness module 1
Disables bandwidth fairness for a module.
Step 3
switch(config)# exit
Note
You must enter this command separately for
each module for which you want to disable
bandwidth fairness. You must reload the
module for the command to take effect.
Exits configuration mode.
Taking Interfaces out of Service
You can take interfaces out of service on Generation 2 and Generation 3 switching modules. When an
interface is out of service, all the shared resources for the interface are released as well as the
configuration associated with those resources.
Prerequisites
•
The interface must be disabled using a shutdown command before it can be taken out of service.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-42
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Restrictions
•
The interface cannot be a member of a PortChannel.
•
Taking interfaces out of service releases all the shared resources to ensure that they are available to
other interfaces. This causes the configuration in the shared resources to revert to default when the
interface is brought back into service. Also, an interface cannot come back into service unless the
default shared resources for the port are available. The operation to free up shared resources from
another port is disruptive.
Detailed Steps
To take an interface out of service, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc 1/1
switch(config-if)#
Selects the interface and enters interface
configuration submode.
Step 3
switch(config-if)# no channel-group
Removes the interface from a PortChannel.
Step 4
switch(config-if)# shutdown
Disables the interface.
Step 5
switch(config-if)# out-of-service
Putting an interface into out-of-service will
cause its shared resource configuration to
revert to default
Do you wish to continue (y/n)? [n] y
Takes the interface out of service.
Use the show port-resources module command to verify the out-of-service configuration for interfaces
on a Generation 2 and Generation 3 switching module.
This example shows a 24-port 4-Gbps module:
switch# show port-resources module 9
Module 9
Available dedicated buffers are 5429
Port-Group 1
Total bandwidth is 12.8 Gbps
Total shared bandwidth is 12.8 Gbps
Allocated dedicated bandwidth is 0.0 Gbps
-------------------------------------------------------------------Interfaces in the Port-Group
B2B Credit Bandwidth Rate Mode
Buffers
(Gbps)
-------------------------------------------------------------------fc9/1
16
4.0 shared
fc9/2 (out-of-service)
fc9/3
16
4.0 shared
fc9/4
16
4.0 shared
fc9/5
16
4.0 shared
fc9/6
16
4.0 shared
...
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-43
Chapter 2
Configuring Fibre Channel Interfaces
Configuring Fibre Channel Interfaces
Releasing Shared Resources in a Port Group
When you want to reconfigure the interfaces in a port group on a Generation 2 or Generation 3 module,
you can return the port group to the default configuration to avoid problems with allocating shared
resources.
Restrictions
•
The interface cannot be a member of a PortChannel.
•
Releasing shared resources disrupts traffic on the port. Traffic on other ports in the port group is not
affected.
Detailed Steps
To release the shared resources for a port group, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc 1/1
switch(config-if)#
Selects the interface and enters interface
configuration submode.
Tip
You can use an interface range to
release the resources for all interfaces
in a port group.
Step 3
switch(config-if)# no channel-group
Removes the interface from a PortChannel.
Step 4
switch(config-if)# shutdown
Disables the interface.
Step 5
switch(config-if)# out-of-service
Putting an interface into out-of-service will
cause its shared resource configuration to
revert to default
Do you wish to continue(y/n)? [n] y
Takes the interface out of service.
Step 6
switch(config-if)# no out-of-service
Makes the interface available for service.
Repeat Step 2 through Step 6 for all the
interfaces in the port group.
Disabling ACL Adjacency Sharing for System Image Downgrade
Fibre Channel ACL adjacency sharing is enabled by default on the switches with an active Generation 2
switching module as of Cisco MDS SAN-OS Release 3.0(3), and with an active Generation 3 module as
of MDS NX-OS Release 4.1(1). Fibre Channel ACL adjacency sharing improves the performance for
zoning and inter-VSAN routing (IVR) network address translation (NAT). To prevent disruptions when
downgrading the system image on your switch to a release prior to Cisco SAN-OS Release 3.0(3), enter
the following command in EXEC mode:
switch# system no acl-adjacency-sharing
To reenable Fibre Channel ACL adjacency sharing on your switch, enter the following command in
EXEC mode:
switch# system acl-adjacency-sharing
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-44
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Verifying Fibre Channel Interfaces Configuration
Verifying Fibre Channel Interfaces Configuration
To display Fibre Channel interface configuration information, perform one of the following tasks:
Command
Purpose
show module
Displays the module.
show module slot recovery-steps
Displays the slot for the module.
show port-resources module slot
Displays the port resources for the slot.
show interface fc slot/port
Displays the slot or port information. FEC admin
and operational states are displayed.
show interface brief
Displays the interface.
show port index-allocation
Displays the port in the index allocation.
show port index-allocation startup
Displays the startup port in the index allocation.
show port-channel compatibility parameters
Displays the PortChannel compatibility
parameters.
show module slot bandwidth-fairness
Displays the module slot bandwidth fairness
information.
For detailed information about the fields in the output from these commands, refer to the Cisco MDS
NX-OS Command Reference.
Displaying Interface Capabilities
Before configuring a Generation 2 or Generation 3 interface, you can use the show interface
capabilities command to display detailed information about the capabilities of the interface.
This example shows the capabilities of a Generation 2 Fibre Channel interface:
switch# show interface fc 9/1 capabilities
Min Speed is 1 Gbps
Max Speed is 4 Gbps
FC-PH Version (high, low)
Receive data field size (max/min)
Transmit data field size (max/min)
Classes of Service supported are
Class 2 sequential delivery
Class 3 sequential delivery
Hold time (max/min)
BB state change notification
Maximum BB state change notifications
Rate Mode change
Rate Mode Capabilities
Receive BB Credit modification supported
FX mode Receive BB Credit (min/max/default)
ISL mode Receive BB Credit (min/max/default)
Performance buffer modification supported
Out of Service capable
Beacon mode configurable
(0,6)
(2112/256) bytes
(2112/128) bytes
Class 2, Class 3, Class F
supported
supported
(100/1) micro sec
supported
14
supported
Shared
yes
(1/16/16)
-no
Dedicated
yes
(1/250/16)
(2/250/250)
no
yes
yes
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-45
Chapter 2
Configuring Fibre Channel Interfaces
Verifying Fibre Channel Interfaces Configuration
This example shows the capabilities of an interface on the 48-port 8-Gbps Fibre Channel switching
module:
switch# show interface fc 4/1 capabilities
Min Speed is 1 Gbps
Max Speed is 8 Gbps
FC-PH Version (high, low)
Receive data field size (max/min)
Transmit data field size (max/min)
Classes of Service supported are
Class 2 sequential delivery
Class 3 sequential delivery
Hold time (max/min)
BB state change notification
Maximum BB state change notifications
Rate Mode change
Rate Mode Capabilities
Receive BB Credit modification supported
FX mode Receive BB Credit (min/max/default)
ISL mode Receive BB Credit (min/max/default)
Performance buffer modification supported
Out of Service capable
Beacon mode configurable
Extended B2B credit capable
(0,6)
(2112/256) bytes
(2112/128) bytes
Class 2, Class 3, Class F
supported
supported
(100000/1) micro sec
supported
14
supported
Shared
yes
(1/32/32)
-no
Dedicated
yes
(1/500/32)
(2/500/250)
no
yes
yes
yes
Displaying SFP Diagnostic Information
You can use the show interface transceiver command to display small form-factor pluggable (SFP)
diagnostic information for Generation 2 switching modules.
switch# show interface transceiver
...
fc12/12 sfp is present
name is CISCO-FINISAR
part number is FTRJ-8519-7D2CS1
revision is A
serial number is H11TVQB
fc-transmitter type is short wave laser w/o OFC (SN)
fc-transmitter supports intermediate distance link length
media type is multi-mode, 62.5m (M6)
Supported speed is 200 MBytes/sec
Nominal bit rate is 2100 MBits/sec
Link length supported for 50/125mm fiber is 300 m(s)
Link length supported for 62.5/125mm fiber is 150 m(s)
cisco extended id is unknown (0x0)
no tx fault, rx loss, no sync exists, Diag mon type 104
SFP Diagnostics Information
Temperature
: 24.33 Celsius
Voltage
:
3.33 Volt
Current
:
0.04 mA
-Optical Tx Power : N/A
dBm
-Optical Rx Power : N/A
dBm
Note: ++ high-alarm; + high-warning; -- low-alarm; -
low-warning
...
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-46
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuration Examples for Fibre Channel Interfaces
Configuration Examples for Fibre Channel Interfaces
This section describes example configurations and includes the following sections:
•
Configuration Example for FEC Module Interfaces, page 2-47
•
Configuration Example for 48-Port 8-Gbps Module Interfaces, page 2-47
•
Configuration Example for 24-Port 8-Gbps Module Interfaces, page 2-48
•
Configuration Example for 4/44-Port 8-Gbps Module Interfaces, page 2-49
•
Configuration Example for 48-Port 4-Gbps Module Interfaces, page 2-50
•
Configuration Example for 24-Port 4-Gbps Module Interfaces, page 2-51
Configuration Example for FEC Module Interfaces
These steps describe how to configure FEC module interfaces:
Step 1
Select the interfaces fc 4/1 through fc 4/2.
switch# config t
switch(config)# interface fc 4/1 - 2
Step 2
Configure the FEC on the interfaces.
switch(config-if)# switchport speed 16000
switch(config-if)# switchport fec
Step 3
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Step 4
Select the interfaces fc 4/3 through fc 4/4.
switch# config t
switch(config)# interface fc 4/3 - 4
Step 5
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed 16000
switch(config-if)# switchport fec
Configuration Example for 48-Port 8-Gbps Module Interfaces
These steps describe how to configure the 48-port 8-Gbps module interfaces:
Step 1
Select the interfaces fc 4/1 through fc 4/2.
switch# config t
switch(config)# interface fc 4/1 - 2
Step 2
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed 8000
switch(config-if)# switchport rate-mode shared
switch(config-if)# switchport mode f
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-47
Chapter 2
Configuring Fibre Channel Interfaces
Configuration Examples for Fibre Channel Interfaces
Step 3
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Step 4
Select the interfaces fc 4/3 through fc 4/4.
switch# config t
switch(config)# interface fc 4/3 - 4
Step 5
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed auto max 4000
switch(config-if)# switchport rate-mode dedicated
switch(config-if)# switchport mode f
Step 6
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Step 7
Select the interfaces fc 4/5 through fc 4/6.
switch# config t
switch(config)# interface fc 4/5 - 6
Step 8
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed auto max 4000
switch(config-if)# switchport rate-mode shared
switch(config-if)# switchport mode f
Step 9
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Configuration Example for 24-Port 8-Gbps Module Interfaces
These steps describe how to configure the 24-port 8-Gbps module interfaces:
Step 1
Select interfaces fc 3/1.
switch# config t
switch(config)# interface fc 3/1
Step 2
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed 8000
switch(config-if)# switchport rate-mode dedicated
switch(config-if)# switchport mode f
Step 3
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-48
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuration Examples for Fibre Channel Interfaces
Step 4
Select the interfaces fc 3/2 through fc 3/3.
switch# config t
switch(config)# interface fc 3/2 - 3
Step 5
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed 8000
switch(config-if)# switchport rate-mode shared
switch(config-if)# switchport mode f
Step 6
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Configuration Example for 4/44-Port 8-Gbps Module Interfaces
These steps describe how to configure the 4/44-port 8-Gbps module interfaces:
Step 1
Select interfaces fc 4/11 through fc 4/12.
switch# config t
switch(config)# interface fc 4/11 - 12
Step 2
Disable the interfaces and take them out of service.
switch(config-if)# shutdown
switch(config-if)# out-of-service
Step 3
Return to configuration mode.
switch(config-if)# exit
switch#
Step 4
Select the interfaces fc 4/1.
switch# config t
switch(config)# interface fc 4/1
Step 5
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed 8000
switch(config-if)# switchport rate-mode shared
switch(config-if)# switchport mode f
Step 6
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Step 7
Select the interfaces fc 4/2 through fc 4/10.
switch# config t
switch(config)# interface fc 4/2 - 10
Step 8
Configure the port speed, rate mode, and port mode on the interfaces.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-49
Chapter 2
Configuring Fibre Channel Interfaces
Configuration Examples for Fibre Channel Interfaces
switch(config-if)# switchport speed auto max 4000
switch(config-if)# switchport rate-mode shared
switch(config-if)# switchport mode f
Step 9
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Configuration Example for 48-Port 4-Gbps Module Interfaces
These steps describe how to configure the example shown in Figure 4-6 on page 4-12:
Step 1
Select interfaces fc 4/11 through fc 4/12.
switch# config t
switch(config)# interface fc 4/11 - 12
Step 2
Disable the interfaces and take them out of service.
switch(config-if)# shutdown
switch(config-if)# out-of-service
Step 3
Return to configuration mode.
switch(config-if)# exit
switch#
Step 4
Select the interfaces fc 4/1 through fc 4/6.
switch# config t
switch(config)# interface fc 4/1 - 6
Step 5
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed auto max 2000
switch(config-if)# switchport rate-mode dedicated
switch(config-if)# switchport mode e
Step 6
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Step 7
Select the interfaces fc 4/7 through fc 4/10.
switch# config t
switch(config)# interface fc 4/7 - 10
Step 8
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed 1000
switch(config-if)# switchport rate-mode shared
switch(config-if)# switchport mode f
Step 9
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-50
OL-29284-01, Release 6.x
Chapter 2
Configuring Fibre Channel Interfaces
Configuration Examples for Fibre Channel Interfaces
switch#
Configuration Example for 24-Port 4-Gbps Module Interfaces
These steps describe how to configure the example shown in Figure 4-8 on page 4-14:
Step 1
Select interfaces fc 3/1 through fc 3/3.
switch# config t
switch(config)# interface fc 3/1 - 3
Step 2
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed 4000
switch(config-if)# switchport rate-mode dedicated
switch(config-if)# switchport mode e
Step 3
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
Step 4
Select the interfaces fc 3/4 through fc 3/6.
switch# config t
switch(config)# interface fc 3/4 - 6
Step 5
Configure the port speed, rate mode, and port mode on the interfaces.
switch(config-if)# switchport speed 1000
switch(config-if)# switchport rate-mode shared
switch(config-if)# switchport mode f
Step 6
Enable the interfaces and return to configuration mode.
switch(config-if)# no shutdown
switch(config-if)# exit
switch#
1
1.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
2-51
Chapter 2
Configuring Fibre Channel Interfaces
Configuration Examples for Fibre Channel Interfaces
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
2-52
OL-29284-01, Release 6.x
CHAPTER
3
Configuring Interfaces
This document provides information about configuring and verifying interfaces on Cisco MDS 9000
Series Multilayer switches.
This chapter includes the following topics:
•
Finding Feature Information, page 3-1
•
Feature Information for Interfaces, page 3-1
•
Prerequisites for Interfaces, page 3-4
•
Guidelines and Limitations for Interfaces, page 3-4
•
Default Settings for Interface Parameters, page 3-6
•
Information About Interfaces, page 3-7
•
Configuring Interfaces, page 3-27
•
Verifying Interfaces Configuration, page 3-52
Finding Feature Information
Your software release might not support all the features documented in this module. For the latest caveats
and feature information, see the Bug Search Tool at https://tools.cisco.com/bugsearch/ and the release
notes for your software release. To find information about the features documented in this chapter, and
to see a list of the releases in which each feature is supported, see the Chapter 1, “New and Changed
Information” or the Feature Information table below.
Feature Information for Interfaces
Table 3-1 lists the new and changed features for Cisco MDS NX-OS Release 6.2(x).
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-1
Chapter 3
Configuring Interfaces
Feature Information for Interfaces
Table 3-1
New and Changed Features for Cisco MDS NX-OS Release 6.2(x)
Feature Name Release
Feature Information
Port Monitor
Port Monitor Counter—Users can configure the state-change
counter to be monitored. This counter records port down-to-port up
as one state change.
Cisco MDS
NX-OS
Release
6.2(17)
The following command was modified:
monitor counter state-change (port-monitor configuration mode)
Port Monitor
Cisco MDS
NX-OS
Release
6.2(15)
•
Port Monitor Warning Threshold—Users can configure an
optional lower threshold value than the rising threshold value
(in addition to the rising and falling threshold) to generate
syslogs.
The following command was introduced:
counter counter_name poll-interval time interval delta rising-threshold
value event event number warning-threshold value falling-threshold
value event event number
Set the numerical warning-threshold limit in the range 0 to
9223372036854775807.
•
Port Monitor Check Interval—Users can check errors at
frequent intervals before a poll interval expires.
The following commands were introduced:
– port-monitor check-interval time interval
– no port-monitor check-interval time interval
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-2
Chapter 3
Configuring Interfaces
Feature Information for Interfaces
Table 3-1
New and Changed Features for Cisco MDS NX-OS Release 6.2(x) (continued)
Feature Name Release
Slow-Drain
Detection and
Mitigation
Enhancements
Cisco MDS
NX-OS
Release
6.2(13)
Feature Information
•
Slow-Drain Device Detection—Users can detect slow-drain
devices that cause congestion in a network. The feature also
provides a congestion-mitigation function.
Slow-port monitoring is supported on 8 Gbps and advanced
8-Gbps modules.
The following commands were introduced:
– show process creditmon slowport-monitor-events
– system timeout slowport-monitor
•
TxWait—The advanced 8 and 16 Gbps modules support
slow-port monitoring using the transmit wait feature. The
transmit credit unavailable history is graphically represented
with a transmit-wait history graph.
The following commands were introduced:
– show interface fcx/y counters
– show process creditmon txwait-history
•
On-Board Failure Logging—The slow-port monitor events and
the TxWait delta values were logged in Onboard Failure
Logging (OBFL) periodically.
The following commands were introduced:
– show logging onboard slowport-monitor-events
– show logging onboard txwait
•
Port Monitor Alerting—Three new counters were added to the
port monitor policy:
– tx-slowport-count
– tx-slowport-oper-delay
– txwait
The following commands were introduced:
– counter tx-slowport-count
– counter tx-slowport-oper-delay
– counter txwait
•
Slow Drain—Users can display various debug logs related to
slow drain.
The following command was introduced:
show tech-support slowdrain
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-3
Chapter 3
Configuring Interfaces
Prerequisites for Interfaces
Prerequisites for Interfaces
Before you begin configuring interfaces, ensure that the modules in the chassis are functioning as
designed. To verify the status of a module, enter the show module command in user EXEC mode. For
information about verifying the module status, refer to the Cisco MDS 9000 Series NX-OS Fundamentals
Configuration Guide.
Guidelines and Limitations for Interfaces
When you activate a port-monitor policy using the port-monitor activate policyname command, a
syslog is generated to display that the policy is activated successfully. However, when you disable the
policy using the no port-monitor activate policyname command and enable the policy again, there is
no syslog message displayed about the policy activation. Use the no logging rate-limit command in the
configuration mode to ensure that all syslogs are logged.
The guidelines and limitations for interfaces configuration are listed in the following topics:
•
Guidelines for Configuring Port Monitor Interval, page 3-4
•
Guidelines for Local Switching, page 3-5
•
Guidelines for 10-Gbps Fibre Channel Mode, page 3-5
•
Guidelines for VSAN Interface Configuration, page 3-6
Guidelines for Configuring Port Monitor Interval
•
Check interval should be configured before activating port monitor policies.
Note
The value of the check interval is common across counters and policies.
•
Check interval should be less than the poll interval.
•
Check interval is applicable to all the active port monitor policies configured.
•
Users should deactivate all the active port monitor policies before enabling, modifying, or disabling
the check interval functionality.
•
Check interval cannot be enabled when an active policy is configured.
•
Software downgrade to a version that does not support the check interval functionality is restricted
when the check interval functionality is enabled.
•
We recommend that you do not have a port guard action set to the state-change counter when an
interface state is changed from down state to up state.
•
We recommend that you do not use the default policy when the check interval is configured.
Example 3-1
Check Interval
Let us consider a scenario where the poll interval, rising threshold, and check interval are configured
with the following values:
•
Poll interval is 100 seconds
•
Rising threshold is 30
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-4
Chapter 3
Configuring Interfaces
Guidelines and Limitations for Interfaces
Check interval is 20 seconds
354416
•
P1=0
C1=0
C2=20
C3=40
C4=60
C5=80
P2=100
C6=100
C7=120
C8=140
C9=160 C10=180 P3=200
C11=200
P = Poll interval = 100 seconds
C = Check interval = 20 seconds
The check interval starts its interval, C1, along with the poll interval at P1. If an error occurs between
the check intervals C2 and C3, the check intervals C2 and C3 are higher than the configured rising
threshold value of 30, an alert (syslog or trap or both) is generated at C3, alerting the user that an error
has occurred at that particular port.
Note
You can configure longer poll intervals to capture events across poll intervals. For example, configure a
poll interval of 24 hours with a check interval of 30 seconds, with the rising threshold value being
checked cumulatively every 30 seconds.
Guidelines for Local Switching
Note
•
All the ports should be in shared mode, which is usually the default state. To place a port in shared
mode, enter the switchport rate-mode shared command.
•
E ports are not allowed in the module because they must be in dedicated mode.
Local switching is not supported on the Cisco MDS 9710 switch.
Guidelines for 10-Gbps Fibre Channel Mode
•
To change the port speed from 10 to 16 Gbps, use only the no 10g-speed-mode command. We do
not recommend using the 16g-speed-mode command because the ports will move to unrecoverable
state, and the only way to recover these ports is to issue the no 10g-speed-mode command.
•
For Cisco MDS 9513, the ports in the module can be configured to 10-Gbps speed only when the
DS-13SLT-FAB3 (fabric 3) module bandwidth is 256 Gbps. Any other combination of fabric
modules or Cisco MDS 9506 or Cisco MDS 9509 will not let the ports come up in 10 Gbps.
•
When the 8-Gbps modules are in 10-Gbps mode, the ports in the module that are not 10-Gbps
capable are disabled and are in the out-of-service state. For DS-X9232-256K9, the ASIC range is
eight ports, of which two ports will be out of service. For DS-X9248-256K9, the ASIC range is 12
ports, of which six ports will be out of service. For the 16-Gbps modules and fabric switch, all the
ports have 10-G speed mode.
•
The ports function only in full rate mode. They cannot be moved to shared rate mode.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-5
Chapter 3
Configuring Interfaces
Default Settings for Interface Parameters
•
The ports cannot be configured in any other speed other than the speed values provided in the
switchport speed command.
•
Ports that are 10 Gbps capable and are disabled or are out of service cannot be put back in service
using the no out-of-service command. To put these ports back in service, all the ports in the ASIC
range need to be reconfigured with the no 10g-speed-mode command.
•
Local switching must be disabled. Otherwise, ports cannot be configured in dedicated mode.
Thus, for interconnecting 16-Gbps Fibre Channel modules, 16 Gbps is the preferred speed. However, for
interconnecting 8-Gbps modules, or for interconnecting 16-Gbps modules and 8-Gbps modules, we
recommend 10 Gbps as the preferred speed.
Guidelines for VSAN Interface Configuration
Tip
•
Create a VSAN before creating the interface for that VSAN. If a VSAN does not exist, the interface
cannot be created.
•
Create the interface VSAN; it is not created automatically.
•
If you delete a VSAN, the attached interface is automatically deleted.
•
Configure each interface only in one VSAN.
After configuring a VSAN interface, you can configure an IP address or the Virtual Router Redundancy
Protocol (VRRP) feature. See the Cisco MDS 9000 Series NX-OS IP Services Configuration Guide.
Default Settings for Interface Parameters
Table 3-2 lists the default settings for interface parameters.
Table 3-2
Default Settings for Interface Parameters
Parameter
Default
Interface mode
Auto
Interface speed
Auto
Administrative state
Shutdown (unless changed during initial setup)
Trunk mode
On (unless changed during initial setup) in
non-NPV and NPIV core switches. Off in NPV
switches.
Trunk-allowed VSANs or VF-IDs
1 to 4093
Interface VSAN
Default VSAN (1)
Beacon mode
Off (disabled)
EISL encapsulation
Disabled
Data field size
2112 bytes
Internal CRC error handling
Disabled
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-6
Chapter 3
Configuring Interfaces
Information About Interfaces
Information About Interfaces
The main function of a switch is to relay frames from one data link to another. To relay frames, the
characteristics of the interfaces through which the frames are received and sent must be defined. The
configured interfaces can be Fibre Channel interfaces, Gigabit Ethernet interfaces, the management
interface, or VSAN interfaces.
This section includes the following topics:
•
Interface Description, page 3-7
•
Interface Modes, page 3-7
•
Interface States, page 3-11
•
Graceful Shutdown, page 3-14
•
10-Gbps Fibre Channel Mode, page 3-15
•
Port Administrative Speeds, page 3-17
•
Frame Encapsulation, page 3-17
•
Debounce Timer, page 3-18
•
Bit Error Rate Threshold, page 3-18
•
SFP Transmitter Types, page 3-19
•
Port Guard, page 3-19
•
Port Monitor, page 3-21
•
Port Group Monitor, page 3-25
•
Local Switching, page 3-26
•
Slow-Drain Device Detection and Congestion Avoidance, page 3-26
•
Interface Types, page 3-26
Interface Description
For Fibre Channel interfaces, you can configure the description parameter to provide a recognizable
name for an interface. Using a unique name for each interface allows you to quickly identify an interface
when you are looking at a listing of multiple interfaces. You can also use the description to identify the
traffic or the use for a specific interface.
Interface Modes
Each physical Fibre Channel interface in a switch operates in one of the following port modes
(see Figure 3-1):
•
E Port, page 3-9
•
F Port, page 3-9
•
FL Port, page 3-9
•
NP Ports, page 3-9
•
TE Port, page 3-9
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-7
Chapter 3
Configuring Interfaces
Information About Interfaces
•
TF Port, page 3-10
•
TNP Port, page 3-10
•
SD Port, page 3-10
•
ST Port, page 3-10
•
Fx Port, page 3-10
•
B Port, page 3-11
•
Auto Mode, page 3-11
Note
Besides these modes, each interface can be configured in auto port mode or Fx port mode. These
two modes determine the port type during interface initialization.
Figure 3-1
Cisco MDS 9000 Series Switch Port Modes
NL port
p
N port
Private
loop
ISL lin k
F port
FL port
E port
NL port
E port
TL port
Public
loop
NL port
79528
NL port
Note
Interfaces are created in VSAN 1 by default. For information about VSANs, see the Cisco MDS 9000
Series NX-OS Fabric Configuration Guide.
Each interface has an associated administrative configuration and an operational status:
Note
•
The administrative configuration does not change unless you modify it. This configuration has
various attributes that you can configure in administrative mode.
•
The operational status represents the current status of a specified attribute, such as the interface
speed. This status cannot be changed and is read-only. Some values, for example, operational speed,
may not be valid when the interface is down.
When a module is removed and replaced with the same type of module, the original configuration is
retained. If a different type of module is inserted, the original configuration is no longer retained.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-8
Chapter 3
Configuring Interfaces
Information About Interfaces
E Port
In expansion port (E port) mode, an interface functions as a fabric expansion port. This port can be
connected to another E port to create an Inter-Switch Link (ISL) between two switches. E ports carry
frames between switches for configuration and fabric management. They serve as a conduit between
switches for frames destined for remote N ports and NL ports. E ports support Class 2, Class 3, and Class
F services.
An E port connected to another switch can also be configured to form a port channel. For more details
about configuring a port channel, see Chapter 6, “Configuring PortChannels”.
F Port
In fabric port (F port) mode, an interface functions as a fabric port. This port can be connected to a
peripheral device (host or disk) operating as an N port. An F port can be attached to only 1 N port. F
ports support Class 2 and Class 3 services.
FL Port
In fabric loop port (FL port) mode, an interface functions as a fabric loop port. This port can be
connected to one or more NL ports (including FL ports in other switches) to form a public, arbitrated
loop. If more than one FL port is detected on the arbitrated loop during initialization, only one FL port
becomes operational and the other FL ports enter nonparticipating mode. FL ports support Class 2 and
Class 3 services.
Note
FL port mode is not supported on 4-port 10 Gbps switching module interfaces.
NP Ports
An NP port is a port on a device that is in NPV mode and connected to the core switch via an F port. NP
ports function like N ports, except that in addition to providing N port operations, they also function as
proxies for multiple physical N ports.
For more details about NP ports and NPV, see Chapter 7, “Configuring N Port Virtualization.”
TE Port
In trunking E port (TE port) mode, an interface functions as a trunking expansion port. It can be
connected to another TE port to create an extended ISL (EISL) between two switches. TE ports are
specific to Cisco MDS 9000 Series Multilayer Switches. These switches expand the functionality of E
ports to support the following:
•
VSAN trunking
•
Transport quality of service (QoS) parameters
•
Fibre Channel Trace (fctrace) feature
In TE port mode, all the frames are transmitted in EISL frame format, which contains VSAN
information. Interconnected switches use the VSAN ID to multiplex traffic from one or more VSANs
across the same physical link. This feature is referred to as trunking in the Cisco MDS 9000 Series
Multilayer Switches. For more details about trunking, see Chapter 5, “Configuring Trunking”. TE ports
support Class 2, Class 3, and Class F services.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-9
Chapter 3
Configuring Interfaces
Information About Interfaces
TF Port
In trunking F port (TF port) mode, an interface functions as a trunking expansion port. It can be
connected to another trunked N port (TN port) or trunked NP port (TNP port) to create a link between a
core switch and an NPV switch or a host bus adapter (HBA) in order to carry tagged frames. TF ports
are specific to Cisco MDS 9000 Series Multilayer Switches. They expand the functionality of F ports to
support VSAN trunking.
In TF port mode, all the frames are transmitted in EISL frame format, which contains VSAN
information. Interconnected switches use the VSAN ID to multiplex traffic from one or more VSANs
across the same physical link. This feature is referred to as trunking in the Cisco MDS 9000 Series
Multilayer Switches. For more details about trunking, see Chapter 5, “Configuring Trunking”. TF ports
support Class 2, Class 3, and Class F services.
TNP Port
In trunking NP port (TNP port) mode, an interface functions as a trunking expansion port. It can be
connected to a trunked F port (TF port) to create a link to a core NPIV switch from an NPV switch in
order to carry tagged frames.
SD Port
In SPAN destination port (SD port) mode, an interface functions as a switched port analyzer (SPAN).
The SPAN feature is specific to switches in the Cisco MDS 9000 Series. It monitors network traffic that
passes though a Fibre Channel interface. This is done using a standard Fibre Channel analyzer (or a
similar switch probe) that is attached to an SD port. SD ports do not receive frames; they only transmit
a copy of the source traffic. The SPAN feature is non-intrusive and does not affect switching of network
traffic in SPAN source ports. For more details about SPAN, see the Cisco MDS 9000 Series NX-OS
System Management Configuration Guide.
ST Port
In the SPAN tunnel port (ST port) mode, an interface functions as an entry point port in the source switch
for the RSPAN Fibre Channel tunnel. The ST port mode and the remote SPAN (RSPAN) feature are
specific to switches in the Cisco MDS 9000 Series Multilayer Switches. When configured in ST port
mode, the interface cannot be attached to any device, and thus cannot be used for normal Fibre Channel
traffic. For more details about SPAN, see the Cisco MDS 9000 Series NX-OS System Management
Configuration Guide.
Note
ST port mode is not supported on the Cisco MDS 9124 Fabric Switch, the Cisco Fabric Switch for HP
c-Class BladeSystem, and the Cisco Fabric Switch for IBM BladeCenter.
Fx Port
Interfaces configured as Fx ports can operate in either F port mode or FL port mode. The Fx port mode
is determined during interface initialization depending on the attached N port or NL port. This
administrative configuration disallows interfaces to operate in any other mode, for example, preventing
an interface to connect to another switch.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-10
Chapter 3
Configuring Interfaces
Information About Interfaces
B Port
While E ports typically interconnect Fibre Channel switches, some SAN extender devices, such as the
Cisco PA-FC-1G Fibre Channel port adapter, implement a bridge port (B port) model to connect
geographically dispersed fabrics. This model uses B ports as described in the T11 Standard FC-BB-2.
If an FCIP peer is a SAN extender device that supports only Fibre Channel B ports, you should enable
the B port mode for the FCIP link. When a B port mode is enabled, the E port functionality is also
enabled and they coexist. Even if the B port mode is disabled, the E port functionality remains enabled.
For more details about SPAN, see the Cisco MDS 9000 Series NX-OS IP Services Configuration Guide.
Auto Mode
Interfaces configured in auto mode can operate in F port, FL port, E port, TE port, or TF port mode. The
port mode is determined during interface initialization. For example, if the interface is connected to a
node (host or disk), it operates in F port mode or FL port mode depending on the N port mode or NL port
mode. If the interface is attached to a third-party switch, it operates in E port mode. If the interface is
attached to another switch in the Cisco MDS 9000 Series Multilayer Switches, it may become
operational in TE port mode. For more details about trunking, see Chapter 5, “Configuring Trunking”.
TL ports and SD ports are not determined during initialization and are administratively configured.
Note
Fibre Channel interfaces on Storage Services Modules (SSMs) cannot be configured in auto mode.
Interface States
An interface state depends on the administrative configuration of the interface and the dynamic state of
the physical link.
The section includes the following topics:
•
Administrative States, page 3-11
•
Operational States, page 3-11
•
Reason Codes, page 3-12
Administrative States
Administrative state refers to the administrative configuration of an interface, as described in Table 3-3.
Table 3-3
Administrative States
Administrative
State
Description
Up
Interface is enabled.
Down
Interface is disabled. If you administratively disable an interface by shutting
down that interface, the physical link layer state change is ignored.
Operational States
Operational state indicates the current operational state of an interface, as described in Table 3-4.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-11
Chapter 3
Configuring Interfaces
Information About Interfaces
Table 3-4
Operational States
Operational State Description
Up
Interface is transmitting or receiving traffic, as required. To be in this state, an
interface must be administratively up, the interface link layer state must be up, and
the interface initialization must be completed.
Down
Interface cannot transmit or receive (data) traffic.
Trunking
Interface is operational in TE mode or TF mode.
Reason Codes
Reason codes are dependent on the operational state of an interface, as described in Table 3-5.
Table 3-5
Reason Codes for Interface States
Administrative
Configuration
Operational
Status
Reason Code
Up
Up
None.
Down
Down
Administratively down—If you administratively configure an interface
as down, you disable the interface. No traffic is received or transmitted.
Up
Down
See Table 3-6. Only some of the reason codes are listed in Table 3-6.
If the administrative state is up and the operational state is down, the reason code differs based on the
nonoperational reason code, as described in Table 3-6.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-12
Chapter 3
Configuring Interfaces
Information About Interfaces
Table 3-6
Reason Codes for Nonoperational States
Reason Code (Long Version)
Description
Applicable
Modes
Link failure or Not connected
The physical layer link is not operational.
All
SFP not present
The small form-factor pluggable (SFP) hardware is not
plugged in.
Initializing
The physical layer link is operational and protocol
initialization is in progress.
Reconfigure fabric in progress The fabric is currently being reconfigured.
Offline
The Cisco NX-OS software waits for the specified
R_A_TOV time before retrying initialization.
Inactive
The interface VSAN is deleted or is in a suspended
state.
To make the interface operational, assign that port to a
configured and active VSAN.
Hardware failure
A hardware failure is detected.
Error disabled
Error conditions require administrative attention.
Interfaces may be error-disabled for various reasons:
•
Configuration failure
•
Incompatible buffer-to-buffer credit configuration
To make the interface operational, fix the error
conditions causing this state, and administratively shut
down or enable the interface.
Fibre Channel redirect failure
A port is isolated because a Fibre Channel redirect is
unable to program routes.
No port activation license
available
A port is not active because it does not have a port
license.
SDM failure
A port is isolated because SDM is unable to program
routes.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-13
Chapter 3
Configuring Interfaces
Information About Interfaces
Table 3-6
Reason Codes for Nonoperational States (continued)
Applicable
Modes
Reason Code (Long Version)
Description
Isolation due to ELP failure
The port negotiation failed.
Isolation due to ESC failure
The port negotiation failed.
Isolation due to domain
overlap
The Fibre Channel domains (fcdomain) overlap.
Isolation due to domain ID
assignment failure
The assigned domain ID is not valid.
Isolation due to the other side
of the link E port isolated
The E port at the other end of the link is isolated.
Isolation due to invalid fabric
reconfiguration
The port is isolated due to fabric reconfiguration.
Isolation due to domain
manager disabled
The fcdomain feature is disabled.
Isolation due to zone merge
failure
The zone merge operation failed.
Isolation due to VSAN
mismatch
The VSANs at both ends of an ISL are different.
Nonparticipating
Only FL
FL ports cannot participate in loop operations. This
might occur if more than one FL port exists in the same ports and TL
ports
loop, in which case, all but one FL port in that loop
automatically enters nonparticipating mode.
Port Channel administratively
down
The interfaces belonging to a port channel are down.
Suspended due to incompatible The interfaces belonging to a port channel have
speed
incompatible speeds.
Only E ports
and TE ports
Only port
channel
interfaces
Suspended due to incompatible The interfaces belonging to a port channel have
mode
incompatible modes.
Suspended due to incompatible An improper connection is detected. All the interfaces
remote switch WWN
in a port channel must be connected to the same pair of
switches.
Graceful Shutdown
Interfaces on a port are in a shut-down state by default (unless you modified the initial configuration).
The Cisco NX-OS software implicitly performs a graceful shutdown in response to either of the
following actions for interfaces operating in E port mode:
•
If you shut down an interface.
•
If a Cisco NX-OS software application executes a port shutdown as part of its function.
A graceful shutdown ensures that no frames are lost when the interface is shutting down. When a
shutdown is triggered either by you or the Cisco NX-OS software, the switches connected to the
shutdown link coordinate with each other to ensure that all the frames in the ports are safely sent through
the link before shutting down. This enhancement reduces the chance of frame loss.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-14
Chapter 3
Configuring Interfaces
Information About Interfaces
A graceful shutdown is not possible in the following situations:
Note
•
If you physically remove the port from the switch.
•
If In-Order Delivery (IOD) is enabled. For more details about IOD, see the Cisco MDS 9000 Series
NX-OS Fabric Configuration Guide.
•
If the Min_LS_interval is higher than 10 seconds. For information about Fabric Shortest Path First
(FSPF) global configuration, see the Cisco MDS 9000 Series NX-OS Fabric Configuration Guide.
This feature is triggered only if both the switches at either end of the E port interface are Cisco MDS
switches and are running Cisco SAN-OS Release 2.0(1b) or later, or Cisco MDS NX-OS Release 4.1(1a)
or later.
10-Gbps Fibre Channel Mode
Some Cisco MDS Fibre Channel 8 and 16-Gbps modules and the Cisco MDS 9396S 16-Gbps Fabric
Switch have the capability to run at 10-Gbps speed, and in two modes:
•
1/2/4/8-Gbps (for 8-Gbps modules) or 2/4/8/16-Gbps (for 16-Gbps modules and 9396S 16-Gbps
Fabric Switch).
•
10 Gbps
This section includes the following topics:
•
Benefits of 10-Gbps Fibre Channel Mode, page 3-15
•
Supported Modules and Switches, page 3-15
Benefits of 10-Gbps Fibre Channel Mode
A 10-Gbps Fibre Channel uses a more efficient encoding and a faster clock rate than an 8-Gbps Fibre
Channel. Therefore, it has an approximately 50-percent throughput advantage over an 8-Gbps Fibre
Channel. Consequently, fewer links are needed to achieve a given bandwidth.
Supported Modules and Switches
The following modules and switches support 10-Gbps mode:
Note
•
32-port Cisco MDS 1/2/4/8/10-Gbps Advanced Fibre Channel Module (DS-X9232-256K9)
•
48-port Cisco MDS 1/2/4/8/10-Gbps Advanced Fibre Channel Module (DS-X9248-256K9)
•
48-port Cisco MDS 2/4/8/10/16-Gbps Advanced Fibre Channel Module (DS-X9448-768K9)
•
96-port Cisco MDS 9396S 2/4/8/10/16-Gbps Fabric Switch (DS-C9396S-96EK9)
By default, all the above are in their native Fibre Channel speed (1/2/4/8 or 2/4/8/16 Gbps) mode.
The following tables contain information about each module and the port ranges that need to be
configured in 10-Gbps speed:
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-15
Chapter 3
Configuring Interfaces
Information About Interfaces
Table 3-7
32-Port Cisco MDS 1/2/4/8/10-Gbps Advanced Fibre Channel Module (DS-X9232-256K9)
ASIC Port Range
10-G Port
Offline Port
1-8
2-6,8
1,7
9-16
10-14,16
9,15
17-24
18-22,24
17,23
25-32
26-30,32
25,31
Table 3-8
48-Port Cisco MDS 1/2/4/8/10-Gbps Advanced Fibre Channel Module (DS-X9248-256K9)
ASIC Port Range
10-G Port
Offline Port
1-12
4-8,10
1-3,9,11-12
13-24
16-20,22
13-15,21,23-24
25-36
28-32,34
25-27,33,35-36
37-48
40-44,46
37-39, 45,47-48
Table 3-9
48-Port Cisco MDS 2/4/8/10/16-Gbps Advanced Fibre Channel Module (DS-X9448-768K9)
ASIC Port Range
Offline Port
1-8
None
9-16
None
17-24
None
25-32
None
33-40
None
41-48
None
Table 3-10
96-Port Cisco MDS 9396S 2/4/8/10/16-Gbps Fabric Switch (DS-C9396S-96EK9)
ASIC Port Range
Offline Port
1-8
None
9-16
None
17-24
None
25-32
None
33-40
None
41-48
None
49-56
None
57-64
None
65-72
None
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-16
Chapter 3
Configuring Interfaces
Information About Interfaces
Table 3-10
96-Port Cisco MDS 9396S 2/4/8/10/16-Gbps Fabric Switch (DS-C9396S-96EK9) (continued)
ASIC Port Range
Offline Port
73-80
None
81-88
None
89-96
None
Port Administrative Speeds
By default, the port administrative speed for an interface is automatically calculated by the switch.
For internal ports on the Cisco Fabric Switch for HP c_Class BladeSystem and Cisco Fabric Switch for
IBM BladeCenter, a port speed of 1 Gbps is not supported. Auto negotiation is supported between 2 and
4 Gbps only. Also, if the BladeCenter is a T chassis, then port speeds are fixed at 2 Gbps, and auto
negotiation is not enabled.
Auto Sensing
Auto sensing speed is enabled on all 4 and 8-Gbps switching module interfaces by default. This
configuration enables the interfaces to operate at speeds of 1, 2, or 4 Gbps on 4 Gbps switching modules,
and 8 Gbps on 8-Gbps switching modules. When auto sensing is enabled for an interface operating in
dedicated rate mode, 4 Gbps of bandwidth is reserved even if the port negotiates at an operating speed
of 1 or 2 Gbps.
To avoid wasting unused bandwidth on 48-port and 24-port 4 and 8 Gbps Fibre Channel switching
modules, you can specify that only 2 Gbps of required bandwidth be reserved, not the default of 4 or 8
Gbps. This feature shares the unused bandwidth within the port group, provided the bandwidth does not
exceed the rate limit configuration for the port. You can also use this feature for shared rate ports that
are configured for auto sensing.
Tip
When migrating a host that supports up to 2-Gbps traffic (that is, not 4 Gbps with auto-sensing
capabilities) to the 4 Gbps switching modules, use auto sensing with a maximum bandwidth of 2 Gbps.
When migrating a host that supports up to 4-Gbps traffic (that is, not 8 Gbps with auto-sensing
capabilities) to the 8 Gbps switching modules, use auto sensing with a maximum bandwidth of 4 Gbps.
Frame Encapsulation
The switchport encap eisl command applies only to SD port interfaces. This command determines the
frame format for all the frames transmitted by the interface in SD port mode. If the encapsulation is set
to EISL, all outgoing frames are transmitted in the EISL frame format, regardless of the SPAN sources.
For information about encapsulation, see the Cisco MDS 9000 Series NX-OS System Management
Configuration Guide.
The switchport encap eisl command is disabled by default. If you enable encapsulation, all outgoing
frames are encapsulated, and you will see a new line (Encapsulation is eisl) in the show interface
SD_port_interface command output. For information about encapsulation, see the Cisco MDS 9000
Series NX-OS System Management Configuration Guide.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-17
Chapter 3
Configuring Interfaces
Information About Interfaces
Debounce Timer
The debounce timer delays the notification of a link change that can decrease traffic loss due to network
reconfiguration. The default value for debounce timer link down is 100 ms for Fibre Channel interfaces.
This value cannot be configured. If there is a synchronization loss for less than 100 ms, the Fibre Channel
interface will not bounce.
Bit Error Rate Threshold
The bit error rate (BER) threshold is used by a switch to detect an increased error rate before
performance degradation seriously affects traffic.
Bit errors occur because of the following reasons:
•
Faulty or bad cable
•
Faulty or bad Gigabit Interface Converter (GBIC) or Small Form-Factor Pluggable (SFP)
•
GBIC or SFP is specified to operate at 1 Gbps, but is used at 2 Gbps
•
GBIC or SFP is specified to operate at 2 Gbps, but is used at 4 Gbps
•
Short-haul cable is used for long haul or long-haul cable is used for short haul
•
Momentary synchronization loss
•
Loose cable connection at one end or both ends
•
Improper GBIC or SFP connection at one end or both ends
A BER threshold is detected when 15 error bursts occur in a 5-minute period. By default, the switch
disables the interface when the threshold is reached. Use the shutdown and no shutdown command
sequence to re-enable the interface.
Disabling the Bit Error Rate Action
By default, the threshold disables the interface. However, you can configure the switch to not disable an
interface when the threshold is crossed.
To disable the BER threshold for an interface, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc1/1
Step 3
Prevent the detection of BER events from disabling the interface:
switch(config-if)# switchport ignore bit-errors
(Optional) Prevent the detection of BER events from enabling the interface:
switch(config-if)# no switchport ignore bit-errors
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-18
Chapter 3
Configuring Interfaces
Information About Interfaces
Note
Regardless of the setting of the switchport ignore bit-errors command, a switch generates a syslog
message when the BER threshold is exceeded.
SFP Transmitter Types
The SFP hardware transmitters are identified by their acronyms when displayed using the show
interface brief command. If the related SFP has a Cisco-assigned extended ID, the show interface and
show interface brief commands display the ID instead of the transmitter type. The show interface
transceiver and show interface fc slot/port transceiver commands display both values (ID and
transmitter type) for Cisco-supported SFPs. Table 3-11 defines the acronyms used in the command
output. For information about how to display interface information, see the Chapter 3, “Displaying
Interface Information”.
Table 3-11
SFP Transmitter Acronym Definitions
Definition
Acronym
Standard transmitters defined in the GBIC specifications
Short wave laser
swl
Long wave laser
lwl
Long wave laser cost reduced
lwcr
Electrical
elec
Extended transmitters assigned to Cisco-supported SFPs
CWDM-1470
c1470
CWDM-1490
c1490
CWDM-1510
c1510
CWDM-1530
c1530
CWDM-1550
c1550
CWDM-1570
c1570
CWDM-1590
c1590
CWDM-1610
c1610
Port Guard
The Port Guard feature is intended for use in environments where systems do not adapt quickly to a port
going down and up (single or multiple times). For example, if a large fabric takes 5 seconds to stabilize
after a port goes down, but the port actually goes up and down once per second, a severe failure might
occur in the fabric, including devices becoming permanently unsynchronized.
The Port Guard feature provides the SAN administrator with the ability to prevent this issue from
occurring. A port can be configured to stay down after a specified number of failures in a specified time
period. This allows the SAN administrator to automate fabric stabilization, thereby avoiding problems
caused by the up-down cycle.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-19
Chapter 3
Configuring Interfaces
Information About Interfaces
Using the Port Guard feature, the SAN administrator can restrict the number of error events and bring a
malfunctioning port to down state dynamically once the error events exceed the event threshold. A port
can be configured such that it shuts down when specific failures occur.
There are two types of port guard, Port Level type and Port Monitor type. The former is a basic type
where event thresholds are configurable on a per port basis, the latter allows the configuration of policies
that are applied to all the ports of the same type, for example, all E ports or all F ports.
Note
We recommend against the simultaneous use of both types of port guard for a given port.
This section includes the following topics:
•
Port-Level Port Guard, page 3-20
•
Port Monitor Port Guard, page 3-20
Port-Level Port Guard
The following is the list of events that can be used to trigger port-level port guard actions:
•
TrustSec violation—Link fails because of excessive TrsustSec violation events.
•
Bit errors—Link fails because of excessive bit error events.
•
Signal loss—Link fails because of excessive signal loss events.
•
Signal synchronization loss—Link fails because of excessive signal synchronization events.
•
Link reset—Link fails because of excessive link reset events.
•
Link down—Link fails because of excessive link down events.
•
Credit loss (Loop F ports only)—Link fails because of excessive credit loss events.
A link failure occurs when it receives two bad frames in an interval of 10 seconds and the respective
interface will be error disabled. A general link failure caused by link down is the superset of all other
causes. The sum of the number of all other causes equals the number of link-down failures. This means
that a port is brought to down state when it reaches the maximum number of allowed link failures or the
maximum number of specified causes.
Port-level Port Guard can be used to shut down misbehaving ports based on certain link event types.
Event thresholds are configurable for each event type per port which makes them customizable between
host, array, and tape F ports, or between intra- and inter-data center E ports, for example.
The events listed above might get triggered by certain events on a port, such as:
– Receipt of Not Operational Signal (NOS)
– Too many hardware interrupts
– The cable is disconnected
– The detection of hardware faults
– The connected device is rebooted (F ports only)
– The connected modules are rebooted (E ports only)
Port Monitor Port Guard
The Port Monitor Port Guard feature allows a port to be automatically error disabled or flapped when a
given event threshold is reached.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-20
Chapter 3
Configuring Interfaces
Information About Interfaces
Note
The Port Monitor Port Guard is not available for absolute counters.
The following is the list of events that can be used to trigger the Port Monitor Port Guard actions:
•
Port-to-forwarding engine frame error
•
Crossbar-to-forwarding engine frame error
•
Forwarding engine-to-crossbar frame error
•
Credit loss
•
Link loss
•
Signal loss
•
Signal synchronization loss
•
Received data rate
•
Received invalid CRC
•
Received invalid words
•
Received logical link resets
•
Transmit credit not available
•
Transmit data rate
•
Transmit discards
•
Transmit logical link resets
•
Transmit slow port events
•
Transmit wait
•
Transit timeout discards
Port Monitor
The Port Monitor feature can be used to monitor the performance and status of ports and generate alerts
when problems occur. You can configure thresholds for various counters and enable event triggers when
the values cross the threshold.
For rising and falling thresholds, a syslog is generated only when the error count crosses these threshold
values.
Table 3-12 displays the default port monitor policy with threshold values. The unit for threshold values
(rising and falling) differs across different counters.
Table 3-12
Counter
Default Port Monitor Policy with Threshold Values
Threshold Interval Rising
Falling
Warning PMON Port
Type
(Seconds) Threshold Event Threshold Event Threshold Guard
Link Loss Delta
60
5
4
1
4
Not
enabled
Not enabled
Sync Loss Delta
60
5
4
1
4
Not
enabled
Not enabled
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-21
Chapter 3
Configuring Interfaces
Information About Interfaces
Table 3-12
Counter
Default Port Monitor Policy with Threshold Values (continued)
Threshold Interval Rising
Falling
Warning PMON Port
Type
(Seconds) Threshold Event Threshold Event Threshold Guard
Signal
Loss
Delta
60
5
4
1
4
Not
enabled
Not enabled
State
change
Delta
60
5
4
0
4
Not
enabled
Not enabled
Invalid
Words
Delta
60
1
4
0
4
Not
enabled
Not enabled
Invalid
CRCs
Delta
60
5
4
1
4
Not
enabled
Not enabled
Transmit
(TX)
Discards
Delta
60
200
4
10
4
Not
enabled
Not enabled
Link
Reset
(LR)
Receive
(RX)
Delta
60
5
4
1
4
Not
enabled
Not enabled
LR TX
Delta
60
5
4
1
4
Not
enabled
Not enabled
Timeout
Discards
Delta
60
200
4
10
4
Not
enabled
Not enabled
Credit
Delta
Loss Reco
1
1
4
0
4
Not
enabled
Not enabled
TX Credit Delta
Not
Available
1
10%
4
0%
4
Not
enabled
Not enabled
RX
Datarate
Delta
60
80%
4
20%
4
Not
enabled
Not enabled
TX
Datarate
Delta
60
80%
4
20%
4
Not
enabled
Not enabled
1
5
4
0
4
Not
enabled
Not enabled
1
50 ms
4
0 ms
4
Not
enabled
—
4
0%
4
Not
enabled
Not enabled
TX-Slowp Delta
ort-Count
1
TX-Slowp Absolute
ort-OperDelay2
80 ms
(Advanced
8-Gbps
modules)
TXWait3
Delta
1
40%
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-22
Chapter 3
Configuring Interfaces
Information About Interfaces
Table 3-12
Counter
Default Port Monitor Policy with Threshold Values (continued)
Threshold Interval Rising
Falling
Warning PMON Port
Type
(Seconds) Threshold Event Threshold Event Threshold Guard
err-pkt-fr —
om-port_
ASIC
Error Pkt
from Port4
—
—
—
—
—
—
—
err-pkt-to
-xbar_
ASIC
Error Pkt
to xbar4
—
—
—
—
—
—
—
—
err-pkt-fr
om-xbar_
ASIC
Error Pkt
from
xbar4
—
—
—
—
—
—
—
—
1. For all platforms, if the default value for tx-slowport-count is modified, ISSD will be restricted. To proceed with ISSD, use the
no form of the counter tx-slowport-count command to roll back to the default value.
2. For all platforms, if the default value for tx-slowport-oper-delay is modified, ISSD will be restricted. To proceed with ISSD,
use the no form of the counter tx-slowport-oper-delay command to roll back to the default value.
3. For all platforms, if the default value for TxWait is modified, ISSD will be restricted. To proceed with ISSD, use the no form
of the counter txwait command to roll back to the default value.
4. The counter was introduced in Cisco NX-OS Release 5.2(2a).
Note
•
TX-Slowport-Count is applicable only for 8-Gbps modules (DS-X9224-96K9, DS-X9248-96K9,
and DS-X9248-48K9) in the Cisco MDS 9500 Series switches. In the default configuration, the port
monitor sends an alert when a slow-port condition is detected 5 times in 1 second for the configured
slow-port monitor timeout. (See the system timeout slowport-monitor command in the Cisco MDS
9000 Series Command Reference).
•
TX-Slowport-Oper-Delay is applicable only for advanced 8 and 16 Gbps modules. There are two
defaults based on the module type:
– For advanced 8-Gbps modules, the default rising threshold is 80 ms in a 1-second polling
interval.
– For 16-Gbps modules, the default rising threshold is 50 ms in a 1-second polling interval.
•
Configuring slow-port monitoring using the system timeout slowport-monitor command in order
to get alerts for TX-Slowport-Count and TX-Slowport-Oper-Delay for a particular port type. (See
the system timeout slowport-monitor command in the Cisco MDS 9000 Series Command
Reference).
•
Port guard action for TX-Slowport-Oper-Delay (for Absolute type counter) is not supported.
•
TXWait is applicable only for advanced 8 and 16 Gbps modules. In the default configuration, the
port monitor sends an alert if the transmit credit is not available for 400 ms (40%) in 1 second.
TXWait sends alerts when there are multiple slow-port events that have not hit the slow-port monitor
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-23
Chapter 3
Configuring Interfaces
Information About Interfaces
threshold, but have together hit the TXWait threshold configured. For example, if there are 40
discrete 10-ms intervals of 0 TX credits in 1 second, TX-Slowport-Oper-Delay does not find these
credits; TXWait finds the credits and sends an alert.
•
The state-change counter records the port down-to-port up action as one state change that is similar
to flap. This is the reason the state-change counter does not have the port guard action set as flap.
•
When the port guard action is set as flap, users get alerts only through syslog.
Three more counters were added in Cisco Release NX-OS 5.2(2a); these are not included in the default
policy:
•
err-pkt-from-port_ASIC Error Pkt from port
•
err-pkt-to-xbar_ ASIC Error Pkt to xbar
•
err-pkt-from-xbar_ ASIC Error Pkt from xbar
Table 3-13 displays the threshold value of the slow-drain port-monitor policy:
Table 3-13
Threshold
Type
Interval
(Seconds)
Rising
Threshold Event
Falling
Threshold Event
PMON Port
Guard
Credit Loss
Reco
Delta
1
1
4
0
4
Not enabled
TX Credit Not
Available
Delta
1
10
4
0
4
Not enabled
Counter
Note
Slow-Drain Port-Monitor Policy Threshold Value
If no other Port Monitor policy is explicitly activated, the slow-drain policy is activated. The default
policy shows only the default counter-monitor values.
This section includes the following topics:
•
Warning Threshold, page 3-24
•
Check Interval, page 3-25
Warning Threshold
From Cisco MDS NX-OS Release 6.2(15), the warning threshold functionality is available for each
counter in a Port Monitor policy.
Port Monitor warning thresholds can be used to generate syslog messages before rising and falling
thresholds are reached. A single threshold is configurable per Port Monitor counter. A syslog is
generated whenever the counter crosses the configured warning threshold in either the rising or falling
direction. This allows the user to track counters that are not severe enough to hit the rising threshold, but
where nonzero events are of interest.
The warning threshold must be equal or less than the rising threshold and equal or greater than the falling
threshold.
The warning threshold is optional; warning syslogs are only generated when it is specified in a counter
configuration.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-24
Chapter 3
Configuring Interfaces
Information About Interfaces
Check Interval
From Cisco MDS NX-OS Release 6.2(15), a new functionality called check interval is introduced to
check errors at a shorter time interval than the poll interval.
Check interval polls for values more frequently within a poll interval so that the errors are detected much
earlier and appropriate action can be taken.
With the existing poll interval, it is not possible to detect errors at an early stage. Users have to wait until
the completion of the poll interval to detect the errors.
By default, the check interval functionality is not enabled.
Note
•
The port monitor check interval feature is supported only on the Cisco MDS 9710 Multilayer
Director, Cisco MDS 9718 Multilayer Directors, and Cisco MDS 9706 Multilayer Directors.
•
Check interval is supported on both counters, absolute and delta.
•
We recommend that you configure the poll interval as a multiple of the check interval.
•
Check interval is supported on the Cisco MDS 9700 Series Multilayer Directors from Cisco MDS
NX-OS Release 6.2(15) onwards, and on the Cisco MDS 9250i Multiservice Fabric Switch from
Cisco MDS NX-OS Release 6.2(17) onwards.
•
When a port comes up, the check interval will not provide an alert about invalid words for the port
until the poll interval expires. We recommend that you bring up a set of ports at a given time in the
module instead of all the ports.
Port Group Monitor
Note
Port Group Monitor functionality only applies to line cards that support oversubscription.
The ports on a line card are divided into fixed groups called port groups that share a link of fixed
bandwidth to the backplane. Since the total port bandwidth can exceed the backplane link bandwidth,
frames will be queued, introducing traffic delays. The Port Group Monitor functionality can be used to
monitor this oversubscription in both the transmit and receive directions to allow ports to be rebalanced
between port groups before the delays become unacceptable.
When the Port Group Monitor feature is enabled and when a policy consisting of polling interval in
seconds and the rising and falling thresholds in percentage are specified, the port group monitor
generates a syslog if port group traffic goes above the specified percentage of the maximum supported
bandwidth for that port group (for receive and for transmit). Another syslog is generated if the value falls
below the specified threshold.
Table 3-14 shows the threshold values for the default Port Group Monitor policy:
Table 3-14
Default Port Group Monitor Policy Threshold Values
Counter
Threshold
Type
Interval
(Seconds)
% Falling
% Rising Threshold Threshold
RX Performance
Delta
60
80
20
TX Performance
Delta
60
80
20
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-25
Chapter 3
Configuring Interfaces
Information About Interfaces
Note
Port group monitor is not supported on a 1-rack box when any of the threshold values is reached for the
receive performance and transmit performance counters.
Local Switching
Local switching can be enabled in advanced 8-Gbps modules. This allows traffic to be switched directly
with a local crossbar when the traffic is directed from one port to another on the same line card. By using
local switching, an extra switching step is avoided, which in turn decreases the latency.
Slow-Drain Device Detection and Congestion Avoidance
Most SAN edge devices use Class 2 or Class 3 Fibre Channel services that have link-level flow control.
This feature allows a receiving port to back pressure the sending peer port when the receiving port
reaches its capacity to accept frames. When an edge device does not accept frames from the fabric for
an extended time, it creates a condition in the fabric known as slow drain. If the upstream source of a
slow-edge device is an ISL, it results in credit starvation in that ISL. This credit starvation then affects
the unrelated flows that use the same ISL link.
Congestion avoidance focuses on minimizing or completely avoiding the held frames from consuming
all the egress buffers of an edge port attached to a slow-drain device. To achieve congestion avoidance,
configure a no-credit frame timeout value that is lower than the default 500-ms frame timeout, which in
turn reduces the effects of the slow-drain device on the fabric. Thus, the slow-moving frames get dropped
faster than the general frame timeout, freeing buffers in the upstream ISL and allowing the unrelated
flows to move continuously.
Note
The no-credit timeout functionality is used for edge ports because these ports are directly connected to
slow-drain devices. Although the no-credit timeout functionality can be applied to core ports, we
recommend that you do not use it. The no-credit timeout functionality is not supported on Generation 1
modules.
Interface Types
The following topics provide information about the interfaces types.
•
Management Interface, page 3-26
•
VSAN Interfaces, page 3-27
Management Interface
You can remotely configure a switch through the management interface (mgmt0). To configure a
connection on the mgmt0 interface, configure either the IPv4 parameters (IP address, subnet mask, and
default gateway), or the IPv6 parameters (IP address, subnet mask, and default gateway) so that the
switch is reachable.
Before you configure the management interface manually, obtain the switch’s IPv4 address, subnet
mask, and default gateway, or the IPv6 address, depending on which IP version you are configuring.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-26
Chapter 3
Configuring Interfaces
Configuring Interfaces
The management port (mgmt0) auto senses and operates in full-duplex mode at a speed of 10, 100, or
1000 Mbps. Auto sensing supports both the speed mode and the duplex mode. On a Supervisor-1 module,
the default speed is 100 Mbps and the default duplex mode is auto. On a Supervisor-2 module, the default
speed and the default duplex mode are set to auto.
Note
Explicitly configure a default gateway to connect to the switch and send IP packets or add a route for
each subnet.
VSAN Interfaces
VSANs are applicable to Fibre Channel fabrics and enable you to configure multiple isolated SAN
topologies within the same physical infrastructure. You can create an IP interface on top of a VSAN, and
then use this interface to send frames to the corresponding VSAN. To use this feature, configure the IP
address for this VSAN.
Note
VSAN interfaces cannot be created for non existing VSANs.
Configuring Interfaces
This section includes the following topics:
•
Configuring a Fibre Channel Interface, page 3-28
•
Setting the Interface Administrative State, page 3-28
•
Configuring an Interface Mode, page 3-29
•
Configuring the MAX NPIV Limit, page 3-30
•
Configuring the System Default F Port Mode, page 3-30
•
Configuring ISL Between Two Switches, page 3-31
•
Configuring the 10-Gbps Fibre Channel Mode via the CLI, page 3-32
•
Configuring the 10-Gbps Fibre Channel Mode via the Device Manager, page 3-32
•
Configuring the Port Administrative Speed, page 3-33
•
Configuring the Interface Description, page 3-33
•
Specifying a Port Owner, page 3-34
•
Configuring Beacon Mode, page 3-34
•
Configuring a Switch Port Attribute Default Value, page 3-35
•
Configuring the Port Guard, page 3-35
•
Configuring Port Monitor, page 3-37
•
Configuring a Port Monitor Port Actions, page 3-39
•
Configuring Port Group Monitor, page 3-41
•
Configuring the Management Interface, page 3-44
•
Creating a VSAN Interface, page 3-45
•
Configuring Slow-Drain Device Detection and Congestion Avoidance, page 3-45
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-27
Chapter 3
Configuring Interfaces
Configuring Interfaces
For more details on configuring an mgmt0 interface, see the Cisco MDS 9000 Series NX-OS
Fundamentals Configuration Guide and the Cisco MDS 9000 Series NX-OS IP Services Configuration
Guide.
For more details on configuring a Gigabit Ethernet interface, see the Cisco MDS 9000 Series NX-OS IP
Services Configuration Guide.
Configuring a Fibre Channel Interface
To configure a Fibre Channel interface, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc1/1
When a Fibre Channel interface is configured, it is automatically assigned a unique world wide name
(WWN). If the interface’s operational state is up, it is also assigned a Fibre Channel ID (FC ID).
To configure a range of interfaces, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select the range of Fibre Channel interfaces and enter interface configuration submode3:
switch(config)# interface fc1/1 - 4 , fc2/1 - 3
Note
When using this command, provide a space before and after the comma.
For the Cisco Fabric Switch for HP c-Class BladeSystem and the Cisco Fabric Switch for IBM
BladeCenter, you can configure a range of interfaces in internal ports or external ports, but you cannot
mix both interface types within the same range. For example, bay 1-10 , bay 12 or ext 0 , ext 15-18 are
valid ranges, but bay 1-5 , ext 15-17 is not.
Setting the Interface Administrative State
To set the interface administrative state, you must first gracefully shut down the interface and enable
traffic flow.
To gracefully shut down an interface, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc1/1
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-28
Chapter 3
Configuring Interfaces
Configuring Interfaces
Step 3
Gracefully shut down the interface and administratively disable the traffic flow; this is the default state:
switch(config-if)# shutdown
To enable traffic flow, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc1/1
Step 3
Enable traffic flow to administratively allow traffic when the no prefix is used (provided the operational
state is up):
switch(config-if)# no shutdown
Configuring an Interface Mode
To configure an interface mode, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc1/1
Step 3
Configure the administrative mode of the port. You can set the operational state to auto, E, F, FL, Fx, TL,
NP, or SD port mode:
switch(config-if)# switchport mode {E | F | FL | Fx | NP | SD}
Note
Step 4
Fx ports refer to an F port or an FL port (host connection only), but not E ports.
Configure interface mode to auto negotiate an E, F, FL, or TE port mode (not TL or SD port modes) of
operation:
switch(config-if)# switchport mode auto
Note
•
TL ports and SD ports cannot be configured automatically. They must be administratively
configured.
•
You cannot configure Fibre Channel interfaces on Storage Services Modules (SSM) in auto mode.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-29
Chapter 3
Configuring Interfaces
Configuring Interfaces
Configuring the MAX NPIV Limit
Both the max-npiv-limit and trunk-max-npiv-limit can be configured on a port or port channel. If the
port or port channel becomes a trunking port, trunk-max-npiv-limit is used for limit checks.
To configure the maximum NPIV limit, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc 3/29
Step 3
Configure switch port mode F on the Fibre Channel interface:
switch(config-if)# switchport mode F
Step 4
Specify the maximum login value for this port:
switch(config-if)# switchport max-npiv-limit 100
The valid range is from 1 to 256.
Configuring the System Default F Port Mode
The system default switchport mode F command sets the administrative mode of all Fibre Channel
ports to mode F, while avoiding traffic disruption caused by the formation of unwanted ISLs. This
command is part of the setup utility that runs during bootup after a write erase or reload command is
issued. It can also be executed from the command line in configuration mode. This command changes
the configuration of the following ports to administrative mode F:
•
All ports that are down and are not out of service.
•
All F ports that are up, whose operational mode is F, and whose administrative mode is not F.
The system default switchport mode F command does not affect the configuration of the following
ports:
•
All user-configured ports, even if they are down.
•
All non-F ports that are up. However, if non-F ports are down, this command changes the
administrative mode of those ports.
Guidelines and Restrictions
•
To ensure that ports that are a part of ISLs do not get changed to port mode F, configure the ports in
port mode E, rather than in auto mode.
•
When the command is executed from the command line, the switch operation remains graceful. No
ports are flapped.
To set the administrative mode of Fibre Channel ports to mode F in the CLI, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-30
Chapter 3
Configuring Interfaces
Configuring Interfaces
Step 2
Sets administrative mode of Fibre Channel ports to mode F (if applicable):
switch(config)# system default switchport mode F
(Optional) Set the administrative mode of Fibre Channel ports to the default (unless user configured),
use the following command:
switch(config)# no system default switchport mode F
Note
For detailed information about the switch setup utility, see the Cisco MDS 9000 Series NX-OS
Fundamentals Configuration Guide.
Example 3-2 shows the command in the setup utility, and Example 3-3 shows the command from the
command line.
Example 3-2
Setup Utility
Configure default switchport mode F (yes/no) [n]: y
Example 3-3
Command Line
switch(config)# system default switchport mode F
Configuring ISL Between Two Switches
Note
Ensure that the Fibre Channel cable is connected between the ports and perform a no-shut operation on
each port.
E-port mode is used when a port functions as one end of an ISL setting. When you set the port mode to
E, you restrict the port coming up as an E port (trunking or nontrunking, depending on the trunking port
mode).
To configure the port mode to E:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc 3/29
Step 3
Configure switch port mode E on the Fibre Channel interface:
switch(config-if)# switchport mode E
Note
Ensure that you perform the task of setting the port mode to E on both the switches between which you
are attempting to bring up the ISL link.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-31
Chapter 3
Configuring Interfaces
Configuring Interfaces
Configuring the 10-Gbps Fibre Channel Mode via the CLI
There are two ways to configure the port speed to the 10-Gbps speed mode:
•
Use the 10g-speed mode command, which is the recommended method.
Note
•
When 10-G speed mode is configured in an interface mode for 16-Gbps modules, all the
ports in an interface mode will be in 10-Gbps mode, whereas in 8-Gbps modules, only
certain ports in an interface mode will be in 10-Gbps mode and the rest will be in the
out-of-service state.
Use the generic switchport speed command.
To configure interface mode, perform these steps. The following is an example on a Cisco MDS 9396S
DS-C9396S-96EK9.
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration mode:
switch(config)# interface fc1/1-8
Ensure that a full ASIC range of ports is selected before executing this command. For example, fcy/1-12
for a 48-port 8-Gbps module or fcy/1-8 for an 8-Gbps 32-port, 48-port 16-Gbps module.
Step 3
Configure all the ports (1 to 8) in Fibre Channel module 1 to 10 Gbps:
switch(config-if)# 10g-speed-mode
For the DS-X9248-256K9 module, the 10g-speed-mode command works only for interface ranges 1–12,
13–24, 25–36, or 37–48.
For the DS-X9232-256K9 module, the 10g-speed-mode command works only for interface ranges 1–8,
9–16, 17–24, or 25–32.
For the DS-X9448-768K9 module, the 10g-speed-mode command works only for interface ranges 1–8,
9–16, 17–24, 25–32, 33–40, or 41–48.
For the DS-C9396S-96EK9 module, the 10g-speed-mode command works only for interface ranges 1-8,
9-16, 17-24, 25-32, 33-40, 41-48, 49-56, 57-64, 65-72, 73-80, 81-88, or 89-96.
(Optional) Revert the settings and put all the ports (1 to 8) in the Out-of-service state and move them to
the In-service state:
switch(config-if)# no 10g-speed-mode
Configuring the 10-Gbps Fibre Channel Mode via the Device Manager
Perform these steps to convert a defined range of interfaces to 10-G mode for a module with
2//4/8/10/16-Gbps Advanced Fibre Channel module (DS-X9448-768K9):
Step 1
Launch the Device Manager for the device supporting 10-G speed.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-32
Chapter 3
Configuring Interfaces
Configuring Interfaces
Step 2
Right-click the module and select Configure bandwidth Reservation.
Step 3
Select one or more ASIC port ranges and click Apply. By default, all the ports are 1/2/4/8 or
2/4/8/16-Gbps speed capable.
Configuring the Port Administrative Speed
Caution
Changing the port administrative speed is a disruptive operation.
To configure the port speed of an interface, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select the Fibre Channel interface and enter interface configuration mode:
switch(config)# interface fc 1/1
Step 3
Configure the port speed of the interface to 1000 Mbps:
switch(config-if)# switchport speed 1000
All the 10 Gbps-capable interfaces, except the interface that is being configured, must be in the
Out-of-service state. At least one other 10 Gbps-capable interface must be in the In-service state.
(Optional) Revert to the factory default (auto) administrative speed of the interface:
switch(config-if)# no switchport speed
Configuring the Interface Description
The interface description can be any alphanumeric string that is up to 80 characters long.
To configure a description for an interface, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc1/1
Step 3
Configure the description of the interface:
switch(config-if)# switchport description cisco-HBA2
(Optional) Clear the description of the interface:
switch(config-if)# no switchport description
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-33
Chapter 3
Configuring Interfaces
Configuring Interfaces
Specifying a Port Owner
Using the Port Owner feature, you can specify the owner of a port and the purpose for which a port is
used so that the other administrators are informed.
Note
The Port Guard and Port Owner features are available for all ports regardless of the operational mode.
To specify or remove a port owner, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select the port interface:
switch(config)# interface fc1/1
Step 3
Specify the owner of the switch port:
switch(config)# switchport owner description
The description can include the name of the owner and the purpose for which the port is used, and can
be up to 80 characters long.
(Optional) Remove the port owner description:
switch(config)# no switchport owner
(Optional) Display the owner description specified for a port, use one of the following commands:
•
switch# show running interface fc module-number/interface-number
•
switch# show port internal info interface fc module-number/interface-number
Configuring Beacon Mode
By default, the beacon mode is disabled on all switches. The beacon mode is indicated by a flashing
green light that helps you identify the physical location of the specified interface. Note that configuring
the beacon mode has no effect on the operation of the interface.
To configure a beacon mode for a specified interface or range of interfaces, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select a Fibre Channel interface and enter interface configuration submode:
switch(config)# interface fc1/1
Step 3
Enable the beacon mode for the interface:
switch(config-if)# switchport beacon
(Optional) Disable the beacon mode for the interface:
switch(config-if)# no switchport beacon
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-34
Chapter 3
Configuring Interfaces
Configuring Interfaces
Note
The flashing green light turns on automatically when an external loopback that causes the interfaces to
be isolated is detected. The flashing green light overrides the beacon mode configuration. The state of
the LED is restored to reflect the beacon mode configuration after the external loopback is removed.
Configuring a Switch Port Attribute Default Value
You can configure default values for various switch port attributes. These attributes will be applied
globally to all future switch port configurations, even if you do not individually specify them at that time.
To configure a default value for a switch port attribute, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Configure the default setting for the administrative state of an interface as up (the factory default setting
is down):
switch(config)# no system default switchport shutdown
Note
This command is applicable only to interfaces for which no user configuration exists for the
administrative state.
(Optional) Configure the default setting for the administrative state of an interface as down:
switch(config)# system default switchport shutdown
Note
This command is applicable only to interfaces for which no user configuration exists for the
administrative state.
(Optional) Configure the default setting for the administrative trunk mode state of an interface as Auto:
switch(config)# system default switchport trunk mode auto
Note
The default setting is On.
Configuring the Port Guard
All port guard causes are monitored over a common time interval with the same start and stop times. The
link down counter is not a specific event, but the aggregation of all other cause counters in the same time
interval.
To configure a port-level port guard for an interface, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select the interface:
switch(config)# interface fc1/1
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-35
Chapter 3
Configuring Interfaces
Configuring Interfaces
Step 3
Enable port guard error disabling of the interface if the link goes down once:
switch(config-if)# errdisable detect cause link-down
(Optional) Enable port guard error disabling of the interface if the link flaps a certain number of times
within the specified time, in seconds:
switch(config-if)# errdisable detect cause link-down [num-times number duration seconds]
(Optional) Remove the port guard configuration for the interface:
switch(config-if)# no errdisable detect cause link-down
The link resumes flapping and sending error reports normally.
Step 4
Enable port guard error disabling of the interface if the specified error occurs once:
switch(config-if)# errdisable detect cause {bit-errors | credit-loss | link-down |
link-reset | signal-loss | sync-loss | trustsec-violation}
(Optional) Enable port guard error disabling of the interface if the specified error occurs a certain
number of times within the specified time, in seconds:
switch(config-if)# errdisable detect cause {bit-errors | credit-loss | link-down |
link-reset | signal-loss | sync-loss | trustsec-violation} [num-times number duration
seconds]
(Optional) Remove the port guard configuration for the interface:
switch(config-if)# no errdisable detect cause {bit-errors | credit-loss | link-down |
link-reset | signal-loss | sync-loss | trustsec-violation}
The link resumes flapping and sending error reports normally.
Note
The port guard credit loss event is triggered only on loop interfaces; it is not triggered on point-to-point
interfaces.
This example shows how to configure port guard to set an interface to Error Disabled state if the link
flaps five times within 120 seconds due to multiple causes. The port guard controls the interface in the
following manner:
•
The interface will be error disabled due to link down if there are link failures due to bit errors 2 times
and link failures due to credit loss 3 times in 120 seconds.
•
The interface will be error disabled due to bit errors if there are link failures due to bit errors 5 times
in 120 seconds.
•
The interface will be error disabled due to credit loss if there are link failures due to credit loss 5
times in 120 seconds.
Switch# configure terminal
Switch (config)# interface fc1/1
Switch (config-if)# errdisable detect cause link-down num-times 5 duration 120
Switch (config-if)# errdisable detect cause bit-errors num-times 5 duration 120
Switch (config-if)# errdisable detect cause credit-loss num-times 5 duration 120
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-36
Chapter 3
Configuring Interfaces
Configuring Interfaces
Configuring Port Monitor
Configuring a port guard action is optional for each counter in a port monitor policy, and is disabled by
default.
This section includes the following topics:
•
Enabling Port Monitor, page 3-37
•
Configuring the Check Interval, page 3-37
•
Configuring a Port Monitor Policy, page 3-37
•
Configuring a Port Monitor Port Actions, page 3-39
•
Activating a Port Monitor Policy, page 3-39
•
Warning Threshold Example, page 3-40
Enabling Port Monitor
To enable or disable port monitor, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Enable port monitoring:
switch(config)# port-monitor enable
(Optional) Disable port monitoring:
switch(config)# no port-monitor enable
Configuring the Check Interval
To configure the check interval, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Configure the check interval time to 30 seconds:
switch(config)# port-monitor check-interval 30
(Optional) Disable the check interval, use the following command:
switch(config)# no port-monitor check-interval
Configuring a Port Monitor Policy
To configure a port monitor policy, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-37
Chapter 3
Configuring Interfaces
Configuring Interfaces
Step 2
Specify the policy name and enter port monitoring policy configuration mode:
switch(config)# port-monitor name policyname
(Optional) Remove the policy name:
switch(config)# no port-monitor name policyname
Step 3
Apply policy type:
switch(config-port-monitor)# port-type {access-port | trunks | all}
Step 4
Specify the counter parameters:
switch(config-port-monitor)# counter {credit-loss-reco | err-pkt-from-port |
err-pkt-from-xbar | err-pkt-to-xbar | invalid-crc | invalid-words | link-loss | lr-rx |
lr-tx | rx-datarate | signal-loss | state-change | sync-loss | timeout-discards |
tx-credit-not-available | tx-datarate | tx-discards | tx-slowport-count |
tx-slowport-oper-delay | txwait} poll-interval seconds {absolute | delta} rising-threshold
count1 event RMON-ID warning-threshold count2 falling threshold count3 event RMON-ID
portguard {errordisable | flap}
Note
•
You must activate the err-pkt-from-port, err-pkt-from-xbar, and err-pkt-to-xbar counters using
the monitor counter name command, before specifying the counter parameters.
•
Counters err-pkt-from-xbar, err-pkt-from-port, and err-pkt-to-xbar support delta threshold
type only.
•
Counter tx-slowport-oper-delay supports absolute threshold type only.
•
Counter tx-slowport-oper-delay does not support port guard action.
•
Counter tx-slowport-count is supported only on DS-X9224-96K9, DS-X9248-96K9, and
DS-X9248-48K9 modules.
(Optional) Revert to the default values for a counter:
switch(config-port-monitor)# no counter {credit-loss-reco | err-pkt-from-port |
err-pkt-from-xbar | err-pkt-to-xbar | invalid-crc | invalid-words | link-loss | lr-rx |
lr-tx | rx-datarate | signal-loss | state-change | sync-loss | timeout-discards |
tx-credit-not-available | tx-datarate | tx-discards | tx-slowport-count |
tx-slowport-oper-delay | txwait} poll-interval seconds {absolute | delta} rising-threshold
count1 event RMON-ID warning-threshold count2 falling threshold count3 event RMON-ID
portguard {errordisable | flap}
(Optional) Monitor a counter:
switch(config-port-monitor)# monitor counter {credit-loss-reco | err-pkt-from-port |
err-pkt-from-xbar | err-pkt-to-xbar | invalid-crc | invalid-words | link-loss | lr-rx |
lr-tx | rx-datarate | signal-loss | state-change | sync-loss | timeout-discards |
tx-credit-not-available | tx-datarate | tx-discards | tx-slowport-count |
tx-slowport-oper-delay | txwait}
A port monitor currently recognizes two kinds of ports:
•
Port type access ports are normally F ports with a single end device logged in. However, a port
monitor considers TF ports and F ports with multiple logins to be port type access as well.
•
Port type trunk ports are ports that are E ports (ISLs) regardless of whether they are actually carrying
multiple VSANs (TE, trunking) or not. Some of the access port counter thresholds and port guard
actions might not be appropriate on the TF ports in port monitor configurations. Specifically, port
guard disable or flap actions can affect multiple end devices on the F ports with multiple logins.
Therefore, performing disable of flap actions should be avoided on an N Port Identifier
Virtualization (NPIV) system.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-38
Chapter 3
Configuring Interfaces
Configuring Interfaces
Configuring a Port Monitor Port Actions
To configure a port monitor port guard action, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Specify the policy name and enter port monitoring policy configuration mode:
switch(config)# port-monitor name policyname
Step 3
Specify the delta link loss poll interval (in seconds), threshold limits, and event IDs of the events to be
triggered:
switch(config-port-monitor)# counter link-loss poll-interval seconds delta
rising-threshold count1 event event-id warning-threshold count2 falling-threshold count3
event event-id portguard flap
This command also specifies if the port is flapped (port goes down and up) when the event occurs. It also
specifies if the port guard action is set to flap for the port when the rising threshold is reached.
Step 4
Specify the delta link loss poll interval (in seconds), threshold limits, and event IDs of the events to be
triggered:
switch(config-port-monitor)# counter link-loss poll-interval seconds delta
rising-threshold count1 event event-id warning-threshold count2 falling-threshold count3
event event-id portguard errordisable
This command also specifies if the interface is down (error disabled) when the event occurs. It also
specifies if the port guard action set to error disable for the port when the rising threshold is reached.
Note
Port guard action is not supported for absolute type counters.
Activating a Port Monitor Policy
To activate a port monitor policy, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Activate the specified port monitor policy:
switch(config)# port-monitor activate policyname
(Optional) Activate the default port monitor policy:
switch(config)# port-monitor activate
(Optional) Deactivate the specified port monitoring policy:
switch(config)# no port-monitor activate policyname
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-39
Chapter 3
Configuring Interfaces
Configuring Interfaces
Warning Threshold Example
Let us consider two scenarios with the following configurations:
•
Rising threshold is 30
•
Warning threshold is 10
•
Falling threshold is 0
This example displays the syslog generated when the error count is less than the rising threshold value,
but has reached the warning threshold value:
Example 3-4
Syslog Generated When the Error Count is Less Than the Rising Threshold Value
%PMON-SLOT2-4-WARNING_THRESHOLD_REACHED_UPWARD: Invalid Words has reached
warning threshold in the upward direction (port fc2/18 [0x1091000], value = 10).
%PMON-SLOT2-5-WARNING_THRESHOLD_REACHED_DOWNWARD: Invalid Words has
reached warning threshold in the downward direction (port fc2/18 [0x1091000], value = 5).
In the first polling interval, the errors triggered for the counter (Invalid Words) are 10, and have reached
the warning threshold value. A syslog is generated, indicating that the error count is increasing (moving
in the upward direction).
In the next polling interval, the error count decreases (moves in the downward direction), and a syslog
is generated, indicating that the error count has decreased (moving in the downward direction).
This example displays the syslog that is generated when the error count crosses the rising threshold
value:
Example 3-5
Syslog Generated When the Error Count Crosses the Rising Threshold Value
%PMON-SLOT2-4-WARNING_THRESHOLD_REACHED_UPWARD: Invalid Words has reached
warning threshold in the upward direction (port fc2/18 [0x1091000], value = 30).
%PMON-SLOT2-3-RISING_THRESHOLD_REACHED: Invalid Words has reached the rising
threshold (port=fc2/18 [0x1091000], value=30).
%SNMPD-3-ERROR: PMON: Rising Alarm Req for Invalid Words counter for port fc2/18(1091000),
value is 30 [event id 1 threshold 30 sample 2 object 4 fcIfInvalidTxWords]
%PMON-SLOT2-5-WARNING_THRESHOLD_REACHED_DOWNWARD: Invalid Words has
reached warning threshold in the downward direction (port fc2/18 [0x1091000], value = 3).
%PMON-SLOT2-5-FALLING_THRESHOLD_REACHED: Invalid Words has reached the falling
threshold (port=fc2/18 [0x1091000], value=0).
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-40
Chapter 3
Configuring Interfaces
Configuring Interfaces
%SNMPD-3-ERROR: PMON: Falling Alarm Req for Invalid Words counter for port fc2/18(1091000),
value is 0 [event id 2 threshold 0 sample 2 object 4 fcIfInvalidTxWords]
This example displays the syslog generated when the error count is more than the warning threshold
value and less than the rising threshold value:
Example 3-6
Syslog Generated When the Error Count is More than the Warning Threshold Value and Less than
the Rising Threshold Value
%PMON-SLOT2-4-WARNING_THRESHOLD_REACHED_UPWARD: Invalid Words has reached
warning threshold in the upward direction (port fc2/18 [0x1091000], value = 15).
%PMON-SLOT2-5-WARNING_THRESHOLD_REACHED_DOWNWARD: Invalid Words has
reached warning threshold in the downward direction (port fc2/18 [0x1091000], value = 3).
The errors generated for the counter (Invalid Words) are 30 when the counter has crossed both the
warning and rising threshold values. A syslog is generated when no further errors are triggered.
As there are no further errors in this poll interval, the consecutive polling interval will have no errors,
and the error count decreases (moves in downward direction) and reaches the falling threshold value,
which is zero. A syslog is generated for the falling threshold.
Configuring Port Group Monitor
This section includes the following topics:
•
Enabling Port Group Monitor, page 3-41
•
Configuring Port Group Monitor Policy, page 3-42
•
Reverting to the Default Value for a Specific Counter, page 3-42
•
Turning Off Specific Counter Monitoring, page 3-43
•
Activating Port Group Monitor Policy, page 3-43
Enabling Port Group Monitor
To enable port group monitor, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Enable port group monitoring:
switch(config)# port-group-monitor enable
(Optional) Disable port group monitoring:
switch(config)# no port-group-monitor enable
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-41
Chapter 3
Configuring Interfaces
Configuring Interfaces
Configuring Port Group Monitor Policy
To configure port group monitor policy, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Specify the policy name and enter port group monitoring policy configuration mode:
switch(config)# port-group-monitor name policyname
(Optional) Remove the policy:
switch(config)# no port-group-monitor name policyname
Step 3
Specify the delta receive or transmit counter poll interval (in seconds) and thresholds (in percentage):
switch(config-port-group-monitor)# counter {rx-performance | tx-performance} poll-interval
seconds delta rising-threshold percentage1 falling-threshold percentage2
(Optional) Revert to the default policy:
switch(config-port-group-monitor)# no counter tx-performance
For more information on reverting to the default policy, see Chapter 3, “Reverting to the Default Value
for a Specific Counter” and Chapter 3, “Port Group Monitor”.
Step 4
Turn on performance monitoring:
switch(config-port-group-monitor)# monitor counter {rx-performance | tx-performance}
(Optional) Turn off performance monitoring:
switch(config-port-group-monitor)# no monitor counter {rx-performance | tx-performance}
For more information on turning off transmit performance monitoring, see Chapter 3, “Turning Off
Specific Counter Monitoring”.
Note
On 8 Gbps and higher speed modules, port errors are monitored using the invalid-crc and
invalid-words counters. The err-pkt-from-port counter is supported only on 4-Gbps modules.
Reverting to the Default Value for a Specific Counter
The following examples display the default values for counters:
switch(config)# port-group-monitor name PGMON_policy
switch(config-port-group-monitor)# counter rx-datarate poll-interval 200 delta
rising-threshold 75 falling-threshold 0
switch(config)# show port-group-monitor PGMON_policy
Policy Name : PGMON_policy
Admin status : Not Active
Oper status : Not Active
Port type
: All Port Groups
-----------------------------------------------------------------------------------------Counter
Threshold Interval %ge Rising Threshold %ge Falling Threshold
--------------- -------- -------------------- ---------------------RX Datarate
Delta
200
75
0
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-42
Chapter 3
Configuring Interfaces
Configuring Interfaces
TX Datarate
Delta
60
80
20
-----------------------------------------------------------------------------------------switch(config-port-group-monitor)# no counter rx-datarate poll-interval 200 delta
rising-threshold 75 falling-threshold 0
switch(config)# show port-group-monitor PGMON_policy
Policy Name : PGMON_policy
Admin status : Not Active
Oper status : Not Active
Port type
: All Port Groups
-----------------------------------------------------------------------------------------Counter
Threshold Interval %ge Rising Threshold %ge Falling Threshold
--------------- -------- -------------------- ---------------------RX Datarate
Delta
60
80
20
TX Datarate
Delta
60
80
20
------------------------------------------------------------------------------------------
Turning Off Specific Counter Monitoring
The following examples display turning off counter monitoring:
switch(config)# port-group-monitor name PGMON_policy
switch(config-port-group-monitor)# no monitor counter rx-performance
switch(config)# show port-group-monitor PGMON_policy
Policy Name : PGMON_policy
Admin status : Not Active
Oper status : Not Active
Port type
: All Port Groups
-----------------------------------------------------------------------------------------Counter
Threshold Interval %ge Rising Threshold %ge Falling Threshold
--------------- -------- -------------------- ---------------------TX Performance
Delta
60
100
80
------------------------------------------------------------------------------------------
Activating Port Group Monitor Policy
To activate port group monitor policy, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Activate the specified port group monitor policy:
switch(config)# port-group-monitor activate policyname
(Optional) Activate the default port group monitor policy:
switch(config)# port-group-monitor activate
(Optional) Deactivate the specified port group monitor policy:
switch(config)# no port-group-monitor activate policyname
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-43
Chapter 3
Configuring Interfaces
Configuring Interfaces
Configuring the Management Interface
To configure the mgmt0 Ethernet interface to connect over IPv4, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select the management Ethernet interface on the switch and enter interface configuration submode:
switch(config)# interface mgmt0
Step 3
Configure the IPv4 address and IPv4 subnet mask:
switch(config-if)# ip address 10.16.1.2 255.255.255.0
Step 4
Enable the interface:
switch(config-if)# no shutdown
Step 5
Return to configuration mode:
switch(config-if)# exit
Step 6
Configure the default gateway IPv4 address:
switch(config)# ip default-gateway 1.1.1.4
Step 7
Return to user EXEC mode:
switch(config)# exit
(Optional) Save your configuration changes to the file system:
switch# copy running-config startup-config
To configure the mgmt0 Ethernet interface to connect over IPv6, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Select the management Ethernet interface on the switch and enter interface configuration submode:
switch(config)# interface mgmt0
Step 3
Enable IPv6 and assign a link-local address on the interface:
switch(config-if)# ipv6 enable
Step 4
Specify an IPv6 unicast address and prefix length on the interface:
switch(config-if)# ipv6 address 2001:0db8:800:200c::417a/64
Step 5
Enable the interface:
switch(config-if)# no shutdown
Step 6
Return to user EXEC mode:
switch(config-if)# end
(Optional) Save your configuration changes to the file system:
switch# copy running-config startup-config
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-44
Chapter 3
Configuring Interfaces
Configuring Interfaces
Creating a VSAN Interface
To create a VSAN interface, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Configure a VSAN with the ID 2:
switch(config)# interface vsan 2
Step 3
Enable the VSAN interface:
switch(config-if)# no shutdown
Configuring Slow-Drain Device Detection and Congestion Avoidance
Slow-drain devices are devices that do not accept frames at the configured rate. The presence of these
slow-drain devices leads to traffic congestion in the Fibre Channel or Fibre Channel over Ethernet
(FCoE) fabric. This traffic congestion can affect the unrelated flows in the fabric that use the same ISLs
for its traffic as the slow-drain device. This is true although the destination devices are not slow-drain
devices.
From Cisco MDS NX-OS Release 4.2(1), slow-drain device detection and congestion avoidance is
supported on all Fibre Channel switching modules.
From Cisco MDS NX-OS Release 5.2(1), slow-drain device detection and congestion avoidance is
supported on all FCoE switch modules.
From Cisco MDS NX-OS Release 5.2(1), slow-drain detection and congestion avoidance functionality
for edge ports was enhanced.
Multiple features are available on the Cisco MDS 9000 Series Multilayer Switches to detect slow drain
and avoid the resulting effects.
Table 3-15 describes the features that help detect slow drain:
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-45
Chapter 3
Configuring Interfaces
Configuring Interfaces
Table 3-15
Features to Detect Slow Drain
Feature Name
Description
TX Credit Not Available Counter
Fibre Channel port monitor—Transmit credit not
available is the continuous no-transmit credit
condition. When the configured period expires,
trap, flap, or error-disable the port.
Congestion Drop
Fibre Channel system—Congestion drop timeout
is the maximum lifetime of a frame in a switch.
When the configured period expires, the frames
are dropped.
FCoE system—Congestion drop timeout is the
maximum lifetime of a frame in the switch. When
the configured period expires, the frames are
dropped.
Slow-Port Monitor
Fibre Channel system—Slow-port monitor credits
return to a switch slowly; logs only events.
No-Credit Drop
Fibre Channel system—No-credit drop is the
continuous no-transmit credit condition. All the
queued and incoming frames for a port are
dropped immediately.
Pause Timeout
FCoE system—Pause timeout is a continuous
pause condition. When the configured period
expires, all the queued and incoming frames for a
port are dropped.
Credit Loss Recovery
Fibre Channel system—Credit loss recovery is its
continuous no-transmit credit condition; credit
loss recovery resets the port.
This section includes the following topics:
•
Configuring the Congestion Frame Timeout Value for FCoE, page 3-46
•
Configuring Pause Frame Timeout Value for FCoE, page 3-47
•
Configuring the Congestion Drop Timeout Value for Fibre Channel, page 3-48
•
Configuring the No-Credit Frame Timeout Value for Fibre Channel, page 3-48
•
Configuring the Slow-Port Monitor Timeout Value for Fibre Channel, page 3-49
•
Displaying Credit Loss Recovery Actions, page 3-50
•
Configuring the Transmit Average Credit-Not-Available Duration Threshold and Action, page 3-51
Configuring the Congestion Frame Timeout Value for FCoE
When an FCoE frame takes longer than the congestion timeout period to be transmitted by the egress
port, the frame is dropped. This dropping of the frames is useful in controlling the effect of slow egress
ports that are paused almost continuously (long enough to cause congestion), but not long enough to
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-46
Chapter 3
Configuring Interfaces
Configuring Interfaces
trigger the pause timeout drop. Dropping of frames is counted as egress discard on the egress port. Egress
discard releases buffers in the upstream ingress ports of the switch, allowing the unrelated flows to move
continuously through them.
The congestion timeout value is 500 ms by default for all port types. We recommend that you retain the
default timeout for core ports and consider configuring a lower value for edge ports. This value should
be equal to or greater than the pause frame timeout value for that port type.
To configure the congestion frame timeout value for FCoE, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Configure the system-wide FCoE congestion timeout, in milliseconds, for either core or edge ports:
switch(config)# system default interface congestion timeout milliseconds mode {core |
edge}
The range is 100-1000 ms
Configuring Pause Frame Timeout Value for FCoE
When an FCoE port is in a state of continuous pause for the pause frame timeout period, all the frames
that are queued to that port are dropped immediately. As long as the port continues to remain in the pause
state, the newly arriving frames destined for the port are dropped immediately. These drops are counted
as egress discards on the egress port, and create buffers in the upstream ingress ports of the switch,
allowing unrelated flows to continue moving through them.
To reduce the effect of a slow-drain device on unrelated traffic flows, configure a lower-pause frame
timeout value than the congestion frame timeout value, for edge ports. This causes the frames destined
for a slow port to be dropped immediately after the pause timeout period has occurred, rather than
waiting for the congestion timeout period to drop them.
Pause timeout dropping can be enabled and disabled. By default, frame dropping is enabled. The pause
timeout value is 500 ms by default for all ports. We recommend that you retain the default timeout core
ports and consider configuring a lower value for edge ports.
To configure the pause frame timeout value for FCoE, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Configure the system-wide FCoE pause timeout, in milliseconds, for either edge or core ports:
switch(config)# system default interface pause timeout milliseconds mode {core | edge}
The range is 100–500 ms.
(Optional) Revert to the default pause timeout, in milliseconds:
switch(config)# no system default interface pause timeout milliseconds mode {core | edge}
Step 3
Enable the pause timeout drops for edge or core ports:
switch(config)# system default interface pause mode {core | edge}
(Optional) Disable the pause timeout drops for edge or core ports:
switch(config)# no system default interface pause mode {core | edge}
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-47
Chapter 3
Configuring Interfaces
Configuring Interfaces
Configuring the Congestion Drop Timeout Value for Fibre Channel
When a Fibre Channel frame takes longer than the congestion timeout period to be transmitted by the
egress port, the frame is dropped. This option of the frames being dropped is useful for controlling the
effect of slow egress ports that lack transmit credits almost continuously; long enough to cause
congestion, but not long enough to trigger the no-credit timeout drop. These drops are counted as egress
discards on the egress port, and release buffers into the upstream ingress ports of the switch, allowing
unrelated flows to continue moving through them.
By default, the congestion timeout value is 500 ms for all port types. We recommend that you retain the
default timeout for core ports and configure a lower value (not less than 200 ms) for edge ports. The
congestion timeout value should be equal to or greater than the no-credit frame timeout value for that
port type.
To configure the congestion frame timeout value for the Fibre Channel, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Configure the Fibre Channel congestion drop timeout value, in milliseconds, for the specified port type:
switch(config)# system timeout congestion-drop milliseconds mode E | F
The range is 100-500 ms.
(Optional) Revert to the default value for the congestion timeout for the specified port type:
switch(config)# system timeout congestion-drop default mode E | F
Configuring the No-Credit Frame Timeout Value for Fibre Channel
When a Fibre Channel egress port has no transmit credits continuously for the no-credit timeout period,
all the frames that are already queued to that port are dropped immediately. As long as the port remains
in this condition, newly arriving frames destined for that port are dropped immediately. These drops are
counted as egress discards on the egress port, and release buffers in the upstream ingress ports of the
switch, allowing unrelated flows to continue moving through them.
No-credit dropping can be enabled or disabled. By default, frame dropping is disabled and the frame
timeout value is 500 ms for all port types. We recommend that you retain the default frame timeout for
core ports and configure a lower value (300 ms) for edge ports. If the slow-drain events continue to affect
unrelated traffic flows, the frame timeout value for the edge ports can be lowered to drop the previous
slow-drain frames. This frees the ingress buffers for frames of unrelated flows, thus reducing the latency
of the frames through the switch.
Note
•
The no-credit frame timeout value should always be less than the congestion frame timeout for the
same port type, and the edge port frame timeout values should always be lower than the core port
frame timeout values.
•
The slow-port monitor delay value should always be less than the no-credit frame timeout value for
the same port type.
For pre-16-Gbps-capable modules and systems, the no-credit timeout value can be 100 to 500 ms in
multiples of 100 ms. On these systems, the no-credit condition is checked only at 100-ms intervals. At
this point, if the no-credit condition exists, dropping starts. Depending on the timing of the actual onset
of the no-credit condition, the task of checking port dropping can be delayed by up to 100 ms later than
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-48
Chapter 3
Configuring Interfaces
Configuring Interfaces
the configured value. On 16 Gbps and later modules and systems, the no-credit timeout value can be 1
to 500 ms in multiples of 1 ms. Dropping starts immediately after the no-credit condition comes into
existence for the configured timeout value.
To configure the no-credit timeout value, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Specify the no-credit timeout value for the switch’s F ports:
switch(config)# system timeout no-credit-drop milliseconds mode F
(Optional) Revert to the default no-credit timeout value (500 ms) for edge ports:
switch(config)# system timeout no-credit-drop default mode F
The no-credit drop action is not changed.
(Optional) Disable no-credit dropping for edge ports:
switch(config)# no system timeout no-credit-drop mode F
Configuring the Slow-Port Monitor Timeout Value for Fibre Channel
The slow-port monitor functionality is similar to the no-credit frame timeout and drop functionality,
except that it does not drop frames; it only logs qualifying events. When a Fibre Channel egress port has
no transmit credits continuously for the slow-port monitor timeout period, the event is logged. No frames
are dropped unless the no-credit frame timeout period is reached and no-credit frame timeout drop is
enabled. If the no-credit frame timeout drop is not enabled, no frames are dropped until the congestion
frame timeout period is reached.
Slow-port monitoring is implemented in the hardware, with the slow-port monitor functionality being
slightly different in each generation of hardware. The 8-Gbps modules report a single slow-port monitor
event for each 100-ms window in which the slow-port monitor threshold has crossed one or more times.
They do not have the ability to report the exact number of slow-port events. The advanced 8 and 16 Gbps
modules and switches are not restricted and can detect each instance of the slow-port monitor threshold
being crossed. The slow-port monitor log is updated at 100-ms intervals. A log entry for a slow port on
an 8-Gbps module can increment by a maximum of one. A log for a slow-port event on an advanced 8
or 16 Gbps module or system increments the exact number of times the threshold is reached.
Modules and switches that currently support slow-port monitor are:
•
8-Gbps modules:
– Cisco MDS 9500 1, 2, 4, or 8-Gbps Fibre Channel Module DS-X9248-48K9
– Cisco MDS 9500 1, 2, 4, or 8-Gbps Fibre Channel Module DS-X9224-96K9
– Cisco MDS 9500 1, 2, 4, or 8-Gbps Fibre Channel Module DS-X9248-96K9
•
Advanced 8-Gbps modules:
– Cisco MDS 9500 1, 2, 4, 8, or 10-Gbps Advanced Fibre Channel Module DS-X9232-256K9
– Cisco MDS 9500 1, 2, 4, 8, or 10-Gbps Advanced Fibre Channel Module DS-X9248-256K9
•
16-Gbps modules or switches:
– Cisco MDS 9700 2, 4, 8, 10, or 16-Gbps Advanced Fibre Channel Module DS-X9448-768K9
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-49
Chapter 3
Configuring Interfaces
Configuring Interfaces
– Cisco MDS 9250i Fabric Switch
– Cisco MDS 9148S Fabric Switch
– Cisco MDS 9396S Fabric Switch
Table 3-16 displays the slow port features supported on different Fibre Channel switching modules for
Cisco MDS NX-OS Release 6.2(13):
Table 3-16
Slow-Port Support on Fibre Channel Switching Modules
Hardware Support
Function
8-Gbps
Modules
Advanced
8-Gbps
Modules
16-Gbps
Modules and
Switches
Slow-port monitor1
Yes
Yes
Yes
Transmit-wait history graph
No
Yes
Yes
Transmit-wait OBFL logging
Yes
Yes
Yes
Port monitor slow-port counter
No
Yes
Yes
Port monitor transmit-wait counter
Yes
Yes
Yes
Transmit-wait interface counter
No
Yes
Yes
1. From Cisco MDS NX-OS Release 6.2(9), slow-port monitoring is supported on 16-Gbps modules and switches. From Cisco
MDS NX-OS Release 6.2(13), slow-port monitoring is supported on 8-Gbps modules.
To configure the slow-port monitor timeout value, perform these steps:
Step 1
Enter configuration mode:
switch# configure terminal
Step 2
Specify the slow-port monitor timeout value for E or F port mode for the switch:
switch(config)# system timeout slowport-monitor milliseconds mode E | F
Valid values for the slow-port monitor timeout are:
•
16-Gbps modules or switches—1 to 500 ms in 1-ms increments.
•
8-Gbps and advanced 8-Gbps modules—1 to 100 ms in 1-ms increments.
(Optional) Revert to the default slow-port monitor timeout value (500 ms) for the specified port type:
switch(config)# system timeout slowport-monitor default mode E | F
Displaying Credit Loss Recovery Actions
When a port is at zero transmit credits for 1 full second (F ports) and 1.5 seconds (E ports), it is called
a credit loss condition. Cisco MDS initiates credit loss recovery by transmitting a Link Credit Reset
(LCR). If the end device responds with a Link Credit Reset Response (LCRR), the port is back at its fully
agreed number of B2B credits in both directions. If an LRR is not received, the port is shut down.
When the port detects the credit loss condition and recovers, some of the following actions might occur:
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-50
Chapter 3
Configuring Interfaces
Configuring Interfaces
•
An SNMP trap with interface details can be sent, indicating the credit loss event.
•
The port can be error disabled.
•
The port can be flapped.
When the configured threshold is exceeded, one or more of these actions can be combined together.
These actions can be turned on or off depending on the situation. The Port Monitor feature provides the
CLI to configure the thresholds and action.
The 1 second (F ports) and 1.5 seconds (E ports) timers that are set for the switch to initiate CLR are
fixed and cannot be changed.
To verify a port monitor policy to generate SNMP alerts and take other actions in the quantity and timing
of these events, perform these steps:
•
Display the last 10 credit loss events per interface per module:
switch# show process creditmon credit-loss-events [module x]
•
Display a chronological log of credit loss events per module:
switch# show process creditmon credit-loss-event-history [module x]
Note
When a port sees the credit loss condition and fails to recover, the port flaps. This function is already a
part of the port guard, and you can configure the supported actions using the Port Guard feature.
Configuring the Transmit Average Credit-Not-Available Duration Threshold and Action
Cisco MDS monitors its ports that are at zero transmit credits for 100 ms or more. This is called transmit
average credit-not-available duration. The Port Monitor feature can monitor this using the TX Credit Not
Available counter. When the transmit average credit-not-available duration exceeds the threshold set in
the port monitor policy, some or all the following actions might occur:
•
An SNMP trap with interface details can be sent, indicating the transmit average credit not available
duration event.
•
The port can be error disabled.
•
The port can be flapped.
When the configured threshold is exceeded, one or more of these actions can be combined together.
These actions can be turned on or off depending on the situation. The Port Monitor feature provides the
CLI to configure the thresholds and action. The threshold configuration is configured as a percentage of
the interval. The thresholds can be 0 to 100 percent in multiples of 10, and the interval can be 1 second
to 1 hour. The default is 10 percent of a 1-second interval and generates a trap when the
transmit-average-credit-not-available duration hits 100 ms.
For information about configuring the average-credit-not-available-duration threshold and action, refer
to the Chapter 3, “Port Monitor”.
The following example shows how to configure credit loss recovery and the average credit-not-available
duration threshold and action:
switch# show port-monitor PMON_policy
Policy Name : PMON_policy
Admin status : Not Active
Oper status : Not Active
Port type
: All Ports
-----------------------------------------------------------------------------------------Counter
Threshold Interval Rising Threshold
event Falling Threshold
event Warning Threshold
PMON Portguard
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-51
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
--------------- -------- ----------------------------------------- ------------------------------Link Loss
Delta
60
5
4
1
4
Not enabled
Not enabled
Sync Loss
Delta
60
5
4
1
4
Not enabled
Not enabled
Signal Loss
Delta
60
5
4
1
4
Not enabled
Not enabled
Invalid Words
Delta
60
1
4
0
4
Not enabled
Not enabled
Invalid CRC's
Delta
60
5
4
1
4
Not enabled
Not enabled
State Change
Delta
60
5
4
0
4
Not enabled
Not enabled
TX Discards
Delta
60
200
4
10
4
Not enabled
Not enabled
LR RX
Delta
60
5
4
1
4
Not enabled
Not enabled
LR TX
Delta
60
5
4
1
4
Not enabled
Not enabled
Timeout Discards
Delta
60
200
4
10
4
Not enabled
Not enabled
Credit Loss Reco
Delta
1
1
4
0
4
Not enabled
Not enabled
TX Credit Not Available Delta
1
10%
4
0%
4
Not enabled
Not enabled
RX Datarate
Delta
60
80%
4
20%
4
Not enabled
Not enabled
TX Datarate
Delta
60
80%
4
20%
4
Not enabled
Not enabled
TX-Slowport-Oper-Delay
Absolute
1
50ms
4
0ms
4
Not enabled
Not enabled
TXWait
Delta
1
40%
4
0%
4
Not enabled
Not enabled
------------------------------------------------------------------------------------------
The following edge port monitor policy is active by default. No port monitor policy is enabled for core
ports by default.
switch# show port-monitor slowdrain
Policy Name : slowdrain
Admin status : Not Active
Oper status : Not Active
Port type
: All Access Ports
-----------------------------------------------------------------------------------------Counter
Threshold Interval Rising Threshold event Falling Threshold event
PMON Portguard
-----------------------------------------------------------------------------------------Credit Loss Reco
Delta
1
1
4
0
4
Not enabled
TX Credit Not
Available
Delta
1
10%
4
0%
4
Not enabled
------------------------------------------------------------------------------------------
Verifying Interfaces Configuration
This section includes the following topics:
•
Displaying Interface Information, page 3-53
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-52
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
•
Displaying the Port Monitor Status and Policies, page 3-62
•
Displaying Port Group Monitor Status and Policies, page 3-66
•
Displaying the Management Interface Configuration, page 3-67
•
Displaying VSAN Interface Information, page 3-67
•
Displaying the Congestion Frame Timeout Value for FCoE, page 3-67
•
Displaying the Pause Frame Timeout Value for FCoE, page 3-67
•
Displaying the Congestion Drop Timeout Value for Fibre Channel, page 3-68
•
Displaying the No-Credit Frame Timeout Value for Fibre Channel, page 3-68
•
Displaying Slow-Port Monitor Events, page 3-68
Displaying Interface Information
Run the show interface command from user EXEC mode. This command displays the interface
configurations. Without any arguments, this command displays the information for all the configured
interfaces in the switch.
The following example displays the status of interfaces:
switch# show interface
fc1/1 is up
Hardware is Fibre Channel, SFP is short wave laser
Port WWN is 20:0b:00:05:30:00:8d:de
Admin port mode is F
Port mode is F, FCID is 0x610000
Port vsan is 2
Speed is 2 Gbps
Transmit B2B Credit is 3
Receive B2B Credit is 16
Receive data field Size is 2112
Beacon is turned off
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
134 frames input, 8468 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
154 frames output, 46072 bytes
0 discards, 0 errors
1 input OLS, 1 LRR, 0 NOS, 0 loop inits
1 output OLS, 0 LRR, 1 NOS, 0 loop inits
16 receive B2B credit remaining
3 transmit B2B credit remaining.
.
.
.
fc1/9 is trunking
Hardware is Fibre Channel, SFP is long wave laser cost reduced
Port WWN is 20:09:00:05:30:00:97:9e
Peer port WWN is 20:0b:00:0b:5f:a3:cc:00
Admin port mode is E, trunk mode is on
Port mode is TE
Port vsan is 100
Speed is 2 Gbps
Transmit B2B Credit is 255
Receive B2B Credit is 255
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-53
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
Receive data field Size is 2112
Beacon is turned off
Trunk vsans (admin allowed and active) (1,100,3000)
Trunk vsans (up)
(1,100,3000)
Trunk vsans (isolated)
()
Trunk vsans (initializing)
()
5 minutes input rate 280 bits/sec, 35 bytes/sec, 0 frames/sec
5 minutes output rate 176 bits/sec, 22 bytes/sec, 0 frames/sec
4609939 frames input, 8149405708 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
4638491 frames output, 7264731728 bytes
0 discards, 0 errors
3 input OLS, 9 LRR, 1 NOS, 0 loop inits
9 output OLS, 7 LRR, 1 NOS, 0 loop inits
16 receive B2B credit remaining
3 transmit B2B credit remaining.
.
.
.
fc1/13 is up
Hardware is Fibre Channel, SFP is short wave laser
Port WWN is 20:0d:00:05:30:00:97:9e
Admin port mode is auto, trunk mode is on
Port mode is F, FCID is 0x650100
Port vsan is 100
Speed is 2 Gbps
Transmit B2B Credit is 3
Receive B2B Credit is 16
Receive data field Size is 2112
Beacon is turned off
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
8696 frames input, 3227212 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
16799 frames output, 6782444 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
1 output OLS, 1 LRR, 0 NOS, 1 loop inits
16 receive B2B credit remaining
3 transmit B2B credit remaining.
.
.
.
sup-fc0 is up
Hardware is Fibre Channel
Speed is 1 Gbps
139597 packets input, 13852970 bytes
0 multicast frames, 0 compressed
0 input errors, 0 frame, 0 overrun 0 fifo
139516 packets output, 16759004 bytes, 0 underruns
0 output errors, 0 collisions, 0 fifo
0 carrier errors
You can also specify arguments (a range of interfaces or multiple specified interfaces) to display
interface information. You can specify a range of interfaces by issuing a command in the following
format:
interface fc1/1 - 5 , fc2/5 - 7
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-54
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
Note
The spaces are required before and after the dash ( - ) and before and after the comma (, ).
The following example displays the status of a range of interfaces:
switch# show interface fc3/13 , fc3/16
fc3/13 is up
Hardware is Fibre Channel, SFP is short wave laser
Port WWN is 20:8d:00:05:30:00:97:9e
Admin port mode is FX
Port mode is F, FCID is 0x7b0300
Port vsan is 1
Speed is 2 Gbps
Transmit B2B Credit is 3
Receive B2B Credit is 12
Receive data field Size is 2112
Beacon is turned off
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1856 frames input, 116632 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
1886 frames output, 887712 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 1 loop inits
1 output OLS, 1 LRR, 0 NOS, 1 loop inits
16 receive B2B credit remaining
3 transmit B2B credit remaining.
fc3/16 is up
Hardware is Fibre Channel, SFP is short wave laser
Port WWN is 20:90:00:05:30:00:97:9e
Admin port mode is FX
Port mode is F, FCID is 0x7d0100
Port vsan is 3000
Speed is 2 Gbps
Transmit B2B Credit is 3
Receive B2B Credit is 12
Receive data field Size is 2112
Beacon is turned off
5 minutes input rate 504 bits/sec, 63 bytes/sec, 0 frames/sec
5 minutes output rate 520 bits/sec, 65 bytes/sec, 0 frames/sec
47050 frames input, 10311824 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
62659 frames output, 10676988 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
1 output OLS, 1 LRR, 0 NOS, 1 loop inits
16 receive B2B credit remaining
3 transmit B2B credit remaining.
The following example displays the status of a specified interface:
switch# show interface fc2/2
fc2/2 is trunking
Port description is Trunk to Core-4
Hardware is Fibre Channel, SFP is short wave laser
Port WWN is 20:42:00:05:30:00:97:9e
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-55
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
Peer port WWN is 20:cc:00:05:30:00:50:9e
Admin port mode is E, trunk mode is on
Port mode is TE
Port vsan is 1
Speed is 2 Gbps
Transmit B2B Credit is 255
Receive B2B Credit is 255
Receive data field Size is 2112
Beacon is turned off
Belongs to port-channel 2
Trunk vsans (admin allowed and active) (1,100,3000)
Trunk vsans (up)
(1)
Trunk vsans (isolated)
(100,3000)
Trunk vsans (initializing)
()
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 32 bits/sec, 4 bytes/sec, 0 frames/sec
2214834 frames input, 98673588 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
2262415 frames output, 343158368 bytes
0 discards, 0 errors
1 input OLS, 1 LRR, 1 NOS, 0 loop inits
2 output OLS, 1 LRR, 0 NOS, 0 loop inits
16 receive B2B credit remaining
3 transmit B2B credit remaining.
The following example displays the description of interfaces:
switch# show interface description
------------------------------------------------------------------------------Interface
Description
------------------------------------------------------------------------------fc3/1
test intest
fc3/2
-fc3/3
-fc3/4
TE port
fc3/5
-fc3/6
-fc3/10
Next hop switch 5
fc3/11
-fc3/12
-fc3/16
-------------------------------------------------------------------------------Interface
Description
------------------------------------------------------------------------------port-channel 1
-port-channel 5
-port-channel 6
--
The following example displays a summary of information:
switch# show interface brief
------------------------------------------------------------------------------Interface Vsan
Admin Admin
Status
SFP
Oper Oper
Port
Mode
Trunk
Mode Speed Channel
Mode
(Gbps)
------------------------------------------------------------------------------fc1/1
1
E
on
trunking
swl
TE
2
1
fc1/2
1
E
on
trunking
swl
TE
2
1
fc1/3
1
auto
on
SFPAbsent
----
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-56
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
fc1/4
1
auto
on
SFPAbsent
---fc1/5
3000
auto
on
up
swl
F
2
-...
fc2/2
1
E
on
trunking
swl
TE
2
2
fc2/3
1
auto
on
down
c1610 --fc2/4
1
auto
on
down
c1590 --fc2/5
3000
auto
on
notConnected
lwcr
--fc2/6
1
auto
on
SFPAbsent
---...
fc3/16
3000
FX
-up
swl
F
2
-fc3/17
1
FX
-SFPAbsent
---...
------------------------------------------------------------------------------Interface
Status
IP Address
Speed
MTU
------------------------------------------------------------------------------GigabitEthernet4/1
SFPAbsent -auto
1500
...
GigabitEthernet4/6
down
10.1.1.2/8
auto
3000
GigabitEthernet4/7
down
10.1.1.27/24
auto
1500
GigabitEthernet4/8
down
-auto
1500
------------------------------------------------------------------------------Interface
Status
Oper Mode
Oper Speed
(Gbps)
------------------------------------------------------------------------------iscsi4/1
down
-...
------------------------------------------------------------------------------Interface
Status
Speed
(Gbps)
------------------------------------------------------------------------------sup-fc0
up
1
------------------------------------------------------------------------------Interface
Status
IP Address
Speed
MTU
------------------------------------------------------------------------------mgmt0
up
172.19.48.96/25
100 Mbps
1500
------------------------------------------------------------------------------Interface
Vsan
Admin
Status
Oper
Oper
Trunk
Mode
Speed
Mode
(Gbps)
------------------------------------------------------------------------------port-channel 1
1
on
trunking
TE
4
port-channel 2
1
on
trunking
TE
4
------------------------------------------------------------------------------Interface Vsan
Admin Admin
Status
Oper Profile Port-channel
Mode
Trunk
Mode
Mode
------------------------------------------------------------------------------fcip10
1
auto
on
notConnected -10
--
The following example displays a summary of information:
switch# show interface counters
fc3/1
5 minutes input rate 24 bits/sec, 3 bytes/sec, 0 frames/sec
5 minutes output rate 16 bits/sec, 2 bytes/sec, 0 frames/sec
3502 frames input, 268400 bytes
0 discards, 0 CRC, 0 unknown class
0 too long, 0 too short
3505 frames output, 198888 bytes
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-57
Chapter 3
Verifying Interfaces Configuration
0 discards
1 input OLS, 1 LRR, 1 NOS, 0 loop inits
2 output OLS, 1 LRR, 1 NOS, 0 loop inits
1 link failures, 1 sync losses, 1 signal losses
.
.
.
fc9/8
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
0 frames input, 0 bytes
0 class-2 frames, 0 bytes
0 class-3 frames, 0 bytes
0 class-f frames, 0 bytes
0 discards, 0 CRC, 0 unknown class
0 too long, 0 too short
0 frames output, 0 bytes
0 class-2 frames, 0 bytes
0 class-3 frames, 0 bytes
0 class-f frames, 0 bytes
0 discards
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
0 output OLS, 0 LRR, 0 NOS, 0 loop inits
0 link failures, 0 sync losses, 0 signal losses
16 receive B2B credit remaining
3 transmit B2B credit remaining.
.
.
.
sup-fc0
114000 packets input, 11585632 bytes
0 multicast frames, 0 compressed
0 input errors, 0 frame, 0 overrun 0 fifo
113997 packets output, 10969672 bytes, 0 underruns
0 output errors, 0 collisions, 0 fifo
0 carrier errors
mgmt0
31557 packets input, 2230860 bytes
0 multicast frames, 0 compressed
0 input errors, 0 frame, 0 overrun 0 fifo
26618 packets output, 16824342 bytes, 0 underruns
0 output errors, 0 collisions, 7 fifo
0 carrier errors
vsan1
0 packets input, 0 bytes, 0 errors, 0 multicast
0 packets output, 0 bytes, 0 errors, 0 dropped
.
.
.
port-channel 1
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
0 frames input, 0 bytes
0 class-2 frames, 0 bytes
0 class-3 frames, 0 bytes
0 class-f frames, 0 bytes
0 discards, 0 CRC, 0 unknown class
0 too long, 0 too short
0 frames output, 0 bytes
0 class-2 frames, 0 bytes
0 class-3 frames, 0 bytes
0 class-f frames, 0 bytes
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-58
Configuring Interfaces
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
0 discards
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
0 output OLS, 0 LRR, 0 NOS, 0 loop inits
0 link failures, 0 sync losses, 0 signal losses
Note
Interfaces 9/8 and 9/9 are not trunking ports and display Class 2, 3, and F information as well.
The following example displays the brief counter information of interfaces:
switch# show interface counters brief
------------------------------------------------------------------------------Interface
Input (rate is 5 min avg)
Output (rate is 5 min avg)
----------------------------- ----------------------------Rate
Total
Rate
Total
Mbits/s Frames
Mbits/s Frames
------------------------------------------------------------------------------fc3/1
0
3871
0
3874
fc3/2
0
3902
0
4232
fc3/3
0
3901
0
4138
fc3/4
0
3895
0
3894
fc3/5
0
3890
0
3897
fc9/8
0
0
0
0
fc9/9
0
5
0
4
fc9/10
0
4186
0
4182
fc9/11
0
4331
0
4315
------------------------------------------------------------------------------Interface
Input (rate is 5 min avg)
Output (rate is 5 min avg)
----------------------------- ----------------------------Rate
Total
Rate
Total
Mbits/s Frames
Mbits/s Frames
------------------------------------------------------------------------------port-channel 1
0
0
0
0
port-channel 2
0
3946
0
3946
You can run the show interface transceiver command only on a switch in the Cisco MDS 9100 Series
if the SFP is present, as shown in the following example:
switch# show interface transceiver
fc1/1 SFP is present
name is CISCO-AGILENT
part number is QFBR-5796L
revision is
serial number is A00162193
fc-transmitter type is short wave laser
cisco extended id is unknown (0x0)
.
.
.
fc1/9 SFP is present
name is FINISAR CORP.
part number is FTRJ-1319-7D-CSC
revision is
serial number is H11A6ER
fc-transmitter type is long wave laser cost reduced
cisco extended id is unknown (0x0)
.
.
.
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-59
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
The following example displays the entire running configuration, with information about all the
interfaces. The interfaces have multiple entries in the configuration files to ensure that the interface
configuration commands execute in the correct order when the switch reloads.
switch# show running-config
.
.
.
interface fc9/1
switchport speed 2000
.
.
.
interface fc9/1
switchport mode E
.
.
.
interface fc9/1
channel-group 11 force
no shutdown
The following example displays the running configuration information for a specified interface. The
interface configuration commands are grouped:
switch# show running-config interface fc1/1
interface fc9/1
switchport speed 2000
switchport mode E
channel-group 11 force
no shutdown
The following example displays the running configuration after the system default switchport mode F
command is executed:
switch# show running-config
version 3.1(3)
system default switchport mode F
interface fc4/1
interface fc4/2
interface fc4/3
interface fc4/4
interface fc4/5
interface fc4/6
interface fc4/7
interface fc4/8
interface fc4/9
interface fc4/10
The following example displays the running configuration after two interfaces are individually
configured for FL mode:
switch# show running-config
version 3.1(3)
system default switchport mode F
interface fc4/1
switchport mode FL
interface fc4/2
interface fc4/3
switchport mode FL
interface fc4/4
interface fc4/5
interface fc4/6
interface fc4/7
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-60
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
interface fc4/8
interface fc4/9
interface fc4/1
The following example displays interface information in a brief format after the system default
switchport mode F command is executed:
switch# show interface brief
------------------------------------------------------------------------------Interface Vsan
Admin Admin
Status
SFP
Oper Oper
Port
Mode
Trunk
Mode Speed Channel
Mode
(Gbps)
------------------------------------------------------------------------------fc4/1
1
F
-notConnected
swl
--fc4/2
1
F
-notConnected
swl
--fc4/3
1
F
-notConnected
swl
--fc4/4
1
F
-notConnected
swl
--fc4/5
1
F
-sfpAbsent
---fc4/6
1
F
-sfpAbsent
---fc4/7
1
F
-sfpAbsent
---fc4/8
1
F
-sfpAbsent
---fc4/9
1
F
-sfpAbsent
----
The following example displays interface information in a brief format after two interfaces are
individually configured for FL mode:
switch# show interface brief
------------------------------------------------------------------------------Interface Vsan
Admin Admin
Status
SFP
Oper Oper
Port
Mode
Trunk
Mode Speed Channel
Mode
(Gbps)
------------------------------------------------------------------------------fc4/1
1
FL
-notConnected
swl
--fc4/2
1
F
-notConnected
swl
--fc4/3
1
FL
-notConnected
swl
--fc4/4
1
F
-notConnected
swl
--fc4/5
1
F
-sfpAbsent
---fc4/6
1
F
-sfpAbsent
---fc4/7
1
F
-sfpAbsent
---fc4/8
1
F
-sfpAbsent
---fc4/9
1
F
-sfpAbsent
---fc4/10
1
F
-sfpAbsent
----
Displaying the Port-Level Port Guard
The following command displays information about an interface that is set to error-disabled state by the
port guard because of a TrustSec violation:
switch# show interface fc8/3
fc8/3 is down (Error disabled - port down due to trustsec violation) Hardware is Fibre
Channel, SFP is short wave laser w/o OFC (SN) Port WWN is 21:c3:00:0d:ec:10:57:80
Admin port mode is E, trunk mode is on snmp link state traps are enabled
Port vsan is 1
Receive data field Size is 2112 Beacon is turned off
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
11274 frames input, 1050732 bytes
0 discards, 0 errors
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-61
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
0 CRC, 0 unknown class
0 too long, 0 too short
11242 frames output, 971900 bytes
0 discards, 0 errors
11 input OLS, 34 LRR, 10 NOS, 0 loop inits
72 output OLS, 37 LRR, 2 NOS, 0 loop inits
Interface last changed at Sun Nov 27 07:34:05 1988
Troubleshooting Tips
An interface may be error disabled for several reasons. To recover an error-disabled interface, use the
shutdown and no shutdown commands in interface configuration mode to re-enable the link.
Displaying the Port Monitor Status and Policies
The following commands display information about the Port Monitor feature:
switch# show port-monitor status
Port Monitor
: Enabled
Active Policies : test
Last 100 logs :
send_alarm_tosup, the if_index is 1000000 (hex), value is 7 event id 4 high 5 low 0 sample
2 object link-loss prev_value is 2, curr_value is 9 alarm_sent is 1 portguard_type is 0
20:45:55 UTC Oct 17 2
016
send_alarm_tosup, the if_index is 1000000 (hex), value is 7 event id 4 high 5 low 0 sample
2 object state-change prev_value is 3, curr_value is 10 alarm_sent is 1 portguard_type is
0 20:45:55 UTC Oct
17 2016
switch# show port-monitor PMON_policy
-----------------------------------------------------------------------------------------Port Monitor : enabled
-----------------------------------------------------------------------------------------Policy Name : PMON_policy
Admin status : Not Active
Oper status : Not Active
Port type
: All Ports
-----------------------------------------------------------------------------------------Counter
Threshold
Interval
Rising Threshold event
Falling Threshold
event
PMON Portguard
-----------------------------------------------------------------------------------------Link Loss
Delta
60
5
4
1
4
Not enabled
Sync Loss
Delta
60
5
4
1
4
Not enabled
Signal Loss
Delta
60
5
4
1
4
Not enabled
Invalid Words
Delta
60
1
4
0
4
Not enabled
Invalid CRC's
Delta
60
5
4
1
4
Not enabled
State Change
Delta
60
5
4
0
4
Not enabled
TX Discards
Delta
60
200
4
10
4
Not enabled
LR RX
Delta
60
5
4
1
4
Not enabled
LR TX
Delta
60
5
4
1
4
Not enabled
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-62
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
Timeout
Discards
Delta
60
200
4
10
4
Not enabled
Credit Loss
Reco
Delta
1
1
4
0
4
Not enabled
TX Credit Not
Available
Delta
1
10%
4
0%
4
Not enabled
RX Datarate
Delta
60
80%
4
20%
4
Not enabled
TX Datarate
Delta
60
80%
4
20%
4
Not enabled
TX-Slowport
-Count
Delta
1
5
4
0
4
Not enabled
TX-Slowport
-Oper-Delay
Absolute
1
50ms
4
0ms
4
Not enabled
TXWait
Delta
1
40%
4
0%
4
Not enabled
-----------------------------------------------------------------------------------------switch# show port-monitor active
Policy Name : sample
Admin status : Active
Oper status : Active
Port type
: All Access Ports
-----------------------------------------------------------------------------------------Counter
Threshold Interval Rising Threshold event
Falling Threshold event
portguard
-----------------------------------------------------------------------------------------Link Loss
Delta
60
5
4
1
4
Not enabled
Sync Loss
Delta
60
5
4
1
4
Not enabled
Signal Loss
Delta
60
5
4
1
4
Not enabled
Invalid Words
Delta
60
1
4
0
4
Not enabled
Invalid CRC's
Delta
60
5
4
1
4
Not enabled
State Change
Delta
60
5
4
0
4
Not enabled
TX Discards
Delta
60
200
4
10
4
Not enabled
LR RX
Delta
60
5
4
1
4
Not enabled
LR TX
Delta
60
5
4
1
4
Not enabled
Timeout
Discards
Delta
60
200
4
10
4
Not enabled
Credit Loss
Reco
Delta
1
1
4
0
4
Not enabled
TX Credit Not
Available
Delta
1
10%
4
0%
4
Not enabled
RX Datarate
Delta
60
80%
4
20%
4
Not enabled
TX Datarate
Delta
60
80%
4
20%
4
Not enabled
TX-Slowport
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-63
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
-Count
Delta
1
5
4
0
4
Not enabled
TX-SlowportOper-Delay
Absolute
1
50ms
4
0ms
4
Not enabled
TXWait
Delta
1
40%
4
0%
4
Not enabled
-----------------------------------------------------------------------------------------switch# show port-monitor sample
Policy Name : sample
Admin status : Active
Oper status : Active
Port type
: All Access Ports
-----------------------------------------------------------------------------------------Counter
Threshold Interval Rising Threshold event
Falling Threshold
event
portgurard
-----------------------------------------------------------------------------------------Link Loss
Delta
60
5
4
1
4
Not enabled
Sync Loss
Delta
60
5
4
1
4
Not enabled
Signal Loss
Delta
60
5
4
1
4
Not enabled
Invalid Words
Delta
60
1
4
0
4
Not enabled
Invalid CRC's
Delta
60
5
4
1
4
Not enabled
State Change
Delta
60
5
4
0
4
Not enabled
TX Discards
Delta
60
200
4
10
4
Not enabled
LR RX
Delta
60
5
4
1
4
Not enabled
LR TX
Delta
60
5
4
1
4
Not enabled
Timeout Discards
Delta
60
200
4
10
4
Not enabled
Credit Loss Reco
Delta
1
1
4
0
4
Not enabled
TX Credit Not
Available
Delta
1
10%
4
0%
4
Not enabled
RX Datarate
Delta
60
80%
4
20%
4
Not enabled
TX Datarate
Delta
60
80%
4
20%
4
Not enabled
TX-Slowport-Count Delta
1
5
4
0
4
Not enabled
TX-Slowport-Oper
-Delay
Absolute
1
50ms
4
0ms
4
Not enabled
TXWait
Delta
1
40%
4
0%
4
Not enabled
-----------------------------------------------------------------------------------------switch# show port-monitor default
Policy Name : default
Admin status : Not Active
Oper status : Not Active
Port type
: All Ports
------------------------------------------------------------------------------------------
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-64
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
Counter
Threshold Interval Rising Threshold
event
Falling Threshold
event
PMON Portguard
-----------------------------------------------------------------------------------------Link Loss
Delta
60
5
4
1
4
Not enabled
Sync Loss
Delta
60
5
4
1
4
Not enabled
Signal Loss
Delta
60
5
4
1
4
Not enabled
Invalid Words
Delta
60
1
4
0
4
Not enabled
Invalid CRC's
Delta
60
5
4
1
4
Not enabled
State Change
Delta
60
5
4
0
4
Not enabled
TX Discards
Delta
60
200
4
10
4
Not enabled
LR RX
Delta
60
5
4
1
4
Not enabled
LR TX
Delta
60
5
4
1
4
Not enabled
Timeout Discards
Delta
60
200
4
10
4
Not enabled
Credit Loss Reco
Delta
1
1
4
0
4
Not enabled
TX Credit Not
Available
Delta
1
10%
4
0%
4
Not enabled
RX Datarate
Delta
60
80%
4
20%
4
Not enabled
TX Datarate
Delta
60
80%
4
20%
4
Not enabled
TX-Slowport-Count Delta
1
5
4
0
4
Not enabled
TX-Slowport-Oper
-Delay
Absolute
1
50ms
4
0ms
4
Not enabled
TXWait
Delta
1
40%
4
0%
4
Not enabled
------------------------------------------------------------------------------------------
Note
TX-Slowport-Count is displayed only on switches that use DS-X9224-96K9, DS-X9248-96K9, or
DS-X9248-48K9 modules.
switch# show port-monitor slowdrain
Policy Name : slowdrain
Admin status : Not Active
Oper status : Not Active
Port type
: All Access Ports
-----------------------------------------------------------------------------------------Counter
Threshold
Interval
Rising Threshold
event
Falling Threshold event
PMON Portguard
-----------------------------------------------------------------------------------------Credit Loss Reco Delta
1
1
4
0
4
Not enabled
TX Credit Not
Available
Delta
1
10%
4
0%
4
Not enabled
------------------------------------------------------------------------------------------
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-65
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
Displaying Port Group Monitor Status and Policies
The following examples display information about the port group monitor:
switch# show port-group-monitor status
Port Group Monitor : Enabled
Active Policies : pgm2
Last 100 logs :
switch#
switch# show port-group-monitor
-----------------------------------------------------------------------------------------Port Group Monitor : enabled
-----------------------------------------------------------------------------------------Policy Name : pgm1
Admin status : Not Active
Oper status : Not Active
Port type
: All Port Groups
-----------------------------------------------------------------------------------------Counter
Threshold Interval %ge Rising Threshold %ge Falling Threshold
--------------- -------- -------------------- ---------------------RX Performance
Delta
60
50
10
TX Performance
Delta
60
50
10
-----------------------------------------------------------------------------------------Policy Name : pgm2
Admin status : Active
Oper status : Active
Port type
: All Port Groups
-----------------------------------------------------------------------------------------Counter
Threshold Interval %ge Rising Threshold %ge Falling Threshold
--------------- -------- -------------------- ---------------------RX Performance
Delta
60
80
10
TX Performance
Delta
60
80
10
-----------------------------------------------------------------------------------------Policy Name : default
Admin status : Not Active
Oper status : Not Active
Port type
: All Port Groups
-----------------------------------------------------------------------------------------Counter
Threshold Interval %ge Rising Threshold %ge Falling Threshold
--------------- -------- -------------------- ---------------------RX Performance
Delta
60
80
20
TX Performance
Delta
60
80
20
-----------------------------------------------------------------------------------------switch# show port-group-monitor active
Policy Name : pgm2
Admin status : Active
Oper status : Active
Port type
: All Port Groups
-----------------------------------------------------------------------------------------Counter
Threshold Interval %ge Rising Threshold %ge Falling Threshold
--------------- -------- -------------------- ---------------------RX Performance
Delta
60
80
10
TX Performance
Delta
60
80
10
-----------------------------------------------------------------------------------------switch# show
Policy Name
Admin status
Oper status
Port type
port-group-monitor PGMON_policy
: PGMON_policy
: Not Active
: Not Active
: All Port Groups
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-66
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
-----------------------------------------------------------------------------------------Counter
Threshold Interval %ge Rising Threshold %ge Falling Threshold
--------------- -------- -------------------- ---------------------RX Datarate
Delta
60
80
20
TX Datarate
Delta
60
80
20
------------------------------------------------------------------------------------------
Displaying the Management Interface Configuration
The following command displays the management interface configuration:
switch# show interface mgmt 0
mgmt0 is up
Hardware is FastEthernet
Address is 000c.30d9.fdbc
Internet address is 10.16.1.2/24
MTU 1500 bytes, BW 100 Mbps full Duplex
26388 packets input, 6101647 bytes
0 multicast frames, 0 compressed
0 input errors, 0 frame, 0 overrun 0 fifo
10247 packets output, 2389196 bytes, 0 underruns
0 output errors, 0 collisions, 0 fifo
0 carrier errors
Displaying VSAN Interface Information
The following example displays the VSAN interface information:
switch# show interface vsan 2
vsan2 is up, line protocol is up
WWPN is 10:00:00:05:30:00:59:1f, FCID is 0xb90100
Internet address is 10.1.1.1/24
MTU 1500 bytes, BW 1000000 Kbit
0 packets input, 0 bytes, 0 errors, 0 multicast
0 packets output, 0 bytes, 0 errors, 0 dropped
Displaying the Congestion Frame Timeout Value for FCoE
The following commands display the congestion frame timeout value for FCoE (pause counter log and
pause event log, respectively, with timeout value):
(Optional) Display the pause counter log with time-stamp information:
switch# show logging onboard flow-control pause-count
(Optional) Display the pause event log with time-stamp information:
switch# show logging onboard flow-control pause-events
Displaying the Pause Frame Timeout Value for FCoE
The following commands display the pause frame timeout value for FCoE:
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-67
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
(Optional) Display the pause counter log with time-stamp information:
switch# show logging onboard flow-control pause-count
(Optional) Display the pause counters per module per interface with time-stamp information:
switch# show logging onboard flow-control pause-events
(Optional) Display the timeout drops per module per interface with time-stamp information:
switch# show logging onboard flow-control timeout-drops [module x] [last mm minutes] [last
hh hours] [last dd days]
Displaying the Congestion Drop Timeout Value for Fibre Channel
The following command displays the timeout drops per module per interface with time-stamp
information:
switch# show logging onboard flow-control timeout-drops [module x] [last mm minutes] [last
hh hours] [last dd days]
Displaying the No-Credit Frame Timeout Value for Fibre Channel
The following command displays various error statistics per module per interface with time-stamp
information:
switch# show logging onboard [module x] [starttime mm/dd/yy-hh:mm:ss] error-stats
The following counters indicate that the no-credit drop threshold has been reached:
•
FCP_CNTR_FORCE_TIMEOUT_ON
•
AK_FCP_CNTR_FORCE_TIMEOUT_ON
•
FCP_SW_CNTR_FORCE_TIMEOUT_ON
The following counters indicate that a credit has been received on the interface, and the port no longer
drops packets because of the no-credit drop condition:
•
FCP_CNTR_FORCE_TIMEOUT_OFF
•
AK_FCP_CNTR_FORCE_TIMEOUT_OFF
•
FCP_SW_CNTR_FORCE_TIMEOUT_OFF
Displaying Slow-Port Monitor Events
The following commands display slow-port monitor events:
Note
These commands are applicable for both supervisor and module prompts.
Display slow-port monitor events per module:
switch# show process creditmon slowport-monitor-events [module x [port y]]
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-68
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
Display the slow-port monitor events on the Onboard Failure Logging (OBFL):
switch# show logging onboard slowport-monitor-events
Note
The slow-port monitor events are logged periodically into the OBFL.
The following example displays the credit monitor or output of the creditmon slow-port
monitor-events command for the 16-Gbps modules:
switch# show process creditmon slowport-monitor-events
Module: 06
Slowport Detected: YES
=========================================================================
Interface = fc6/3
---------------------------------------------------------------| admin | slowport | oper |
Timestamp
|
| delay | detection | delay |
|
| (ms)
| count
| (ms) |
|
---------------------------------------------------------------|
1
|
46195 |
1 | 1. 10/14/12 21:46:51.615
|
|
1
|
46193 |
50 | 2. 10/14/12 21:46:51.515
|
|
1
|
46191 |
50 | 3. 10/14/12 21:46:51.415
|
|
1
|
46189 |
50 | 4. 10/14/12 21:46:51.315
|
|
1
|
46187 |
50 | 5. 10/14/12 21:46:51.215
|
|
1
|
46185 |
50 | 6. 10/14/12 21:46:51.115
|
|
1
|
46183 |
50 | 7. 10/14/12 21:46:51.015
|
|
1
|
46181 |
50 | 8. 10/14/12 21:46:50.915
|
|
1
|
46179 |
50 | 9. 10/14/12 21:46:50.815
|
|
1
|
46178 |
50 |10. 10/14/12 21:46:50.715
|
----------------------------------------------------------------
Note
For 16-Gbps modules and Cisco MDS 9700, 9148S, 9250i, and 9396S switches, if no-credit-drop
timeout is configured, the maximum value of tx-slowport-oper-delay as shown in slow-port monitor
events is limited by the no-credit-drop timeout. So, the maximum value for tx-slowport-oper-delay can
reach the level of the no-credit-drop timeout even if the actual slow-port delay from the device is higher
because the frames are forcefully dropped by the hardware when tx-slowport-oper-delay reaches the
level of the no-credit-drop timeout.
The following example displays the output of the creditmon slowport-monitor-events command for the
Cisco MDS 9500 switches (8-Gbps modules):
switch# show process creditmon slowport-monitor-events
Module: 04
Slowport Detected: YES
========================================================================
Interface = fc4/13
-------------------------------------------------------| admin | slowport |
Timestamp
|
| delay | detection |
|
| (ms)
| count
|
|
-------------------------------------------------------| 1
|
194 | 1. 04/29/15 17:19:13.345
|
| 1
|
193 | 2. 04/29/15 17:19:13.245
|
| 1
|
192 | 3. 04/29/15 17:19:13.145
|
| 1
|
191 | 4. 04/29/15 17:19:13.045
|
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-69
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
| 1
|
190 | 5. 04/29/15 17:19:12.945
|
| 1
|
189 | 6. 04/29/15 17:19:12.845
|
| 1
|
188 | 7. 04/29/15 17:19:12.745
|
| 1
|
187 | 8. 04/29/15 17:19:12.645
|
| 1
|
186 | 9. 04/29/15 17:19:12.545
|
| 1
|
185 |10. 04/29/15 17:19:12.445
|
-------------------------------------------------------========================================================================
Note
The Cisco MDS 9500 Series 8-Gbps modules can only detect whether the slow-port monitor has reached
the threshold (admin delay) or not for every 100-ms polling interval. The modules cannot determine the
actual length of time, whether it is higher than the threshold, or when the port is in the
zero-transmit-credits-remaining condition. Also, the modules cannot determine if the slow-port monitor
has reached the threshold (admin delay) multiple times. The modules can record only one event per
100-ms polling interval.
The following example displays output of the creditmon slow-port-monitor-events command for the
Cisco MDS 9500 switches (advanced 8-Gbps modules):
Note
The Cisco MDS 9500 Series advanced 8-Gbps modules utilize the transmit wait functionality to
implement slow-port monitoring. Hence, tx-slowport-oper-delay is the total amount of time the port
was in the zero-transmit-credits-remaining condition during the 100-ms polling interval. No specific
duration of time is indicated.
switch# show process creditmon slowport-monitor-events module 1
Module: 01
Slowport Detected: YES
===================================================================
Interface = fc1/5
---------------------------------------------------------------| admin | slowport | txwait|
Timestamp
|
| delay | detection | oper |
|
| (ms)
| count
| delay |
|
|
|
| (ms) |
|
---------------------------------------------------------------| 10
|
888 |
93 | 1. 04/30/15 21:33:42.561
|
| 10
|
887 |
81 | 2. 04/30/15 21:33:42.461
|
| 10
|
886 |
76 | 3. 04/30/15 21:33:42.361
|
| 10
|
885 |
99 | 4. 04/30/15 21:33:42.261
|
| 10
|
884 |
99 | 5. 04/30/15 21:33:42.161
|
-------------------------------------------------------========================================================================
Note
For advanced 8-Gbps modules, the transmit-wait value does not increment after the no-credit-drop
threshold has been reached because the frames are forcefully dropped by the hardware, and no more
frames are queued for transmit. Consequently, when slow-port monitor is used with no-credit-drop, the
tx-slowport-oper-delay value, as shown in the output of the slow-port monitor events command may
be lower than expected.
The following example displays the transmit-wait statistics for a particular interface for the Cisco MDS
9500 switches (advanced 8-Gbps modules and 16-Gbps modules):
switch# show interface fc1/1 counters
or
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-70
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
switch# show interface fc1/1 counters details
switch(config)# show int fc1/81 counters
fc1/81
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 280288 bits/sec, 35036 bytes/sec, 15 frames/sec
5206879 frames input, 11142684612 bytes
0 class-2 frames, 0 bytes
5206879 class-3 frames, 11142684612 bytes
0 class-f frames, 0 bytes
171 discards, 175 errors, 0 CRC/FCS
0 unknown class, 0 too long, 4 too short
2498081 frames output, 5345868788 bytes
0 class-2 frames, 0 bytes
2498081 class-3 frames, 5345868788 bytes
0 class-f frames, 0 bytes
7260927715 discards, 0 errors
7260927715 timeout discards, 0 credit loss
2 input OLS, 272 LRR, 0 NOS, 0 loop inits
3 output OLS, 3 LRR, 2 NOS, 0 loop inits
2 link failures, 0 sync losses, 0 signal losses
2498321 Transmit B2B credit transitions to zero
275 Receive B2B credit transitions to zero
54867361792 2.5us TxWait due to lack of transmit credits
Percentage Tx credits not available for last 1s/1m/1h/72h: 50%/50%/92%/52%
32 receive B2B credit remaining
0 transmit B2B credit remaining
0 low priority transmit B2B credit remaining
Last clearing of "show interface" counters :never
=================================================================================
switch(config)# show int fc1/81 counters details
fc1/81
5206879 frames, 11142684612 bytes received
0 class-2 frames, 0 bytes received
0 class-2 discards
0 F_BSY frames, 0 F_RJT frames
generated against class-2 frames
0 port reject frames
5206879 class-3 frames, 11142684612 bytes received
0 class-f frames, 0 bytes received
171 discards, 175 errors received
7273423181 discards, 0 errors transmitted
2499069 frames, 5347983108 bytes transmitted
0 class-2 frames, 0 bytes transmitted
2499069 class-3 frames, 5347983108 bytes transmitted
171 class-3 frames discarded
0 class-f frames, 0 bytes transmitted
0 class-f frames discarded
0 multicast packets received, 0 transmitted
0 broadcast packets received, 0 transmitted
5206879 unicast packets received, 2499069 transmitted
7273423181 timeout discards, 0 credit loss
2 link failures, 0 sync losses,
0 signal losses
0 primitive sequence protocol errors
31822 invalid transmission words
0 invalid CRCs, 0 Delimiter Errors
0 address identifier errors
0 link reset received while link is active
272 link reset transmitted while link is active
2 Offline Sequence errors received
3 Offline Sequence errors transmitted
0 frames received that are shorter than
the minimum allowable frame length
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-71
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
regardless of the CRC/FCS error
0 frames received that are longer than
the maximum frame length and also have a
CRC/FCS error
54879203328 2.5us TxWait due to lack of transmit credits
0 frames received with length greater
than what was agreed to in FLOGI/PLOGI
4 frames received with length less than
the minimum indicated by the frame header
272 link reset responses received
3 link reset responses transmitted
0 non-operational sequences received
2 non-operational sequences transmitted
0 fragmented frames received
171 frames received with EOF aborts
0 unknown class frames received
0 8b10b disparity errors
0 frames discarded
0 Exchange Link Parameters switch fabric
internal link service request failures
2499309 Transmit B2B credit transitions to zero
275 Receive B2B credit transitions to zero
0 Enhanced Inter Switch Link (EISL) frames
discarded
0 framing errors
0 F8 type LIP sequence errors received
0 F8 type LIP sequence errors issued
0 Non F8 type LIP sequence errors received
0 Non F8 type LIP sequence errors issued
0 fec corrected blocks
0 fec uncorrected blocks
Percentage Tx credits not available for last 1s/1m/1h/72h: 50%/50%/92%/52%
Transmit-Wait History Graph
The transmit-wait history for the slow ports on advanced 8 and 16 Gbps modules and switches can be
displayed in the form of a graph over a period of time. The total transmit-wait time for each time period
is displayed as a column of #. The actual value appears above each column as a vertically printed number.
The following graphs can be displayed:
•
Seconds scale—The transmit-wait history for the port over the last 60 seconds. The Y-axis value is
the total transmit-wait time for each second, in milliseconds.
•
Minutes scale—The transmit-wait history for the port over the last 60 seconds. The Y-axis value is
the total transmit-wait time for each minute, in seconds, to one decimal place.
•
Hours scale—The transmit-wait history for the port over the last 60 seconds. The Y-axis value is the
total transmit-wait time for each hour, in minutes.
To display the transmit-wait history for a given interval of time, use the following commands:
Display the transmit-wait history graph for the period when transmit credit is not available for a given
interval of time (seconds, minutes, or hours):
switch# show process creditmon txwait-history [module x [port y]]
Display the transmit-wait time in 2.5-microsecond units, as well as in seconds:
switch# show logging onboard txwait
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-72
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
Note
The transmit-wait delta values are logged periodically (every 20 seconds) into the OBFL when
transmit wait increases by at least 100 ms in the 20-second interval.
Display the total transmit-wait value for a particular interface in 2.5-microsecond units:
switch# show interface fcx/y counters
The following example displays the transmit-wait history graph, in seconds, for advanced 8-Gbps
modules and 16-Gbps modules:
switch(config)# show process creditmon txwait-history module 1 port 81
TxWait history for port fc1/81:
==============================
455555555455554555554555599999999999999999999999999999999999
900000000800009000008100011111231922322211321121112112113111
433799991899990359909838608935137962088988254848894870461938
1000
#
900
###################################
800
###################################
700
###################################
600
###################################
500 ############################################################
400 ############################################################
300 ############################################################
200 ############################################################
100 ############################################################
0....5....1....1....2....2....3....3....4....4....5....5....6
0
5
0
5
0
5
0
5
0
5
0
Tx Credit Not Available per second (last 60 seconds)
# = TxWait (ms)
The following example displays the transmit-wait history graph, in minutes, for advanced 8-Gbps
modules and 16-Gbps modules:
555555555555555555555555555555555555555555555555555555555555
055555555555555555555555555555555555555555555555555555555555
............................................................
035444445444445445454444444445635363534375434453343554545344
60
54 ###########################################################
48 ############################################################
42 ############################################################
36 ############################################################
30 ############################################################
24 ############################################################
18 ############################################################
12 ############################################################
6 ############################################################
0....5....1....1....2....2....3....3....4....4....5....5....6
0
5
0
5
0
5
0
5
0
5
0
Tx Credit Not Available per minute (last 60 minutes)
# = TxWait (secs)
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-73
Chapter 3
Configuring Interfaces
Verifying Interfaces Configuration
The following example displays the transmit wait history graph, in hours, for advanced 8-Gbps modules
and 16-Gbps modules:
33333333333333333333333333333333333333333
33333333333333333333333333333333333333332
22222222222222222222222222222222222222229
777788777777877877778777877677777777876790000000000000000000000000000000
3600
3240
2880
2520
2160
1800
1440
1080
720
360
#########################################
#########################################
#########################################
#########################################
#########################################
#########################################
#########################################
#########################################
#########################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.7
0
5
0
5
0
5
0
5
0
5
0
5
0 2
Tx Credit Not Available per hour (last 72 hours)
# = TxWait (secs)
The following example displays the transmit-wait OBFL logging for advanced 8-Gbps modules and
16-Gbps modules:
switch# show logging onboard txwait
Notes:
- sampling period is 20 seconds
- only txwait delta value >= 100 ms are logged
--------------------------------Module: 1 txwait count
------------------------------------------------------------------------------------------------------------| Interface | Delta TxWait Time
| Congestion | Timestamp
|
|
| 2.5us ticks | seconds |
|
|
----------------------------------------------------------------------------|
fc1/11 | 3435973
| 08
|
42%
| Sun Sep 30 05:23:05 2001 |
|
fc1/11 | 6871947
| 17
|
85%
| Sun Sep 30 05:22:25 2001 |
Cisco MDS 9000 Series NX-OS Fabric Configuration Guide
3-74
CHAPTER
4
Configuring Interface Buffers
•
Information About Interface Buffers, page 4-1
•
Configuring Interface Buffers, page 4-23
•
Verifying BB_Credit Configuration, page 4-27
Information About Interface Buffers
Fibre Channel interfaces use buffer credits to ensure all packets are delivered to their destination.
This section includes the following topics:
•
Buffer-to-Buffer Credits, page 4-1
•
Performance Buffers, page 4-2
•
Buffer Pools, page 4-2
•
BB_Credit Buffers for Switching Modules, page 4-5
•
BB_Credit Buffers for Fabric Switches, page 4-16
•
Extended BB_Credits, page 4-20
•
Buffer-to-Buffer Credit Recovery, page 4-22
•
Buffer-to-Buffer State Change Number, page 4-22
•
Receive Data Field Size, page 4-23
Buffer-to-Buffer Credits
Buffer-to-buffer credits (BB_credits) are a flow-control mechanism to ensure that Fibre Channel
switches do not run out of buffers, so that switches do not drop frames. BB_credits are negotiated on a
per-hop basis.
The receive BB_credit (fcrxbbcredit) value may be configured for each Fibre Channel interface. In most
cases, you do not need to modify the default configuration.
The receive BB_credit values depend on the module type and the port mode, as follows:
•
For 16-port switching modules and full rate ports, the default value is 16 for Fx mode and 255 for
E or TE modes. The maximum value is 255 in all modes. This value can be changed as required.
•
For 32-port switching modules and host-optimized ports, the default value is 12 for Fx, E, and TE
modes. These values cannot be changed.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-1
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
•
For 4-Gbps, 8-Gbps, advanced 8-Gbps switching modules, see the “Buffer Pools” section on
page 4-2.
Note
In the Cisco MDS 9100 Series switches, the groups of ports on the left outlined in white are in dedicated
rate mode. The other ports are host-optimized. Each group of 4 host-optimized ports have the same
features as for the 32-port switching module.
Note
Because Generation 1 modules do not support as many buffer-to-buffer credits as advanced 8-Gbps
modules support, you cannot configure an ISL on E or TE ports between a Generation 1 module such as
the 16-port 1-, 2-Gbps Fibre Channel Switching Module (DS-X9016) and a advanced 8-Gbps module
such as the 48 port 8-Gbps Advanced Fibre Channel module (DS-X9248-256K9) or the 32-port 8-Gbps
Advanced Fibre Channel module (DS-X9232-256k9).
Performance Buffers
Regardless of the configured receive BB_credit value, additional buffers, called performance buffers,
improve switch port performance. Instead of relying on the built-in switch algorithm, you can manually
configure the performance buffer value for specific applications (for example, forwarding frames over
FCIP interfaces).
Note
Performance buffers are not supported on the Cisco MDS 9148 Fabric Switch, Cisco MDS 9124 Fabric
Switch, the Cisco Fabric Switch for HP c-Class BladeSystem, and the Cisco Fabric Switch for IBM
BladeCenter.
For each physical Fibre Channel interface in any switch in the Cisco MDS 9000 Family, you can specify
the amount of performance buffers allocated in addition to the configured receive BB_credit value.
The default performance buffer value is 0. If you set the performance buffer value to 0, the built-in
algorithm is used. If you do not specify the performance buffer value, 0 is automatically used.
The default performance buffer value is 0. If you use the default option, the built-in algorithm is used.
If you do not specify this command, the default option is automatically used.
Buffer Pools
In the architecture of 4-Gbps, 8-Gbps, and 16-Gbps modules, receive buffers shared by a set of ports are
called buffer groups. The receive buffer groups are organized into global and local buffer pools.
The receive buffers allocated from the global buffer pool to be shared by a port group are called a global
receive buffer pool. Global receive buffer pools include the following buffer groups:
•
Reserved internal buffers
•
Allocated BB_credit buffers for each Fibre Channel interface (user configured or assigned by
default)
•
Common unallocated buffer pool for BB_credits, if any, to be used for additional BB_credits as
needed
•
Performance buffers (only used on 12-port 4-Gbps and 4-port 10-Gbps switching modules)
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-2
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Note
The 48-port and 24-port 8-Gbps modules have dual global buffer pools. Each buffer pool in the 48-port
modules support 24 ports and in the 24-port modules each buffer pool supports 12 ports.
Figure 4-1 shows the allocation of BB_credit buffers on line cards (24-port and 48-port 4-Gbps line
cards).
Figure 4-1
Receive Buffers for Fibre Channel Ports in a Global Buffer Pool
Common unallocated buffer pool for BB_credits
1
2
3
Allocated BB_credit buffers
for each front panel FC ports
N
185164
Performance Buffers (Shared Pool)
Total BB_credit
buffers
Maximim Receive
buffers
Reserved Internal Buffers (not user configurable)
Figure 4-2 shows the default BB_credit buffer allocation model for 48-port 8-Gbps switching modules.
The minimum BB_credits required to bring up a port is two buffers.
Figure 4-2
BB_Credit Buffer Allocation in 48-Port 8-Gbps Switching Modules
48-port module
Mixed
Dedicated
1 Gbps
250 BB credits
All ports
dedicated (2 Gbps)
250 BB credits
2 Gbps
250 BB credits
Shared
32 BB credits
4 Gbps
250 BB credits
8 Gbps
250 BB credits
189048
All ports
shared (8 Gpbs)
32 BB credits
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-3
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Figure 4-3 shows the default BB_credit buffer allocation model for 24-port 8-Gbps switching modules.
The minimum BB_credits required to bring up a port is two buffers.
Figure 4-3
BB_Credit Buffer Allocation in 24-Port 8-Gbps Switching Modules
24-port module
All ports
shared (8 Gpbs)
32 BB credits
All ports
dedicated (2 Gbps)
500 BB credits
Mixed
1 Gbps
500 BB credits
2 Gbps
500 BB credits
8 Gbps
500 BB credits
4 Gbps
500 BB credits
189047
Shared
32 BB credits
Dedicated
Figure 4-4 shows the default BB_credit buffer allocation model for 4/44-port 8-Gbps host-optimized
switching modules. The minimum BB_credits required to bring up a port is two buffers.
Figure 4-4
BB_Credit Buffer Allocation in 4/44-Port 8-Gbps Switching Modules
4/44-port module
Dedicated
1 Gbps
250 BB credits
All ports
dedicated (2 Gbps)
125 BB credits
Mixed
2 Gbps
250 BB credits
Shared
32 BB credits
4 Gbps
250 BB credits
8 Gbps
125 BB credits
189049
All ports
shared (8 Gpbs)
32 BB credits
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-4
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Figure 4-5 shows the default BB_credit buffer allocation model for 24-port 4-Gbps switching modules.
The minimum BB_credits required to bring up a port is two buffers.
Figure 4-5
BB_Credit Buffer Allocation in 24-Port 4-Gbps Switching Modules
4
24-port module
Mixed
Dedicated
1 Gbps
250 BB credits
Note
All ports
dedicated (2 Gbps)
250 BB credits
2 Gbps
250 BB credits
Shared
16 BB credits
4 Gbps
250 BB credits
144856
All ports
shared (4 Gpbs)
16 BB credits
The default BB_credit buffer allocation is the same for all port speeds.
BB_Credit Buffers for Switching Modules
This section describes how buffer credits are allocated to Cisco MDS 9000 switching modules, and
includes the following topics:
•
Configuring Buffer Credits on a 4-Gbps, 8-Gbps, or Advanced 8-Gbps Module, page 4-5
•
48-Port 16-Gbps Fibre Channel Module BB_Credit Buffers, page 4-6
•
48-Port 8-Gbps Advanced Fibre Channel Module BB_Credit Buffers, page 4-7
•
48-Port 8-Gbps Fibre Channel Module BB_Credit Buffers, page 4-8
•
24-Port 8-Gbps Fibre Channel Module BB_Credit Buffers, page 4-9
•
4/44-Port 8-Gbps Host-Optimized Fibre Channel Module BB_Credit Buffers, page 4-10
•
48-Port 4-Gbps Fibre Channel Module BB_Credit Buffers, page 4-11
•
24-Port 4-Gbps Fibre Channel Module BB_Credit Buffers, page 4-13
•
18-Port Fibre Channel/4-Port Gigabit Ethernet Multiservice Module BB_Credit Buffers, page 4-14
•
4-Port 10-Gbps Switching Module BB_Credit Buffers, page 4-16
Configuring Buffer Credits on a 4-Gbps, 8-Gbps, or Advanced 8-Gbps Module
When you configure port mode to auto or E on a 4-Gbps module, one of the ports will not come up for
the following configuration:
•
Port Mode: auto or E for all of the ports
•
Rate Mode: dedicated
•
Buffer Credits: default value
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-5
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
When you configure port mode to auto or E on a 8-Gbps module, one or two of the ports will not come
up for the following configuration:
•
Port Mode: auto or E for the first half of the ports, the second half of the ports or for all of the ports
•
Rate Mode: dedicated
•
Buffer Credits: default value
When you configure port mode to auto or E for all ports in the global buffer pool, you need to reconfigure
buffer credits on one or more of the ports. The total number of buffer credits configured for all the ports
in the global buffer pool should be reduced by 64.
48-Port 16-Gbps Fibre Channel Module BB_Credit Buffers
Table 4-1 lists the BB_credit buffer allocation for the 48-port 16-Gbps Fibre Channel Switching Module
(DS-X9448-768K9).
Table 4-1
48-Port 16-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate Mode
16-Gbps Speed
Note
BB_Credit Buffer Allocation
ISL
Fx Port
Default BB_credit buffers
500
32
Maximum BB_credit buffers
500
500
Cisco MDS 9700 linecard is a full-rate card.
The following guidelines apply to BB_credit buffers on the 48-port 16-Gbps Fibre Channel Switching
Modules:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 500 buffers for a dedicated rate mode.
•
BB_credit buffers for Fx port mode connections can be configured from a minimum of 2 buffers to
a maximum of 500 buffers for a dedicated rate mode.
•
Per-port credits can be increased up to 4095 using extended buffer to buffer credits if the user has
installed an enterprise license.
•
The extended buffer feature can be activated by using the following commands:
Switch(config)#int fc1/5
switch(config-if)#switchport fcrxbbcredit extended 4095.
Note
In MDS 9700 Series Switching Modules, total buffer available are 49800 for 12 port group. One
port group comprises of 4 ports, and there are 2 port groups per ASIC. Each port-group consists
of total 4150 buffers. These buffers can be allocated to any combination of port(s) using
extended buffer configuration. Please see show port-resource module module_number
command for details about buffers supported by port-groups.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-6
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
48-Port 8-Gbps Advanced Fibre Channel Module BB_Credit Buffers
Table 4-2 lists the BB_credit buffer allocation for the 48-port 8-Gbps Advanced Fibre Channel switching
module.
Table 4-2
48-Port 8-Gbps Advanced Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate
Mode
8-Gbps Speed
Shared Rate Mode
8-Gbps Speed
BB_Credit Buffer Allocation
ISL
Fx Port
Default BB_credit buffers
250 for 48 32
port
Fx Port
32
500 for 32
port
Maximum BB_credit buffers
500
500
32
The following guidelines apply to BB_credit buffers on 32/48-port Advanced 8-Gbps Fibre Channel
switching modules:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 500 buffers for dedicated rate mode.
•
BB_credit buffers for Fx port mode connections can be configured. The minimum is 2 buffers and
the maximum of 500 buffers for dedicated rate mode or 32 buffers for shared rate mode.
•
Performance buffers are not supported on this module.
•
The buffers should not be allocated automatically.
Each port group on the 32/48-port Advanced 8-Gbps Fibre Channel switching module consists of
four/six ports. The ports in shared rate mode in a port group can have a maximum bandwidth
oversubscription of 1.5:1 considering that each port group has 32-Gbps bandwidth. In case of 32 Port
version, each port group of 4 ports has sufficient bandwidth (32 Gbps) to handle the line rate traffic
without any oversubscription.
The following example configurations are supported by the 48-port Advanced 8-Gbps Fibre Channel
switching modules:
•
Six ports with shared rate mode and 8-Gbps speed (1.5:1 oversubscription) (default).
•
Two port with dedicated rate mode and 8-Gbps speed plus four ports with shared rate mode and
8-Gbps speed (2:1 oversubscription).
•
Two ports with dedicated rate mode and 8-Gbps speed plus four ports with shared rate mode and
8-Gbps speed (2:1 oversubscription) .
•
One port with dedicated rate mode and 8-Gbps speed plus three ports with dedicated rate mode and
4-Gbps speed plus two ports with shared rate mode and 8-Gbps speed (1.33:1 oversubscription).
•
Six ports with dedicated rate mode and 8-Gbps speed.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-7
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
48-Port 8-Gbps Fibre Channel Module BB_Credit Buffers
Table 4-3 lists the BB_credit buffer allocation for the 48-port 8-Gbps Fibre Channel switching module.
Table 4-3
48-Port 8-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate
Mode
8-Gbps Speed
Shared Rate Mode
8-Gbps Speed
BB_Credit Buffer Allocation
ISL
Fx Port
Fx Port
Default BB_credit buffers
250
32
32
Maximum BB_credit buffers
500
500
32
Total Number of BB_Credit Buffers per Module
Ports 1 through 24
6000
Ports 25 through 48
6000
The following guidelines apply to BB_credit buffers on 48-port 8-Gbps Fibre Channel switching
modules:
•
BB_credit buffers allocated for ports 1 through 24 and 25 through 48 can be a maximum of 6000
each so that the load is distributed.
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 500 buffers for dedicated rate mode.
•
BB_credit buffers for Fx port mode connections can be configured. The minimum is 2 buffers and
the maximum of 500 buffers for dedicated rate mode or 32 buffers for shared rate mode.
•
Performance buffers are not supported on this module.
•
The buffers should not be allocated automatically.
Each port group on the 48-port 8-Gbps Fibre Channel switching module consists of six ports. The ports
in shared rate mode in a port group can have a maximum bandwidth oversubscription of 10:1 considering
that each port group has 12.8-Gbps bandwidth.
The following example configurations are supported by the 48-port 8-Gbps Fibre Channel switching
modules:
•
Six ports with shared rate mode and 8-Gbps speed (4:1 oversubscription) (default)
•
One port with dedicated rate mode and 8-Gbps speed plus
five ports with shared rate mode and 8-Gbps speed (10:1 oversubscription)
•
Two ports with dedicated rate mode and 4-Gbps speed plus
four ports with shared rate mode and 4-Gbps speed (4:1 oversubscription)
•
One port with dedicated rate mode and 4-Gbps speed plus
three ports with dedicated rate mode and 2-Gbps speed plus
two ports with shared rate mode and 4-Gbps speed (4:1 oversubscription)
•
Six ports with dedicated rate mode and 2-Gbps speed
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-8
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
24-Port 8-Gbps Fibre Channel Module BB_Credit Buffers
Table 4-4 lists the BB_credit buffer allocation for the 24-port 8-Gbps Fibre Channel switching module.
Table 4-4
24-Port 8-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate
Mode
8-Gbps Speed
Shared Rate Mode
8-Gbps Speed
BB_Credit Buffer Allocation
ISL
Fx Port
Fx Port
Default BB_credit buffers
500
32
32
Maximum BB_credit buffers
500
1
500
1
32
Total Number of BB_Credit Buffers per Module
Ports 1 through 12
6000
Ports 13 through 24
6000
1. When connected to Generation 1 modules, reduce the maximum BB_credit allocation to 250.
The following guidelines apply to BB_credit buffers on 24-port 8-Gbps Fibre Channel switching
modules:
•
BB_credit buffers allocated for ports 1 through 12 and 13 through 24 can be a maximum of 6000
each so that the load is distributed.
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 500 buffers for dedicated rate mode.
•
BB_credit buffers for Fx port mode connections can be configured. The minimum is 2 buffers and
the maximum of 500 buffers for dedicated rate mode or 32 buffers for shared rate mode.
•
Performance buffers are not supported on this module.
Each port group on the 24-port 8-Gbps Fibre Channel switching module consists of three ports. The ports
in shared rate mode in a port group can have a maximum bandwidth oversubscription of 10:1 considering
that each port group has 12.8-Gbps bandwidth.
The following example configurations are supported by the 24-port 8-Gbps Fibre Channel switching
modules:
•
Three ports with shared rate mode and 8-Gbps speed (2:1 oversubscription) (default)
•
One port with dedicated rate mode and 8-Gbps speed plus
two ports with shared rate mode and 8-Gbps speed (4:1 oversubscription)
•
One port with dedicated rate mode and 8-Gbps speed plus
one port with dedicated rate mode and 4-Gbps speed plus
one port with shared rate mode and 8-Gbps speed (10:1 oversubscription)
•
Two ports with dedicated rate mode and 4-Gbps speed plus
one port with shared rate mode and 8-Gbps speed (2:1 oversubscription)
•
Three ports with dedicated rate mode and 4-Gbps speed
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-9
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
4/44-Port 8-Gbps Host-Optimized Fibre Channel Module BB_Credit Buffers
Table 4-5 lists the BB_credit buffer allocation for the 4/44-port 8-Gbps Fibre Channel switching module.
Table 4-5
4/44-Port 8-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate
Mode
8-Gbps Speed
Shared Rate Mode
8-Gbps Speed
BB_Credit Buffer Allocation
ISL
Fx Port
Fx Port
Default BB_credit buffers
125
32
32
Maximum BB_credit buffers
250
250
32
Total number of BB_credit buffers per module
6000
The following guidelines apply to BB_credit buffers on 4/44-port 8-Gbps Fibre Channel switching
modules:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 500 buffers for dedicated rate mode.
•
BB_credit buffers for Fx port mode connections can be configured. The minimum is 2 buffers and
the maximum of 250 buffers for dedicated rate mode or 32 buffers for shared rate mode.
•
Performance buffers are not supported on this module.
Each port group on the 24-port 8-Gbps Fibre Channel switching module consists of 12 ports. The ports
in shared rate mode in a port group can have a maximum bandwidth oversubscription of 10:1 considering
that each port group has 12.8-Gbps bandwidth.
The following example configurations are supported by the 4/44-port 8-Gbps Fibre Channel switching
modules:
•
Twelve ports with shared rate mode and 4-Gbps speed (5:1 oversubscription) (default)
•
One port with dedicated rate mode and 8-Gbps speed plus
eleven ports with shared rate mode and 4-Gbps speed (10:1 oversubscription)
•
One port with dedicated rate mode and 4-Gbps speed plus
three ports with dedicated rate mode and 3-Gbps speed plus
eight ports with shared rate mode and 4-Gbps speed (2:1 oversubscription)
•
Twelve ports with dedicated rate mode and 1-Gbps speed
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-10
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
48-Port 4-Gbps Fibre Channel Module BB_Credit Buffers
Table 4-6 lists the BB_credit buffer allocation for 48-port 4-Gbps Fibre Channel switching modules.
Table 4-6
48-Port 4-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate Mode
4-Gbps Speed
Shared Rate
Mode
4-Gbps Speed
BB_Credit Buffer Allocation
ISL1
Fx Port
Fx Port
Default BB_credit buffers
125
16
16
Maximum BB_credit buffers
250
250
16
Total number of BB_credit buffers per module
6000
1. ISL = E port or TE port.
The following considerations apply to BB_credit buffers on 48-port 4-Gbps Fibre Channel switching
modules:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode.
•
BB_credit buffers for Fx port mode connections can be configured. The minimum is 2 buffers and
the maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode.
•
Performance buffers are not supported on this module.
Each port group on the 48-port 4-Gbps Fibre Channel switching module consists of 12 ports. The ports
in shared rate mode have bandwidth oversubscription of 2:1 by default. However, some configurations
of the shared ports in a port group can have maximum bandwidth oversubscription of 4:1 (considering
that each port group has 12.8-Gbps bandwidth).
The following example configurations are supported by the 48-port 4-Gbps Fibre Channel switching
modules:
•
Twelve ports with shared rate mode and 4-Gbps speed (4:1 oversubscription) (default)
•
One port with dedicated rate mode and 4-Gbps speed plus
11 ports with shared rate mode and 4-Gbps speed (5:1 oversubscription)
•
One port with dedicated rate mode and 4-Gbps speed plus
11 ports with shared rate mode and 2-Gbps speed (2.5:1 oversubscription)
•
Two ports with dedicated rate mode and 2-Gbps speed plus
10 ports with shared rate mode and 4-Gbps speed (5:1 oversubscription)
•
Two ports with dedicated rate mode and 2-Gbps speed plus
10 ports with shared rate mode and 2-Gbps speed (2.5:1 oversubscription)
•
Twelve ports with dedicated rate mode and 1-Gbps speed
•
Three ports with dedicated rate mode and 4-Gbps speed plus
four ports with shared rate mode and 1-Gbps speed plus
five ports put out-of-service (see Figure 4-6)
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-11
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
•
1
3
5
7
9
11
4-Gbps
4-Gbps
1-Gbps
1-Gbps
Dedicated
Dedicated
Shared
Shared
Out of
Service
Out of
Service
2
4
6
8
10
12
4-Gbps
1-Gbps
1-Gbps
Dedicated
Shared
Shared
Out of
Service
Out of
Service
Out of
Service
Six ports with dedicated rate mode and 2-Gbps speed plus
four ports with shared rate mode and 1-Gbps speed plus
two ports put out-of-service (see Figure 4-7)
For detailed configuration steps of this example, see “Configuration Example for 48-Port 4-Gbps
Module Interfaces” section on page 2-50.
Figure 4-7
Example Speed and Rate Configuration on a 48-Port 4-Gbps Switching Module
1
3
5
7
9
2-Gbps
2-Gbps
2-Gbps
1-Gbps
1-Gbps
Dedicated
Dedicated
Dedicated
Shared
Shared
2
4
6
8
10
2-Gbps
2-Gbps
2-Gbps
1-Gbps
1-Gbps
Dedicated
Dedicated
Dedicated
Shared
Shared
11
Out of
Service
12
Out of
Service
144859
Note
Example Speed and Rate Configuration on a 48-Port 4-Gbps Switching Module
144858
Figure 4-6
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-12
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
24-Port 4-Gbps Fibre Channel Module BB_Credit Buffers
Table 4-7 lists the BB_credit buffer allocation for 24-port 4-Gbps Fibre Channel switching modules.
Table 4-7
24-Port 4-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate Mode
4-Gbps Speed
Shared Rate
Mode
4-Gbps Speed
BB_Credit Buffer Allocation
ISL1
Fx Port
Fx Port
Default BB_credit buffers
250
16
16
Maximum BB_credit buffers
250
250
16
Total number of BB_credits buffers per module
6000
1. ISL = E port or TE port.
The following considerations apply to BB_credit buffers on 24-port 4-Gbps Fibre Channel switching
modules:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode.
•
BB_credit buffers for Fx port mode connections can be configured. The minimum is 2 buffers and
the maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode.
•
Performance buffers are not supported on this module.
Each port group on the 24-port 4-Gbps Fibre Channel switching module consists of six ports. The ports
in shared rate mode have a bandwidth oversubscription of 2:1 by default. However, some configurations
of the shared ports in a port group can have a maximum bandwidth oversubscription of 4:1 (considering
that each port group has 12.8-Gbps bandwidth).
The following example configurations are supported by the 24-port 4-Gbps Fibre Channel switching
modules:
Note
•
Six ports with shared rate mode and 4-Gbps speed (2:1 oversubscription) (default)
•
Two ports with dedicated rate mode and 4-Gbps speed plus
four ports with shared rate mode and 4-Gbps speed (with 4:1 oversubscription)
•
One port with dedicated rate mode and 4-Gbps speed plus
three ports with dedicated rate mode and 2-Gbps speed plus
two ports with shared rate mode and 4-Gbps speed (4:1 oversubscription)
•
Six ports with dedicated rate mode and 2-Gbps speed
•
Three ports with dedicated rate mode and 4-Gbps speed plus
three ports with shared rate mode and 1-Gbps speed (see Figure 4-8)
For detailed configuration steps of this example, see the “Configuration Example for 24-Port 4-Gbps
Module Interfaces” section on page 2-51.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-13
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Example Speed and Rate Configuration on a 24-Port 4-Gbps Switching Module
1
2
3
4
5
6
4-Gbps
4-Gbps
4-Gbps
1-Gbps
1-Gbps
1-Gbps
Dedicated
Dedicated
Dedicated
Shared
Shared
Shared
144857
Figure 4-8
18-Port Fibre Channel/4-Port Gigabit Ethernet Multiservice Module BB_Credit Buffers
Table 4-8 lists the BB_credit buffer allocation for 18-port 4-Gbps multiservice modules.
Table 4-8
18-Port 4-Gbps Multiservice Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate
Mode
4-Gbps Speed
Shared Rate Mode
4-Gbps Speed
BB_Credit Buffer Allocation
ISL1
Fx Port
ISL1
Fx Port
Default BB_credit buffers
250
16
16
16
Maximum BB_credit buffers
250
250
16
16
Total number of BB_credit buffers per module
4509
1. ISL = E port or TE port.
The following considerations apply to BB_credit buffers on18-port 4-Gbps Fibre Channel switching
modules:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode.
•
BB_credit buffers for Fx port mode connections can be configured. The minimum is 2 buffers and
the maximum of 250 buffers for dedicated rate mode or 16 buffers for shared rate mode.
•
Performance buffers are not supported on this module.
12-Port 4-Gbps Switching Module BB_Credit Buffers
Table 4-9 lists the BB_credit buffer allocation for 12-port 4-Gbps switching modules.
Table 4-9
12-Port 4-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate Mode
4-Gbps Speed
BB_Credit Buffer Allocation Type
ISL 1
Fx Port
Default BB_credit buffers
250
16
Maximum BB_credit buffers
250
16
Default Performance buffers
145
12
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-14
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Table 4-9
12-Port 4-Gbps Switching Module BB_Credit Buffer Allocation (continued)
BB_Credit Buffers Per Port
Dedicated Rate Mode
4-Gbps Speed
BB_Credit Buffer Allocation Type
ISL 1
Total number of BB_credit buffers per module
5488
Total number of performance buffers per module
512 (shared)
Fx Port
1. ISL = E port or TE port.
The following considerations apply to BB_credit buffers on 12-port 4-Gbps switching modules:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 250 buffers.
•
BB_credit buffers for Fx port mode connections can be configured from a minimum of 2 buffers to
a maximum of 250 buffers.
•
By default, 512 performance buffers are preallocated and are shared by all the ports. These buffers
are configurable and the buffers are assigned to the port based on the availability of the buffers in
the shared pool.
•
There are 2488 extra buffers available as extended BB_credit buffers after allocating all the default
BB_credit buffers for all the ports in ISL mode (5488 - (250 * 12)).
Note
Note
Extended BB_credits are allocated across all ports on the switch. That is, they are not allocated
by port group.
By default, the ports in the 12-port 4-Gbps switching modules come up in 4-Gbps dedicated rate mode
but can be configured as 1-Gbps and 2-Gbps dedicated rate mode. Shared mode is not supported.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-15
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
4-Port 10-Gbps Switching Module BB_Credit Buffers
Table 4-10 lists the BB_credit buffer allocation for 4-port 10-Gbps switching modules.
Table 4-10
4-Port 10-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate Mode
10-Gbps Speed
BB_Credit Buffer Allocation Type
ISL1
F port2
Default BB_credit buffers
250
16
Maximum BB_credit buffers
750
16
Maximum BB_credit buffers on one of the ports with
Enterprise license
4095
Total number of BB_credit buffers per module
5488
Default Performance buffers
145
Total number of performance buffers per module
512 (shared)
12
1. ISL = E port or TE port.
2. Ports on the 4-port 10-Gbps cannot operate in FL port mode.
Note
The ports in the 4-port 10-Gbps switching module only support 10-Gbps dedicated rate mode. FL port
mode and shared rate mode are not supported.
The following considerations apply to BB_credit buffers on 4-port 10-Gbps switching modules:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 750 buffers.
•
BB_credit buffers for Fx port mode connections can be configured from a minimum of 2 buffers to
a maximum of 750 buffers.
•
By default, 512 performance buffers are preallocated and are shared by all the ports. These buffers
are configurable and the buffers are assigned to the port based on the availability of the buffers in
the shared pool.
•
There are 2488 extra buffers available as extended BB_credits after allocating all the default
BB_credit buffers for all the ports in ISL mode (5488 - (750 * 4)).
Note
Extended BB_credits are allocated across all ports on the switch. That is, they are not allocated
by port group.
BB_Credit Buffers for Fabric Switches
This section describes how buffer credits are allocated to Cisco MDS 9000 Fabric switches, and includes
the following topics:
•
Cisco MDS 9396s Fabric Switch BB_Credit Buffers, page 4-17
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-16
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
•
Cisco MDS 9250i and Cisco MDS 9148S Fabric Switch BB_Credit_Buffers, page 4-17
•
Cisco MDS 9148 Fabric Switch BB_Credit Buffers, page 4-18
•
Cisco MDS 9148 Fabric Switch BB_Credit Buffers, page 4-18
•
Cisco MDS 9124 Fabric Switch BB_Credit Buffers, page 4-19
•
Cisco MDS 9222i Multiservice Modular Switch BB_Credit Buffers, page 4-19
Cisco MDS 9396s Fabric Switch BB_Credit Buffers
Table 4-11 lists the BB_credit buffer allocation for the 96-port 16-Gbps Fabric switch.
.
Table 4-11
96-Port 16-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate Mode
16-Gbps Speed
Note
BB_Credit Buffer Allocation
ISL
Fx Port
Default BB_credit buffers
500
32
Maximum BB_credit buffers
500
500
Cisco MDS 9396s is a full-rate fabric switch.
The following guidelines apply to BB_credit buffers on the 96-port 16-Gbps Fabric switch:
•
BB_credit buffers for ISL connections can be configured from a minimum of 2 buffers to a
maximum of 500 buffers for a dedicated rate mode.
•
BB_credit buffers for Fx port mode connections can be configured from a minimum of 2 buffers to
a maximum of 500 buffers for a dedicated rate mode.
•
Per-port credits can be increased up to 4095 using extended buffer to buffer credits if the user has
installed an enterprise license.
•
The extended buffer feature can be activated by using the following commands:
Switch(config)#int fc1/5
switch(config-if)#switchport fcrxbbcredit extended 4095.
Note
In MDS 9396s Fabric Switches, total buffer available are 99600 for 24 port group. One port
group comprises of 4 ports, and there are 2 port groups per ASIC. Each port-group consists of
total 4150 buffers. These buffers can be allocated to any combination of port(s) using extended
buffer configuration. Please see show port-resource module module_number command for
details about buffers supported by port-groups.
Cisco MDS 9250i and Cisco MDS 9148S Fabric Switch BB_Credit_Buffers
Table 4-12lists the BB_credit buffer allocation for 40/48-port 16-Gbps Cisco MDS 9250i and 9148s
Fabric switches.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-17
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Table 4-12
40/48-Port 16-Gbps Switching Module BB_Credit Buffer Allocation
BB_Credit Buffers Per Port
Dedicated Rate Mode
16-Gbps Speed
Note
BB_Credit Buffer Allocation
ISL
Fx Port
Default BB_credit buffers
64
64
Maximum BB_credit buffers
252
252
Cisco MDS 9148s and Cisco MDS 9250i are full-rate fabric switches.
The following guidelines apply to BB_credit buffers on the 40/48-port 9250i/9148S Fabric switches:
•
BB_credit buffers can be configured from a minimum of 1 buffer to a maximum of 64 buffers per
port when the ports are in F or FL mode.
•
BB_credit buffers can be configured from a minimum of 2 buffers to a maximum of 64 buffers per
port when the ports are in E or TE mode.
•
BB_credit buffers for F or FL port can be configured for a single port in a port group from a
minimum of 1 buffer to a maximum of 252 buffers when all other ports in a port group are moved
to out of service.
•
BB_credit buffers for E or TE port can be configured for a single port in a port group from a
minimum of 2 buffer to a maximum of 252 buffers when all other ports in a port group are moved
to out of service.
Note
The ports that are moved to out-of-service need not be licensed.
Cisco MDS 9148 Fabric Switch BB_Credit Buffers
Table 4-13 lists the BB_credit buffer allocation for 48-port 8-Gbps Fabric switches.
Table 4-13
48-Port 8-Gbps Fabric Switch BB_Credit Buffer Allocation
BB_Credit Buffer Allocation Type
Default BB_credit buffers
BB_Credit
Buffers Per
Port Group
128
Maximum configurable BB_credit buffers on 8-Gbps 128
mode
BB_Credit Buffers Per Port
ISL1
Fx Port
32
32
125
125
1. ISL = E port or TE port.
The following considerations apply to BB_credit buffers on 48-port 8-Gbps Fabric switches:
•
BB_credit buffers can be configured from a minimum of 1 buffer to a maximum of 32 buffers per
port when the ports are in F or FL mode.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-18
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
•
BB_credit buffers can be configured from a minimum of 2 buffers to a maximum of 32 buffers per
port when the ports are in E or TE mode.
•
BB_credit buffers for F or FL port can be configured for a single port in a port group from a
minimum of 1 buffer to a maximum of 125 buffers when all other ports in a port group are moved
to out of service.
•
BB_credit buffers for E or TE port can be configured for a single port in a port group from a
minimum of 2 buffer to a maximum of 125 buffers when all other ports in a port group are moved
to out of service.
Note
The ports that are moved to out-of-service need not be licensed.
Cisco MDS 9134 Fabric Switch BB_Credit Buffers
Table 4-14 lists the BB_credit buffer allocation for MDS 9134 Fabric switches.
Table 4-14
MDS 9134 Fabric Switch BB_Credit Buffer Allocation
BB_Credit
Buffers Per
Port Group
BB_Credit Buffer Allocation Type
Maximum user-configurable BB_credit buffers
64
Minimum user-configurable BB_credit buffers
BB_Credit Buffers Per Port
ISL1
Fx Port
61
61
2
1
Default BB_credit buffers on 10-Gbps mode
64
64
64
Default BB_credit buffers on 4-Gbps mode
64
16
16
1. ISL = E port or TE port.
Cisco MDS 9124 Fabric Switch BB_Credit Buffers
Table 4-15 lists the BB_credit buffer allocation for MDS 9124 Fabric switches.
Table 4-15
MDS 9124 Fabric Switch BB_Credit Buffer Allocation Defaults
BB_Credit
Buffers Per
Port Group
BB_Credit Buffer Allocation Type
Maximum user-configurable BB_credit buffers
64
Minimum user-configurable BB_credit buffers
Default BB_credit buffers
64
BB_Credit Buffers Per Port
Defaults
ISL1
Fx Port
61
61
2
1
16
16
1. ISL = E port or TE port.
Cisco MDS 9222i Multiservice Modular Switch BB_Credit Buffers
Table 4-16 lists the BB_credit buffer allocation for 18-port 4-Gbps Multiservice Modular switches.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-19
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Table 4-16
18-Port 4-Gbps Fabric Switch BB_Credit Buffer Allocation Defaults
BB_Credit Buffer Allocation Type
User-configurable BB_credit buffers
BB_Credit
Buffers Per
Port Group
4509
BB_Credit Buffers Per Port
Defaults
ISL1
Fx Port
250
16
1. ISL = E port or TE port.
Extended BB_Credits
To facilitate BB_credits for long-haul links, the extended BB_credits feature allows you to configure the
receive buffers above the maximum value on all 4-Gbps, 8-Gbps, advanced 8-Gbps switching modules.
When necessary, you can reduce the buffers on one port and assign them to another port, exceeding the
default maximum. The minimum extended BB_credits per port is 256 and the maximum is 4095.
Note
Extended BB_credits are not supported on the Cisco MDS 9148 Fabric Switch, Cisco MDS 9134 Fabric
Switch, Cisco MDS 9124 Fabric Switch, the Cisco Fabric Switch for HP c-Class BladeSystem, and the
Cisco Fabric Switch for IBM BladeCenter.
In general, you can configure any port in a port group to dedicated rate mode. To do this, you must first
release the buffers from the other ports before configuring larger extended BB_credits for a port.
Note
The ENTERPRISE_PKG license is required to use extended BB_credits on 4-Gbps, 8-Gbps, and
advanced 8-Gbps switching modules. Also, extended BB_credits are not supported by ports in shared
rate mode.
All ports on the 4-Gbps, and 8-Gbps switching modules support extended BB_credits. There are no
limitations for how many extended BB_credits you can assign to a port (except for the maximum and
minimum limits). If necessary, you can take interfaces out of service to make more extended BB_credits
available to other ports.
You can use the extended BB_credits flow control mechanism in addition to BB_credits for long-haul
links.
This section includes the following topics:
•
Extended BB_credits on Generation 1 Switching Modules, page 4-20
•
Extended BB_credits on 4-Gbps and 8-Gbps Switching Modules, page 4-21
Extended BB_credits on Generation 1 Switching Modules
The BB_credits feature allows you to configure up to 255 receive buffers on Generation 1 switching
modules. To facilitate BB_credits for long haul links, you can configure up to 3,500 receive BB_credits
on a Fibre Channel port on a Generation 1 switching module.
To use this feature on Generation 1 switching modules, you must meet the following requirements:
•
Obtain the ENTERPRISE_PKG license. See the Cisco MDS 9000 Family NX-OS Licensing Guide.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-20
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Configure this feature in any port of the full-rate 4-port group in either the Cisco MDS 9216i Switch
or in the MPS-14/2 module (see Figure 4-9).
Figure 4-9
Port Group Support for the Extended BB_Credits Feature
1
LINK-
1
2
3
4
5
6
7
8
9
10
11
12
13
2
LINK-
14
GIGABIT E THERNET
STATUS
LINK—
—SPEED
Group 1
LINK—
—SPEED
Group 2
LINK—
—SPEED
Group 3
Extended credits
not supported
120479
•
The port groups that support extended credit configurations are as follows:
– Any one port in ports 1 to 4 (identified as Group 1).
– Any one port in ports 5 to 8 (identified as Group 2).
– Any one port in ports 9 to 12 (identified as Group 3).
Note
The last two Fibre Channel ports (port 13 and port 14) and the two Gigabit Ethernet ports
do not support the extended BB_credits feature.
•
Explicitly enable this feature in the required Cisco MDS switch.
•
Disable the remaining three ports in the 4-port group if you need to assign more than 2,400
BB_credits to the first port in the port group.
– If you assign less than 2,400 extended BB_credits to any one port in a port group, the remaining
three ports in that port group can retain up to 255 BB_credits based on the port mode.
Note
The receive BB_credit value for the remaining three ports depends on the port mode.
The default value is 16 for the Fx mode and 255 for E or TE modes. The maximum value
is 255 in all modes. This value can be changed as required without exceeding the
maximum value of 255 BB_credits.
– If you assign more than 2,400 (up to a maximum of 3,500) extended BB_credits to the port in
a port group, you must disable the other three ports.
•
If you change the BB_credit value the port is disabled, and then reenabled.
– Disable (explicitly) this feature if you need to nondisruptively downgrade to Cisco SAN-OS
Release 1.3 or earlier. When you disable this feature, the existing extended BB_credit
configuration is completely erased.
Note
The extended BB_credit configuration takes precedence over the receive BB_credit and performance
buffer configurations.
Extended BB_credits on 4-Gbps and 8-Gbps Switching Modules
To use this feature on 4-Gbps or, 8-Gbps switching modules, you must meet the following requirements:
•
Display the interface configuration in the Information pane.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-21
Chapter 4
Configuring Interface Buffers
Information About Interface Buffers
Note
•
Obtain the Enterprise package (ENTERPRISE_PKG) license (see the NX-OS Family Licensing
Guide).
•
Configure this feature in any port on a 4-Gbps switch module. See the “Extended BB_Credits”
section on page 4-20 for more information on extended BB_credits on 2-Gbps switching modules.
Extended BB_credits are not supported on the Cisco MDS 9124 Fabric Switch, Cisco MDS 9134 Fabric
Switch, the Cisco Fabric Switch for HP c-Class BladeSystem, and the Cisco Fabric Switch for IBM
BladeCenter.
Buffer-to-Buffer Credit Recovery
Although the Fibre Channel standards require low bit error rates, bit errors do occur. Over time, the
corruption of receiver-ready messages, known as R_RDY primitives, can lead to a loss of credits, which
can eventually cause a link to stop transmitting in one direction. The Fibre Channel standards provide a
feature for two attached ports to detect and correct this situation. This feature is called buffer-to-buffer
credit recovery.
Buffer-to-buffer credit recovery functions as follows: the sender and the receiver agree to send
checkpoint primitives to each other, starting from the time that the link comes up. The sender sends a
checkpoint every time it has sent the specified number of frames, and the receiver sends a checkpoint
every time it has sent the specified number of R_RDY primitives. If the receiver detects lost credits, it
can retransmit them and restore the credit count on the sender.
The buffer-to-buffer credit recovery feature can be used on any non arbitrated loop link. This feature is
most useful on unreliable links, such as MANs or WANs, but can also help on shorter, high-loss links,
such as a link with a faulty fiber connection.
Note
The buffer-to-buffer credit recovery feature is not compatible with the distance extension (DE) feature,
also known as buffer-to-buffer credit spoofing. If you use intermediate optical equipment, such as
DWDM transceivers or Fibre Channel bridges, on ISLs between switches that use DE, then
buffer-to-buffer credit recovery on both sides of the ISL needs to be disabled.
Buffer-to-Buffer State Change Number
The BB_SC_N field (word 1, bits 15-12) specifies the buffer-to-buffer state change (BB_SC) number.
The BB_SC_N field indicates that the sender of the port login (PLOGI), fabric login (FLOGI), or ISLs
(E or TE ports) frame is requesting 2^SC_BB_N number of frames to be sent between two consecutive
BB_SC send primitives, and twice the number of R_RDY primitives to be sent between two consecutive
BB_SC receive primitives.
For 4-Gbps and 8-Gbps modules, the BB_SCN on ISLs (E or TE ports) is enabled by default. This can
fail the ISLs if used with optical equipment using distance extension (DE), also known as
buffer-to-buffer credit spoofing.
On a 4-Gbps module, one port will not come up for the following configuration for all ports:
•
Port Mode: auto or E for all the ports
•
Rate Mode: dedicated
•
Buffer Credits: default value
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-22
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Configuring Interface Buffers
On a 8-Gbps module, one or two ports will not come up for the following configuration for the first half
of the ports, the second half of the ports, or all ports:
•
Port Mode: auto or E for the first half of the ports, the second half of the ports, or for all of the ports
•
Rate Mode: dedicated
•
Buffer Credits: default value
When you configure port mode to auto or E and rate-mode to dedicated for all ports in the global buffer
pool, you need to reconfigure buffer credits on one or more ports (other than default).
Note
If you use distance extension (buffer-to-buffer credit spoofing) on ISLs between switches, the BB_SCN
parameter on both sides of the ISL needs to be disabled.
Receive Data Field Size
You can also configure the receive data field size for Fibre Channel interfaces. If the default data field
size is 2112 bytes, the frame length will be 2148 bytes.
Configuring Interface Buffers
This section includes the following topics:
•
Configuring Buffer-to-Buffer Credits, page 4-23
•
Configuring Performance Buffers, page 4-24
•
Configuring Extended BB_credits, page 4-25
•
Enabling Buffer-to-Buffer Credit Recovery, page 4-25
•
Enabling the Buffer-to-Buffer State Change Number, page 4-26
•
Configuring Receive Data Field Size, page 4-26
Configuring Buffer-to-Buffer Credits
Detailed Steps
To configure BB_credits for a Fibre Channel interface, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc1/1
switch(config-if)#
Selects a Fibre Channel interface and enters
interface configuration submode.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-23
Chapter 4
Configuring Interface Buffers
Configuring Interface Buffers
Step 3
Step 4
Command
Purpose
switch(config-if)# switchport fcrxbbcredit
default
Applies the default operational value to the
selected interface. The operational value depends
on the port mode. The default values are assigned
based on the port capabilities.
switch(config-if)# switchport fcrxbbcredit 5
Assigns a BB_credit of 5 to the selected interface.
The range to assign BB_credits is between 1 and
255.
switch(config-if)# switchport fcrxbbcredit 5
mode E
Assigns this value if the port is operating in E or
TE mode. The range to assign BB_credits is
between 1 and 255.
switch(config-if)# switchport fcrxbbcredit 5
mode Fx
Assigns this value if the port is operating in F or
FL mode. The range to assign BB_credits is
between 1 and 255.
switch(config-if# do show int fc1/1
Displays the receive and transmit BB_credit
along with other pertinent interface information
for this interface.
Note
The BB_credit values are correct at the
time the registers are read. They are
useful to verify situations when the data
traffic is slow.
This example shows the output of the show int fc1/1 command:
intfc1/1 is up
...
16 receive B2B credit remaining
3 transmit B2B credit remaining
Configuring Performance Buffers
Detailed Steps
To configure performance buffers for a Fibre Channel interface, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# interface fc1/1
switch(config-if)#
Selects a Fibre Channel interface and enters interface
configuration submode.
Step 3
switch(config-if)# switchport
fcrxbbcredit performance-buffers 45
Assigns a performance buffer of 45 to the selected
interface. The value ranges from 1 to 145.
switch(config-if)# switchport
fcrxbbcredit performance-buffers default
Reverts to the factory default of using the built-in
algorithm.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-24
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Configuring Interface Buffers
Note
Use the show interface bbcredit command to display performance buffer values and other BB_credit
information.
Configuring Extended BB_credits
Detailed Steps
To configure extended BB_credits for a MDS-14/2 interface, for a 4-Gbps switching module interface
(not including the Cisco MDS 9124 Fabric Switch), or for an interface in a Cisco MDS 9216i switch,
follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch (config)# feature fcrxbbcredit
extended
Enables a feature or service on the switch.
switch (config)# no feature fcrxbbcredit
extended
Disables a feature or service on the switch.
switch(config)# fcrxbbcredit extended enable
Enables the extended BB_credits feature.
switch(config)# no fcrxbbcredit extended
enable
Disables (default) the extended BB_credits
feature.
Step 4
switch(config)# interface fc1/1
switch(config-if)#
Selects a Fibre Channel interface and enters
interface configuration submode.
Step 5
switch(config-if)# switchport fcrxbbcredit
extended 1500
Applies the extended BB_credit value of 1,500
credits to the selected interface. The valid range is
from 256 to 3,500 credits.
switch(config-if)# no switchport
fcrxbbcredit extended 1500
Clears the configured extended BB_credit
configuration for this port.
Step 3
Step 6
switch# do show interface fc3/2
Displays the receive and transmit BB_credit
fc3/2 is trunking
values along with other pertinent interface
Hardware is Fibre Channel, SFP is short wave
information for this interface if the interface is in
laser w/o OFC (SN)
the up state.
Port WWN is 20:82:00:05:30:00:2a:1e
Peer port WWN is 20:42:00:0b:46:79:f1:80
Note
The receive BB_credit value reflects the
Admin port mode is auto, trunk mode is
extended BB_credit configuration, if
on
applicable.
Port mode is TE
Port vsan is 1
Speed is 2 Gbps
Transmit B2B Credit is 255
Receive B2B Credit is 1500
Receive data field Size is 2112
...
Enabling Buffer-to-Buffer Credit Recovery
Buffer-to-buffer credit recovery on ISLs (E or TE ports) is enabled by default.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-25
Chapter 4
Configuring Interface Buffers
Configuring Interface Buffers
Detailed Steps
To use buffer-to-buffer credit recovery on a port, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc 1/1
switch(config-if)#
Selects the interface and enters interface
configuration submode.
Step 3
switch(config-if)# switchport fcbbscn
Enables buffer-to-buffer credit recovery on the
interface.
switch(config-if)# no switchport fcbbscn
Disables (default) buffer-to-buffer credit recovery
on the interface.
Enabling the Buffer-to-Buffer State Change Number
Detailed Steps
To use the BB_SC_N field during PLOGI or FLOGI, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc 1/1
switch(config-if)#
Selects the interface and enters interface
configuration submode.
Step 3
switch(config-if)# switchport fcbbscn
Enables the use of buffer-to-buffer state change
number for PLOGIs and FLOGIs on the interface.
switch(config-if)# no switchport fcbbscn
Disables (default) the use of buffer-to-buffer state
change number for PLOGIs and FLOGIs on the
interface.
Configuring Receive Data Field Size
Detailed Steps
To configure the receive data field size, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# interface fc1/1
switch(config-if)#
Selects a Fibre Channel interface and enters
interface configuration submode.
Step 3
switch(config-if)# switchport fcrxbufsize 2000
Reduces the data field size for the selected
interface to 2000 bytes. The default is 2112
bytes and the range is from 256 to 2112 bytes.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-26
OL-29284-01, Release 6.x
Chapter 4
Configuring Interface Buffers
Verifying BB_Credit Configuration
Verifying BB_Credit Configuration
To display BB_credit configuration information, perform one of the following tasks:
Command
Purpose
show interface
Displays the interface configuration.
show interface bbcredit
Displays the BB_credit configuration for all the
interfaces.
show interface fc numbers bbcredit
Displays the BB_credit configuration for the
specified interfaces.
For detailed information about the fields in the output from these commands, refer to the Cisco NX-OS
Command Reference.
To display the BB_credit information, use the show interface bbcredit command (see Example 4-1 and
Example 4-2).
Example 4-1
Displays BB_credit Information
switch# show interface bbcredit
fc2/1 is down (SFP not present)
...
fc2/17 is trunking
Transmit B2B Credit is 255
Receive B2B Credit is 12
Receive B2B Credit performance buffers is 375
12 receive B2B credit remaining
255 transmit B2B credit remaining
fc2/18 is down (SFP not present)
fc2/19 is down (SFP not present)
fc2/20 is down (SFP not present)
fc2/21 is down (Link failure or not-connected)
...
fc2/31 is up
Transmit B2B Credit is 0
Receive B2B Credit is 12
Receive B2B Credit performance buffers is 48
12 receive B2B credit remaining
0 transmit B2B credit remaining
fc2/32 is down (Link failure or not-connected)
Example 4-2
Displays BB_credit Information for a Specified Fibre Channel Interface
switch# show interface fc2/31 bbcredit
fc2/31 is up
Transmit B2B Credit is 0
Receive B2B Credit is 12
Receive B2B Credit performance buffers is 48
12 receive B2B credit remaining
0 transmit B2B credit remaining
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
4-27
Chapter 4
Configuring Interface Buffers
Verifying BB_Credit Configuration
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
4-28
OL-29284-01, Release 6.x
CHAPTER
5
Configuring Trunking
•
Information About Trunking, page 5-1
•
Guidelines and Limitations, page 5-7
•
Default Settings, page 5-11
•
Configuring Trunking, page 5-11
•
Verifying Trunking Configuration, page 5-13
•
Configuration Example for F Port Trunking, page 5-14
Information About Trunking
Trunking, also known as VSAN trunking, is a feature specific to switches in the Cisco MDS 9000
Family. Trunking enables interconnect ports to transmit and receive frames in more than one VSAN, over
the same physical link. Trunking is supported on E ports and F ports (See Figure 5-1 and Figure 5-2).
This section includes the following topics:
•
Trunking E Ports, page 5-2
•
Trunking F Ports, page 5-2
•
Key Concepts, page 5-3
•
Trunking Protocols, page 5-4
•
Trunk Modes, page 5-5
•
Trunk-Allowed VSAN Lists and VF_IDs, page 5-5
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5-1
Chapter 5
Configuring Trunking
Information About Trunking
Trunking E Ports
Trunking the E ports enables interconnect ports to transmit and receive frames in more than one VSAN,
over the same physical link, using enhanced ISL (EISL) frame format.
Trunking E Ports
Any other
switch
Switch 1
ISL
E port
Switch 1
Switch 2
EISL
E port
TE port
TE port
79938
Figure 5-1
Trunking
Note
Trunking is not supported by internal ports on both the Cisco Fabric Switch for HP c_Class BladeSystem
and the Cisco Fabric Switch for IBM BladeCenter.
Trunking F Ports
Trunking F ports allows interconnected ports to transmit and receive tagged frames in more than one
VSAN, over the same physical link.
Figure 5-2 represents the possible trunking scenarios in a SAN with MDS core switches, NPV switches,
third-party core switches, and HBAs.
Figure 5-2
Trunking F Ports
MDS Core
Switch
3rd party Core
Switch
TF
F
TF
EPP
NP
TN
2
4
EPP
EVFP
1a
TF
3
5
EVFP
HB A
EPP
TNP
TNP
NPV Switch
TNP
F
TF
1b
N
TN
HB A
HB A
192090
EVFP
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
5-2
OL-29284-01, Release 6.x
Chapter 5
Configuring Trunking
Information About Trunking
Link Number
Link Description
1a and 1b
F port trunk with N port.1
2
F port trunk with NP port.
3
F PortChannnel with NP port.
4
Trunked F PortChannel with NP port.
5
Trunking NP port with third-party core switch F port.1
1. These features are not supported currently.
Key Concepts
The trunking feature includes the following key concepts:
•
TE port—If trunk mode is enabled in an E port and that port becomes operational as a trunking E
port, it is referred to as a TE port.
•
TF port—If trunk mode is enabled in an F port (see the link 2 in Figure 5-2) and that port becomes
operational as a trunking F port, it is referred to as a TF port.
•
TN port—If trunk mode is enabled (not currently supported) in an N port (see the link 1b in
Figure 5-2) and that port becomes operational as a trunking N port, it is referred to as a TN port.
•
TNP port—If trunk mode is enabled in an NP port (see the link 2 in Figure 5-2) and that port
becomes operational as a trunking NP port, it is referred to as a TNP port.
•
TF PortChannel—If trunk mode is enabled in an F PortChannel (see the link 4 in Figure 5-2) and
that PortChannel becomes operational as a trunking F PortChannel, it is referred to as TF
PortChannel. Cisco Port Trunking Protocol (PTP) is used to carry tagged frames.
•
TF-TN port link—A single link can be established to connect an F port to an HBA to carry tagged
frames (see the link 1a and 1b in Figure 5-2) using Exchange Virtual Fabrics Protocol (EVFP). A
server can reach multiple VSANs through a TF port without inter-VSAN routing (IVR).
•
TF-TNP port link—A single link can be established to connect an TF port to an TNP port using the
PTP protocol to carry tagged frames (see the link 2 in Figure 5-2). PTP is used because PTP also
supports trunking PortChannels.
Note
•
The TF-TNP port link between a third-party NPV core and a Cisco NPV switch is
established using the EVFP protocol.
A Fibre Channel VSAN is called Virtual Fabric and uses a VF_ID in place of the VSAN ID. By
default, the VF_ID is 1 for all ports. When an N port supports trunking, a pWWN is defined for each
VSAN and called a logical pWWN. In the case of MDS core switches, the pWWNs for which the N
port requests additional FC_IDs are called virtual pWWNs.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5-3
Chapter 5
Configuring Trunking
Information About Trunking
Trunking Protocols
The trunking protocol is important for trunking operations on the ports. The protocols enable the
following activities:
•
Dynamic negotiation of operational trunk mode.
•
Selection of a common set of trunk-allowed VSANs.
•
Detection of a VSAN mismatch across an ISL.
Table 5-1 specifies the protocols used for trunking and channeling.
Table 5-1
Supported Trunking Protocols
Trunk Link
Default
TE-TE port link
TF-TN port link
Cisco EPP (PTP)
1
FC-LS Rev 1.62 EVFP
TF-TNP port link
Cisco EPP (PTP)
E or F PortChannel
Cisco EPP (PCP)
TF Port Channel
Third-party TF-TNP port link
Cisco EPP (PTP and PCP)
1
FC-LS Rev 1.62 EVFP
1. These features are not currently supported.
By default, the trunking protocol is enabled on E ports and disabled on F ports. If the trunking protocol
is disabled on a switch, no port on that switch can apply new trunk configurations. Existing trunk
configurations are not affected. The TE port continues to function in trunk mode, but only supports
traffic in VSANs that it negotiated with previously (when the trunking protocol was enabled). Also, other
switches that are directly connected to this switch are similarly affected on the connected interfaces. In
some cases, you may need to merge traffic from different port VSANs across a non-trunking ISL. If so,
disable the trunking protocol.
Note
We recommend that both ends of a trunking link belong to the same port VSAN. On certain switches or
fabric switches where the port VSANs are different, one end returns an error and the other end is not
connected.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
5-4
OL-29284-01, Release 6.x
Chapter 5
Configuring Trunking
Information About Trunking
Trunk Modes
By default, trunk mode is enabled on all Fibre Channel interfaces (Mode: E, F, FL, Fx, ST, and SD) on
non-NPV switches. On NPV switches, by default, trunk mode is disabled. You can configure trunk mode
as on (enabled), off (disabled), or auto (automatic). The trunk mode configuration at the two ends of an
ISL, between two switches, determine the trunking state of the link and the port modes at both ends (see
Table 5-2).
Table 5-2
Trunk Mode Status Between Switches
Your Trunk Mode Configuration
Resulting State and Port Mode
Port Type
Switch 1
Switch 2
Trunking State
Port Mode
E ports
On
Auto or on
Trunking (EISL)
TE port
Off
Auto, on, or off
No trunking (ISL)
E port
Auto
Auto
No trunking (ISL)
E port
Port Type
Core Switch NPV Switch
Trunking State
Link Mode
F and NP
ports
On
Auto or on
Trunking
TF-TNP link
Auto
On
Trunking
TF-TNP link
Off
Auto, on, or off
No trunking
F-NP link
Tip
The preferred configuration on the Cisco MDS 9000 Family switches is one side of the trunk set to auto
and the other side set to on.
Note
When connected to a third-party switch, the trunk mode configuration on E ports has no effect. The ISL
is always in a trunking disabled state. In the case of F ports, if the third-party core switch ACC's physical
FLOGI with the EVFP bit is configured, then EVFP protocol enables trunking on the link.
Trunk-Allowed VSAN Lists and VF_IDs
Each Fibre Channel interface has an associated trunk-allowed VSAN list. In TE-port mode, frames are
transmitted and received in one or more VSANs specified in this list. By default, the VSAN range (1
through 4093) is included in the trunk-allowed list.
The common set of VSANs that are configured and active in the switch are included in the trunk-allowed
VSAN list for an interface, and they are called allowed-active VSANs. The trunking protocol uses the
list of allowed-active VSANs at the two ends of an ISL to determine the list of operational VSANs in
which traffic is allowed.
Switch 1 (see Figure 5-3) has VSANs 1 through 5, switch 2 has VSANs 1 through 3, and switch 3 has
VSANs 1, 2, 4, and 5 with a default configuration of trunk-allowed VSANs. All VSANs configured in
all three switches are allowed-active. However, only the common set of allowed-active VSANs at the
ends of the ISL become operational (see Figure 5-3).
For all F, N, and NP ports, the default VF_ID is 1 when there is no VF_ID configured. The trunk-allowed
VF_ID list on a port is same as the list of trunk-allowed VSANs. VF_ID 4094 is called the control VF_ID
and it is used to define the list of trunk-allowed VF-IDs when trunking is enabled on the link.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5-5
Chapter 5
Configuring Trunking
Information About Trunking
If F port trunking and channeling is enabled, or if switchport trunk mode on is configured in NPV mode
for any interface, or if NP PortChannel is configured, the VSAN and VF-ID ranges available for the
configuration are as described in Table 5-3.
Table 5-3
VSAN or VF-ID
Description
000h
Cannot be used as virtual fabric identifier.
001h(1) to EFFh(3839)
This VSAN range is available for user configuration.
F00h(3840) to FEEh(4078)
Reserved VSANs and they are not available for user
configuration.
FEFh(4079)
EVFP isolated VSAN.
FF0h(4080) to FFEh(4094)
Used for vendor-specific VSANs.
FFFh
Cannot be used as virtual fabric identifier.
If the VF_ID of the F port and the N port do not match, then no tagged frames can be exchanged.
AN
Switch 1 VS
VSAN1
VSAN2
VSAN3
VSAN4
VSAN5 VS
AN
s
Default Allowed-Active VSAN Configuration
s1 ,
nd
2, a
1, 2
, 4,
re
3a
5a
.
nal
atio
r
e
op
re o
per
atio
nal
.
Switch 2
VSAN1
VSAN2
VSAN3
Switch 3
VSAN1
VSAN2
VSAN4
VSAN5
79945
Figure 5-3
VSANs 1 and 2 are operational.
Note
VSAN and VF-ID Reservations
You can configure a select set of VSANs (from the allowed-active list) to control access to the VSANs
specified in a trunking ISL.
Using Figure 5-3 as an example, you can configure the list of allowed VSANs on a per-interface basis
(see Figure 5-4). For example, if VSANs 2 and 4 are removed from the allowed VSAN list of ISLs
connecting to switch 1, the operational allowed list of VSANs for each ISL would be as follows:
•
The ISL between switch 1 and switch 2 includes VSAN 1 and VSAN 3.
•
The ISL between switch 2 and switch 3 includes VSAN 1 and VSAN 2.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
5-6
OL-29284-01, Release 6.x
Chapter 5
Configuring Trunking
Guidelines and Limitations
•
The ISL between switch 3 and switch 1 includes VSAN 1, 2, and 5.
Consequently, VSAN 2 can only be routed from switch 1 through switch 3 to switch 2.
Operational and Allowed VSAN Configuration
Switch 2
VSAN1
VSAN2
VSAN3
Switch 1
VSAN1
VSAN2
VSAN3
VSAN4
VSAN5
VS
VS
AN ANs 1
s1
, 2, , 2, 5 a
5a
re o re ope
ra
n th
e a tional
llow
.
ed
list.
Switch 3
VSAN1
VSAN2
VSAN4
VSAN5
79946
list.
ed
w
allo
l.
the ationa
n
o
r
e
e
p
r
3a
re o
and nd 3 a
1
s
a
AN Ns 1
VS
A
S
V
VSANs 1 and 2 are operational.
VSANs 1 and 2 are on the allowed list.
Figure 5-4
Guidelines and Limitations
Trunking has the following configuration guidelines and limitations:
•
General Guidelines and Limitations, page 5-7
•
Upgrade and Downgrade Limitations, page 5-8
•
Difference Between TE Ports and TF-TNP Ports, page 5-8
•
Trunking Misconfiguration Examples, page 5-10
General Guidelines and Limitations
The trunking feature has the following general configuration guidelines and limitations:
•
F ports support trunking in Fx mode.
•
The trunk-allowed VSANs configured for TE, TF, and TNP links are used by the trunking protocol
to determine the allowed active VSANs in which frames can be received or transmitted.
•
If a trunking enabled E port is connected to a third-party switch, the trunking protocol ensures
seamless operation as an E port.
•
Trunking F ports and trunking F PortChannels are not supported on the following hardware:
– 91x4 switches, if NPIV is enabled and used as the NPIV core switch.
– Generation 1 2-Gbps Fibre Channel switching modules.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5-7
Chapter 5
Configuring Trunking
Guidelines and Limitations
Note
•
On core switches, the FC-SP authentication will be supported only for the physical FLOGI from the
physical pWWN.
•
No FC-SP authentication is supported by the NPV switch on the server F ports.
•
MDS does not enforce the uniqueness of logical pWWNs across VSANs.
•
DPVM is not supported on trunked F port logins.
•
The DPVM feature is limited to the control of the port VSAN, since the EVFP protocol does not
allow changing the VSAN on which a logical pWWN has done FLOGI.
•
The port security configuration will be applied to both the first physical FLOGI and the per VSAN
FLOGIs.
•
Trunking is not supported on F ports that have FlexAttach enabled.
•
On MDS 91x4 core switches, hard zoning can be done only on F ports that are doing either NPIV
or trunking. However, in NPV mode, this restriction does not apply since zoning is enforced on the
core F port.
Fibre Channel Security Protocol (FC-SP) is not supported for 6.2(1) release on MDS 9710, but targeted
for a future release.
Upgrade and Downgrade Limitations
The trunking and channeling feature includes the following upgrade and downgrade limitations:
•
When F port trunking or channeling is configured on a link, the switch cannot be downgraded to
Cisco MDS SAN-OS Release 3.x and NX-OS Release 4.1(1b), or earlier.
•
If you are upgrading from a SAN-OS Release 3.x to NX-OS Release 5.0(1), and you have not created
VSAN 4079, the NX-OS software will automatically create VSAN 4079 and reserve it for EVFP
use.
If VSAN 4079 is reserved for EVFP use, the switchport trunk allowed vsan command will filter
out VSAN 4079 from the allowed list, as shown in the following example:
switch(config-if)# switchport trunk allowed vsan 1-4080
1-4078,4080
switch(config-if)#
– If you have created VSAN 4079, the upgrade to NX-OS Release 5.0(1) will have no affect on
VSAN 4079.
– If you downgrade after NX-OS Release 5.0(1), the VSAN will no longer be reserved for EVFP
use.
Difference Between TE Ports and TF-TNP Ports
In case of TE ports, the VSAN will in be initializing state when VSAN is coming up on that interface
and when peers are in negotiating phase. Once the handshake is done, VSAN will be moved to up state
in the successful case, and isolated state in the case of failure. Device Manager will show the port status
as amber during initializing state and it will be green once VSANs are up.
This example shows the trunk VSAN states of a TE port:
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
5-8
OL-29284-01, Release 6.x
Chapter 5
Configuring Trunking
Guidelines and Limitations
Switch# show interface fc2/15
fc2/15 is trunking
Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
Port WWN is 20:4f:00:0d:ec:6d:2b:40
Peer port WWN is 20:0a:00:0d:ec:3f:ab:80
Admin port mode is auto, trunk mode is on
snmp link state traps are enabled
Port mode is TE
Port vsan is 1
Speed is 2 Gbps
Rate mode is dedicated
Transmit B2B Credit is 16
Receive B2B Credit is 250
B2B State Change Number is 14
Receive data field Size is 2112
Beacon is turned off
Trunk vsans (admin allowed and active) (1,100-101,1101,1163-1166,1216,2172,2182-2183)
Trunk vsans (up)
(1,1101,1163-1166,1216,2172,2182-2183)
Trunk vsans (isolated)
(100-101)
Trunk vsans (initializing)
()
In case of TF ports, after the handshake, one of the allowed VSANs will be moved to the up state. All
other VSANs will be in initializing state even though the handshake with the peer is completed and
successful. Each VSAN will be moved from initializing state to up state when a server or target logs in
through the trunked F or NP ports in the corresponding VSAN.
Note
In case of TF or TNP ports, the Device Manager will show the port status as amber even after port is up
and there is no failure. It will be changed to green once all the VSAN has successful logins.
This example shows a TF port information after the port is in the up state:
sw7# show interface fc1/13
fc1/13 is trunking (Not all VSANs UP on the trunk)
Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
Port WWN is 20:0d:00:0d:ec:6d:2b:40
Admin port mode is FX, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 1
Speed is 4 Gbps
Rate mode is shared
Transmit B2B Credit is 16
Receive B2B Credit is 32
Receive data field Size is 2112
Beacon is turned off
Trunk vsans (admin allowed and active) (1,100-101,1101,1163-1166,1216,2172,2182-2183)
Trunk vsans (up)
(1)
Trunk vsans (isolated)
()
Trunk vsans (initializing)
(1101,1163-1166,1216,2172,2182)
This example shows the TF port information when a server logs in on noninternal FLOGI VSAN. VSAN
2183 is moved to the up state when the server logs in to VSAN 2183.
w7# show interface fc1/13
fc1/13 is trunking (Not all VSANs UP on the trunk)
Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
Port WWN is 20:0d:00:0d:ec:6d:2b:40
Admin port mode is FX, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 1
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5-9
Chapter 5
Configuring Trunking
Guidelines and Limitations
Speed is 4 Gbps
Rate mode is shared
Transmit B2B Credit is 16
Receive B2B Credit is 32
Receive data field Size is 2112
Beacon is turned off
Trunk vsans (admin allowed and active) (1,100-101,1101,1163-1166,1216,2172,2
182-2183)
Trunk vsans (up)
(1,2183)
Trunk vsans (isolated)
()
Trunk vsans (initializing)
(1101,1163-1166,1216,2172,2182)
Trunking Misconfiguration Examples
If you do not configure the VSANs correctly, issues with the connection may occur. For example, if you
merge the traffic in two VSANs, both VSANs will be mismatched. The trunking protocol validates the
VSAN interfaces at both ends of a link to avoid merging VSANs (see Figure 5-5).
Figure 5-5
VSAN Mismatch
Switch 1
Switch 2
Isolated
E port
VSAN 3
85471
E port
VSAN 2
VSAN mismatch
The trunking protocol detects potential VSAN merging and isolates the ports involved (see Figure 5-5).
The trunking protocol cannot detect merging of VSANs when a third-party switch is placed in between
two Cisco MDS 9000 Family switches (see Figure 5-6).
Figure 5-6
Third-Party Switch VSAN Mismatch
Switch 1
VSAN 3
E port
Switch 2
Switch 3
85472
Third-party switches
VSAN 2
E port
VSAN 2 and VSAN 3 are effectively merged with overlapping entries in the name server and the zone
applications. Cisco DCNM-SAN helps detect such topologies.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
5-10
OL-29284-01, Release 6.x
Chapter 5
Configuring Trunking
Default Settings
Default Settings
Table 5-4 lists the default settings for trunking parameters.
Table 5-4
Default Trunk Configuration Parameters
Parameters
Default
Switch port trunk mode
ON on non-NPV and MDS core switches.
OFF on NPV switches.
Allowed VSAN list
1 to 4093 user-defined VSAN IDs.
Allowed VF-ID list
1 to 4093 user-defined VF-IDs.
Trunking protocol on E ports
Enabled.
Trunking protocol on F ports
Disabled.
Configuring Trunking
This section includes the following topics:
•
Enabling the Cisco Trunking and Channeling Protocols, page 5-11
•
Enabling the F Port Trunking and Channeling Protocol, page 5-12
•
Configuring Trunk Mode, page 5-12
•
Configuring an Allowed-Active List of VSANs, page 5-12
Enabling the Cisco Trunking and Channeling Protocols
This section describes how to enable the required trunking and channeling protocols.
Prerequisites
•
To avoid inconsistent configurations, disable all ports with a shutdown command before enabling
or disabling the trunking protocols.
Detailed Steps
To enable or disable the Cisco trunking and channeling protocol, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# trunk protocol enable
switch(config)#
Enables the Cisco PTP trunking protocol
(default).
switch(config)# no trunk protocol enable
switch(config)#
Disables the Cisco PTP trunking protocol.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5-11
Chapter 5
Configuring Trunking
Configuring Trunking
Enabling the F Port Trunking and Channeling Protocol
This section describes how to enable the F port trunking and channeling protocol.
Prerequisites
•
To avoid inconsistent configurations, shut all ports before enabling or disabling the trunking
protocols.
Detailed Steps
To enable or disable the F port trunking and channeling protocol, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# feature fport-channel-trunk
switch(config)#
Enables the F port trunking and channeling
protocol (default).
switch(config)# no feature fport-channel-trunk
switch(config)#
Disables the F port trunking and channeling
protocol.
Configuring Trunk Mode
Detailed Steps
To configure trunk mode, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# interface fc1/1
switch(config-if)#
Configures the specified interface.
Step 3
switch(config-if)# switchport trunk mode on
Enables (default) the trunk mode for the
specified interface.
switch(config-if)# switchport trunk mode off
Disables the trunk mode for the specified
interface.
switch(config-if)# switchport trunk mode auto
Configures the trunk mode to auto mode, which
provides automatic sensing for the interface.
Configuring an Allowed-Active List of VSANs
Detailed Steps
To configure an allowed-active list of VSANs for an interface, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# interface fc1/1
switch(config-if)#
Configures the specified interface.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
5-12
OL-29284-01, Release 6.x
Chapter 5
Configuring Trunking
Verifying Trunking Configuration
Step 3
Command
Purpose
switch(config-if)# switchport trunk allowed vsan 2-4
Changes the allowed list for the
specified VSANs.
switch(config-if)# switchport trunk allowed vsan add 5
updated trunking membership
Expands the specified VSAN (5) to
the new allowed list.
switch(config-if)# no switchport trunk allowed vsan 2-4
Deletes VSANs 2, 3, and 4.
switch(config-if)# no switchport trunk allowed vsan add 5
Deletes the expanded allowed list.
Verifying Trunking Configuration
To display trunking configuration information, perform one of the following tasks:
Command
Purpose
show interface fc slot/port
Displays the interface configuration information
that includes trunking, trunk mode, allowed
VSANs, and status.
show trunk protocol
Displays whether the trunk protocol is enabled.
show interface trunk vsan numbers
Displays whether the interface is trunking, and the
allowed VSAN list for each trunking interface.
For detailed information about the fields in the output from these commands, refer to the Cisco MDS
NX-OS Command Reference.
The show interface command is invoked from the EXEC mode and displays trunking configurations for
a TE port. Without any arguments, this command displays the information for all of the configured
interfaces in the switch. See Examples 5-1 to 5-3.
Example 5-1
Displays a Trunked Fibre Channel Interface
switch# show interface fc1/13
fc1/13 is trunking
Hardware is Fibre Channel
Port WWN is 20:0d:00:05:30:00:58:1e
Peer port WWN is 20:0d:00:05:30:00:59:1e
Admin port mode is auto, trunk mode is on
Port mode is TE
Port vsan is 1
Speed is 2 Gbps
Receive B2B Credit is 255
Beacon is turned off
Trunk vsans (admin allowed and active) (1)
Trunk vsans (up)
(1)
Trunk vsans (isolated)
()
Trunk vsans (initializing)
()
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
233996 frames input, 14154208 bytes, 0 discards
0 CRC, 0 unknown class
0 too long, 0 too short
236 frames output, 13818044 bytes, 0 discards
11 input OLS, 12 LRR, 10 NOS, 28 loop inits
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5-13
Chapter 5
Configuring Trunking
Configuration Example for F Port Trunking
34 output OLS, 19 LRR, 17 NOS, 12 loop inits
Example 5-2
Displays the Trunking Protocol
switch# show trunk protocol
Trunk protocol is enabled
Example 5-3
Displays Per VSAN Information on Trunk Ports
switch# show interface trunk vsan 1-1000
fc3/1 is not trunking
...
fc3/7 is trunking
Vsan 1000 is down (Isolation due to vsan not configured on peer)
...
fc3/10 is trunking
Vsan 1 is up, FCID is 0x760001
Vsan 2 is up, FCID is 0x6f0001
fc3/11 is trunking
Belongs to port-channel 6
Vsan 1 is up, FCID is 0xef0000
Vsan 2 is up, FCID is 0xef0000
...
port-channel 6 is trunking
Vsan 1 is up, FCID is 0xef0000
Vsan 2 is up, FCID is 0xef0000
Configuration Example for F Port Trunking
This example shows how to configure trunking and bring up the TF-TNP link between an F port in the
NPIV core switch and an NP port in the NPV switch:
Step 1
Enable the F port trunking and channeling protocol on the MDS core switch:
switch(config)# feature fport-channel-trunk
Step 2
Enable NPIV on the MDS core switch:
switch(config)# feature npiv
Step 3
Configure the port mode to auto, F, or Fx on the MDS core switch:
switch(config)# interface fc1/2
switch(config-if)# switchport mode F
Step 4
Configure the trunk mode to ON on the MDS core switch:
switch(config-if)# switchport trunk mode on
Step 5
Configure the port mode to NP on the NPV switch:
switch(config)# interface fc1/2
switch(config-if)# switchport mode NP
Step 6
Configure the trunk mode to ON on the NPV switch:
switch(config-if)# switchport trunk mode on
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
5-14
OL-29284-01, Release 6.x
Chapter 5
Configuring Trunking
Configuration Example for F Port Trunking
Step 7
Set the port administrative state on NPIV and NPV switches to ON:
switch(config)# interface fc1/2
switch(config-if)# shut
switch(config-if)# no shut
Step 8
Save the configuration.
switch(config)# copy running-config startup-config
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
5-15
Chapter 5
Configuring Trunking
Configuration Example for F Port Trunking
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
5-16
OL-29284-01, Release 6.x
CHAPTER
6
Configuring PortChannels
•
Information About PortChannels, page 6-1
•
Prerequisites for PortChannels, page 6-10
•
Guidelines and Limitations, page 6-10
•
Default Settings, page 6-13
•
Configuring PortChannels, page 6-14
•
Verifying PortChannel Configuration, page 6-16
•
Configuration Examples for F and TF PortChannels, page 6-19
Information About PortChannels
This section includes the following topics:
•
PortChannels Overview, page 6-1
•
E PortChannels, page 6-2
•
F and TF PortChannels, page 6-3
•
PortChanneling and Trunking, page 6-3
•
Load Balancing, page 6-4
•
PortChannel Modes, page 6-6
•
PortChannel Deletion, page 6-7
•
Interfaces in a PortChannel, page 6-7
•
PortChannel Protocols, page 6-9
PortChannels Overview
PortChannels refer to the aggregation of multiple physical interfaces into one logical interface to provide
higher aggregated bandwidth, load balancing, and link redundancy (See Figure 6-1). PortChannels can
connect to interfaces across switching modules, so a failure of a switching module cannot bring down
the PortChannel link. They are referred to as PortChannels in Fibre Channels, and as Ethernet
PortChannel in Fibre over Ethernet (FCoE).
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-1
Chapter 6
Configuring PortChannels
Information About PortChannels
Figure 6-1
PortChannel Flexibility
Switch 1
Switch 2
PortChannel A
PortChannel C
79529
PortChannel B
PortChannels on Cisco MDS 9000 Family switches allow flexibility in configuration. This illustrates
three possible PortChannel configurations:
•
PortChannel A aggregates two links on two interfaces on the same switching module at each end of
a connection.
•
PortChannel B also aggregates two links, but each link is connected to a different switching module.
If the switching module goes down, traffic is not affected.
•
PortChannel C aggregates three links. Two links are on the same switching module at each end,
while one is connected to a different switching module on switch 2.
E PortChannels
An E PortChannel refers to the aggregation of multiple E ports into one logical interface to provide higher
aggregated bandwidth, load balancing, and link redundancy. PortChannels can connect to interfaces across
switching modules, so a failure of a switching module cannot bring down the PortChannel link.
A PortChannel has the following features and restrictions:
Note
•
Provides a point-to-point connection over ISL (E ports) or EISL (TE ports). Multiple links can be
combined into a PortChannel.
•
Increases the aggregate bandwidth on an ISL by distributing traffic among all functional links in the
channel.
•
Load balances across multiple links and maintains optimum bandwidth utilization. Load balancing
is based on the source ID, destination ID, and exchange ID (OX ID).
•
Provides high availability on an ISL. If one link fails, traffic previously carried on this link is switched
to the remaining links. If a link goes down in a PortChannel, the upper protocol is not aware of it. To
the upper protocol, the link is still there, although the bandwidth is diminished. The routing tables
are not affected by link failure. PortChannels may contain up to 16 physical links and may span
multiple modules for added high availability.
See the Cisco MDS 9000 Family NX-OS Fabric Configuration Guide for information about failover
scenarios for PortChannels and FSPF links.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-2
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Information About PortChannels
F and TF PortChannels
An F PortChannel is also a logical interface that combines a set of F ports connected to the same Fibre
Channel node and operates as one link between the F ports and the NP ports. The F PortChannels support
bandwidth utilization and availability like the E PortChannels. F PortChannels are mainly used to
connect MDS core and NPV switches to provide optimal bandwidth utilization and transparent failover
between the uplinks of a VSAN.
An F PortChannel trunk combines the functionality and advantages of a TF port and an F PortChannel.
This logical link uses the Cisco PTP and PCP protocols over Cisco EPP (ELS).
Note
If a Cisco MDS 9124 or 9134 switch is used as a core switch, only a nontrunking F PortChannel is
supported. Trunking is not supported on this platform when NPIV enabled.
PortChanneling and Trunking
Trunking is a commonly used storage industry term. However, the Cisco NX-OS software and switches
in the Cisco MDS 9000 Family implement trunking and PortChanneling as follows:
•
PortChanneling enables several physical links to be combined into one aggregated logical link.
•
Trunking enables a link transmitting frames in the EISL format to carry (trunk) multiple VSAN
traffic. For example, when trunking is operational on an E port, that E port becomes a TE port. A
TE port is specific to switches in the Cisco MDS 9000 Family. An industry standard E port can link
to other vendor switches and is referred to as a nontrunking interface (See Figure 6-2 and
Figure 6-3). See Chapter 5, “Configuring Trunking,” for information on trunked interfaces.
Trunking Only
Any other
switch
ISL
E port
Switch 1
Switch 2
EISL
E port
TE port
TE port
79938
Switch 1
Trunking
Figure 6-3
PortChanneling and Trunking
Switch 1
Switch 2
ISL 1
ISL 2
ISL 3
Port channel
Switch 1
Switch 2
EISL 1
EISL 2
EISL 3
Port channel
and trunking
79939
Figure 6-2
PortChanneling and trunking are used separately across an ISL.
•
PortChanneling—Interfaces can be channeled between the following sets of ports:
– E ports and TE ports
– F ports and NP ports
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-3
Chapter 6
Configuring PortChannels
Information About PortChannels
– TF ports and TNP ports
•
Trunking—Trunking permits carrying traffic on multiple VSANs between switches.
See the Cisco MDS 9000 Family NX-OS Fabric Configuration Guide.
•
Both PortChanneling and trunking can be used between TE ports over EISLs.
Load Balancing
Two methods support the load-balancing functionality:
•
Flow based—All frames between source and destination follow the same links for a given flow. That
is, whichever link is selected for the first exchange of the flow is used for all subsequent exchanges.
•
Exchange based—The first frame in an exchange picks a link and subsequent frames in the exchange
follow the same link. However, subsequent exchanges can use a different link. This provides more
granular load balancing while preserving the order of frames for each exchange.
Figure 6-4 illustrates how source ID 1 (SID1) and destination ID1 (DID1) based load balancing works.
When the first frame in a flow is received on an interface for forwarding, link 1 is selected. Each
subsequent frame in that flow is sent over the same link. No frame in SID1 and DID1 utilizes link 2.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-4
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Information About PortChannels
Figure 6-4
SID1 and DID1-Based Load Balancing
Frame 1
Frame 2
SID1, DID1,
Exchange 1
Link 1
Link 2
Frame 3
Frame n
Frame 1
Frame 2
SID1, DID1,
Exchange 2
Link 1
Link 2
Frame 3
Frame n
Frame 1
Link 1
Frame 2
Frame 3
Link 2
Frame n
79530
SID2, DID2
Exchange 1
Figure 6-5 illustrates how exchange-based load balancing works. When the first frame in an exchange is
received for forwarding on an interface, link 1 is chosen by a hash algorithm. All remaining frames in
that particular exchange are sent on the same link. For exchange 1, no frame uses link 2. For the next
exchange, link 2 is chosen by the hash algorithm. Now all frames in exchange 2 use link 2.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-5
Chapter 6
Configuring PortChannels
Information About PortChannels
Figure 6-5
SID1, DID1, and Exchange-Based Load Balancing
Frame 1
Frame 2
SID1, DID1,
Exchange 1
Link 1
Link 2
Frame 3
Frame n
Frame 1
Link 1
Frame 2
Frame 3
Link 2
Frame n
79531
SID1, DID1,
Exchange 2
For more information on configuring load balancing and in-order delivery features, see the Cisco MDS
9000 Family NX-OS Fabric Configuration Guide.
PortChannel Modes
You can configure each PortChannel with a channel group mode parameter to determine the PortChannel
protocol behavior for all member ports in this channel group. The possible values for a channel group
mode are as follows:
•
ON (default)—The member ports only operate as part of a PortChannel or remain inactive. In this
mode, the PortChannel protocol is not initiated. However, if a PortChannel protocol frame is
received from a peer port, the software indicates its nonnegotiable status. This mode is backward
compatible with the existing implementation of PortChannels in releases prior to Release 2.0(1b),
where the channel group mode is implicitly assumed to be ON. In Cisco MDS SAN-OS Releases
1.3 and earlier, the only available PortChannel mode was the ON mode. PortChannels configured in
the ON mode require you to explicitly enable and disable the PortChannel member ports at either
end if you add or remove ports from the PortChannel configuration. You must physically verify that
the local and remote ports are connected to each other.
•
ACTIVE—The member ports initiate PortChannel protocol negotiation with the peer port(s)
regardless of the channel group mode of the peer port. If the peer port, while configured in a channel
group, does not support the PortChannel protocol, or responds with a nonnegotiable status, it will
default to the ON mode behavior. The ACTIVE PortChannel mode allows automatic recovery
without explicitly enabling and disabling the PortChannel member ports at either end.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-6
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Information About PortChannels
Table 6-1 compares ON and ACTIVE modes.
Table 6-1
Channel Group Configuration Differences
ON Mode
ACTIVE Mode
No protocol is exchanged.
A PortChannel protocol negotiation is performed
with the peer ports.
Moves interfaces to the suspended state if its
operational values are incompatible with the
PortChannel.
Moves interfaces to the isolated state if its
operational values are incompatible with the
PortChannel.
When you add or modify a PortChannel member
port configuration, you must explicitly disable
(shut) and enable (no shut) the PortChannel
member ports at either end.
When you add or modify a PortChannel interface,
the PortChannel automatically recovers.
Port initialization is not synchronized.
There is synchronized startup of all ports in a
channel across peer switches.
All misconfigurations are not detected as no
protocol is exchanged.
Consistently detect misconfigurations using a
PortChannel protocol.
Transitions misconfigured ports to the suspended Transitions misconfigured ports to the isolated
state to correct the misconfiguration. Once you
state. You must explicitly disable (shut) and
enable (no shut) the member ports at either end. correct the misconfiguration, the protocol ensures
automatic recovery.
This is the default mode.
You must explicitly configure this mode.
PortChannel Deletion
When you delete the PortChannel, the corresponding channel membership is also deleted. All interfaces
in the deleted PortChannel convert to individual physical links. After the PortChannel is removed,
regardless of the mode used (ACTIVE and ON), the ports at either end are gracefully brought down,
indicating that no frames are lost when the interface is going down (see the “Graceful Shutdown” section
on page 3-14).
If you delete the PortChannel for one port, then the individual ports within the deleted PortChannel
retain the compatibility parameter settings (speed, mode, port VSAN, allowed VSAN, and port security).
You can explicitly change those settings as required.
•
If you use the default ON mode to avoid inconsistent states across switches and to maintain
consistency across switches, then the ports shut down. You must explicitly enable those ports again.
•
If you use the ACTIVE mode, then the PortChannel ports automatically recover from the deletion.
Interfaces in a PortChannel
You can add or remove a physical interface (or a range of interfaces) to an existing PortChannel. The
compatible parameters on the configuration are mapped to the PortChannel. Adding an interface to a
PortChannel increases the total bandwidth of the PortChannel. Removing an interface from a
PortChannel decreases the total bandwidth of the PortChannel.
This section describes interface configuration for a PortChannel and includes the following topics:
•
Interface Addition to a PortChannel, page 6-8
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-7
Chapter 6
Configuring PortChannels
Information About PortChannels
Note
•
Forcing an Interface Addition, page 6-9
•
Interface Deletion from a PortChannel, page 6-9
For information about PortChannel support on Generation 2 switching modules, see the “PortChannel
Limitations” section on page 2-22.
Interface Addition to a PortChannel
You can add a physical interface (or a range of interfaces) to an existing PortChannel. The compatible
parameters on the configuration are mapped to the PortChannel. Adding an interface to a PortChannel
increases the total bandwidth of the PortChannel.
A port can be configured as a member of a static PortChannel only if the following configurations are
the same in the port and the PortChannel:
•
Speed
•
Mode
•
Rate mode
•
Port VSAN
•
Trunking mode
•
Allowed VSAN list or VF-ID list
After the members are added, regardless of the mode (ACTIVE and ON) used, the ports at either end are
gracefully brought down, indicating that no frames are lost when the interface is going down (see the
“Generation 1 PortChannel Limitations” section on page 6-11 and “Graceful Shutdown” section on
page 3-14).
Compatibility Check
A compatibility check ensures that the same parameter settings are used in all physical ports in the
channel. Otherwise, they cannot become part of a PortChannel. The compatibility check is performed
before a port is added to the PortChannel.
The check ensures that the following parameters and settings match at both ends of a PortChannel:
•
Capability parameters (type of interface, Gigabit Ethernet at both ends, or Fibre Channel at both
ends).
•
Administrative compatibility parameters (speed, mode, rate mode, port VSAN, allowed VSAN list,
and port security).
Note
•
Ports in shared rate mode cannot form a PortChannel or a trunking PortChannel.
Operational parameters (remote switch WWN and trunking mode).
A port addition procedure fails if the capability and administrative parameters in the remote switch are
incompatible with the capability and administrative parameters in the local switch. If the compatibility
check is successful, the interfaces are operational and the corresponding compatibility parameter
settings apply to these interfaces.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-8
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Information About PortChannels
Suspended and Isolated States
If the operational parameters are incompatible, the compatibility check fails and the interface is placed
in a suspended or isolated state based on the configured mode:
•
An interface enters the suspended state if the interface is configured in the ON mode.
•
An interface enters the isolated state if the interface is configured in the ACTIVE mode.
Forcing an Interface Addition
You can force the port configuration to be overwritten by the PortChannel. In this case, the interface is
added to a PortChannel.
Note
•
If you use the default ON mode to avoid inconsistent states across switches and to maintain
consistency across switches, then the ports shut down. You must explicitly enable those ports again.
•
If you use the ACTIVE mode, then the PortChannel ports automatically recover from the addition.
When PortChannels are created from within an interface, the force option cannot be used.
After the members are forcefully added, regardless of the mode (ACTIVE and ON) used, the ports at
either end are gracefully brought down, indicating that no frames are lost when the interface is going
down (see the “Graceful Shutdown” section on page 3-14).
Interface Deletion from a PortChannel
When a physical interface is deleted from the PortChannel, the channel membership is automatically
updated. If the deleted interface is the last operational interface, then the PortChannel status is changed
to a down state. Deleting an interface from a PortChannel decreases the total bandwidth of the
PortChannel.
•
If you use the default ON mode to avoid inconsistent states across switches and to maintain
consistency across switches, then the ports shut down. You must explicitly enable those ports again.
•
If you use the ACTIVE mode, then the PortChannel ports automatically recover from the deletion.
After the members are deleted, regardless of the mode (ACTIVE and ON) used, the ports at either end
are gracefully brought down, indicating that no frames are lost when the interface is going down (see the
“Generation 1 PortChannel Limitations” section on page 6-11 and “Graceful Shutdown” section on
page 3-14).
PortChannel Protocols
In earlier Cisco SAN-OS releases, PortChannels required additional administrative tasks to support
synchronization. The Cisco NX-OS software provides robust error detection and synchronization
capabilities. Any change in configuration applied to the associated PortChannel interface is propagated
to all members of the channel group.
A protocol to exchange PortChannel configurations is available in all Cisco MDS switches. This addition
simplifies PortChannel management with incompatible ISLs.
The PortChannel protocol is enabled by default.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-9
Chapter 6
Configuring PortChannels
Prerequisites for PortChannels
The PortChannel protocol expands the PortChannel functional model in Cisco MDS switches. It uses the
exchange peer parameters (EPP) services to communicate across peer ports in an ISL. Each switch uses
the information received from the peer ports along with its local configuration and operational values to
decide if it should be part of a PortChannel. The protocol ensures that a set of ports are eligible to be
part of the same PortChannel. They are only eligible to be part of the same PortChannel if all the ports
have a compatible partner.
The PortChannel protocol uses the following subprotocol:
•
Bringup protocol—Automatically detects misconfigurations so you can correct them. This protocol
synchronizes the PortChannel at both ends so that all frames for a given flow (as identified by the
source FC ID, destination FC ID and OX_ID) are carried over the same physical link in both
directions. This helps make applications such as write acceleration, work for PortChannels over
FCIP links.
Prerequisites for PortChannels
Before configuring a PortChannel, consider the following guidelines:
Note
•
Configure the PortChannel across switching modules to implement redundancy on switching
module reboots or upgrades.
•
Ensure that one PortChannel is not connected to different sets of switches. PortChannels require
point-to-point connections between the same set of switches.
On switches with Generation 1 switching modules, or a combination of Generation 1 and Generation 2
switching modules, you can configure a maximum of 128 PortChannels. On switches with only
Generation 2 switching modules, or Generation 2 and Generation 3 switching modules, you can
configure a maximum of 256 PortChannels.
If you misconfigure PortChannels, you may receive a misconfiguration message. If you receive this
message, the PortChannel’s physical links are disabled because an error has been detected.
A PortChannel error is detected if the following requirements are not met:
•
Each switch on either side of a PortChannel must be connected to the same number of interfaces.
•
Each interface must be connected to a corresponding interface on the other side (see Figure 6-7 for
an example of an invalid configuration).
•
Links in a PortChannel cannot be changed after the PortChannel is configured. If you change the
links after the PortChannel is configured, be sure to reconnect the links to interfaces within the
PortChannel and reenable the links.
If all three conditions are not met, the faulty link is disabled.
Enter the show interface command for that interface to verify that the PortChannel is functioning as
required.
Guidelines and Limitations
This section includes the guidelines and limitations for this feature:
•
General Guidelines and Limitations, page 6-11
•
Generation 1 PortChannel Limitations, page 6-11
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-10
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Guidelines and Limitations
•
F and TF PortChannel Limitations, page 6-11
•
Valid and Invalid PortChannel Examples, page 6-12
General Guidelines and Limitations
Cisco MDS 9000 Family switches support the following number of PortChannels per switch:
•
Switches with only Generation 1 switching modules do not support F and TF PortChannels.
•
Switches with Generation 1 switching modules, or a combination of Generation 1 and Generation 2
switching modules, support a maximum of 128 PortChannels. Only Generation 2 ports can be
included in the PortChannels.
•
Switches with only Generation 2 switching modules or Generation 2 and Generation 3 modules
support a maximum of 256 PortChannels with 16 interfaces per PortChannel.
•
A PortChannel number refers to the unique identifier for each channel group. This number ranges
from of 1 to 256.
Generation 1 PortChannel Limitations
This section includes the restrictions on creation and addition of PortChannel members to a PortChannel
on Generation 1 hardware:
•
The 32-port 2-Gbps or 1-Gbps switching module.
•
The MDS 9140 and 9120 switches.
When configuring the host-optimized ports on Generation 1 hardware, the following PortChannel
guidelines apply:
•
If you execute the write erase command on a 32-port switching module, and then copy a saved
configuration to the switch from a text file that contains the no system default switchport
shutdown command, you need to copy the text file to the switch again for the E ports to come up
without manual configuration.
•
Any (or all) full line rate port(s) in the Cisco MDS 9100 Series can be included in a PortChannel.
•
The host-optimized ports in the Cisco MDS 9100 Series are subject to the same PortChannel rules
as 32-port switching modules; only the first port of each group of 4 ports is included in a
PortChannel.
– You can configure only the first port in each 4-port group as an E port (for example, the first
port in ports 1–4, the fifth port in ports 5–8, and so on). If the first port in the group is configured
as a PortChannel, the other three ports in each group (ports 2–4, 6–8, and so on) are not usable
and remain in the shutdown state.
– If any of the other three ports are configured in a no shutdown state, you cannot configure the
first port to be a PortChannel. The other three ports continue to remain in a no shutdown state.
F and TF PortChannel Limitations
The following guidelines and restrictions are applicable for F and TF PortChannels:
•
The ports must be in F mode.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-11
Chapter 6
Configuring PortChannels
Guidelines and Limitations
•
The PortChannel interface must be in ACTIVE mode when multiple FCIP interfaces are grouped
with WA.
•
ON mode is not supported. Only ACTIVE-ACTIVE mode is supported. By default, the mode is
ACTIVE on the NPV switches.
•
Devices logged in through F PortChannel on an MDS switch are not supported in IVR non-NAT
configuration. The devices are supported only in IVR NAT configuration.
•
Port security rules are enforced only on physical pWWNs at the single link level.
•
FC-SP authenticates only the first physical FLOGI of every PortChannel member.
•
Since the FLOGI payload carries only the VF bits to trigger the use of a protocol after the FLOGI
exchange, those bits will be overridden. In the case of the NPV switches, the core has a Cisco WWN
and will try to initiate the PCP protocol.
•
The name server registration of the N ports logging in through an F PortChannel will use the fWWN
of the PortChannel interface.
•
DPVM configuration is not supported.
•
The PortChannel port VSAN cannot be configured using DPVM.
•
The Dynamic Port VSAN Management (DPVM) database will be queried only for the first physical
FLOGI of each member, so that the port VSAN can be configured automatically.
•
DPVM does not bind FC_IDs to VSANs, but pWWNs to VSANs. It will be queried only for the
physical FLOGI.
Valid and Invalid PortChannel Examples
PortChannels are created with default values. You can change the default configuration just like any
other physical interface.
Figure 6-6 provides examples of valid PortChannel configurations.
Valid PortChannel Configurations
Channel Group 10
Cisco MDS
Switch A
Channel Group 20
Channel Group 10
Channel Group 20
1
1
1
1
2
2
2
2
3
Cisco MDS
3 Switch B
3
Cisco MDS
3 Switch B
4
4
4
4
Cisco MDS
Switch A
Channel Group 1
120480
Figure 6-6
Channel Group 2
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-12
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Default Settings
Figure 6-7 provides examples of invalid configurations. Assuming that the links are brought up in the 1,
2, 3, 4 sequence, links 3 and 4 will be operationally down as the fabric is misconfigured.
Misconfigured Configurations
Channel Group 10
Cisco MDS
Switch A
Channel Group 20
1
1
2
2
Cisco MDS
3 Switch B
3
4
X
X
Channel Group 10
4
4
1
1
2
2
Cisco MDS
3 Switch B
3
4
X
X
4
1
Cisco MDS
Switch B
2
2
3
Cisco MDS
Switch A
Channel Group 20
Channel Group 20
1
Cisco MDS
Switch A
Channel Group 10
X
X
3
Cisco MDS
Switch C
4
120488
Figure 6-7
Default Settings
Table 6-2 lists the default settings for PortChannels.
Table 6-2
Default PortChannel Parameters
Parameters
Default
PortChannels
FSPF is enabled by default.
Create PortChannel
Administratively up.
Default PortChannel mode
ON mode on non-NPV and NPIV core switches.
ACTIVE mode on NPV switches.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-13
Chapter 6
Configuring PortChannels
Configuring PortChannels
Configuring PortChannels
This section includes the following topics:
•
Configuring PortChannels Using the WizardCreating a PortChannel, page 6-14
•
Configuring the PortChannel Mode, page 6-14
•
Deleting PortChannels, page 6-15
•
Adding an Interface to a PortChannel, page 6-15
•
Forcing an Interface Addition, page 6-16
•
Deleting an Interface from a PortChannel, page 6-16
Configuring PortChannels Using the WizardCreating a PortChannel
Detailed Steps
To create a PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface port-channel 1
switch(config-if)#
Configures the specified PortChannel (1) using the
default ON mode.
Configuring the PortChannel Mode
By default, the CLI and the Device Manager create the PortChannel in ON mode in the NPIV core
switches and ACTIVE mode on the NPV switches. DCNM-SAN creates all PortChannels in ACTIVE
mode. We recommend that you create PortChannels in ACTIVE mode.
Restrictions
•
An F PortChannel is supported only on ACTIVE mode.
Detailed Steps
To configure ACTIVE mode, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface port-channel 1
switch(config-if)#
Configures the specified PortChannel (1) using the
default ON mode.
Step 3
switch(config-if)# channel mode active
Configures the ACTIVE mode.
switch(config-if)# no channel mode active
Reverts to the default ON mode.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-14
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Configuring PortChannels
Deleting PortChannels
Detailed Steps
To delete a PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# no interface port-channel 1
port-channel 1 deleted and all its members
disabled
please do the same operation on the switch at the
other end of the port-channel
switch(config)#
Deletes the specified PortChannel (1), its
associated interface mappings, and the
hardware associations for this PortChannel.
Adding an Interface to a PortChannel
Detailed Steps
To add an interface to a PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc1/15
switch(config-if)#
Configures the specified port interface (fc1/15).
Step 3
switch(config-if)# channel-group 15
Adds physical Fibre Channel port 1/15 to
channel group 15. If channel group 15 does not
exist, it is created. The port is shut down.
To add a range of ports to a PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc1/1 - 5
switch(config-if)#
Configures the specified range of interfaces. In
this example, interfaces from 1/1 to 1/5 are
configured.
Step 3
switch(config-if)# channel-group 2
Adds physical interfaces 1/1, 1/2, 1/3, 1/4, and
1/5 to channel group 2. If channel group 2 does
not exist, it is created.
If the compatibility check is successful, the
interfaces are operational and the
corresponding states apply to these interfaces.
Note
By default, the CLI adds a interface normally to a PortChannel, while DCNM-SAN adds the interface
by force, unless specified explicitly.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-15
Chapter 6
Configuring PortChannels
Verifying PortChannel Configuration
Forcing an Interface Addition
Detailed Steps
To force the addition of a port to a PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface fc1/1
switch(config-if)#
Specifies the interface fc1/1.
Step 3
switch(config-if)# channel-group 1 force
Forces the addition of the physical port for
interface fc1/1 to channel group 1. The port is
shut down.
Deleting an Interface from a PortChannel
Detailed Steps
To delete a physical interface (or a range of physical interfaces) from a PortChannel, follow these steps:
Step 1
Step 2
Command
Purpose
switch(config)# interface fc1/1
switch(config-if)#
Enters the selected physical interface level.
switch(config)# interface fc1/1 - 5
switch(config-if)#
Enters the selected range of physical interfaces.
switch(config-if)# no channel-group 2
switch(config-if)#
Deletes the physical Fibre Channel interfaces in
channel group 2.
Verifying PortChannel Configuration
To display PortChannel configuration information, perform one of the following tasks:
Command
Purpose
show port-channel summary
Displays the Ethernet PortChannel group, port
channel, type, protocol and member ports.
show port-channel database
Displays the PortChannel configured in the
default ON mode and ACTIVE mode.
show port-channel consistency
Displays the consistency status without details.
show port-channel consistency detail
Displays the consistency status with details.
show port-channel usage
Displays the PortChannel usage.
show port-channel compatibility-parameters
Displays the PortChannel compatibility.
show port-channel database interface
port-channel number
Displays the specified PortChannel interface.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-16
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Verifying PortChannel Configuration
For detailed information about the fields in the output from these commands, refer to the Cisco MDS
NX-OS Command Reference.
You can view specific information about existing PortChannels at any time from EXEC mode. The
following show commands provide further details on existing PortChannels. You can force all screen
output to go to a printer or save it to a file. See Examples 6-1 to 6-6.
Example 6-1
Displays the PortChannel Summary
switch# show port-channel summary
-----------------------------------------------------------------------------Interface
Total Ports
Oper Ports
First Oper Port
-----------------------------------------------------------------------------port-channel 77
2
0
-port-channel 78
2
0
-port-channel 79
2
2
fcip200
Example 6-2
Displays the PortChannel Configured in the Default ON Mode
switch# show port-channel database
port-channel 77
Administrative channel mode is on
Operational channel mode is on
Last membership update succeeded
2 ports in total, 0 ports up
Ports:
fcip1
[down]
fcip2
[down]
port-channel 78
Administrative channel mode is on
Operational channel mode is on
Last membership update succeeded
2 ports in total, 0 ports up
Ports:
fc2/1
[down]
fc2/5
[down]
port-channel 79
Administrative channel mode is on
Operational channel mode is on
Last membership update succeeded
First operational port is fcip200
2 ports in total, 2 ports up
Ports:
fcip101 [up]
fcip200 [up] *
Example 6-3
Displays the PortChannel Configured in the ACTIVE Mode
switch# show port-channel database
port-channel 77
Administrative channel mode is active
Operational channel mode is active
Last membership update succeeded
2 ports in total, 0 ports up
Ports:
fcip1
[down]
fcip2
[down]
port-channel 78
Administrative channel mode is active
Operational channel mode is active
Last membership update succeeded
2 ports in total, 0 ports up
Ports:
fc2/1
[down]
fc2/5
[down]
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-17
Chapter 6
Configuring PortChannels
Verifying PortChannel Configuration
port-channel 79
Administrative channel mode is active
Operational channel mode is active
Last membership update succeeded
First operational port is fcip200
2 ports in total, 2 ports up
Ports:
fcip101 [up]
fcip200 [up] *
The show port-channel consistency command has two options: without details and with details.
Example 6-4
Displays the Consistency Status without Details
switch# show port-channel consistency
Database is consistent
Example 6-5
Displays the Consistency Status with Details
switch# show port-channel consistency detail
Authoritative port-channel database:
================================================
totally 3 port-channels
port-channel 77:
2 ports, first operational port is none
fcip1
[down]
fcip2
[down]
port-channel 78:
2 ports, first operational port is none
fc2/1
[down]
fc2/5
[down]
port-channel 79:
2 ports, first operational port is fcip200
fcip101 [up]
fcip200 [up]
================================================
database 1: from module 5
================================================
totally 3 port-channels
port-channel 77:
2 ports, first operational port is none
fcip1
[down]
fcip2
[down]
port-channel 78:
2 ports, first operational port is none
fc2/1
[down]
fc2/5
[down]
port-channel 79:
2 ports, first operational port is fcip200
fcip101 [up]
fcip200 [up]
================================================
database 2: from module 4
================================================
totally 3 port-channels
port-channel 77:
2 ports, first operational port is none
fcip1
[down]
fcip2
[down]
port-channel 78:
2 ports, first operational port is none
fc2/1
[down]
fc2/5
[down]
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-18
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Configuration Examples for F and TF PortChannels
port-channel
2 ports,
fcip101
fcip200
...
79:
first operational port is fcip200
[up]
[up]
The show port-channel usage command displays details of the used and unused PortChannel numbers.
Example 6-6
Displays the PortChannel Usage
switch# show port-channel usage
Totally 3 port-channel numbers used
===================================
Used :
77 - 79
Unused:
1 - 76 , 80 - 256
Example 6-7
Displays the PortChannel Compatibility
switch# show port-channel compatibility-parameters
physical port layer
fibre channel or ethernet
port mode
trunk mode
speed
port VSAN
port allowed VSAN list
Example 6-8
Displays the PortChannel Summary
switch# show port-channel summary
-----------------------------------------------------------------------------Interface
Total Ports
Oper Ports
First Oper Port
-----------------------------------------------------------------------------port-channel 1
1
0
-port-channel 2
1
1
fc8/13
port-channel 3
0
0
-port-channel 4
0
0
-port-channel 5
1
1
fc8/3
port-channel 6
0
0
--
Configuration Examples for F and TF PortChannels
This example shows how to configure F PortChannel in shared mode and bring up the link (not supported
on the MDS 91x4 switches) between F ports on the NPIV core switches and NP ports on the NPV
switches:
Step 1
Enable the F port trunking and channeling protocol on the MDS core switch.
switch(config)# feature fport-channel-trunk
Step 2
Enable NPIV on the MDS core switch:
switch(config)# feature npiv
Step 3
Create the PortChannel on the MDS core switch:
switch(config)# interface port-channel 1
switch(config-if)# switchport mode F
switch(config-if)# channel mode active
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-19
Chapter 6
Configuring PortChannels
Configuration Examples for F and TF PortChannels
switch(config-if)# switchport trunk mode off
switch(config-if)# switchport rate-mode shared
switch(config-if)# exit
Step 4
Configure the PortChannel member interfaces on the core switch:
switch(config)# interface fc2/1-3
switch(config-if)# shut
switch(config-if)# switchport mode F
switch(config-if)# switchport trunk mode off
switch(config-if)# switchport speed 4000
switch(config-if)# switchport rate-mode shared
switch(config-if)# channel-group 1
switch(config-if)# no shut
switch(config-if)# exit
Step 5
Create the PortChannel on the NPV switch:
switch(config)# interface port-channel 1
switch(config-if)# switchport mode NP
switch(config-if)# switchport rate-mode shared
switch(config-if)# exit
Step 6
Configure the PortChannel member interfaces on the NPV switch:
switch(config)# interface fc2/1-3
switch(config-if)# shut
switch(config-if)# switchport mode NP
switch(config-if)# switchport speed 4000
switch(config-if)# switchport rate-mode shared
switch(config-if)# switchport trunk mode off
switch(config-if)# channel-group 1
switch(config-if)# no shut
switch(config-if)# exit
Step 7
Set the administrative state of all the PortChannel member interfaces in both NPIV core switch and the
NPV switch to ON:
switch(config)# interface fc1/1-3
switch(config-if)# shut
switch(config-if)# no shut
switch(config)# interface fc2/1-3
switch(config-if)# shut
switch(config-if)# no shut
Note
The speed configuration must be the same for all member interfaces in a PortChannel. While configuring
the channel in dedicated mode, ensure that required bandwidth is available to the ports.
This example shows how to configure channeling in dedicated mode and bring up the TF-TNP
PortChannel link between TF ports in the NPIV core switch, and TNP ports in the NPV switch:
Step 1
Enable the F port trunking and channeling protocol on the MDS core switch:
switch(config)# feature fport-channel-trunk
Step 2
Enable NPIV on the MDS core switch:
switch(config)# feature npiv
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-20
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Configuration Examples for F and TF PortChannels
Step 3
Create the PortChannel on the MDS core switch:
switch(config)# interface port-channel 2
switch(config-if)# switchport mode F
switch(config-if)# switchport rate-mode dedicated
switch(config-if)# channel mode active
switch(config-if)# exit
Step 4
Configure the PortChannel member interfaces on the MDS core switch in dedicated mode:
switch(config)# interface fc1/4-6
switch(config-if)# shut
switch(config-if)# switchport mode F
switch(config-if)# switchport speed 4000
switch(config-if)# switchport rate-mode dedicated
switch(config-if)# switchport trunk mode on
switch(config-if)# channel-group 2
switch(config-if)# no shut
switch(config-if)# exit
Step 5
Create the PortChannel in dedicated mode on the NPV switch:
switch(config)# interface port-channel 2
switch(config-if)# switchport rate-mode dedicated
switch(config-if)# switchport mode NP
switch(config-if)# no shut
switch(config-if)# exit
Step 6
Configure the PortChannel member interfaces on the NPV switch in dedicated mode:
switch(config)# interface fc3/1-3
switch(config-if)# shut
switch(config-if)# switchport mode NP
switch(config-if)# switchport speed 4000
switch(config-if)# switchport rate-mode dedicated
switch(config-if)# switchport trunk mode on
switch(config-if)# channel-group 2
switch(config-if)# no shut
switch(config-if)# exit
Step 7
Set the administrative state of all the PortChannel member interfaces in both NPIV core switch and the
NPV switch to ON:
switch(config)# interface fc1/4-6
switch(config-if)# shut
switch(config-if)# no shut
switch(config)# interface fc3/1-3
switch(config-if)# shut
switch(config-if)# no shut
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-21
Chapter 6
Configuring PortChannels
Configuring Ethernet PortChannels
Configuring Ethernet PortChannels
Configuring Ethernet PortChannels
Detailed Steps
To create a Ethernet PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface ethernetport-channel 1
switch(config-if)#
Configures the specified Ethernet PortChannel
using the default ON mode.
Note
Ethernet port-channels are not supported on the following Line Cards:
•
Cisco MDS 9000 18/4-Port Multiservice Module (DS-X9304-18K9) and
•
Cisco MDS 9000 16-Port Storage Services Node (DS-X9316-SSNK9).
Configuring the Ethernet PortChannel Mode
By default, the CLI and the Device Manager creates the Ethernet PortChannel. DCNM-SAN creates all
Ethernet PortChannels in ACTIVE mode. We recommend that you create Ethernet PortChannels in
ACTIVE mode.
Detailed Steps
To configure ON mode, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface
ethernet-port-channel 513
switch(config-if)#
Configures the specified Ethernet PortChannel
using the default ON mode.
Detailed Steps
To configure ACTIVE mode, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface
ethernet-port-channel 513 mode active
switch(config-if)#
Configures the specified Ethernet PortChannel
using the ACTIVE mode.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-22
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Configuring Ethernet PortChannels
Detailed Steps
To configure PASSIVE mode, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface
ethernet-port-channel 513 mode passive
switch(config-if)#
Configures the specified Ethernet PortChannel
using the PASSIVE mode.
Deleting Ethernet PortChannels
Detailed Steps
To delete a Ethernet PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# no interface
ethernet-port-channel 513
switch(config)#
Deletes the specified Ethernet PortChannel,
its associated interface mappings, and the
hardware associations for this Ethernet
PortChannel.
Adding an Interface to a Ethernet PortChannel
Detailed Steps
To add an interface to a Ethernet PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface ethernet 5/19
switch(config-if)#
Adds an Ethernet port to the Ethernet
PortChannel.
Step 3
switch(config-if)# channel-group 513
Adds the interface to a channel group.
To add a range of ports to a Ethernet PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface ethernet 1/3-5
switch(config-if-range)#
Configures the specified range of interfaces.
Step 3
switch(config-if-range)# channel-group 513
Adds interfaces 3, 4 and 5 to a channel group.
By default, the CLI adds a interface normally to a Ethernet PortChannel, while DCNM-SAN adds the
interface by force, unless specified explicitly.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-23
Chapter 6
Configuring PortChannels
Verifying Ethernet PortChannel Configuration
Forcing an Interface Addition to an Ethernet PortChannel
Detailed Steps
To force the addition of a port to a Ethernet PortChannel, follow these steps:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface ethernet 1/3-5
switch(config-if-range)#
Specifies the specified range of interfaces.
Step 3
switch(config-if)# channel-group 513 force
Forces the addition of a port for interface
ethernet-port-channel to a channel group. The
port is shut down.
Deleting an Interface from an Ethernet PortChannel
Detailed Steps
To delete an interface (or a range of physical interfaces) from a Ethernet PortChannel, follow these steps:
Command
Purpose
Step 1
switch(config)# interface ethernet 1/3
switch(config-if-range)#
Adds the Ethernet port 1/3 to Ethernet PortChannel.
Step 2
switch(config-if)# no channel-group 2
switch(config-if-range)#
Deletes the Ethernet PortChannel interfaces in
channel group 2.
Verifying Ethernet PortChannel Configuration
To display Ethernet PortChannel configuration information, perform one of the following tasks:
Command
Purpose
show ethernet-port-channel summary
Displays the Ethernet PortChannel group, port
channel, type, protocol and member ports.
show ethernet-port-channel database
Displays the Ethernet PortChannel database.
show ethernet-port-channel usage
Displays the Ethernet PortChannel usage.
show ethernet-port-channel
compatibility-parameters
Displays the Ethernet PortChannel compatibility.
show ethernet-port-channel database interface Displays the specified Ethernet PortChannel
port-channel number
interface.
You can view specific information about existing Ethernet PortChannels at any time from EXEC mode.
The following show commands provide further details on existing Ethernet PortChannels. You can force
all screen output to go to a printer or save it to a file.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-24
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Verifying Ethernet PortChannel Configuration
Example 6-9
Displays the Ethernet PortChannel Summary
switch# show ethernet-port-channel summary
Flags:
D - Down
P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended
r - Module-removed
b - BFD Session Wait
S - Switched
R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
----------------------------------------------------------------------------Group PortType
Protocol Member Ports
Channel
----------------------------------------------------------------------------513
Epo513(SD) Eth
NONE
-514
Epo514(SD) Eth
NONE
-515
Epo515(SU) Eth
LACP
Eth3/7(P)
Eth3/8(P)
Eth3/9(P)
Eth3/10(P)
516
Epo516(SU) Eth
LACP
Eth2/3(P)
Eth2/4(P)
Eth2/5(P)
Eth2/6(P)
531
Epo531(SU) Eth
NONE
Eth10/13(P) Eth10/14(P) Eth10/15(P)
Eth10/16(P)
671
Epo671(SD) Eth
NONE
-672
Epo672(SU) Eth
LACP
Eth2/17(P)
Eth2/18(P)
Eth3/19(P)
Eth3/20(P)
4090 Epo4090(SD) Eth
NONE
Eth10/1(D)
Eth10/2(D)
Eth10/3(D)
Eth10/4(D)
Eth10/5(D)
Eth10/6(D)
Eth10/7(D)
Eth10/8(D)
4093 Epo4093(SD) Eth
NONE
-4094 Epo4094(SD) Eth
NONE
--
Example 6-10
Displays the Ethernet PortChannel Database
switch# show ethernet-port-channel database
ethernet-port-channel513
Last membership update is successful
0 ports in total, 0 ports up
Age of the port-channel is 0d:13h:34m:03s
ethernet-port-channel514
Last membership update is successful
0 ports in total, 0 ports up
Age of the port-channel is 0d:13h:34m:02s
The show ethernet-port-channel command displays details of used and unused Ethernet PortChannel
numbers:
Example 6-11
Displays the Ethernet PortChannel Usage
switch# show ethernet-port-channel usage
Total 11 port-channel numbers used
====================================
Used :
513 - 516 , 671 - 672 , 4090 , 4093 - 4094 , 255 - 256
Unused:
1 - 512 , 517 - 670 , 673 - 4089 , 4091 - 4092 , 4095 - 254
257 – 4096
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-25
Chapter 6
Configuring PortChannels
Verifying Ethernet PortChannel Configuration
Example 6-12
Displays the Ethernet PortChannel Compatibility
switch# show ethernet-port-channel compatibility-parameters
* port mode
Members must have the same port mode configured, either E,F or AUTO. If
they are configured in AUTO port mode, they have to negotiate E or F mode
when they come up. If a member negotiates a different mode, it will be
suspended.
* speed
Members must have the same speed configured. If they are configured in AUTO
speed, they have to negotiate the same speed when they come up. If a member
negotiates a different speed, it will be suspended.
* MTU
Members have to have the same MTU configured. This only applies to ethernet
port-channel.
* MEDIUM
Members have to have the same medium type configured. This only applies to
ethernet port-channel.
* Span mode
Members must have the same span mode.
* load interval
Member must have same load interval configured.
* span port headers
The exclude header span config must either be present or absent on all
members.
* sub interfaces
Members must not have sub-interfaces.
* Duplex Mode
Members must have same Duplex Mode configured.
* Ethernet Layer
Members must have same Ethernet Layer (switchport/no-switchport) configured.
* Span Port
Members cannot be SPAN ports.
* Storm Control
Members must have same storm-control configured.
* Flow Control
Members must have same flowctrl configured.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-26
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Verifying Ethernet PortChannel Configuration
* Capabilities
Members must have common capabilities.
* Capabilities speed
Members must have common speed capabilities.
Members must have common speed duplex capabilities.
* rate mode
Members must have the same rate mode configured.
* Capabilities FabricPath
Members must have common fabricpath capability.
* Port has PVLAN config
Members must have same pvlan configuration
* 1G port is not capable of acting as peer-link
Members must be 10G to become part of a vPC peer-link.
* EthType
Members must have same EthType configured.
* shared interface
Members can not be shared-interfaces.
* Capabilities SpanDest
Members must be capable of span destination configuration
* Module Type Incompatible
Module type for interfaces is not compatible.
* Port Mode Fabricpath Incompatible
Members are Fabricpath Enforce locked, not compatible.
* Rewrite Translate - Rate Mode shared, Incompatible
Members have rate mode shared,and rewrite translate exists on the channel
group,Not compatible.
* FEX - PVLAN promiscuous config
FEX ports cannot have private VLAN promiscuous or promiscuous trunk configs
* vmtracker enable mismatch
Members have vmtracker enable mismatch
* port
Members port VLAN info.
* port
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-27
Chapter 6
Configuring PortChannels
Verifying Ethernet PortChannel Configuration
Members port does not exist.
* switching port
Members must be switching port, Layer 2.
* port access VLAN
Members must have the same port access VLAN.
* port native VLAN
Members must have the same port native VLAN.
* port allowed VLAN list
Members must have the same port allowed VLAN list.
* VLAN translation mapping list
Members must have the same VLAN translation list.
* Members should have same fex config
Members must have same FEX configuration.
* FEX pinning max-links not one
FEX pinning max-links config is not one.
* Multiple port-channels with same Fex-id
Multiple port-channels to same FEX not allowed.
* Pinning Params
Members must have the same pinning parameters.
* All HIF member ports not in same pinning group
All HIF member ports not in same pinning group
* Slot in host vpc mode
Cannot add cfged slot member to fabric po vpc.
* Members in multiple FEX
Members must belong to same FEX.
* Members are of different type
Members must of same interface type.
* port egress queuing policy
10G port-channel members must have the same egress queuing policy as the
port-channel.
* Port Security policy
Members must have the same port-security enable status as port-channel
Members must have the same port-security enable status as port-channel
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-28
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Verifying Ethernet PortChannel Configuration
* Port priority-flow-control
PFC config should be the same for all the members
* Dot1x policy
Members must have host mode as multi-host with no mab configuration. Dot1X
cannot be enabled on members when Port Security is configured on port
channel
* PC Queuing policy
Queuing policy for Non-DCE PC should be non-dce
* PC Queuing policy
Queuing policy for the PC should be same as system queuing policy
* PVLAN port config
Members must have same PVLAN port configuration.
* Emulated switch port type policy
vPC ports in emulated switch complex should be L2MP capable.
* VFC bound to port
Members cannot have VFCs bound to them.
* VFC bound to port channel
Port Channels that have VFCs bound to them cannot have more than one member
* VFC bound to FCoE capable port channel
Port Channels that have VFCs bound to them cannot have non fcoe capable
member
* VFC bound to FCoE capable port channel
Port Channels that have VFCs bound to them cannot have non fcoe licensed
member
* EVC EFP configured under port channel member
Port Channel members cannot have EVC EFP configured under them
* Fex ports for span
Port-Channel is already a SPAN source. Cannot add FEX ports connected
through F2/F2E series line cards to this PC
* shut lan
Members cannot have shut lan configured
* VPC orphan port suspend configured under port channel member
* VPC orphan port suspend configured under port channel member
Port channel members cannot have vPC orphan port suspend configured under
them
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-29
Chapter 6
Configuring PortChannels
Verifying Ethernet PortChannel Configuration
* tag native
Members must have same native vlan tagging configured
* VNSEG VSI configured under port channel member
Port Channel members cannot have VNSEG VSI configured under them
* VNSEG VSI Id for FEX > 62
VNSEG VSI Id greater than 62 not allowed on FEX Ports
* VNSEG VSI configured under port channel and VSI under FEX member have overlap
ing mappings
VNSEG VSI configured under port channel and VSI under FEX member should not
have overlapping mappings
* VNSEG VSI Id and profile name mismatch on FEX member with those on Port-chann
l
FEX Port Channel members and port-channel configuration for VSI Id and
profile name should match
* VNSEG VSI type mismatch on FEX member with those on Port-channel
All FEX ports should have same VSI Type - STATIC/VDP/DFS
* PLSM EDP port type policy
EDP ports should fail to be part of Port Channel
* Port priority-flow-control long-distance
PFC_LD config should be the same for all the members
* Port channel LFC-PFC compat check fail
PFC of PC to be turned OFF when LFC is ON or LFC of PC to be default, to
add DCE member
* CTS mode
Members must have the same CTS mode configured (either "cts manual" or "cts
dot1x" or no cts)
* CTS SGT propagation
SGT propagation must either be enabled or disabled on all members
* CTS SGT policy
Members must all have either "policy static" or "policy dynamic" or no
policy configured
* CTS peer identity
Members must all have the same peer identity configured
* CTS SGT configuration
Members must all have the same SGT configured
* CTS replay protection
Replay protection must either be enabled or disabled on all members
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-30
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Verifying Ethernet PortChannel Configuration
switch# show interface ethernet-port-channel 515
ethernet-port-channel515 is up
admin state is up
Hardware: Port-Channel, address: 046c.9d32.6bb3 (bia 046c.9d32.6bb3)
MTU bytes (CoS values): MTU 1500(0-2,4-7) bytes MTU 2112(3) bytes
BW 160000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/25
5
Encapsulation ARPA, medium is p2p
Port mode is trunk
full-duplex, 40 Gb/s
Input flow-control is off, output flow-control is off
Auto-mdix is turned off
Switchport monitor is off
EtherType is 0x8100
Members in this channel: Eth3/7, Eth3/8, Eth3/9, Eth3/10
Last clearing of "show interface" counters never
2 interface resets
Load-Interval #1: 30 seconds
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 0 bits/sec, 0 packets/sec
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
Load-Interval #2: 5 minute (300 seconds)
300 seconds input rate 0 bits/sec, 0 packets/sec
300 seconds output rate 0 bits/sec, 0 packets/sec
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
RX
54416 unicast packets 448442 multicast packets 12810 broadcast packets
515678 input packets 45314396 bytes
1 jumbo packets 0 storm suppression packets
0 runts 1 giants 0 CRC/FCS 0 no buffer
1 input error 0 short frame 0 overrun
0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
654 unicast packets 513120 multicast packets 0 broadcast packets
513774 output packets 139758400 bytes
5 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
switch# show ethernet-port-channel summary
Flags: D - Down
P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended
r - Module-removed
b - BFD Session Wait
S - Switched
R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
------------------------------------------------------------------------------Group PortType
Protocol Member Ports
Channel
------------------------------------------------------------------------------513
Epo513(SD) Eth
NONE
-514
Epo514(SD) Eth
NONE
-515
Epo515(SU) Eth
LACP
Eth3/7(P)
Eth3/8(P)
Eth3/9(P)
Eth3/10(P)
516
Epo516(SU) Eth
LACP
Eth2/3(P)
Eth2/4(P)
Eth2/5(P)
Eth2/6(P)
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-31
Chapter 6
Configuring PortChannels
Configuration Examples for F and TF Ethernet PortChannels
531
Epo531(SU)
Eth
NONE
671
672
Epo671(SD)
Epo672(SU)
Eth
Eth
NONE
LACP
4090
Epo4090(SD)
Eth
NONE
4093
4094
Epo4093(SD)
Epo4094(SD)
Eth
Eth
NONE
NONE
Eth10/13(P)
Eth10/16(P)
-Eth2/17(P)
Eth3/20(P)
Eth10/1(D)
Eth10/4(D)
Eth10/7(D)
---
Eth10/14(P)
Eth10/15(P)
Eth2/18(P)
Eth3/19(P)
Eth10/2(D)
Eth10/5(D)
Eth10/8(D)
Eth10/3(D)
Eth10/6(D)
Configuration Examples for F and TF Ethernet
PortChannels
This example shows how to configure F Ethernet PortChannel in shared mode and bring up the link (not
supported on the MDS 91x4 switches) between F Ethernet ports on the NPIV core switches and NP ports
on the NPV switches:
Step 1
Enable the F Ethernet port trunking and channeling protocol on the MDS core switch.
switch(config)# feature fport-channel-trunk
Step 2
Enable NPIV on the MDS core switch:
switch(config)# feature npiv
switch(config)# feature lldp
switch(config)# feature lacp
Step 3
Create the Ethernet PortChannel on the MDS core switch:
switch(config)# interface ethernet-port-channel 513
switch(config-if)# switchport trunk mode off
switch(config-if)# exit
Step 4
Create the PortChannel on the NPV switch:
switch(config)# interface ethernet-port-channel 513
switch(config-if)# exit
Step 5
Configure the Ethernet PortChannel member interfaces on the NPV switch:
switch(config)# interface interface 1/3-5
switch(config-if-range)# channel-group 513
switch(config-if-range)# no shut
switch(config-if-range)# exit
Step 6
Set the administrative state of all the Ethernet PortChannel member interfaces in both NPIV core switch
and the NPV switch to ON:
switch(config)# interface ethernet 1/3-5
switch(config-if-range)# no shut
switch(config)# interface ethernet 2/3-5
switch(config-if-range)# no shut
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-32
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Configuration Examples for F and TF Ethernet PortChannels
Note
The speed configuration must be the same for all member interfaces in a Ethernet PortChannel. While
configuring the channel in dedicated mode, ensure that required bandwidth is available to the ports.
This example shows how to configure channeling in dedicated mode and bring up the TF-TNP
PortChannel link between TF ports in the NPIV core switch, and TNP ports in the NPV switch:
Step 1
Enable the F port trunking and channeling protocol on the MDS core switch:
switch(config)# feature fport-channel-trunk
Step 2
Enable NPIV on the MDS core switch:
switch(config)# feature npiv
Step 3
Create the Ethernet PortChannel on the MDS core switch:
switch(config)# interface ethernet-port-channel 513
switch(config-if)# exit
Step 4
Configure the Ethernet PortChannel member interfaces on the MDS core switch in dedicated mode:
switch(config)# interface ethernet 1/3-5
switch(config-if-range)# shut
switch(config-if-range)# channel-group 2
switch(config-if-range)# no shut
switch(config-if-range)# exit
Step 5
Create the Ethernet PortChannel in dedicated mode on the NPV switch:
switch(config)# interface ethernet-port-channel 513
switch(config-if)# no shut
switch(config-if)# exit
Step 6
Configure the Ethernet PortChannel member interfaces on the NPV switch in dedicated mode:
switch(config)# interface ethernet 1/3-5
switch(config-if-range)# no shut
switch(config-if-range)# exit
Step 7
Set the administrative state of all the Ethernet PortChannel member interfaces in both NPIV core switch
and the NPV switch to ON:
switch(config)# interface ethernet 1/3-5
switch(config-if-range)# no shut
switch(config)# interface ethernet 2/3-5
switch(config-if-range)# no shut
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-33
Chapter 6
Configuring PortChannels
Configuring Virtual Interfaces
Configuring Virtual Interfaces
Mapping a VSAN to a VLAN
A unique, dedicated VLAN must be configured at every converged access switch to carry traffic for each
virtual fabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 1002, VLAN 1003 for VSAN
2, and so on). If you enable MST, you must use a separate Multiple Spanning Tree (MST) instance for
FCoE VLANs.
Before You Begin
•
Ensure you have installed the correct license for FCoE.
•
Ensure you have enabled FCoE.
For Cisco Nexus 7000 Series Switches, ensure that you are in the storage VDC.
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# vsan database
switch(config-vsan-db)#
Enters VSAN database configuration mode.
Step 3
switch(config-vsan-db)# vsan 200
Defines the VSAN. The VSAN number range is
from 1 to 4094.
Step 4
switch(config-vsan-db)# vlan 200
switch(config-vlan)#
Enters VLAN configuration mode. The VLAN
number range is from 1 to 4096.
Step 5
switch(config-vlan)# fcoe vsan 200
Enables FCoE for the specified VLAN and
configures the mapping from this VLAN to the
specified VSAN. If you do not specify a VSAN
number, a mapping is created from this VLAN
to the VSAN with the same number.
Step 6
switch(config-vlan)# exit
switch(config)#
Exits VLAN configuration mode. You must exit
this mode to execute the configured commands
on the Cisco Nexus 7000 Series Switches.
Step 7
switch(config-vlan)# show vlan fcoe
(Optional)
Displays information about the FCoE
configuration for a VLAN.
Step 8
switch(config-vlan)# copy running-config
startup-config
(Optional)
Copies the running configuration to the startup
configuration.
This example shows how to map VLAN 200 to VSAN 200 on a Cisco MDS 9000 Series Switch:
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-34
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Configuring Virtual Interfaces
switch(config)# vlan 200
switch(config-vlan)# fcoe vsan 200
This example shows how to map VLAN 300 to VSAN 300 on a Cisco Nexus 7000 Series Switches:
switch(config)# switchto vdc fcoe_vdc
switch-fcoe_vdc# configure terminal
switch-fcoe_vdc(config)# vlan 300
switch-fcoe_vd(config-vlan)# fcoe vsan 300
Creating an Explicit Virtual Fibre Channel Interface
You can create an explicit virtual Fibre Channel interface. You must bind the virtual Fibre Channel
interface to a physical interface before it can be used.
Before You Begin
•
Ensure you have installed the correct license for FCoE.
•
Ensure you have enabled FCoE
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface vfc 4
switch(config-if)#
Creates a virtual Fibre Channel interface (if it
does not already exist) and enters interface
configuration mode. The vfc-id range is from 1
to 8192.
Step 3
switch(config-if)# switchport mode e
Configures the switchport mode for a virtual
Fibre Channel interface. The mode is E or F.
The default is F mode.
Step 4
switch(config-if)# bind interface
ethernet-port-channel 513
switch(config-if)# no shutdown
Binds the virtual Fibre Channel interface to the
specified interface. Use ? to see the supported
interfaces and port channels. Use the no form of
this command to unbind the virtual Fibre
Channel interface from the specified interface.
Step 5
switch(config-if)# show interface vfc
(Optional)
Displays information about the virtual Fibre
Channel interfaces.
Step 6
switch(config)# copy running-config
startup-config
(Optional)
Copies the running configuration to the startup
configuration.
This example shows how to bind a virtual Fibre Channel interface to an Ethernet interface:
switch# configure terminal
switch(config)# interface vfc 4
switch(config-if)# bind interface ethernet-port-channel 513
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-35
Chapter 6
Configuring PortChannels
Configuring Virtual Interfaces
This example shows how to delete a virtual Fibre Channel interface:
switch# configure terminal
switch(config)# no interface vfc 4
Creating an Implicit Virtual Fibre Channel Port Channel Interface
You can create a virtual Fibre Channel port channel interface that automatically binds to the port channel
with the same interface number.
Before You Begin
•
For the Cisco Nexus 7000 Series, ensure that you create the port channel interface before you create
the virtual Fibre Channel port channel interface.
•
For the Cisco MDS 9700 switches, MDS 9500 switches, and MDS 9250i switch, ensure that you
create the Ethernet port channel interface before you create the virtual Fibre Channel port channel
interface.
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# interface vfc-port-channel
513
switch(config-if)#
Creates a virtual Fibre Channel interface (if it
does not already exist) that is bound to the port
channel with the same interface number and
enters interface configuration mode. The
int-numberrange is from 1 to 4096 (for Cisco
Nexus 7000) or from 257 to 4095 (for Cisco
MDS 9500). The default switchport mode for
this interface is F.
Note
Starting from the MDS 9710 switch and
newer chassis, we support only the
Ethernet port channel or the channel
group ID numbers ranging from 513 to
4096. However, we support 257 to 4096
ID numbers on the MDS Ethernet port
channel.
Step 3
switch(config-if)# switchport mode e
Configures the switchport mode for a virtual
Fibre Channel interface. The mode is E or F.
The default is F mode.
Step 4
switch(config-if)# show interface
vfc-port-channel 2
(Optional)
Displays information about the virtual Fibre
Channel interfaces bound to port channel
interfaces.
Step 5
switch(config)# copy running-config
startup-config
(Optional)
Copies the running configuration to the startup
configuration.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-36
OL-29284-01, Release 6.x
Chapter 6
Configuring PortChannels
Configuring Virtual Interfaces
Associating a Virtual Fibre Channel Interface to a VSAN
You must configure unique, dedicated VLAN at every converged access switch to carry traffic for each
Virtual Fabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 1, VLAN 1003 for VSAN 2,
and so on). If you enable MST, you must use a separate MST instance for FCoE VLANs.
Before You Begin
For Cisco Nexus 7000 Series Switches, ensure that you are in the storage VDC.
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# vsan database
switch(config-vsan-db)#
Enters VSAN configuration mode.
Step 3
switch(config-vsan-db)# vsan 2
(Optional)
Creates the VSAN. The vsan-id range is from 1
to 4094 and must map to a VLAN on the
physical Ethernet interface that is bound to the
virtual Fibre Channel interface.
Step 4
switch(config-vsan-db)# vsan 2 interface vfc
4
Configures the association between the VSAN
and virtual Fibre Channel interface or virtual
Fibre Channel port channel. Thevsan-id range is
from 1 to 4094 and must map to a VLAN on the
physical Ethernet interface or port channel that
is bound to the virtual Fibre Channel interface
or virtual Fibre Channel port channel. The
vfc-idrange is from 1 to 8192. Us the no form of
this command to dissassociate the connection
between the VSAN and virtual Fibre Channel
interface or virtual Fibre Channel port channel.
Step 5
switch(config-vsan-db)# show vsan
(Optional)
Displays information about the VSAN.
Step 6
switch(config-vsan-db)# copy running-config
startup-config
(Optional)
Copies the running configuration to the startup
configuration.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
6-37
Chapter 6
Configuring PortChannels
Configuring Virtual Interfaces
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
6-38
OL-29284-01, Release 6.x
CHAPTER
7
Configuring N Port Virtualization
•
Information About N Port Virtualization, page 7-1
•
Guidelines and Limitations, page 7-8
•
Configuring N Port Virtualization, page 7-11
•
Verifying NPV Configuration, page 7-14
Information About N Port Virtualization
This section includes the following topics:
•
NPV Overview, page 7-1
•
N Port Identifier Virtualization, page 7-2
•
N Port Virtualization, page 7-3
•
NPV Mode, page 7-4
•
NP Ports, page 7-5
•
NP Links, page 7-5
•
Default Port Numbers, page 7-7
•
NPV CFS Distribution over IP, page 7-7
•
NPV Traffic Management, page 7-7
•
Multiple VSAN Support, page 7-8
NPV Overview
N port virtualization (NPV) reduces the number of Fibre Channel domain IDs in SANs. Switches
operating in the NPV mode do not join a fabric. They pass traffic between NPV core switch links and
end devices, which eliminates the domain IDs for these edge switches.
NPV is supported by the following Cisco MDS 9000 switches and Cisco Nexus 5000 Series switches
only:
•
Cisco MDS 9124 Multilayer Fabric Switch
•
Cisco MDS 9134 Fabric Switch
•
Cisco MDS 9148 Multilayer Fabric Switch
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-1
Chapter 7
Configuring N Port Virtualization
Information About N Port Virtualization
Note
•
Cisco MDS 9148S Multilayer Fabric Switch
•
Cisco Fabric Switch for HP c-Class BladeSystem
•
Cisco Fabric Switch for IBM BladeCenter
•
Cisco MDS 9396S Multilayer Fabric Switch
•
Cisco Nexus 5000 Series switches
NPV is available on these switches only while in NPV mode; if in switch mode, NPV is not available.
N Port Identifier Virtualization
N port identifier virtualization (NPIV) provides a means to assign multiple FC IDs to a single N port.
This feature allows multiple applications on the N port to use different identifiers and allows access
control, zoning, and port security to be implemented at the application level. Figure 7-1 shows an
example application using NPIV.
Figure 7-1
NPIV Example
NPV-Core Switch
(MDS or 3rd party switch
with NPIV support)
FC
NP-port
VS
5
NPV Device uses
the same domains
as the NPV-core
switches
Cisco Fabric Switch
for HP c-Class BladeSystem
Cisco Fabric Switch
for IBM BladeCenter
in a Blade Chassis
20.2.1
AN
Can have
multiple uplinks
on different
VSANs
15
Up to 100
NPV switches
F-port (server port)
Blade Server 1
Blade Server 2
10.5.2
10.5.7
FC
Target
20.5.1
Initiator
(no FL ports)
184639
V
N
SA
10.1.1
VSAN 10
F-port
FC
Blade Server n
You must globally enable NPIV for all VSANs on the MDS switch to allow the NPIV-enabled
applications to use multiple N port identifiers.
Note
All of the N port identifiers are allocated in the same VSAN.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-2
OL-29284-01, Release 6.x
Chapter 7
Configuring N Port Virtualization
Information About N Port Virtualization
N Port Virtualization
Typically, Fibre Channel networks are deployed using a core-edge model with a large number of fabric
switches connected to edge devices. Such a model is cost-effective because the per port cost for director
class switches is much higher than that of fabric switches. However, as the number of ports in the fabric
increases, the number of switches deployed also increases, and you can end up with a significant increase
in the number of domain IDs. This challenge becomes even more difficult when additional blade chassis
are deployed in Fibre Channel networks.
NPV addresses the increase in the number of domain IDs needed to deploy a large number of the ports
by making a fabric or blade switch appear as a host to the core Fibre Channel switch, and as a Fibre
Channel switch to the servers in the fabric or blade switch. NPV aggregates multiple locally connected
N ports into one or more external NP links, which shares the domain ID of the NPV core switch among
multiple NPV switches. NPV also allows multiple devices to attach to same port on the NPV core switch,
which reduces the need for more ports on the core
For more information on scalability limits, see the Cisco MDS NX-OS Release 6.2 Configuration Limits
guide.
Figure 7-2
Cisco NPV Fabric Configuration
NPV-Core Switch
(MDS or 3rd party switch
with NPIV support)
FC
V
NP-port
N
SA
10.1.1
VS
5
NPV Device uses
the same domains
as the NPV-core
switches
Blade Server 1
Blade Server 2
AN
Can have
multiple uplinks
on different
VSANs
15
Up to 100
NPV switches
F-port (server port)
10.5.2
10.5.7
FC
Target
20.5.1
Initiator
(no FL ports)
184639
Cisco Fabric Switch
for HP c-Class BladeSystem
Cisco Fabric Switch
for IBM BladeCenter
in a Blade Chassis
20.2.1
VSAN 10
F-port
FC
Blade Server n
While NPV is similar to N port identifier virtualization (NPIV), it does not offer exactly the same
functionality. NPIV provides a means to assign multiple FC IDs to a single N port, and allows multiple
applications on the N port to use different identifiers. NPIV also allows access control, zoning, and port
security to be implemented at the application level. NPV makes use of NPIV to get multiple FCIDs
allocated from the core switch on the NP port.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-3
Chapter 7
Configuring N Port Virtualization
Information About N Port Virtualization
Note
For the Cisco MDS 9124, Cisco MDS 9134, and Cisco MDS 9148 legacy switches and the Cisco MDS
9148S and Cisco MDS 9396S switches supporting NPV mode, the nested NPV switches are not
supported in the topology.
Figure 7-3 shows a more granular view of an NPV configuration at the interface level.
Figure 7-3
Cisco NPV Configuration–Interface View
Host
N-Port
NPV Device
NPV Core Switch
F-Port
F-Port
NP-Port
184631
NPIV enabled
N-Port
Host
NPV Mode
A switch is in NPV mode after a user has enabled NPV and the switch has successfully rebooted. NPV
mode applies to an entire switch. All end devices connected to a switch that is in NPV mode must log in
as an N port to use this feature (loop-attached devices are not supported). All links from the edge
switches (in NPV mode) to the NPV core switches are established as NP ports (not E ports), which are
used for typical interswitch links. NPIV is used by the switches in NPV mode to log in to multiple end
devices that share a link to the NPV core switch.
Note
In-order data delivery is not required in NPV mode because the exchange between two end devices
always takes the same uplink to the core from the NPV device. For traffic beyond the NPV device, core
switches will enforce in-order delivery if needed and/or configured.
After entering NPV mode, only the following commands are available:
Command
Description
aaa
Configure aaa functions.
banner
Configure banner message.
boot
Configure boot variables.
callhome
Enter the callhome configuration mode.
cfs
CFS configuration commands.
cli
Configure CLI commands.
clock
Configure time-of-day clock.
crypto
Set crypto settings.
event
Event Manager commands.
fcanalyzer
Configure cisco fabric analyzer.
feature
Command to enable/disable features.
fips
Enable/Disable FIPS mode.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-4
OL-29284-01, Release 6.x
Chapter 7
Configuring N Port Virtualization
Information About N Port Virtualization
Command
Description
flex-attach
Configure Flex Attach.
hardware
Hardware Internal Information.
hw-module
Enable/Disable OBFL information.
interface
Configure interfaces.
ip
Configure IP features.
ipv6
Configure IPv6 features.
license
Modify license features.
line
Configure a terminal line.
logging
Modify message logging facilities.
module
Configure for module.
no
Negate a command or set its defaults.
npv
Config commands for FC N_port Virtualizer.
ntp
NTP Configuration.
password
Password for the user
port-group-monitor
Configure port group monitor.
port-monitor
Configure port monitor.
power
Configure power supply.
poweroff
Power off a module in the switch.
radius
Configure RADIUS configuration.
radius-server
Configure RADIUS related parameters.
rate-mode
Configure rate mode oversubscription limit.
rmon
Remote Monitoring.
role
Configure roles.
snmp
Configure snmp.
snmp-server
Configure snmp server.
span
Enter SPAN configuration mode.
ssh
SSH to another system.
switchname
Configure system's network name.
system
System management commands.
terminal
Configure terminal settings.
this
Shows info about current object (mode's instance).
username
Configure user information.
vsan
Enter the vsan configuration mode.
wwn
Set secondary base MAC addr and range for additional WWNs.
NP Ports
An NP port (proxy N port) is a port on a device that is in NPV mode and connected to the NPV core
switch using an F port. NP ports behave like N ports except that in addition to providing N port behavior,
they also function as proxies for multiple, physical N ports.
NP Links
An NP link is basically an NPIV uplink to a specific end device. NP links are established when the uplink
to the NPV core switch comes up; the links are terminated when the uplink goes down. Once the uplink
is established, the NPV switch performs an internal FLOGI to the NPV core switch, and then (if the
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-5
Chapter 7
Configuring N Port Virtualization
Information About N Port Virtualization
FLOGI is successful) registers itself with the NPV core switch’s name server. Subsequent FLOGIs from
end devices in this NP link are converted to FDISCs. For more details refer to the “Internal FLOGI
Parameters” section on page 7-6.
Server links are uniformly distributed across the NP links. All the end devices behind a server link will
be mapped to only one NP link.
Internal FLOGI Parameters
When an NP port comes up, the NPV device first logs itself in to the NPV core switch and sends a FLOGI
request that includes the following parameters:
•
The fWWN (fabric port WWN) of the NP port used as the pWWN in the internal login.
•
The VSAN-based sWWN (switch WWN) of the NPV device used as nWWN (node WWN) in the
internal FLOGI.
After completing its FLOGI request, the NPV device registers itself with the fabric name server using
the following additional parameters:
Note
•
Switch name and interface name (for example, fc1/4) of the NP port is embedded in the symbolic
port name in the name server registration of the NPV device itself.
•
The IP address of the NPV device is registered as the IP address in the name server registration of
the NPV device.
The BB_SCN of internal FLOGIs on NP ports is always set to zero. The BB_SCN is supported at the
F-port of the NPV device.
Figure 7-4 shows the internal FLOGI flows between an NPV core switch and an NPV device.
Figure 7-4
Internal FLOGI Flows
fc 5/10
fwwn
fc 1/5
pwwn
nwwn
NPV Device
184572
NPV Core Switch
Table 7-1 identifies the internal FLOGI parameters that appear in Figure 7-4.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-6
OL-29284-01, Release 6.x
Chapter 7
Configuring N Port Virtualization
Information About N Port Virtualization
Table 7-1
Internal FLOGI Parameters
Parameter
Derived From
pWWN
The fWWN of the NP port.
nWWN
The VSAN-based sWWN of the NPV device.
fWWN
The fWWN of the F port on the NPV core switch.
symbolic port name
The switch name and NP port interface string.
Note
If there is no switch name available, then the output will display
“switch.” For example, switch: fc1/5.
IP address
The IP address of the NPV device.
symbolic node name
The NPV switch name.
Although fWWN-based zoning is supported for NPV devices, it is not recommended because:
•
Zoning is not enforced at the NPV device (rather, it is enforced on the NPV core switch).
•
Multiple devices behind an NPV device log in via the same F port on the core (they use same fWWN
and cannot be separated into different zones).
•
The same device might log in using different fWWNs on the core switch (depending on the NPV
link it uses) and may need to be zoned using different fWWNs.
Default Port Numbers
Port numbers on NPV-enabled switches will vary depending on the switch model. For details about port
numbers for NPV-eligible switches, see the Cisco NX-OS Family Licensing Guide.
NPV CFS Distribution over IP
NPV devices use only IP as the transport medium. CFS uses multicast forwarding for CFS distribution.
NPV devices do not have ISL connectivity and FC domain. To use CFS over IP, multicast forwarding has
to be enabled on the Ethernet IP switches all along the network that physically connects the NPV switch.
You can also manually configure the static IP peers for CFS distribution over IP on NPV-enabled
switches. For more information, see the Cisco MDS 9000 Family NX-OS System Management
Configuration Guide.
NPV Traffic Management
This sections discusses the following aspects of load balancing:
•
Auto, page 7-8
•
Traffic Map, page 7-8
•
Disruptive, page 7-8
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-7
Chapter 7
Configuring N Port Virtualization
Guidelines and Limitations
Auto
Before Cisco MDS SAN-OS Release 3.3(1a), NPV supported automatic selection of external links.
When a server interface is brought up, an external interface with the minimum load is selected from the
available links. There is no manual selection on the server interfaces using the external links. Also, when
a new external interface was brought up, the existing load was not distributed automatically to the newly
available external interface. This newly brought up interface is used only by the server interfaces that
come up after this interface.
Traffic Map
As in Cisco MDS SAN-OS Release 3.3(1a) and NX-OS Release 4.1(1a), NPV supports traffic
management by allowing you to select and configure the external interfaces that the server uses to
connect to the core switches.
Note
When the NPV traffic management is configured, the server uses only the configured external interfaces.
Any other available external interface will not be used.
The NPV traffic management feature provides the following benefits:
•
Facilitates traffic engineering by providing dedicated external interfaces for the servers connected
to NPV.
•
Uses the shortest path by selecting external interfaces per server interface.
•
Uses the persistent FC ID feature by providing the same traffic path after a link break, or reboot of
the NPV or core switch.
•
Balances the load by allowing the user to evenly distribute the load across external interfaces.
Disruptive
Disruptive load balance works independent of automatic selection of interfaces and a configured traffic
map of external interfaces. This feature forces reinitialization of the server interfaces to achieve load
balance when this feature is enabled and whenever a new external interface comes up. To avoid flapping
the server interfaces too often, enable this feature once and then disable it whenever the needed load
balance is achieved.
If disruptive load balance is not enabled, you need to manually flap the server interface to move some
of the load to a new external interface.
Multiple VSAN Support
By grouping devices into different NPV sessions based on VSANs, it is possible to support multiple
VSANs on the NPV-enabled switch. The correct uplink must be selected based on the VSAN that the
uplink is carrying.
Guidelines and Limitations
This section includes the guidelines and limitations for this feature:
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-8
OL-29284-01, Release 6.x
Chapter 7
Configuring N Port Virtualization
Guidelines and Limitations
•
NPV Guidelines and Requirements, page 7-9
•
NPV Traffic Management Guidelines, page 7-9
•
DPVM Configuration Guidelines, page 7-10
•
NPV and Port Security Configuration Guidelines, page 7-10
•
Connecting an NPIV-enabled Cisco MDS 9396S Multilayer Fabric Switch to an NPV switch
running Cisco MDS NX-OS version earlier than 6.2(13), page 7-10
NPV Guidelines and Requirements
Following are recommended guidelines and requirements when deploying NPV:
•
NPV core switches must support NPIV.
•
You can have up to 105 NPV devices.
•
Nondisruptive upgrades are supported. See the Cisco MDS 9000 Family NX-OS Fundamentals
Configuration Guide.
•
Port tracking is supported. See the Cisco MDS 9000 Family NX-OS Security Configuration Guide.
•
You can configure zoning for end devices that are connected to NPV devices using all available
member types on the NPV core switch. If fWWN, sWWN, domain, or port-based zoning is used,
then fWWN, sWWN or the domain/port of the NPV core switch should be used.
•
Port security is supported on the NPV core switch for devices logged in via NPV.
•
NPV uses a load-balancing algorithm to automatically assign end devices in a VSAN to one of the
NPV core switch links (in the same VSAN) upon initial login. If there are multiple NPV core switch
links in the same VSAN, then you cannot assign a specific one to an end device.
•
Both servers and targets can be connected to an NPV device.
•
Remote SPAN is not supported.
•
Local switching is not supported; all traffic is switched using the NPV core switch.
•
NPV devices can connect to multiple NPIV core switches. In other words, different NP ports can be
connected to different NPIV core switches.
•
NPV supports NPIV-capable servers. This capability is called nested NPIV.
•
Connecting two Cisco NPV switches together is not supported.
•
Only F, NP, and SD ports are supported in NPV mode.
•
In the case of servers that are booted over the SAN with NPV, if an NPV link failover occurs, servers
will lose access to their boot LUN temporarily.
•
NPV switches do not recognize the BB_SCN configuration on the xNP ports because of
interoperability issues with the third-party core switches.
NPV Traffic Management Guidelines
When deploying NPV traffic management, follow these guidelines:
•
Use NPV traffic management only when the automatic traffic engineering by the NPV device is not
sufficient for the network requirements.
•
Do not configure traffic maps for all the servers. For non-configured servers, NPV will use
automatic traffic engineering.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-9
Chapter 7
Configuring N Port Virtualization
Guidelines and Limitations
•
Configure the Persistent FC ID on the core switch. Traffic engineering directs the associated server
interface to external interfaces that lead to the same core switch. The server will be assigned the
same FC ID for every log in. This guideline is not applicable if a 91x4 switch is used as the core
switch.
•
Server interfaces configured to a set of external interfaces cannot use any other available external
interfaces, even if the configured interfaces are not available.
•
Do not configure disruptive load balancing because this involves moving a device from one external
interface to another interface. Moving the device between external interfaces requires NPV relogin
to the core switch through F port leading to traffic disruption.
•
Link a set of servers to a core switch by configuring the server to a set of external interfaces that are
linked to the core switch.
DPVM Configuration Guidelines
When NPV is enabled, the following requirements must be met before you configure DPVM on the NPV
core switch:
•
You must explicitly configure the WWN of the internal FLOGI in DPVM. If DPVM is configured
on the NPV core switch for an end device that is connected to the NPV device, then that end device
must be configured to be in the same VSAN. Logins from a device connected to an NPV device will
fail if the device is configured to be in a different VSAN. To avoid VSAN mismatches, ensure that
the internal FLOGI VSAN matches the port VSAN of the NP port.
•
The first login from an NP port determines the VSAN of that port. If DPVM is configured for this
first login, which is the internal login of the NPV device, then the NPV core switch’s VSAN F port
is located in that VSAN. Otherwise, the port VSAN remains unchanged.
For details about DPVM configuration, see the Cisco MDS 9000 Family NX-OS Fabric Configuration
Guide.
NPV and Port Security Configuration Guidelines
Port security is enabled on the NPV core switch on a per interface basis. To enable port security on the
NPV core switch for devices logging in via NPV, you must adhere to the following requirements:
•
The internal FLOGI must be in the port security database so that, the port on the NPV core switch
will allow communications and links.
•
All of the end device pWWNs must also be in the port security database.
Once these requirements are met, you can enable port security as you would in any other context. For
details about enabling port security, see the Cisco MDS 9000 Family NX-OS Security Configuration
Guide.
Connecting an NPIV-enabled Cisco MDS 9396S Multilayer Fabric Switch to an NPV switch
running Cisco MDS NX-OS version earlier than 6.2(13)
When trunking is enabled on the NPV ports of any MDS switch (released before the Cisco MDS 9396S
Multilayer Fabric Switch) that runs on an MDS NX-OS release earlier than 6.2(13), and you connect an
NPIV enabled Cisco MDS 9396S Multilayer Fabric Switch, use ports fc1/1 through fc1/63.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-10
OL-29284-01, Release 6.x
Chapter 7
Configuring N Port Virtualization
Configuring N Port Virtualization
Note
Trunking failure can occur in both non-portChannel (individual physical NP uplinks) and portChannel
NP uplinks. To avoid trunking failure, ensure that you upgrade the NPV switch to Cisco MDS NX-OS
Release 6.2(13) or later.
Configuring N Port Virtualization
This section includes the following topics:
•
Enabling N Port Identifier Virtualization, page 7-11
•
Configuring NPV, page 7-11
•
Configuring NPV Traffic Management, page 7-13
Enabling N Port Identifier Virtualization
You must globally enable NPIV for all VSANs on the MDS switch to allow the NPIV-enabled
applications to use multiple N port identifiers.
Note
All of the N port identifiers are allocated in the same VSAN.
Detailed Steps
To enable or disable NPIV on the switch, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# feature npiv
Enables NPIV for all VSANs on the switch.
Step 3
switch(config)# no feature npiv
Disables (default) NPIV on the switch.
Configuring NPV
When you enable NPV, the system configuration is erased and the system reboots with the NPV mode
enabled.
Note
We recommend that you save the current configuration either on bootflash or a TFTP server before NPV
(if the configuration is required for later use). Use the following commands to save either your non-NPV
or NPV configuration:
switch# copy running bootflash:filename
The configuration can be reapplied later using the following command:
switch# copy bootflash:filename running-config
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-11
Chapter 7
Configuring N Port Virtualization
Configuring N Port Virtualization
Note
NPV cannot be enabled or disabled from the ASCII configuration file. You can enable or disable only
from the command line.
Detailed Steps
To configure NPV using the CLI, perform the following tasks:
Command
Purpose
Step 1
switch# config t
switch(config)#
On the NPV core switch, enters configuration
mode.
Step 2
switch(config)# feature npiv
switch (config)#
Enables NPIV mode on the NPV core switch.
switch (config)# no feature npiv
Disables NPIV mode on the NPV core switch.
switch(config)# interface fc 2/1
switch(config-if)# switchport mode F
Configures the NPIV core switch port as an F
port.
switch(config-if)# no shutdown
Changes Admin status to bring up the interfaces.
Step 4
switch(config)# vsan database
switch(config-vsan-db)# vsan 8 interface fc
2/1
Configures the port VSANs for the F port on the
NPIV core switch.
Step 5
switch(config)# npv enable
Enables NPV mode on a NPV device (module,
Cisco MDS 9124, Cisco MDS 9134, Cisco MDS
9148 Fabric Switch, Cisco MDS 9148S
Multilayer Fabric Switch Cisco MDS 9250i
Multilayer Fabric Switch, and Cisco MDS
9396S Multilayer Fabric Switch). The module
or switch is rebooted, and when it comes back
up, is in NPV mode.
Step 3
Note
Step 6
switch(config)# interface fc 1/1
switch(config-if)# switchport mode NP
switch(config-if)# no shutdown
A write-erase is performed during the
reboot.
On the NPV device, selects the interfaces that
will be connected to the aggregator switch and
configure them as NP ports.
Changes Admin status to bring up the interfaces.
Step 7
switch(config)# vsan database
switch(config-vsan-db)# vsan 9 interface fc
1/1
Configures the port VSANs for the NP port on
the NPV device.
Step 8
switch(config-if)# exit
Exits interface mode for the port.
Step 9
switch(config)# interface fc 1/2 - 6
switch(config-if)# switchport mode F
switch(config-if)# no shutdown
Selects the remaining interfaces (2 through 6)
on the NPV-enabled device and configures them
as F ports.
Changes Admin status to bring up the interfaces.
Step 10
switch(config)# vsan database
switch(config-vsan-db)# vsan 12 interface fc
1/1 - 6
Configures the port VSANs for the F ports on
the NPV device.
Step 11
switch(config-npv)# no npv enable
switch(config)#
Terminates session and disables NPV mode,
which results in a reload of the NPV device.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-12
OL-29284-01, Release 6.x
Chapter 7
Configuring N Port Virtualization
Configuring N Port Virtualization
Configuring NPV Traffic Management
The NPV traffic management feature is enabled after configuring NPV. Configuring NPV traffic
management involves configuring a list of external interfaces to the servers, and enabling or disabling
disruptive load balancing.
This section includes the following topics:
•
Configuring List of External Interfaces per Server Interface, page 7-13
•
Enabling the Global Policy for Disruptive Load Balancing, page 7-13
Configuring List of External Interfaces per Server Interface
Allows you to configure a list of external FC interfaces per server or range of interfaces by specifying
the external interfaces in the ext-fc-if-range. The servers to be linked are specified in the svr-if-range.
Detailed Steps
To configure the list of external interfaces per server interface, perform the following tasks:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode on the NPV.
Step 2
switch(config)# npv traffic-map
server-interface svr-if-range
external-interface fc ext-fc-if-range
switch (config)#
Allows you to configure a list of external FC
interfaces per server or range of interfaces by
specifying the external interfaces in the
ext-fc-if-range. The servers to be linked are
specified in the svr-if-range.
switch(config)# npv traffic-map
server-interface svr-if-range
external-interface port-channel
ext-pc-if-range
switch (config)#
Allows you to configure a list of external
PortChannel1 interfaces per server or range of
interfaces by specifying the external interfaces
in the ext-fc-if-range. The servers to be linked
are specified in the svr-if-range.
switch(config)# no npv traffic-map
server-interface svr-if-range
external-interface ext-if-range
switch (config)#
Disables the NPV traffic management feature on
the NPV.
1. While mapping non-PortChannel interfaces and PortChannel interfaces to the server interfaces, include them separately in two steps.
Enabling the Global Policy for Disruptive Load Balancing
Disruptive load balancing allows you to review the load on all the external interfaces and balance the
load disruptively. Disruptive load balancing is done by moving the servers using heavily loaded external
interfaces, to the external interfaces running with fewer loads.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-13
Chapter 7
Configuring N Port Virtualization
Verifying NPV Configuration
Detailed Steps
To enable or disable the global policy for disruptive load balancing, perform the following tasks:
Command
Purpose
Step 1
switch# config t
switch(config)#
Enters configuration mode on the NPV.
Step 2
switch(config)# npv auto-load-balance disruptive
switch (config)#
Enables disruptive load balancing on the NPV
core switch.
Step 3
switch (config)# no npv auto-load-balance
disruptive
Disables disruptive load balancing on the NPV
core switch.
Verifying NPV Configuration
This section includes the following topics:
•
Verifying NPV, page 7-14
•
Verifying NPV Traffic Management, page 7-16
To display NPV configuration information, perform one of the following tasks:
Command
Purpose
show fcns database
Displays all the NPV devices in all the VSANs that
the aggregator switch belongs to.
show fcns database detail
Displays additional details such as IP addresses,
switch names, interface names about the NPV
devices.
show npv flogi-table
Displays a list of the NPV devices that are logged
in, along with VSANs, source information,
pWWNs, and FCIDs.
show npv status
Displays the status of the different servers and
external interfaces.
show npv traffic-map
Displays the NPV traffic map.
show npv internal info traffic-map
Displays the NPV internal traffic details.
For detailed information about the fields in the output from these commands, refer to the Cisco MDS
NX-OS Command Reference.
Verifying NPV
To view all the NPV devices in all the VSANs that the aggregator switch belongs to, enter the show fcns
database command.
switch# show fcns database
VSAN 1:
-------------------------------------------------------------------------FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-14
OL-29284-01, Release 6.x
Chapter 7
Configuring N Port Virtualization
Verifying NPV Configuration
-------------------------------------------------------------------------0x010000 N 20:01:00:0d:ec:2f:c1:40 (Cisco) npv
0x010001 N 20:02:00:0d:ec:2f:c1:40 (Cisco) npv
0x010200 N 21:00:00:e0:8b:83:01:a1 (Qlogic) scsi-fcp:init
0x010300 N 21:01:00:e0:8b:32:1a:8b (Qlogic) scsi-fcp:init
Total number of entries = 4
For additional details (such as IP addresses, switch names, interface names) about the NPV devices you
see in the show fcns database output, enter the show fcns database detail command.
switch# show fcns database detail
-----------------------VSAN:1 FCID:0x010000
-----------------------port-wwn (vendor) :20:01:00:0d:ec:2f:c1:40 (Cisco)
node-wwn :20:00:00:0d:ec:2f:c1:40
class :2,3
node-ip-addr :172.20.150.38
ipa :ff ff ff ff ff ff ff ff
fc4-types:fc4_features :npv
symbolic-port-name :para-3:fc1/1
symbolic-node-name :para-3
port-type :N
port-ip-addr :0.0.0.0
fabric-port-wwn :20:01:00:0d:ec:04:99:40
hard-addr :0x000000
permanent-port-wwn (vendor) :20:01:00:0d:ec:2f:c1:40 (Cisco)
connected interface
:port-channel6
switch name (IP address)
:switch (192.0.2.1)
-----------------------VSAN:1 FCID:0x010001
-----------------------port-wwn (vendor) :20:02:00:0d:ec:2f:c1:40 (Cisco)
node-wwn :20:00:00:0d:ec:2f:c1:40
class :2,3
node-ip-addr :172.20.150.38
ipa :ff ff ff ff ff ff ff ff
fc4-types:fc4_features :npv
symbolic-port-name :para-3:fc1/2
symbolic-node-name :para-3
port-type :N
port-ip-addr :0.0.0.0
fabric-port-wwn :20:02:00:0d:ec:04:99:40
hard-addr :0x000000
permanent-port-wwn (vendor) :20:02:00:0d:ec:2f:c1:40 (Cisco)
connected interface
:port-channel6
switch name (IP address)
:switch (192.0.2.1)
If you need to contact support, enter the show tech-support NPV command and save the output so that
support can use it to troubleshoot, if necessary.
To display a list of the NPV devices that are logged in, along with VSANs, source information, pWWNs,
and FCIDs, enter the show npv flogi-table command.
switch# show npv flogi-table
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-15
Chapter 7
Configuring N Port Virtualization
Verifying NPV Configuration
---------------------------------------------------------------------------------------SERVER
VSAN
FCID
PORT NAME
NODE NAME
INTERFACE
EXTERNAL
INTERFACE
---------------------------------------------------------------------------------------fc1/19
1
0xee0008
10:00:00:00:c9:60:e4:9a
20:00:00:00:c9:60:e4:9a
fc1/9
fc1/19
1
0xee0009
20:00:00:00:0a:00:00:01
20:00:00:00:c9:60:e4:9a
fc1/1
fc1/19
1
0xee000a
20:00:00:00:0a:00:00:02
20:00:00:00:c9:60:e4:9a
fc1/9
fc1/19
1
0xee000b
33:33:33:33:33:33:33:33
20:00:00:00:c9:60:e4:9a
fc1/1
Total number of flogi = 4.
To display the status of the different servers and external interfaces, enter the show npv status command.
switch# show npv status
npiv is enabled
External Interfaces:
====================
Interface: fc1/1, VSAN: 2, FCID: 0x1c0000, State: Up
Interface: fc1/2, VSAN: 3, FCID: 0x040000, State: Up
Number of External Interfaces: 2
Server Interfaces:
==================
Interface: fc1/7, VSAN: 2, NPIV: No, State: Up
Interface: fc1/8, VSAN: 3, NPIV: No, State: Up
Number of Server Interfaces: 2
Verifying NPV Traffic Management
To display the NPV traffic map, enter the show npv traffic-map command.
switch# show npv traffic-map
NPV Traffic Map Information:
----------------------------Server-If
External-If(s)
----------------------------fc1/1
fc1/5
-----------------------------
To display the NPV internal traffic details, enter the show npv internal info traffic-map command.
switch# show npv internal info traffic-map
NPV Traffic Map Information:
-----------------------------------------------------------Server-If
Last Change Time
External-If(s)
-----------------------------------------------------------fc1/1
2015-01-15 03:24:16.247856
fc1/5
------------------------------------------------------------
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-16
OL-29284-01, Release 6.x
Chapter 7
Configuring N Port Virtualization
Verifying NPV Configuration
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
7-17
Chapter 7
Configuring N Port Virtualization
Verifying NPV Configuration
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
7-18
OL-29284-01, Release 6.x
CHAPTER
8
Configuring FlexAttach Virtual pWWN
•
Information About FlexAttach Virtual pWWN, page 8-1
•
Guidelines and Limitations, page 8-3
•
Configuring FlexAttach Virtual pWWN, page 8-3
•
Verifying FlexAttach Virtual pWWN Configuration, page 8-5
•
Monitoring FlexAttach Virtual pWWN, page 8-6
Information About FlexAttach Virtual pWWN
This section includes the following topics:
•
FlexAttach Virtual pWWN, page 8-1
•
Difference Between San Device Virtualization and FlexAttach Port Virtualization, page 8-2
•
FlexAttach Virtual pWWN CFS Distribution, page 8-2
•
Security Settings for FlexAttach Virtual pWWN, page 8-3
FlexAttach Virtual pWWN
FlexAttach virtual pWWN feature facilitates server and configuration management. In a SAN
environment, the server installation or replacement, requires interaction and coordination among the
SAN and server administrators. For coordination, it is important that the SAN configuration does not
change when a new server is installed, or when an existing server is replaced. FlexAttach virtual pWWN
minimizes the interaction between the server administrator and the SAN administrator by abstracting the
real pWWN using virtual pWWNs.
When FlexAttach virtual pWWN is enabled on an interface, a virtual pWWN is assigned to the server
interface. The real pWWN is replaced by a virtual pWWN, which is used for a SAN configuration such
as zoning.
Server administrators can benefit from FlexAttach in the following scenarios:
•
Pre-configure—Pre-configure SAN for new servers that are not available physically yet. For
example, they may be on order. FlexAttach can be enabled on the ports designated for the new
servers and use the virtual WWNs assigned for configuring SAN. The new servers are then plugged
into the fabric without any change needed in the SAN.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
8-1
Chapter 8
Configuring FlexAttach Virtual pWWN
Information About FlexAttach Virtual pWWN
•
Replacement to the same port—A failed server can be replaced onto the same port without changing
the SAN. The new server gets a same pWWN as the failed server because the virtual pWWN is
assigned to the port.
•
Replacement to (spare)—A spare server, which is on the same NPV device or a different NPV
device) can be brought online without changes to the SAN. This action is achieved by moving the
virtual port WWN from the current server port to the spare port.
•
Server Mobility—A server can be moved to another port on the same NPV device or another NPV
device without changing the SAN. This is accomplished by moving the virtual pWWN to the new
port. No change is needed if FlexAttach was configured using the physical port WWN of the server
to the virtual port WWN mapping.
Difference Between San Device Virtualization and FlexAttach Port
Virtualization
Table 8-1 describes the difference between SAN device virtualization (SDV) and FlexAttach port
virtualization.
Table 8-1 Difference Between SDV and FlexAttach Virtualization
SAN Device Virtualization (SDV)
FlexAttach Virtualization
Facilitates target and disk management, and only Facilitates server management and has no
facilitates disk and data migration.
restriction on the end devices used.
WWN NAT and Fibre Channel ID (FC-ID) are
allocated on the virtual device, both primary and
secondary.
WWN and Network Address Transport (NAT) is
allocated to host bus adapter (HBA).
FC-ID rewrite on the switch indicates a
rewrite-capable switch on the path.
No rewrite requirements.
Configuration is distributed. This allows
programming rewrites and connectivity
anywhere.
Configuration distribution is not required for any
of the interface-based configurations.
Configuration is secured to device alias.
Does not require device alias for virtual pWWN.
Does not allow automapping to the secondary
device.
Allows automapping to the new HBA. Mapping
process is manual for NPIV.
FlexAttach Virtual pWWN CFS Distribution
The FlexAttach virtual pWWN configuration is distributed for CFS through IPv4, and is enabled by
default. The FlexAttach virtual pWWN distribution, by default, is on CFS region 201. The CFS region
201 links only to the NPV-enabled switches. Other CFS features such as syslog is on region 0. Region 0
will be linked through IPv4 for all NPV switches on the same physical fabric. If CFS has an option to
link through IPv4 or ISL, then CFS will select the ISL path.
Note
NPV switches do not have ISL (E or TE ports) and are linked through IPv4.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
8-2
OL-29284-01, Release 6.x
Chapter 8
Configuring FlexAttach Virtual pWWN
Guidelines and Limitations
Security Settings for FlexAttach Virtual pWWN
Security settings for the FlexAttach virtual pWWN feature are done by port security at the NPV core.
Node WWN of the end device is used to provide physical security.
For more details on enabling port security, refer to the Cisco MDS 9000 Family NX-OS Security
Configuration Guide.
Guidelines and Limitations
Following are recommended guidelines and requirements when deploying FlexAttach virtual pWWN:
•
FlexAttach configuration is supported only on NPV switches.
•
Cisco Fabric Services (CFS) IP version 4 (IPv4) distribution should be enabled.
•
Virtual WWNs should be unique across the fabric.
Configuring FlexAttach Virtual pWWN
This section includes the following topics:
•
Automatically Assigning FlexAttach Virtual pWWN, page 8-3
•
Manually Assigning FlexAttach Virtual pWWN, page 8-4
•
Mapping pWWN to Virtual pWWN, page 8-4
Automatically Assigning FlexAttach Virtual pWWN
Automatic assignment of virtual pWWN can be configured on an NPV switch globally, per VSAN, or
per port. When assigned automatically, a virtual WWN is generated from the device local switch WWN.
Prerequisites
The port must be in a shut state when the virtual pWWN is enabled.
Detailed Steps
To assign a virtual pWWN automatically, perform this task:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch# (config)# flex-attach virtual-pwwn auto
[interface interface-list]
Assigns FlexAttach virtual pWWN
automatically for the interfaces.
switch# (config)# flex-attach virtual-pwwn auto
[vsan vsan-range]
Assigns FlexAttach virtual pWWN
automatically for the VSANs.
switch# (config)# flex-attach commit
Commits the configuration.
Step 3
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
8-3
Chapter 8
Configuring FlexAttach Virtual pWWN
Configuring FlexAttach Virtual pWWN
Troubleshooting Tips
•
When the interface-list value is not included in the command, virtual pWWN is assigned globally.
•
All the interfaces mentioned in the interface-list value must be in a shut state.
Manually Assigning FlexAttach Virtual pWWN
You can manually assign a WWN to the interface, without generating it through the switch. Several
checks are done by the NPV core to ensure the uniqueness of virtual pWWNs in the switch. When
duplicate virtual pWWNs are configured, the subsequent logins are rejected by the NPV core switch.
Prerequisites
•
Some ports may be in automode, some in manual mode, and the virtual pWWNs need not be
assigned.
•
The port must be in a shut state when a virtual pWWN is assigned.
•
The interface mentioned in the interface value must be in a shut state.
Restrictions
Detailed Steps
To assign virtual pWWN manually, perform this task:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch# (config)# flex-attach virtual-pwwn
vpwwn interface interface
Configures the FlexAttach virtual pWWN for
the interface.
switch# (config)# flex-attach virtual-pwwn
vpwwn interface interface [vsan vsan]
Configures the FlexAttach virtual pWWN for
the interface in the VSAN.
switch# (config)# flex-attach commit
Commits the configuration.
Step 3
Mapping pWWN to Virtual pWWN
You can configure virtual pWWNs through real pWWNs. This process is required for NPIV hosts
containing multiple pWWNs, of which only FLOGI is mapped to the virtual pWWN. Subsequent
FDSIDs will have different mappings.
Several checks are done by the NPV core to ensure the uniqueness of virtual pWWNs in the switch across
the NPV switches. When duplicate virtual pWWNs are configured, the subsequent logins are rejected by
the NPV core switch.
Prerequisites
•
The interface must be in a shut state and the specified virtual pWWN should not be logged in.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
8-4
OL-29284-01, Release 6.x
Chapter 8
Configuring FlexAttach Virtual pWWN
Verifying FlexAttach Virtual pWWN Configuration
Restrictions
•
The specified virtual pWWN and the real pWWN must not be logged in.
Detailed Steps
To map pWWN to virtual pWWN, perform this task:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch# (config)# flex-attach virtual-pwwn
vpwwn pwwn pwwn
Maps the pWWN to the virtual pWWN.
Step 3
switch# (config)# flex-attach commit
Commits the configuration.
Verifying FlexAttach Virtual pWWN Configuration
To display FlexAttach configuration information, perform one of the following tasks:
Command
Purpose
show flex-attach virtual-pwwn
Displays the type and value of virtual pWWNs.
show fcns database
Displays if the end device is logged with the
correct virtual WWNs.
For detailed information about the fields in the output from these commands, refer to the Cisco MDS
NX-OS Command Reference.
To view and confirm that the type and value of virtual pWWNs are correct, enter the show flex-attach
virtual-pwwn command. (See Example 8-1.)
Example 8-1
Displaying the Type and Value of Virtual pWWNs
switch# show flex-attach virtual-pwwn
VIRTUAL PORT WWNS ASSIGNED TO INTERFACES
---------------------------------------------------------------------VSAN
INTERFACE VIRTUAL-PWWN
AUTO
LAST-CHANGE
---------------------------------------------------------------------1
fc1/1
00:00:00:00:00:00:00:00
1
fc1/2
22:73:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/3
22:5e:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/4
22:5f:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/5
22:74:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:26:24 2008
1
fc1/6
22:60:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/7
22:61:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/8
22:62:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/9
22:63:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/10
22:64:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/11
22:65:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
1
fc1/12
22:66:00:05:30:01:6e:1e
TRUE
Thu Jan 31 01:58:52 2008
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
8-5
Chapter 8
Configuring FlexAttach Virtual pWWN
Monitoring FlexAttach Virtual pWWN
Verifying the End Device
To verify that the end device is logged with the correct virtual WWNs, use the show fcns database
command on the NPV core. (See Example 8-2.)
Example 8-2
Verifying the End Device
switch# show fcns database
VSAN 1:
-------------------------------------------------------------------------FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE
-------------------------------------------------------------------------0x010000 N 20:01:00:0d:ec:2f:c1:40 (Cisco) npv
0x010001 N 20:02:00:0d:ec:2f:c1:40 (Cisco) npv
0x010200 N 21:00:00:e0:8b:83:01:a1 (Qlogic) scsi-fcp:init
0x010300 N 21:01:00:e0:8b:32:1a:8b (Qlogic) scsi-fcp:init
Total number of entries = 4
Monitoring FlexAttach Virtual pWWN
Table 8-2 lists the errors that might be displayed and provides the workarounds.
Table 8-2 FlexAttach Errors and Workarounds
Error
Description
Workaround
fc1/1 : interface is not FlexAttach configuration To move the port to the shut state, enable the
FlexAttach configuration, and then move the port
down
fails because the
to no shut state.
configuration is enabled
for an active interface with
the operation state as up.
FlexAttach
The FlexAttach
configuration is not
configuration on one peer
distributed to the peers NPV is not available to any
other peer NPV.
FlexAttach configuration will not be distributed
if cfs ipv4 distribute, or cfs ipv6 distribute is
disabled. Enable cfs ipv4 distribute, or cfs ipv6
distribute.
Even with CFS
distribution enabled
Inagua does not
become a peer with
other NPVs
CFS over IP is enabled,
and the Inagua in one
blade center is not the peer
NPV for other NPVs.
CFS over IP uses IP multicast to discover the
NPV peers in the network. IBM MM does not
support multicast and cannot act as a peer with
NPV. This prevents the FlexAttach configuration
from getting distributed to other peer NPVs in the
network.
NP port uses physical
pWWN instead of
virtual pWWN
configured through
FlexAttach
This occurs when NP port
uses physical pWWN
instead of virtual pWWN,
that is configured through
FlexAttach.
FlexAttach is supported on server interfaces such
as F ports, and not on external interfaces such as
NP ports.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
8-6
OL-29284-01, Release 6.x
Chapter 8
Configuring FlexAttach Virtual pWWN
Monitoring FlexAttach Virtual pWWN
Error
Description
Workaround
real port WWN and
virtual WWN cannot
be same
This occurs when you try
to configure FlexAttach
with a similar value for
pWWN and virtual
pWWN.
Use different values for pWWN and virtual
pWWN, as similar values for pWWN and virtual
pWWn are not allowed.
Virtual port WWN
already exists
This occurs when you try
to configure an already
defined pWWN to a
different interface.
Use an undefined virtual pWWN for a new
interface.
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
8-7
Chapter 8
Configuring FlexAttach Virtual pWWN
Monitoring FlexAttach Virtual pWWN
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
8-8
OL-29284-01, Release 6.x
CHAPTER
9
Configuring Port Tracking
The port tracking feature is unique to the Cisco MDS 9000 Family of switches. This feature uses
information about the operational state of the link to initiate a failure in the link that connects the edge
device. This process of converting the indirect failure to a direct failure triggers a faster recovery process
towards redundant links. When enabled, the port tracking feature brings down the configured links based
on the failed link and forces the traffic to be redirected to another redundant link.
This chapter includes the following sections:
•
Information About Port Tracking, page 9-1
•
Guidelines and Limitations, page 9-2
•
Default Settings, page 9-2
•
Configuring Port Tracking, page 9-3Configuring Port Tracking, page 9-3
•
Displaying Port Tracking Information, page 9-6
Information About Port Tracking
Generally, hosts can instantly recover from a link failure on a link that is immediately (direct link)
connected to a switch. However, recovering from an indirect link failure between switches in a WAN or
MAN fabric with a keep-alive mechanism is dependent on several factors such as the time out values
(TOVs) and on registered state change notification (RSCN) information.
In Figure 9-1, when the direct link 1 to the host fails, recovery can be immediate. However, when the
ISL 2 fails between the two switches, recovery depends on TOVs, RSCNs, and other factors.
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
OL-29284-01
9-1
Chapter 9
Configuring Port Tracking
Guidelines and Limitations
Figure 9-1
Traffic Recovery Using Port Tracking
ISL2
X
Direct link 1
WAN or
MAN
X
FC
FC
120490
WAN or
MAN
The port tracking feature monitors and detects failures that cause topology changes and brings down the
links connecting the attached devices. When you enable this feature and explicitly configure the linked
and tracked ports, the Cisco NX-OS software monitors the tracked ports and alters the operational state
of the linked ports on detecting a link state change.
The following terms are used in this chapter:
•
Tracked ports—A port whose operational state is continuously monitored. The operational state of
the tracked port is used to alter the operational state of one or more ports. Fibre Channel, VSAN,
PortChannel, FCIP, or a Gigabit Ethernet port can be tracked. Generally, ports in E and TE port
modes can also be Fx ports.
•
Linked ports—A port whose operational state is altered based on the operational state of the tracked
ports. Only a Fibre Channel port can be linked.
Guidelines and Limitations
Before configuring port tracking, consider the following guidelines:
•
Verify that the tracked ports and the linked ports are on the same Cisco MDS switch.
•
Do not track a linked port back to itself (for example, Port fc1/2 to Port fc2/5 and back to Port fc1/2)
to avoid recursive dependency.
•
Be aware that the linked port is automatically brought down when the tracked port goes down. Be
aware that the linked port is automatically brought down when the tracked port goes down.
Default Settings
Table 9-1 lists the default settings for port tracking parameters.
Table 9-1
Default Port Tracking Parameters
Parameters
Default
Port tracking
Disabled.
Operational binding
Enabled along with port tracking.
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
9-2
OL-29284-01
Chapter 9
Configuring Port Tracking
Configuring Port Tracking
Configuring Port Tracking
Port tracking has the following features:
•
The application brings the linked port down when the tracked port goes down. When the tracked port
recovers from the failure and comes back up again, the tracked port is also brought up automatically
(unless otherwise configured).
•
You can forcefully continue to keep the linked port down, even though the tracked port comes back
up. In this case, you must explicitly bring the port up when required.
This section includes the following topics:
•
Enabling Port Tracking, page 9-3
•
Information About Configuring Linked Ports, page 9-3
•
Binding a Tracked Port Operationally, page 9-4
•
Information About Tracking Multiple Ports, page 9-4
•
Tracking Multiple Ports, page 9-5
•
Information About Monitoring Ports in a VSAN, page 9-5
•
Monitoring Ports in a VSAN, page 9-5
•
Information AboutForceful Shutdown, page 9-6
•
Forcefully Shutting Down a Tracked Port, page 9-6
Enabling Port Tracking
The port tracking feature is disabled by default in all switches in the Cisco 9000 Family. When you
enable this feature, port tracking is globally enabled for the entire switch.
To configure port tracking, enable the port tracking feature and configure the linked port(s) for the
tracked port.
Detailed Steps
To enable port tracking, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# feature port-track
Enables port tracking.
switch(config)# no feature
port-track
Removes the currently applied port tracking configuration
and disables port tracking.
Information About Configuring Linked Ports
You can link ports using one of two methods:
•
Operationally binding the linked port(s) to the tracked port (default).
•
Continuing to keep the linked port down forcefully—even if the tracked port has recovered from the
link failure.
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
OL-29284-01
9-3
Chapter 9
Configuring Port Tracking
Configuring Port Tracking
Binding a Tracked Port Operationally
When you configure the first tracked port, operational binding is automatically in effect. When you use
this method, you have the option to monitor multiple ports or monitor ports in one VSAN.
Detailed Steps
To operationally bind a tracked port, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# interface fc8/6
switch(config-if)#
Configures the specified interface and enters the interface
configuration submode. You can now configure tracked ports.
Note
Step 3
switch(config-if)# port-track
interface port-channel 1
Tracks interface fc8/6 with interface port-channel 1. When
port-channel 1 goes down, interface fc8/6 is also brought down.
Note
switch(config-if)# no port-track
interface port-channel 1
This link symbolizes the direct link (1) in Figure 9-1.
This link symbolizes the ISL (2) in Figure 9-1.
Removes the port tracking configuration that is currently
applied to interface fc8/6.
Information About Tracking Multiple Ports
You can control the operational state of the linked port based on the operational states of multiple tracked
ports. When more than one tracked port is associated with a linked port, the operational state of the
linked port will be set to down only if all the associated tracked ports are down. Even if one tracked port
is up, the linked port will stay up.
In Figure 9-2, only if both ISLs 2 and 3 fail, will the direct link 1 be brought down. Direct link 1 will
not be brought down if either 2 or 3 are still functioning as desired.
Figure 9-2
Traffic Recovery Using Port Tracking
Port Channel
2
fc 8/6
1
FC
X
X
WAN or
MAN
FCIP
3
FC
WAN or
MAN
120491
X
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
9-4
OL-29284-01
Chapter 9
Configuring Port Tracking
Configuring Port Tracking
Tracking Multiple Ports
Detailed Steps
To track multiple ports, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# interface fc8/6
Configures the specified interface and enters the interface
configuration submode. You can now configure tracked ports.
Note
Step 3
switch(config-if)# port-track
interface port-channel 1
Tracks interface fc8/6 with interface port-channel 1. When
port-channel 1 goes down, interface fc8/6 is also brought down.
Note
Step 4
switch(config-if)# port-track
interface fcip 5
This link symbolizes the direct link (1) in Figure 9-2.
This link symbolizes the ISL (2) in Figure 9-2.
Tracks interface fc8/6 with interface fcip 5. When FCIP 5 goes
down, interface fc8/6 is also brought down.
Note
This link symbolizes the ISL (3) in Figure 9-2.
Information About Monitoring Ports in a VSAN
You can optionally configure one VSAN from the set of all operational VSANs on the tracked port with
the linked port by specifying the required VSAN. This level of flexibility provides higher granularity in
tracked ports. In some cases, when a tracked port is a TE port, the set of operational VSANs on the port
can change dynamically without bringing down the operational state of the port. In such cases, the port
VSAN of the linked port can be monitored on the set of operational VSANs on the tracked port.
If you configure this feature, the linked port is up only when the VSAN is up on the tracked port.
Tip
The specified VSAN does not have to be the same as the port VSAN of the linked port.
Monitoring Ports in a VSAN
Detailed Steps
To monitor a tracked port in a specific VSAN, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# interface fc8/6
Configures the specified interface and enters the interface
configuration submode. You can now configure tracked ports.
Step 3
switch(config-if)# port-track
interface port-channel 1 vsan 2
Enables tracking of the PortChannel in VSAN 2.
switch(config-if)# no port-track
interface port-channel 1 vsan 2
Removes the VSAN association for the linked port. The
PortChannel link remains in effect.
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
OL-29284-01
9-5
Chapter 9
Configuring Port Tracking
Displaying Port Tracking Information
Information AboutForceful Shutdown
If a tracked port flaps frequently, then tracking ports using the operational binding feature may cause
frequent topology change. In this case, you may choose to keep the port in the down state until you are
able to resolve the reason for these frequent flaps. Keeping the flapping port in the down state forces the
traffic to flow through the redundant path until the primary tracked port problems are resolved. When
the problems are resolved and the tracked port is back up, you can explicitly enable the interface.
Tip
If you configure this feature, the linked port continues to remain in the shutdown state even after the
tracked port comes back up. You must explicitly remove the forced shut state (by administratively
bringing up this interface) of the linked port once the tracked port is up and stable.
Forcefully Shutting Down a Tracked Port
Detailed Steps
To forcefully shut down a tracked port, follow these steps:
Command
Purpose
Step 1
switch# config t
Enters configuration mode.
Step 2
switch(config)# interface fc1/5
Configures the specified interface and enters the
interface configuration submode. You can now
configure tracked ports.
Step 3
switch(config-if)# port-track force-shut
Forcefully shuts down the tracked port.
switch(config-if)# no port-track force-shut
Removes the port shutdown configuration for the
tracked port.
Displaying Port Tracking Information
Examples
The show commands display the current port tracking settings for the Cisco MDS switch (see Examples
9-1 to 9-4).
Example 9-1
Displays the Linked and Tracked Port Configuration
switch# show interface
...
fc8/6 is down (All tracked ports down)
<-------------------------------------Linked port
Hardware is Fibre Channel, FCOT is short wave laser
Port WWN is 21:c6:00:05:30:00:37:1e
Admin port mode is auto, trunk mode is on
Port vsan is 1
Receive data field Size is 2112
Beacon is turned off
Port tracked with interface port-channel 1 vsan 2 (trunking) <-----Tracked port
Port tracked with interface fcip 5 <------------------------------------------Tracked port
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
9-6
OL-29284-01
Chapter 9
Configuring Port Tracking
Displaying Port Tracking Information
269946 frames input, 22335204 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
205007 frames output, 10250904 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
2 output OLS, 2 LRR, 0 NOS, 1 loop inits
0 receive B2B credit remaining
0 transmit B2B credit remaining
...
Example 9-2
Displays a Tracked Port Configuration for a Fibre Channel Interface
switch# show interface fc1/1
fc1/1 is down (Administratively down)
Hardware is Fibre Channel, FCOT is short wave laser w/o OFC (SN)
Port WWN is 20:01:00:05:30:00:0d:de
Admin port mode is FX
Port vsan is 1
Receive data field Size is 2112
Beacon is turned off
Port tracked with interface fc1/2 (down)
Port tracked with interface port-channel 1 vsan 2 (down)
Port tracked with interface fcip1 (down)
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
1 frames input, 128 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
1 frames output, 128 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
0 output OLS, 0 LRR, 0 NOS, 0 loop inits
0 receive B2B credit remaining
0 transmit B2B credit remaining
Example 9-3
Displays a Tracked Port Configuration for a PortChannel Interface
switch# show interface port-channel 1
port-channel 1 is down (No operational members)
Hardware is Fibre Channel
Port WWN is 24:01:00:05:30:00:0d:de
Admin port mode is auto, trunk mode is on
Port vsan is 2
Linked to 1 port(s)
Port linked to interface fc1/1
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
0 frames input, 0 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
0 frames output, 0 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
0 output OLS, 0 LRR, 0 NOS, 0 loop inits
No members
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
OL-29284-01
9-7
Chapter 9
Example 9-4
Configuring Port Tracking
Displays a Forced Shutdown Configuration
switch# show interface fc 1/5
fc1/5 is up
Hardware is Fibre Channel, FCOT is short wave laser
Port WWN is 20:05:00:05:30:00:47:9e
Admin port mode is F
Port mode is F, FCID is 0x710005
Port vsan is 1
Speed is 1 Gbps
Transmit B2B Credit is 64
Receive B2B Credit is 16
Receive data field Size is 2112
Beacon is turned off
Port track mode is force_shut <--this port remains shut
even if the tracked port is back up
Cisco MDS 9000 Family NX-OS Quality of Service Configuration Guide
9-8
OL-29284-01
INDEX
example configurations
Numerics
See also switching modules
12-port 4-Gbps switching modules
4-port 10-Gbps switching modules
4-14
BB_credit buffers
configuration guidelines
default settings
2-47, 4-8
4-16
BB_credit buffers
2-27
2-28
configuration guidelines
2-24
default settings
See also switching modules
2-24
See also switching modules
16-port switching modules
configuring BB_credits
4-1
24-port 4-Gbps switching modules
bandwidth fairness
default settings
2-17
A
ACL adjacency sharing
2-24
example configurations
disabling for downgrading
2-48, 2-51, 4-13
administrative speeds
oversubscription
2-11
configuring
shared resources
2-11
administrative states
See also switching modules
2-25
example configurations
setting
4-9
3-31
auto port mode
4-1
2-25
example configurations
3-30
configuring
3-11
description
4/44-port 8-Gbps switching modules
default settings
3-11
auto mode
32-port switching modules
configuring BB_credits
3-17, 3-34
description
24-port 8-Gbps switching modules
default settings
2-44
interface configuration
3-8
autosensing speed
4-10
Generation 2 switching modules
3-17
48-port 4-Gbps switching modules
bandwidth fairness
default settings
2-17
B
2-24
example configurations
2-49, 2-50, 4-11
bandwidth fairness
oversubscription
2-11
disabling
2-42
shared resources
2-11
enabling
2-42
See also switching modules
48-port 8-Gbps switching modules
default settings
BB_credit buffers
12-port 4-Gbps switching module allocations
4-14
2-25
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
IN-1
Index
12-port 4-Gbps switching module
considerations 4-15
C
24-port 4-Gbps switching module allocations
4-13
24-port 4-Gbps switching module
considerations 4-13, 4-14
Cisco MDS 9216i switches
configuring extended BB_credits
24-port 8-Gbps switching module considerations
4-9
4/44-port 8-Gbps switching module
considerations 4-10
48-port 8-Gbps switching module considerations
4-port 10-Gbps switching module allocations
Configuring Congestion Frame Timeout Value for
FC 3-47
4-8
4-16
D
dedicated rate mode
migrating from shared rate mode
4-6, 4-7, 4-8, 4-9, 4-10, 4-11,
4-17, 4-18
migrating to shared rate mode
2-26
2-27
destination IDs
BB_credits
configuring
4-23
description
4-1
exchange based
domain IDs
domain manager
4-22, 4-25, 4-26
isolation
beacon modes
3-14
downgrading
3-35
configuring
3-14
assignment failures
BB_SC
enabling
6-5
6-4
flow based
3-13
reason codes
disabling ACL adjacency sharing
bit errors
reasons
2-7
description
4-port 10-Gbps switching module
considerations 4-16, 4-18
allocation defaults (table)
7-11
configuring NPV
48-port 4-Gbps switching module
considerations 4-11
4-20
2-44
dynamic bandwidth management
3-18
configuring
3-18
description
3-18
2-10
description
bit error thresholds
E
B port mode
description
EISLs
3-11
interface modes
PortChannel links
3-11
6-2
enhanced ISLs. See EISLs
bridge port mode. See B port mode
E port mode
buffer pools
Generation 2 switching modules
4-2
buffer-to-buffer credits. See BB_credits
buffer-to-buffer start change. See BB_SC
classes of service
3-9
3-9
description
E ports
32-port switching module configuration
guidelines 6-11
configuring
isolation
3-30, 3-31
3-14
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
IN-2
OL-29284-01, Release 6.x
Index
3-12
exchange link parameter. See ELP
reason codes
expansion port mode. See E port mode
states
extended BB_credits
taking out of service on Generation 2 switching
modules 2-42
configuring
4-25
troubleshooting operational states
4-27
displaying information
3-13
3-11
See also interfaces
4-20
Generation 2 switching modules
FL port mode
4-20
licensing
3-11
3-9
classes of service
description
F
3-9
FL ports
fabric loop port mode. See FL port mode
fabric port mode. See F port mode
3-9
configuring system default port mode f
description
4-1
configuring
3-29
configuring auto port mode
configuring bit error thresholds
default settings
3-9
3-17
Fx ports
3-17
3-30, 3-31
configuring receive data field sizes
configuring speeds
description
configuring
3-18
configuring frame encapsulation
configuring port modes
3-30, 3-31
frame encapsulation
3-34
configuring descriptions
configuring
See also Fx ports
3-31
3-35
configuring beacon modes
3-9
F ports
3-11
administrative states
3-9
classes of service
3-31
Fibre Channel interfaces
BB_credits
3-14
F port mode
Fiber Channel interfaces
configuring
3-30, 3-31
description
3-10
interface modes
4-23
3-10
3-10
See also F ports; FL ports
3-17, 3-34
2-24, 3-6, 5-11, 6-13
deleting from PortChannels
6-9
G
3-30
displaying capabilities on Generation 2 switching
modules 2-45
3-52 to 3-59
displaying information
3-30
graceful shutdown
modes
description
See also Fx ports
3-14
overlap isolation
enabling
3-30, 3-31
nonparticipating code
fcdomains
disabling
configuring
performance buffers
extended BB_credits
port index allocations
4-20
2-19
Generation 2 switching modules
3-14
3-7 to 3-11
operational states
Generation 1 switching modules
3-11
4-2
buffer groups
4-2 to 4-16
configuring port speeds
2-28, 2-30
configuring rate modes
2-32
default settings
2-24
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
IN-3
Index
description
2-1 to 2-3
link failures
disabling ACL adjacency sharing
2-44
2-43
displaying port resources
load balancing
dynamic bandwidth management
extended BB_credits
4-20, 4-21
interface capabilities
2-45
2-10
2-19
port index allocations
management interfaces
2-5
port rate modes
configuring
recovering from powered-down state
2-21
Generation 3 switching modules
3-66
3-28
features
mgmt0 interfaces
2-25
default settings
2-24, 3-6, 5-11, 6-13
displaying information
2-42
taking interfaces out of service
3-28, 3-43
default settings
2-44
releasing shared resources
6-2
M
2-3
port groups
6-4
description
PortChannels
2-11
out-of-service interfaces
9-1
recovering
configuring
3-28, 3-43
2-24, 3-6, 5-11, 6-13
default settings
3-28
features
I
MPS-14/2 modules
configuring extended BB_credits
indirect link failures
recovering
4-20
9-1
interfaces
6-8, 6-15, 6-23
adding to PortChannels
configuring descriptions
3-34
deleting from PortChannels
6-9
forced addition to PortChannels
6-9, 6-16
6-9
suspended states
3-14
description
NPIV
7-2
description
6-9
7-11
enabling
IPv4 default gateways
NP links
configuring mgmt0 interfaces
3-11
interface modes
nonparticipating codes
3-52 to 3-59
displaying information
isolated states
NL ports
2-24, 3-6, 5-11, 6-13
default settings
N
3-28
ISLs
7-5
N port identifier virtualization. See NPIV
NL ports
PortChannel links
6-2
See also Nx ports
NP-ports
7-5
NPV, configuring
L
NPV mode
7-11
7-4
licenses
extended BB_credits
4-20
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
IN-4
OL-29284-01, Release 6.x
Index
4-20
O
assigning extended BB_credits
operational states
Generation 2 Fibre Channel switching modules
2-3
Generation 3 Fibre Channel switching modules
2-8,
2-3
description
configuring on Fibre Channel interfaces
3-30, 3-31
2-9
3-11
description
port indexes
out-of-service interfaces
port modes
oversubscription
3-11
disabling restrictions
2-38
auto
enabling restrictions
2-41
description
Generation 2 switching modules
ratios
2-19
description
2-11
description
3-7 to 3-11
port rate modes
2-11
2-32
configuring
2-11
2-7
dedicated
2-5
description
P
2-8
oversubscribed
shared
performance buffers
configuring
4-24
description
4-2
See also rate modes
port speeds
configuring on Generation 2 switching module
interfaces 2-28, 2-30
6-6
PortChannel Protocol
displaying configuration
6-9
description
comparison with trunking
examples
enabling
guidelines
6-14
9-6
9-3
9-2
monitoring ports in a VSAN
6-13
multiple ports
6-7
deleting interfaces
9-1
displaying information
6-3
6-8
compatibility checks
deleting
description
3-14
administratively down
9-2
default settings
6-8, 6-15, 6-23
adding interfaces
default settings
2-29, 2-31
port tracking
PortChannels
creating
3-17, 3-34
configuring
PortChannel modes
description
2-8
9-4
shutting down ports forcefully
6-9
9-5
9-6
6-2
forcing interface additions
Generation 2 switching module interfaces
interface states
6-9
load balancing
6-4
2-22
rate modes
configuring on Generation 2 switching module
interfaces 2-32
misconfiguration error detection
verifying configurations
R
6-9, 6-16
6-10
6-17 to 6-19
verifying configuration
2-32
See also port rate modes
port groups
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
IN-5
Index
reason codes
description
T
3-12
TE port mode
receive buffer groups. See buffer groups
configuring
3-9
classes of service
receive data field sizes
3-9
description
4-23
TE ports
recovery
from powered-down state
5-3
trunking restrictions
2-21
TF port mode
3-10
classes of service
S
3-10
description
TL ports
SD port mode
description
3-10
interface modes
3-10
configuring
3-30, 3-31
description
3-19
tracked ports
SD ports
configuring
9-4
binding operationally
3-30, 3-31
trunk-allowed VSAN lists
SFPs
transmitter types
trunking
shared rate mode
description
5-5 to 5-7
description
3-19
migrating from dedicated rate mode
migrating to dedicated rate mode
oversubscription
2-27
2-26
configuration guidelines
configuring modes
merging traffic
flow based
5-5
5-10
5-3, 5-7
restrictions
6-5
5-10
5-5
link state
2-11
source IDs
exchange based
6-3
comparison with PortChannels
2-8
trunking E port mode. See TE port mode
6-4
SPAN destination port mode. See SD port mode
trunking F port mode. See TF port mode
SPAN tunnel port mode. See ST port mode
trunking protocol
default settings
ST port mode
description
interface modes
limitations
default state
3-10
ST ports
disabling
5-11, 5-12
enabling
5-11, 5-12
5-10
trunk mode
3-10
configuring
subnet masks
configuring mgmt0 interfaces
5-4
detecting port isolation
3-10
3-10
interface modes
5-11
status
3-28
5-5, 5-12
5-5
trunk ports
switch ports
configuring attribute default values
3-36
displaying information
5-14
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
IN-6
OL-29284-01, Release 6.x
Index
V
VSAN IDs
5-11
allowed list
multiplexing traffic
3-9, 3-10
VSAN interfaces
creating
3-44
description
3-28
displaying information
3-66, 3-67
VSANs
allowed-active
5-7
configuring allowed-active lists
configuring trunk-allowed lists
mismatches
3-14
port tracking
9-5
TE port mode
3-9
TF port mode
3-10
trunk-allowed
5-7
5-12
5-5 to ??
VSAN trunking. See trunking
W
WWNs
suspended connections
3-14
Z
zones
merge failures
3-14
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
OL-29284-01, Release 6.x
IN-7
Index
Cisco MDS 9000 Family NX-OS Interfaces Configuration Guide
IN-8
OL-29284-01, Release 6.x
Download PDF
Similar pages