Cisco Nexus 4001I Switch Module for IBM BladeCenter

Product Guide

The Cisco Nexus 4001I Switch Module is a blade switch solution for the BladeCenter H and HT chassis providing the server I/O solution required for high-performance, scale-out, virtualized, and non-virtualized x86 computing architectures. It is a line rate, extremely low-latency, non-blocking, Layer 2, 10 Gigabit Ethernet blade switch that is fully compliant with Fibre Channel over Ethernet (FCoE) and IEEE Data Center Bridging standards.

The Cisco Nexus 4001I enables a standards-based, high-performance Unified Fabric running over 10 Gigabit Ethernet in the blade server environment. This Unified Fabric enables consolidation of LAN traffic, storage traffic (IP-based such as iSCSI, NAS, and Fibre Channel SAN), and high-performance computing (HPC) traffic over a single 10 Gigabit Ethernet server network.

This offering works with BladeCenter Open Fabric Manager, providing all the benefits of I/O virtualization at 10 Gbps speed. Figure 1 shows the switch module.

Figure 1. Cisco Nexus 4001I Switch Module for IBM BladeCenter

Did you know?

Clients considering Fibre Channel over Ethernet as a fabric consolidation solution can implement a 10 Gigabit Ethernet infrastructure as the basis of the solution. The solution uses FCoE between the converged network adapter (CNA) in each server and a top-of-rack (TOR) switch such as the Cisco Nexus 5000 Series, with the Cisco Nexus 4001I blade switch at the access layer between them.

With this solution, clients gain most of the cost savings from the use of fewer adapters, cables, and switch ports. The

FCoE connection between the CNA and the 4001I and 5000 Series switches will carry both Fibre Channel and Ethernet traffic on a single link. The Cisco Nexus 5000 Series switch will then separate LAN and Ethernet traffic to the Cisco

Nexus 7000 Series switch upstream and SAN traffic to the Cisco MDS 9000 Family switch upstream.

Check for Updates

Cisco Nexus 4001I Switch Module for IBM BladeCenter 1

Part number information

Table 1 shows the part numbers for ordering these modules and additional options for them.

Table 1. IBM part numbers and feature codes for ordering

Description

Cisco Nexus 4001I Switch Module

Software Upgrade License for Cisco Nexus 4001I

IBM part number

46M6071

49Y9983

IBM feature code

0072

1744

Cisco part number

N4K-4001I-XPX

N4K-4001I-SSK9

The module part numbers include the following items:

One Cisco Nexus 4001I Switch Module

Cisco Console Cable RJ45-to-DB9

One filler panel

Important Notices document

Documentation CD-ROM

Software Upgrade License for Cisco Nexus 4001I

The Cisco Nexus 4001I Switch Module is designed to support both 10 Gb Ethernet and Fibre Channel over Ethernet.

Software Upgrade License for Cisco Nexus 4001I, part number 49Y9983, enables the switch to work in FCoE mode.

When connected to a converged adapter in the server, this switch can route CEE packets to an upstream FCoE switch, which can then route the packets to the LAN or SAN.

SFP+ Transceivers and Copper Cables

The Cisco Nexus 4001I Switch Module does not ship with either SFP+ Transceivers or SFP+ Copper Cables. These can be ordered directly from Cisco Systems or authorized Cisco Systems resellers as listed in Table 2. Certain transceivers can also be ordered directly from IBM.

Table 2. SFP+ Transceivers and Copper Cables

Description

Cisco 10GBASE-SR SFP+ Transceiver

Cisco 10GBASE-LR SFP+ Transceiver

Cisco 10GBASE-CU SFP+ Cable 1 Meter

Cisco 10GBASE-CU SFP+ Cable 3 Meter

Cisco 10GBASE-CU SFP+ Cable 5 Meter

Cisco 1000BASE-T SFP Transceiver

Cisco 1000BASE-SX SFP Transceiver (GE SFP, LC connector)

Cisco GE SFP, LC connector LX/LH transceiver

IBM part number

88Y6054

Not available

Not available

Not available

Not available

88Y6058

88Y6062

Not available

Cisco part number

SFP-10G-SR(=)

SFP-10G-LR(=)

SFP-H10GB-CU1M(=)

SFP-H10GB-CU3M(=)

SFP-H10GB-CU5M(=)

GLC-T(=)

GLC-SX-MM(=)

GLC-LH-SM(=)

The Cisco 10GBASE-SR SFP+ Transceiver (88Y6054) supports a link length of 26 meters on standard Fiber Distributed

Data Interface (FDDI)-grade multimode fiber (MMF).

The Cisco 1000BASE-T SFP Transceiver (88Y6058) operates on standard Category 5 unshielded twisted pair copper cabling of up to 100 meters (328 ft) link length. Cisco 1000BASE-T SFP modules support 10/100/1000 autonegotiation and Auto MDI/MDIX.

The Cisco 1000BASE-SX SFP Transceiver (88Y6062) is compatible with the IEEE 802.3z 1000BASE-SX standard and operates on 50 µm multimode fiber links up to 550 m and on 62.5 µm Fiber Distributed Data Interface (FDDI)-grade multimode fibers up to 220 m. It can support up to 1km over laser-optimized 50 µm multimode fiber cable.

Cisco Nexus 4001I Switch Module for IBM BladeCenter 2

Benefits

The benefits of the Cisco Nexus 4001I Switch Module include the following:

Lower total cost of operation (TCO): Deployment of Unified Fabric with the Nexus 4001I on the blade server access leverages a significant reduction in the number of switches, network interface cards (LAN and SAN), ports, optic modules, and cables. This consolidation of server access network elements significantly reduces the overall capital and operation costs of the data center network through the reduction of network elements to purchase, manage, power, and cool.

High performance: Nexus 4001I is a line rate, feature-rich, extremely low-latency switch capable of enabling server access migration from 1 GbE to 10 GbE to lossless 10 GbE, as well as, supporting the demanding latency requirements of High Performance Compute clusters or high-frequency trading applications.

Enhanced server virtualization: Utilizing Unified Fabric on the server access with Nexus 4001I provides uniform interfaces, simplified cabling, and consistent server access design required to leverage the advantages of automated virtual machine mobility. Using the Nexus 4001I in conjunction with the Nexus 1000V delivers the most operationally consistent and transparent server access design for virtual machine deployments, substantially reducing the overhead to configure, troubleshoot, and repair the server access link between the vNIC, virtual switch, and blade switch.

Increased resilience: The Nexus 4001I extends NX-OS to blade server access, providing a fault-tolerant network with a single modular operating system across the data center.

Features and specifications

The Cisco Nexus 4001I Switch Module includes the following features and functions:

Form-factor

Single-height high-speed switch module

External ports

Six 10 Gb SFP+ ports operating at wire speed. Also designed to support 1 Gb SFP if required, with the flexibility of mixing 1 Gb/10 Gb. Table 2 lists supported transceivers and cables.

One 10/100/1000 Mb copper RJ-45 used for out-of-band management.

An RS-232 RJ-45 connector for serial port that provides an additional means to configure the switch module. The console cable is supplied with the switch module.

Internal ports

Fourteen internal auto-negotiating ports: 1 Gb or 10 Gb to the server blades

One internal full-duplex 100 Mbps port connected to the management module

Scalability and performance

Autosensing 1 Gb/10 Gb internal and external Ethernet ports for bandwidth optimization

Non-blocking architecture with wire-speed forwarding of traffic and full line rate performance of 400 Gbps full duplex

Forwarding rate of 300 million packets per second (mpps)

Low, predictable, and consistent latency of 1.5 microseconds regardless of packet size, traffic pattern, or enabled features on 10 Gigabit Ethernet interface

Media access control (MAC) address learning: automatic update, supports up to 8 Kb MAC addresses

EtherChannels and LACP (IEEE 802.3ad) link aggregation, up to 60 Gb of total uplink bandwidth per switch, up to seven trunk groups, and up to six ports per group

Support for jumbo frames (up to 9216 bytes)

Traffic suppression (unicast, multicast, and broadcast)

IGMP snooping to limit flooding of IP multicast traffic (IGMP V2 and V3)

Configurable traffic distribution schemes over EtherChannel links based on source/destination IP addresses, MAC addresses, or ports

Spanning Tree edge ports (formerly PortFast) for rapid STP convergence

Availability and redundancy

IEEE 802.1D-2004 Rapid and Multiple Spanning Tree Protocols (802.1w and 802.1s)

Layer 2 Trunk Failover to support active/standby configurations of network adapter teaming on blades

VLAN support

Up to 512 VLANs supported per switch; VLAN numbers ranging from 1 to 4000

802.1Q VLAN tagging support on all ports

Private VLANs

Cisco Nexus 4001I Switch Module for IBM BladeCenter 3

Security

VLAN-based, MAC-based, and IP-based access control lists (ACLs)

Role-based access control

Radius, TACACS+

Quality of service (QoS)

Support for IEEE 802.1p CoS, IP ToS/DSCP, Protocol, IP Real Time Protocol, and ACL-based (MAC/IP source and destination addresses, VLANs) traffic classification and processing

Traffic shaping and re-marking based on defined policies

Eight Weighted Round Robin (WRR) priority queues per port for processing qualified traffic

Fibre Channel over Ethernet

Support for T11-compliant FCoE on all 10-Gigabit Ethernet interfaces

FCoE Initialization Protocol (FIP): Converged Enhanced Ethernet Data Center Bridging Exchange (CEE-

DCBX) protocol supports T11-compliant Gen-2 CNAs

802.1Q VLAN tagging for FCoE frames

Priority-based flow control (IEEE 802.1Qbb) simplifies management of multiple traffic flows over a single network link and creates lossless behavior for Ethernet by allowing class-of-service (CoS)-based flow control

Enhanced Transmission Selection (IEEE 802.1Qaz) enables consistent management of QoS at the network level by providing consistent scheduling of different traffic types (IP, storage, and so on)

Data Center Bridging Exchange (DCBX) Protocol (IEEE 802.1AB) simplifies network deployment and reduces configuration errors by providing autonegotiation of IEEE 802.1 DCB features between the network interface card (NIC) and the switch and between switches

Manageability

Command-line interface (CLI): You can configure switches using the CLI from an SSH V2 session, a Telnet session, or the console port. SSH provides a secure connection to the device.

XML Management Interface over SSH: You can configure switches using the XML management interface, which is a programming interface based on the NETCONF protocol that complements the CLI functionality. For more information, see the

Cisco NX-OS XML Interfaces User Guide .

Cisco Data Center Manager support.

Monitoring

Switch LEDs for external port status and switch module status indication.

RMON.

Change tracking and remote logging with syslog feature.

Online diagnostics.

Cisco Fabric Services.

Session Manager.

Special functions

Serial over LAN (SOL)

The switch module supports the following IEEE standards:

? IEEE 802.1D: Spanning Tree Protocol

? IEEE 802.1p: CoS Prioritization

? IEEE 802.1Q: VLAN Tagging

? IEEE 802.1s: Multiple VLAN Instances of Spanning Tree Protocol

? IEEE 802.1w: Rapid Reconfiguration of Spanning Tree Protocol

? IEEE 802.3: Ethernet

? IEEE 802.3ad: Link Aggregation Control Protocol (LACP)

? IEEE 802.3ae: 10 Gigabit Ethernet

Supported BladeCenter chassis and expansion cards

The Cisco Nexus 4001I Switch Module is supported in the IBM BladeCenter chassis listed in Table 3.

Table 3. IBM BladeCenter chassis that supports the Cisco Nexus 4001I Switch Module

Cisco Nexus 4001I Switch Module for IBM BladeCenter 4

Cisco Nexus 4001I Switch Module 46M6071 N N Y N Y N N

The Cisco Nexus 4001I Switch Module supports the expansion cards listed in Table 4. Table 4 also lists the chassis bays in which the switch module must be installed when used with each expansion card.

Table 4. Cisco Nexus 4001I Switch Module and BladeCenter chassis I/O bays support

Gigabit Ethernet integrated in the server

Ethernet Expansion Card (CFFv)

Ethernet Expansion Card (CIOv)

QLogic Ethernet and 4Gb FC Card (CFFh)

2/4 Port Ethernet Expansion Card (CFFh)

QLogic Ethernet and 8Gb FC Card (CFFh)

NetXen 10Gb Ethernet Expansion Card (CFFh)

Broadcom 2-port 10Gb Ethernet Exp. Card (CFFh)

Broadcom 4-port 10Gb Ethernet Exp. Card (CFFh)

Broadcom 10 Gb Gen 2 2-port Ethernet Expansion Card (CFFh)

Broadcom 10 Gb Gen 2 4-port Ethernet Expansion Card (CFFh)

QLogic 2-port 10Gb Converged Network Adapter (CFFh)

Emulex Virtual Fabric Adapter (CFFh)

None

39Y9310

N

N

N

N

N

N

N

N

N

N

N

N

N

N

N

N

N

N

N

N

44W4475 N N N N N N N N N N

39Y9306 N N N N N N N N N N

44W4479 N N N N N N Y Y Y Y

44X1940 N N N N N N N N N N

39Y9271 N N N N N N Y N Y N

44W4466 N N N N N N Y N Y N

44W4465 N N N N N N Y Y Y Y

46M6168 N N N N N N Y N Y N

46M6164 N N N N N N Y Y Y Y

42C1830 N N N N N N Y N Y N

49Y4235 N N N N N N Y* N Y* N

* The Cisco Nexus 4001I Switch Module supports the Emulex Virtual Fabric Adapter (CFFh) only in the physical NIC

(pNIC) mode of the adapter

The BladeCenter chassis have the following bays:

BladeCenter S, E, and T have four standard I/O bays (1, 2, 3, and 4).

BladeCenter H has four standard I/O bays (1, 2, 3, and 4), two bridge bays (5 and 6), and four high-speed bays (7,

8, 9, and 10).

Cisco Nexus 4001I Switch Module for IBM BladeCenter 5

BladeCenter HT has four standard I/O bays (1, 2, 3, and 4) and four high-speed bays (7, 8, 9, and 10).

The Cisco Nexus 4001I Switch Module fits in one of the high-speed I/O bay (bays 7-10).

Popular configurations

The Cisco Nexus 4001I Switch Module can be used in various configurations.

Fibre Channel over Ethernet configuration

Figure 2 shows the use of Cisco Nexus 4001I Switch Modules to route two Ethernet ports from the QLogic 2-port 10Gb

Converged Network Adapter (CFFh) installed into each server. Two Cisco Nexus 4001I Switch Modules are installed in bay 7 and bay 9 of the BladeCenter H chassis. All connections between the controller, card, and the switch modules are internal to the chassis. No cabling is needed.

The Cisco Nexus 4001I Switch Modules are connected to the Cisco Nexus 5000 TOR switches that can have native FC interfaces for storage attachments (not shown).

Figure 2. FCoE solution using two Cisco Nexus 4001I Switch Modules

Table 5 lists the components used in this configuration.

Table 5. Components used when connecting QLogic 2-port 10Gb Converged Network Adapter (CFFh) to two Cisco

Nexus 4001I Switch Modules

Diagram reference

Part number/ machine type

Description Quantity

Varies IBM BladeCenter HS22 or other supported server 1 to 14

42C1830 QLogic 2-port 10Gb Converged Network Adapter (CFFh)

8852 or 8740/8750

46M6071

49Y9983

Varies

BladeCenter H or BladeCenter HT

Cisco Nexus 4001I Switch Module

Software Upgrade License for Cisco Nexus 4001I

External cables to connect to Cisco Nexus 5000 (not shown)

2

2

1 per server

1

Up to 12*

*The Cisco Nexus 4001I Switch Module has six external 10 Gb ports. To communicate outside of the chassis, you must have either one SFP+ transceiver or SFP+ direct-attached cable (DAC) connected. You have the flexibility to expand your bandwidth as you see fit using from one to six connections per switch.

4-port 10 Gb Ethernet LAN-only configuration

Cisco Nexus 4001I Switch Module for IBM BladeCenter 6

Figure 3 shows the use of Cisco Nexus 4001I Switch Modules to route four Ethernet ports from the Broadcom 4-port

10Gb Ethernet Expansion Card (CFFh) installed into each server. Four Cisco Nexus 4001I Switch Modules are installed in bay 7, bay 8, bay 9, and bay 10 of the BladeCenter H chassis. All connections between the controller, card, and switch modules are internal to the chassis. No cabling is needed.

Figure 3. A 40 Gb solution using four Cisco Nexus 4001I Switch Modules

Table 6 lists the components used in this configuration.

Table 6. Components used when connecting Broadcom 4-port 10Gb Ethernet Expansion Card (CFFh) to four Cisco

Nexus 4001I Switch Modules

Diagram reference

Part number/ machine type

Description Quantity

Varies IBM BladeCenter HS22 or other supported server 1 to 14

44W4465 Broadcom 4-port 10Gb Ethernet Expansion Card (CFFh)

8852 or 8740/8750

46M6071

Varies

BladeCenter H or BladeCenter HT

Cisco Nexus 4001I Switch Module

External cables to connect to external LAN infrastructure (not shown)

1 per server

1

4

Up to 24*

*The Cisco Nexus 4001I Switch Module has six external 10 Gb ports. In order to communicate outside of the chassis you must have either one SFP+ transceiver or SFP+ direct-attached cable (DAC) connected. You have the flexibility to expand your bandwidth as you see fit using from one to six connections per switch.

Connectors and LEDs

Cisco Nexus 4001I Switch Module for IBM BladeCenter 7

Connectors and LEDs

Figure 4 shows the front panel of the Cisco Nexus 4001I Switch Module .

Figure 4. Front panel of the Cisco Nexus 4001I Switch Module

The front panel contains the components listed in Table 7.

Table 7. Callouts in Figure 4

Callout Description

1, 2, 3, 4, 5, 6 10 Gb Ethernet Small Form Factor Pluggable (SFP+) ports

7, 10

8

9

Release latches

Out-of-band management RJ45 port (labeled Management); supports 10/100/1000 Mbps speeds

Serial console port has RJ45 connector for management console (labeled Console)

Network cabling requirements

Cisco Nexus 4001I Switch Module for IBM BladeCenter 8

Network cabling requirements

The network cables required for the switch module are:

10GBASE-SR

850 nm communication using multimode fiber cable (50 µ or 62.5 µ) up to 300 m, LC duplex connector

Requires 10 GbE SFP+ transceiver modules, part number SFP-10G-SR(=)

10GBASE-LR

1310 nm communication using single-mode fiber cable up to 10 km, LC duplex connector

Requires 10 GbE SFP+ transceiver modules, part number SFP-10G-LR(=)

1000BASE-X:

850 nm communication using multimode fiber cable (50 µ or 62.5 µ) up to 550 m, LC duplex connector

(requires GbE SFP transceiver modules, part number GLC-SX-MM(=))

1310 nm communication using single-mode fiber cable up to 10 km, LC duplex connector (requires GbE

SFP transceiver modules, part number GLC-LH-SM(=)

1000BASE-T:

UTP Category 6

UTP Category 5e (100 meters maximum)

UTP Category 5 (100 meters maximum)

EIA/TIA-568B 100-ohm STP (100 meters maximum)

Requires GbE SFP transceiver modules, part number GLC-T(=)

RS-232 serial cable: RJ-45-to-DB-9 console cable that comes with the switch module

Related publications

For more information see the following documents:

Cisco Nexus 4001I product publications:

Getting Started Guide for Cisco Nexus 4001I Switch Module

Configuration Guide for Cisco Nexus 4001I Switch Module

Command Reference Guide for Cisco Nexus 4001I Switch Module

Hardware Installation Guide for Cisco Nexus 4001I Switch Module

http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5082494

Implementing FCoE with IBM BladeCenter using Cisco Nexus 4001i/5000 Solution http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101688

QLogic 2-port 10Gb Converged Network Adapter (CFFh) for IBM BladeCenter http://www.redbooks.ibm.com/abstracts/tips0716.html

Cisco Nexus 4000 Series Solution Overview http://cisco.com/en/US/prod/collateral/switches/ps9441/ps10596/solution_overview_c22-560880.pdf

Cisco Nexus 4000 Series Design Guide http://cisco.com/en/US/prod/collateral/switches/ps9441/ps10596/deployment_guide_c07-574724.pdf

IBM U.S. Announcement Letter http://ibm.com/common/ssi/cgi-bin/ssialias?infotype=dd&subtype=ca&&htmlfid=897/ENUS109-553

IBM BladeCenter Interoperability Guide: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073016

IBM Redbooks publication

IBM BladeCenter Products and Technology , SG24-7523: http://www.redbooks.ibm.com/abstracts/sg247523.html

Cisco Nexus 4001I Switch Module for IBM BladeCenter 9

Notices

Lenovo may not offer the products, services, or features discussed in this document in all countries. Consult your local Lenovo representative for information on the products and services currently available in your area. Any reference to a Lenovo product, program, or service is not intended to state or imply that only that Lenovo product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any Lenovo intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any other product, program, or service.

Lenovo may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to:

Lenovo (United States), Inc.

1009 Think Place - Building One

Morrisville, NC 27560

U.S.A.

Attention: Lenovo Director of Licensing

LENOVO PROVIDES THIS PUBLICATION ”AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,

INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS

FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. Lenovo may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

The products described in this document are not intended for use in implantation or other life support applications where malfunction may result in injury or death to persons. The information contained in this document does not affect or change

Lenovo product specifications or warranties. Nothing in this document shall operate as an express or implied license or indemnity under the intellectual property rights of Lenovo or third parties. All information contained in this document was obtained in specific environments and is presented as an illustration. The result obtained in other operating environments may vary. Lenovo may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Any references in this publication to non-Lenovo Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this Lenovo product, and use of those Web sites is at your own risk. Any performance data contained herein was determined in a controlled environment.

Therefore, the result obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment.

© Copyright Lenovo 2016. All rights reserved.

This document, TIPS0754, was created or updated on July 25, 2013.

Send us your comments in one of the following ways:

Use the online Contact us review form found at: http://lenovopress.com/TIPS0754

Send your comments in an e-mail to: [email protected]

This document is available online at http://lenovopress.com/TIPS0754 .

Cisco Nexus 4001I Switch Module for IBM BladeCenter 10

Trademarks

Lenovo, the Lenovo logo, and For Those Who Do are trademarks or registered trademarks of Lenovo in the United

States, other countries, or both. A current list of Lenovo trademarks is available on the Web at http://www.lenovo.com/legal/copytrade.html

.

The following terms are trademarks of Lenovo in the United States, other countries, or both:

Lenovo®

BladeCenter®

BladeCenter Interoperability Guide™

BladeCenter Open Fabric™ vNIC™

The following terms are trademarks of other companies:

Other company, product, or service names may be trademarks or service marks of others.

Cisco Nexus 4001I Switch Module for IBM BladeCenter 11